EP2974322A1 - Système à modes multiples et procédé de fourniture de contenu vidéo sélectionnable par un utilisateur - Google Patents

Système à modes multiples et procédé de fourniture de contenu vidéo sélectionnable par un utilisateur

Info

Publication number
EP2974322A1
EP2974322A1 EP14722897.7A EP14722897A EP2974322A1 EP 2974322 A1 EP2974322 A1 EP 2974322A1 EP 14722897 A EP14722897 A EP 14722897A EP 2974322 A1 EP2974322 A1 EP 2974322A1
Authority
EP
European Patent Office
Prior art keywords
video
content item
video content
client device
transmission mode
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
EP14722897.7A
Other languages
German (de)
English (en)
Inventor
Ronald Alexander Brockmann
Anuj DEV
Gerrit Hiddink
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
ActiveVideo Networks Inc
Original Assignee
ActiveVideo Networks Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by ActiveVideo Networks Inc filed Critical ActiveVideo Networks Inc
Priority claimed from PCT/US2014/030773 external-priority patent/WO2014145921A1/fr
Publication of EP2974322A1 publication Critical patent/EP2974322A1/fr
Ceased legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
    • G06F3/04842—Selection of displayed objects or displayed text elements
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs
    • H04N21/23412—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs for generating or manipulating the scene composition of objects, e.g. MPEG-4 objects
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/60—Network streaming of media packets
    • H04L65/70—Media network packetisation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs
    • H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs
    • H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
    • H04N21/234309—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements by transcoding between formats or standards, e.g. from MPEG-2 to MPEG-4 or from Quicktime to Realvideo
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs
    • H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
    • H04N21/234363—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements by altering the spatial resolution, e.g. for clients with a lower screen resolution
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/236—Assembling of a multiplex stream, e.g. transport stream, by combining a video stream with other content or additional data, e.g. inserting a URL [Uniform Resource Locator] into a video stream, multiplexing software data into a video stream; Remultiplexing of multiplex streams; Insertion of stuffing bits into the multiplex stream, e.g. to obtain a constant bit-rate; Assembling of a packetised elementary stream
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/236—Assembling of a multiplex stream, e.g. transport stream, by combining a video stream with other content or additional data, e.g. inserting a URL [Uniform Resource Locator] into a video stream, multiplexing software data into a video stream; Remultiplexing of multiplex streams; Insertion of stuffing bits into the multiplex stream, e.g. to obtain a constant bit-rate; Assembling of a packetised elementary stream
    • H04N21/23614—Multiplexing of additional data and video streams
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/25—Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication, learning user preferences for recommending movies
    • H04N21/258—Client or end-user data management, e.g. managing client capabilities, user preferences or demographics, processing of multiple end-users preferences to derive collaborative data
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/25—Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication, learning user preferences for recommending movies
    • H04N21/258—Client or end-user data management, e.g. managing client capabilities, user preferences or demographics, processing of multiple end-users preferences to derive collaborative data
    • H04N21/25808—Management of client data
    • H04N21/25825—Management of client data involving client display capabilities, e.g. screen resolution of a mobile phone
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/25—Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication, learning user preferences for recommending movies
    • H04N21/258—Client or end-user data management, e.g. managing client capabilities, user preferences or demographics, processing of multiple end-users preferences to derive collaborative data
    • H04N21/25808—Management of client data
    • H04N21/25833—Management of client data involving client hardware characteristics, e.g. manufacturer, processing or storage capabilities
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/25—Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication, learning user preferences for recommending movies
    • H04N21/258—Client or end-user data management, e.g. managing client capabilities, user preferences or demographics, processing of multiple end-users preferences to derive collaborative data
    • H04N21/25808—Management of client data
    • H04N21/25858—Management of client data involving client software characteristics, e.g. OS identifier

Definitions

  • the present invention relates to providing user selectable content with a graphical user interface in a streaming multimedia system, and more particularly to a multiple mode system with automatic control logic for determining which mode to implement based upon a plurality of characteristics including capabilities of the decoding device and the selected user- selectable multimedia content.
  • a method provides an audiovisual experience to an individual having a client device that is capable of decoding audiovisual data using a video codec, and an audiovisual display coupled to the client device for display of decoded audiovisual data.
  • the method includes first providing the client device with a first graphical user interface (GUI) that indicates a plurality of videos and includes an input for selecting a video from the plurality of videos.
  • GUI graphical user interface
  • the method in response to receiving a selection of a video in the plurality of videos by the individual using the input, includes determining a transmission mode as a function of: 1) the decoding capabilities of the client device, 2) a video encoding format of the selected video, 3) whether the selected video should be displayed full screen or partial screen, and 4) whether the client device is capable of overlaying image data into a video stream. Then, in a server device remote from the client device, the method calls for preparing, for transmission according to the determined transmission mode, audiovisual data that include the selected video. Finally, the method requires transmitting the prepared audiovisual data, from the server device to the client device, according to the predetermined transmission mode, for display on the audiovisual device associated with said client.
  • the audiovisual data includes the first GUI
  • preparation further includes: rendering the first GUI according to a previously determined screen resolution and stitching the selected video into the previously rendered first GUI where stitching is a method of combining previously encoded video streams by any of a variety of suitable processes.
  • preparation includes transcoding the selected video where transcoding is a method of altering already encoded video by changing format or changing encoding means or both.
  • a third transmission mode when the selected video will be displayed full screen and the client device can decode the format of the selected video, and further, no image data will be overlaid on the selected video then preparation includes repackaging the selected video.
  • preparation when the selected video should be displayed full screen and the client device can decode the selected video, and further, the client device is capable of overlaying image data onto the selected video, and still further, the audiovisual data from the server includes a second GUI that provides various GUI elements such as video playback controls, then preparation includes rendering the second GUI according to the client overlay resolution.
  • preparation includes: rendering the second GUI according to the video resolution, video size, and video frame rate compatible with the client device; decoding a portion of the selected video; blending the rendered second GUI into the decoded portion; and re-encoding the blended portion according to the video encoding format.
  • a computer program product provides an audiovisual experience to an individual having a client device that is capable of decoding audiovisual data using a video codec, and an audiovisual display coupled to the client device for display of decoded audiovisual data.
  • the computer program product has a computer useable medium on which is stored non-transitory computer program code for executing the above-described method in its various transmission modes.
  • a computer system for providing an audiovisual experience to an individual having a client device that is capable of decoding audiovisual data using a video codec, and an audiovisual display coupled to the client device for display of decoded audiovisual data.
  • the computer system has an application engine for providing a first graphical user interface (GUI) that indicates a plurality of videos and includes an input for selecting a video from the plurality of videos, and for providing a second GUI that includes video playback controls.
  • GUI graphical user interface
  • the computer system also has control logic for determining a transmission mode in response to receiving a selection of a video in the plurality of videos by the individual using the input.
  • Determining the transmission mode is a function of: 1) the decoding capabilities of the client device, 2) a video encoding format of the selected video, 3) whether the selected video should be displayed full screen or partial screen, and 4) whether the client device is capable of overlaying image data into a video stream.
  • the computer system also has a transcoder for transcoding the selected video from a second encoding format into the first encoding format, according to the determined transmission mode.
  • the computer system also has a blender for blending the second GUI into the selected video using the first encoding format, according to the determined transmission mode.
  • the computer system also has a stitcher for stitching the output of the application engine with the output of the transcoder and the blender, according to the determined transmission mode.
  • the computer system also has a packager for packaging audiovisual data according to the determined transmission mode.
  • the computer system has a transmitter for transmitting the packaged audiovisual data, toward the client device, according to the determined transmission mode, for display on the audiovisual display.
  • the components of the computer system may be configured according to the transmission mode.
  • the audiovisual data further include the first GUI
  • the application engine is configured to render the first GUI according to a previously set screen resolution
  • the transcoder is configured to transcode the selected video
  • the stitcher is configured to stitch the transcoded video into the rendered first GUI, when the video should be displayed partial screen.
  • the transcoder is configured to transcode the selected video, when the selected video should be displayed full screen and the client device cannot decode the video encoding format of the selected video.
  • the packager is configured to repackage the selected video, when the selected video should be displayed full screen, the client device can decode the video encoding format of the selected video, and no image data should be overlaid on the selected video.
  • the audiovisual data further include a second GUI that includes video playback controls, and wherein the application engine is configured to render the second GUI according to a client overlay resolution, when the selected video should be displayed full screen, the client device can decode the video encoding format of the selected video, and the client device is capable of overlaying image data into the selected video.
  • the application engine is configured to render the second GUI according to a video resolution, video size, and video frame rate; the transcoder is configured to decode a portion of the selected video; the blender is configured to blend the rendered second GUI into the decoded portion; and the transcoder is further configured to re-encode the blended portion according to the video encoding format, when the selected video should be displayed full screen, the client device can decode the video encoding format of the selected video, and the client device is not capable of overlaying image data into the selected video. It should be clear that not all of these components must be active in each transmission mode. Therefore, operation of each of the transcoder, blender, stitcher, and packager may be optional according to the determined transmission mode.
  • a method for streaming user-selected video content encoded in a first protocol format having a protocol container.
  • the method requires first receiving a request for streaming the user-selected video content, and obtaining the user-selected video content from a first source.
  • the method calls for removing the protocol container from the user-selected video content and repackaging the user-selected video content into an MPEG-2 transport stream.
  • the method requires transmitting the MPEG-2 transport stream with the user-selected video content encoded in the first protocol wherein the first protocol is different than MPEG and the client device is capable of decoding the first protocol.
  • the method may also include adapting the presentation and synchronization timing of the stream based upon the presentation and synchronization timing of the user-selected video content.
  • the method may be performed within a cable television network.
  • the first protocol container may be MP4, DASH, or HTTP, and the first protocol container and the first encoded protocol may be the same.
  • a method for adaptation of a stream for streaming a user-selected video asset includes first streaming a graphical user interface from a server to a client device wherein the stream has a plurality of stream characteristics. Next, the method includes receiving a user request for playback of encoded video content encoded with one or more different streaming
  • the method includes generating graphical user interface elements in accordance with the one or more different streaming characteristics.
  • the method includes combining the encoded video content and the generated graphical user interface elements to form an encoded transport stream.
  • the user requested encoded elements may have a picture size that is less than a full video frame and the generated user elements when combined with the user requested encoded video may form a complete video frame.
  • a different streaming characteristic between the graphical user interface and the requested encoded video content may be the frame rate and the generated graphical user interface elements may have the same frame rate as the requested encoded video content.
  • the generated graphical user interface elements may have the same sampling rate as the requested encoded video content.
  • FIG. 1 is an illustration of a screen layout
  • Fig. 2 is an environment for implementation of at least one embodiment of the invention.
  • Fig. 3 is a flow chart that discloses the control logic sequence for switching between different modes of operation
  • Fig.4 shows the steps for repackaging and resynchronizing a full-frame video sequence
  • FIG. 5 shows an exemplary functional architecture for implementing a multimodal platform for providing user-selectable video content
  • Fig. 5 A shows the functional architecture for supporting mode 1, which is a partial screen video with a stitched graphical user interface
  • Fig. 5B shows the functional architecture to support modes 2 and 5, which are the display of full-screen video with and without blended overlays;
  • Fig. 5C shows the functional architecture for mode 3, which is a full-screen pass through where encoded video content is repackaged and re-streamed;
  • Fig. 5D shows the functional architecture for supporting mode 4, which is a full screen transcode due to decoding limitation of the client device.
  • Fig. 6 shows the source architecture with overlays showing structural elements.
  • Embodiments of the present invention are directed to network transmission of user-selected multimedia content (video, audio, and audiovisual content).
  • the selected multimedia content may be encoded data that may be encoded with a codec.
  • Various embodiments also use one or more container protocols for putting encoded data into a format for transmission in a network and use transport protocols for transferring the containers of encoded video and audio data to a client device within a network.
  • container protocols for putting encoded data into a format for transmission in a network
  • transport protocols for transferring the containers of encoded video and audio data to a client device within a network.
  • embodiments of the present invention operate on multimedia content. For simplicity, disclosed embodiments in general describe video
  • Fig. 1 is an illustration of a frame layout that includes graphical user interface elements for allowing a subscriber to select video content to be distributed to a client device in a content distribution network, such as a cable television network.
  • the frame layout provides the location of various graphical user interface elements and video content that are to be added to the frame layout.
  • the graphical user interface elements include buttons for the selection of SD (standard definition) and HD (high definition) content for both renting and purchasing.
  • the video content elements to be added include a scaled movie preview which is full-motion video, along with text based video content elements such as "Movie Title", "Actor, Year, Duration, Director", and "Synopsis".
  • video elements since the content distribution network transmits video content in a video stream. As a result, even static elements are displayed as frames of video.
  • video content is distributed via a transport protocol, using a container protocol, wherein the video content is encoded in an encoded format (e.g., MPEG-2, H.264 etc.).
  • an encoded format e.g., MPEG-2, H.264 etc.
  • the content distribution network includes a multitude of components including a central platform that includes a plurality of processors for serving content.
  • the processors generally perform the functions of providing broadcast video, user-interface guides, interactive content, and video- on-demand.
  • the processors that are part of a content distribution platform are coupled to a number of nodes that broadcast and stream on-demand video content to a plurality of subscriber client devices.
  • the subscriber client devices may include set-top boxes, tablets, televisions and other electronic communications devices.
  • Each client device has certain capabilities based upon both the hardware and software that are available to the client device. For example, disparate client devices may have different processors, memory, codecs, and capabilities to download and execute programs.
  • MPEG-2 Transport streams and decode MPEG-2 elementary streams In a cable television environment, most, if not all, devices can access MPEG-2 Transport streams and decode MPEG-2 elementary streams. Some devices may have more advanced capabilities, including a local operating system, software components, and the ability to download and execute additional programs. Further client device may be able to receive and work with different transport protocols such as UDP, HLS, HTTP, MPEG-DASH, and smooth streaming, work with different content containers, such as MP4, MPEG-2 transport stream, MPEG-2 program stream and decode different codecs including H.264, MPEG-2.
  • transport protocols such as UDP, HLS, HTTP, MPEG-DASH, and smooth streaming
  • Fig. 2 shows an exemplary environment including a content distribution platform for providing a multi-modal operation of selectable video content to be streamed to a client device.
  • the platform is a structure that includes a plurality of components.
  • the platform includes an application engine for selection of a graphical user interface to be provided to a client device in response to a request from the client device.
  • the application engine responds to requests from the client device for content.
  • the application engine may include an HTML5 application that defines a graphical user interface (GUI).
  • GUI may include a frame layout along with position information for insertion of content, for example as shown in Fig. 1.
  • the layout includes a plurality of blocks (movie title, movie preview (video), SD price etc.) for video elements to be inserted into the layout.
  • encoded video elements such as MPEG encoded fragments, may be referenced at the block locations.
  • the HTML5 application keeps track of state information regarding the elements of the graphical user interface.
  • the HTML5 application can be reused for presenting different content to the user in a graphical display.
  • the HTML5 application may access the encoded elements and may cause elements that are not already properly encoded to be encoded in an MPEG encoder.
  • the MPEG elements of the HTML5 application may include MPEG fragments of buttons, sliders, switches, etc. that may be part of the HTML5 application.
  • the MPEG elements may be static images and scaled versions of video content, such as movie previews.
  • the HTML5 application may include encoded fragments that represent the layout.
  • the layout may be considered a background and therefore the layout may include a plurality of encoded elements to represent the layout. Similar content distribution platforms that can be used with embodiments of the present invention can be found in U.S. Patent Application 12/008,697 and U.S. Patent Application 13/445,104 both of which are incorporated herein by reference in their entirety.
  • the HTML5 application may also include reference to video content from an outside source or stored at a different location.
  • a content source which may be on a content source server.
  • content from the content source server is provided to the content distribution platform.
  • the platform determines if the content needs to be transcoded based upon the capabilities of the client device. If necessary, the content from the content server is provided to a transcoder. The transcoder then scales and/or transcodes the video from the content source, so that the video from the content source can be stitched together with other encoded elements in a compositor module. If the content does not require scaling or transcoding, the content will be provided directly to a compositor. Whether a source is transcoded is determined by control logic that is part of the platform and will be explained in further detail below. The compositor receives in encoded fragments, such as encoded MPEG fragments and may receive in encoded video content.
  • encoded fragments such as encoded MPEG fragments
  • the compositor takes the various encoded elements and creates an MPEG elementary stream based upon the encoded elements and the frame layout from the HTML5 application. If a request for full frame encoded video content is received, the compositor may receive in the encoded video content in its native encoded format and may package the encoded video content in an MPEG transport stream without requiring the encoded video content to be transcoded. For example, if the client device is capable of decoding an H.264 encoded file and a full screen video is requested from a source, the H.264 video will not be transcoded and will only be encapsulated into an MPEG-2 transport stream for transmission to the client device.
  • the type of request, the content to be presented, along with the available processing resources at both the server and on the client device are used in determining the mode of operation and the format of the data to be transmitted to a requesting client device.
  • the client device in general includes an MPEG decoder and optionally may include a bitmap decoder for an overlay plane.
  • the MPEG decoder receives an MPEG transport stream that contains one or more MPEG elementary streams or other encoded streams (e.g., H.264 etc.).
  • the MPEG decoder decodes the encoded stream and presents the output to a display device.
  • the bitmap decoder receives in a bitmap overlay stream separate from the full screen MPEG video content.
  • the client device receives the bitmap overlay and displays the bitmap overlay on top of the decoded MPEG video content.
  • the bitmap and the decoded MPEG video may be blended together in the spatial domain by the client device or elements of the bitmap may replace elements of the spatially decoded MPEG video content.
  • a decoded MPEG video frame may have elements replaced wherein the bitmap represent a graphical user interface.
  • Fig. 2 represents one version of a content distribution platform and should not be viewed as limiting the scope of the present invention.
  • resources at both the server-side and client-side are relevant to determining how to efficiently process requests from client devices.
  • the user can select from a plurality of different content and therefore the content distribution platform will operate in one of a plurality of different modes.
  • a first mode provides a graphical user interface for selection of content to be displayed (e.g., movies, television shows, specials etc.) which may either be static or have less-than full frame video streaming using an HTML5 application wherein MPEG elements are stitched into a layout to form MPEG video frames.
  • a second mode provides a full screen trick-play mode wherein full-screen video is overlaid with graphical user interface controls (e.g., fast forward, rewind, pause, stop etc.) where at least partial decoding and re-encoding of the video content and blending occurs.
  • a third mode provides a full screen display wherein the video content is provided to the client device for full-screen playback in an encoded format compatible with the client device without transcoding.
  • a fourth mode provides a full-screen trick play mode wherein the client device performs the blending and encoding graphical controls.
  • a fifth mode provides a full screen transcode of the source material dependent in part on the client device's decoding capabilities. In various embodiments of the present invention, these modes of operation can be selectively and automatically switched between based upon both requests from the client device, and the capabilities of the client device wherein the control is performed on the server-side.
  • Fig. 3 is a flow chart showing the operation of a control processing logic within the content distribution platform.
  • the control processing logic queries if the requested content by the client device contains visible video. If the answer to the query is no, the graphical user interface is processed at the server wherein a selection of encoded elements, such as MPEG elements, are selected based upon a frame layout of an HTML5 application and the MPEG elements are stitched together to form a full video frame (static) that can then be transmitted to the client device and displayed on a display device.
  • encoded elements such as MPEG elements
  • the logic queries if full screen video is to be shown. If there is only a partial screen of video to be shown, the server switches to mode 1 and identifies an HTML5 application and frame layout. It then accesses source content that is scaled and stitched together to form a series of MPEG encoded video frames defining an MPEG elementary stream. If, however, the video content that has been selected by the user is full screen video, for example if the user indicates a desire to view a movie, TV show, fullscreen video clip, or full-screen promotional content, the logic performs further queries.
  • the client device identifies itself and thus, identifies its capabilities.
  • the client capabilities may be transmitted during the communication or may be stored in a user/device profile that can be accessed by the server and the control logic.
  • the logic queries whether the video properties of the selected video content are compatible with the capabilities of the client device. For example, if the client device can only decode MPEG-2 and the selected video content is encoded using H.264, the logic switches to mode 4 and a full-screen transcode is performed, so that the selected video content will be transmitted to the client device in a format that the client device can decode (e.g., MPEG-2).
  • the logic determines if a graphical user interface element is required to be placed on top of the video.
  • a graphical user interface element may be required based upon signals received from the client device. For example, a user may have a full screen video playing and may use a remote control coupled to the client device to pause or fast- forward the video content. The client device initiated signal informs the control logic that graphical user interface elements should be placed on top of the full screen video to enable trick-play features (fast forward, rewind, play, pause, skip etc.).
  • the logic will initiate mode 3, which provides a video pass-through capability.
  • the client device does not require content to be transcoded and therefore the content will remain in its native format.
  • the video content will then be repackaged and streamed to the client device.
  • the logic determines that the graphical user interface is to be placed on top of the video, the logic then queries if the client and control protocol support a local overlay.
  • the control logic looks at the control protocol as implemented by the platform and the connecting network between the control logic (i.e. server) and the client device.
  • Certain networks will provide for more than one channel of communication with a client device, so that control data may be transmitted separately from MPEG elementary stream data (the requested video content). The control data may be transmitted using a different signaling system or may be provided in a separate MPEG elementary stream.
  • the client device must be capable of receiving instructions that a local overlay should be created.
  • the control logic determines that the client and the control protocol support a local overlay, either graphical or full-motion video, the control logic switches to mode 5.
  • mode 5 the system renders GUI elements in accordance with the overlay resolution, size, and frame rate for the video content. For example, if the full screen video is being rendered as, for example, 720p, the GUI elements will be scaled and rendered as 720p elements.
  • These graphical user interface elements may be transmitted as encoded fragments.
  • the client device is capable of receiving encoded fragments or spatially rendered fragments and includes local software for stitching the encoded fragments or spatially rendered fragments with the full screen encoded video content.
  • the client device may have a predetermined layout using templates that define screen locations and other parameters for adding in the GUI elements for the interface or the platform may transmit a layout for the interface.
  • the client device will then insert the GUI elements onto the full screen video. This may be performed either in the encoded domain or in the spatial domain. If the client device performs the combination in the spatial domain, blending may occur wherein the GUI elements and the video content may be blended using an alpha layer.
  • the control logic will cause the graphical user interface elements to be added on the server side (at the platform) in mode 2.
  • a partial decode of the selected encoded video content may occur.
  • macroblock locations of the video content that will include GUI elements may be decoded to the spatial domain and then alpha blended together with the graphical user interface elements.
  • the GUI elements either may be stored locally or retrieved from a remote location.
  • the GUI elements may be saved at a high resolution and then scaled as needed to meet the requirements of the respective client.
  • the blended elements will then be encoded and a compatible MPEG elementary stream will be formed and placed into an MPEG-2 transport stream.
  • the platform sends the MPEG-2 transport stream to the client device and the client device decodes and displays the user- selected video content with embedded user-interface overlays.
  • One benefit of the present multi-modal system is that the number of transcoding steps is reduced compared to existing content distribution platforms and especially to content distribution platforms coupled to cable television networks. Rather than having to transcode encoded video content selected by a user, the content may be passed-through in its native encoding. This assumes that the client device is capable of decoding the native format. In this configuration, processor resources are conserved on the content distribution platform and therefore, more video streams can be processed simultaneously. Additionally, the present multi-modal system allows for all of the user-interface graphics to be centrally controlled by the content distribution platform. The multi-modal system passes the graphical user interface elements from the platform to the client device or incorporates the graphical user interface elements into the video that is being streamed to the client.
  • a further advantage of the present multi-modal systems is that content providers do not need to re-author their applications (YouTube, Netflix, Amazon etc.) or transcode their content for operation within this system.
  • the native applications (ex. YouTube, Netflix, Amazon) can be run on the content distribution platform in their native operating system and language (Java, iOS, Linux etc.) and the content will either be passed through or transcoded without intervention by the content provider.
  • Yet another advantage of the present multi-modal system occurs because full-screen playback and control of full-screen playback can be controlled by the content distribution platform without requiring client devices to take over control.
  • the client device becomes another processing element in load balancing and both legacy client devices and modern client devices can be serviced with comparable content and consistent graphical presentations.
  • the client devices on the content distribution network do not need to be updated before more advanced features can be presented to a user.
  • the content distribution system will automatically adapt to the capabilities of the client devices within the network and therefore, updates can be made on a rolling basis.
  • video content can be repackaged and re-streamed without requiring transcoding.
  • a content source having video content for streaming in a particular container format and with audio and video synchronization information will have the source video container format, such as MP4, DASH, HTTP, Smooth Streaming and HLS removed so that only the actual encoded video and audio data remains.
  • the encoded audio and video data are repackaged in a transport container that is compatible with the content distribution network. For example, the compressed video and audio content is repackaged into an MPEG-2 transport stream container. Additionally, the audio and video
  • the video stream from the content distribution platfonn to the client device is adapted based upon the audio and video synchronization data so that the stream timing complies with the transport protocol specifications (e.g., MPEG transport stream specifications).
  • transport protocol specifications e.g., MPEG transport stream specifications
  • FIG. 4 shows an example of the repackaging and re-streaming process.
  • Element 400 shows video content from a content source in its native format.
  • the video content has a first container format having a video bit stream, an audio bit stream, synchronization information and header information.
  • the container is removed from the video content and the audio and video bit streams are extracted and stored to a memory location. Additionally, the synchronization data is extracted and stored to a memory location. Other information may also be extracted and saved such as subtitles for the bit stream.
  • a new container is created. Synchronization is re-done using the new container's format.
  • Element 430 shows a representation of video frames that have been decoded from a compressed format and are transformed into the spatial domain for presentation on a video playback display associated with the requesting client device.
  • a stream may be adapted for streaming a user-selected video asset.
  • a stream may be adapted when a graphical user interface, having certain streaming characteristics, is streamed from a server to a client, and a user requests playback of video content encoded with streaming characteristics different from those of the graphical user interface.
  • a method includes first streaming a graphical user interface from a server to a client device wherein the stream has a plurality of streaming
  • the method includes receiving a user request for playback of encoded video content encoded with one or more streaming characteristics different from the plurality of streaming characteristics of the graphical user interface stream. Then, the method includes generating graphical user interface elements in accordance with the one or more different streaming characteristics. Finally, the method includes combining the encoded video content and the generated graphical user interface elements to form an encoded transport stream.
  • the user requested encoded video content may have a picture size that is less than a full video frame, and the generated user interface elements, when combined with the user requested encoded video content, may form a complete video frame.
  • the characteristic between the graphical user interface and the user requested encoded video content may be the frame rate, and in one embodiment, the generated graphical user interface elements may have the same frame rate as the user requested encoded video content.
  • the generated graphical user interface elements may have the same sampling rate as the user requested encoded video content.
  • Fig. 5 shows an architecture of one embodiment of the content distribution platform in a network that can support all of the modes of delivering a graphical user interface and video content to a client device.
  • the architecture presents functional blocks and data paths for video, audio and graphical user interface elements. To support each specific mode only a subset of the possible data paths and functional blocks are required.
  • a user may select between various types of content and the associated screens (VOD screen, content selection screen, video content information screen, ordering screen, full screen playback etc.).
  • control logic automatically selects an appropriate application and corresponding frame layout for presentation of the desired content/screen to the client device, selects an appropriate mode of operation, and controls the various functional blocks.
  • the control logic of the application content distribution platform determines how to process the received request based upon capacity of the network and capabilities of the client device.
  • control logic 5 includes several functional blocks controlled by the control logic, including a source content network, an application execution environment, an audio encoder, a bitmap (image) encoder, an audiovisual (MPEG) encoder, a transcoder, a blender, a stitcher, and a packager / multiplexer / streamer. These functional blocks are now described in more detail.
  • the source content network provides various audio, video and audiovisual content to be supplied to the client device as part of an audiovisual experience.
  • Content is provided by various content providers, each of whom may have a different storage and streaming format for their content.
  • the content may be encoded in a variety of different formats, including MPEG-4, Flash video, AVI, RTMP, MKV, and others.
  • the architecture includes an application execution environment for generation of a graphical user interface in response to requests received from a client device. Based on the correct application, the application execution environment selects the appropriate graphical user interface (frame layout along with references/addresses of elements to be stitched into the frame layout) and provides audio and screen updates to data paths of the architecture. For example, state of a button may have changed in response to action by a user, and therefore the application will have a screen update for the graphics of the button and perhaps play an audible sound.
  • the application execution environment requests video elements, such as encoded fragments, for example, encoded MPEG fragments, for incorporation into the frame layout from one or more sources including from a source content network and one or more source content servers.
  • the AEE may provide actual spatial data for the screen updates or the AEE may provide pointers or addresses to content that is to be combined with a frame layout.
  • Other examples of content that may be combined under the direction of the AEE is full-motion video such as MPEG2 or animated graphic elements which are encoded as MPEG2.
  • the AEE can embed said alpha channel information either as a coded frame which is then not directly displayed or as non-displaying coded information embedded in a portion of a frame (e.g., as an MPEG custom format).
  • the alpha mask information is extracted, by the client device, as illustrated in Figure 5, from the stream upon detection of an alpha channel mask identifier and an empty frame or empty macroblocks are substituted by the receiving software of the client prior to the decoding of said video information.
  • the application execution environment may include links to one or more graphical elements that may be either in the encoded domain (MPEG fragments i.e. groups of encoded macroblocks) or the spatial domain.
  • a screen update may be either a full frame or part of a frame.
  • An API can be used such as the OpenGL API, where scene information (including bitmaps) is exchanged.
  • the screen update may be in a format where an abstract scene description (application description) is passed to the modules along with references to bitmaps/encoded fragments and textures.
  • the audio encoder receives audio output provided by the executing application and encodes it according to an audio encoding format supported by the client device.
  • the audio format may be, for example, MP3, AAC, AC3, or others known in the art.
  • the audio encoder may be employed if the client device is capable of mixing audio into a video stream; if not, then the audio encoder is not employed, and all audio received by the client device from the content distribution platform occurs as an integral part of an audiovisual stream.
  • the image encoder receives screen updates provided by the executing application and encodes them according to an image encoding format supported by the client device.
  • the image format may be, for example, PNG, BMP, GIF, or JPG, or others known in the art.
  • the image encoder may be employed if the client device is capable of overlaying image graphics onto a video stream.
  • the images are directed to the client device through the network infrastructure.
  • the images are received by the client device, which combines them with decoded MPEG content so that the bitmap is used as an overlay, and blending may be efficiently done in the spatial domain.
  • the method comprises the steps of adding a tag, such as a URL or other means, for identification of graphic fragments to said fragments.
  • a tag such as a URL or other means
  • This enables the tracking of data relating to the frequency of use of a given fragment, and on this basis a certain priority can be given to a fragment which further determines how long said fragment will remain in said cache.
  • FurtheiTnore a method is provided for associating the data related to where said fragments are used on a client display in order to reuse said fragments correctly in other parts of the respective user interface of said client display.
  • systems for performing methods described herein include fast access memory, such as a cache memory for temporary storing of encoded fragments.
  • fast access memory such as a cache memory for temporary storing of encoded fragments.
  • the MPEG encoder receives both audio and screen updates provided by the executing application and encodes them according to an MPEG format into a stitchable MPEG format.
  • the MPEG encoder may be employed if the user has selected a mode in which a selected video is displayed on a partial screen only.
  • the architecture also includes a transcoder.
  • the transcoder receives audio and video content from the source content network, and transcodes it when the source content is in an audio or video format that is not supported by the client device. Once transcoded if required, the audiovisual content may be blended using a blender that receives graphical user interface audio and screen updates from the application execution environment.
  • the output of the transcoder and blender is also in a stitchable MPEG format.
  • the screen elements will be retrieved from a source (application execution environment or the content distribution network) and the screen elements may be transcoded into a stitchable MPEG element or resized for the frame layout (e.g., 480p to 200x200 pixels).
  • a source application execution environment or the content distribution network
  • the screen elements may be transcoded into a stitchable MPEG element or resized for the frame layout (e.g., 480p to 200x200 pixels).
  • a stitching module receives stitchable MPEG from the MPEG encoder and from the transcoder and blender, and stitches them into a standards-compliant MPEG transport stream.
  • the application changes the state of a button in response to a user input.
  • the graphical element for the changed state of the button will be provided to the stitching module if the graphic is already an MPEG fragment, or if the graphical element is spatially encoded, the graphical element will be encoded as an MPEG fragment and passed to the stitching module that stitches together the frame layout.
  • the MPEG fragments and the frame layout may be stitched together to form a complete MPEG frame in the stitching module.
  • the complete MPEG frames are packaged into an MPEG elementary stream and then into an MPEG transport stream container in a packaging and multiplexing stream module. More than one MPEG elementary stream may be multiplexed together and there may be multiple audio and/or video streams.
  • the graphical user interface elements that are to be placed on top of a video element can be sent either as an overlay graphic to the client (e.g., a bitmap) or the overlay graphical user interface elements can be blended with the video element in the transcoding and blending module.
  • the packaged MPEG transport stream (e.g., MPEG2 transport stream) is then sent through the network infrastructure to the client device.
  • Figs. 5A-5D show the functional blocks and data paths that may be used for each of the modes of operation.
  • Fig. 5 A shows the functional architecture for supporting mode 1, which is a partial screen video with a stitched graphical user interface.
  • Fig. 5B shows the functional architecture to support modes 2 and 5, which are the display of full-screen video with and without blended overlays.
  • Fig. 5C shows the functional architecture for mode 3, which is a full-screen pass through where encoded video content is repackaged and re- streamed.
  • Fig. 5D shows the functional architecture for supporting mode 4, which is a full screen transcode due to a decoding limitation of the client device.
  • Fig. 5 A is now described in more detail with respect to mode 1 and operation of its relevant functional blocks.
  • the user has requested an application that provides partial screen video with a stitched graphical user interface.
  • the application execution environment provides the graphical user interface, including screen updates and audio updates that make up the portion of the screen not occupied by the video, while the partial screen video itself is provided by the source content network.
  • the screen updates are either provided to the image encoder (if the client is capable of performing graphics overlay) or to the MPEG encoder (if the client is incapable of performing graphics overlay).
  • Video controls that overlay the video itself, for example to permit a trick-play mode may be provided by the application execution environment to the transcoder, and blended into the video received from the source content network.
  • the encoded screen updates and the transcoded and blended video are stitched together to form MPEG video frames, which are packaged, multiplexed with audio, and streamed to the client. If the client is capable of performing a graphics overlay function, then the transcoded and blended video is provided separately from screen updates to the graphical user interface.
  • Fig. 5B provides the functional architecture for providing full-screen video content to a requesting client device wherein the capabilities of the client device do not permit the client device to decode the content of the selected full-screen video in its native encoding scheme.
  • the control logic determines that either mode 2 or mode 5 is required.
  • the stitching components of the functional architecture of Fig. 5 are not used. Instead, the control logic evaluates whether selected video content to be displayed on the display device through the client device of the user is in a format that is compatible with the decoder of the client device.
  • the content is provided to either a blending module if the requested full screen video content is to have a graphical user overlay (e.g., trick -play, captions etc.) and then to a transcoding module, or the video content is provided directly to the transcoding module.
  • the transcoding module transcodes the full screen video content from the native format to a format that can be decoded by the requesting client device.
  • a user may use a device such as a remote control to request a graphical user interface for changing a parameter of the full-motion video.
  • the application execution environment will send screen update commands to the transcode and blend module.
  • the transcode and blend module will first decode the full-screen video into the spatial domain and will obtain the graphical elements for the graphical user interface overlay.
  • the transcode and blend module will then alpha blend the graphical elements and the portion of the underlying full-motion video and then the transcode and blend module will take the spatially encoded full-motion video with the graphical user interface overlaid and will encode the data using an encoding format that can be decoded by the client device.
  • the transcoded video data either with the overlay or without the overlay is presented to a repackaging, re- streaming, and re-multiplexing module.
  • the module will encapsulate the encoded video content using a container and transport protocol that can be transmitted through the network infrastructure and be extracted and decoded by the client device.
  • Fig. 5C shows the functional architecture wherein selected full-screen video content is passed through the system to the client device without transcoding.
  • the user selects full-screen playback content and the control logic determines that the selected video content is in an encoding format that can be decoded by the requesting client device.
  • the full-screen video does not need to be transcoded and can simply be passed to the repackaging, re-multiplexing, and re-streaming module.
  • the repackaging, re- multiplexing, and re-streaming module performs the functions as described with respect to Fig.
  • the full screen pass-through can also enable overlays where the client device is capable of receiving separate bitmap encoded data, JPEG, GIF or other data in an encoded data format for a graphical user interface element.
  • the control logic confirms whether the client device can add overlays before initiating this mode.
  • the full-screen video content can be passed through from the content distribution network to the client device, without transcoding the full-screen video content. The client device thus receives both the full-screen video content and any required overlays, removes the video content from its container, and decodes and displays the video content on the user's display device.
  • this can be further optimized. For example, in mode 6 of operation, if the client has the capability to parse the container format of the source video, then there is no need to repackage and this step can be omitted. The client then fetches the source video directly from the source content server. The client then resumes playing the source asset using the local video player.
  • HTTP Live Streaming HLS
  • the GUI decoding can be done with very shallow buffers, allowing a more responsive user interface with less latency.
  • Fig. 5D shows the functional architecture for supporting mode 4, which is a fullscreen transcode due to a decoding limitation of the client device.
  • mode 4 is a fullscreen transcode due to a decoding limitation of the client device.
  • the user selects to view a source video that is encoded using a codec that cannot be decoded by the client device.
  • the full-screen video must be transcoded, and cannot be simply passed to the repackaging, re-multiplexing, and re-streaming module. Therefore, the video is obtained from the source content network and transcoded in the transcoder.
  • the video may be blended with a graphical user interface overlay for GUI elements such as video controls (e.g., with start, stop, rewind, captions and etc.) and any associated sounds if the client is incapable of performing graphics overlay and/or audio mixing, or these may be transmitted separately to the client (if the client is capable of performing these functions).
  • GUI elements such as video controls (e.g., with start, stop, rewind, captions and etc.) and any associated sounds if the client is incapable of performing graphics overlay and/or audio mixing, or these may be transmitted separately to the client (if the client is capable of performing these functions).
  • Fig. 6 shows the source architecture with overlays showing structural elements.
  • the structural elements may include one or more hardware processors for performing the functions of the architecture.
  • the structural elements include an application engine, media farm, a compositor and a set-top box (i.e. a client device).
  • the application engine is a structural element that encapsulates the application execution environment, audio encoder, image encoder, and MPEG encoder. These functions are tightly coupled in that the outputs of the encoders are all encoded data that are used by other functional components in the system. These functions may be advantageously distributed among the one or more hardware processors in a parallel fashion to improve response time and maximize the use of the processors. As executing graphical applications and encoding are CPU-intensive operations, the application engine may include a great deal of computational power, with less emphasis on storage and input/output operations.
  • the media farm controls bulk expensive media operations, including transcoding and blending.
  • the media farm receives audio, video, and audiovisual content from the source content network and receives audio and screen updates from the application engine.
  • Transcoding and blending must be performed in real time, while screen updates may be pre- encoded, and much more data passes through the media farm than is generated by the application engine. Therefore, managing operation of the media farm structural element is different from managing the application engine, and requires more storage and network bandwidth.
  • the output of the media farm is stitchable MPEG.
  • the compositor receives stitchable MPEG from the application engine and the media farm, and stitches it together. Because the compositor outputs standards-compliant MPEG transport streams, it also includes the packager, multiplexer, and streamer. As with the other two structural elements, the compositor has its own unique responsibilities. All source video passes through the compositor, which must therefore have a great deal of network bandwidth available to it.
  • the client device may or may not include graphics overlay capabilities and audio mixing capabilities. However, it can decode video content according to at least one codec, for example MPEG-2. As described above, any given client device may decode a variety of video formats, and the network infrastructure connects the content distribution framework to a wide variety of client devices in a heterogeneous network.
  • the control logic in accordance with various embodiments of the invention is flexible enough to accommodate this variety.
  • the present invention may be embodied in many different forms, including, but in no way limited to, computer program logic for use with a processor (e.g., a microprocessor, microcontroller, digital signal processor, or general purpose computer), programmable logic for use with a programmable logic device (e.g., a Field Programmable Gate Array (FPGA) or other PLD), discrete components, integrated circuitry (e.g., an Application Specific
  • Integrated Circuit any other means including any combination thereof.
  • predominantly all of the reordering logic may be implemented as a set of computer program instructions that is converted into a computer executable form, stored as such in a computer readable medium, and executed by a microprocessor within the array under the control of an operating system.
  • Source code may include a series of computer program instructions implemented in any of various programming languages (e.g., an object code, an assembly language, or a high-level language such as Fortran, C, C++, JAVA, or HTML) for use with various operating systems or operating environments.
  • the source code may define and use various data structures and
  • the source code may be in a computer executable form (e.g., via an interpreter), or the source code may be converted (e.g., via a translator, assembler, or compiler) into a computer executable form.
  • the computer program may be fixed in any form (e.g., source code form, computer executable form, or an intermediate form) either permanently or transitorily in a tangible storage medium, such as a semiconductor memory device (e.g., a RAM, ROM, PROM, EEPROM, or Flash-Programmable RAM), a magnetic memory device (e.g., a diskette or fixed disk), an optical memory device (e.g., a CD-ROM), a PC card (e.g., PCMCIA card), or other memory device.
  • a semiconductor memory device e.g., a RAM, ROM, PROM, EEPROM, or Flash-Programmable RAM
  • a magnetic memory device e.g., a diskette or fixed disk
  • an optical memory device e.g., a CD-ROM
  • PC card e.g., PCMCIA card
  • the computer program may be fixed in any form in a signal that is transmittable to a computer using any of various communication technologies, including, but in no way limited to, analog technologies, digital technologies, optical technologies, wireless technologies, networking technologies, and internetworking technologies.
  • the computer program may be distributed in any form as a removable storage medium with accompanying printed or electronic documentation (e.g., shrink wrapped software or a magnetic tape), preloaded with a computer system (e.g., on system ROM or fixed disk), or distributed from a server or electronic bulletin board over the communication system (e.g., the Internet or World Wide Web.)
  • Hardware logic including programmable logic for use with a programmable logic device
  • implementing all or part of the functionality previously described herein may be designed using traditional manual methods, or may be designed, captured, simulated, or documented electronically using various tools, such as Computer Aided Design (CAD), a hardware description language (e.g., VHDL or AHDL), or a PLD programming language (e.g., PALASM, ABEL, or CUPL.)
  • CAD Computer Aided Design
  • a hardware description language e.g., VHDL or AHDL
  • PLD programming language e.g., PALASM, ABEL, or CUPL.
  • Embodiments of the present invention may be described, without limitation, by the following clauses. While these embodiments have been described in the clauses by process steps, an apparatus comprising a computer with associated display capable of executing the process steps in the clauses below is also included in the present invention. Likewise, a computer program product including computer executable instructions for executing the process steps in the clauses below and stored on a computer readable medium is included within the present invention.

Abstract

L'invention concerne un procédé de fourniture de contenu audiovisuel à un dispositif client configuré pour être relié à un écran. Le procédé consiste à détecter une sélection d'un élément graphique correspondant à un élément de contenu vidéo. En réponse à la détection de la sélection de l'élément graphique, un mode de transmission est déterminé. Le mode de transmission dépend : (i) d'une ou plusieurs capacités de décodage du dispositif client ; (ii) d'un format de codage vidéo de l'élément de contenu vidéo ; (ii) du fait que l'élément de contenu vidéo doive être affiché en format plein écran ou écran partiel ; et (iv) du fait que le dispositif client soit en mesure de superposer des données d'image dans un flux vidéo. Ensuite, les données audiovisuelles qui contiennent l'élément de contenu vidéo sont préparées en vue de leur transmission selon le mode de transmission déterminé. Enfin, les données audiovisuelles préparées sont transmises du serveur au dispositif client, selon le mode de transmission déterminé, en vue de leur affichage sur l'écran.
EP14722897.7A 2013-03-15 2014-03-17 Système à modes multiples et procédé de fourniture de contenu vidéo sélectionnable par un utilisateur Ceased EP2974322A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US201361793898P 2013-03-15 2013-03-15
PCT/US2014/030773 WO2014145921A1 (fr) 2013-03-15 2014-03-17 Système à modes multiples et procédé de fourniture de contenu vidéo sélectionnable par un utilisateur

Publications (1)

Publication Number Publication Date
EP2974322A1 true EP2974322A1 (fr) 2016-01-20

Family

ID=54842945

Family Applications (1)

Application Number Title Priority Date Filing Date
EP14722897.7A Ceased EP2974322A1 (fr) 2013-03-15 2014-03-17 Système à modes multiples et procédé de fourniture de contenu vidéo sélectionnable par un utilisateur

Country Status (1)

Country Link
EP (1) EP2974322A1 (fr)

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
None *
See also references of WO2014145921A1 *

Similar Documents

Publication Publication Date Title
US11073969B2 (en) Multiple-mode system and method for providing user selectable video content
US10200744B2 (en) Overlay rendering of user interface onto source video
KR102246002B1 (ko) 가상 현실 미디어 콘텐트의 스트리밍을 개선하는 방법, 디바이스, 및 컴퓨터 프로그램
US9712890B2 (en) Network video streaming with trick play based on separate trick play files
JP5795404B2 (ja) 被管理ネットワークを介したテレビ放送および非被管理ネットワークを介した双方向コンテンツのクライアントデバイスへの提供
US9021541B2 (en) Streaming digital video between video devices using a cable television system
KR101968070B1 (ko) 데이터를 스트리밍하기 위한 방법, 데이터를 제공하기 위한 방법, 데이터를 획득하기 위한 방법, 컴퓨터 판독 가능 저장 매체, 서버 장치, 및 클라이언트 장치
GB2506911A (en) Streaming data corresponding to divided image portions (tiles) via a description file including spatial and URL data
JPWO2019031469A1 (ja) 送信装置、送信方法、受信装置および受信方法
US11653054B2 (en) Method and apparatus for late binding in media content
US9648320B2 (en) Methods and systems for processing content
KR101944601B1 (ko) 기간들에 걸쳐 오브젝트들을 식별하기 위한 방법 및 이에 대응하는 디바이스
EP4128808A1 (fr) Appareil, procédé et programme informatique pour le codage et le décodage vidéo
EP2974322A1 (fr) Système à modes multiples et procédé de fourniture de contenu vidéo sélectionnable par un utilisateur
KR20180003608A (ko) 오디오-비디오 컨텐츠를 렌더링하는 방법, 이 방법을 구현하는 디코더 및 오디오-비디오 컨텐츠를 렌더링하는 렌더링 장치
Cardoso et al. Hyperlinked video with moving objects in digital television
EP3160156A1 (fr) Système, dispositif et procédé pour améliorer du contenu audio-vidéo à l'aide d'images d'application
Quax et al. A practical and scalable method for streaming omni-directional video to web users

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

17P Request for examination filed

Effective date: 20151015

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

AX Request for extension of the european patent

Extension state: BA ME

DAX Request for extension of the european patent (deleted)
17Q First examination report despatched

Effective date: 20160629

REG Reference to a national code

Ref country code: DE

Ref legal event code: R003

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE APPLICATION HAS BEEN REFUSED

18R Application refused

Effective date: 20190602