Classifications

• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
  • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
  • H04N21/83—Generation or processing of protective or descriptive data associated with content; Content structuring
  • H04N21/845—Structuring of content, e.g. decomposing content into time segments
  • H04N21/8456—Structuring of content, e.g. decomposing content into time segments by decomposing the content in the time domain, e.g. in time segments
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
  • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
  • H04L65/60—Network streaming of media packets
  • H04L65/65—Network streaming protocols, e.g. real-time transport protocol [RTP] or real-time control protocol [RTCP]
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
  • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
  • H04L65/60—Network streaming of media packets
  • H04L65/75—Media network packet handling
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
  • H04L67/00—Network arrangements or protocols for supporting network services or applications
  • H04L67/01—Protocols
  • H04L67/02—Protocols based on web technology, e.g. hypertext transfer protocol [HTTP]
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
  • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
  • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
  • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
  • H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
  • H04N21/23439—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements for generating different versions
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
  • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
  • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
  • H04N21/431—Generation of visual interfaces for content selection or interaction; Content or additional data rendering
  • H04N21/4312—Generation of visual interfaces for content selection or interaction; Content or additional data rendering involving specific graphical features, e.g. screen layout, special fonts or colors, blinking icons, highlights or animations
  • H04N21/4316—Generation of visual interfaces for content selection or interaction; Content or additional data rendering involving specific graphical features, e.g. screen layout, special fonts or colors, blinking icons, highlights or animations for displaying supplemental content in a region of the screen, e.g. an advertisement in a separate window
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
  • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
  • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
  • H04N21/431—Generation of visual interfaces for content selection or interaction; Content or additional data rendering
  • H04N21/4318—Generation of visual interfaces for content selection or interaction; Content or additional data rendering by altering the content in the rendering process, e.g. blanking, blurring or masking an image region
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
  • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
  • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
  • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
  • H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
  • H04N21/440281—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display by altering the temporal resolution, e.g. by frame skipping
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
  • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
  • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
  • H04N21/442—Monitoring of processes or resources, e.g. detecting the failure of a recording device, monitoring the downstream bandwidth, the number of times a movie has been viewed, the storage space available from the internal hard disk
  • H04N21/44213—Monitoring of end-user related data
  • H04N21/44222—Analytics of user selections, e.g. selection of programs or purchase activity
  • H04N21/44224—Monitoring of user activity on external systems, e.g. Internet browsing
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
  • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
  • H04N21/47—End-user applications
  • H04N21/482—End-user interface for program selection
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
  • H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client 
  • H04N21/63—Control signaling related to video distribution between client, server and network components; Network processes for video distribution between server and clients or between remote clients, e.g. transmitting basic layer and enhancement layers over different transmission paths, setting up a peer-to-peer communication via Internet between remote STB's; Communication protocols; Addressing
  • H04N21/637—Control signals issued by the client directed to the server or network components
  • H04N21/6373—Control signals issued by the client directed to the server or network components for rate control, e.g. request to the server to modify its transmission rate
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
  • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
  • H04N21/83—Generation or processing of protective or descriptive data associated with content; Content structuring
  • H04N21/845—Structuring of content, e.g. decomposing content into time segments
  • H04N21/8455—Structuring of content, e.g. decomposing content into time segments involving pointers to the content, e.g. pointers to the I-frames of the video stream

Definitions

• TITLE Restitution of content in the background or in the form of overlay as part of an adaptive progressive download of the HAS type
• the field of the invention is that of the broadcasting of digital multimedia content, namely digital audio and / or video content. More precisely, the invention relates to the restitution of such content in the background or in the form of an overlay of PiP (for English "Picture in Picture") type, in the context of the routing of digital content according to a technique known as adaptive progressive downloading, or HAS.
• PiP for English "Picture in Picture”
• local such as a home network.
• the terminal generally sends a request to a server, indicating the content chosen, and in return it receives a stream of digital data relating to this content.
• a request passes through the network access gateway, for example the residential gateway.
• the terminal is suitable for receiving this digital content in the form of multimedia data and for rendering it back.
• This restitution consists in providing the digital content at the terminal level in a form accessible to the user.
• data received corresponding to a video is generally decoded, then restored at the level of the terminal in the form of a display of the corresponding video with its associated soundtrack.
• the digital content will be assimilated to a video and the reproduction by the terminal, or consumption by the user of the terminal, to a display on the screen of the terminal.
• the distribution of digital content on the Internet is often based on client-server protocols of the HTTP family (standing for “Hyper Text Transport Protocol”).
• the progressive downloading of digital content also called streaming, makes it possible to transport and consume the data in real time, that is to say that the digital data is transmitted over the network and returned by the terminal as it goes. and as they arrive.
• the terminal receives and stores part of the digital data in a buffer memory before returning them.
• This distribution mode is particularly useful when the bit rate available to the user is not guaranteed for the real-time transfer of the video.
• Adaptive progressive downloading, in English HTTP Adaptive Streaming, abbreviated HAS also makes it possible to broadcast and receive data in different qualities corresponding for example to different bit rates.
• this description file makes it possible to select the right format for the content to be consumed as a function of the available bandwidth or the storage and decoding capacities of the client terminal.
• This type of technique makes it possible in particular to take account of the variations in bandwidth on the link between the client terminal and the content server.
• the MPEG-DASH standard (for English “Dynamic Adaptive Streaming over HTTP", in French “dynamic adaptive streaming over HTTP") is a standard for audiovisual broadcasting format on the Internet. It is based on the preparation of the content in different presentations of varying quality and speed, cut into short segments (of the order of a few seconds), also called “chunks”. Each of these segments is made available individually by means of an exchange protocol.
• the mainly targeted protocol is HTTP, but other protocols (eg FTP) can also be used.
• the organization of the segments and the associated parameters are published in a manifest in XML format.
• the principle underlying this standard is that the MPEG-DASH client makes an estimate of the bandwidth available for the reception of segments, and, depending on the filling of its reception buffer, chooses, for the next segment to load, a representation whose flow:
• the rendering terminal must therefore find a compromise between the overall quality of the content, and its uninterrupted reproduction, by carefully selecting the next segment to download, from among the various encoding rates offered. To do this, there are different algorithms for selecting the quality of the content as a function of the available bandwidth, which may have more or less aggressive, or more or less secure, strategies.
• HAS adaptive progressive download
• VOD video on demand
• Replay delayed broadcasting of television programs
• Network PVR type offers for “Network Personal Video Recorder”, ie a service
• real-time media streaming devices also access digital content in progressive adaptive download mode for real-time (or Live) television content. This is the case, for example, with the Chromecast ® device developed by Google ® , or the CléTV ® from Orange ® .
• Such devices are conventionally connected to the HDMI port of a television set and communicate, by Wi-Fi ® connection , with another device of the home communication network connected to a wide Internet-type communication network (residential gateway, computer, telephone smart device (smartphone, tablet, etc.), in order to reproduce, on the television, the multimedia content received by a compatible software application.
• a wide Internet-type communication network residential gateway, computer, telephone smart device (smartphone, tablet, etc.
• playback terminals which do not use this adaptive progressive downloading technique, but instead allow the user to access the content using a so-called “multicast” technology, or in French “grouped distribution”.
• multicast a set of receiving terminals interested in the same content form a broadcast group.
• the content server sends the data packets relating to this content only once, and they are routed to all the terminals of the broadcast group. Their replication takes place as close as possible to the receiving terminals. This technology therefore has little impact on end-to-end bandwidth consumption, because it is fully distributed.
• This multicast technology is used in particular by television systems over IP (in English "Internet Protocol") to broadcast the content of television channels to a large number of subscribers, most often through the use of set top boxes (in French “Adapter box”, or “decoder”).
• the corresponding playback terminals most often offer their users a user interface, or HMI, allowing them to access the main menu of the television over IP service, in which the stream of one of the television channels is displayed in the background.
• menu map or in the form of a PiP (for English "Picture in Picture") type overlay on the display terminal screen.
• This user interface which for example allows the user to view a video in the background of the main menu, or to view a secondary on-screen TV channel that plays a main TV channel, offers an experience and a service particularly appreciated by users. It also has little impact on the consumption of stream and bandwidth in the network, due to the multicast technology used.
• HAS type download because it is customary, with this type of service, to prevent the client from consuming bandwidth when it is not necessary. Indeed, broadcasting a stream in the background or in PiP in a menu is considered, in this type of “unicast” (or point to point) technology, as a waste of overall bandwidth, insofar as this stream of background, or overlay, is not the user's primary focus.
• the invention responds to this need by proposing a method for managing the adaptive progressive download (HAS) of digital content within a multimedia stream reader terminal in real time, associated with a terminal for rendering the digital content.
• a method comprises a step of obtaining a description file of the digital content, comprising a list of time segments of the content each associated with several encoding rates of the content.
• such a method implements during a concomitant rendering, by the rendering terminal, of the digital content and of an element, called the main element, belonging to the group comprising a menu and other digital content, a partial downloading of at least some of the time segments, and restitution of a decoded image from the downloaded part of the corresponding time segment.
• the invention is based on a completely new and inventive approach to adaptive progressive downloading of content within the framework of an OTT service, making it possible to offer the user an interface experience similar to that proposed in the framework of multicast content distribution techniques.
• the invention proposes to offer the user, on the rendering terminal, the concomitant rendering, on the one hand, of a main element, which can be content chosen by the user, or a menu. access to the service, and on the other hand, other content, for example a real-time video stream, which is offered in the background of the main element (for example with an opacity of 50%), or as an overlay on the screen (for example in a small window at the bottom left of the screen).
• a main element which can be content chosen by the user, or a menu.
• other content for example a real-time video stream, which is offered in the background of the main element (for example with an opacity of 50%), or as an overlay on the screen (for example in a small window at the bottom left of the screen).
• a time segment corresponds for example to a portion of the content of one or two seconds: it conventionally begins with an I image, also called an intra image. It then contains information making it possible to reconstruct, by motion estimation / compensation, the following images of the content, from this image I. This information relates to the parts of the image which change or evolve, with respect to the image I, for the duration of the time segment.
• the start of the chunk also called the introductory portion of the chunk, which makes it possible to decode the image I, or intra.
• This introductory portion of the time segment represents approximately 20% of the total segment size in bytes. After decoding of the image I, the latter is restored to the user throughout the duration of the time segment from which it is extracted.
• the image returned to the user in the background or in PiP is therefore refreshed approximately every 2 seconds, each time a new intra image is decoded, associated with a new one. time segment.
• This makes it possible to achieve an interesting compromise between a rich user interface, making it possible to display real-time content in the background or in overlay on the screen of the rendering terminal, and a limitation of the consumption of network resources by the server. content.
• the video segments broadcast in HAS are made up, like conventional videos, of GOP (for English "Group Of Pictures", in French “Group of Pictures”).
• a GOP consists of a series of images grouped together in an encoded video stream which is repeated periodically until the end of encoding. Visible images are generated from images contained in a GOP.
• Each HAS segment begins with a GOP which itself begins with an I frame (l-Frame, or internally coded frame).
• a GOP contains a single I image which corresponds to the first image to be encoded. It is followed by a succession of images P (images with predictive coding) and B (images with bidirectional predictive coding), the pattern of which is repeated until the end of the GOP.
• the P images are at regular intervals and the B images complete these intervals.
• I image is a reference image whose decoding does not depend on a previous or next image. It can be completely decoded
• the objective of an embodiment of the invention is therefore to recover the beginning, or the introductory portion, of the video segment, which makes it possible to extract the image I (in the form of a JPG for example).
• this partial downloading of the chunks takes place at the end of a first period of determined duration of absence of interaction of a user with the multimedia stream reader terminal in real time and / or with the return terminal.
• it is decided not to download all of the time segments, but only their introductory portion, when it is detected that the user has no longer interacted with the playback terminal or the multimedia stream reader terminal for a few moments (e.g. example a few minutes).
• the partial download (as opposed to the download of the entire segment), can be implemented: either when a concomitant restitution of a content and a main element is required, that is to say immediately as soon as the user accesses the service,
• the partial download relates to an introductory portion of at least some of said time segments; in this configuration, at the end of a second period of fixed duration of absence of user interaction with the multimedia stream reader terminal in real time and / or with the reproduction terminal, the image is decoded from the introductory portion of one of the time segments and is restored for a duration of at least two consecutive time segments.
• the consumption of bandwidth is thus further reduced, by displaying the intra decoded image from a time segment of 2 seconds for the duration of two consecutive time segments, ie 4 seconds, or even for 6 seconds or more.
• the partially downloaded time segments are associated with an encoding rate lower than an optimum encoding rate for restitution of the content.
• optimum encoding rate is meant here the maximum rate of a segment to be downloaded to which the user can claim depending on network conditions, according to the general principle of HAS downloading.
• the method according to one embodiment of the invention acts on the resolution and the temporal sampling of the stream, by only downloading the beginning of some of the time segments associated with reduced encoding rates, for example to offer the user a reproduction at 10 images per minute with a resolution of 576 pixels.
• a graphic transition between said decoded image and a previous decoded image improves the user's experience: in fact, the display in the background of the screen of real-time content of poor quality or resolution can be unattractive.
• the use of a technique of blurring the displayed image is a subterfuge to improve the rendering of the image for the client, who can furthermore better highlight and highlight the menu buttons, in the case where content is displayed in the background of service access control buttons.
• the method described above can also have an adaptive character, in the sense that its various aspects and characteristics can be activated gradually and successively, depending on the behavior of the user, and the duration of his period of time. 'inactivity or interaction with the terminal. For example, first of all, we start by displaying only the intra images; then, secondly, the encoding rate of the time segments of which the introductory portion is downloaded is reduced; finally, in a third step, the display duration of the intra decoded image is increased, therefore reducing the number of partially downloaded time segments, and for example graphic post-processing of the blur or opacity type is introduced. Different time counters can be implemented to determine the moment at which it is appropriate to add an additional measurement among those listed above, depending for example on the interactions detected by the user with the playback terminal or the reader terminal. multimedia stream.
• the decoded image prior to the restitution of the decoded image, it implements a determination of the chrominance of the decoded image and, if the determined chrominance is greater than a chrominance threshold, it implements the partial download. for at least one time segment adjacent to the time segment from which the decoded image is extracted, and a determination of the chrominance of at least one decoded image from said at least one neighboring time segment, called neighboring decoded image, and a restitution of one of the neighboring decoded images of which the determined chrominance is less than the chrominance threshold.
• the digital content is rendered, on a screen of the rendering terminal, in the background of the main element and / or in a window of the screen.
• the invention also relates to a computer program product comprising program code instructions for implementing a method as described above, when it is executed by a processor.
• the invention also relates to a recording medium readable by a computer on which is recorded a computer program comprising program code instructions for the execution of the steps of the adaptive progressive download management method according to the invention such as described above.
• Such a recording medium can be any entity or device capable of storing the program.
• the medium can comprise a storage means, such as a ROM, for example a CD ROM or a microelectronic circuit ROM, or else a magnetic recording means, for example a USB key or a hard disk.
• such a recording medium can be a transmissible medium such as an electrical or optical signal, which can be conveyed via an electrical or optical cable, by radio or by other means, so that the program computer it contains can be executed remotely.
• the program according to the invention can in particular be downloaded over a network, for example the Internet network.
• the recording medium can be an integrated circuit in which the program is incorporated, the circuit being adapted to execute or to be used in the execution of the aforementioned display control method.
• the invention also relates to a device for managing the adaptive progressive download (HAS) of digital content within a multimedia stream reader terminal in real time, associated with a terminal for rendering said digital content.
• a device for managing the adaptive progressive download (HAS) of digital content within a multimedia stream reader terminal in real time associated with a terminal for rendering said digital content.
• Such a device comprises a module for obtaining a description file of the digital content, comprising a list of time segments of the content each associated with several encoding rates of the content.
• such a device comprises a module for concomitant rendering of the digital content and an element, called the main element, belonging to the group comprising a menu and other digital content, and a downloading module configured to implement, during a concomitant restitution, a partial downloading of at least some of the time segments, and the restitution module is configured to restore a decoded image from the downloaded part of the corresponding time segment.
• the invention also relates to a real-time multimedia stream reader terminal, which comprises a device for managing the adaptive progressive download of digital content as described above.
• the adaptive download management device and the multimedia stream reader terminal have, in combination, all or part of the characteristics set out in the whole of this document in relation to the adaptive download management method.
• the adaptive download management device, the multimedia stream reader terminal and the corresponding computer program mentioned above have at least the same advantages as those conferred by the adaptive download management method according to the present invention.
• FIG 1 presents a progressive download architecture based on the use of adaptive streaming according to an embodiment of the invention
• FIG 2 shows an architecture of a multimedia stream reader terminal in real time according to one embodiment of the invention
• FIG 3 illustrates the user interface proposed according to one embodiment of the invention
• FIG. 4 presents a method allowing the display of the content C1 on the interface of FIG. 3 according to one embodiment of the invention.
• the general principle of the invention is based, in order to be able to offer the user an interface experience as rich as in the context of video services based on multicast technology, on an adaptation of the nature of the video stream to be displayed behind.
• plan or overlay as part of an OTT service based on an adaptive progressive download of the HAS type. More precisely, it is proposed to download only the start of the time segments representative of the flow to be displayed in the background or in PiP, in order to reduce the consumption of network flows, and not to restore, with a display refresh rate. adaptive, than the first frame decoded from this chunk start.
• FIG. 1 a progressive download architecture based on the use of adaptive streaming according to the invention.
• Terminal 3 for example an intelligent telephone of the “smartphone” type
• terminal 4 for example a laptop computer
• terminal 8 for example an HDMI key connected to a television 5
• LAN network local
• the context of the local network is given by way of example and could easily be transposed to a “best effort” type Internet network, a company network, and so on.
• a digital content server 2 is located according to this example in the wide area network (WAN, GSM, GSM, GSM, GSM, GSM, GSM, GSM, GSM
• the content server 2 receives, for example, digital television content channels originating from a broadcast television network, not shown, and / or videos on demand, and makes them available to the client terminals.
• Client terminals 3, 4 and 8 can enter into communication with content server 2 in order to receive one or more content (films, documentaries, advertising sequences, etc.).
• an adaptive progressive download technique in English “adaptive streaming”, abbreviated as HAS based on HTTP protocol.
• HAS adaptive progressive download technique
• This type of technique makes it possible in particular to offer a good quality of content to the user, taking into account the variations in bandwidth which may occur on the link between the client terminal 3, 4 and 8 and the service gateway 6, or between the latter and the content server 2.
• different qualities can be encoded for the same content of a channel, corresponding for example to different bit rates. More generally, we will speak of quality to refer to a certain resolution of the digital content (spatial, temporal resolution, level of quality associated with video and / or audio compression) with a certain bit rate. Each quality level is itself split on the content server into time segments (or
• URI Universal Resource Identifier
• All of these parameters are generally grouped together in a parameters file, called a description file. It will be noted that this parameter file can be a computer file or a set of descriptive information about the content, accessible at a certain address.
• Terminals 3, 4 and 8 have their own characteristics in terms of decoding, display, etc. In a progressive adaptive download context, they can adapt their requests to receive and decode the content requested by the user at the quality that best suits them. In our example, if the contents are available at 512 kb / s (kilobits per second) (Resolution 1, or level 1, denoted NI), 1024 kb / s (N2), 2048 kb / s (N3) speeds and the client terminal has a bandwidth of 3000 kb / s, it can request content at any rate below this limit, for example 2048 kb / s.
• the content number i with the quality j is denoted “Ci @ Nj” (for example the j-th quality level Nj described in the description file).
• the service gateway 6 is in this example a domestic gateway which ensures the routing of data between the wide area network 1 and the local network 10, manages the digital contents by ensuring in particular their reception from the network and their decoding thanks to the decoders that l 'it is assumed here that they are integrated into the gateway 6 or into the client terminals 3, 4 and 8.
• the decoders can be located elsewhere in the extended network 1 or local 10, in particular at the level of an element of STB type (of the 'English Set-Top-Box) (not shown) associated with a television set.
• the terminal 3, 4 or 8 firstly queries the service gateway 6 to obtain an address of the description document 7 of the desired content (for example, C1).
• the service gateway 6 responds by providing the terminal with the address of the description file 7.
• this file is a manifest type file according to the MPEG-DASH standard (denoted “C.mpd”) and we will refer indifferently, depending on the context, to the expression “description file” or “manifest”.
• this file can be retrieved directly from a local Internet server or external to the local network, or be already on the service gateway or on the terminal at the time of the request.
• MPD MPEG-DASH
• NI 512 kb / s
• N2 1024 kb / s
• N3 2048 kb / s
• the service gateway 6 proceeds to obtain the fragments via a download at these addresses. It will be noted that this download takes place here, traditionally, through an HTTP URL, but could also take place through a universal address (URI) describing another protocol (dvb: // my content segment for example).
• URI universal address
• the HDMI key 8 is connected to the television 5 by connection to the latter's HDMI port, and is used to reproduce, on the screen of the television 5, a television program broadcast live, which therefore has no by nature, no predetermined duration, nor end date.
• this television program is designated under the name of content C1.
• Such content C1 is described in a manifest file 7.
• the content C1 can also be a deferred television program, or a video on demand, or a personal video of the user, or any other multimedia content of determined duration, for which the invention s. 'also applies.
• the HDMI 8 key is connected via WiFi ® directly to the residential gateway 6.
• the HDMI 8 key could also be connected via WiFi ® to another nomadic device on the home network, for example to the laptop computer. 4 or smart phone 3, through which it could access the extended communication network 1.
• the HDMI 8 key can also be controlled by the user by means of the smart phone 3, on which a software application for controlling the HDMI 8 key is installed.
• the content fragments obtained by the residential gateway 6 are, for example, transmitted via WiFi® to the HDMI key 8, which controls their display on the television screen 5, for return to the user.
• the figure fig. 2 represents an architecture of a multimedia stream reader terminal in real time according to an embodiment of the invention, for example the HDMI key 8 of FIG. 1.
• the HDMI key 8 communicates with the local network 10 and the extended Internet network 1 via the WIFI module for local wireless communication with the residential gateway 6 or another communication terminal of the local network 10, for example the smart phone 3.
• the HDMI 8 key also includes a download module progressive adaptive HAS capable of requesting a progressive download of one of the contents at one of the qualities proposed in a description file 7. This description file 7 can be recorded for example in the memories M of the HDMI key 8 or be find outside.
• the HDMI 8 key also includes a module for progressive adaptation of the quality of ADAP_Q content, which controls the HAS adaptive progressive download module.
• This ADAP_Q module commands the adaptive progressive download module HAS to download only certain portions of time segments, ideally the introductory portions of these segments, in order to extract therefrom the first image representative of the segment, or intra image. In addition, it can force the latter to request a progressive download of these segments at a quality, proposed in the description file 7, lower than the optimum quality to which it could claim. It can also select only some of the time segments to be partially downloaded, according to the refresh rate of the display of the desired intra images, in particular according to the interactions detected by the user with the HDMI 8 key or the playback terminal, for example TV 5.
• the HDMI 8 key also includes a DEC module for decoding intra images from the preliminary portions of downloaded time segments, which is also able to perform processing on these decoded images, such as a calculation of the chrominance of these images, for decide on their representativeness of the content, and various graphic post-processing, such as an application of a blur, an opacity, or a fade.
• DEC module for decoding intra images from the preliminary portions of downloaded time segments, which is also able to perform processing on these decoded images, such as a calculation of the chrominance of these images, for decide on their representativeness of the content, and various graphic post-processing, such as an application of a blur, an opacity, or a fade.
• the HDMI key 8 can also contain other modules such as a hard disk (not shown) for storing video fragments, a content access control module, a module for processing commands received from a tablet or a smartphone on which the application for controlling the HDMI key is installed, thanks to which the user can control its operation, etc.
• a hard disk not shown
• a content access control module a module for processing commands received from a tablet or a smartphone on which the application for controlling the HDMI key is installed, thanks to which the user can control its operation, etc.
• such an HDMI 8 key does not generally contain an I / O interface module, and it is the I / O interface module of the user's smart phone or tablet that is used by the latter. to choose for example its content.
• the HDMI Key 8 also integrates an INT module for managing the interface with the television 5, by which it obtains, for example, information on any user interactions with the television 5 (action on the television remote control for example by pressing the volume key or a channel change key), and by which it can control, for example, the simultaneous display on the television screen 5 of a menu and real-time content in the background, or in a PiP-type window, in pop-up, for example in the bottom left corner of the screen.
• an INT module for managing the interface with the television 5, by which it obtains, for example, information on any user interactions with the television 5 (action on the television remote control for example by pressing the volume key or a channel change key), and by which it can control, for example, the simultaneous display on the television screen 5 of a menu and real-time content in the background, or in a PiP-type window, in pop-up, for example in the bottom left corner of the screen.
• module can correspond just as well to a software component as to a hardware component or a set of hardware and software components, a software component itself corresponding to one or more computer programs or subroutines or more generally to any element of a program capable of implementing a function or a set of functions as described for the modules concerned.
• a hardware component corresponds to any element of a hardware set (or hardware) capable of implementing a function or a set of functions for the module concerned (integrated circuit, smart card, memory card, etc. .).
• HDMI 8 key comprises a random access memory (for example a RAM memory), a processing unit equipped for example with a processor CPU, and controlled by a computer program, representative of the progressive download management module.
• adaptive HAS stored in read only memory (eg ROM or hard disk).
• the code instructions of the computer program are for example loaded into the random access memory before being executed by the processor CPU of the processing unit.
• the random access memory contains in particular the description file manifest 7.
• the processor of the processing unit controls the evaluation of the presence or absence.
• the HDMI key 8 or the television 5 the display of content in the background or overlay on the television screen 5, any graphic post-processing applied to it before display, and the progressive reduction of the quality and the refresh rate of the display of the downloaded content, through the choice of the time segments and the associated encoding rates to download, the decoding of the intra images which are extracted therefrom, and 'issuance of corresponding commands to the HAS client module.
• Figure Fig. 2 only illustrates one particular way, among several possible, of making the HDMI key 8, so that it carries out the steps of the method detailed below, in relation to FIG. 4 (in any of the different embodiments, or in a combination of these embodiments). In fact, these steps can be carried out either on a reprogrammable computing machine (a PC computer, a DSP processor or a microcontroller) executing a program comprising a sequence of instructions, or on a dedicated computing machine (for example a set of logic gates such as an FPGA or ASIC, or any other hardware module).
• a reprogrammable computing machine a PC computer, a DSP processor or a microcontroller
• a program comprising a sequence of instructions
• a dedicated computing machine for example a set of logic gates such as an FPGA or ASIC, or any other hardware module.
• Figure Fig. 3 provides an overview of the user interface proposed according to one embodiment of the invention, for example on the screen of the television 5.
• a main menu also called a “desk” as part of the service offered by Orange ® .
• the television screen 5 displays only a set of buttons M, allowing him to access the various functionalities of the service (choice of content, adjustment of the noise level, etc.).
• the real-time content C1 is also displayed in the background on the screen, in superposition (“overlay”) with the buttons M of the main menu, for example with an opacity by 50%.
• this content C1 can be displayed, not in the background, but in a dedicated window on the screen, in the form
• This PiP type inlay technique can also be used, according to one embodiment of the invention, to allow the user to view C2 content on the screen, in high resolution, while keeping an eye on a device.
• content C1 displayed in a window of reduced size in a corner of the screen, for example to allow it to detect the start of a specific television program on another channel.
• the flowchart in FIG. 4 details more specifically the method allowing the display of this content C1 according to one embodiment of the invention.
• This method is implemented by a real-time multimedia stream reader terminal 30, for example the HDMI key 8 in FIGS. 1 and fig. 2.
• the real-time multimedia stream reader terminal 30 is connected to a television set TV 5, via an HDMI link. It comprises an INT module for interfacing with the television set 301, an ADAP_Q 302 module for progressive adaptation of the quality of content rendered in the background or in PiP, and a HAS client module 303.
• a HAS content server 2 exposes a video C1 in the form of “chunks” C1 j @Nj encoded at different encoding rates Nj, where the index i designates a temporal identifier of the “chunk” C1 j @Nj.
• a HAS client module is responsible for coming to retrieve these “chunks” from the HAS content server 2 by choosing the video quality Nj according to the available network resource.
• the way in which the HAS client module chooses the encoding rate of the next video fragment to download is not described in more detail here: there are indeed many algorithms allowing this choice to be made, the strategies of which are more or less secure. or aggressive.
• the general principle of such algorithms is based on the downloading of a first fragment at the lowest encoding rate proposed in the manifesto, and on the evaluation of the recovery time of this first fragment. . Based on this, the HAS client module evaluates whether, depending on the size of the fragment and the time taken to retrieve it, the network conditions allow the next fragment to be downloaded at a higher encoding rate.
• Some algorithms are based on a gradual increase in the quality level of the downloaded content fragments; others propose more risky approaches, with jumps in the levels of the encoding rates of successive fragments.
• the HAS client module In the classic case, if a video "chunk" lasts 3 seconds, the recovery of the "chunk” by the HAS client module must not exceed 3 seconds, in order to allow a restitution without interruption of the content by the terminal 30. It is therefore necessary for the HAS client module to operate the best compromise between a quality of reproduction, and therefore an encoding rate, as high as possible, and the download time of the fragment, which must be low enough to allow continuous playback on the television 5.
• the HAS client module 303 does not control the downloading of the entire fragment, nor at the optimal encoding rate, in order to optimize the consumption of network resources, in particular in the case where it is detected that the The user does not interact with the playback terminal 5, or the HDMI 8 key.
• the HAS module 303 recovers the manifest file 7 in order to discover the available fragments of the video content C1, and the various associated video qualities Nj.
• the HAS 303 module downloads, for example, successive fragments Cl ⁇ Nl (i.e. the first temporal fragment at an encoding rate of 500 kb / s) , then C1 2 @ N3 (i.e. the second temporal fragment at an encoding rate of 2000 kb / s), then C1 3 @ N3 (i.e. the third temporal fragment at an encoding rate of 2000 kb / s),. ..
• the algorithm implemented by the HAS 303 client module to determine which fragment at which encoding rate must be downloaded in normal operating mode can be one of the algorithms already in existence of the prior art. This algorithm will therefore not be described in more detail here.
• the various fragments downloaded by the HAS client module 303 are transmitted to the interface module INT 301 for their return to the user on the screen of the television set TV 5.
• the INT module 301 monitors the user's interactions with the television 5, and receives for example, via the HDMI port, information relating to the actions performed by the user on the remote control on TV 5 and on TV 5 itself. It transmits this information to the ADAP_Q module 302, which also receives information relating to any interactions of the user with the terminal 30 itself, via its remote control or a control terminal of the smartphone or tablet type.
• the ADAP_Q 302 module initiates for example a counter, or "timer", for example of a duration of ten or fifteen minutes, during the activation of the HDMI 8 key and of the television 5.
• the INT 301 module monitors user activity. If, at the expiration of this counter, no user interaction with the content restitution equipment has been detected, the ADAP_Q module 302 initiates the adaptation of the nature of the video stream C1, in order to reduce consumption network resources.
• this adaptation of the nature of the video stream can be implemented immediately, as soon as the user accesses the service, regardless of whether or not he interacts with the terminals.
• this adaptation consists, for the ADAP_Q 302 module, in ordering the HAS 303 download module to download only the introductory portions of the time segments Cl j @Nj, for example only 20% of the size of the chunk in bytes , corresponding to the start of the time segment, making it possible to decode the intra I image that it contains.
• the ADAP_Q 302 module can decide to download portions of time segments corresponding, in volume, to 10% of the main stream of the content Cl.
• the ADAP_Q 302 module implements an adaptive algorithm making it possible to determine the optimal portion of chunk to download, ie the portion of the chunk necessary and sufficient for the extraction of the Intra image.
• the ADAP_Q 302 module can, on a first time segment, initialize at 20% the portion of the segment to be downloaded.
• the ADAP_Q 302 module can incrementally reduce the portion to download for the next segment, for example by reducing by 1% the amount of data to be recovered on this next segment, i.e. 19%. If the extraction of the intra image from this 19% portion is successful, the ADAP_Q 302 module can continue to reduce the portion of the next segment to download, incrementally, until the image I extracted from this portion is not complete, because the ADAP_Q 302 module did not retrieve enough bytes from the start of the segment. The ADAP_Q 302 module then returns to the previous value, and freezes this value as being the optimal value, that is to say necessary and sufficient, to completely extract the Intra image of the segment.
• the ADAP_Q module 302 then initiates, for example, a second counter, and the INT module 301 continues to monitor the possible actions of the user.
• the ADAP_Q 302 module controls the restitution, in the background of the screen of the television 5, of an intra I image extracted from the segment Cl j @Nj for the entire duration of this segment , for example two seconds. Every two seconds, the display is refreshed, offering the user the intra I image extracted from the various successive time segments.
• the user is therefore not allowed to view the real video stream associated with the content C1, but an interface is offered to him allowing him to have an evolving overview of this content.
• This second counter can also be independent of the detection or not of user interactions with the terminals, and serve only to trigger a new content adaptation step.
• the ADAP_Q module 302 can decide on a new adaptation of the quality of the reproduction of the content C1 to the user.
• this adaptation may consist of no longer displaying one intra I image per time segment, but only downloading the start of one out of two time segments or the start of one out of three time segments, in order to display an intra I image. , no longer for the duration of one but of two or three successive segments, that is to say for four or six seconds. This reduces the refresh rate of the display of the content in the background or overlay on the screen of the television 5, and therefore the consumption of network resources.
• the DEC 304 module Before the INT 301 module controls the display of the decoded I picture from one of the time segments on the television screen 5, the DEC 304 module performs a chrominance calculation on the decoded intra picture .
• a threshold is determined, for example of the order of 50%. If the calculated chroma value for the image is less than the 50% threshold, the image contains little color information, and is therefore too dark to be effectively representative of the content. It is therefore not displayed on the TV screen 5.
• the ADAP_Q module 302 commands the HAS module 303 to download the start of one or two time segment (s) preceding and / or following the chunk from which the too dark image was extracted. Corresponding intra images are extracted from these partially downloaded time segments.
• the DEC 304 module calculates their chrominance, to determine which of these images is the most representative of the content C1, and which should therefore be displayed for better rendering to the user. More simply, the module DEC 304 successively calculates the chrominance of these different images, and as soon as it identifies an image whose chrominance value is satisfactory to faithfully represent the content C1, the INT module 301 controls the display of this image on the TV screen 5.
• a third counter is triggered during this adaptation of the refresh rate and that a new change is decided by the ADAP_Q 302 module at the end of this third "timer" if the user does not interact. still not with return terminals.
• This new development may for example consist of a reduction in the encoding rate of the time segments of the content C1 partially downloaded by the HAS module 303.
• the HAS module 303 changes from downloading preliminary portions of time segments to 3000 kb / s a download of introductory portions of time segments at 500kb / s.
• the HAS 303 client module will successively download (312) the beginnings of the fragments C1 2 @ N4, C1 3 @ N3, C1 4 @ N2 and C1 5 @ N1, to reduce the quality of reproduction of the content C1, as illustrated by the arrow referenced 311, down to the lowest quality proposed in manifesto 7, corresponding to a bit rate of 500 kb / s.
• the reduction 311 of the encoding rates of the temporal fragments is not necessarily so rapid, but can be spread out over time, for example over a period of one hour.
• the figure fig. 4 therefore corresponds to a simplified illustration of this principle, making it possible to apprehend it visually.
• the HAS 303 client module controls the partial download of successive time fragments at quality level N4 for a period of fifteen minutes, then lowers this level by downloading the start of the time fragments at quality level N3 for the quarter of an hour which follows, and so on, until the lowest NI quality level expected in manifest 7 is reached after about an hour.
• the ADAP_Q module 302 can also control the implementation of graphic post-processing of the intra I images decoded and extracted from the time segments by the DEC 304 module. For example, a blur can be applied to the intra I image restored backwards. shot to the user on the television screen 5. This is particularly advantageous when the resolution of the content C1 is reduced, in order to improve the rendering and the user experience. This technique also makes it possible to better highlight the buttons M of the main menu superimposed on the content C1, and therefore to draw the user's attention to them.
• Other graphics processing techniques can also be implemented, such as the application of a fade black, or the realization of a fade between the successive intra I images displayed, in order to increase the user's experience. that it is in the presence of a real video stream.
• the INT 301 module detects that the user manifests his presence by operating the remote control of the television 5, or by pressing the smartphone 3 controlling the terminal 30, the ADAP_Q 302 module can send the HAS client module 303 a order back to normal. If the user has for example chosen, via one of the buttons in the menu M, to view the real-time content C1 in full screen, the HAS client module 303 then determines, depending on the network conditions, the optimal bit rate of the time fragment which it can claim, and starts downloading the next fragment at this optimal speed.

Landscapes

• Engineering & Computer Science (AREA)
• Multimedia (AREA)
• Signal Processing (AREA)
• Computer Networks & Wireless Communication (AREA)
• Health & Medical Sciences (AREA)
• General Health & Medical Sciences (AREA)
• Social Psychology (AREA)
• Business, Economics & Management (AREA)
• Marketing (AREA)
• Databases & Information Systems (AREA)
• Human Computer Interaction (AREA)
• Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)

Applications Claiming Priority (2)

Publications (1)

Family

ID=67957059

Family Applications (1)

Country Status (4)

Family Cites Families (7)

• 2019
  • 2019-05-20 FR FR1905266A patent/FR3096540A1/fr active Pending
• 2020
  • 2020-05-12 EP EP20723886.6A patent/EP3973714A1/de active Pending
  • 2020-05-12 US US17/612,814 patent/US11736548B2/en active Active
  • 2020-05-12 WO PCT/EP2020/063109 patent/WO2020234030A1/fr unknown

Also Published As

Similar Documents

Legal Events