FR3013932A1 - - Google Patents

Abstract

A video encoder includes: a segment generation module (202) arranged to generate a first video segment sequence (SEQ (1)) based on a first video stream (EVS (1) ) encoded at a first bit rate and a second video segment sequence (SEQ (2)) based on a second video stream (EVS (2)) encoded at a second bit rate different from the first bit rate; and a segment transmission module (204) arranged to receive, from a video decoder, a segment request signal (REQ) identifying a selected video segment (SEG) of the first or second sequence segments; generate forward error correction code for the selected video segment; and transmit the selected video segment and the generated forward error correction code.

Description

The present disclosure relates to the field of video transmissions, and in particular a video encoder for encoding a video stream and a video decoder for reconstructing the video stream. BACKGROUND It is known to transmit a video stream comprising frames of video data over data networks such as the internet and / or one or more local area networks using transport layer protocols such as TCP ( transmission control protocol). Such a protocol ensures reliable transmission of data packets by detecting - packet loss, and requesting retransmission of these packets. The actual transmission bit rate obtained using the TCP protocol, in other words the rate at which data is successfully transmitted over the network, is a function of the transmission bit rate, the latency between the data points and the data rate. In some cases, such as when there is a relatively high rate of packet loss and / or latency in the network, this actual bit rate can become very bad. There is therefore a need in the art for a video transmission system and method that provides improved effective baud rate under at least some network conditions.

SUMMARY An object of embodiments of the present invention is to at least partially solve one or more needs of the prior art: According to one aspect, there is provided a video encoder comprising: a segment generation module arranged to generate a first video segment sequence based on a first video stream encoded at a first bit rate and a second video segment sequence based on a second video stream encoded at a second bit rate different from the first bit rate and a segment transmission module arranged to receive, from a video decoder, a segment request signal identifying a selected video segment of the first or second sequence of segments; generate forward error correction code for the selected video segment; and transmit the selected video segment and the generated forward error correction code. According to one embodiment, the segment request signal indicates a forward error correction decoding level to be used for the selected video segment, and the segment transmission module is arranged to generate the error correction code. directly on the basis of said level. According to one embodiment, the segment transmission module is arranged to generate the forward error correction code with a number of bits based on said level. According to one embodiment, the segment request signal indicates a transmission bit rate to be used for the transmission of the selected video segment, and the segment transmission module is arranged to transmit the selected video segment to the bit rate. indicated. According to one embodiment, the segment transmission module is arranged to separate the selected video segment into a plurality of data packets, and the transmission of the selected video segment to the transmission bit rate comprises a transmission of each packet of data. the plurality of data packets at the transmission bit rate. According to one embodiment, the first and second coded video streams are generated based on the same initial video stream. According to one embodiment, the video segments of the first sequence of video segments correspond to time periods of the initial video stream, and the video segments of the second sequence of video segments correspond to the same time periods of the stream. initial video. In another aspect, there is provided a video decoder 20 comprising: a segment receiving module arranged to at least partially reconstruct a video segment based on a received data block including segment data encoded at a first rate binary and direct error correction code; and a .section request module arranged to generate, on the basis of the reconstruction of one or more previous video segments, a request for another video segment encoded at a second selected bit rate different from the first one. bit rate. According to one embodiment, the segment data comprises a plurality of data packets, and the second bit rate is selected based on a loss rate of the data packets. According to one embodiment, the request further indicates a forward error correction level to be used for the other video segment. In one embodiment, the segment data includes a plurality of data packets. , and the level of, forward error correction is selected based on a loss rate of the data packets.

According to one embodiment, the video decoder further comprises a video segment buffer arranged to store segments successfully received by the segment receiving module, the request further indicating a transmission bit rate to be used for transmission. of the other video segment, the transmission bit rate being selected based on the amount of data stored in the video segment buffer. In another aspect, there is provided a video transmission system comprising: the aforementioned video encoder; the aforementioned video decoder; and one or more intermediate networks, interconnecting the video encoder and the video decoder, through which the selected video segment is transmitted by the video encoder and the data block is received by the video decoder. In another aspect, there is provided a video encoding method comprising: generating, by a segment generation module, a first video segment sequence based on a first video stream encoded at a first bit rate and 4 a second sequence of video segments based on a second. coded video stream at a second bit rate different from the first bit rate; receiving, by a segment transmission module from a video decoder, a segment request signal identifying a selected video segment of the first or second segment sequence; generating, by the segment transmission module, the forward error correction code for the selected video segment; and transmitting, by the segment transmission module, the selected video segment and the generated forward error correction code.

B13020 In another aspect, there is provided a video decoding method comprising: at least partially reconstructing, by a segment receiving module, a video segment based on a received data block comprising coded segment data at a first bit rate and the forward error correction code; and generating, by a segment request module based on the reconstruction, one or more previous video segments, a request for another video segment encoded at a second selected bit rate different from the first bit rate. BRIEF DESCRIPTION OF THE DRAWINGS The above-mentioned and other features and advantages will become apparent from the following detailed description of embodiments, given by way of illustration and without limitation, with reference to the accompanying drawings, in which: Figure 1 schematically illustrates a video transmission system according to an exemplary embodiment of the present description; Fig. 2 schematically illustrates in more detail a video encoder of the video transmission system of Fig. 1 according to an exemplary embodiment of the present disclosure; Figure 3 shows schematically video segments stored in a video encoder file system of Figure 2 according to an exemplary embodiment of the present description; FIG. 4 schematically illustrates a direct error correction code (FEC) generation circuit according to an exemplary embodiment of the present description; Fig. 5 schematically illustrates in more detail a video decoder of the video transmission system of Fig. 1 according to an exemplary embodiment of the present disclosure; and FIG. 6 schematically illustrates a device for implementing in software at least some parts of the video encoder or decoder of the video transmission system of FIG. 1 according to an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION FIG. 1 illustrates a video transmission system 100 for transmitting a video stream from a TX transmit side to a receive side RX. As illustrated, the video stream (VIDEO) is for example received by a video encoder 102 (ENCODER) on the transmitter side, which encodes the video stream and generates data packets for transmission to a video decoder 104 ( DECODER) on the receiver side, via one or more intermediate networks 106. The intermediate network or networks 106 may comprise, for example, one or more packet-switched networks such as the Internet network, wired or wireless local networks, etc., telecommunications networks, and / or one or more satellite data transmissions. The decoder 104 decodes the data packets to reconstruct the video stream. A return path is illustrated from decoder 104 to encoder 102 through intermediate network (s) 106 to provide control data described in more detail below. The reconstructed video stream (VIDEO) may be substantially identical to the original video stream encoded by the encoder 102, but will generally be modified to some extent by the encoding and decoding operation.

For example, in one embodiment, the video transmission system of FIG. 1 is used for transmitting, live and / or over the Internet, live video from a remote location, such as an event. to a control center where the video can be processed and broadcast. Figure 2 illustrates schematically in more detail the encoder 102 of Figure 1 according to an example embodiment. The encoder 102 comprises a segment generation module 202, and a segment transmission module 204. The segment generation module 202 uses, for example, a dynamic adaptive streaming (DAS) function. For this, the module 202 comprises N video encoders (ENCODER 1 to ENCODER N) arranged in parallel, of which three are illustrated in the example of Figure 2, denoted 206; 208 and 210. In alternative embodiments, N could take any value between 2 and several hundred. Each of the N encoders receives the same incoming video stream (VIDEO) via an input line 211, and encodes the incoming stream at a different bit rate, to generate EVS encoded video streams (1). at EVS (N) respectively. In particular, the N video encoders use for example the same video coding standard for encoding the video stream, but generate video streams having for example different image sizes, different frame rates and / or different metadata, to produce video streams encoded with different bit rates. In some embodiments, two or more N encoders may generate encoded video streams having substantially the same bit rate but which are differentiated by other characteristics such as audio language and / or audio. presence of subtitles, etc. For example, the N video encoders can encode the video stream according to the ISO H264 MPEG4 AVC standard, or the MPEG HEVC (High Efficiency Video Coding) standard, referenced ISO / IEC 25 23008-2. The specification describing the HEVC standard is for example published under the title "High Efficiency Video Coding (HEVC) text specifipation draft 10 (for FDIS & Consent)", B.Bross et al., Joint Collaborative Team on Video Coding (JCTVC) , document JCTVC-K1003, 12th Meeting: Geneva, CH, 14-23 January 30, 2013, the contents of which form part of the present description to the extent permitted by law. A document giving an overview of this standard is titled "Overview of the High Efficiency Video Coding (HEVC) Standard", G.Suilivan et al., Pre-publication draft to appear in IEEE Trans. the content of which forms part of the present description within the limits where this is permitted by law. The N EVS encoded video streams (1) to EVS (N) are respectively provided to N segmentation devices (SEGMENTER 1 to 5 SEGMENTER N), three of which are referenced 212, 214 and 216, are illustrated in FIG. segmentation devices 212 to 216 separate the corresponding video stream into video segments to form segment sequences SEQ (1) to SEQ (N). Each segment contains the video data for a time period of, for example, between 1 and 10 seconds, for example 5 seconds, of the original video stream. In addition, the N segmentation devices 212 to 216, for example, have a synchronized operation so that each generates segments based on the same time periods of the original video stream. For example, each of the N segmentation devices generates a first segment corresponding to a first time period tpl of the video stream, which is for example the period from 0 to 5 seconds, then a second segment corresponding to a second time period tp2 of the stream of video, which is for example the period of 5 to 10 seconds, and so on, until the video transmission operation is stopped. The video stream on the 211 line, the EVS (1) to EVS (N) coded video streams, and the video data of each video segment may comprise video alone, or a combination of video and video. other data such as one or more audio streams, text such as subtitles, and / or other useful data. In the example of Fig. 2, the video stream on the input line 211 includes only video, and each of the N segmentation devices 212-216 receives a separate stream of audio and / or video data. other data (AUDIO / OTHER DATA) associated with the video stream and coded by encoder 218 (ENCODER). Audio data can include a single audio stream, or multiple streams of audio in different languages. The other 35 data may contain text as B13020 subtitles in. one or more languages. Segmentation devices 212 to 216, for example, also separate this coded stream of audio data and / or other data into segments to be combined with the corresponding video segments.

Sequences of video segments SEQ (1) to SEQ (N) from segmentation devices 212 to 216 are for example supplied to a file system 220 (FILE SYSTEM) and stored therein. The file system 220, for example, buffers the segment sequences so that each segment remains available in the buffer for a minimum time interval. For example, the file system 220 is capable of storing the segments of each sequence corresponding to a duration of between 20 seconds and several minutes of the original video stream.

The file system 220 provides, in response to a selection signal, selected segments to the segment transmission module 204. In particular, each selected segment is for example supplied to a packet builder 222 of the segment transmission module 204, which separates the segment into data packets, and provides these data packets to an output buffer 224, through a circuit 226 for adding forward error correction (FEC) code. In particular, the data packets are provided to an FEC code generation module 226 (FEC GEN), which generates FEC code based on the video data of each segment. The FEC code and the segment packets are then stored in the output buffer 224, which is for example capable of storing a plurality of segments that are ready for transmission. Each segment is then transmitted, via a transmission interface 230 (TRANSMISSION INTERFACE), from the output buffer 224 to the video decoder 104. For example, the transmission interface operates in accordance with the user datagram protocol (UDP). In some embodiments, the FEC code is integrated with the packets of each video segment, and transmitted through the transmission interface 230 using the same output port as the video segments. In alternative embodiments, the FEC code has the form of a separate packet stream, which is stored in the output buffer 224 in a separate queue of the video segment packets, and these packets, FEC are by example transmitted by the transmission interface through one or more output ports separate from that used for the transmission of video packets.

The segment transmission module 204 further comprises an input 232 for receiving, from the video decoder 104, a request signal REQ comprising segment selection information identifying a segment to be transmitted to the decoder. In particular, the request signal REQ for example indicates the time of the segment, in other words the moment in the video stream, and a video quality level, in other words, the coding bit rate. For example, the video quality level is simply indicated by an identification in the request signal of the sequence of segments of the requested segment. In some embodiments, the request signal may further indicate another feature of the requested segment, such as the language of the audio or the presence or absence of subtitles. This segment request is processed by a request server 234 (REQUEST SERVER) of the segment transmission module 204, which is for example a TCP server, the request being received via a TOP interface. The request server 234 in turn provides the file system 220 with an appropriate read instruction so that the selected segment is provided to the packet constructor 222. In some embodiments, the request may identify a plurality of segments to be transmitted, and these requests can be pipelined in the segment transmission module 204.

B13020 The request signal REQ from the decoder also indicates, for example, a level of FEC coding to be used for the transmission of the segment, and as represented by an arrow 236 in FIG. 2, the FEC generation module 226 is controlled accordingly. to adapt its FEC coding level. In addition, the request signal REQ also indicates, for example, a transmission bit rate with which the segment is to be transmitted, and as represented by an arrow 238 in FIG. 2, the transmission interface 230 is consequently controlled for adapt the transmission bit rate. FIG. 3 diagrammatically represents the segment data stored in the file system 220 of FIG. 2 according to an exemplary embodiment. In the example of FIG. 3, the file system 220 stores, for each of the 15 segment sequences SEQ (1) to SEQ (N) corresponding to the rows in FIG. 3, video segments 302 corresponding to time periods tpl at tp (m) of the video stream corresponding to the columns in FIG. 3. The segments of the sequence SEQ (1) have for example the highest bit rate, and the segments of the other sequences have progressively lower bit rates up to to the N sequence, which has segments having the lowest bit rate. The request received by the request server 232 of FIG. 2 for example identifies the time period of the segment 25 to be transmitted, for example one of the time periods tpl at tp (m) in FIG. 3, and also the bit rate of the segment to transmit, in other words by identifying one of the sequences SEQ (1) to SEQ (N). Fig. 4 schematically illustrates the FEC code generation module 226 of Fig. 2 according to an exemplary embodiment. For example, the FEC code is generated as described in SMPTE 2022-1-2007 of the Society of Motion Picture and Television Engineers (SMPTE) published under the title "Forward Error Correction for Audio-Video / Audio Transport". Over IP Networks ", version of May 24, 2007, the contents of which, B13020 12 part of the present description to the extent permitted by law. As is illustrated in FIG. 4, for a segment Si comprising data packets P1 to PK, some or all the data bits from each of a first group of L of data packets P1 to PL are used, for example. to generate a FEC packet, FEC 1, some or all of the data bits from each of a second L group of PL + 1 to P2L data packets are used to generate a FEC packet, FEC 2, etc., up to a final FEC packet FEC M, where M is equal to K / L. Instead, or in addition, a FEC FEC'l packet may be generated based on the first Pi, P -L + 1 packets of each group of L packets, and similarly for the second pack of each L group. packets, etc., up to a FEC FEC 'L packet based on L-rank packets, PLI P2L, packets. As is known in the art, the FEC codes add redundancy to the data, which makes it possible to reconstruct on the receiver side one or more lost packets. According to the embodiments described herein, the level of FEC coding may be adapted based on such factors as the packet loss rate detected by the decoder. The level of the FEC coding is for example determined by at least the amount of redundancy that is added by the FEC codes, for example the number of bits of the FEC code. For example, a relatively low FEC coding level may be able to reconstruct a burst of a relatively small number of consecutive lost packets in each segment, whereas a relatively high FEC coding level may make it possible to reconstruct a relatively large number of packets. consecutive lost packets. In the example of FIG. 4, the level of the FEC code is for example adapted by varying the number of packets L in each group. The higher the value of L, the greater the number of packets that can be reconstructed in a burst. + In addition or to. Instead, the level of the FEC code may be, etc., of each group of L 3013 suitable by providing one or both of the types of FEC packets, FEC 1 to FEC M and FEC11 to FEC'L. Figure 5 illustrates in more detail the video decoder 104 of Figure 1 according to an exemplary embodiment. As illustrated, the data packets received from the video encoder 102 through the intermediate network (s) 106 are provided to a segment receiving module 502, which is for example a UDP receiver. The segment receiving module 502 reconstructs the video segments based on the video data and the FEC code. As previously indicated, this video data may comprise only video data, or a combination of video data and other data such as audio and / or text data. For example, video data (VIDEO) and code. FECs are separated and provided to an FEC 503 decoding module, which checks for missing packets, and attempts to reconstruct any missing packets based on the FEC code. If all the packets of a given video segment have been successfully received or reconstructed, the entire segment is transmitted to a video stream generation module 504, and in particular to a receive buffer 506 of that module. However, if it is not possible to reconstruct all the packets of the segment, the complete segment is for example abandoned. The segments from buffer 506 are then processed by a desegmentation device 508 (DESEGMENTER), which recombines the segments to form a coded video stream, which is then decoded by a decoder 510 (DECODER) to provide a video stream (VIDEO ) at an output of the decoder.

The segment receiving module 502 for example provides segment reception information to a request generation module 512, and in particular to a memory 514 (HISTORY) 'storing a reception information history. This segment receive information includes, for example, one or more of an upload time, latency, the number of packets lost, and so on. for each segment received. For example, download time and latency could be calculated using time stamping of packets by encoder 102. The reception information history also indicates, for example, whether a segment has not been received. successfully because of the number of packets lost. This information stored in the memory 514 is used by a request generator 516 (REQUEST GEN) to request one or more future segments. In particular, the request identifies for example the time period of the segment that is requested, as well as the segment encoding bit rate, and one or more of the FEC coding level and the transmission bit rate to be used for the segment. This request is then transmitted via the intermediate networks to the input 232 of the segment transmission module 204 as previously described. For example, if one or more past segments have been lost and / or are close to being lost due to the packet loss rate, the request generator 516 may request a segment having a lower coding bit rate and / or a higher FEC coding level. In some cases, for example in the case of transmission by a satellite or other wireless interface, the packet loss rate may be a percentage of transmitted packets that will depend on transmission conditions such as cloud cover. Therefore, increasing the FEC coding level without reducing the coding bit rate may be the preferred solution for maintaining video quality and maintaining low latency. Alternatively, in the case of transmission via a packet-switched network, the packet loss rate can be very high if the bandwidth capacity is exceeded, and in such a case a reduction in the coding bit rate may be the preferred solution. Further, as shown by an arrow 518 in FIG. 5, the request veneration module 512 also receives B13020, for example, an indication of the number of segments in the buffer 506, and if this number falls below a lower threshold or comes above an upper threshold, the request may indicate a corresponding increase or decrease in the transmission bit rate for a future segment. It will be clear to those skilled in the art that certain elements of the encoder and decoder may be implemented by a specific hardware circuit, such as an ASIC (application specific integrated circuit) or an FPGA, while other functionalities may be implemented by software executed by a suitable processor. FIG. 6 illustrates an exemplary processing device 600 adapted to a software-based implementation of at least some of the functions of the video encoder 102, or at least some of the functions of the decoder 104. For example the processing device 600 may be used to at least partially implement the functions of the request server 234 and / or the FEC code generation module 226 of the encoder 102, or the request generation module 512 of the decoder 104. The processing device 600 comprises a processing device 602 (P) comprising one or more software instruction control processors stored in an instruction 604 (MEM INSTR). The processing device 602 is also coupled to a memory 604 (M), which, in the case of the video encoder 102, can store, for example, the file system 220, and in the case of the decoder 104, can store, for example the historical data 514. A user interface 606 (USER INTERFACE) is also for example provided, which allows an operator to adapt certain parameters, such as the algorithm applied by the request generator 516 to determine the segment query parameters. In addition, a communication interface 608 (COMS INTERFACE) may be coupled to the processing device 602 which allows the reception and / or transmission of data. An advantage of the embodiments described herein is that a video stream can be transmitted at a relatively high effective bit rate. In particular, because of the combined functionality of providing a plurality of segment sequences each having a different coding bit rate, and of using FEC code with each segment, the actual transmission bit rate can be very high, even when there is relatively high and variable latency in the network, and / or a relatively high and variable packet loss rate in the network. With the description thus made of at least one illustrative embodiment, various alterations, modifications and improvements will readily occur to those skilled in the art.

Claims (15)

REVENDICATIONS1. A video encoder comprising a segment generation module (202) arranged to generate a first sequence of video segments (SEQ (1)) based on a first video stream (EVS (1)) encoded at a first rate binary and a second sequence of video segments (SEQ (2)) based on a second video stream (EVS (2)) encoded at a second bit rate different from the first bit rate; and a segment transmission module (204) arranged to receive, from a video decoder, a segment request signal (REQ) identifying a selected video segment (SEG) of the first or second sequence segments; generate forward error correction code for the selected video segment; and transmit the selected video segment and the generated forward error correction code. The video encoder of claim 1, wherein the segment request signal (REQ) indicates a forward error correction decoding level to be used for the selected video segment, and wherein the transmission module segments (204) is arranged to generate the forward error correction code based on said level. 25 The video encoder of claim 2, wherein the segment transmission module (204) is arranged to generate the forward error correction code with a number of bits based on said level. A video encoder according to any one of claims 1 to 3, wherein the segment request signal (REQ) indicates a transmission bit rate to be used for the transmission of the selected video segment, and wherein the Segment transmission module (204) is arranged to transmit the selected video segment to the indicated transmission bit rate. The video encoder according to claim 4, wherein the segment transmission module (204) is arranged to separate the selected video segment into a plurality of data packets, and wherein the transmission of the selected video segment to said transmission bit rate comprises transmitting each packet of the plurality of data packets to said transmission bit rate. 10 The video encoder of any one of claims 1 to 5, wherein the first and second encoded video streams are generated based on the same initial video stream. The video encoder of claim 6, wherein the video segments of the first video segment sequence (SEQ (1)) correspond to time periods (tp1 to tp (m)) of the initial video stream, and the video segments of the second sequence of video segments (SEQ (2)) correspond to the same time periods of the initial video stream. A video decoder comprising: a segment receiving module (502) arranged to at least partially reconstruct a segment of video based on a received data block comprising segment data encoded at a first bit rate and the direct error correction code; and a segment request module (512) arranged to generate, based on the reconstruction of one or more previous video segments, a request for another video segment encoded at a second selected bit rate different from the first bit rate . The video decoder of claim 8, wherein the segment data comprises a plurality of data packets, and the second bit rate is selected based on a loss rate of the data packets. The video decoder of claim 8 or 9, wherein the request further indicates a forward error correction level to be used for the other video segment. The video decoder of claim 10, wherein the segment data comprises a plurality of data packets, and said forward error correction level is selected based on a loss rate of the data packets. . The video decoder of any one of claims 8 to 11, further comprising: a video segment buffer (506) arranged to store segments successfully received by the segment receiving module (502), the further indicating a transmission bit rate to be used for transmission of the other video segment, the transmission bit rate being selected on the basis of the amount of data stored in the video segment buffer. A video transmission system comprising: the video encoder according to any one of claims 1 to 7; the video decoder according to any one of claims 8 to 12; and one or more intermediate networks (106), interconnecting the video encoder and the video decoder, through which the selected video segment is transmitted by the video encoder and the data block is received by the decoder of video. A video coding method comprising: generating, by a segment generation module (202), a first video segment sequence (SEQ (1)) based on a first video stream (EVS (1); )) encoded at a first bit rate and a second video segment sequence (SEQ (2)) based on a second video stream (EVS (2)) encoded at a second bit rate different from the first bit rate; B13020 receive, by a segment transmission module (204) from a video decoder, a segment request signal (REQ) identifying a selected video segment (SEG) of the first or the second sequence of segments; generating, by the segment transmission module (204), forward error correction code for the selected video segment; and transmitting, by the segment transmission module (204), the selected video segment and the forward error correction code generated. A video decoding method comprising: at least partially reconstructing, by a segment receiving module (502), a video segment based on a received data block including segment data encoded at a first rate binary and direct error correction code; and generating, by a segment request module (512) based on the reconstruction of one or more previous video segments, a request for another video segment encoded at a second selected bit rate different from the first bit rate .