RU2787846C1 - Sample arrige encoding for low delay - Google Patents

Abstract

FIELD: video encoding.

SUBSTANCE: invention relates to means for video encoding. A plurality of entropy slices is entropy encoded into an entropy encoded data stream, each of said plurality of entropy slices containing entropy encoded data for a corresponding row of the array of samples. Entropy encoding includes encoding into the data stream, for the current entropy slice, using Exponential Golomb coding, a position difference value associated with a start position within the data stream of the current entropy slice, where the position difference value indicates the difference between the start position within the data stream of the previous entropy slice and the starting position of the current entropy slice. The current entropy slice corresponds to the current row of the array of samples, and the previous entropy slice corresponds to the previous row of the array of samples, and the current and previous rows are consecutive rows of said array of samples. The start position of the current entropy slice within the data stream is to be obtained based on the sum of the start position of the previous entropy slice and the position difference value.

EFFECT: improving coding efficiency.

27 cl, 20 dwg, 4 tbl

Description

The present application relates to array coding of samples (bins), such as image or video coding.

The parallelization of the encoder and decoder is very important due to the increase in processing requirements due to the HEVC standard, as well as the expected increase in video resolution. Multi-core architecture is becoming available in a wide range of modern electronic devices. Therefore, efficient methods for enabling the use of a multi-core architecture are required.

The encoding or decoding of the LCU occurs with a raster scan, with which the CABAC probability is adapted to the specific features of each image. Spatial dependencies exist between adjacent LCUs. Each LCU (largest coding unit) depends on its left, top, top-left, and top-right neighbor LCUs, due to various components such as motion vector, prediction, intra prediction, and others. Due to the possibility of parallelization in decoding, these dependencies usually need to be broken or are broken in modern applications.

Some parallelization concepts have been proposed, namely wave data processing. The motivation for further research is the use of techniques that reduce the loss of coding efficiency and thus reduce the bitstream cost for parallelization approaches in the encoder and decoder. Also, low latency processing was not possible with available methods.

Thus, it is an object of the present invention to provide an array-of-sample coding concept that allows for lower delay with comparatively less degradation in coding efficiency.

This objective is provided by the contents of the appended independent claims.

If the entropy encoding of the current part of the predefined entropy slice is based on the corresponding probability estimates of the predefined entropy slice that are adapted using the previously encoded part of the predefined entropy slice, but not only but also on the probability estimates that are used in the entropy encoding of the spatially adjacent, in order of entropy slices of the previous entropy slice in its adjacent portion, then the probability estimates used in entropy coding adapt to the actual symbol statistics more closely, thus reducing the reduction in coding efficiency typically caused by lower delay concepts. Temporal relationships may be used additionally or alternatively.

For example, dependency on probability estimates that are used in entropy encoding of a spatially adjacent entropy slice order of a previous entropy slice may involve initializing probability estimates at the start of entropy encoding of a predefined entropy slice. Usually, the probability estimates are initialized to values adapted to the statistical information of the symbols of the characteristic mixture of the material of the array of samples. To avoid passing initialization values of probability estimates, they are known to the encoder and decoder by definition. However, such predefined initialization values are naturally just a compromise between the bitrate (bitrate) of the side information on the one hand and the encoding efficiency on the other hand, since the initialization values naturally - more or less - deviate from the actual type statistical information of the array material currently being encoded. samples. Probability adaptation during entropy slice encoding adapts the probability estimates to the actual symbol statistics. This process is accelerated by initializing the probability estimates at the beginning of the entropy encoding of the current/predefined entropy slice, using the already adapted probability estimates of the just mentioned spatially neighboring, in the order of entropy slices, of the previous entropy slice, since the latter values are already, to some extent, adapted to the actual statistical information of the sample array symbol currently being considered. Low latency encoding may, however, be made available by using initialization probability estimates for predefined/current entropy slices, the probability estimate data being used in its adjacent portion instead of being found at the end of the entropy encoding of the previous entropy slice. Wave processing is still possible with these measures.

Additionally, the aforementioned reliance on probability estimates that are used in the entropy coding of a spatially adjacent entropy slice order of the previous entropy slice may involve an adaptation process consisting of adapting the probability estimates directly used in the entropy coding of the current/predetermined entropy slice. Adapting the probability estimate involves using the freshly encoded portion, ie the newly encoded symbol(s), to adapt the current state of the probability estimates to the actual symbol statistics. With these measures, the initialized probability estimates are adapted at some adaptation rate to the actual symbol statistics. This adaptation rate is increased by performing the just-mentioned adaptation of the probability estimate, based not only on the currently encoded symbol of the current/predetermined entropy slice, but also depending on the probability estimates that have been found in the neighboring part of the spatially neighboring, in order of entropy slices of the previous entropy slice. Again, by selecting the spatially adjacent current portion of the current entropy slice and the adjacent portion of the previous entropy slice, respectively, wave processing is still possible. The advantage of coupling adapting one's own probability estimate on the current entropy slice with adapting the probability of the previous entropy slice is an increase in the speed at which adaptation occurs to the actual symbol statistics, since the many symbols moved through in the current and previous entropy slices contribute to the adaptation. and not just symbols of the current entropy slice.

Preferred embodiments of the embodiments of the present invention are the subject of the dependent claims. Additionally, preferred embodiments are described with respect to the figures, among which:

fig. 1 shows a block diagram of an exemplary encoder;

fig. 2 shows a schematic diagram of the division of an image into slices and portions of slices (ie blocks or coding units) along with the coding order defined between them;

fig. 3 shows the operation flow of an exemplary encoder, such as the encoder of FIG. 1;

fig. 4 shows a schematic diagram for explaining the operation of an exemplary encoder such as the encoder in FIG. 1;

fig. 5 shows a schematic diagram of a parallel operating implementation of an encoder and decoder;

fig. 6 shows a block diagram of an exemplary decoder;

fig. 7 shows the flow of operation of an exemplary decoder, such as the decoder of FIG. 6;

fig. 8 shows a schematic diagram of an exemplary bitstream resulting from the coding scheme of FIG. sixteen;

fig. 9 schematically shows an example of probability calculation with other LCUs;

fig. 10 shows a graph illustrating RD results for the internal method (4 streams) compared to HM3.0;

fig. 11 shows a graph illustrating RD results for a low latency (1 stream) method compared to HM3.0;

fig. 12 shows a graph illustrating RD results for random access method (1 stream) compared with HM3.0;

fig. 13 shows a graph illustrating RD results for a low latency (4 stream) method compared to HM3.0;

fig. 14 schematically and roughly illustrates possible entropy slice connections;

fig. 15 schematically and roughly illustrates a possible entropy slice signaling;

fig. 16 schematically and exemplarily illustrates the encoding, segmentation, interleaving, and decoding of entropy slice data across slices;

fig. 17 schematically and roughly illustrates a possible connection between frames;

fig. 18 schematically and roughly illustrates the possible use of related information;

fig. 19 schematically shows the possibility of a wave passing obliquely within a spatial/temporal space filled with successive arrays of samples; and

fig. 20 schematically shows another example for subdividing entropy slices into fragments.

In order to facilitate understanding of the following measures to improve the provision of low delay with fewer drawbacks in relation to coding efficiency, the encoder in FIG. 1 is first described more generally without prior discussion of the preferred concepts of the embodiments of the present application and how it can be incorporated into the embodiment of FIG. 1. It should be mentioned, however, that the structure shown in FIG. 1 merely serves as an illustrative environment in which embodiments of the present application may be used. Generalizations and alternatives for encoders and decoders in accordance with embodiments of the present invention are also briefly discussed.

Fig. 1 shows an encoder for encoding an array of samples 10 into an entropy encoded data stream 20. As shown in FIG. 1, sample array 10 may be one of sample array sequences 30, and an encoder may be configured to encode sequence 30 into data stream 20.

The encoder in Fig. 1 is generally denoted by reference symbol 40 and contains a pre-coder 42 followed by an entropy encoding stage 44 whose output outputs a data stream 20. The precoder 42 is configured to receive and process an array 10 of samples to describe its contents in terms of syntax elements with a predefined syntax, each syntax element being a corresponding one of a predefined set of syntax element types, which in turn are associated with the corresponding semantics.

When describing the sample array 10 using syntax elements, the precoder 42 may subdivide the sample array 10 into coding units 50 . The term "coding unit" may, for reasons discussed in more detail below, alternatively be referred to as "coding tree units" (CTUs). One possibility of how the precoder 42 can subdivide the sample array 10 into coding units 50 is shown in FIG. 2. In accordance with this example, this division regularly divides the 10 sample array into coding units 50, so that the latter are arranged in rows and columns to cover the entire array of 10 samples without gap without overlap. In other words, precoder 42 may be configured to describe each coding unit 50 in terms of syntax elements. Some of these syntax elements may form subdivision information for the additional subdivision of the corresponding coding unit 50 . For example, by division into multiple trees, the division information may describe the division of the corresponding coding unit 50 into prediction blocks 52, with precoder 42 associating a prediction mode with associated prediction parameters for each of these prediction blocks 52. This prediction subdivision may allow prediction blocks 52 to be of different sizes, as illustrated in FIG. 2. Precoder 42 may also associate subdivision residual information with prediction blocks 52 to further subdivide prediction blocks 52 into residual blocks 54 to describe the prediction error for each prediction block 52. Thus, the precoder may be configured to generate a syntactic description of the array of 10 samples in accordance with the hybrid coding scheme. However, as noted above, the just-mentioned way in which the precoder 42 describes the sample array 10 in terms of syntax elements has been presented merely for purposes of illustration, and may also be implemented differently.

The precoder 42 may use the spatial relationships between the contents of neighboring coding units 50 from the array 10 samples. For example, precoder 42 may predict syntax elements for a certain coding unit 50 from syntax elements defined for previously encoded coding units 50 that are spatially adjacent to the currently encoded coding unit 50. In FIG. 1 and 2, for example, the top and left adjacent elements serve for prediction, which is illustrated by arrows 60 and 62. In addition, the pre-coder 42 in the intra prediction method can already extrapolate the encoded contents of adjacent coding units 50 to the current coding unit 50 to obtain prediction samples current unit 50 coding. As illustrated in FIG. 1, precoder 42 may, without using spatial relationships, in the time domain predict samples and/or syntax elements for the current coding unit 50 from previously encoded arrays of samples, as illustratively shown in FIG. 1 by arrow 64. That is, a given motion compensated prediction may be used by precoder 42, and the motion vectors themselves may be the object of temporal prediction from motion vectors of previously encoded arrays of samples.

That is, the precoder 42 may describe the contents of the sample array 10 with respect to coding units, and for this purpose it may use spatial prediction. Spatial prediction is limited for each coding unit 50 to spatially adjacent coding units of the same sample array 10 such that when order 66 of the coding among the coding units 50 of the 10 sample array is followed, through adjacent coding units serving as a prediction reference for spatial prediction, in the general case have already moved in order 66 coding before the current unit 50 coding. As shown in FIG. 2, the coding order 66 defined among the coding units 50 may, for example, be a raster scan order in which line by line is moved through the coding units 50 from top to bottom. Optionally, subdividing the array 10 into an array of tiles may cause the scan order 66 to move—in raster scan order—through the coding units 50 that create one tile, first before moving on to the next in order tile, which in turn may also have a raster scan type. For example, spatial prediction may involve simply neighboring coding units 50 within a coding unit row above the coding unit row within which the current coding unit 50 resides, and a coding unit within the same coding unit row but to the left of the current coding unit. As will be explained in more detail below, this constraint on spatial relationship/spatial prediction enables parallel waveform data processing.

The pre-coder 42 directs the syntax elements to the entropy encoding stage 44 . As just stated, some of these syntax elements are predictively encoded, that is, they represent prediction errors. The precoder 42 can thus be regarded as a predictive encoder. In addition, the pre-coder 42 may be a transform encoder configured to convert the prediction code residuals of the contents of the coding units 50 .

An exemplary internal structure of entropy encoding stage 44 is also shown in FIG. 1. As shown, the entropy encoding stage 44 may optionally comprise a character representer for converting each of the syntax elements received from the precoder 42 whose number of possible states exceeds the number of character alphabet elements into a character alphabet character sequence s _i based on which the means (engine) 44 entropy coding. In addition to this optional character representation means 70, entropy encoding means 44 may comprise a context selector 72 and an initialization means 74, a probability estimate control means 76, a probability estimate adaptation means 78, and an entropy encoding kernel 80 . The output of the entropy encoding core forms the output of the entropy encoding stage 44 . In addition, the entropy encoding core 80 contains two inputs, namely, one for receiving symbols s _i of a sequence of symbols, and the other for receiving a probability estimate p _i for each of the symbols.

Due to the properties of entropy coding, the coding efficiency with respect to the compression ratio increases as the probability estimate improves: the better the probability estimate matches the actual symbol statistical information, the better the compression ratio.

In the example in FIG. 1, the context selector 72 is configured to select, for each symbol s _i , a corresponding context c _i from the set of available contexts managed by the manager 76 . It should be noted, however, that context selection is merely an optional feature and can be omitted, for example, by using the same context for each character. However, if context selection is used, then context selector 72 may be configured to perform context selection based at least in part on information that pertains to coding units outside the current coding unit, namely that pertains to adjacent coding units within a limited range. proximity, as discussed above.

The control means 76 includes a memory that stores, for each available context, an associated probability score. For example, the character alphabet may be a binary alphabet, so just one probability value may need to be stored for each available context.

The initializer 74 may periodically initialize or reinitialize the probability estimates stored in the control 76 for the available contexts. Possible times at which such initialization may be performed are discussed further below.

The adaptor 78 has access to the symbol pairs s _i and the corresponding probability estimates p _i and adapts the probability estimates in the control means 76 accordingly. That is, each time a probability estimate is applied by the entropy encoding core 80 to entropy encode the corresponding symbol s _i into the data stream 20, the adaptor 78 may change that probability estimate according to the value of the given current symbol s _i , such that the estimate p _i probability was better adapted to the actual symbol statistics when encoding the next symbol that is associated with a given probability estimate (via its context). That is, the adaptor 78 receives a probability estimate for the selected context from the control 76 along with the corresponding symbol s _i and adapts the probability estimate _pi accordingly so that an adaptive probability estimate is used for the next symbol s _i of the same context c _i .

The entropy coding core 80 is, for example, configured to operate in accordance with an arithmetic coding scheme or with a probability interval division entropy coding scheme. In arithmetic encoding, the entropy encoding core 80, for example, continuously updates its state when encoding a symbol sequence with a state, for example, determined by a probability interval determined by a probability interval width value and a probability interval offset value. When operating in a pipelined concept, the entropy encoding core 80, for example, divides the range of possible probability estimates into different bins, performing fixed probability entropy coding with respect to each of those bins, thereby obtaining a substream for each of the sub-bins whose coding efficiency, accordingly, tuned for the associated probability interval. In the case of entropy encoding, the output of the data stream 20 is an arithmetically encoded data stream conveying information to the decoding side, which allows the process of subdividing a given slot to be emulated or re-executed.

Naturally, it is possible for the entropy encoding state 44 to entropy encode all information, i.e. all syntax elements/symbols s _i , related to the sample array 10 by initializing the probability estimates just once at its beginning, and then continuously updating the probability estimate with the means 78 adaptations. This, however, would result in a data stream 20 that would have to be sequentially decoded on the decoding side. In other words, it would not be possible for any decoder to subdivide the resulting data stream into multiple sub-portions and decode the sub-portions in parallel. This, in turn, would hinder all efforts to reduce latency.

Accordingly, as will be discussed in more detail below, it is convenient to subdivide the data set describing the sample array 10 into so-called entropy slices. Each of these entropy slices, respectively, covers a different set of syntax elements related to the array of 10 samples. If the entropy encoding stage 44 were, however, to entropy encode each entropy slice completely independently of each other, first initializing the probability estimate once and then constantly updating the probability estimates separately for each entropy slice, then the encoding efficiency would decrease due to the increased percentage of data , which refer to and describe an array of 10 samples for which the probability estimates used are (yet) less closely adapted to the actual symbol statistical information.

In order to overcome the problems just mentioned in reconciling the need for low coding delay on the one hand and high coding gain on the other hand, the following coding scheme can be used, which is further described with respect to FIG. 3.

First, the data describing the array of 10 samples is subdivided into portions, hereinafter referred to as "entropy slices". Subdivision 80 need not be without overlaps. On the other hand, this subdivision may at least partially correspond to the spatial subdivision of the array of 10 samples into different chunks. That is, according to subdivision 80, the syntax elements describing the sample array 10 may be distributed into different entropy slices depending on the location of the coding unit 50 that belongs to the corresponding syntax element. See, for example, FIG. 2. FIG. 2 shows an exemplary subdivision of an array of 10 samples into different chunks 12. Each chunk corresponds to a corresponding entropy slice. As an example, each chunk 12 is shown to correspond to a string of coding units 50. Other divisions, however, may also be performed. However, it is preferable if the division of the array 10 of samples into chunks 12 adheres to the aforementioned coding order 66, so that chunks 12 cover successive runs of coding units 12 in coding order 66 . Even so, however, it is not required that the start and end positions of chunk 12 in coding order 66 coincide with the left and right edges of the coding unit strings 50, respectively. It is not even necessary to coincide with the borders of coding units 50 that immediately follow each other and the coding order 66 .

With this division of the array 10 samples among the portions 12 determine the order of 16 entropy slices, in which the portions 12 follow each other in order 66 coding. In addition, for each entropy slice, a corresponding entropy encoding path 14 is determined, namely, a fragment of the encoding path 66 directed towards the corresponding chunk 12. In the example of FIG. 2, where chunks 12 coincide with strings of coding units 50, the entropy encoding paths 14 from each point of the entropy slice along the string direction are parallel to each other, that is, in this case, from the left side to the right side.

It should be noted that it would be possible to restrict the spatial predictions performed by the precoder 42 and the context acquisition performed by the context selector 72 so that they do not cross slice boundaries, i.e. so that the spatial predictions and the results of the context selections are not data dependent. corresponding to another entropy slice. Thus, "entropy slices" correspond to the usual definition of "slices" in H.264, for example, which are decoded completely independently of each other, except for the probability initialization/adaptation dependency below. However, it is also acceptable to allow spatial predictions and context selection results, i.e. dependencies in general, to cross slice boundaries to exploit local/spatial dependencies, since WPP processing can still be performed while precoding inversion is being considered, then there is a recovery based on syntactic elements and an entropy context choice. Until now, entropy slices corresponded in one way or another to "dependent slices".

Subdivision 80 can, for example, be performed using the cascade 44 entropy encoding. The subdivision may be fixed or may vary among the plurality of sequences 30. The subdivision may be set by definition or may be communicated in data stream 20.

Based on the entropy slices, the actual entropy encoding can occur, ie 82. For each entropy slice, the entropy encoding can be structured into an initial phase 84 and a continuation phase 86. The initial phase 84 involves, for example, initializing the probability estimates as well as invoking the actual entropy encoding process for the corresponding entropy slice. The actual entropy encoding is then performed during the continuation phase 86 . Entropy encoding during phase 86 is performed along the corresponding entropy encoding path 14 . The start phase 84 for each entropy slice is controlled such that entropy encoding of a plurality of entropy slices is started sequentially using an order of 16 entropy slices.

Then, in order to avoid the above disadvantage, which is the result of entropy encoding each entropy slice completely independently of each other, the entropy encoding process 82 is controlled such that the current part, for example, the current coding unit of the current entropy slice, is entropy encoded based on the respective probability estimates of the current entropy slice that are adapted using the previously encoded portions of the current entropy slice, ie the portions of the current entropy slice to the left of the current coding unit 50 in the case of FIG. 2, and on the probability estimates that are used in the entropy coding of spatially adjacent, in the order of 16 entropy slices, of the previous entropy slices in its adjacent part, i.e., in the adjacent coding unit.

In order to describe the above relationship more clearly, reference is made to FIG. 4. FIG. 4 illustrates the n-1st, nth and n+1st entropy slices in order of 16 entropy slices with reference sign 90. Each entropy slice 90 spans a sequence of syntax elements describing a chunk 12 of a 10 sample array with which the corresponding entropy slice 90. Along the entropy encoding path 14, the entropy slice 19 is segmented into a sequence of segments 92, each of which corresponds to a respective one of the coding units 50 of the chunk 12 to which the entropy slice 90 belongs.

As described above, the probability estimates used in the entropy encoding of the entropy slices 90 are continuously updated during the continuation phase 86 along the entropy encoding path 14 so that the probability estimates are better and better adapted to the actual symbol statistical information of the corresponding entropy slice 90 - that is, the estimates the probabilities are associated with the corresponding entropy slice. Although the probability estimates 94 used to entropy encode the entropy slice 90 are continuously updated during the continuation phase 86, in FIG. 4, simply the states of the probability estimates 94 that appear at the start and end positions of the segments 92 are illustrated and referred to further below. In particular, the state before the entropy encoding of the first segment 92, which is initialized during the initial phase 84, is shown as 96, the state found after the encoding of the first fragment is illustrated as 98, and the state found after the encoding of the first two fragments is indicated as 100. Those the same elements are shown in FIG. 4 also for entropy slice n-1 in order of 16 entropy slices, and the next entropy slice, i.e. entropy slice n+1.

Now, to ensure the above relationship, the initial state 96 for the entropy encoding of the nth entropy slice 90 is set depending on any intermediate state of the probability estimates 94 found during encoding of the previous n-1 entropy slice. "Intermediate state" shall mean any state of the probability estimates 94, excluding the initial state 96 and the final state found after the entropy encoding of the entire n-1 entropy slice. By doing so, the entropy encoding of a sequence of entropy slices 90 in order of 16 entropy slices can be parallelized with a degree of parallelization that is determined by the ratio of the number of segments 92 preceding the state used to initialize the probability estimates 94 for the entropy encoding of the next entropy slice, i.e. a, and the number of segments 92 following this stage, i.e. b. In particular, in FIG. 4 a is set, for example, by initialization, i.e. adaptation, to state 100, so as to set state 96 of the current entropy slice to be equal to state 100 of the previous entropy slice, illustrated by arrow 104.

With these measures, the entropy encoding of any segment 92 after state 100 in order 14 of the entropy encoding path will depend on the probability estimate 94 that is adapted during the continuation phase 86 based on previous fragments of the same entropy slice, as well as on the probability estimate, which used in the entropy encoding of the third segment 92 of the previous entropy slice 90.

Accordingly, the entropy encoding of the entropy slices 90 can be performed in parallel with pipelined scheduling. The only restrictions imposed on the scheduling time is that the entropy encoding of an entropy slice can start immediately after the end of the entropy encoding of the a-th segment 92 of the previous entropy slice. Entropy slices 90 immediately after each other in order of 16 entropy slices are not subject to any other restrictions regarding the time alignment of the entropy encoding procedure during the continuation phase 86 .

However, in accordance with another embodiment, additionally and/or alternatively, a stronger connection is used. In particular, as illustrated in FIG. 4 with conventional arrows 106, adapting the probability estimate during the continuation phase 86 causes the coding unit data corresponding to a segment 92 to change the probability estimates 94 from the state at the beginning of the corresponding segment 92 to the end of that segment 92, thereby improving the approximation the actual statistical information of the symbol, as indicated above. That is, for entropy slice n-1, adaptation 106 is performed simply depending on the data of entropy slice n-1, and the same applies to adaptation 106 of the probability estimate of entropy slice n, and so on. For example, it is possible to perform initialization as explained above with arrows 104 by performing probability estimate adaptation 106 without further interference between entropy slices 90. However, in order to speed up the approximation of the probability estimate to the actual symbol historical information, probability estimate adaptation 106 of successive entropy slices may be coupled such that the adaptation 106 of the probability estimate of the previous n-1 entropy slice is also affected or taken into account when the adaptation result of the probability estimate of the current entropy slice n is adapted. This is illustrated in FIG. 4 with an arrow 108 pointing from the state 110 of the spatially adjacent probability estimates 94 for the entropy encoding of the n-1 entropy slice 90 to the state 100 of the probability estimates 94 of the entropy encoding of the nth entropy slice 90. When using the above state initialization 96, connection 108 The probability adaptation may, for example, be used in any of the b probability estimation states found after the entropy encoding of the b segments 92 of the previous entropy slice. More precisely, the probability estimates found immediately after the entropy encoding of the first segment 92 of the current entropy slice may be the result of the usual probability adaptation 106 and by taking into account the 108 probability estimate states resulting from the adaptation 106 of the probability estimates during entropy encoding of the (a+1)th segment 92 of the previous entropy slice n-1. "Accounting" may, for example, involve some sort of averaging operation. An example will be further set forth below. In other words, the state 98 of the entropy encoding probability estimates 94 of the nth entropy slice 90 at the beginning of the entropy encoding of its segment 92 may be the result of averaging the previous state 96 of the entropy encoding probability estimates 94 of the current entropy slice n, which are adapted using adaptation 106, and the states before the entropy encoding of the (a+1)th segment 92 of the previous n-1 entropy slice modified according to the probability adaptation 106 . Likewise, state 100 may be the result of averaging the result of adaptation 106 performed during the entropy encoding of the current entropy slice n and the result of probability adaptation during the entropy encoding of the (a+2)th segment 92 of the previous entropy slice n-1, etc. .

More specifically, it is assumed that

p(n)->{i, j}, where i, j denotes the position of any coding unit (where (0,0) is the upper left position and (I, J) is the lower right position), i∈{1.. .I} and j∈{1...J}, I is the number of columns, J is the number of rows and p() defines the order 66 of the trajectory,

P _{{i, j}} is the probability estimate used in the entropy encoding of the coding unit {i, j}; and

T(P _{{i, j}} ) is the result of adapting 106 the probability P _{{i, j}} based on the coding unit {i, j};

Then, the probability estimates 106 of successive entropy slices 90 can be combined to replace the usual internal adaptation of the entropy slice according to P _p(n+1) =T(P _p(n) ), with

P _p(n+1) =mean (T(P _p(n) ), T(P _{{i, j}1} ), ..., T(P _{{i, j}N} ))

where N can be 1 or greater than 1 and {i, j} _1...N are selected from (are within) any previous (in order of 16 entropy slices) entropy slice 90 and its associated portion 12, respectively. The "mean" function can be one of a weighted summation, median function, etc. p(n)={i, j} is the current coding unit and p(n+1) is the next one according to coding order 14 and 66, respectively. In the embodiments shown, p(n+1)={i+1, j}.

Preferably, {i, j} _1...N corresponds for each k∈{1...N} {i, j} _1...N ={i _k , j _k } and i _k <i+3 and j _k <j, where p(n)={i, j} is the current coding unit (ie any of the second or subsequent coding units of the current entropy slice), i.e. they are not located outside the wave.

In the last alternative, the entropy encoding timing, when entropy encoding of the entropy slices 90 is performed in parallel, is connected to each other more intensively. That is, during the continuation phase 86, the next segment 92 on the current entropy slice line may begin immediately after the end of the corresponding segment of the previous entropy slice that is further downstream of the entropy encoding path 14.

In other words, the above discussion shows an example in which the decoder 40, and in particular the entropy encoding stage 44, is configured to perform, for an entropy slice 90 such as the nth one, initializing its probability estimates 94 before decoding the first slice 92 corresponding to the first coding unit. /to the first block 50 of chunk 12 corresponding to the n-th entropy slice along the corresponding coding path 14 using probability estimates found after entropy decoding of the second coding unit/block 50 of chunk 12 corresponding to the order of 16 entropy slices of the previous entropy slice along the corresponding trajectory 14 coding. Additionally, or alternatively, the decoder 40, and in particular the entropy encoding stage 44, may be configured to perform, for each entropy slice 90, entropy decoding and adapting the probability estimate such that, after the current coding portion/block/unit 50 of the current entropy slice has been entropy decoded based on the corresponding probability estimates 94 from the current entropy slice 90, the corresponding probability estimates 94 from the current entropy slice are adapted depending on this current part of the current entropy slice and the probability estimates that are found in the entropy decoding of the neighboring part / block / coding units 50 of a spatially adjacent previous entropy slice, such as the slice in the aforementioned adjacent row in the second column to the right of the current portion/block/coding unit of the current entropy slice.

As has become clear from the above discussion, the probability estimate must be adapted/managed for each entropy slice 50 separately. This can be done by sequentially processing the entropy slices and storing the probability estimation states as states, which for example are shown and mentioned with respect to FIG. 4, that is, 96, 98, 100, 110, and 102, on the respective probability estimation memory 120 (see FIG. 1). Alternatively, more than one entropy encoding stage 44 may be provided to decode entropy slices in parallel. This is illustrated in FIG. 5, which shows a plurality of implementations of entropy encoding stages 44, each associated with a respective one of the entropy slices and respective portions 12 of an array of 10 samples, respectively. Fig. 5 also illustrates the decoding process and its possible implementations using parallel implementations of the corresponding entropy decoding stage 130. Each entropy decoding stage 130 is provided with a corresponding one of the entropy slices transmitted via the data stream 20 .

Fig. 5 shows that the entropy encoding stages 44 and the corresponding decoding stages 130 do not operate in parallel completely independently of each other. Instead, stored probability estimation states, such as those stored in memory 120, are transferred from one stage corresponding to the corresponding entropy slice to another stage corresponding to the next entropy slice according to the order of 16 entropy slices.

Fig. 5 also shows, for illustrative purposes, a possible move order for moving through the results of a possible division of coding units 50, such as a move order 140 determined among prediction units 52 within one coding unit 50. For all of these prediction blocks 52, the corresponding syntax elements are contained within the corresponding segment 92 and correspond to the values of these syntax elements, the probability estimates 94 are adapted during movement along the path 140, with the adaptation during movement through the coding units 50 being determined by the aforementioned "T". In CABAC according to H.264 and HEVC, "T" is performed based on a table, i.e. a "route table" defining transitions from the current state of the probability estimate for the current context to the next state depending on the current symbol value associated with that context.

Before going to FIG. 6 which shows an embodiment of a decoder corresponding to the encoder embodiment of FIG. 1, it should be noted that the predictive entity of the precoder 42 is merely an illustrative embodiment. In accordance with even alternative embodiments, the precoder 42 may be left with the syntax elements that the entropy encoding stage 44 operates on, which are, for example, the original sample values of the array 10 of samples. Even alternatively, the precoder 42 may be configured to generate a subband decomposition of the 10 sample array, as in JPEG. The additional nature of the context selector 72 has already been mentioned above. The same applies to the initialization means 74. The same can be implemented differently.

Fig. 6 shows a decoder 200 corresponding to encoder 40 in FIG. 1. From FIG. 6, the design of decoder 200 essentially mirrors that of encoder 40. That is, decoder 200 includes an input 202 for receiving data stream 20, followed by a serial connection of entropy decoding stage 204 and creator 206. The entropy decoding stage 204 entropy decodes the entropy slices transmitted in the data stream 20 and sends, in turn, the decoded symbols s_i and syntax elements, respectively, to the creator 206, which in turn requests the syntax elements from the entropy decoding stage 204 with the appropriate request 208. In other words, the creator 206 is also responsible for parsing the stream of syntax elements created by the precoder 42 in within the encoder. Accordingly, the creator 206 queries the syntax elements sequentially from the entropy decoding stage 204 . Entropy decoding stage 204 has essentially the same structure as entropy encoding stage 44 . Accordingly, the same reference characters of the internal blocks of the entropy decoding stage 204 are used again. Symbolizer 70, if present, converts requests for syntax elements into requests for symbols, and entropy decoding engine 80 responds with the corresponding symbol value s_i, the character representer 70 maps the received character sequences forming valid character words with syntax elements and forwards them to the creator 206 . Creator 206 reconstructs an array 10 of samples from the stream of syntax elements received from entropy decoding stage 204, such as described above, using predictive decoding, and so on. More specifically, creator 206 also uses coding order 66 and performs decoding on coding units to perform predictions 60, 62, and 64. One or more predictions for syntax elements or for sample values are combined, e.g., summed, optionally using the prediction residual , obtained from the syntax elements of the stream of syntax elements. The entropy decoding core 80 as well as the entropy encoding core 80 support the concept of arithmetic decoding or the concept of probability division entropy decoding. In the case of arithmetic decoding, the entropy decoding core 80 may continuously update an internal state in the form of a partial interval width value and a value such as an offset value indicating that partial interval. The update is performed using the incoming data stream. The current partial interval is subdivided similarly to the entropy encoding kernel 80 using the estimate p_i probability provided for each character s_i through a context selector 72 along with a probability estimation control 76 . The adaptation means 78 performs adaptation of the probability estimate using the value of s_i decoded character to update p values_i context probability estimates c_i, associated with the symbol s_i, using the context selector 72 . Initialization with the initializer 74 is performed in the same implementations and in the same manner as on the encoding side.

The decoder in Fig. 6 operates similarly to the operation of the encoder as described above with respect to FIG. 3. In step 230, the sample array 10 is subdivided into entropy slices. See, for example, FIG. 8. FIG. 8 shows the data stream 20 that arrives at the input 202 and in particular the entropy slices 90 contained therein. At step 230, each of these entropy slices 90 is associated with chunk 12, this corresponding entropy slice 90 is associated so that the above-described initialization of the probability estimate and adaptation of the probability estimate based on the corresponding previous entropy slice can be performed. Subdivision of the array of samples, or more precisely, the association of entropy slices 90 with their respective portions 12, can be performed using the creation tool 206 . Linking can be provided by various measures, such as side information contained in the data stream 20 in non-entropy coded chunks, or by definition.

The entropy slices 90 are then entropy decoded in the entropy decoding process 232 in a manner that reflects the encoding processing 82, namely, performing for each entropy slice 90 an initial phase 234 and a continuation phase 236 with initialization of the probability estimate and adaptation in the same manner and in the same implementations, as in the coding procedure.

The same parallelization as described above with respect to encoding is possible on the decoding side. The entropy decoding stage implementations 130 shown in FIG. 5 may each be implemented as shown with respect to the entropy decoding stage 204 in FIG. 6. The probability estimate memory 240 may be used to store the state of the probability estimates for use in the entropy decoding stage 130 responsible for entropy decoding a subsequent entropy slice in order 16 entropy encoding.

After describing the embodiments of the present application, the discussed concepts are described below, again using different wording. In the following, several additional aspects of the present application are described. In the following, the above-mentioned coding units 50 are referred to as LCU (largest coding unit), thus adapting the wording to the future HEVC standard.

First, the probability adaptation 106 discussed above is briefly discussed again with respect to FIG. nine.

The current LCU uses probability estimates, such as CABAC probabilities, available after encoding the previous left LCU. For example, the LCU in FIG. 9, labeled x, is assumed to be entropy encoded using a probability estimate p ₁ that is adapted to complete the entropy encoding of the left-hand LCUs to the left of LCU x. However, if information is used not only from the left, but also from one or more other LCUs that are already processed and available, then better results in probability adaptation can be provided.

As described above, in entropy decoding and entropy coding of entropy slices, new probability estimates are computed before each LCU is coded or decoded using already existing probabilities (probability estimates) from other LCUs. More specifically, the probability adaptation is performed not only from any preceding LCUs of the current entropy slice, but also from LCUs of entropy slices preceding in order the entropy slices. This approach is again shown in Fig. 9. The initial probability of the current LCU, which is denoted by X in FIG. 9 can be calculated accordingly:

where a ₁ , ..., a _k are the weighting factors of the LCU.

Tested which probability weighting provides the best results. In this experiment, only neighboring LCUs were used. The study shows the use of weighting: 75% of the left LCU and 25% of the upper right LCU. In FIG. 10-13 show the results. Graphs titled using "prob. Adapt., use the probability adaptation described above.

However, when adapting the probability estimate, more than just adjacent blocks may be used. Each nearest LCU has its own neighbors, the use of which for probability optimization can be essential. In other words, not only the LCUs from the nearest upper row can be applied. In FIG. 9, an example can be seen in which the neighbors were taken as sources of probability estimates first, and each upper right LCU of each next upper row can be taken, compare p ₅ and p _k .

It must be recognized that some complexity is introduced by the above recalculation of the probability or adaptation of the probability estimate. The new calculation of the probability score occurs, for example, in three steps: first, the probability estimates of each candidate must be obtained from each state of the context. This is done by storing to memory 120 and 240, respectively, or by directing parallel entropy slice n decoding processes such that these states are available simultaneously. Then, using equation (1), an optimized probability (p _new ) will be created. That is, some averaging, for example, can be used to combine the adapted probability estimates of different entropy slices. As a final step, the new context state is converted from p _new and replaces the old one. That is, the probability estimate control means 76 borrows the new probability estimates thus obtained. This execution for each syntactic element, in particular using multiplication operations, can increase complexity tremendously. The only way to eliminate this shortcoming is to try to avoid these three steps. If the number of candidates and their weights are defined, then a pre-calculated table can be approximated for each situation. Thus, only one simple access to the table data is needed using candidate indexes (context states).

It is claimed that this technique can provide good results for both applications - with and without the use of entropy slices. The first application uses only one slice per frame, so the probability adaptation is optimized without any other changes. In the case of entropy slices, the adaptation of probabilities occurs within each slice, independently of other slices. This allows for a quick learning of the probabilities of the current LCU.

The above description also presents the use of the second LCU of the top line, that is, the use of the second LCU to initialize the probability estimate. Parallelization of encoding and decoding is possible if some of the bitstream conditions mentioned above (entropy slices) are provided. The dependency of CABAC probabilities between LCUs must be eliminated. Wave parallel processing is important to make the first LCU of each line independent of the last LCU of the previous line. This can be achieved if, for example, the CABAC probabilities are reinitialized at the start of each LCU line. However, this method is not optimal because each reinitialization loses the provided CABAC probabilities that are adapted to the specifics of the image. This drawback can be mitigated if the initialization of the CABAC probabilities of the first LCU of each link is done with the probabilities obtained after the second LCU of the previous link.

As described above, an increase in the probability adaptation rate can be achieved by connecting the probability adaptation of spatially adjacent entropy slices. In particular, in other words, the above discussion also anticipates a decoder such as the decoder in FIG. 6, for recovering an array (10) of samples from an entropy encoded data stream, configured to entropy decode (performed by an entropy decoding stage) a plurality of entropy slices in an entropy encoder data stream to recover different portions (12) of an array of samples associated with entropy sequences, respectively, by performing, for each entropy slice, entropy decoding along the corresponding entropy encoding trajectory (14) using the corresponding probability estimates, adapting (performed by adaptation means 78) the corresponding probability estimates along the corresponding entropy encoding trajectory, using the previously decoded part of the corresponding entropy coding slice , run entropy decoding of a plurality of entropy slices sequentially using the order (16) of entropy slices, and entropy decode a predefined entr opium slice of entropy decoding the current part (x) of the predefined entropy slice, based on the respective probability estimates of the predefined entropy slice, which are adapted using the previously decoded part of the predefined entropy slice (including p1, for example), and the probability estimates that are used in entropy decoding spatially neighboring, in the order of entropy slices, of the previous entropy slice (the slice containing X, for example) in a neighboring part (such as p4) of the spatially neighboring entropy slice.

The various chunks may be rows of blocks (eg, LCUs or macroblocks) of the array of samples. The last one could be video. The entropy encoding trajectory can propagate across lines. Entropy encoding, and thus also probability adaptation, can be adapted to the context. The order of the entropy slices can generally be chosen such that, in the order of the entropy slices, the different chunks follow each other in a direction (16) rotated about the entropy encoding paths (14) of the entropy slices, which in turn propagate substantially parallel to each other. friend. With these measures, the "wave" of currently decodable portions (such as p1, p4, p5+1, and pk in this figure) of entropy slices can generally be along a line forming an angle to the direction of the entropy slice that is less than the direction portion sequences. The wave will probably have to have a 1y by 2x slope for block positions in order for the top left pivot to always be there for all threads processing slices in parallel.

The decoder may be configured to perform, for each entropy slice, entropy decoding along the corresponding entropy encoding trajectory, in units of portions of a portion of an array of samples of the corresponding entropy slice, so that the entropy slices consist of the same number of portions, respectively, and the sequence of portions of the portions along the trajectories of the entropy slice was aligned relative to each other in the direction transverse to the trajectories of the entropy slice. The current part of a portion of a predefined entropy slice belongs to the resulting grid of parts (eg, LCUs or macroblocks). When performing entropy decoding for each entropy slice along the corresponding entropy encoding path, the decoder may store offsets/shifts between start times between immediately consecutive entropy slices in entropy slice order such that the wave of the currently decoded portions of the entropy slices forms a diagonal such as a corresponding line with a 0.5x block position slope with respect to the entropy slice trajectory and direction along the order of the entropy slices. Offset/shift may correspond to two parts for all pairs of immediately consecutive entropy slices. Alternatively, the decoder may simply prevent the distance between currently decodable portions of immediately consecutive (and immediately adjacent, as long as their sample array portions 12 are considered) entropy slices from becoming less than two portions. See the figure above: once the part/block p4 is decoded, the part/block to the right of it is decoded according to the order of 16 paths, and at the same time, if any, X or any of the preceding parts/blocks is decoded). With these measures, the decoder can use probability estimates that are already adapted based on the contents of the part/block p4, i.e. the part within a spatially adjacent chunk aligned to the part of chunk 12 of the predefined entropy slice following the current part X in the order of 16 trajectories for determining the probability estimates to be used in decoding X. In the case of a constant bias when decoding immediately consecutive entropy slices of two parts, the decoder is able to use the probability estimates that are already adapted based on the contents of the part/block p4 simultaneously to entropy decode the subsequent part (that is, the part to the right of p4) of a spatially adjacent entropy slice.

As described above, a weighted sum of already adapted probability estimates may be used to determine the probability estimates to be used to decode X.

As also described above, the order of entropy slices may also extend across frame boundaries.

It should be noted that the borrowing of probability from previous entropy slices just described can be performed for each part of the current/predefined entropy slice for which such neighboring parts are available in the previous entropy slices. That is, it is also true for the first part along the trajectory direction 16, and for this very first part/block (leftmost in each entropy slice in this figure), the borrow equals the initialization described above.

For better adaptation, also in this case, the 2 above methods can also be combined together. The results of this process with 1 and 4 threads, i.e. using processing units in parallel, are illustrated in FIG. 10 - fig. 13 (plots 2LCU+Prob.Adap or 2LCU).

To better understand the context of the above embodiments, and in particular the additional embodiments described below, namely the use of an LCU, one may first look at the H.264/AVC structure.

An encoded video sequence in H.264/AVC consists of a sequence of access units that are assembled in a stream of NAL units, and they use only one set of sequence parameters. Each video sequence can be decoded independently. An encoded sequence consists of a sequence of encoded pictures. The encoded frame may be the entire frame or one field. Each image is divided into macroblocks of a fixed size (in HEVC: LCU). Several macroblocks or LCUs may be combined together into one slice. The image is therefore a collection of one or more slices. The purpose of this data partitioning is to allow independent decoding of the samples in the region of the image that is represented by a given slice, without using data from other slices.

A technique that is also often referred to as "entropy slices" is the division of a traditional slice into additional subslices. In particular, this means splitting the entropy encoded data of one slice. The arrangement of entropy slices in a slice can have different varieties. The simplest variation is to use each row of LCUs/macroblocks in a frame as one entropy slice. Alternatively, columns or individual areas can be used as entropy slices, which can even be interrupted and switched with each other, such as slice 1 in FIG. fourteen.

The obvious purpose of the entropy slice concept is to enable parallel use of the CPU/GPU and multi-core architecture to improve the time of the decoding process, that is, to speed up the process. The current slice can be divided into partitions that can be parsed and reconstructed independently of other slice data. While some advantages may be provided by the entropy slice approach, there are some disadvantages associated with this.

First of all, the main goal is to create a bitstream that is suitable for a parallel encoding and decoding process. It should be appreciated that LCUs can only be encoded if neighboring LCUs (left, top, top-right) are already available as a coded version to use spatial and motion information for prediction. In order to enable parallelism with partitioning, a shift between slice processing must be implemented (eg, a shift of 2 LCUs, which is typical of the wave approach). Due to the adaptation of the CABAC probabilities, the LCU uses the probabilities that are available from the previous encoded LCU. With regard to the raster scan order, a problem that arises with image splitting is avoidance of parallelism, since the first LCU of each line depends on the last LCU of the previous line. This is affected by the fact that dependencies of CABAC probabilities between slices need to be removed so that several slices can start at the same time. One way to do this is to simply re-initialize CABAC, whereby, however, all borrowed data will be lost. As a result, the bitrate may increase.

Each slice then generates its own sub-bit stream, which can be placed sequentially into the main stream. But it is necessary to pass special information to the decoder so that these slices and their positions in the main stream can be correctly identified. Two signaling scenarios are supported. The location information may be stored in the image header (slice length information) or in each slice header (dots as code start). The byte alignment at the end of each entropy slice and location information increase the overhead.

To overcome the shortcomings introduced by signaling for entropy slices, it is important to use a good coding technique for signaling. A significant lack of entropy slice signaling is introduced into the frame if start codes are used for each slice, i.e. too many extra bytes (eg, a minimum of 4 bytes per slice) are added to the bitstream. Of course, inserting entropy slices 90 using seed codes has an advantage in low latency scenarios where the encoder needs to be able to output the entropy slices instantly. In such cases, there is no possibility of forwarding entry point information.

In less stringent low latency scenarios, however, the ability to store slice lengths (offsets) seems more appropriate. One known method for encoding such information is variable length coding (VLC) or Exponential Golomb coding. The main feature of VLC is the addition of empty information (zeroes) before the real information. With these zeros, code can be defined that stores information about the length of the offset. The authors propose another technique for doing this, the schematic of which is shown in FIG. 15, where =EntropySliceSize is the number of bytes contained in the entropy slice. Each subsequent X (offset) is defined as the difference in size between the previous already encoded and transmitted entropy slice offset and the current slice. The main features of this concept are formed from the difference values, depending on the size of X, so the amount of data can be reduced to 3 bits, which makes it possible to extract accurate information about the size of each entropy slice using a decoder. Compared to VLC, savings in the number of bits in the entropy slice header can be obtained.

That is, in accordance with the aspect of FIG. 15, for example, provides the concept of entropy encoding of the information signal, which allows a higher degree of compression, despite the possibility of parallel processing, compared to currently available concepts. In accordance with this aspect, the entropy encoded section 20 of the data stream into which the array of samples is encoded contains entropy slices such as 90 in FIG. 8 into which the various chunks 12 of the sample array are entropy encoded, and the header illustrated by dotted lines 300 in FIG. 9 contains information that shows the starting positions 302 of the entropy slices 90 measured in the entropy decoding region within the entropy encoded data stream, this information contains, for a predetermined entropy slice, a difference value revealing the difference between the starting position of the previous entropy slice n-1 and the starting position of a predefined entropy slice n, the data stream includes these delta values as a sequence of VLC bits.

In particular, the VLC bit sequence may have a variable length prefix y indicating that the difference x falls in the y-th sequence of the number z of intervals [0,2 ^a -1], [2 ^a , 2 ^b +2 ^a -1], [ 2 ^b +2 ^a , 2 ^c +2 ^b +2 ^a -1] and so on, and the PCM encoded suffix of the yth sequence of lengths a, b, c... If a, b, c... choose so that they are a power of two, if the corresponding y is added, that is, so that a+1, b+2, c+3, and so on, are all powers of two, then the byte alignment can be preserved. The number z is not limited to three, as is exemplified in FIG. 15.

An encoder such as the encoder in FIG. 1 is suitably designed to convert the difference of successive starting positions into a VLC bit sequence, i.e. by first determining the prefix, i.e. in which subinterval the difference (y-th) is located, and then setting the suffix plus the difference starting position minus the y-th of 0 , 2 ^a , 2 ^b +2 ^a and so on. A decoder such as in FIG. 6 is suitably designed to obtain the starting position of the current entropy slice n from the VLC bit sequence, that is, by first learning the prefix to obtain y, then setting the delta value for the suffix value plus the yth 0, 2 ^a , 2 ^b +2 ^a and so on, and then adding the difference for the starting point of the previous n-1 entropy slice.

With entropy slice segmentation, an additional advantage can be provided, namely, low latency transmission and faster decoding.

In a video stream, enabling higher resolutions (Full-HD, QUAD-HD, etc.) results in more data to be transmitted. For time-sensitive scenarios, so-called low latency (<145 milliseconds) use cases, transfer time becomes a critical factor. Consider ADSL data line for video conferencing applications. In this case, the so-called random access flow points, typically referred to as I-frames, would be candidates for bottlenecking during transmission.

In order to solve this problem and minimize transmission delay and decoding time, that is, end-to-end delay, a new technique for an entropy slice interleaving scheme for parallel transmission and processing can be applied.

HEVC provides for the so-called wave processing on the decoder side. They are going to provide it with the help of entropy slices. In the normal case, the data of the entire slice is delivered at one point in time. The decoding process starts as soon as the encoded data arrives at the wave decoder means. In order to reduce the time a decoder can start and end a frame, segmentation of entropy slices into small fragments using an interleaving approach is used in accordance with the present embodiment. Therefore, the encoder can deliver data corresponding to a certain entropy slice to the transport layer earlier than in the usual case. This then leads to faster transmission and earlier start of the parallel decoding process in the client.

The creation of slice fragments can be further provided by dividing the entropy slice into additional slices, keeping all dependencies (dependent slices). If this is done in each largest coding unit (LCU)/coding tree unit (CTU), then these fragments can be interleaved, additionally using system level techniques that consider transmitting the fragments in interleaved form and restoring or at least providing information about the original decoding order fragments through additional signaling. Such signaling may be a decoding sequence number (DON) such as the IETF RTP Payload Format for H.264/AVC (RFC 3984) defined. Another systemic method may be to apply wave substream fragments to another transport stream, as in MEPG-2 systems, by assigning a different PID to each of them, then multiplexing and thereby interleaving them in the transport channel.

This approach can also be applied across frame boundaries, in case the slice(s) of the next frame or the entropy slice(s) can already be decoded, for example, in a waveform, based on knowledge of the requested information to decode the entropy slice of the next frame due to the availability of interframe links. This data, which can already be decoded, of the next frame in decoding order, can be derived from the maximum allowed/transmitted motion vector length or additional information in the stream indicating the dependencies of parts of the data on the previous frame(s)), or from a fixed reference scheme, indicating the position passed in a position fixed in the sequence, such as a set of parameters. This will be further outlined below.

A picture may be encoded with one entropy slice per line(s) of largest coding units (LCUs), as depicted in FIG. 2 and 9. It is preferable to use wave methods on the decoder side. During the encoding process, the bitstream of each slice may be divided into segments of constant size. The resulting segments are then interleaved and may be transmitted for transmission. The constant size of the segments can create a problem at the end of the bitstream due to its variable length.

There are two possible generalized solutions. The first solution is to generate one-byte segments (usually the bitstream representing the slice is byte-aligned), and control the consumption of bytes with each decoder means, that is, the decoder determines the completion of the entropy slice.

The second is to use an exit code at the end of the slice. In the latter case, variable length segments may be used, but this may also result in more data. Another way is to pass the length of the entropy slice. One such alternative is described below.

The segment size and interleaving method may be transmitted either in a single SEI message or in an SPS.

The transmission scheme is shown in Fig. sixteen.

Thus, in accordance with an aspect of FIG. 16, for example, provides a sample array entropy coding concept that provides lower latency than currently available concepts. In accordance with this aspect, the encoded data stream 20 into which the information signal is encoded contains slices, such as entropy slices 90 or simply completely independently decodable slices (shown on the left side), into which different portions 12 of the information signal (predictively and/or entropy) are encoded, wherein the data slices 90 are subdivided into chunks (shaded boxes 310) which are arranged in the encoded data stream 20 in an interleaved manner (shown on the right side), the interleaving being represented by bracket 312.

As indicated above and described with respect to other aspects, slices may be entropy slices 90, which in turn may be corresponding subsets of frame slices, and accordingly, the encoded data stream may be entropy encoded data stream 20.

Interleaving 312 of slices 90 allows for lower latency since the decoder responsible for decoding any of the slices 90 does not have to wait for the amount of time spent on previous slices of other decoders (according to the order of 16 slices). Instead, all available decoders have the ability to start decoding their associated slices as soon as the first chunk 310 is available, and potential dependencies between chunks are determined, cf. the wave approach.

Different chunks 12 may be entropy encoded into entropy slices 90 using probability scores that are established independently among entropy slices, or using borrowing between entropy slices of probability scores as described above, for example, in which different chunks 12 are encoded into entropy slices 90 along respective entropy encoding trajectories 14, and the dependent entropy slices have their respective chunks 12 encoded therein using probability estimates derived from among other things the probability estimates used in higher ranking entropy slices in spatially adjacent portions of the respective chunk, as described previously.

The entropy encoded data stream 20 may further comprise a header, as shown as an optional feature in FIG. 16. Header 300 may be associated with a frame (array of samples) of sequence 30, header 300 contains information that indicates the length of the entropy slices. Information relating to the length of the entropy slices 90 may be encoded within header 300 as described above using VLC codes. Using the knowledge of the length of the entropy slices, on the decoding side, the last fragment associated with each of the entropy slices 90 and its length can be identified. However, start codes or other indication schemes may also be used. Slice start positions can also be identified simply by a decoding process that knows the end of the slice. Therefore, it may be possible to simply rely on an indication from the decoder, but this requires signaling between decoders and in some cases if the "earlier" entropy slice is completed later than the "later" slice in the stream. This may require, for certain cases, "adaptive" in-stream signaling, which may be based on start codes.

For example, entropy slices 90 may be placed after header 300, as illustrated in FIG. sixteen.

Fragments 310 may be of equal length, at least as long as it concerns the initial adjacent portion of the sequence of fragments 310 from the first of the fragments in the order in which the fragments are located in the entropy encoded data stream 20. Subsequent fragments may vary in length. Subsequent fragments may be equal to or less than the length of the initial adjacent portion of the sequence. The length of the subsequent fragments can be obtained from the above information in the header 300, which reveals the length of the entropy slices 90 or the position of the start. Fragments 310 may be arranged in entropy encoded data stream 20 cyclically according to an order defined among the entropy slices. For entropy slices, the fragments of which are completely in previous cycles, they can be skipped in the current and subsequent cycles.

Information signals other than the sample array sequence, such as a video signal, may also be transmitted via data stream 20. The various chunks 12 thus need not be chunks of a predefined array of samples, such as an image/frame.

As described above, entropy slices 90 may have different portions 12 of sample array 10 encoded therein by predictive coding using inter-entropy slice prediction and/or inter-picture prediction, entropy prediction error coding for inter-entropy slice prediction, and/or inter-picture prediction. That is, as described above, the different chunks may be spatially different chunks of frame 10 or a plurality of frames 30. The latter case applies if the slice(s) of the next frame(s) or entropy slice(s) can already be decoded, e.g., in a waveform. , based on knowledge of the requested information to decode the entropy slice of the next frame due to available inter-frame references. The already decoded data of the next frame in decoding order may be obtained from the maximum allowed/transmitted motion vector length or from additional information in the stream indicating the dependencies of the data pieces on the previous frame(s)), and prediction between entropy slices may involve intra prediction, in while inter-picture prediction may involve motion-compensated prediction. An example is shown below.

The aforementioned independence of setting a probability score among entropy slices can apply to both probability adaptation as well as content modeling. That is, the content selected within an entropy slice may be selected independently from other entropy slices, and the content probability score may also be initialized and adapted independently of any other entropy slice.

The corresponding entropy decoder can be constructed as follows.

An entropy decoder configured to decode an entropy encoded data stream 20 that contains entropy slices 90 into which various portions of a frame are entropy encoded, the entropy slices being subdivided into fragments 310 that are interleaved in the entropy encoded data stream 20, may be configured as shown in FIG. 6 and may further comprise a deinterleaver configured to deinterleave tiles 310, shown with symbol 314 in FIG. sixteen.

In particular, as illustrated in FIG. 5, an entropy decoder may comprise a plurality of entropy decoders 130, such as threads executing in different processing cores, where a deinterleaver may be configured, for each entropy slice, to direct its fragments 310 to the entropy decoder 44 associated with the corresponding entropy slice.

In other words, the entropy slices may be divided into slices, which in turn may be interleaved, and the decoder may contain a deinterleaver to deinterleave the slices, and may begin processing the entropy slices in parallel along paths 16 even before receiving any of the entropy slices as a whole. It should be remembered that the length of the fragments is preferably measured in the entropy coding domain rather than in the syntactic domain so as to correspond to, for example, a plurality of defined spatial parts/blocks in an image, etc., although the latter possibility will also be available.

The following describes the possible use of time dependencies. The same can be used in addition to, or alternatively to, the options for improving the probability estimate already described.

The waveform data processing pattern as described herein can be extended to entropy coding with a new calculation of the probabilities for each LCU to also use inter-frame temporal dependencies.

As you know, the probabilities will be reinitialized at the beginning of each frame (first LCU). Thus, the probabilities that were already obtained in the previous frame are lost. To reduce the loss in coding efficiency, the end state of the picture (cf. 320) or, in the case of an entropy slice, the end state of the slice (cf. 322) of the probabilities from the reference frame 324 to the first LCU 50 of the current frame 10 or entropy slice 12, respectively, can be transmitted (FIG. .17). Such data of the corresponding slice in the reference frame can be obtained not only at the final position, but also at the previous position in the reference slices, since the parallel waveform processing of data can also pass through frame boundaries, that is, while a frame slice is being encoded, the process of encoding the slice of the previous frame may not be finished yet. Therefore, the signaling may be used to indicate a reference position, or it may be indicated by a diagram.

Using the above remark, it is thus possible to set P _{{i, j}} for initialization in the initial phase 84 and 234, respectively, where {i, j} denotes the first CU 50 in the k-th entropy slice of the current array of samples, equal to, or depending at least on any T(P _{{i, j}'} ), where {i, j}' denotes the CU within the previous (in order of encoding array of samples, which may be equal to the order of presentation) array of samples, or a combination of several T (P _{{i, j}'} ). This can only be done for k=0, or for each entropy slice k∈{1...K}, where K denotes the number of entropy slices in the current frame. Temporal initialization may be done in addition to or alternatively to the spatial initialization described above. That is, P{i, j}, where {i, j} denotes the first CU 50 in the kth entropy slice, may be set to some combination (such as the mean value of sme) T(P _{{i, j}'} ) and T (P _{{i, j}spatial} ), where {i, j}' denotes the CU within the previous (previously decoded) array of samples, or a combination of several T(P _{{i, j}'} ), and {i, j} _spatial denotes CU within the previous entropy slice of the current array of samples. While {i, j}' is being considered, P _{{i, j}} , where {i, j} denotes the first (entropy-coded order 14) CU 50 in the k-th (entropy-coded order 14) entropy slice of the current array of samples, can be set equal to T(P _{{i, j}'} ), where {i, j}' denotes the last (in order of entropy coding) CU within the k-th (in order of entropy slices) entropy slice in the previous (in order of entropy coding) sample array encoding) sample array, or the last CU within the last (in entropy slice order) entropy slice in the previous (sample array encoding order) sample array. Again, this temporary initialization can only be performed for the first entropy slice in the array of samples.

The end state analysis process of the reference frame was tested with the probability adaptation method, these results are illustrated in FIG. 10 - fig. 19 (timeline).

Another possibility of using data from other frames is to exchange received probabilities between related LCUs. The main idea is based on the assertion that the properties of the key frame do not differ very much from the current frame. To speed up the learning of probabilities over an LCU in a frame, one can try to pass the end state of each LCU to the corresponding LCU in the current frame. This proposal is illustrated in Fig. eighteen.

Under the reference frame can be understood various possibilities. For example, the frame that is encoded last may be used as the reference frame. Otherwise, the last encoded frame only from the same temporal level may be assigned as a reference.

In addition, this approach can be combined with the methods already proposed above, such as using the last (slice) information from the reference frame, adapting the probabilities, and using the second upper line LCU.

The above spatial adaptation process can be modified to be

where N can be 1 or greater than 1 and {i, j} _1...N is selected from (is within) any previous (in order of 16 entropy slices) entropy slice 90 in the current sample array 10 and its associated chunk 12, respectively, and M may be 1 or greater than 1 and {i, j}' _1...M is within the previous array of samples 350. It may be that at least one of the CU 50 {i, j}' _{1. ..M is} combined with p(n). Regarding the possible choices of CU 50 {i, j} _1...N , reference is made to the above description. The "mean" function can be one of a weighted summation, median function, etc.

The above spatial adaptation process can be replaced by

where M can be 1 or greater than 1 and {i, j}' _1...M is within the previous array of samples. It may happen that at least {i, j}' _1...M is aligned with p(n). Regarding the possible choices {i, j} _1...N , reference is made to the above description. The "mean" function can be one of a weighted summation, median function, etc. It may happen that at least {i, j}' _1...M is aligned with p(n).

As a certain extension of the use of correlated information, an approach may be taken to use received data from other blocks from one or even more reference frames.

The previously mentioned methods use received information only from immediate neighbors in the current frame or in reference frames. However, this does not mean that the obtained probabilities in this case are the best. Adjacent LCUs, according to image divisions (difference information), do not always have the best probability models. It is assumed that the best results can be provided with the blocks on which the prediction will be performed. And thus, this corresponding block can be used as a reference for the current LCU.

Thus, in the above adaptation example, {i, j} _1...N and/or {i, j}' _1...M may be selected depending on the use of CUs as sources of prediction parameters for p(n).

The presented temporal probability adaptation/initialization schemes can also be used without entropy slices or with one entropy slice per frame.

In accordance with the latter aspect, an increase in the rate of probability adaptation is provided by combining the probability adaptation in the time domain of adjacent/related frames. What is described is a decoder, such as the decoder in FIG. 6, wherein the decoder is configured to recover a sequence of sample arrays from the entropy encoder data stream, and is configured to entropy decode the current frame of the entropy encoder data stream to recover the current sample array of the sequence of sample arrays, perform entropy decoding along the entropy encoding path, and use probability and adaptation estimates. probability estimates along the entropy encoding path using a previously decoded portion of the current frame, the entropy decoding stage being configured to initialize or determine a probability estimate for the current frame based on the probability estimates used in decoding the previously decoded frame of the entropy encoded data stream.

That is, for example, the probability estimates for the current frame are initialized based on the probability estimates resulting from the completion of decoding a previously decoded frame of the entropy encoded data stream. Therefore, the buffer requirements are low, as simply the final state of the probability estimates must be buffered before decoding of the current frame begins. Of course, this aspect is combined with the aspect in FIG. 1-9 in that for the first parts of each chunk 12, not only the probability estimates used for spatially adjacent parts in previous entropy slices (if available) are used, but also in a weighted manner, e.g., the end state of the probability estimates (e.g., spatially) corresponding entropy slice in the previous frame. Such data of the corresponding part in the reference frame can be obtained not only at the final position, but also from the previous position in the reference slices, since the data parallel waveform processing can also pass through frame boundaries, that is, when a frame slice is encoded, the previous slice encoding process frame may not be finished yet. Therefore, the signaling may be used to indicate the reference position, or it may be indicated by a diagram.

Additionally, for example, the probability estimates used to encode portions/blocks of a previously decoded frame are all buffered, not just the final state, and the decoder will entropy decode a predefined entropy slice (with reference to the above description of the spatially combined probability acquisition), perform entropy decoding of the current part (X) of the predefined entropy slice, based on the corresponding probability estimates of the predefined entropy slice, which are adapted using the previously decoded part of the predefined entropy slice (including p1, for example), and on the probability estimates that are used in the entropy decoding of the spatially corresponding part of the entropy slice of a previously decoded frame, if desired, additionally using the probability estimates that are used in the entropy decoding of a spatially neighboring, in the order of entropy slices, of the previous entropy a slice (a slice containing X, for example) in an adjacent part (such as p4) of a spatially adjacent entropy slice, as described above. As also described above, the spatial correspondence between the parts and the identification of the corresponding part to borrow the probability for the current frame among the previously decoded frame can be determined using motion information such as motion indices, motion vectors, etc., of the current part/block.

So far, a wave propagating during wave processing has been first described as propagating obliquely through one array of 10 samples, with encoding/decoding performed one array of samples after another. However, this is not necessary. Reference is made to FIG. 19. FIG. 19 shows a portion of a sequence of sample arrays, with the sample arrays of the sequence defined among it and depicted as arranged in sample array encoding order 380, which may or may not coincide with the presentation time order. Fig. 19, for example, shows the division of arrays of 10 samples into four entropy slices each. Already encoded/decoded entropy slices are shown as shaded. Four encoding/decoding streams (encoding/decoding stage) 382 are currently running on these four entropy slices 12 of the sample array having index n. However, FIG. 19 shows the remaining stream number 5, and it is possible that this additional stream 382, numbered 5 in FIG. 19 operates to encode/decode the next array of samples on the line, i.e. n+1, in chunks for which it is guaranteed that the corresponding reference chunks in the currently encoded/decoded frame n are already available, i.e. already processed by any of streams 1 -four. These chunks are reference, for example, in the predictions shown as 64 in FIG. 1.

Fig. 19 shows by way of example with a dotted line 384 propagating through an n+1 sample array that is already aligned with the boundary between the processed, i.e. encoded/decoded portion of the n sample array, i.e. the hatched portion within the n sample array on one side, and not yet processed portion, that is, the unshaded portion of the n-sample array on the other side. With the aid of the double-headed arrows of Fig. 19 also shows the maximum possible length of the motion vectors measured in the column and row directions, ie y _max and x _max , respectively. Accordingly, FIG. 19 also shows, with dotted line 386, an offset version of line 384, namely line 386 that is spaced apart from line 384 at the minimum possible distance so that the distance does not fall below y _max in the column direction and x _max in the row direction. As can be seen, there are coding units 50 in an n+1 sample array for which any reference chunk in the n sample array is guaranteed to be found to be entirely contained within the already processed chunk of that n sample array, namely those that are in the upper half of the n+1 sample array with respect to line 386. Accordingly, stream 5 is already in a position to work to decode/encode these coding units, as shown in FIG. 19. As can be seen, even the sixth thread can process the second entropy slice in the order of 16 entropy slices of an array of n+1 samples. In this regard, the wave propagates not only spatially, but also in the time domain through space-time space filled with a sequence of 30 sample arrays.

It should be noted that the wave aspect just mentioned does also work in combination with the above relationships of probability estimates over entropy slice boundaries.

Additionally, with respect to the above aspect of the fragment, it should also be noted that the division of entropy slices into smaller parts, ie fragments, is not limited to being performed in the entropy coding domain, ie in the entropy compressed domain. Thinking about the above discussion: the entropy slices as described above have the advantage of reducing the loss of coding efficiency despite the possibility of data wave processing due to the origin of probability estimates from previously encoded/decoded entropy slices of the same or previously encoded/decoded frame, i.e. initializing and/or adapting the probability estimates based on the probability estimates of such previous entropy slices. Each of these entropy slices is assumed to be entropy encoded/decoded by one stream in case of wave processing. That is, when entropy slices are divided, it is not necessary to process slices that can be encoded/decoded in parallel. Instead, the encoder is simply provided with the ability to output sections of the bitstream of its entropy slice before entropy encoding is completed, and the decoder is provided with the ability to process these sections, ie fragments, before receiving the remaining fragments of the same entropy slice. In addition, interleaving must be enabled on the receive side. To allow the last deinterleaving, however, it is not necessary to perform separation in the entropy coding domain. In particular, it is possible to perform the above division of entropy slices into smaller fragments without serious loss of coding efficiency by periodically simply reloading the internal state of the probability interval, i.e., the probability interval width value and the offset value, respectively, of the entropy encoding/decoding kernel. Probability estimates, however, are not reloaded. Instead, they are continuously updated/adapted from start to finish of entropy slices, respectively. With these measures, it is possible to divide entropy slices into separate fragments, and the division is carried out in the area of syntactic elements, and not in the area of the compressed bitstream. The partitioning may be the result of a spatial partitioning, as discussed below, to facilitate transmission to the decoder of boundaries between tiles. Each of the tiles may be provided with its own tiles header that reveals, for example, its starting position in the array of samples measured, for example, in coding order 14 relative to the starting position of the corresponding entropy slice along with its entropy slice index, or relative to the known location of the array 10 of samples. , such as the top left corner.

For a clearer description of the division of entropy slices into fragments in accordance with the latter embodiment, reference is made to FIG. 20. FIG. 20 shows, simply for illustrative purposes, an array of 10 samples as subdivided into four entropy sequences. The currently encoded portions of the 10-sample array are shown as shaded. Three threads currently perform entropy encoding of the 10 sample array and derive entropy slice fragments based on direct examination: see for example the first entropy slice in order of 16 entropy slices, which corresponds to chunk 12 of the 10 sample array. After section 12a of chunk 12 is encoded, the encoder generates chunk 390 from it, i.e., the entropy encoding core 80 performs some finishing routine to complete the arithmetic bitstream obtained from sub chunk 12a in the case of arithmetic encoding to generate chunk 390. The encoding routine then resumes with respect to the subsequent subportion 12b of the entropy slice 12 in order 14 coding, starting a new entropy bitstream. This implies, for example, that internal states such as the probability interval width value and the probability interval offset value of the entropy encoding core 80 are reloaded. Probability estimates, however, are not reloaded. They are left unchanged. This is illustrated in FIG. 20 with arrow 392. In FIG. 20 shows, for example, that entropy slice or chunk 12 is divided into more than two sub-chunks, and accordingly, even the second 1b chunk is subject to some entropy completion before reaching final chunk 12 in coding order 14, after which the next chunk on the line begins, and so on. d.

Simultaneously, another thread is processing the second entropy slice or chunk 12 in order of 16 entropy slices. After the end of the first subportion of this second entropy slice/portion 12, fragment 2a is output, after which the entropy encoding of the remainder of the second entropy slice begins, keeping, however, the probability estimate that is valid at the end of fragment 2a.

With the time axis 394 of FIG. 20 illustrates that fragments 390 are output as soon as they are completed. This results in an interleaving similar to the interleaving shown in FIG. 16. Each fragment can be packaged into a packet and transported to the decoding side through some transport layer in any order. The transport layer is illustrated using arrow 396.

The decoder must reassign tiles to sub-chunks 12a, 12b, and so on. For this purpose, each fragment 390 may have a header section 398 that reveals the initial location of its associated subportion 12a or 12b, that is, subportions of syntax elements describing the same thing are entropy encoded in the corresponding fragment. Using this information, the decoder is able to associate each chunk 390 with its entropy sequence and with its subportion within chunk 12 of that entropy slice.

For the purpose of illustrating FIG. 20 also shows by way of example the possibility that the transition between successive sub-portions 12a and 12b of the entropy slice 12 is not required to coincide with the boundary between successive coding units 50. Instead, this transition may be defined at a deeper level of the above exemplary division of coding units into multiple trees. The location information contained in headers 398 may indicate the start of the subportion associated with the current chunk 390 precisely enough to identify the corresponding subblock of the corresponding coding block, i.e., the location within the sequence of syntax elements that describes the corresponding subblock.

As it became clear from the above discussion, almost no loss in coding efficiency results from dividing entropy slices into fragments. It's just that the entropy termination and packetization processes may involve some loss in coding efficiency, but on the other hand the delay reduction is huge.

Again, it should be noted that the just mentioned spatial aspect of fragmentation does also work in combination with the above connections of probability estimation along entropy slice boundaries, in the spatial and temporal domains.

A decoder, such as the decoder in FIG. 6 may cancel the fragment transmission as follows. In particular, the decoder may check to which entropy slice the current fragment belongs. This check may be performed based on the above location information. It can then be checked whether or not the current fragment corresponds to the first sub-portion of the corresponding entropy slice portion along the entropy encoding path 14 . If yes, then the decoder may entropy decode the current slice while adapting the respective probability estimates and take into account the state of the respective probability estimates that are found at the end of entropy decoding of the current slice when another slice is entropy decoded that corresponds to a second sub-portion of a portion of a predefined entropy slice along the path of the entropy slice. coding. "Accounting" may involve setting the probability estimates at the beginning of fragment 1b equal to the probability estimates, finding them directly, by adapting the probability, starting from the state of the probability estimate at the beginning of fragment 1a, at the end of the 1a subportion of fragment 12a, or equal to combining it with entropy estimates probabilities from other entropy slices, as described above. While the probability initialization at the beginning of the first fragment 12a is considered, reference is made to the above discussion since it also forms the beginning of the corresponding entropy slice. In other words, if the current slice is the second or subsequent slice in order 14, then the decoder may entropy decode the current slice using probability estimates that depend on the probability estimates found at the end of the entropy-decoded slice that corresponds to the sub-chunk of the predefined entropy slice slice preceding the sub-chunk corresponding to the current fragment, along the trajectory 14 entropy encoding.

The above description shows various methods that can be used for parallel encoding and decoding, as well as used to optimize existing processes in the upcoming HEVC video coding standard. A short overview of entropy slices was presented. It is shown how they can be formed, what advantages can be provided by slicing, and what disadvantages can result from these methods. Many methods have been proposed that are believed to improve the process of learning the probabilities over the LCU (largest coding unit) in a frame, by better exploiting local dependencies between LCUs as well as temporal dependencies between LCUs of different frames. Various combinations are said to provide improvements to both concepts with and without parallelization of encoding and decoding.

Efficiency gains for high-efficiency coding, such as using the best combination of proposed approaches, are -0.4% for internal, -0.78% for low latency, and -0.63% for random access compared to HM3.0 without the use of entropy slices or -0.7% for internal, -1.95% for low latency, and -1.5% for random access compared to the entropy slice approach with normal re-initialization.

In particular, the following methods were presented above, including:

to use not only the local but also the temporal dependencies of the LCU to optimize the adaptation of the CABAC probabilities before encoding each LCU, see FIG. 1-9, 17 and 18;

to provide more flexibility in decoding, also entropy slices can be used so that certain areas in the frame become independent of each other;

to allow minimal signaling of the start position of the slice/entropy slices for parallelization, eg, waveform data processing, see FIG. 15;

to enable low latency transport in a parallelized encoder-transmitter-receiver-decoder environment by interleaved transport of entropy slices/slices, see FIG. sixteen.

All the methods mentioned above have been combined and tested in HM3.0. The results obtained, where the breakpoint is HM3.0 without any implementation of the entropy slice, are presented in Tables 1 and 2 (where 2LCU is the use of the second upper line LCU; 2LCU+Prob.Adap is the 2LCU method combined with the probability adaptation method; related to time - use of time dependencies (end state of the reference frame) with adaptation of probabilities for each LCU).

Table 1 Summarized RD results with 1 thread 1 stream 2LCU 2LCU+Prob.Adap Prob.Adap pertaining to time Interior 0.14 -0.31 -0.4 Low latency 0.66 0.09 -0.12 -0.78 random access 0.48 -0.1 -0.24 -0.63

table 2 Generalized RD results with 4 threads 4 threads 2LCU 2LCU+Prob.Adap Prob.Adap pertaining to time Interior 0.19 -0.27 -0.27 Low latency 1.01 0.54 0.63 0.17 random access 0.74 0.2 0.24 0.01

However, it is interesting to know how the proposed approaches affect wave processing with re-initialization of probabilities at the beginning of each LCU line. These results are illustrated in Tables 3 and 4 (where orig_neiInit is a comparison of HM3.0 without using entropy slices using reinitialized entropy slices).

Table 3 Summarized RD results with 1 thread. The reference element is the new initialization 1 stream orig_
neiInit 2LCU 2LCU+ProbAdap ProbAdap pertaining to time Interior 0.3 -0.15 -0.62 -0.7 random access 0.9 -0.4 -0.99 -1.13 -1.51 Low latency 1.19 -0.51 -1.08 -1.29 -1.95

Table 4 Generalized RD results with 4 threads. The reference element is the new initialization 4 threads orig_
neiInit 2LCU 2LCU+ProbAdap ProbAdap pertaining to time Interior 0.34 -0.14 -0.61 -0.61 random access 1.18 -0.43 -0.96 -0.96 -1.15 Low latency 1.61 -0.5 -0.92 -1.05 -1.41

The above results show that a much greater use of dependencies within and between frames and the rational application of the information already obtained prevents the average loss.

The waveform approach for encoding and decoding HEVC video combines the ability to exploit dependencies between adjacent LCUs as well as frame time dependencies with the concept of wavelet parallel processing. In this way, losses can be reduced and an increase in efficiency can be achieved.

An increase in the probability adaptation rate is provided by computing the probability adaptation of spatially adjacent entropy slices.

As already mentioned above, all of the aforementioned aspects can be combined with each other, and thus the mention of certain embodiments in relation to a certain aspect must, of course, also refer to other aspects.

Although some aspects are described in the context of a device, it is clear that these aspects also represent a description of the corresponding method, where the block or device corresponds to a method step or a feature of a method step. Likewise, the aspects described in the context of a method step also provide a description of the corresponding block or element or feature of the corresponding device. Some or all of the steps of the method may be performed (or used) by a hardware device such as a microprocessor, programmable computer, or electronic circuit. In some embodiments, one or more of the most important steps of the method may be performed using such a device.

The inventive coded signals mentioned above may be stored on a digital storage medium or may be transmitted over a transmission medium such as a wireless transmission medium or a wired transmission medium such as the Internet.

Depending on certain implementation requirements, embodiments of the invention may be implemented in hardware or software. Embodiment may be performed using a digital storage medium such as floppy disk, DVD (Digital Versatile Disk), Blu-ray, CD (Compact Disk), ROM (Read Only Memory), PROM (Programmable ROM), EPROM (Erasable Programmable ROM) , EEPROM (Electrically Erasable Programmable ROM) or flash memory having stored thereon electronically readable control signals that interact (or can interact) with a programmable computer system in such a way that the corresponding method is performed. Therefore, the digital storage medium may be a computer-readable medium.

Some embodiments of the invention comprise a storage medium having electronically readable control signals that can interact with a programmable computer system such that one of the described methods is performed.

In general, embodiments of the present invention may be implemented as a computer program product with program code, the program code being used to perform one of the methods when the computer program product is running on a computer. The program code may, for example, be stored on a computer-readable medium.

Other embodiments include a computer program for performing one of the described methods, stored on a computer-readable medium.

In other words, an embodiment of the inventive method is therefore a computer program having program code for performing one of the described methods when the computer program is executed on a computer.

A further embodiment of the inventive methods is therefore a storage medium (either a digital storage medium or a computer-readable medium) containing a computer program recorded thereon for performing one of the described methods. The storage medium, digital storage medium, or recorded medium is typically tangible and/or non-transitory.

An additional embodiment of the inventive method is therefore a data stream or signal sequence representing a computer program for performing one of the described methods. The data stream or signal sequence may, for example, be configured to travel over a data connection, such as the Internet.

An additional embodiment comprises processing means, such as a computer or programmable logic device, configured or adapted to perform one of the described methods.

An additional embodiment includes a computer, a computer program installed on it for performing one of the described methods.

An additional embodiment according to the invention comprises an apparatus or system configured to move to the receiver (eg, electronically or optically) a computer program to perform one of the described methods. The receiver may, for example, be a computer, mobile device, storage device, or the like. The device or system may, for example, include a file server for transferring a computer program to a receiver.

In some embodiments, a programmable logic device (eg, a field programmable gate array) may be used to perform some or all of the functionality of the described methods. In some embodiments, a user-programmable gate array may work with a microprocessor to perform one of the methods described. In general, the methods are preferably performed using any hardware device.

In a first aspect of the present invention, a decoder is provided for recovering an array of video samples from an entropy encoded data stream, the decoder comprising: contains entropy encoded data for the corresponding row of the array of samples, wherein to entropy decode the current entropy slice from the plurality of entropy slices, the entropy decoder is configured to: and indicates the difference between the starting position of the previous entropy slice in the data stream and the starting position of the current entropy slice, with the current entropy slice corresponding to t the current row of the array of samples, and the previous entropy slice corresponds to the previous row of the array of samples, and the current and previous rows are consecutive rows of the array of samples, and the previous row is spatially above the current row and getting the starting position in the data stream of the current entropy slice based on the sum of the starting position the previous entropy slice and a position difference value, wherein the entropy decoding of the previous entropy slice precedes the entropy decoding of the current entropy slice.

In said decoder, the entropy decoding of the previous entropy slice immediately precedes the entropy decoding of the current entropy slice, so that no other entropy slice is entropy decoded between the entropy decoding of the current and previous entropy slices.

In said decoder, exponential Golomb encoding provides a prefix bin string and a suffix bin string associated with the position delta value, and the prefix bin string identifies one of the plurality of value ranges to which the position delta value belongs.

In said decoder, each row has the same number of sample array blocks.

In said decoder, the entropy decoder is configured to perform entropy decoding for each entropy slice along a corresponding entropy encoding path running parallel along the block rows.

In said decoder, the entropy decoder is configured to entropy decode a plurality of entropy slices in slice subset order, wherein the slice subset order is chosen in such a way that, according to the slice subset order, different portions follow each other in a direction rotated relative to the entropy coding trajectories of the entropy slices.

In said decoder, the entropy slices are subdivided into fragments, and the decoder further comprises a deinterleaver for deinterleaving the fragments, and the entropy decoder is configured to start entropy decoding of the entropy slices in parallel along the entropy encoding paths, even before obtaining all of any of the entropy slices.

In said decoder, the sample array is the current sample array from a sequence of sample arrays, and the entropy decoder is configured to entropy decode an entropy slice using a first probability estimate based on a previously decoded portion of that entropy slice and a second probability estimate obtained from entropy decoding of a previously decoded frame. A that refers to one of the arrays of selections other than the current array of selections.

Said decoder is at least part of a programmable logic device, a user-programmable gate array, a microprocessor, a computer, or a hardware component.

In a second aspect of the present invention, there is provided a method for recovering an array of video samples from an entropy encoded data stream, comprising: entropy decoding a plurality of entropy slices in the data stream to recover different portions of the array of video samples, each of the plurality of entropy slices containing entropy encoded data for the corresponding row of the sample array, wherein for entropy decoding the current entropy slice of the plurality of entropy slices, the entropy decoding includes: decoding the position difference value of the current entropy slice using exponential Golomb coding, wherein the position difference value is extracted from the data stream and indicates the difference between the starting position of the previous entropy slice in the data stream and the starting position of the current entropy slice, while the current entropy slice corresponds to the current row of the array of samples, and the previous the entropy slice corresponds to the previous row of the array of samples, where the current and previous rows are consecutive rows of the array of samples, and the previous row is located spatially above the current row, and obtaining the initial position in the data stream of the current entropy slice based on the sum of the initial position of the previous entropy slice and the position difference value, wherein the entropy decoding of the previous entropy slice precedes the entropy decoding of the current entropy slice.

In said method, the entropy decoding of the previous entropy slice immediately precedes the entropy decoding of the current entropy slice, so that no other entropy slice is entropy decoded between the entropy decoding of the current and previous entropy slices.

In said method, exponential Golomb encoding provides a prefix bin string and a suffix bin string associated with the position delta value, and the prefix bin string identifies one of the plurality of value ranges to which the position delta value belongs.

In said method, each row has the same number of sample array blocks.

In said method, entropy decoding of each of the plurality of entropy slices is performed along a respective entropy encoding path running parallel along the block rows.

In said method, the entropy slices are subdivided into slices, and the method further comprises deinterleaving the slices and starting entropy decoding of the entropy slices in parallel along the entropy encoding paths, even before obtaining all of any of the entropy slices.

In the mentioned method, the entropy decoding of a plurality of entropy slices is performed in the order of the subset of slices, and the order of the subset of slices is chosen in such a way that, in the order of the subset of slices, different portions follow each other in a direction rotated relative to the trajectories of the entropy coding of the entropy slices.

In said method, the sample array is the current sample array from a sequence of sample arrays, and the entropy decoding of the entropy slice uses a first probability estimate based on a previously decoded portion of the entropy slice and a second probability estimate obtained from entropy decoding of a previously decoded frame belonging to one of the arrays. samples other than the current array of samples.

In a third aspect of the present invention, there is provided a method for encoding an array of video samples into an entropy encoded data stream, comprising the step of: entropy encoding a plurality of entropy slices into an entropy encoded data stream, each of the plurality of entropy slices containing entropy encoded data for a corresponding row of the array of samples, wherein the entropy encoding includes encoding into the data stream, for the current entropy slice of the plurality of entropy slices, using exponential Golomb coding, a position difference value associated with a starting position in the data stream of the current entropy slice, the position difference value indicating the difference between the starting position in the data stream of the previous entropy slice and the starting position of the current entropy slice, while the current entropy slice corresponds to the current row of the array of samples, and the previous entropy slice corresponds to the previous st sample array, where the current and previous rows are consecutive rows of the sample array, and the previous row is spatially above the current row, and the starting position of the current entropy slice in the data stream is to be obtained based on the sum of the starting position of the previous entropy slice and the position difference value, and entropy encoding the previous entropy slice precedes the entropy encoding of the current entropy slice.

In said method, the entropy encoding of the previous entropy slice immediately precedes the entropy encoding of the current entropy slice, so that no other entropy slice is entropy encoded between the entropy encoding of the current and previous entropy slices.

In said method, exponential Golomb encoding provides a prefix bin string and a suffix bin string associated with the position delta value, and the prefix bin string identifies one of the plurality of value ranges to which the position delta value belongs.

In said method, each row has the same number of sample array blocks.

In said method, the entropy encoding of each of the plurality of entropy slices is performed along a respective entropy encoding path running parallel along the rows of blocks.

In said method, the entropy slices are subdivided into fragments, and the method further comprises deinterleaving the fragments into a data stream.

In the mentioned method, the entropy coding of a plurality of entropy slices is performed in the order of a subset of slices, and the order of the subset of slices is chosen in such a way that, in the order of the subset of slices, different portions follow each other in a direction rotated relative to the trajectories of entropy coding of entropy slices.

In said method, the array of samples is the current array of samples from a sequence of arrays of samples, and the entropy encoding of the entropy slice uses a first probability estimate based on the previously encoded portion of the entropy slice and a second probability estimate obtained by entropy encoding the previously encoded frame belonging to one of the arrays. samples other than the current array of samples.

In a fourth aspect of the present invention, a durable computer-readable storage medium for storing video related data is provided, comprising: a data stream stored on a durable computer-readable medium, the data stream comprising entropy encoded information associated with an array of video samples, the entropy encoded information including: a plurality of entropy slices, each of the plurality of entropy slices containing entropy encoded data for a corresponding row of the array of samples, wherein the entropy encoding of the array of samples includes encoding into a data stream, for the current entropy slice of the plurality of entropy slices, using exponential Golomb coding, the values position difference associated with the start position in the data stream of the current entropy slice, and the position difference value indicates the difference between the start position in the data stream of the previous entropy slice and the start position of the current entropy slice, where the current entropy slice corresponds to the current row of the array of samples, and the previous entropy slice corresponds to the previous row of the array of samples, and the current and previous rows are consecutive rows of the array of samples, and the previous row is located spatially above the current row, the starting position of the current entropy slice in the data stream is to be obtained based on the sum of the starting position of the previous entropy slice and the position difference value, and the entropy encoding of the previous entropy slice precedes the entropy encoding of the current entropy slice.

In said durable computer-readable medium, exponential Golomb encoding provides a prefix bin string and a suffix bin string associated with the position delta value, and the prefix bin string identifies one of the plurality of value ranges to which the position delta value belongs.

The above described embodiments are merely illustrative of the principles of the present invention. It is contemplated that modifications and changes to the structures and details described will be apparent to those skilled in the art. Therefore, it is intended that the invention be limited only by the scope of the claims of the patent in question, and not by the specific details provided through the description and explanation of the embodiments in this work.

Claims (45)

1. A decoder for restoring an array of video samples from an entropy encoded data stream, the decoder comprising: an entropy decoder configured to entropy decode a plurality of entropy slices in the data stream to recover different portions of the video sample array, each of said plurality of entropy slices containing entropy encoded data for a corresponding row of the sample array, wherein, for entropy decoding of the current entropy slice from said plurality of entropy slices, the entropy decoder is configured to: decoding the position difference value of the current entropy slice using exponential Golomb coding, wherein the position difference value indicates the difference between the start position of the previous entropy slice within said data stream and the start position of the current entropy slice, the current entropy slice corresponding to the current row of the array of samples, and the previous the entropy slice corresponds to the previous row of the array of samples, where the current and previous rows are consecutive rows of said array of samples, and obtaining an initial position within the data stream of the current entropy slice based on the sum of the initial position of the previous entropy slice and the position difference value, wherein the entropy decoding of the previous entropy slice precedes the entropy decoding of the current entropy slice. 2. The decoder of claim 1, wherein the entropy decoding of the previous entropy slice immediately precedes the entropy decoding of the current entropy slice such that no other entropy slice is entropy decoded between the entropy decoding of the current and previous entropy slices. 3. The decoder of claim 1, wherein the exponential Golomb encoding provides a prefix bin string and a suffix bin string associated with the position difference value, and the prefix bin string identifies one of the plurality of value ranges to which the position difference value belongs. 4. The decoder according to claim 1, wherein each row has the same number of blocks of said array of samples. 5. The decoder of claim 1, wherein the entropy decoder is configured to perform, for each entropy slice, entropy decoding along a corresponding entropy encoding path running parallel along the block rows. 6. The decoder of claim 5, wherein the entropy decoder is configured to entropy decode the plurality of entropy slices in slice subset order, the order of the slice subset being chosen such that, in the order of the slice subset, different portions follow each other in a direction rotated relative to the trajectories of the entropy encoding of entropy slices. 7. The decoder of claim 5, wherein the entropy slices are subdivided into slices, and the decoder further comprises a deinterleaver for deinterleaving the slices, and the entropy decoder is configured to start entropy decoding of the entropy slices in parallel along the entropy encoding paths, even before all of any of the entropy slices is obtained. . 8. The decoder of claim 1, wherein the sample array is a current sample array from a sequence of sample arrays, and the entropy decoder is configured to entropy decode an entropy slice using a first probability estimate based on a previously decoded portion of that entropy slice and a second probability estimate, obtained by entropy decoding a previously decoded frame belonging to one of the sample arrays other than the current sample array. 9. The decoder of claim 1, wherein the decoder is at least part of a programmable logic device, field-programmable gate array, microprocessor, computer, or hardware component. 10. A method for recovering an array of video samples from an entropy encoded data stream, the method comprising: entropy decoding a plurality of entropy slices in the data stream to reconstruct different portions of the video sample array, each of said plurality of entropy slices containing entropy encoded data for a respective row of the sample array, wherein, for entropy decoding of the current entropy slice from said plurality of entropy slices, the entropy decoding includes: decoding the position difference value of the current entropy slice using exponential Golomb coding, wherein the position difference value indicates the difference between the start position of the previous entropy slice within said data stream and the start position of the current entropy slice, the current entropy slice corresponding to the current row of the array of samples, and the previous the entropy slice corresponds to the previous row of the array of samples, where the current and previous rows are consecutive rows of said array of samples, and obtaining an initial position within the data stream of the current entropy slice based on the sum of the initial position of the previous entropy slice and the position difference value, wherein the entropy decoding of the previous entropy slice precedes the entropy decoding of the current entropy slice. 11. The method of claim 10, wherein the entropy decoding of the previous entropy slice immediately precedes the entropy decoding of the current entropy slice such that no other entropy slice is entropy decoded between the entropy decoding of the current and previous entropy slices. 12. The method of claim 10, wherein exponential Golomb encoding provides a prefix bin string and a suffix bin string associated with the position delta value, and the prefix bin string identifies one of a plurality of value ranges to which the position delta value belongs. 13. The method of claim 10, wherein each row has the same number of blocks of said array of samples. 14. The method of claim 10, wherein the entropy decoding of each of the plurality of entropy slices is performed along a corresponding entropy encoding path running parallel along the rows of blocks. 15. The method of claim 14, wherein the entropy slices are subdivided into slices, and the method further comprises deinterleaving the slices, and starting entropy decoding of the entropy slices in parallel along the entropy encoding paths, even before obtaining all of any of the entropy slices. 16. The method of claim 14, wherein the entropy decoding of the plurality of entropy slices is performed in slice subset order, wherein the order of the slice subset is chosen such that, in the order of the slice subset, different portions follow each other in a direction rotated relative to the entropy encoding paths of the entropy slices. . 17. The method of claim 10, wherein the sample array is the current sample array from a sequence of sample arrays, and the entropy decoding of the entropy slice uses a first probability estimate based on a previously decoded portion of that entropy slice and a second probability estimate obtained from the entropy decoding previously a decoded frame that belongs to one of the sample arrays other than the current sample array. 18. A method for encoding an array of video samples into an entropy encoded data stream, the method comprising: entropy encoding a plurality of entropy slices into an entropy encoded data stream, wherein each of said plurality of entropy slices contains entropy encoded data for a corresponding row of the sample array, wherein the entropy encoding includes encoding into the data stream, for the current entropy slice of the plurality of entropy slices, using exponential Golomb coding, a position difference value associated with an initial position within the data stream of the current entropy slice, the position difference value indicating the difference between the initial position within the data stream of the previous entropy slice and the starting position of the current entropy slice, while the current entropy slice corresponds to the current row of the array of samples, and the previous entropy slice corresponds to the previous row of the array of samples, and the current and previous rows are consecutive rows of the mentioned array of samples, the start position of the current entropy slice within the data stream is to be obtained based on the sum of the start position of the previous entropy slice and the position difference value, and the entropy encoding of the previous entropy slice precedes the entropy encoding of the current entropy slice. 19. The method of claim 18, wherein the entropy encoding of the previous entropy slice immediately precedes the entropy encoding of the current entropy slice such that no other entropy slice is entropy encoded between the entropy encoding of the current and previous entropy slices. 20. The method of claim 18, wherein exponential Golomb encoding provides a prefix bin string and a suffix bin string associated with the position delta value, and the prefix bin string identifies one of a plurality of value ranges to which the position delta value belongs. 21. The method of claim 18, wherein each row has the same number of blocks of said sample array. 22. The method of claim 18, wherein the entropy encoding of each of the plurality of entropy slices is performed along a corresponding entropy encoding path running parallel along the rows of blocks. 23. The method of claim 22, wherein the entropy slices are subdivided into chunks, and the method further comprises deinterleaving the chunks into a data stream. 24. The method of claim 22, wherein the entropy encoding of the plurality of entropy slices is performed in slice subset order, wherein the order of the slice subset is chosen such that, in the order of the slice subset, different portions follow each other in a direction rotated relative to the entropy encoding paths of the entropy slices. . 25. The method of claim 18, wherein the sample array is the current sample array from a sequence of sample arrays, and the entropy encoding of the entropy slice uses a first probability estimate based on a previously coded portion of that entropy slice and a second probability estimate obtained from the entropy encoding previously an encoded frame that belongs to one of the sample arrays other than the current sample array. 26. Non-volatile computer-readable media for storing video-related data, comprising: a data stream stored on that durable computer-readable medium, wherein the data stream contains entropy encoded information associated with an array of video samples, the entropy encoded information including a plurality of entropy slices, each of said plurality of entropy slices containing entropy encoded data for a respective row array of samples, wherein the entropy encoding of the array of samples includes encoding into the data stream, for the current entropy slice of the plurality of entropy slices, using exponential Golomb coding, a position difference value associated with an initial position within the data stream of the current entropy slice, the position difference value indicating the difference between the starting position within the data stream of the previous entropy slice and the starting position of the current entropy slice, while the current entropy slice corresponds to the current row of the array of samples, and the previous entropy slice corresponds to the previous row of the array of samples, and the current and previous rows are consecutive rows of the said array of samples, the start position of the current entropy slice within the data stream is to be obtained based on the sum of the start position of the previous entropy slice and the position difference value, and the entropy encoding of the previous entropy slice precedes the entropy encoding of the current entropy slice. 27. The durable computer-readable medium of claim 26, wherein the exponential Golomb encoding provides a prefix bin string and a suffix bin string associated with the position difference value, and the prefix bin string identifies one of the plurality of value ranges to which the position difference value belongs.

Images