JP6664454B2 - Method and apparatus for data hiding of a multilayer structured coding unit - Google Patents

Description

  The present invention relates to the provision of a method for hiding the value of a hierarchically structured coding unit (hierarchically layered coding unit) among other values included in a coding unit (coding method). I do. Furthermore, the invention relates to a method (decoding method) for reconstructing hidden data from encoded coding units. The invention also relates to the implementation of these encoding and / or decoding methods in the device and on the (non-transitory) computer readable medium.

  Lossy data compression has numerous applications, especially in communications, broadcasting, entertainment, and security. Video compression is a difficult task because transmitting high quality, high resolution pictures over existing communication channels requires a large compression ratio. This task becomes even more difficult in the context of wireless and mobile communications, or real-time encoding of media.

  The recently adopted ITU-T H.265 / HEVC standard (ISO / IEC 23008-2: 2013, `` Information technology-High efficiency coding and media delivery in heterogeneous environments-Part 2: High efficiency video coding '', November 2013) Declares a set of modern video coding tools that justify the trade-off between coding efficiency and computational complexity. An overview of the ITU-T H.265 / HEVC standard can be found in a paper by Gary J. Sullivan, `` Overview of the High Efficiency Video Coding (HEVC) Standard '', in IEEE Transactions on Circuits and Systems for Video Technology, Vol. 22, No. 12, Dec. 2012, which is incorporated herein by reference in its entirety.

  Like the ITU-T H.264 / AVC video coding standard, the HEVC / H.265 video coding standard specifies that a source picture be divided into blocks, eg, coding units (CUs). Each of the CUs may be further divided into smaller CUs or prediction units (PUs). PUs may be intra or inter predicted depending on the type of processing applied to the pixels of the PU. For inter prediction, PU represents a region of pixels that is processed by motion compensation using the motion vector specified for the PU. In intra prediction, a PU specifies a prediction mode for a set of transform units (TUs). The TUs can have different sizes (eg, 4x4, 8x8, 16x16, and 32x32 pixels) and can be processed differently. Transform coding is performed on the TU, that is, the prediction error is transformed by a discrete cosine transform (DCT) and quantized. The resulting quantized transform coefficients are grouped into CGs, each CG having 16 quantized transform coefficients.

  As pointed out above, the core tools of these standards or similar dedicated codecs for encoding blocks of pictures include inter and intra prediction, spectral transforms (e.g., discrete cosine transform or its integer approximation), And quantization. Inter and intra prediction tools are used to generate a prediction signal for a given block. On the encoder side, the difference between the source block and its prediction, the so-called residual signal, is transformed into a spectrum, i.e. the source block pixels are represented by transform coefficients in the frequency domain. Further, the coefficients are quantized. Non-zero and zero quantized transform coefficients are often referred to as significant and insignificant coefficients, respectively. All syntax elements including the quantized transform coefficients and side information (eg, an intra prediction mode for intra coding and a motion vector for inter coding) are binarized and entropy coded. The portion of the entropy coded coefficients in the compressed H.265 / HEVC bit stream can exceed 80%.

The steps for encoding the quantized transform coefficients are as follows.
Encoding the position of the last significant coefficient, ie the last non-zero quantized transform coefficient.
-Encoding a significance map used to recover the positions of all non-zero coefficients.
-Sign encoding of significant coefficients.
-Significant coefficient magnitude encoding.

  These steps are performed in situations where the quantized transform coefficients are divided into so-called coefficient groups (CG). Each CG is typically a subset of 4x4 coefficients.

  Explicit code encoding requires that one sign bit is encoded per significant coefficient. However, a new tool called Sign Bit Hiding (SBH) has been adopted for the ITU-T H.265 / HEVC standard. The basic idea behind this technique is that the sign of the first significant coefficient in a given CG is implicitly determined using a parity check of the sum of the least significant bits of the significant coefficients belonging to that CG. It is to instruct. This tool is applied not to all CGs but only to CGs that meet the threshold condition, ie the difference between the positions of the first significant coefficient and the last significant coefficient should be 4 or more. According to the results presented by the Joint Collaborative Team on Video Coding (JCT-VC), which was responsible for developing the H.265 / HEVC standard, this tool supports a wide range of video sequences used in JCT-VC testing. The bit rate so that the same quality is obtained. This confirms that especially SBH, and generally data hiding, can be an efficient compression tool.

  In addition to video compression, data hiding may be used in various digital rights management (DRM) applications (eg, digital watermarking). This is one of the DRM techniques used to embed digital information into carrier signals (e.g., Tirkel et al., "Electronic Water Mark", Digital Image Computing: Techniques and Applications (DICTA), 1993, Macquarie University 666-673). This concealment information can also be used to verify the authenticity or integrity of the carrier signal or to indicate the identity of its owner.

  In that case, if several concealment operations are to be performed on the same set of target values, these operations can potentially interfere with each other. This interference occurs when the concealment operation modifies this set of values without considering the effect of this modification on the data that was concealed during the previous concealment operation. Thus, a simple combination of concealment operations may result in a non-decodable bitstream.

  As an example of such a situation, we could consider concealment of a set of flags within the TU's quantized transform coefficients when SBH should be performed on these coefficients as well. is there. If the flag concealment operation is performed independently (ie, does not match the SBH), the extraction of the concealed code may be performed incorrectly.

ISO / IEC 23008-2: 2013, `` Information technology-High efficiency coding and media delivery in heterogeneous environments-Part 2: High efficiency video coding '', November 2013 A paper by Gary J. Sullivan, "Overview of the High Efficiency Video Coding (HEVC) Standard", in IEEE Transactions on Circuits and Systems for Video Technology, Vol. 22, No. 12, December 2012. Tirkel et al., `` Electronic Water Mark, '' Digital Image Computing: Techniques and Applications (DICTA), 1993, Macquarie University, 666-673.

  One object proposes a novel method for coding a plurality of data of a data unit to be coded (coding unit) based on an input set of data (values) contained in said coding unit. That is. Another object is to propose a method for decoding such data again from an encoded data unit. Yet a further object is to propose a data concealment mechanism in which several concealment operations can each be performed without causing interference, resulting in coded data that cannot be decoded. .

  According to the first aspect of the invention, the data units to be coded have a hierarchical structure. This means, for example, that the individual values of the data belonging to the coding unit correspond to a hierarchical structure comprising several layers of the coding unit. The idea of this first aspect is to use a data concealment pattern to conceal information (eg, values) belonging to each given layer based on other information of the coding unit. Other information to which each data hiding pattern applies may be considered an input set of values for each data hiding pattern. Some data hiding patterns may be based on the same input set of values, partially overlapping input sets of values, or input sets of distinctly different values, or combinations thereof.

In accordance with a first exemplary implementation of the first aspect, a method is provided for hiding values of a hierarchically structured coding unit among other values included in the coding unit (encoding method). In this way, a layered stack of data hiding patterns is realized. The data concealment pattern is for concealing the value of the coding unit in different layers of the coding unit. Each of the data concealment patterns has a check function associated with it that is used to conceal one or more of said values of the coding unit in one of the layers of the coding unit. For each of the data hiding patterns, the following is performed.
(i) calculating a check function for each one of the data hiding patterns based on the value selected by the respective data hiding pattern from the other values of the coding unit;
(ii) determining whether the result of the check function corresponds to the value of the coding unit to be hidden by a respective data hiding pattern, and
(iii) if not, respectively, from the other values of the coding unit such that the result of the check function of step (ii) corresponds to the value of the coding unit to be concealed by the respective data concealment pattern Modifying at least one of said values selected by said data hiding pattern.

  The encoded coding unit includes other values of the coding unit as modified in step (iii) for all data hiding patterns.

  In another second implementation of the first implementation of the first aspect, the layered stack of data hiding patterns defines a decoupled data hiding pattern. The decoupled data hiding pattern is a combination of the remaining data hiding patterns whose hidden values restored using any of the data hiding patterns of the stack are defined in the stack (e.g., linear combinations). Can be considered to satisfy the criterion that it cannot be derived from If the layered stack of data hiding patterns is decoupled, applying it to the same input data set of values does not cause interference between the different data hiding patterns in the stack, and is therefore The values can be restored appropriately. Such interference can occur, for example, without considering the effects of modifying one or more values of the input set of values in a previous data hiding operation, such that the concealment operation is performed on the values of the input set of values (i.e., , The value selected by the data concealment pattern used in the concealment operation.

  Further, in a third implementation of the previous implementation of the first aspect, each layer of the coding unit may be associated with one or more data hiding patterns.

  In a further fourth implementation of the previous implementation of the first aspect, the data hiding pattern is a decimation-based data hiding pattern, one or more regular data hiding patterns, and one or more pseudo-random data hiding patterns. At least one of the following.

  According to a fifth implementation of the previous implementation of the first aspect, the method may further include performing for each of the data hiding patterns: If n, the algorithm used to modify at least one of the values selected by the respective data hiding pattern from the other values of the coding unit in step (iii) is the same as in step (ii) Without modifying the value so that the result of the check function corresponds to the value of the coding unit to be hidden by the respective data hiding pattern, the result of the check function in step (ii) is hidden by the respective data hiding pattern. Steps (i) to (iii) are repeated until it corresponds to the value of the coding unit that should be.

  According to a sixth implementation of the previous implementation of the first aspect, using the first data concealment pattern and the second data concealment pattern of the data concealment patterns, respectively, using steps (i) to (iii) ), One or more other values of the coding unit that have been selected by the first data hiding pattern and that have been modified in step (iii), When selected by the data hiding pattern, when the steps (i) to (iii) are performed on the second data hiding pattern, they are not corrected again in the step (iii).

  According to a seventh implementation of the previous implementation of the first aspect, the at least one value modified in step (iii) for one of the data hiding patterns is another one of the data hiding patterns. It is considered instead of the original value when performing steps (i) to (iii) on the pattern.

  According to an eighth possible implementation of the first aspect, there is provided a method of reconstructing a hidden value from an encoded coding unit (decoding method). Again, a layered stack of data hiding patterns is realized. This stack is used by the encoder to hide the value of the coding unit. Each of the data hiding patterns has a check function associated with it. In this method, for each of the data hiding patterns, the check function of the respective one of the data hiding patterns is based on a value selected by the respective data hiding pattern from the other value of the coding unit. The result of the calculated and checked function corresponds to one of the reconstructed hidden values.

  According to a ninth implementation of the preceding implementation of the first aspect, the value included in the coding unit based on when the data concealment pattern check function is calculated is a value of a lowest layer of the coding unit. .

  According to the tenth implementation of the ninth implementation of the first aspect, the check function of the data hiding pattern associated with a layer other than the lowest layer of the coding unit, each of the values of the lowest layer of the coding unit Is calculated for the value selected by each one of the data hiding patterns from the subset of.

  According to the eleventh implementation of the tenth implementation of the first aspect, the size of each one of the data hiding patterns is determined based on the size of the subset for which the check function of each data hiding pattern is calculated. Is done.

  According to the twelfth implementation of the tenth or eleventh implementation of the first aspect, each subset in a layer other than the lowest layer is the lowest of coding units hierarchically belonging to the organizational data unit of each layer. Contains the value of the hierarchy.

  According to a thirteenth implementation of the twelfth implementation of the first aspect, the number and / or size of the data hiding patterns applicable to each organizational data unit of each layer is determined by the respective check function being calculated. The determination is based on the size of the subset of values belonging to each organizational data unit of the layer.

  According to the fourteenth implementation of the tenth to thirteenth implementations of the first aspect, the size of each one of the data hiding patterns is the size of the subset for which the check function of each data hiding pattern is calculated. Is equal to or a divisor of the size. In other words, the size of the subset for which the check function of each data hiding pattern is calculated is equal to or an integer multiple of the size of each one of the data hiding patterns.

  According to a fifteenth implementation of the previous implementation of the first aspect, the value of the coding unit represents a block of pixels of an image, the coding unit comprising one or more prediction units, one or more transform units , And one or more coefficient groups, wherein the data concealment pattern is associated with a layer of coding units corresponding to the prediction unit, the transform unit, and the coding unit, respectively. Further, each of the data hiding patterns hides one or more values of the associated layer of the coding unit.

  According to a sixteenth implementation of the fifteenth implementation of the first aspect, the data concealment pattern is associated with a coefficient group, a prediction unit, a transform unit, and a layer of coding units corresponding to the coding units, respectively, wherein the coefficient group is It is part of the bottom layer of the coding unit.

  According to a seventeenth implementation of the previous implementation of the first aspect, the data concealment pattern check function is calculated based on the value of the coefficient group.

  According to the eighteenth implementation of the fifteenth to seventeenth implementations of the first aspect, the data concealment pattern is for concealing a reference sample filtering flag for intra prediction for the transform unit of the coding unit. One or more data hiding patterns.

  According to a nineteenth implementation of the fifteenth to eighteenth implementations of the first aspect, the data concealment pattern comprises one or more data concealments for concealing at least one of a prediction mode index and a size of a prediction unit. Including patterns.

  According to a twentieth implementation of the fifteenth to nineteenth implementations of the first aspect, the data concealment pattern comprises a data concealment pattern for embedding a watermark in a coding unit.

  According to the fifteenth to twentieth implementations of the first aspect, the data concealment pattern is included in the data concealment pattern for concealing the code bits of the coefficients of the coefficient group.

  According to a twenty-second implementation of the first to fourteenth implementations of the first aspect, the value of the coding unit represents a block of image data, a block of audio data, a block of a document to be encoded.

  According to a twenty-third implementation of the first to twenty-second implementations of the first aspect, one of the concealed values allows to confirm the authenticity of the value contained in the coding unit.

A twenty-fourth implementation of the first aspect of the present invention comprises an encoding device for concealing a value of a hierarchical coding unit among other values included in the coding unit. The encoding apparatus is provided with a layered stack of data concealment patterns for concealing said values of the coding units at different layers of the coding units, each of the data concealment patterns being coded at one of the layers of the coding unit. It has a check function associated with it to hide one or more of said values of the unit. The encoding device further comprises a processing unit configured to perform the following steps for each of the data concealment patterns.
(i) calculating a check function for each one of the data hiding patterns based on the value selected by the respective data hiding pattern from the other values of the coding unit;
(ii) determining whether the result of the check function corresponds to the value of the coding unit to be hidden by a respective data hiding pattern, and
(iii) if not, respectively, from the other values of the coding unit such that the result of the check function of step (ii) corresponds to the value of the coding unit to be concealed by the respective data concealment pattern Modifying at least one of said values selected by said data hiding pattern.

  The encoding device further comprises an output unit configured to output the encoded coding unit, wherein the encoded coding unit is modified in step (iii) for all data concealment patterns. Includes other values of the coding unit.

  A twenty-fifth implementation of the first aspect of the present invention implements an encoding device configured to perform a method according to any implementation of the first aspect.

  A twenty-sixth implementation of the first aspect of the present invention implements a decoding device for reconstructing a hidden value from an encoded coding unit. The decoding device is provided with a layered stack of data concealment patterns used by the encoder to conceal said values of the coding units, each of the data concealment patterns having a check function associated therewith. The decoding device selects, for each of the data concealment patterns, a check function of a respective one of the data concealment patterns by a respective data concealment pattern from the other value of the encoded coding unit. A processing unit configured to calculate based on the calculated value, wherein a result of the check function corresponds to one of the reconstructed concealed values, and a processing unit of the decoded data. In part, an output unit for outputting a decoded coding unit comprising the reconstructed concealed value.

  According to the twenty-seventh implementation of the first to twenty-sixth implementations of the first aspect, the value selected by the respective data concealment pattern from the encoded coding unit is the value of the lowest layer of the coding unit.

  According to a twenty-eighth implementation of the first to twenty-seventh implementations of the first aspect, the processing unit performs a check function of the data concealment pattern associated with a layer other than the lowest layer of the encoded coding unit. , Further configured to calculate for a value selected by a respective one of the data hiding patterns from a respective subset of the values of the lowest layer of the coding unit.

  According to the twenty-ninth implementation of the first to twenty-eighth implementations of the first aspect, the processing unit calculates a size of each one of the data hiding patterns, and a check function of each data hiding pattern. Is configured to be determined based on the size of the subset.

  According to the first to the twenty-eighth or thirtieth implementations of the twenty-ninth implementation of the first aspect, each of the subsets in layers other than the lowest layer is a coding unit that hierarchically belongs to the organizational data unit of each layer. Contains the value of the lowest level of.

  According to a thirty first implementation of the first to thirtieth implementations of the first aspect, the processing unit checks the number and / or size of data concealment patterns applicable to each organized data unit of each tier. The function is configured to determine based on the size of the subset of values belonging to each organizational data unit of each tier for which it is calculated.

  A thirty-second implementation of the first aspect of the present invention implements a decoding device configured to execute a decoding method according to any implementation of the first aspect.

A thirty-third implementation of the first aspect of the present invention stores instructions that, when executed by a processing unit of an encoding device, cause the encoding device to perform the following steps for each of the data hiding patterns. Provide (non-transitory) computer readable media.
(i) calculating a check function for each one of the data hiding patterns based on the value selected by the respective data hiding pattern from the other values of the coding unit;
(ii) determining whether the result of the check function corresponds to the value of the coding unit to be hidden by a respective data hiding pattern, and
(iii) if not, respectively, from the other values of the coding unit such that the result of the check function of step (ii) corresponds to the value of the coding unit to be concealed by the respective data concealment pattern Modifying at least one of said values selected by said data hiding pattern.

  Further, the instructions further comprise instructing the processing unit of the encoding device to include other values of the coding unit as modified in step (iii) for all data hiding patterns in the encoded coding unit. Can be done.

  A thirty-fourth implementation of the first aspect of the present invention stores instructions that, when executed by a processing unit of the encoding device, cause the encoding device to perform an encoding method according to any implementation of the first aspect. (Non-transitory) computer readable media.

  A thirty-fourth implementation of the first aspect of the present invention stores instructions that, when executed by a processing unit of the encoding device, cause the encoding device to provide a layered stack of data hiding patterns ( Provide a (non-transitory) computer readable medium, wherein the stack is used by an encoder to hide said value of a coding unit. Each of the data hiding patterns has a check function associated with it. The instructions, to the decoding device, for each of the data concealment patterns, a check function for each one of the data concealment patterns, selected by the respective data concealment pattern from the other value of the coding unit. Having a further calculation based on the value, the result of the check function corresponds to one of the reconstructed hidden values.

  A thirty-sixth implementation of the first aspect of the present invention stores instructions that, when executed by a processing unit of the decoding device, cause the decoding device to execute the decoding method according to any implementation of the first aspect ( Provide a (non-transitory) computer readable medium.

  Hereinafter, embodiments of the present invention will be described in more detail with reference to the accompanying drawings and drawings. In the figures, similar or corresponding details are provided with the same reference numerals.

FIG. 4 illustrates an exemplary structure (coding tree) of a coding unit according to an exemplary embodiment of the present invention. FIG. 4 is a diagram illustrating an exemplary structure of a coding unit according to an embodiment of the present invention, which is similar to a coding unit structure in the H.265 standard. FIG. 9 is a diagram illustrating a different example of the one-dimensional decoupling data hiding pattern according to the embodiment of the present invention. FIG. 7 is a diagram illustrating a different example of a two-dimensional decoupling data hiding pattern according to an embodiment of the present invention. FIG. 3 illustrates an exemplary embodiment of the structure of a layered stack of data hiding patterns defined for the different layers of FIG. 2 according to one embodiment of the present invention. FIG. 4 illustrates an exemplary hierarchical structure of a data hiding pattern according to an exemplary embodiment of the present invention. 5 is a flowchart of a procedure for encoder-side coefficient adjustment for a multilayer hierarchically structured data hiding pattern according to an exemplary embodiment of the present invention. 5 is a flowchart of a procedure for encoder-side coefficient adjustment for a multilayer hierarchically structured data hiding pattern according to an exemplary embodiment of the present invention. 5 is a flowchart of a simplified procedure for encoder-side coefficient adjustment for a two-layer data hiding pattern according to an exemplary embodiment of the present invention. 5 is a flowchart of a process for decoding a video stream according to an embodiment of the present invention. FIG. 3 is a diagram illustrating a structure of an encoder and a decoder according to an exemplary embodiment of the present invention. FIG. 7 illustrates an exemplary modification of implicit flag signaling of the flowchart for reference sample filtering according to ITU T H.256 / HEVC intra prediction according to an exemplary embodiment of the present invention; FIG. 7 illustrates an example use case of the inventive concept according to different further embodiments. FIG. 7 illustrates an example use case of the inventive concept according to different further embodiments. FIG. 7 illustrates an example use case of the inventive concept according to different further embodiments. FIG. 2 is a diagram illustrating an exemplary encoding device according to an embodiment of the present invention. FIG. 7 is a diagram illustrating an exemplary decoding device according to an embodiment of the present invention.

  The following paragraphs describe various implementations and embodiments of the different aspects. As already pointed out above, one aspect of the present invention involves performing multiple data hiding operations on the data of the coding unit.

  A coding unit generally refers to a structure that contains a set of data to be coded. The coding units are assumed to have a hierarchical structure. This means, for example, that the individual values representing data belonging to a coding unit correspond to a hierarchical structure comprising several layers of the coding unit. Such stratification may be realized, for example, by a structure defining a coding tree, where the coding unit layer defines the root of the tree, and the data for each branch in the tree is the next lower layer. Belongs to. The data in the coding unit is represented by data values (values for short) that represent information in different layers of the structure. Data values may be represented in binary format. Although the data hiding operations, and especially the computation of the check functions, can be performed on the values, they can also be performed on the binary representation (bit level), if desired.

  An exemplary coding tree is shown in FIG. Therefore, the structure of the coding unit (layer n unit, top layer unit) defines the top layer in the hierarchical order of the layers of the coding tree. Each unit in each layer (excluding the lowest layer) may include side information of each layer and one or more lower layer units. Note that depending on the application of the invention, there may not be side information in one or more layers. The lowest layer unit (layer 0 unit) may include or consist of a value, for example, and may have side information as appropriate.

  The idea of the first aspect of the present invention is to use data hiding pattern (DHP) to disseminate information belonging to distinct layers (e.g., one or more values of the coding unit) based on other information of the coding unit. It is to conceal. For example, the data to be concealed may be one or more side information of each layer that would otherwise have to be explicitly coded in the encoded data. In one implementation, side information of each layer's data structure (eg, layer n-1 units or other lower layer units) may be hidden in values belonging to each data structure.

  Which information (or value) of which layer is concealed in the coded coding unit can be predetermined or signaled to a decoding device, for example. Also, the data concealment pattern used in each different layer may be predetermined or may be derivable from the parameters of the encoded coding unit, eg, their respective sizes. The same one or more data hiding patterns may be reused in one layer X for each of the layer X units in this layer.

  In some embodiments, the value on which the data hiding operation is based belongs to the lowest layer of the coding unit. These values to which each data hiding pattern applies may be considered an input set of values for each data hiding pattern. Some data hiding patterns may be based on the same input set of values, partially overlapping input sets of values, or distinctly different input sets, or combinations thereof.

  In general, assuming a check function with a binary result for exemplary purposes only, one data hiding pattern can hide one bit of information. Thus, the number of data concealment patterns is the number of bits to be concealed for one respective layer unit, unless the input data set is large enough so that the same data concealment pattern can be applied to its respective subset. Corresponds to a number. The data hiding pattern for one layer of data units may thus define a stack of decoupling data hiding patterns. The decoupling of the data concealment pattern ensures that the concealed bits can be reconstructed from the coded coding unit. Advantageously, not only is the decoupling of the data concealment pattern given to the data concealment pattern for one single layer, but also the data concealment pattern is decoupled across all layers of the coding unit. As already pointed out, "decoupled" means that the value recovered from any layer using any data hiding pattern is the rest of the data hiding pattern defined for this layer. Cannot be derived by a combination (eg, a linear combination). In other words, a stack of decoupled data hiding patterns can be described by a non-degenerate equation system that derives an extracted value from a given set of target values.

  The layered stack of decoupling data hiding patterns can be constructed, for example, in various ways. FIG. 3 shows different examples of one-dimensional decoupling data hiding patterns that can be classified into three groups: decimation-based, regular, and irregular (pseudo-random). In FIG. 3, (a) and (b) present two examples of decimation-based DHP, and (c), (d), and (e) show three different examples of other regular DHP. And (f) shows an example of an irregular (pseudo-random) DHP.

  A decimation-based data hiding pattern can be constructed in such a way that a given set of elements is used for data hiding every nth element. In the more general case, if the elements of this set are selected regularly by data hiding patterns, ie according to some regular order, then these data hiding patterns can be classified as regular data hiding patterns . In all other cases that do not define a regular order inside a data hiding pattern, they can be grouped into classes of irregular or pseudo-random data hiding patterns.

  The stack of layers can consist of different classes of data hiding patterns, but the constraints on these data hiding patterns are decoupled. Further, the data hiding pattern can be a two-dimensional or even multi-dimensional pattern. An example of a two-dimensional layered stack of data hiding patterns is given in FIG. The two-dimensional data concealment pattern can be applied, for example, to quantized transform coefficients in the same way as the one-dimensional pattern, in which case the data concealment and extraction procedure is performed before the coefficient scan.

  In the following examples, it is assumed, for the most part, that the invention is used in the context of video coding. Thus, the data of the coding unit represents a block of pixels of the image. For exemplary purposes, the coding units are assumed to have a coding tree structure similar to the H.265 standard. A coding unit has one or more prediction units, one or more transform units, and one or more coefficient groups. This exemplary layered structure is shown in FIG. 2, where each layer of the coding unit (CU) is highlighted. Further, the parameters identified for the individual layers are a list of exemplary side information that may be hidden for each layer data unit (CU, PU, TU, and CG). Further, it is illustratively assumed that the data hiding operation on each layer is performed based on the transform coefficient values of CGs belonging to the data units (CU, PU, TU, and CG) of each layer. Therefore, the input data set for the data hiding operation performed on the data units of each layer may be different. This may allow to reuse the same data hiding pattern for different respective layers of data units.

  FIG. 5 illustrates an exemplary embodiment of the structure of a layered stack of data hiding patterns defined for the different layers of FIG. In this example, for illustrative purposes only, it is only assumed that there is one data hiding pattern for each respective layer of data units in each of the four layers.

  In the lowest layer (layer 0), the CG layer, one value of each CG should be hidden (for example, the sign flag of the (significant) transform coefficient of each CG). Note that a full set of transform coefficients is considered (e.g., for a CG of size 4x4, the code unit in a coding unit where all 16 significant and insignificant coefficients are considered, but only significant coefficients are coded) Can be converted).

  Using the respective data concealment patterns and check functions (eg, parity check functions), the sign of one or more significant coefficients in a given CG may be coded. The size of the data hiding pattern is (or a submultiple of) the size of the CG (e.g., 16 values for a 4x4 CG) and selects a subset of the CG's coefficient values for application of the check function I do. The data hiding operation is not applied to all CGs, but may be applied only to CGs that satisfy a threshold condition. For example, data concealment is used when the difference between the positions of the first and last significant coefficients is greater than or equal to four.

  In one example, for each value of the CG to be hidden, a corresponding data hiding pattern (and check function) is defined. Thus, for example, if the signs of four significant coefficients are to be concealed, four distinct data concealment patterns are defined, each used to conceal one sign bit (data concealment). Assume that the size of the pattern is equal to the size of the CG (ie, the number of its coefficients). In the example of FIG. 5, one sign bit of one of the significant coefficients can be hidden for one CG. Note that the set of data concealment patterns used to conceal the sign bits of different CGs can be the same for all CGs of the coding unit. However, different sets of data hiding patterns can also be used.

  Referring now to the TU level, ie layer 1 in the example of FIG. 5, some CGs typically belong to a single TU. Thus, to conceal TU side information (eg, TU reference sample filtering flag), the input data set for the data concealment operation can then be the transform coefficients of all CGs belonging to each TU. Therefore, the data hiding pattern of the TU layer may have a size equal to the size of the CG, as shown in FIG. If multiple bits need to be concealed in the TU layer, the corresponding number of decoupled data concealment patterns is defined.

  Alternatively, the size of the data hiding pattern of the TU layer may be a divisor of the size of the CG (not shown in FIG. 5). As described below, the data hiding pattern may have a maximum size. In this case, an integer multiple of the size of the data concealment pattern may be equal to the size of the CG belonging to the TU (ie, the number of coefficients of different CGs belonging to the TU). In this case, the same data concealment pattern can be applied multiple times to different subsets of CGs belonging to the TU to conceal multiple bits.

  Similar to the TU layer, some CGs may belong to a single PU also in the PU layer, that is, in layer 2 in the example of FIG. Therefore, the input data set for the data hiding operation on the PU layer may again be large compared to the TU level. The data hiding pattern on the PU layer may again be equal in terms of the size and size of the CG belonging to each PU, or alternatively, the size of the data hiding pattern in the PU layer is a divisor of the size of the CG (Not shown in FIG. 5) to facilitate reuse of the data concealment pattern for different subsets of CG coefficient values belonging to each PU when concealing a plurality of bits.

  In general, a data concealment pattern may be used to conceal non-binary values, such as a prediction mode index or motion vector (index) for a PU, for example. Depending on the size of the data concealment pattern and the number of coefficients belonging to each layer unit, one or more data concealment patterns are needed to do so. Assuming that the PU has multiple TUs, the number of transform coefficients of the CG belonging to the CG of the PU is significantly larger than the (maximum) size of the data concealment pattern for the PU layer, so the prediction mode index or motion vector for the PU It can be expected that even one data hiding pattern can be used to hide non-binary values, such as (index).

  Finally, at the CU layer, ie, layer 3 in the example of FIG. 5, all coefficients of all CGs of the CU may be used for the data hiding process. Therefore, it may be possible to hide one or more side information on the CU layer. For example, loop filter parameters, SAO offset values, or deblocking filter parameters may be hidden using data hiding operations within the CU's CG coefficients, as described above for the TU and PU layers.

  In addition, it is possible to add a watermark (fingerprint) to any of the layers, as described below.

  As will become clearer, the data hiding operation needs to modify the individual values at the lowest layer (for example, the CG layer involvement in the example described earlier in FIGS. 2 and 5). (Numerical value), the check function applied to the value selected by the respective data hiding pattern may give the correct result for hiding the data to be hidden in different layers. Thus, for example, when considering audio, picture, or video coding, modification of the bottom layer data may result in distortion of the decoded signal. The distortion caused by the value modification may be, for example, as defined in terms of "cost", and the algorithm for determining which of the lowest layer values requires modification is based on the input of the data hiding operation. A decision can be made based on the cost of modification (ie, the estimated decrease in quality) of each value forming the data set (ie, the value selected by the data hiding pattern). The distortion introduced by the concealment operation depends on the number of values selected by each data concealment pattern (i.e., the number of candidates available for correction), and their values over the entire set of values of the lowest layer value of the coding unit. Also depends on how they are distributed. Generally, the distortion introduced by the data hiding operation can be expected to decrease with the size of the data hiding pattern, as more values are selected and / or distributed over a larger set of values. . Another factor is, of course, the number of data hiding operations performed on the coding unit.

  However, the lower the reduction in distortion of the decoded signal between different lengths of the data concealment pattern, the larger the data concealment pattern. Therefore, it makes sense to define the maximum length of the data hiding pattern, but as described above in relation to the CU, PU, and TU layers, it is (typically at the upper layer) Can be applied more than once on the lowest layer value associated with each layer value. This may reduce distortion introduced by data concealment and may also reduce complexity in defining a decoupled stack of data concealment patterns.

  Nevertheless, the number of bits to be concealed per layer may be predetermined, and the size of the units in each layer (and thus the number of lowest layer values associated with them) may be determined by the coding process. , The encoding device or even the decoding device may provide a data concealment pattern stack (e.g., a data concealment pattern for each layer and its size) to be used for encoding and decoding each coding unit. ) Can be determined dynamically. Nevertheless, the selection of a stack of data concealment patterns to be used to encode and decode each coding unit depends on the structure of the coding unit (e.g., the unit of each layer unit), as outlined in more detail below. (Size), thereby allowing the decoding device to dynamically use different stacks of data to reduce distortion introduced by data concealment, while still allowing the decoding device to There is no additional signaling overhead required to signal the stack of concealment patterns.

  Then, considering the decoding of the concealed data, i.e. the reconstruction of the concealed values in the different layers from the coded coding units, the decoupled stack of the data concealment patterns used in the different layers is It need not be determined but may be derivable from the structure of the encoded coding unit. For example, the data concealment pattern stack is adaptively derived by the encoding and decoding device based on coding unit parameters, e.g., CG, TU, PU, and CU sizes, as outlined below. It is possible to

  To illustrate this concept, the following simplified example is considered without loss of generality. The data concealment pattern is assumed to have a maximum size of 16 bits (ie, select a value from a set of 16 CG coefficients). Further, the CG is assumed to have either a size of 4x4, producing 16 coefficients per CG, or a size of 8x8, producing 64 coefficients per CG. Furthermore, the sign of the first four significant coefficients in the CG is assumed to be encoded by data concealment (four bits are concealed). Further, for the CG layer, there are four predetermined data hiding patterns DHP1, DHP2, DHP3, and DHP4 that are decoupled and each have a size of 16 bits. Each data concealment pattern DHP1, DHP2, DHP3, and DHP4 uses a parity check function as a check function for data concealment, so that each data concealment pattern can be used to conceal one single bit.

  In the encoding process, the coding unit has a structure that produces 4x4 CG, i.e. if there are 16 coefficients in the CG, the processing unit of the encoding device will have 4 data hiding patterns DHP1, DHP2, DHP3, and Perform 4 data concealment operations to conceal 4 code bits for 16 coefficients in CG using DHP4. Similarly, on the decoding device side, in the decoding process, the processing unit of the decoding device recognizes that the size of the CG is 4x4, and for each of DHP1, DHP2, DHP3, and DHP4, four data hiding patterns DHP1, DHP2. , DHP3, and DHP4 calculate four results of the parity check function based on a coefficient selected from the 16 coefficients of the CG.

  If the coding unit has a structure that yields an 8x8 CG, i.e., there are 64 coefficients in the CG, the processing unit of the encoder splits the 64 coefficients into different subsets of the 16 coefficients, and then Perform four data concealment operations to conceal the four code bits on each subset of the sixteen coefficients. In these four data concealment operations based on the four different subsets of coefficients, the processing unit of the encoder uses one of the four data concealment patterns DHP1, DHP2, DHP3, and DHP4, or an alternative. Or a subset of all four data hiding patterns DHP1, DHP2, DHP3, and DHP4. Which one or more of the four data hiding patterns DHP1, DHP2, DHP3, and DHP4 are used in the encoding device is predetermined so that this information need not be signaled to the decoding device. obtain.

  Similarly, on the decoding device side, in the decoding process, the processing unit of the decoding device recognizes that the size of the CG is 8x8. The decoding device forms 4 subsets of each of the 16 coefficients from the 64 coefficients in the CG (note that the division is, of course, the same as in the coding device and can be predetermined). . The decoding device then uses the same one or more of the four data concealment patterns DHP1, DHP2, DHP3, and DHP4 used by the encoding device to generate a parity check based on each subset of the 16 coefficients. Calculate the four results of the check function.

  Concealing four bits in a 4x4 CG by using four data concealment operations performed on the same set of coefficients, and four data concealment operations performed on different subsets of coefficients Modification of the coefficients to ensure that the parity check function produces the correct sign flag (bit value) for each of the four codes when compared to concealing the four bits in an 8x8 CG by using It will be clearer that the distortion introduced by can cause greater distortion in the decoded signal for a 4x4 CG as compared to an 8x8 CG.

  Thus, it is possible to consider foreseeing a flag in the 4x4 CG side information that can signal whether data concealment was used when encoding the CG. Alternatively, this flag may be foreseen in the TU-side information and may signal whether data concealment is used in encoding the TU's 4x4 CG. The encoder can determine the use of data concealment for a 4x4 CG based on a predetermined or configurable distortion threshold.

  For reconstruction of the concealed data, the decoding device can determine only the 16 coefficients of the CG from the encoded CG (i.e. significant and insignificant coefficients), but from the spectrum to the time domain Does not need to be performed.

  This latter point may be particularly advantageous when using data concealment in watermarking applications (e.g., on layers above the CG layer, e.g., on the CU layer), where the watermarking is performed from the spectral to the time domain. Because it can be checked without performing the inverse transformation of the coefficients to. Accordingly, decoding of the lowest layer data (e.g., inverse transform) may be performed by the decoding device only when it is possible to verify the authenticity of the encoded data, whereby: Significant processing resources may be saved if the watermark is not verified.

  Although the above example involves adaptively hiding data in the CG layer, a similar mechanism may be used, for example, in one or more of the TU, PU, and CU layers. For example, in FIG. 2, some of the TUs have 4x4 = 16 CGs associated with them, while other TUs have 8x8 = 64 CGs associated with them. Given that there are 4x4 = 16 or 8x8 = 64 coefficients per CG, a possible input data set for the data hiding operation may be based on a subset of the total number of coefficients belonging to the TU, but not all of them. It is clear.

  In one exemplary implementation, it is assumed that, for exemplary purposes, eight bits should be hidden per TU. For a TU with 16 CGs of size 4x4, this yields a total of 256 coefficient values on which a data hiding operation can be performed. For example purposes, considering only a 32-bit data concealment pattern, this single data concealment pattern is sufficient to conceal eight bits in 256 coefficients. The data concealment pattern is applied in eight data concealment operations on distinctly different subsets of 32 of the 256 coefficients. Of course, for example, consider defining a plurality of 32-bit long data concealment patterns to ensure that the stack of data concealment patterns across all layers is decoupled. However, on the decoding device side, a 4x4 TU size and a 4x4 CG size are used by the decoding device to select a predetermined (one or more) data concealment pattern of 32 bits each (used by the ), Reconstructing the 8 concealment bits from a predetermined subset of each of the 32 coefficients of the 256 coefficients belonging to the TU.

  For a TU with 16 CGs of size 8x8 or a TU with 64 CGs of size 4x4, this yields a total of 1024 coefficient values on which a data hiding operation can be performed. One possibility is to increase the size of the data hiding pattern (subset of the coefficients to which they apply accordingly) as the size of the CG increases, i.e., of 128 of the 1024 coefficients belonging to the TU. It may be to define one or more data concealment patterns of 128 bits each, which are used to conceal and reconstruct the eight concealment bits using each predetermined subset.

  Alternatively, the same data concealment patterns used for a 4x4 TU with a 4x4 CG can be used, which are then 1024 pre-determined coefficients belonging to the TU. Operated at 256. The remaining 768 coefficients of the TU cannot be used.

  In another alternative, the same data hiding pattern that is used for a 4x4 TU with a 4x4 CG can be used, but the remaining 768 coefficients of the TU have additional bits. (Ie, a total of 32 bits can be concealed).

  For a TU having 64 CGs of size 8x8, the number of coefficients available for data hiding increases to 4096 coefficients. As in the above case, this may be to increase the size of the data hiding pattern accordingly, use only a subset of the coefficients for data hiding (while maintaining the size of the data hiding pattern), or Can be used to hide a lot of data.

  The determination of how many bits are concealed in each of the layers should also take into account the overall distortion introduced by the data concealment operation into the decoded signal. However, if the data hidden in each layer is predetermined, the size of the data unit of each layer (CU, PU, TU, CG) and / or the data unit of each lower layer (PU, TU, CG) Parameters such as one or more of the sizes may implicitly inform the decoding device of the stack of data concealment patterns used by the encoding device when decoding the encoded coding unit. .

  Now consider the data hiding operation, which can be implemented as follows. It is assumed that the check function associated with the data concealment pattern has a binary result, which can conceal one bit of information in the input data set (eg, CG coefficients). For each data hiding pattern in each layer, each data hiding pattern selects a subset of the values of the input data set (or all of its values) and computes a check function on the selected set of values. The binary result of this calculation is compared with the binary information to be hidden. If the result matches the binary information to be concealed, none of the selected values require modification. If the result matches the binary information to be concealed, one or more of the values selected by the data concealment pattern from the input data set need to be modified, so that the result of the check function is Matches binary information to be hidden.

  In one exemplary implementation, the necessary modification of the value selected by the data hiding pattern from the input data set determines the value to be modified while minimizing the distortion introduced by the modification of the selected value. It is performed by using an algorithm that can do it.

  When using several data concealment patterns on different hierarchies with partial overlap in the selection of values (see FIGS. 4 and 5), the encoding device is selected and modified in the first data concealment operation. Value is not modified again by another second data hiding operation that selects this value again. Further, the second data hiding operation needs to take into account the selected and modified values when calculating the check function. This is important for implementations where data hiding is performed sequentially on individual data hiding patterns (and layers).

  Alternatively, all check functions for the data concealment pattern are calculated first, and then to modify the values of the lowest layer of the coding unit for all data concealment patterns so that each of the check functions produces the correct result. Implementations where one algorithm is used are possible.

  A hybrid implementation of the two alternatives is also possible, where, for example, the individual layers are processed sequentially, but for a single layer, for all data hiding patterns of a single layer All check functions are calculated, and the modification algorithm modifies the value of the lowest layer of the coding unit for all data hiding patterns of a single layer so that each of the check functions produces the correct result.

  For a hierarchical structure as shown in FIG. 1, special rules of coefficient adjustment can be applied to the DHP that guarantees correct coefficient adjustment (FIG. 6). For a given layer of DHP 601, it is possible to find a set of child DHPs 602,603. Within this set of DHPs, one special DHP 603 can be selected. Other child DHPs 602 control their own check function values and their respective child DHPs, but the selected DHP 603 should additionally control the value of a given parent DHP 601. The same rule applied recursively to each of the child DHPs yields a target check function value for all DHPs 601-605.

Exemplary flowcharts of the data concealment procedure including encoder-side coefficient adjustment for the multi-layer hierarchically structured DHP are shown in FIG. 7 and FIG. The encoding process may be based, for example, on a stack of DHPs structured as shown in FIG. 6, but the invention is not limited thereto. The resulting set of adjusted coefficients is obtained by performing a matching concealment search 701 for each of the top layer DHPs. This procedure is recursive except for the lowest layer that has the next input.
Index of layer i Current DHP of layer i
A stack of DHPs built on a higher layer representing the chain of ancestor DHPs for the current DHP (see Figure 6). When a matched hidden search is performed on the top layer DHP, the DHP stack is empty.
Target value associated with each of the DHPs. These target values are, in effect, the values to be concealed and define the results to be produced by the check function applied to the value selected by the respective DHP.

  Step 702 performs a check whether the lowest layer is being processed. Assuming that all layers are numbered starting from zero to the lowest layer, the input layer has index i, and block 702 compares i to zero.

  If not the lowest layer to be processed (i is not equal to zero), the search for the best DHP chain continues (or starts if i equals the index of the top layer). Steps, best cost initialization 703, save state of search 704, DHP scan 705, and coefficient update 718 are performed for both start and continue search cases.

  Best cost initialization 703 should ensure that best cost check 715 returns the value true unless the best concealment cost has been redefined. For example, if the best concealment cost can be stored in a variable, initialization 703 can assign the highest possible value of this variable that should be greater than the concealment cost calculated in step 714. is there.

  Another possible method of initialization 703 is to first introduce a special initialization flag equal to false before check 715 and switch it to true after check 715 has been performed. In this case, check 715 should always be true if the initialization flag is false.

  Saving the state of the search 704 is necessary to maintain the consistency of the DHP scan 705, ie, all independent iterations of the scan 705 should start from the same state of the stack of coefficients and DHPs. The coefficients of both the current DHP and the subset corresponding to the stack of DHPs can be stored in temporary arrays at step 704 and can be restored from these temporary arrays at restore step 706. Alternatively, the restoration step 706 can be performed before starting the next iteration of 705, ie, after completion of 717 or when 715 is evaluated to be false. However, placing 706 should not affect the results obtained, as it results in the same initial state of coefficients and the same initial state of the stack of DHPs for all iterations of 705.

  A DHP is a child DHP of the current DHP if the subset of coefficients for the child DHP is included in the subset of coefficients corresponding to the current DHP. Step 705 scans all DHPs in layer i-1 that are child DHPs of the current DHP to select a child DHP that gives the value of the target check function to itself as well as to the chain of ancestor DHPs .

  When a candidate child DHP is selected, the other child DHPs should be adjusted first, since adjusting the coefficient of any of the child DHPs will affect the adjustment to be performed on the selected DHP . Step 707 iterates over all child DHPs except the selected DHP, performing a matched concealment search for each of them. However, in iteration 707 the current stack of the DHP is not passed to the matched concealment search 709, and a new stack is built in step 708. This newly constructed set contains the child DHPs currently repeated in step 707, so that all DHPs repeated by 707 are correct check function values for themselves and their inheritors DHP, not the ancestor DHP give.

  The target check function value of the ancestor DHP should only be provided by the child DHP selected in iteration step 705. Thus, upon completion of step 707, the DHP's input stack is updated with the selected child DHP 711 and the current input DHP 712 that is the parent of the selected child DHP.

  When all coefficient adjustments have been completed, a concealment cost 714 can be calculated for a given stack of DHPs. By comparing this cost with the best cost (715), the best coefficient adjustment variant may be selected. This selection may include a step 716 of saving the coefficient adjustments in a temporary array and a step 717 of redefining the best cost with the current cost calculated in step 714. The best adjustment of the coefficients is restored from this temporary array at step 718 as all modifications to the current DHP are processed by iteration at step 705. Further actions 721 depend on the current layer i. If i is equal to the top layer index, the matched concealment search is completed and the DHP coefficients are adjusted. Otherwise, the recursive call is completed and other steps after 709 should be performed accordingly.

  For the lowest layer (if i is zero), the matched concealment search 701 should perform a subset adjustment 720 according to the input stack of the DHP. The coefficients selected for subset adjustment in step 719 should belong to the subset corresponding to the top layer DHP in the DHP's input stack.

  FIG. 8 shows the process of coefficient adjustment 801 for a given subset of coefficients, a given stack of DHPs containing M DHPs, and target values to be provided by each if the DHPs belong to a stack of DHPs. Shows the stack given. The recursive call counter is initialized to zero in step 802. However, this step 802 should be skipped if the subset adjustment 801 is called recursively. Thereafter, a subset check 803 is performed which verifies whether the result of the check function is appropriately equal to the value from the stack of target values. This check 803 consists of calculating the check function values for each of the DHPs 804 and comparing these values with the target value 805.

  Subset adjustment is complete when this equality relationship holds for all DHPs in the DHP's input stack. Immediate termination 813 should be performed even if subset adjustment 801 is recursively invoked from step 815.

  Otherwise, the search for coefficient adjustments continues. Note that because the DHPs in the stack are decoupled, the target value of the check function can be achieved by adjusting the maximum of the M coefficients. However, depending on the situation, the number of necessary adjustments may be smaller than M.

  Coefficient adjustments begin by storing the values of the coefficients in a temporary array 806, so that the adjustments in the iterative search 807 can be restored to the initial state. This iterative search 807 is performed for all coefficients of the input subset except those already modified in the previous recursive call. 808 is performed for the selected coefficient adjustment. In particular, the adjustment may consist of adding some constant value (positive or negative) or changing the value of a bit of the coefficient. In addition, at step 808, the cost of the adjustment is calculated. The set of coefficients is then rolled back to the initial state 809 and the next coefficient is selected.

  When all coefficients are repeated, in step 811, the counter of the recursive loop is incremented. A special check 810 is performed on this counter to detect possible errors. Step 810 may be omitted when it is only an option. In particular, if the recursion counter exceeds the number of DHPs in the stack M, this means that the subset check 803 did not succeed for the M adjusted coefficients. This situation is only possible if the DHPs in the stack are not decoupled. When that occurs, check 810 is not successful and error 812 should be handled. Normally, check 810 should be evaluated and true in all cases. In addition to incrementing the recursive counter 811, the input coefficients are adjusted according to the best cost selected from the costs calculated in step 808. The coefficient adjustment that yields the best cost value is performed in step 814, the adjusted coefficient is excluded from the check 807 performed in the next iteration, and finally the subset adjustment is recursively called (step 815). .

  Clearly, the subset adjustment 801 described above modifies M or fewer coefficients at the minimum adjustment cost. The tuning procedure 801 can be sub-optimal, but should always yield valid results for valid input arguments. Note that this procedure can also be implemented in the form of a loop, as the recursive call 815 is the last step in the procedure.

  For the special case of data hiding in the two layers, a simpler scheme 901 can be considered, as illustrated in FIG. In this example, it is assumed that for exemplary purposes only, the parent DHP corresponds to the TU and the child DHP corresponds to the CG. The first step in the procedure is the initialization of the best cost and the concealment flag 902 is executed. The method of initializing the best cost is the same as that in step 703 of the generalized method. The initial value of the matching concealment flag is zero. When the repetition 903 that loops the steps 904 to 908 ends, the CG having the minimum adjustment cost is determined. Checking step 906 controls further steps of the scheme depending on whether there is a need for coefficient adjustment for layer 0. If this need is detected for at least one of the CGs, at step 908, the matched concealment flag is changed to one.

  The action after step 903 depends on whether layer 0 needs adjustment. If so, steps 909 and 910 adjust all necessary CGs except the CG with the least cost. Otherwise, the coefficient with the minimum adjustment cost is searched in all CGs (step 911).

  The next step 912 is the step of calculating the check function value at layer 1 (for the TU). This value is checked by comparison with the target value (913). If adjustment should be performed on the TU, steps 914 and 916 adjust the coefficient with the minimum cost selected by the DHP of the TU. The second coefficient should be adjusted in steps 915 and 917 depending on the matched concealment flag value. This adjustment is necessary to match the adjustment for layer 1 with the adjustment for layer 0.

  If the evaluation 913 of the TU check function is successful and the matching concealment flag is non-zero, it is possible to adjust a single coefficient that does not belong to the TU's DHP. Therefore, steps 915, 917 are performed in that case.

  If the evaluation 913 of the TU check function is successful and the matching concealment flag is 0, no adjustment is necessary, and the procedure 901 is terminated (918).

  In the above description, reference is made to the concealment costs resulting from the modification of the input dataset. In the context of video or image coding, the concealment cost may be determined, for example, by any cost function that estimates or determines image quality degradation resulting from a given modification of the input dataset. Such a decrease in image quality may be estimated, for example, using a cost function that determines or approximates the rate distortion resulting from modification of the input data set. In particular, rate-distortion calculations can consume large amounts of resources, so it is preferable to select a cost function that allows cost, low-computation estimation and can only provide estimation of image quality degradation. Seem.

  The process of decrypting the concealed data can be implemented as follows. The decoding device obtains each encoded coding unit from the coded stream of data. The stack of decoupled data hiding patterns used by the encoder may be static, configurable, or derivable from the structure of the encoded coding unit (as described above). As such), the decoding device recognizes the data concealment pattern and the input data set that forms the basis of the data concealment operation in the encoding device. Therefore, the decoding device can simply calculate the check function of each data hiding pattern using the appropriate input data set. These output values of the check function represent the recovered concealed data.

  Regarding the choice of the check function for the data hiding operation, the only requirement for the check function is that it should have a single output value (binary or non-binary) for a set within the input target values. The most common type of check function is a parity check function that derives a binary output by checking whether the sum of the input target values is even or odd. The following list contains some other valid examples of check functions.

Equal to zero of the remainder from the division of the result of some distance function (e.g., the summation function) by
Parity of the number of zero (or non-zero) values Comparison of some statistical property value (mean, deviation, etc.) with a threshold

  The output value of the check function can also be defined as non-binary. For example,

, The remainder from the division of some distance function (eg, summation function) value by x> 1. In this case, the output value of the check function is limited to the range [0, ..., x-1].
Quantized value of some statistical property (mean, deviation, etc.)
A function of the number of elements in a given set of values satisfying some criteria

  A more detailed process of decoding a video stream according to an exemplary embodiment of the present invention is shown in FIG. This decoding process is suitable for decoding an encoded video stream that has been encoded using the methods described in connection with FIGS. 7 and 8 above. The encoded input bit stream 1001 is processed by an entropy decoder 1002. Some of the decoded values correspond to the absolute values of the quantized transform coefficients 1005. These values are further processed by an inverse quantizer. The result of this process is an absolute value of a coefficient that is suitable for the inverse transformation process 1004 after the correct code has been assigned.

  The sign of the value may be explicitly signaled, for example, in the encoded input bitstream 1001 or may be included as concealed data therein. In the first case, the derivation of the code is performed explicitly by the entropy decoder 1002, and in the second case, implicitly by applying the appropriate DHP to the corresponding set of quantized transform coefficients. Can be performed. DHP selector 1006 performs DHP selection in response to available quantized transform coefficients 1005 provided by entropy decoder 1002. When the coefficients are only available for the coefficient group (CG), the DHP selector 1006 extracts the code value (s) for the CG, provides these codes 1008, and provides the inverse transform 1004. To get the correct value of the coefficient for.

  The DHP selector 1006 is applied to a given (sub) set of coefficients when quantized transform coefficients are available for a larger structuring element that can contain multiple CGs Apply all possible DHP. The output value is dispatched by the level switch 1007 depending on what DHP is selected. As described for coefficient code 1008, the output of 1007 may provide a portion of the decoded data. In that case, the rest of the data should be decoded in some other way. One of the most obvious ways is to explicitly decode this part and recover it using the entropy decoder 1002.

As shown in FIG. 10, possible output data for DHP selector 1007 can be one or more of the following:
Coefficient code 1008
Flag to filter reference sample 1009 (its use is further illustrated in FIG. 12)
Intra prediction mode 1010
Loop filter parameter 1011
Watermark bit 1012

  However, FIG. 10 only covers some examples of data that can possibly be concealed within the quantized transform coefficients. The possible types of hidden data are not limited to the list shown above.

  After the transform coefficients have been inversely transformed, the residual signal is restored, which is further added to the prediction signal 1019. The prediction signal is generated using intra prediction 1013 or inter prediction 1014 mechanism depending on the selected prediction mode. This selection can be implemented in the form of a switch 1018. Both inter prediction and intra prediction use previously decoded pixels, which are stored in frame buffer 1017. However, the intra prediction mechanism can use pixels that have not been modified by the loop filter 1015.

  In addition to video coding applications, the proposed invention can also solve additional tasks, such as watermarking. The DHP selector 1006 recovers the concealed watermark bits 1012, and in response to the results returned by the watermark validator 1016 for these bits, an output frame is output as illustrated by the switch 1020 Is prevented. If the validation is successful, switch 1020 is "closed" and output picture 1021 is output by the decoder. Otherwise, switch 1020 is "opened" and watermark validator 1016 can signal a watermark mismatch.

  FIG. 11 illustrates an exemplary structure of a video encoding device according to an embodiment of the present invention. The encoding device also comprises decoding blocks (1112 to 1115), which may further be provided in the decoding device, further comprising an entropy decoder 1002. The apparatus of FIG. 11 may be implemented, for example, on an application-specific or general-purpose computing device (see FIGS. 14 and 15). The encoder receives a video signal, including individual frames / slices 1102 to be encoded, and generates an encoded bitstream 1110. A decoding device such as the one shown in FIG. 15 is capable of receiving an encoded bit stream 1110 and outputs a decoded video frame / slice for each frame / slice of the video signal.

  The encoding device includes a motion estimation / motion compensation block 1101, a mode determination block 1105, a transform block 1106, a quantizer 1107, and an entropy encoder (CABAC) 1109. Mode determination block 1105 may determine an appropriate coding mode for the video source. The mode decision block 1105 determines, for example, whether the subject frame / slice is an I, P, or B frame / slice, and / or that a particular coding unit within the frame / slice is inter or intra coded. You can decide whether or not.

  The conversion block 1106 performs conversion on the spatial domain data. Such a transform of transform block 1106 may be a block-based transform for transforming spatial domain data into spectral components. For example, a discrete cosine transform (DCT) may be used for the transform. Other transforms such as a discrete sine transform or others may also be used. Block-based transforms are performed on coding units depending on the size of the coding unit. The block-based transform of the block of pixel data results in a set of transform coefficients as described above. The transform coefficient is quantized by a quantizer 1107. The quantized coefficients and associated side information are then encoded by an entropy encoder (CABAC) 1109. A block or matrix of quantized transform coefficients may be referred to as a transform unit (TU). In some cases, the TU may be non-square, for example, a non-square transform (NSQT).

  Intra-coded frames / slices (ie, type I) are coded without reference to other frames / slices 1103 and therefore do not use temporal prediction. Intra-coded frames (see block 1104) rely on spatial prediction within the frame / slice. During encoding, a particular block of pixels in a block may be compared to data of nearby pixels in the block that have already been encoded for that frame / slice. By using a prediction algorithm, the source data of the block can be converted to residual data. Next, transform block 1106 encodes the residual data.

  Intra prediction is performed on the reference samples generated by the reference sample generation block 1111. Block 1111 can output filtered or unfiltered reference samples for intra prediction, and the selection that determines which of them is used is signaled using a filtering flag. In the example of FIG. 11, it is illustratively assumed that the reference sample filter flag is added as data hidden in output bitstream 1110. The RSAF flag concealment block 1108 performs a concealment operation as described in connection with FIGS. 7 and 8 above, and optionally, some of the signs of the transform coefficients obtained from the transform block 1106, or Additional side information, such as all, may also be hidden (not shown in FIG. 11).

  To support motion prediction / compensation and utilize temporal prediction, the encoder includes a feedback including an inverse quantizer 1112, an inverse transform block 1113, a deblocking filter 1114, and optionally, a sample adaptive offset block 1114. Has a loop. The deblocking filter 1114 can include a deblocking processor and a filtering processor. Elements 1112-1115 reflect the decoding process implemented by the decoding device to reproduce the original frame / slice, as outlined above. The frame store (reference frame 1103) is used to store the reproduced frame. In this manner, motion prediction is based on what would be the reconstructed frame 1116 and not the original frame, which may differ from the frame 1116 reconstructed by lossy compression involving encoding / decoding. Motion estimation / motion compensation 1101 uses the frames / slices stored in the frame store as source frames / slices for comparison with the current frame for the purpose of identifying similar blocks. The data provided by motion estimation / motion compensation 1101 includes, as side information, information about reference frames, motion vectors, and residual pixel data representing the difference (if any) between the reference block and the current block. obtain. The residual signal supplied to the motion estimation / motion compensation 1101 is further encoded. Information about reference frames and / or motion vectors cannot be processed by transform block 1106 and / or quantizer 1107, but instead is entropy encoder for encoding as part of the bitstream with the quantized coefficients. (CABAC) 1109.

  The decoding device can include an entropy decoder that performs a process opposite to the entropy coding on the encoder side, and further includes blocks 1112-1115 shown in FIG.

  The invention has a wide range of uses. As pointed out above, an example of such an application implicit flag signaling of reference sample filtering for ITU T H.265 / HEVC intra prediction may also be contemplated as described above. . However, as pointed out above, other side information that forms part of the data structure generated upon encoding can also be hidden using the present invention. According to ITU T H.265 / HEVC, the decision whether to apply a low-pass filter to reference samples is made according to the selected prediction mode and the size of the predicted block. This decision can be overridden by a flag that can be signaled implicitly in the transform coefficients quantized using the proposed invention. This modification is illustrated in FIG.

  In addition to media compression, many use cases for the present invention can be found in watermarking applications. Figures 13a and 13b and 13c represent some of these embodiments. The use cases meant in FIGS. 13a and 13b and 13c are extracted by applying DHP to either the pixels or the quantized transform coefficients of a digital photo stored on an ID card. Represents an authentication procedure consisting of confirming the identity of this card using concealed data that is possible. Validation can be performed in different ways, for example, by calculating a checksum value and comparing it to the extracted data. Authentication fails if a comparison mismatch occurs. This mismatch indicates that either the digital photo or the identifier has been changed without permission. If the comparison is successful, the validation should result in a positive authentication.

  FIG. 13b shows an example of the falsification detection use. In that case, it is assumed that the digital picture is prepared to have key data concealed in the block, for example, in pixel values or transform coefficients. The watermark reading operation retrieves this hidden key from the block of pictures using DHP and is further validated using a security key. The validation procedure is the same as that presented in FIG. 8a, and if there is a mismatch, the result is the detection of a particular tampered picture block.

  Another use case (FIG. 13c) assumes that the implicit message needs to be sent in an explicit message, making the presence of the implicit message difficult to detect. In this example use case shown in FIG. 8c, the implicit message is restored by applying DHP to the quantized value during decoding of the explicit message.

  FIG. 14 shows a simplified block diagram of an exemplary embodiment of an encoder 1400. The encoder 1400 includes a processor 1401, a memory 1402, and an encoding application 1406. Encoding application 1403 may include a computer program or application stored in memory 1402 and including instructions for configuring processor 1401 to perform operations as described herein. For example, encoding application 1403 encodes a coded bitstream according to an encoding process, according to various embodiments, for example, as described herein in connection with FIGS. 7 and 8 above. Can output. It will be appreciated that encoding application 1403 may be stored in a computer-readable medium. The output bitstream may be transmitted via communication system 1404 or may be stored on a computer-readable medium, for example.

  Reference is also made to FIG. 15, which shows a simplified block diagram of an exemplary embodiment of the decoder 1500. The decoder 1500 includes a processor 1501, a memory 1502, and a decoding application 1503. The decoding application 1503 decodes the input bitstream stored in the memory 1501 according to one of the various embodiments herein, for example, as outlined in connection with FIG. 10 above. May include a computer program or application that includes instructions for configuring the processor 1501 to perform It will be appreciated that the decryption application 1503 can be stored in a computer readable medium. The input bitstream may be received, for example, via communication system 1404 or read from a computer-readable medium.

  Although some aspects are described in the context of a method, it is clear that these aspects also represent a description of the corresponding apparatus, which is suitably adapted to perform such a method. In such an apparatus, a block (functional or tangible) may correspond to one or more method steps or features of a method step. Similarly, aspects described in the context of corresponding blocks or items or features of corresponding devices may also correspond to individual methods of the corresponding method.

  Further, the methods described herein may also be performed by (by using) a hardware device, such as a processor, a microprocessor, a programmable computer, or an electronic circuit. Any one or more of the most important method steps may be performed by such a device. It should be further understood that the apparatus has been described herein in terms of functional blocks, and that those elements of the apparatus may be fully or partially implemented in hardware elements / circuits. Separate hardware, such as a processor or a microprocessor, may be used to implement the functions of one or more elements of the device.

  In addition, here the information or data should be stored in the process of implementing the method steps of the functional elements of the hardware device, and the device communicates with one or more hardware elements / circuits of the device. It may comprise a memory or storage medium, which may be operably coupled.

  It is also contemplated that aspects of the invention may be implemented in hardware, or in software, or a combination thereof. This is a digital storage medium, in which electronically readable control signals or instructions are stored, associated with (or capable of being associated with) a programmable computer system such that the respective method is performed, for example, It may use a floppy disk, DVD, Blu-Ray, CD, ROM, PROM, EPROM, EEPROM, or FLASH memory. A data carrier can be provided having electronically readable control signals or instructions that can be associated with a programmable computer system such that the methods described herein are performed.

  It is also contemplated that aspects of the invention may be implemented in the form of a computer program product with program code, the program code being operable to perform the method when the computer program product runs on a computer. The program code may be stored on a machine readable carrier.

  It is understood that the above description is merely illustrative, and that modifications and changes in the arrangement and details described herein will be apparent to those skilled in the art. Accordingly, the intent is limited only by the claims that follow, and not by the specific details presented using the above description and description.

601 DHP
602-605 DHP
701 Hidden Search
703 Best cost initialization
704 Saving search state
705 DHP scan
711 DHP
712 Input DHP
715 Best cost check
718 Coefficient update
720 subset adjustment
801 Coefficient adjustment
803 Subset check
804 DHP
805 target value
806 temporary array
807 iterative search
809 Initial state
810 checks
812 error
901 method
902 Concealment flag
913 Ratings
1001 encoded input bitstream
1002 entropy decoder
1004 Inversion process
1005 Quantized transform coefficients
1006 DHP selector
1007 Level switch
1008 sign
1009 Reference Sample
1010 Intra prediction mode
1011 Loop filter parameters
1012 watermark bit
1013 Intra prediction
1014 Inter prediction
1015 Loop filter
1016 Watermark validator
1017 Frame buffer
1018 switch
1019 Prediction signal
1020 switch
1021 output picture
1101 motion estimation / motion compensation block
1102 frames / slice
1105 Mode decision block
1106 Transform block
1107 Quantizer
1109 Entropy encoder (CABAC)
1110 Encoded bitstream
1111 Reference sample generation block
1112 Inverse quantizer
1113 Inversion block
1114 Deblocking filter
1114 sample adaptive offset block
1116 Reconstructed frame
1400 encoder
1401 processor
1402 memory
1403 Encoding application
1404 Communication system
1406 Encoding Application
1500 decoder
1501 processor
1502 memory
1503 Decryption application

Claims (29)

A method for hiding a value of a hierarchically structured coding unit among other values included in the coding unit,
Providing a layered stack of data concealment patterns corresponding to different layers of the coding unit for concealing the values of the coding units at different layers of the coding unit , wherein each of the data concealment patterns is Having a check function associated with it to conceal one or more of said values of said coding unit in one of said layers of said coding unit;
For each of the data hiding patterns,
(i) calculating the check function of the respective one pattern among the data hiding pattern based on the value selected by data hiding pattern of other values or Raso respectively of the coding unit,
(ii) determining whether the result of the check function corresponds to a value of the coding unit to be hidden by the respective data hiding pattern; and
(iii) if not, the other of the coding units such that the result of the check function of step (ii) corresponds to the value of the coding unit to be concealed by the respective data concealment pattern. Modifying at least one of the values selected by the respective data hiding pattern from the values.
Marks Goka been coded unit, the other values look including the said coding unit, such as modified for all data hiding pattern in step (iii),
The value of the coding unit represents a block of pixels of an image, the coding unit having one or more prediction units, one or more transform units, and one or more coefficient groups; The data concealment pattern is associated with a layer of the coding unit corresponding to the prediction unit, the transform unit, and the coding unit, respectively,
A method wherein each of the data concealment patterns conceals one or more values of the associated layer of the coding unit . The method of claim 1, wherein the layered stack of data hiding patterns defines a decoupled data hiding pattern.   3. The method of claim 1 or 2, wherein each layer is associated with one or more of the data hiding patterns. The data hiding pattern is a decimation-based data hiding pattern constructed in such a way that the elements of a given set of values are used for data hiding for every nth element , wherein the elements of the set have some regularity. 4. The method according to any one of claims 1 to 3, comprising one or more regular data hiding patterns selected according to an order and at least one of one or more pseudo-random data hiding patterns. The method, for each of the data hiding patterns,
The algorithm used in step (iii) to modify the at least one of the values selected by the respective data hiding pattern from the other values of the coding unit, the algorithm in step (ii) If the result of the check function does not modify the value to correspond to the value of the coding unit to be concealed by the respective data concealment pattern, the result of the check function in step (ii) is the respective data 5. The method according to any of the preceding claims, further comprising repeating steps (i) to (iii) until corresponding to the value of the coding unit to be concealed by a concealment pattern.   Using the first data concealment pattern and the second data concealment pattern of the data concealment patterns, when performing steps (i) to (iii), respectively, selecting by the first data concealment pattern The one or more other values of the coding unit being modified in step (iii), wherein the second value is selected by the second data hiding pattern in the second iteration. The method according to any one of claims 1 to 5, wherein the steps (i) to (iii) are not modified again in the step (iii) when the steps (i) to (iii) are performed on the data hiding pattern. A method of reconstructing said hidden value from a coded coding unit, wherein the values of a hierarchically structured coding unit are hidden among other values included in said coding unit ,
Providing a layered stack of data concealment patterns corresponding to different layers of the coding unit, which is used by an encoder to conceal the value of the coding unit, wherein each of the data concealment patterns comprises: A step having a check function associated with it,
Wherein for each of the data hiding pattern, each of the one pattern the other values or Raso respectively the value selected by data hiding pattern of the check function the coding unit of one of the data hiding pattern and calculating on the basis of the result of the check function, corresponding to the one of the reconstructed hidden value, and a step seen including,
The value of the coding unit represents a block of pixels of an image, the coding unit having one or more prediction units, one or more transform units, and one or more coefficient groups; The data concealment pattern is associated with a layer of the coding unit corresponding to the prediction unit, the transform unit, and the coding unit, respectively,
A method wherein each of the data concealment patterns conceals one or more values of the associated layer of the coding unit . The method according to any one of claims 1 to 7 , wherein the value included in the coding unit based on which the check function of the data concealment pattern is calculated is a value of a lowest layer of the coding unit. . The check function of the data concealment pattern associated with a layer other than the lowest tier of the coding unit, the check function of each of the data concealment patterns from a respective subset of the values of the lowest tier of the coding unit. 9. The method of claim 8 , wherein the method is calculated with respect to a value selected by one of the following patterns: The size of each of the one pattern of the data hiding pattern, according to claim 9, wherein the checking function of the data hiding pattern of their respective is determined based on the size of the subset of values to be calculated Method. Wherein each of the subsets in a layer other than the lowermost hierarchy The method of claim 9 or 10 including the values of the lowermost layer of the coding units belonging to hierarchically organized data units of a layer of their respective . The number and / or size of the data concealment patterns applicable to each of the respective organized data units of the respective tiers is determined by the value of the value belonging to the respective organized data units of the respective tier for which the check function is calculated. the method of claim 11 which is determined based on a subset of size. Size of each of the one pattern of the data hiding pattern is equal to the size of a subset of the values that the said checking function of the respective data hiding pattern is calculated, or about the number of the size A method according to any one of claims 9 to 12 . The data hiding pattern, the coefficient group, the prediction unit, the conversion unit, and associated with the layer of the coding units corresponding to each of the coding unit, wherein the coefficient group of the lowest hierarchy of the coding unit 14. The method according to any one of claims 8 to 13, which is a part. The method according to any one of claims 1 to 14 , wherein the check function of the data hiding pattern is calculated based on the value of the coefficient group. The method according to any one of claims 1 to 15 , wherein the data concealment pattern comprises one or more data concealment patterns for concealing a reference sample filtering flag for intra prediction of the coding unit for the transform unit. . The data concealment pattern according to any one of claims 1 to 16 , comprising one or more data concealment patterns for concealing at least one of a prediction mode index and a prediction unit size of the prediction unit. Method. The data hiding pattern A method according to any one of claims 1 to 17 including a data hiding pattern for embedding a watermark in the coding unit. The data hiding pattern A method according to any one of claims 1 to 18 comprising a data hiding pattern for concealing sign bit of the coefficient of the coefficient group. Wherein the value of the coding units, the method according to any one of claims 1 to 13 which represents the block of image data, a block of audio data, a block of the document to be encoded. Wherein one of the hidden value, method according to any one of claims 1 to 20 which makes it possible to verify the authenticity of the value contained in the coding unit. An encoding device for concealing a value of a hierarchically structured coding unit among other values included in said coding unit, said encoding device comprising: Providing a layered stack of data concealment patterns corresponding to different layers of the coding unit, wherein each of the data concealment patterns is provided in one of the layers of the coding unit. Having a check function associated with it to conceal one or more of the values, the encoding device comprising:
For each of the data hiding patterns,
(i) calculating the check function of the respective one pattern among the data hiding pattern based on the value selected by data hiding pattern of other values or Raso respectively of the coding unit,
(ii) determining whether the result of the check function corresponds to a value of the coding unit to be hidden by the respective data hiding pattern; and
(iii) if not, the other of the coding units such that the result of the check function of step (ii) corresponds to the value of the coding unit to be concealed by the respective data concealment pattern. A processing unit configured to perform a step of modifying at least one of the values selected by the respective data hiding pattern from the values,
An output unit configured to output an encoded coding unit, wherein the encoded coding unit is modified in step (iii) for all data concealment patterns. comprising said other value, and an output unit,
The value of the coding unit represents a block of pixels of an image, the coding unit having one or more prediction units, one or more transform units, and one or more coefficient groups; The data concealment pattern is associated with a layer of the coding unit corresponding to the prediction unit, the transform unit, and the coding unit, respectively,
It said data each of hiding pattern, said one of its dependent layers or more values you hide the encoding device of the coding unit. The encoding device, encoding device of claim 22 configured to perform the way of any one of claims 1 to 6. A decoding device for reconstructing said concealed value from a coded coding unit, wherein the value of the hierarchically structured coding unit is concealed among other values contained in said coding unit, The apparatus is provided with a layered stack of data concealment patterns corresponding to different layers of the coding unit, which are used by an encoder to conceal the values of the coding unit, wherein each of the data concealment patterns is Having a check function associated therewith, wherein the decoding device comprises:
Wherein for each of the data hiding pattern, each one of said check function of the pattern, data hiding pattern of said other value or Raso respectively of the encoded coding unit of the data hiding pattern A processing unit configured to calculate based on the value selected by the processing unit, wherein the calculation result of the check function corresponds to one of the reconstructed concealed values, ,
As part of decrypt data, comprising said reconstructed hidden value, and an output unit for outputting the decoded coding unit,
The value of the coding unit represents a block of pixels of an image, the coding unit having one or more prediction units, one or more transform units, and one or more coefficient groups; The data concealment pattern is associated with a layer of the coding unit corresponding to the prediction unit, the transform unit, and the coding unit, respectively,
It said data each of hiding pattern, the decoding device you conceal one or more values of its dependent layers of the coding unit. Said value said selected by the respective data hiding pattern from the encoded coding unit, decoding apparatus according to claim 24 which is the value of the most lower level of the coding unit. The processing unit is further configured to execute the check function of the data concealment pattern associated with a layer other than the lowest layer of the encoded coding unit, the subset of each of the values of the lowest layer of the coding unit. 26. The decoding device according to claim 25 , further configured to calculate with respect to a value selected by a respective one of the data hiding patterns. Wherein the processing unit is configured the the size of each of the one pattern of the data hiding pattern, as determined based on the size of the subset the said checking function of the respective data hiding pattern is calculated The decoding device according to claim 26 . Each subset in the layer other than the lowermost hierarchy decoding according to claim 26 or 27, including the value of the lowermost layer of the coding units belonging to hierarchically organized data units of a layer of their respective apparatus. The processing unit may determine the number and / or size of data concealment patterns applicable to each of the respective organized data units of the respective layer, the respective organized data units of the respective layer for which the check function is calculated. 29. The decoding device according to claim 28 , wherein the decoding device is configured to determine based on the size of the subset of values belonging to.