WO2020066641A1

WO2020066641A1 - Image processing device and method

Info

Publication number: WO2020066641A1
Application number: PCT/JP2019/035818
Authority: WO
Inventors: 健史筑波
Original assignee: ソニー株式会社
Priority date: 2018-09-25
Filing date: 2019-09-12
Publication date: 2020-04-02
Also published as: JP2022002352A

Abstract

The present disclosure relates to an image processing device and method that make it possible to more easily perform a one-dimensional transform or an inverse one-dimensional transform. A sign inversion operation is performed with respect to a one-dimensional signal train of coefficient data. When a one-dimensional transform of a first transform type is implemented with respect to the one-dimensional signal train that has been subjected to the sign inversion operation, a transform matrix of a second transform type for implementing the one-dimensional transform of the first transform type by an FTS operation is defined as a base transform matrix, whereas when a one-dimensional transform of a third transform type is implemented, a transform matrix which is of a fourth transform type for implementing the one-dimensional transform of the third transform type by the FTS operation and which is a symmetric matrix is defined as the base transform matrix. A matrix computation is performed using the base transform matrix, and a flip operation is performed with respect to the one-dimensional signal train that has been subjected to the matrix computation. The present disclosure is applicable, for example, to an image processing device, an image encoding device, or an image decoding device.

Description

Image processing apparatus and method

The present disclosure relates to an image processing apparatus and method, and more particularly, to an image processing apparatus and method that can perform one-dimensional conversion or inverse one-dimensional conversion more easily.

Conventionally, for the TU (Transform @ Unit) unit, a plurality of different values are adaptively different for each of the horizontal primary transform PThor (also referred to as primary horizontal transform) and the vertical primary transform PTver (also referred to as primary vertical transform). An adaptive primary transform (AMT: Adaptive Multiple Core Transforms) for selecting a primary transform from an orthogonal transform has been disclosed (for example, see Non-Patent Document 1). In Non-Patent Document 1, there are five one-dimensional transforms (also referred to as one-dimensional orthogonal transform) of DCT-II, DST-VII, DCT-VIII, DST-I, and DST-VII as candidates for the primary transform.

Further, it has been proposed that two one-dimensional orthogonal transforms of DST-IV and IDT (Identity Transform: one-dimensional transform skip) are added, and a total of seven one-dimensional orthogonal transforms are set as candidates for a primary transform (for example, , Non-Patent Document 2).

{Also, type2 / type4} AMT in which the orthogonal transform used in AMT is {DCT4 / DST4 / DCT2 / DST2} has been proposed (for example, see Non-Patent Document 3). In Non-Patent Document 3, furthermore, a transform matrix of 2 ^ N-pt DCT2 is converted / sampled / sign-inverted / flipped, and a transform matrix of 22M-ptpt2 DCM4 / DST4 / DCT2 / DST2 smaller than 2 ^ N-pt It was proposed to derive

Furthermore, it has been proposed to realize DST4 by STF operation of DCT4 and DST2 by FTS operation of DCT2 (for example, see Non-Patent Document 4).

For example, by applying DST2 or DST4 realized using the FTS operation or STF operation described in Non-Patent Document 4 to the type2 / type4 AMT described in Non-Patent Document 3, the primary using the FTS operation or STF operation is Conversion (reverse primary conversion) can be realized.

However, in this case, it is necessary to be able to perform the flip operation and the sign inversion operation in both the pre-processing on the input side and the post-processing on the output side of the matrix operation, and it is necessary to perform one-dimensional conversion (or inverse one-dimensional conversion). Conversion) and the circuit configuration may be more complicated.

The present disclosure has been made in view of such a situation, and is intended to facilitate one-dimensional conversion or inverse one-dimensional conversion.

An image processing device according to an aspect of the present technology includes a decoding unit that decodes a bit stream to generate coefficient data regarding an image, and a one-dimensional signal sequence of the coefficient data generated by the decoding unit. And a flip section for performing a flip operation for rearranging the order of the first order, and performing an inverse one-dimensional conversion of the first conversion type on the one-dimensional signal sequence subjected to the flip operation by the flip section. When the conversion matrix of the second conversion type that realizes the inverse one-dimensional conversion of the first conversion type is used as the base conversion matrix, and the inverse one-dimensional conversion of the third conversion type is realized, A conversion matrix that is a symmetric matrix of a fourth conversion type that realizes an inverse one-dimensional conversion of the conversion type is a base conversion matrix, and a matrix operation is performed using a transpose of the base conversion matrix. A matrix operation unit, and a sign inversion unit that performs a sign inversion operation of inverting a sign of an odd-numbered signal of the one-dimensional signal sequence with respect to the one-dimensional signal sequence on which the matrix operation is performed by the matrix operation unit. An image processing apparatus comprising:

An image processing method according to an embodiment of the present technology decodes a bit stream to generate coefficient data related to an image, and rearranges the order of each coefficient in a one-dimensional signal sequence of the generated coefficient data in reverse order. When a flip operation is performed and an inverse one-dimensional conversion of a first conversion type is realized on the flip-operated one-dimensional signal sequence, an inverse one-dimensional conversion of the first conversion type is realized by an STF operation In the case where the transformation matrix of the second transformation type to be used is a base transformation matrix and the inverse one-dimensional transformation of the third transformation type is realized, the fourth one of realizing the inverse one-dimensional transformation of the third transformation type by an FTS operation. A conversion type, and a conversion matrix that is a symmetric matrix as a base conversion matrix, performs a matrix operation using a transposed matrix of the base conversion matrix, for the one-dimensional signal sequence subjected to the matrix operation, The image processing method of performing a sign inversion operation to invert the sign of the odd-numbered signal dimensional signal sequence.

An image processing device according to another aspect of the present technology includes a sign inverting unit that performs a sign inversion operation of inverting a sign of an odd-numbered signal of the one-dimensional signal sequence for a one-dimensional signal sequence of coefficient data regarding an image; When realizing a one-dimensional conversion of the first conversion type on the one-dimensional signal sequence subjected to the sign inversion operation by the sign inversion unit, realizing the one-dimensional conversion of the first conversion type by an FTS operation When a conversion matrix of the second conversion type is used as a base conversion matrix and a one-dimensional conversion of the third conversion type is realized, a fourth conversion type of realizing the one-dimensional conversion of the third conversion type by STF operation And, a transformation matrix that is a symmetric matrix as a base transformation matrix, a matrix operation unit that performs a matrix operation using the base transformation matrix, and the one-dimensional signal sequence on which the matrix operation is performed by the matrix operation unit hand, A flip unit that performs a flip operation of rearranging the order of coefficients in a reverse order, and an encoding unit that encodes coefficient data including the one-dimensional signal sequence subjected to the flip operation by the flip unit and generates a bit stream. It is an image processing apparatus provided.

An image processing method according to another aspect of the present technology includes performing a sign inversion operation for inverting a sign of an odd-numbered signal of the one-dimensional signal sequence on a one-dimensional signal sequence of coefficient data relating to an image, and performing the sign inversion operation. When realizing one-dimensional conversion of the first conversion type for the one-dimensional signal sequence thus performed, a conversion matrix of a second conversion type for realizing one-dimensional conversion of the first conversion type by an FTS operation is obtained. When a one-dimensional conversion of the third conversion type is realized as a base conversion matrix, a conversion matrix of a fourth conversion type and a symmetric matrix for realizing the one-dimensional conversion of the third conversion type by an STF operation Is a base conversion matrix, a matrix operation is performed using the base conversion matrix, and a flip operation for rearranging the order of each coefficient on the one-dimensional signal sequence on which the matrix operation is performed is performed, and the flip operation is performed. Operation Encodes the coefficient data including the 1-dimensional signal sequence is performed, an image processing method for generating a bit stream.

In the image processing apparatus and method according to one aspect of the present technology, a bit stream is decoded to generate coefficient data regarding an image, and the order of each coefficient is reversed in a one-dimensional signal sequence of the generated coefficient data. When an inverse one-dimensional conversion of the first conversion type is realized for the flip-operated one-dimensional signal sequence, the inverse one-dimensional conversion of the first conversion type is performed by the STF operation. When the conversion matrix of the second conversion type that realizes the above is used as the base conversion matrix, and the inverse one-dimensional conversion of the third conversion type is realized, the inverse one-dimensional conversion of the third conversion type is realized by the FTS operation. 4, a conversion matrix that is a symmetric matrix is used as a base conversion matrix, a matrix operation is performed using a transposed matrix of the base conversion matrix, and a one-dimensional signal sequence on which the matrix operation is performed is And, sign inversion operation is performed to invert the sign of the odd-numbered signal of the one-dimensional signal sequence.

In the image processing device and method according to another aspect of the present technology, a sign inversion operation of inverting a sign of an odd-numbered signal of the one-dimensional signal sequence is performed on a one-dimensional signal sequence of coefficient data regarding an image, When realizing one-dimensional conversion of the first conversion type for the one-dimensional signal sequence subjected to the sign inversion operation, conversion of the second conversion type for realizing one-dimensional conversion of the first conversion type by the FTS operation When the matrix is a base conversion matrix and a one-dimensional conversion of a third conversion type is realized, the matrix is a symmetric matrix of a fourth conversion type that realizes a one-dimensional conversion of the third conversion type by an STF operation. The transformation matrix is used as a base transformation matrix, a matrix operation is performed using the base transformation matrix, and a flip operation for rearranging the order of each coefficient is performed on the one-dimensional signal sequence on which the matrix operation has been performed. We Coefficient data including a one-dimensional signal sequence flip operation is performed is encoded, the bit stream is generated.

FIG. 3 is a diagram illustrating an example of one-dimensional conversion of DST2. FIG. 9 is a diagram illustrating an example of one-dimensional conversion of DST4. FIG. 4 is a diagram illustrating an example of one-dimensional conversion using an STF operation / FTS operation. FIG. 3 is a block diagram illustrating a main configuration example of a conversion device. 13 is a flowchart illustrating an example of the flow of a conversion process. FIG. 9 is a diagram for describing an example of one-dimensional conversion using an FTS operation. FIG. 3 is a block diagram illustrating a main configuration example of a conversion device. 13 is a flowchart illustrating an example of the flow of a conversion process. FIG. 9 is a diagram illustrating an example of inverse one-dimensional conversion using an STF operation. It is a block diagram which shows the main structural examples of an inversion apparatus. It is a flowchart explaining the example of the flow of an inverse conversion process. FIG. 9 is a diagram illustrating an example of deriving a base transformation matrix. FIG. 9 is a diagram illustrating an example of deriving a base transform matrix of DCT2. FIG. 9 is a diagram illustrating an example of deriving a base transform matrix of DCT4. FIG. 3 is a block diagram illustrating a main configuration example of a conversion device. FIG. 4 is a block diagram illustrating a main configuration example of a base transformation matrix derivation unit. 13 is a flowchart illustrating an example of the flow of a conversion process. It is a flowchart explaining the example of the flow of a base transformation matrix derivation process. It is a block diagram which shows the main structural examples of an inversion apparatus. It is a flowchart explaining the example of the flow of an inverse conversion process. FIG. 35 is a block diagram illustrating a main configuration example of an image encoding device. It is a block diagram which shows the main structural examples of an orthogonal transformation part. FIG. 3 is a block diagram illustrating a main configuration example of a primary conversion unit. It is a block diagram which shows the main structural examples of a primary horizontal conversion part. FIG. 3 is a block diagram illustrating a main configuration example of a primary vertical conversion unit. 15 is a flowchart illustrating an example of the flow of an image encoding process. It is a flowchart explaining the example of the flow of an orthogonal transformation process. It is a flowchart explaining the example of the flow of a primary conversion process. It is a flowchart explaining the example of the flow of a primary horizontal conversion process. 15 is a flowchart illustrating an example of the flow of a primary vertical conversion process. FIG. 35 is a block diagram illustrating a main configuration example of an image decoding device. It is a block diagram which shows the main structural examples of an inverse orthogonal transformation part. It is a block diagram which shows the main structural examples of an inverse primary conversion part. It is a block diagram which shows the main structural examples of an inverse primary vertical conversion part. It is a block diagram which shows the main structural examples of an inverse primary horizontal conversion part. It is a flowchart explaining an example of the flow of an image decoding process. It is a flowchart explaining the example of the flow of an inverse orthogonal transformation process. It is a flowchart explaining the example of the flow of an inverse primary conversion process. 15 is a flowchart illustrating an example of the flow of an inverse primary vertical conversion process. It is a flowchart explaining the example of the flow of an inverse primary horizontal conversion process. FIG. 18 is a block diagram illustrating a main configuration example of a computer.

Hereinafter, embodiments for implementing the present disclosure (hereinafter, referred to as embodiments) will be described. The description will be made in the following order.
1. 1. Documents supporting technical contents and technical terms 2. adaptive primary conversion Concept 4. First embodiment (conversion device)
5. Second embodiment (inverse conversion device)
6. Third Embodiment (Conversion device (base conversion matrix derivation))
7. Fourth Embodiment (Inverse Transformation Device (Derivation of Base Transformation Matrix))
8. Fifth embodiment (application example)
9. Sixth embodiment (image coding apparatus)
10. Seventh embodiment (image decoding device)
11. Note

<1. Documents that support technical content and technical terms>
The scope disclosed by the present technology includes not only the contents described in the embodiments but also the contents described in the following non-patent documents that are known at the time of filing.

Non-patent document 1: (described above)
Non-patent document 2: (described above)
Non-patent document 3: (described above)
Non-patent document 4: (described above)
Non-Patent Document 5: TELECOMMUNICATION STANDARDIZATION SECTOR OF ITU (International Telecommunication Union), "Advanced video coding for generic audiovisual services", H.264, 04/2017
Non-Patent Document 6: TELECOMMUNICATION STANDARDIZATION SECTOR OF ITU (International Telecommunication Union), "High efficiency video coding", H.265, 12/2016

That is, the contents described in the above-mentioned non-patent literature also serve as the basis for determining the support requirements. For example, even when the Quad-Tree Block Structure described in Non-Patent Document 6 and the Quad Tree Plus Binary Tree (Block Tree) Structure described in Non-Patent Document 1 are not directly described in the embodiments, the present invention is not limited to this. It is within the scope of disclosure of the technology and satisfies the support requirements of the claims. Similarly, for example, technical terms such as parsing, syntax, and semantics are within the disclosure range of the present technology even if there is no direct description in the embodiment. Support requirements in the range of

In this specification, a “block” (not a block indicating a processing unit) used in the description as a partial region or a processing unit of an image (picture) indicates an arbitrary partial region in a picture unless otherwise specified. The size, shape, characteristics, and the like are not limited. For example, “block” includes a TB (Transform @ Block), a TU (Transform @ Unit), a PB (Prediction @ Block), and a PU (Prediction @ Unit) described in Non-Patent Documents 1, 5 and 6 described above. ), SCU (Smallest Coding Unit), CU (Coding Unit), LCU (Largest Coding Unit), CTB (Coding Tree Unit), CTU (Coding Tree Unit), conversion block, sub block, macro block, tile, slice, etc. , An arbitrary partial area (processing unit).

In specifying such a block size, not only the block size may be specified directly, but also the block size may be specified indirectly. For example, the block size may be specified using identification information for identifying the size. Further, for example, the block size may be specified by a ratio or a difference from the size of a reference block (for example, an LCU or an SCU). For example, when transmitting information for specifying a block size as a syntax element or the like, the information for indirectly specifying a size as described above may be used as the information. By doing so, the amount of information can be reduced, and the coding efficiency can be improved in some cases. The designation of the block size also includes designation of a range of block sizes (for example, designation of a range of allowable block sizes, etc.).

符号 In this specification, the term “encoding” includes not only the entire process of converting an image into a bit stream but also a part of the process. For example, prediction processing, orthogonal transformation, quantization, not only includes a comprehensive process such as arithmetic coding, etc., but also includes a process generically referred to quantization and arithmetic coding, prediction processing, quantization and arithmetic coding Processing, etc. Similarly, decoding includes not only the entire process of converting a bit stream into an image, but also some processes. For example, it includes not only the processing including the inverse arithmetic decoding, the inverse quantization, the inverse orthogonal transform, and the prediction processing, but also the processing including the inverse arithmetic decoding and the inverse quantization, the inverse arithmetic decoding, the inverse quantization, and the prediction processing. And comprehensive processing.

<2. Adaptive Primary Conversion>
<Conversion type setting>
In a test model (JEM4 (Joint Exploration Test Model 4)) described in Non-Patent Document 1, a horizontal primary conversion PThor (also referred to as a primary horizontal conversion) and a vertical primary conversion PTver (vertical primary conversion PTver) are used for a luminance conversion block. An adaptive primary transform (AMT (Adaptive Multiple core Transforms)) for selecting a primary transform from a plurality of different one-dimensional orthogonal transforms adaptively for each primary vertical transform is disclosed. AMT is also referred to as EMT (Explicit Multiple Core Transforms).

Specifically, when the adaptive primary conversion flag apt_flag indicating whether or not to perform the adaptive primary conversion on the luminance conversion block is 0 (false), DCT (Discrete Cosine Transform) -II, Alternatively, DST (Discrete Sine Transform) -VII is uniquely determined by mode information (TrSetIdx = 4).

If the adaptive primary conversion flag apt_flag is 1 (true) and the current CU (Coding @ Unit) including the luminance conversion block to be processed is an intra CU, the horizontal direction (x direction) and the vertical direction (y direction) ) Are selected from among three transform sets TrSet (TrSetIdx = 0, 1, 2), each of which includes an orthogonal transform that is a candidate for a primary transform. Note that DST-VII and DCT-VIII described above indicate orthogonal transform types.

変換 The conversion set TrSet is uniquely determined based on (intra prediction mode information of) the correspondence table between the mode information and the conversion set. For example, as shown in the following Expressions (1) and (2), the conversion set TrSetH, TrSetV is set so that the conversion set identifier TrSetIdx that specifies the corresponding conversion set TrSet is set.

Here, TrSetH indicates a conversion set of the primary horizontal conversion PThor, TrSetV indicates a conversion set of the primary vertical conversion PTver, and a lookup table LUT_IntraModeToTrSet is a correspondence table between mode information and the conversion set. The first array of the lookup table LUT_IntraModeToTrSet [] [] has an intra prediction mode IntraMode as an argument, and the second array has {H = 0, V = 1} as arguments.

For example, in the case of the intra prediction mode number 19 (IntraMode == 19), the conversion set with the conversion set identifier TrSetIdx = 0 is selected as the conversion set TrSetH (also referred to as the primary horizontal conversion set) of the primary horizontal conversion PThor, and the primary vertical conversion A conversion set with a conversion set identifier TrSetIdx = 2 is selected as a conversion set TrSetV (also referred to as a primary vertical conversion set) of PTver.

When the adaptive primary conversion flag apt_flag is 1 (true) and the current CU including the luminance conversion block to be processed is the inter CU, the primary horizontal conversion conversion set TrSetH and the primary vertical conversion conversion set The conversion set InterTrSet (TrSetIdx = 3) dedicated to the Inter CU is assigned to TrSetV.

Subsequently, for each of the horizontal direction and the vertical direction, of the selected transformation set TrSet, which orthogonal transformation is to be applied is determined by a corresponding one of the primary horizontal transformation designation flag pt_hor_flag and the primary vertical transformation designation flag pt_ver_flag. select.

For example, as shown in the following equations (3) and (4), the primary {horizontal, vertical} conversion set TrSet {H, V}, the primary {horizontal, vertical} conversion designation flag pt_ {hor, ver} _flag Is derived from a predetermined conversion set definition table (LUT_TrSetToTrTypeIdx).

The primary conversion identifier pt_idx is derived from the primary horizontal conversion designation flag pt_hor_flag and the primary vertical conversion designation flag pt_ver_flag based on the following equation (5). That is, the upper 1 bit of the primary conversion identifier pt_idx corresponds to the value of the primary vertical conversion designation flag, and the lower 1 bit corresponds to the value of the primary horizontal conversion designation flag.

Encoding is performed by applying arithmetic coding to the derived bin string of the primary conversion identifier pt_idx to generate a bit string. The adaptive primary conversion flag apt_flag and the primary conversion identifier pt_idx are signaled in a luminance conversion block.

As described above, in Non-Patent Document 1, DCT-II (DCT2), ΔDST-VII (DST7), ΔDCT-VIII (DCT8), ΔDST-I (DST1), ΔDCT-V (DCT5) ) Were proposed. In this method, when AMT is applied, a 2-bit index indicating which orthogonal transform is to be applied horizontally / vertically is signaled from a transform set determined by a prediction mode, and two candidates are set for each direction. One transform is selected.

In addition, in Non-Patent Document 2, two one-dimensional orthogonal transforms of DST-IV (DST4) and IDT (Identity @ Transform: one-dimensional transform skip) are added in addition to them, for a total of seven one-dimensional orthogonal transforms. Was proposed as a candidate for primary conversion.

<Type2 / type4 AMT>
Non-Patent Document 3 proposes a type2 / type4 AMT in which the orthogonal transform used in the AMT is {DCT4 / DST4 / DCT2 / DST2}. Furthermore, it has been proposed to derive a transform matrix of DCT4 / DST4 / DCT2 / DST2 of 2 ^ M-pt smaller than 2 ^ N-pt by sampling / sign inversion / flip the transform matrix of 2 ^ N-pt DCT2. Was.

<FTS operation / STF operation>
Furthermore, Non-Patent Document 4 proposes that DST4 be realized by an STF operation of DCT4 and DST2 be realized by an FTS operation of DCT2.

The FTS operation means the sign inversion operation (S) for inverting the sign of the signal at the odd position of the input signal, the orthogonal transformation (T) of the input signal after the sign inversion operation, and the reverse order of the transformation coefficients after the orthogonal transformation. Indicates that three processes (processes in the order of S → T → F) of the flip operation (F) for rearranging are performed.

For example, as shown in FIG. 1A, an input signal X is subjected to orthogonal transform processing 11 of a conversion type DST (Discrete Sine Transform) 2 and a process of outputting an output signal Y is shown in FIG. As described above, the FTS operation using the orthogonal transform processing 13 of the transform type DCT (Discrete Cosine Transform) 2 can be realized. That is, a sign inversion operation (S) 12 is performed on the input signal X, and an orthogonal transformation process (T) 13 of the transform type DCT2 is performed on the input signal X on which the sign inversion operation is performed. By performing the flip operation (F) 14 on the, a process equivalent to the orthogonal transform process 11 of the transform type DST2 can be performed.

Therefore, a transformation matrix T _DST2 (hereinafter, also referred to as a transformation matrix of the transformation type DST2) representing the orthogonal transformation of the transformation type DST2 is a sign inversion matrix S representing the sign inversion operation, and a transformation matrix T representing the orthogonal transformation of the transformation type DCT2. _DCT2 (hereinafter also referred to as a transform matrix of transform type DCT2) and a flip matrix F representing a flip operation can be represented as in the following equation (6).

The STF operation is a flip operation (F) that rearranges the order of the input signal, an orthogonal transformation (T) of the input signal after the flip operation, and a code that inverts a transformation coefficient at an odd position after the orthogonal transformation. This indicates that three processes (in the order of F → T → S) of the inversion operation (S) are performed.

For example, as shown in FIG. 2A, the input signal X is subjected to the orthogonal transform process 21 of the conversion type DST (Discrete Sine Transform) 4 and the process of outputting the output signal Y is shown in FIG. As can be seen, it can be realized by the STF operation using the orthogonal transform processing 23 of the transform type DCT (Discrete Cosine Transform) 4. That is, a flip operation (F) 22 is performed on the input signal X, and an orthogonal transformation process (T) 23 of the transform type DCT4 is performed on the input signal X on which the flip operation has been performed. By performing the sign inversion operation (S) 24 in this way, a process equivalent to the orthogonal transform process 21 of the transform type DST4 can be performed.

Therefore, a transformation matrix T _DST4 (hereinafter, also referred to as a transformation matrix of the transformation type DST4) representing the orthogonal transformation of the transformation type DST4 is a sign inversion matrix S representing the sign inversion operation, and a transformation matrix T representing the orthogonal transformation of the transformation type DCT4. _The following equation (7) can be used using _DCT4 (hereinafter, also referred to as a transform matrix of transform type DCT4) and a flip matrix F representing a flip operation.

The flip matrix F can be expressed as in the following equation (8). The sign inversion matrix S can be expressed as in the following equation (9).

<Type2 / type4 AMT using FTS operation and STF operation>
By applying the one-dimensional orthogonal transform of the conversion types DST2 and DST4 realized using the FTS operation and the STF operation described in Non-Patent Document 4 to the type2 / type4 AMT described in Non-Patent Document 3, Primary conversion (reverse primary conversion) using an operation or STF operation can be realized. For example, in the primary conversion, control is performed as shown in the table of FIG.

In the table of FIG. 3, for example, when the conversion type identifier trTypeIdx for specifying the conversion type of the one-dimensional conversion (one-dimensional orthogonal conversion) is 0, that is, when the one-dimensional conversion of the conversion type DCT2 is specified, the pre-processing for the input signal is performed. Both the flip operation (F) and the sign inversion operation (S) are skipped (omitted) (False), and the conversion type is DCT2. Only a matrix operation (one-dimensional transformation) using a transformation matrix transMatrix _nTbS, DCT2 of size nTbS as a base transformation matrix T _base is performed.

Similarly, when the conversion type identifier trTypeIdx is 1, that is, when the one-dimensional conversion of the conversion type DCT4 is specified, both the flip operation (F) and the sign inversion operation (S) are skipped (pre-processing and post-processing) (omitted). ) Is performed (False), and only a matrix operation (one-dimensional conversion) using a transformation matrix transMatrix _{nTbS, DCT4} of a transformation type of DCT4 and a size of _nTbS as a base transformation matrix T _base is performed.

On the other hand, when the conversion type identifier trTypeIdx is 2, that is, one-dimensional conversion of the conversion type DST4 is specified, a flip operation (F) is performed on the input signal as preprocessing (True), and the sign is inverted. The operation (S) is skipped (omitted) (False). Then, a matrix operation (one-dimensional conversion) is performed on the flip-operated input signal using the transform matrix transMatrix _{nTbS, DCT4} of transform type DCT4 and size _nTbS as the base transform matrix T _base . Further, as post processing, a sign inversion operation (S) is performed on the transform coefficient obtained by the matrix operation (True), and a flip operation (F) is skipped (omitted) (False).

Similarly, when the conversion type identifier trTypeIdx is 3, that is, one-dimensional conversion of the conversion type DST2 is specified, the input inversion operation (S) is performed on the input signal as preprocessing (True), and the flip operation ( F) is skipped (omitted) (False). Then, a matrix operation (one-dimensional conversion) is performed on the input signal subjected to the input inversion operation, using a conversion matrix transMatrix _nTbS, DCT2 _having a conversion type of DCT2 and a size of nTbS as a base conversion matrix T _base . Further, as post processing, a flip operation (F) is performed on the transform coefficient obtained by the matrix operation (True), and a sign inversion operation (S) is skipped (omitted) (False).

FIG. 4 shows an example of a hardware configuration for realizing such processing. In the case of FIG. 4, the conversion device 50 includes a control unit 51, a pre-processing unit 52, a matrix operation unit 53, and a post-processing unit 54.

Based on parameters such as the conversion type identifier trTypeIdx, the width log2TBWidth of the input signal processing target block, and the height log2TBHeight of the input signal processing target block, the control unit 51 executes processing executed as pre-processing or post-processing, Select a base conversion matrix (that is, a conversion type) to be used for dimensional conversion. The control unit 51 supplies preprocessing selection information indicating a preprocessing selection result to the preprocessing unit 52. Further, the control unit 51 supplies base matrix selection information indicating the result of base matrix selection to the matrix calculation unit 53. Further, the control unit 51 supplies post processing selection information indicating the result of the post processing selection to the post processing unit 54.

The pre-processing unit 52 has a sign inversion unit 61 that performs a sign inversion operation and a flip unit 62 that performs a flip operation, selects one of the processing units according to the pre-processing selection information, and performs a processing on the input coefficient data Xin. To perform coefficient inversion operation or flip operation to generate coefficient data X ′ (X ′ = S · Xin or X ′ = F · Xin).

The matrix operation unit 53 has a base conversion matrix LUT (Look Up Table) 70. The base transform matrix LUT 70 stores a transform matrix 71 of transform type DCT2 and a transform matrix 72 of transform type DCT4, which are candidates for the base transform matrix. The matrix calculation unit 53 reads a conversion matrix specified by the base conversion matrix selection information from the candidates from the base conversion matrix LUT 70, and uses the _base conversion matrix T _base to perform a matrix calculation (one-dimensional) on the coefficient data X ′. Conversion) to generate coefficient data X ″ (X ″ = T _{base ×} X ′).

The post-processing unit 54 has a flip unit 81 for performing a flip operation and a sign-reversing unit 82 for performing a sign-reversing operation, selects one of the processing units according to the post-processing selection information, and converts the selected processing unit into coefficient data X ''. The output coefficient data Xout is generated by performing a flip operation or a sign inversion operation (Xout = F ・ X ″ or Xout = S ・ X ″).

In other words, when the conversion device 50 that performs the one-dimensional conversion as described above is configured by hardware, both the sign inversion unit 61 and the flip unit 62 are required as the pre-processing unit 52. Similarly, the post-processing unit 54 requires both the configuration of the flip unit 81 and the sign inversion unit 82. Therefore, there is a possibility that the circuit scale increases and the mounting cost increases.

(5) An example of the flow of such a conversion process will be described with reference to the flowchart of FIG. When the conversion processing is started, the control unit 51 determines the base conversion matrix selection information, the pre-processing selection information, and the post-processing selection information based on the specified conversion type and size (ie, trTypeIdx, log2TBWidth, log2TBHeight) and the like. Is set (step S51).

The pre-processing unit 52 determines whether or not to perform the pre-processing based on the pre-processing selection information (step S52), and when performing the pre-processing, further determines the processing content (sign inversion operation or flip operation). (Step S53). The pre-processing unit 52 performs a sign inversion operation on the input coefficient data Xin (step S54), performs a flip operation (step S55), or skips the pre-processing according to these determination results.

The matrix calculation unit 53 performs a matrix calculation (one-dimensional conversion) on the coefficient data X ′ using the selected base conversion matrix T _base according to the base conversion matrix selection information (step S56).

The post-processing unit 54 determines whether or not to perform post-processing based on the post-processing selection information (step S57). If the post-processing is to be performed, the post-processing unit 54 further determines the processing content (flip operation or sign inversion operation). (Step S58). The post-processing unit 54 performs a flip operation on the coefficient data X ″ (step S59), performs a sign inversion operation (step S60), or skips the post-processing according to these determination results.

As described above, it is necessary to determine whether or not to perform the pre-processing and the post-processing, and to perform the pre-processing and the post-processing, determine the content of the processing (whether to perform the sign inversion operation or the flip operation). . Therefore, even when the one-dimensional conversion as described above is realized by software, control of pre-processing and post-processing is complicated, processing load is increased, and mounting cost may be increased.

制御 Also, in the case of the inverse primary conversion, the same control as in the case of the above-described primary conversion is required. That is, both the flip operation and the sign inversion operation need to be candidates as the pre-processing and the post-processing. Therefore, as in the case of the primary conversion described above, there is a possibility that the circuit scale and the processing load increase, and the mounting cost increases.

<3. Concept>
Therefore, the type2 / type4 AMT using the FTS operation and the STF operation is simplified to suppress an increase in mounting cost.

変換 The transformation matrix TDCT4 of the transformation type DCT4, the transformation matrix TDST4 of the transformation type DST4, the flip matrix F, and the sign inversion matrix S have characteristics as shown in the following equations (10) to (13). That is, transform matrix TDCT4 of transform type DCT4, transform matrix TDST4 of transform type DST4, flip matrix F, and sign inversion matrix S are symmetric matrices.

Therefore, for example, as shown in FIG. 2A, a process of performing an orthogonal transformation process 21 of a conversion type DST4 on an input signal X and outputting an output signal Y is performed as shown in FIG. 2C. This can be realized by an FTS operation using the orthogonal transform processing 26 of the transform type DCT4. That is, a sign inversion operation (S) 25 is performed on the input signal X, and an orthogonal transformation process (T) 26 of the transform type DCT4 is performed on the input signal X on which the sign inversion operation is performed. By performing a flip operation (F) 27 on the orthogonal transform process 21, a process equivalent to the orthogonal transform process 21 of the transform type DST4 can be performed. Note that the orthogonal transformation processing 26 and the orthogonal transformation processing 23 are equivalent. That is, the transformation matrix T _DST4 of the transformation type DST4 is represented by the following equation (14) using the flip matrix F representing the flip operation, the transformation matrix T _DCT4 of the transformation type _DCT4 , and the sign inversion matrix S representing the sign inversion operation. Can be expressed as

In this way, when the one-dimensional conversion of the conversion type DST4 and the one-dimensional conversion of the conversion type DST2 are selectively performed, the sign inversion operation (S) 12 (B in FIG. 1) and the sign inversion operation (B in FIG. 1) are performed. S) 25 (C in FIG. 2). Similarly, the flip operation (F) 14 (B in FIG. 1) and the flip operation (F) 27 (C in FIG. 2) can be shared. In other words, the pre-processing performed before the orthogonal transformation processing can be unified (to the sign inversion operation (S)), and the post-processing performed after the orthogonal transformation processing can be unified (to the flip operation (F)). .

The same applies to the case of inverse one-dimensional conversion. For example, the transpose matrix T _DST2 ^t of the transform matrix of the transform type DST2 is obtained by using the sign inversion matrix S and the transpose matrix of the transform matrix of the transform type DCT2 based on the above-described equations (6), (12), and (13). Using T _DCT2 ^t and the flip matrix F, it can be expressed as the following equation (15).

In addition, the transposed matrix T _DST4 ^t of the transform matrix of the transform type DST4 is obtained by using the sign inversion matrix S and the transposed matrix of the transform matrix of the transform type DCT4 from Equations (7), (12), and (13) described above. Using T _DCT4 ^t and the flip matrix F, it can be expressed as the following equation (16).

Accordingly, similarly to the case of the expression (14), the expression can be modified as the following expression (17).

As shown in Expressions (15) and (17), in the case of the inverse one-dimensional conversion, when the inverse one-dimensional conversion of the conversion type DST4 and the inverse one-dimensional conversion of the conversion type DST2 are selectively performed, a code is used. The reversing operation (S) and the flip operation (F) can be shared. In other words, unifying the pre-processing performed before the inverse orthogonal transformation processing (to the flip operation (F)) and unifying the post-processing performed after the inverse orthogonal transformation processing (to the sign inversion operation (S)). Can be.

As described above, the selection of the processing contents in the pre-processing and post-processing (whether to perform the sign inversion operation (S) or the flip operation (F)) can be omitted, so that one-dimensional conversion or inverse one-dimensional conversion is performed. Can be suppressed (simplification of the configuration), and one-dimensional conversion or inverse one-dimensional conversion can be performed more easily. That is, it is possible to suppress an increase in circuit scale and processing load, and to suppress an increase in mounting cost.

<4. First Embodiment>
<Conversion device>
FIG. 6 is a block diagram illustrating an example of a main configuration of a conversion device that is an aspect of an image processing device to which the present technology is applied. The conversion device 100 shown in FIG. 6 is a device that performs one-dimensional conversion of conversion types DCT2, DST2, DCT4, and DST4 on input coefficient data. As illustrated in FIG. 6, the conversion device 100 includes a control unit 101, a sign inversion unit 102, a matrix operation unit 103, and a flip unit 104.

The control unit 101 performs processing related to one-dimensional conversion control. For example, the control unit 101 sets a sign inversion flag (signChangeFlag), which is flag information indicating whether to perform sign inversion, based on parameters such as the input conversion type identifier trTypeIdx, and sets the sign inversion unit 102 to the sign inversion flag. To control the sign inversion operation (S). Further, for example, the control unit 101 may use the base used for matrix calculation based on parameters such as the input conversion type identifier trTypeIdx, the width log2TBWidth of the input signal processing target block, and the height log2TBHeight of the input signal processing target block. By setting base conversion matrix selection information that specifies the conversion matrix T _base and supplying it to the matrix calculation unit 103, matrix calculation using the _base conversion matrix T _base is controlled. Further, for example, the control unit 101 sets a flip flag (flipFlag), which is flag information indicating whether or not to perform a flip operation (F), based on parameters such as the input conversion type identifier trTypeIdx. By supplying the signal to the flip unit 104, the flip operation (F) is controlled.

The control unit 101 includes a sign inversion flag setting unit 111, a base conversion matrix selection unit 112, and a flip flag setting unit 113. The sign inversion flag setting unit 111 sets a sign inversion flag (signChangeFlag) based on parameters such as the conversion type identifier trTypeIdx. The base conversion matrix selection unit 112 sets base conversion matrix selection information based on parameters such as the conversion type identifier trTypeIdx, the width log2TBWidth of the input signal processing target block, and the height log2TBHeight of the input signal processing target block. The flip flag setting unit 113 sets a flip flag (flipFlag) based on parameters such as the conversion type identifier trTypeIdx.

The control unit 101 has an optional configuration. For example, the control unit 101 may be configured by a logic circuit that implements the above processing. In addition, the control unit 101 has, for example, a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), and the like, and executes the program using them to realize the above-described processing. You may do so. Of course, the control unit 101 may have both of the configurations, and a part of the above processing may be realized by a logic circuit, and the other may be realized by executing a program.

Each processing unit of the sign inversion flag setting unit 111, the base conversion matrix selection unit 112, and the flip flag setting unit 113 has an arbitrary configuration. For example, each processing unit may be configured by a logic circuit that realizes the above-described processing. In addition, each processing unit may include, for example, a CPU, a ROM, a RAM, and the like, and may execute the program by using the CPU, the ROM, the RAM, and the like, thereby realizing the above-described processing. Of course, each processing unit may have both configurations, and a part of the above-described processing may be realized by a logic circuit, and the other may be realized by executing a program. The configuration of each processing unit may be independent from each other. For example, some of the processing units may realize a part of the above-described processing by a logic circuit, and some of the other processing units may execute a program. May be implemented, and another processing unit may implement the above-described processing by both the logic circuit and the execution of the program.

The sign inversion unit 102 performs a process related to the sign inversion operation (S). For example, the sign inversion unit 102 performs a sign inversion operation (S) on the input coefficient data Xin to invert the sign of the coefficient data at the odd-numbered position to generate coefficient data X ′. Note that the sign inversion unit 102 can also skip (omit) the sign inversion operation (S). In that case, the input coefficient data Xin is used as it is as the coefficient data X ′. The sign inversion unit 102 selects whether or not to execute a sign inversion operation (S) based on a sign inversion flag (signChangeFlag) supplied from the control unit 101. In either case, the sign inversion unit 102 supplies the coefficient data X ′ to the matrix operation unit 103.

Sign inverting section 102 has an arbitrary configuration. For example, the sign inversion unit 102 may be configured by a logic circuit that implements the above-described processing. In addition, the sign inversion unit 102 may include, for example, a CPU, a ROM, a RAM, and the like, and may execute the program using the CPU, the ROM, the RAM, and the like, thereby realizing the above-described processing. Of course, the sign inverting unit 102 may have both configurations, and a part of the above processing may be realized by a logic circuit, and the other may be realized by executing a program.

The matrix operation unit 103 performs a process related to the matrix operation. For example, the matrix operation unit 103 performs a matrix operation (one-dimensional conversion) using the _base conversion matrix T _base on the coefficient data X ′ supplied from the sign inversion unit 102 to generate coefficient data X ″. . The matrix calculation unit 103 performs a matrix calculation using a conversion matrix of the conversion type specified by the base conversion matrix selection information supplied from the control unit 101. The matrix operation unit 103 has a base conversion matrix LUT120. In the base transform matrix LUT 120, a transform matrix 121 of a transform type DCT2 and a transform matrix 122 of a transform type DCT4 are registered (stored). A conversion matrix other than the conversion matrix 121 and the conversion matrix 122 may be registered in the base conversion matrix LUT 120. The matrix calculation unit 103 reads a conversion matrix of the conversion type specified by the base conversion matrix selection information from the base conversion matrix LUT 120, and uses the conversion matrix as a base conversion matrix in the matrix calculation for the coefficient data X ′. The matrix operation unit 103 supplies the generated coefficient data X ″ to the flip unit 104.

The matrix operation unit 103 has an arbitrary configuration. For example, the matrix operation unit 103 may be configured by a logic circuit that implements the above-described processing. In addition, the matrix operation unit 103 may include, for example, a CPU, a ROM, a RAM, and the like, and execute the program using the CPU, the ROM, the RAM, and the like, thereby realizing the above-described processing. Of course, the matrix operation unit 103 may have both configurations, and a part of the above processing may be realized by a logic circuit, and the other may be realized by executing a program. In any case, the matrix operation unit 103 has a storage area such as a RAM, for example, and forms the base conversion matrix LUT 120.

The flip unit 104 performs a process related to the flip operation (F). For example, the flip unit 104 performs a flip operation (F) on the coefficient data X ″ to rearrange the order of the coefficient data in the reverse order, and generates output coefficient data Xout. The flip unit 104 can skip (omit) the flip operation (F). In that case, the coefficient data X ″ is directly used as the output coefficient data Xout. The flip unit 104 selects whether to execute a flip operation (F) based on a flip flag (flipFlag) supplied from the control unit 101. In either case, the flip unit 104 outputs the output coefficient data Xout to the outside of the conversion device 100.

The flip unit 104 has an optional configuration. For example, the flip unit 104 may be configured by a logic circuit that realizes the above processing. Further, the flip unit 104 may include, for example, a CPU, a ROM, a RAM, and the like, and may execute the program using the CPU, the ROM, the RAM, and the like, thereby realizing the above-described processing. Of course, the flip unit 104 may have both configurations, and a part of the above processing may be realized by a logic circuit, and the other may be realized by executing a program.

<Control example>
In such a conversion device 100, for example, the control unit 101 performs control as shown in the table in FIG. For example, when the input conversion type identifier trTypeIdx is 0, the control unit 101 controls the conversion type (trType) to perform one-dimensional conversion of DCT2. That is, the control unit 101 sets the sign inversion flag (signChangeFlag) to false (for example, 0) using the sign inversion flag setting unit 111. Further, the control unit 101 uses the base transformation matrix selection unit 112 to generate base transformation matrix selection information that specifies a base transformation matrix transMatrix _nTbS, DCT2 of the transformation type DCT2 and the size of nTbS × nTbS. Further, the control unit 101 sets the flip flag (flipFlag) to false (for example, 0) using the flip flag setting unit 113. That is, in this case, only the matrix operation using the transform matrix of the transform type DCT2 is performed, and the sign inversion operation (S) on the input coefficient data Xin and the flip operation (F) on the coefficient data X ″ that is an orthogonal transformation coefficient are performed. Is skipped.

For example, when the input conversion type identifier trTypeIdx is 1, the control unit 101 controls so that the conversion type (trType) performs one-dimensional conversion of DCT4. That is, the control unit 101 uses the sign inversion flag setting unit 111 to set the sign inversion flag to False (for example, 0). In addition, the control unit 101 uses the base transformation matrix selection unit 112 to generate base transformation matrix selection information that specifies a base transformation matrix transMatrix _{nTbS, DCT4} of the transformation type DCT4 and the size of nTbS × nTbS. Further, the control unit 101 sets the flip flag to False (for example, 0) using the flip flag setting unit 113. That is, in this case, only the matrix operation using the transform matrix of the transform type DCT4 is performed, and the sign inversion operation (S) on the input coefficient data Xin and the flip operation (F) on the coefficient data X ″ that is the orthogonal transformation coefficient are performed. Is skipped.

For example, when the input conversion type identifier trTypeIdx is 2, the control unit 101 controls so that the conversion type (trType) performs one-dimensional conversion of DST4. That is, the control unit 101 sets the sign inversion flag to True (for example, 1) by using the sign inversion flag setting unit 111. In addition, the control unit 101 uses the base transformation matrix selection unit 112 to generate base transformation matrix selection information that specifies a base transformation matrix transMatrix _{nTbS, DCT4} of the transformation type DCT4 and the size of nTbS × nTbS. Further, the control unit 101 sets the flip flag to True (for example, 1) using the flip flag setting unit 113. That is, in this case, the sign inversion operation (S) on the input coefficient data Xin, the matrix operation using the conversion matrix of the conversion type DCT4, and the flip operation (F) on the coefficient data X ″ that is the orthogonal transformation coefficient are executed. You.

For example, when the input conversion type identifier trTypeIdx is 3, the control unit 101 controls so that the conversion type (trType) is one-dimensional conversion of DST2. That is, the control unit 101 sets the sign inversion flag to True (for example, 1) by using the sign inversion flag setting unit 111. Further, the control unit 101 uses the base transformation matrix selection unit 112 to generate base transformation matrix selection information that specifies a base transformation matrix transMatrix _nTbS, DCT2 of the transformation type DCT2 and the size of nTbS × nTbS. Further, the control unit 101 sets the flip flag to True (for example, 1) using the flip flag setting unit 113. That is, in this case, the sign inversion operation (S) on the input coefficient data Xin, the matrix operation using the conversion matrix of the conversion type DCT2, and the flip operation (F) on the coefficient data X ″ that is the orthogonal transformation coefficient are executed. You.

As described above, the conversion apparatus 100 can perform one-dimensional conversion of the conversion type DCT2 or DCT4 by skipping the sign inversion operation (S) and the flip operation (F). In addition, the conversion apparatus 100 can perform the sign inversion operation (S) and the flip operation (F), and perform one-dimensional conversion of the conversion type DST2 or DST4 by the FTS operation.

In other words, as shown in FIG. 6, the conversion apparatus 100 can use the pre-processing unit for the conversion type DST2 and the pre-processing unit for the conversion type DST4 in common with the sign inversion unit 102. Similarly, a post-processing unit for the conversion type DST2 and a post-processing unit for the conversion type DST4 can be shared by the flip unit 104. Therefore, it is possible to suppress an increase in circuit scale and an increase in mounting cost (the circuit scale can be reduced and the mounting cost can be reduced).

<Conversion process flow>
Next, an example of the flow of a conversion process performed by the conversion device 100 will be described with reference to the flowchart in FIG.

When the conversion process is started, the control unit 101 (the sign inversion flag setting unit 111, the base conversion matrix selection unit 112, and the flip flag setting unit 113) determines in step S101 that the conversion type supplied from outside the conversion device 100 Based on trTypeIdx and size (log2TBWidth, log2TBHeight), base conversion matrix selection information, sign inversion flag (signChangeFlag), and flip flag (flipFlag) are set as described above.

In step S102, the sign inversion unit 102 determines whether or not to perform the sign inversion operation (S) based on the sign inversion flag set in step S101 (signChangeFlag == True?). When the value of the sign inversion flag is true and it is determined that the sign inversion operation (S) is to be performed, the process proceeds to step S103.

In step S103, the sign inversion unit 102 performs a sign inversion operation (S) on the input coefficient data Xin that is a one-dimensional signal sequence, and generates coefficient data X ′ that is a one-dimensional signal sequence. This sign inversion operation (S) can be expressed, for example, as in the following Expression (18).

すると When the process in step S103 ends, the process proceeds to step S104. If it is determined in step S102 that the value of the sign inversion flag is false (False) and the sign inversion operation (S) is not to be performed, the process of step S103 is skipped, and the input coefficient data Xin is directly used as the coefficient data. X 'is set, and the process proceeds to step S104.

In step S104, the matrix computing unit 103 obtains the base transform matrix T _base selected, i.e., the base transform matrix T _base specified by the base transform matrix selected information set in step S101 from the base transformation matrix LUT 120, A matrix operation (one-dimensional conversion) is performed on the coefficient data X ′, which is a one-dimensional signal sequence, using the data to generate coefficient data X ″, which is a one-dimensional signal sequence. This matrix operation can be represented, for example, by the following equation (19).

In step S105, the flip unit 104 determines whether to perform a flip operation (F) based on the flip flag set in step S101 (FlipFlagF == True?). If the value of the flip flag is True and it is determined that the flip operation (F) is to be performed, the process proceeds to step S106.

In step S106, the flip unit 104 performs a flip operation (F) on the coefficient data X ″ that is the one-dimensional signal sequence obtained in step S104, and generates output coefficient data Xout that is a one-dimensional signal sequence. . This flip operation (F) can be expressed, for example, as in the following equation (20).

The flip unit 104 outputs the generated output coefficient data Xout to the outside of the conversion device 100. When the processing in step S106 ends, the conversion processing ends. If it is determined in step S105 that the value of the flip flag is false (False) and the flip operation (F) is not to be performed, the process of step S106 is skipped, and the coefficient data X ″ is output as is as the output coefficient data. Xout is output to the outside of the conversion device 100. When the output coefficient data Xout is output, the conversion process ends.

In other words, in this case, it is not necessary to check the contents of the pre-processing and post-processing, and it is only necessary to control whether or not to execute the sign inversion operation (S) and the flip operation (F). Therefore, the complexity of the control of the pre-processing and the post-processing can be suppressed. Therefore, an increase in processing load can be suppressed, and an increase in mounting cost can be suppressed (processing load can be reduced and mounting cost can be reduced).

As described above, the conversion device 100 can suppress the complexity of the configuration of the one-dimensional conversion (simplify the configuration), and can perform the one-dimensional conversion more easily.

<Conversion type>
In the above description, the conversion apparatus 100 realizes the one-dimensional conversion of the conversion type DST2 by the FTS operation including the one-dimensional conversion of the conversion type DCT2, and the conversion device 100 performs the one-dimensional conversion of the conversion type DST4 by the FTS operation including the one-dimensional conversion of the conversion type DCT4. Although an example of implementing one-dimensional conversion has been described, the conversion type applicable to the conversion device 100 is not limited to the above-described example.

For example, a one-dimensional conversion using a conversion matrix of a first conversion type is equivalent to an FTS operation including a one-dimensional conversion using a conversion matrix of a second conversion type different from the first conversion type. , The first conversion type and the second conversion type can be applied to the conversion device 100. In the above example, the conversion type DST2 is the first conversion type, and the conversion type DCT2 is the second conversion type.

As described above, the relationship that can realize the one-dimensional conversion using the conversion matrix of the first conversion type by the FTS operation including the one-dimensional conversion using the conversion matrix of the second conversion type is described as “the FTS operation. Also referred to as a “paired relationship”. In addition, the first conversion type and the second conversion type having such a relationship are also referred to as “conversion types that are paired by the FTS operation”. For example, the conversion type paired by the FTS operation of the first conversion type is the second conversion type. Therefore, the one-dimensional conversion using the conversion matrix of the first conversion type is realized by the FTS operation including the one-dimensional conversion using the conversion matrix of the second conversion type, which is the conversion type paired by the FTS operation. Can be.

Also, for example, one-dimensional conversion using a conversion matrix of a third conversion type different from the first conversion type and the second conversion type, and a fourth conversion type different from the first to third conversion types. A third conversion type and a fourth conversion type, in which an STF operation including a one-dimensional conversion using a conversion matrix of the following conversion type is equivalent, and the conversion matrix of the fourth conversion type is a symmetric matrix, It can be applied to the conversion device 100. In the above example, the conversion type DST4 is the third conversion type, and the conversion type DCT4 is the fourth conversion type.

As described above, the relationship that can realize the one-dimensional conversion using the conversion matrix of the third conversion type by the STF operation including the one-dimensional conversion using the conversion matrix of the fourth conversion type is described as “by the STF operation. Also referred to as a “paired relationship”. Further, the third conversion type and the fourth conversion type having such a relationship are also referred to as “conversion types paired by the STF operation”. For example, the conversion type paired by the STF operation of the third conversion type is the fourth conversion type. Therefore, the one-dimensional transformation using the transformation matrix of the third transformation type includes the FTS including the one-dimensional transformation using the transformation matrix that is the symmetric matrix of the fourth transformation type that is the transformation type paired by the STF operation. It can be realized by operation.

Additionally, as in the above-described example, the one-dimensional conversion using the conversion matrix of the conversion type DST4 can be realized by an FTS operation including the one-dimensional conversion using the conversion matrix of the conversion type DCT4. That is, the one-dimensional conversion using the conversion matrix of the third conversion type can be realized by an FTS operation including the one-dimensional conversion using the conversion matrix of the fourth conversion type. That is, the fourth conversion types are the “conversion types paired by the STF operation” and the “conversion types paired by the FTS operation” of the third conversion type.

As described above, the conversion device 100
A sign inversion unit that performs a sign inversion operation (S) for inverting the sign of the odd-numbered signal of the one-dimensional signal sequence with respect to the one-dimensional signal sequence of the coefficient data regarding the image;
For the one-dimensional signal sequence subjected to the sign inversion operation by the sign inversion unit,
When realizing a one-dimensional conversion of the first conversion type, a conversion matrix of a second conversion type for realizing a one-dimensional conversion of the first conversion type by an FTS operation is set as a base conversion matrix,
When a one-dimensional conversion of the third conversion type is realized, a conversion matrix that is a symmetric matrix of a fourth conversion type that realizes a one-dimensional conversion of the third conversion type by an STF operation is a base conversion matrix,
A matrix operation unit that performs a matrix operation using the base transformation matrix;
And a flip unit that performs a flip operation (F) for rearranging the order of each coefficient in the reverse order on the one-dimensional signal sequence on which the matrix operation is performed by the matrix operation unit.

In other words,
A sign inversion operation (S) for inverting the sign of the odd-numbered signal of the one-dimensional signal sequence is performed on the one-dimensional signal sequence of the coefficient data relating to the image,
For the sign-inverted one-dimensional signal sequence,
When realizing a one-dimensional conversion of the first conversion type, a conversion matrix of a second conversion type for realizing a one-dimensional conversion of the first conversion type by an FTS operation is set as a base conversion matrix,
When a one-dimensional conversion of the third conversion type is realized, a conversion matrix that is a symmetric matrix of a fourth conversion type that realizes a one-dimensional conversion of the third conversion type by an STF operation is a base conversion matrix,
Perform a matrix operation using the base transformation matrix,
A flip operation (F) for rearranging the order of each coefficient in reverse order may be performed on the one-dimensional signal sequence on which the matrix operation has been performed.

変換 By doing so, the conversion apparatus 100 can perform one-dimensional conversion more easily.

Note that when the “conversion type paired by the STF operation” and the “conversion type paired by the FTS operation” are not distinguished from each other, they are also referred to as “paired conversion type”.

When realizing the one-dimensional conversion of the second conversion type or the fourth conversion type, the conversion apparatus 100 skips the sign inversion operation and the flip operation (F), and performs the conversion on the one-dimensional signal sequence of the coefficient data. The matrix operation may be performed using a conversion matrix of the second or fourth conversion type as a base conversion matrix. By doing so, the conversion device 100 can easily realize one-dimensional conversion of the second conversion type or the fourth conversion type.

The conversion apparatus 100 further includes a sign inversion flag setting unit that sets a sign inversion flag indicating whether or not to perform the sign inversion operation (S) based on the designated conversion type of the one-dimensional conversion. However, based on the sign inversion flag set by the sign inversion flag setting unit, the sign inversion operation (S) may be performed or skipped (select and execute).

The conversion apparatus 100 further includes a flip flag setting unit that sets a flip flag indicating whether or not to perform a flip operation (F) based on the specified conversion type of the one-dimensional conversion. The flip operation (F) may be performed or skipped (either selected and executed) based on the flip flag set by the flag setting unit.

By doing so, the conversion apparatus 100 can easily control whether to execute or skip the sign inversion operation and the flip operation (F) based on the designation of the conversion type of the one-dimensional conversion. Therefore, the conversion device 100 can more easily realize each one-dimensional conversion of the first to fourth conversion types.

In addition, the conversion apparatus 100 selects which of the second conversion type conversion matrix and the fourth conversion type conversion matrix is to be used as the base conversion matrix, based on the specified one-dimensional conversion conversion type. A base conversion matrix selection unit may be provided, and a matrix operation may be performed using the base conversion matrix selected by the base conversion matrix selection unit. By doing so, the conversion apparatus 100 can easily select the base conversion matrix to be used based on the specification of the conversion type of the one-dimensional conversion. Therefore, the conversion device 100 can more easily realize each one-dimensional conversion of the first to fourth conversion types.

<5. Second Embodiment>
<Inverse conversion device>
FIG. 9 is a block diagram illustrating an example of a main configuration of an inverse conversion device that is an aspect of an image processing device to which the present technology is applied. The inverse transform device 150 shown in FIG. 6 is a device that performs an inverse one-dimensional transform of transform types DCT2, DST2, DCT4, and DST4 on input coefficient data (orthogonal transform coefficients). The inverse conversion device 150 is a device corresponding to the conversion device 100 described above in the first embodiment, and performs an inverse one-dimensional conversion which is an inverse process of the one-dimensional conversion performed by the conversion device 100. As illustrated in FIG. 9, the inverse transform device 150 includes a control unit 151, a flip unit 152, a matrix operation unit 153, and a sign inversion unit 154.

The control unit 151 performs a process related to control of the inverse one-dimensional conversion. For example, the control unit 151 sets a flip flag (flipFlag), which is flag information indicating whether or not to perform a flip operation (F), based on the input parameters such as the conversion type identifier trTypeIdx, and sets the flip flag. The flip operation (F) is controlled by supplying the signal to the terminal 152. Further, for example, the control unit 151 may use the base used for matrix calculation based on parameters such as the input conversion type identifier trTypeIdx, the width log2TBWidth of the input signal processing target block, and the height log2TBHeight of the input signal processing target block. By setting base conversion matrix selection information specifying the conversion matrix T _base and supplying it to the matrix calculation unit 153, matrix calculation using the _base conversion matrix T _base is controlled. Further, for example, the control unit 151 sets a sign inversion flag (signChangeFlag) which is flag information indicating whether or not to perform sign inversion based on parameters such as the input conversion type identifier trTypeIdx, and sets the sign inversion to sign inversion. The sign inversion operation (S) is controlled by supplying the signal to the unit 154.

The control unit 151 includes a flip flag setting unit 161, a base conversion matrix selection unit 162, and a sign inversion flag setting unit 163. The flip flag setting unit 161 sets a flip flag (flipFlag) based on parameters such as the conversion type identifier trTypeIdx. The base conversion matrix selection unit 162 sets base conversion matrix selection information based on parameters such as the conversion type identifier trTypeIdx, the width log2TBWidth of the input signal processing target block, and the height log2TBHeight of the input signal processing target block. The sign inversion flag setting unit 163 sets a sign inversion flag (signChangeFlag) based on parameters such as the conversion type identifier trTypeIdx.

The control unit 151 has an optional configuration. For example, the control unit 151 may be configured by a logic circuit that implements the above processing. Further, the control unit 151 may include, for example, a CPU, a ROM, a RAM, and the like, and may execute the program by using the CPU, the ROM, the RAM, and the like to realize the above-described processing. Of course, the control unit 151 may have both of the configurations, and a part of the above-described processing may be realized by a logic circuit, and the other may be realized by executing a program.

Each processing unit of the flip flag setting unit 161, the base conversion matrix selecting unit 162, and the sign inversion flag setting unit 163 has an arbitrary configuration. For example, each processing unit may be configured by a logic circuit that realizes the above-described processing. In addition, each processing unit may include, for example, a CPU, a ROM, a RAM, and the like, and may execute the program by using the CPU, the ROM, the RAM, and the like, thereby realizing the above-described processing. Of course, each processing unit may have both configurations, and a part of the above-described processing may be realized by a logic circuit, and the other may be realized by executing a program. The configuration of each processing unit may be independent from each other. For example, some of the processing units may realize a part of the above-described processing by a logic circuit, and some of the other processing units may execute a program. May be implemented, and another processing unit may implement the above-described processing by both the logic circuit and the execution of the program.

The flip unit 152 performs a process related to the flip operation (F). For example, the flip unit 152 performs a flip operation (F) for rearranging the order of the coefficient data on the input coefficient data Xin in reverse order, and generates coefficient data X ′. The flip unit 152 can skip (omit) the flip operation (F). In that case, the input coefficient data Xin is used as it is as the coefficient data X ′. The flip unit 152 selects whether to execute a flip operation (F) based on the flip flag (flipFlag) supplied from the control unit 151. In any case, the flip unit 152 supplies the coefficient data X ′ to the matrix operation unit 153.

The flip section 152 has an optional configuration. For example, the flip unit 152 may be configured by a logic circuit that implements the above-described processing. In addition, the flip unit 152 may include, for example, a CPU, a ROM, a RAM, and the like, and may execute a program using the CPU, the ROM, the RAM, and the like, thereby realizing the above-described processing. Of course, the flip unit 152 may have both configurations, and a part of the above processing may be realized by a logic circuit, and the other may be realized by executing a program.

The matrix calculation unit 153 performs a process related to the matrix calculation. For example, the matrix operation unit 153 performs a matrix operation (inverse one-dimensional conversion) on the coefficient data X ′ supplied from the flip unit 152 using the transposed matrix T _base ^t of the base conversion matrix, and obtains the coefficient data X ′. 'Is generated. The matrix calculation unit 153 performs a matrix calculation using the transposed matrix T _base ^t of the base conversion matrix of the conversion type specified by the base conversion matrix selection information supplied from the control unit 151. The matrix operation unit 153 has a base conversion matrix LUT 170. In the base transformation matrix LUT 170, a transformation matrix 171 of the transformation type DCT2 and a transformation matrix 172 of the transformation type DCT4 are registered (stored). A conversion matrix other than the conversion matrix 171 and the conversion matrix 172 may be registered in the base conversion matrix LUT 170. The matrix calculation unit 153 reads a conversion matrix of the conversion type specified by the base conversion matrix selection information from the base conversion matrix LUT 170, and uses the conversion matrix as a base conversion matrix in the matrix calculation for the coefficient data X ′. That is, the matrix calculation unit 153 performs a matrix calculation on the coefficient data X ′ using the transposed matrix T _base ^t of the base conversion matrix read from the base conversion matrix LUT 170. The matrix operation unit 153 supplies the generated coefficient data X ″ to the sign inversion unit 154.

The matrix operation unit 153 has an arbitrary configuration. For example, the matrix operation unit 153 may be configured by a logic circuit that implements the above-described processing. Further, the matrix operation unit 153 may include, for example, a CPU, a ROM, a RAM, and the like, and may execute the program using the CPU, the ROM, the RAM, and the like, thereby realizing the above-described processing. Of course, the matrix operation unit 153 may have both configurations, and a part of the above processing may be realized by a logic circuit, and the other may be realized by executing a program. In any case, the matrix operation unit 153 has a storage area such as a RAM, for example, and forms the base conversion matrix LUT 170.

The sign inversion unit 154 performs a process related to the sign inversion operation (S). For example, the sign inversion unit 154 performs a sign inversion operation (S) on the coefficient data X ″ to invert the sign of the coefficient data at the odd-numbered position to generate output coefficient data Xout. Note that the sign inversion unit 154 can also skip (omit) the sign inversion operation (S). In that case, the coefficient data X ″ is directly used as the output coefficient data Xout. The sign inversion unit 154 selects whether or not to execute the sign inversion operation (S) based on the sign inversion flag (signChangeFlag) supplied from the control unit 151. In any case, the sign inversion unit 154 outputs the output coefficient data Xout to the outside of the inverse transform device 150.

The sign inversion unit 154 has an optional configuration. For example, the sign inversion unit 154 may be configured by a logic circuit that implements the above-described processing. In addition, the sign inverting unit 154 may include, for example, a CPU, a ROM, a RAM, and the like, and may execute the program using the CPU, the ROM, the RAM, and the like, thereby realizing the above-described processing. Of course, the sign inverting unit 154 may have both configurations, and a part of the above processing may be realized by a logic circuit, and the other may be realized by executing a program.

<Control example>
In such an inverse conversion device 150, for example, the control unit 151 performs control as shown in the table in FIG. For example, when the input conversion type identifier trTypeIdx is 0, the control unit 151 controls so that the conversion type (trType) performs the inverse one-dimensional conversion of DCT2. That is, the control unit 151 sets the flip flag (flipFlag) to False (for example, 0) by using the flip flag setting unit 161. Further, the control unit 151 uses the base transformation matrix selection unit 162 to generate base transformation matrix selection information that specifies the base transformation matrix transMatrix _nTbS, DCT2 of the transformation type DCT2 and the size of nTbS × nTbS. Further, the control unit 151 sets the sign inversion flag (signChangeFlag) to false (for example, 0) using the sign inversion flag setting unit 163. That is, in this case, only the matrix operation using the transposed matrix (transMatrix _{nTbS, DCT2} ) ^t of the transform matrix of the transform type _DCT2 is performed, and the flip operation (F) on the input coefficient data Xin and the code on the coefficient data X ″ are performed. The inversion operation (S) is skipped.

For example, when the input conversion type identifier trTypeIdx is 1, the control unit 151 controls the conversion type (trType) to perform inverse one-dimensional conversion of DCT4. That is, the control unit 151 sets the flip flag to False (for example, 0) using the flip flag setting unit 161. In addition, the control unit 151 uses the base transformation matrix selection unit 162 to generate base transformation matrix selection information that specifies the base transformation matrix transMatrix _{nTbS, DCT4} of the transformation type DCT4 and the size of nTbS × nTbS. Further, the control unit 151 sets the sign inversion flag to False (for example, 0) using the sign inversion flag setting unit 163. That is, in this case, only the matrix operation using the transposed matrix (transMatrix _{nTbS, DCT4} ) ^t of the transform matrix of the transform type DCT4 is performed, and the flip operation (F) for the input coefficient data Xin and the code for the coefficient data X ″ are performed. The inversion operation (S) is skipped.

For example, when the input conversion type identifier trTypeIdx is 2, the control unit 151 controls so that the conversion type (trType) performs the inverse one-dimensional conversion of DST4. That is, the control unit 151 sets the flip flag to True (for example, 1) using the flip flag setting unit 161. In addition, the control unit 151 uses the base transformation matrix selection unit 162 to generate base transformation matrix selection information that specifies the base transformation matrix transMatrix _{nTbS, DCT4} of the transformation type DCT4 and the size of nTbS × nTbS. Further, the control unit 151 sets the sign inversion flag to True (for example, 1) by using the sign inversion flag setting unit 163. That is, in this case, a flip operation (F) on the input coefficient data Xin _, a matrix operation using the transposed matrix (transMatrix _{nTbS, DCT4} ) ^t of the conversion matrix of the conversion type DCT4, and a sign inversion operation on the coefficient data X ″ ( S) is executed.

For example, when the input conversion type identifier trTypeIdx is 3, the control unit 151 controls so that the conversion type (trType) performs the inverse one-dimensional conversion of DST2. That is, the control unit 151 sets the flip flag to True (for example, 1) using the flip flag setting unit 161. Further, the control unit 151 uses the base transformation matrix selection unit 162 to generate base transformation matrix selection information that specifies the base transformation matrix transMatrix _nTbS, DCT2 of the transformation type DCT2 and the size of nTbS × nTbS. Further, the control unit 151 sets the sign inversion flag to True (for example, 1) by using the sign inversion flag setting unit 163. That is, in this case, the flip operation (F) on the input coefficient data Xin _, the matrix operation using the transposed matrix (transMatrix _{nTbS, DCT2} ) ^t of the conversion matrix of the conversion type DCT2, and the sign inversion operation ( S) is executed.

As described above, the inverse transform device 150 can perform the inverse one-dimensional transform of the transform type DCT2 or DCT4 by skipping the flip operation (F) and the sign inversion operation (S). In addition, the inverse transform device 150 executes the flip operation (F) and the sign inversion operation (S), and can perform the inverse one-dimensional conversion of the conversion type DST2 or DST4 by the STF operation.

In other words, as shown in FIG. 9, the inverse conversion device 150 can share the pre-processing unit for the conversion type DST2 and the pre-processing unit for the conversion type DST4 with the flip unit 152. Similarly, the post-processing unit for the conversion type DST2 and the post-processing unit for the conversion type DST4 can be shared by the sign inversion unit 154. Therefore, it is possible to suppress an increase in circuit scale and an increase in mounting cost (the circuit scale can be reduced and the mounting cost can be reduced).

<Flow of inverse transformation process>
Next, an example of the flow of the conversion process executed by the inverse conversion device 150 will be described with reference to the flowchart in FIG.

When the inverse conversion processing is started, the control unit 151 (the flip flag setting unit 161, the base conversion matrix selection unit 162, and the sign inversion flag setting unit 163) in step S151, converts the conversion type trTypeIdx and the size (log2TBWidth, log2TBHeight). ), The base conversion matrix selection information, the flip flag (flipFlag), and the sign inversion flag (signChangeFlag) are set as described above.

において In step S152, the flip unit 152 determines whether or not to perform a flip operation (F) based on the flip flag set in step S151 (FlipFlag == True?). If the value of the flip flag is true and it is determined that the flip operation (F) is to be performed, the process proceeds to step S153.

In step S153, the flip unit 152 performs a flip operation (F) on the input coefficient data Xin that is a one-dimensional signal sequence, and generates coefficient data X ′ that is a one-dimensional signal sequence. This flip operation (F) can be expressed, for example, as in the following equation (21).

すると When the process of step S153 ends, the process proceeds to step S154. If it is determined in step S152 that the value of the flip flag is false (False) and the flip operation (F) is not to be performed, the process of step S153 is skipped, and the input coefficient data Xin is directly used as the coefficient data X ′. , And the process proceeds to step S154.

In step S154, the matrix calculator 153 obtains the base transform matrix T _base selected, i.e., the base transform matrix T _base specified by the base transform matrix selected information set in step S151 from the base transformation matrix LUT 170, A matrix operation (inverse one-dimensional conversion) is performed on the coefficient data X ′ that is a one-dimensional signal sequence using the transposed matrix to generate coefficient data X ″ that is a one-dimensional signal sequence. This matrix operation can be represented, for example, by the following equation (22).

{At step S155, the sign inversion unit 154 determines whether or not to perform the sign inversion operation (S) based on the sign inversion flag set at step S151 (signChangeFlag == True?). If the value of the sign inversion flag is true and it is determined that the sign inversion operation (S) is to be performed, the process proceeds to step S156.

In step S156, the sign inversion unit 154 performs a sign inversion operation (S) on the coefficient data X ″ that is the one-dimensional signal sequence obtained in step S154, and outputs the output coefficient data Xout that is the one-dimensional signal sequence. Generate. This sign inversion operation (S) can be expressed, for example, as in the following Expression (23).

The sign inversion unit 154 outputs the generated output coefficient data Xout to the outside of the inverse transform device 150. When the processing in step S156 ends, the conversion processing ends. If it is determined in step S155 that the value of the sign inversion flag is false (False) and the sign inversion operation (S) is not performed, the process in step S156 is skipped, and the coefficient data X ″ is output as it is. The data is set as coefficient data Xout and output to the outside of the inverse transform device 150. When the output coefficient data Xout is output, the inverse conversion processing ends.

In other words, in this case, it is not necessary to check the contents of the pre-processing and the post-processing, and it is only necessary to control whether or not to execute the flip operation (F) and the sign inversion operation (S). Therefore, the complexity of the control of the pre-processing and the post-processing can be suppressed. Therefore, an increase in processing load can be suppressed, and an increase in mounting cost can be suppressed (processing load can be reduced and mounting cost can be reduced).

As described above, the inverse transform device 150 can suppress the complexity of the configuration of the inverse one-dimensional transform (simplify the configuration), and can more easily perform the inverse one-dimensional transform.

<Conversion type>
In the above, the inverse transform apparatus 150 realizes the inverse one-dimensional transform of the transform type DST2 by the STF operation including the inverse one-dimensional transform of the transform type DCT2, and the STF operation including the inverse one-dimensional transform of the transform type DCT4. Although the example of implementing the inverse one-dimensional conversion of the conversion type DST4 has been described, the conversion type applicable to the inverse conversion device 150 is not limited to the above-described example.

For example, an inverse one-dimensional transform using a transform matrix of a first transform type is equivalent to an STF operation including an inverse one-dimensional transform using a transform matrix of a second transform type different from the first transform type. The first conversion type and the second conversion type can be applied to the inverse conversion device 150. In the above example, the conversion type DST2 is the first conversion type, and the conversion type DCT2 is the second conversion type.

In the case of the inverse one-dimensional conversion, similarly to the case of the one-dimensional conversion described in the first embodiment, the first transfer is performed by the STF operation including the inverse one-dimensional conversion using the conversion matrix of the second conversion type. The relationship that can realize the inverse one-dimensional transformation using the transformation type transformation matrix is also referred to as “the relationship paired by the STF operation”. Further, the first conversion type and the second conversion type having such a relationship are also referred to as “conversion types paired by the STF operation”. For example, in the case of the inverse one-dimensional conversion, the conversion type paired by the STF operation of the first conversion type is the second conversion type. Therefore, the inverse one-dimensional transformation using the transformation matrix of the first transformation type is realized by the STF operation including the inverse one-dimensional transformation using the transformation matrix of the second transformation type, which is the transformation type paired by the STF operation. can do.

Also, for example, an inverse one-dimensional conversion using a conversion matrix of a third conversion type different from the first conversion type and the second conversion type, and a first conversion type different from the first to third conversion types. A third transformation type and a fourth transformation in which the FTS operation including the inverse one-dimensional transformation using the transformation matrix of the fourth transformation type is equivalent, and the transformation matrix of the fourth transformation type is a symmetric matrix The type can also be applied to the inverse converter 150. In the above example, the conversion type DST4 is the third conversion type, and the conversion type DCT4 is the fourth conversion type.

In the case of the inverse one-dimensional conversion, similarly to the case of the one-dimensional conversion described in the first embodiment, the FTS operation including the inverse one-dimensional conversion using the conversion matrix of the fourth conversion type performs the third operation. The relationship that can implement the inverse one-dimensional transformation using the transformation type transformation matrix is also referred to as “the relationship paired by the FTS operation”. Further, the third conversion type and the fourth conversion type having such a relationship are also referred to as “conversion types paired by the FTS operation”. For example, in the case of this inverse one-dimensional conversion, the conversion type paired by the FTS operation of the third conversion type is the fourth conversion type. Therefore, the inverse one-dimensional transformation using the transformation matrix of the third transformation type is the inverse one-dimensional transformation using the transformation matrix that is the symmetric matrix of the fourth transformation type, which is the transformation type paired by the FTS operation. It can be realized by including FTS operation.

Additionally, as in the example described above, the inverse one-dimensional transform using the transform matrix of the transform type DST4 can be realized by an STF operation including the inverse one-dimensional transform using the transform matrix of the transform type DCT4. That is, the inverse one-dimensional conversion using the conversion matrix of the third conversion type can be realized by the STF operation including the one-dimensional conversion using the conversion matrix of the fourth conversion type. That is, the fourth conversion types are the “conversion types paired by the FTS operation” and the “conversion types paired by the STF operation” of the third conversion type.

As described above, the inverse conversion device 150 performs a flip operation for rearranging the order of each coefficient on the one-dimensional signal sequence of the coefficient data relating to the image in the reverse order,
For the one-dimensional signal sequence flipped by the flip part,
When realizing the inverse one-dimensional conversion of the first conversion type, a conversion matrix of the second conversion type that realizes the inverse one-dimensional conversion of the first conversion type by the STF operation is a base conversion matrix,
When implementing the inverse one-dimensional transformation of the third transformation type, a transformation matrix that is a symmetric matrix of the fourth transformation type that implements the inverse one-dimensional transformation of the third transformation type by an FTS operation is a base transformation matrix. age,
A matrix operation unit that performs a matrix operation using the transpose of the base transformation matrix;
A sign inverting unit for performing a sign inversion operation (S) for inverting the sign of the odd-numbered signal of the one-dimensional signal sequence with respect to the one-dimensional signal sequence subjected to the matrix operation by the matrix operation unit. Good.

In other words,
A one-dimensional signal sequence of coefficient data relating to an image is subjected to a flip operation of rearranging the order of each coefficient in reverse order,
For the flip-operated one-dimensional signal sequence,
When realizing the inverse one-dimensional conversion of the first conversion type, a conversion matrix of the second conversion type that realizes the inverse one-dimensional conversion of the first conversion type by the STF operation is a base conversion matrix,
When implementing the inverse one-dimensional transformation of the third transformation type, a transformation matrix that is a symmetric matrix of the fourth transformation type that implements the inverse one-dimensional transformation of the third transformation type by an FTS operation is a base transformation matrix. age,
Performs a matrix operation using the transposed matrix of the base transformation matrix,
A sign inversion operation (S) for inverting the sign of the odd-numbered signal in the one-dimensional signal sequence may be performed on the one-dimensional signal sequence on which the matrix operation has been performed.

にする By doing so, the inverse conversion device 150 can more easily perform the inverse one-dimensional conversion.

逆 In the case of the inverse one-dimensional conversion, when the “conversion type paired by the STF operation” and the “conversion type paired by the FTS operation” are not distinguished from each other, they are also referred to as “paired conversion type”.

When implementing the inverse one-dimensional conversion of the second conversion type or the fourth conversion type, the inverse conversion device 150 skips the flip operation (F) and the sign inversion operation (S) and performs one-dimensional conversion of the coefficient data. A matrix operation may be performed on a signal sequence using a conversion matrix of the second conversion type or the fourth conversion type as a base conversion matrix. By doing so, the inverse conversion device 150 can easily realize the inverse one-dimensional conversion of the second conversion type or the fourth conversion type.

Further, the inverse conversion device 150 includes a flip flag setting unit that sets a flip flag indicating whether to perform a flip operation based on the specified conversion type of the inverse one-dimensional conversion, and the flip unit includes the flip flag. A flip operation may be performed or skipped (either selected or executed) based on the flip flag set by the setting unit.

In addition, the inverse conversion device 150 includes a sign inversion flag setting unit that sets a sign inversion flag indicating whether or not to perform a sign inversion operation (S) based on the designated inverse one-dimensional conversion type. The reversing unit may perform the sign reversing operation (S) or skip (select and execute one) based on the sign reversal flag set by the sign reversal flag setting unit.

By doing so, the inverse conversion device 150 easily controls whether to execute or skip the flip operation (F) and the sign inversion operation (S) based on the designation of the conversion type of the inverse one-dimensional conversion. can do. Therefore, the inverse conversion device 150 can more easily realize the inverse one-dimensional conversion of each of the first to fourth conversion types.

Further, the inverse transform device 150 determines which of the second transform type transform matrix and the fourth transform type transform matrix is the base transform matrix based on the designated inverse one-dimensional transform type. A base conversion matrix selection unit for selection may be provided, and a matrix operation may be performed using the base conversion matrix selected by the base conversion matrix selection unit. By doing so, the inverse transform device 150 can easily select the base transform matrix to be used based on the designation of the transform type of the inverse one-dimensional transform. Therefore, the inverse conversion device 150 can more easily realize the inverse one-dimensional conversion of each of the first to fourth conversion types.

<6. Third Embodiment>
<Derivation of base transformation matrix>
In the first embodiment and the second embodiment, either the transform matrix of the transform type DCT2 registered in the base transform matrix LUT or the transform matrix of the transform type DCT4 is selected to perform the matrix operation. Although described as being used, the present invention is not limited to this, and a base transformation matrix may be derived.

For example, the transform matrix of the transform type DCT2 and the transform matrix of the transform type DCT4 are derived by sampling (extracting) matrix elements by a predetermined method from the transform matrix of the transform type DCT2 having a larger size. can do. Therefore, if the transform matrix of this large size transform type DCT2 is stored in advance, a base transform matrix (transform matrix of transform type DCT2 or transform matrix of transform type DCT4) used for matrix operation is derived from the transform matrix. be able to.

For example, the maximum size of a transform block (for example, 64) is maxTbS, the maximum size of a transform block to which AMT can be applied (for example, 32) is maxTbAMT, and the size nTbS of one-dimensional transform (also referred to as 1D transform) is nTbS <= maxTbAMT. It is assumed that <maxTbS. In this case, the size of the derived base transform matrix is (nTbS) × (nTbS), and the transform matrix of the transform type DCT2 of (maxTbS) × (maxTbS) may be stored (prepared) in advance.

The base transformation matrix used for the matrix operation is obtained by sampling the transformation matrix of the transformation type DCT2 (derived transformation matrix maxTbS-pt DCT2) of (maxTbS) × (maxTbS) prepared in this manner based on a predetermined sampling parameter. ((NTbS) × (nTbS) transform type DCT2 or transform type DCT4 transform matrix) can be derived as a submatrix.

<Sampling parameters>
Here, the sampling parameters will be described. The sampling parameters may be any. For example, a sampling interval stepsize indicating a sampling row interval, a row offset offsetCol indicating a sampling offset (row position), and a column offset offsetRow indicating a sampling offset (column position) may be included.

The sampling interval stepsize is a parameter indicating how many lines are sampled. The row offset offsetCol is a parameter indicating the position of the first row at which sampling is started (the row number). The column offset offsetRow is a parameter that indicates the position of the first column from which sampling is started (the number of the column). In this specification, the row numbers and column numbers of the transformation matrix start from “0” (that is, 0 rows and 0 columns).

<Example of derivation for each conversion type>
The sampling method (that is, the value of the sampling parameter) is determined by the conversion type of the derived conversion matrix as shown in the table shown in FIG. For example, as shown in the second row from the bottom of the table shown in FIG. 12, when the base conversion matrix nTbS-pt DCT2 whose conversion type trType is DCT2 is derived from the prepared derivation source conversion matrix maxTbS-pt DCT2, sampling is performed. The interval stepsize may be set to “1 << (Log2 (maxTbS) −Log2 (nTbS))”, the row offset offsetCol may be set to “0”, and the column offsetoffsetRow may be set to “0 (low order)”. By doing so, a transformation matrix nTbS-pt DCT2 can be derived from the base transformation matrix maxTbS-pt DCT2.

Further, for example, as shown in the first row from the bottom of the table shown in FIG. 12, when a base transformation matrix nTbS-pt DCT4 whose transformation type trType is DCT4 is derived from the prepared derivation source transformation matrix maxTbS-pt DCT2. , The sampling interval stepsize is “1 <<<< (Log2 (maxTbS) −Log2 (nTbS))”, the row offset offsetCol is “stepsize >> 1 (that is, one half of the stepsize)”, and the column offset offsetRow is “ 0 (low order) ". By doing so, a transformation matrix nTbS-pt DCT4 can be derived from the base transformation matrix maxTbS-pt DCT2.

<Details of each derivation method>
<For conversion type DCT2>
Next, a method for deriving a transformation matrix of each transformation type will be described more specifically. First, a method of deriving a base transform matrix nTbS-pt DCT2 with a transform type trType of DCT2 will be described. As shown in FIG. 13A, the size of the prepared derivation source transformation matrix maxTbS-pt DCT2 is set to 16 × 16.

In this case, by sampling the matrix elements of the gray portion of the derivation source transformation matrix maxTbS-pt2DCT2 shown in FIG. 13A, the conversion type DCT2 as shown in FIG. The matrix (8-pt DCT2) is obtained. Further, by sampling the matrix elements surrounded by the thick line frame of the derivation source transformation matrix maxTbS-pt2DCT2 shown in FIG. 13A, a transform type DCT2 of size 4 × 4 as shown in FIG. A transformation matrix (4-pt DCT2) is obtained.

Thus, in the case of the method of deriving the transform matrix of DCT2, the sampling interval stepsize is every two rows in the case of 8 × 8 (one row is sampled every two rows) and every four rows in the case of 4 × 4. (One row is sampled every four rows). That is, the sampling interval stepsize is a value raised to the power of the difference between the logarithmic value whose base is 2 of the maximum size maxTbS of the transform block and the logarithmic value whose base is 2 of the size nTbS of the transformation matrix to be derived. Note that the row offset offsetCol and the column offset offsetRow are both “0” in both cases.

That is, the transformation matrix nTbS-pt DCT2 can be derived by the derivation processing represented by the equation (X1) of D in FIG. The formula (X1) is also shown below.

transMatrix DCT2 _{, nTbS} [j] [i]
= transMatrix DCT2 _{, maxTbS} [j * stepsize + offsetCol] [i + offsetRow]
= transMatrix DCT2 _{, maxTbS} [j * stepsize] [i]
... (X1)
However,
stepsize = 1 << (log2 (maxTbS)-log2 (nTbS))
offsetCol = 0
offsetRow = 0

That is, the element of the j-th row and the ith column of the DCT2 transformation matrix of (nTbS) x (nTbS) is the element of the (j * stepsize) row and the ith column of the DCT2 transformation matrix of (maxTbS) x (maxTbS). . In other words, the DCT2 transformation matrix of (maxTbS) x (maxTbS) is calculated at a sampling interval stepsize = (1 << (log2 (maxTbS)-log2 (nTbS))), row offset offsetCol = 0, column offset offsetRow = 0. The submatrix obtained by sampling is a DCT2 transform matrix of (nTbS) × (nTbS).

により By performing the derivation process in this manner, a transformation matrix nTbS-pt DCT2 can be derived from the base transformation matrix maxTbS-pt DCT2.

<For conversion type DCT4>
Next, a method for deriving the base transform matrix nTbS-pt DCT4 whose transform type trType is DCT4 will be described. As shown in FIG. 14A, the size of the prepared derivation source transformation matrix maxTbS-pt DCT2 is set to 16 × 16.

In this case, by sampling the matrix elements of the gray portion of the derivation source transformation matrix maxTbS-pt2DCT2 shown in FIG. 14A, the conversion type DCT4 as shown in FIG. A matrix (8-pt DCT4) is obtained. Further, by sampling the matrix elements enclosed by the thick line frame of the derived transformation matrix maxTbS-pt DCT2 shown in FIG. 14A, the transformation type DCT4 as shown in FIG. A transformation matrix (4-pt DCT4) is obtained.

As described above, in the case of the method of deriving the transform matrix of DCT4, the sampling interval stepsize is every two rows when 8 × 8 (one row is sampled every two rows), and every four rows when 4 × 4. (One row is sampled every four rows). That is, the sampling interval stepsize is a value raised to the power of the difference between the logarithmic value whose base is 2 of the maximum size maxTbS of the transform block and the logarithmic value whose base is 2 of the size nTbS of the transformation matrix to be derived. The row offset offsetCol is “1” (ie, one row) in the case of 8 × 8, and “2” (ie, two rows) in the case of 4 × 4. That is, in the vertical direction in the figure, sampling is started from the second row (row of row number “1”) in the case of 8 × 8, and from the third row (row of row number “2”) in the case of 4 × 4. Sampling is started. That is, the row offset offsetCol is one half of the sampling interval stepsize. Note that the column offset offsetRow is “0” in both cases.

That is, the transformation matrix nTbS-pt DCT4 can be derived by the derivation processing represented by the equation (X2) of D in FIG. The formula (X2) is also shown below.

transMatrix _{DCT4, nTbS} [j] [i]
= transMatrix DCT2 _{, maxTbS} [j * stepsize + offsetCol] [i + offsetRow]
= transMatrix DCT2 _{, maxTbS} [j * stepsize + offsetCol] [i]
... (X2)
However,
stepsize = 1 << (log2 (maxTbS)-log2 (nTbS))
offsetCol = stepsize >> 1
offsetRow = 0

That is, the element of the j-th row and the ith column of the DCT4 transform matrix of (nTbS) x (nTbS) is the element of the (j * stepsize + offsetCol) row i-th column of the DCT2 transform matrix of (maxTbS) x (maxTbS). It is. In other words, the DCT2 transformation matrix of (maxTbS) x (maxTbS), sampling interval stepsize = (1 << (log2 (maxTbS)-log2 (nTbS))), row offset offsetCol = (stepsize >> 1), column The submatrix obtained by sampling at offset offsetRow = 0 is a DCT4 transform matrix of (nTbS) × (nTbS).

により By performing the derivation processing in this way, a transformation matrix nTbS-pt DCT4 can be derived from the base transformation matrix maxTbS-pt DCT2.

<Conversion device>
Next, a description will be given of the conversion apparatus 100 when the base conversion matrix is derived in this manner. FIG. 15 is a block diagram illustrating a main configuration example of the conversion device 100 in this case. As shown in FIG. 15, also in this case, the conversion device 100 has basically the same configuration as in the case of the first embodiment (FIG. 6). However, in this case, the matrix calculation unit 103 includes the base transformation matrix derivation unit 220.

The base conversion matrix deriving unit 220 performs processing related to derivation of the base conversion matrix. For example, the base conversion matrix deriving unit 220 converts a conversion matrix (base conversion matrix used for matrix calculation) specified by the base conversion matrix selection information supplied from the control unit 101 into a conversion matrix prepared in advance (derivation source conversion). Matrix). The matrix operation unit 103 performs a matrix operation on the coefficient data X ′, for example, as described in the above equation (19), using the base transformation matrix derived by the base transformation matrix deriving unit 220.

That is, the base transformation matrix deriving unit 220 derives a base transformation matrix based on the designated transformation type of the inverse one-dimensional transformation. The matrix calculation unit 103 performs a matrix calculation using the base conversion matrix derived by the base conversion matrix derivation unit 220.

{For example, the base transform matrix deriving unit 220 derives the base transform matrix by using a source transform matrix of a second transform type (for example, DCT2) having a size equal to or larger than the derived base transform matrix.

For example, the base transform matrix deriving unit 220 samples the second transform type (for example, DCT2) or the second transform type (for example, DCT2) by sampling the derived transform matrix of the second transform type (for example, DCT2) having a size equal to or larger than the derived base transform matrix. A base transform matrix of a fourth transform type (for example, DCT4) is derived.

ベース The base transformation matrix deriving unit 220 has an arbitrary configuration. For example, the base conversion matrix deriving unit 220 may be configured by a logic circuit that implements the above-described processing. Further, the base conversion matrix deriving unit 220 may include, for example, a CPU, a ROM, a RAM, and the like, and may execute the program using the CPU, the ROM, the RAM, and the like, thereby realizing the above-described processing. Of course, the base conversion matrix deriving unit 220 may have both of the configurations, and a part of the above-described processing may be realized by a logic circuit, and the other may be realized by executing a program.

<Base transformation matrix derivation unit>
FIG. 16 is a block diagram illustrating a main configuration example of the base transform matrix deriving unit 220 in FIG. As shown in FIG. 16, the base transformation matrix derivation unit 220 includes a sampling unit 231 and a derivation source transformation matrix LUT232.

The sampling unit 231 performs a process related to sampling. For example, the sampling unit 231 sets a sampling parameter in accordance with the base transformation matrix selection information, and converts the conversion type trType size (nTbS) × (nTbS) from the derivation source transformation matrix maxTbS-pt DCT2 by a method according to the sampling parameter. And derives a base conversion matrix T _base of The sampling unit 231 has a sampling parameter derivation unit 241 and a partial matrix extraction unit 242.

The sampling parameter deriving unit 241 performs a process related to deriving a sampling parameter. For example, the sampling parameter deriving unit 241 acquires base transformation matrix selection information. The base transformation matrix selection information is information for specifying a base transformation matrix used for matrix operation. That is, the conversion type trType, the maximum size maxTbS of the conversion block, the size nTbS of the derivation target base conversion matrix, and the like are specified by the base conversion matrix selection information. The sampling parameter deriving unit 241 sets sampling parameters such as a sampling interval stepsize, a row offset offsetCol, and a column offset offsetRow based on such information specified by such base transformation matrix selection information. For example, the sampling parameter deriving unit 241 sets the sampling parameters as described with reference to the table in FIG. The sampling parameter derivation unit 241 supplies the derived sampling parameters to the partial matrix extraction unit 242.

The sub-matrix extraction unit 242 performs a process related to the extraction of the sub-matrix. For example, the sub-matrix extraction unit 242 acquires the sampling parameters derived by the sampling parameter derivation unit 241. In addition, the sub-matrix extraction unit 242 acquires the source transformation matrix (maxTbS-pt DCT2) 251 registered in the source transformation matrix LUT232. Then, the sub-matrix extraction unit 242 samples the derived transformation matrix (maxTbS-pt DCT2) 251 by a method according to the sampling parameter. By this sampling, the sub-matrix extraction unit 242 obtains a sub-matrix of the conversion type trType and size (nTbS) × (nTbS) specified by the base conversion matrix selection information. The sub-matrix extraction unit 242 supplies the sub-matrix as the base conversion matrix T _base to the matrix calculation unit 103.

The derivation source transformation matrix LUT 232 registers (stores) a derivation source transformation matrix (maxTbS-pt２５DCT2) 251 having a conversion type DCT2 and a size (maxTbS) × (maxTbS). The source transform matrix LUT 232 supplies the source transform matrix (maxTbS-ptmaxDCT2) 251 to the sub-matrix extractor 242 in response to a request from the sub-matrix extractor 242.

The sampling unit 231 has an optional configuration. For example, the sampling unit 231 may be configured by a logic circuit that implements the above processing. Further, the sampling unit 231 may include, for example, a CPU, a ROM, a RAM, and the like, and may execute the program using the CPU, the ROM, the RAM, and the like to realize the above-described processing. Needless to say, the sampling unit 231 may have both configurations, and a part of the above processing may be realized by a logic circuit, and the other may be realized by executing a program.

The source transformation matrix LUT 232 has a storage area formed by a RAM or the like, and stores the source transformation matrix (maxTbS-pt DCT2) 251 therein.

The sampling parameter deriving unit 241 has an arbitrary configuration. For example, the sampling parameter deriving unit 241 may be configured by a logic circuit that implements the above-described processing. In addition, the sampling parameter deriving unit 241 may include, for example, a CPU, a ROM, a RAM, and the like, and may execute the program using the CPU, the ROM, the RAM, and the like, thereby realizing the above-described processing. Of course, the sampling parameter deriving unit 241 may have both of the configurations, and a part of the above-described processing may be realized by a logic circuit, and the other may be realized by executing a program.

The sub-matrix extraction unit 242 has an arbitrary configuration. For example, the sub-matrix extraction unit 242 may be configured by a logic circuit that implements the above-described processing. In addition, the sub-matrix extracting unit 242 may include, for example, a CPU, a ROM, a RAM, and the like, and execute the program using the CPU, the ROM, the RAM, and the like, thereby realizing the above-described processing. Of course, the sub-matrix extraction unit 242 may have both of the configurations, and a part of the above-described processing may be realized by a logic circuit, and the other may be realized by executing a program.

With such a configuration, the base transformation matrix deriving unit 220 can derive the base transformation matrix specified by the base transformation matrix selection information. In other words, since one derived source transformation matrix (maxTbS-pt DCT2) 251 may be stored in the derived transformation matrix LUT 232, it is possible to suppress an increase in the size of the LUT (reduce the size of the LUT). Further, the matrix operation of the (maxTbS) × (maxTbS) conversion matrix and the matrix operation of the (nTbS) × (nTbS) conversion matrix of the conversion type trType can be shared. That is, the matrix calculation unit 103 can perform a matrix calculation using the base conversion matrix of each conversion type trType using the same calculation circuit. Therefore, it is possible to suppress an increase in the circuit scale (reduce the circuit scale).

<Conversion process flow>
Next, an example of the flow of the conversion process in this case will be described with reference to the flowchart in FIG.

When the conversion process is started, the control unit 101 (the sign inversion flag setting unit 111, the base conversion matrix selection unit 112, and the flip flag setting unit 113) determines in step S201 that the conversion type supplied from outside the conversion device 100 Based on trTypeIdx and size (log2TBWidth, log2TBHeight), base conversion matrix selection information, sign inversion flag (signChangeFlag), and flip flag (flipFlag) are set as described above.

{At step S202, the sign inversion unit 102 determines whether or not to perform a sign inversion operation based on the sign inversion flag set at step S201 (signChangeFlag == True?). If the value of the sign inversion flag is true and it is determined that the sign inversion operation is to be performed, the process proceeds to step S203.

In step S203, the sign inversion unit 102 performs a sign inversion operation (S) on the input coefficient data Xin, which is a one-dimensional signal sequence, as in Equation (18) described above, for example, Generate data X '.

すると When the process in step S203 is completed, the process proceeds to step S204. Further, in step S202, when it is determined that the value of the sign inversion flag is false (False) and the sign inversion operation is not performed, the process of step S203 is skipped, and the input coefficient data Xin is directly used as the coefficient data X ′. Then, the process proceeds to step S204.

In step S204, the base transformation matrix deriving unit 220 performs a base transformation matrix deriving process, and derives a _base transformation matrix T _base based on the base transformation matrix selection information set in step S201.

In step S205, the matrix operation unit 103 uses the base transformation matrix T _base derived in step S204 to perform a matrix operation (1) on the coefficient data X ′ that is a one-dimensional signal sequence, for example, as in Expression (19) described above. Dimensional conversion) to generate coefficient data X ″ that is a one-dimensional signal sequence.

In step S206, the flip unit 104 determines whether to perform a flip operation based on the flip flag set in step S201 (FlipFlagF == True?). If the value of the flip flag is true and it is determined that a flip operation is to be performed, the process proceeds to step S207.

In step S207, the flip unit 104 performs a flip operation (F) on the coefficient data X ″ that is the one-dimensional signal sequence obtained in step S205, for example, as in Expression (20) described above, and performs one-dimensional Generate output coefficient data Xout as a signal sequence.

The flip unit 104 outputs the generated output coefficient data Xout to the outside of the conversion device 100. When the processing in step S207 ends, the conversion processing ends. If it is determined in step S206 that the value of the flip flag is false (False) and the flip operation is not performed, the process of step S207 is skipped, and the coefficient data X ″ is directly used as the output coefficient data Xout. Are output to the outside of the conversion device 100. When the output coefficient data Xout is output, the conversion process ends.

<Flow of base transformation matrix derivation process>
Next, an example of the flow of the base transformation matrix derivation process executed in step S204 of FIG. 17 will be described with reference to the flowchart of FIG.

When the base conversion matrix derivation process is started, the sampling parameter derivation unit 241 of the base conversion matrix derivation unit 220 corresponds to the conversion type trType and the block size (nTbS) specified by the base conversion matrix selection information in step S221. The sampling parameters to be derived are derived.

において In step S222, the sub-matrix extraction unit 242 reads the derived transformation matrix (maxTbS-pt DCT2) 251 from the derived transformation matrix LUT232.

In step S223, the sub-matrix extracting unit 242 extracts a sub-matrix from the derived transformation matrix (maxTbS-pt @ DCT2) 251 read in step S222 using the sampling parameters derived in step S221.

In step S224, the partial matrix extraction unit 242 supplies the partial matrix extracted in step S223 to the matrix calculation unit 103 as a base transformation matrix.

すると When the process of step S224 ends, the base transformation matrix derivation process ends, and the process returns to FIG.

各 By executing each process as described above, the base transformation matrix specified by the base transformation matrix selection information can be derived. In other words, since one derived source transformation matrix (maxTbS-pt DCT2) 251 may be stored in the derived transformation matrix LUT 232, it is possible to suppress an increase in the size of the LUT (reduce the size of the LUT). Further, the matrix operation of the (maxTbS) × (maxTbS) conversion matrix and the matrix operation of the (nTbS) × (nTbS) conversion matrix of the conversion type trType can be shared. That is, the matrix calculation unit 103 can perform a matrix calculation using the base conversion matrix of each conversion type trType using the same calculation circuit. Therefore, it is possible to suppress an increase in the circuit scale (reduce the circuit scale).

<7. Fourth Embodiment>
<Inverse conversion device>
The derivation of the base transform matrix described in the third embodiment can be similarly applied to the inverse one-dimensional transform described in the second embodiment.

FIG. 19 is a block diagram showing a main configuration example of the inverse conversion device 150 in this case. As shown in FIG. 15, also in this case, the inverse conversion device 150 has basically the same configuration as in the case of the second embodiment (FIG. 9). However, in this case, the matrix operation unit 153 includes the base transformation matrix derivation unit 270.

ベース The base transformation matrix deriving unit 270 has the same configuration as the base transformation matrix deriving unit 220 described in the third embodiment, and performs the same processing. Therefore, the configuration example of the base transform matrix deriving unit 220 described with reference to FIG. 16 can be applied to the description of the base transform matrix deriving unit 270.

That is, the base conversion matrix deriving unit 270 derives a base conversion matrix based on the specified inverse one-dimensional conversion conversion type. The matrix calculation unit 153 performs a matrix calculation using the base conversion matrix derived by the base conversion matrix derivation unit 270.

{For example, the base transformation matrix deriving unit 270 derives the base transformation matrix by using a derived transformation matrix of a second transformation type (for example, DCT2) having a size equal to or larger than the derived base transformation matrix.

For example, the base transform matrix deriving unit 270 samples the second transform type (for example, DCT2) and the second transform type (for example, DCT2) by sampling the derived transform matrix of the second transform type (for example, DCT2) having a size equal to or larger than the derived base transform matrix. A base transform matrix of a fourth transform type (for example, DCT4) is derived.

With such a configuration, the base transformation matrix deriving unit 270 can derive the base transformation matrix specified by the base transformation matrix selection information. In other words, since one derived source transformation matrix (maxTbS-pt DCT2) 251 may be stored in the derived transformation matrix LUT 232, it is possible to suppress an increase in the size of the LUT (reduce the size of the LUT). Further, the matrix operation of the (maxTbS) × (maxTbS) conversion matrix and the matrix operation of the (nTbS) × (nTbS) conversion matrix of the conversion type trType can be shared. That is, the matrix calculation unit 153 can perform a matrix calculation using the base conversion matrix of each conversion type trType using the same calculation circuit. Therefore, it is possible to suppress an increase in the circuit scale (reduce the circuit scale).

<Flow of inverse transformation process>
Next, an example of the flow of the inverse conversion process in this case will be described with reference to the flowchart in FIG.

When the inverse conversion process is started, the control unit 151 (the flip flag setting unit 161, the base conversion matrix selecting unit 162, and the sign inversion flag setting unit 163) in step S251 performs the conversion type trTypeIdx and the size (log2TBWidth, log2TBHeight). ), The base conversion matrix selection information, the flip flag (flipFlag), and the sign inversion flag (signChangeFlag) are set as described above.

In step S252, the flip unit 152 determines whether to perform a flip operation (F) based on the flip flag set in step S251 (FlipFlagF == True?). If the value of the flip flag is true and it is determined that the flip operation (F) is to be performed, the process proceeds to step S253.

In step S253, the flip unit 152 performs a flip operation (F) on the input coefficient data Xin, which is a one-dimensional signal sequence, for example, as in the above-described equation (21), to execute the coefficient data X, which is a one-dimensional signal sequence. 'Is generated.

When the process of step S253 is completed, the process proceeds to step S254. If it is determined in step S252 that the value of the flip flag is false (False) and the flip operation (F) is not to be performed, the process of step S253 is skipped, and the input coefficient data Xin is directly used as the coefficient data X ′. And the process proceeds to step S254.

In step S254, the base transformation matrix derivation unit 270 performs a base transformation matrix derivation process, and derives a _base transformation matrix T _base based on the base transformation matrix selection information set in step S251. This base transformation matrix derivation process is executed in the same flow as in the case of the flowchart in FIG. Therefore, the description is omitted.

In step S255, the matrix operation unit 153 determines the base transformation matrix T _base derived in step S254, that is, the base specified by the base transformation matrix selection information set in step S251, as in the above-described equation (22), for example. Using the transposed matrix of the transformation matrix T _base , a matrix operation (inverse one-dimensional conversion) is performed on the coefficient data X ′ that is a one-dimensional signal sequence to generate coefficient data X ″ that is a one-dimensional signal sequence.

{At step S256, the sign inversion unit 154 determines whether or not to perform the sign inversion operation (S) based on the sign inversion flag set at step S251 (signChangeFlag == True?). If the value of the sign inversion flag is true and it is determined that the sign inversion operation (S) is to be performed, the process proceeds to step S257.

In step S257, the sign inversion unit 154 performs a sign inversion operation (S) on the coefficient data X ″ that is the one-dimensional signal sequence obtained in step S255, for example, as in Expression (23) described above. The output coefficient data Xout which is a one-dimensional signal sequence is generated. The sign inverting unit 154 outputs the generated output coefficient data Xout to the outside of the inverse transform device 150. When the processing in step S257 ends, the conversion processing ends.

If it is determined in step S256 that the value of the sign inversion flag is false (False) and the sign inversion operation (S) is not performed, the process in step S257 is skipped, and the coefficient data X ″ is output as it is. The data is set as coefficient data Xout and output to the outside of the inverse transform device 150. When the output coefficient data Xout is output, the inverse conversion processing ends.

各 By executing each process as described above, the base transformation matrix specified by the base transformation matrix selection information can be derived. In other words, since one derived source transformation matrix (maxTbS-pt DCT2) 251 may be stored in the derived transformation matrix LUT 232, it is possible to suppress an increase in the size of the LUT (reduce the size of the LUT). Further, the matrix operation of the (maxTbS) × (maxTbS) conversion matrix and the matrix operation of the (nTbS) × (nTbS) conversion matrix of the conversion type trType can be shared. That is, the matrix calculation unit 153 can perform a matrix calculation using the base conversion matrix of each conversion type trType using the same calculation circuit. Therefore, it is possible to suppress an increase in the circuit scale (reduce the circuit scale).

<8. Fifth Embodiment>
<Application example>
In the above, an example has been described in which the (inverse) one-dimensional conversion of the conversion types DST2 and DST4 is realized by the FTS operation or the STF operation including the one-dimensional conversion of the conversion types DCT2 and DCT4. The present technology can be applied to other examples.

For example, the (reverse) one-dimensional conversion of the conversion types DCT2 and DCT4 may be realized by an FTS operation or STF operation including the (reverse) one-dimensional conversion of the conversion types DST2 and DST4.

Transformation matrix T _DCT2 conversion type DCT2 used for one-dimensional transform is a transformation matrix T _DST2 conversion type DST2, flip matrix F, and the sign inversion matrix with S, be expressed as the following equation (24) it can.

Similarly, the transformation matrix T _DCT4 of the transformation type DCT4 used for the one-dimensional transformation is represented by the following equation (25) using the transformation matrix T _DST4 , the flip matrix F, and the sign inversion matrix S of the transformation type DST4. Can be represented.

Therefore, one-dimensional conversion of conversion types DCT2 and DCT4 can be realized by STF operations including one-dimensional conversion of conversion types DST2 and DST4. For example, in the conversion apparatus 100 (FIG. 6) described in the first embodiment, the sign inversion section 102 and the flip section 104 are exchanged, and the base conversion matrix LUT 120 is converted to the conversion matrix of the conversion type DST2 and the conversion matrix of the conversion type DST4. May be stored, and the matrix operation unit 103 may perform the matrix operation using the transformation matrices as the base transformation matrix. Further, for example, in the transform apparatus 100 (FIG. 15) described in the third embodiment, the sign inverting section 102 and the flip section 104 are exchanged, and the base transform matrix deriving section 220 performs the transform type DST2 base transform matrix or transform. What is necessary is just to derive a base transformation matrix of type DST4, and to make the matrix operation unit 103 perform a matrix operation using the derived base transformation matrix.

In this manner, when the one-dimensional conversion of the transform type DCT4 and the one-dimensional transform of the transform type DCT2 are selectively performed, the pre-process performed before the orthogonal transform process is performed by (the sign inversion operation (S) ) It is possible to unify the post-processing performed after the orthogonal transformation processing (to the flip operation (F)).

Furthermore, the transposed matrix T _DCT2 ^t of the transformation matrix of the conversion type DCT2 used in inverse one-dimensional transform is the transposed matrix T _DST2 ^t of the transformation matrix of the conversion type DST2, using flip matrix F, and sign inversion matrix S, the following Equation (26) can be expressed.

Similarly, the transformation matrix T _DCT4 of the transformation type DCT4 used for the inverse one-dimensional transformation is represented by the following equation (using the transposed matrix T _DST4 ^t , the flip matrix F, and the sign inversion matrix S of the transformation matrix of the transformation type DST4). 27).

Therefore, the inverse one-dimensional transform of the transform types DCT2 and DCT4 can be realized by the FTS operation including the inverse one-dimensional transform of the transform types DST2 and DST4. For example, in the inverse conversion device 150 (FIG. 9) described in the second embodiment, the flip unit 152 and the sign inversion unit 154 are exchanged, and the base conversion matrix LUT 170 is converted between the conversion matrix of the conversion type DST2 and the conversion matrix of the conversion type DST4. The matrix may be stored, and the matrix operation unit 153 may use the transformation matrices as base transformation matrices and perform the matrix computation using the transposed matrix of the base transformation matrix. Also, for example, in the conversion apparatus 100 (FIG. 19) described in the fourth embodiment, the flip unit 152 and the sign inversion unit 154 are exchanged, and the base conversion matrix derivation unit 270 converts the base conversion matrix of the conversion type DST2 or the conversion. What is necessary is just to derive a base transformation matrix of type DST4, and to make the matrix operation unit 103 perform a matrix operation using the derived base transformation matrix.

In this manner, when the inverse one-dimensional transform of the transform type DCT4 and the inverse one-dimensional transform of the transform type DCT2 are selectively performed, the pre-process performed before the inverse orthogonal transform process is performed by the (flip operation (F )), And post-processing performed after the inverse orthogonal transformation processing (to the sign inversion operation (S)).

By doing so, it is possible to omit the selection of the processing content in the pre-processing and post-processing (whether to perform the sign inversion operation (S) or the flip operation (F)), so that one-dimensional conversion or reverse one It is possible to suppress the complexity of the configuration of the dimensional conversion (simplify the configuration), and it is possible to easily perform the one-dimensional conversion or the inverse one-dimensional conversion. That is, also in this case, similarly to the first to fourth embodiments, it is possible to suppress an increase in circuit scale and processing load, and to suppress an increase in mounting cost.

<9. Sixth embodiment>
<Image coding device>
The present technology described above can be applied to any device, device, system, and the like. For example, the present technology described above can be applied to an image encoding device that encodes image data.

FIG. 21 is a block diagram illustrating an example of a configuration of an image encoding device that is an aspect of an image processing device to which the present technology is applied. An image encoding device 300 illustrated in FIG. 21 is an device that encodes image data of a moving image. For example, the image encoding device 300 implements the technology described in Non-Patent Document 1, Non-Patent Document 5, or Non-Patent Document 6, and employs a method based on a standard described in any of those documents. The image data of the moving image is encoded.

FIG. 21 shows main components such as the processing unit and the flow of data, and the components shown in FIG. 21 are not necessarily all. That is, in the image encoding device 300, a processing unit not illustrated as a block in FIG. 21 may exist, or a process or data flow not illustrated as an arrow or the like in FIG. 21 may exist. This is the same in other drawings for explaining the processing unit and the like in the image encoding device 300.

As illustrated in FIG. 21, the image encoding device 300 includes a control unit 301, a rearrangement buffer 311, an arithmetic unit 312, an orthogonal transformation unit 313, a quantization unit 314, an encoding unit 315, a storage buffer 316, and an inverse quantization unit. 317, an inverse orthogonal transform unit 318, an operation unit 319, an in-loop filter unit 320, a frame memory 321, a prediction unit 322, and a rate control unit 323.

<Control unit>
The control unit 301 divides the moving image data held by the rearrangement buffer 311 into processing unit blocks (CU, PU, conversion block, etc.) based on an external or pre-designated processing unit block size. . In addition, the control unit 301 determines coding parameters (header information Hinfo, prediction mode information Pinfo, conversion information Tinfo, filter information Finfo, and the like) to be supplied to each block based on, for example, RDO (Rate-Distortion Optimization). I do.

詳細 The details of these encoding parameters will be described later. After determining the above-described encoding parameters, the control unit 301 supplies the parameters to each block. Specifically, it is as follows.

(4) The header information Hinfo is supplied to each block. The prediction mode information Pinfo is supplied to the encoding unit 315 and the prediction unit 322. The transform information Tinfo is supplied to an encoding unit 315, an orthogonal transformation unit 313, a quantization unit 314, an inverse quantization unit 317, and an inverse orthogonal transformation unit 318. The filter information Finfo is supplied to the in-loop filter unit 320.

<Sort buffer>
Each field (input image) of the moving image data is input to the image encoding device 300 in the order of reproduction (display order). The reordering buffer 311 acquires and holds (stores) each input image in its reproduction order (display order). The rearrangement buffer 311 rearranges the input image in an encoding order (decoding order) or divides the input image into blocks in processing units based on the control of the control unit 301. The rearrangement buffer 311 supplies the processed input images to the calculation unit 312. The reordering buffer 311 also supplies the input images (original images) to the prediction unit 322 and the in-loop filter unit 320.

<Operation part>
The calculation unit 312 receives the image I corresponding to the block of the processing unit and the prediction image P supplied from the prediction unit 322, and subtracts the prediction image P from the image I as shown in the following equation (28). Then, the prediction residual D is derived and supplied to the orthogonal transform unit 313.

<Orthogonal transformer>
The orthogonal transform unit 313 receives the prediction residual D supplied from the calculation unit 312 and the conversion information Tinfo supplied from the control unit 301 as inputs, and performs orthogonal transform on the prediction residual D based on the conversion information Tinfo. Conversion is performed to derive a conversion coefficient Coeff. The orthogonal transform unit 313 supplies the obtained transform coefficient Coeff to the quantization unit 314.

<Quantizer>
The quantization unit 314 receives the transform coefficient Coeff supplied from the orthogonal transform unit 313 and the transform information Tinfo supplied from the control unit 301, and scales the transform coefficient Coeff based on the transform information Tinfo (quantization). ). The rate of this quantization is controlled by the rate control unit 323. The quantization unit 314 supplies the quantized transform coefficient obtained by such quantization, that is, the quantized transform coefficient level level, to the encoding unit 315 and the inverse quantization unit 317.

<Encoding unit>
The encoding unit 315 includes a quantization transform coefficient level supplied from the quantization unit 314 and various encoding parameters (header information Hinfo, prediction mode information Pinfo, conversion information Tinfo, and filter information Finfo supplied from the control unit 301). ), Information on filters such as filter coefficients supplied from the in-loop filter unit 320, and information on the optimal prediction mode supplied from the prediction unit 322. The encoding unit 315 performs variable-length encoding (for example, arithmetic encoding) on the quantized transform coefficient level level to generate a bit string (encoded data).

{Encoding section 315 also derives residual information Rinfo from the quantized transform coefficient level level, encodes residual information Rinfo, and generates a bit string.

{Furthermore, the encoding unit 315 includes information about the filter supplied from the in-loop filter unit 320 in the filter information Finfo, and includes information about the optimal prediction mode supplied from the prediction unit 322 in the prediction mode information Pinfo. Then, the coding unit 315 codes the above-described various coding parameters (header information Hinfo, prediction mode information Pinfo, conversion information Tinfo, filter information Finfo, and the like), and generates a bit sequence.

{Encoding section 315 also multiplexes the bit strings of the various information generated as described above to generate encoded data. The encoding unit 315 supplies the encoded data to the storage buffer 316.

<Accumulation buffer>
The accumulation buffer 316 temporarily stores the encoded data obtained by the encoding unit 315. At a predetermined timing, the accumulation buffer 316 outputs the held encoded data to the outside of the image encoding device 300 as, for example, a bit stream. For example, the encoded data is transmitted to the decoding side via an arbitrary recording medium, an arbitrary transmission medium, an arbitrary information processing device, or the like. That is, the accumulation buffer 316 is also a transmission unit that transmits encoded data (bit stream).

<Inverse quantization unit>
The inverse quantization unit 317 performs a process related to inverse quantization. For example, the inverse quantization unit 317 receives as input the quantized transform coefficient level supplied from the quantization unit 314 and the transform information Tinfo supplied from the control unit 301, and performs quantization based on the transform information Tinfo. Scale (inverse quantization) the value of the transform coefficient level level. Note that the inverse quantization is an inverse process of the quantization performed in the quantization unit 314. The inverse quantization unit 317 supplies the transform coefficient Coeff_IQ obtained by such inverse quantization to the inverse orthogonal transform unit 318.

<Inverse orthogonal transform unit>
The inverse orthogonal transform unit 318 performs a process related to the inverse orthogonal transform. For example, the inverse orthogonal transform unit 318 receives as input the transform coefficient Coeff_IQ supplied from the inverse quantization unit 317 and the transform information Tinfo supplied from the control unit 101, and converts the transform coefficient Coeff_IQ based on the transform information Tinfo. An inverse orthogonal transform is performed on the result to derive a prediction residual D ′. Note that the inverse orthogonal transform is an inverse process of the orthogonal transform performed in the orthogonal transform unit 313. The inverse orthogonal transform unit 318 supplies the prediction residual D ′ obtained by such an inverse orthogonal transform to the calculation unit 319. Since the inverse orthogonal transform unit 318 is similar to the inverse orthogonal transform unit (described later) on the decoding side, the description (described later) performed on the decoding side can be applied to the inverse orthogonal transform unit 318.

<Operation part>
The calculation unit 319 receives as input the prediction residual D ′ supplied from the inverse orthogonal transform unit 318 and the prediction image P supplied from the prediction unit 322. The calculation unit 319 adds the prediction residual D ′ and the prediction image P corresponding to the prediction residual D ′ to derive a local decoded image Rlocal. The operation unit 319 supplies the derived local decoded image Rlocal to the in-loop filter unit 320 and the frame memory 321.

<In-loop filter section>
The in-loop filter unit 320 performs a process related to the in-loop filter process. For example, the in-loop filter unit 320 converts the local decoded image Rlocal supplied from the arithmetic unit 319, the filter information Finfo supplied from the control unit 301, and the input image (original image) supplied from the rearrangement buffer 311. Take as input. Note that information input to the in-loop filter unit 320 is arbitrary, and information other than these information may be input. For example, if necessary, the prediction mode, motion information, code amount target value, quantization parameter QP, picture type, block (CU, CTU, etc.) information and the like may be input to the in-loop filter unit 320. Good.

The in-loop filter unit 320 appropriately performs a filtering process on the locally decoded image Rlocal based on the filter information Finfo. The in-loop filter unit 320 also uses an input image (original image) and other input information for the filtering process as needed.

For example, as described in Non-Patent Document 1, the in-loop filter unit 320 includes a bilateral filter, a deblocking filter (DBF (DeBlocking Filter)), an adaptive offset filter (SAO (Sample Adaptive Offset)), and an adaptive loop filter. Four in-loop filters (ALF (Adaptive Loop Loop Filter)) are applied in this order. It is to be noted that which filter is applied and in which order are applied are arbitrary and can be selected as appropriate.

Of course, the filter processing performed by the in-loop filter unit 320 is arbitrary, and is not limited to the above example. For example, the in-loop filter unit 320 may apply a Wiener filter or the like.

The in-loop filter unit 320 supplies the filtered local decoded image Rlocal to the frame memory 321. Note that when information about a filter such as a filter coefficient is transmitted to the decoding side, the in-loop filter unit 320 supplies information about the filter to the encoding unit 315.

<Frame memory>
The frame memory 321 performs processing relating to storage of data relating to an image. For example, the frame memory 321 receives the local decoded image Rlocal supplied from the arithmetic unit 319 and the filtered local decoded image Rlocal supplied from the in-loop filter unit 320, and stores (stores) them. Further, the frame memory 321 reconstructs and holds the decoded image R for each picture using the local decoded image Rlocal (stores the decoded image R in a buffer in the frame memory 321). The frame memory 321 supplies the decoded image R (or a part thereof) to the prediction unit 322 in response to a request from the prediction unit 322.

<Prediction unit>
The prediction unit 322 performs a process related to generation of a predicted image. For example, the prediction unit 322 includes the prediction mode information Pinfo supplied from the control unit 301, the input image (original image) supplied from the rearrangement buffer 311 and the decoded image R (or a part thereof) read from the frame memory 321. Is input. The prediction unit 322 performs prediction processing such as inter prediction or intra prediction using the prediction mode information Pinfo and the input image (original image), performs prediction with reference to the decoded image R as a reference image, and performs prediction based on the prediction result. To perform a motion compensation process to generate a predicted image P. The prediction unit 322 supplies the generated prediction image P to the calculation unit 312 and the calculation unit 319. Further, the prediction unit 322 supplies the prediction mode selected by the above processing, that is, information on the optimal prediction mode to the encoding unit 315 as necessary.

<Rate control section>
The rate control unit 323 performs processing related to rate control. For example, the rate control unit 323 controls the rate of the quantization operation of the quantization unit 314 based on the code amount of the coded data stored in the storage buffer 316 so that overflow or underflow does not occur.

<Details of orthogonal transform unit>
FIG. 22 is a block diagram illustrating a main configuration example of the orthogonal transform unit 313. As shown in FIG. 22, the orthogonal transform unit 313 includes a switch 351, a primary transform unit 352, and a secondary transform unit 353.

The switch 351 receives the prediction residual D and the conversion skip flag ts_flag [compID] corresponding to the component identifier compID, and when the value of the conversion skip flag ts_flag [compID] is NO_TS (= 0) (when the conversion skip is not applied). ), And supplies the prediction residual D to the primary conversion unit 352. When the value of the conversion skip flag ts_flag [compID] is 2D_TS (= 1) (indicating that the two-dimensional conversion skip is to be applied), the primary conversion unit 352 and the secondary conversion unit 353 are skipped, and the prediction residual D Is output to the outside of the orthogonal transform unit 313 as a transform coefficient Coeff (supplied to the quantization unit 314).

The primary conversion unit 352 performs a process related to primary conversion, which is a predetermined conversion process such as orthogonal conversion. For example, the primary conversion unit 352 converts the component identifier compID, the adaptive primary conversion flag apt_flag [compID] of the component identifier compID, the primary conversion identifier pt_idx [compID] of the component identifier compID, the prediction mode information PInfo, the size of the conversion block (the pair of the horizontal width). The numerical value log2TBWSize, the logarithmic value of the vertical width log2TBHSize) and the prediction residual D are input. Note that the horizontal width TBWSize of the conversion block is also referred to as TBWidth, and the logarithmic value thereof is also referred to as log2TBWidth. Similarly, the vertical width TBHSize of the conversion block is also referred to as TBHeight, and the logarithmic value thereof is also referred to as log2TBHeight.

The primary conversion unit 352 refers to the prediction mode information PInfo, the component identifier compID, the adaptive primary conversion flag apt_flag [compID] of the component identifier compID, and the primary conversion identifier pt_idx [compID] of the component identifier compID to generate a component identifier compID. The corresponding primary horizontal conversion conversion type TrTypeH (and the primary horizontal conversion type identifier TrTypeIdxH indicating the conversion type) and the primary vertical conversion conversion type TrTypeV (and the primary vertical conversion type identifier TrTypeIdxV indicating the conversion type) are selected.

Further, the primary conversion unit 352 converts the prediction residual D into a primary horizontal conversion type identifier TrTypeIdxH (or a primary horizontal conversion type TrTypeH) and a horizontal width log2TBWSize of the conversion block, and a primary vertical conversion type identifier. Perform primary vertical conversion determined by TrTypeIdxV (or primary vertical conversion type TrTypeV) and vertical width log2TBHSize of the conversion block, and derive a conversion coefficient Coeff_P after the primary conversion. The primary horizontal transform is a one-dimensional orthogonal transform in the horizontal direction, and the primary vertical transform is a one-dimensional orthogonal transform in the vertical direction.

The primary conversion unit 352 supplies the derived conversion coefficient Coeff_P to the secondary conversion unit 353.

The secondary conversion unit 353 performs a process related to a secondary conversion, which is a predetermined conversion process such as an orthogonal conversion. For example, the secondary conversion unit 353 receives a secondary conversion identifier st_idx, a scan identifier scanIdx indicating a method of scanning a conversion coefficient, and a conversion coefficient Coeff_P. The secondary conversion unit 353 performs secondary conversion on the conversion coefficient Coeff_P based on the secondary conversion identifier st_idx and the scan identifier scanIdx, and derives a conversion coefficient Coeff_S after the secondary conversion.

More specifically, when the secondary conversion identifier st_idx indicates that the secondary conversion is applied (st_idx> 0), the secondary conversion unit 353 converts the conversion coefficient Coeff_P of the secondary conversion corresponding to the secondary conversion identifier st_idx. The processing is executed to derive a conversion coefficient Coeff_S after the secondary conversion.

The secondary transform unit 353 outputs the secondary transform coefficient Coeff_S to the outside of the orthogonal transform unit 313 as a transform coefficient Coeff (supplies it to the quantization unit 314).

If the secondary conversion identifier st_idx indicates that the secondary conversion is not to be applied (st_idx == 0), the secondary conversion unit 353 skips the secondary conversion and converts the conversion coefficient Coeff_P after the primary conversion into the conversion coefficient Coeff (secondary conversion coefficient). It is output to the outside of the orthogonal transform unit 313 as a subsequent transform coefficient Coeff_S (supplied to the quantization unit 314).

<Primary converter>
FIG. 23 is a block diagram illustrating a main configuration example of the primary conversion unit 352 in FIG. As shown in FIG. 23, the primary conversion unit 352 includes a primary conversion selection unit 361, a primary horizontal conversion unit 362, and a primary vertical conversion unit 363.

The primary conversion selection unit 361 receives as input the prediction mode information PInfo, the component identifier compID, the adaptive primary conversion flag apt_flag [compID], and the primary conversion identifier pt_idx [compID]. The primary conversion selection unit 361 derives a conversion type identifier TrTypeIdxH for primary horizontal conversion and a conversion type identifier TrTypeIdxV for primary vertical conversion with reference to the information. The primary conversion selection unit 361 supplies the derived conversion type identifier TrTypeIdxH of the primary horizontal conversion to the primary horizontal conversion unit 362. In addition, the primary conversion selection unit 361 supplies the derived conversion type identifier TrTypeIdxV of the primary vertical conversion to the primary vertical conversion unit 363.

The primary horizontal conversion unit 362 receives as input the prediction residual D, the conversion type identifier TrTypeIdxH of the primary horizontal conversion, and information (not shown) on the size of the conversion block. The information on the size of the transform block may be a natural number N indicating the horizontal or vertical size (the number of coefficients) of the transform block, or log2TBWSize (the logarithmic value of the transverse width) indicating the lateral width of the transform block. (N あっ = 1 << log2TBWSize). The primary horizontal transform unit 362 performs a primary horizontal transform Phor determined on the prediction residual D by the transform type identifier TrTypeIdxH and the size of the transform block, and derives a transform coefficient Coeff_Phor after the primary horizontal transform. The primary horizontal conversion unit 362 supplies the conversion coefficient Coeff_Phor after the primary horizontal conversion to the primary vertical conversion unit 363.

The primary vertical conversion unit 363 receives as input the conversion coefficient Coeff_Phor after the primary horizontal conversion, the conversion type identifier TrTypeIdxV of the primary vertical conversion, and information (not shown) on the size of the conversion block. The information on the size of the transform block may be a natural number N indicating the horizontal or vertical size (the number of coefficients) of the transform block, or log2TBHSize (the logarithmic value of the vertical width) indicating the vertical width of the transform block. ) (N = 1 << log2TBHSize). The primary vertical conversion unit 363 performs a primary vertical conversion Pver determined by the conversion type identifier TrTypeIdxV and the size of the conversion block on the conversion coefficient Coeff_Phor after the primary horizontal conversion, and derives the conversion coefficient Coeff_Pver after the primary vertical conversion. . The primary vertical conversion unit 363 outputs the conversion coefficient Coeff_Pver after the primary vertical conversion to the outside of the primary conversion unit 352 as the conversion coefficient Coeff_P after the primary conversion (supplies it to the secondary conversion unit 353).

<Primary horizontal conversion unit>
FIG. 24 is a block diagram illustrating a main configuration example of the primary horizontal conversion unit 362 in FIG. As shown in FIG. 24, the primary horizontal conversion unit 362 includes a signal sequence extraction unit 371, a one-dimensional conversion unit 372, a scaling unit 373, a clip unit 374, and a two-dimensional data sequence generation unit 375.

The signal sequence extraction unit 371 performs a process related to signal sequence extraction. For example, the signal sequence extraction unit 371 acquires and stores input coefficient data Xin (prediction residual D) of a two-dimensional data sequence (matrix) input to the primary horizontal conversion unit 362. Signal sequence extraction unit 371, each line of the input coefficient data Xin extracted one line, supplied as a one-dimensional signal sequence X ₁ in the one-dimensional conversion unit 372.

The one-dimensional conversion unit 372 performs processing related to one-dimensional conversion. For example, the one-dimensional conversion unit 372 acquires the conversion type identifier TrTypeIdxH of the primary horizontal conversion and information (log2TBWSize and log2TBHSize) regarding the size of the conversion block supplied from the primary conversion selection unit 361. Also, one-dimensional transform unit 372 obtains a one-dimensional signal sequence X ₁ supplied from the signal sequence extraction unit 371. 1-dimensional conversion unit 372 performs for the one-dimensional signal sequence X _1, and conversion type identifier TrTypeIdxH primary horizontal transform, a one-dimensional transformation corresponding to the information (Log2TBWSize and Log2TBHSize) about the size of the transform block, one-dimensional generating a signal sequence X _2. 1-dimensional conversion unit 372 supplies the one-dimensional signal sequence X ₂ to the scaling unit 373.

For example, the signal sequence extraction unit 371 extracts one-dimensional signal sequence X ₁ from the target block of coefficient data. Sign inversion unit 102 of the one-dimensional conversion unit 372, to the extracted by the signal string extraction unit 371 one-dimensional signal sequence X _1, performs sign inversion operation.

Further, for example, the two-dimensional data sequence generation unit 375 uses the one-dimensional signal sequence X ₂ (corresponding to the one-dimensional signal sequence X ₄ ) on which the flip operation has been performed by the flip unit 104 of the one-dimensional conversion unit 372. Generate a dimensional data string.

The scaling unit 373 performs processing related to scaling. For example, the scaling unit 373 obtains a one-dimensional signal sequence X ₂ supplied from the one-dimensional conversion unit 372. Scaling unit 373, the coefficients of the one-dimensional signal sequence X _2, scaled by a predetermined shift amount fwdShift1 generating a one-dimensional signal sequence X _3. Scaling unit 373 supplies the one-dimensional signal sequence X ₃ in the clip portion 374.

The clip unit 374 performs processing related to clip processing. For example, the clip portion 374 obtains a one-dimensional signal sequence X ₃ supplied from the scaling unit 373. Clip portion 374, each coefficient of the one-dimensional signal sequence X _3, clipped using the minimum value minCoefVal and maximum MaxCoefVal, to generate a one-dimensional signal sequence X _4. Clip portion 374 supplies the one-dimensional signal sequence X ₄ in the two-dimensional data string generator 375.

The two-dimensional data string generation unit 375 performs processing related to generation of a two-dimensional data string. For example, 2-dimensional data string generator 375 stores a one-dimensional signal sequence X ₄ supplied from the clip portion 374. 2-dimensional data string generator 375 generates an output coefficient data Xout is collectively two-dimensional data sequence to its one-dimensional signal sequence X ₄ by a predetermined number. The two-dimensional data string generation unit 375 outputs the output coefficient data Xout (the conversion coefficient Coeff_Phor after the primary horizontal conversion) to the outside of the primary horizontal conversion unit 362 (supplies it to the primary vertical conversion unit 363).

Each processing unit of the signal sequence extraction unit 371 to the two-dimensional data sequence generation unit 375 has an arbitrary configuration. For example, each processing unit may be configured by a logic circuit that realizes the above-described processing. In addition, each processing unit may include, for example, a CPU, a ROM, a RAM, and the like, and may execute the program by using the CPU, the ROM, the RAM, and the like, thereby realizing the above-described processing. Of course, each processing unit may have both configurations, and a part of the above-described processing may be realized by a logic circuit, and the other may be realized by executing a program. The configuration of each processing unit may be independent from each other. For example, some of the processing units may realize a part of the above-described processing by a logic circuit, and some of the other processing units may execute a program. May be implemented, and another processing unit may implement the above-described processing by both the logic circuit and the execution of the program.

<Primary vertical conversion unit>
FIG. 25 is a block diagram illustrating a main configuration example of the primary vertical conversion unit 363 of FIG. As shown in FIG. 25, the primary vertical conversion unit 363 includes a signal sequence extraction unit 381, a one-dimensional conversion unit 382, a scaling unit 383, a clip unit 384, and a two-dimensional data sequence generation unit 385.

The signal sequence extraction unit 381 performs processing related to signal sequence extraction. For example, the signal sequence extraction unit 381 acquires and stores input coefficient data Xin (transformation coefficient Coeff_Phor after primary horizontal transformation) of a two-dimensional data sequence (matrix) input to the primary vertical transformation unit 363. Signal sequence extraction unit 381, each column of the input coefficient data Xin extracted one by one row, and supplies a 1-dimensional signal sequence X ₁ in the one-dimensional conversion unit 382.

One-dimensional conversion section 382 performs processing related to one-dimensional conversion. For example, the one-dimensional conversion unit 382 acquires the conversion type identifier TrTypeIdxV of the primary vertical conversion and information (log2TBWSize and log2TBHSize) regarding the size of the conversion block, which are supplied from the primary conversion selection unit 361. Also, one-dimensional transform unit 382 obtains a one-dimensional signal sequence X ₁ supplied from the signal sequence extraction unit 381. 1-dimensional conversion unit 382 performs for the one-dimensional signal sequence X _1, and conversion type identifier TrTypeIdxV primary vertical conversion, the one-dimensional transform corresponding to the information (Log2TBWSize and Log2TBHSize) about the size of the transform block, one-dimensional generating a signal sequence X _2. 1-dimensional conversion unit 382 supplies the one-dimensional signal sequence X ₂ to the scaling unit 383.

The scaling unit 383 performs a process related to scaling. For example, the scaling unit 383 obtains a one-dimensional signal sequence X ₂ supplied from the one-dimensional conversion unit 382. Scaling unit 383, the coefficients of the one-dimensional signal sequence X _2, scaled by a predetermined shift amount fwdShift2 generating a one-dimensional signal sequence X _3. Scaling unit 383 supplies the one-dimensional signal sequence X ₃ in the clip portion 384.

The clip unit 384 performs processing related to clip processing. For example, the clip portion 384 obtains a one-dimensional signal sequence X ₃ supplied from the scaling unit 383. Clip portion 384, each coefficient of the one-dimensional signal sequence X _3, clipped using the minimum value minCoefVal and maximum MaxCoefVal, to generate a one-dimensional signal sequence X _4. Clip portion 384 supplies the one-dimensional signal sequence X ₄ in the two-dimensional data string generator 385.

The two-dimensional data string generation unit 385 performs processing related to generation of a two-dimensional data string. For example, 2-dimensional data string generator 385 stores a one-dimensional signal sequence X ₄ supplied from the clip portion 384. 2-dimensional data string generator 385 generates an output coefficient data Xout is collectively two-dimensional data sequence to its one-dimensional signal sequence X ₄ by a predetermined number. The two-dimensional data sequence generation unit 385 outputs the output coefficient data Xout (the conversion coefficient Coeff_P after the primary conversion (the conversion coefficient Coeff_Pver after the primary vertical conversion)) to the outside of the primary vertical conversion unit 363 (to the secondary conversion unit 353). Supply).

Each processing unit of the signal sequence extraction unit 381 to the two-dimensional data sequence generation unit 385 has an arbitrary configuration. For example, each processing unit may be configured by a logic circuit that realizes the above-described processing. In addition, each processing unit may include, for example, a CPU, a ROM, a RAM, and the like, and may execute the program by using the CPU, the ROM, the RAM, and the like, thereby realizing the above-described processing. Of course, each processing unit may have both configurations, and a part of the above-described processing may be realized by a logic circuit, and the other may be realized by executing a program. The configuration of each processing unit may be independent from each other. For example, some of the processing units may realize a part of the above-described processing by a logic circuit, and some of the other processing units may execute a program. May be implemented, and another processing unit may implement the above-described processing by both the logic circuit and the execution of the program.

<Application of this technology>
In the image encoding device 300 having the above-described configuration, for example, the one-dimensional conversion unit 372 (FIG. 24) and the one-dimensional conversion unit 382 (FIG. 25) are used as the conversion device 100 (FIG. 24) described in the first embodiment. 6) may be applied. The conversion device 100 (FIG. 15) described in the third embodiment may be applied as the one-dimensional conversion unit 372 (FIG. 24) or the one-dimensional conversion unit 382 (FIG. 25). Furthermore, the conversion device 100 described in the fifth embodiment may be applied as the one-dimensional conversion unit 372 (FIG. 24) or the one-dimensional conversion unit 382 (FIG. 25).

That is, for example, the 1-dimensional conversion unit 372 and the one-dimensional transform unit 382, for one-dimensional signal sequence X _1, as described with reference to the tables, and the like in FIG. 7, performs one-dimensional transform of each conversion type To do. Further, for example, the one-dimensional conversion unit 372 and the one-dimensional conversion unit 382 derive the base conversion matrix used for the matrix operation as described with reference to the table in FIG.

With such a configuration, the one-dimensional conversion unit 372 and the one-dimensional conversion unit 382 perform the primary conversion (the one-dimensional conversion in the horizontal or vertical direction) on (the prediction residual D of) the image data to be encoded. In this case, the same effect as in the case of the first embodiment, the third embodiment, or the fifth embodiment can be obtained. That is, the image encoding device 300 can suppress the complexity of the configuration of the one-dimensional conversion (simplify the configuration). That is, the image encoding device 300 can more easily perform the one-dimensional conversion. Therefore, the image encoding device 300 can suppress an increase in circuit scale and processing load and an increase in mounting cost.

<Flow of image encoding process>
Next, the flow of each process executed by the image encoding device 300 as described above will be described. First, an example of the flow of the image encoding process will be described with reference to the flowchart in FIG.

When the image encoding process starts, in step S301, the rearrangement buffer 311 is controlled by the control unit 301 to rearrange the order of the frames of the input moving image data from the display order to the encoding order.

In step S302, the control unit 301 sets a processing unit (performs block division) for the input image held by the rearrangement buffer 311.

In step S303, the control unit 301 determines (sets) an encoding parameter for the input image held by the rearrangement buffer 311.

In step S304, the prediction unit 322 performs a prediction process to generate a prediction image or the like in an optimal prediction mode. For example, in this prediction processing, the prediction unit 322 performs intra prediction to generate a prediction image or the like in an optimal intra prediction mode, performs inter prediction to generate a prediction image or the like in an optimal inter prediction mode, From among them, an optimal prediction mode is selected based on a cost function value or the like.

In step S305, the calculation unit 312 calculates the difference between the input image and the prediction image in the optimal mode selected by the prediction processing in step S304. That is, the calculation unit 312 generates a prediction residual D between the input image and the prediction image. The data amount of the prediction residual D obtained in this manner is reduced as compared with the original image data. Therefore, the data amount can be compressed as compared with the case where the image is directly encoded.

において In step S306, the orthogonal transform unit 313 performs an orthogonal transform process on the prediction residual D generated by the process in step S305, and derives a transform coefficient Coeff.

In step S307, the quantization unit 314 quantizes the transform coefficient Coeff obtained by the process in step S306 by using the quantization parameter calculated by the control unit 301, and derives a quantized transform coefficient level level. .

In step S308, the inverse quantization unit 317 inversely quantizes the quantized transform coefficient level generated by the process in step S307 by using a characteristic corresponding to the quantization characteristic in step S307, and derives a transform coefficient Coeff_IQ. .

In step S309, the inverse orthogonal transform unit 318 performs inverse orthogonal transform on the transform coefficient Coeff_IQ obtained in step S308 by a method corresponding to the orthogonal transform process in step S306, and derives a prediction residual D ′. Since the inverse orthogonal transform process is the same as the inverse orthogonal transform process (described later) performed on the decoding side, the description (described later) performed on the decoding side is applied to the inverse orthogonal transform process in step S309. can do.

In step S310, the arithmetic unit 319 adds the prediction image obtained by the prediction processing in step S304 to the prediction residual D ′ derived in the processing in step S309, to obtain a locally decoded image. Generate.

In step S311, the in-loop filter unit 320 performs an in-loop filter process on the locally decoded image derived in step S310.

In step S312, the frame memory 321 stores the locally decoded image derived in step S310 and the locally decoded image filtered in step S312.

において In step S313, the encoding unit 315 encodes the quantized transform coefficient level level obtained by the processing in step S307. For example, the encoding unit 315 encodes a quantized transform coefficient level, which is information about an image, by arithmetic encoding or the like, and generates encoded data. At this time, the encoding unit 315 encodes various encoding parameters (header information Hinfo, prediction mode information Pinfo, and conversion information Tinfo). Further, the encoding unit 315 derives residual information RInfo from the quantized transform coefficient level level, and encodes the residual information RInfo.

In step S314, the accumulation buffer 316 accumulates the encoded data thus obtained, and outputs the encoded data to the outside of the image encoding device 300, for example, as a bit stream. This bit stream is transmitted to the decoding side via a transmission path or a recording medium, for example. Further, the rate control unit 323 performs rate control as needed.

すると When the processing in step S314 ends, the image encoding processing ends.

<Flow of orthogonal transformation processing>
Next, an example of the flow of the orthogonal transformation process executed in step S306 in FIG. 26 will be described with reference to the flowchart in FIG.

When the orthogonal transform processing is started, the switch 351 determines in step S331 that the transform skip flag ts_flag is 2D_TS (when indicating a two-dimensional transform skip) (for example, 1 (true)) or the transform quantization bypass flag transquant_bypass_flag is 1 (True), it is determined whether or not. When the transform skip flag ts_flag is determined to be 2D_TS (for example, 1 (true)) or the transform quantization bypass flag is 1 (true), the orthogonal transform process ends, and the process returns to FIG. In this case, the orthogonal transform processing (primary transform and secondary transform) is omitted, and the input prediction residual D is used as the transform coefficient Coeff.

Also, in step S331 of FIG. 27, it is determined that the conversion skip flag ts_flag is not 2D_TS (not a two-dimensional conversion skip) (for example, 0 (false)) and the transform quantization bypass flag transquant_bypass_flag is 0 (false). If so, the process proceeds to step S332. In this case, a primary conversion process and a secondary conversion process are performed.

In step S332, the primary transform unit 352 performs a primary transform process on the input prediction residual D based on the adaptive primary transform information specified by the component identifier compID, and derives a transform coefficient Coeff_P after the primary transform. I do.

In step S333, the secondary conversion unit 353 performs a secondary conversion process on the conversion coefficient Coeff_P, and derives a conversion coefficient Coeff_S (transform coefficient Coeff) after the secondary conversion.

(4) When the processing in step S333 ends, the orthogonal transformation processing ends.

<Flow of primary conversion process>
Next, an example of the flow of the primary conversion process executed in step S332 of FIG. 27 will be described with reference to the flowchart of FIG.

When the primary conversion process is started, the primary conversion selecting unit 361 (FIG. 23) of the primary conversion unit 352 determines in step S341 that the conversion type identifier TrTypeIdxH of the primary horizontal conversion (and the conversion type TrTypeH specified by the identifier). , And the conversion type identifier TrTypeIdxV of the primary vertical conversion (and the conversion type TrTypeV specified by the identifier), respectively, are selected as described above.

In step S342, the primary horizontal conversion unit 362 performs primary horizontal conversion processing corresponding to the conversion type identifier TrTypeIdxH of the primary horizontal conversion obtained in step S341 on the prediction residual D, and performs a conversion coefficient Coeff_Phor after the primary horizontal conversion. Is derived.

In step S343, the primary vertical conversion unit 363 performs the primary vertical conversion processing corresponding to the conversion type identifier TrTypeIdxV of the primary vertical conversion obtained in step S341 on the primary horizontal conversion result (the conversion coefficient Coeff_Phor after the primary horizontal conversion). Then, a conversion coefficient Coeff_Pver after the primary vertical conversion (a conversion coefficient Coeff_P after the primary conversion) is derived.

すると When the processing in step S343 ends, the primary conversion processing ends, and the processing returns to FIG.

<Flow of primary horizontal conversion process>
Next, an example of the flow of the primary horizontal conversion process executed in step S342 in FIG. 28 will be described with reference to the flowchart in FIG.

When the primary horizontal conversion process is started, the signal sequence extraction unit 371 (FIG. 24) of the primary horizontal conversion unit 362 acquires input coefficient data Xin (prediction residual D) as a two-dimensional data sequence in step S351. , Memorize (hold).

In step S352, the signal sequence extraction unit 371, for example, as shown in the following expression (29), and extracts a row of the process target of the input coefficient data Xin holding the (j) as a one-dimensional signal sequence X _1.

In step S353, one-dimensional conversion unit 372 performs conversion processing using the base transform matrix T _base in accordance with the converted type identifier trTypeIdxH the transform size (NTBS), a one-dimensional transformation for the one-dimensional signal sequence X ₁ Do.

In step S354, the scaling unit 373, for example, as shown in the following expression (30), scaling the coefficients X ₂ of the one-dimensional signal sequence X ₂ [i] in the shift amount FwdShift1, a one-dimensional signal sequence X ₃ Derive.

In step S355, the clip portion 374, for example, as shown in the following expression (31), the coefficients of the one-dimensional signal sequence X _₃ X ₃ [i], clipped between the minimum minCoefVal and maximum maxCoefVal , to derive a one-dimensional signal sequence X _4.

In step S356, the two-dimensional data array generating unit 375 generates two-dimensional data string Xout by using a one-dimensional signal sequence X _4. That is, two-dimensional data string generator 375, holding a one-dimensional signal sequence X ₄ and (storage), by assembling a predetermined column fraction 1 dimensional signal sequence X _4, to produce a 2-dimensional data string Xout. This processing can be represented, for example, by the following equation (32).

(2) In step S357, the two-dimensional data string generation unit 375 determines whether or not each processing in steps S352 to S357 has been performed for all rows. That is, the processes in steps S352 to S357 are performed for each row of the input data Xin held in step S351. The two-dimensional data string generation unit 375 determines whether or not all the rows have been processed.

If it is determined that there is an unprocessed line, the process returns to step S352, and the subsequent process is repeated for the next unprocessed line. If it is determined in step S357 that all the rows have been processed, the two-dimensional data sequence generation unit 375 converts the generated two-dimensional data sequence Xout (the conversion coefficient Coeff_Phor after the primary horizontal conversion) into the primary vertical conversion unit 363. It is output to the outside (supplied to the primary vertical conversion unit 363). When the two-dimensional data string Xout is output, the primary horizontal conversion processing ends, and the processing returns to FIG.

<Flow of primary vertical conversion processing>
Next, an example of the flow of the primary vertical conversion process executed in step S343 of FIG. 28 will be described with reference to the flowchart of FIG.

When the primary vertical conversion process is started, the signal sequence extraction unit 381 (FIG. 25) of the primary vertical conversion unit 363 determines in step S361 that the input coefficient data Xin (the conversion coefficient Coeff_Phor after the primary horizontal conversion) is a two-dimensional data sequence. ) Is acquired and stored (held).

In step S362, the signal sequence extraction unit 381, for example, as shown in the following expression (33), extracts the column to be processed of the input coefficient data Xin holding the (j) as a one-dimensional signal sequence X _1.

In step S363, one-dimensional conversion unit 382 performs conversion processing using the base transform matrix T _base in accordance with the converted type identifier trTypeIdxV the transform size (NTBS), a one-dimensional transformation for the one-dimensional signal sequence X ₁ Do.

In step S364, the scaling unit 383, for example, as shown in the following expression (34), scaling the coefficients X ₂ of the one-dimensional signal sequence X ₂ [i] in the shift amount FwdShift2, a one-dimensional signal sequence X ₃ Derive.

In step S365, the clipping unit 384 clips each coefficient X ₃ [i] of the one-dimensional signal sequence X ₃ between the minimum value minCoefVal and the maximum value maxCoefVal, for example, as in Expression (31) above. , to derive a one-dimensional signal sequence X _4.

In step S366, the two-dimensional data array generating unit 385 generates two-dimensional data string Xout by using a one-dimensional signal sequence X _4. That is, two-dimensional data string generator 385, holding a one-dimensional signal sequence X ₄ and (storage), by assembling a predetermined column fraction 1 dimensional signal sequence X _4, to produce a 2-dimensional data string Xout. This processing can be represented, for example, by the following equation (35).

において In step S367, the two-dimensional data string generation unit 385 determines whether or not each processing in steps S362 to S367 has been performed for all the columns. That is, the processes in steps S362 to S367 are performed for each column of the input data Xin held in step S361. The two-dimensional data sequence generation unit 385 determines whether or not all the columns have been processed.

If it is determined that there is an unprocessed column, the process returns to step S362, and the subsequent process is repeated for the next unprocessed column. If it is determined in step S367 that all columns have been processed, the primary vertical conversion process ends, and the process returns to FIG.

<Application of this technology>
In step S353 of the above-described primary horizontal conversion processing (FIG. 29), for example, the one-dimensional conversion unit 372 executes the conversion processing in the same flow as in the case of the first embodiment (FIG. 8). You may. In addition, the one-dimensional conversion unit 372 may execute the conversion process in the same flow as in the case of the third embodiment (FIG. 17). Further, the one-dimensional conversion unit 372 may execute the conversion processing in the same flow as in the fifth embodiment.

Further, in step S363 of the primary vertical conversion process (FIG. 30) as described above, for example, the one-dimensional conversion unit 382 executes the conversion process in the same flow as in the case of the first embodiment (FIG. 8). You may do so. Also, the one-dimensional conversion unit 382 may execute the conversion process in the same flow as in the case of the third embodiment (FIG. 17). Further, the one-dimensional conversion unit 382 may execute the conversion process in the same flow as in the fifth embodiment.

By executing each process in this way, the one-dimensional conversion unit 372 or the one-dimensional conversion unit 382 performs one-dimensional conversion in the horizontal or vertical direction on the primary conversion (for the prediction residual D of the encoded image data). In the conversion, it is possible to obtain the same effect as in the case of the first embodiment, the third embodiment, or the fifth embodiment. That is, the image encoding device 300 can suppress the complexity of the configuration of the one-dimensional conversion (simplify the configuration). That is, the image encoding device 300 can more easily perform the one-dimensional conversion. Therefore, the image encoding device 300 can suppress an increase in circuit scale and processing load and an increase in mounting cost.

<10. Seventh embodiment>
<Image decoding device>
FIG. 31 is a block diagram illustrating an example of a configuration of an image decoding device that is an aspect of an image processing device to which the present technology is applied. The image decoding apparatus 400 illustrated in FIG. 31 is an apparatus that decodes encoded data in which a prediction residual between an image and a prediction image thereof is encoded, such as AVC or HEVC. For example, the image decoding apparatus 400 implements the technology described in Non-Patent Document 1, Non-Patent Document 5, or Non-Patent Document 6, and performs moving image decoding by a method based on a standard described in any of those documents. The encoded data obtained by encoding the image data of the image is decoded. For example, the image decoding device 400 decodes the encoded data (bit stream) generated by the image encoding device 300 described above.

Note that FIG. 31 shows main components such as the processing unit and the flow of data, and the components shown in FIG. 31 are not necessarily all. That is, in the image decoding device 400, a processing unit not illustrated as a block in FIG. 31 may exist, or a process or data flow not illustrated as an arrow or the like in FIG. 31 may exist. This is the same in other drawings for explaining the processing unit and the like in the image decoding device 400.

In FIG. 31, the image decoding device 400 includes an accumulation buffer 411, a decoding unit 412, an inverse quantization unit 413, an inverse orthogonal transform unit 414, an operation unit 415, an in-loop filter unit 416, a rearrangement buffer 417, a frame memory 418, and A prediction unit 419 is provided. Note that the prediction unit 419 includes an intra prediction unit and an inter prediction unit (not shown). The image decoding device 400 is a device for generating moving image data by decoding encoded data (bit stream).

<Accumulation buffer>
The accumulation buffer 411 acquires the bit stream input to the image decoding device 400 and holds (stores) the bit stream. The storage buffer 411 supplies the stored bit stream to the decoding unit 412 at a predetermined timing or when a predetermined condition is satisfied.

<Decoding unit>
The decoding unit 412 performs a process related to image decoding. For example, the decoding unit 412 receives the bit stream supplied from the accumulation buffer 411 as input, performs variable length decoding of the syntax value of each syntax element from the bit string according to the definition of the syntax table, and derives parameters. I do.

パラメータ The parameters derived from the syntax elements and the syntax values of the syntax elements include, for example, information such as header information Hinfo, prediction mode information Pinfo, conversion information Tinfo, residual information Rinfo, and filter information Finfo. That is, the decoding unit 412 parses (analyzes and acquires) such information from the bit stream. The information will be described below.

<Header information Hinfo>
The header information Hinfo includes, for example, header information such as VPS (Video Parameter Set) / SPS (Sequence Parameter Set) / PPS (Picture Parameter Set) / SH (slice header). The header information Hinfo includes, for example, an image size (horizontal width PicWidth, vertical width PicHeight), bit depth (luminance bitDepthY, chrominance bitDepthC), chrominance array type ChromaArrayType, and CU size maximum value MaxCUSize / minimum value MinCUSize and quadtree division ( Maximum depth MaxQTDepth / minimum depth MinQTDepth of Quad-tree partition) Maximum depth MaxBTDepth / minimum depth MinBTDepth of binary tree partition (Binary-tree partition), maximum value MaxTSSize of conversion skip block (also called maximum conversion skip block size) ), And information specifying an on / off flag (also referred to as a valid flag) of each encoding tool.

For example, as the on / off flag of the encoding tool included in the header information Hinfo, there are on / off flags related to the following conversion and quantization processing. Note that the on / off flag of the encoding tool can also be interpreted as a flag indicating whether or not syntax related to the encoding tool exists in encoded data. When the value of the on / off flag is 1 (true), it indicates that the encoding tool is usable. When the value of the on / off flag is 0 (false), the encoding tool is not usable. Show. Note that the interpretation of the flag value may be reversed.

間 Inter-component prediction enabled flag (ccp_enabled_flag): Flag information indicating whether or not inter-component prediction (CCP (Cross-Component Prediction), also referred to as CC prediction) is available. For example, when the flag information is “1” (true), it indicates that it can be used, and when it is “0” (false), it indicates that it cannot be used.

CCP This CCP is also called inter-component linear prediction (CCLM or CCLMP).

<Prediction mode information Pinfo>
The prediction mode information Pinfo includes, for example, information such as size information PBSize (prediction block size) of the processing target PB (prediction block), intra prediction mode information IPinfo, and motion prediction information MVinfo.

The 予測 intra prediction mode information IPinfo includes, for example, JCTVC-W1005, 7.3.8.5 coding Unit syntax, prev_intra_luma_pred_flag, mpm_idx, rem_intra_pred_mode, and a luminance intra prediction mode IntraPredModeY derived from the syntax.

The intra prediction mode information IPinfo includes, for example, an inter-component prediction flag (ccp_flag (cclmp_flag)), a multi-class linear prediction mode flag (mclm_flag), a chrominance sample position type identifier (chroma_sample_loc_type_idx), a chrominance MPM identifier (chroma_mpm_idx), and , And a luminance intra prediction mode (IntraPredModeC) derived from these syntaxes.

The inter-component prediction flag (ccp_flag (cclmp_flag)) is flag information indicating whether or not to apply inter-component linear prediction. For example, when ccp_flag == 1, it indicates that inter-component prediction is applied, and when ccp_flag == 0, it indicates that inter-component prediction is not applied.

The multi-class linear prediction mode flag (mclm_flag) is information on the mode of linear prediction (linear prediction mode information). More specifically, the multi-class linear prediction mode flag (mclm_flag) is flag information indicating whether to set the multi-class linear prediction mode. For example, “0” indicates a one-class mode (single-class mode) (for example, CCLMP), and “1” indicates a two-class mode (multi-class mode) (for example, MCLMP). .

The chrominance sample position type identifier (chroma_sample_loc_type_idx) is an identifier for identifying the type of the pixel position of the chrominance component (also referred to as a chrominance sample position type). For example, when the chrominance array type (ChromaArrayType), which is information on the color format, indicates the 420 format, the chrominance sample position type identifier is assigned as in the following Expression (36).

The chrominance sample position type identifier (chroma_sample_loc_type_idx) is transmitted (stored in) as information (chroma_sample_loc_info ()) regarding the pixel position of the chrominance component.

The chrominance MPM identifier (chroma_mpm_idx) is an identifier indicating which prediction mode candidate in the chrominance intra prediction mode candidate list (intraPredModeCandListC) is designated as the chrominance intra prediction mode.

The motion prediction information MVinfo includes, for example, information such as merge_idx, merge_flag, inter_pred_idc, ref_idx_LX, mvp_lX_flag, X = {0,1}, mvd, and the like (for example, see JCTVC-W1005, 7.3.8.6 Prediction Unit Syntax). .

Of course, the information included in the prediction mode information Pinfo is arbitrary, and information other than these information may be included.

<Conversion information Tinfo>
The conversion information Tinfo includes, for example, the following information. Of course, the information included in the conversion information Tinfo is arbitrary, and information other than these information may be included.

The horizontal width size TBWSize and the vertical width TBHSize of the conversion block to be processed (or each TBWSize having a base of 2, and the logarithmic value log2TBWSize or log2TBHSize of TBHSize may be used). Conversion skip flag (ts_flag): This flag indicates whether or not (reverse) primary conversion and (reverse) secondary conversion are skipped.
Scan identifier (scanIdx)
Quantization parameter (qp)
Quantization matrix (scaling_matrix (eg, JCTVC-W1005, 7.3.4 Scaling list data syntax))

<Residual information Rinfo>
The residual information Rinfo (for example, refer to 7.3.8.11 Residual Coding syntax of JCTVC-W1005) includes, for example, the following syntax.

cbf (coded_block_flag): residual data presence flag last_sig_coeff_x_pos: last non-zero coefficient X coordinate last_sig_coeff_y_pos: last non-zero coefficient Y coordinate coded_sub_block_flag: sub-block non-zero coefficient presence flag sig_coeff_flag: non-zero coefficient presence flag gr1_flag: non-zero coefficient Flag indicating whether it is greater than 1 (also called GR1 flag)
gr2_flag: Flag indicating whether the level of the non-zero coefficient is greater than 2 (also referred to as GR2 flag)
sign_flag: code indicating the sign of the non-zero coefficient (also called sign code)
coeff_abs_level_remaining: residual level of non-zero coefficient (also called non-zero coefficient residual level)
Such.

Of course, the information included in the residual information Rinfo is arbitrary, and information other than these information may be included.

<Filter information Finfo>
The filter information Finfo includes, for example, control information on each filter process described below.

Control information about deblocking filter (DBF) Control information about pixel adaptive offset (SAO) Control information about adaptive loop filter (ALF) Control information about other linear and nonlinear filters

More specifically, for example, a picture to which each filter is applied, information for specifying a region in the picture, filter On / Off control information for each CU, filter on / off control information for slices, and tile boundaries are included. included. Of course, the information included in the filter information Finfo is arbitrary, and information other than these information may be included.

Returning to the description of the decoding unit 212, the decoding unit 212 derives the quantized transform coefficient level level at each coefficient position in each transform block with reference to the residual information Rinfo. The decoding unit 212 supplies the quantized transform coefficient level level to the inverse quantization unit 213.

{Decoding section 412 supplies the parsed header information Hinfo, prediction mode information Pinfo, quantized transform coefficient level level, transform information Tinfo, and filter information Finfo to each block. Specifically, it is as follows.

The header information Hinfo is supplied to the inverse quantization unit 413, the inverse orthogonal transform unit 414, the prediction unit 419, and the in-loop filter unit 416. The prediction mode information Pinfo is supplied to the inverse quantization unit 413 and the prediction unit 419. The transform information Tinfo is supplied to the inverse quantization unit 413 and the inverse orthogonal transform unit 414. The filter information Finfo is supplied to the in-loop filter unit 416.

Of course, the above example is an example, and the present invention is not limited to this example. For example, each encoding parameter may be supplied to an arbitrary processing unit. Further, other information may be supplied to an arbitrary processing unit.

<Inverse quantization unit>
The inverse quantization unit 413 performs a process related to inverse quantization. For example, the inverse quantization unit 413 receives the transform information Tinfo and the quantized transform coefficient level supplied from the decoding unit 412 as inputs, and scales the value of the quantized transform coefficient level (inverse) based on the transform information Tinfo. Quantization), and derives a transform coefficient Coeff_IQ after inverse quantization.

逆 Note that this inverse quantization is performed as inverse processing of quantization by the quantization unit 314. This inverse quantization is the same processing as the inverse quantization by the inverse quantization unit 317. That is, the inverse quantization unit 317 performs the same processing (inverse quantization) as the inverse quantization unit 413.

The inverse quantization unit 413 supplies the derived transform coefficient Coeff_IQ to the inverse orthogonal transform unit 414.

<Inverse orthogonal transform unit>
The inverse orthogonal transform unit 414 performs a process related to the inverse orthogonal transform. For example, the inverse orthogonal transform unit 414 receives the transform coefficient Coeff_IQ supplied from the inverse quantization unit 413 and the transform information Tinfo supplied from the decoding unit 412 as inputs, and converts the transform coefficient Coeff_IQ based on the transform information Tinfo. An inverse orthogonal transformation process is performed on the result to derive a prediction residual D ′.

逆 Note that this inverse orthogonal transform is performed as an inverse process of the orthogonal transform by the orthogonal transform unit 313. The inverse orthogonal transform is a process similar to the inverse orthogonal transform performed by the inverse orthogonal transform unit 318. That is, the inverse orthogonal transform unit 318 performs the same processing (inverse orthogonal transform) as the inverse orthogonal transform unit 414.

The inverse orthogonal transform unit 414 supplies the derived prediction residual D ′ to the calculation unit 415.

<Operation part>
The calculation unit 415 performs a process related to addition of information on an image. For example, the calculation unit 415 receives the prediction residual D ′ supplied from the inverse orthogonal transform unit 414 and the prediction image P supplied from the prediction unit 419 as inputs. The calculation unit 415 adds the prediction residual D ′ and the prediction image P (prediction signal) corresponding to the prediction residual D ′ to derive the local decoded image Rlocal as shown in the following Expression (37). I do.

The operation unit 415 supplies the derived local decoded image Rlocal to the in-loop filter unit 416 and the frame memory 418.

<In-loop filter section>
The in-loop filter unit 416 performs processing related to in-loop filter processing. For example, the in-loop filter unit 416 receives the local decoded image Rlocal supplied from the arithmetic unit 415 and the filter information Finfo supplied from the decoding unit 412 as inputs. Note that information input to the in-loop filter unit 416 is arbitrary, and information other than these information may be input.

The in-loop filter unit 416 appropriately performs a filtering process on the locally decoded image Rlocal based on the filter information Finfo.

For example, as described in Non-Patent Document 1, the in-loop filter unit 416 includes a bilateral filter, a deblocking filter (DBF (DeBlocking @ Filter)), an adaptive offset filter (SAO (Sample @ Adaptive @ Offset)), and an adaptive loop filter. Four in-loop filters (ALF (Adaptive Loop Loop Filter)) are applied in this order. It is to be noted that which filter is applied and in which order are applied are arbitrary and can be selected as appropriate.

The in-loop filter unit 416 performs a filter process corresponding to the filter process performed by the encoding side (for example, the in-loop filter unit 320 of the image encoding device 300). Of course, the filtering process performed by the in-loop filter unit 416 is optional, and is not limited to the above example. For example, the in-loop filter unit 416 may apply a Wiener filter or the like.

The in-loop filter unit 416 supplies the filtered local decoded image Rlocal to the reordering buffer 417 and the frame memory 418.

<Sort buffer>
The reordering buffer 417 receives the local decoded image Rlocal supplied from the in-loop filter unit 416 as an input, and holds (stores) it. The reordering buffer 417 reconstructs and holds the decoded image R for each picture unit using the local decoded image Rlocal (stores the decoded image R in the buffer). The rearrangement buffer 417 rearranges the obtained decoded images R from decoding order to reproduction order. The rearrangement buffer 417 outputs the rearranged decoded image group R to the outside of the image decoding device 200 as moving image data.

<Frame memory>
The frame memory 418 performs a process related to storage of data related to an image. For example, the frame memory 418 receives the local decoded image Rlocal supplied from the arithmetic unit 415 as an input, reconstructs a decoded image R for each picture unit, and stores the reconstructed image R in a buffer in the frame memory 418.

The frame memory 418 receives the in-loop filtered local decoded image Rlocal supplied from the in-loop filter unit 416 as an input, reconstructs a decoded image R for each picture unit, and To store. The frame memory 418 appropriately supplies the stored decoded image R (or a part thereof) to the prediction unit 419 as a reference image.

Note that the frame memory 418 may store header information Hinfo, prediction mode information Pinfo, conversion information Tinfo, filter information Finfo, and the like related to generation of a decoded image.

<Prediction unit>
The prediction unit 419 performs a process related to generation of a predicted image. For example, the prediction unit 419 receives the prediction mode information Pinfo supplied from the decoding unit 412, performs prediction using the prediction method specified by the prediction mode information Pinfo, and derives a predicted image P. At the time of the derivation, the prediction unit 419 uses a decoded image R (or a part thereof) before or after the filter, which is specified by the prediction mode information Pinfo and stored in the frame memory 418, as a reference image. The prediction unit 419 supplies the derived prediction image P to the calculation unit 415.

<Details of inverse orthogonal transform unit>
FIG. 32 is a block diagram illustrating a main configuration example of the inverse orthogonal transform unit 414 in FIG. As shown in FIG. 32, the inverse orthogonal transform unit 414 includes a switch 451, an inverse secondary transform unit 452, and an inverse primary transform unit 453.

The switch 451 receives as input the conversion coefficient Coeff_IQ and the conversion skip flag ts_flag [compID]. When the value of the conversion skip flag ts_flag [compID] is NO_TS (= 0), that is, when the conversion skip is not applied, the switch 451 supplies the conversion coefficient Coeff_IQ to the inverse secondary conversion unit 452. When the value of the conversion skip flag ts_flag [compID] is 2D_TS (= 1), that is, when it indicates that the two-dimensional conversion skip is to be applied, the switch 451 sets the inverse secondary conversion unit 452 and the inverse primary conversion unit 453 to each other. Skipping is performed and the transform coefficient Coeff_IQ is output to the outside of the inverse orthogonal transform unit 414 as the prediction residual D ′ (supplied to the arithmetic unit 415).

The inverse secondary transform unit 452 performs a process related to an inverse secondary transform, which is an inverse process of the secondary transform performed on the encoding side (for example, the secondary transform unit 353 of the image encoding device 300). For example, the inverse secondary transform unit 452 receives as input the secondary transform identifier st_idx, the scan identifier scanIdx indicating the scan method of the transform coefficient, and the transform coefficient Coeff_IQ supplied from the switch 451.

The inverse secondary transform unit 452 performs inverse secondary transform on the transform coefficient Coeff_IQ based on the secondary transform identifier st_idx and the scan identifier scanIdx, and derives a transform coefficient Coeff_IS after the inverse secondary transform.

More specifically, when the secondary transform identifier st_idx indicates that the inverse secondary transform is applied (st_idx> 0), the inverse secondary transform unit 452 performs the inverse transform corresponding to the secondary transform identifier st_idx on the transform coefficient Coeff_IQ. The secondary conversion processing is executed to derive a conversion coefficient Coeff_IS after the inverse secondary conversion. The inverse secondary transform unit 452 supplies the inverse secondary transform coefficient Coeff_IS to the inverse primary transform unit 453.

If the secondary transform identifier st_idx indicates that the inverse secondary transform is not applied (st_idx == 0), the inverse secondary transform unit 452 skips the inverse secondary transform and converts the transform coefficient Coeff_IQ into the transform coefficient after the inverse secondary transform. It is supplied to the inverse primary conversion unit 453 as Coeff_IS.

The inverse primary conversion unit 453 performs a process related to an inverse primary conversion, which is an inverse process of the primary conversion performed on the encoding side (for example, the primary conversion unit 352 of the image encoding device 300). For example, the inverse primary conversion unit 453 generates the component identifier compID, the adaptive primary conversion flag apt_flag [compID] of the component identifier compID, the primary conversion identifier pt_idx [compID] of the component identifier compID, the prediction mode information PInfo, the size of the conversion block (the width of the conversion block). The logarithmic value log2TBWSize, (the vertical logarithmic value log2TBHSize) and the conversion coefficient Coeff_IS after the inverse secondary conversion are input.

The inverse primary conversion unit 453 refers to the prediction mode information PInfo, the component identifier compID, the adaptive primary conversion flag apt_flag [compID] of the component identifier compID, and the primary conversion identifier pt_idx [compID] of the component identifier compID, and refers to the component identifier compID. The conversion type TrTypeH of the inverse primary horizontal conversion corresponding to (and the inverse primary horizontal conversion type identifier TrTypeIdxH indicating the conversion type), and the conversion type TrTypeV of the inverse primary vertical conversion (and the inverse primary vertical conversion type identifier TrTypeIdxV indicating the conversion type) ).

In addition, the inverse primary conversion unit 453 determines the inverse primary vertical conversion type identifier TrTypeIdxV (or the inverse primary vertical conversion type TrTypeV) and the inverse primary logarithm determined by the vertical width log2TBHSize of the conversion block for the conversion coefficient Coeff_IS after the inverse secondary. The vertical conversion and the inverse primary horizontal conversion type identifier TrTypeIdxH (or the inverse primary horizontal conversion type TrTypeH) and the inverse primary horizontal conversion determined by the horizontal width log2TBWSize of the conversion block are performed to derive a conversion coefficient Coeff_IP after the inverse primary conversion. The inverse primary vertical transform is an inverse one-dimensional orthogonal transform in the vertical direction, and the inverse primary horizontal transform is an inverse one-dimensional orthogonal transform in the horizontal direction.

The inverse primary transform unit 453 outputs the transform coefficient Coeff_IP after the inverse primary transform to the outside of the inverse orthogonal transform unit 414 as a prediction residual D ′ (supplies it to the arithmetic unit 415).

<Reverse primary conversion unit>
FIG. 33 is a block diagram illustrating a main configuration example of the inverse primary conversion unit 453 (FIG. 32) in this case. As shown in FIG. 33, the inverse primary conversion unit 453 has an inverse primary conversion selection unit 461, an inverse primary vertical conversion unit 462, and an inverse primary horizontal conversion unit 463.

The inverse primary conversion selecting unit 461 receives as input the prediction mode information PInfo, the component identifier compID, the adaptive primary conversion flag apt_flag [compID], and the primary conversion identifier pt_idx [compID]. The inverse primary conversion selecting unit 461 derives the conversion type identifier TrTypeIdxV of the inverse primary vertical conversion and the conversion type identifier TrTypeIdxH of the inverse primary vertical conversion with reference to the information. The inverse primary conversion selecting unit 461 supplies the derived conversion type identifier TrTypeIdxV of the inverted primary vertical conversion to the inverted primary vertical conversion unit 462. Further, the inverse primary conversion selecting unit 461 supplies the derived conversion type identifier TrTypeIdxH of the inverted primary horizontal conversion to the inverted primary horizontal conversion unit 463.

The inverse primary vertical conversion unit 462 receives as input the conversion coefficient Coeff_IS after the inverse secondary conversion, the conversion type identifier TrTypeIdxV of the inverse primary vertical conversion, and information on the size of the conversion block. The information on the size of the transform block may be a natural number N indicating the horizontal or vertical size (the number of coefficients) of the transform block, or log2TBHSize (the logarithmic value of the vertical width) indicating the vertical width of the transform block. ) (N = 1 << log2TBHSize). The inverse primary vertical conversion unit 462 performs an inverse primary vertical conversion IPver determined by the conversion type identifier TrTypeIdxV and the size of the conversion block on the conversion coefficient Coeff_IS after the inverse secondary conversion, and converts the conversion coefficient Coeff_IPver after the inverse primary vertical conversion. Derive. The inverse primary vertical conversion unit 462 supplies the conversion coefficient Coeff_IPver after the inverse primary vertical conversion to the inverse primary horizontal conversion unit 463.

The inverse primary horizontal conversion unit 463 receives as input the conversion coefficient Coeff_IPver after the inverse primary vertical conversion, the conversion type identifier TrTypeIdxH of the inverse primary horizontal conversion, and information on the size of the conversion block. The information on the size of the transform block may be a natural number N indicating the horizontal or vertical size (the number of coefficients) of the transform block, or log2TBWSize (the logarithmic value of the transverse width) indicating the lateral width of the transform block. (N あっ = 1 << log2TBWSize). The inverse primary horizontal conversion unit 463 performs the inverse primary horizontal conversion IPhor determined by the conversion type identifier TrTypeIdxH and the size of the conversion block on the conversion coefficient Coeff_IPver after the inverse primary vertical conversion supplied from the inverse primary vertical conversion unit 462. , The conversion coefficient Coeff_IPhor after the inverse primary horizontal conversion (that is, the conversion coefficient Coeff_IP after the inverse primary conversion) is derived. The inverse primary horizontal transform unit 463 outputs the transform coefficient Coeff_IPhor after the inverse primary horizontal transform as a prediction residual D ′ to the outside of the inverse primary transform unit 453 (supplies it to the arithmetic unit 415).

<Inverse primary vertical conversion unit>
FIG. 34 is a block diagram illustrating a main configuration example of the inverse primary vertical conversion unit 462 in FIG. As shown in FIG. 34, the inverse primary vertical conversion unit 462 includes a signal sequence extraction unit 471, an inverse one-dimensional conversion unit 472, a scaling unit 473, a clip unit 474, and a two-dimensional data sequence generation unit 475.

The signal sequence extraction unit 471 performs a process related to signal sequence extraction. For example, the signal sequence extracting unit 471 acquires and stores input coefficient data Xin (transform coefficient Coeff_IS after inverse secondary transform) of a two-dimensional data sequence (matrix) input to the inverse primary vertical transform unit 462. Signal sequence extraction unit 471, each column of the input coefficient data Xin extracted one column, and supplies the inverse one-dimensional transform unit 472 as a one-dimensional signal sequence X _1.

For example, the signal sequence extraction unit 471 obtains a one-dimensional signal sequence X from the processing target block of the quantized transform coefficient level level (the transform coefficient Coeff_IS corresponding to the coefficient data generated by decoding the bit stream by the decoding unit 412). Extract ₁ The flip unit 152 of the inverse one-dimensional conversion unit 472 performs a flip operation on the one-dimensional signal sequence extracted by the signal sequence extraction unit 471.

The inverse one-dimensional conversion unit 472 performs a process related to the inverse one-dimensional conversion. For example, the inverse one-dimensional conversion unit 472 acquires the conversion type identifier TrTypeIdxV of the inverse primary vertical conversion and the information (log2TBWSize and log2TBHSize) regarding the size of the conversion block, which are supplied from the inverse primary conversion selection unit 461. Also, inverse one-dimensional transform unit 472 obtains a one-dimensional signal sequence X ₁ supplied from the signal sequence extraction unit 471. Inverse one-dimensional transform unit 472, for the one-dimensional signal sequence X _1, and conversion type identifier TrTypeIdxV inverse primary vertical conversion, an inverse one-dimensional transform corresponding to the information (Log2TBWSize and Log2TBHSize) about the size of the transform block performs , and it generates a one-dimensional signal sequence X _2. Inverse one-dimensional transform unit 472 supplies the one-dimensional signal sequence X ₂ to the scaling unit 473.

The scaling unit 473 performs processing related to scaling. For example, the scaling unit 473 obtains a one-dimensional signal sequence X ₂ supplied from the inverse one-dimensional transform unit 472. Scaling unit 473, the coefficients of the one-dimensional signal sequence X _2, scaled by a predetermined shift amount invShift1 generating a one-dimensional signal sequence X _3. Scaling unit 473 supplies the one-dimensional signal sequence X ₃ in the clip portion 474.

The clip unit 474 performs processing related to clip processing. For example, the clip portion 474 obtains a one-dimensional signal sequence X ₃ supplied from the scaling unit 473. Clip portion 474, each coefficient of the one-dimensional signal sequence X _3, clipped using the minimum value minCoefVal and maximum MaxCoefVal, to generate a one-dimensional signal sequence X _4. Clip portion 474 supplies the one-dimensional signal sequence X ₄ in the two-dimensional data string generator 475.

The two-dimensional data string generation unit 475 performs processing related to generation of a two-dimensional data string. For example, 2-dimensional data string generator 475 stores a one-dimensional signal sequence X ₄ supplied from the clip portion 474. 2-dimensional data string generator 475 generates an output coefficient data Xout is collectively two-dimensional data sequence to its one-dimensional signal sequence X ₄ by a predetermined number. The two-dimensional data string generation unit 475 outputs the output coefficient data Xout (the conversion coefficient Coeff_IPver after inverse primary vertical conversion) to the outside of the inverse primary vertical conversion unit 462 (supplies it to the inverse primary horizontal conversion unit 463).

For example, the two-dimensional data sequence generation unit 475 uses the one-dimensional signal sequence X ₂ (corresponding to the one-dimensional signal sequence X ₄ ) on which the sign inversion operation has been performed by the sign inversion unit 154 of the inverse one-dimensional conversion unit 472. Generate a two-dimensional data sequence.

Each processing unit of the signal sequence extraction unit 471 to the two-dimensional data sequence generation unit 475 has an arbitrary configuration. For example, each processing unit may be configured by a logic circuit that realizes the above-described processing. In addition, each processing unit may include, for example, a CPU, a ROM, a RAM, and the like, and may execute the program by using the CPU, the ROM, the RAM, and the like, thereby realizing the above-described processing. Of course, each processing unit may have both configurations, and a part of the above-described processing may be realized by a logic circuit, and the other may be realized by executing a program. The configuration of each processing unit may be independent from each other. For example, some of the processing units may realize a part of the above-described processing by a logic circuit, and some of the other processing units may execute a program. May be implemented, and another processing unit may implement the above-described processing by both the logic circuit and the execution of the program.

<Inverse primary horizontal conversion unit>
FIG. 35 is a block diagram illustrating a main configuration example of the inverse primary horizontal conversion unit 463 of FIG. As shown in FIG. 35, the inverse primary horizontal conversion unit 463 includes a signal sequence extraction unit 481, an inverse one-dimensional conversion unit 482, a scaling unit 483, a clip unit 484, and a two-dimensional data sequence generation unit 485.

The signal sequence extraction unit 481 performs a process related to signal sequence extraction. For example, the signal sequence extraction unit 481 acquires and stores input coefficient data Xin (transform coefficient Coeff_IPver after inverse primary vertical transform) of a two-dimensional data sequence (matrix) input to the inverse primary horizontal transform unit 463. Signal sequence extraction unit 481, each line of the input coefficient data Xin extracted one line, and supplies the inverse one-dimensional transform unit 482 as a one-dimensional signal sequence X _1.

The inverse one-dimensional conversion unit 482 performs a process related to the inverse one-dimensional conversion. For example, the inverse one-dimensional conversion unit 482 acquires the conversion type identifier TrTypeIdxH of the inverse primary horizontal conversion and the information (log2TBWSize and log2TBHSize) related to the size of the conversion block, which are supplied from the inverse primary conversion selection unit 461. Also, inverse one-dimensional transform unit 482 obtains a one-dimensional signal sequence X ₁ supplied from the signal sequence extraction unit 481. Inverse one-dimensional transform unit 482, for the one-dimensional signal sequence X _1, and conversion type identifier TrTypeIdxH inverse primary horizontal transform, an inverse one-dimensional transform corresponding to the information (Log2TBWSize and Log2TBHSize) about the size of the transform block performs , and it generates a one-dimensional signal sequence X _2. Inverse one-dimensional transform unit 482 supplies the one-dimensional signal sequence X ₂ to the scaling unit 483.

The scaling unit 483 performs processing related to scaling. For example, the scaling unit 483 obtains a one-dimensional signal sequence X ₂ supplied from the inverse one-dimensional transform unit 482. Scaling unit 483, the coefficients of the one-dimensional signal sequence X _2, scaled by a predetermined shift amount invShift2 generating a one-dimensional signal sequence X _3. Scaling unit 483 supplies the one-dimensional signal sequence X ₃ in the clip portion 484.

The clip unit 484 performs processing related to clip processing. For example, the clip portion 484 obtains a one-dimensional signal sequence X ₃ supplied from the scaling unit 483. Clip portion 484, each coefficient of the one-dimensional signal sequence X _3, clipped using the minimum value minCoefVal and maximum MaxCoefVal, to generate a one-dimensional signal sequence X _4. Clip portion 484 supplies the one-dimensional signal sequence X ₄ in the two-dimensional data string generator 485.

The two-dimensional data string generation unit 485 performs processing related to generation of a two-dimensional data string. For example, 2-dimensional data string generator 485 stores a one-dimensional signal sequence X ₄ supplied from the clip portion 484. 2-dimensional data string generator 485 generates an output coefficient data Xout is collectively two-dimensional data sequence to its one-dimensional signal sequence X ₄ by a predetermined number. The two-dimensional data sequence generation unit 485 converts the output coefficient data Xout (prediction residual D ′, the transform coefficient Coeff_IPhor after inverse primary horizontal transform, or the transform coefficient Coeff_IP after inverse primary transform) into the inverse primary horizontal transform unit 463. Output to the outside (supply to the operation unit 415).

Each processing unit of the signal sequence extraction unit 481 to the two-dimensional data sequence generation unit 485 has an arbitrary configuration. For example, each processing unit may be configured by a logic circuit that realizes the above-described processing. In addition, each processing unit may include, for example, a CPU, a ROM, a RAM, and the like, and may execute the program by using the CPU, the ROM, the RAM, and the like, thereby realizing the above-described processing. Of course, each processing unit may have both configurations, and a part of the above-described processing may be realized by a logic circuit, and the other may be realized by executing a program. The configuration of each processing unit may be independent from each other. For example, some of the processing units may realize a part of the above-described processing by a logic circuit, and some of the other processing units may execute a program. May be implemented, and another processing unit may implement the above-described processing by both the logic circuit and the execution of the program.

<Application of this technology>
In the image decoding device 400 configured as described above, for example, the inverse one-dimensional conversion unit 472 (FIG. 34) or the inverse one-dimensional conversion unit 482 (FIG. 35) is used as the inverse conversion device 150 described in the second embodiment. (FIG. 9) may be applied. In addition, the inverse transform device 150 (FIG. 19) described in the fourth embodiment may be applied as the inverse one-dimensional transform unit 472 (FIG. 34) or the inverse one-dimensional transform unit 482 (FIG. 35). . Further, the inverse transform device 150 described in the fifth embodiment may be applied as the inverse one-dimensional transform unit 472 (FIG. 34) or the inverse one-dimensional transform unit 482 (FIG. 35).

That is, for example, inverse one-dimensional transform unit 472 and the inverse one-dimensional transform unit 482, for one-dimensional signal sequence X _1, as described with reference to the tables, and the like in FIG. 10, inverse one-dimensional of each conversion type Perform the conversion. Further, for example, the inverse one-dimensional conversion unit 472 and the inverse one-dimensional conversion unit 482 derive the base conversion matrix used for the matrix operation as described with reference to the table in FIG.

With such a configuration, the inverse one-dimensional conversion unit 472 and the inverse one-dimensional conversion unit 482 can decode coefficient data obtained by decoding a bit stream in which image data is encoded (the conversion coefficient Coeff_IS after inverse secondary conversion). ), The same effect as in the second, fourth, or fifth embodiment can be obtained. Can be. That is, the image decoding apparatus 400 can suppress the configuration of the inverse one-dimensional conversion from becoming complicated (simplify the configuration). That is, the image decoding apparatus 400 can perform the inverse one-dimensional conversion more easily. Therefore, the image decoding device 400 can suppress an increase in circuit scale and processing load, and can suppress an increase in mounting cost.

<Image decoding process flow>
Next, the flow of each process executed by the image decoding device 400 as described above will be described. First, an example of the flow of the image decoding process will be described with reference to the flowchart in FIG.

When the image decoding process starts, the accumulation buffer 411 acquires and stores (accumulates) the encoded data (bit stream) supplied from outside the image decoding device 400 in step S401.

In step S402, the decoding unit 412 decodes the encoded data (bit stream) to obtain a quantized transform coefficient level level. In addition, the decoding unit 412 parses (analyzes and acquires) various encoding parameters from the encoded data (bit stream) by this decoding.

In step S403, the inverse quantization unit 413 performs inverse quantization, which is an inverse process of the quantization performed on the encoding side, on the quantized transform coefficient level obtained by the process in step S402, and performs transform. Obtain the coefficient Coeff_IQ.

In step S404, the inverse orthogonal transform unit 414 performs an inverse orthogonal transform process, which is an inverse process of the orthogonal transform process performed on the encoding side, on the transform coefficient Coeff_IQ obtained by the process in step S403, and performs prediction prediction. Obtain the difference D '.

In step S405, the prediction unit 419 performs a prediction process based on the information parsed in step S402 using a prediction method designated by the encoding side, and refers to a reference image stored in the frame memory 418, and the like. Then, a predicted image P is generated.

In step S406, the calculation unit 415 adds the prediction residual D 'obtained by the processing of step S404 and the prediction image P obtained by the processing of step S405 to derive a local decoded image Rlocal.

In step S407, the in-loop filter unit 416 performs an in-loop filter process on the locally decoded image Rlocal obtained by the process in step S406.

In step S408, the reordering buffer 417 derives the decoded image R using the filtered local decoded image Rlocal obtained in the process of step S407, and arranges the order of the decoded image R group from decoding order to reproduction order. Replace. The decoded image R group rearranged in the reproduction order is output to the outside of the image decoding device 400 as a moving image.

In step S409, the frame memory 418 stores at least one of the local decoded image Rlocal obtained by the processing in step S406 and the local decoded image Rlocal obtained by the filtering in step S407. .

(4) When the processing in step S409 ends, the image decoding processing ends.

<Process flow of inverse orthogonal transformation>
Next, an example of the flow of the inverse orthogonal transform process performed in step S404 in FIG. 36 will be described with reference to the flowchart in FIG. When the inverse orthogonal transform process is started, the switch 451 determines in step S431 that the transform skip flag ts_flag is 2D_TS (two-dimensional transform skip mode) (for example, 1 (true)) or the transform quantization bypass flag transquant_bypass_flag Is 1 (true). When it is determined that the transform skip identifier ts_idx is 2D_TS or the transform quantization bypass flag is 1 (true), the inverse orthogonal transform process ends, and the process returns to FIG. In this case, the inverse orthogonal transform processing (the inverse primary transform and the inverse secondary transform) is omitted, and the transform coefficient Coeff_IQ is set to the prediction residual D ′.

In step S431, it is determined that the conversion skip identifier ts_idx is not 2D_TS (a mode other than the two-dimensional conversion skip) (for example, 0 (false)) and the conversion quantization bypass flag is 0 (false). If so, the process proceeds to step S432. In this case, an inverse secondary conversion process and an inverse primary conversion process are performed.

In step S432, the inverse secondary transform unit 452 performs an inverse secondary transform process on the transform coefficient Coeff_IQ based on the secondary transform identifier st_idx to derive and output a transform coefficient Coeff_IS.

In step S433, the inverse primary transform unit 453 performs an inverse primary transform process on the transform coefficient Coeff_IS, and derives a transform coefficient Coeff_IP (prediction residual D ′) after the inverse primary transform.

(6) When the process in step S433 ends, the inverse orthogonal transform process ends, and the process returns to FIG.

<Flow of reverse primary conversion process>
Next, an example of the flow of the inverse primary conversion process executed in step S433 in FIG. 37 will be described with reference to the flowchart in FIG.

When the inverse primary conversion process is started, the inverse primary conversion selecting unit 461 (FIG. 33) of the inverse primary conversion unit 453, in step S441, converts the inverse primary vertical conversion conversion type identifier TrTypeIdxV (or the conversion type TrTypeV) with the inverse type. The conversion type identifier TrTypeIdxH (or the conversion type TrTypeH) of the primary horizontal conversion is selected.

In step S442, the inverse primary vertical conversion unit 462 performs an inverse primary vertical conversion process corresponding to the conversion type identifier TrTypeIdxV of the inverse primary vertical conversion obtained in step S441 on the conversion coefficient Coeff_IS after the inverse secondary conversion. The conversion coefficient Coeff_IPver after the primary vertical conversion is derived.

In step S443, the inverse primary horizontal conversion unit 463 performs an inverse primary horizontal conversion process corresponding to the conversion type identifier TrTypeIdxH of the inverse primary horizontal conversion obtained in step S441 on the conversion coefficient Coeff_IPver after the inverse primary vertical conversion. The transformation coefficient Coeff_IPhor after the inverse primary horizontal transformation (that is, the transformation coefficient Coeff_IP (prediction residual D ′) after the inverse primary transformation) is derived.

すると When the processing in step S443 ends, the inverse primary conversion processing ends, and the processing returns to FIG.

<Flow of inverse primary vertical conversion process>
Next, an example of the flow of the inverse primary vertical conversion process executed in step S442 in FIG. 38 will be described with reference to the flowchart in FIG.

When the inverse primary vertical conversion process is started, the signal sequence extraction unit 471 (FIG. 34) of the inverse primary vertical conversion unit 462 determines in step S451 that input coefficient data Xin (conversion after inverse secondary conversion) is a two-dimensional data sequence. The coefficient Coeff_IS is obtained and stored (held).

In step S452, the signal sequence extraction unit 471, for example, as in the above Expression (33), extracts the column to be processed of the input coefficient data Xin holding the (j) as a one-dimensional signal sequence X _1.

In step S453, inverse one-dimensional transform unit 472 performs an inverse transform process, using a base transform matrix T _base in accordance with the converted type identifier trTypeIdxV the transform size (NTBS), reverse 1 for a one-dimensional signal sequence X ₁ Perform dimension conversion.

In step S454, the scaling unit 473, for example, as shown in the following expression (38), scaling the coefficients X ₂ of the one-dimensional signal sequence X ₂ [i] in the shift amount InvShift1, a one-dimensional signal sequence X ₃ Derive.

In step S455, the clip portion 474, for example, as the above equation (31), the coefficients of the one-dimensional signal sequence X _₃ X ₃ [i], clipped between the minimum minCoefVal and maximum maxCoefVal , to derive a one-dimensional signal sequence X _4.

In step S456, the two-dimensional data array generating unit 475 generates two-dimensional data string Xout by using a one-dimensional signal sequence X _4. That is, two-dimensional data string generator 475, holding a one-dimensional signal sequence X ₄ and (storage), as above Expression (35), by assembling a predetermined column fraction 1 dimensional signal sequence X ₄ A two-dimensional data sequence Xout is generated.

(2) In step S457, the two-dimensional data string generation unit 475 determines whether or not each processing in steps S452 to S457 has been performed for all the columns. That is, each processing of steps S452 to S457 is performed for each column of the input data Xin held in step S451. The two-dimensional data sequence generation unit 475 determines whether all the columns have been processed.

If it is determined that there is an unprocessed column, the process returns to step S452, and the subsequent process is repeated for the next unprocessed column. If it is determined in step S457 that all columns have been processed, the primary vertical conversion process ends, and the process returns to FIG.

<Flow of inverse primary horizontal conversion process>
Next, an example of the flow of the inverse primary horizontal conversion process executed in step S443 in FIG. 38 will be described with reference to the flowchart in FIG.

When the inverse primary horizontal conversion process is started, the signal sequence extraction unit 481 (FIG. 35) of the inverse primary horizontal conversion unit 463 determines in step S461 that the input coefficient data Xin (two-dimensional data sequence) A conversion coefficient (Coeff_IPver) is acquired and stored (held).

In step S462, the signal sequence extraction unit 481, for example, as in the above Expression (29), and extracts processed line of the input coefficient data Xin holding the (j) as a one-dimensional signal sequence X _1.

In step S463, the inverse one-dimensional conversion unit 482 performs an inverse conversion process, and uses the conversion type identifier trTypeIdxH and the transposed matrix T _base ^t of the base conversion matrix corresponding to the conversion size (nTbS) to generate the one-dimensional signal sequence X performing inverse one-dimensional transform for _1.

In step S464, the scaling unit 483, for example, as shown in the following expression (39), scaling the coefficients X ₂ of the one-dimensional signal sequence X ₂ [i] in the shift amount InvShift2, a one-dimensional signal sequence X ₃ Derive.

In step S465, the clipping unit 484 clips each coefficient X ₃ [i] of the one-dimensional signal sequence X ₃ between the minimum value minCoefVal and the maximum value maxCoefVal, for example, as in Expression (31) above. , to derive a one-dimensional signal sequence X _4.

In step S466, the two-dimensional data array generating unit 485 generates two-dimensional data string Xout by using a one-dimensional signal sequence X _4. That is, two-dimensional data string generator 485, as shown in Equation (32) described above, holds the one-dimensional signal sequence X ₄ and (storage), by assembling a predetermined column fraction 1 dimensional signal sequence X ₄ A two-dimensional data sequence Xout is generated.

(2) In step S467, the two-dimensional data string generation unit 485 determines whether or not the processing in steps S462 to S467 has been performed for all rows. That is, each processing of steps S462 to S467 is performed for each row of the input data Xin held in step S461. The two-dimensional data string generation unit 485 determines whether or not all the rows have been processed.

If it is determined that there is an unprocessed line, the process returns to step S462, and the subsequent processes are repeated with the next unprocessed line as a processing target. If it is determined in step S467 that all the rows have been processed, the two-dimensional data sequence generation unit 485 generates the two-dimensional data sequence Xout (the prediction residual D ′, the conversion coefficient Coeff_IPhor after inverse primary horizontal conversion, Alternatively, the conversion coefficient Coeff_IP after the inverse primary conversion is output to the outside of the inverse primary horizontal conversion unit 463 (supplied to the calculation unit 415). When the two-dimensional data string Xout is output, the inverse primary horizontal conversion processing ends, and the processing returns to FIG.

<Application of this technology>
In step S453 of the above-described inverse primary vertical conversion process (FIG. 39), for example, the inverse one-dimensional conversion unit 472 executes the inverse conversion process in the same flow as in the case of the second embodiment (FIG. 11). You may make it. Further, the inverse one-dimensional conversion unit 472 may execute the inverse conversion process in the same flow as in the case of the fourth embodiment (FIG. 20). Further, the inverse one-dimensional conversion unit 472 may execute the inverse conversion process in the same flow as in the fifth embodiment.

In addition, in step S463 of the above-described inverse primary horizontal conversion process (FIG. 40), for example, the inverse one-dimensional conversion unit 482 performs the inverse conversion process in the same flow as in the case of the second embodiment (FIG. 11). May be executed. Further, the inverse one-dimensional conversion unit 482 may execute the inverse conversion process in the same flow as in the case of the fourth embodiment (FIG. 20). Further, the inverse one-dimensional conversion unit 482 may execute the inverse conversion process in the same flow as in the fifth embodiment.

By executing each process in this way, the inverse one-dimensional transform unit 472 and the inverse one-dimensional transform unit 482 perform inverse primary transform (in horizontal inverse transform) on coefficient data obtained by decoding a bit stream in which image data is encoded. In the direction or the vertical one-dimensional inverse one-dimensional conversion), the same effect as in the second embodiment or the fourth embodiment can be obtained. That is, the image decoding apparatus 400 can suppress the configuration of the inverse one-dimensional conversion from becoming complicated (simplify the configuration). That is, the image decoding apparatus 400 can perform the inverse one-dimensional conversion more easily. Therefore, the image decoding device 400 can suppress an increase in circuit scale and processing load, and can suppress an increase in mounting cost.

As described above, the image encoding device 300 also has the inverse orthogonal transform unit 318, has the same configuration as the inverse orthogonal transform unit 414 of the image decoding device 400, and performs the same processing. That is, the inverse orthogonal transform unit 318 also performs the inverse primary transform (the horizontal or vertical inverse one-dimensional transform in the transform coefficient) on the inversely quantized transform coefficient Coeff_IQ in the second embodiment and the fourth embodiment. Alternatively, the same effect as that of the fifth embodiment can be obtained. That is, the image encoding device 300 can suppress the complexity of the configuration of the inverse one-dimensional transform (simplify the configuration). That is, the image encoding device 300 can more easily perform the inverse one-dimensional conversion. Therefore, the image encoding device 300 can suppress an increase in circuit scale and processing load and an increase in mounting cost.

<11. Appendix>
<Computer>
The series of processes described above can be executed by hardware or can be executed by software. When a series of processing is executed by software, a program constituting the software is installed in a computer. Here, the computer includes a computer incorporated in dedicated hardware, a general-purpose personal computer that can execute various functions by installing various programs, and the like.

FIG. 41 is a block diagram showing a configuration example of hardware of a computer that executes the above-described series of processing by a program.

In the computer 800 shown in FIG. 41, a CPU (Central Processing Unit) 801, a ROM (Read Only Memory) 802, and a RAM (Random Access Memory) 803 are interconnected via a bus 804.

The input / output interface 810 is also connected to the bus 804. An input unit 811, an output unit 812, a storage unit 813, a communication unit 814, and a drive 815 are connected to the input / output interface 810.

The input unit 811 includes, for example, a keyboard, a mouse, a microphone, a touch panel, an input terminal, and the like. The output unit 812 includes, for example, a display, a speaker, an output terminal, and the like. The storage unit 813 includes, for example, a hard disk, a RAM disk, a nonvolatile memory, and the like. The communication unit 814 includes, for example, a network interface. The drive 815 drives a removable medium 821 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory.

In the computer configured as described above, the CPU 801 loads a program stored in the storage unit 813 into the RAM 803 via the input / output interface 810 and the bus 804, and executes the program, for example. Is performed. The RAM 803 also appropriately stores data necessary for the CPU 801 to execute various processes.

The program executed by the computer (CPU 801) can be recorded on a removable medium 821 as a package medium or the like and applied. In that case, the program can be installed in the storage unit 813 via the input / output interface 810 by attaching the removable medium 821 to the drive 815.

This program can also be provided via a wired or wireless transmission medium such as a local area network, the Internet, or digital satellite broadcasting. In that case, the program can be received by the communication unit 814 and installed in the storage unit 813.

In addition, this program can be installed in the ROM 802 or the storage unit 813 in advance.

<Unit of information and processing>
The data units in which the various types of information described above are set and the data units targeted for various types of processing are arbitrary, and are not limited to the examples described above. For example, these pieces of information and processing are respectively TU (Transform Unit), TB (Transform Block), PU (Prediction Unit), PB (Prediction Block), CU (Coding Unit), LCU (Largest Coding Unit), and sub-block. , A block, a tile, a slice, a picture, a sequence, or a component, or the data of these data units may be targeted. Of course, this data unit can be set for each information or process, and it is not necessary that all information and data units of the process be unified. The storage location of these pieces of information is arbitrary, and may be stored in the above-described data unit header or parameter set. Further, the information may be stored at a plurality of locations.

<Control information>
The control information related to the present technology described in each of the above embodiments may be transmitted from the encoding side to the decoding side. For example, control information (for example, enabled_flag) for controlling whether to apply (or prohibit) applying the present technology described above may be transmitted. Further, for example, control information indicating a target to which the present technology is applied (or a target to which the present technology is not applied) may be transmitted. For example, control information specifying a block size (upper or lower limit, or both) to which the present technology is applied (or application is permitted or prohibited), a frame, a component, a layer, or the like may be transmitted.

<Applicable target of this technology>
The present technology can be applied to any image encoding / decoding method. That is, as long as there is no contradiction with the present technology described above, the specifications of various processes related to image encoding / decoding such as conversion (inverse transformation), quantization (inverse quantization), encoding (decoding), prediction, and the like are arbitrary. However, the present invention is not limited to this example. Further, some of these processes may be omitted as long as they do not conflict with the present technology described above.

{In addition, the present technology can be applied to a multi-view image encoding / decoding system that performs encoding / decoding of a multi-view image including images of a plurality of viewpoints (views). In that case, the present technology may be applied to encoding / decoding of each viewpoint (view).

Furthermore, the present technology is applied to a hierarchical image encoding (scalable encoding) / decoding system that encodes / decodes a hierarchical image that is multi-layered (hierarchized) so as to have a scalability function for a predetermined parameter. can do. In this case, the present technology may be applied to encoding / decoding of each layer (layer).

The image processing device, the image encoding device, and the image decoding device according to the above-described embodiments are used, for example, in satellite broadcasting, cable broadcasting such as cable TV, distribution on the Internet, and distribution to terminals by cellular communication. A device (eg, a hard disk recorder) that records an image on a medium such as a transmitter or a receiver (eg, a television receiver or a mobile phone) or an optical disk, a magnetic disk, and a flash memory, and reproduces an image from these storage media And cameras).

In addition, the present technology is applicable to any configuration mounted on an arbitrary device or a device configuring a system, for example, a processor (eg, a video processor) as a system LSI (Large Scale Integration), a module using a plurality of processors (eg, video Module), a unit using a plurality of modules (eg, a video unit), a set in which other functions are added to the unit (eg, a video set), and the like (ie, a configuration of a part of the apparatus).

Furthermore, the present technology can be applied to a network system including a plurality of devices. For example, the present invention can be applied to a cloud service that provides a service related to an image (moving image) to an arbitrary terminal such as a computer, an AV (Audio Visual) device, a portable information processing terminal, and an IoT (Internet of Things) device. it can.

In addition, the system, apparatus, processing unit, etc. to which this technology is applied may be used in any fields such as traffic, medical care, crime prevention, agriculture, livestock industry, mining, beauty, factories, household appliances, weather, nature monitoring, etc. Can be. Further, its use is arbitrary.

For example, the present technology can be applied to systems and devices provided for providing ornamental content and the like. Further, for example, the present technology can also be applied to systems and devices used for traffic, such as traffic condition management and automatic driving control. Further, for example, the present technology can also be applied to systems and devices provided for security. Further, for example, the present technology can be applied to a system or a device provided for automatic control of a machine or the like. Further, for example, the present technology can also be applied to systems and devices provided for use in agriculture and livestock industry. Further, the present technology can also be applied to a system or a device that monitors a natural state such as a volcano, a forest, and the ocean, a wildlife, and the like. Further, for example, the present technology can also be applied to systems and devices provided for sports.

<Others>
In this specification, “flag” is information for identifying a plurality of states, and is not limited to information used for identifying two states of true (1) or false (0), as well as three or more. Information that can identify the state is also included. Therefore, the value that the “flag” can take may be, for example, a binary value of 1/0 or a ternary value or more. That is, the number of bits constituting the “flag” is arbitrary, and may be 1 bit or a plurality of bits. Also, the identification information (including the flag) may include not only a form in which the identification information is included in the bit stream but also a form in which the difference information of the identification information with respect to a certain reference information is included in the bit stream. In the above, “flag” and “identification information” include not only the information but also difference information with respect to reference information.

{Circle around (4)} Various types of information (metadata and the like) relating to the encoded data (bit stream) may be transmitted or recorded in any form as long as the information is associated with the encoded data. Here, the term “associate” means, for example, that one data can be used (linked) when one data is processed. That is, the data associated with each other may be collected as one data, or may be individual data. For example, the information associated with the encoded data (image) may be transmitted on a different transmission path from the encoded data (image). Further, for example, information associated with encoded data (image) may be recorded on a recording medium different from the encoded data (image) (or another recording area of the same recording medium). Good. The “association” may be a part of the data instead of the entire data. For example, an image and information corresponding to the image may be associated with each other in an arbitrary unit such as a plurality of frames, one frame, or a part of the frame.

In the present specification, “combining”, “multiplexing”, “adding”, “integrating”, “include”, “store”, “insert”, “insert”, “insert” "Means that a plurality of things are put together into one, such as putting encoded data and metadata into one data, and means one method of the above-mentioned" association ".

The embodiments of the present technology are not limited to the above-described embodiments, and various modifications can be made without departing from the gist of the present technology.

Also, for example, the configuration described as one device (or processing unit) may be divided and configured as a plurality of devices (or processing units). Conversely, the configuration described above as a plurality of devices (or processing units) may be configured as one device (or processing unit). Also, a configuration other than those described above may be added to the configuration of each device (or each processing unit). Further, if the configuration and operation of the entire system are substantially the same, a part of the configuration of a certain device (or processing unit) may be included in the configuration of another device (or other processing unit). .

In the present specification, a system means a set of a plurality of components (devices, modules (parts), etc.), and it does not matter whether all the components are in the same housing. Therefore, a plurality of devices housed in separate housings and connected via a network, and one device housing a plurality of modules in one housing are all systems. .

Also, for example, the present technology can take a configuration of cloud computing in which one function is shared by a plurality of devices via a network and processed jointly.

In addition, for example, the above-described program can be executed in any device. In that case, the device only has to have necessary functions (functional blocks and the like) and can obtain necessary information.

Also, for example, each step described in the above-described flowchart can be executed by a single device, or can be shared and executed by a plurality of devices. Further, when a plurality of processes are included in one step, the plurality of processes included in the one step can be executed by one device or can be shared and executed by a plurality of devices. In other words, a plurality of processes included in one step can be executed as a plurality of steps. Conversely, the processing described as a plurality of steps can be collectively executed as one step.

Note that the computer-executable program may be configured so that the processing of the steps for describing the program is executed in chronological order according to the order described in this specification, or may be executed in parallel or by calling. It may be executed individually at a necessary timing such as time. That is, as long as no contradiction occurs, the processing of each step may be performed in an order different from the order described above. Further, the processing of the steps for describing the program may be executed in parallel with the processing of another program, or may be executed in combination with the processing of another program.

技術 Note that the present technology, which has been described in plural in this specification, can be implemented independently and independently as long as no inconsistency arises. Of course, it is also possible to carry out the present invention by using a plurality of the present technologies in combination. For example, some or all of the present technology described in any of the embodiments may be combined with some or all of the present technology described in other embodiments. In addition, some or all of the above-described arbitrary technology may be implemented in combination with another technology that is not described above.

{100} conversion device, {101} control unit, {102} sign inversion unit, {103} matrix operation unit, {104} flip unit, {111} sign inversion flag setting unit, {112} base conversion matrix selection unit, {113} flip flag setting unit, {120} base conversion matrix LUT, {150} inverse Conversion device, {151} control unit, {152} flip unit, {153} matrix operation unit, {154} sign inversion unit, {161} flip flag setting unit, {162} base conversion matrix selection unit, {163} sign inversion flag setting unit, {170} base conversion matrix LUT, {220} base conversion Matrix derivation unit, {231} sampling unit, {232} derivation source transformation matrix LUT, {241} sampling parameter derivation unit, {242} submatrix extraction unit, {270} base transformation matrix derivation unit, {300} image encoding device, 301 control unit, {313} orthogonal transform unit, {315} encoding unit, {318} inverse orthogonal transform unit, {352} primary transform unit, {353} secondary transform unit, {362} primary horizontal transform unit, {363} primary vertical transform unit, {371} signal sequence extractor, {372} 1 Dimension conversion unit, {373} scaling unit, {374} clipping unit, {375} two-dimensional data sequence generation unit, {381} signal sequence extraction unit, {382} one-dimensional conversion unit, {383} scaling unit, {384} clipping unit, {385} two-dimensional data sequence generation unit, {400} image Decoding device, {412} decoding unit, {414} inverse orthogonal transform unit, {452} inverse secondary transform unit, {453} inverse primary transform unit, {461} inverse primary transform selecting unit, {462} inverse primary vertical transform unit, {463} inverse transform Imari horizontal conversion unit, {471} signal sequence extraction unit, {472} inverse one-dimensional conversion unit, {473} scaling unit, {474} clip unit, {475} two-dimensional data sequence generation unit, {481} signal sequence extraction unit, {482} inverse one-dimensional conversion unit, {483} scaling unit , {484} clip part, {485} two-dimensional data string generation part

Claims

A decoding unit that decodes the bit stream and generates coefficient data related to the image;
A one-dimensional signal sequence of the coefficient data generated by the decoding unit, a flip unit performing a flip operation to rearrange the order of each coefficient in reverse order,
For the one-dimensional signal sequence subjected to the flip operation by the flip unit,
When realizing the inverse one-dimensional conversion of the first conversion type, a conversion matrix of the second conversion type that realizes the inverse one-dimensional conversion of the first conversion type by an STF operation is a base conversion matrix,
When realizing the inverse one-dimensional conversion of the third conversion type, a base transformation is performed on a conversion matrix that is a symmetric matrix of the fourth conversion type that realizes the inverse one-dimensional conversion of the third conversion type by an FTS operation. Matrix
A matrix operation unit that performs a matrix operation using the transpose of the base transformation matrix,
A sign inverting unit that performs a sign inverting operation to invert the sign of an odd-numbered signal of the one-dimensional signal sequence on the one-dimensional signal sequence on which the matrix operation has been performed by the matrix operation unit. .
When implementing the inverse one-dimensional conversion of the second conversion type or the fourth conversion type,
The flip section skips the flip operation,
The matrix operation unit performs the matrix operation on a one-dimensional signal sequence of the coefficient data generated by the decoding unit, using a conversion matrix of the second conversion type or the fourth conversion type as a base conversion matrix. Do
The image processing device according to claim 1, wherein the sign inverting unit skips the sign inversion operation.
A flip flag setting unit that sets a flip flag indicating whether or not to perform the flip operation, based on the designated inverse one-dimensional conversion type;
A sign inversion flag setting unit that sets a sign inversion flag indicating whether or not to perform the sign inversion operation based on the conversion type.
The flip unit performs or skips the flip operation based on the flip flag set by the flip flag setting unit,
The image processing device according to claim 2, wherein the sign inversion unit performs or skips the sign inversion operation based on the sign inversion flag set by the sign inversion flag setting unit.
A base conversion matrix for selecting which of the second conversion type conversion matrix and the fourth conversion type conversion matrix is to be the base conversion matrix based on the specified inverse one-dimensional conversion conversion type Further comprising a selection unit,
The image processing device according to claim 1, wherein the matrix operation unit performs the matrix operation using the base transformation matrix selected by the base transformation matrix selection unit.
A base transformation matrix deriving unit that derives the base transformation matrix based on a designated inverse one-dimensional transformation type;
The image processing device according to claim 1, wherein the matrix operation unit performs the matrix operation using the base conversion matrix derived by the base conversion matrix derivation unit.
The image processing device according to claim 5, wherein the base transformation matrix derivation unit derives the base transformation matrix using a derivation source transformation matrix of the second transformation type having a size equal to or larger than the base transformation matrix.
The image processing device according to claim 6, wherein the base transformation matrix deriving unit derives the base transformation matrix of the second transformation type or the fourth transformation type by sampling the derived transformation matrix.
A one-dimensional signal sequence extracting unit that extracts a one-dimensional signal sequence from the coefficient data generated by the decoding unit;
A two-dimensional data sequence generation unit that generates a two-dimensional data sequence using the one-dimensional signal sequence on which the sign inversion operation has been performed by the sign inversion unit;
The image processing device according to claim 1, wherein the flip unit performs the flip operation on the one-dimensional signal sequence extracted by the one-dimensional signal sequence extraction unit.
The first conversion type is DST2;
The second transform type is DCT2;
The third conversion type is DST4;
The image processing device according to claim 1, wherein the fourth conversion type is DCT4.
Decoding the bitstream to generate coefficient data for the image,
A flip operation is performed on the generated one-dimensional signal sequence of the coefficient data to rearrange the order of each coefficient in reverse order,
For the one-dimensional signal sequence subjected to the flip operation,
When realizing the inverse one-dimensional conversion of the first conversion type, a conversion matrix of the second conversion type that realizes the inverse one-dimensional conversion of the first conversion type by an STF operation is a base conversion matrix,
When realizing the inverse one-dimensional conversion of the third conversion type, a base transformation is performed on a conversion matrix that is a symmetric matrix of the fourth conversion type that realizes the inverse one-dimensional conversion of the third conversion type by an FTS operation. Matrix
Perform a matrix operation using the transposed matrix of the base transformation matrix,
An image processing method for performing a sign inversion operation for inverting a sign of an odd-numbered signal of the one-dimensional signal sequence on the one-dimensional signal sequence on which the matrix operation has been performed.
A sign inverting unit that performs a sign inverting operation for inverting the sign of an odd-numbered signal of the one-dimensional signal string with respect to the one-dimensional signal string of coefficient data relating to an image;
For the one-dimensional signal sequence subjected to the sign inversion operation by the sign inversion unit,
When realizing a one-dimensional conversion of the first conversion type, a conversion matrix of a second conversion type that realizes the one-dimensional conversion of the first conversion type by an FTS operation is a base conversion matrix,
When a one-dimensional conversion of the third conversion type is realized, a conversion matrix that is a symmetric matrix of a fourth conversion type that realizes the one-dimensional conversion of the third conversion type by an STF operation is defined as a base conversion matrix. ,
A matrix operation unit that performs a matrix operation using the base conversion matrix,
For the one-dimensional signal sequence on which the matrix operation has been performed by the matrix operation unit, a flip unit that performs a flip operation of rearranging the order of each coefficient in reverse order,
An encoding unit that encodes coefficient data including the one-dimensional signal sequence on which the flip operation has been performed by the flip unit and generates a bit stream.
When implementing the one-dimensional conversion of the second conversion type or the fourth conversion type,
The sign inversion unit skips the sign inversion operation,
The matrix operation unit performs the matrix operation on the one-dimensional signal sequence of the coefficient data, using a conversion matrix of the second conversion type or the fourth conversion type as a base conversion matrix,
The image processing device according to claim 11, wherein the flip unit skips the flip operation.
A sign inversion flag setting unit that sets a sign inversion flag indicating whether to perform the sign inversion operation based on a designated conversion type of the one-dimensional conversion;
A flip flag setting unit that sets a flip flag indicating whether to perform the flip operation based on the conversion type,
The sign inversion unit performs or skips the sign inversion operation based on the sign inversion flag set by the sign inversion flag setting unit,
The image processing device according to claim 12, wherein the flip unit performs or skips the flip operation based on the flip flag set by the flip flag setting unit.
Base conversion matrix selection for selecting which of the second conversion type conversion matrix and the fourth conversion type conversion matrix is to be the base conversion matrix based on the specified one-dimensional conversion type conversion type Part further,
The image processing device according to claim 11, wherein the matrix operation unit performs the matrix operation using the base conversion matrix selected by the base conversion matrix selection unit.
A base conversion matrix deriving unit that derives the base conversion matrix based on the specified conversion type of the one-dimensional conversion,
The image processing device according to claim 11, wherein the matrix operation unit performs the matrix operation using the base conversion matrix derived by the base conversion matrix derivation unit.
The image processing device according to claim 15, wherein the base transformation matrix deriving unit derives the base transformation matrix using a derivation source transformation matrix of the second transformation type having a size equal to or larger than the base transformation matrix.
The image processing device according to claim 16, wherein the base transformation matrix deriving unit derives the base transformation matrix of the second transformation type or the fourth transformation type by sampling the derived transformation matrix.
A one-dimensional signal sequence extracting unit for extracting a one-dimensional signal sequence from the coefficient data;
A two-dimensional data sequence generation unit that generates a two-dimensional data sequence using the one-dimensional signal sequence subjected to the flip operation by the flip unit;
The image processing device according to claim 11, wherein the sign inverting unit performs the sign inverting operation on the one-dimensional signal sequence extracted by the one-dimensional signal sequence extracting unit.
The first conversion type is DST2;
The second transform type is DCT2;
The third conversion type is DST4;
The image processing device according to claim 11, wherein the fourth conversion type is DCT4.
Performing a sign inversion operation for inverting the sign of the odd-numbered signal of the one-dimensional signal sequence on the one-dimensional signal sequence of the coefficient data relating to the image,
For the one-dimensional signal sequence subjected to the sign inversion operation,
When realizing a one-dimensional conversion of the first conversion type, a conversion matrix of a second conversion type for realizing the one-dimensional conversion of the first conversion type by an FTS operation is set as a base conversion matrix,
When a one-dimensional conversion of the third conversion type is realized, a conversion matrix that is a symmetric matrix of a fourth conversion type that realizes the one-dimensional conversion of the third conversion type by an STF operation is defined as a base conversion matrix. ,
Perform a matrix operation using the base transformation matrix,
For the one-dimensional signal sequence subjected to the matrix operation, perform a flip operation to rearrange the order of each coefficient in reverse order,
An image processing method for encoding coefficient data including the one-dimensional signal sequence subjected to the flip operation and generating a bit stream.