JP2014220624A - Image processing apparatus, encoding apparatus, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processing apparatus, an encoding apparatus, and a program which improve processing efficiency in selection of proper orthogonal transformation from plural orthogonal transformations.SOLUTION: The image processing apparatus, which selects one of a first orthogonal transformation and one or more other orthogonal transformations and performs processing, comprises: an analyzer which analyzes a first orthogonal transformation coefficient obtained by the first orthogonal transformation of image data, in accordance with a predetermined rule; and a selector which selects one of the first orthogonal transformation and the one or more other orthogonal transformations, on the basis of a result of analysis by the analyzer.

Description

  The present invention relates to an image processing apparatus, an encoding apparatus, and a program that adaptively select orthogonal transform in image encoding.

  A pixel value of a frame image such as a moving image is orthogonally transformed and represented by an orthogonal transformation coefficient. By performing quantization and entropy coding on the orthogonal transform coefficient as necessary, the amount of information is compressed without reducing image quality as much as possible. In particular, this image encoding technique is used in the field of video encoding.

  The main purpose of using the orthogonal transform is to eliminate the spatial correlation of images and concentrate the information (energy) of the video signal on a small number of orthogonal transform coefficients. The frame image is divided into small areas, and orthogonal transformation is performed for each block of 8 × 8 pixels, for example. In this encoding, DCT (Discrete Cosine Transform) capable of consolidating information in a region close to a DC component is most widely used.

  Optimal orthogonal transformation for an input image can be realized by designing an orthogonal basis that has been decorrelated by principal component analysis. However, in reality, it is difficult to design an orthogonal basis in real time according to an input signal. As transformation using principal component analysis, KL transformation (KLT: Karhuen Loeve transform) is known. KL conversion provides optimal conversion efficiency for an input signal. However, hardware implementation is very difficult, and currently it is not adopted as an encoding method.

  Therefore, conventionally, when an image is encoded, orthogonal transformation optimized by the statistical properties of an input signal known in advance is used. For this reason, orthogonal transform is predetermined for each image coding method.

  Typical orthogonal transforms used in image coding techniques include DST (Discrete Sine Transform), DWT (Discrete Wavelet Transform), DFT (Discrete Fourier Transform), DHT (Hadamard Transform), etc. in addition to the above-mentioned DCT. It has been.

  There are eight standard types of DCT and DST, respectively. In DCT, the most common type is DCT-II (type 2). The inverse transform of DCT-II is DCT-III. DCT-I and DCT-IV themselves are inversely transformed. DCT and DST have different properties depending on their types.

  Note that these transformations are generally called orthogonal transformations. However, when implemented, integer precision calculation is adopted or rounding error occurs, so the orthogonality in the strict sense is not guaranteed. In the present specification, the term “orthogonal transformation” is used for explanation. However, it should be noted that the term does not mean only a transformation in which orthogonality in a strict sense is ensured.

  MPEG is known as a widely used video encoding. MPEG-2 and H.264 In almost all image coding techniques such as H.264 / AVC (Advance Video Coding) (for example, see Non-Patent Document 1) and HEVC, an input signal is converted into a frequency domain, and signals in the conversion domain are converted using various processes. Realizes capacity compression.

  Note that in the video encoding scheme HEVC, DST-VII (type 7) orthogonal transform is used for intra-frame coding (intra-screen coding) in a block of 4 × 4 pixels. .

  Therefore, HEVC intra coding uses two DCT-II and DST-VII as orthogonal transforms. On the other hand, in inter coding, since the residual signal does not have a characteristic tendency as in intra coding, coding using only DCT-II is used.

  In addition, intra coding (intra coding) encodes an image by performing spatial signal prediction in the screen. This technique is also referred to as intra prediction (intra prediction). Also, inter-screen prediction (inter-coding) searches for similar regions between different frames and encodes an image for prediction. This technique is also referred to as motion compensation prediction (inter prediction).

  In the method of selectively switching between DCT-II and DST-VII adopted in the above-mentioned HEVC, the signal bias by the intra prediction method using extrapolation is statistically analyzed by prior analysis. Then, which orthogonal transform is selected for each intra prediction mode is determined in advance on the table (see Non-Patent Document 3).

  Further, it has been reported that further improvement in coding efficiency can be expected by using various other orthogonal transforms (DST-II, DCT-III, DCT-IV, DCT-VII, etc.) adaptively ( Non-patent document 2).

  However, since the input signal to be actually encoded may differ from the statistically known tendency, the predetermined orthogonal transform may not be an appropriate orthogonal transform.

  In order to solve such a problem, it is possible to select a more appropriate orthogonal transform by a technique such as an RD (Rate-Distortion) optimization method, for example, as shown below. However, it is necessary to perform processing until each prepared orthogonal transform becomes a compressed signal in advance by a completely or simplified method.

  FIG. 1 shows an example in which two types of orthogonal transforms are selected. A first orthogonal transform is performed on the input signal 102 by the first orthogonal transform unit 112. Thereafter, the entropy encoding unit 114 performs entropy encoding. By this processing, a compressed signal 116 is obtained. Then, the second orthogonal transform unit 122 performs second orthogonal transform on the input signal 102. Thereafter, the entropy encoding unit 124 performs entropy encoding. A compressed signal 126 is obtained by this processing.

  The compressed signal 116 and the compressed signal 126 are input to the RD determination unit 130. The RD determination unit 130 determines which orthogonal transform is appropriate by using an RD optimization method that is used for optimization in consideration of both information amount (Rate) and distortion (Distortion). In this case, almost twice as many resources are required as compared with the case of using one type of orthogonal transform (see Non-Patent Document 4).

Supervised by Satoshi Okubo, “Revised 3rd edition H.264 / AVC textbook”, Impress R & D, p124, January 1, 2009 JCTVC-E027 CE-7: Summary report of core experiment on alternative transforms) (March 2011) A. Saxena and F. C. Fernandes, "Mode dependent DCT / DST for intra prediction in block-based image / video coding" 18th IEEE International Conference on Image Processing, PP / 1721-1724, 2011 Goro Ichigaya, Yasuko Sugito, Shinichi Sakaida, "Inter-coding using DCT and DST adaptively", Eiji-gakugakuho, vol.36, no.9 ME2012-59, PP.213-218,2012

  As described above, when an appropriate one of a plurality of orthogonal transforms is selected, executing each orthogonal transform increases processing performance and power consumption, and is not realistic. For this reason, in consideration of implementation in real-time signal processing, a technique for selecting an optimal orthogonal transform at high speed is required.

  Therefore, an object of the present invention is to provide an image processing device, an encoding device, and a program that improve processing efficiency when selecting an appropriate orthogonal transform from a plurality of orthogonal transforms.

  An image processing apparatus according to an aspect of the present invention is an image processing apparatus that performs processing by selecting any one of a first orthogonal transform and one or more other orthogonal transforms, An analysis unit that analyzes the first orthogonal transform coefficient obtained by the first orthogonal transform according to a predetermined rule, the first orthogonal transform based on the analysis result of the analysis unit, and the 1 And a selection unit that selects any one of the other orthogonal transforms.

  Further, the predetermined rule is that an energy ratio between an odd-order and an even-order in the horizontal direction of the first orthogonal transform coefficient and / or an odd-order and an even-order in the vertical direction of the first orthogonal transform coefficient. An energy ratio may be used.

  In addition, the analysis unit stores the orthogonal transform coefficients obtained by transforming each of a predetermined number of the plurality of base images included in each of the one or more other orthogonal transforms by the first orthogonal transform. A similarity calculation unit that calculates a similarity between the first orthogonal transform coefficient and the plurality of orthogonal transform coefficients stored in the storage unit, and the selection unit includes the similarity calculation unit If the highest similarity among the similarities obtained by the above-mentioned is over a predetermined threshold, the image data is selectively given to the other orthogonal transformation corresponding to the highest similarity, When the orthogonal transform coefficient obtained by transforming the image data is used for encoding the image data, and the highest similarity is equal to or less than the predetermined threshold, the first orthogonal transform coefficient is used as the image data. May be used for encoding

  In addition, an encoding device including any one of the above image processing devices is provided.

  A program according to another embodiment of the present invention causes a computer to function as the encoding device.

  According to the present invention, it is possible to improve efficiency when performing image processing by selecting an appropriate orthogonal transform from a plurality of orthogonal transforms.

It is a figure which shows the example in the case of selecting two types of orthogonal transformation in a prior art. It is a graph which shows the example of the orthogonal transformation coefficient of DCT and DST. 1 is a block diagram illustrating an example of a configuration of an image processing apparatus according to a first embodiment. It is a figure which shows the example of the horizontal component of the orthogonal transformation coefficient of DCT and DST. 3 is a flowchart illustrating an example of image processing according to the first exemplary embodiment. 2 is a diagram illustrating an example of a basic principle of Embodiment 1. FIG. FIG. 10 is a block diagram illustrating an example of a configuration of an image processing apparatus according to a second embodiment. It is a figure explaining the pattern for orthogonal transformation coefficient comparison memorize | stored in a memory | storage part. 10 is a flowchart illustrating an example of image processing in the second embodiment. 10 is a block diagram illustrating an example of a schematic configuration of an image processing apparatus according to Embodiment 3. FIG. FIG. 10 is a block diagram illustrating an example of a schematic configuration of an image processing device according to a fourth embodiment.

  First, taking DCT and DST as an example, an example of the relationship between different orthogonal transforms is shown. The energy distribution in the DCT region is obtained as Equation (1). For example, assuming that a sine wave signal orthogonal to the DCT base is input, the input signal is a continuous signal for simplicity, and the normalization coefficient is also omitted. In addition, a one-dimensional signal is taken up.

入力信号を式（２）とすると、

If the input signal is expressed by equation (2),

式（１）は、式（３）のように表せる。

Expression (1) can be expressed as Expression (3).

この式（３）を用いて、以下の２つの場合についてＤＣＴを計算する。
ｉ）u_i+1＝uの場合

Using this equation (3), DCT is calculated for the following two cases.
i) When u _i + 1 = u

ｉｉ）u_i+1≠uの場合
ア）u_i+u＝2n−1の場合

ii) When u _i + 1 ≠ u a) When u _i + u = 2n−1

イ）u_i+u＝2nの場合

B) When u _i + u = 2n

したがって、式（４）、式（５）、式（６）より、正弦波信号の次数u_iとＤＣＴ基底の次数uの和が偶数となる次数に、ＤＣＴ係数のエネルギーが広く分布することが分かる。

Therefore, from Equations (4), (5), and (6), the energy of the DCT coefficient is widely distributed in the order in which the sum of the order u _i of the sine wave signal and the order u of the DCT base is an even number. I understand.

FIG. 2 shows DCT coefficients and DST coefficients when a sine wave signal (u _i = 1) is input. FIG. 2 (A) shows the DCT coefficient, and a spectrum having a large absolute value is distributed on the coefficient of u (u = 1, 3, 5, 7) that makes the sum of u _i = 1 even. Yes. Here, the spectrum means an orthogonal transform coefficient.

FIG. 2B shows a spectrum of DST. In this case, the spectrum appears only when u _i = 1. Therefore, the input signal can be reconstructed even if the third and subsequent coefficients of the DST coefficient are removed.

  Returning to FIG. 2A, in this case, unlike FIG. 2B, the spectrum concentration of DCT coefficients is low and the spectrum is widely distributed. It can be seen that DCT has a complementary property to DST.

As can be seen from the above example, for a sine wave signal (u _i = 1), DST can collect information in fewer spectral components than DCT, and from the viewpoint of encoding efficiency, Is appropriate.

[Example 1]
In the first embodiment, the characteristics of the two orthogonal transforms of DCT and DST described above are used. Then, which orthogonal transformation is an appropriate orthogonal transformation can be specified by a simple method.

<Configuration>
FIG. 3 is a block diagram illustrating an example of the image processing apparatus 300 according to the first embodiment. In the example illustrated in FIG. 3, the image processing apparatus 300 includes a first orthogonal transform unit 320, an analysis unit 322, a selection unit 330, a second orthogonal transform unit 340, a first switch 328, and a second switch 329. Have. An outline of each part will be described below.

  The first orthogonal transform unit 320 transforms the input image 341 by the first orthogonal transform. Note that the input image 341 may be a prediction error signal obtained by subtracting the predicted image from the original image. The unit to be converted is a unit divided into blocks such as 32 × 32, 16 × 16, and 8 × 8 pixels that are a part of the input image. The first orthogonal transform unit 320 may have a function of executing DCT, for example.

  The analysis unit 322 receives the first orthogonal transform coefficient 321 transformed by the first orthogonal transform unit 320. Specific processing of the analysis unit 322 will be described later with reference to FIG.

  The selection unit 330 receives the analysis result 321 of the analysis unit 322. The selection unit 330 provides a selection instruction signal 327 to the first switch 328 and the second switch 329 based on the analysis result 321.

  When the selection instruction signal 327 is a signal that gives the input image 341 to the second orthogonal transform unit 340 to the first switch 328, the second orthogonal transform unit 340 performs orthogonal transform on the input image 341. Execute the process. Then, the second orthogonal transformation unit 340 outputs the second orthogonal transformation coefficient 331 to the second switch 329.

  As described above, the first orthogonal transform coefficient 321 or the second orthogonal transform coefficient 331 is selectively output to the output 351 by the selection instruction signal 327. In this case, when the first orthogonal transform coefficient is output to the output 351, the processing of the second orthogonal transform unit 340 can be omitted. In this case, resources are saved.

<Description of Analysis Unit 322>
Next, the analysis unit 322 will be described. As an example, DCT-II (hereinafter may be simply referred to as DCT) is used for the first orthogonal transform unit 320, and DST-II (hereinafter simply referred to as DST) is used for the second orthogonal transform unit 340. The case where is used is assumed. Although DST is type 2 for ease of mounting, other types may be used.

  DCT and DST have the relationship of the above-mentioned formula (4), formula (5), and formula (6). That is, an orthogonal transform coefficient obtained by transforming one of the DST base images by DCT has a power spectrum only in one of the even order and the odd order, and the other power spectrum becomes zero. Similarly, an orthogonal transform coefficient obtained by transforming one of the DCT base images with the DST has a spectrum only in one of the even order and the odd order, and the other spectrum becomes zero.

  FIG. 4 shows an example of the relationship between DST and DCT. FIG. 4 is a diagram showing a horizontal row of two-dimensional orthogonal transform coefficients. 4A and 4B are diagrams showing one of the horizontal components of the spectrum when the same image is converted by DCT and DST, respectively. The color intensity represents the magnitude of the absolute value of the spectrum.

  FIG. 4C and FIG. 4D are diagrams showing one of the horizontal components of the spectrum when the same image is converted by DCT and DST, respectively, under other conditions.

  In the signal compression coding system, it is assumed that the spectrum is generally concentrated in a low band in the orthogonal transform. Therefore, when spectrum spread occurs as shown in FIG. 4A or FIG. 4C, it means that the orthogonal transform (that is, DCT) does not function properly for the target image. In other words, in this case, even if DCT is used, it means that the subsequent information compression cannot be sufficiently expected because the spectrum is spread.

  On the other hand, in the case of an image in which spectrum concentration is obtained by DST as shown in FIG. 4B or FIG. 4D, it is shown that it is appropriate to use DST.

  Due to these characteristics, when a spectrum pattern similar to FIG. 4A or 4C is observed in the first orthogonal transform unit 320 using DCT, the second orthogonal transform by DST is performed. It can be easily predicted that the concentration of the spectrum can be expected by using the unit 340 rather than the DCT. This means that by examining the characteristics of the DCT spectrum, it is possible to determine which DCT or DST orthogonal transform is appropriate without trying DST.

<Operation>
Next, the operation of the image processing apparatus 300 according to the first embodiment will be described. FIG. 5 is a flowchart illustrating an example of image processing according to the first embodiment.

  In step S502, the first orthogonal transform unit 320 transforms the input image by DCT-II, which is the first orthogonal transform, and obtains a first orthogonal transform coefficient 321.

  In step S504, the analysis unit 322 may integrate the horizontal power spectrum (or the absolute value of the spectrum) for each even-order and odd-order.

  In step S506, the analysis unit 322 calculates the ratio between the even-order integration in the horizontal direction and the odd-order integration in the horizontal direction obtained in step S504. In the calculation of the ratio in this case, the large value is calculated as the denominator and the small value is calculated as the numerator so that the ratio is always obtained as a value of 1 or less.

  An example of the calculation in step 4 will be described with reference to FIG. FIG. 6 shows a power spectrum 600 obtained from orthogonal transform coefficients obtained by orthogonally transforming an 8 × 8 pixel block by DCT-II. For ease of explanation, the power spectrum 600 expresses the value of each power spectrum using the codes of columns 1 to 8 and rows A to H. For example, element A1 in column 1 row A is the power spectrum value of the DC component of DCT-II. The other elements also represent the power spectrum values in the respective frequency regions.

If the total value of the even-order power spectra in the horizontal direction is set to H _even in step S504, H _even is specifically calculated as follows.

H _even = A2 + A4 + A6 + A8 + B2 + …… .. + H6 + H8
Further, in step S504, if the total value of the odd-order power spectra in the horizontal direction is H _odd , H _odd is specifically calculated as follows.

H _odd = A1 + A3 + A5 + A7 + B1 + …… .. + H5 + H7
Then, the ratio H _rate is calculated as follows using the larger value of H _even and H _odd as the denominator.

H _rate = H _even / H _odd (when H _even <H _odd )
Returning to FIG.

  In step S508, the analysis unit 322 integrates the power spectrum in the vertical direction (or the absolute value of the spectrum) for each even-order and odd-order.

  In step S510, the analysis unit 322 calculates the ratio of the even-order integration in the vertical direction and the odd-order integration in the vertical direction obtained in step S508. In the calculation of the ratio in this case, the large value is calculated as the denominator and the small value is calculated as the numerator so that the ratio is always obtained as a value of 1 or less.

  If step S508 and step S510 are similarly calculated using FIG. 6, it will be as follows.

If the total value of the even-order power spectra in the vertical direction is V _even , V _even is specifically calculated as follows.

V _even = B1 + D1 + F1 + H1 + B2 + …… .. + F8 + H8
Further, in step S508, if the total value of the odd-order power spectra in the vertical direction is V _odd , V _odd is specifically calculated as follows.

V _odd = A1 + C1 + E1 + G1 + A2 + …… .. + E8 + G8
Then, the ratio V _rate is calculated as follows using the larger value of V _even and V _odd as the denominator.

V _rate = V _even / V _odd (when H _even <H _odd )
Returning to FIG.

In step S512, an average value AVE _rate of the horizontal ratio H _rate and the vertical ratio V _rate may be calculated. The average ratio AVE _rate can be calculated as follows.

AVE _rate = (H _rate + V _rate ) / 2
In step S514, the selection unit 330 compares the average ratio value AVE _rate with a predetermined threshold value. For example, when 0.5 is adopted as the threshold value, the following determination may be made.

AVE _rate > 0.5
The larger the AVE _rate , the smaller the difference between the even and odd orders of the power spectrum of the first orthogonal transform. In this case, DCT is more likely to be more appropriate orthogonal transform than DST.

Therefore, if AVE _rate > 0.5 holds, it may be determined that DCT is appropriate. If AVE _rate > 0.5 does not hold, it may be determined that DST is appropriate.

Note that the threshold value 0.5 adopted here is merely an example. Other values may be used. In the above example, both the horizontal ratio H _rate and the straight vertical ratio V _rate are used, but either one of them may be used.
Although in the example the two-dimensional orthogonal transformation for the sake of simplicity, in this embodiment, the horizontal, in the case of coding the application of different one-dimensional orthogonal transform in the vertical depending respectively H _rate and V _rate is equal to or greater than a threshold value Each orthogonal transform may be determined.

  If the determination result is “Yes” in step S514, the process proceeds to step S516. If the determination result is “No”, the process proceeds to step S518.

  In step S516, the selection unit 330 outputs the orthogonal transform coefficient obtained by the DCT of the first orthogonal transform unit 320 to the output 351.

  In step S518, the selection unit 330 provides the input image 341 to the second orthogonal transform unit 340. Then, the second orthogonal transform unit 340 transforms the input image 341 using DST.

  In step S520, the selection unit 330 outputs the orthogonal transform coefficient obtained by the second orthogonal transform unit 340 to the output 351.

  As described above, according to the first embodiment, it is possible to determine which orthogonal transform of DCT or DST is preferable while reducing the amount of calculation. Then, while reducing the amount of calculation, an orthogonal transform coefficient can be acquired by using an appropriate orthogonal transform between DCT and DST. If it is determined that DCT is appropriate, the calculation of DST can be omitted.

[Example 2]
The second embodiment is an example in which the first embodiment is more generalized.

<Configuration>
FIG. 7 is a block diagram illustrating a configuration of the image processing apparatus 700 according to the second embodiment. Constituent elements having the same functions as those in FIG. In order to avoid duplication, description of the constituent elements related to the same reference numerals is omitted.

  In FIG. 7, the analysis unit 322 may include a similarity calculation unit 724 and a storage unit 726. In addition, in the example of FIG. 7, in addition to the first orthogonal transform unit 320 and the second orthogonal transform unit 340, a third orthogonal transform unit 743 is provided. For this reason, the first switch 328 is modified so that the third orthogonal transform unit 743 can be selected.

  Assume that the first orthogonal transform unit 320, the second orthogonal transform unit 340, and the third orthogonal transform unit 743 employ, for example, DCT-II, DST-II, and DST-VII, respectively. The following description will be given. In addition, Example 2 is not restricted to this. Therefore, four or more orthogonal transform units may exist.

  FIG. 8 is a diagram for explaining orthogonal transform coefficient comparison patterns stored in the storage unit. The basic principle of Example 2 is demonstrated using FIG.

  Taking the base image 820 of the second orthogonal transform in FIG. 8, a plurality of orthogonal transform coefficient comparison patterns stored in the storage unit will be described. The base image 820 has a total of 64 base images, and each base image has 8 × 8 pixels.

  In this example, four low-frequency component base images of the second orthogonal transform (DST-II) are used. First, a first orthogonal transform (DCT-II) is performed on the base image 821 by the first orthogonal transform unit 320 to obtain an orthogonal transform coefficient 872. The orthogonal transform coefficient 872 may be stored in the storage unit 726. Similarly, the other three base images 822, 823, and 824 are also subjected to the first orthogonal transform to obtain orthogonal transform coefficients 871, 874, and 873, and are stored in the storage unit 726.

  The orthogonal transform coefficients 871 to 874 obtained by this operation can be used as an orthogonal transform coefficient comparison pattern.

  Similarly, the low-frequency component base images 831 to 834 of the base of the third orthogonal transform are also subjected to the first orthogonal transform. As a result, orthogonal transform coefficients 881 to 884 are obtained and stored in the storage unit 726. The orthogonal transform coefficients 881 to 884 can be used as orthogonal transform coefficient comparison patterns.

  It is assumed that the above processing is performed in advance before performing actual image processing. Therefore, it is assumed that the storage unit 726 stores a plurality of orthogonal transform coefficient comparison patterns.

  Next, the properties of the orthogonal transform coefficient comparison pattern will be described. For example, it is assumed that the input image is transformed by the first orthogonal transformation and the orthogonal transformation coefficient 890 is obtained. Assume that the orthogonal transform coefficient 890 is a pattern that is very similar to the orthogonal transform coefficient comparison pattern 871 stored in the storage unit 726. For calculating the similarity, a correlation value may be calculated.

  In this case, when the input image 341 is converted by the second orthogonal transform unit 340 (DST-II), a spectrum pattern having a high absolute value of the orthogonal transform coefficient corresponding to the base image 822 is obtained. . In this case, it is shown that the second orthogonal transform has a higher possibility of collecting the spectrum than the first orthogonal transform. Therefore, in this case, it is desirable to use the second orthogonal transform rather than the first orthogonal transform. Thereby, improvement in compression efficiency can be expected.

  Note that this similarity comparison may be performed, for example, by calculating a correlation value between the orthogonal transform coefficient comparison pattern and the orthogonal transform coefficient obtained from the actual image by the first orthogonal transform unit 320. Then, an orthogonal transform coefficient comparison pattern having the highest correlation among the calculated correlation values may be searched. Based on the result of comparing the correlation value corresponding to the searched orthogonal transformation coefficient comparison pattern with a predetermined threshold, the first orthogonal transformation is used, or the orthogonal transformation corresponding to the searched orthogonal transformation comparison pattern is used. It may be determined whether or not. That is, when the highest correlation value exceeds a predetermined threshold, the orthogonal transformation corresponding to the searched orthogonal transformation comparison pattern is used. If the correlation value is less than or equal to a predetermined threshold, the first orthogonal transform is used.

  The similarity may be calculated using any of existing methods such as SAD (Sum of absolute difference) and SSD (Sum of Squared difference).

  The number of orthogonal transform coefficient comparison patterns stored in the storage unit 726 may be determined in consideration of the number of orthogonal transforms to be prepared, the amount of resources required for similarity calculation, and the like.

  In FIG. 7, the base of the low frequency component is selected as the base of the second orthogonal transformation. This is because the orthogonal transform is often designed so that the power spectrum is concentrated in the low band. Therefore, it is desirable to consider the nature or design philosophy of the orthogonal transformation to be used as the basis for creating the orthogonal transformation coefficient comparison pattern based on the nature of the orthogonal transformation.

  In the second embodiment, the first orthogonal transform unit 320 is used to create one orthogonal transform coefficient comparison pattern from one base image. However, the second embodiment is not limited to this. An image obtained by combining a plurality of base images is created, and one orthogonal transform coefficient comparison pattern is created using the first orthogonal transform unit 320. Also good.

<Operation>
FIG. 9 is a flowchart illustrating an example of image processing according to the second embodiment. The operation of the image processing apparatus according to the second embodiment will be described below.

  In step S <b> 902, the first orthogonal transform unit 320 performs a first orthogonal transform (DCT-II) on the input image 341 and obtains a first orthogonal transform coefficient 321.

  In step S904, the similarity calculation unit 724 extracts one of the orthogonal transform comparison patterns from the storage unit 726.

  In step S906, the similarity calculation unit 724 calculates the similarity (eg, correlation value) between the first orthogonal transform coefficient 321 and the read orthogonal transform comparison pattern.

  In step S908, steps S904 and S906 are repeated for the number of patterns stored in the storage unit 726.

  In step S910, the selection unit 330 specifies an orthogonal transform comparison pattern having the maximum similarity (for example, correlation value).

  In step S912, the selection unit 330 determines whether the maximum similarity exceeds a predetermined threshold. If the determination is “No”, the process moves to step S914. If the determination is “Yes”, the process proceeds to step S916.

  In step S914, the selection unit outputs the orthogonal transformation coefficient obtained by the first orthogonal transformation unit to the output 351.

  In step S916, the selection unit specifies an orthogonal transformation unit (either the second orthogonal transformation unit 340 or the third orthogonal transformation unit 743) corresponding to the selected orthogonal transformation comparison pattern.

  The selection unit 330 operates the first switch 328 and the second switch 329 to provide the input image 341 to the specified orthogonal transform unit. The identified orthogonal transform unit performs the orthogonal transform of the identified orthogonal transform unit on the input image 341, and acquires the orthogonal transform coefficient 331. The selection unit 330 instructs the second switch 329 to output the obtained orthogonal transform coefficient 331 to the output 351.

  As described above, according to the second embodiment, it is possible to obtain an orthogonal transform coefficient by performing an appropriate orthogonal transform among a plurality of orthogonal transforms while suppressing a calculation amount. Even if there are three or more orthogonal transforms that are candidates for use, it is only necessary to perform two orthogonal transforms at maximum. If it is determined that the first orthogonal transformation is appropriate, the orthogonal transformation only needs to be executed once.

[Example 3]
In the third embodiment, an image processing device (image encoding device) including the image processing device 300 in the first embodiment or the image processing device 700 in the second embodiment in the orthogonal transform unit 202 will be described.

<Configuration>
FIG. 10 is a block diagram illustrating an example of a schematic configuration of the image processing apparatus 20 according to the third embodiment. In the example illustrated in FIG. 10, the image processing apparatus 20 includes a preprocessing unit 200, a prediction error signal generation unit 201, an orthogonal transformation unit 202, a quantization unit 203, an entropy encoding unit 204, and an inverse quantization unit. 205, an inverse orthogonal transform unit 206, a decoded image generation unit 207, a loop filter unit 209, a decoded image storage unit 210, an intra prediction unit 211, an inter prediction unit 212, a motion vector calculation unit 213, and a prediction And an image selection unit 215. An outline of each part will be described below.

  The preprocessing unit 200 rearranges the pictures according to the picture type, and sequentially outputs the picture type and the frame image for each frame. The preprocessing unit 200 may also perform block division and the like, and output block division boundary information to the loop filter unit 209.

  The prediction error signal generation unit 201 obtains block data obtained by dividing the encoding target image of the input moving image data into blocks such as 32 × 32, 16 × 16, and 8 × 8 pixels, for example.

  The prediction error signal generation unit 201 generates a prediction error signal based on the block data and the block data of the prediction image output from the prediction image selection unit 215. The prediction error signal generation unit 201 outputs the generated prediction error signal to the orthogonal transformation unit 202.

  The orthogonal transform unit 202 can include the image processing device 300 in the first embodiment or the image processing device 700 in the second embodiment. The orthogonal transform unit 202 performs an orthogonal transform process on the input prediction error signal. The orthogonal transform unit 202 outputs a signal indicating the transformed coefficient value to the quantization unit 203. In addition, a selection signal indicating the selected orthogonal transform is sent to the entropy coding unit and the inverse orthogonal transform unit.

  The quantization unit 203 quantizes the output signal from the orthogonal transform unit 202. The quantization unit 203 reduces the code amount of the output signal by quantization, and outputs this output signal to the entropy encoding unit 204 and the inverse quantization unit 205. The quantization unit 203 may output the QP value of the quantization parameter to the loop filter unit 209.

  The entropy encoding unit 204 entropy-encodes and outputs the output signal from the quantization unit 203, the motion vector information output from the motion vector calculation unit 213, the filter coefficient from the loop filter unit 209, and the like.

  Further, the entropy encoding unit 204 may entropy encode the difference value of the intra prediction direction acquired from the intra prediction unit 211, the difference value of the motion vector and the prediction vector acquired from the inter prediction unit 212, and the like.

  The entropy encoding unit 204 encodes the orthogonal transformation information 250 (orthogonal transformation selection signal) used in the orthogonal transformation unit 202, and transmits the coded information to the decoding device side as an index of the selected orthogonal transformation. Entropy coding is a method of assigning variable-length codes according to the appearance frequency of symbols.

  The inverse quantization unit 205 performs inverse quantization on the output signal from the quantization unit 203 and outputs the result to the inverse orthogonal transform unit 206. The inverse orthogonal transform unit 206 performs an inverse orthogonal transform process on the output signal from the inverse quantization unit 205 and then outputs the output signal to the decoded image generation unit 207. The inverse orthogonal transform unit 206 selects an inverse orthogonal transform method using the input orthogonal transform selection signal. By performing decoding processing by the inverse quantization unit 205 and the inverse orthogonal transform unit 206, a signal having the same level as the prediction error signal before encoding is obtained.

  The decoded image generation unit 207 is decoded by the block data of the image predicted in the screen by the intra prediction unit 211 or the motion compensated image by the inter prediction unit 212, and the inverse quantization unit 205 and the inverse orthogonal transform unit 206. Add the prediction error signal. The decoded image generation unit 207 outputs the decoded image block data generated by addition to the loop filter unit 209.

  The loop filter unit 209 is, for example, an ALF (Adaptive Loop Filter) or a deblocking filter. The loop filter unit 209 outputs the filter processing result to the decoded image storage unit 210, and stores the accumulated filter processing result for one image as a reference image.

  The decoded image storage unit 210 stores the input block data of the decoded image as new reference image data, and outputs the data to the intra prediction unit 211, the inter prediction unit 212, and the motion vector calculation unit 213.

  The intra prediction unit 211 generates block data of the predicted image from the already-encoded reference pixels for the processing target block of the encoding target image. The intra prediction unit 211 performs prediction using a plurality of prediction directions, and determines an optimal prediction direction. With respect to the prediction direction, the difference value is output to the entropy encoding unit 204 in order to include the difference value with the prediction direction of the encoded block in the bitstream.

  The inter prediction unit 212 performs motion compensation on the reference image data acquired from the decoded image storage unit 210 with the motion vector provided from the motion vector calculation unit 213. Thereby, block data as a motion-compensated reference image is generated. For the motion vector, the difference value with the motion vector (predicted vector) of the encoded block is output to the entropy encoding unit 204 in order to include the difference value in the bitstream.

  The motion vector calculation unit 213 obtains a motion vector using the block data in the encoding target image and the reference image acquired from the decoded image storage unit 210.

  The motion vector calculation unit 213 outputs the obtained motion vector to the inter prediction unit 212 and outputs motion vector information including information indicating the reference image to the entropy encoding unit 204.

  The block data output from the intra prediction unit 211 and the inter prediction unit 212 is input to the predicted image selection unit 215.

  The predicted image selection unit 215 selects one of the block data acquired from the intra prediction unit 211 and the inter prediction unit 212 as a predicted image. The selected prediction image is output to the prediction error signal generation unit 201.

  Depending on which of the intra prediction unit 211 and the inter prediction unit 212 is used, a plurality of orthogonal transform candidates used in the orthogonal transform unit 202 may be different.

  Note that the configuration of the image processing apparatus 20 illustrated in FIG. 10 is an example, and the configurations may be combined or the configurations may be appropriately changed as necessary.

  As described above, according to the third embodiment, orthogonal transform coefficients can be obtained by appropriate orthogonal transform from among a plurality of orthogonal transforms while suppressing the amount of computation of orthogonal transform during image coding.

[Example 4]
FIG. 11 is a block diagram illustrating an example of a schematic configuration of the image processing device 40 according to the fourth embodiment. An image processing apparatus 40 shown in FIG. 11 is an example of an apparatus in which the image processing described in the first to third embodiments is implemented by software.

  As shown in FIG. 11, the image processing apparatus 40 includes a control unit 401, a main storage unit 402, an auxiliary storage unit 403, a drive device 404, a network I / F unit 406, an input unit 407, and a display unit. 408. These components are connected to each other via a bus so as to be able to transmit and receive data.

  The control unit 401 is a CPU (Central Processing Unit) that controls each device, calculates data, and processes in a computer. The control unit 401 is an arithmetic device that executes an image processing program stored in the main storage unit 402 or the auxiliary storage unit 403. The control unit 401 receives data from the input unit 407 and the storage device, calculates and processes the data, and then outputs the data to the display unit 408 and the storage device.

  Also, the control unit 401 can realize the processing described in the first to third embodiments by executing an image processing program.

  The main storage unit 402 is a ROM (Read Only Memory), a RAM (Random Access Memory), or the like. The main storage unit 402 is a storage device that stores or temporarily stores programs and data such as OS (Operating System) and application software that are basic software executed by the control unit 401.

  The auxiliary storage unit 403 is an HDD (Hard Disk Drive) or the like, and is a storage device that stores data related to application software or the like.

  The drive device 404 reads the program from the recording medium 405, for example, a flexible disk, and installs it in the storage unit.

  A predetermined program is stored in the recording medium 405, and the program stored in the recording medium 405 is installed in the image processing apparatus 40 via the drive device 404. The installed predetermined program can be executed by the image processing apparatus 40.

  The network I / F unit 406 is a peripheral having a communication function connected via a network such as a LAN (Local Area Network) or a WAN (Wide Area Network) constructed by a data transmission path such as a wired and / or wireless line. This is an interface between the device and the image processing apparatus 40.

  The input unit 407 includes a keyboard having cursor keys, numeric input, various function keys, and the like, and a mouse and a slide pad for performing key selection on the display screen of the display unit 408. The display unit 408 is configured by an LCD (Liquid Crystal Display) or the like, and performs display according to display data input from the control unit 401.

  Each unit of the image processing apparatus 300 in FIG. 3 and the image processing apparatus 700 in FIG. 7 can be realized by, for example, the control unit 401 and the main storage unit 402 as a work memory.

  Also, the decoded image storage unit 210 illustrated in FIG. 10 is realized by, for example, the main storage unit 402 or the auxiliary storage unit 403, and the configuration other than the decoded image storage unit 210 illustrated in FIG. This can be realized by the main storage unit 402.

  The program executed by the image processing apparatus 40 has a module configuration including each unit described in the first to third embodiments. As actual hardware, when the control unit 401 reads out and executes a program from the auxiliary storage unit 403, one or more of the above-described units are loaded onto the main storage unit 402, and one or more of the respective units are main. It is generated on the storage unit 402.

  As described above, the image processing described in the first to third embodiments may be realized as a program for causing a computer to execute the image processing. The processing described in the first to third embodiments can be realized by installing this program from a server or the like and causing the computer to execute the program.

  It is also possible to record the program in the recording medium 405 and cause the processing unit such as a computer or a portable terminal to read the recording medium 405 on which the program is recorded, thereby realizing the above-described image processing.

  Note that the recording medium 405 is a recording medium that records information optically, electrically, or magnetically, such as a CD-ROM, a flexible disk, or a magneto-optical disk, and information that is electrically stored such as a ROM or a flash memory. Various types of recording media such as a semiconductor memory for recording can be used.

  Further, the image processing described in each of the above embodiments may be implemented in one or a plurality of integrated circuits. Note that, as described above, the image processing apparatus 40 according to the fourth embodiment may have a function as at least one of the image processing apparatuses 300, 700, and 20.

  In addition, the image processing apparatus in each embodiment described above can be applied to an encoding technique that selects and uses orthogonal transform. It is not limited to H.264 / AVC or HEVC.

  Each embodiment has been described in detail above. However, the present invention is not limited to the specific embodiment, and various modifications and changes other than the above-described modification are possible within the scope described in the claims. .

320 First orthogonal transform unit 321 Analysis result 322 Analysis unit 327 Selection instruction signal 328 First switch 329 Second switch 330 Selection unit 340 Second orthogonal transform unit 341 Input image 351 Output 724 Similarity calculation unit 726 Storage unit 743 Third orthogonal transform unit

Claims (5)

An image processing apparatus that performs processing by selecting any one of a first orthogonal transformation and one or more other orthogonal transformations,
An analysis unit that analyzes a first orthogonal transform coefficient obtained by transforming image data by the first orthogonal transform according to a predetermined rule;
A selection unit that selects any one of the first orthogonal transform and the one or more other orthogonal transforms based on the analysis result of the analysis unit;
An image processing apparatus. The predetermined rule is: an energy ratio between an odd-order and an even-order in the horizontal direction of the first orthogonal transform coefficient and / or an energy ratio between an odd-order and an even-order in the vertical direction of the first orthogonal transform coefficient. Use
The image processing apparatus according to claim 1. The analysis unit
A storage unit that stores orthogonal transform coefficients obtained by transforming each of a predetermined number of the plurality of base images of each of the one or more other orthogonal transforms by the first orthogonal transform;
A similarity calculation unit that calculates the similarity between the first orthogonal transform coefficient and the plurality of orthogonal transform coefficients stored in the storage unit;
Including
The selection unit, when the highest similarity among the similarities obtained by the similarity calculation unit exceeds a predetermined threshold, the image is subjected to the other orthogonal transformation corresponding to the highest similarity. By selectively providing data, an orthogonal transform coefficient obtained by transforming the image data is used for encoding the image data,
If the highest similarity is less than or equal to the predetermined threshold, the first orthogonal transform coefficient is used for encoding the image data;
The image processing apparatus according to claim 1.   An encoding device comprising the image processing device according to any one of claims 1 to 3.   A program for causing a computer to function as the encoding device according to claim 4.

Images