JP5375676B2 - Image processing apparatus, image processing method, and image processing program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To reduce an operation amount of two-dimensional orthogonal transformation. <P>SOLUTION: This image processing device calculates difference between a first pixel value matrix (Bc_l) regarding a pixel which does not overlap a first prediction block in a second prediction block and a second pixel value matrix (Bp_l) regarding a pixel which does not overlap the second prediction block in the first prediction block. Then, by using a calculated differential value matrix as a target matrix for orthogonal transformation, the image processing device performs first orthogonal transformation and second orthogonal transformation to the differential value matrix using an orthogonal transformation matrix. The image processing device calculates a second transformation coefficient matrix regarding a second block by complementing conversion coefficient matrixes after the first and second orthogonal transformation, using a first conversion coefficient matrix regarding the first prediction block. <P>COPYRIGHT: (C)2011,JPO&amp;INPIT

Description

  The present invention relates to an image processing apparatus, an image processing method, and an image processing program that perform two-dimensional orthogonal transform processing on blocks in an image.

  Conventionally, H.M. An image processing apparatus that uses inter-frame prediction as a means for generating a predicted image such as H.264 or MPEG-2 includes a process of calculating a motion vector used in inter-frame prediction (see, for example, Patent Document 1 below). ). The motion vector is motion position information indicating from which block in the reference image the target block in the original image has moved.

  In the motion vector calculation process, the similarity between the target block in the original image and the predicted block in the reference image indicated by the candidate motion vector is determined. The most widely used evaluation index for similarity determination is SAD (Sum Of Absolute Difference). The SAD is obtained by taking the absolute value of the difference between corresponding pixels of the target block and the prediction block and summing the absolute values of the differences.

  However, as a motion vector evaluation index, SATD (Sum Of Absolute Transformed Difference) (for example, refer to Patent Document 2 below) is more accurate than SAD. The SATD is obtained by orthogonally transforming the difference between the pixels of the prediction block and the target block and summing the absolute values of the respective coefficients after the transformation.

  The evaluation using the SAD corresponds to the evaluation of the result of the encoding process up to the difference process, whereas the SATD is the evaluation of the result of the encoding process up to the orthogonal transform process. Therefore, in SATD, since the encoding process is advanced to a position closer to the stream that is the encoding result, it is possible to accurately evaluate the predicted image as compared with SAD.

  In orthogonal transform, Hadamard transform with a small amount of calculation is often used. H. The Hadamard transform is also used in the H.264 reference encoder. Furthermore, a process for reducing the amount of computation in the Hadamard transform by modifying the Hadamard transform matrix used in the Hadamard transform is known (for example, see Patent Document 3 below).

JP 2006-186504 A JP-A-3-185986 JP 2004-241957 A

  However, since SATD requires more calculations than SAD and the ratio of calculation processing occupied by SAD / SATD is very large in the calculation processing of the entire encoding process, either SAD or SATD is adopted. Depending on whether or not, the calculation amount of the entire encoding process is greatly different.

  H. In the H.264 reference encoder, the amount of calculation is reduced while using SATD by using a technique of evaluating vectors by SAD in integer precision search and using SATD in decimal precision search. However, since there is a problem that the amount of calculation of SATD is large, it is common to adopt SAD with a small amount of calculation even if the evaluation accuracy is lowered in an actual product.

  In order to solve the above-described problems caused by the prior art, the present invention provides an image processing apparatus, an image processing method, and an image processing apparatus that can reduce the amount of computation of two-dimensional orthogonal transformation by reducing the number of elements of a matrix to be subjected to orthogonal transformation. An object is to provide a processing program.

  According to one aspect of the present invention, an N × N element first block in an image is shifted by M (1 ≦ M <N / 2) elements in an arbitrary search direction. The difference value matrix that is the difference between the first feature quantity matrix that does not overlap the first block and the second feature quantity matrix that does not overlap the second block in the first block. A difference value calculation means for calculating the difference value, and a one-dimensional orthogonal transformation of the difference value matrix of N × M elements calculated by the difference value calculation means using an orthogonal transformation matrix of N × N elements to convert N × M elements A first orthogonal transform unit that calculates a coefficient matrix and a N × M element transform coefficient matrix calculated by the first orthogonal transform unit are subjected to a one-dimensional orthogonal transform using an M × N element orthogonal transform matrix. Second orthogonal transform means for calculating a transform coefficient matrix of N × N elements, and the first block A second N × N element related to the second block based on the first N × N element first transform coefficient matrix and the N × N element transform coefficient matrix transformed by the second orthogonal transform means. There is provided an image processing apparatus comprising: a coefficient value calculating unit that calculates the conversion coefficient matrix of

  According to the image processing apparatus, the image processing method, and the image processing program, there is an effect that it is possible to reduce the calculation amount of the two-dimensional orthogonal transformation by reducing the number of elements of the matrix to be subjected to the orthogonal transformation.

It is explanatory drawing (the 1) which shows one Example of this invention. It is explanatory drawing (the 2) which shows one Example of this invention. It is explanatory drawing which shows the case where x component of a current vector is one larger than x component of a previous vector. It is explanatory drawing which shows the case where the x component of the current vector is 1 smaller than the x component of the previous vector. It is explanatory drawing which shows the case where y component of the current vector is one larger than y component of the previous vector. It is explanatory drawing which shows the case where y component of the current vector is 1 smaller than y component of the previous vector. 1 is an explanatory diagram illustrating an example of an image processing apparatus according to a first embodiment; It is explanatory drawing which shows the example of a motion vector search by a spiral search. FIG. 11 is an explanatory diagram (part 1) illustrating a specific processing example of the second orthogonal transform circuit 303 in the second embodiment. FIG. 10 is an explanatory diagram (part 2) of a specific process example of the second orthogonal transform circuit 303 in the second embodiment. FIG. 11 is an explanatory diagram (part 3) illustrating a specific processing example of the second orthogonal transform circuit 303 in the second embodiment. FIG. 12 is an explanatory diagram (part 4) illustrating a specific processing example of the second orthogonal transform circuit 303 in the second embodiment. It is explanatory drawing (the 1) which shows the example by which the multiplication process with the permutation matrix was replaced by the permutation process and the code conversion process. It is explanatory drawing (the 2) which shows the example by which the multiplication process with the permutation matrix was replaced by the permutation process and the code conversion process. It is explanatory drawing (the 3) which shows the example by which the multiplication process with the permutation matrix was replaced by the permutation process and the code conversion process. It is explanatory drawing (the 4) which shows the example by which the multiplication process with the permutation matrix was replaced by the permutation process and the code conversion process. It is explanatory drawing which shows an example of a macroblock. FIG. 9 is an explanatory diagram illustrating an example of an image processing apparatus according to a third embodiment. It is explanatory drawing which shows the detailed description of unit 0_1-unit 0_7. It is explanatory drawing which shows the detailed description of unit 1_1-unit 1_4. It is explanatory drawing which shows the detailed description of the unit 2-3_1 to the unit 2-3_9. 1 is a block diagram illustrating a hardware configuration of an image processing apparatus according to an embodiment. FIG. 10 is a block diagram illustrating a functional configuration of an image processing apparatus according to a fourth embodiment. FIG. 11 is an explanatory diagram (part 1) illustrating a detailed configuration of a coefficient value calculation unit 1304; 6 is an explanatory diagram showing a detailed configuration of a first orthogonal transform unit 1302; FIG. FIG. 10 is an explanatory diagram (part 2) illustrating a detailed configuration of a coefficient value calculation unit 1304; It is a flowchart which shows an example of the image processing procedure by an image processing apparatus. 16 is a flowchart showing a detailed description of the first two-dimensional orthogonal transformation process (step S1503) shown in FIG. 16 is a flowchart showing a detailed description of the second two-dimensional orthogonal transformation process (step S1504) shown in FIG. 16 is a flowchart showing a detailed description of the second two-dimensional orthogonal transformation process (step S1505) shown in FIG.

  Exemplary embodiments of an image processing apparatus, an image processing method, and an image processing program according to the present invention are explained in detail below with reference to the accompanying drawings.

  FIGS. 1-1 and 1-2 are explanatory views showing an embodiment of the present invention. First, FIG. 1-1 shows a first prediction block indicated by a previous vector in a reference image and a second prediction block indicated by a current vector obtained by shifting the first prediction block by 1 in the search direction. Has been. Here, the block of N × N elements indicates a block of N × N pixels, and in FIGS. 1-1 and 1-2, 4 × 4 pixels are handled as one block. The feature amount related to the pixel is a pixel value.

  First, the image processing apparatus overlaps a first pixel value matrix (Bc_l) that does not overlap with the first prediction block in the second prediction block and a second prediction block in the first prediction block. A difference value matrix that is a difference from the second pixel value matrix (Bp_l) not calculated is calculated.

  Next, the image processing apparatus calculates a 4 × 1 element transform coefficient matrix by performing one-dimensional orthogonal transform on the 4 × 1 element difference value matrix using the 4 × 4 element orthogonal transform matrix. Regarding the orthogonal transformation, an existing technique may be used, and the orthogonal transformation matrix is determined by the orthogonal transformation.

  Then, the image processing apparatus calculates a 4 × 4 element conversion coefficient matrix by performing one-dimensional orthogonal transformation on the 4 × 1 element conversion coefficient matrix using the 1 × 4 element orthogonal conversion matrix.

  Next, moving to FIG. 1-2, the image processing apparatus includes a 4 × 4 element first transform coefficient matrix (Rp) related to the first prediction block, a calculated 4 × 4 element transform coefficient matrix, and Based on the above, a 4 × 4 second transform coefficient matrix for the second prediction block is calculated. Specifically, for example, the image processing apparatus adds the first conversion coefficient matrix and the calculated 4 × 4 element conversion coefficient matrix, and multiplies the replacement matrix for replacing row vectors. As a result, a 4 × 4 second transform coefficient matrix is calculated.

  When the second prediction block is a block that is shifted by 1 in the horizontal direction with respect to the first prediction block, the first feature matrix and the second feature matrix are N rows and one column. The one-dimensional orthogonal transformation is performed using an N-row N-column orthogonal transformation matrix and a 1-row N-column orthogonal transformation matrix. Further, when shifting in the horizontal direction by M (1 ≦ M <N / 2), the first feature matrix and the second feature matrix are N rows and M columns. The one-dimensional orthogonal transformation is performed using an N-row N-column orthogonal transformation matrix and an M-row N-column orthogonal transformation matrix.

  On the other hand, when the second prediction block is a block shifted by 1 in the vertical direction with respect to the first prediction block, the first feature matrix and the second feature matrix are 1-row N-column matrices. One-dimensional orthogonal transformation is performed using an N-row N-column orthogonal transformation matrix and an N-row 1-column orthogonal transformation matrix. When the vertical shift is performed by M (1 ≦ M <N / 2), the first feature matrix and the second feature matrix are M rows and N columns. The one-dimensional orthogonal transformation is performed using an N-row N-column orthogonal transformation matrix and an N-row M-column orthogonal transformation matrix.

  Detailed description of the present invention will be described using Embodiments 1 to 4. First, in the first embodiment, an example in which the second prediction block of N × N pixels indicated by the current vector is two-dimensionally orthogonally transformed using the first prediction block of N × N pixels indicated by the previous vector. Will be explained.

  Next, in the second embodiment, the N × N pixel second prediction block indicated by the current vector is subjected to two-dimensional Hadamard transform using the N × N pixel first prediction block indicated by the previous vector. An example will be described.

  In the third embodiment, an example in which the SATD for the second prediction block of N × N pixels indicated by the current vector is calculated when N = 4 will be described.

  Finally, in the fourth embodiment, an example in which the processing by the image processing apparatus shown in the first embodiment is processed by a computer will be described.

(Embodiment 1)
First, the first prediction block indicated by the previous vector and the second prediction block indicated by the current vector will be described with reference to FIGS.

  FIG. 2A is an explanatory diagram illustrating a case where the x component of the current vector is one greater than the x component of the previous vector. The arrow from the target block in the original image to the first prediction block in the reference image is the previous vector. The arrow from the target block in the original image to the second prediction block in the reference image is the current vector. The x component of the current vector is one greater than the x component of the previous vector. Therefore, the second prediction block is a block shifted by +1 in the horizontal direction from the first prediction block.

  In the first embodiment, one block is 4 pixels × 4 pixels. The vector x is a pixel in the horizontal direction, and y is a pixel in the vertical direction. The vector (0, 0) is a vector indicating a prediction block in the reference image at the same position as the target block in the original image.

  Bc_l is a column vector of pixels that do not overlap the first prediction block in the second prediction block. On the other hand, Bp_l is a column vector of pixels that do not overlap with the second prediction block in the first prediction block.

  In the first embodiment, the second prediction coefficient matrix (Rc) obtained by two-dimensional orthogonal transformation of the second prediction block is used as the first transformation coefficient matrix (Rp) obtained by two-dimensional orthogonal transformation of the first prediction block. ) To calculate. If the x component of the current vector is one greater than the x component of the previous vector, Rc is calculated using the following equation (1).

  Rc = [Rp + {T1. (Bc_1-Bp_1)}. T2_u]. (IT2.Sr.T2) (1)

  T1, T2, T2_u, and Sr will be described later.

  FIG. 2B is an explanatory diagram of a case where the x component of the current vector is one smaller than the x component of the previous vector. The arrow from the target block in the original image to the first prediction block in the reference image is the previous vector. The arrow from the target block in the original image to the second prediction block in the reference image is the current vector. The x component of the current vector is one less than the x component of the previous vector. Therefore, the second prediction block is a block shifted by −1 in the horizontal direction from the first prediction block. In the first embodiment, one block is 4 pixels × 4 pixels.

  Bc_l is a column vector of pixels that do not overlap the first prediction block in the second prediction block. On the other hand, Bp_l is a column vector of pixels that do not overlap with the second prediction block in the first prediction block.

  A second transform coefficient matrix (Rc) relating to the second predicted block is calculated using the first transform coefficient matrix (Rp) obtained by two-dimensionally orthogonally transforming the first predicted block. When the x component of the current vector is one smaller than the x component of the previous vector, Rc is calculated using the following equation (2).

  Rc = [Rp + {T1. (Bc_1-Bp_1)}. T2_d]. (IT2.S1.T2) (2)

  T2_d and Sl will be described later.

  FIG. 2-3 is an explanatory diagram illustrating a case where the y component of the current vector is one greater than the y component of the previous vector. The arrow from the target block in the original image to the first prediction block in the reference image is the previous vector. The arrow from the target block in the original image to the second prediction block in the reference image is the current vector. The y component of the current vector is one greater than the y component of the previous vector. Therefore, the second prediction block is a block shifted by +1 in the vertical direction from the first prediction block.

  Bc_r is a column vector of pixels that do not overlap the first prediction block in the second prediction block. On the other hand, Bp_r is a column vector of pixels that do not overlap with the second prediction block in the first prediction block.

  When the y component of the current vector is one greater than the y component of the previous vector, Rc is calculated by the following equation (3).

  Rc = (T1 · Sd · iT1) · [Rp + T1 — l · {(Bc_r−Bp_r) · T2}] (3)

  T1_l and Sd will be described later.

  FIG. 2-4 is an explanatory diagram illustrating a case where the y component of the current vector is one smaller than the y component of the previous vector. The arrow from the target block in the original image to the first prediction block in the reference image is the previous vector. The arrow from the target block in the original image to the second prediction block in the reference image is the current vector. The y component of the current vector is one less than the y component of the previous vector. Therefore, the second prediction block is a block shifted by −1 in the vertical direction from the first prediction block.

  Bc_r is a column vector of pixels that do not overlap the first prediction block in the second prediction block. On the other hand, Bp_r is a column vector of pixels that do not overlap with the second prediction block in the first prediction block.

  When the y component of the current vector is one smaller than the y component of the previous vector, Rc is calculated by the following equation (4).

  Rc = (T1 · Su · iT1) · [Rp + T1_r · {(Bc_r−Bp_r) · T2}] (4)

  T1_r and Su will be described later.

Below, it shows about each matrix shown by said Formula (1)-(4).
T1: One-dimensional orthogonal transformation matrix T2: One-dimensional orthogonal transformation matrix iT1: Inverse matrix of T1 iT2: Inverse matrix of T2 Sr: When applied from the right, the rightmost column becomes the leftmost column and the remaining columns A matrix that moves one to the right ・ Sl: If applied from the right, the leftmost column is moved to the rightmost column, and the remaining column is moved one to the left A matrix that moves the next row down by one. Su: When applied from the left, the top row is the bottom row, and the remaining rows are moved up by one. Bc_l: indicated by the previous vector in the prediction block indicated by the current vector N × 1 pixel value matrix (first pixel value matrix) that does not overlap with the predicted block to be displayed, column vector Bp — l: overlaps with the prediction block indicated by the current vector in the prediction block indicated by the previous vector No N × 1 pixel value matrix (second pixel value matrix), column Kutor ・ Bc_r: 1 × N pixel value matrix (first pixel value matrix) that does not overlap the prediction block indicated by the previous vector in the prediction block indicated by the current vector, row vector ・ Bp_r: the previous vector A pixel value matrix (second pixel value matrix) of 1 × N pixels that does not overlap the prediction block indicated by the current vector in the indicated prediction block, a row vector T2_u: a row vector having the top row of T2 as an element T2_d: a row vector having the bottom row of T2 as an element T1_1: a column vector having the leftmost column of T1 as an element T1_r: a column vector having the rightmost column of T1 as an element Rp: relating to the first prediction block First transform coefficient matrix Rc: second transform coefficient matrix for the second prediction block

  (IT2 · Sr · T2) of the above formula (1), (iT2 · Sl · T2) of the above formula (2), (T1 · Sd · iT1) of the above formula (3), and the above formula (4) (T1 · Su · iT1) is a replacement matrix related to matrix replacement.

  Here, the derivation process of the formula (1) will be described. T1, Pc, and T2 are N rows and N columns, and T1 and T2 are regular matrices. Let Pc be the predicted image indicated by the current vector, and Pp be the predicted image indicated by the previous vector. The following formula (5) is an example in which Rc is calculated by orthogonal transformation of Pc.

  Rc = T1, Pc, T2 (5)

  For example, it is set as the following formula (6).

  Pm = Pc · Sr (6)

  Therefore, the following formula (7) can be obtained from the above formula (6).

  Pc = Pm · iSr (7)

  Then, the following formula (8) can be obtained by substituting the above formula (7) into the above formula (5).

  Rc = T1, Pm, iSr, T2 (8)

  When the upper left coordinate of Pp is (0, 0), the pixel in the first prediction block is expressed as p (x, y) using the relative coordinate from (0, 0), and Pm and Pp The components are as follows.

  The relationship between Pm and Pp is represented by the following formula (9).

  Pm = Pp + Z (9)

  Z is a matrix in which the first column is equal to (Bc_l-Bp_l) and the remaining columns are all 0 components. Therefore, the following formula (10) can be obtained.

Rc = T1. (Pp + Z) .iSr.T2
= T1 ・ (Pp + Z) ・ I ・ iSr ・ T2
= T1 ・ (Pp + Z) ・ T2 ・ iT2 ・ iSr ・ T2
= (T1 · (Pp + Z) · T2) · (iT2 · iSr · T2)
= (T1 · Pp · T2 + T1 · Z · T2) · (iT2 · iSr · T2)
(10)

  Note that I is a unit matrix of N × N elements. Since Rp is T1, Pp, and T2, Rc can be expressed by the following formula (11).

  Rc = (Rp + T1 · Z · T2) · (iT2 · iSr · T2) (11)

  Z · T2 in the above formula (1) is the above formula (12). In addition, T22,..., T2n are the second matrix of T2,..., Nth row, and 0 are n matrixes of n rows and n−1 columns, respectively. Therefore, the following formula (13) is obtained, and the above formula (1) is obtained.

  Rc = {Rp + T1. (Bc_1-Bp_1) .T2_u}. (IT2.iSr.T2) (13)

  Here, a conventional calculation amount will be described. Conventionally, the calculation of the matrix of N × N elements is performed twice as in the above equation (5). In multiplication of matrices of N × N elements, a conventional image processing apparatus performs multiplication N times and addition N−1 times to obtain each element of the multiplication result. Therefore, the number of multiplications between N × N matrixes is N ^ 3 times ("^" indicates a multiplier, N ^ 3 is the third power of N), and addition is (N-1). N ^ 2 times. In order to calculate the conversion result, the matrix of N × N elements is multiplied twice, so in the orthogonal transformation of one block, multiplication is performed 2N ^ 3 times and addition is performed (N-1) · 2N ^ 2 times. carry out. “^ 2” indicates a square.

Next, the calculation amount in the present invention will be described by taking the above equation (1) as an example.
Bc_l-Bp_l: N times of subtraction. Multiplication of T1 and (Bc_l-Bp_l): N times "multiplication N times, N-1 times addition", that is, N ^ 2 times multiplication and N (N-1) times addition. Multiplication of (T1 · (Bc_l−Bp_l)) and T2_u: multiplication N ^ 2 times Rp and {T1 · (Bc_l−Bp_l) · T2_u} addition: addition N ^ 2 times [Rp + {T1 · (Bc_l -Bp_l) · T2_u}] and (iT2 · Sr · T2): Multiplication N ^ 3 times, addition (N-1) · N ^ 2 times

  Therefore, multiplication is 2N ^ 2 times + N ^ 3 times, and addition / subtraction is N times + N · (N−1) times + N ^ 2 times + (N−1) · N ^ 2 times, which is faster than the conventional method. be able to.

  (IT2 · Sr · T2) of the above formula (1), (iT2 · Sl · T2) of the above formula (2), (T1 · Sd · iT1) of the above formula (3), and the above formula (4) (T1 · Su · iT1) is determined when the orthogonal transformation matrix (T1, T2) is determined. Therefore, it is actually a matrix of N × N elements.

  In the figure, although it is a 4 × 4 block, it may be an N × N block (N> = 4), and the orthogonal transform in the above formulas (1) to (4) is not particularly defined.

  FIG. 3 is an explanatory diagram of an example of the image processing apparatus according to the first embodiment. The image processing apparatus 300 includes a difference value calculation circuit 301, a first orthogonal transformation circuit 302, a second orthogonal transformation circuit 303, and a coefficient value calculation circuit 304. Here, the above formula (1) will be described as an example.

  The difference value calculation circuit 301 performs the process of (Bc_l−Bp_l) in the above equation (1), and is specifically a subtraction circuit.

  Next, the first orthogonal transform circuit 302 performs a multiplication process of T2 of the above formula (1) and the difference value matrix, and specifically includes, for example, a multiplication circuit and an addition circuit.

  Next, the second orthogonal transformation circuit 303 performs a multiplication process with T2_u in the above equation (1), and specifically includes, for example, a multiplication circuit and an addition circuit.

  The coefficient value calculation circuit 304 is addition with Rp in the above equation (1) and multiplication with (iT2 · Sr · T2), and specifically, a multiplication circuit and an addition circuit.

  Although not shown, the search method is not particularly limited as long as a conventionally known motion vector search (for example, full search) process is used. Note that the image processing apparatus 300 executes the processing of the above formula (5) for at least one motion vector candidate that is a search start point.

  In the above example, the second prediction block is a block in which the first prediction block is shifted by one pixel in the search direction, but the speed is increased even when M (1 ≦ M <N / 2) pixels are shifted. Can be done.

  First, for (Bc_l−Bp_l) in the above formula (1), Bc_l and Bp_l are N rows × M columns. T2_u is M rows and N columns. For (iT2 · Sr · T2), for example, when (iT2 · Sr · Sr · T2) is shifted by 2 pixels, it is (iT2 · Sr · Sr · T2) when T1 and T2 are determined. ) Is also determined, so N rows and N columns. Therefore, the calculation amount is as follows.

Bc_l-Bp_l: Subtraction N × M times Multiplication of T1 and (Bc_l−Bp_l): Multiplication N ^ 2 times, addition N (N−1) times Multiplication of (T1 · (Bc_l−Bp_l)) and T2_u: Multiplication N ^ 2 · Rp and {T1 · (Bc_l-Bp_l) · T2_u} addition: addition N ^ 2 times · [Rp + {T1 · (Bc_l-Bp_l) · T2_u}] and (iT2 · Sr · T2 ) Multiplication: multiplication N ^ 3 times, addition (N-1) · N ^ 2 times

  Multiplication is M * N ^ 2 times + M * N ^ 2 times + N ^ 3 times, and addition / subtraction is N * M times + M * N (N-1) + (M-1) * N ^ 2 times + N ^ 2 times. + (N−1) × N ^ 2 times.

  Therefore, in the case of N = 8, it is possible to increase the speed when shifting by 1 to 3 pixels. Conventionally, when N = 8, multiplication is 1024 times and addition is 896 times. In the first embodiment, when N = 8 and the second prediction block is shifted by one pixel in the horizontal direction from the first prediction block, the multiplication is 640 times and the addition is 512 times. When the second prediction block is shifted by two pixels in the horizontal direction from the first prediction block, the multiplication is 768 times and the addition is 640 times. When the second prediction block is shifted by 3 pixels in the horizontal direction from the first prediction block, the multiplication is 896 times and the addition is 768 times.

  Also, at least one vector must be subjected to the same two-dimensional orthogonal transformation as before. The number of vectors to be subjected to the same two-dimensional orthogonal transformation as in the past differs depending on the search method of motion vector search.

  FIG. 4 is an explanatory diagram showing an example of motion vector search by spiral search. For example, when motion vector search is performed by spiral search, the upper left pixel of the reference image indicated by the vector changes as shown in the figure. Each vector forms a vector sequence in which the x components or y components of adjacent vectors differ by one. Therefore, in the spiral search, the same two-dimensional orthogonal transformation as before may be performed only at the center point of the search start.

(Embodiment 2)
Next, in the second embodiment, an example in which the amount of calculation is reduced when M = 1 and T1 and T2 in the equations (1) to (4) shown in the first embodiment are Hadamard transform matrices will be described. The Hadamard transform matrix is a matrix having elements “−1” and “1”. In the second embodiment, the same reference numerals are given to the same components as those shown in the first embodiment, and the description of the components is omitted.

  Further, as described in the first embodiment, T2_u is a row vector having the uppermost row of T2 as an element, and T2_d is a row vector having the lowermost row of T2 as an element. Each element of T2_u and T2_d is “1” or “−1”.

  T2_u and T2_d are constants because they are determined when T2 is determined at the time of design. Therefore, since the difference (T1 · (Bc — 1−Bp — l)) after the one-dimensional orthogonal transformation of the above equations (1) and (2) is an N × 1 transform coefficient matrix, the N × 1 transform coefficient matrix and the T2_u Alternatively, multiplication with T2_d can be realized simply by duplicating the column vector and code conversion.

  As described in the first embodiment, T1_l is a column vector having the leftmost column of T1 as an element, and T1_r is a column vector having the rightmost column of T1 as an element. Each element of T1_l and T1_r is “1” or “−1”.

  T1_l and T1_r are constants because they are determined when T1 is determined at the time of design. Therefore, since the difference ((Bc_r−Bp_r) · T2) after the one-dimensional orthogonal transformation in the above equations (3) and (4) is a 1 × 1 transformation coefficient matrix, a 1 × N transformation coefficient matrix and T1_l And multiplication with T1_r can be realized simply by duplication of row vectors and code conversion.

  In the second embodiment, for example, when N = 4 and T1 and T2 are represented by the following equation (14), T2_d, T2_u, T1_l, and T1_r are respectively represented by the following equations (15) to (18).

  FIG. 5A is an explanatory diagram (part 1) of a specific processing example of the second orthogonal transform circuit 303 in the second embodiment. First, a column vector whose elements are k1_0 to k1_3 is the difference (T1 · (Bc_l−Bp_l)) after the one-dimensional orthogonal transformation of the above equation (1). Since each element of T2_u is “1” as in the above equation (16), three column vectors having k1_0 to k1_3 as elements are duplicated to generate a 4 × 4 element matrix.

  FIG. 5-2 is an explanatory diagram (part 2) of a specific process example of the second orthogonal transform circuit 303 in the second embodiment. First, a column vector whose elements are k2_0 to k2_3 is the difference (T1 · (Bc_l−Bp_l)) after the one-dimensional orthogonal transformation of the above equation (2). First, three column vectors whose elements are k2_0 to k2_3 are duplicated to generate a 4 × 4 matrix. Next, since the elements of T2_d are “1” in the first column and the third column and “−1” in the second column and the fourth column as shown in the above formula (15), The sign of each element in the fourth column is converted.

  FIG. 5C is an explanatory diagram (part 3) of a specific process example of the second orthogonal transform circuit 303 in the second embodiment. First, the row vector whose elements are k3_0 to k3_3 is the difference {(Bc_r−Bp_r) · T2} after the one-dimensional orthogonal transformation of the above equation (3). Since each element of T1_l is “1” as in the above equation (17), three row vectors having k3_0 to k3_3 as elements are duplicated to generate a 4 × 4 matrix.

  FIG. 5-4 is an explanatory diagram (part 4) of a specific process example of the second orthogonal transform circuit 303 in the second embodiment. First, the row vector whose elements are k4_0 to k4_3 is the difference {(Bc_r−Bp_r) · T2} after the one-dimensional orthogonal transformation of the above equation (4). First, three column vectors whose elements are k4_0 to k4_3 are duplicated to generate a 4 × 4 matrix.

  Next, since the elements of T1_r are “1” in the first and third lines and “−1” in the second and fourth lines as in the above equation (18), the second and The sign of each element in the fourth row is converted.

  Therefore, in the image processing apparatus according to the second embodiment, the amount of calculation can be reduced by setting the specific processing of the second orthogonal transform circuit 303 as replication processing and code conversion processing.

  In the above description, Rp is added after the second orthogonal transformation and the N × N element transformation coefficient matrix after the second orthogonal transformation processing. Here, instead of sign conversion as described above, addition to Rp can be replaced with subtraction for a row or column corresponding to an element of the transform coefficient matrix of “−1” instead of addition to Rp. .

  Taking the above equation (1) as an example, the image processing apparatus duplicates (N−1) N × 1 element conversion coefficient matrices by the second orthogonal conversion circuit 303 to obtain an N × N element conversion coefficient matrix. Is generated.

  Next, since T2_u is all “1”, the image processing apparatus adds the N × N element conversion coefficient matrix and the first conversion coefficient matrix by the coefficient value calculation circuit 304. Then, a second conversion coefficient matrix (Rc) is calculated by multiplying the added N × N element conversion coefficient matrix by the replacement matrix (iT 2 · Sr · T 2) related to the replacement of the matrix.

  Taking the above equation (2) as an example, the image processing apparatus 300 duplicates (N−1) N × 1 element conversion coefficient matrices by the second orthogonal transformation circuit 303 to convert N × N element conversion coefficients. Generate a matrix.

  Next, since the second column and the third column of the image processing apparatus 300 are “−1”, the coefficient value calculation circuit 304 determines that the first column of the N × N element first transform coefficient matrix is the T2_d. The first column of the N × N element conversion coefficient matrix is added. Then, the coefficient value calculation circuit 304 subtracts the second column of the N × N element first conversion coefficient matrix and the second column of the N × N element conversion coefficient matrix, and the coefficient value calculation circuit 304 calculates N × N. The third column of the first transformation coefficient matrix of the element is subtracted from the third column of the N × N element transformation coefficient matrix. Further, the coefficient value calculation circuit 304 adds the fourth column of the N × N element first conversion coefficient matrix and the fourth column of the N × N element conversion coefficient matrix. Then, the second transformation coefficient matrix (Rc) is calculated by multiplying the transformation coefficient matrix of N × N elements after addition and subtraction by the exchange matrix (iT2, S1, and T2) related to the matrix exchange.

  Taking the above equation (3) as an example, the image processing apparatus 300 uses the second orthogonal transform circuit 303 to replicate (N−1) 1 × N element conversion coefficient matrices to generate N × N element conversion coefficients. Generate a matrix.

  Next, since all of the elements of T1_l are “1”, the image processing apparatus 300 adds the N × N element conversion coefficient matrix and the first conversion coefficient matrix by the coefficient value calculation circuit 304. Then, a second conversion coefficient matrix (Rc) is calculated by multiplying the added N × N element conversion coefficient matrix by the replacement matrix (T1, Sd, iT1) related to the replacement of the matrix.

  Taking the above equation (2) as an example, the image processing apparatus 300 duplicates (N−1) N × 1 element conversion coefficient matrices by the second orthogonal transformation circuit 303 to convert N × N element conversion coefficients. Generate a matrix.

  Next, since the image processing apparatus 300 determines that T1_r is “−1” in the second and fourth rows, the coefficient value calculation circuit 304 determines that the first row of the first transformation coefficient matrix of N × N elements The first row of the N × N element conversion coefficient matrix is added. Then, the coefficient value calculation circuit 304 subtracts the second row of the N × N element first conversion coefficient matrix and the second row of the N × N element conversion coefficient matrix, and the coefficient value calculation circuit 304 calculates N × N. The third row of the first transformation coefficient matrix of the element and the third row of the N × N element transformation coefficient matrix are added. Further, the coefficient value calculation circuit 304 subtracts the fourth row of the N × N element first transformation coefficient matrix and the fourth row of the N × N element transformation coefficient matrix. Then, the second transformation coefficient matrix (Rc) is calculated by multiplying the transformation coefficient matrix of N × N elements after addition / subtraction by the exchange matrix (T1 · Su · iT1) relating to the matrix exchange.

  Therefore, in the image processing apparatus 300 according to the second embodiment, by using an orthogonal transformation matrix as a Hadamard matrix, the processing of the first orthogonal transformation circuit 302 can be realized only by addition processing or subtraction processing, and the amount of computation can be reduced. . Furthermore, the specific processing of the second orthogonal transformation circuit 303 is a replication processing, and the addition processing in the coefficient value calculation circuit 304 is an addition processing or a subtraction processing based on the second orthogonal transformation matrix, thereby reducing the amount of computation. Can be reduced.

(Embodiment 3)
Next, in the third embodiment, the speed-up process in the case of only N = 4 will be described. In Embodiment 3, the process of calculating the orthogonal transformation result of the current vector from the orthogonal transformation of the target block in the original image is speeded up without calculating the orthogonal transformation result of the current vector.

  First, Sr, Sd, Su, and S1 are the following expressions (19) and (20), respectively. Furthermore, since T1 and T2 are determined at the time of design, (iT2 · Sr · T2) in the above formula (1) is the following formula (21), and (iT2 · Sl · T2) in the above formula (2). Is the following formula (22). And (T1 * Sd * iT1) of the above-mentioned formula (3) is the following formula (23), and (T1 * Su * iT1) of the above-mentioned formula (4) is the following formula (24).

  The above formulas (1) to (4) are formulas related to the calculation of Rc. In SATD, Rc is subtracted from the conversion result (C) of the target block in the original image, and the absolute values of the respective elements are summed.

  Since (iT2 · Sr · T2) in the above equation (1) is the above equation (21), (iT2 · Sr · T2) is to the right of the 4 × 4 matrix as in the above equation (1). When multiplied, the following equation (25) is obtained.

  A matrix of 4 × 4 elements having T1_0 to T1_15 as elements is the calculation result of [Rp + {T1 · (Bc_l−Bp_l)} · T2_u] in the above equation (1). Therefore, multiplication from the right of (iT2, Sr, T2) is performed by switching the third and fourth columns of the 4 × 4 element matrix and inverting the signs of the second and fourth columns after the replacement. It corresponds to doing.

  FIG. 6A is an explanatory diagram (part 1) of an example in which multiplication processing with a replacement matrix is replaced with replacement processing and code conversion processing. H1_0 to H1_3 are the first column, H1_4 to H1_7 are the second column, H1_8 to H1_11 are the third column, and H1_12 to H1_15 are the fourth column.

  First, the third and fourth columns are interchanged, and then the codes of the second and fourth columns are converted. As a result, the multiplication process of the 4 × 4 element matrix and (iT2, Sr, T2) is only the column replacement process and the sign inversion process, and the speed can be increased.

  Then, the 4 × 4 element matrix after the inversion process is subtracted from the target block.

  Next, since (iT2 · S1 · T2) in the above equation (2) is the above equation (22), the (iT2 · S1 · T2) is calculated for a 4 × 4 element matrix as in the above equation (2). ) From the right, the following equation (26) is obtained.

  Therefore, multiplication from the right of (iT2, S1, T2) is performed by switching the third and fourth columns of the 4 × 4 element matrix and inverting the signs of the second and third columns after the replacement. It corresponds to doing.

  FIG. 6B is an explanatory diagram (part 2) of the example in which the multiplication process with the exchange matrix is replaced with the exchange process and the code conversion process. H2_0 to H2_3 are the first column, H2_4 to H2_7 are the second column, H2_8 to H2_11 are the third column, and H2_12 to H2_15 are the fourth column. First, the third and fourth columns are interchanged, and then the codes in the second and third columns are converted. As a result, the multiplication process of the 4 × 4 element matrix and (iT2, S1, T2) is only the column replacement process and the sign inversion process, and the speed can be increased.

  Next, since (T1 · Sd · iT1) in the above equation (3) is the above equation (23), (T1 · Sd · iT1) is applied to a 4 × 4 element matrix as in the above equation (3). ) From the left, the following equation (27) is obtained.

  Therefore, (T1 · Sd · iT1) is multiplied by the 4 × 4 element matrix from the left to replace the third and fourth columns of the 4 × 4 element matrix, and the second column after the replacement. Is equivalent to reversing the sign of the third column.

  FIG. 6C is an explanatory diagram (part 3) of the example in which the multiplication process with the exchange matrix is replaced with the exchange process and the code conversion process. H3_0, H3_4, H3_8, H3_12 are the first row, H3_1, H3_5, H3_9, H3_13 are the second row, H3_2, H3_6, H3_10, H3_14 are the third row, H3_3, H3_7, H3_11 , H3_15 is the fourth row. First, the third and fourth lines are interchanged, and then the codes of the second and third lines are converted. As a result, the multiplication process of the 4 × 4 element matrix and (T1, Sd, iT1) is only the column replacement process and the sign inversion process, and the speed can be increased.

  Next, since (T1 · Su · iT1) in the above equation (4) is the above equation (24), (T1 · Su · iT1) is applied to a 4 × 4 element matrix as in the above equation (4). ) From the left, the following equation (28) is obtained.

  Therefore, (T1 · Su · iT1) is multiplied by the 4 × 4 element matrix from the left to replace the third row and the fourth row of the 4 × 4 element matrix, and the second row after the replacement. Is equivalent to reversing the sign of the fourth row.

  6-4 is explanatory drawing (the 4) which shows the example by which the multiplication process with the permutation matrix was replaced by the permutation process and the code conversion process. H4_0, H4_4, H4_8, H4_12 are the first row, H4_1, H4_5, H4_9, H4_13 are the second row, H4_2, H4_6, H4_10, H4_14 are the third row, H4_3, H4_7, H4_11 , H4_15 is the fourth row.

  First, the third and fourth lines are interchanged, and then the codes of the second and fourth lines are converted. As a result, the multiplication process of the 4 × 4 element matrix and (T1, Su, iT1) is only the column replacement process and the sign inversion process, and the speed can be increased.

  In the multiplication processing related to the calculation of the coefficient value described above, the code is converted after the replacement processing. In the motion vector detection process, actually, after the second transform coefficient matrix relating to the second prediction block is calculated, the second transform coefficient matrix is subtracted from the third transform coefficient matrix relating to the target block. In the multiplication process relating to the calculation of the coefficient value, only the replacement process is performed, and a part of the process when the second conversion coefficient matrix is subtracted from the third conversion coefficient matrix (Rp) is replaced with addition, whereby the third conversion is performed. The difference between the coefficient matrix (Rp) and the second transform coefficient matrix can be calculated.

  Taking the above equation (1) as an example, the third column and the fourth column of the N × N element conversion coefficient matrix are exchanged as described above. Then, the first column of the second conversion coefficient matrix is subtracted from the first column of the third conversion coefficient matrix (Rp), and the second conversion coefficient matrix is calculated from the second column of the third conversion coefficient matrix (Rp). Are added to the second column. Further, the third column of the second conversion coefficient matrix is subtracted from the third column of the third conversion coefficient matrix (Rp), and the second conversion coefficient matrix is calculated from the fourth column of the third conversion coefficient matrix (Rp). Are added to the fourth column.

  Taking the above equation (2) as an example, the third column and the fourth column of the N × N element conversion coefficient matrix are exchanged as described above. Then, the first column of the second conversion coefficient matrix is subtracted from the first column of the third conversion coefficient matrix (Rp), and the second conversion coefficient matrix is calculated from the second column of the third conversion coefficient matrix (Rp). Are added to the second column. Furthermore, the third column of the second transform coefficient matrix is added from the third column of the third transform coefficient matrix (Rp), and the second transform coefficient matrix is added from the fourth column of the third transform coefficient matrix (Rp). The fourth column is subtracted.

  Taking the above equation (3) as an example, the third row and the fourth row of the N × N element conversion coefficient matrix are interchanged as described above. Then, the first row of the second transformation coefficient matrix is subtracted from the first row of the third transformation coefficient matrix (Rp), and the second transformation coefficient matrix is obtained from the second row of the third transformation coefficient matrix (Rp). Add the second line of. Further, the third row of the second transformation coefficient matrix is added from the third row of the third transformation coefficient matrix (Rp), and the second transformation coefficient matrix is obtained from the fourth row of the third transformation coefficient matrix (Rp). The 4th line is subtracted.

  Note that the N × N element transform coefficient matrix after replacement is different from the second transform coefficient matrix related to the second prediction block.

  Taking the above equation (4) as an example, the third row and the fourth row of the N × N element conversion coefficient matrix are interchanged as described above. Then, the first row of the second transformation coefficient matrix is subtracted from the first row of the third transformation coefficient matrix (Rp), and the second transformation coefficient matrix is obtained from the second row of the third transformation coefficient matrix (Rp). Add the second line of. Further, the third row of the second transformation coefficient matrix is subtracted from the third row of the third transformation coefficient matrix (Rp), and the second transformation coefficient matrix is obtained from the fourth row of the third transformation coefficient matrix (Rp). Add the 4th line. Thereby, the amount of calculation can be reduced.

  Further, the N × N element transform coefficient matrix after replacement is different from the second transform coefficient matrix related to the second prediction block. Therefore, since the second transform coefficient matrix (Rc) is not calculated, another value is used to obtain the SATD of the next vector using the current vector.

  Here, only the above formula (2) will be described as an example. The above equation (22) can be decomposed into two matrices of S and L as in the following equation (29).

  When (T1 · (Bc — 1−Bp — 1)) · T2_d is B, the following equation (30) is obtained.

  C−Rc = C− (Rp + B) · S · L (30)

  In the above equation (29), L can be realized by replacing a part of subtraction with C by addition. Here, the addition / subtraction with C and the replacement processing are expressed by using ◎, as shown in the following formula (31).

  C−Rc = C ◎ (Rp + B) · S (31)

  In order to calculate the conversion result of the next vector using the conversion result of the current vector, iL (inverse matrix of L) · Rc is used instead of Rc. Even if the next vector is moved by one pixel in either the up, down, left, or right direction of the current vector, the operation L · (iM · Rp) + Pc or Pc + M · (iM · Rp) is performed, and the left of M The multiplication from can be replaced with subtraction as part of the addition of Pc.

  Next, the third embodiment is described in H.264. An example of the configuration of a motion vector search unit when applied to an H.264 encoder will be described. Here, the circuit configuration from the acquisition of the original image and the reference image from the frame memory to the portion where the SATD calculation is performed, instead of the entire motion vector search unit will be described.

  FIG. 7 is an explanatory diagram illustrating an example of a macroblock. FIG. 7 shows the current macroblock in the original image and the reference macroblock in the same position as the target macroblock in the reference image. The current macroblock size and the reference macroblock size are a set of 4 × 4 blocks, and each block size is 4 × 4 pixels.

  In Embodiment 3, motion vector search is performed for each macroblock, and SATDs of five vectors arranged in the horizontal direction are calculated in parallel. Here, an example of five vectors 0 to 4 arranged in the horizontal direction will be described.

  There are BK1_1 to BK1_4 in the macroblock in the original image. There are BK2_1 to BK2_4 in the macroblock in the reference image. Here, BK2_1 to BK2_4 are blocks pointed to by vector 0 from BK1_1 to BK1_4 in the original image. The block in the reference image at the same position as BK1_1 is BK2_1. Therefore, the motion vector indicating BK2_1 is the vector 0 regarding BK1_1.

  Further, the block in the reference image at the same position as BK1_2 is BK2_2, and the motion vector indicating BK2_2 is the vector 0 regarding BK1_2. The block in the reference image at the same position as BK1_3 is BK2_3, and the motion vector indicating BK2_3 is the vector 0 regarding BK1_3. The block in the reference image at the same position as BK1_4 is BK2_4, and the motion vector indicating BK2_4 is vector 0 regarding BK1_4.

  Taking a vector related to BK1_1 as an example, a motion vector indicating a block obtained by shifting BK2_1 by -1 in the horizontal direction is a vector 1 related to BK1_1. Furthermore, a motion vector indicating a block shifted by −1 in the horizontal direction is a vector 2 regarding BK1_1, and thereafter, vectors 3 and 4 regarding BK1_1 are illustrated. As described above, in the third embodiment, SATDs for five vectors 0 to 4 are calculated in parallel.

  Here, it is assumed that a conversion result of four points on the left side of the rightmost vector is obtained based on the conversion result of the rightmost vector among the five vectors. The subscripts in the following mathematical expressions represent the difference of the x coordinate from the rightmost vector of the target vector.

  The difference Dx between the conversion result of the block in the original image of each vector and the conversion result of the block in the reference image is as follows.

・ D0 = C−R0
· D1 = C-R1 = C-(R0ΔB1) · S · L = C ◎ (R0ΔB1) · S
· D2 = C-R2 = C-(R1ΔB2) · S · L = C ◎ (R1ΔB2) · S
D3 = C−R3 = C− (R2ΔB3) S S L = C ◎ (R2ΔB3) S
・ D4 = C−R4 = C− (R3ΔB4) • S • L = C ◎ (R3ΔB4) • S

  Here, Bn is a 4 × 4 matrix in which four column vectors (T1 · (Bc_l−Bp_l)) are arranged, Δ and ◎ are matrix components at the same position, and the first and third columns are added, Columns 2 and 4 are operators that perform an operation of subtracting the right side component from the left side component. The sum of absolute values of D0 to D4 is SATD in each vector.

  If R1 = (R0ΔB1) · S · L and P1 = (R0ΔB1) · S, then R1ΔB2 = P1 · LΔB2 = P1１B2. When M is multiplied from the right, it is a matrix that inverts the sign of the second column and the fourth column, so ▲ is equal to the normal matrix addition operator +. D2 to D3 are modified by this method to finally obtain:

  D0 = C−R0 (32)

  D1 = C−R1 = C ◎ (R0ΔB1) · S (33)

  D2 = C−R2 = C ◎ (P1 + B2) · S (34)

  D3 = C−R3 = C ◎ (P2 + B3) · S (35)

  D4 = C−R4 = C ◎ (P3 + B4) · S (36)

  FIG. 8 is an explanatory diagram of an example of the image processing apparatus according to the third embodiment. The image processing apparatus 800 illustrated in FIG. 8 includes a selector, a two-dimensional Hadamard transform circuit, a predicted block transform result holding circuit, a target block transform result holding circuit, a predicted block holding circuit, and units 0 to 4. It is the composition provided. Next, unit 0 to unit 4 will be described.

  Unit 0 includes units 0_1 to unit 0_4, and each has the same configuration. In unit 0, the SATD corresponding to vector 0 is calculated.

  The unit 1 includes units 1_1 to 1_4, each having the same configuration. In unit 1, SATD corresponding to vector 1 is calculated.

  The unit 2 includes units 2-3_1 to 2-3_4, the unit 3 includes units 2-3_5 to 2-3_8, and the units 2-3_1 to 2-3_8 have the same configuration. In unit 2, the SATD corresponding to vector 2 is calculated, and in unit 3, the SATD corresponding to vector 3 is calculated.

  The unit 4 includes a unit 2-3_9 and units 0_5 to 0_7. The unit 2-3_9 has the same configuration as the unit 2-3_1, and the units 0_5 to 0_7 have the same configuration as the unit 0_1. Unit 4 calculates SATD corresponding to vector 4.

  FIG. 9 is an explanatory diagram showing a detailed description of the units 0_1 to 0_7. Each of the units 0_1 to 0_7 constitutes the above equation (32), and each includes four subtraction circuits, four absolute value calculation circuits, and a sum calculation circuit.

  In units 0_1 to 0_7, the difference between the two-dimensional orthogonal transformation result of the target block in the original image and the two-dimensional orthogonal transformation result of the block in the reference image is calculated, the respective absolute values are taken, and the sum is calculated. Output SATD.

  FIG. 10 is an explanatory diagram showing a detailed description of the units 1_1 to 1_4. Each of the units 1_1 to 1_4 constitutes the above equation (33), and includes a subtraction circuit, a difference holding circuit, a one-dimensional Hadamard transform circuit, a Δ operation circuit, a column replacement circuit, an ◎ operation circuit, and an absolute value. Four arithmetic circuits and a total arithmetic circuit are provided.

  In the subtraction circuit, a subtraction process corresponding to Bc_l−Bp_l is performed, and a difference value is calculated. Then, the subtraction result is output to the difference holding circuit. The difference holding circuit has a double buffer configuration for holding the subtraction result, and simultaneously receiving the difference for a certain block and outputting the difference for the next one-dimensional Hadamard transformation.

  The one-dimensional Hadamard transform circuit performs a first orthogonal transform process. Therefore, in the one-dimensional Hadamard transform circuit, addition / subtraction corresponding to T1 is performed. In the Δ arithmetic circuit, as described above, the first column and the third column are added, and the second column and the fourth column are calculated by subtracting the right side component from the left side component.

  Next, the column replacement circuit performs processing to replace the columns, and in the arithmetic circuit, as described above, the first column and the third column are added, and the second column and the fourth column are calculated by subtracting the right side component from the left side component. To do. Since the absolute value calculation circuit and the sum calculation circuit are known, the description thereof is omitted.

  Returning to FIG. 8, in the unit 1, an operation corresponding to ΔB <b> 1 is performed on the two-dimensional orthogonal transformation result (R <b> 0) of the prediction block passed from the unit 0. B1 is obtained by performing one-dimensional Hadamard transformation on the difference held by the difference holding circuit by the one-dimensional Hadamard transformation circuit. A column replacement process corresponding to S is performed on R0ΔB1, and P1 is obtained. P1 is obtained together with the original image conversion result. The arithmetic circuit obtains D1 from P1 and the original image conversion result, and the sum of absolute values is calculated as SATD. P1 is also sent to unit 2. The SATD calculated by this unit is a vector 1 that puts one vector 0 to the left.

  Therefore, unit 1 receives R0 as the result of the two-dimensional orthogonal transformation of the previous vector, and Pn is passed from unit n to unit n + 1.

  FIG. 11 is an explanatory diagram showing a detailed description of the units 2-3_1 to 2-3_9. In the units 2-3_1 to 2-3_9, the above formulas (34) to (36) are configured, respectively, and a subtraction circuit, a difference holding circuit, a one-dimensional Hadamard transform circuit, a + arithmetic circuit, and a column replacement circuit, , ◎ A calculation circuit, four absolute value calculation circuits, and a sum calculation circuit are provided.

The units 2-3_1 to 2-3_9 are 1) a + arithmetic circuit (adder circuit) instead of the Δ arithmetic circuit.
2) P1 is input instead of R0.
3) A difference result at a position where the input to the difference holding circuit is shifted to the left by one pixel.

  Except for the above 1) to 3), the same calculation is performed at the same timing as the units 1_1 to 1_4.

  Returning to FIG. 8, first, macroblocks in the processing target original image and macroblocks in the reference image are acquired from the frame memory. The macroblocks in the original image are subjected to two-dimensional Hadamard transform processing for each block while being fetched from the frame memory, and stored in the target block transform result holding circuit.

  Also, the four blocks that are vector 0 while being fetched from the frame memory for the macroblocks in the reference image are subjected to the two-dimensional Hadamard transform process for each block and stored in the predicted block transform result holding circuit. Here, Hadamard transform processing is performed on B2_1 to B2_4. Furthermore, the macroblock in the reference image is stored in the prediction block holding circuit even in a state where it is not converted.

  The image in the prediction block holding circuit is used during the subsequent SATD calculation process. For example, if the image has a prefetch mechanism or if a sufficient bandwidth with the frame memory can be secured, the prediction block holding is performed. A configuration without a circuit is also possible.

  In the image processing apparatus 800 of FIG. 8, five vectors having the same vertical component (y component) are processed in parallel based on the full search, and further, the SATD for 20 vectors is calculated by repeating four times. Conventionally, the number of additions / subtractions is 30400, but in the image processing apparatus 800 of FIG. 8, the number of additions / subtractions is 24256. Note that the absolute value calculation count is 5120 times in all cases.

  Further, the two-dimensional Hadamard transformation circuit that converts the original image and the reference image shown in FIG. 8 when fetching from the frame memory can be shared with the one-dimensional Hadamard transformation circuit used in each unit.

  In the third embodiment, the above formula (2) is taken as an example, and the configuration of the image processing apparatus is described based on the full search. However, the present invention is not limited to this, and the above formulas (1) to (4) and Other search methods may be used.

(Embodiment 4)
Next, a fourth embodiment will be described. In the fourth embodiment, an example in which each function of the image processing apparatus shown in the first embodiment is realized by processing by a computer will be described.

(Hardware configuration of image processing device)
FIG. 12 is a block diagram of a hardware configuration of the image processing apparatus according to the embodiment. In FIG. 12, an image processing apparatus includes a CPU (Central Processing Unit) 1201, a ROM (Read-Only Memory) 1202, a RAM (Random Access Memory) 1203, a magnetic disk drive 1204, a magnetic disk 1205, and an optical disk drive. 1206, an optical disc 1207, a display 1208, an I / F (Interface) 1209, a keyboard 1210, a mouse 1211, a scanner 1212, and a printer 1213. Each component is connected by a bus 1200.

  Here, the CPU 1201 controls the entire image processing apparatus. The ROM 1202 stores programs such as a boot program. The RAM 1203 is used as a work area for the CPU 1201. The magnetic disk drive 1204 controls reading / writing of data with respect to the magnetic disk 1205 under the control of the CPU 1201. The magnetic disk 1205 stores data written under the control of the magnetic disk drive 1204.

  The optical disk drive 1206 controls reading / writing of data with respect to the optical disk 1207 according to the control of the CPU 1201. The optical disk 1207 stores data written under the control of the optical disk drive 1206, and causes the computer to read data stored on the optical disk 1207.

  A display 1208 displays data such as a document, an image, and function information as well as a cursor, an icon, or a tool box. As the display 1208, for example, a CRT, a TFT liquid crystal display, a plasma display, or the like can be adopted.

  An interface (hereinafter abbreviated as “I / F”) 1209 is connected to a network 1214 such as a LAN (Local Area Network), a WAN (Wide Area Network), and the Internet through a communication line, and the other via the network 1214. Connected to other devices. The I / F 1209 controls an internal interface with the network 1214 and controls input / output of data from an external device. For example, a modem or a LAN adapter may be employed as the I / F 1209.

  The keyboard 1210 includes keys for inputting characters, numbers, various instructions, and the like, and inputs data. Moreover, a touch panel type input pad or a numeric keypad may be used. The mouse 1211 performs cursor movement, range selection, window movement, size change, and the like. A trackball or a joystick may be used as long as they have the same function as a pointing device.

  The scanner 1212 optically reads an image and takes in image data into the image processing apparatus. The scanner 1212 may have an OCR (Optical Character Reader) function. The printer 1213 prints image data and document data. As the printer 1213, for example, a laser printer or an ink jet printer can be employed.

(Functional configuration of image processing apparatus)
FIG. 13 is a block diagram of a functional configuration of the image processing apparatus according to the fourth embodiment. The image processing apparatus 1300 includes a difference value calculation unit 1301, a first orthogonal transformation unit 1302, a second orthogonal transformation unit 1303, a coefficient value calculation unit 1304, a coefficient difference value calculation unit 1305, and an absolute value calculation unit. 1306, a sum operation unit 1307, and a two-dimensional orthogonal transform unit 1308. Specifically, each function (difference value calculation unit 1301 to two-dimensional orthogonal transformation unit 1308) is, for example, a program stored in a storage device such as the ROM 1202, the RAM 1203, the magnetic disk 1205, and the optical disk 1207 shown in FIG. Are executed by the CPU 1201 or by the I / F 1209.

  Here, N × N pixels are defined as one block. In the fourth embodiment, the two-dimensional orthogonal transform process of the second prediction block indicated by the current vector when the first prediction block indicated by the previous vector in the reference image is shifted +1 in the horizontal direction will be described. . In the fourth embodiment, only the process of calculating the sum of absolute value differences (SATD) related to motion vector candidates will be described.

  First, the two-dimensional orthogonal transform unit 1308 has a function of performing two-dimensional orthogonal transform on the first prediction block indicated by the previous block in the reference image using an N × N element transform coefficient matrix. Specifically, for example, the first transform coefficient matrix related to the first prediction block is calculated by multiplying the pixel value matrix related to the first prediction block by T1 from the left and T2 from the right.

  Further, the two-dimensional orthogonal transform unit 1308 has a function of performing a two-dimensional orthogonal transform on a third block (target block in the above description) in the original image using an N × N element transform coefficient matrix. Specifically, for example, the above-described T1 is applied from the left and T2 is applied from the right to the pixel value matrix related to the target block to calculate the third transform coefficient matrix related to the target block.

  Then, the coefficient difference value calculation unit 1305 performs the first prediction in the reference image calculated by the two-dimensional orthogonal transformation unit from the third transformation coefficient matrix related to the target block in the original image calculated by the two-dimensional orthogonal transformation unit. A coefficient difference value with respect to the first transform coefficient matrix relating to the block is calculated.

  The absolute value calculation unit 1306 has a function of calculating the absolute value of the coefficient difference value. Further, the sum calculation unit 1307 calculates the sum of elements in the calculated coefficient difference value to calculate a first absolute value difference sum (SATD).

  Next, the difference value calculation unit 1301 overlaps the first pixel value matrix that does not overlap the first prediction block in the second prediction block and the second prediction block in the first prediction block. And a function of calculating a difference value matrix that is a difference from the second pixel value matrix that has not been performed. Specifically, for example, the CPU 1201 calculates a difference value matrix by subtracting Bp_l from Bc_l.

  Next, the first orthogonal transform unit 1302 performs one-dimensional orthogonal transform on the N × 1 element difference value matrix calculated by the difference value calculation unit 1301 using an N × N element orthogonal transform matrix, thereby obtaining N × 1. It has a function of calculating a conversion coefficient matrix of elements. Specifically, for example, the CPU 1201 performs one-dimensional orthogonal transformation by multiplying the difference value matrix by an orthogonal transformation matrix of N × N elements from the left.

  The second orthogonal transformation unit 1303 performs one-dimensional orthogonal transformation on the N × 1 element transformation coefficient matrix calculated by the first orthogonal transformation unit 1302 using a 1 × N element orthogonal transformation matrix, and obtains N × N. It has a function of calculating a conversion coefficient matrix of elements. Specifically, for example, the CPU 1201 performs one-dimensional orthogonal transformation by multiplying the difference value matrix by an orthogonal transformation matrix of 1 × N elements from the right.

  The coefficient value calculation unit 1304 is based on the N × N element first transform coefficient matrix related to the first prediction block and the N × N element transform coefficient matrix transformed by the first orthogonal transform unit 1302. A function of calculating a second transform coefficient matrix of N × N elements for two prediction blocks;

  Next, the coefficient difference value calculation unit 1305 outputs the second value in the reference image calculated by the coefficient value calculation unit 1304 from the third conversion coefficient matrix related to the target block in the original image calculated by the two-dimensional orthogonal transformation unit. A coefficient difference value with respect to the second transform coefficient matrix related to the prediction block is calculated.

  The absolute value calculation unit 1306 has a function of calculating the absolute value of the coefficient difference value. Further, the sum calculation unit 1307 calculates the sum of the elements in the calculated coefficient difference value to calculate a second absolute value difference sum (SATD).

  Next, detailed examples of the coefficient value calculation unit 1304 and the first orthogonal transform unit 1302 and other examples of the coefficient difference value calculation unit 1305 are shown.

(Example 1)
Here, the two-dimensional orthogonal transformation of the second prediction block of N × N pixels obtained by shifting the first prediction block of N × N pixels by M (1 <= M <N) elements will be described. Since the functional units other than the first orthogonal transform unit 1302 and the coefficient value calculation unit 1304 are the same as described above, the description thereof is omitted.

  Taking the above equation (1) as an example, the first orthogonal transform unit 1302 multiplies an N × M element transform coefficient matrix by an M × N element orthogonal transform matrix to obtain an N × N element transform coefficient matrix. Is calculated.

  Next, FIG. 14A is an explanatory diagram (part 1) of a detailed configuration of the coefficient value calculation unit 1304. The coefficient value calculation unit 1304 includes an addition unit 1401 and a replacement unit 1402. The adding unit 1401 has a function of adding the N × N element first transform coefficient matrix related to the first prediction block and the N × N element transform coefficient matrix transformed by the first orthogonal transform unit 1302. .

  Then, the replacement unit 1402 multiplies the N × N element conversion coefficient matrix after the addition related to the first prediction block by the N × N element replacement matrix from the right to thereby calculate N × N related to the second prediction block. Calculate a second transform coefficient matrix of the elements.

(Example 2)
Here, when the orthogonal transformation matrix (T1 and T2) is a Hadamard transformation matrix, the two-dimensional orthogonal transformation of the second prediction block of N × N pixels obtained by shifting the first prediction block of N × N pixels by one element. explain. Since the functional units other than the first orthogonal transform unit 1302 and the coefficient value calculation unit 1304 are the same as described above, the description thereof is omitted.

  FIG. 14B is an explanatory diagram of a detailed configuration of the first orthogonal transform unit 1302. The first orthogonal transform unit 1302 includes a generation unit 1403 and a code conversion unit 1404. The generation unit 1403 generates an N × N element transform coefficient matrix by arranging N N × 1 element transform coefficient matrices calculated by the first orthogonal transform unit 1302 in the search direction. Detailed processing will be described by taking the above formula (1) as an example.

  Specifically, for example, the CPU 1201 replicates N−1 transform coefficient matrices of N × 1 elements calculated by the first orthogonal transform unit 1302, and replicates N−1 transform coefficient matrices. One N × 1 element transform coefficient matrix is arranged in the search direction (here, the horizontal direction).

  Next, the code conversion unit 1404 converts the code of the N × N element conversion coefficient matrix generated by the generation unit 1403 based on the 1 × N element orthogonal conversion matrix (T2_u in the above equation (1)). . Specifically, for example, the CPU 1201 outputs all the elements of T2_u without conversion because the codes of the N × N element conversion coefficient matrix are all “1”. When there is “−1” as in T2_d, the sign of the corresponding column vector is inverted in the N × N element conversion coefficient matrix.

  Then, the coefficient value calculation unit 1304 performs the second calculation based on the N × N element first conversion coefficient matrix related to the first prediction block and the N × N element conversion coefficient matrix converted by the code conversion unit 1404. A second transformation coefficient matrix of N × N elements related to the prediction block is calculated. Specifically, it is the same as the processing in Example 1.

(Example 3)
Here, although the same conditions as Example 2 are shown, an example of a different process is shown. Since the functional units other than the first orthogonal transform unit 1302 and the coefficient value calculation unit 1304 are the same as described above, the description thereof is omitted.

  The first orthogonal transform unit 1302 is configured by a generation unit 1403. The generation unit 1403 generates an N × N element transform coefficient matrix by arranging N N × 1 element transform coefficient matrices calculated by the first orthogonal transform unit 1302 in the search direction. Since the processing of the generation unit 1403 is the same as that in Example 2, description thereof is omitted.

  FIG. 14C is an explanatory diagram (part 2) of the detailed configuration of the coefficient value calculation unit 1304. The coefficient value calculation unit 1304 includes an addition / subtraction unit 1405 and an exchange unit 1406. The addition / subtraction unit 1405 converts the N × N element first transform coefficient matrix related to the first prediction block and the N × N element transform coefficient matrix generated by the generation unit 1403 into the N × N element transform coefficient matrix. When the element in the orthogonal transformation matrix of 1 × N elements corresponding to is 1, addition is performed, and when the element is −1, subtraction is performed.

  Then, the replacement unit 1406 multiplies the N × N element conversion coefficient matrix added / subtracted by the addition / subtraction unit 1405 by the N × N element replacement matrix to calculate a second conversion coefficient matrix related to the second prediction block. To do. Since the addition / subtraction unit 1405 and the replacement unit 1406 are the same as the processing described in the third embodiment, detailed description thereof is omitted.

(Example 4)
Here, a case where N is 4 and the orthogonal transformation matrices (T1 and T2) are Hadamard transforms will be described. Since only the replacement unit 1402 shown in Example 1 is different, description of other functional units is omitted.

  The replacement unit 1402 replaces the N × N element conversion coefficient matrix added by the addition unit 1401 with the row or column order based on the N × N element replacement matrix, and the N × N element conversion coefficient after the replacement. The code is converted based on the permutation matrix of N × N elements for each row or column in the matrix to calculate a second conversion coefficient matrix for the second prediction block. Specifically, since it is the same as the processing shown in FIGS.

(Example 5)
Here, although the same conditions as Example 4 are shown, an example of a different process is shown. Since the functional units other than the switching unit 1402 and the coefficient difference value calculating unit 1305 shown in Example 4 are the same as described above, the description thereof will be omitted.

  The replacement unit 1402 switches the order of the rows or columns of the N × N element conversion coefficient matrix added by the addition unit 1401 based on the N × N element replacement matrix. Specifically, among the processes shown in FIGS. 6-1 to 6-4, the process is the same as the process related to matrix replacement, and thus the description thereof is omitted.

  The coefficient difference value calculation unit 1305 corresponds to the N × N element conversion coefficient matrix replaced by the third conversion coefficient matrix related to the target block in the original image by the replacement unit 1402. When the value of the replacement matrix to be calculated is 1, subtraction is performed, and when the replacement matrix is −1, addition is performed to calculate a coefficient difference value amount sequence. Since a specific processing example is the same as the example shown in the third embodiment, the description thereof is omitted.

(Image processing procedure by image processing device)
FIG. 15 is a flowchart illustrating an example of an image processing procedure performed by the image processing apparatus. Here, a case where the motion vector search method is a spiral search will be described as an example. As described in the first embodiment, in the case of spiral search, the conventional two-dimensional orthogonal transformation only needs to be performed on the prediction block at the search start point in the reference image. First, the image processing apparatus determines the next motion vector candidate as the current vector (step S1501), and determines whether it is the first vector (step S1502). Whether or not it is the first vector indicates whether or not it is a search start point.

  Next, when the image processing apparatus determines that the vector is the first vector (step S1502: Yes), the first two-dimensional orthogonal transformation process is performed (step S1503). On the other hand, when the image processing apparatus determines that the vector is not the first vector (step S1502: No), the second two-dimensional orthogonal transformation process is performed (step S1504).

  After step S1503 or step S1504, the image processing apparatus calculates the absolute value of each difference (step S1506), calculates the sum of absolute values (SATD) (step S1507), and stores it (step S1508).

  Then, the image processing apparatus determines whether or not the search end condition is satisfied (step S1509). When it is determined that the search end condition is not satisfied (step S1509: No), the process returns to step S1501 and when it is determined that it is satisfied (step S1509). : Yes), a series of processing ends.

  Next, FIG. 16 is a flowchart showing a detailed description of the first two-dimensional orthogonal transform process (step S1503) shown in FIG. Here, regarding the transformation matrix of the two-dimensional orthogonal transformation, the processing is the same regardless of whether it is a Hadamard transformation matrix or another transformation matrix. First, the image processing apparatus performs two-dimensional orthogonal transform on the target block in the original image (step S1601), and stores a transform coefficient matrix related to the target block (step S1602).

  Next, the image processing apparatus performs two-dimensional orthogonal transform on the prediction block indicated by the current vector in the reference image (step S1603), and stores a transform coefficient matrix related to the prediction block (step S1604). Then, the image processing apparatus calculates the difference between the transform coefficient matrix related to the target block and the transform coefficient matrix related to the prediction block (step S1605), and proceeds to step S1506.

  Next, FIG. 17 is a flowchart showing a detailed description of the second two-dimensional orthogonal transformation process (step S1504) shown in FIG. First, the image processing apparatus acquires a first pixel value matrix and a second pixel value matrix (step S1701), and calculates a difference between the first pixel value matrix and the second pixel value matrix (step S1702). ). Next, the image processing apparatus performs one-dimensional orthogonal transformation of the difference value matrix using the N × N transformation matrix (step S1703), and N × 1 after the one-dimensional orthogonal transformation using the 1 × N transformation coefficient matrix. Are transformed by one-dimensional orthogonal transformation (step S1704).

  Then, the image processing apparatus calculates an orthogonal transform coefficient matrix related to the prediction block indicated by the current vector (step S1705) and stores it (step S1706). Next, the image processing apparatus calculates the difference between the transform coefficient related to the target block and the transform coefficient related to the prediction block (step S1707), and proceeds to step S1506.

  FIG. 18 is a flowchart showing a detailed description of the second two-dimensional orthogonal transform process (step S1505) shown in FIG. Here, in the case where N = 4, M = 1, and the transformation matrix is a Hadamard matrix, it is shown that the amount of calculation is reduced compared to the conventional case. First, the image processing apparatus acquires a first pixel value matrix and a second pixel value matrix (step S1801), and calculates a difference between the first pixel value matrix and the second pixel value matrix (step S1802). ). Next, the image processing apparatus performs one-dimensional orthogonal transformation of the difference value matrix using the N × N transformation matrix (step S1803), and duplicates the vector to generate an N × N transformation coefficient matrix (step S1804). .

  Then, the image processing apparatus performs code conversion (step S1805), calculates an orthogonal transformation coefficient matrix related to the prediction block indicated by the current vector (step S1806), and stores it (step S1807). Next, the image processing apparatus calculates the difference between the transform coefficient related to the target block and the transform coefficient related to the prediction block (step S1808), and proceeds to step S1506.

  Here, in the case where N = 4, M = 1, and the transformation matrix is a Hadamard matrix, it indicates that the amount of calculation is reduced as compared with the conventional case.

First, the amount of calculation is described for the conventional case.
Hadamard transformation: subtraction 16 times. Two-dimensional Hadamard transformation: 32 additions and subtractions for each of T1 and T2. Total 64 times addition / subtraction ・ Absolute value calculation: 16 times ・ Addition: 15 times

On the other hand, the present invention is as follows.
Two-dimensional Hadamard transform for the target block in the original image and at least the block of the search point in the reference image: 64 additions and subtractions as in the prior art Other blocks in the reference image: first pixel value and second block Pixel value subtraction 4 times, Hadamard transform addition / subtraction 8 times, previous vector conversion result addition / subtraction 16 times ・ Difference value calculation process with target block: addition / subtraction 16 times ・ Absolute value calculation process: Absolute value calculation 16 times ・ Total: 15 additions

  Taking a spiral search as an example of the search method, when 20 vectors are evaluated for each macroblock, the number of additions / subtractions is 30400 in the conventional SATD calculation, but 19264 in the present invention. The amount is reduced. The absolute value calculation is 5120 times in the conventional and the present invention.

  As described above, according to the image processing device, the image processing method, and the image processing program, the orthogonal transformation is performed by using the difference value of the non-overlapping regions in the first and second prediction blocks as the orthogonal transformation target matrix, It complements using the conversion factor matrix regarding a 1st prediction block. Thereby, the number of elements of the target matrix and the transformation matrix is reduced, and the amount of calculation can be reduced.

  When M = 1 and the orthogonal transform matrix is a Hadamard transform matrix, N transform coefficient matrices obtained by the first orthogonal transform process are arranged in the search direction during the second orthogonal transform process, and the orthogonal transform coefficients are arranged. The amount of calculation can be reduced by converting the code based on the matrix.

  Further, the transform coefficient matrix obtained by orthogonally transforming the difference value and the transform coefficient matrix related to the first prediction block are added, and the second transform coefficient matrix is multiplied by the replacement matrix related to matrix replacement by the added transform coefficient matrix. A second transform coefficient matrix for the prediction block can be calculated. Thereby, the amount of calculation can be reduced by reducing the number of elements by orthogonal transformation of only the difference value.

  When M = 1 and the orthogonal transform matrix is a Hadamard transform matrix, N transform coefficient matrices obtained by the first orthogonal transform process are arranged in the search direction during the second orthogonal transform process. The amount of calculation can be reduced by adding or subtracting the arranged N × N transform coefficient matrix and the first transform coefficient matrix based on the orthogonal transform coefficient matrix.

  Further, when N is 4 and the orthogonal transformation matrix is a Hadamard transformation matrix, the transformation coefficient matrix after addition with the first transformation coefficient matrix is replaced based on the replacement matrix, and the order of the rows or columns is changed. The second conversion coefficient matrix is calculated by converting the code based on the above. As a result, the second conversion coefficient matrix can be calculated without performing addition / subtraction between the conversion coefficient matrix after the addition and the replacement matrix, thereby reducing the amount of calculation.

  Also, the difference between the third transform coefficient matrix related to the target block in the original image and the second transform coefficient matrix related to the second prediction block is calculated, and the sum is calculated after the absolute value calculation, thereby calculating SATD. To do. Thereby, the calculation amount regarding SATD calculation can be reduced.

Further, when N is 4 and the orthogonal transformation matrix is a Hadamard transformation matrix, the transformation coefficient matrix after addition with the first transformation coefficient matrix is replaced based on the replacement matrix, and the order of the rows or columns is changed. A difference value between the transform coefficient matrix of the second transform coefficient matrix related to the second prediction block and
The amount of calculation can be reduced by adding or subtracting based on the replacement matrix.

  The image processing method described in the present embodiment can be realized by executing a program prepared in advance on a computer such as a personal computer or a workstation. The image processing program is recorded on a computer-readable recording medium such as a hard disk, a flexible disk, a CD-ROM, an MO, and a DVD, and is executed by being read from the recording medium by the computer. The image processing program may be distributed via a network such as the Internet.

  In addition, the image processing apparatus 1300 described in the fourth embodiment includes an application-specific IC (hereinafter simply referred to as “ASIC”) such as a standard cell or a structured ASIC (Application Specific Integrated Circuit), or a PLD (Programmable) such as an FPGA. It can also be realized by Logic Device). Specifically, for example, the functions of the image processing apparatus 1300 described above (difference value calculation unit 1301 to two-dimensional direct conversion unit 1308) are defined by HDL description, and the HDL description defined by the function is logically synthesized to perform ASIC or By giving to the PLD, the image processing apparatus 1300 can be manufactured.

  In addition, only a part of the functions (difference value calculation unit 1301 to two-dimensional orthogonal conversion unit 1308) of the image processing apparatus 1300 described in the fourth embodiment may be realized by a PLD such as an ASIC or FPGA.

  The following additional notes are disclosed with respect to the embodiment described above.

(Supplementary Note 1) Two-dimensional orthogonal transformation of a second block of N × N elements obtained by shifting the first block of N × N elements in the image by M (1 ≦ M <N / 2) elements in an arbitrary search direction. An image processing apparatus to execute,
A first feature matrix that does not overlap the first block in the second block, and a second feature matrix that does not overlap the second block in the first block. A difference value calculating means for calculating a difference value matrix as a difference;
First, the N × M element difference value matrix calculated by the difference value calculating means is subjected to one-dimensional orthogonal transformation using an N × N element orthogonal transformation matrix to calculate an N × M element transformation coefficient matrix. Orthogonal transformation means,
The N × M element transform coefficient matrix calculated by the first orthogonal transform unit is subjected to one-dimensional orthogonal transform using the M × N element orthogonal transform matrix to calculate an N × N element transform coefficient matrix. A second orthogonal transform means;
Based on the first transform coefficient matrix of N × N elements related to the first block and the transform coefficient matrix of N × N elements transformed by the second orthogonal transform unit, N related to the second block A coefficient value calculating means for calculating a second transform coefficient matrix of × N elements;
An image processing apparatus comprising:

(Supplementary Note 2) M = 1, the N × N element orthogonal transformation matrix is a Hadamard transformation matrix, and the previous M × N element transformation matrix is a matrix obtained by extracting a part of the Hadamard transformation matrix,
The second orthogonal transform means includes:
Generating means for generating an N × N element transform coefficient matrix by arranging N N × M element transform coefficient matrices calculated by the first orthogonal transform means in the search direction;
Code conversion means for converting the code of each element in the N × N element conversion coefficient matrix generated by the generation means based on the M × N element orthogonal conversion matrix;
The coefficient value calculating means includes
Based on the N × N element first transform coefficient matrix related to the first block and the N × N element transform coefficient matrix whose code has been converted by the code conversion means, N × N elements related to the second block. The image processing apparatus according to appendix 1, wherein a second transformation coefficient matrix of N elements is calculated.

(Supplementary Note 3) The coefficient value calculating means includes:
Adding means for adding the N × N element first transform coefficient matrix and the N × N element transform coefficient matrix transformed by the second orthogonal transform means;
Replacement means for calculating a second transformation coefficient matrix of N × N elements related to the second block by multiplying the transformation coefficient matrix of N × N elements added by the addition means and a replacement matrix of N × N elements. When,
The image processing apparatus according to appendix 1 or 2, further comprising:

(Supplementary note 4) M = 1, and the orthogonal transformation matrix of N × N elements and the orthogonal transformation matrix of M × N elements are Hadamard transformation matrices,
The second orthogonal transform means includes:
An N × N element transform coefficient matrix is generated by arranging N transform coefficient matrices calculated by the first orthogonal transform means in the search direction,
The coefficient value calculating means includes
The N × N element first transform coefficient matrix related to the first block and the N × N element transform coefficient matrix generated by the second orthogonal transform means are converted into the N × N element transform coefficient matrix. Addition / subtraction means for adding when the corresponding element in the orthogonal transformation matrix of the M × N element is 1, and for subtracting when the element is −1,
An N × N element second transformation coefficient matrix related to the second block is calculated by multiplying the N × N element transformation coefficient matrix added / subtracted by the addition / subtraction means with an N × N element replacement matrix. The image processing apparatus according to appendix 1, further comprising: replacement means.

(Supplementary Note 5) N = 4, and the N × N element orthogonal transform matrix and the M × N element orthogonal transform matrix are Hadamard transform matrices,
The replacement means includes
The order of the rows or columns of the N × N element conversion coefficient matrix added by the adding means is replaced based on the N × N element replacement matrix, and the row or column in the N × N element conversion coefficient matrix after the replacement. 5. The image processing apparatus according to appendix 3 or 4, wherein a second transform coefficient matrix of N × N elements related to the second block is calculated by transforming the element code for each element based on the replacement matrix.

(Supplementary Note 6) Coefficient difference that is the difference between the third transform coefficient matrix relating to the third block of N × N elements in the image different from the image and the second transform coefficient matrix calculated by the coefficient value calculating means Coefficient difference value calculating means for calculating a value matrix;
Absolute value calculating means for calculating an absolute value of the coefficient difference value matrix calculated by the coefficient difference value calculating means;
A sum calculating means for calculating a sum of elements in the absolute value matrix calculated by the absolute value calculating means;
The image processing apparatus according to any one of appendices 1 to 5, further comprising:

(Supplementary Note 7) N = 4, and the orthogonal transformation matrix of N × N elements and the orthogonal transformation matrix of M × N elements are Hadamard transformation matrices, and
The 4 × 4 element conversion coefficient matrix replaced by the replacement unit with respect to the third conversion coefficient matrix relating to the third block of N × N elements in an image different from the image corresponds to the conversion coefficient matrix. Coefficient difference value calculating means for calculating a coefficient difference value matrix by subtraction when the element of the 4 × 4 element replacement matrix is 1, and addition when the element is −1;
Absolute value calculating means for calculating an absolute value of the coefficient difference value matrix calculated by the coefficient difference value calculating means;
A sum calculating means for calculating a sum of elements in the absolute value matrix calculated by the absolute value calculating means,
The replacement means includes
The image processing apparatus according to appendix 3 or 4, wherein the N × N element transform coefficient matrix added by the adding means is switched in order of rows or columns based on the N × N element replacement matrix. .

(Supplementary Note 8) Two-dimensional orthogonal transformation of a second block of N × N elements obtained by shifting the first block of N × N elements in the image by M (1 ≦ M <N / 2) elements in an arbitrary search direction. The computer that runs is
A first feature matrix that does not overlap the first block in the second block, and a second feature matrix that does not overlap the second block in the first block. A difference value calculating step for calculating a difference value matrix as a difference;
First, the N × M element difference value matrix calculated in the difference value calculating step is subjected to one-dimensional orthogonal transformation using an N × N element orthogonal transformation matrix to calculate an N × M element transformation coefficient matrix. Orthogonal transformation process of
The N × M element transform coefficient matrix calculated in the first orthogonal transform step is subjected to one-dimensional orthogonal transform using an M × N element orthogonal transform matrix to calculate an N × N element transform coefficient matrix. A second orthogonal transformation step;
Based on the first transform coefficient matrix of N × N elements related to the first block and the transform coefficient matrix of N × N elements transformed by the second orthogonal transform step, N related to the second block A coefficient value calculating step of calculating a second transformation coefficient matrix of × N elements;
The image processing method characterized by performing.

(Supplementary Note 9) Two-dimensional orthogonal transformation of a second block of N × N elements obtained by shifting the first block of N × N elements in the image by M (1 ≦ M <N / 2) elements in an arbitrary search direction. The computer to run,
A first feature matrix that does not overlap the first block in the second block, and a second feature matrix that does not overlap the second block in the first block. A difference value calculating means for calculating a difference value matrix as a difference;
First, the N × M element difference value matrix calculated by the difference value calculating means is subjected to one-dimensional orthogonal transformation using an N × N element orthogonal transformation matrix to calculate an N × M element transformation coefficient matrix. Orthogonal transformation means,
The N × M element transform coefficient matrix calculated by the first orthogonal transform unit is subjected to one-dimensional orthogonal transform using the M × N element orthogonal transform matrix to calculate an N × N element transform coefficient matrix. Second orthogonal transform means;
Based on the first transform coefficient matrix of N × N elements related to the first block and the transform coefficient matrix of N × N elements transformed by the second orthogonal transform unit, N related to the second block X coefficient value calculating means for calculating a second transform coefficient matrix of N elements;
An image processing program that functions as an image processing program.

300 Image processing device 301 Difference value calculation circuit 302 First orthogonal transformation circuit 303 Second orthogonal transformation circuit 304 Coefficient value calculation circuit 800 Image processing device 1300 Image processing device 1301 Difference value calculation unit 1302 First orthogonal transformation unit 1303 First Two orthogonal transform units 1304 Coefficient value calculation unit 1305 Coefficient difference value calculation unit 1306 Absolute value calculation unit 1307 Summation calculation unit 1308 Two-dimensional orthogonal transform unit

Claims (5)

Image processing for performing two-dimensional orthogonal transformation of a second block of N × N elements obtained by shifting the first block of N × N elements in the image by M (1 ≦ M <N / 2) elements in an arbitrary search direction A device,
A first feature matrix that does not overlap the first block in the second block, and a second feature matrix that does not overlap the second block in the first block. A difference value calculating means for calculating a difference value matrix as a difference;
First, the N × M element difference value matrix calculated by the difference value calculating means is subjected to one-dimensional orthogonal transformation using an N × N element orthogonal transformation matrix to calculate an N × M element transformation coefficient matrix. Orthogonal transformation means,
The N × M element transform coefficient matrix calculated by the first orthogonal transform unit is subjected to one-dimensional orthogonal transform using the M × N element orthogonal transform matrix to calculate an N × N element transform coefficient matrix. A second orthogonal transform means;
Based on the first transform coefficient matrix of N × N elements related to the first block and the transform coefficient matrix of N × N elements transformed by the second orthogonal transform unit, N related to the second block A coefficient value calculating means for calculating a second transform coefficient matrix of × N elements;
An image processing apparatus comprising: M = 1, and is said orthogonal transform matrix is a Hadamard transform matrix of N × N elements, and orthogonal transform matrix of the M × N element was removed portion of the Hadamard transform matrix matrix,
The second orthogonal transform means includes:
Generating means for generating an N × N element transform coefficient matrix by arranging N N × M element transform coefficient matrices calculated by the first orthogonal transform means in the search direction;
Code conversion means for converting the code of each element in the N × N element conversion coefficient matrix generated by the generation means based on the M × N element orthogonal conversion matrix;
The coefficient value calculating means includes
Based on the N × N element first transform coefficient matrix related to the first block and the N × N element transform coefficient matrix whose code has been converted by the code conversion means, N × N elements related to the second block. The image processing apparatus according to claim 1, wherein a second transform coefficient matrix of N elements is calculated. Calculate a coefficient difference value matrix that is a difference between a third transform coefficient matrix relating to a third block of N × N elements in an image different from the image and the second transform coefficient matrix calculated by the coefficient value calculation means. Coefficient difference value calculating means to perform,
Absolute value calculating means for calculating an absolute value of the coefficient difference value matrix calculated by the coefficient difference value calculating means;
A sum calculating means for calculating a sum of elements in the absolute value matrix calculated by the absolute value calculating means;
The image processing apparatus according to claim 1, further comprising: A computer that performs two-dimensional orthogonal transformation of a second block of N × N elements obtained by shifting a first block of N × N elements in an image by M (1 ≦ M <N / 2) elements in an arbitrary search direction. ,
A first feature matrix that does not overlap the first block in the second block, and a second feature matrix that does not overlap the second block in the first block. A difference value calculating step for calculating a difference value matrix as a difference;
First, the N × M element difference value matrix calculated in the difference value calculating step is subjected to one-dimensional orthogonal transformation using an N × N element orthogonal transformation matrix to calculate an N × M element transformation coefficient matrix. Orthogonal transformation process of
The N × M element transform coefficient matrix calculated in the first orthogonal transform step is subjected to one-dimensional orthogonal transform using an M × N element orthogonal transform matrix to calculate an N × N element transform coefficient matrix. A second orthogonal transformation step;
Based on the first transform coefficient matrix of N × N elements related to the first block and the transform coefficient matrix of N × N elements transformed by the second orthogonal transform step, N related to the second block A coefficient value calculating step of calculating a second transformation coefficient matrix of × N elements;
The image processing method characterized by performing. A computer that performs a two-dimensional orthogonal transformation of a second block of N × N elements obtained by shifting the first block of N × N elements in the image by M (1 ≦ M <N / 2) elements in an arbitrary search direction. ,
A first feature matrix that does not overlap the first block in the second block, and a second feature matrix that does not overlap the second block in the first block. A difference value calculating means for calculating a difference value matrix as a difference;
First, the N × M element difference value matrix calculated by the difference value calculating means is subjected to one-dimensional orthogonal transformation using an N × N element orthogonal transformation matrix to calculate an N × M element transformation coefficient matrix. Orthogonal transformation means,
The N × M element transform coefficient matrix calculated by the first orthogonal transform unit is subjected to one-dimensional orthogonal transform using the M × N element orthogonal transform matrix to calculate an N × N element transform coefficient matrix. Second orthogonal transform means;
Based on the first transform coefficient matrix of N × N elements related to the first block and the transform coefficient matrix of N × N elements transformed by the second orthogonal transform unit, N related to the second block X coefficient value calculating means for calculating a second transform coefficient matrix of N elements;
An image processing program that functions as an image processing program.

Images