JP3175914B2 - Image encoding method and image encoding device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method for detecting a motion vector for efficiently detecting a motion in a moving image signal.

[0002]

2. Description of the Related Art In highly efficient coding of moving images, motion compensation prediction is often used to remove inter-frame correlation. Motion compensation prediction utilizes similarity between the current frame and another frame that is temporally close.
In order to perform efficient motion compensation prediction, the direction and magnitude (motion vector) of motion indicating where a certain region of the encoding target frame exists in another frame that is temporally close to the frame must be accurately determined. It is important to ask.

In selecting a motion vector, the following method is usually used. The image data of the motion vector detection target small block is set to an N × N size, and B (i,
j). With respect to B (i, j), a small block B ′ (i + j) in which B (i, j) is spatially and horizontally shifted by (u, v) on the reference image in the frame memory.
An evaluation value D (u, v) representing the degree of similarity to (u, j + v) is calculated. As the evaluation value, a sum of absolute differences and a sum of squares of differences are often used. (U, v) is changed in a predetermined search area, and (u, v) that gives the minimum value of D (u, v) is set as the final motion vector (U, V), and the motion vector (U , V) in the reference image are set as reference small blocks.

FIG. 3 is a block diagram showing a configuration of an apparatus according to a conventional typical motion vector detecting method. In this conventional example, the reference image is an image one frame before in time, and an absolute difference value is used as an evaluation function.

In the figure, reference numeral 101 denotes an input terminal.
Is divided into small blocks 104, which are units of motion search. On the other hand, the input image 102 is also input to the frame memory 105, and the frame memory 105 outputs the reference image 106 one frame before. Reference numeral 107 denotes a candidate vector setting unit that has a function of generating candidate vectors in a predetermined search range and transmitting the generated candidate vectors to the shift unit 108 and the vector memory 112. The shift unit 108 converts the reference image 106 stored in the frame memory 105 into the candidate vector setting unit 10
According to the candidate vector set in step 7, the image is spatially shifted by that amount. In the subtracter 109, the difference between the shifted small block and the small block to be encoded 104 is calculated, and the calculation result is input to the absolute value sum calculation unit 110. The absolute value sum calculator 110 calculates the sum of the absolute values of the differences in the small block, sends the calculated absolute value sum to the minimum value memory 111, and stores the minimum value stored in the minimum value memory 111 at that time. Is compared with. If the newly sent absolute value sum is smaller than the currently stored minimum value, the minimum value is updated to the newly sent absolute value sum and an update instruction is issued to the vector memory 112. An order is sent. Upon receiving the update instruction command, the vector memory 112 replaces the stored vector with the currently input candidate vector. When the generation of all the vectors in the search range determined by the candidate vector setting unit 107 ends, the vector memory 11
2 are output to the output terminal 114 as the motion vector 113 of the small block to be encoded, and the minimum plant memory 111 and the vector memory 112
Is reset.

According to the conventional method, as an evaluation value of the similarity between a small block to be detected with a motion vector and a small block indicated by a candidate vector on a reference image, a difference absolute value using all pixels in the small block is used. Sums and sums of squares are used. That is, when the size of the small block is N × N, the evaluation value D (u, v) in the case of the sum of absolute differences is

[0007]

(Equation 1) Becomes The above equation calculates the sum of N ² pieces, which leads to an increase in the amount of calculation.

In order to solve this, it is often the case that the sum is calculated by thinning out the pixels at predetermined predetermined intervals without calculating the sum of all the pixels in the above equation. According to such a method, the amount of calculation required for processing can be reduced, but an incorrect motion vector may be obtained as compared with the case where the evaluation value is calculated using all pixels. The important thing is how to perform the calculation by thinning out the pixels.Conventionally, in the past, this thinning pattern is generally set in advance in such a manner that the thinning pattern is evenly thinned in the horizontal and vertical directions according to the thinning rate. Met. FIG. 4 shows a conventional example of the thinning method to be set.

[0009]

In the above-described conventional motion vector detection of a difference in image coding, in order to reduce the amount of calculation, pixels are thinned out uniformly in the horizontal and vertical directions according to the thinning rate. Was. However, some thinning methods are optimal depending on the local properties (movement and resolution) of the image. In the conventional method, the thinning method is determined in advance without using such information at all. However, there is a problem that the probability of selecting a motion vector that has been increased increases.

SUMMARY OF THE INVENTION The present invention has been made in view of the above-mentioned problems of the prior art, and achieves high reproduction with a small amount of computation by performing optimal thinning in accordance with local characteristics of an image. It is an object of the present invention to realize an image encoding method and an image encoding device capable of obtaining image quality.

[0011]

According to an image encoding method of the present invention, image data of a current frame is divided into small blocks to be subjected to motion detection with respect to a digital image signal such as a television signal, and the divided motion detection is performed. The similarity between the target small block and a plurality of candidate blocks in the past or future reference frame stored in the frame memory is calculated based on the absolute difference value, and the evaluation value is used to determine the most similar candidate block. In an image coding method in which motion compensation prediction is performed using a vector as a motion vector and coding is performed, when the similarity between a motion detection processing target small block and a certain candidate block is evaluated, the motion detection target processing target block and the candidate block are in the same frame. Selection of pixels in small block used for vector evaluation according to the direction of motion vector obtained in small block near Law and switches to.

In this case, when the motion vector obtained in a nearby small block in the same frame as the motion detection processing target block is in the horizontal direction, the vector in the small block used for vector evaluation is obtained by thinning out in the vertical direction. When the motion vector obtained in the nearby small block in the same frame as the motion detection processing target block is in the vertical direction, the vector evaluation is performed by decimating in the horizontal direction as pixels in the small block used for vector evaluation. If no directionality is found in the motion vector obtained in a nearby small block in the same frame as the motion detection processing target block, the pixels in the small block used for vector evaluation are equally used in the horizontal and vertical directions. Alternatively, the vector evaluation may be performed by using the data thinned out.

Further, the direction of the motion vector (U, V) may be determined according to the value obtained by | U | / | V |.

Further, | U | / | V | is obtained for the direction of the motion vector (U, V). If the obtained value is larger than 5, the motion vector is set to the horizontal direction, and the obtained value is 1
If it is smaller than / 5, the motion vector may be set to the vertical method.

An image encoding apparatus according to the present invention comprises: an input terminal to which an image is input; a small block division unit for dividing the input image input to the input terminal into small blocks, which are units of motion search; A frame memory that inputs an image and outputs a reference image one frame before the image divided by the small block division unit, and a candidate vector setting unit that generates and outputs a candidate vector within a predetermined search range. A shift unit that spatially shifts the reference image by the frame memory by the amount of the candidate vector set by the candidate vector setting unit; and a reference image shifted by the shift unit and the small block division unit. A subtracter for calculating a difference between the divided input image and an output of the subtractor, wherein a pixel thinning pattern is set in accordance with a direction of a motion vector of a neighboring small block to form an image. A pixel thinning unit that performs thinning and outputs a signal after pixel thinning; an absolute value sum calculating unit that receives the output of the pixel thinning unit and calculates the sum of absolute differences in a small block; The sum of absolute values obtained by the calculation unit is inputted and stored, and each time a new sum of absolute values is inputted, the sum is compared with the currently stored absolute value sum. If the value is smaller than the value, the newly transmitted absolute planting is updated and stored, and the minimum value memory for outputting an update instruction command indicating the update is output from the candidate vector setting unit. When a candidate vector is input and stored, and when an update instruction command by the minimum value memory is received, the stored content is replaced with the candidate vector input at that time.
A vector memory that outputs a vector stored at the time when the generation of all the vectors in a predetermined search range is completed as a motion vector of the encoding target small block, and a motion vector of the encoding target small block output by the vector memory. A direction detecting unit that inputs the direction, obtains the direction, and outputs the direction of the motion vector of the neighboring small block for performing pixel thinning of the subsequent block to the pixel thinning unit.

In this case, the direction detection unit obtains | U | / | V | for the direction of the motion vector (U, V), and if the obtained value is larger than 5, sets the motion vector to the horizontal direction. When the calculated value is smaller than 1/5, the motion vector may be set to the vertical method.

When the direction of the motion vector of the neighboring small block is the horizontal direction, the pixel thinning section performs vector evaluation by vertically decimating the pixels in the small block used for the vector evaluation. When the motion vector obtained in the nearby small block in the same frame is in the vertical direction, the vector evaluation is performed by thinning out the pixels in the horizontal direction as pixels in the small block used for vector evaluation, and the same as the motion detection processing target block. If no particular directionality is found in the motion vector obtained in a nearby small block in the frame, vector evaluation is performed using pixels in the small block used in the vector evaluation, which are evenly thinned out in the horizontal and vertical directions. It may be that.

[Effect] The present invention evaluates the similarity between a small block to be subjected to motion detection processing and a candidate block.
The method is characterized by switching the method of selecting a pixel in a small block used for vector evaluation by referring to the direction of a motion vector obtained in a nearby small block in the same frame as the motion detection processing target block.

When the similarity is obtained based on the absolute value of the difference, the inter-pixel correlation of the motion compensation difference signal is examined. If the motion is close to the horizontal direction, the correlation is high in the vertical direction and the motion is close to the vertical direction. In this case, it is known that the correlation is high in the horizontal direction.

In the present invention, since the thinning method is adaptively set according to the direction of motion, it is possible to operate so that the optimum thinning method is always used depending on the local nature of the image.

[0021]

Next, an embodiment of the present invention will be described with reference to the drawings.

FIG. 1 is a block diagram showing the configuration of one embodiment of the present invention. In the figure, reference numeral 301 denotes an input terminal.
At 3, it is divided into small blocks 304, which are units of motion search. On the other hand, the input image 302
5, the frame memory 305 outputs a reference image 306 one frame before. Reference numeral 307 denotes a candidate vector setting unit that has a function of generating candidate vectors within a predetermined search range and transmitting the generated candidate vectors to the shift unit 308 and the vector memory 313. The shift unit 308 includes a frame memory 305
Is spatially shifted by the candidate vector set by the candidate vector setting unit 307 by that amount. In the subtracter 309, the difference between the shifted small block and the small block to be encoded 304 is calculated, and the calculation result is sent to the pixel thinning unit 310. In the pixel thinning unit 310, a pixel thinning pattern is set according to the direction 318 of the motion vector of the neighboring small block that has already been obtained, pixel thinning is performed, and a signal after pixel thinning is input to the absolute value sum calculating unit 311. You. The absolute value sum calculator 311 calculates the sum of the absolute values of the differences in the small block, sends the calculated sum of absolute values to the minimum value memory 312, and stores the minimum value stored in the minimum value memory 312 at that time. Is compared with. When the newly transmitted absolute planting is smaller than the currently stored minimum value, the minimum value is updated and an update instruction command is sent to the vector memory 313. Upon receiving the update instruction command, the vector memory 313 replaces the stored vector with the candidate vector input at that time. When the generation of all the vectors in the search range determined by the candidate vector setting unit 307 is completed, the candidate vector setting unit 307 outputs a signal indicating that to the minimum value memory 111 and the vector memory 112, and the vector memory The vector stored in 313 is output to the output terminal 315 as the motion vector 314 of the small block to be encoded, and the minimum value memory 31
2. The vector memory 313 is reset. Further, the obtained motion vector is sent to the direction detection unit 317 via the delay circuit 316, and the direction detection unit 317 thins out the pixels of the subsequent block. The direction 318 of the detected vector used as the information for
Output to 0. Here, the delay circuit 316 is provided to match the timing of the direction 318 of the vector output from the direction detection unit 317 to the pixel thinning unit 310.

The motion vector is used for motion compensation prediction by a known encoding unit (not shown) provided at a stage subsequent to the illustrated circuit.

Next, the operation of the main part of the embodiment constructed as described above will be described in detail. In this embodiment, the sum of absolute differences is used as an evaluation function.

The motion compensation difference signal E (i, j) -B '(i
+ U, j + v), the motion (u, j, v)
It is known that when v) is close to the horizontal direction, the correlation is high in the vertical direction, and when the motion (u, v) is close to the vertical direction, the correlation is high in the horizontal direction. The case where the sum of absolute values calculated by thinning out pixels in the direction of high correlation is used as the evaluation value, and the case where the sum of absolute values calculated by thinning out pixels in the direction of low correlation is used as the evaluation value, Compared to the case where the sum of absolute values is calculated without thinning out at all and used as the evaluation value, the former has less influence of being erroneous. Therefore, the motion direction of the small block to be subjected to the motion detection processing is predicted to be almost the same direction as the already detected motion vector spatially nearby, and if it is horizontal, it is thinned out in the vertical direction as a vector evaluation pixel. In the case of the vertical direction, the evaluation is performed by thinning out the vector evaluation pixels in the horizontal direction.

FIG. 2 is a diagram for explaining an evaluation method performed in this embodiment, and shows a case where the thinning rate is set to be 1/4.

The evaluation is specifically performed by the following method. First, the motion directions of the motion vectors (U, V) which are spatially close and have been obtained are classified into three types by the following procedure. That is, when | U | / | V |> 5, the motion vector detection processing target small block moves in the horizontal direction, when | U | / | V | <1/5, moves in the vertical direction. In this case, it is determined that the movement is other than that. Based on the motion direction obtained in this procedure, the thinning method is D ₁ for horizontal motion, D _{2 for} vertical motion, and D _{3 for} other motions. D ₁ ,
D ₂ and D ₃ are determined as follows.

[0028]

(Equation 2) In the example shown in FIG. 2, among the pixels used for calculating the sum of absolute values defined in D ₁ , D ₂ , and D ₃ , in the case of D ₁ , all the sums are calculated in the horizontal direction, but the sum is calculated in the vertical direction. The sum is taken every four pixels in the direction. In the case of D ₂ is vertically reversed to calculate the sum of all, taking the sum every four pixels in the horizontal direction. Further, in the case of D ₃ means calculating a sum in the horizontal direction, both vertically uniform intervals.

By performing the above calculation, it is possible to set an appropriate thinning method for each small block of the image. Therefore, while reducing the total calculation amount to 1/4, The probability that a vector is required can be reduced.

In the embodiment described above, the motion detection target small block divided by the small block dividing unit 303 is compared with the previous (past) reference frame stored in the frame memory 305. Although it has been described that the similarity with a plurality of candidate blocks is obtained, the reference frame one frame before is set as the divided motion detection target small block, and the frame memory is the current, that is, the future from the viewpoint of the motion detection target small block. May be output. What is important in the present invention is to use frames before and after detecting a motion vector, and the positions before and after the frame are not particularly limited.

In detecting motion, | U | / | V
The case where |> 5 and the case where | U | / | V | <1/5 are used, but these values are set based on experience and are not particularly limited.

[0032]

Since the present invention is configured as described above, it has the following effects.

A pixel thinning method for evaluating a motion vector is adaptively set according to the direction of motion, and an optimum thinning method is always used depending on the characteristics of each region of an image. As a result, it is possible to detect a correct motion vector even if the amount of calculation required for the motion detection process is reduced, and it is possible to provide high reproduction image quality with a small amount of calculation when applied to high-efficiency encoding of an image or the like. Becomes

[Brief description of the drawings]

FIG. 1 is a block diagram showing the configuration of an embodiment of the present invention.

FIG. 2 is a diagram for explaining a pixel thinning method in the embodiment shown in FIG.

FIG. 3 is a block diagram showing a configuration of a conventional example.

FIG. 4 is a diagram showing how to thin out pixels used for calculating an evaluation value in a conventional example.

[Explanation of symbols]

 301 Input terminal 302 Input image 303 Small block division unit 304 Motion vector detection target small block 305 Frame memory 306 One frame previous image 307 Candidate vector setting unit 308 Shift unit 309 Subtractor 310 Decimation setting unit 311 Absolute value sum calculation unit 312 Minimum Value memory 313 Vector memory 314 Obtained motion vector 315 Output terminal 316 Delay circuit 317 Direction detector 318 Motion vector direction

Claims (7)

(57) [Claims] 1. A digital image signal such as a television signal is divided into image data of a current frame as a small block for motion detection, and the divided small motion detection target block is stored in a frame memory. The degree of similarity to a plurality of candidate blocks in a past or future reference frame is calculated based on the absolute value of the difference, and the vector to the most similar candidate block is subjected to motion compensation prediction based on the evaluation value as a motion vector, and coding is performed. When evaluating the similarity between a small block to be subjected to motion detection processing and a certain candidate block, a motion vector obtained by a nearby small block in the same frame as the block to be subjected to motion detection processing is evaluated. Image coding characterized by switching a method of selecting a pixel in a small block used for vector evaluation according to a direction Law. 2. The image encoding method according to claim 1, wherein when a motion vector obtained in a nearby small block in the same frame as the motion detection processing target block is in the horizontal direction, a small block used in vector evaluation is used. When the motion vector obtained in a nearby small block in the same frame as the motion detection target block is in the vertical direction, a vector evaluation is performed by vertically decimating the pixel as a pixel, and the pixel in the small block used for the vector evaluation is used. The vector evaluation is performed by using the thinned-out blocks in the horizontal direction, and the small blocks used for the vector evaluation when the direction of the motion vector obtained in the nearby small block in the same frame as the motion detection processing target block is not particularly recognized. The vector evaluation is based on pixels that are evenly thinned in the horizontal and vertical directions as inner pixels. Picture coding method and performing. 3. The image encoding method according to claim 2, wherein the direction of the motion vector (U, V) is | U | / | V |.
Determining the direction of a motion vector according to the value obtained from the image encoding method. 4. The image encoding method according to claim 2, wherein the direction of the motion vector (U, V) is | U | / | V |.
An image coding method characterized in that when the obtained value is larger than 5, the motion vector is set in the horizontal direction, and when the obtained value is smaller than 1/5, the motion vector is set in the vertical direction. . 5. An input terminal to which an image is input, a small block dividing unit that divides the input image input to the input terminal into small blocks, which are units of motion search, A frame memory for outputting a reference image one frame before the image divided by the block division unit, a candidate vector setting unit for generating and outputting a candidate vector within a predetermined search range, and a reference image by the frame memory And a shift unit for spatially shifting the candidate image set by the candidate vector setting unit, and a reference image shifted by the shift unit and an input image divided by the small block division unit. A subtracter for calculating a difference; and setting the pixel subtraction pattern according to the direction of the motion vector of the neighboring small block to the output of the subtracter to perform pixel thinning. A pixel decimating unit that outputs a signal after the subtraction, an absolute value sum calculating unit that receives the output of the pixel decimating unit and calculates the sum of absolute differences in the small block, and is calculated by the absolute value sum calculating unit. Input and store the sum of absolute values, and compare it with the sum of absolute values currently stored each time a new sum of absolute values is input. If the value is also smaller, the newly sent absolute planting is updated and stored, and a minimum value memory for outputting an update instruction command indicating that, and a candidate vector output by the candidate vector setting unit are input. When an update instruction command by the minimum value memory is received, the stored content is replaced with the candidate vector input at that time, and the generation of all the vectors in the predetermined search range is performed by the candidate vector setting unit. finished A vector memory that outputs a vector stored at a point as a motion vector of a small block to be coded, and a motion vector of the small block to be coded that is output from the vector memory, obtains the direction, and performs pixel thinning of a subsequent block. And a direction detecting unit that outputs the direction of the motion vector of the neighboring small block to the pixel thinning unit for performing the following. 6. The image encoding apparatus according to claim 5, wherein the direction detection unit obtains | U | / | V | for the direction of the motion vector (U, V), and the obtained value is larger than 5. , Wherein the motion vector is set in the horizontal direction, and when the obtained value is smaller than 1/5, the motion vector is set in the vertical direction. 7. The image coding apparatus according to claim 5, wherein the pixel thinning section includes a vertical direction as a pixel in a small block used for vector evaluation when a direction of a motion vector of a neighboring small block is a horizontal direction. When the motion vector obtained in a nearby small block in the same frame as the motion detection processing target block is in the vertical direction, the vector is thinned in the horizontal direction as a pixel in the small block used for vector evaluation. If the direction of the motion vector obtained in the neighboring small block in the same frame as the motion detection processing target block does not have any particular direction, the pixel in the small block used for vector evaluation is horizontally The feature is that vector evaluation is performed by using a uniform thinning in the vertical and vertical directions. That the image coding apparatus.