JPH10341440A - Moving image encoding method and device therefor - Google Patents

Abstract

PROBLEM TO BE SOLVED: To reduce the computing amount of the motion vector detection of inter-frame encoding and the number of times of reference. SOLUTION: The position of a prediction block is calculated from the motion vector to an adjacent frame of an encoding object block (step 101). One or more reference blocks inside an adjacent reference frame for constituting the prediction block are extracted (step 102). Whether or not the motion vector of the respective adjacent reference blocks indicates an actual motion is evaluated and the motion vector of a low evaluation value is excluded (step 103). The weighting coefficient of the motion vector of the respective adjacent reference blocks is calculated (step 104). The motion vector of the encoding object block is calculated from the motion vector to the adjacent frame of the encoding object block, the motion vector of the respective adjacent reference blocks and the weighting coefficient (step 105).

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention divides an input frame into blocks, detects a motion vector for a past or future adjacent frame for each block, and performs motion compensation inter-frame prediction using the detected motion vector. The present invention relates to a moving image encoding method to be performed.

[0002]

2. Description of the Related Art In general, in a moving picture coding method, motion compensated inter-frame prediction is used to remove temporal redundancy. The motion compensated inter-frame prediction is a method of calculating a prediction error of an encoding target block by using an image signal obtained by performing motion compensation on a reference frame using a motion vector as a prediction signal. When motion compensation inter-frame prediction is used, it is necessary to search for a motion vector.

Normally, the search range for a motion vector is expanded in proportion to the distance between the current frame and the reference frame. The search range of the motion vector when the interframe distance is 1 is ± S × ± S. When the distance between frames is d, ± d
Search the range of S × ± d · S. Therefore, the search range at the inter-frame distance d is d × d times the motion vector search in the adjacent frame with the inter-frame distance of 1. Search range of motion vector at interframe distance 1 is ± S × ± S
Assuming that the number of calculations at U is U, the number of calculations at the interframe distance d is d ² · U.

Since the search for a motion vector involves an enormous number of calculations, a method for reducing the number of calculations is required.
Generally, a telescopic search is used as a method of reducing the number of operations. The telescopic search is a technique of detecting a motion vector by sequentially performing a motion search on a frame existing between a reference frame and an encoding target frame. The amount of computation at the distance d between frames is d
-U is obtained, and the number of operations can be reduced.

FIG. 9 shows an example of a telescopic search.
Now, frame n + 3 is the frame to be encoded, frame n
Is used as a reference frame to perform motion vector detection. First, a motion vector v1 to a frame n + 2 is detected within the search range ± S × ± S. Subsequently, the motion vector v2 from the frame n + 3 to the frame n + 1 in the search range ± S × ± S around the position moved by the motion vector v1.
+ V1 is detected. Similarly, a search is performed around the position moved by the motion vector v2 + v1, and a motion vector v3 + v2 + v1 from the frame n + 3 to the frame n is detected. This motion vector is set as a motion vector v0 of the current block. The number of operations in this example is 3U.

As described above, the number of calculations can be reduced by using the telescopic search.

[0007]

In the conventional method, a motion search is sequentially performed on a frame between an encoding target frame and a reference frame.

Now, consider a case where encoding processing is performed with reference relationships as shown in FIG. The search range of the motion vector at the inter-frame distance 1 is ± S × ± S, and the number of calculations is U. In the telescopic search, the number of operations of the reference relation P1 is U, the number of operations of the reference relation P2 is 2U,
There is a problem that the amount of calculation of the encoding target frame increases in proportion to the inter-frame distance between the encoding target frame and the reference frame, such as the number of operations of 3 is 3U.

In the reference relations P2 and P3, frames which are not related to the reference relation must be referred to.
The number of references is proportional to the distance between frames.

Thus, in the telescopic search,
If the reference relation of each frame is not uniform, there is a problem that the amount of calculation and the number of references increase in proportion to the distance between frames of the reference relation, and the amount of calculation and the number of references of each frame are not uniform. For this reason, for example, when the encoding process is performed in real time, it is necessary to design a moving image encoding processing unit based on the maximum calculation amount and the maximum number of times of reference.

An object of the present invention is to provide a moving picture coding method and apparatus which reduce the amount of calculation and the number of references for detecting a motion vector in inter-frame coding.

Another object of the present invention is to provide a moving picture coding method and apparatus for equalizing the calculation amount and the number of times of reference of the motion vector detection of the inter-frame coding irrespective of the distance between the frame to be coded and the reference frame. Is to provide.

[0013]

According to the moving picture coding method of the present invention, when a frame other than the frame adjacent to the frame to be coded is used as a reference frame, the frame exists between the reference frame and the frame to be coded. The motion vector between the reference frame and the encoding target frame is estimated from the motion vector detected for each block.

According to the embodiment of the present invention, when estimating the motion vector of each block of the encoding target frame, a motion vector from the encoding target block to an adjacent frame is detected, and the motion vector is obtained from the detected motion vector. A block of an adjacent frame constituting the prediction block is extracted, and a motion vector from the reference frame to the encoding target frame is estimated for each encoding target block based on information of the extracted block.

According to the embodiment of the present invention, when estimating the motion vector of the current block based on the motion vectors of the current frame and the blocks of adjacent frames constituting the predicted block, For each block of the adjacent frame that constitutes, the composition ratio of the prediction block is calculated, a weight coefficient is calculated based on the calculated composition ratio, the motion vector of each block in the adjacent frame is weighted, and the The motion vector to the encoding target frame is estimated for each encoding target block.

According to the embodiment of the present invention, when estimating the motion vector of each block of the encoding target frame based on the motion vector of the encoding target frame and the extracted block of the adjacent frame, Evaluate how much the motion vector of the extracted block represents the actual motion, and estimate the motion vector from the reference frame to the encoding target frame for each encoding target block without using the motion vector of low evaluation value I do.

According to an embodiment of the present invention, in a moving picture coding method using motion-compensated inter-frame prediction, when a frame to be coded refers to a frame other than an adjacent frame, a frame between the reference frame and the frame to be coded is In the case where there are a plurality of frames existing in, a motion vector from the reference frame to the encoding target frame is estimated by repeating any one of the above motion vector estimations.

The moving picture coding apparatus according to the present invention calculates a motion vector of a block existing at a position of a prediction block in an adjacent frame determined from a motion vector to an adjacent frame of each coding target block of the coding target frame. It has a motion vector estimating means for estimating a motion vector from a reference frame to an encoding target frame for each encoding target block.

According to the embodiment of the present invention, the motion vector estimating means calculates a constituent ratio of a predicted block for each block of an adjacent frame forming the predicted block,
A weight coefficient is calculated based on the calculated composition ratio, and a motion vector of each block in an adjacent frame is weighted.
A motion vector from the reference frame to the encoding target frame is estimated for each encoding target block.

According to the embodiment of the present invention, the motion vector estimating means evaluates how much the motion vector of the extracted block of the adjacent frame represents the actual motion,
A motion vector from the reference frame to the encoding target frame is estimated for each encoding target block without using a motion vector having a low evaluation value.

According to the present invention, a motion vector is detected from a frame to be encoded and a frame having a distance between frames of 1 (hereinafter, an adjacent frame), and a motion vector having a reference relationship that refers to a frame other than the adjacent frame is detected from the motion vector. presume.

According to the present invention, a motion vector from a current block to a reference frame is estimated by the procedure shown in FIG.

1. The position of the prediction block is calculated from the motion vector of the current block to the adjacent frame (step 101).

2. One or more reference blocks in adjacent reference frames constituting a prediction block are extracted (step 102).

3. It is evaluated whether the motion vector of each adjacent reference block represents an actual motion, and a motion vector having a low evaluation value is excluded (step 103).

4. The weight coefficient of the motion vector of each adjacent reference block is calculated (step 104).

5. The motion vector of the current block is calculated from the motion vector of the current block to the adjacent frame, the motion vector of each adjacent reference block, and the weight coefficient (step 105).

Next, the basic configuration of the present invention will be described with reference to an example in which a past frame is referred to. FIG. 2 shows the basic configuration of the present invention. First, the input frame is divided into blocks by the block dividing unit 201. The motion vector detection unit 202 detects a motion vector from the immediately preceding frame stored in the frame memory 203. The detected motion vector is stored in the motion vector memory 20.
4 is stored. If the frame in the frame memory 203 is a reference frame, the switch 206 is switched to “D” and the detected motion vector is passed to the encoding processing unit 207 to perform encoding. If the frame in the frame memory 203 is not a reference frame, the motion vector memory 20
4, the motion vector estimating unit 205 estimates a motion vector from the current frame to the reference frame. The switch 206 is switched to “E”, and the estimated motion vector is passed to the encoding processing unit 207 to perform encoding. Here, the capacity of the motion vector memory 204 is proportional to the maximum value of the inter-frame distance between the encoding target frame and the reference frame. For example, in the reference relationship shown in FIG. 10 described above, a capacity that can store the motion vectors for two frames is required.

As an operation example of the proposed method, consider the detection of a motion vector based on the reference relationship shown in FIG. It is assumed that the frame n has already been stored in the frame memory 203. The search range is U, the number of operations per frame at ± S × ± S
Let N be the number of blocks. FIG. 3 shows the relationship between the reference relationship of each frame and the motion vector.

When the frame n + 1 is input, the block i of the encoding target frame is
(I = 1, 2,..., N) motion vector

[0031]

[Outside 1] Is detected. Motion vector

[0032]

[Outside 2] Are stored in the motion vector memory 204. Since the reference relationship P1 is a reference between adjacent frames, the switch 206
To “D”, and passes the detected motion vector to the encoding processing unit 207 to perform encoding.

Next, when frame n + 2 is input,
A motion vector for each block of the encoding target frame between the adjacent frame n + 1

[0034]

[Outside 3] Is detected. Motion vector as before

[0035]

[Outside 4] Are stored in the motion vector memory 204. Reference relationship P
2 is not a reference between adjacent frames, so the motion vector

[0036]

[Outside 5] Is estimated. In the motion vector estimation unit 205

[0037]

[Outside 6] From the motion vector V ₀₁ stored in the motion vector memory 204.

[0038]

(Equation 1)

Here, V ₀₁ is the frame n and the frame n +
It is a motion vector of all blocks detected between one. Since the reference relationship P2 is not a reference between adjacent frames, the switch 206 is switched to “E” and the estimated motion vector

[0040]

[Outside 7] Is passed to the encoding processing unit 207 to perform encoding.

Next, when frame n + 3 is input,
A motion vector for each block of the encoding target frame between the adjacent frame n + 2

[0042]

[Outside 8] Is detected. Motion vector as before

[0043]

[Outside 9] Are stored in the motion vector memory 204. Since the reference relation P3 is not a reference between adjacent frames, the motion vector

[0044]

[Outside 10] Is estimated. In the motion vector estimation unit 205,

[0045]

[Outside 11] From the motion vectors V ₀₁ and V ₁₂ stored in the motion vector memory 204.

[0046]

(Equation 2)

Here, V ₀₁ is the frame n and the frame n +
Motion vectors of all blocks detected between 1, V ₁₂ represents a motion vector of the entire blocks detected between frame n + 1 and frame n + 2. Since the reference relation P3 is not a reference between adjacent frames, the switch 206 is switched to “E” and the estimated motion vector

[0048]

[Outside 12] Is passed to the encoding processing unit 207 to perform encoding.

Next, the motion vector estimating unit 205 will be described. FIG. 4 shows the relationship between the current block and the reference frame. This figure shows the reference relationship P2 in FIG. The encoding target frame is n + 2, and the reference frame is n.

First, a motion vector to the adjacent frame of the encoding target block

[0051]

[Outside 13] , The position of the prediction block in the adjacent frame is obtained. Motion vectors of one or more blocks (hereinafter, adjacent reference blocks) existing at the position of the prediction block

[0052]

[Outside 14] Is extracted from the encoded data. The number of adjacent reference blocks is
It is determined by the block shape and the arrangement of the prediction block and the adjacent reference block. In the example of FIG. 4, the block shape is a square, and the prediction block includes four adjacent reference blocks (bloc
k _i ) (i = 1 to 4) and four motion vectors

[0053]

[Outside 15] Exists. The motion vector of this adjacent reference block

[0054]

[Outside 16] And a new motion vector is calculated from the following equation.

[0055]

[Outside 17]

Is calculated.

(Equation 3)

Here, the weight w _i is calculated from the relationship between the adjacent reference block and the prediction block. For example, it can be calculated from the area ratio of each adjacent reference block block _i shown in the prediction block. In addition, there is a correlation coefficient between the image signal of the current block and the image signal of each adjacent reference block. In addition, by calculating the reliability of the motion vector of the adjacent reference block and excluding the motion vector having a low reliability, the prediction efficiency can be improved. The reliability of the motion vector includes a prediction error at the time of motion detection, a coding mode, and a direction and a size of the motion vector.

Next, a case where the distance between the encoding target frame and the reference frame is 3 or more will be described. In the case of the reference relationship P3, first, from frame n + 3 to frame n
Motion vector to +1

[0059]

[Outside 18] Is estimated by the above method. And the motion vector

[0060]

[Outside 19] Of FIG.

[0061]

[Outside 20] And a motion vector based on the motion vector V ₀₁ of the frame n + 1.

[0062]

[Outside 21] Is estimated. Similarly, when the frame interval is 3 or more,
By repeatedly performing the estimation, a motion vector between the encoding target frame and the reference frame can be estimated.

The prediction in the future direction can be easily realized by reversing the order of the above-mentioned reference.

The amount of calculation in the proposed method is shown below. First, the number of motion vector searches between adjacent frames is only the search between adjacent frames, and is three times V ₀₁ , V ₁₂ , and V ₂₃ . Therefore, the calculation amount is 3U. The number of times of reference to each frame (n, n + 1, n + 2) is one.

For reference, the amount of calculation and the number of times of reference when the same reference relation is used for motion vector detection by telescopic search are shown. The amount of calculation is U + (U + U) + (U + U
+ U) = 6U, and each frame (n, n + 1, n +
The number of times of reference in 2) is 3, 2, and 1, respectively.

[0066]

Next, embodiments of the present invention will be described with reference to the drawings.

FIG. 1 is a block diagram of a moving picture coding apparatus according to one embodiment of the present invention. The encoding algorithm of the present embodiment includes motion compensation inter-frame prediction and discrete cosine transform (DC
T). In the motion compensation inter-frame prediction, the reference relations P1 and P2 among the reference relations shown in FIG.
Do with. The block shape is a square, and the weight coefficient for estimating the motion vector is calculated from the area ratio. In the present embodiment, there are two types of coding modes: an inter-frame coding mode (hereinafter, an inter mode) and an intra-frame coding mode (hereinafter, an intra mode). The encoding mode is switched to the intra mode depending on the magnitude of the prediction error at the time of detecting the motion vector.

In FIG. 6, the same reference numerals as those in FIG. 2 indicate the same components.

The encoding processing unit 207 includes the block dividing unit 30
1, a frame memory 302, a motion compensation unit 303, a switch 304, a subtractor 305, a DCT unit 306, and a quantization unit 3.
07, the inverse quantization unit 308, the IDCT unit 309, and the adder 3
10 and an encoding unit 311.

First, the input frame is divided into blocks by the block dividing section 201. The motion vector detection unit 202 detects a motion vector v ₀ from the immediately preceding frame stored in the frame memory 203. At this time, the coding mode m ₀ of the current block is determined from the prediction error Err by the following equation.

[0071]

(Equation 4)

Here, TH is a threshold value for determining a coding mode. The detected motion vector v ₀ is stored in the motion vector memory 204, and the coding mode m ₀ is stored in the coding mode memory 208. If the frame in the frame memory 203 is a reference frame such as the reference relationship P1, the switches 206A and 206B are set to "D".
And the detected motion vector and the encoding mode m ₀
Is passed to the encoding processing unit 207 to perform encoding processing. When the reference relationship is a frame other than the reference frame, such as P2, the motion vector stored in the motion vector memory 204 and the coding mode stored in the coding mode memory 208 are changed. The motion vector estimation unit 205 estimates a motion vector from the encoding target frame to the reference frame. The switch 206A and the switch 206B are switched to “E”, and the estimated motion vector

[0073]

[Outside 22] And the coding mode m _new are passed to the coding processing unit 207 to perform coding processing.

The encoding processing unit 207 divides the encoding target frame into blocks, and performs an encoding process based on the encoding mode determined previously. If the encoding mode is inter, the motion compensation unit 303 performs motion compensation on the encoded frame in the frame memory 302 using the previously calculated motion vector, and extracts a prediction block. When the encoding mode is intra, the signal of the prediction block is 0. The DCT section 306 performs DCT on the difference between the prediction block and the current block, quantizes the result with the quantization section 307, and performs variable length coding with the coding section 311.
The quantized signal is inversely quantized by an inverse quantization unit 308, subjected to inverse DCT by an IDCT unit 309, added by a predictor block signal by an adder 310, and decoded. This decoded signal is stored in the frame memory 302, and then used for motion compensation of the encoding target frame.

FIG. 7 is a flowchart showing the procedure for determining the coding mode of the current block.

1. Motion vector of the current block to the adjacent frame

[0077]

[Outside 23] And the encoding mode m ₀ (step 401).

2. In step 402, if the coding mode m ₀ of the current block is the intra mode,
Encoding is performed by intra-frame encoding.

3. In step 403, if the reference frame is an adjacent frame, the detected motion vector

[0080]

[Outside 24] Is used for motion compensated inter-frame prediction.

4. From the coding mode and motion vector of each reference block in the adjacent frame, the motion vector to the reference frame

[0082]

[Outside 25] And the encoding mode m _new are calculated (step 404).

5. By the calculated encoding mode m _new ,
The coding mode of the current block is determined.

Motion vector in procedure 4 above

[0085]

[Outside 26] The method for calculating the encoding mode m _new is as follows. FIG. 8 shows a flowchart of the motion vector calculation method.

1. Motion vector of the block to be coded

[0087]

[Outside 27] (Step 50)
1).

2. Extract one or more reference blocks in adjacent frames where the prediction block overlaps (step 50)
2). The number of reference blocks that may overlap with one to four adjacent reference blocks block _i depending on the position of the prediction block (FIG. 5) is NB.

3. Predicted block and each adjacent reference block bl
The area p _i where the ock _i (i = 1,..., NB) overlaps is calculated (step 503).

4. Examine the coding mode m _i of each adjacent reference blocks block _i, excludes a reference block of an intra mode (step 504).

5. If the occupied area of the adjacent block in the intra mode exceeds 50% in step 505, the coding mode m _new of the current block is set to the intra mode.

6. Inter mode neighbor block Inter bl
The total area p _all occupied by ock _i (i = 1,..., NIB ≦ NB) is calculated (step 506). Where NIB
Is the number of inter mode neighbor blocks, Inter p _i is the area occupied by each inter-mode adjacent block.

[0093]

(Equation 5) 7. Area occupied by adjacent blocks in each inter mode p _i ,
Motion vector of each adjacent block

[0094]

[Outside 28] Motion vector of the current block to the adjacent frame

[0095]

[Outside 29] From the motion vector of the current block

[0096]

[Outside 30] Is calculated.

[0097]

(Equation 6)

Here, in the procedure 4, the blocks in the intra mode are excluded. The fact that the encoding mode is the intra mode indicates that the prediction error at the time of detecting the motion vector is large, and it is considered that the reliability of the motion vector is low. Therefore, in order to exclude a motion vector with low reliability, the intra mode is excluded.

In the embodiment described above, the area ratio is used as the weighting coefficient. However, the correlation coefficient between the image signal of the block to be encoded and the image signal of the reference block may be used. Although the coding mode was used as the evaluation value of the correctness of the motion vector,
The prediction error at the time of detecting the motion vector or the direction of the motion vector may be used. Also, although the blocks are square, this is not a limitation.

[0100]

As described above, according to the present invention,
The calculation amount of the motion vector search and the number of references to the frame memory can be reduced, and the calculation amount and the number of references do not depend on the distance between the encoding target frame and the reference frame. As a result, the amount of calculation and the number of references are reduced, and a motion vector search with a uniform amount of calculation and the number of references can be performed.

[Brief description of the drawings]

FIG. 1 is a flowchart of a motion vector estimation method according to the present invention.

FIG. 2 is a block diagram of a video encoding device according to the present invention.

FIG. 3 is a diagram showing estimation of a motion vector.

FIG. 4 is a diagram illustrating a relationship between a current block and a reference frame.

FIG. 5 is a diagram showing a reference block overlapping a prediction block.

FIG. 6 is a block diagram of a video encoding device according to an embodiment of the present invention.

FIG. 7 is a flowchart showing a coding mode determination procedure of a current block in the embodiment of FIG. 6;

FIG. 8 is a flowchart of a motion vector calculation in the embodiment of FIG. 6;

FIG. 9 is a diagram showing a telescopic search.

FIG. 10 is a diagram showing a reference relationship between frames.

[Explanation of symbols]

 101 to 105 Step 201 Block dividing unit 202 Motion vector detecting unit 203 Frame memory 204 Motion vector memory 205 Motion vector estimating unit 206, 206A, 206B Switch 207 Coding processing unit 301 Block dividing unit 302 Frame memory 303 Motion compensating unit 304 Switch 305 Subtractor 306 DCT section 307 Quantization section 308 Dequantization section 309 IDCT section 310 Adder 311 Encoding section 401-405 step 501-507 step

 ──────────────────────────────────────────────────続 き Continuing on the front page (72) Inventor Jun Sagata 3-19-2 Nishishinjuku, Shinjuku-ku, Tokyo Nippon Telegraph and Telephone Corporation

Claims (8)

[Claims] 1. A moving picture coding method which divides an input frame into blocks, detects a motion vector for a past or future adjacent frame for each block, and performs motion compensation inter-frame prediction based on the detected motion vector. In the method, when a frame other than the frame to which the encoding target frame is adjacent is used as a reference frame, the reference frame and the encoding target frame are determined from the motion vector detected for each block existing between the reference frame and the encoding target frame. A moving picture coding method characterized by estimating a motion vector between the moving pictures. 2. When estimating a motion vector of each block of an encoding target frame, a motion vector from the encoding target block to an adjacent frame is detected, and an adjacent frame forming a prediction block obtained from the detected motion vector. 2. The method according to claim 1, further comprising: extracting a block of (i), and estimating a motion vector from the reference frame to the encoding target frame for each encoding target block based on information of the extracted block. 3. When estimating a motion vector of an encoding target block based on a motion vector of an encoding target frame and a motion vector of an adjacent frame constituting the prediction block, each block of the adjacent frame configuring the prediction block is used. , Calculate the composition ratio of the prediction block, calculate the weighting factor based on the calculated composition ratio, weight the motion vector of each block in the adjacent frame, and calculate the motion from the reference frame to the encoding target frame. 3. The method according to claim 2, wherein the vector is estimated for each current block. 4. When estimating a motion vector of an encoding target block based on a motion vector of an encoding target frame and an extracted block of an adjacent frame, how much the motion vector of the extracted block of the adjacent frame is actually Is evaluated, and without using a motion vector having a low evaluation value, a motion vector from the reference frame to the encoding target frame is estimated for each encoding target block.
The method of claim 2. 5. In a moving picture coding method using motion compensated inter-frame prediction, when a current frame refers to a frame other than an adjacent frame, there are a plurality of frames existing between the reference frame and the current frame. If
5. A moving picture coding method comprising: estimating a motion vector from a reference frame to a coding target frame by repeating the motion vector estimation according to claim 1 a plurality of times. 6. A moving picture coding apparatus which divides an input frame into blocks, detects a motion vector for a past or future adjacent frame for each block, and performs motion compensation inter-frame prediction based on the detected motion vector. The target frame to be encoded from the reference frame based on the motion vector of the block existing at the position of the prediction block in the adjacent frame determined from the motion vector to the adjacent frame of each encoding target block of the encoding target frame. A moving picture coding apparatus comprising a motion vector estimating means for estimating a motion vector for each coding target block. 7. The motion vector estimating means calculates, for each block of an adjacent frame forming a prediction block, a configuration ratio of the prediction block, calculates a weight coefficient based on the calculated configuration ratio, and 7. The apparatus according to claim 6, wherein the motion vector of each block in the block is weighted, and the motion vector from the reference frame to the encoding target frame is estimated for each encoding target block. 8. The motion vector estimating means evaluates to what extent a motion vector of a block extracted from an adjacent frame represents an actual motion, and uses a motion vector having a low evaluation value to calculate a motion vector from a reference frame. 7. The apparatus according to claim 6, wherein a motion vector to the encoding target frame is estimated for each encoding target block.