JPH07143492A - Video coding method by automatic motion search range adjustment with respect to motion estimation - Google Patents

Abstract

PURPOSE:To obtain a video coding method which a motion search range is adjusted according to the magnitude of motion and the processing time is reduced while keeping excellent motion prediction. CONSTITUTION:After all motion vectors are checked, a motion retrieval range to be a base is set based on the magnitude of the motion vectors. A matching block is retrieved within the motion retrieval range to obtain a minimum prediction error. Whether the minimum prediction error is smaller than a threshold is checked, and the motion retrieval range is extended to be adaptive from the base range or unchanged depending on the result of check.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to video signal compression, and more particularly to a video coding method using motion estimation and compensation of moving images.

[0002]

2. Description of the Prior Art Digital transmission and storage of television or video signals have become more and more attentional these days because of their higher quality and greater flexibility as compared to analog signals.

Due to the large amount of image data and the limited transmission bandwidth of digital video images, video signal coding techniques need to be extremely powerful. The most efficient compression systems utilize not only spatial correlation similar to that performed in still image processing, but also temporal correlation between adjacent frames. A system that considers both spatial and temporal correlation is also called a hybrid coding system.

Redundancy removal in the spatial domain has been performed for a long time, various types of transform methods have been tried, and DCT (Discrete Cosine Transform) is the best view in terms of efficiency and implementation complexity. It turned out that
The DCT is capable of compressing the energy of the transformed data into a small region, thus requiring less data to represent the original data.

Removal of temporal redundancies is basically implemented using differential coding techniques. That is, first a prediction is made based on information derived from previous frames of the same video sequence, and the difference between the actual frame and the actual frame prediction is transmitted.

Since the amount of movement in a video frame varies significantly in different parts of the frame, the video frame is processed by dividing the frame into many pixel data blocks. For each block of the frame, the motion vector is derived by comparing the pixel data block of the current frame with the pixel data block of the previous frame, and the motion compensation is similar to the block data of the current frame and the previous frame. Is performed by obtaining the difference between the block data of Therefore, differential encoding is achieved by performing motion compensation on each pixel data block for the frame.

The process of determining a motion vector is called motion estimation, and as is well known, efficient motion estimation leads to efficient motion compensation with excellent prediction accuracy, and A highly efficient coding system will be obtained.

Further, if the processing time for motion estimation is shortened, the system can be easily realized and the cost of the system can be reduced. A good system with high coding efficiency and low cost is a good product feature.

[0009]

Current compression algorithms utilize the same algorithm to encode different types of video sequences, but some sequences do not contain motion. , Some motions are slow, but some motions are extremely fast.

MPEG (Moving Picture)
The Expert Group coding standard utilizes a search window matching method to search for similar blocks of pixel data in the previous or subsequent frames. The size of the search window is called the search range. The search range is fixed (+/-) irrespective of the motion content of the image frames in the sequence, i.e. whether it is stationary, slow motion or fast motion, over the entire frame and even over the process. -15 pixels or +/-
30 pixels).

The wider the search range, the more the amount of calculation required to carry out the search. If the search range is too narrow,
There is a possibility that the motion block is located outside the search range.

A fixed search range will cause some problems in the sequence depending on the degree of motion variability, eg very fast motion in a frame (ie a large actual motion vector value).
In that case, the fixed motion search range is not sufficient to cover the motion block and the motion compensation becomes less efficient. FIG. 1 shows the problems that occur when the +/− 15 motion search range is insufficient.

The reason for the inefficiency is that in the motion search range of +/− 15, the real matching pixel data block (A ′ in FIG. 9) cannot be found in the preceding frame, so that the false matching pixel data block (see FIG. 10 B)
It is to be taken as an alternative to the real matched block of A. As a result, the prediction error is so large that its coding is difficult. Moreover, the block reconstructed from the false matched block is significantly different from the real block of A in the current frame, so significant distortion may be observed.

On the other hand, when the motion in the frame is extremely slow or close to a stationary state, an extremely narrow motion search range is sufficient.
Motion compensation with a search window of +/- 15 pixels would waste a lot of unnecessary computational time.

FIG. 10 shows the problem in this case. A is the current block, A ₀ ′ is the block in the reference frame at the same position as A in the current frame, and A ′ is the matching block of A. For slow motion sequences, the A, A'matched block is in the reference frame at about the same position as A in the current block. That is, in the case of such a sequence, a very narrow search range is sufficient, and in the case of a wider fixed search range, a lot of processing time is required.

The present invention considers the above problems of the conventional video coding method, and according to the magnitude of the motion,
Video that can adjust the motion search range and reduce processing time while maintaining good motion prediction
It is intended to provide a coding method.

[0017]

SUMMARY OF THE INVENTION The present invention, in a video coding method utilizing motion estimation and compensation with adaptive motion search range, partitions a current frame of an input video signal into many blocks of pixel data. Partitioning the reference frame of the input video signal into a number of blocks of pixel data, and searching for a block within the basic motion search range that best matches the current block of the reference frame (most in the reference frame). A vector representing a matching block is called a motion vector), a step of detecting a moving state in the current frame based on the search result, and if there is a high-speed motion according to the detection result, Expand the basic motion search range to the applicable value So, there are no motion, or, for slow, and steps to reduce the basic motion search range to fit values, and a video coding method comprising.

[0018]

The present invention detects, for example, slow or fast motion sequences in the encoding process and assigns a small or intermediate value as the initial base of the motion search range for slow or fast motion, respectively. assign. Next, the base compares the prediction error with a threshold to determine whether the base value of the motion search range is sufficient to maximize the efficiency of motion estimation, that is, the processing time to be used is shortened as much as possible. Adjusted to see if it is sufficient to obtain the best image quality for video coding.

[0019]

DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention will be described below with reference to the drawings showing its embodiments.

In the present invention, the motion
It also accommodates different modes of motion compensation that require vector search, but these include frame-based motion compensation, field-based motion compensation, and adaptive frame / field motion compensation.

1 and 2 are diagrams for explaining a video coding method according to an embodiment of the present invention. 1 and 2, MSR in step S11 represents a motion search range, and min_error in step S18.
Is the minimum error of all prediction errors in the constant motion search range, and each prediction error is the sum of absolute differences of each pixel pair between the block of the current frame and the block of the reference frame.

The threshold value in step S18 in FIG. 2 is a preset value, and MSR_exp in step S19 is
It is the motion search range after expansion, and step S
Exp of 19 is a value that is expanded based on the initial base value of the motion search range, and T_p of step S23.
Is the time period that needs to be determined before the start of encoding.

The processing in this embodiment has the following two main steps.

A) The first step is step S in FIG.
At all defined times shown in FIG. 23, step S in FIG.
11, all motions of the frame shown in S12, S13
Finding the minimum base motion search range by checking the vector (eg, T_p
= 1 second).

The detailed processing is as follows, starting from the first frame.

First, in steps S11 and S12 of FIG. 1, the first frame of the video sequence is coded to remove spatial correlation, and each pixel data block of the second frame is converted to the first frame. Predicted from the frame, i.e. the intermediate motion search range (eg + /
A matching pixel data block within -15 pixels) is found. Next, in step S13, a motion vector indicating the position of each current block in the reference frame is calculated, and a smaller value (for example, +/-) is calculated.
8 pixels). If the absolute value of the motion vector is much smaller than the smaller value,
The motion search range is set to the intermediate value (+/− 15) for the next other frame, as shown in step S14 of FIG.
Pixel) to a smaller value (+/− 8 pixels), or it remains at an intermediate value (+/− 15 pixels) as in step S15 of FIG.

B) The second step is to search for a real matched block based on the base motion search range obtained from the first step by expanding the motion search range to accommodate it. The detailed process below is included.

In steps S16 and S17 of FIG. 1, reference is made to the intermediate motion search range (+/− 15 pixels) for fast motion and a smaller value (+/− 8 pixel) for slow motion. Search for a matching block in the frame.

FIG. 3 shows a detailed diagram of steps S16 and S17. As shown in FIG. 3, step S
31 is to get the block from the current frame,
Step S32 is to obtain the block at the same position from the reference frame, and Step S33 is to calculate the absolute error between the two blocks to obtain the minimum error (min1, min2 regarding field 1 and field 2). Yes, minimum error (min on frame)
Calculates the sum of the minimum errors for field 1 and field 2. Step S34 is to obtain the next block from the reference frame within the motion search range,
Step S35 is to calculate the absolute error (err1, err2 for field 1 and field 2) and the error for frame (err) is the sum of the errors for field 1 and field 2. In step S36, err1, err2, and err
Are respectively compared with min1, min2, and min, and if any of them are smaller than the corresponding one, they replace min1, min2, and min. These steps are performed until the end of the motion search range is reached, as shown in step S37.
When the end of the motion search range is not reached, steps S34 to S37 are repeatedly executed, and when the end of the motion search range is reached, the process returns to step S31,
Following the same procedure as above, the other blocks in the current frame are processed.

In FIG. 4, min1, min2, min,
The definitions of err1, err2, and err are shown. 3 min1 is the absolute error of field 1 between two blocks, the current block (A) from the current frame and the block (B) from the reference frame at the same position as the current block, 3 mi
n2 is the absolute error of field 2 between the two blocks, min in FIG. 3 is the absolute error of the frame between the two blocks, min is min1 and m
It is the sum of in2.

Err1 in FIG. 3 is the absolute error for field 1 between the current block (A) and another block (B1) from the reference frame within the constant motion search range, and err2 in FIG. , Er is the absolute error for field 2 between the current block (A) and another block (B1) from the reference frame within the constant motion search range, where err in FIG. It is the absolute error for the frame between the current block (A) and another block (B1) from the reference frame.

The minimum prediction error (min_error) is
As shown in step S18 of FIG. 2, a predetermined threshold (Th
It is checked whether it is less than the threshold value, and if it is not less than the threshold value, as shown in step S19 of FIG.
If it is necessary to expand the motion search range, and if it is equal to or greater than the threshold value, as shown in step S20 of FIG.
It will be kept the same.

In FIG. 5, steps S18, S19 and
A detailed diagram of S20 is shown. As shown in FIG.
When the end of the motion search range is reached, step S40
, Min1, min2, and min for field 1, field 2, and frame, respectively.
Are calculated and compared with threshold 1, threshold 2 and threshold in step S41, respectively. min1 is the threshold 1
Is less than, and at the same time, min2 is less than the threshold value 2,
Alternatively, if min is less than the threshold value, it is not necessary to expand the MSR (motion search range), and therefore the next block is continued (step S43),
If not, it is necessary to expand the MSR (step S42).

FIG. 6 shows a predetermined threshold value and its adjusting method. In this case, the threshold is slightly adjusted by the relative deviation of the current block. If the deviation of the block is large, that is, in the case of a detailed area having a large deviation, the matching error (prediction error) becomes large especially when the area has high-speed motion. In this case, as shown in FIG. 6, the relative deviation is raised. That is, adjustment is performed.

Dev shown in step S50 of FIG. 6 is the standard deviation of the current block, and dev_t of step S
Is the total deviation for all the blocks so far, dev_a in step S52 is the average deviation divided by the number of blocks, and dev_m in step S53 is an adjustment value obtained from dev and dev_a. The above-mentioned respective deviations are respectively obtained by (Equation 1), (Equation 2), (Equation 3).

[0036]

[Equation 1]

Here,

[0038]

[Equation 2]

[0039]

[Equation 3]

Here, num_mb is the number of blocks so far.

The threshold is used to check the motion search range for frame motion estimation,
Threshold 1 and Threshold 2 are used to check the motion search range for adaptive frame / field motion estimation.

As shown in FIG. 6, step S50 is to calculate the deviation of the current block, the result of which is
Shown as dev. Average deviation of current block de_
a is obtained from steps S51 and S52, and is the adjustment value de
_M is obtained from step S53. Threshold 1, threshold 2
Also, the threshold is adjusted by de_m in step S54.

The motion search range is step S2 in FIG.
When expanded in 1, the search for a matching block in the expanded search range is continued.

System implementation of motion estimation by adaptive motion search range.

An implementation diagram of motion estimation by the adaptive motion search range is shown as follows.

The method of the present invention is particularly applicable to MPEG-II (Moving Picture) as shown in FIG.
Expert Group, Phase II),
"Coded Representation of
Picture and Audio Information
"ion", ISO-IEC / JTC1 / SC29 / W
G11 MPEG93 / 225, January 19
When used to encode a video or television signal used in 93, the input video is input to the system one frame at a time and then the frame is subdivided into many pixel data blocks (step S1). ).

When the second frame is input, in step S2, a full pixel motion estimation is performed with the initial intermediate motion search range (in this case, the selected +/− 15 pixels) and is obtained for this frame. All motion vectors are checked to see if they are significantly smaller than a smaller value (in this case, the selected +/- 8 pixels), which results in the motion search range becoming the base motion search range. To be reset. In step S3, a half pixel motion estimation is performed based on the motion vector resulting from the full pixel motion estimation. Furthermore, for this frame, motion compensation (step S4), DCT
(Step S5), quantization (step S6), run length coding and variable length coding (step S7) are performed.

FIG. 8 shows a detailed implementation of the motion estimation in FIG.

When the third frame is input, all-pixel motion estimation is performed by this base motion search range (step S61 in FIG. 8), and when all blocks of all frames are coded, the base motion is estimated. The search range is checked to confirm whether it is sufficient (step S62 in FIG. 8). If yes, continue motion estimation for other blocks, if not,
The base motion search range is expanded (step S63 in FIG. 8), and the search is continued for the matching block within the expanded motion search range (step S64 in FIG. 8).
When all-pixel motion estimation is complete, half-pixel motion estimation, motion compensation, DCT, quantization, run length coding, and variable length coding are performed as usual.

In FIG. 8, "inverse quantization" (step S
12), "inverse DCT" (step S11), and "locally decoded frame memory" (step S10) together constitute a local decoder for motion compensation,
The "rate controller" (step S13) allocates bits for each frame, the "frame interpolation" (step S9) relates to half-pixel motion estimation, and the "reference frame memory" (step S8) Store the immediately preceding frame for motion estimation.

Thus, the present invention has two major improvements over the prior art, which has a constant motion search range throughout the encoding process. For very fast motions where the constant motion search range is not enough to get a good matched block, the adaptive extension of the motion search range improves the efficiency of motion estimation,
Therefore, the coding efficiency and the image quality are improved. This improvement is for MPs like "Marble" and "Firework"
It can be found in the EG test sequence.

On the other hand, in the case of slow motion or still scenes, the motion estimation is performed by reducing the search range to a narrower range (for example, +/− 8 pixels) so as to adapt without lowering the efficiency of motion estimation. Save a lot of processing time. This improvement can be found in the MPEG test sequence "Mobile &Calendar".

The test experiments were carried out with the "Marble" and "Firework" sequences (60 frames) for the fast motion sequence and the "Mobile &Calendar" sequence (60 frames) for the slow motion sequence. However, here, only the case of high-speed motion is illustrated in (Table 1).

[0054]

[Table 1]

The processing time% saved is compared with the constant motion search range (+/− 30) plan, and the motion search range (+/− 15 or +/− 30) is selected to adapt to the motion. It shows how much of the processing time can be saved when the estimation is performed.

As is clear from (Table 1), when the motion search range is fixed at +/- 15, the prediction error is "Marble" and "Firework", respectively.
When the motion search range is fixed to +/− 30, which is extremely large as “1373” and “2093”, the prediction error becomes small as “809” and “1608”, respectively, but the processing time significantly increases and the motion When the search range is changed to +/− 15 or +/− 30 to adapt, the prediction error is almost the same as when fixed at +/− 30, but the processing time is 51% and 48%, respectively. Saves

In conclusion, the prediction error when the motion search range is selected to be adapted is as small as when the motion search range is fixed to a large value, and the processing time when adapted is +/- 30. 4 when fixed
9% and 52%. Therefore, the concept of automatic motion search range adjustment is to reduce the processing time and enable good prediction of motion estimation.

As described above, first, all motion vectors of the frame are checked, and a narrow or intermediate motion search range (base) is set depending on whether the motion is slow or fast, Motion estimation is done by using this base as the search window size. Next, the motion estimation error (prediction error) of a certain area is checked, the error is compared with a predetermined threshold, and the motion search range (base) is adjusted according to the result of the comparison. If the prediction error exceeds the threshold, the motion search range is expanded, and if it is less than the threshold, the motion search range is reduced. Further, the characteristics of the processing area are examined, and the thresholds are slightly adjusted so that the areas having different characteristics have different thresholds.

[0059]

As is apparent from the above description, the present invention has the advantage that the motion search range can be adjusted according to the magnitude of motion and the processing time can be shortened while maintaining good motion prediction. Have.

[Brief description of drawings]

FIG. 1 is a diagram illustrating a part of a video coding method according to an embodiment of the present invention.

FIG. 2 is a diagram illustrating a part of the rest of the embodiment.

FIG. 3 is a diagram showing a method for searching a matching block in the embodiment.

FIG. 4 is a diagram showing a calculation of a field error between two blocks in the embodiment.

FIG. 5 is a diagram showing a method of checking whether or not the motion search range is sufficient in the embodiment.

FIG. 6 is a diagram showing a method of determining and adjusting a threshold value in the embodiment.

FIG. 7 is a diagram showing a coding system based on MPEG-II based adaptive motion estimation.

FIG. 8 is a detailed diagram for explaining an all-pixel motion estimation portion of FIG. 7;

FIG. 9 is a diagram showing an example in which a motion matching range of a conventional fixed motion searching range is too narrow to find a real matching block of a current block.

FIG. 10 is a diagram showing an example in which a motion search range based on a conventional fixed motion search range is too wide and unnecessary processing time is wasted significantly.

[Explanation of symbols]

 1 Reference frame 2 Current frame

 ─────────────────────────────────────────────────── ─── Continued Front Page (72) Inventor Lee Chak Joo Singapore 1544 Laguna Park # 05-13 Marine Parade Road Block 5000D

Claims (5)

[Claims] 1. A video coding method utilizing motion estimation and compensation with adaptive motion search range, partitioning a current frame of an input video signal into many blocks of pixel data, and a reference frame of the input video signal. Partitioning into a large number of blocks of pixel data, and searching a block in the basic motion search range that best matches the current block of the reference frame (the vector representing the best matching block in the reference frame is , Called a motion vector)
And a step of detecting a moving state in the current frame based on the search result, and if there is a high-speed motion according to the detection result,
Expanding the basic motion search range to a compatible value, and reducing the basic motion search range to a compatible value when there is no motion or at low speed; Encoding method. 2. The movement state is detected by obtaining a prediction error between the current block and the matching block, and comparing the prediction error with a preset threshold value to determine whether or not the motion is fast. And, as a result of the comparison, if there is a high-speed motion in the input image, the basic motion search range is expanded to a suitable value, and a motion vector is detected from the corrected search range. The video encoding method according to claim 1, further comprising: 3. The basic motion search range is a step of obtaining all motion vectors of a frame corresponding to the lowest prediction error, and checking all the motion vectors, and all of the motion vectors are less than a preset value. Is determined, and as a result of the determination, all of the motion vectors are
The video coding method according to claim 1 or 2, wherein when the value is less than a preset value, the motion search range is further adjusted based on the step of reducing the motion search range. 4. Finding the corresponding matching block from the frame decoded by the local decoder using the motion vector obtained from the motion estimation, and the current block of the current frame and the matching block from the decoded frame. 4. The method according to claim 1, further comprising: a step of obtaining a prediction error between the two and a step of encoding the prediction error in preparation for transmission.
Video encoding method described. 5. A step of calculating a standard deviation of the pixel data of the current block, a step of calculating an average standard deviation of the pixel data of the current frame, and a step of calculating an adjustment value from the standard deviation and the average standard deviation. The video coding method according to claim 2, wherein the threshold value is adjusted based on the step of adjusting the threshold value using the adjustment value.