JP2702139B2 - Video predictive coding - Google Patents

Description

Detailed Description of the Invention [Table of Contents] Overview Industrial application field Conventional technology (Fig. 12) Problems to be solved by the invention Means for solving the problem (Figs. 1-3) FIG. 3) Embodiments (FIGS. 1 to 11) Effects of the Invention [Summary] The present invention relates to a predictive encoding method for image information, particularly moving image information, and aims at making a block size to be subjected to encoding processing variable. A plurality of types of blocks each having a required size including the image information are determined, and the inter-frame prediction, the motion compensation prediction, and the intra-frame prediction are performed on the blocks of each size, thereby obtaining the blocks of the respective sizes. , And calculate these prediction errors as the first
Of the prediction errors, a prediction error obtained by any of the inter-frame prediction, the motion compensation prediction, and the intra-frame prediction is selected from the prediction errors. Is evaluated by the second evaluation means in consideration of the result of the discrete cosine transform, so that an optimal block size is determined from the plurality of types of block sizes.

[Industrial applications]

The present invention relates to a predictive coding method for image information, particularly for moving image information.

For example, regarding an image signal in a videophone call or a videoconference, corresponding images between the two frames generally have similar values, and thus information between such frames has a strong correlation. For this reason, at this time, it is necessary to make the prediction error smaller for the moving image and perform the band-band compression encoding of the image signal more efficiently.

[Conventional technology]

FIG. 12 shows a conventional predictive coding scheme with motion compensation, in which 1 is a quantizer, 2 is a frame memory, 3 is a variable delay, 4 is a motion detector, 5 Is a discrete cosine transformer (DCT) as an orthogonal transformer, and 6 is a discrete inverse cosine transformer (DCT ^-1 ). In this scheme,
The motion detector 4 takes the absolute value of the difference in pixel units between the input screen block and the block at the same position on the reference screen stored in the frame memory 2 and its surroundings, and accumulates one block. The block having the smallest value is predicted as a block before moving, and the predicted block is delayed by the amount of movement in the variable delay unit 3, and then a difference from the next input screen block is obtained.

The difference signal obtained in this manner is decomposed into frequency components by performing a cosine transform in DCT5, quantization of the coefficients is performed in the quantizer 1 in the frequency domain, and quantization transmission is performed in order from the coefficients of the low frequency components. When all of the remaining quantized DCT coefficients become "0", "invalid" information is sent and block transmission is terminated, thereby compressing high-frequency components.

That is, the blocks subjected to DCT and quantized have a two-dimensional array as shown in FIG. 13 (1) in order to increase the coding efficiency because the probability that "0" is generated increases as the frequency component increases. As shown in FIG.
The dimension is expanded, and scanning is performed in order from low frequency components to high frequency components. Scan, and if the remaining frequency components are all “0”, instead of transmitting “0”, transmit the subsequent components as “invalid” with a codeword such as EOB (end of bound) Are terminated to improve the coding efficiency.

 On the receiving side, the reverse operation is performed.

[Problems to be solved by the invention]

In the conventional motion compensation method, an I frame is divided into a plurality of blocks, and motion compensation is performed for each block. there were.

On the other hand, the larger the block size of motion compensation, the smaller the amount of information for transmitting motion compensation motion information, and the greater the difference (error) between the screen obtained by motion compensation and the original image.

Therefore, conventionally, as described above, motion compensation is performed using the same block size in any part of the screen. For example, when the block size is uniformly large, the error that occurs in the flat part of the screen is small, When the block size is large, the error is large, and conversely, when the block size is uniformly small, the error that occurs in the portion where the screen changes rapidly is small, but in a flat portion of the screen, the error is almost the same as when the block size is large. Nevertheless, there is a problem that the amount of extra information increases.

An object of the present invention is to solve such a trade-off problem, and an object of the present invention is to provide a motion-compensated motion-compensated predictive coding method capable of changing a block size for performing a coding process. And

[Means for solving the problem]

FIGS. 1 to 3 show a predictive coding method in a predictive coding method of a moving image according to the present invention for achieving the above-mentioned object. In FIG. In order to determine, for each of a plurality of types of block sizes including the input image information, each prediction method of inter-frame prediction, motion compensation prediction, and intra-frame prediction is executed, so that each prediction error for each size block is determined. , And a first evaluation unit 17 for selecting a minimum prediction error and a prediction method corresponding to the minimum prediction error from the prediction errors.

FIG. 2 shows a second evaluator 22 that determines an optimum block size from the plurality of types of blocks based on the prediction error obtained by the evaluator 17 of FIG. The evaluation means 22 is the larger block size invalidated as a result of DCT sequentially performed on two adjacent large and small blocks of the entire block size, or the larger one from the first evaluation means 17. When the absolute value of the difference between the prediction error between the block size and the small block size per unit pixel is smaller than a predetermined threshold value, the larger block size is determined and transmitted.

In this case, in the present invention, the second evaluating means 22 compares the sum of the DCT information amount of both large and small blocks with the information amount of the prediction method from the first evaluating means 17 as shown in FIG. If the sum of the information amounts of the larger block size is smaller, the larger block size can be determined and transmitted.

[Action]

In FIG. 2, the second evaluation means 22 determines the block size when the DCT results of the two large and small blocks to be compared are invalid for a large block size, and determines the block size even when the DCT results are not invalid. If the absolute value of the difference between the prediction errors of both large and small block sizes per unit is smaller than a predetermined threshold value, the larger block size is determined. A large block is selected for a portion where the change is flat, and a small block is selected for a portion where the screen changes greatly.

Further, as shown in FIG. 3, the second evaluating means compares the sum of the DCT information amount of both the large and small blocks and the information amount of the prediction method from the first evaluating means 17 with each other. If the relationship of the sum of the information amounts of the block sizes <the sum of the information amounts of the smaller block sizes is satisfied, it is possible to select the larger block size as a flat portion as in the case where the result of the DCT becomes invalid. it can.

In this way, the block size is adaptively switched according to the partial state of the screen.

〔Example〕

First, an embodiment of the present invention will be described with reference to FIG. 1 and FIG.

First, referring to FIG. 1, in FIG.
Is a variable delay means (VDLY), and 4 is a motion detection / compensation means (M
C), 13 is an average value calculating means, 14 is an inter-frame prediction means, 1
5 is a motion compensation prediction means, 16 is an intra-frame prediction means, and 17 is a first evaluation means.

Here, the motion compensating unit 4 receives the input image information and the reproduced image information from the frame memory (not shown) and outputs the motion vector information to the variable delay unit 3.

The variable delay unit 3 delays the reproduced image according to the motion vector information and outputs the delayed image to the motion compensation prediction unit 15.

The average value calculation means 13 calculates the average value of the input image information and outputs the calculated average value to the intra-frame prediction means 16.

The inter-frame prediction unit 14 receives the reproduced image information and the input image information, and outputs a prediction error due to the inter-frame prediction to the first evaluation unit 17, and the motion compensation prediction unit 15 And the prediction error due to the motion compensation prediction is received by the first evaluator 17.
The intra-frame prediction unit 16 receives the input image information and the output from the average value calculation unit 13, and outputs a prediction error due to intra-frame prediction to the first evaluation unit 17.

The first evaluator 17 receives the prediction error from the inter-frame predictor 14, the prediction error from the motion compensation predictor 15, and the prediction error from the intra-frame predictor 16. By evaluating the error, an optimal prediction method is selected, information about the optimal prediction method is output from the output line 17a, and a prediction error of the selected prediction method is output from the output line 17b.

In FIG. 2, reference numeral 20 denotes a line for outputting prediction error information for a block having a relatively large size from the first evaluation means 17, and reference numeral 21 denotes prediction error average value calculation means. The calculating means 21 is the first evaluating means 17
Calculates the average value (prediction error when converted to a large block) of prediction errors for n blocks of relatively small size (equivalent to the size of a large block). The average value information of the prediction error is input to the second evaluation means 22 through the output line 21a.

The second evaluation means 22 shown in FIG.
The T result (valid / invalid result when quantization is performed by performing DCT) is input from line 31, and DCT of a small block is also input.
The result is entered from line 32. The DCT result is D
It is "invalid" when the quantized coefficient after CT becomes "0", and "valid" otherwise.

In response to such input information, the second evaluation means 22
The optimum block size is determined, and the information is output from the output line 22a.

 Hereinafter, this embodiment will be described in more detail.

First, for example, the block size is changed as shown in FIG.
(D), 32 × 32 (pixels), 16 × 16 as shown in FIG.
(Pixels), 8 × 8 (pixels), 4 × 4 (pixels), and the like. For each block size, the inter-frame prediction unit 14, the motion compensation prediction unit 15, and the In the intra-frame prediction means 16,
Each of the inter-frame prediction, the motion compensation prediction, and the intra-frame prediction is performed, and the prediction error is input to the first evaluation unit 17 having a predetermined evaluation function, and as a result of the evaluation, which prediction method is selected. Is output from an output line 17a of the first evaluation means 17, and an error of the selected prediction method is output from another output line 17b.

As described above, there are four types of block sizes, and 32 × 32 (pixels), 16 × 16, 8 × 8, 4 × 4 [fourth
(A) to (d), FIG. 5], the maximum block size is 32 × 32, and the same processing is repeated at this size. Further, since these block sizes are in a multiple relationship, relative conversion within one frame is possible.

In FIG. 1, the intra-frame prediction is a method of calculating its own average value (of the block to be coded). However, the present invention is not limited to this. A method using the average value of the processed blocks (for example, the left and upper blocks) may be used. When using the average value of the own block in intra-frame prediction, information indicating which prediction method is selected and the prediction error thereof, and the average value of the own block must be output. Must.

Next, an algorithm for selecting an optimal prediction method for each block size will be described with reference to the flowchart shown in FIG.

First, in step a1, prediction errors D _KAN , D _MC , and D _NAI of inter-frame prediction, motion compensation prediction, and intra-frame prediction are calculated and input, and a threshold value TH is determined in advance.

Then, in the next step a2, it is determined whether the prediction error D _{KAN of the} inter-frame prediction is a minimum value,
If the prediction error D _{KAN of the} inter-frame prediction is the minimum value,
In step a2, a YES route is taken. In step a4, the inter-frame prediction method is adopted, and the prediction error _DKAN is output from the output lines 17a and 17b of the first evaluation means 17, respectively.

If the prediction error D _{KAN of the} inter-frame prediction is not the minimum value, in step a3, the difference (absolute value) between the prediction error D _{KAN of the} inter-frame prediction and the prediction error D _MC of the motion compensation prediction is set to a threshold. Although it is determined whether within TH, if YES at step a3, at step a4, and adopting a prediction method between frames, the prediction error D _KAN each line
Output from 17a and 17b.

On the other hand, if NO in step a3, in step a5,
Comparing the prediction error D _NAI of the prediction error D _MC intra-frame prediction of the motion compensation prediction, if towards the prediction error D _MC motion compensated prediction is smaller, at step a7, and adopting a motion compensation prediction system, the The prediction error D _{MC is applied} to lines 17a and 17b, respectively.
Output from

Otherwise, at step a6, if the prediction error D _MC motion compensated prediction, the difference (absolute value) between the prediction error D _NAI prediction in a frame within the threshold TH, at step a7, the motion compensated prediction method and adopting in, and outputs the prediction error D _MC from each line 17a and 17b.

If NO in step a6, in step a8, the intra-frame prediction method is adopted, and the prediction error _DNAI is output from lines 17a and 17b, respectively.

In this way, the minimum prediction error and the prediction method corresponding thereto are selected. In this case, if the motion is small, the inter-frame prediction is selected, and as the motion increases, the motion compensation prediction and the intra-frame prediction are sequentially performed. Is selected.

Next, how to determine the block size in the second evaluation means 22 will be described.

Such a block size determination operation is performed first in 32 × 32 and 16
X16, then 16x16 and 8x8, 8x8 and 4x
Perform in the order of 4 in the relation of large block to small block,
The same processing is repeated.

That is, when the prediction method of each block size and the prediction error are determined by the first evaluation means 17 as described above, the block size is determined next. In this case, the prediction error of the large block is determined. And the average value of the prediction error of the next smaller small block (this is the size of the large block, and n small blocks are assumed).
(The evaluation function of the second evaluation means 22 may be the same as that of the first evaluation means 17), and the DCT results of the large block and the small block are input. The block size selected by the evaluation function of the second evaluation means 22 and its prediction error are output to an output line.
Output from 22a and 22b.

Next, the algorithm of the second evaluation means 22 for determining the block size will be described with reference to the flowchart shown in FIG.

First, in step b1, if the DCT result for the prediction error obtained in the large block of 32 × 32 is an invalid block (a block in which the quantized coefficient of the quantizer is “0”), the block size is set to 32 × It is determined as 32 (step b2).

Otherwise, if the DCT result for the next largest 16 × 16 block becomes an invalid block, the block size is determined to be 16 × 16 (steps b3 and b4). This step is performed for each of the four 16 × 16 blocks in the 32 × 32 block.

Otherwise, in step b5, the average value of prediction errors of 16 × 16 blocks (prediction error per pixel)
When the difference between the average value of the prediction error blocks of smaller 8 × 8 (4 pieces) of the block size if the threshold TH ₁ or less 1
Determined as 6 × 16 (step b4).

Otherwise, if the DCT result for the 8 × 8 block becomes an invalid block in step b6, the block size is determined to be 8 × 8 (step b7). This step is also performed for each 8 × 8 block in the 16 × 16 block because there are four 8 × 8 blocks.

Otherwise, 8 and the average value of the prediction error blocks of × 8, if the difference between the average value of prediction errors of the 4 × 4 blocks (4) the threshold TH ₂ or less, 8 × block size 8
(Steps b8 and b7), otherwise, the block size is determined as 4 × 4 (step b9).

The block size selected as described above is output from the output line 22a of the second evaluation means 22. In addition, the fact that no comparison between the prediction errors of 32 × 32 and 16 × 16 is provided between the steps b1 and b3 is apparent from experiments, in most cases, the difference between the two exceeds the threshold value. Because it is.

FIG. 8 is a flowchart showing an embodiment of the present invention for determining a block size in the second evaluation means 22 shown in FIG. 3. In this embodiment, a comparison is made between two large and small block sizes. While the DCT result for the large block is invalidated, the large block size is selected (steps c1 and c2).

Otherwise, the information amount of the large block prediction method (motion vector information for the motion compensation method, the method information for the inter-frame prediction method, the average value information for the intra-frame prediction method) and the prediction error Is compared with the sum of the information amount obtained by applying DCT to the sum of the information amount of the small block prediction method and the DCT information amount. If the large block is smaller, the large block is selected (steps c3 and c1). ), If not,
A small block is selected (steps c3 and c4).

These steps initially consist of a large 32x32 block and a 16x
The 16 small blocks are compared, and when a large block is selected, the block size information is sent to the receiving side. Otherwise, the 16 × 16 large block and the 8 × 8 large block are selected.
With the small block of. Thus, the adjacent blocks are compared with each other, and the larger block size or 4
The information of the minimum block size of × 4 will be sent.

In addition, it is necessary to transmit the sum of the information amount of the prediction method of the block size and the information amount of the DCT together with the selected block size.

In this way, an optimal block size is selected.
In this case, if there is little motion, the maximum block size (32
× 32) is selected, and as the motion increases, a smaller block size (16 × 16 → 8 × 8 → 4 × 4) is selected in order.

Since the method of calculating the average value of the prediction errors for n small blocks differs depending on the method of calculating the prediction errors, there is no particular limitation such as having to be as shown in FIG. It suffices if the average value of the prediction error when n small blocks are collected can be calculated.

FIG. 1 shows a calculation method when the prediction error is an absolute value error.

By the way, the flow of this embodiment and the like in FIGS.
Assuming that 32 × 32 is a unit of one large block based on (d) and the block size and the corresponding positional relationship in FIG. 5, the data structure according to this method is a four-branch four-stage tree as shown in FIG. In this state, the optimum path is followed by the evaluation function.

 FIGS. 10 and 11 show examples of the processing results of this embodiment.

FIG. 10 shows an example of the processing result following FIG. 9, FIG. 11 (a) shows the positional relationship at 16 × 16 in FIG. 10, and FIG. 7 is an example of screen division corresponding to.

In this embodiment, the determined prediction method and the size of the block size (and the average value information when the intra-frame prediction method is adopted) are also transmitted to the receiving side.

As described above, in this method, although the information of the block size and the type of the prediction method are transmitted to the receiving side, the block size is increased in the flat portion, the block size is reduced in the portion where the change is rapid, and the local area of the screen is reduced. Therefore, since the prediction method is adaptively switched according to the characteristic, the transmission efficiency as a whole can be improved.

In addition, in addition to the above embodiment, the first and second evaluation means use the number of determinations obtained by multiplying the DCT information amount by the code length of each prediction method and its weighting coefficient instead of the DCT information amount. Can obtain the same result.

〔The invention's effect〕

As described above in detail, according to the present invention, although the information on the block size and the type of the prediction method are transmitted to the receiving side, the flat portion or the change based on the calculation result of the DCT conventionally used is used. Detects intense parts and increases or decreases the block size accordingly.
The optimum block size is selected for both the flat part and the rapidly changing part of the screen, and there is an advantage that the transmission efficiency can be improved as a whole.

[Brief description of the drawings]

FIG. 1 is a diagram illustrating the principle for determining a prediction method in the present invention, FIG. 2 is a diagram illustrating the principle for determining a block size in the present invention, and FIG. 3 is a diagram illustrating the principle for determining a block size in the present invention. FIGS. 4 (a) to (d) and FIG. 5 are diagrams each illustrating a method of dividing a block, FIG. 6 is a flowchart for determining a prediction method in the present invention, FIG. 7 is a flowchart for determining a block size in the present invention, FIG. 8 is a flowchart for determining a block size in the present invention, FIG. 9 is a data structure diagram in an embodiment of the present invention, FIG. FIG. 11 is a schematic diagram for explaining a path defined by a certain evaluation function in the embodiment of the present invention. FIGS. 11 (a) and 11 (b) show processing results in the embodiment of the present invention. FIG. 12 is a block diagram showing a conventionally well-known moving picture predictive coding method, and FIG. 13 is an explanatory diagram of a discrete cosine transform. In the figure, 3 is a variable delay unit, 4 is a motion compensation unit, 14 is an inter-frame prediction unit, 15 is a motion compensation prediction unit, 16 is an intra-frame prediction unit, 17 is a first evaluation unit, and 22 is a second evaluation unit. means. In the drawings, the same reference numerals indicate the same or corresponding parts.

Claims (2)

(57) [Claims] 1. A plurality of types of blocks each having a required size including input image information are determined, and for each block of each size, each prediction method of inter-frame prediction, motion compensation prediction, and intra-frame prediction is executed. By doing
Means (14, 15, 16) for obtaining respective prediction errors for blocks of each size, and first evaluation means for selecting a minimum prediction error and a prediction method corresponding thereto from among these prediction errors (17) and a second estimating means (22) for determining an optimum block size from among the plurality of types of blocks based on the selected prediction error. The second evaluation means (22) invalidates the larger block size as a result of the discrete cosine transform sequentially performed on two adjacent large and small block sizes of the entire block size, or the first evaluation. When the absolute value of the difference per unit pixel of the prediction error between the large block size and the small block size from the means (17) is smaller than a predetermined threshold, the larger block size is determined and transmitted. System that was characterized by the door. 2. The method according to claim 1, wherein said second evaluating means (22) invalidates a larger block size as a result of a discrete cosine transform performed sequentially on two adjacent large and small block sizes among all block sizes. Alternatively, the sum of the information amount of the discrete cosine transform of the large block size and the small block size and the information amount of the prediction method from the first evaluation means (17) are compared with each other, and the sum of the information amounts of the large block size is compared. 2. The method according to claim 1, wherein if the value is smaller, the larger block size is determined and transmitted.