JPH01238389A - Predictive coding system for moving image - Google Patents

Abstract

PURPOSE:To make the optimum block size selectable even to both parts of a picture where picture variation is flat and sharp so as to improve the transmission efficiency by detecting the flat or sharply varying part based on the calculation result of a discrete cosine transformer (DCT) and making a block size larger or smaller in accordance with the flat or sharply varying parts. CONSTITUTION:The 1st evaluating means 17 selects one of an inter-frame prediction, movement compensating prediction, and inter-frame prediction out of predicted errors found by means 14-16 and the predicted error of the selected one. The 2nd evaluating means 22 decides the optimum block size out of plural kinds of blocks by using the predicted error found by the means 17. In other words, the means 22 decides the block size when each DCT result of a large and small blocks to be compared becomes invalid to the larger block and at the same time, decides the larger block size when the absolute value of the difference per unit of each predicted error of both larger and smaller block sizes is smaller than a prescribed threshold value even when each DCT result does not become invalid. When such operations are successively performed on all blocks, a larger block is selected to a part where tbh variation of the picture is flat and a smaller block is selected to a part where the variation is sharp.

Description

[Detailed description of the invention] [Table of contents] Overview Industrial field of application Prior art (Fig. 12) Means for solving the problem to be solved by the invention (Figs. 1 to 3) Function (
(Figures 1 to 3) Example (Figures 1 to 11) Effects of the invention [Summary] Regarding the predictive encoding method for image information, especially moving image information, the block size for encoding processing is made variable. For the purpose of this, we define multiple types of blocks of a required size that contain the image information, and perform interframe prediction, motion compensation prediction, and intraframe prediction on each of these blocks of different sizes. By determining prediction errors for each human-sized block and evaluating these prediction errors using the first evaluation means, the inter-frame prediction, motion compensation prediction, and frame prediction errors are determined from among these prediction errors. By selecting the prediction error obtained by one of the inner predictions, and then evaluating the selected prediction error in the second evaluation means by taking into account the results of the discrete cosine transform, the block sizes of the plurality of types are evaluated. The configuration is configured to determine the optimal block size from among the following.

[Industrial application field]

The present invention relates to a predictive coding method for image information, particularly moving image information.

For example, when it comes to image signals from video calls or video conferences, the corresponding images between two frames generally have similar values, so the information between such frames has a strong correlation. For this reason, it has become necessary to further reduce prediction errors for moving images and perform band ring coding of image signals more efficiently.

[Conventional technology]

FIG. 12 shows a motion-compensated predictive coding system that has been used in the past. In the figure, 1 is a quantizer, 2 is a frame memory, 3 is a variable delay device, 4 is a motion detector, and 5 is a discrete cosine transformer (DCT) as an orthogonal transformer, and 6 is a discrete inverse cosine transformer (DC"l"). In this method, a motion detector 4 takes the absolute value of the difference in pixel units between an input screen block and blocks at the same position on the reference screen stored in the frame memory 2 and its surroundings, and accumulates the difference for l blocks. The block with the smallest cumulative value is predicted to be the block before movement, and the predicted block is delayed by the amount of movement in the variable delay unit 3, and then the difference from the next input screen block is calculated.

The difference signal obtained in this way is decomposed into frequency components by cosine transformation in the DCT 5, quantization of the coefficients is performed in the frequency domain by a quantizer l, and the coefficients are quantized and transmitted in order from the low frequency component coefficients, When all remaining quantized DCT coefficients become "O", "invalid" information is sent and block transmission is terminated, thereby compressing high frequency components.

On the receiving side, exactly the opposite operation is performed.

[Problem that the invention sought to solve]

In conventional motion compensation methods, one frame is divided into multiple blocks and motion compensation is applied to each block, but the block size to which motion compensation is applied is the same whether it is a flat part of the screen or a part with rapid changes. there were.

On the other hand, the larger the block size for motion compensation, the smaller the amount of information to transmit motion information for motion compensation, and the greater the difference (error) between the screen obtained by motion compensation and the original image.

Therefore, in the past, as mentioned above, motion compensation was performed using the same block size for any part of the screen. For example, when the block size is uniformly large, there is less error in flat areas of the screen, but when the screen changes On the other hand, when the block size is uniformly small, the error is small in areas where the screen changes rapidly, but in flat areas of the screen, the error is almost the same as when the block size is large. However, there was a problem in that the amount of redundant information increased.

The present invention is an attempt to solve such trade-off problems, and aims to provide a motion-compensated predictive coding method for moving images that allows variable block size for coding processing. It is said that

[Means to solve the problem]

1 to 3 show a predictive coding method in a moving picture predictive coding method according to the present invention to achieve the above object. 13. Means for determining prediction errors for blocks of each size by performing interframe prediction, motion compensation prediction, and intraframe prediction for a plurality of types of block sizes including input image information in order to determine the size; 15.16, and a first evaluation means 1 for selecting one of the interframe prediction, motion compensation prediction, and intraframe prediction and its prediction error from among these prediction errors.
7.

FIG. 2 shows a second evaluation means 22 that determines the optimal block size from among the plurality of types of blocks based on the prediction error obtained by the evaluation means 17 of FIG.
This second evaluation means 22 is.

Either the larger block size is invalid as a result of DCT sequentially performed on two adjacent large and small blocks among all block sizes, or the first evaluation means 17
If the absolute value of the difference per unit pixel between the prediction error between the larger block size and the smaller block size is smaller than a predetermined threshold, the larger block size is determined and transmitted.

In this case, in the present invention, the second evaluation means 22 compares the sum of the DCT information amount of both large and small blocks and the information amount of the prediction method from the first evaluation means 17, as shown in FIG. Then, if the sum of the information amounts of the larger block size is smaller, the larger block size can be determined and transmitted.

[Function] In FIG. 2, the second evaluation means 22 determines the block size when the DCT results of two large and small blocks to be compared become invalid for a large block size, and also determines the block size when the DCT results of two large and small blocks are invalid. Even if there is no block size, if the absolute value of the difference per unit of each prediction error for both large and small block sizes is smaller than a predetermined threshold, the larger block size is determined, so this should be done sequentially for all blocks. As a result, large blocks are selected for portions of the screen where changes are flat, and small blocks are selected for portions of the screen where changes are rapid.

Further, as shown in FIG. 3, the second evaluation means further compares the sum of the DCT information amount of both large and small blocks and the information amount of the prediction method from the first evaluation means 17, and selects the larger one. If the relationship is that the sum of the information amount of the block size is equal to the sum of the information amount of the smaller block size, it is possible to select the larger block size as a flat part, as in the case where the DCT result is invalid. can.

In this way, the block size is adaptively switched depending on the partial state of the screen.

〔Example〕

First, one embodiment of the present invention will be described with reference to FIGS. 1 and 2.

First, to explain Fig. 1, in this Fig. 1, 3 is a variable delay means (VDLY), 4 is a motion detection/compensation means (
MC), 13 is an average value calculation means, 14 is an interframe prediction means, 15 is a motion compensation prediction means, 16 is an intraframe prediction means, and 17 is a first evaluation means.

Here, the motion compensation means 4 receives input image information and reproduced image information from a frame memory (not shown), and outputs motion vector information to the variable delay means 3.

The variable delay means 3 delays the reproduced image according to the motion vector information and outputs the delayed image to the motion compensation prediction means 15.

The average value calculation means 13 calculates the average value of the input image information and outputs it to the intra-frame prediction means 16.

The interframe prediction means 14 receives the reproduced image information and the input image information, and outputs the prediction error resulting from the interframe prediction to the first evaluation means 17, and the motion compensation prediction means 15 receives the input image information and the variable delay means 12 The intra-frame prediction means 16 receives the input image information and the output from the average value calculation means 13, and outputs the prediction error by the motion compensation prediction to the first evaluation means 17. Then, the prediction error resulting from the intra-frame prediction is output to the first evaluation means 17.

The first evaluation means 17 receives the prediction error from the interframe prediction means 14, the prediction error from the motion compensated prediction means 15, and the prediction error from the intraframe prediction means 16, and performs the above prediction using a predetermined evaluation function. By evaluating the error, the optimum prediction method is selected and information regarding the optimum prediction method is outputted from the output line 17a, and the prediction error of the selected prediction method is outputted from the output line 17b.

Further, in FIG. 2, 20 is a line for outputting prediction error information for relatively large blocks from the first evaluation means 17, and 21 is a prediction error average value calculation means, which outputs the prediction error average value. The calculation means 21 calculates n blocks of relatively small size from the first evaluation means 17 (
It calculates the average value of prediction errors (prediction error when converted to large blocks) for n small blocks (corresponding to the size of a large block), and outputs average value information of the prediction errors through its output line 21a. is the second evaluation means 22
is input to.

The second evaluation means 22 in FIG.
CT results (effective when performing OCT and quantization)
An invalid result) is input from line 31, and a DCT result of the small block is also input from line 32. still,
The DCT result is "invalid" when the quantized coefficient after OCT is "0", and otherwise "valid".

Upon receiving such input information, the second evaluation means 22
The optimal block size is determined and the information is output from the output line 22a.

This example will be explained in more detail below.

First, the block size is determined as shown in FIGS. 4(a) to (d), for example.
As shown in Figure 5 <32X32 (pixels), 16X16
(pixels), 8x8 (pixels), and 4x4 (pixels), and for each block size, interframe prediction means 14, motion compensation prediction means 15, and intraframe prediction means are used for each block. In step 16, interframe prediction, motion compensation prediction, and intraframe prediction are performed, and the prediction errors are input to the first evaluation means 17 having a certain evaluation function.As a result of the evaluation, which prediction The selection of the prediction method is output from the output line 17a of the first evaluation means 17, and the error of the selected prediction method is output from the other output line 17b.

As mentioned above, there are four types of block sizes: 32 x 32 (pixels), 16 x 16.8 x 8.4 x
4 [Fig. 4 (a) to (d), Fig. 5], but the block size is 32 x 32 as the maximum block size,
The same process is repeated with this size. Furthermore, since these block sizes are in a multiple relationship, relative conversion within one frame is possible.

In addition, intra-frame prediction is a method that calculates its own average value (of the block to be encoded) in Fig. 1, but it is not limited to that, and the prediction is based on the image to be encoded (input image). A method using the average value of the processed blocks (for example, the left and upper blocks) may be used. When using the average value of your own block in intra-frame prediction, in addition to outputting information about which prediction method was selected and its prediction error, you must also output the average value of your own block. Must be.

Next, an algorithm for selecting an optimal prediction method for each block size will be explained using the flowchart shown in FIG.

First, in step a1, the prediction error D of interframe prediction, motion compensation prediction, and intraframe prediction is
In addition to calculating and inputting DH□, the threshold value TH
Decide.

Then, in the next step a2, it is determined whether the prediction error D XAN of the inter-frame prediction is the minimum value. If the prediction error D KAII of the inter-frame prediction is the minimum value, the YES route is taken in step a2. Take the
In step a4, the interframe prediction method is adopted and its prediction error D□ is outputted from the output lines 17a and 17b of the first evaluation means 11, respectively.

Moreover, if the prediction error of inter-frame prediction is not the minimum value, 8 is not the minimum value, in step a3, the prediction error of inter-frame prediction DIIAM and the prediction error of motion-compensated prediction DK
It is determined whether the difference (absolute value) with c is within the threshold TH. If YES in step a3, step a4
Then, the inter-frame prediction method is adopted, and its prediction error DKAN is output from lines 17a and 17b, respectively.

On the other hand, if NO in step a3, the prediction error Dxc of the motion compensation prediction is compared with the prediction error I), Ia+ of the intraframe prediction in step a5, and if the prediction error DMC of the motion compensation prediction is smaller, step a7 Then, the motion compensated prediction method is adopted and the prediction error DNC thereof is outputted from lines 17a and 17b, respectively.

Otherwise, in step a6, if the difference (absolute value) between the prediction error DNC of the motion compensation prediction and the prediction error □ of the intra-frame prediction is within the threshold TH, in step a7,
Adopting a motion compensation prediction method and its prediction error DM
C from lines 17a and 17b, respectively.

Further, if NO in step a6, in step a8, the intra-frame prediction method is adopted and its prediction error DM is determined.
□ are output from lines 17a and 17b, respectively.

In this way, the optimal prediction method is selected.

In this case, if there is little motion, interframe prediction is chosen,
As the amount of motion increases, motion compensation prediction and intraframe prediction are selected in this order.

Next, a method of determining the block size in the second evaluation means 22 will be explained.

This block size determination operation is first performed for 32x32 and 16x16, and then for 16x16 and 8x8.8.
The same process is repeated in the order of x8 and 4x4 in the large block versus small block relationship.

That is, once the prediction method and prediction error of each block size are determined by the first evaluation means 17 as described above, the block size is determined next, but in this case, the prediction error of the large block and the average value of the prediction error of the next smallest small block (this is the size of the large block, which corresponds to n small blocks) from the first evaluation means 17 to the second evaluation means 22 (this second evaluation means 22). The evaluation function possessed by the second evaluation means 22 may be the same as that possessed by the first evaluation means 17).
By inputting the results, the block size selected by the evaluation function of this second evaluation means 22 and its prediction error are outputted from output lines 22a and 22b.

Next, the algorithm of the second evaluation means 22 for determining the block size will be explained using the flowchart shown in FIG.

First, in step bl, if the DCT result for the prediction error obtained for the large block 32x32 is an invalid block (when the quantization coefficient of the quantizer becomes "0"), the block size is changed to 32x32. 32 (step b2).

Otherwise, if the DCT result for the next largest size 16x16 block is an invalid block, the block size is determined as 16x16 (step b3,
b4), this step
There are four 6x16 blocks, so do this for each one.

Otherwise, in step b5, the difference between the average prediction error value (prediction error per pixel) of the 16×16 block and the average value of the prediction error of the smaller 8×8 (4 blocks) is determined. 2 threshold TH, the block size is determined to be 16×16 (step b4).

Otherwise, in step b6, if the DCT result for the 8×8 block is an invalid block, the block size is determined as 8×8 (step b7);
Since there are four 8x8 staves in the 16x16 block, this step is performed for each one.

Otherwise, if the difference between the average value of the prediction errors of the 8×8 block and the average value of the prediction errors of the 4×4 blocks (4 blocks) is less than or equal to the threshold value TH, then the block size is changed to 8× 8
If so, step 1)) (, bT), otherwise the block size is determined as 4×4 (step b9).

Note that the block size selected as described above is output from the output line 22a of the second evaluation means 22. Also,
The reason why we do not specifically compare the prediction errors of 32x32 and 16x16 between steps b1 and b3 is because experiments have shown that in most cases the difference between the two exceeds the threshold. Because there is.

FIG. 8 is a flowchart showing an embodiment of the present invention for determining the block size in the second evaluation means 22 shown in FIG. 3. In this embodiment, two block sizes, large and small, are compared. First, the DCT for the large block is determined.
If the result is invalid, select a large block size (steps C1 and C2).

Otherwise, the information amount of the large block prediction method (motion vector information in the case of motion compensation method, method information in the case of interframe prediction method, average value information in the case of intraframe or measurement method) and Compare the sum of the information amount when multiplying the error &'Th D CT with the sum of the information amount of the small block P measurement method and the DCT information amount, and if the large block is smaller, select the large block. 76, 1″ Ruga (Step C3, C
I), otherwise select a small block (steps C3, C4).

These steps first start with a 32x32 large block and 1
When a large block is selected, the block size information is sent to the receiving side, but if not, the current case is a 16x16 large block and an 8x8 small block. Compare. Adjacent blocks are thus compared, and information about the larger block size or the smallest 4×4 block size is sent.

Note that it is necessary to simultaneously transmit the selected block size and the sum of the amount of information of the prediction method of the selected block size and the amount of information of the DCT.

In this way, the optimal block size is chosen. In this case, if there is little movement, the maximum block size (32
X32) is selected, and as the amount of movement increases, smaller block sizes (16×16 → 8×8 → 4×4) are selected in order.

Note that the method for calculating the average value of the prediction errors for n small blocks differs depending on the calculation method of the prediction errors.
There is no restriction that it has to be as shown in the diagram, but it is sufficient that the average value of the prediction error when n small blocks of the size of the large block are gathered can be calculated.

In addition, in this Figure 1, the prediction error is an absolute value error,
This is the calculation method when

By the way, the flow etc. of this example and FIGS. 4(a) to (
d) and from the block size and corresponding positional relationship in Figure 5, if 32x32 is taken as a unit of one large block, the data structure according to this method GJ7 will be in a tree shape with 4 branches and 4 stages as shown in Figure 9, Among these, the optimal route is traced using the evaluation function.

An example of the processing results of this embodiment is shown in FIG. 10.11.

10 shows an example of the processing result in accordance with FIG. 9, FIG. 11(a) shows the positional relationship at 16×16 in FIG. This is an example of screen division corresponding to .

In this embodiment, the determined prediction method and block size (as well as average value information when the intra-frame measurement method is adopted) are also transmitted to the receiving side.

In this way, in this method, information on the block size and the type of prediction method are transmitted to the receiving side, but the block size is increased in flat areas, decreased in areas with rapid changes, and the block size is Since the prediction method is adaptively switched according to the characteristics of the network, overall transmission efficiency can be improved.

In addition to the above embodiment, in the first and second evaluation means, the number of judgments obtained by multiplying the DCT information amount by the code length of each prediction method and its weighting coefficient is used instead of the DCT information amount. gives similar results.

[Effects of the Invention] As detailed above, according to the present invention, although information on the block size and the type of prediction method are transmitted to the receiving side, information on the block size and the type of prediction method are transmitted based on the calculation results of the conventionally used DCT. It detects flat areas or areas with rapid changes and increases or decreases the block size accordingly.
The optimum block size is selected for both flat areas and rapidly changing areas of the screen, which has the advantage of improving overall transmission efficiency.

[Brief explanation of the drawing]

Figure 1 is a diagram of the principle of determining the prediction method in the present invention. Figure 2 is a diagram of the principle of determining block size in the present invention. Figure 3 is a diagram of the principle of determining block size in the present invention. 4 (a) to (d) and 5 are diagrams explaining the block division method, and FIG. 6 is a flowchart diagram for determining the prediction method in the present invention. , FIG. 7 is a flowchart for determining the block size in the present invention, FIG. 8 is a flowchart for determining the block size in the present invention, FIG. 9 is a data structure diagram in an embodiment of the present invention, 10
The figure is a schematic diagram for explaining a route determined by a certain evaluation function in an embodiment of the present invention.
b) is a schematic diagram showing processing results in an embodiment of the present invention, and FIG. 12 is a block diagram showing a conventionally well-known predictive encoding method for moving images. In the figure, 3 is a variable delay means, 4 is a motion compensation means, 14 is an interframe prediction means, 15 is a motion compensation prediction means, 16 is an intraframe prediction means, 17 is a first evaluation means, and 22 is a second evaluation means. means. In the figures, the same reference numerals indicate the same or corresponding parts. 7-σ' 74--1-Frame darkness prediction means 16--Frame prediction) column means In the present invention, the principle diagram for determining the prediction method/law method is shown. α) (b) Cc) (
d) Without explaining how to separate the blocks in Figure 4, Figure 5, Figure 5, in the main explanation, 7, Flowchart for determining the preliminary regulation method, Figure 6, Figure 7, Figure 9, e4 measurement '5% i' prediction. amount of information for rules
4x4(pet)C: redundant block e: tLt'n inclined flock (α)
(b) σ for $ invention
A schematic diagram of the book size showing the results of the process. Figure 11. Conventional diagram. Figure 12.

Claims (2)

[Claims] (1) By defining multiple types of blocks of the required size that contain input image information, and performing interframe prediction, motion compensation prediction, and intraframe prediction on each of these blocks, each size of block is determined. means (14, 15,
16), and from among these prediction errors, one of the interframe prediction, motion compensation prediction, and intraframe prediction,
a first evaluation means (17) for selecting the prediction error; and a second evaluation means (17) for determining the optimal block size from among the plurality of types of blocks based on the selected prediction error.
22), in which the second evaluation means (22) performs discrete cosine transformation sequentially on two adjacent large and small block sizes among all block sizes; When the larger block size that has become invalid or the absolute value of the difference per unit pixel between the prediction error between the larger block size and the smaller block size from the first evaluation means (17) is smaller than a predetermined threshold value. A method characterized by determining the larger block size and transmitting it. (2) The second evaluation means (22) evaluates the larger block size that has become invalid as a result of the discrete cosine transform sequentially performed on two adjacent large and small block sizes among all block sizes, or The sum of the discrete cosine transform information amount of the block size and the smaller block size, and the information amount of the prediction method from the first evaluation means (17) are compared with each other, and the sum of the information amount of the larger block size is 2. The method according to claim 1, wherein if the block size is small, the larger block size is determined and transmitted.