JPH0289481A - Moving picture coding system - Google Patents

Abstract

PURPOSE:To execute prediction of movement, deformation or background or the like suitable for block location while utilizing prediction in timewise direction by splitting a pattern into coded blocks and storing data in the timewise direction for each block. CONSTITUTION:An inputted picture is segmented into a unit block for coding by a block processing circuit 101 and the result is fed to a significant block discrimination circuit 102, the inputted block address is led to an index register 103. Then the signal of the circuit 102 and the input block are compared and whether or not the input block is significant is discriminated. When the input block is significant, the input signal is compared with an output of a code book 107 by a selection circuit 108, distortion is evaluated and the count representing distortion and its order in the index is fed to a comparison/control circuit 109. An output difference signal is subject to discrete cosine variable processing by a discrete cosine variable DCT circuit 113 independently of the level of the distortion in comparison with a threshold value and the result is fed to a multiplex circuit 116. The circuit 116 outputs a normalized coefficient or the like of significant information of a counter value dynamic area as a multiplexed signal.

Description

DETAILED DESCRIPTION OF THE INVENTION [Object of the Invention] (Field of Industrial Application) The present invention relates to a video encoding method used for transmitting video images in video conferences, video telephones, and the like.

(Prior art) In encoding image signals when transmitting moving images in video conferences, etc., motion-compensated inter-frame predictive encoding is used for moving areas, and new images are generated depending on the movement of people, etc. Background predictive coding is used for each uncovered background, contributing to improving the compression rate of the amount of transmitted information. Recently, a new method of deformation prediction has been proposed for deformed portions of moving regions.

Among these, motion compensation performs prediction by subjecting decoded information one frame before to parallel movement.

Good predictions are made for motion regions consisting only of translation. However, in addition to the moving area due to parallel movement, the area recognized as a moving area by the encoding device includes the above-mentioned uncovered background, moving area due to deformation (
(turning the head, opening and closing the eyes, mouth, etc.)

In background prediction, in order to predict the former region, only the background is stored in a background memory, and prediction is made based on the contents. Further, in deformation compensation, an area that is not sufficiently compensated by motion compensation is considered to be a deformed area, and is stored in a memory that stores deformation patterns, and predictions are made based on the contents. It can be said that these two methods utilize the correlation with a frame that is one frame or more earlier.

In other words, motion-compensated inter-frame prediction, which has become established as a conventional prediction method, does not use information from one frame or more earlier, and therefore cannot use temporal inter-pixel correlation.
It cannot deal with uncovered background and deformation, although background prediction and deformation compensation have been devised to compensate for this.

These conventional methods require hardware compatible with each prediction method, and have the disadvantage that the prediction is adaptive and divided into several stages, which tends to make the device complex. In addition, in deformation compensation, all deformed patterns are treated the same, and each deformed pattern must be matched by motion-compensated parallel movement, which has the disadvantage of an extremely large amount of calculation. .

(Problem M to be solved by the invention) As mentioned above, hardware is required to compensate for the drawbacks of conventional motion compensated interframe prediction, such as hardware for background prediction and deformation compensation. However, there is a problem in that it also causes an increase in the amount of calculations.

Therefore, the present invention has been made to solve the above-mentioned problems, and aims to automatically perform predictions that integrate three types of predictions in accordance with block positions. This provides an image encoding method.

[Structure of the invention]

(Means for Solving the Problems) In the present invention, a screen is divided into blocks to be encoded, and data is accumulated in the time direction for each block, thereby achieving movement and deformation adapted to the block position.

It is possible to predict the background, etc. by using correlation in the time direction. Furthermore, data storage is divided into a codebook common to each block and an index indicating the codeword of the codebook, and only the index is held for each block, thereby reducing the burden on storage capacity.

(Function) Scenes that appear in videophone calls, video conferences, etc. are thought to involve little movement of the positions of people, etc., and movements such as parallel translation and transformation are often repeated. This means that
By observing, classifying, and accumulating long-term appearing patterns for the same block position, a predicted pattern containing all motion compensation elements, background memory elements, and deformation compensation elements will be created depending on the block position. It is thought that it accumulates.

In the present invention, such a pattern is prepared according to the position of each block, so it is not possible to search for the mouth pattern when predicting the eye pattern, as is done in deformation compensation. becomes less likely to be carried out.

Furthermore, in conventional motion compensation and deformation compensation, codes of equal length are often assigned to each compensation pattern.

In the present invention, the contents of the index register are rearranged each time they are used, and are controlled so that the first register is always most likely to be used. By encoding this with a variable length code to which a short code word such as the beginning is assigned,
This means that the selected pattern can be sent with a small amount of information.

(Example) Hereinafter, one embodiment of the present invention will be described with reference to the drawings.

FIG. 1 is a block diagram showing a configuration example (transmission side) of a moving image transmission device according to the present invention.

In FIG. 1, the input image is first processed by the blocking circuit 1.
At 01, it is cut out into unit blocks to be encoded,
It is sent to the significant block determination circuit 102. On the other hand, the block address corresponding to the input block is input to the index register 103. The block address and the address from the counter 104 representing the order of the index in the index register are input to the index register.

At the time of significant block determination, the counter 104 is set to "1", and the switching circuit 106 connects the output of the index register to be input to the codebook 107. As a result, the pattern selected in the previous frame at the same block position is input from the codebook 107 to the significant block determination circuit 102, and by comparing this with the input block, it is determined whether the input block is a significant block. Ru. If the input block is a significant block, a signal indicating significance is sent to counter 104 and block address encoding circuit 117, and encoding in the index register is started. The input block determined to be a significant block is input to the selection circuit 108, compared with the output from the codebook 107, distortion is evaluated, and the input block is sent to the counter 104 and the index register 1.
The code word with the least distortion among the code words indicated by 03 is output to the block memory 110, and its distortion and the counter value at that time (representing the order of the index in the register) are output to the comparison/control circuit 109. Comparison/control circuit 109 compares the input distortion with a set threshold Th. If the strain is less than Th, the comparison/
The control circuit 109 outputs a counter value to the variable length code circuit 111 and the index register 103, and a signal instructing the difference circuit 112 to output a difference.The control circuit 109 also outputs a signal to the multiplexing circuit 116 from the selection circuit 108. Outputs a signal instructing not to multiplex the codebook index. The index register 103 to which the counter value has been manually input arranges and updates the order of the indexes in the register according to predetermined rules. As a predetermined sorting rule, there is a rule called "Moueto Front" which moves the used index to the top and shifts the others one by one to the rear. Further, the variable length encoding circuit 111 that receives the counter value outputs a code corresponding to the input counter value to the multiplexing circuit 116.

On the other hand, if the distortion is larger than Th, the comparison/control times j%1
09 sends to the variable length encoding circuit 111 a signal indicating that a sufficient S/N cannot be obtained by encoding with the code word indicated by the index in the index register. The variable length encoding circuit Ut, which has received the above signal, outputs to the multiplexing circuit 116 a code indicating that the index of the codebook 107 will be transmitted subsequently. Comparison/control circuit 109 also. A start signal is sent to the counter 105 to move the counter 105, a switching signal is sent to the switching circuit 106 to connect the counter 105 and the code book 107, and the food book 107 is completely searched. The selection circuit 108 again evaluates the distortion between the input block and the code word, outputs the code word with the minimum distortion to the block memory 110, and outputs the index in that case to the comparison/control circuit 18I109. The control circuit 109 outputs the index to the multiplexing circuit 116 following a signal instructing to multiplex the index.The control circuit 109 also outputs the same index to the index register 103.The index register 103 that received the index has a distortion of Th. The order of the indexes in the register is rearranged and updated according to a predetermined rule as in the case where the difference is smaller than .Whether the distortion is larger or smaller than Th, the output difference signal is converted to DC by the DCT (, 1 loss cosine transform) circuit 113.
A significant motion region is selected by the significant motion region determination circuit 114 and normalized by the normalization/quantization circuit 115.

The signal is quantized and output to multiplexing circuit 116. To summarize the above, the multiplexing circuit 116 uses the block address, the counter value representing the order of the index in the index register (or the signal indicating that the encoding is insufficient in the index register and the codebook index that follows), The region's significant/insignificant information normalization coefficient and quantization output are multiplexed and output.

Note that a DCT circuit 113 for encoding prediction errors,
Significant motion region determination circuit 114, normalization/quantization circuit 115
Consists of.

Circuit 118 can also be replaced in other ways.

For example, instead of the circuit 118, VQ (vector quantization) can be performed, or only scalar quantization can be performed. In either case, the features of this method are not impaired. Also, the structure of Food Book 107 is simple VQ.
The configuration may be based on the average value separation normalization VQ, or the configuration may be based on the average value separation normalization VQ.

A detailed description of some of the elements of FIG. 1 is now provided below.

First, index register 103 and codebook 107
Explain the relationship between Index register 103
The block specified by is the same size as the block blocked by the blocking circuit 101, but the codewords in the codebook do not need to be blocks of the same size. An example is shown in FIG. 2, where (a) is the case where both blocks have the same size, and (b) is an example where the block size of the index register is four times the block size of the code word. in this case,
The contents of the index register consist of four indexes, and the four indexes are collectively encoded, rearranged, and updated.

In addition, distortion evaluation is also performed using the block size of the index register. (c) is an example in which the block size is the same for both, but the encoding, updating, and distortion evaluation are performed for the luminance signal and color signal together.

In general, increasing the codeword block size
In order to obtain the same S/N, it becomes necessary to use a codebook with a large number of codewords, which makes codebook design difficult. Therefore, an example like (b) can be said to be important in that the block size of the codebook does not need to be increased.

Next, we will explain the variable length code that encodes the order of indexes.In the present invention, the indexes in the index register are rearranged and updated so that the closer to the beginning they are, the easier it is to be selected, so the closer to the beginning the shorter the code word. A variable-length code that can be assigned is used. As an example of such a variable length code, the codes shown in Table 1 can be considered.

(Margin below) Table 1 This is for use in the embodiment shown in Figure 1. For example, index number 1 means the same as the previous frame and is not a significant block, so no code word is assigned. In the example, index registers are prepared for each block position, but they may be prepared for several block positions collectively for an area.

Next, FIG. 3 shows an embodiment of a decoding device (receiving side) that decodes a signal encoded by the configuration example shown in FIG.
block address, a counter value representing the order of the index in the index register (or a signal indicating that the encoding is not sufficient in the index register and the following codebook index), and the significance/insignificance of the dynamic region. Separated into information, normalization coefficients, and quantization output. The block address is sent to a block address decoding circuit 402, decoded, and then sent to an index register 405 and a frame memory 412.
Only significant blocks designated by the block address are written into the frame memory 412.

The counter value representing the index order is decoded by the variable length decoding circuit 403, and in the case of a signal representing "encoding with codebook" (corresponding to 11111111 in Table 1), a signal indicating that is sent to the control circuit 404. . In this case, following that signal, the index of the food book is input from the separation circuit 401 to the control circuit 404. The control circuit 401 sends a signal to the switching circuit 406 to change the codebook 4.
07 and the control circuit 404 to send the index to the codebook 407. The control circuit 404 also sends the index to the index register 405, and within the index register, the order of the index is rearranged and updated according to the same rules as on the sending side. On the other hand, the signal decoded by the variable length decoding circuit 403 is [encoded with a codebook].
If the signal is any other signal, the signal is a counter value representing the index order, and is input to the index register 405. This index register 40
5 is controlled so that the same index as the index register 103 on the transmitting side is placed in the same position.

The index register 405 outputs the index determined by the counter value and the block address to the codebook 407 through the switching circuit 406, and
The order of the index is rearranged and updated using the same rules as on the sending side. In any of the above cases, the switching circuit 4
06 is input into the codebook 407, and the code word corresponding to the index is output to the adder circuit 411.

Separation circuit 401 also sends the quantization output and normalization coefficients to dequantization/expansion circuit 408 . Next, the conversion plane block configuration circuit 409 receives the significant/
A transform plane block is constructed from a signal that has been dequantized/expanded based on the meaningless information. This signal is the IDC7 circuit 410
The signal is inversely cosine-transformed, added to the output from the codebook in an adder circuit 411, and written into the corresponding block address position of the frame memory 412 to form a decoded frame and output.

Next, FIG. 4 shows a block diagram of another configuration example of the encoding device according to the present invention. In this embodiment, the prediction based on the past appearance pattern at the same block position in the first embodiment is based on parallel movement. This is an embodiment in which only motion compensated interframe prediction is separated from the prediction method according to the present invention, taking into consideration that the performance of motion prediction may be slightly inferior. The input block is detected by the motion detection circuit 5 after determining the significant block.
27 variable delay circuit 526. A motion compensated prediction unit comprising a frame memory 525 and a difference circuit 512 performs motion compensated interframe prediction, and the distortion obtained from the prediction error is compared with a set threshold in a comparison circuit 518. If the distortion is smaller than the threshold, the prediction error is encoded by the prediction error encoding unit connected to the DCT circuit 513 by the subsequent switching circuit 520. If the distortion is greater than the threshold.

A control signal 519 sends a signal indicating that the prediction is to be switched from motion-compensated interframe prediction to prediction using temporal correlation according to the present invention to the multiplexing circuit 516, and a switching circuit 520 switches the prediction from 503 to 511. The prediction unit is switched to a prediction unit having the same configuration as the first embodiment.

Next, FIG. 5 shows a block diagram of another configuration example of the encoding device according to the present invention. In this sixth embodiment, the prediction based on the past appearance pattern at the same block position in the first embodiment is shifted in parallel. In consideration of the possibility that performance may be slightly inferior in motion prediction according to the present invention, this embodiment adds a motion compensation element to the prediction pattern according to the present invention. The input image is temporarily stored in a frame memory 600, and is blocked by a blocking circuit 601 into blocks having the same size as the motion compensation range centered on the encoded block, and is input to a significant block determination circuit 602. The significant block determination circuit 602 evaluates the distortion with respect to the signal transmitted in the previous frame only for the coded block in the center of the block, and determines whether the coded block is significant or insignificant. As in the first embodiment, blocks determined to be significant are first predicted by the codeword in the codebook 607 indicated by the index in the index register 603, the optimal prediction error is evaluated by a set threshold, and the codebook is It is determined whether to perform prediction using the entire codeword within 607.

However, at this time, each pattern indicated by the index has the same size as the block cut out by the blocking circuit 601, as shown in FIG. The central block indicated by a thick line in FIG. 6 is the coding block, and the blocks (nine in the figure) constituting the motion compensation range block are the block size of the code word. As a numerical example, a motion compensation range block may be 12×12, a coding block may be 8×8, and a codeword block may be 4×4. For the coded block portion at the center of the block determined to be a significant block, an optimal pattern is motion-compensated by a motion compensation unit including a motion detection circuit 618 and a variable delay circuit 619, and is output to the selection circuit 608. The obtained motion vectors are multiplexed in multiplexing circuit 616. The subsequent processing is performed in the same manner as in the first embodiment.

Now, in this embodiment, the contents of the index register are updated in almost the same way as in the first embodiment, but since the block size is larger than in the first embodiment, it is easier to update the contents using a simple method. It becomes necessary to send many indexes for updating. For example, in the example shown in FIG. 6, while in the first embodiment it is sufficient to send four indexes at the time of updating, in this embodiment it is necessary to send nine indexes, which is inconvenient. However, in the case of this embodiment, the 8 pixels located around the motion compensation range block
Blocks overlap motion compensation range blocks in adjacent coded blocks. Therefore, for example, when updating, the shaded block position in Figure 6 has already been transmitted in an adjacent encoded block (or it will not be transmitted if it is not a significant block, but in either case, the first block position in the index register If we update by referring to the index (all we have to do is look at the index), and we decide to send update information only for the 4 blocks that are not shaded as for the current block position,
The same transmission burden as in the first embodiment can be achieved.

Next, FIG. 7 shows a block diagram of another example of the configuration of the encoding device according to the present invention. In this embodiment, even if distortion is larger than a set threshold in prediction using a codeword in a codebook indicated by an index register, prediction is not performed using the entire codebook, and the decoded signal (locally decoded signal on the transmitting side) is used. In this method, only the index register is updated by comparing the code with the entire codebook. Therefore, the process up to encoding using the index register is performed in the same manner as in the first configuration example. At this time, whether the prediction was successful or distorted is determined by evaluating the distortion of one block of the prediction error decoded signal outputted from the IDCT circuit 824 in the comparison/control circuit 809. If the distortion is greater than the set threshold, the comparison/control circuit 809 outputs the counter 805, switching circuits 806 and 8121.
A signal indicating this is output to the selection circuit 808 and the index register 803. As a result, the switching circuit 806 connects the counter 805 and the codebook 807, the switching circuit 812 sends the locally decoded signal from the adder circuit 822 to the selection circuit 808, and the selection circuit 808 selects the code word closest to the locally decoded signal in the codebook. The index is selected and sent to the index register 803.

In response to this, the contents of the index register are updated.

This operation is performed in exactly the same way on the receiving side.

〔Effect of the invention〕

According to the present invention, predictive coding using temporal correlation can be realized using only one prediction method. This prediction includes conventional motion compensated interframe prediction, background prediction, and deformation compensation. Also, although this prediction is performed adaptively to the block position, the only data stored for each block position is the codebook index.

There is also less increase in memory load.

[Brief explanation of the drawing]

FIG. 1 is a block diagram showing an example of the configuration of an encoding device according to the present invention, FIG. 2 is a diagram showing an example of the configuration of the contents of an index register, and FIG. 3 is a block diagram showing an example of the configuration of a decoding device according to the present invention. 4 and 5 are block diagrams each showing another example of the configuration of the encoding device according to the present invention, and FIG. 6 is a diagram for explaining patterns indicated by indexes. FIG. 7 is a block diagram showing another example of the configuration of the encoding device according to the present invention. Agent Patent Attorney Ken Nori Chika Yudo Mitsuyuki Matsuyama, 71121 44 Exit Figure

Claims (4)

[Claims] (1) In a video encoding method that divides each frame of a video into blocks of a certain size and makes predictions for each block, past appearance patterns that are sequentially classified and accumulated based on distortion are set in advance. A video encoding method characterized in that the prediction is performed using the stored appearance patterns. (2) The pattern according to claim 1 is constituted by a codebook used in the entire block and an index register for each block, and prediction is performed using a codeword in the codebook indicated by an index in the index register. 2. The video encoding method according to claim 1, wherein if the resulting distortion is larger than a predetermined threshold value, an optimal codeword is selected from the entire codebook and the contents of the index register are updated. (3) The moving image encoding system according to claim 2, wherein in the index register, a plurality of sets of indexes of the codebook are used as units of accumulation and updating. (4) In the index register, a pair of luminance signal index and color signal index is stored and stored.
3. The moving picture encoding method according to claim 2, wherein the moving picture encoding method is a unit of update.