JPH09266575A - Method and device for encoding dynamic image - Google Patents

Abstract

PROBLEM TO BE SOLVED: To decrease the arithmetic quantity of motion vector detection and to improve the efficiency and speed of motion vector detection by performing the detection of a motion vector based on area form information. SOLUTION: A small area dividing means 1 divides an encoding object image into encoding units, and a belonging area discriminating means 2 detects an area, to which the encoding unit as an object to find motion belongs, based on known area form information or area form information detected from an input signal. Besides, a small area dividing means 3 divides a reference image into the small areas of reference objects and a belonging area discriminating means 4 detects an area, to which the small area of the reference image as a reference object belongs, based on the area form information of the reference image and detects whether or not this detected area is equal with the area, to which the encoding unit as the object to find the motion belongs. A motion vector detecting means 5 detects the motion vector by collating the encoding unit of the encoding object image with the small area of reference of the reference image.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a digital compression coding method and a digital compression coding apparatus for an image signal used for image communication, image recording, and the like, and more specifically, for transmitting and receiving an image signal between terminals and a file. The present invention relates to a moving picture coding method and a moving picture coding apparatus used for storing in a storage medium.

[0002]

2. Description of the Related Art Conventionally, motion-compensated inter-frame prediction is often used as a means for suppressing temporal redundancy of a moving picture signal in digital compression coding of a moving picture.

FIG. 7 is a diagram for explaining a conventional motion vector detecting method, and FIG. 8 is a diagram for explaining motion compensation based on the obtained motion vector. In conventional inter-frame prediction,
Normally, as shown in FIG. 7, the encoding target image 71 is divided into rectangular blocks of 16 pixels × 16 lines as the encoding unit, and the encoding target block 72 for which the motion is to be obtained is centered for each block. The search range 73 is set in the reference image 74, and the block that is most similar to the encoding target block 72 is used as an evaluation function with the absolute value difference, the sum of squared differences, etc., while moving the target block in parallel within the range. It is used to search for the entire search range 73 that has been set, and the relative position with respect to the reference image 74 is detected as a motion vector (v _x , v _y ) 75.

The image that best matches the image data on the coordinates (x, y) of the image to be encoded 71 is associated as the image data on the coordinates (x _r , y _r ) in the reference image 74.

[0005]

[Equation 1]

In the motion-compensated inter-frame prediction based on the motion vector, as shown in FIG. 8, instead of the block at the same position (x, y) in the reference image 74, the obtained motion vector (v _x , v _y ) A block at a position parallelly moved by a minute is used as a prediction image, and a difference (motion compensation prediction error) signal between this prediction image and the encoding target image is encoded. As a result, the inter-frame correlation of moving images is dramatically improved, and the inter-frame prediction efficiency can be significantly improved.

FIG. 9 is a block diagram of a conventional motion vector detecting method. The encoding target image 901 is stored in the frame memory 902. The image 906 of the current frame stored in the frame memory 902 reads the value of the address signal 908 (the address corresponding to the upper leftmost pixel of the block) consisting of two coordinate components of (x _cf , y _cf ) as an address value. And input to the small area dividing unit 907. The small area dividing unit 907 reads m × n block data from the image 906 of the current frame (usually m = n = 16), and outputs the encoding target block signal 912.

The image 901 to be coded becomes a reference image 904 after one frame delay 903 and is stored in the frame memory 905. The reference frame image 909 stored in the frame memory 905 is (x _rf , y _rf ) 2
The value of the address signal 911 (address corresponding to the upper leftmost pixel of the block) composed of coordinate components is read as an address value and input to the small area dividing unit 910. The small area dividing unit 910 extracts m × n from the image 909 of the reference frame.
Block data is read (usually m = n = 16),
The reference frame block signal 913 is output.

In the matching error calculation unit 914, the encoding target block signal 912 and the reference frame block signal 913 output from the two small area dividing units 907 and 910 are output.
Each corresponding pixel value of is calculated, and the calculated error evaluation value 915
Is output to the comparator 916.

The comparator 916 has a minimum value memory 917. If the input error evaluation value 915 is smaller than the value stored in the minimum value memory 917, the value in the minimum value memory 917 is updated, An address signal (x _rf , which indicates the position of a reference block composed of two coordinate components x and y, respectively)
The motion vector (v _x , v _y ) 918, which is a value obtained by subtracting the address signal (x _cf , y _cf ) 908 indicating the position of the target block from (y _rf ) 911, is stored in the vector memory 919.

When the above operation for the search range is completed,
The detected motion vector 918 is stored in the vector memory 919.
Is stored.

[0012]

SUMMARY OF THE INVENTION In the above prior art,
When detecting a motion vector, a search range centered on the coding unit (block) that is the target of motion is created in the reference image regardless of the video content and the area shape information accompanying it, and the setting is made. Since it searches all within the specified range, the amount of calculation becomes enormous.

Further, in the conventional motion detecting method, the motion is detected without considering the area shape of the video signal.
There is a possibility that a motion vector may be wrongly detected and a motion vector may be erroneously detected from a region different from the region to which the coding unit for which motion is to be calculated belongs. Often fall apart. In this case, when the vector value of each block is transmitted to the decoder side later, the difference value between the adjacent vectors becomes large, and the coding efficiency is reduced.

An object of the present invention is to solve the above problems and to use the area shape information of the input video signal to reduce the amount of calculation at the time of search without lowering the accuracy of motion vector detection. An object of the present invention is to provide a moving picture coding method and a moving picture coding apparatus using a vector detection method.

[0015]

In order to achieve the above object, the present invention divides an image to be encoded into encoding units, and for each encoding unit, a motion amount between the image to be encoded and a reference image. Is detected, motion compensation is performed using the motion vector obtained from the motion amount, a predicted image is created, and the difference between the predicted image and the image to be coded is coded. When detecting the motion amount for each conversion unit, if the area shape information of the input video signal is known, it is used, and if the area shape information of the input video signal is not known, the area of the input video signal is used. The shape is detected and the motion vector is searched based on the area shape only within the reference image and within the same area as the area to which the coding unit whose motion is to be calculated belongs.

FIG. 1 is a diagram for explaining the principle of the present invention. In the present invention, the known region shape information of the encoding target image or the encoding target image region shape information detected from the input video signal and the corresponding reference image region shape information are input, and the following is performed. By using the means, a motion vector for motion compensation is detected.

In FIG. 1, the small area dividing means 1 is means for dividing an image to be encoded into encoding units. The belonging area determination means 2 detects the area to which the coding unit, which is the target of the motion, belongs, based on the known area shape information of the image to be coded or the area shape information detected from the input video signal. Is a means to do.

The small area dividing means 3 is means for dividing the reference image into small areas to be referred to. The belonging area determination unit 4 detects the area to which the small area of the reference image that is the reference object belongs, based on the area shape information of the reference image, and the coding unit that is the target of the motion belongs. It is a means for detecting whether or not the area is the same as the area.

The motion vector detecting means 5 is a means for detecting the motion vector by collating the coding unit of the image to be coded with the reference small area of the reference image. When the area to which the small area of the reference image belongs is not the same as the area to which the coding unit for which the motion is to be calculated belongs, the search range of the motion vector is changed for other small areas of the reference image. Have means.

With these, in detecting the motion amount for each coding unit, by using the area shape information, the range in which the motion vector is searched is coded within the reference image and for which the motion is to be obtained. It is in the same area as the area to which the unit belongs.

That is, in the present invention, when the area shape information of the input video signal is known, it is used, and when the area shape information is not known, for example, the K-means clustering method or the equal density area is extracted. The area shape is detected by performing Then, the image to be coded is divided into coding units, and when detecting the motion vector for each coding unit, the motion vector search range is set based on the area shape information of the video signal, and the motion vector in the reference image is set to It is limited to the same area as the area to which the coding unit to be obtained belongs.

Consider, for example, the case of the moving image shown in FIG. In the example of FIG. 2, the coding unit is a rectangular block for simplicity. Now, (a), (b), (c) of FIG.
They are arranged in chronological order. It is assumed that (b) is encoded using (a) as a reference image and (c) is predictively encoded using (b) as a reference image. 2 (d),
(E) and (f) are shown in FIGS.
It is the area shape information of (c). The area shape information can be easily acquired by creating an image using a chroma key. In recent years, computer graphics (C
The image according to G) is often created, and the area shape information is known even in such a case.

In the example of FIG. 2, the area shape information is divided into two areas, a person in front of the image and a background. Now, in FIGS. 2D, 2E, and 2F, the foreground part of the person shown in black is a region α, and the background part shown in white is a region β.

In this case, when detecting the motion vectors of the coding units, blocks b1 and b2, within the human region (region α) of FIG. 2B, among the reference images of FIG. , Only the portion corresponding to the area α in FIG. 2D is searched.

Similarly, when detecting the coding unit in the human area (area α) in FIG. 2C and the motion vector of the block b3, one of the reference images in FIG. Two
Only the portion corresponding to the area α in (e) is searched.

Further, when detecting the coding unit in the background area (area β) of FIG. 2C and the motion vector of the block b4, among the reference images shown in FIG.
Only the portion corresponding to the area β in (e) is searched.

The operation of the present invention will be described below in comparison with the prior art. In the conventional moving image coding method, since the entire search is performed in the vicinity of the coding unit (block) for which the motion vector is obtained regardless of the area shape, the image created by CG, the chroma key, etc. When the area shape information of the video signal is known, such as the video created by using it and the composite video of it and CG, the motion vector search efficiency is extremely poor, and a very big problem when real-time processing is required. It was.

The motion vectors of adjacent blocks are
Originally, the values are close to each other, but in the conventional motion vector detection method, since a full search is performed, a vector with a low correlation may be detected even though it is an adjacent block. This is because in the case of transmitting the motion vector in block units to the decoder, the existing coding method that encodes / transmits the difference between the motion vectors of adjacent blocks has a problem that the amount of information is rather increased. It was

By using the present invention to detect the motion vector based on the area shape information, it is possible to reduce the calculation amount of the motion vector detection and improve the motion vector detection efficiency and speed. INDUSTRIAL APPLICABILITY The present invention can be a basic technology for realizing real-time encoding of video by software. Also, when hardware is used, the amount of heat generated decreases as the amount of calculation decreases, which facilitates chip design.

Further, by using the present invention, the motion vector detection accuracy is improved, and the motion vectors of adjacent coding units (blocks) have relatively close values, which is necessary for the motion vector coding / transmission. It is possible to efficiently encode video even when a small amount of information is required and a low rate is required such as image communication via a wireless transmission line or a modem.

[0031]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the present invention will be described below with reference to the drawings using an example in which a rectangular block is used as an encoding unit.

FIG. 3 is a diagram showing a configuration example of a motion vector detector of the moving picture coding apparatus according to the embodiment of the present invention. The other components of the moving picture coding device may be similar to the conventionally used technique, and therefore their illustration is omitted.

The image 101 to be coded is stored in the frame memory 102. The image 106 of the current frame stored in the frame memory 102 reads the value of an address signal 107 (address corresponding to the upper leftmost pixel of the block) consisting of two coordinate components (x _cf , y _cf ) as an address value. And input to the small area dividing unit 108.

The small area dividing unit 108 reads m × n block data from the image 106 of the current frame (normally,
m = n = 16), and outputs the encoding target block signal 109. The address signal 107 composed of the two coordinate components (x _cf , y _cf ) simultaneously generates the encoding target image area shape information 11
It is input together with 0 to the belonging area determination unit 111.

The belonging area determination unit 111 determines which of the areas indicated by the encoding target image area shape information 110 the target block currently to be encoded is included in the encoding target block. Belonging area flag 1
12 is output.

The image 101 to be coded becomes the reference image 104 after the delay 103 of one frame and is stored in the frame memory 105. The image 113 of the reference frame stored in the frame memory 105 reads the value of the address signal 114 (the address corresponding to the upper leftmost pixel of the block) consisting of two coordinate components (x _rf , y _rf ) as the address value. And input to the small area dividing unit 115.

The small area dividing unit 115 reads m × n block data from the image 113 of the reference frame (usually m = n = 16), and the reference frame block signal 116.
Is output. The address signal 114 composed of two coordinate components of (x _rf , y _rf ) simultaneously generates the reference image area shape information 11
7, together with the encoding target block belonging area flag 112,
It is input to the belonging area determination unit 118.

The belonging area determination unit 118 determines which of the areas indicated by the reference image area shape information 117 the block (reference block) to be referred to when the encoding target block signal 109 is encoded is included. Along with the determination, the area and the encoding target block belonging area flag 11
It is determined whether the areas indicated by 2 are the same, and the same area belonging flag 119, which is a flag indicating whether the encoding target block and the reference block belong to the same area, is output to the matching error calculation unit 120. .

For example, in the case of the moving image shown in FIG.
When encoding the block b5 to be encoded in FIG. 4B, according to FIG. 4D, since the block b5 belongs to the area α, the encoding target block belonging area flag 1
12 is a flag indicating "α".

At this time, when the block b6 of FIG. 4 (a) is searched as a reference block, according to FIG. 4 (c), the block b6 belongs to the area α, so that the same area belonging flag 119 is set to " It becomes a flag indicating "the same area". Further, when the block b7 of FIG. 4A is searched as a reference block, according to FIG. 4C, since the block b7 belongs to the region β, the same region belonging flag 119 indicates “different region”. It becomes the flag shown.

The matching error calculation unit 120 is composed of a unit for checking the value of the same region belonging flag 119 and an arithmetic unit. The matching error calculation unit 120 first determines the value of the same area belonging flag 119.

(I) When the flag is "same area" If the same area belonging flag 119 is a flag indicating "same area", the two small area dividing units 10
Encoding target block signal 10 output from 8 and 115
9 and the difference square value of each corresponding pixel value of the reference frame block signal 116 are calculated, and the sum of them is output to the comparator 122 as an error evaluation value 121.

(Ii) When the flag is "different area" On the other hand, when the same area belonging flag 119 is a flag indicating "different area", the processing relating to this reference frame block is stopped at this point and the processing Cancel flag 12
6 is transmitted to the small area dividing unit 115, and the process proceeds to the processing of the next reference frame block. By this processing, it is possible to greatly reduce the amount of calculation without lowering the accuracy of motion vector detection.

Further, in the matching error calculation unit 120,
Corresponding to one block signal to be encoded 109, a plurality of reference frame block signals 116 within the search range and the flag determination and calculation are performed. The difference value calculation is not limited to the sum of squared differences, and it is also possible to use the sum of absolute differences.

The comparator 122 has a minimum value memory 123. If the input error evaluation value 121 is smaller than the value stored in the minimum value memory 123, the value in the minimum value memory 123 is updated, An address signal (x _rf , which indicates the position of a reference block composed of two coordinate components x and y, respectively)
The motion vector (v _x , v _y ) 124, which is a value obtained by subtracting the address signal (x _cf , y _cf ) 107 indicating the position of the target block from (y _rf ) 114, is stored in the vector memory 125.

[0046]

[Equation 2]

When the above operation for the search range is completed, the detected motion vector 1 is stored in the vector memory 125.
24 are stored. In addition, in the conventional moving image coding method using fixed-size blocks, motion compensation is performed using only one motion vector for one block, so only the motion of one object can be followed. It has already been pointed out that there is a portion where motion compensation does not work well, the prediction error increases and the coding efficiency decreases.

In order to avoid this, when the block b8 to be encoded includes the contour line of the area shape as shown in FIG.
Divide the block into multiple small areas according to the area shape,
A method has already been proposed in which each small area is used as a new coding unit and a motion vector is calculated for each small area and then coded.

Also, as shown in FIG. 6, a method has been proposed in which the block b8 is subdivided into small blocks (encoding units), and similarly, a motion vector is obtained for each small block and encoded.

Even in such a case, in detecting the motion vector for each coding unit generated by dividing the block, the present invention is used to detect the motion vector based on the area shape information of the video signal. The search range is
It can be dealt with by restricting to the same area as the area to which the coding unit for which the motion vector is obtained belongs, and it is possible to greatly improve the calculation amount as in the case where the boundary of the area shape is not included in the block. Become.

Specifically, as shown in FIG. 5D, when the coding target block including the region shape is subdivided into the coding target small area α ₁ and the coding target small area β ₁ , Since the target small area α ₁ belongs to the area α, only the portion corresponding to the area α of FIG. 5C in the reference image of FIG. 5A is searched.

Similarly, since the encoding target small area β ₁ belongs to the area β, only the portion corresponding to the area β of FIG. 5C in the reference image of FIG. 5A is searched. To do. Further, in the method shown in this embodiment, the area shape information of the input image is known, but the area shape information of the input image signal is known as in the case of a video image captured without using a chroma key or the like. If not, the encoding target image area shape information 110 and the reference image area shape information 117 need to be detected in advance using K-means clustering or the like. For details on region segmentation techniques such as K-means clustering, see "Image Analysis Handbook" (supervised by Mikio Takagi and Yohisa Shimoda. The University of Tokyo Press, January 1991).

[0053]

As described above, according to the present invention,
By detecting the motion vector based on the area shape information, it is possible to reduce the calculation amount of the motion vector detection, improve the motion vector detection efficiency and speed, and realize the real-time coding of video by software. It becomes the basic technology of.

Further, by using the present invention, the detection accuracy of the motion vector is improved, and the motion vectors of adjacent coding units (blocks) have relatively close values, so that the low rate is required. Even in this case, the video can be efficiently coded.

[Brief description of drawings]

FIG. 1 is a diagram illustrating the principle of the present invention.

FIG. 2 is a diagram showing area shape information according to the embodiment of the present invention.

[Fig. 3] Fig. 3 is a diagram showing a configuration example of a motion vector detector of the moving picture coding device according to the embodiment of the present invention.

FIG. 4 is an explanatory diagram of whether a coding target block and a reference block are included in the same area.

[Fig. 5] Fig. 5 is a diagram illustrating an example of a coping method when a boundary of a region shape exists in a block to be encoded.

[Fig. 6] Fig. 6 is a diagram illustrating a coping method by small block division when a boundary of a region shape exists in a coding target block.

FIG. 7 is a diagram illustrating a conventional motion vector detection method.

FIG. 8 is a diagram illustrating motion compensation based on the obtained motion vector.

FIG. 9 is a block diagram of a conventional motion vector detection method.

[Explanation of symbols]

 1 small area dividing means 2 belonging area judging means 3 small area dividing means 4 belonging area judging means 5 motion vector detecting means 101 image to be encoded 102 frame memory 103 1 frame delay 104 reference image 105 frame memory 106 current frame image 107 address Signal 108 Small area dividing unit 109 Encoding target block signal 110 Encoding target image area shape information 111 Belonging region determination unit 112 Encoding target block Belonging region flag 113 Reference frame image 114 Address signal 115 Small region dividing unit 116 Reference frame block Signal 117 Reference image region shape information 118 Belonging region determination unit 119 Same region belonging flag 120 Matching error calculation unit 121 Error evaluation value 122 Comparator 123 Minimum value memory 124 Motion vector 125 Vector memory 26 treatment discontinuation flag

 ────────────────────────────────────────────────── ─── Continuing on the front page (72) Inventor Hiroshi Watanabe Nippon Telegraph and Telephone Corporation, 3-19-2 Nishishinjuku, Shinjuku-ku, Tokyo

Claims (2)

[Claims] 1. An image to be encoded is divided into encoding units,
The amount of motion between the target image and the reference image is detected for each coding unit, and the motion vector obtained from the amount of motion is used to perform motion compensation to create a predicted image. In the moving picture coding method for coding the difference from the picture to be coded, when detecting the motion amount for each coding unit, if the area shape information of the input video signal is known, it is used, If the area shape information of the input video signal is not known, the area shape of the input video signal is detected, and based on the area shape, the range in which the motion vector is searched is found in the reference image and the motion is obtained. A moving picture coding method, characterized in that it is only in the same area as the area to which the target coding unit belongs. 2. A prediction image is created by detecting a motion amount between an encoding target image and a reference image and performing motion compensation using a motion vector obtained from the motion amount. In a moving picture coding apparatus including means for coding a difference from an encoding target image, means for dividing the encoding target image into encoding units, and known region shape information of the encoding target image or an input video Based on the area shape information detected from the signal, means for detecting the area to which the coding unit that is the target of the motion belongs, means for dividing the reference image into small areas, and area shape information for the reference image Based on
Means for detecting an area to which a small area of a reference image as a reference object belongs, and detecting whether or not the area is the same as an area to which the coding unit as a target of the motion belongs, If the area to which the small area belongs to is not the same as the area to which the coding unit for which the motion is to be calculated belongs, a means for changing the search range of the motion vector with respect to other small areas of the reference image. , In detecting the motion amount for each coding unit, the range for searching the motion vector based on the area shape information is set in the reference image,
A moving picture coding apparatus, characterized in that it is set in the same area as an area to which a coding unit for which motion is to be obtained belongs.