JP2000308066A - Device and method for encoding moving picture - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a moving picture encoding device detecting a moving vector with less operation quantity while encoding efficiency is improved by detecting the moving vector of shape information in a macro block through the use of the moving vector of texture information which is previously detected. SOLUTION: It is checked whether a texture moving vector can be used or not (S11). When it cannot be used as a result, a search range is set to be a 'search range 3'. When it can be used as the result of checking, it is checked whether the texture moving vector can be relied or not (S12). When it cannot be relied as the result, the search range is set to be a 'search range 2'. When it can be relied as the result of checking, the search range is set to be a 'search range 1'. Thus, the search range corresponding o the prediction precision of a prediction vector can be switched and therefore the calculation quantity of the detection of the moving vector can be reduced without deteriorating encoding efficiency.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

[0001] The present invention relates to the ISO / IEC
A moving image encoding apparatus and a moving image using a function of individually encoding objects of an arbitrary shape realized by the international standard method MPEG4 for moving image encoding, which is being standardized in JTC / SC29 / WG11 It relates to an encoding method.

[0002]

2. Description of the Related Art MPEG standardization is currently underway.
4 (Motion Picture Experts Groupphase 4)
MPEG1 which is the international standard system of the conventional moving picture coding
(Motion Picture Experts Group phase 1) and MPE
A function that cannot be realized by G2 (Motion Picture Experts Group phase 2), that is, a function of individually encoding each object of an arbitrary shape (for example, a person appearing in a screen) is to be realized.

In order to realize this function, information (shape information) indicating the shape and size of each object is required, and this information is encoded together with texture information indicating changes in luminance and color difference inside the object. After transmission,
Stored.

That is, if there is an image composed of a background and an object, this image is divided into a background and an object and encoded. Then, in order to separately encode the background and the object, information (shape information) indicating the shape and size of each object is separately given. This is a shape information signal called an alpha map signal.

[0005] The alpha map signal is given as, for example, binary sub-image information representing the shape of the object and the position in the screen.
Uniquely determined from the object's alpha map signal).

As described above, in MPEG4, information (shape information) representing the shape and size of each object is encoded together with texture information representing changes in luminance and color difference inside the object, thereby reducing the amount of information. On the decoding side, a function that can reproduce the object based on this can be realized.

Here, an outline of an MPEG4 image encoding and decoding apparatus will be described. FIG. 7 is a block diagram of an image encoding apparatus that encodes an image by dividing the screen into a background and an object when encoding the image. As shown in FIG. 7, the image encoding apparatus includes a difference circuit 1100, a motion compensation prediction circuit (MC) 11
10, orthogonal transform circuit (DCT) 1120, quantization circuit 1
130, variable length coding circuit (VLC) 1140, inverse quantization circuit (IQ) 1150, inverse orthogonal transform circuit (IDCT)
1160, an addition circuit 1170, a multiplexing circuit 1180, and an alpha map encoding circuit 1200.

The alpha map encoding circuit 1200 encodes the input alpha map, outputs the encoded signal to the multiplexing circuit 1180 as an alpha map signal, and decodes the alpha map signal to local It has the function of outputting as a decoded signal.

In particular, the present alpha map encoding circuit 120
0 performs a process of reducing the resolution at a given reduction rate (magnification) when encoding the input alpha map, encodes the resolution-reduced image, and combines the encoded image with the encoded image. Reduction rate information (magnification information)
And multiplexing the same as an alpha map signal and outputting the same to the multiplexing circuit 180. Then, as the local decoded signal, a signal obtained by performing a process of returning the resolution reduced process to the original resolution is used.

The difference circuit 1100 includes a motion compensation prediction circuit 1
A difference signal between the motion-compensated prediction signal supplied from 110 and the input image signal is calculated.
Reference numeral 20 denotes a difference signal supplied from the difference circuit 1100,
According to the information of the alpha map, it is converted into orthogonal transform coefficients and output.

The quantizing circuit 1130 is the orthogonal transform circuit 1
The variable length coding circuit 1140 is a circuit for quantizing the orthogonal transform coefficient obtained by the quantization circuit 120.
30 is encoded and output. The multiplexing circuit 1180 multiplexes and multiplexes the signal coded by the variable length coding circuit 1140 and the alpha map signal together with side information such as motion vector information, and outputs the multiplexed signal as a bit stream.

The inverse quantization circuit 1150 includes the quantization circuit 113
0 is inversely quantized, and the inverse orthogonal transform circuit 1
Reference numeral 160 denotes an inverse orthogonal transform for the output of the inverse quantization circuit 1150 based on the alpha map. An adder 1170 outputs the output of the inverse orthogonal transform circuit 1160 and a prediction signal (motion) supplied from the motion compensation prediction circuit 1110. And outputs the result to the difference circuit 1100.

The motion compensation prediction circuit 1110 has a frame memory, and operates based on the local decoded signal supplied from the alpha map decoding circuit 1200, and has a function of accumulating the signal of the object area and the signal of the background area. Also, the motion compensation prediction circuit 1110 predicts a motion compensation value from the stored image of the object area and outputs the predicted value, and also predicts a motion compensation value from the stored image of the background area and outputs the predicted value. Having.

The operation of the apparatus having such a configuration will be described. This apparatus receives an image signal and an alpha map of the image signal. Of these, the image signal has a predetermined pixel size (for example, M × N) for each frame.
After being divided into blocks of pixels (M: the number of pixels in the horizontal direction, N: the number of pixels in the vertical direction), they are supplied to the difference circuit 1100 via the signal line 1010 in the order of the block positions.
Then, in the difference circuit 1100, this input (image signal)
And a prediction signal (the output of the motion compensation prediction signal from the object prediction circuit 1110) is calculated and supplied to the orthogonal transform circuit 1120.

The orthogonal transform circuit 1120 converts the supplied difference signal into orthogonal transform coefficients according to the information of the alpha map supplied from the alpha map encoding circuit 1200 via the signal line 1040, and then converts the signal into a quantized circuit. 1
130. And it is quantized here. The transform coefficient obtained by quantization in the quantization circuit 1130 is
The data is encoded by the variable length encoding circuit 1140 and supplied to the inverse quantization circuit 1150.

The transform coefficient supplied to the inverse quantization circuit 1150 is inversely quantized here, and then the inverse orthogonal transform circuit 116
Inverse conversion is performed at 0. Then, the sum is added to the motion compensation prediction value supplied from the motion compensation prediction circuit 1110 in the addition circuit 1170 and output as a locally decoded image.
It is again input to the motion compensation prediction circuit 1110.

The locally decoded image output from the addition circuit 1170 is stored in a frame memory in the motion compensation prediction circuit 1110.

On the other hand, the motion compensation prediction circuit 1110
Based on the local decoded signal supplied from the alpha map decoding circuit 1200, the processing timing of the block in the object area is "object motion compensation predicted value".
In addition, at other timings, the “motion compensation prediction value of the background portion” is output and given to the difference circuit 1100.

That is, the motion compensation prediction circuit 1110 converts the image signal of the block corresponding portion of the object from the local decoded signal of the alpha map signal to the difference circuit 1100.
Or the image signal of the block corresponding portion of the background portion is input to the difference circuit 1100. If the image signal of the block corresponding portion of the object is being input, the motion compensation of the object is performed. The prediction signal is supplied to the difference circuit 1100 during the image signal input period of the block corresponding portion of the background portion during the input period.

The difference circuit 1100 calculates the difference between the input image signal and the predicted signal corresponding to the area of the image. As a result, if the input image is in the area corresponding to the object, If the difference signal from the predicted value at the corresponding position is the input image of the background area, a difference signal from the predicted value corresponding to the background position is calculated and supplied to the orthogonal transformation circuit 1120. .

The orthogonal transform circuit 1120 converts the supplied difference signal into an orthogonal transform coefficient by performing processing such as discrete cosine transform in accordance with the information of the alpha map supplied via the signal line 1040. This is supplied to the quantization circuit 1130. Then, the orthogonal transform coefficient is quantized by the quantization circuit 1130.

The transform coefficients quantized by the quantization circuit 1130 are encoded by the variable length encoding circuit 1140 and supplied to the inverse quantization circuit 1150. Then, the transform coefficient supplied to the inverse quantization circuit 1150 is inversely quantized here, then inversely transformed in the inverse orthogonal transform circuit 1160 and supplied to the addition circuit 1170. And
Addition circuit 1170 via predicted value switching circuit 1500
Is added to the predicted value supplied to

The signal of the locally decoded image output from the addition circuit 1170 is supplied to a motion compensation prediction circuit 1110.
Then, in the motion compensation prediction circuit 1110, whether the signal corresponding to the block of the object or the signal corresponding to the block of the background portion is currently output from the addition circuit 1170 from the local decoded signal of the alpha map signal When the signal corresponding to the block of the object is being output, the frame memory for the object is output. When the signal corresponding to the background block is being output, the signal is applied to the background memory. And store it in the corresponding memory.

Thus, only the object image is obtained in the object frame memory, and only the background image is obtained in the background memory. Thus, the motion compensation prediction circuit 1110 can obtain the predicted value of the object image using the object image, and can obtain the predicted value of the background image using the image of the background portion.

As described above, the alpha map encoding circuit 1200 encodes the input alpha map and sends the encoded alpha map signal to the signal line 30.
Is supplied to the multiplexing circuit 1180 via the.

The multiplexing circuit 180 is supplied with the transform coefficient output from the variable length coding circuit 140 via a line 1040. Then, the multiplexing circuit 180 multiplexes the supplied alpha map signal and the encoded value of the transform coefficient together with side information such as motion vector information and the like, and outputs the multiplexed value through a signal line 1050 to output the main image code. Bit stream as the final output of the encoding device.

FIG. 8 is a block diagram of the decoding apparatus. The decoding device, as shown in FIG.
00, variable length decoding circuit 2310, inverse quantization circuit 232
0, inverse orthogonal transformation circuit 2330, addition circuit 2340, motion compensation prediction circuit 2350, alpha map decoding circuit 24
00.

The demultiplexing circuit 2300 is a circuit for demultiplexing an input coded bit stream to obtain an alpha map signal and an image coded signal, and the like. This is a circuit for decoding the separated alpha map signal and reproducing the alpha map.

The variable length decoding circuit 2310 decodes the coded signal of the image separated by the separation circuit 2300, and the inverse quantization circuit 2320 inversely quantizes the decoded signal to obtain the original signal. The inverse orthogonal transform circuit 2330 performs an inverse orthogonal transform of the coefficient according to the alpha map and returns the result to a prediction error signal. The addition circuit 2340 adds the motion compensation prediction circuit 2
The motion compensation prediction value from 350 is added and output as a reproduced image signal. This reproduced image signal is the final output of the decoding device.

The motion compensation prediction circuit 2350 includes an addition circuit 2
An object image and a background image are obtained by accumulating the reproduced image signal output from 340 in a frame memory according to an alpha map, and an object motion compensation prediction signal and a background motion compensation are obtained from the accumulated image. Get predictions.

In the decoding device having such a configuration,
The coded bit stream is supplied to a demultiplexing circuit 2300 via a line 2070, and is separated for each piece of information in the demultiplexing circuit 2300, whereby a code relating to the alpha map signal and a variable length code of the image signal are converted. Divided.

The code relating to the alpha map signal is supplied to an alpha map decoding circuit 2400 via a signal line 2080, and the variable length code of the image signal is supplied to a variable length decoding circuit 2310.

The code relating to the alpha map signal is reproduced into an alpha map signal in the alpha map decoding circuit 2400 and output to the inverse orthogonal transform circuit 2330 and the motion compensation prediction circuit 2350 via the signal line 2090.

On the other hand, the variable length decoding circuit 2310 decodes the code supplied from the demultiplexing circuit 2300 and supplies it to the inverse quantization circuit 2320, where it is inversely quantized. The inversely quantized transform coefficients are inversely transformed by an inverse orthogonal transform circuit 2330 according to an alpha map supplied via a line 2090, and supplied to an adder circuit 2340. The addition circuit 2340 adds the signal subjected to the inverse orthogonal transform from the inverse orthogonal transform circuit 2330 to the motion compensation prediction signal supplied from the motion compensation prediction circuit 2350 to obtain a reproduced image.

The above is the outline of the image coding apparatus and the image decoding apparatus for MPEG4.

Here, the encoding of the alpha map will be described. The alpha map, that is, the pixel value of the shape information is a binary signal, which is represented by "0", "1" or "0", "255".

First, as shown in FIG. 9, the encoding of an alpha map is performed by encoding an object to be encoded (VOP in MPEG4) as shown in FIG.
(Called an “ideoObject Plane”). Then, this area is divided into macroblocks of 16 × 16 pixels, and the object is encoded for each macroblock.

Here, for each VOP, bounding
-Rcctangle size (vop-width, vop-heigh
t) and the position vector (Spatial-refference (vop-horizon
tal-mc-spatial-ref, vop-vertical-mc-spatial-ref))
Is encoded.

FIG. 10 is a diagram for explaining the attributes of each macroblock. Macroblocks can be classified into two types: “transparent macroblocks” that do not include objects in the macroblock (there is no pixel in the object), and “opaque macroblocks” in which all macroblocks are included in the object (all of which are objects). (A pixel is filled with pixels) and a “macroblock macroblock” in which a part of the macroblock is included in the object (in which some pixels of the object are included).

Therefore, the macroblock has one of the attributes of "transparent macroblock", "opaque macroblock", and "boundary macroblock".

The encoded data of each macroblock includes shape information and texture information. Note that texture information is not included in the transparent macroblock.

With reference to a reference (edited by Miki, "All about MPEG-4", Chapter 3, Industrial Research Council, 1998)
An encoding method of the shape information in the encoder of the MPEG4 verification model and an encoding method of the texture information of the arbitrary-shaped object will be described.

<Coding Method of Shape Information> A coding method of the shape information (A) will be described. FIG. 11 shows a configuration example of a model system as a conventional technique. The shape encoding is performed by a shape encoding encoder (Binary Shape Encoder 500) having a configuration as shown in FIG. The encoder for shape coding (shape information coding means 500) includes, as shown in FIG.
1. Shape motion vector detection means 502, motion compensation means 5
03, arithmetic coding means 504, motion vector prediction means 5
05, a frame memory 506, a selector 507, a shape motion vector information storage means 508, a difference circuit 509, reduction circuits 510 and 511, an enlargement circuit 512, variable length coding circuits 513 and 514, and a multiplexer 515.

The shape motion vector detecting means 502 detects a coding area (Boundi) for detecting a coding area which is an area where the target object exists in the frame image.
ng-rectangle), a shape motion vector is detected from the alpha map signal (shape information A) inputted from the alpha vector signal (shape information A), the information of the prediction vector obtained by the motion vector prediction means 505, and the shape information signal of the frame memory 506. This is output as shape motion vector information. The motion compensating means 503 compares the shape motion vector information detected by the shape motion vector detecting means 502 with the frame memory 50.
6 for obtaining motion vector information for motion compensation prediction from the shape information signal of No. 6 and the prediction vector output from the motion vector prediction circuit 505.

The shape motion vector information storage means 50
Numeral 8 is for storing the shape motion vector information detected by the shape motion vector detection means 502, and the motion vector prediction means 505 is for storing the shape motion vector information and the luminance signal 53 held by the shape motion vector information storage means 508.
And a prediction vector that is a prediction value of the motion vector is obtained based on the above.

The difference circuit 509 is for obtaining a difference between the shape motion vector obtained by the shape motion vector detection means 502 and the prediction vector obtained by the motion vector prediction means 505. Is to determine the mode from the input shape information, the motion vector information for motion compensation prediction output from the motion compensating means 503, and the shape motion vector information output from the shape motion vector detecting means 502.

Here, there are the following seven types of mode information. (Mode 1): Transparent (Mode 2): Opaque (Mode 3): Binary image coding (within frame) (Mode 4): Motion compensation (MV = 0) (Mode 5) ): Motion compensation (MV = 0) + arithmetic coding (interframe) (mode 6): motion compensation (MVＭ0) (mode 7): motion compensation (MV ≠ 0) + arithmetic coding (interframe) is there.

Of these, "mode 1" is a case where all data in the macroblock is transparent, that is, the macroblock does not include any object, and "mode 2" is a case where all data in the macroblock is not. Transparency, that is, the case where the entire macroblock is an object, and "mode 3" is the case of arithmetic coding (within a frame). “Mode 4” is a case where the motion compensation vector is zero (MV = 0), and “Mode 5” is a case where the motion compensation vector is zero (MV = 0) and performs arithmetic coding (between frames). "Mode 6"
Is the case where the motion compensation vector is not zero (MV ≠ 0), and in “mode 7”, the motion compensation vector is not zero (MV ≠ 0) and arithmetic coding (between frames) is performed. Is the case.

Arithmetic encoding means 504 arithmetically encodes one of the outputs of first and second reduction means 510 and 511 in accordance with the mode judgment result of mode judgment means 501 and outputs the result to multiplexer 515. Further, the selector 507 outputs the motion vector information for motion compensation prediction output from the motion compensation unit 503, the output of the enlargement unit 512, and the value of a transparent pixel which is a preset fixed value (Transparentpixel). value; for example, a value “0”) and a predetermined fixed value of an opaque pixel (Opaque pixel value; for example, a value “1”) are given. Judgment means 50
The mode information is selected and output depending on what mode information is given from step 1.

The frame memory 506 is a memory for holding the output of the selector 507 as a shape information signal.

The first reduction means 510 reduces the shape information input, which is a binary signal of the alpha map, and outputs the result to the arithmetic coding means 504. The second reduction means 511 , The motion vector information output from the motion compensating means 503 is reduced and output to the arithmetic coding means 504. The enlargement circuit 512 outputs the output of the first reducing means 510 to the mode of the mode determining means 501. The data is output to the selector 507 according to the determination result.

The first variable length coding circuit 513 performs variable length coding on the output of the difference circuit 509 and outputs the result. The second variable length coding circuit 514 outputs the result of the determination by the mode determination means 501. The multiplexer 515 receives the outputs of the first and second variable-length encoding means and the output of the arithmetic encoding means 504, multiplexes them, and encodes them into shape information. It is output as an output 52.

In such a configuration, the shape information signal 51 which is a binary signal of the alpha map is input. Then, the mode determination unit 501 that has received this determines the mode based on the supplied shape information, the shape motion vector output from the shape motion vector detection unit 502, and the motion vector information output from the motion compensation unit 503. And
According to the mode of each macroblock determined for each macroblock by the mode determining means 501, information is encoded for each macroblock as follows.

Here, the mode information as a result of the determination by the mode determining means 501 is one of the following seven types. "Mode 1" (Transparent), "Mode 2" (Opaque), "Mode 3" (arithmetic coding (within frame)), "Mode 4" (motion compensation (MV =
0)), “mode 5” ((motion compensation (MV = 0) + arithmetic coding (interframe)), “mode 6” (motion compensation (M
V ≠ 0)) and “mode 7” (motion compensation (MV ≠ 0) + arithmetic coding (between frames)).

The selector 507 outputs a reproduction signal for each macro block in accordance with the mode information determined by the mode determining means 501. Then, the output reproduced signal of each macro block is stored in a frame memory (F
M) 506 and supplied to the texture information encoding means via the output line 54 to be encoded.

Here, depending on what mode information is given from the mode determining means 501 to the selector 507,
The output from the selector 507 is as follows.

In the case of "mode 1": The reproduction pixel values of the shape information in the macro block are all set to "Transparcnt pixe".
l "(for example, each pixel value is set to" 255 ").

In the case of "modes 3, 5, 7": A signal obtained by subjecting the supplied macroblock shape information signal 51 to reduction processing by the first reduction means 510 (the arithmetic coding means 5).
04 is further enlarged by the enlarging means 512 to restore the original size and output a reproduced pixel value.

In the case of "modes 4 and 6": A value obtained by performing motion compensation prediction on the reproduced pixel value of the shape information in the macroblock by the motion compensation means 503 is output.

On the other hand, the shape / motion vector detecting means 502 calculates the shape input signal input from the input line 51 and the prediction vector obtained by the motion vector predicting means 505,
Shape motion vector information is obtained from the shape information signal stored in the frame memory (FM) 506. The shape motion vector information detected by the shape motion vector detection means 502 is used to calculate the shape of each macroblock by the motion compensation means 503.
Is motion vector information used for motion compensated prediction in step (a). The motion vector information is supplied to the motion compensating means 503 and the shape motion vector information storing means (MVmemoly)
8 is stored.

Here, the shape motion vector detecting means 502
Detects a prediction error vector in a range of ± 16 pixels around the prediction vector obtained by the motion vector prediction means 505.

Therefore, as shown in FIG. 3, since the frequency at which the zero vector is detected is the highest, the information as to whether or not the shape motion vector is a zero vector is included in the mode information. The code amount of information is reduced. Also, by using this property (the probability that a vector near the zero vector is detected is high), the processing amount of motion vector detection can be reduced.

The motion vector predicting means 505 calculates the predicted value of the shape motion vector from the shape motion vector stored in the shape motion vector information storage means (MV memoly) 508 and the texture motion vector supplied via the signal line 53. Seeking.

When "modes 3, 5, 7" are selected, the arithmetic coding means 504 converts the signal obtained by the first reduction means 510 to the signal obtained by the reduction processing in the case of intra-frame coding. It will be arithmetically coded. In the case of inter-frame encoding, the arithmetic encoding unit 504 refers to the signal reduced by the second reduction unit 511 for the signal obtained by performing the reduction processing by the first reduction unit 510. Arithmetic coding. On the other hand, "mode 5,
When 7 ″ is selected, the prediction error vector obtained by the difference circuit 509 is subjected to variable-length encoding by the variable-length encoding unit 513.

The arithmetic code obtained by the arithmetic coding means 504 or the variable length code of the prediction error vector obtained by the variable length coding means 513 is subjected to variable length coding by the second variable length coding means 514. Along with the obtained mode information, it is sent to a multiplexing means (MUX) 515,
Here, they are multiplexed and output as a shape information encoded output 52.

By the way, the binary image encoded information encoded by the arithmetic encoding means 504 is information in which a detailed shape in each macroblock is treated as a binary image and encoded. Here, the shape motion vector is obtained by searching for 16 pixels around the prediction vector obtained by the motion vector prediction means 505.

Therefore, since the frequency at which the zero vector is detected is the highest as shown in FIG. 3, the code amount of the shape motion vector information is reduced by including information on whether or not the shape motion vector is the zero vector in the mode information. are doing.

Thus, the shape information encoding means 50
An encoding process of the shape information by 0 is performed. It should be noted that the shape information here is for a two-gradation alpha map, and there is also a multi-tone gray scale alpha map.

<Encoding Method of Texture Information> The alpha map is information (shape information) representing the shape and size of an object, but if there is no texture information which is information representing a change in luminance or color difference inside the object. The image of the object cannot be played. Therefore, in MPEG4, together with the alpha map, the texture information is also encoded and used in pairs with the alpha map.

An encoding method of the texture information (YUV in FIG. 12) will be described with reference to FIG. FIG. 12 is a block diagram of an encoder showing a configuration example as a conventional model system. In FIG. 12, a coding region (Bounding-Rectangle) detection unit 301, a reference image padding unit 302, and LPE, which are components highlighted in the figure,
The padding unit 303, the zero padding unit 304, the vector padding unit 306, and the shape information encoding unit 500 are components necessary only for encoding an object having an arbitrary shape. Therefore, MPEG1 and MP
As in the case of a conventional encoding method such as EG2, in order to encode a rectangular object, other components, that is, the motion vector detecting means 305 in FIG.
Frame memory 307, motion compensation means 308, changeover switches 309 and 310, motion vector prediction means 311, motion vector storage means 312, difference circuit 313, third variable length coding means 314, quantization means 315, inverse quantization means It suffices if an element including 316, orthogonal transform means 317, fourth variable length coding means 318, inverse orthogonal transform means 319, adder circuit 320, switch 321, and multiplexer 322 is provided.

Here, the coding area (Bounding-Rectangl
e) The detecting means 301 is for detecting the coding area which is the area where the target object exists in the frame image (see FIG. 9), and the shape information coding means 50.
0 is as described with reference to FIG.

In the configuration of FIG. 12, the encoding of the texture information (YUV) is performed on the “opaque macroblock” and the “boundary macroblock”.
Of these, "opaque macroblocks"
As in the conventional encoding method, the input macroblock signal is encoded as it is by motion compensation prediction + DCT (discrete cosine transform).

On the other hand, for the “boundary macro block”, the signal outside the object is padded (compensation processing).
After that, coding is performed by the motion compensation prediction + DCT method.

Then, padding (compensation processing) here
Has the following three methods.

[1] Reference image padding: This is a supplementary process by the reference image padding means 302.
The reference image for motion compensation prediction is padded. There are processes for boundary macroblocks and processes for transparent macroblocks.

[2] LPE padding: This is LP
In the replenishment process by the E padding means 303, before performing DCT (Discrete Cosine Transform) on a block in an intra macroblock, the value of a pixel (open circle in FIG. 13) outside the object is converted to a pixel inside the object (black circle in FIG. 13). Part)
This is a process of applying a low-pass filter after replacing with the average value. The processing unit is the same as in DCT.
× 8 pixels.

[3] Zero padding: This is a supplementary process by the zero padding means 304. Before the DCT of the block in the inter macro block, pixels outside the object of the motion compensation prediction error signal (the white circles in FIG. 13) ) The process of replacing the value with a zero value. The processing unit is 8 × 8 pixels, which is the same as DCT.

Here, the padding of the above [1] is the MPE
Although it is an indispensable process in the G4 standard, the padding in [2] and [3] is necessary for improving coding efficiency and is not an indispensable process in the standard. The present invention is not limited to the processing by the constituent elements, and may be performed using other means.

In the motion vector detecting means 305,
Based on the shape signal supplied from the encoding area detecting means 301 via the signal line 31, the luminance signal (YUV) of the original image supplied via the signal line 32 and the frame memory 307 via the signal line 33 Motion vector detection is performed between the supplied reference image and the luminance signal. Then, as a result, the obtained motion vector 34 is stored in the motion compensation unit (MC).
308 and vector padding means 3
06.

Vector paddin means (Vector paddin)
g) 306, based on the reproduction value data 54 of the shape information output and supplied from the shape information encoding means 500, suitable for an 8 × 8 pixel unit block (transparent block or intra macro block) having no motion vector. After filling the motion vectors, the motion vector storage means (MVmemor
y) Store in 312.

When a motion vector is detected, pixel values outside the object are padded so that the reference image is smoothly connected to pixel values inside the object by padding.

On the other hand, in the original image, the pixel value at the object boundary is often the boundary of the edge, and thus the pixel value greatly varies. Therefore, when ordinary block matching is performed on the boundary macroblock, the mismatch between pixel values outside the object is large, and a normal motion vector is often not detected.

Therefore, in the conventional model, for the boundary macroblock, the motion vector is detected by evaluating the error only with the pixel value (black circle in FIG. 14) inside the object with reference to the shape information. (This is called polygon matching). However, in this method, a motion vector is detected for each pixel while determining whether or not the pixel is inside the object, so that the amount of momentum processing increases.

[0084]

As described above, the alpha map is information (shape information) representing the shape and size of an object, but the texture information is information representing the change in luminance and color difference inside the object. Without it, the object image cannot be played. Therefore, in MPEG4, together with the alpha map, the texture information is also encoded and used in pairs with the alpha map.

The encoding of the texture information (YUV) is performed on the “opaque macroblock” and the “boundary macroblock”.
For the “opaque macroblock”, the input macroblock signal is directly encoded by motion compensation prediction + DCT (discrete cosine transform), and for the “boundary macroblock”, the signal outside the object is padded (compensation processing). After that, coding is performed by the motion compensation prediction + DCT method.

That is, for the “boundary macroblock”, motion compensation prediction is performed. Therefore, when detecting a motion vector, the reference image is padded with pixel values outside the object so as to be smoothly connected to pixel values inside the object. That's why.

However, in the original image, the pixel value at the object boundary is often at the boundary of the edge, so that the pixel value greatly fluctuates. Therefore, when ordinary block matching is performed at the boundary macroblock, the pixel value outside the object is reduced. Is large and a normal motion vector is not detected in many cases.

Therefore, polygon matching, which is a method of detecting a motion vector by evaluating an error only with a pixel value (a black circle in FIG. 14) inside an object with reference to shape information for a boundary macroblock, is employed.

However, in this method, since a motion vector is detected while judging whether or not each pixel is inside an object, there is a problem that a processing amount increases.

Accordingly, an object of the present invention is to provide a moving picture coding apparatus and a moving picture coding method capable of detecting a motion vector with a small amount of calculation while improving coding efficiency. To provide.

[0091]

The present invention has the following configuration to achieve the above object.

[1] First, for a desired object constituting an image, a moving image encoding method for encoding the desired object as an arbitrary-shaped object composed of shape information and texture information in macroblock units. In the encoding device, means for detecting a motion vector of texture information for each macroblock, and determination means for judging whether or not to use a motion vector of texture information to detect a motion vector of shape information in the macroblock Evaluation means for evaluating the reliability of the motion vector of the texture information in order to detect the motion vector of the shape information in the macroblock; Sometimes set wider than available,
When the motion vector of the texture information is available, the setting means that is set wider when the reliability of the motion vector of the texture information is low than when the reliability is high, and the motion vector of the already detected texture information is used. And a motion vector detecting means for detecting a motion vector of the shape information in the macro block. In this apparatus, when encoding an arbitrary shape object composed of shape information and texture information, the motion vector detecting means can detect the motion vector of the texture information so that the motion vector of the texture information can be used. Of the input object's shape information and texture information, the identification means determines whether or not the motion vector of the texture information can be used, and the evaluation means evaluates the reliability of the motion vector of the texture information. In coding a macroblock to be coded, a shape motion vector of a macroblock around the macroblock is searched for a shape motion vector. The search range is the motion vector of the texture information. Is narrow when available, and the motion If the motion vector of the texture information is available, it is set wider than the case where the reliability of the motion vector of the texture information is higher than the case where the motion vector of the texture information is available. In this way, the search range of the shape motion vector of the macroblock is set to an appropriate range according to the availability of the motion vector of the texture information and the degree of reliability.

That is, the present invention relates to motion vector detection, and as a result of this, it is possible to provide a technique capable of reducing the calculation amount of motion vector detection without lowering the coding efficiency. .

[2] According to the present invention, in order to achieve the above object, secondly, for a desired object forming an image, the desired object is formed from shape information and texture information in macroblock units. A coding apparatus of a moving image coding system for coding as an arbitrary shape object includes a motion vector detecting unit for detecting a shape motion vector and a motion vector of texture information from shape information of an input object, Threshold setting means for setting a threshold for designating the error; dividing means for dividing the search range into a plurality of search ranges; and a motion compensation prediction error based on the detected motion vector being smaller than the threshold. Determination means for determining whether or not the search range is greater than the maximum value; That is, the motion vector detection is started from the narrowest search range, and if the motion compensation prediction error due to the optimal motion vector detected in the search range is larger than a threshold, the optimal motion vector Is detected, and when the motion compensation prediction error is smaller than the threshold value, the motion vector detection is terminated, and the motion vector is output as a detection result.

In MPEG4, when the prediction vector is a zero vector, the encoding efficiency is improved by incorporating information indicating that the prediction vector is a zero vector into the mode information and separately encoding the information. The ability to efficiently detect the zero vector also greatly contributes to the reduction in the amount of computation. Therefore, in the present invention, a threshold value setting unit is provided to set a threshold value for designating an error at the time of encoding, and the dividing unit divides the search range into a plurality of search ranges. I do. Then, the motion vector detection means starts the motion vector detection from the narrowest search range among the divided search ranges, and the determination means performs the detection based on the motion vector detected by the motion vector detection means. It is determined whether the motion compensation prediction error based on the obtained motion vector is larger than a threshold value. Then, as a result of starting the motion vector detection from the narrowest search range, the determination unit determines that the motion compensation prediction error due to the optimal motion vector detected in the search range is larger than the threshold value. If it is determined, the operation is performed to detect the optimal motion vector in a wider search range. If it is determined that the motion compensation prediction error is smaller than the threshold value, the motion vector detection is terminated, and the motion vector Is output as the detection result. As described above, when the motion compensation prediction error due to the optimal motion vector detected in the search range is large, the optimal motion vector is detected in a wider search range, and when the motion compensation prediction error is small, the motion vector detection is performed. Is ended, and the motion vector is output as a detection result. As a result, if the error is small, the zero vector can be efficiently detected with a small amount of calculation, which greatly contributes to a reduction in the amount of calculation.

[3] Further, in order to achieve the above object, the present invention provides, for a third object, a desired object which forms an image by arbitrarily constructing the desired object in macroblock units from shape information and texture information. Prior to performing shape coding and texture coding, a moving image coding method coding apparatus that codes as a shape object includes an object as a part by referring to a shape signal input to the coding unit. For the boundary macroblock of the form, a means for performing padding processing for eliminating discontinuities in the macroblock to compensate for all pixels in the macroblock to be subjected to texture encoding, and performing shape encoding. The texture encoding is performed using the padded image.

The present invention relates to texture coding in a "boundary macroblock". In an original image, for a boundary macroblock, padding processing capable of eliminating discontinuity of pixel values at an object boundary portion,
That is, padding processing is performed by LPE padding, and then subjected to shape encoding and texture encoding. Therefore, discontinuity of the pixel value at the object boundary is almost eliminated, so that a normal motion vector can be detected even when ordinary block matching is performed. Therefore, it is not necessary to perform polygon matching with a large amount of processing in the “boundary macroblock”, so that the calculation time can be reduced, and a normal motion vector can be detected even when ordinary block matching is performed. become able to.

[0098]

Embodiments of the present invention will be described below with reference to the drawings. The present invention improves the coding efficiency by changing the processing order of the components of the MPEG4 encoder (moving picture coding apparatus), and improves the detection method of the shape motion vector, without reducing the coding efficiency. This enables a motion vector to be detected with a small amount of calculation. The details will be described below.

<First Specific Example> A first specific example of the present invention will be described with reference to the drawings. The technique described in the first specific example is a technique that enables the amount of calculation for motion vector detection to be reduced without lowering the coding efficiency.

As described above, in the configuration of the model system as the prior art shown in FIG. 11, the shape motion vector (MVs) has a size of ± 16 pixels around the prediction vector (MVPs) obtained by the motion vector prediction means 505. (FIG. 3
In the search range 3). That is, the difference vector MVDs from the prediction vector MVPs is detected (Equation 1).

MVDs = MVs−MVPs (Equation 1) Normally, the frequency distribution of the prediction error signal has a high frequency near “0” as shown in FIG. 1, and the frequency rapidly decreases as the distance from “0” increases. Tend to be. In FIG. 1, the horizontal component of the difference vector MVDs is described as “mvds_y” and the vertical component is described as “mvds_y”.

Here, when the prediction accuracy of the prediction vector MVPs is low, the frequency distribution becomes slow, and when the prediction accuracy of the MVPs is high, the frequency distribution becomes steep. FIG. 2 is a diagram for explaining a method for obtaining the prediction vectors MVPs. FIG.
FIG. 2A is a diagram showing information on a shape signal component, and FIG. 2B is a diagram showing information on a luminance signal component in units of macroblocks. Vector MVs
It indicates that there is a macroblock having 1, MVs2, and MVs3. FIG. 2B shows a macroblock MB to be encoded and its neighboring macroblocks.
This indicates that there is a macroblock having the texture motion vectors MV1, MV2, and MV3.

In this case, when encoding the macroblock MB, the system of the present invention first performs the encoding process on the encoding target macroblock in the order of MVs1, MVs2, and MVs3. For the macroblock around the block,
It is checked whether or not a shape motion vector exists, and the shape motion vector that first exists is defined as MVPs.

Shape motion vectors MVs1, MVs2, M
If none of Vs3 is present, then MV1,
It is checked whether or not a texture motion vector exists in the order of MV2 and M3, and the motion vector that first exists is set as a prediction vector MVPs.

The texture motion vectors MV1, MV2,
If neither MV3 exists, the prediction vector MVP
Let s be a zero vector.

In the system of the present invention, the motion vector prediction means 505 has such a function.

As described in the above reference, the texture motion vectors (MV1, MV2, MV
There are cases where 3) can be used and cases where it cannot. For example,
A mode in which only the shape information is encoded without encoding the texture information, and an MV for the encoding target macroblock.
1, MV2, and MV3 do not exist.
That is, assuming that the texture motion vector is reliable, in this method, it can be said that the prediction accuracy of the prediction vector MVps is higher when the texture motion vector can be used than when it cannot be used.

(First Specific Example 1) Therefore, in this embodiment, as shown in FIG. 3, a search range is prepared for several stages, and when a texture motion vector can be used, the prediction accuracy of the prediction vector MVPs is The search range is switched according to. For example, a search range is prepared for several stages, such as “search range 1”, “search range 2”, and “search range 3”. The area size of “search range 1” is 4 × 4 pixels and “search range 1”. The area size of “range 2” is 8 × 8 pixels, and “search range 3”
Area size is 16 × 16 pixels.

In the configuration shown in FIG. 11, the functions of the shape motion vector detecting means 502 include the function of detecting the prediction accuracy of the prediction vector MVPs and the function of switching the motion vector search range according to the prediction accuracy. By improving to the added configuration, the amount of calculation of motion vector detection can be reduced without lowering the coding efficiency. The prediction accuracy is, for example, the motion vector prediction means 505
May be determined based on the magnitude of the error with respect to the predicted vector MVPs obtained in the above.

As a specific example of the motion vector search range,
If the texture motion vector is reliable, the search area 1 is the minimum area size, and if the texture motion vector is unreliable, the search area 2 is a somewhat larger area. Search range. If the texture motion vector cannot be used, the “search range 3”, which is the widest area, is searched as the search area.

That is, the shape motion vector detecting means 50
2 includes an alpha map signal (shape information A) input through an encoding area detection unit (Bounding-rectangle) that detects an encoding area that is an area where the target object exists in the frame image, and a motion. Vector prediction means 5
In detecting a motion vector from the information of the prediction vector obtained in step 05 and the shape information signal of the frame memory 506, the prediction accuracy of the prediction vector MVPs is obtained, and the shape motion is detected in the optimum motion vector search range switched accordingly. Operate to detect vectors. And
Thus, the detected vector is output as shape motion vector information.

As a result of checking the reliability based on the prediction accuracy of the texture motion vector when coding the macroblock, if the reliability is high, a narrow search range around the macroblock to be coded is determined. Can be used to complete the calculation of motion vector detection, and the amount of calculation required for motion vector detection can be reduced. When the texture motion vector is not reliable or when the texture motion vector cannot be used, the search range can be expanded to detect the motion vector.

FIG. 4 is a flowchart of this example.
That is, it is checked whether or not the texture motion vector can be used (step S11). As a result, if the texture motion vector cannot be used, the search range is set to “search range 3”.
If the result of the check in step S11 indicates that the texture motion vector can be used, it is next checked whether or not the texture motion vector is reliable (step S12). If the result is not reliable, the search range is set to "search range 2". It shall be.
As a result of the check in step S12, if it is reliable, the search range is set to "search range 1".

Here, the reliability of the texture motion vector is determined by the search range of the texture motion vector. That is, since the detection of the texture motion vector directly detects the texture motion vector itself, it is possible to follow a large motion when the search range is wide. Therefore, it can be estimated that the reliability is higher than when the search range is narrow. In MPEG4, the search range of the texture motion vector can be extended to ± 1024.

In this way, the prediction vector MV
The search range can be switched according to the prediction accuracy of Ps. By performing the search range switching according to the prediction accuracy of the prediction vector MVPs, the motion vector detection can be performed without lowering the coding efficiency. The amount of calculation can be reduced.

(First Specific Example 2) Prediction Vector MV
When Ds is a zero vector, as described above, information indicating that the prediction vector MVDs is a zero vector is incorporated into mode information and separately encoded to improve encoding efficiency.

Therefore, in the conventional model encoding system for MPEG4 shown in FIG. 12, a predetermined threshold value is set as a comparison criterion so that a zero vector can be easily detected. The error (MC error) is compared with this threshold value, and when the motion compensation prediction error (MC error) at the time of the zero vector is smaller than the threshold value, detection is stopped at that time, and the prediction vector MVPs is shaped. The motion vector is MVs.

In this specific example, the above-mentioned truncation is further extended. For example, when the motion compensation prediction error in the optimal motion vector when detecting up to “search range 1” in FIG. Is smaller, the detection is terminated at that time. When the motion compensation prediction error is larger than the predetermined threshold, the search range is extended to “search range 2”, and the remaining range of “search range 2” is changed in the same manner as in “search range 1”. If the motion compensation prediction error with the optimum motion vector at the time of the search is smaller than a predetermined threshold, the detection is terminated at that time.

That is, the search range is switched in accordance with the prediction accuracy of the prediction vector MVPs, and the search range is switched in accordance with the prediction accuracy of the prediction vector MVPs. The calculation amount of vector detection is reduced.

FIG. 5 is a flowchart of this example. That is, the prediction vector MVDs is initialized to "0" (step S1), and it is checked whether or not the MC (motion compensation) error is larger than a threshold (step S2). As a result, if it is smaller, the process is terminated, and if it is larger, a search is performed within "search range 1" (step S3).

Then, it is checked whether or not the MC (motion compensation) error is larger than a threshold (step S4). As a result, if it is smaller, the process is terminated, and if it is larger, a search is performed within "search range 2-search range 1" (step S5).

Then, it is checked whether or not the MC (motion compensation) error is larger than a threshold (step S6). As a result, if it is smaller, the process is ended, and if it is larger, the search within “search range 3−search range 2” is performed (step S7).

In this way, the prediction vector MV
The search range can be switched according to the prediction accuracy of Ps. By performing the search range switching according to the prediction accuracy of the prediction vector MVPs, the motion vector detection can be performed without lowering the coding efficiency. The amount of calculation can be reduced.

As described above, since there is a high possibility that the optimum value of the difference vector MVDs is near the zero vector, the search range is expanded stepwise from a narrow search range, and when a predetermined condition is satisfied in the middle. By terminating the search, the amount of calculation for motion vector detection can be reduced without lowering the coding efficiency.

In both “Specific Example 1” and “Specific Example 2”, since a vector having a smaller size is selected as compared with the MPEG4 model encoder in FIG. 11, the code amount of the motion vector is reduced. As a result, the coding efficiency may be improved.

By combining the techniques shown in "Specific example 1" and "Specific example 2", the calculation amount of motion vector detection can be further reduced.

The technique described above is a technique in which the amount of calculation in detecting a motion vector can be reduced by performing motion vector detection using a texture motion vector in MPEG4.

Next, texture coding in a “boundary macroblock” will be described.

<Second Specific Example> Next, a second specific example of the present invention will be described. This specific example relates to texture coding in a “boundary macroblock” in which an object is included in a part of a macroblock.

In FIG. 6, a coding area (Bounding-Rectangle) detecting means 3 which is a component highlighted in the figure
01, reference image padding means 302, LPE padding means 303a, zero padding means 304, vector padding means 306, and shape information encoding means 500
Is a component necessary only for encoding an object of an arbitrary shape. Therefore, MPEG1 and MPEG2
In order to encode a rectangular object, as in the conventional encoding method as described above, the other components, that is, the motion vector detecting means 305, the frame memory 307, the motion compensating means 308, the switching Switch 3
09, 310, motion vector prediction means 311, motion vector storage means 312, difference circuit 313, third variable length coding means 314, quantization means 315, inverse quantization means 31
6, orthogonal transform means 317, fourth variable length coding means 31
8. It is sufficient that an element composed of the inverse orthogonal transform means 319, the adder circuit 320, the switch 321, and the multiplexer 322 is provided.

Here, the motion vector detection means 305
Between the LPE padded macroblock data output from the LPE padding means 303, the reproduced value data 54 of the shape information output from the shape information encoding means 500, and the luminance signal of the reference image supplied from the frame memory 307. Perform motion vector detection. Then, as a result, the obtained motion vector is output to the motion compensation means (MC) 308, the vector padding (Vector padding) means 306, and the difference circuit 313.

The frame memory 307 stores the luminance signal of the padded reference image output from the reference image padding means 302. Also,
The motion compensating unit 308 includes a motion vector detected by the motion vector detecting unit 305 and the shape information encoding unit 50.
This is for obtaining motion vector information for motion compensation prediction using the reproduced value data 54 of the shape information output as 0, and the changeover switches 309 and 310 switch the output of the LPE padding means 303 to the zero padding means 304. Path switching switch for selectively switching between giving to the user and detouring.

The orthogonal transform means 317 is for performing a process of orthogonally transforming (discrete cosine transform) the output provided via the changeover switch 310 and decomposing the output into frequency components. Orthogonal transformation means 31
7 is output after being quantized, and the fourth variable-length encoding unit 318 performs variable-length encoding on the quantized output and outputs the result as a texture stream to the multiplexer 3.
22.

The motion vector predicting means 311 uses the reproduced value data 54 of the shape information output from the shape information encoding means 500 and the data held by the motion vector storing means 312 to predict a motion vector. The difference circuit 313 includes a motion vector prediction unit 31
1 to obtain a difference value between the motion vector prediction value output by the motion vector detector 1 and the motion vector output by the motion vector detection means 305. The third variable length encoding means 314 changes the output of the difference circuit 313. The data is subjected to long coding processing and output to the multiplexer 322 as a motion stream.

The inverse quantization means 316 includes a quantization means 315
The inverse orthogonal transform means 319 performs inverse orthogonal transform (inverse discrete cosine transform) on the data output from the inverse quantization means 316. To return to the original data at the time of the zero padding processing.

The multiplexer 322 includes a variable-length encoded motion stream output from the third variable-length encoding unit 314 and a variable-length encoded texture stream output from the fourth variable-length encoding unit 318. Receiving the information of the coding area output from the coding area (Bounding-Rectangle) detection means 301 and the reproduction value data 54 of the shape information output from the shape information coding means 500, and multiplexing and outputting them. It is.

The inverse orthogonal transform means 319 is for performing inverse orthogonal transform (inverse discrete cosine transform) on the data output from the inverse quantization means 316 to return to the original data at the time of the zero padding processing. is there.

The addition circuit 320 includes a switch 321
To add the motion vector information from the motion compensating means 308 provided via the 補償 and the original data at the time of the zero padding processing provided from the inverse orthogonal transform means 319, and to provide the resulting data to the reference image padding means 302. The motion vector storage unit 312 stores the vector padding unit 3
Based on the reproduced value data 54 of the shape information output from the block 06, data obtained by filling a block (transparent block or intra macro block) in 8 × 8 pixels without a motion vector with an appropriate motion vector is received and stored. Things.

Here, the difference between the encoder of this embodiment and the mechanism of the conventional encoder will be described. A conventional encoder is, as shown in FIG.
Reference numeral 5 is such that the motion vector 34 is obtained using the input texture information (YUV), the detection output from the coding area (Bounding-Rectangle) detection means 301 and the storage information from the frame memory 307. Here, the encoding area (Bounding-Rectangle) detecting means 301 is for detecting an encoding area which is an area where the target object exists in the frame image.

Further, the LPE padding means 303 is placed in parallel with the zero padding means 304, and the input texture information (YUV) is given to one of them according to the conditions to perform padding, and the padded output is subjected to orthogonal transform means. 317.

According to the present invention, as shown in FIG.
In the first stage, the E padding means 303 performs LPE padding processing on the input texture information (YUV) irrespective of the condition, and supplies the result to the motion vector detecting means 305 and the zero padding means 304 according to the condition. Here, LPE padding refers to, as described above, a block in an intra macroblock is DC
Before T (discrete cosine transform), the pixel values outside the object (open circles in FIG. 13) are replaced with the average values of the pixels inside the object (black circles in FIG. 13), and a low-pass filter is applied. The processing unit is 8 × 8 pixels as in the case of DCT. Further, zero padding is a process of replacing a pixel (open circle portion in FIG. 13) outside the object of the motion compensation prediction error signal with a zero value before performing DCT on a block in the inter macro block. As in the case of DCT, it is 8 × 8 pixels. These LPE padding and zero padding
Although not an essential process in the MPEG4 standard, it is performed to improve coding efficiency.

The block configuration of this example is as shown in FIG. In the configuration of this specific example, the processing position of the motion vector detecting unit 305 is different from the position of the motion vector detecting unit 305 in the configuration of the conventional model shown in FIG. 12, and the processing position of the LPE padding 303 is Different from the position of LPE padding 303.

<LPE padding means> First, the difference between the processing by the LPE padding means 303a in the system of the present embodiment and the processing by the LPE padding means 303 in the system according to the prior art will be described.

The padding processing by the LPE padding means 303a is performed only on the boundary macroblock of the original signal (YUV) 41 of the texture image supplied via the signal line. Here, whether or not the macroblock is a boundary macroblock is determined by using the original signal 42 of the shape image supplied via the coding area (Bounding-Rectangle) detection means 301. That is, the processing by the LPE padding means 303a is performed by performing LPE padding as preprocessing for encoding.

By performing the preprocessing, the texture image 4 output from the LPE padding means 303a is output.
No. 3 means that all pixel values in the macroblock to be encoded are supplemented before encoding.

In the LPE padding means 303 in the model system as the prior art, an 8 × 8 pixel block is formed based on the local decoded signal of the shape image supplied from the shape information coding means 500 via the signal line 35. If it is a boundary block, padding is applied.

In this specific example, padding is performed in units of 8 × 8 pixels, as shown in the upper left block of FIG.
If the block does not include the pixels in the object,
For example, by padding with the average value of surrounding blocks, all pixels in the macroblock are padded.

As described above, the first example is different from the conventional model system in that padding is performed based on the original signal of the shape image or padding is performed based on the locally decoded signal.

Here, in order to specifically show the effect of the method of “padding based on the original signal of the shape image” according to the present invention, an example in which LPE padding is applied to a one-dimensional signal model will be described. . The luminance signal is Y, the shape signal is A,
The local decoded signal of the shape (No. 1) is A ′, and the local decoded signal of the shape (No. 2) is A ″. It is assumed that these are the following signals, respectively.

In the following example, for simplicity, the shape signal is represented by “0” (outside the object) or “1” (inside the object).

Y = {50,50,50,50,180,190,200,210} A = {0,0,0,0,1,1,1,1,1} A ′ = {0,0,0,1,1,1 , 1,1} A ″ = {0,0,0,0,0,1,1,1} Here, the result of LPE padding of the luminance signal Y with the above-mentioned A, A ′, A ″ is Yp, Assuming that Yp ′ and Yp ″, for example, the following is performed.

Yp = {195,195,195,188,180,190,200,210} Yp ′ = {166,166,108,50,180,190,200,210} Yp ″ = {200,200,200,200,195,190,200,210} That is, in the case of Yp, the original Y data “50”,
“50”, “50”, “50”, “180”, “190”, “20
The data strings “0” and “210” are padded as a result of “19”.
5 ”,“ 195 ”,“ 195 ”,“ 188 ”,“ 180 ”,“ 190 ”,
The data values are "200" and "210". In the case of Yp ', the data values are "166", "166", "108", and "5".
0 "," 180 "," 190 "," 200 ", and" 210 ". In the case of Yp", the data value is "200",
“200”, “200”, “200”, “195”, “190”, “20”
0 ”and“ 210. ”The numerical values shown here are“ 0 ”(white) for the lower limit and“ 255 ”for the upper limit.
It is a gray scale value of 256 steps to be (black).

In general, the boundary of an object is the edge of an image, and the luminance signal Y greatly fluctuates as in the above example. In the processing by the shape information encoding unit 500, if an error occurs in the locally decoded signal of the shape, padding may not be performed smoothly as in the above example (Yp). Therefore, when texture coding is performed on a “boundary macroblock”, if a signal including such an error must be used, a problem that edges become unclear arises.

However, in the case of the method of the present invention, the shape signal is set to “0” (outside the object) and “1” (inside the object).
By doing so, such worries are completely eliminated.

<Motion Vector Detecting Means> Next, differences between the processing of the motion vector detecting means 305a in the system of the present embodiment and the processing of the motion vector detecting means 305 in the system of the conventional model will be described. In the motion vector detecting means 305a in the present system, the reproduction value data 54 of the shape information output from the shape information encoding means 500 and supplied via the signal line 44, and the padded data output from the LPE padding means 303a , A motion vector for a macroblock other than the transparent macroblock is detected.

The original image data (padding-processed data) supplied from the LPE padding means 303a via the signal line 45 and the signal line 46 from the frame memory 307.
, And a motion vector is detected between the reference image supplied through the signal line 47 and a vector padding means 306 via a signal line 47.
To the motion compensating means 308 and the difference circuit 313.

Here, in the original image supplied via the signal line 45, the boundary macroblock is already padded by the LPE padding 403. Therefore, there is almost no discontinuity in the pixel value at the object boundary.

Therefore, unlike the case of the motion vector detecting means 305 in the conventional model system, the motion vector detecting means 305a of the system of the present invention does not need to perform the polygon matching with a large processing amount in the "boundary macroblock". Even when ordinary block matching is performed, a normal motion vector can be detected.

Various embodiments have been described above. In short, the present invention is, first, for a desired object constituting an image, the desired object is constituted by shape information and texture information in macroblock units. A motion vector detecting means for detecting a shape motion vector to be used for motion compensation prediction and a motion vector of texture information from shape information of an input object in a coding apparatus of a moving image coding system for coding as an arbitrary shape object. Identification means for identifying whether or not a motion vector of the texture information can be used from input texture information; evaluation means for evaluating the reliability of the detected motion vector in the texture information; and a search range of the shape motion vector. Can be used when motion vector of texture information is not possible Is wider set, when the motion vector of texture information is available are those having a setting unit for lower than the reliability of the motion vector of texture information are set widely. And
In this apparatus, when encoding an arbitrary shape object composed of shape information and texture information, the motion vector detecting means has a function of detecting a motion vector of texture information so that a motion vector of texture information can be used. Of the input object's shape information and texture information to identify whether or not the motion vector of the texture information can be used by the identification means,
The evaluation unit evaluates the reliability of the motion vector of the texture information, and when coding a macroblock to be coded, searches for a shape motion vector of a macroblock around the macroblock and calculates the shape motion vector. The search range is narrow when the motion vector of the texture information is available, and wider when it is not available, and the motion vector of the texture information is available. In such a case, the lower reliability is set wider than the higher reliability of the motion vector of the texture information so that the use of the motion vector of the texture information and the degree of the reliability depend on the macroblock. The shape motion vector search range is set to an appropriate range. That is, the present invention relates to motion vector detection, and as a result of this, it is possible to provide a technique that can reduce the calculation amount of motion vector detection without lowering the coding efficiency.

Second, the present invention relates to a moving picture code for encoding a desired object constituting an image as an arbitrary-shaped object composed of shape information and texture information in macroblock units. A coding apparatus of a coding method, comprising a motion vector detecting means for detecting a shape motion vector and a motion vector of texture information from input shape information of an object, Threshold setting means for setting, dividing means for dividing the search range into a plurality of search ranges, and determination means for determining whether a motion compensation prediction error due to the detected motion vector is larger than a threshold. Wherein the motion vector detecting means includes a motion vector from a narrowest search range among the divided search ranges. If the motion compensation prediction error due to the optimal motion vector detected in the search range is larger than the threshold, the optimal motion vector is detected in a wider search range, and the motion compensation prediction error Is smaller than the threshold value, the motion vector detection is terminated, and the motion vector is output as a detection result.

In MPEG4, when the prediction vector is a zero vector, the coding efficiency is improved by incorporating information indicating that the prediction vector is a zero vector into the mode information and separately coding the mode information. The ability to efficiently detect the zero vector also greatly contributes to the reduction in the amount of computation. Therefore, in the present invention, a threshold value setting unit is provided to set a threshold value for designating an error at the time of encoding, and the dividing unit divides the search range into a plurality of search ranges. I do. Then, the motion vector detection means starts the motion vector detection from the narrowest search range among the divided search ranges, and the determination means performs the detection based on the motion vector detected by the motion vector detection means. It is determined whether the motion compensation prediction error based on the obtained motion vector is larger than a threshold value. Then, as a result of starting the motion vector detection from the narrowest search range, the determination unit determines that the motion compensation prediction error due to the optimal motion vector detected in the search range is larger than the threshold value. If it is determined, the operation is performed to detect the optimal motion vector in a wider search range. If it is determined that the motion compensation prediction error is smaller than the threshold value, the motion vector detection is terminated, and the motion vector Is output as the detection result. As described above, when the motion compensation prediction error due to the optimal motion vector detected in the search range is large, the optimal motion vector is detected in a wider search range, and when the motion compensation prediction error is small, the motion vector detection is performed. Is ended, and the motion vector is output as a detection result. As a result, if the error is small, the zero vector can be efficiently detected with a small amount of calculation, which greatly contributes to a reduction in the amount of calculation.

Thirdly, the present invention provides a moving picture encoding system which encodes a desired object constituting an image as an arbitrary-shaped object composed of shape information and texture information on a macroblock basis. Prior to performing shape coding and texture coding in a coding apparatus of the system, a boundary macroblock of a form including an object is referred to by referring to a shape signal input to the coding means. Has a means for compensating for all the pixels in the macroblock to be texture-encoded by performing padding processing for eliminating discontinuities in the inside, and the shape encoding and texture encoding have the padding processing. This is a configuration using images.

The present invention relates to texture coding in a "boundary macroblock". In an original image, for a boundary macroblock, padding processing for eliminating discontinuity of pixel values at an object boundary portion,
That is, padding processing is performed by LPE padding, and then subjected to shape encoding and texture encoding. Therefore, discontinuity of the pixel value at the object boundary is almost eliminated, so that a normal motion vector can be detected even when ordinary block matching is performed. Therefore, it is not necessary to perform polygon matching with a large amount of processing in the “boundary macroblock”, so that the calculation time can be reduced, and a normal motion vector can be detected even when ordinary block matching is performed. become able to.

The present invention is not limited to the above-described embodiment, but can be implemented with appropriate modifications within a scope that does not change the gist.

[0165]

As described above in detail, according to the present invention, the coding efficiency can be improved without increasing the processing amount by changing the processing order of the components required for coding the arbitrary-shaped object. Can be achieved. In addition, by adaptively terminating the shape motion vector detection processing, the processing amount can be reduced without lowering the coding efficiency.

[Brief description of the drawings]

FIG. 1 is a view for explaining the characteristics of the frequency distribution of difference vectors MVDs.

FIG. 2 is a view for explaining a method for obtaining prediction vectors MVPs.

FIG. 3 is a diagram illustrating a search range of a shape motion vector according to the present invention.

FIG. 4 is a diagram for explaining the present invention, and is a flowchart showing a processing example in the first specific example 1 of the present invention;

FIG. 5 is a diagram for explaining the present invention, and is a flowchart showing a processing example in the first specific example 2 of the present invention;

FIG. 6 is a diagram for explaining the present invention, and is a block diagram showing a configuration example of an encoder of the present invention.

FIG. 7 is a block diagram of an image encoding apparatus that encodes an image by dividing the screen into a background and an object when encoding the image.

FIG. 8 is a block diagram of a decoding device.

FIG. 9 is a diagram for explaining the present invention, and is a diagram for explaining a coding area including an object.

FIG. 10 is a diagram for explaining the present invention and is a diagram for explaining attributes of each macroblock.

FIG. 11 is a block diagram of a shape encoding encoder (Binary Shape encoder 500) in a model system as a conventional technique.

FIG. 12 is a block configuration diagram of an encoder in a model system as a conventional technique.

FIG. 13 is a diagram illustrating pixels in an object and pixels outside the object.

FIG. 14 is a diagram illustrating LPE padding of the present invention.

[Explanation of symbols]

 301: Bounding-Rectangle detection means 302: Reference image padding means 303 ... LPE padding means 304 ... Zero padding means 306 ... Vector padding means 500 ... Binary Shape encoders 305, 305a ... Motion Vector detecting means 307 frame memories 308, 308a motion compensating means 309, 310, changeover switch 312 motion vector storing means 313 difference circuit 314 third variable length coding means 315 quantizing means 316 inverse quantum Means 317 ... orthogonal transform means (DCT) 318 ... fourth variable length coding means 319 ... inverse orthogonal transform means 320 ... adder circuit 321 ... switch 322 ... multiplexer

Claims (10)

[Claims] An encoding apparatus of a moving image encoding system for encoding a desired object constituting an image as an arbitrary-shaped object composed of shape information and texture information in macroblock units. Means for detecting a motion vector of texture information for each macroblock; determining means for determining whether a motion vector of texture information can be used to detect a motion vector of shape information in the macroblock; Evaluation means for evaluating the reliability of the motion vector of the texture information in order to detect the motion vector of the shape information, and a motion vector detection range of the shape information which is wider when the motion vector of the texture information is unavailable than when it is available Is set and the motion vector of the texture information is Setting means that is set wider when the reliability of the motion vector of the texture information is low than when the reliability is high, and using the motion vector of the already detected texture information in the macroblock. A moving image encoding apparatus comprising: a motion vector detecting unit configured to detect a motion vector of shape information. 2. The moving picture coding apparatus according to claim 1, wherein the reliability of the motion vector of the texture information is evaluated based on a search range of the motion vector of the texture information. Device. 3. A moving picture coding method for coding a desired object constituting an image as an arbitrary-shaped object composed of shape information and texture information in macroblock units. A means for detecting a motion vector of texture information for each macroblock; a threshold setting means for setting a threshold for designating an error at the time of encoding; and a search range divided into a plurality of search ranges And a determining unit for determining whether a motion compensation prediction error due to the detected motion vector is larger than a threshold value. The means for detecting a motion vector includes: Motion vector detection is started from the narrowest search range, and the motion vector detected by the optimal motion vector detected within the search range is determined. If compensation prediction error is greater than the threshold value, it detects the optimal motion vector with a wider search range when the motion compensated prediction error is smaller than the threshold value,
A moving picture encoding apparatus having a function of ending a motion vector detection and outputting the motion vector as a detection result. 4. A moving picture coding method for coding a desired object constituting an image as an arbitrary-shaped object composed of shape information and texture information in macroblock units. Prior to encoding and texture encoding, a boundary macroblock of a form partially including an object is referred to the shape signal input to the encoding means to eliminate discontinuity in the macroblock. A means for performing a padding process to compensate all the pixels in the macroblock to be texture-encoded by performing the padding process, and performing the shape encoding and the texture encoding using the padded image. A moving image encoding apparatus. 5. A moving picture coding apparatus according to claim 4, wherein said motion vector detecting means detects a shape motion vector to be used for motion compensation prediction and a motion vector of texture information from the input shape information of the object. In addition, when detecting the motion vector of the texture information, the motion vector detecting means is configured to detect between the macroblock already compensated by the padding process and the reference image padded with the reference image. A video encoding device characterized by the above-mentioned. 6. A moving picture coding method for coding a desired object constituting an image as an arbitrary-shaped object composed of shape information and texture information in macroblock units. An identification step of determining whether or not the motion vector of the texture information can be used from the texture information to be obtained; an evaluation step of evaluating the reliability of the detected motion vector in the texture information; A setting that sets the search range of the shape motion vector to be set wider when the motion vector of the texture information is available and lower when the reliability of the motion vector of the texture information is higher when the motion vector of the texture information is available. Step and input object shape information A motion vector detecting step of detecting a shape motion vector to be used for motion compensation prediction from the report within the search range. 7. A moving image encoding method for encoding a desired object constituting an image as an arbitrary-shaped object composed of shape information and texture information in macroblock units. A threshold value for designating an error at the time of conversion, dividing the search range into a plurality of search ranges, and calculating a shape motion vector for use in motion compensation prediction from shape information of an input object. It is determined whether or not the motion compensation prediction error due to the detected motion vector is larger than a threshold value, and the motion vector is detected from the narrowest search range among the divided search ranges. Vector detection is started, and a motion compensation prediction error due to an optimal motion vector detected within the search range is a threshold. If the value is larger than the value, the optimum motion vector is detected in a wider search range. If the motion compensation prediction error is smaller than the threshold, the motion vector detection is terminated and the motion vector is output as a detection result. A moving image encoding method. 8. A moving picture encoding method for encoding a desired object constituting an image as an arbitrary-shaped object composed of shape information and texture information in macroblock units. Prior to encoding and texture encoding, a boundary macroblock of a form partially including an object is referred to the shape signal input to the encoding means to eliminate discontinuity in the macroblock. Moving image wherein all pixels in a macroblock to be texture-encoded are compensated by performing padding processing, and shape encoding and texture encoding are performed using the padded image. Image coding method. 9. A moving picture coding apparatus for coding a desired object constituting an image as an arbitrary-shaped object composed of shape information and texture information in macroblock units, wherein a motion vector of the texture information is defined for each macroblock. Detecting means; determining means for using a motion vector of texture information to detect a motion vector of shape information in the macroblock; and means for evaluating reliability of the motion vector of texture information. If the motion vector of the texture information is highly reliable and the motion vector of the texture information can be used to detect the motion vector of the shape information, the detection range of the motion vector of the shape information is limited. Video encoding device. 10. A moving image encoding method for encoding a desired object constituting an image as an arbitrary-shaped object composed of shape information and texture information in macroblock units, wherein a motion vector of texture information is defined for each macroblock. In addition to determining whether or not to use the motion vector of the texture information in order to detect the motion vector of the shape information in the macro block, the reliability of the motion vector of the texture information is evaluated. Moving image coding characterized by limiting the detection range of the motion vector of the shape information when the reliability of the vector is high and the motion vector of the texture information can be used to detect the motion vector of the shape information. Method.