JP2005094803A - Apparatus and method for motion image coding - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To attain reduction of processing amount and improvement of coding efficiency in an arbitrary shaped object coding. <P>SOLUTION: An arbitrary shaped object coding means 502 consisting of shape information and texture information is a shape information coding means having a motion vector detecting means capable of also utilizing a motion vector of the texture information. This means has a means of discriminating whether or not the motion vector of the texture information can be used; a means of evaluating the reliability of the motion vector of the texture information; and a means in which a range of searching a shape vector is set larger when the motion vector of the texture information can be utilized than when it cannot be utilized, and the searching range is set larger when the reliability of the motion vector of the texture information is high than when it is low, if the motion vector of the texture information can be utilized. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention is a moving picture using a function for individually encoding an object of arbitrary shape, which is realized by the MPEG4 international standard system for encoding moving images, which is being standardized in ISO / IEC JTC / SC29 / WG11. The present invention relates to an image encoding device and a moving image encoding method.

  In MPEG4 (Motion Picture Experts Group phase 4), where standardization work is currently in progress, in MPEG1 (Motion Picture Experts Group phase 1) and MPEG2 (Motion Picture Experts Group phase 2), which are international standard systems for conventional video coding, A function that cannot be realized, and a function that individually encodes each object of an arbitrary shape (for example, a person shown in the screen) is to be realized.

  In order to realize this function, information (shape information) representing the shape and size of each object is required, and this information is encoded with texture information representing changes in luminance and color difference inside the object. , Transmitted and stored.

  That is, if there is an image composed of a background and an object, the image is divided into a background and an object and encoded. In order to separately encode the background and the object, information (shape information) indicating the shape and size of each object is separately provided. This is a shape information signal called an alpha map signal.

  This alpha map signal is given as, for example, binary sub-image information representing the shape of the object and the position in the screen (note that the alpha map signal of the background is uniquely obtained from the alpha map signal of the object).

  In this way, MPEG4 encodes information (shape information) representing the shape and size of each object together with texture information representing changes in luminance and color difference inside the object, thereby reducing the amount of information and decoding side. Based on this, a function that can reproduce an object can be realized.

  Here, an outline of an MPEG4 image encoding and image decoding apparatus will be described. FIG. 7 is a block configuration diagram of an image coding apparatus that divides the screen into a background and an object and codes the image when coding the image. As shown in FIG. 7, the image coding apparatus includes a difference circuit 1100, a motion compensation prediction circuit (MC) 1110, an orthogonal transform circuit (DCT) 1120, a quantization circuit 1130, a variable length coding circuit (VLC) 1140, The circuit includes an inverse quantization circuit (IQ) 1150, an inverse orthogonal transform circuit (IDCT) 1160, an adder circuit 1170, a multiplexing circuit 1180, and an alpha map encoding circuit 1200.

  The alpha map encoding circuit 1200 encodes the input alpha map and outputs the encoded signal to the multiplexing circuit 1180 as an alpha map signal, and decodes the alpha map signal as a local decoded signal. Has a function to output.

  In particular, the present alpha map encoding circuit 1200 performs a process of reducing the resolution at a given reduction rate (magnification) when encoding the input alpha map, and encodes the resolution reduced process. At the same time, it has a function of multiplexing the encoded data and reduction rate information (magnification information) and outputting them to the multiplexing circuit 1180 as an alpha map signal. As the local decoded signal, a signal obtained by performing processing for returning the resolution-reduced signal to the original resolution is used.

  The difference circuit 1100 calculates a difference signal between the motion compensation prediction signal supplied from the motion compensation prediction circuit 1110 and the input image signal, and the orthogonal transformation circuit 1120 converts the difference signal supplied from the difference circuit 1100 into the difference signal. According to the information of the alpha map, it is converted into an orthogonal transform coefficient and output.

  The quantization circuit 1130 is a circuit that quantizes the orthogonal transform coefficient obtained by the orthogonal transform circuit 1120, and the variable length coding circuit 1140 encodes and outputs the output of the quantization circuit 1130. The multiplexing circuit 1180 multiplexes and multiplexes the signal encoded by the variable length encoding circuit 1140 and the alpha map signal together with side information such as motion vector information and outputs the result as a bit stream.

  The inverse quantization circuit 1150 performs inverse quantization on the output of the quantization circuit 1130, and the inverse orthogonal transform circuit 1160 performs inverse orthogonal transform on the output of the inverse quantization circuit 1150 based on the alpha map. The adder circuit 1170 adds the output of the inverse orthogonal transform circuit 1160 and the prediction signal (motion compensation prediction signal) given from the motion compensation prediction circuit 1110 and outputs the result to the difference circuit 1100.

  The motion compensation prediction circuit 1110 has a frame memory and has a function of accumulating the object region signal and the background region signal by operating based on the local decoded signal supplied from the alpha map encoding circuit 1200. The motion compensation prediction circuit 1110 also predicts a motion compensation value from the accumulated image of the object region and outputs it as a prediction value, and also predicts a motion compensation value from the image of the accumulated background region and outputs it as a prediction value. Have

  The operation of the apparatus having such a configuration will be described. The apparatus receives an image signal and an alpha map of the image signal. Of these, the image signal is divided into blocks each having a predetermined pixel size (for example, M × N pixels (M: number of pixels in the horizontal direction, N: number of pixels in the vertical direction)) for each frame, and then the blocks The signals are supplied to the difference circuit 1100 via the signal line 1010 in the order of position. Then, the difference circuit 1100 calculates a difference signal between this input (image signal) and the prediction signal (output of the motion compensation prediction signal from the object prediction circuit 1110) and supplies the difference signal to the orthogonal transformation circuit 1120.

  In the orthogonal transform circuit 1120, the supplied differential signal is converted into an orthogonal transform coefficient according to the information of the alpha map supplied from the alpha map encoding circuit 1200 via the signal line 1040, and then supplied to the quantization circuit 1130. To do. And it is quantized here. The transform coefficient obtained by quantization in the quantization circuit 1130 is encoded in the variable length encoding circuit 1140 and supplied to the inverse quantization circuit 1150.

  The transform coefficient supplied to the inverse quantization circuit 1150 is inversely quantized here and then inverse transformed in the inverse orthogonal transform circuit 1160. Then, the addition circuit 1170 adds the motion compensation prediction value supplied from the motion compensation prediction circuit 1110, outputs it as a locally decoded image, and inputs it again to the motion compensation prediction circuit 1110.

  Then, the locally decoded image that is the output of the addition circuit 1170 is stored in a frame memory in the motion compensation prediction circuit 1110.

  On the other hand, the motion compensation prediction circuit 1110 outputs an “object motion compensation prediction value” at the processing timing of the block in the object region based on the local decoded signal supplied from the alpha map decoding circuit 1200, and other than that. At the timing, the “background portion motion compensation prediction value” is output and supplied to the difference circuit 1100.

  That is, in the motion compensation prediction circuit 1110, whether the image signal of the block corresponding portion of the object is currently input to the difference circuit 1100 from the local decoded signal of the alpha map signal or the image signal of the block corresponding portion of the background portion is the difference circuit. If it is during the input period of the image signal of the block corresponding part of the object, the motion compensation prediction signal of the object and the image signal input period of the block corresponding part of the background part may be detected. For example, the background motion compensation prediction signal is supplied to the difference circuit 1100.

  The difference circuit 1100 calculates the difference between the input image signal and the prediction signal corresponding to the region of the image. As a result, if the input image is in the region corresponding to the object, the corresponding position of the object If the difference signal from the predicted value in FIG. 4 is a background region, the difference signal from the predicted value corresponding to the background position is calculated and supplied to the orthogonal transformation circuit 1120.

  The orthogonal transform circuit 1120 converts the supplied difference signal into an orthogonal transform coefficient by performing processing such as discrete cosine transform according to the information of the alpha map supplied via the signal line 1040, and then the quantization circuit. 1130. The orthogonal transform coefficient is quantized by the quantization circuit 1130.

  The transform coefficient quantized by the quantization circuit 1130 is encoded by the variable length encoding circuit 1140 and supplied to the inverse quantization circuit 1150. The transform coefficient supplied to the inverse quantization circuit 1150 is inversely quantized here, and then inverse transformed in the inverse orthogonal transform circuit 1160 and supplied to the adder circuit 1170. Then, it is added to the predicted value supplied to the adding circuit 1170 via the predicted value switching circuit 1500.

  The signal of the locally decoded image that is the output of the adder circuit 1170 is supplied to the motion compensation prediction circuit 1110. In this motion compensated prediction circuit 1110, whether the signal corresponding to the block of the object is currently output from the local decoding signal of the alpha map signal, or whether the signal corresponding to the block of the background portion is output. As a result, if the signal corresponding to the block of the object is being output, it is applied to the frame memory for the object, and if the signal corresponding to the block of the background portion is being output, the operation is performed to the background memory. And store it in the corresponding memory.

  Thus, only the object image is obtained in the object frame memory, and only the background image is obtained in the background memory. Accordingly, the motion compensation prediction circuit 1110 can obtain the predicted value of the object image using the object image, and can obtain the predicted value of the background image using the image of the background portion.

  As described above, the alpha map encoding circuit 1200 encodes an input alpha map and supplies the encoded alpha map signal to the multiplexing circuit 1180 via the signal line 1030.

  Also, the transform coefficient output from the variable length coding circuit 1140 is supplied to the multiplexing circuit 1180 via a line 1040. Then, the multiplexing circuit 1180 multiplexes the supplied alpha map signal and the encoded value of the transform coefficient together with side information such as motion vector information, and then outputs it through the signal line 1050 to output the main image code. The encoded bit stream is the final output of the encoding device.

  On the other hand, FIG. 8 is a block diagram of the decoding apparatus. As shown in FIG. 8, the decoding apparatus includes a separation circuit 2300, a variable length decoding circuit 2310, an inverse quantization circuit 2320, an inverse orthogonal transform circuit 2330, an addition circuit 2340, a motion compensation prediction circuit 2350, an alpha map decoding. A circuit 2400 is included.

  The separation circuit 2300 is a circuit that obtains an alpha map signal and a coded image signal by separating the input encoded bit stream, and the alpha map decoding circuit 2400 is separated by the separation circuit 2300. This circuit decodes the alpha map signal and reproduces the alpha map.

  The variable length decoding circuit 2310 decodes the encoded signal of the image separated by the separation circuit 2300, and the inverse quantization circuit 2320 inversely quantizes the decoded signal to return it to the original coefficient. The inverse orthogonal transform circuit 2330 performs inverse orthogonal transform on this coefficient according to the alpha map to return it to the prediction error signal, and the adder circuit 2340 converts the motion compensation from the motion compensation prediction circuit 2350 into the prediction error signal. The predicted value is added and output as a reproduced image signal. This reproduced image signal becomes the final output of the decoding apparatus.

  The motion compensation prediction circuit 2350 obtains the object image and the background image by accumulating the reproduced image signal output from the adder circuit 2340 in the frame memory according to the alpha map, and obtains the object image from the accumulated image. A motion compensation prediction signal and a background motion compensation prediction are obtained.

  In the decoding apparatus having such a configuration, the encoded bit stream is supplied to the separation circuit 2300 via the line 2070, and is separated for each piece of information by the separation circuit 2300, thereby relating to the alpha map signal. It is divided into a code and a variable length code of the image signal.

  The code relating to the alpha map signal is supplied to the alpha map decoding circuit 2400 via the signal line 2080, and the variable length code of the image signal is supplied to the variable length decoding circuit 2310, respectively.

  The code relating to the alpha map signal is reproduced as an alpha map signal by the alpha map decoding circuit 2400 and output to the inverse orthogonal transform circuit 2330 and the motion compensation prediction circuit 2350 via the signal line 2090.

  On the other hand, the variable length decoding circuit 2310 decodes the code supplied from the separation circuit 2300 and supplies it to the inverse quantization circuit 2320 where it is inversely quantized. The inversely quantized transform coefficient is inversely transformed by the inverse orthogonal transform circuit 2330 according to the alpha map supplied via the line 2090 and supplied to the adder circuit 2340. The adder circuit 2340 adds the inverse orthogonal transform signal from the inverse orthogonal transform circuit 2330 and the motion compensation prediction signal supplied from the motion compensation prediction circuit 2350 to obtain a reproduced image.

  The above is the outline of the MPEG4 image encoding device and image decoding device.

  Here, encoding of the alpha map will be described. The alpha map, that is, the pixel value of the shape information is a binary signal, which is expressed by “0”, “1” or “0”, “255”.

  First, as shown in FIG. 9, the alpha map is encoded including an object (called VOP (Video Object Plane) in MPEG4) to be encoded in a screen (frame). Set a region (called Bounding-Rectangle). The area is divided into 16 × 16 pixel macroblocks, and the object is encoded for each macroblock.

  Here, for each VOP, the size (vop-width, vop-height) of the Bounding-Rcctangle and the position vector (Spatial-refference (vop-horizontal-mc-spatial-ref, vop-vertical-mc-spatial-ref )) Value is encoded.

  FIG. 10 is a diagram for explaining the attributes of each macroblock. Macroblocks can be classified as "transparent macroblocks" (no object pixels) that do not contain objects in macroblocks, and "opaque macroblocks" (all objects that contain objects). And a “boundary macroblock” (a part of the pixel of the object is included) in which a part of the macroblock is included in the object.

  Therefore, the macroblock has an attribute of “transparent macroblock”, “opaque macroblock”, or “boundary macroblock”.

  The encoded data of each macroblock includes shape information and texture information. Note that the transparent macroblock does not include texture information.

  With reference to the reference (edited by Miki, "All about MPEG-4", Chapter 3, Industrial Research Committee, 1998), the encoding method of shape information in the encoder of the MPEG4 Verification model and the texture information of the arbitrary shape object The encoding method will be described.

<Encoding method of shape information>
A coding method of the shape information (A) will be described. FIG. 11 shows a configuration example of a model system as a conventional technique. The shape encoding is performed by a shape encoding encoder (a shape information encoding means (Binary Shape encoder) 500) having a configuration as shown in FIG. As shown in FIG. 11, the shape encoding encoder (shape information encoding unit 500) includes a mode determination unit 501, a shape motion vector detection unit 502, a motion compensation unit 503, an arithmetic encoding unit 504, and a motion vector prediction unit. 505, a frame memory 506, a selector 507, a shape motion vector information storage unit 508, a difference circuit 509, reduction circuits 510 and 511, an enlargement circuit 512, variable length encoding circuits 513 and 514, and a multiplexer 515.

  The shape motion vector detection means 502 is an alpha map signal (shape information) input via an encoding area detection means (Bounding-rectangle) that detects an encoding area that is an area where a target object exists in a frame image. A) a shape motion vector is detected from the prediction vector information obtained by the motion vector prediction means 505 and the shape information signal of the frame memory 506, and this is output as shape motion vector information. Reference numeral 503 is for obtaining motion vector information for motion compensation prediction from the shape motion vector information detected by the shape motion vector detection means 502, the shape information signal of the frame memory 506, and the prediction vector output from the motion vector prediction circuit 505. belongs to.

  The shape motion vector information storage means 508 is for storing the shape motion vector information detected by the shape motion vector detection means 502, and the motion vector prediction means 505 is the shape held by the shape motion vector information storage means 508. Based on the motion vector information and the luminance signal 53, a predicted vector that is a predicted value of the motion vector is obtained.

  The difference circuit 509 is for obtaining a difference between the shape motion vector obtained by the shape motion vector detection means 502 and the prediction vector obtained by the motion vector prediction means 505, and the mode determination means 501 is inputted. The mode is determined from the shape information obtained from the motion, the motion vector information for motion compensation prediction output from the motion compensation means 503, and the shape motion vector information output from the shape motion vector detection means 502.

Here, there are the following seven types of mode information. That is,
(Mode 1): Transparent
(Mode 2): Opaque
(Mode 3): Binary image encoding (within frame)
(Mode 4): Motion compensation (MV = 0)
(Mode 5): Motion compensation (MV = 0) + arithmetic coding (between frames)
(Mode 6): Motion compensation (MV ≠ 0)
(Mode 7): Motion compensation (MV ≠ 0) + arithmetic coding (between frames)
It is.

  Among these, “mode 1” is a case where all data in the macroblock is transparent, that is, no object is included in the macroblock, and “mode 2” is a case where all data in the macroblock is opaque. In this case, all the macroblocks are objects, and “mode 3” is a case of arithmetic coding (in a frame). “Mode 4” is when the motion compensation vector is zero (MV = 0), and “Mode 5” is when the motion compensation vector is zero (MV = 0) and arithmetic coding (between frames) is performed. “Mode 6” is a case where the motion compensation vector is not zero (MV ≠ 0), and “Mode 7” is a case where the motion compensation vector is not zero (MV ≠ 0). This is a case where arithmetic coding (between frames) is performed.

  The arithmetic encoding unit 504 arithmetically encodes one of the outputs of the first and second reduction units 510 and 511 according to the mode determination result of the mode determination unit 501 and outputs the result to the multiplexer 515. In addition, the selector 507 includes motion vector information for motion compensation prediction output from the motion compensation unit 503, an output from the enlargement unit 512, and a transparent pixel value (transparent pixel value; a preset fixed value). For example, a value “0”) and a fixed pixel value (Opaque pixel value; for example, value “1”) which is a preset fixed value are given, and any one of these values is set as a mode determination unit. Depending on what the mode information given from 501 is, it is selected and output.

  The frame memory 506 is a memory that holds the output of the selector 507 as a shape information signal.

  The first reduction unit 510 performs reduction processing on the shape information input, which is a binarized signal of the alpha map, and outputs it to the arithmetic coding unit 504. The second reduction unit 511 is a motion compensation unit. The motion vector information output from the means 503 is reduced and output to the arithmetic encoding means 504. The enlargement circuit 512 converts the output of the first reduction means 510 into the mode determination result of the mode determination means 501. In response, the data is output to the selector 507.

  The first variable length encoding circuit 513 performs variable length encoding on the output of the difference circuit 509 and outputs the result. The second variable length encoding circuit 514 outputs the determination result of the mode determination unit 501 as a variable length code. The multiplexer 515 receives the outputs of the first and second variable length encoding means and the output of the arithmetic encoding means 504, multiplexes them, and forms them as the shape information encoding output 52. Output.

  In such a configuration, a shape information signal 51 that is a binarized signal of an alpha map is input. In response to this, the mode determination unit 501 determines the mode based on the supplied shape information, the shape motion vector output from the shape motion vector detection unit 502, and the motion vector information output from the motion compensation unit 503. Then, according to the mode of each macroblock determined for each macroblock by the mode determining means 501, information is encoded for each macroblock as follows.

  Here, the mode information as a determination result by the mode determination unit 501 is one of the following seven types. “Mode 1” (Transparent), “Mode 2” (Opaque), “Mode 3” (arithmetic coding (intraframe)), “Mode 4” (motion compensation (MV = 0)) , “Mode 5” ((motion compensation (MV = 0) + arithmetic coding (between frames)), “mode 6” (motion compensation (MV ≠ 0)), “mode 7” (motion compensation (MV ≠ 0)) + Arithmetic coding (between frames).

  The selector 507 outputs a reproduction signal for each macroblock according to the mode information determined by the mode determination unit 501. The output reproduction signal of each macroblock is accumulated in a frame memory (FM) 506 and supplied to the texture information encoding means via the output line 54 to be encoded.

  Here, the output from the selector 507 is as follows depending on what mode information is given from the mode determination means 501 to the selector 507.

  In the case of “mode 1”: the reproduction pixel values of the shape information in the macroblock are all set to “Transparent pixel” (for example, each pixel value is “255”).

  In the case of “modes 3, 5, and 7”: a signal obtained by reducing the shape information signal 51 of the supplied macroblock by the first reduction unit 510 (the encoding target of the arithmetic encoding unit 504) Further, enlargement processing is performed by the enlargement means 512, and the original pixel size is restored and the reproduced pixel value is output.

  In the case of “modes 4 and 6”: A value obtained by performing motion compensation prediction on the reproduction pixel value of the shape information in the macroblock by the motion compensation unit 503 is output.

  On the other hand, the shape motion vector detection unit 502 receives the shape input signal input from the input line 51, the prediction vector obtained by the motion vector prediction unit 505, and the shape information signal stored in the frame memory (FM) 506. To obtain the shape motion vector information. The shape motion vector information detected by the shape motion vector detection unit 502 is motion vector information used for motion compensation prediction of the shape of each macroblock in the motion compensation unit 503, and is supplied to the motion compensation unit 503. At the same time, it is stored in the shape motion vector information storage means (MVmemory) 508.

  Here, the shape motion vector detection unit 502 detects a prediction error vector in a range of ± 16 pixels around the prediction vector obtained by the motion vector prediction unit 505.

  Accordingly, since the zero vector is detected most frequently as shown in FIG. 3, the information on whether or not the shape motion vector is the zero vector is included in the mode information. The amount is reduced. Further, by using this property (the probability that a vector near the zero vector is detected is high), it is possible to reduce the amount of motion vector detection processing.

  The motion vector prediction unit 505 obtains a predicted value of the shape motion vector from the shape motion vector stored in the shape motion vector information storage unit (MVmemory) 508 and the texture motion vector supplied via the signal line 53. Yes.

  When “modes 3, 5, and 7” are selected, the arithmetic encoding unit 504 arithmetically encodes the signal obtained by the reduction processing by the first reduction unit 510 in the case of intra-frame encoding. Will do. In the case of inter-frame coding, the arithmetic coding unit 504 refers to the signal reduced by the second reduction unit 511 for the signal obtained by the reduction processing by the first reduction unit 510. However, arithmetic coding is performed. On the other hand, when “modes 5 and 7” are selected, the prediction error vector obtained by the difference circuit 509 is variable-length encoded by the variable-length encoding means 513.

  The arithmetic code obtained from the arithmetic coding unit 504 or the variable length code of the prediction error vector obtained by the variable length coding unit 513 is obtained by variable length coding by the second variable length coding unit 514. Along with the received mode information, it is sent to multiplexing means (MUX) 515 where it is multiplexed and output as shape information encoded output 52.

  By the way, the binary image encoding information encoded by the arithmetic encoding means 504 is information obtained by treating the detailed shape in each macroblock as a binary image and encoding it. Here, the shape motion vector is obtained by searching 16 pixels around the prediction vector obtained by the motion vector prediction means 505.

  Accordingly, since the zero vector is detected most frequently as shown in FIG. 3, the code amount of the shape motion vector information is reduced by including information on whether or not the shape motion vector is the zero vector in the mode information. .

  In this way, the shape information encoding unit 500 performs shape information encoding processing. Note that the shape information here is for an alpha map with two gradations, and some alpha maps have a multi-grayscale gray scale, so they are referred to as shape information to distinguish them.

<Encoding method of texture information>
The alpha map is information (shape information) representing the shape and size of the object, but the image of the object cannot be reproduced without texture information that is information representing changes in luminance and color difference inside the object. Therefore, in MPEG4, texture information is also encoded together with the alpha map and used as a pair with the alpha map.

  An encoding method of texture information (YUV in FIG. 12) will be described with reference to FIG. FIG. 12 is a block diagram of an encoder portion showing a configuration example as a model system of the prior art. In FIG. 12, coding region (Bounding-Rectangle) detection means 301, reference image padding means 302, LPE padding means 303, zero padding means 304, vector padding means 306, and shape information code which are components highlighted in the figure. The converting unit 500 is a component necessary only for encoding an object having an arbitrary shape. Accordingly, in order to encode a rectangular object as in the conventional encoding method such as MPEG1 or MPEG2, other components, that is, the motion vector detecting means 305, the frame memory 307, the motion in FIG. Compensation means 308, changeover switches 309 and 310, motion vector prediction means 311, motion vector storage means 312, difference circuit 313, third variable length coding means 314, quantization means 315, inverse quantization means 316, orthogonal transformation means 317, fourth variable length coding means 318, inverse orthogonal transform means 319, adding circuit 320, switch 321, and multiplexer 322 may be provided.

  Here, the coding area (Bounding-Rectangle) detecting means 301 is for detecting a coding area which is an area where a target object is present in a frame image (see FIG. 9), and shape information coding. The means 500 is as described with reference to FIG.

  In the configuration of FIG. 12, the texture information (YUV) is encoded for the “opaque macroblock” and the “boundary macroblock”. On the other hand, as in the conventional encoding method, the input macroblock signal is directly encoded by the motion compensated prediction + DCT (discrete cosine transform) method.

  On the other hand, for the “boundary macroblock”, a signal outside the object is padded (compensated) and then encoded by motion compensated prediction + DCT method.

  There are the following three methods of padding (compensation processing) here.

  [1] Reference image padding: This is a replenishment process by the reference image padding means 302, and the reference image for motion compensation prediction is padded. There are processing for boundary macroblocks and processing for transparent macroblocks.

  [2] LPE padding: This is a replenishment process by the LPE padding means 303, and before DCT (discrete cosine transform) of the block in the intra macroblock, the pixel (white circle in FIG. 13) value outside the object is set to the object. This is a process of applying a low-pass filter after replacing the average value of the internal pixel (black circle portion in FIG. 13) value. The processing unit is the same 8 × 8 pixels as in the case of DCT.

  [3] Zero padding: This is a replenishment process by the zero padding means 304, and before DCTing the block in the inter macro block, the value of the pixel outside the object (white circle in FIG. 13) of the motion compensated prediction error signal is set. This is a process of replacing with a zero value. The processing unit is 8 × 8 pixels, which is the same as DCT.

  Here, the padding [1] is an indispensable process in the MPEG4 standard, but the padding [2] and [3] are necessary for improving the coding efficiency and are based on the standard. Therefore, the processing is not limited to the above-described components, and may be performed using other means.

  Further, the motion vector detection unit 305, based on the shape signal supplied from the coding region detection unit 301 via the signal line 31, and the luminance signal (YUV) of the original image supplied via the signal line 32, and the signal Motion vector detection is performed between the luminance signal of the reference image supplied from the frame memory 307 via the line 33. As a result, the obtained motion vector 34 is output to the motion compensation means (MC) 308 and the vector padding means (Vector padding) 306.

  In the vector padding unit 306, based on the reproduction value data 54 of the shape information output and supplied from the shape information encoding unit 500, a block (transparent block or intra block) having no motion vector is used. After the macroblocks) are filled with appropriate motion vectors, they are stored in the motion vector storage means (MV memory) 312.

  When the motion vector is detected, pixel values outside the object are padded so that the reference image is smoothly connected to the pixel values in the object by padding.

  On the other hand, in the original image, the pixel value at the object boundary is often an edge boundary, and the pixel value variation is large. Therefore, when normal block matching is performed on the boundary macroblock, the mismatch of pixel values outside the object is large, and a normal motion vector is often not detected.

  Therefore, in the conventional model, for the boundary macroblock, the motion vector is detected by evaluating the error only with the pixel value inside the object (black circle portion in FIG. 14) with reference to the shape information (this is the Polygon matching). However, since this method detects a motion vector for each pixel while determining whether or not it is inside an object, the amount of momentum processing increases.

  As described above, an alpha map is information (shape information) that represents the shape and size of an object, but an object image cannot be reproduced without texture information that is information representing changes in luminance and color difference inside the object. . Therefore, in MPEG4, texture information is also encoded together with the alpha map and used as a pair with the alpha map.

  The encoding of the texture information (YUV) is performed on the “opaque macroblock” and the “boundary macroblock”. Of these, the “opaque macroblock” is input. The macroblock signal is encoded as it is by the motion compensated prediction + DCT (discrete cosine transform) method, and the “boundary macroblock” is encoded by the motion compensated prediction + DCT method after padding the signal outside the object (compensation processing). .

  That is, since motion compensation prediction is performed for the “boundary macroblock”, the pixel values outside the object are padded so that the reference image is smoothly connected to the pixel values in the object when the motion vector is detected. That is why.

  However, in the original image, the pixel value at the object boundary is often an edge boundary, so the pixel value fluctuates greatly. Therefore, when normal block matching is performed on the boundary macroblock, there is a mismatch between the pixel values outside the object. In many cases, a large and normal motion vector is not detected.

  Therefore, for the boundary macroblock, polygon matching, which is a method of detecting a motion vector by evaluating an error only with a pixel value (black circle portion in FIG. 14) inside the object with reference to shape information, is adopted.

  However, this method has a problem that the amount of processing increases because a motion vector is detected while determining whether or not each pixel is inside an object.

  Accordingly, an object of the present invention is to provide a moving picture coding apparatus and a moving picture coding method capable of detecting a motion vector with a small amount of calculation while improving coding efficiency. It is in.

  In order to achieve the above object, the present invention is configured as follows.

  [1] First, a coding apparatus of a moving image coding method for coding a desired object constituting an image as an arbitrary shape object composed of shape information and texture information in units of macroblocks. A means for detecting a motion vector of the texture information for each macroblock, a determination means for determining whether or not the motion vector of the texture information can be used in order to detect a motion vector of the shape information in the macroblock, In order to detect the motion vector of shape information in a macro block, the evaluation means for evaluating the reliability of the motion vector of texture information and the motion vector detection range of the shape information can be used when the motion vector of texture information is not available It is set wider than the time and the motion information motion vector is In the case where the macroblock is available, setting means that is set more widely when the motion vector reliability of the texture information is lower than when the reliability of the motion vector is high, and the motion vector of the texture information that has already been detected is used in the macroblock. And a motion vector detecting means for detecting a motion vector of the shape information. This device is also capable of detecting the motion vector of the texture information in the motion vector detection means so that the motion vector of the texture information can be used when encoding the arbitrary shape object composed of the shape information and the texture information. Of the input object shape information and texture information, the identification means identifies whether the motion vector of the texture information can be used, and the reliability of the motion vector of the texture information is determined by the evaluation means In addition, when encoding a macroblock to be encoded, a shape motion vector is obtained by searching for shape motion vectors of macroblocks around the macroblock, and the search range is a motion vector of texture information. Is narrower when available, and the motion If the texture information motion vector cannot be used, it is wider than when it is available. If the texture information motion vector is available, the lower reliability is set wider than when the texture information motion vector is highly reliable. Thus, the shape motion vector search range of the macroblock is set to an appropriate range according to the availability of the motion vector of the texture information and the degree of reliability.

  That is, the present invention relates to motion vector detection. As a result, it is possible to provide a technique capable of reducing the calculation amount of motion vector detection without reducing coding efficiency.

  [2] In order to achieve the above object, according to the second aspect of the present invention, secondly, for a desired object constituting an image, the desired object is composed of shape information and texture information in units of macroblocks. In the coding apparatus of the moving image coding method, the motion vector detection means for detecting the shape motion vector and the motion vector of the texture information from the input object shape information is provided, and an error at the time of encoding is designated. Threshold value setting means for setting a threshold value for performing the search, dividing means for dividing the search range into a plurality of search ranges, and whether or not the motion compensation prediction error due to the detected motion vector is larger than the threshold value Determination means for determining whether the motion vector detection means is the narrowest of the divided search ranges When motion vector detection is started from the search range, and the motion compensation prediction error due to the optimal motion vector detected within the search range is larger than the threshold, the optimal motion vector is detected in a wider search range, When the motion compensation prediction error is smaller than the threshold value, the motion vector detection is terminated, and the motion vector is output as a detection result.

  In MPEG4, when the prediction vector is a zero vector, information indicating that the prediction vector is a zero vector is incorporated into mode information and separately encoded, thereby improving the encoding efficiency. Making it possible to detect efficiently also greatly contributes to a reduction in the amount of calculation. Therefore, in the present invention, a threshold setting unit is provided to set a threshold for specifying an error during encoding, and the search unit is divided into a plurality of search ranges by the dividing unit. To do. Then, the motion vector detection means starts motion vector detection from the narrowest search range among the divided search ranges, and the determination means detects the detection based on the motion vector detected by the motion vector detection means. It is determined whether the motion compensation prediction error due to the motion vector thus determined is larger than a threshold value. Then, as a result of starting motion vector detection from the narrowest search range, the motion vector detection means determines that the motion compensation prediction error due to the optimal motion vector detected within the search range is greater than the threshold value. If it is determined, it operates so as to detect an optimal motion vector in a wider search range, and if it is determined that the motion compensation prediction error is smaller than the threshold value, the motion vector detection is terminated and the motion vector is detected. Is output as a detection result. As described above, when the motion compensated prediction error due to the optimum motion vector detected within the search range is large, the optimum motion vector is detected within a wider search range, and when the motion compensated prediction error is small, the motion vector detection is performed. As a result of outputting the motion vector as a detection result, if the error is small, the zero vector can be efficiently detected with a small amount of calculation, which greatly contributes to the reduction of the amount of calculation.

  [3] In order to achieve the above object, according to the third aspect of the present invention, thirdly, regarding a desired object constituting an image, the desired object is an arbitrary shape object composed of shape information and texture information in units of macroblocks. In a coding apparatus of a moving image coding method to be coded, before performing shape coding and texture coding, a boundary of a form including an object as a part with reference to a shape signal input to the coding means For macroblocks, it has means to compensate all pixels in the macroblock to be texture encoded by performing padding processing for eliminating discontinuity in the macroblock, and shape coding and texture coding Is configured to use the padded image.

  The present invention relates to texture coding in a “boundary macroblock”, and the original image is a padding process that can eliminate the discontinuity of pixel values at the object boundary for the boundary macroblock, that is, LPE padding. After the padding process, the shape coding and texture coding are used. For this reason, since there is almost no discontinuity of pixel values at the object boundary, a normal motion vector can be detected even when normal block matching is performed. Therefore, it is not necessary to perform polygon matching with a large amount of processing in the “boundary macroblock”, the calculation time can be shortened, and a normal motion vector can be detected even when normal block matching is performed. become able to.

  According to the present invention, it is possible to improve the encoding efficiency without increasing the processing amount by changing the processing order of the components necessary for encoding the arbitrary shape object. In addition, by adaptively terminating the shape motion vector detection process, the processing amount can be reduced without causing a decrease in encoding efficiency.

  Hereinafter, specific examples of the present invention will be described with reference to the drawings. The present invention improves the encoding efficiency by changing the processing order of the components of the MPEG4 encoder (moving image encoding apparatus) and improves the shape motion vector detection method without reducing the encoding efficiency. The motion vector can be detected with a small amount of calculation, and the details will be described below.

<First specific example>
A first specific example of the present invention will be described with reference to the drawings. The technique described in the first specific example is a technique that can reduce the amount of calculation of motion vector detection without reducing the encoding efficiency.

As described above, in the configuration of the model system as the prior art shown in FIG. 11, the shape motion vector (MVs) is ± 16 pixels around the prediction vector (MVPs) obtained by the motion vector prediction means 505 (FIG. 3). This is obtained by searching the search range 3). That is, the difference vector MVDs from the prediction vector MVPs is detected (Formula 1).
MVDs = MVs−MVPs (Formula 1)
Normally, the frequency distribution of the prediction error signal has a high frequency in the vicinity of “0” as shown in FIG. 1, and the frequency tends to decrease rapidly as the distance from “0” increases. In FIG. 1, the horizontal component of the difference vector MVDs is expressed as “mvds_y”, and the vertical component is expressed as “mvds_y”.

  Here, when the prediction accuracy of the prediction vector MVPs is low, the frequency distribution is slow, and when the prediction accuracy of MVPs is high, the frequency distribution is steep. FIG. 2 is a diagram for explaining a method for obtaining the prediction vector MVPs. FIG. 2A is a diagram showing the shape signal component, and FIG. 2B is a diagram showing information on the luminance signal component in units of macroblocks. As the macroblock MB to be encoded and its neighboring macroblocks, FIG. It shows that there is a macroblock having shape motion vectors MVs1, MVs2, and MVs3. FIG. 2B shows that macroblocks having texture motion vectors MV1, MV2, and MV3 exist as macroblocks MB to be encoded and neighboring macroblocks.

  In this case, in encoding the macroblock MB, in the system of the present invention, when encoding the macroblock to be encoded, first, in the order of the MVs1, MVs2, and MVs3, It is checked whether or not a shape motion vector exists for surrounding macroblocks, and the shape motion vector that exists first is defined as MVPs.

  When none of the shape motion vectors MVs1, MVs2, and MVs3 exist, it is checked whether or not the texture motion vector exists in the order of MV1, MV2, and MV3, and the motion vector that exists first is determined as the prediction vector MVPs. And

  When none of the texture motion vectors MV1, MV2, and MV3 exists, the prediction vector MVPs is set to a zero vector.

  In the system of the present invention, the motion vector predicting unit 505 has such a function.

  Note that, as described in the above-mentioned reference, texture motion vectors (MV1, MV2, MV3) may or may not be used. For example, there is a mode in which only the shape information is encoded without encoding the texture information, or the case where none of MV1, MV2, and MV3 exists for the encoding target macroblock. That is, assuming that the texture motion vector is reliable, it can be said that in this method, when the texture motion vector can be used, the prediction accuracy of the prediction vector MVps is higher than when the texture motion vector cannot be used.

(First specific example 1)
Therefore, in this embodiment, as shown in FIG. 3, the search range is prepared for several stages, and when the texture motion vector can be used, the search range is switched according to the prediction accuracy of the prediction vector MVPs. For example, the search range is prepared for several stages such as “search range 1”, “search range 2”, “search range 3”, and the area size of “search range 1” is 4 × 4 pixels. The area size of “range 2” is 8 × 8 pixels, and the area size of “search range 3” is 16 × 16 pixels.

  In the configuration shown in FIG. 11, as a function of the shape motion vector detection unit 502, the function of detecting the prediction accuracy of the prediction vector MVPs and the function of switching the motion vector search range according to the prediction accuracy are added. Thus, the calculation amount of motion vector detection can be reduced without reducing the coding efficiency. The prediction accuracy may be determined, for example, by the magnitude of the error with respect to the prediction vector MVPs obtained by the motion vector prediction unit 505.

  As a specific example of the motion vector search range, if the texture motion vector is reliable, it is within the “search range 1”, which is the minimum region size, and if the texture motion vector is not reliable, a slightly wider region The “search range 2” is set as the search range. If the texture motion vector cannot be used, the search area “search range 3”, which is the widest area, is searched.

  That is, an alpha map signal input to the shape motion vector detection unit 502 via an encoding region detection unit (Bounding-rectangle) that detects an encoding region that is a region where a target object exists in the frame image. In detecting a motion vector from (shape information A), the prediction vector information obtained by the motion vector prediction means 505 and the shape information signal in the frame memory 506, the prediction accuracy of the prediction vector MVPs is obtained and switched accordingly. The motion is detected so as to detect the shape motion vector within the optimum motion vector search range. Then, the detected vector is output as shape motion vector information.

  In this way, when encoding the macroblock, as a result of examining the reliability based on the prediction accuracy of the texture motion vector, if the reliability is high, a narrow search range around the macroblock to be encoded is used. It becomes possible to complete the calculation of motion vector detection, and the amount of calculation required for motion vector detection can be reduced. If the texture motion vector is not reliable or the texture motion vector cannot be used, the motion vector can be detected by expanding the search range.

  FIG. 4 is a flowchart of this example. That is, it is checked whether or not the texture motion vector can be used (step S11). As a result, if the texture motion vector cannot be used, the search range is set to “search range 3”. If the texture motion vector can be used as a result of the check in step S11, it is next checked whether or not the texture motion vector is reliable (step S12). If the result is not reliable, the search range is set to "search range 2". I will do it. If the result of the check in step S12 is reliable, the search range is set to “search range 1”.

  Here, the reliability of the texture motion vector is determined by the search range of the texture motion vector. That is, the texture motion vector is detected directly because the texture motion vector itself is directly detected, and can follow a large motion when the search range is wide. Therefore, it can be estimated that the reliability is higher than that when the search range is narrow. In MPEG4, the search range of texture motion vectors can be extended to ± 1024.

  In this way, the search range can be switched according to the prediction accuracy of the prediction vector MVPs, and encoding is performed by performing such search range switching according to the prediction accuracy of the prediction vector MVPs. It is possible to reduce the calculation amount of motion vector detection without reducing efficiency.

(First specific example 2)
When the prediction vector MVDs is a zero vector, as described above, information indicating that the prediction vector MVDs is a zero vector is incorporated into mode information and separately encoded to improve encoding efficiency.

  Therefore, in the model encoding system as the prior art for MPEG4 shown in FIG. 12, a predetermined threshold value is set as a comparison reference so that the zero vector is easily detected, and a motion compensation prediction error (MC at the time of the zero vector is determined. Error) is compared with this threshold value, and when the motion compensated prediction error (MC error) at the time of the zero vector is smaller than the threshold value, detection is aborted at that time, and the predicted vector MVPs is converted into the shape motion vector MVs. It is said.

  In this specific example, the above truncation is further expanded. For example, when the motion compensation prediction error with the optimum motion vector when detecting up to “search range 1” in FIG. 3 is smaller than a predetermined threshold value. , At that point, stop the detection. When the motion compensation prediction error is larger than the predetermined threshold value, the search range is expanded to “search range 2”, and the remaining range of “search range 2” is expanded as in the case of “search range 1”. When the motion compensation prediction error with the optimum motion vector at the time of the search is smaller than a predetermined threshold value, the detection is stopped at that point.

  That is, the search range is switched according to the prediction accuracy of the prediction vector MVPs, and the search range switching according to the prediction accuracy of the prediction vector MVPs is performed, so that motion vector detection can be performed without reducing the coding efficiency. Try to reduce the amount of calculation.

  FIG. 5 is a flowchart of this example. That is, the prediction vector MVDs is initialized to “0” (step S1), and then it is checked whether the MC (motion compensation) error is larger than a threshold (step S2). As a result, if it is small, the process is terminated, and if it is large, the search is made in “search range 1” (step S3).

  Then, it is checked whether the MC (motion compensation) error is larger than a threshold value (step S4). As a result, if it is small, the process is terminated, and if it is large, the search is made within “search range 2—search range 1” (step S5).

  Then, it is checked whether the MC (motion compensation) error is larger than a threshold value (step S6). As a result, if it is small, the process is terminated, and if it is large, the search is made within “search range 3—search range 2” (step S7), and when it is completed, the process is terminated.

  In this way, the search range can be switched according to the prediction accuracy of the prediction vector MVPs, and encoding is performed by performing such search range switching according to the prediction accuracy of the prediction vector MVPs. The calculation amount of motion vector detection can be reduced without reducing the efficiency.

  As described above, since there is a high possibility that there is an optimum value of the difference vector MVDs in the vicinity of the zero vector, the search range is expanded step by step from a narrow search range, and the search is terminated when a predetermined condition is satisfied in the middle step. Thus, it is possible to reduce the calculation amount of motion vector detection without lowering the encoding efficiency.

  In both “specific example 1” and “specific example 2”, a vector having a smaller size is selected as compared with the MPEG4 model encoder of FIG. In some cases, the conversion efficiency is improved.

  If the techniques shown in “Specific Example 1” and “Specific Example 2” are combined, the calculation amount of motion vector detection can be further reduced.

  The above is a technique in which the amount of calculation for motion vector detection can be reduced by performing motion vector detection using texture motion vectors in MPEG4.

Next, texture coding in the “boundary macroblock” will be described.
<Second specific example>
Next, a second specific example of the present invention will be described. This specific example relates to texture coding in a “boundary macroblock” which is a form in which an object is included in a part of a macroblock.

  In FIG. 6, coding region (Bounding-Rectangle) detecting means 301, reference image padding means 302, LPE padding means 303a, zero padding means 304, vector padding means 306, and shape information code which are components highlighted in the figure. The converting unit 500 is a component necessary only for encoding an object having an arbitrary shape. Therefore, in order to encode a rectangular object as in the conventional encoding method such as MPEG1 or MPEG2, other components, that is, the motion vector detecting means 305, the frame memory 307, the motion in FIG. Compensation means 308, changeover switches 309 and 310, motion vector prediction means 311, motion vector storage means 312, difference circuit 313, third variable length coding means 314, quantization means 315, inverse quantization means 316, orthogonal transformation means 317, fourth variable length coding means 318, inverse orthogonal transform means 319, adding circuit 320, switch 321, and multiplexer 322 may be provided.

  Here, the motion vector detection means 305 is supplied from the LPE padded macroblock data output from the LPE padding means 303, the shape information reproduction value data 54 output from the shape information encoding means 500, and the frame memory 307. Motion vector detection is performed with the luminance signal of the reference image. As a result, the obtained motion vector is output to the motion compensation means (MC) 308, the vector padding means 306, and the difference circuit 313.

  The frame memory 307 is for holding a luminance signal of a reference image that has been padded and output from the reference image padding unit 302. Also, the motion compensation unit 308 uses the motion vector detected by the motion vector detection unit 305 and the reproduction value data 54 of the shape information output from the shape information encoding unit 500 to provide a motion vector for motion compensation prediction. The changeover switches 309 and 310 are path changeover switches for selectively switching whether the output of the LPE padding means 303 is given to the zero padding means 304 or to be bypassed.

  The orthogonal transform means 317 is for performing an orthogonal transform (discrete cosine transform) on the output given via the changeover switch 310 and decomposing the output into frequency components. The quantization means 315 is an orthogonal transform means. The fourth variable length coding means 318 performs variable length coding processing on the quantized output and outputs the quantized output to the multiplexer 322 as a texture stream.

  The motion vector predicting means 311 is for predicting a motion vector using the reproduction value data 54 of the shape information output from the shape information encoding means 500 and the data held in the motion vector storage means 312. The difference circuit 313 is used to obtain a difference value between the motion vector prediction value output from the motion vector prediction means 311 and the motion vector output from the motion vector detection means 305. Third difference length encoding means 314 is a variable-length encoding process for the output of the difference circuit 313 and outputs it as a motion stream to the multiplexer 322.

  The inverse quantization means 316 returns the quantized output of the quantization means 315 to the original data after being inversely quantized, and the inverse orthogonal transform means 319 outputs the data output from the inverse quantization means 316. Is subjected to inverse orthogonal transform (inverse discrete cosine transform) to restore the original data at the time of zero padding processing.

  The multiplexer 322 encodes the variable-length encoded motion stream output from the third variable-length encoding unit 314, the variable-length encoded texture stream output from the fourth variable-length encoding unit 318, and encoding. It receives the information of the coding area output from the region (Bounding-Rectangle) detection means 301 and the reproduction value data 54 of the shape information output from the shape information encoding means 500, multiplexes them and outputs them.

  The inverse orthogonal transform unit 319 is for performing inverse orthogonal transform (inverse discrete cosine transform) on the data output from the inverse quantization unit 316 to restore the original data at the time of zero padding processing.

  Further, the adder circuit 320 adds the motion vector information from the motion compensation unit 308 given through the switch 321 and the original zero padding process data given from the inverse orthogonal transform unit 319 to add the reference image padding unit. The motion vector storage unit 312 is based on the reproduction value data 54 of the shape information output from the vector padding unit 306. Based on the reproduction value data 54 of the shape information, the motion vector storage unit 312 Macroblock) is received and stored with an appropriate motion vector.

  Here, the difference in the mechanism between the encoder in the present embodiment and the conventional encoder will be mentioned. In the conventional encoder, as shown in FIG. 12, the motion vector detection means 305 includes input texture information (YUV), detection output from the coding area (Bounding-Rectangle) detection means 301, and storage information from the frame memory 307. Is used to obtain the motion vector 34. Here, the coding area (Bounding-Rectangle) detection means 301 is for detecting a coding area that is an area where a target object exists in a frame image.

  Further, the LPE padding means 303 is placed in parallel with the zero padding means 304, and the input texture information (YUV) is given to one of them according to the condition for padding, and the padded output is given to the orthogonal transformation means 317. It was a configuration.

  In the present invention, as shown in FIG. 6, the LPE padding means 303 performs the LPE padding process at the first stage regardless of the condition of the input texture information (YUV), and this is processed into the motion vector detection means 305 and the condition. Accordingly, the zero padding means 304 is provided. Here, as described above, LPE padding refers to pixel values outside an object (white circles in FIG. 13) values inside the object (FIG. 13) before DCT (Discrete Cosine Transform) of the block in the intra macroblock. The processing unit is 8 × 8 pixels, as in the case of DCT. Also, zero padding is a process of replacing pixel values outside the object (white circles in FIG. 13) of the motion compensated prediction error signal with zero values before DCTing the blocks in the inter macroblock. As in the case of DCT, it is 8 × 8 pixels. These LPE padding and zero padding are not indispensable processes in the MPEG4 standard, but are implemented to improve coding efficiency.

  The block configuration of this example is as shown in FIG. In the configuration of this specific example, the processing position of the motion vector detection unit 305 is different from the position of the motion vector detection unit 305 in the configuration of the conventional model shown in FIG. 12, and the processing position of the LPE padding 303 is the same as that of the conventional model. Different from the position of the LPE padding 303.

<LPE padding means>
First, the difference between the processing in the LPE padding means 303a in the system of this embodiment and the processing in the LPE padding means 303 in the system as the prior art will be described.

  The padding process in the LPE padding unit 303a is performed only on the boundary macroblock of the original image (YUV) 41 of the texture image supplied via the signal line. Here, whether or not it is a boundary macroblock is determined using the original signal 42 of the shape image supplied via the coding region (Bounding-Rectangle) detection means 301. That is, the LPE padding means 303a performs LPE padding as a pre-coding process.

  By performing the preprocessing, the texture image 43 output from the LPE padding unit 303a is compensated for all the pixel values in the macroblock to be encoded before being encoded.

  In the LPE padding means 303 in the model system as the prior art, an 8 × 8 pixel block is a boundary block based on the local decoded signal of the shape image supplied from the shape information encoding means 500 via the signal line 35. If there is any padding.

  In this specific example, when padding is performed in block units of 8 × 8 pixels and the pixel in the object is not included in the block as in the upper left block in FIG. 14, for example, padding is performed with the average value of surrounding blocks. As a result, all the pixels in the macroblock are padded.

  As described above, the first specific example and the conventional model system differ in that padding is performed based on the original signal of the shape image or padding based on the local decoded signal.

  Here, an example in which LPE padding is applied to a one-dimensional signal model will be shown in order to specifically show the effect of the method of “padding based on the original signal of the shape image” which is the technique of the present invention. The luminance signal is Y, the shape signal is A, the shape local decoded signal (Part 1) is A ', and the shape local decoded signal (Part 2) is A ". These are the following signals, respectively. Suppose.

  In the following example, for simplicity, the shape signal is expressed by “0” (outside the object) or “1” (inside the object).

Y = {50,50,50,50,180,190,200,210}
A = {0,0,0,0,1,1,1,1,1}
A ′ = {0,0,0,1,1,1,1,1}
A ″ = {0,0,0,0,0,1,1,1}
Here, assuming that the results of LPE padding of the luminance signal Y with the above-described A, A ′, A ″ are Yp, Yp ′, Yp ″, for example, the following results.

Yp = {195,195,195,188,180,190,200,210}
Yp ′ = {166,166,108,50,180,190,200,210}
Yp ″ = {200,200,200,200,195,190,200,210}
That is, in the case of Yp, the data string of “50”, “50”, “50”, “50”, “180”, “190”, “200”, “210” of the original Y data is padded. As a result, “195”, “195”, “195”, “188”, “180”, “190”, “200”, “210” are indicated, and in the case of Yp ′, the data value Indicates “166”, “166”, “108”, “50”, “180”, “190”, “200”, “210”, and in the case of Yp ″, the data value is “ 200, 200, 200, 200, 195, 190, 200, and 210. The numerical values shown here are lower limit values. It is a 256-step gray scale value where “0” (white) and the upper limit value are “255” (black).

  In general, the boundary of an object is an edge portion of an image, and the luminance signal Y varies greatly as in the above example. When an error occurs in the shape local decoding signal in the processing by the shape information encoding unit 500, the padding may not be smoothly performed as in the above example (Yp). Therefore, when texture encoding is performed on the “boundary macroblock”, a problem that an edge becomes unclear becomes apparent when a signal including such an error must be used.

  However, in the case of the method of the present invention, such a worry is completely eliminated by setting the shape signal to “0” (outside the object) and “1” (inside the object).

<Motion vector detection means>
Next, the difference between the processing of the motion vector detection means 305a in the system of this embodiment and the processing of the motion vector detection means 305 in the system of the conventional model will be described. In the motion vector detection unit 305a in this system, the reproduction value data 54 of the shape information output from the shape information encoding unit 500 and supplied via the signal line 44, and the padded processed data output from the LPE padding unit 303a Based on, motion vectors for macroblocks other than the transparent macroblock are detected.

  A motion vector is detected between the original image data (padding processed data) supplied from the LPE padding means 303a via the signal line 45 and the reference image supplied from the frame memory 307 via the signal line 46; The detected motion vector is output to the vector padding means 306, the motion compensation means 308, and the difference circuit 313 via the signal line 47.

  Here, the boundary macroblock is already padded by the LPE padding 403 in the original image supplied via the signal line 45. For this reason, there is almost no discontinuity of pixel values at the object boundary.

  Therefore, unlike the case of the motion vector detection unit 305 in the conventional model system, the motion vector detection unit 305a of the system of the present invention does not need to perform polygon matching with a large amount of processing in the “boundary macroblock”. Even when block matching is performed, a normal motion vector can be detected.

  As described above, various embodiments have been described. In short, the present invention firstly relates to a desired object constituting an image, and the desired shape object is formed of shape information and texture information in units of macroblocks. A motion vector detecting means for detecting a shape motion vector and a texture information motion vector to be used for motion compensation prediction from input object shape information, and an input device Identification means for identifying whether or not the motion vector of the texture information can be used from the texture information, an evaluation means for evaluating the reliability of the detected motion vector in the texture information, a search range of the shape motion vector, When information motion vector is not available, set wider than when it can be used. Is, in case the motion vector of texture information is available are those having a setting unit for lower than the reliability of the motion vector of texture information are set widely.

  This device is also capable of detecting the motion vector of the texture information in the motion vector detection means so that the motion vector of the texture information can be used when encoding the arbitrary shape object composed of the shape information and the texture information. Of the input object shape information and texture information, the identification means identifies whether the motion vector of the texture information can be used, and the reliability of the motion vector of the texture information is determined by the evaluation means In addition, when encoding a macroblock to be encoded, a shape motion vector is obtained by searching for shape motion vectors of macroblocks around the macroblock, and the search range is a motion vector of texture information. Is narrower when available, and the motion If the texture information motion vector cannot be used, it is wider than when it is available. If the texture information motion vector is available, the lower reliability is set wider than when the texture information motion vector is highly reliable. Thus, the shape motion vector search range of the macroblock is set to an appropriate range according to the availability of the motion vector of the texture information and the degree of reliability. That is, the present invention relates to motion vector detection. As a result, it is possible to provide a technique capable of reducing the calculation amount of motion vector detection without reducing coding efficiency.

  Second, the present invention relates to a moving image coding system that encodes a desired object constituting an image as an arbitrary shape object composed of shape information and texture information in units of macroblocks. The encoding apparatus includes a motion vector detection means for detecting a shape motion vector and a texture information motion vector from the shape information of an input object, and sets a threshold value for specifying an error in encoding. A threshold value setting unit, a dividing unit that divides the search range into a plurality of search ranges, and a determination unit that determines whether the motion compensation prediction error due to the detected motion vector is larger than a threshold value, The motion vector detection means detects a motion vector from the narrowest search range among the divided search ranges. When the motion compensated prediction error due to the optimal motion vector detected within the search range is larger than the threshold, the optimal motion vector is detected in a wider search range, and the motion compensated prediction error is the threshold. When the value is smaller than the value, the motion vector detection is terminated, and the motion vector is output as a detection result.

  In MPEG4, when the prediction vector is a zero vector, information indicating that the prediction vector is a zero vector is incorporated into mode information and separately encoded, thereby improving the encoding efficiency. Making it possible to detect efficiently also greatly contributes to a reduction in the amount of calculation. Therefore, in the present invention, a threshold setting unit is provided to set a threshold for specifying an error during encoding, and the search unit is divided into a plurality of search ranges by the dividing unit. To do. Then, the motion vector detection means starts motion vector detection from the narrowest search range among the divided search ranges, and the determination means detects the detection based on the motion vector detected by the motion vector detection means. It is determined whether the motion compensation prediction error due to the motion vector thus determined is larger than a threshold value. Then, as a result of starting motion vector detection from the narrowest search range, the motion vector detection means determines that the motion compensation prediction error due to the optimal motion vector detected within the search range is greater than the threshold value. If it is determined, it operates so as to detect an optimal motion vector in a wider search range, and if it is determined that the motion compensation prediction error is smaller than the threshold value, the motion vector detection is terminated and the motion vector is detected. Is output as a detection result. As described above, when the motion compensated prediction error due to the optimum motion vector detected within the search range is large, the optimum motion vector is detected within a wider search range, and when the motion compensated prediction error is small, the motion vector detection is performed. As a result of outputting the motion vector as a detection result, if the error is small, the zero vector can be efficiently detected with a small amount of calculation, which greatly contributes to the reduction of the amount of calculation.

  In the third aspect of the present invention, there is provided a moving image coding method code that encodes a desired object constituting an image as an arbitrary shape object composed of shape information and texture information in units of macroblocks. Before performing shape coding and texture coding in the coding device, the shape signal input to the coding means is referred to, and boundary macroblocks that include an object as a part of the macroblock are not processed. It has a means to compensate all the pixels in the macroblock to be texture encoded by performing padding processing to eliminate continuity, and shape coding and texture coding use the padded image. Thus, the configuration is performed.

  The present invention relates to texture coding in a “boundary macroblock”, and the original image is a padding process that can eliminate the discontinuity of pixel values at the object boundary for the boundary macroblock, that is, LPE padding. After the padding process, the shape coding and texture coding are used. For this reason, since there is almost no discontinuity of pixel values at the object boundary, a normal motion vector can be detected even when normal block matching is performed. Therefore, it is not necessary to perform polygon matching with a large amount of processing in the “boundary macroblock”, the calculation time can be shortened, and a normal motion vector can be detected even when normal block matching is performed. become able to.

  In addition, this invention is not limited to the Example mentioned above, In the range which does not change a summary, it can deform | transform suitably and can be implemented.

The figure explaining the characteristic of frequency distribution of difference vector MVDs. The figure explaining the method of calculating | requiring the prediction vector MVPs. The figure explaining the search range of the shape motion vector of this invention. It is a figure for demonstrating this invention, Comprising: The flowchart which shows the process example in the 1st specific example 1 of this invention. It is a figure for demonstrating this invention, Comprising: The flowchart which shows the process example in the 1st specific example 2 of this invention. It is a figure for demonstrating this invention, Comprising: The block diagram which shows the structural example of the encoder of this invention. The block block diagram of the image encoding apparatus of the system which divides | segments the inside of a screen into a background and an object, and encodes, when encoding an image. The block diagram of a decoding apparatus. The figure for demonstrating this invention, Comprising: The figure explaining the encoding area | region containing an object. It is a figure for demonstrating this invention, Comprising: The figure explaining the attribute of each macroblock. The block block diagram of the shape encoding encoder (Binary Shape encoder 500) part in the model system as a prior art. The block block diagram of the encoder in the model system as a prior art. The figure explaining the pixel in an object, and the pixel outside an object. The figure explaining the LPE padding of this invention.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 301 ... Coding area | region (Bounding-Rectangle) detection means, 302 ... Reference image padding means, 303 ... LPE padding means, 304 ... Zero padding means, 306 ... Vector padding means, 500 ... Shape information encoding means (Binary Shape encoder) 305, 305a ... motion vector detection means, 307 ... frame memory, 308, 308a ... motion compensation means 309, 310, ... changeover switch, 312 ... motion vector storage means, 313 ... difference circuit, 314 ... third variable length code 315 ... quantization means, 316 ... inverse quantization means, 317 ... orthogonal transform means (DCT), 318 ... fourth variable length coding means, 319 ... inverse orthogonal transform means, 320 ... adder circuit, 321 ... Switch, 322 ... Multiplexer

Claims (3)

In a coding apparatus of a moving image coding method for coding a desired object constituting an image as an arbitrary shape object composed of shape information and texture information in units of macroblocks, shape coding and texture coding Prior to conversion, the shape signal input to the encoding means is referred to, and boundary macroblocks that include an object as a part are subjected to padding processing for eliminating discontinuities in the macroblock. It has a means for compensating all the pixels in the macroblock to be texture-encoded by rubbing, and shape encoding and texture encoding are performed using the padded image. A moving image encoding device. 2. The moving image encoding apparatus according to claim 1, further comprising motion vector detecting means for detecting a shape motion vector and a motion vector of texture information to be used for motion compensation prediction from the shape information of the input object. The moving image characterized in that the vector detection means detects between the macroblock already compensated by the padding process and the reference image padded reference image when detecting the motion vector of the texture information. Image encoding device. Shape coding and texture coding in a coding method of a moving image coding method for coding a desired object constituting an image as an arbitrary shape object composed of shape information and texture information in units of macro blocks Prior to conversion, the shape signal input to the encoding means is referred to, and boundary macroblocks that include an object as a part are subjected to padding processing for eliminating discontinuities in the macroblock. A moving picture coding method characterized in that all pixels in a macroblock to be texture coded are compensated by rubbing, and shape coding and texture coding are performed using the padded image.

Images