JP2008011564A - Image encoding device and image decoding device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a device which encodes the shape information of an object efficiently. <P>SOLUTION: An encoding circuit for an alphamap representing an object (Ob) shape in a motion image encoding device that encodes a multi-frame motion image signal obtained as time-series data per an arbitrary shaped object has a means to split a rectangular region including the Ob for a block (BL) composed of M×N pixels and a means 220 to sequentially encode the BL with a fixed rule in the rectangular region, and a binarization image encoding device which applies relative address encoding for all or a part of the BL has a means to store a BL neighborhood regenerative value, a regenerative signal retention means FM to store a regenerative signal for the already-encoded frame (FM), a predicting means 250 to generate a motion compensation predictive value using the regenerative signal in the retention method, and a means to detect a changing pixel including the BL neighborhood regenerative value, and a reference changing pixel for the relative address encoding is calculated from the motion compensation predictive value not from a pixel value in the BL. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an image encoding device and an image decoding device for encoding, transmitting, storing, and decoding an image signal with high efficiency.

  Since an image signal has an enormous amount of information, it is generally compressed and encoded when used for transmission or storage. In order to encode an image signal with high efficiency, an image in units of frames is converted into a required number of pixels (for example, M × N pixels (M: number of pixels in the horizontal direction, N: number of pixels in the vertical direction)). The image is divided into blocks, orthogonally transformed for each of the divided blocks, the spatial frequency of the image is separated into frequency components, acquired as transform coefficients, and encoded.

  By the way, as one of the image coding, “JYAWang et.al.“ Applying Mid-level Vision Techniques for Video Data Compression and Manipulation ”, MITMedia Lab.Tech.Report No.263, Feb.1994,” An image encoding method belonging to a category called mid-level encoding has been proposed.

  In this method, for example, assuming that there is an image composed of a background and a subject (hereinafter, the subject is referred to as an object) as shown in FIG. 16A, the background and the object are shown in FIGS. 16B and 16C. They are encoded separately.

  Thus, in order to separately encode the background (FIG. 16C) and the object (FIG. 16B), for example, binary sub-image information representing the shape of the object and the position in the screen is used. An alpha map signal (see FIG. 16D, in which white pixels indicate object pixels) is required. The background alpha map signal (FIG. 16E) is uniquely obtained from the alpha map signal of the object.

  By the way, as a method of efficiently encoding the alpha map signal, a binary image encoding method (for example, MMR (Modified Modified READ) encoding) or a line figure encoding method (chain encoding or the like) is used. Is used.

  In order to further reduce the amount of alpha map code, J. Ostermann, “Object-based analysis-synthesis coding based on the souece model of moving rigid” 3D objects ”, Signal Process.:Image Comm.Vol.6 No.2 pp.143-161,1994) and encoding method by reducing the alpha map and approximating the curve when enlarging (Japanese Patent Application No. Hei 5- No. 297133).

  Here, an example of a method of approximating a curve when the alpha map is reduced and encoded and enlarged will be described.

<Specific Example of MMR Encoding Circuit>
(Specific example of block-based coding)
FIG. 18A shows the relationship of change pixels when encoding is performed in block units (for example, block units in an M × N pixel configuration (M: number of pixels in the horizontal direction, N: number of pixels in the vertical direction)). FIG. FIG. 18B is a diagram illustrating a reference area for detecting the reference change pixel b1.
In block-based encoding, encoding of changed pixels may be simplified as follows. The following processing may be performed by switching the scan order or may be applied to a reduced block.

The simplified change pixel encoding is performed as follows.
Now, the addresses (or pixel order) of the change pixel ai (i = 0 to 1) and the reference change pixel b1 obtained in raster order from the upper left of the screen are abs_ai (i = 0 to 1), abs_b1, respectively. When the line to which the change pixel a0 belongs is expressed as a0_line, the values of a0_line, r_ai (i = 0 to 1), and r_b1 are obtained by the following equations.

a0_line = (int) ((abs_a0 + WIDTH) / WIDTH) -1
r_a0 = abs_a0-a0_line * WIDTH
r_a1 = abs_a1-a0_line * WIDTH
r_b1 = abs_b1− (a0_line−1) * WIDTH
In the above formula, * means multiplication, (int) (X) means truncation after the decimal point of X, and WIDTH indicates the number of pixels in the horizontal direction of the block.

  Then, the reproduction value is obtained by encoding the value of the relative address “r_a1-r_b1” or “r_a1-r_a0” of the change pixel. This is the above-described “simplified change pixel encoding”.

  FIG. 19 is a flowchart showing the flow of processing when MMR encoding, which is a binary image encoding method, is performed on a block basis. Hereinafter, the encoding process will be described according to the flowchart. In this process, first, the position of the starting point change pixel is initialized (S501), the pixel value at the initial position (the upper left pixel of the block) is encoded with 1 bit (S502), and then the reference change is performed at the initial position. The pixel b1 is detected (S503).

  Here, when the reference change pixel b1 is not detected, since the change mode does not exist in the reference area, the vertical mode cannot be used. Therefore, the state of the vertical pass mode is set to “TRUE” (true) and b1 is detected. In this case, since the vertical mode can be used, the state of the vertical path mode is set to “FALSE” (false).

The initial setting is thus completed, and the process proceeds to the encoding loop process.
First, the change pixel a1 is detected (S505), and it is determined whether or not the change pixel a1 is detected (S506). If the change pixel a1 is not detected, then there is no change pixel, so that encoding is performed. An encoding process end code (EOMB; End of MB) indicating the end of is encoded (S507).

  If the change pixel a1 is detected as a result of the determination in S506, the state of the vertical pass mode is determined (S508). If the vertical path mode state is “TRUE”, vertical path mode encoding processing (S516) is performed. If the vertical path mode state is “FALSE”, b1 is detected (S509).

  Next, it is determined whether or not b1 has been detected (S510). If the reference change pixel b1 has not been detected, the process proceeds to step S513 in the horizontal mode, and if the reference change pixel b1 has been detected. Determines whether or not the absolute value of “r_a1−r_b1” is larger than the threshold value (VTH) (S511). The process proceeds to S512), and if larger than the threshold value (VTH), the process proceeds to step S513 in the horizontal mode.

  In the horizontal mode step (S513), the value of “r_a1-r_a0” is encoded. Here, it is determined whether or not the value of “r_a1-r_a0” is smaller than “WIDTH” (S514). As a result, if the value is “WIDTH” or more, the state of the vertical path mode is set to “TRUE” (true ) (S515), the process proceeds to the step (S516) of the vertical pass mode. When the step (S516) of the vertical pass mode is completed, the state of the vertical pass mode is set to “FALSE” (false).

  As described above, after any of the vertical mode, the horizontal mode, and the vertical pass mode is completed (after the encoding up to al is completed), the position of a1 is set as a new position of a0 (S518), and the process returns to S505.

FIG. 17 is an example of a VLC table (variable length code table).
Here, when the state of the vertical path mode is “TRUE”, only three types of codes, V0 (mode V0), H (horizontal mode), and EOMB (encoding processing end code), occur. The VLC can be switched according to the state of the pass mode. When the state of the vertical path mode is “TRUE”, EOMB occurs only when a0 is at the upper left position (initial position) of the block. Therefore, in this case, the code in “0” of FIG. 17 is used.
JYA Wang et. Al. “Applying Mid-level Vision Techniques for Video Data Compression and Manipulation”, MIT Media Lab. Tech. Report No. 263, Feb. 1994 J. Ostermann, “Object-based analysis-synthesis coding based on the souece model of moving rigid 3D objects”, Signal Process.:Image Comm. Vol. 6 No. 2 pp. 143-161,1994

  When encoding an image, there is a method that divides the inside of the screen into a background and an object, and in this case, in order to separate the background and the object, an alpha map signal indicating the shape of the object and the position in the screen is used. Is required. The information of the alpha map as well as the encoded information of the image is also encoded to form a bit stream for transmission and storage.

  However, in the method of encoding by dividing the inside of the screen into the background and the object, the amount of code increases by the amount of alpha map compared to the case of encoding in the screen all at once as in the conventional encoding method. There is a problem, and a decrease in encoding efficiency due to an increase in the code amount of the alpha map becomes a problem.

  Accordingly, an object of the present invention is to efficiently encode the information of the alpha map, which is sub-image information representing the shape of the object, the position in the screen, and the like, and an image code that can be decoded. It is to provide an encoding device and an image decoding device.

  In order to achieve the above object, the present invention is configured as follows.

First, in an image encoding apparatus that encodes an image together with an alpha map that is information for distinguishing the object region and the background region of the image, the alpha map is encoded using relative address encoding.
A change pixel that has already been encoded is used as a reference change pixel, and a symbol representing the relative position of the reference change pixel and the change pixel to be encoded next is encoded using a variable-length encoding table; Means for holding two or more long coding tables and switching the variable length coding table according to the already-encoded alpha map pattern.

The second is a decoding device for decoding the encoded bitstream obtained by encoding by this encoding device,
Means for decoding the symbols using a variable-length coding table; and means for holding two or more variable-length coding tables and switching the variable-length coding table according to the already-decoded alpha map pattern. It is characterized by providing.

  Furthermore, the means for switching the variable length coding table is switched by a pattern near the reference change pixel.

  This apparatus configured as described above is capable of performing a plurality of types of variable in encoding / decoding in which the amount of code is reduced by encoding a symbol that specifies the position of a change pixel using a variable-length encoding table. A long encoding table is prepared, and the variable length encoding table is switched according to the already encoded alpha map pattern. According to the present invention, the alpha map An effect of further reducing the code amount can be obtained.

Second, the present invention relates to an alpha map representing the shape of an object in a moving image encoding apparatus that encodes a plurality of frames (image frames) of moving image signals obtained as time series data for each object having an arbitrary shape. Means for dividing a rectangular region including an object into blocks each composed of M × N pixels (M: number of pixels in the horizontal direction, N: number of pixels in the vertical direction); A binary image code which has means for sequentially encoding the blocks obtained by the division in the square area according to a certain rule, and applies relative address encoding to all or a part of the blocks; In the conversion device,
Using reproduction value storage means for storing the reproduction values in the vicinity of the block, image holding means (frame memory) for storing a reproduction signal of an already encoded frame (image frame), and reproduction signals in the image holding means (frame memory) A reference value for relative address encoding, and a motion compensation prediction circuit for generating a motion compensation prediction value and a means for detecting a change pixel including a reproduction value near a block with reference to the reproduction value storage means. The pixels are obtained from the motion compensated prediction signal, not from the pixel values in the block.

  Further, in the alpha map decoding circuit, for each block composed of M × N pixels, means for sequentially decoding the rectangular area including the object according to a certain rule, means for storing a reproduction value in the vicinity of the block, and code An image holding means (frame memory) for storing a reproduction signal of the converted frame (image frame), a motion compensation prediction circuit for generating a motion compensation prediction value using the reproduction signal in the image holding means (frame memory), Means for detecting a change pixel including a reproduction value in the vicinity of the block, and the reference change pixel of the relative address encoding is obtained not from the pixel value in the block but from the motion compensated prediction signal. is there.

  As a result, it is possible to efficiently encode and decode the alpha map information, which is the sub-image information representing the shape of the object, the position in the screen, and the like.

  Also, using means for storing reproduction values in the vicinity of the block, image holding means (frame memory) for storing reproduction signals of already encoded frames (image frames), and reproduction signals in the image holding means (frame memory) A motion compensation prediction circuit for generating a motion compensation prediction value, means for detecting a change pixel including a reproduction value near the block, and a reference change pixel of relative address encoding obtained from the reproduction pixel value in the block And means for switching the reference change pixel of the relative address encoding obtained from the motion compensation prediction signal, and encodes the relative address encoding information together with the switching information.

  Further, in the alpha map decoding circuit, for each block composed of M × N pixels, means for sequentially decoding the rectangular area including the object according to a certain rule, means for storing a reproduction value in the vicinity of the block, and code An image holding means (frame memory) for storing a reproduction signal of the converted frame (image frame), a motion compensation prediction circuit for generating a motion compensation prediction value using the reproduction signal in the image holding means (frame memory), Means for detecting a change pixel including a reproduction value in the vicinity of the block, and obtained from a reference change pixel of relative address encoding obtained from the reproduction pixel value in the block and a motion compensation prediction signal. Means for switching the reference change pixel of relative address encoding is provided, and the reference change pixel is obtained according to the switching information.

  In this case, in relative address encoding, whether the reference change pixel b1 is detected from the “current block” that is the block of the image currently being processed or the “compensated block” that is the block of the previously processed image Can be switched and processed in units of blocks, the encoding side also encodes this switching information, the decoding side decodes this, and the decoding information is converted into the switching information during decoding processing. On the basis of this, it is possible to switch the reference change pixel b1 from the “current block” or the “compensated block” to be detected in units of blocks. Optimal processing is possible based on the image content, and more efficient encoding is possible.

The present invention is also a moving image encoding apparatus that encodes a plurality of frames of moving image signals obtained as time-series data for each object having an arbitrary shape, and a rectangular region including the object is represented by M × N pixels (M: horizontal). An image that is divided into blocks each having a number of pixels in the direction, N: the number of pixels in the vertical direction), and that is obtained by sequentially encoding the inside of the rectangular region according to a certain rule for each of the blocks obtained by the division. In the encoding device,
For an alpha map signal representing the shape of an object, a frame memory that stores a reproduction signal of the frame including a reproduction signal near the block, and a reproduction signal of a frame that has been encoded;
Means for replacing all pixel values in a block with one of two values;
Motion-compensated prediction means for generating a motion-compensated prediction value using a reproduction signal of a frame that has already been encoded in the frame memory;
Means for reducing / enlarging the binary image for each block;
Means for encoding the reduction / enlargement ratio as side information;
An alpha map encoding means comprising a binary image encoding means for encoding a binary image reduced for each block is provided.

  Then, the alpha map encoding means reduces or enlarges the reproduction image of the block for each block, a reproduction value obtained by replacing all the values in the block with one of the two values, a motion compensation prediction value, and the like. One of the obtained reproduction values is selected. Therefore, the encoding of the alpha map signal can be performed in a high-quality and efficient form, the high-quality image quality can be maintained, and the encoding can be performed with a high compression rate.

Further, it is a moving picture decoding apparatus that decodes a plurality of frames of moving picture signals obtained as time series data for each object having an arbitrary shape, and includes M × N pixels (M: number of pixels in the horizontal direction, N : In an image decoding apparatus that sequentially decodes the rectangular area according to a certain rule for each block composed of:
A frame memory for storing a reproduction signal of the frame including a reproduction signal in the vicinity of the block, and a reproduction signal of an encoded frame;
Means for replacing all pixel values in a block with one of two values;
Motion-compensated prediction means for generating a motion-compensated prediction value using a reproduction signal of a frame that has already been encoded in the frame memory;
Means for reducing / enlarging the binary image for each block;
An alpha map decoding unit comprising a binary image decoding unit for decoding a binary image reduced for each block is provided.

  The alpha map decoding means reduces or enlarges the reproduction image of the block for each block, a reproduction value obtained by replacing all of the inside of the block with one of the two values, a motion compensation prediction value, and the like. It was made to select from any one of the reproduction values obtained in (1). Therefore, a high-quality image can be reproduced.

Further, when encoding an alpha map for each block, a method for encoding an attribute for each block, including an object, a means for setting an encoding area represented by a multiple of a block size,
Means for dividing the area into blocks;
Labeling means for assigning a unique label to each attribute for each block; storage means for holding the label for each frame;
Determining means for determining a reference block of a previous frame corresponding to an encoded block of a current frame;
Prediction means for determining a prediction value by at least the label of the previous frame held in the storage means and the reference block;
The encoding block label information is configured to include encoding means for encoding using the prediction value.

In addition, as a decoding device that reproduces the attribute of each block of the alpha map,
Storage means for holding the decrypted label for each frame;
Determining means for determining a reference block of a previous frame corresponding to a decoded block of a current frame;
Prediction means for determining a prediction value by at least the label of the previous frame held in the storage means and the reference block;
The decoding block label information is configured to include decoding means for decoding using the predicted value.

  As a result, when the alpha map is encoded for each macro block (the divided unit image block when the image is divided into a predetermined plurality of pixel configurations such as 16 × 16 pixels), it is specific to the attribute of each block. By attaching a label and encoding it, and reproducing the label and reproducing the original alpha map data, efficient encoding can be performed.

Further, the present invention encodes an image together with an alpha map that is information for distinguishing the object area and the background area of the image. When the alpha map is encoded for each block, the attribute for each block is set. An image encoding device of a method for encoding,
Means for setting an encoding area including an object expressed by a multiple of the block size, means for dividing the area into blocks, and labeling means for assigning a unique label to each attribute for each block Storage means for holding the label or the alpha map for each frame, determination means for determining a reference block of the previous frame corresponding to the encoded block of the current frame, and at least the previous frame held in the storage means And an alpha map, a predicting unit that determines a predicted value by the reference block, and an encoding unit that encodes the label information of the encoded block using the predicted value.

  Furthermore, storage means for holding the reduction / enlargement ratio in units of frames is provided, and the encoding means can change the reduction / enlargement ratio of the frames in units of frames, and encodes corresponding to the reduction / enlargement / reduction ratios. And the determining means uses the reduction / enlargement ratio of the current frame and the reduction / enlargement ratio of the previous frame obtained from the storage means to determine the previous frame corresponding to the encoded block of the current frame. Means for determining a reference block are provided.

  Alternatively, storage means for holding the reduction / enlargement ratio in units of frames is provided, and the encoding means can change the reduction / enlargement ratio of the frames in units of frames and performs encoding corresponding to the reduction / enlargement / reduction ratios. Means for determining a reference to a previous frame corresponding to an encoded block of the current frame using the reduction / enlargement ratio of the current frame and the reduction / enlargement ratio of the previous frame obtained from the storage means. A means for determining a block is provided, and when there are a plurality of reference blocks, the prediction means is provided with a means for setting a label having a large number among the labels of the plurality of reference blocks as a predicted value.

  Alternatively, storage means for holding the reduction / enlargement ratio in units of frames is provided, and the encoding means encodes the reduction / enlargement ratio corresponding to the reduction / enlargement / reduction ratios in which the reduction / enlargement ratio of the frames is variable in units of frames. Means for encoding a coding block using a variable length coding table selected from a plurality of types according to the reduction / enlargement ratio of the previous frame or the current frame or both. And the determination means uses the current frame reduction / enlargement ratio and the previous frame reduction / enlargement ratio obtained from the storage means to determine the reference block of the previous frame corresponding to the encoded block of the current frame. Means for determining;

Also, in a decoding device that reproduces the attribute of each block of the alpha map,
Storage means for holding the decoded label or alpha map for each frame, determination means for determining a reference block of the previous frame corresponding to the decoded block of the current frame, and label of the previous frame held by at least the storage means Or it has the prediction means which determines a predicted value with an alpha map and the said reference block, and the decoding means which decodes the label information of the said decoding block using the said predicted value, It is characterized by the above-mentioned.

Further, the frame reduction / enlargement rate is variable in units of frames, and means for decoding the reduction / enlargement rate information,
Storage means for holding the reduction / enlargement information,
The determination means has a function of determining a reference block of the previous frame corresponding to a decoded block of the current frame using the reduction / enlargement ratio of the current frame and the reduction / enlargement ratio of the previous frame read from the storage means. It is characterized by comprising.

Alternatively, the frame reduction / enlargement rate is variable on a frame basis, and means for decoding the reduction / enlargement rate information;
Storage means for holding the reduction / enlargement information,
The determination means has a function of determining a reference block of the previous frame corresponding to a decoded block of the current frame using the reduction / enlargement ratio of the current frame and the reduction / enlargement ratio of the previous frame read from the storage means. The predicting means is characterized in that, when there are a plurality of reference blocks, a label having a large number among the labels of the plurality of reference blocks is used as a predicted value.

As an enlargement circuit for enlarging a block of a binary image reduced to 1 / 2N (N = 1, 2, 3,...) In both the horizontal and vertical directions,
A memory for holding the reproduction value near the block, a means for obtaining a reference pixel value by reducing the reproduction value held in the memory to 1 / 2N according to the reduction ratio of the block, and a horizontal and vertical double. Means for enlarging to the original size by repeating the process of enlarging N times N times, and the enlarging means always uses a reference pixel value reduced to 1 / 2N.

Further, the present invention is a moving image encoding apparatus that encodes a plurality of frames of moving image signals obtained as time series data for each object having an arbitrary shape, and includes a square area including the object of M × N pixels (M: horizontal pixels). And N: the number of pixels in the vertical direction), and an image encoding device that sequentially encodes the inside of the rectangular area according to a predetermined rule for each of the blocks obtained by the division. Because
Setting means for setting an area represented by a multiple of the block size including the object in the frame, a dividing means for dividing the area set by the setting means into blocks, and a motion in the divided block In an image coding apparatus having means for predictively coding a motion vector necessary for compensated prediction,
A memory for holding a first position vector representing the position of the region in the reference frame in the frame; an encoding means for encoding a second position vector representing the position of the region in the reference frame in the frame; A motion vector memory that holds a motion vector of a reproduced block near the encoding target block, and means for predicting a motion vector of the encoding target block using the motion vector stored in the motion vector memory. And
When the motion vector used in the prediction means does not exist in the motion vector memory, a default motion vector is used as a predicted value, and the default motion vector is a difference vector between the first position vector and the second position vector. The zero vector is switched and used.

Further, it is a moving picture decoding apparatus that decodes a plurality of frames of moving picture signals obtained as time series data for each object having an arbitrary shape, and includes M × N pixels (M: number of pixels in the horizontal direction, N : A picture decoding apparatus that sequentially decodes the rectangular area in accordance with a certain rule for each block composed of: (number of pixels in the vertical direction), and is represented by a multiple of the block size including the object in the frame. In an image decoding device that reproduces a block area for each block,
Means for decoding motion-predicted motion vectors necessary for motion compensation prediction in the block, and means for decoding motion-predicted motion vectors necessary for motion compensation prediction in the reference frame And a memory for holding a first position vector representing the position of the region in the reference frame within the frame, and a means for decoding the second position vector representing the position of the region within the frame within the frame; A motion vector memory that holds a motion vector of a corrected block near the decoding target block, and a prediction unit that predicts a motion vector of the decoding target block using the motion vector held in the motion vector memory. Have
If a motion vector to be used by the prediction means does not exist in the motion vector memory, a default motion vector is used as a predicted value, and the default motion vector is a combination of the first position vector and the second position vector. It is characterized by being either a difference vector or a zero vector.

  According to the present invention, since the amount of alpha map code can be reduced, encoding is performed separately for each object without a significant decrease in encoding efficiency compared to the conventional encoding method in which encoding is performed in units of frames. Will be able to.

  Hereinafter, specific examples of the present invention will be described with reference to the drawings. First, an outline of an image encoding and image decoding apparatus to which the present invention is applied will be described.

(Image coding and image decoding apparatus to which the present invention is applied)
FIG. 1 is a block diagram of an image encoding apparatus to which the present invention is applied, which is a method of encoding an image by dividing the screen into a background and an object. As shown in FIG. 1, the image coding apparatus according to the present invention includes a difference circuit 100, a motion compensation prediction circuit (MC) 110, an orthogonal transformation circuit 120, a quantization circuit 130, a variable length coding circuit (VLC) 140, and an inverse circuit. A quantization circuit (IQ) 150, an inverse orthogonal transform circuit 160, an adder circuit 170, a multiplexing circuit 180, and an alpha map encoding circuit 200 are configured.

  The alpha map encoding circuit 200 encodes the input alpha map and outputs the encoded signal to the multiplexing circuit 180 as an alpha map signal, and decodes the alpha map signal as a local decoded signal. Has a function to output.

  In particular, the present alpha map encoding circuit 200 performs a process of reducing the resolution at a given reduction rate (magnification) when encoding the input alpha map, and encodes the resolution reduced process. At the same time, it has a function of multiplexing the encoded data and reduction rate information (magnification information) and outputting the multiplexed data to the multiplexing circuit 180 as an alpha map signal. As the local decoded signal, a signal obtained by performing processing for returning the resolution-reduced signal to the original resolution is used.

  The difference circuit 100 calculates a difference signal between the motion compensation prediction signal supplied from the motion compensation prediction circuit 110 and the input image signal, and the orthogonal transformation circuit 120 converts the difference signal supplied from the difference circuit 100 into the difference signal. According to the information of the alpha map, it is converted into an orthogonal transform coefficient and output.

  The quantization circuit 130 is a circuit that quantizes the orthogonal transform coefficient obtained by the orthogonal transform circuit 120, and the variable length coding circuit 140 codes and outputs the output of the quantization circuit 130. The multiplexing circuit 180 multiplexes and multiplexes the signal encoded by the variable length encoding circuit 140 and the alpha map signal together with side information such as motion vector information and outputs the result as a bit stream.

  The inverse quantization circuit 150 performs inverse quantization on the output of the quantization circuit 130, and the inverse orthogonal transform circuit 160 performs inverse orthogonal transform on the output of the inverse quantization circuit 150 based on the alpha map. The adder circuit 170 adds the output of the inverse orthogonal transform circuit 160 and the prediction signal (motion compensation prediction signal) given from the motion compensation prediction circuit 110 and outputs the result to the difference circuit 100.

  The motion compensation prediction circuit 110 has a frame memory and has a function of accumulating an object area signal and a background area signal by operating based on a local decoded signal supplied from the alpha map decoding circuit 200. The motion compensation prediction circuit 110 also predicts a motion compensation value from the accumulated image of the object region and outputs it as a prediction value, and predicts a motion compensation value from the accumulated image of the background region and outputs it as a prediction value. Have

  The operation of the apparatus having such a configuration will be described. The apparatus receives an image signal and an alpha map of the image signal.

  Of these, the image signal is divided into blocks each having a predetermined pixel size (for example, M × N pixels (M: number of pixels in the horizontal direction, N: number of pixels in the vertical direction)) for each frame, and then the blocks The signals are supplied to the difference circuit 100 via the signal line 10 in the order of position. The difference circuit 100 calculates a difference signal between the input (image signal) and the prediction signal (output of the motion compensation prediction signal from the object prediction circuit 110) and supplies the difference signal to the orthogonal transformation circuit 120.

  In the orthogonal transform circuit 120, the supplied difference signal is converted into an orthogonal transform coefficient in accordance with the information of the alpha map supplied from the alpha map encoding circuit 200 via the signal line 40, and then supplied to the quantization circuit 130. To do. And it is quantized here. The transform coefficient obtained by quantization by the quantization circuit 130 is encoded by the variable length encoding circuit 140 and supplied to the inverse quantization circuit 150.

  The transform coefficient supplied to the inverse quantization circuit 150 is inversely quantized here, and then inverse transformed in the inverse orthogonal transform circuit 160. Then, the addition circuit 170 adds the motion compensation prediction value supplied from the motion compensation prediction circuit 110, outputs it as a locally decoded image, and inputs it again to the motion compensation prediction circuit 110.

  The locally decoded image that is the output of the adder circuit 170 is stored in the frame memory in the motion compensation prediction circuit 110.

  On the other hand, the motion compensation prediction circuit 110 generates an “object motion compensation prediction value” at the processing timing of the block in the object region based on the local decoded signal given from the alpha map decoding circuit 200, and other than that. At the timing, the “background portion motion compensation predicted value” is output and supplied to the difference circuit 100.

  That is, in the motion compensation prediction circuit 110, whether the image signal of the block corresponding portion of the object is currently input to the difference circuit 100 from the local decoded signal of the alpha map signal, or the image signal of the block corresponding portion of the background portion is the difference circuit. If it is during the input period of the image signal of the object corresponding to the block of the object, the motion compensation prediction signal of the object and the image signal input period of the block corresponding part of the background portion may be detected. For example, the background motion compensation prediction signal is supplied to the difference circuit 100.

  The difference circuit 100 calculates the difference between the input image signal and the prediction signal corresponding to the region of the image. As a result, if the input image is in the region corresponding to the object, the corresponding position of the object If the difference signal from the predicted value in FIG. 4 is a background region, the difference signal from the predicted value corresponding to the background position is calculated and supplied to the orthogonal transformation circuit 120.

  In the orthogonal transform circuit 120, the supplied differential signal is converted into an orthogonal transform coefficient according to the information of the alpha map supplied via the signal line 40, and then supplied to the quantization circuit 130. The orthogonal transform coefficient is quantized by the quantization circuit 130.

  The transform coefficient quantized by the quantization circuit 130 is encoded by the variable length encoding circuit 140 and supplied to the inverse quantization circuit 150. The transform coefficient supplied to the inverse quantization circuit 150 is inversely quantized here, and then inverse transformed in the inverse orthogonal transform circuit 160 and supplied to the adder circuit 170. Then, the predicted value supplied to the adding circuit 170 via the predicted value switching circuit 500 is added.

  The signal of the locally decoded image that is the output of the adder circuit 170 is supplied to the motion compensation prediction circuit 110. In this motion compensation prediction circuit 110, whether the signal corresponding to the block of the object is currently output from the local decoding signal of the alpha map signal, or whether the signal corresponding to the block of the background portion is output. As a result, if the signal corresponding to the block of the object is being output, it is applied to the frame memory for the object, and if the signal corresponding to the block of the background portion is being output, the operation is performed to the background memory. And store it in the corresponding memory.

  Thus, only the object image is obtained in the object frame memory, and only the background image is obtained in the background memory. As a result, the motion compensation prediction circuit 110 can obtain the predicted value of the object image using the object image, and can obtain the predicted value of the background image using the image of the background portion.

  As described above, the alpha map encoding circuit 200 encodes an input alpha map, and supplies the encoded alpha map signal to the multiplexing circuit 180 via the signal line 30.

  Further, the transform coefficient output from the variable length coding circuit 140 is supplied to the multiplexing circuit 180 via the line 40. The multiplexing circuit 180 multiplexes the supplied alpha map signal and the encoded value of the transform coefficient together with side information such as motion vector information, and then outputs the multiplexed image signal via the signal line 50 to output the main image code. The encoded bit stream is the final output of the encoding device.

  The above is the configuration and operation of the encoding device. When obtaining an error signal of an image, the current block position of the image being processed according to the alpha map is the object region in order to perform motion compensation prediction using the object and background images. If the current block position of the image being processed is the object area position, the predicted value obtained from the object image is used while determining whether the current position is the position or the background area position. The difference is obtained using the predicted value obtained from the image for use.

  Then, for object and background prediction, the motion compensation prediction circuit holds the images of the corresponding region portions of the images obtained from the differences according to the alpha map, and is used for the prediction. As a result, optimal motion compensation prediction can be performed for each object and background, enabling high-quality image compression encoding and decoding.

  On the other hand, FIG. 2 is a block diagram of a decoding apparatus in which the present invention is used. As shown in FIG. 2, the decoding apparatus includes a separation circuit 300, a variable length decoding circuit 310, an inverse quantization circuit 320, an inverse orthogonal transform circuit 330, an addition circuit 340, a motion compensation prediction circuit 350, an alpha map decoding. The circuit 400 is constituted.

  The separation circuit 300 is a circuit that obtains an alpha map signal and a coded image signal by separating the input encoded bit stream, and the alpha map decoding circuit 400 is separated by the separation circuit 300. This circuit decodes the alpha map signal and reproduces the alpha map.

  The variable length decoding circuit 310 decodes the encoded signal of the image separated by the separation circuit 300, and the inverse quantization circuit 320 inversely quantizes the decoded signal to return to the original coefficient. The inverse orthogonal transform circuit 330 performs inverse orthogonal transform on the coefficients in accordance with the alpha map to return the prediction error signal to the prediction error signal. The adder circuit 340 applies the motion compensation from the motion compensation prediction circuit 350 to the prediction error signal. The predicted value is added and output as a reproduced image signal. This reproduced image signal becomes the final output of the decoding apparatus.

  The motion compensation prediction circuit 350 obtains an object image and a background image by accumulating the reproduced image signal output from the adder circuit 340 in a frame memory according to an alpha map, and obtains an object image from the accumulated image. A motion compensation prediction signal and a background motion compensation prediction are obtained.

  In the decoding apparatus having such a configuration, the encoded bit stream is supplied to the demultiplexing circuit 300 via the line 70 and is separated for each piece of information by the demultiplexing circuit 300, so that the alpha map signal is related. It is divided into a code and a variable length code of the image signal.

  The code relating to the alpha map signal is supplied to the alpha map decoding circuit 400 via the signal line 80, and the variable length code of the image signal is supplied to the variable length decoding circuit 310, respectively.

  The code relating to the alpha map signal is reproduced as an alpha map signal by the alpha map decoding circuit 400 and output to the inverse orthogonal transform circuit 330 and the motion compensation prediction circuit 350 via the signal line 90.

  On the other hand, the variable length decoding circuit 310 decodes the code supplied from the demultiplexing circuit 300 and supplies it to the inverse quantization circuit 320, where it is inversely quantized. The inversely quantized transform coefficient is inversely transformed by the inverse orthogonal transform circuit 330 according to the alpha map supplied via the line 90 and supplied to the adder circuit 340. The adder circuit 340 adds the inversely orthogonally transformed signal from the inverse orthogonal transform circuit 330 and the motion compensation prediction signal supplied from the motion compensation prediction circuit 350 to obtain a reproduced image.

  The above is the outline of the image coding apparatus and the image decoding apparatus to which the present invention is applied. The present invention relates to an alpha map encoding circuit 200 that is a component of the encoding device shown in FIG. 1 and an alpha map decoding circuit 400 that is a component of the decoding device shown in FIG. An embodiment is shown.

  Prior to describing the details of the main part of the present invention, the alpha map encoding circuit 200 as the prior art will be mentioned. FIG. 3 is a block diagram showing a configuration of an alpha map encoding circuit 200 in the prior art (Japanese Patent Application No. 8-237053). As shown in the figure, an alpha map encoding circuit 200 in the prior art includes a resolution conversion circuit (reduction circuit) 210, an (enlargement circuit) 230, a binary image encoding circuit (block-based MMR encoder) 220, a multiplexing circuit. 240.

  Among these, the resolution conversion circuit 210 is a conversion circuit for resolution reduction conversion, encodes an alpha map at a reduction rate according to a given enlargement rate, and the resolution conversion circuit 230 is a conversion circuit for resolution reduction / enlargement conversion. And it has a function which encodes an alpha map with the expansion rate according to the expansion rate given.

  The resolution conversion circuit 230 is provided in order to restore the resolution converted by the resolution conversion circuit 210 to the original size, and the alpha map restored to the original size by the resolution conversion circuit 230 is transmitted via the signal line 40. It becomes an alpha map local decoded signal given to the orthogonal transformation circuit 120 and the inverse orthogonal transformation circuit 160.

  The binary image encoding circuit 220 performs binary image encoding on the alpha map signal that has been subjected to resolution reduction conversion output from the resolution conversion circuit 210, and outputs the binary image encoding output. The information of the enlargement ratio given through the line 60 is multiplexed and output.

  In the alpha map encoding circuit 200 having such a configuration, the alpha map input via the alpha map signal input line 20 is reduced and encoded at a specified enlargement rate by the resolution conversion circuit 210, and the encoded alpha map is displayed. The map signal is output via the signal line 30, and the local decoded signal obtained by decoding the reduced-encoded alpha map signal to the original resolution by the resolution conversion circuit 230 is converted to the orthogonal conversion circuit via the signal line 40. 120, and output to the inverse orthogonal transform circuit 160.

  That is, by supplying the desired reduction / enlargement ratio setting information to the alpha map encoding circuit 200 via the signal line 60, the above trade-off can be achieved.

  The setting information signal of the reduction / enlargement ratio supplied via the signal line 60 is supplied to the resolution conversion circuits 210 and 230 and the binary image encoding circuit 220, and the generated code amount of the alpha map signal can be controlled. It becomes. Further, the code of the reduction / enlargement ratio (setting information signal) supplied via the signal line 60 is multiplexed with the encoded alpha map signal by the multiplexing circuit 240 and output via the signal line 30. Then, it is supplied as an alpha map encoded signal to the multiplexing circuit 180 which is the final output stage of the image encoding apparatus.

  Next, the alpha map decoding circuit 400 as a prior art will be mentioned. FIG. 4 shows a specific alpha map decoding circuit 400 in the prior art (Japanese Patent Application No. 8-237053).

  As shown in the figure, the alpha map decoding circuit 400 includes a binary image decoding circuit 410 (block-based MMR decoder), a resolution conversion circuit 420, and a separation circuit 430.

  The demultiplexing circuit 430 determines the sign of the alpha map signal and the reduction / enlargement ratio from the alpha map signal separated by the demultiplexing circuit 300 in the image decoding apparatus shown in FIG. 2 and input to the alpha map decoding circuit 400. The binary image decoding circuit 410 converts the code of the alpha map signal into the code of the reduction / enlargement ratio that is given separately from the separation circuit 430. Therefore, this is a circuit for converting back to a binary image, and the resolution conversion circuit 420 converts the binary image according to the sign of the reduction / enlargement ratio given separately from the separation circuit 430 and outputs the result.

  In FIG. 4, the code supplied to the alpha map decoding circuit 400 via the signal line 80 is separated into the code of the alpha map signal and the code of the reduction / enlargement ratio by the separation circuit 430. It is output via line 82.

  The binary image decoding circuit 410 reproduces a reduced alpha map signal from the code of the alpha map signal supplied via the signal line 81 and the code of the reduction / enlargement ratio supplied via the signal line 82. , And supplied to the resolution conversion circuit 420 via the signal line 83. The resolution conversion circuit 420 reproduces the alpha map signal by enlarging the reduced alpha map signal to the original size from the sign of the reduction / enlargement ratio supplied via the signal line 82, and then via the signal line 90. Output.

  The above is the outline of the encoding circuit and the decoding circuit on which the present invention is applied.

  Next, details of the present invention applied to such an encoding circuit and decoding circuit will be described. First, a variable length coding table (VLC table) is created so that a short code is assigned to a symbol having a high appearance frequency when a symbol that specifies the position of a change pixel is coded using a variable length coding table. Thus, when the total code amount is reduced, the code amount is further reduced by adaptively switching the variable length coding table (VLC table) according to the already encoded / decoded alpha map pixel pattern. A specific example will be described.

(First specific example)
The present invention shown in the first specific example encodes a symbol specifying the position of a change pixel using a variable-length encoding table, and switches the variable-length encoding table according to the already-encoded alpha map pattern. It is characterized by this.

  A first specific example of the present invention will be described.

  In the binary image encoding circuit 220 in FIG. 3, a symbol (Mode) representing the position of the changed pixel is encoded using the variable length encoding table (VLC table) shown in FIG. 17. For example, when the vertical path mode is “FALSE” and “V0 (r_a1-r_b1 = 0 in the vertical mode)”, “1” is output as a code. Alternatively, when "V1 (| r_a1-r_b1 | = 1 in vertical mode)", "01s" (s: "0" or "1" depending on the sign of r_a1-r_b1) is output, and "EOMB (end of encoding)" ) ”,“ 0001 ”is output.

  On the other hand, the binary image decoding circuit 410 in FIG. 4 uses the same VLC table in FIG. 17 as that used for encoding. For example, when the input code is “1”, “V0 is regenerated”. If the input code is “0001”, “EOMB” is reproduced.

  In this case, if a variable length coding table (VLC table) is created so that a short code is assigned to a symbol having a high appearance frequency, the total code amount can be reduced.

  The present invention further reduces the code amount by adaptively switching the variable length coding table (VLC table) according to the pixel pattern of the alpha map that has already been coded / decoded.

<Configuration of Encoding Circuit of First Specific Example>
FIG. 5 is a block diagram showing a specific example of the present invention, showing the alpha map encoding circuit 200 of FIG. 1 or the binary image encoding circuit 220 of FIG. 3 in more detail.

  The alpha map signal 1 is input to an a1 detection circuit 2 and a memory 3 that holds an encoded alpha map. In the a1 detection circuit 2, the position 4 of the change pixel a1 described with reference to FIG. 18 and the like is detected and sent to the mode determination circuit 5. At the same time, the position 6 of the reference change pixel b 1 is sent from the memory 3 to the mode determination circuit 5.

  In the mode determination circuit 5, the mode is determined by the algorithm described with reference to FIG. 19, and the mode is sent to the encoding circuit 8 as a symbol 7 to be encoded.

  From the memory 3, the pattern 9 around the encoded reference change pixel b1 is sent to the table determination circuit 1-10. The table determination circuit 1-10 selects and outputs one of a plurality of variable length coding tables.

  Here, for example, as shown in FIG. 7, when there is an edge from the upper right to the lower left above the reference change pixel b1, the same edge often extends linearly below the reference change pixel b1. Among the pixels x1, x2, and x3, there is a high probability that a1 is in x1.

  Therefore, when the pattern above the reference change pixel b1 has such a pattern, a table in which a short code is assigned to VL1 (r_a1-r_b1 = −1) is used.

<Detailed example of table determination method>
A more specific example of the table determination method is shown in FIGS. Here, attention is focused on the upper two lines c0 to c5 of the reference change pixel b1 shown in FIG. If these pixels have the same value as the reference change pixel b1, “1” is set, and if they are different, “0” is set, and “0” and “1” are arranged in the order of c0 to c5 as shown in FIG.
The binary number converted to a decimal number is called a context number.

And corresponding to each context number, for example,
[When context number = 0]
V0 1
VL1 010
VR1 011
VL2 000010
VR2 000011
EOMB 0001
H 001
[When context number = 1]
V0 010
VL1 1
VR1 000010
VL2 011
VR2 000011
EOMB 0001
H 001
[When context number = 2]
A variable length coding table is prepared so that it is omitted below.

In this table,
VL1 represents r_a1-r_b1 = −1,
VL2 is r_a1-r_b1 = −2,
VR1 is r_a1-r_b1 = 1,
VR2 represents r_a1-r_b1 = 2.

  In FIG. 7, since the context number = 1, a table in which the above VL1 can be encoded with 1 bit is selected.

  Returning to FIG. 5 again, the description will be continued. The encoding circuit 8 determines the code 12 using the selected table 11 sent from the table determination circuit 1-10. Then, the determined code 12 is output.

  FIG. 6 shows in more detail the alpha map decoding circuit 400 as a component of the decoding device shown in FIG. 2 or the binary image decoding circuit 410 as a component of the decoding device shown in FIG. A block configuration diagram is shown. This decodes the code | symbol 12 produced | generated by the specific example of FIG.

  The code 12 is input to the decoding circuit 13.

  The memory 14 holds the alpha map decoded so far, and the pattern 15 around the reference change pixel b1 is sent to the table determination circuit 16.

  In the table determination circuit 16, one of a plurality of variable length coding tables is sent to the decoding circuit 13 as a selected table 17. The table determination algorithm is the same as that of the table determination circuit 1-10 in FIG.

  The symbol 18 is decoded by the table 17 and sent to the a1 reproducing circuit 19. The a1 reproduction circuit 19 obtains the position of a1 based on the symbol 18 and the position 1-20 of b1 sent from the memory 14, and reproduces the alpha map 1-21 up to a1.

  The reproduced alpha map 1-21 is output and held in the memory 14 for future decoding.

  As described above, the first specific example switches a plurality of predetermined variable length coding tables. FIG. 10 shows a specific example in which the table is dynamically modified according to the frequency of the symbols actually generated. Is shown as a second specific example.

(Second specific example)
<Configuration of Encoding Device of Second Specific Example>
In the first specific example described above, a plurality of predetermined variable length coding tables are switched. FIG. 10 shows a specific example in which the table is dynamically corrected according to the frequency of symbols that have actually occurred. . This is a configuration in which a counter 22 and a Huffman table generation circuit 23 are added to the configuration of FIG. 5 which is the configuration of the first specific example.

  The counter 22 receives the symbol 7 from the mode determination circuit 5 and the context number 24 from the table determination circuit 1-10. The counter 22 holds the number of occurrences of each symbol for each context number.

  After a certain period of time has elapsed, the number of occurrences 25 of each symbol is sent to the Huffman table generation circuit 23 for each context number.

In the Huffman table generation circuit 23, the encoding table 26 is generated by Huffman encoding (Fujita "Basic Information Theory" (Shokodo) pp. 52-53, 1987). The table 26 is sent to the table determination circuit 1-10, and the table of the corresponding context number is replaced with the table 26. This Huffman table is generated and replaced for all context numbers.

<Configuration of Decoding Device of Second Specific Example>
FIG. 11 shows a decoding apparatus for decoding the code generated in the specific example of FIG. The decoding apparatus as the first specific example shown in FIG. 11 is also configured by adding a counter 27 and a Huffman table generation circuit 28 to FIG.

  The operations of the counter 27 and the Huffman table generation circuit 28 are the same as those in FIG. As described above, in the first and second specific examples, encoding / decoding is performed in which the code amount is reduced by encoding the symbol specifying the position of the changed pixel using the variable-length encoding table. In the conversion, a plurality of types of variable-length coding tables are prepared, and the variable-length coding tables are switched according to the already-encoded alpha map pattern. According to the above, an effect of further reducing the code amount of the alpha map can be obtained.

  Next, a reference change pixel of relative address encoding is determined not from a pixel value in a block having a configuration of M × N pixels (M: the number of pixels in the horizontal direction and N: the number of pixels in the vertical direction) but from a motion compensation prediction signal. A specific example obtained will be described as a third specific example.

(Third example)
The third specific example is characterized in that a reference change pixel of relative address encoding is obtained not only from a pixel value in the block but also from a motion compensation prediction signal.

  FIG. 12 is a block diagram illustrating an alpha map encoding circuit as a third specific example. FIG. 13 is a block diagram illustrating an alpha map decoding circuit as a third specific example.

  The alpha map encoding circuit 200 and the alpha map decoding circuit 400 of the present invention will be described with reference to FIGS.

  In the third specific example, the alpha map encoding circuit 200 shown in FIG. 3 is configured as shown in FIG. 12, and the alpha map decoding circuit 400 shown in FIG. 4 is configured as shown in FIG.

  As shown in FIG. 12, the alpha map encoding circuit 200 of this specific example includes a resolution conversion circuit (reduction processing circuit) 210, a resolution conversion circuit (enlargement processing circuit) 230, and a binary image encoding circuit (block-). based MMR encoder) 220, a multiplexing circuit 240, and a motion compensation prediction circuit 250 and a reduction circuit 260.

  Among these, the resolution conversion circuit 210 is a conversion circuit for resolution reduction conversion, encodes an alpha map at a reduction rate according to a given reduction / enlargement rate setting information signal, and the resolution conversion circuit 230 reduces the resolution. This is a conversion circuit for enlargement conversion, and has a function of encoding an alpha map at an enlargement rate according to a given enlargement rate.

  The resolution conversion circuit 230 is provided in order to restore the resolution converted by the resolution conversion circuit 210 to the original size, and the alpha map restored to the original size by the resolution conversion circuit 230 is transmitted via the signal line 40. It becomes an alpha map local decoded signal given to the orthogonal transformation circuit 120 and the inverse orthogonal transformation circuit 160.

  The binary image encoding circuit 220 encodes and outputs the alpha map signal that has been subjected to resolution reduction conversion output from the resolution conversion circuit 210 and outputs the binary image. Details will be described later, but resolution conversion for reduction processing will be described later. The encoding is performed using the motion compensated prediction signal of the alpha map that has been resolution-reduced and converted supplied from the circuit 260 through the signal line 42. The multiplexing circuit 240 multiplexes and outputs the binary image encoded output and the given enlargement ratio information.

  The configuration of the encoding circuit in the third specific example is different from the configuration of the prior art (the circuit in FIG. 3) described above in that it includes a motion compensation prediction circuit 250 and a resolution conversion circuit 260 for reduction processing. In addition, the motion compensation prediction circuit 250 includes a frame memory that stores a reproduction image of a previously encoded frame, and can store a reproduction signal supplied from the enlargement circuit 230. Further, a motion vector signal (not shown) is supplied to the motion compensation prediction circuit 250, a motion compensation prediction signal is generated according to this motion vector signal, and a resolution conversion circuit 260 for reduction processing is provided via the signal line 41. It is as composition to supply to.

  The resolution conversion circuit 260 for the reduction process uses the motion compensation signal from the motion compensation prediction circuit 250 supplied via the signal line 41 in accordance with the setting information signal for the reduction / enlargement ratio supplied via the signal line 60. And then output to the binary image encoding circuit 220 via the signal line 42.

  In the case of configuring as the binary image encoding circuit 220, the alpha map signal subjected to resolution reduction conversion from the resolution conversion circuit 210 supplied via the signal line 21 is binary image encoded and output.

  In the alpha map encoding circuit 200 having such a configuration, the setting information signal of the reduction / enlargement ratio supplied via the signal line 60 is sent to the resolution conversion circuits 210, 230, 260, and the binary image encoding circuit 220. The generated code amount of the alpha map signal can be controlled. Further, the code of the reduction / enlargement ratio (setting information signal) supplied via the signal line 60 is multiplexed with the encoded alpha map signal by the multiplexing circuit 240 and output via the signal line 30. Then, it is supplied as an alpha map encoded signal to the multiplexing circuit 180 of FIG. 1 which is the final output stage of the image encoding apparatus.

  In the present apparatus, the resolution conversion circuit 210 reduces and encodes the alpha map input via the alpha map signal input line 20 according to the desired reduction / enlargement ratio setting information given via the signal line 60. The signal is supplied to the encoding circuit 220.

  The binary image encoding circuit 220 converts the resolution-reduced and converted alpha-map signal obtained from the resolution conversion circuit 210 to the resolution-reduced and converted alpha-map supplied from the resolution conversion circuit 260 for reduction processing through the signal line 42. Are encoded using the motion compensated prediction signal, and supplied to the multiplexing circuit 240 and the resolution conversion circuit 230 as a binary image encoding output. The multiplexing circuit 240 multiplexes the encoded alpha map signal, which is the binary image encoding output, and the enlargement ratio information given through the signal line 60 and outputs the multiplexed signal to the signal line 30.

  On the other hand, the resolution conversion circuit 230 reduces / enlarges the reduced-encoded alpha map signal (binary image encoded output) given from the binary image encoding circuit 220 via the signal line 60. 1 is decoded to the original resolution in accordance with the rate setting information signal and obtained as a local decoded signal. The obtained local decoded signal is obtained through the signal line 40 through the motion compensation prediction circuit 250 and the orthogonal transform circuit 120 in FIG. Output to the circuit 160.

  On the other hand, the motion compensation prediction circuit 250 is provided with a frame memory for storing a reproduction image of a previously encoded frame, and stores a reproduction signal supplied from the resolution conversion circuit 230 for enlargement processing. Then, the motion compensation prediction circuit 250 generates an alpha map motion compensation prediction signal in accordance with a separately supplied motion vector signal, and supplies it to the resolution conversion circuit 260 for reduction processing via the signal line 41. The resolution conversion circuit 260 performs resolution reduction conversion on the supplied motion compensation prediction signal according to the setting information signal of the reduction / enlargement ratio obtained via the signal line 60, and provides the binary image encoding circuit 220 with the resolution reduction conversion.

  The binary image encoding circuit 220 uses the motion-compensated prediction signal of the alpha map that has undergone resolution reduction conversion provided from the resolution conversion circuit 260 for reduction processing, and uses the resolution reduction-converted alpha obtained from the resolution conversion circuit 210. Encode the map signal.

  The above is the outline of the alpha map encoding circuit of the third specific example. The alpha map decoding circuit is as follows.

  As shown in FIG. 13, the alpha map decoding circuit 400 of this specific example includes a binary image decoding circuit 410 (block-based MMR decoder), a resolution conversion circuit (enlargement processing circuit) 420, a separation circuit 430, Further, it includes a motion compensation prediction circuit 440 and a resolution conversion circuit (reduction processing circuit) 450.

  Among these, the separation circuit 430 is separated by the separation circuit 300 in the image decoding apparatus shown in FIG. 2, and the sign and reduction of the alpha map signal from the alpha map signal input to the alpha map decoding circuit 400. A circuit that separates the code of the enlargement ratio, and the binary image decoding circuit 410 sets the code of the reduction / enlargement ratio given by separating the code of the alpha map signal from the separation circuit 430 (setting of the reduction / enlargement ratio) In accordance with (information signal), a circuit for returning to a binary image, which will be described in detail later, is a resolution-compensated alpha map motion compensated prediction signal supplied from a resolution conversion circuit 450 for reduction processing through a signal line 92. It is used for decryption.

  The resolution conversion circuit 420 for enlargement processing is provided with a code of reduction / enlargement ratio (reduction / enlargement ratio) given by separating the binary image, which is the code of the alpha map signal from the binary image decoding circuit 410, from the separation circuit 430. (Enlargement rate setting information signal) and output after resolution enlargement conversion.

  The configuration of the decoding circuit in the third specific example is different from the configuration of the prior art (the circuit in FIG. 4) described above in that it includes a motion compensation prediction circuit 440 and a resolution conversion circuit 450 for reduction processing. is there. The motion compensation prediction circuit 440 includes a frame memory that stores the reproduction image of the previously encoded frame, stores the reproduction signal supplied from the resolution conversion circuit 420 for enlargement processing, and also provides a motion vector. A signal (not shown) is supplied, a motion compensated prediction signal is generated according to this motion vector signal, and is supplied to the resolution conversion circuit 450 for reduction processing via the signal line 91.

  The resolution conversion circuit 450 reduces the motion compensated prediction signal in accordance with the setting information signal of the reduction / enlargement ratio supplied via the signal line 82, and then sends it to the binary image decoding circuit 410 via the signal line 92. Output.

  In the alpha map decoding circuit 400 having such a configuration, the code supplied to the alpha map decoding circuit 400 via the signal line 80 is the code of the alpha map signal and the code of the reduction / enlargement ratio by the separation circuit 430. And are output via the signal line 81 and the signal line 82, respectively.

  As will be described in detail later, the binary image decoding circuit 410 uses a resolution-compensated alpha map motion compensated prediction signal supplied from a resolution conversion circuit 450 for reduction processing on a signal line 92 to obtain a signal line. A decoding process for returning to a binary image is performed in accordance with the code of the alpha map signal supplied through 81 and the code of the reduction / enlargement ratio (reduction / enlargement ratio setting information signal) supplied through the signal line 82. Thus, the reduced alpha map signal is reproduced and supplied to the resolution conversion circuit 420 via the signal line 83.

  The resolution conversion circuit 420 expands the reduced alpha map signal reproduced by the binary image decoding circuit 410 to the original size based on the sign of the reduction / enlargement ratio supplied via the signal line 82. After the alpha map signal is reproduced, it is output via the signal line 90.

  The binary image encoding circuit 220 converts the resolution-reduced and converted alpha-map signal obtained from the resolution conversion circuit 210 to the resolution-reduced and converted alpha-map supplied from the resolution conversion circuit 260 for reduction processing through the signal line 42. Are encoded using the motion compensated prediction signal, and supplied to the multiplexing circuit 240 and the resolution conversion circuit 230 as a binary image encoding output. The multiplexing circuit 240 multiplexes the encoded alpha map signal, which is the binary image encoding output, and the enlargement ratio information given through the signal line 60 and outputs the multiplexed signal to the signal line 30.

  On the other hand, the resolution conversion circuit 420 reduces / enlarges the reduced-encoded alpha map signal (binary image encoded output) given from the binary image decoding circuit 410 via the signal line 82. It decodes to the original resolution according to the rate setting information signal, obtains it as a local decoded signal, and outputs the obtained local decoded signal to the motion compensation prediction circuit 440.

  On the other hand, the motion compensation prediction circuit 440 is provided with a frame memory that stores a reproduction image of the previously encoded frame, and stores a reproduction signal supplied from the resolution conversion circuit 420 for enlargement processing. Then, the motion compensation prediction circuit 420 generates an alpha map motion compensation prediction signal according to a separately supplied motion vector signal, and supplies it to the resolution conversion circuit 450 for reduction processing via the signal line 91. The resolution conversion circuit 450 performs resolution reduction conversion on the supplied motion compensation prediction signal in accordance with the setting information signal of the reduction / enlargement ratio obtained via the signal line 82, and provides it to the binary image decoding circuit 410.

  The binary image decoding circuit 410 uses the alpha map motion compensated prediction signal subjected to resolution reduction conversion given from the resolution conversion circuit 450 for reduction processing, and sets the reduction / enlargement ratio setting information from the separation circuit 430. The alpha map signal from the separation circuit 430 is decoded according to the signal.

  The above is the outline of the decoding circuit to which the present invention is applied.

  As already described, the configuration of the encoding circuit in the third specific example to which the present invention is applied is that the configuration of the prior art (the circuit of FIG. 3) includes the motion compensation prediction circuit 250 and the reduction circuit 260. In addition, the configuration of the decoding circuit is different from that of the prior art (the circuit of FIG. 4) in that it includes a motion compensation prediction circuit 440 and a reduction circuit 450.

  The motion compensation prediction circuit 250 or 440 is provided with a frame memory that stores a reproduction image of a previously encoded frame, and stores a reproduction signal supplied from the enlargement circuit 230 or 420. Further, a motion vector signal (not shown) is supplied to the motion compensation prediction circuit 250 or 440, and a motion compensation prediction signal is generated in accordance with the motion vector signal, via the signal line 41 and the signal line 91. This is supplied to the reduction circuit 260 or 450.

  Here, as the motion vector signal, a motion vector signal used in the motion compensated prediction circuit 110 or 350 included in the apparatus of FIGS. 1 and 2 may be used. In the alpha map encoding circuit 200, An alpha map motion vector signal may be obtained by providing an alpha map motion vector detection circuit.

  That is, various methods for obtaining the motion vector signal supplied to the motion compensation prediction circuit 250 or 440 are known and are not related to the present invention, and thus will not be further described here.

  In the reduction circuit 260 or 450, the motion compensation signal supplied via the signal line 41 and the signal line 91 is converted into the reduction / enlargement ratio setting information signal supplied via the signal line 60 and the signal line 82. After the reduction, the signal is output via the signal line 42 and the signal line 92.

  In the case where the binary image encoding circuit 220 is configured, the alpha map signal subjected to resolution reduction conversion supplied via the signal line 21 is binary image encoded and output.

Here, the binary image encoding circuit 220 according to the present specific example is fundamentally different from the above-described prior art in that the motion compensation prediction signal of the resolution-converted alpha map supplied via the signal line 42 is used. It has the function to encode using.
This will be described in detail.

  FIG. 14 is a diagram for explaining a method of encoding using a motion compensated prediction signal, and shows one of the divided N × M pixel configuration image blocks in an image in units of frame images.

  In FIG. 14, “current block” is a processing target block, which is a block of an input current processed image. Further, “compensated block” is a compensation block, which is a block of an image that has been processed last time.

  In the prior art which is the application premise of the present invention, the reference change pixel b1 on the corresponding block of the alpha map corresponding to the block of the current processing target image is detected in the same “current block” as the change pixels a0 and a1. It was.

  On the other hand, in the present invention, the reference change pixel b1 is detected from the “compensated block” that is a motion compensated prediction signal, which is a new concept. That is, the reference change pixel b1 on the corresponding block of the alpha map corresponding to the block of the current processing target image is detected from the “compensated block” that is the motion compensated prediction signal.

  The present invention is the same as the prior art in that encoding / decoding is performed using the relative addresses of a0, a1 and b1, except that the detection means for the reference change pixel b1 is different.

  In FIG. 14, a0 is a starting point change pixel, and encoding up to the starting point change pixel a0 has already been completed. Further, a1 is a change pixel next to the start point change pixel a0, and b0 is a pixel (not necessarily a change pixel) at the same position as a0 in the "compensated block". If the line to which a0 (b0) belongs is expressed as “a0-line”, the reference change pixel b1 is defined as follows.

  Here, abs_X is the address of the pixel X when the block is scanned in raster order from the upper left pixel of the block. Note that the address of the pixel at the upper left of the block is “0”.

  If abs_b0 <abs_b1 and the pixel indicated by the symbol “x” is a change pixel and the change pixel x is on “a0-line”, the first change of the opposite color to “a0” If the pixel is the reference change pixel b1 and the change pixel is not on "a0-line", the first change pixel on the line is set as the reference change pixel b1.

  FIG. 14A shows a case where the change pixel is not on “a0_line”. In this case, the first change pixel of the next line is set to “b1”.

  FIG. 14B shows a case where the change pixel is on “a 0 -line”. Since this change pixel x is not the opposite color to “a”, it is not set to “b 1”. The first changed pixel of the line is “b1”.

  Note that the values of “a0-line”, “r_ai (i = 0 to 1)”, and “r_b1” are obtained by the following equations.

a0-line = (int) ((abs-a0 + WIDTH) / WIDTH) -1
r-a0 = abs-a0-a0-line * WIDTH
r- a1 = abs-a1-a0-line * WIDTH
r-b1 = abs-b1-a0-line * WIDTH
In the above formula, * means multiplication, (int) (X) means truncation after the decimal point of X, and WIDTH indicates the number of pixels in the horizontal direction of the block.

  In the present invention, since the definition of the reference change pixel b1 is different from the prior art, the definition of “r_b1” is also changed as shown in the above equation.

  The example described with reference to FIG. 14 is an example of a means for obtaining the reference change pixel b1 from the “compensated block”, and various modifications can be made for the detection of the reference change pixel b1.

  Further, in the binary image decoding circuit, a binary image encoding circuit 220 is used by using a motion compensated prediction signal (“compensated block”) of an alpha map subjected to resolution reduction conversion supplied via a signal line 92. The reference change pixel b1 is detected by the same procedure.

  Furthermore, whether the reference change pixel b1 is detected from the “current block” or the “compensated block” can be switched, for example, in units of blocks. At this time, the binary image encoding circuit 220 also encodes the switching information, and the binary image decoding circuit 410 also decodes the switching information, and at the time of decoding processing, the switching information. Based on the information, whether the reference change pixel b1 is detected from the “current block” or the “compensated block” is switched, for example, in units of blocks.

  In this way, it is possible to perform optimum processing based on the image content in units of blocks, and it is possible to perform more efficient encoding.

  Also, as in the prior art, a means for switching the scan order is provided, and the scan order is switched to the horizontal scan as shown in FIG. 15 (a), or the vertical scan is used as shown in FIG. 15 (b). By switching, the number of changed pixels is reduced, and the amount of codes is further reduced, which also leads to more efficient encoding.

As described above, the third specific example is a code for encoding an alpha map representing the shape of an object in a moving image encoding apparatus that encodes a plurality of frames of moving image signals obtained as time-series data for each object having an arbitrary shape. And a means for dividing a rectangular region including an object into blocks each composed of M × N pixels (M: number of pixels in the horizontal direction, N: number of pixels in the vertical direction). Means for sequentially encoding the block in the square area according to a certain rule, and applying a relative address encoding to all or a part of the block,
Means for storing reproduction values in the vicinity of a block, frame memory for storing a reproduction signal of an already encoded frame, a motion compensation prediction circuit for generating a motion compensation prediction value using the reproduction signal in the frame memory, and the vicinity of the block Means for detecting a change pixel including the reproduction value of the reference value, and the reference change pixel of the relative address encoding is obtained not from the pixel value in the block but from the motion compensated prediction signal.

  Further, in the alpha map decoding circuit, for each block composed of M × N pixels, means for sequentially decoding the rectangular area including the object according to a certain rule, means for storing a reproduction value in the vicinity of the block, and code Frame memory for storing the regenerated signal of the converted frame, a motion compensation prediction circuit for generating a motion compensated prediction value using the regenerated signal in the frame memory, and means for detecting a change pixel including a replay value near the block The reference change pixel of relative address encoding is obtained from the motion compensated prediction signal instead of the pixel value in the block.

  As a result, it is possible to efficiently encode and decode the alpha map information, which is the sub-image information representing the shape of the object, the position in the screen, and the like.

  Also, a motion compensation prediction that generates a motion compensation prediction value using a playback value storage means for storing playback values in the vicinity of a block, a frame memory for storing playback signals of already encoded frames, and playback signals in the frame memory. Relative address encoding obtained from a reproduction pixel value in the block, a circuit, means for detecting a change pixel including reproduction values in the vicinity of the block with reference to information of the accumulation reproduction value of the reproduction value accumulation means And a means for switching the reference change pixel of the relative address encoding obtained from the motion compensated prediction signal, and encodes the relative address encoding information together with the switching information.

  Further, in the alpha map decoding circuit, means for sequentially decoding the rectangular area including the object in a predetermined rule for each block composed of M × N pixels, and reproduction value accumulation means for storing reproduction values near the block; A frame memory that stores a reproduction signal of an already encoded frame, a motion compensation prediction circuit that generates a motion compensation prediction value using the reproduction signal in the frame memory, and a change pixel including a reproduction value near the block. Means for detecting the relative address encoding reference change pixel obtained from the reproduction pixel value in the block and the relative address encoding reference change pixel obtained from the motion compensation prediction signal. The reference change pixel is obtained according to the switching information.

  In this case, in relative address encoding, whether the reference change pixel b1 is detected from the “current block” or the “compensated block” can be switched and processed in units of blocks. The switching information is also encoded and decoded on the decoding side, and the reference change pixel b1 is set to “current” based on the switching information at the time of decoding the encoded alpha map. Whether to detect from “block” or from “compensated block” can be switched on a block-by-block basis, which enables optimal processing based on the image content of each block. Thus, more efficient encoding becomes possible.

  As described above, according to the present invention, it is possible to efficiently encode alpha map information, which is sub-image information representing the shape of an object, a position in a screen, and the like, and to perform the decoding thereof. And an image decoding apparatus is obtained.

  The specific example described above is an example in which MMR (Modified Modified READ) encoding is used in the configuration of the alpha map encoding circuit 200. However, the present invention is not limited to MMR encoding, and can be implemented using any other binary image encoding circuit. Such an example will be described below.

(Fourth specific example)
Specific configurations of the alpha map encoding circuit 200 and the alpha map decoding circuit 400 will be described with reference to FIGS.

  FIG. 20 is a diagram in which the screen of the alpha map is divided into macroblock (MB) units having a predetermined plural pixel configuration such as 16 × 16 pixels, for example, and the areas shown by squares are division boundaries. Each cell is a macro block (MB).

  In the case of an alpha map expressed in binary (in some cases, it may be expressed in multiple values including a weighting factor when compositing the object), the object shape information is expressed in either white or black for each pixel. Is done. Therefore, as shown in FIG. 20, the contents of each macro block (MB) on the alpha map screen are “allW” (all white), “allB” (all black), “Multi” (other). It is classified into one of three types.

  In the case of a screen as shown in FIG. 20 which is an alpha map of a human image, the background is “white” and the human part is “black”, so that the macro block (MB) including the boundary part of the object classified as “Multi” is included. ) Only for binary image coding. Also, among the blocks (MB) classified as “Multi”, when the motion compensation prediction error value is equal to or less than the set value (threshold value), the block (MB) is copied with the motion compensation prediction value. Thus, when the mode of a macroblock (MB) to be copied is expressed as “copy”, and the mode of a macroblock (MB) subjected to binary plane image encoding is expressed as “coded”, the code of the macroblock (MB) is coded. The following four modes are available.

(1) “allW”
(2) “allB”
(3) “copy”
(4) “coded”
The encoding method or decoding method of each mode will be described in the alpha map encoding circuit 200 and the alpha map decoding circuit 400.

<Configuration Example of Alpha Map Encoding Circuit in Fourth Specific Example>
FIG. 21 is a detailed configuration diagram of the alpha map encoding circuit 200. In the configuration of FIG. 21, a mode determination circuit 1100, a CR (reduction / enlargement ratio) determination circuit 1110, a selector 1200, intra-block pixel value setting circuits 1400 and 1500, a motion compensation prediction circuit 1600, a binary image encoding circuit 1700, Reduction circuit 1710, 1730, 1740, enlargement circuit 1720, frame memory 1300, transposition circuit 1750, 1760, scan type (ST) determination circuit 1770, motion vector detection circuit (MVE) 1780, MV encoding circuit 1790, VLC (variable length) Encoding) / multiplexing circuit 1800.

  Among these, the in-block pixel value setting circuit 1400 is a circuit that generates pixel data in which all the pixel values in the macroblock are white, and the in-block pixel value setting circuit 1500 sets all the pixel values in the macroblock to black. This is a circuit for generating pixel data.

  A CR (reduction / enlargement ratio) determination circuit 1110 analyzes an alpha map signal supplied via the alpha map signal input line 20 and determines at what reduction / enlargement ratio an alpha map image for one frame should be processed. And the determination result is output as the reduction / enlargement ratio information b2. The reduction circuit 1710 outputs the alpha map signal supplied via the alpha map signal input line 20 for the entire frame, and the scan type (ST) determination circuit 1770 outputs the output encoded output from the binary image encoding circuit 1700. Based on this, the scan direction is determined and the scan direction information b4 is output.

  Further, the transposition circuit 1750 transposes the position of each macroblock in the alpha map signal for one frame subjected to the reduction processing by the reduction circuit 1710 based on the scan direction information b4 output from the scan type (ST) determination circuit 1770. The transposition circuit 1760 is a circuit that transposes and outputs the outputs of the reduction circuits 1710, 1720, and 1730 based on the scan direction information b4 output from the scan type determination circuit 1770, and outputs the binary image. The encoding circuit 1700 encodes the reduced alpha map signal given through these transposition circuits 1750 and 1760 and outputs the result.

  The enlargement circuit 1720 is a circuit that enlarges the alpha map signal supplied through the reduction circuit 1710 at the reduction / enlargement ratio output from the CR determination circuit 1110. The motion compensation prediction circuit 1600 is stored in the frame memory 1300. A motion compensated prediction signal is generated using the reproduced image of the reference frame, and is output to the mode determination circuit 1100 and the reduction circuit 1740. The reduction circuit 1740 outputs the motion compensation prediction signal to the output of the CR determination circuit 1110. The reduction circuit 1730 is a circuit that reduces the reproduction image of the reference frame stored in the frame memory 1300 with the reduction / enlargement ratio output from the CR determination circuit 1110. is there.

  In addition, the selector 1200 follows the classification information output from the mode determination circuit 1100, the reproduction signal m 0 from the in-block pixel value setting circuit 1400, the reproduction signal m 1 from the in-block pixel value setting circuit 1500, and the motion compensation from the motion compensation prediction circuit 1600. Of the prediction signal m2 and the reproduction signal m3 from the expansion circuit 1720, a circuit that selects and outputs a required signal, and the frame memory 1300 is a memory that stores the output of the selector 1200 in units of frames.

  The mode determination circuit 1100 is supplied via the alpha map signal input line 20 with reference to the motion compensation prediction signal from the motion compensation prediction circuit 1600 and the encoded signal b4 from the binary image encoding circuit 1700. The alpha map signal is analyzed to determine whether each macroblock is classified into “allW”, “allB”, “copy”, or “coded”, and the determination result is output as mode information b0.

  The motion vector detection circuit (MVE) 1780 detects a motion vector from the alpha map signal supplied via the alpha map signal input line 20, and the MV encoding circuit 1790 is a motion vector detection circuit (MVE) 1780. The motion vector detected in step 1 is encoded, and the encoded result is output as motion vector information b1. For example, when predictive encoding is applied to the MV encoding circuit 1970, the prediction error signal is output as motion vector information b1.

  A VLC (variable length encoding) / multiplexing circuit 1800 is provided with mode information b0 from the mode determination circuit 1100, motion vector information b1 from the MV encoding circuit 1790, and reduction / enlargement from the CR (reduction / enlargement ratio) determination circuit 1110. Upon receiving the enlargement factor information b2, the scan direction information b3 from the scan type (ST) determination circuit 1770, and the binary encoded information b4 from the binary image encoding circuit 1700, these are variable length encoded, multiplexed, and signal lines 30 is output.

  In such a configuration, the alpha map signal to be encoded is supplied to the alpha map encoding circuit 200 via the alpha map signal input line 20. In response to this, the alpha map encoding circuit 200 analyzes the alpha map signal in the mode determination circuit 1100 and classifies it into “allW”, “allB”, “copy”, or “coded” for each macroblock. Determine whether it will be done. Here, for example, the number of mismatched pixels is used as the classification evaluation / value criterion.

Specifically, it is as follows.
First, the mode determination circuit 1100 calculates the number of mismatched pixels when all signals in the input macroblock are replaced with white, and classifies the macroblock whose number is equal to or less than the threshold value as “allW”. Similarly, the mode determination circuit 1100 classifies the macroblock as “allW” when all the macroblocks are replaced with black. Similarly, a macroblock in which the number of mismatch pixels when a macroblock is entirely replaced with black is equal to or less than a threshold value is classified as “allB”.
Next, the mode determination circuit 1100 calculates the number of mismatched pixels with the motion compensated prediction value supplied via the signal line 1010 for a macroblock that is not classified as “allW” or “allB”. A block whose number is less than or equal to the threshold is classified as “copy”.
Here, macroblocks that are not classified as “allW”, “allB”, or “copy” are classified as “coded”.

  This classification information b0 from the mode determination circuit 1100 is supplied to the selector 1200 via the signal line 1020. When the mode of the block is “allW” in the selector 1200, the pixel value in the block is determined in the pixel value setting circuit 1400 in the block. Is selected as white, supplied to the frame memory 1300 via the signal line 40, stored in the storage area of the frame, and output as the output of the alpha map encoding circuit 200.

  Similarly, when the mode of the macroblock is “allB”, the selector 1200 selects the reproduction signal m1 in which the pixel values in the macroblock are all black in the macroblock pixel value setting circuit 1500, and the macroblock When the block mode is “copy”, the motion compensation prediction signal m2 generated by the motion compensation prediction circuit 1600 supplied via the signal line 1010 is selected, and the macroblock mode is “coded”. In this case, the reproduction signal m3 supplied through the reduction circuit 1710 and the enlargement circuit 1720 is selected, supplied to the frame memory 1300 via the signal line 40, stored in the storage area of the frame, and the alpha map. This is output as an output signal of the encoding circuit 200.

  Further, the pixel value of the macroblock classified as “coded” in the mode determination circuit 1110 is reduced in the reduction circuit 1710 and then encoded in the binary image encoding circuit 1700. Here, the setting information of the reduction / enlargement ratio (CR (Conversion Ratio)) used in the reduction circuit 1710 is obtained in the CR determination circuit 1110. For example, when the reduction ratio is set to three types of “1 (not reduced)”, “1/2 (1/2 for both horizontal and vertical)” and “1/4 (for both horizontal and vertical)”, The CR determination circuit 1110 obtains CR (reduction / enlargement ratio) in the next step.

      (1) The number of mismatches between the reproduction signal when the macroblock is reduced to “1/4” and the macroblock is calculated. If this number is equal to or less than the threshold value, the reduction ratio is set to “1/1”. 4 ".

      (2) If the number of mismatches is larger than the threshold value in (1) above, calculate the number of mismatches between the playback signal when the macroblock is reduced to “1/2” and the macroblock, When this number is less than or equal to the threshold value, the reduction ratio is set to “1/2”.

      (3) If the number of mismatches is larger than the threshold value in (2) above, the reduction ratio of the macroblock is set to “1”.

  The CR (reduction / enlargement ratio) value obtained in this manner is supplied to the reduction circuits 1710, 1730, 1740, the enlargement circuit 1720, and the binary image encoding circuit 1700 via the signal line 1030, and also the VLC. (Variable Length Coding) • After being supplied to the multiplexing circuit 1800 and encoded, it is multiplexed with other codes. Also, the transposition circuit 1750 transposes the position of the reduced signal of the block supplied from the reduction circuit 1710 (the horizontal address and the vertical address are exchanged).

  As a result, the encoding order can be switched between the horizontal scanning order and the vertical scanning order. Further, the transposition circuit 1760 transposes the position of each signal between the reproduction pixel value near the macroblock reduced by the reduction circuit 1730 and the motion compensated prediction signal reduced by the reduction circuit 1740.

  In transposition circuits 1750 and 1760, whether or not to perform transposition processing is determined by, for example, encoding in horizontal scan and vertical scan in binary image encoding circuit 1700, respectively, and output via signal line 1040. By supplying the conversion information b4 to an ST (Scan Type) determination circuit 1770, the ST determination circuit 1770 is determined by selecting a scan direction in which the code amount decreases.

  In the binary image encoding circuit 1700, the signal of the macroblock supplied from the transposing circuit 1750 is encoded using the reference signal supplied from the transposing circuit 1760.

  An example of a specific binary image encoding method is the method used in the third specific example described above, but is not limited thereto, and this binary image encoding circuit in the fourth specific example is used. In 1700, other binary image coding can be applied.

  Among the blocks classified as “coded”, there are an “intra” coding mode using a reference pixel in a frame and an “inter” coding mode referring to a motion compensation prediction signal. Here, the intra / inter switching is performed by, for example, supplying the encoding information from the binary image encoding circuit 1700 supplied via the line 1040 to the mode determination circuit 1100 and selecting a mode with a small code amount. That's fine. Here, information on the selected encoding mode (intra / inter) is supplied to the binary image encoding circuit 1700 via the signal line 1050.

  The encoded information of the binary image encoding circuit 1700 encoded in the optimum mode selected by the above means is supplied to the VLC / multiplexing circuit 1800 via the signal line 1040, and together with other codes Multiplexed. Further, the mode determination circuit 1100 supplies information on the optimum mode to the VLC / multiplexing circuit 1800 via the signal line 1060, and after encoding, the information is multiplexed together with other codes. The motion vector detection circuit (MVE) 1780 detects an optimal motion vector. Here, there are various motion vector detection methods, and since the motion vector detection method itself is not a main part of the present invention, it is omitted here. The detected motion vector is supplied to the motion compensated prediction circuit 1600 via the signal line 1070 and encoded by the MV encoding circuit 1790 and then supplied to the VLC / multiplexing circuit 1800 and encoded. And then multiplexed with other codes.

  The multiplexed code is output via the signal line 30. The motion compensation prediction circuit 1600 generates a motion compensation prediction signal using a reconstructed image of the reference frame stored in the frame memory 1300 based on the motion vector signal supplied via the signal line 1070, To the mode determination circuit 1100 and the reduction circuit 1740.

  As a result of the above processing, it is possible to perform image coding with good image quality and high compression rate.

  Next, decoding will be described.

<Example of configuration of alpha map decoding circuit>
FIG. 22 is a detailed configuration diagram of the alpha map decoding circuit 400.
As shown in the figure, an alpha map decoding circuit 400 includes a VLC (variable length coding) / separation circuit 2100, a mode reproduction circuit 2200, a selector 2300, intra-macroblock pixel value setting circuits 2400 and 2500, and a motion compensation prediction circuit. 2600, frame memory 2700, binary image decoding circuit 2800, expansion circuit 2810, transposition circuits 2820 and 2850, reduction circuits 2830 and 2840, binary image decoding circuit 2800, and motion vector reproduction circuit 2900.

  Among these, the VLC (variable length coding) / separation circuit 2100 decodes the encoded bit stream of the alpha map that has been multiplexed and sent, and mode information b0, motion vector information b1, reduction / enlargement rate This is a circuit that separates information b2, scan direction information b3, and binary image encoding information b4. The mode reproduction circuit 2200 receives the separated mode information b0 and receives “allW”, “allB”, “copy”, This is a circuit that reproduces which one of the four modes of “coded”.

  The binary image decoding circuit 2800 receives the binary image coding information b4 separated by the VLC (variable length coding) / separation circuit 2100 from the separated reduction / enlargement ratio information b2 and mode reproduction circuit 2200. Is a circuit that decodes and outputs a binary image using the reproduced mode information and information of the transposition circuit 2850, and the enlargement circuit 2810 uses the information of the separated reduction / enlargement ratio information b2 to perform the decoding. The transposed circuit 2820 is a circuit that transposes the enlarged image according to the separated scan direction information b3 and outputs it as a reproduction signal m3.

  The macroblock pixel value setting circuit 2400 is a circuit that generates a reproduction signal m0 in which all macroblock pixel values are white. The macroblock pixel value setting circuit 2500 includes all macroblock pixel values. This is a circuit for generating a black reproduction signal m1.

  The motion vector reproduction circuit 2900 is a circuit that reproduces the motion vector of the macroblock using the motion vector information b1 separated by the VLC (variable length coding) / separation circuit 2100. The motion compensation prediction circuit 2600 This is a circuit for generating a motion compensated prediction value m2 from a reproduced image by using a reconstructed motion vector and a reference frame stored in the frame memory 2700, and a reduction circuit 2840 uses the information of the separated reduction / enlargement ratio information b2. The motion compensation prediction value m2 is reduced, and the reduction circuit 2830 reduces the image of the stored reference frame in the frame memory 2700 using the separated reduction / enlargement ratio information b2. The transposition circuit 2850 is a circuit separated by the VLC (variable length coding) / separation circuit 2100. Based on the emission direction information b3, a circuit for outputting a reduced image permutation to the binary image decoding circuit 2800 outputs to the reduction circuit 2830,2840.

  In addition, the selector 2300 generates reproduction signals m0 and m1 from the intra-macroblock pixel value setting circuits 2400 and 2500, a motion compensation prediction value m2 from the motion compensation prediction circuit 2600, and a transposition circuit according to mode information output from the mode reproduction circuit 2200. One of the 2820 reproduction signals m3 is selected and output, and the frame memory 2700 is a memory that holds the image signal output from the selector 2300 in units of frames.

  The alpha map decoding circuit 400 having such a configuration is supplied with the encoded bit stream of the alpha map via the signal line 80. This bit stream is supplied to a VLD (variable length decoding) / separation circuit 2100.

  Then, the VLD (variable length decoding) / separation circuit 2100 decodes this bit stream, and mode information b0, motion vector information b1, reduction / enlargement ratio information b2, scan direction information b3, binary image code. The information is separated into conversion information b4. The separated information is managed in units of macro blocks.

  Of the separated information, the mode information b0 is supplied to the mode reproduction crossover 2200, and classifies to which of the following modes the macroblock belongs.

(1) “allW”
(2) “allB”
(3) “copy”
(4) “coded”
Here, “coded” has an “intra” mode and an “inter” mode as described in the specific example of the encoding circuit of FIG. That is, the “intra” coding mode uses the reference pixels in the frame, and the “inter” coding mode refers to the motion compensated prediction signal.

  In the selector 2300, when the macroblock mode is “allW” in accordance with the encoding mode supplied via the signal line 2010, all the pixel values in the macroblock are set in the pixel value setting circuit 2400 in the block. The reproduction signal m0 made white is selected, supplied to the frame memory 2700 via the signal line 90, stored in the storage area of the image frame to which the macroblock belongs, and the alpha map decoding circuit 400. Is output as an alpha map image decoded.

  Similarly, when the mode of the macroblock is “allB”, the reproduction signal m1 in which the pixel values in the block are all black is selected in the pixel value setting circuit 2500 in the macroblock, and the mode of the block is In the case of “copy”, the motion compensation prediction signal m2 generated in the motion compensation prediction circuit 2600 supplied via the signal line 2020 is selected, and when the mode of the block is “coded”, A reproduction signal m3 supplied through the enlargement circuit 2810 and the transposition circuit 2820 is selected, supplied to the frame memory 2700 via the signal line 90, and stored in the storage area of the image frame to which the macroblock belongs in the frame memory 2700. At the same time, the image is output as a decoded alpha map image from the alpha map decoding circuit 400. That.

  On the other hand, the separated motion vector information b1 is supplied to the motion vector reproduction circuit 2900, and the motion vector of the macroblock is reproduced. Here, the reproduced motion vector is supplied to the motion compensation prediction circuit 2600 via the signal line 2030, and the motion compensation prediction circuit 2600 reproduces the reference frame stored in the frame memory 2700 based on the motion vector. A motion compensated prediction value is generated from the surface image. Then, the signal is output to the reduction circuit 2840 and the selector 2300 via the signal line 2020.

  The separated reduction / enlargement ratio information b2 is supplied to the reduction circuit 2830 and the enlargement circuit 2810, and is also supplied to the binary image decoding circuit 2800, and the scan direction information b3 is supplied to the transposition circuits 2820 and 2850. Is done.

  The transposition circuit 2820 that has received the scan direction information b3 transposes the position of the reproduction signal of the macroblock supplied from the enlargement circuit 2810 and returned to the original size (the horizontal address and the vertical direction). Addresses are swapped). Also, in the transposition circuit 2850 that has received the scan direction information b3, the position of each of the reproduced pixel values near the block reduced by the reduction circuit 2830 and the motion compensation prediction signal reduced by the reduction circuit 2840 are the positions of the signals. Transposed.

  In the binary image decoding circuit 2800 that has received the reduction / enlargement ratio information b2, the binary image encoding information b4 of the macroblock separated by the separation circuit 2100 is used as a reference signal supplied from the transposition circuit 2850. To decrypt.

  By the way, as described in the fourth specific example of Japanese Patent Application No. 8-237053 proposed by the present inventors as the prior art, the output of the enlargement circuits 1720 and 2810 has an image quality caused by oblique discontinuity. Although deterioration may occur, in order to solve this problem, the expansion circuits 1720 and 2810 may be provided with a filter that suppresses the discontinuity in the oblique direction.

  Similar to the binary image encoding circuit 1700, the configuration of the binary image decoding circuit 2800 to be used is not limited to the configuration example shown in the third specific example.

<Specific example of switching a plurality of binary image encoding methods>
In the above specific examples, the specific examples of the binary image encoding circuit 1700 and the binary image decoding circuit 2800 are described as not limited to the third specific example. Therefore, other examples will be shown.

As another binary image encoding method, for example, there is a Markov model encoding method (see: Television Society, “Image Information Compression”, pp. 171 to 176). FIG. 23 is a diagram for explaining an example of the Markov model encoding method. A pixel x in the drawing is a target pixel to be encoded, and pixels a to f are reference pixels that are referred to when encoding the pixel x.
Here, the pixels a to f are pixels that have already been encoded when the pixel x is encoded. For the pixel of interest x, depending on the states of the reference pixels a to f, a variable length code table is adaptively used when VLC (variable length coding) is used, and a probability table is adaptively used when arithmetic code is used. It is switched and encoded.

  When this specific example is applied to the present invention, “intra” encoding uses a pixel encoded before the target pixel in a macroblock to be processed as a reference pixel, and “inter” encoding is In the macroblock to be processed, not only the pixel encoded before the target pixel but also the pixel in the motion compensation prediction error signal may be used as the reference pixel.

In this example, for example,
[Method 1] Method of the third specific example (encoding method based on MMR)
When,
[Method 2] It is characterized in that both methods of the method of adaptively switching the Markov model encoding are provided, and a mode in which these are adaptively selected and used is adopted.

  FIG. 24A is a block diagram of a binary image encoding circuit 1700 used in this specific example. As illustrated, the binary image encoding circuit 1700 includes a pair of selectors 4110 and 4140 and a pair of binary image encoding circuits. Among these, the selector 4101 is an input side selection circuit, and the selector 4102 is an output side selection circuit. The binary image encoding circuit 4120 is a first binary image encoding circuit that encodes according to the [Method 1], and the binary image encoding circuit 4130 is a second 2 that encodes according to the [Method 2]. It is a value image encoding circuit.

  In the binary image encoding circuit 1700, a selector 4110, which selectively switches the signal of the processing target macroblock supplied from the transposition circuit 1750 via the signal line 4101 according to the switching signal supplied via the signal line 4103. By 4140, the first binary image encoding circuit 4120 and the second binary image encoding circuit 4130 are adaptively allocated, and the allocated macroblock signal is encoded.

  The encoded information is output via the signal line 4102. In FIG. 24A, signal lines 1030 and 1050 for signals supplied to the binary image encoding circuit 1700 and signal lines from the transposition circuit 1760 are omitted.

  Here, the switching information supplied via the signal line 4103 may be a preset default value, or appropriate switching information is obtained based on the image content, separately encoded as side information, and transmitted to the decoding side. You may make it do.

  As a result, it is possible to perform optimum processing based on the image content, and it is also possible to select and use an appropriate one from a plurality of encoding methods according to the application.

  Similarly, FIG. 24B is a block diagram of a binary image decoding circuit 2800 used in this specific example. As shown, the binary image decoding circuit 2800 includes a pair of selectors 4210 and 4240 and a pair of binary image decoding circuits 4220 and 4230. Among these, the selector 4210 is an input side selection circuit, and the selector 4240 is an output side selection circuit. The binary image decoding circuit 4220 is a first binary image decoding circuit that decodes the image encoded by the [Method 1]. The binary image decoding circuit 4230 is the above [Method 2] code. 2 is a second binary image decoding circuit for decoding the converted image.

  In the binary image decoding circuit 2800 having such a configuration, the binary image encoding information b4 of the processing target macroblock supplied via the signal line 4201 is used as the switching signal supplied via the signal line 4203. Accordingly, the selectors 4210 and 4240 adaptively distribute the data to the first binary image decoding circuit 4220 and the second binary image decoding circuit 4230 for decoding.

  The decoded signal is output via a signal line 4202. In FIG. 24B, the signal line for signal supply from the separation circuit 2100 to the binary image decoding circuit 2800, the signal line from the mode reproduction circuit 2200, and the signal line from the transposition circuit 2850 are omitted. .

  Here, the switching information supplied via the signal line 4203 may be a default value, or may be information sent from an encoder that obtains the switching information from an encoded bit stream.

  Next, a specific example of a circuit for performing reduction / enlargement will be described.

<< Specific examples of reduction / enlargement circuit >>
[Example of enlargement processing]
In the present invention, rate control is performed by performing reduction / enlargement processing in units of coding regions or blocks (macroblock units). As an example of a technique used for the reduction / enlargement processing, there is “bilinear interpolation” disclosed in Japanese Patent Application No. 8-237053 by the inventors of the present application. Let me touch on this technology. The bilinear interpolation is illustrated in FIG. 35 based on the reference document “Onoe: Image Processing Handbook, p.630, Shosodo”.

  In FIG. 35A, Pex is a pixel position after conversion, and the Pex indicates a real pixel position as shown in FIG.

  Therefore, from the distance relationship with the integer pixel positions A, B, C, and D of the input signal, the pixel values Ia to Id of A to D are changed from Pex to Pex according to the logical expression shown in FIG. A pixel value Ip is obtained.

  Such a process is called “bilinear interpolation”, and the pixel value Ip of Pex can be easily obtained from the pixel values Ia to Id of A to D.

  By the way, since this bilinear interpolation process is performed using only the surrounding four pixels, a change in a wide range is not reflected, and discontinuity in an oblique direction is particularly likely to occur. As an example for solving this problem, an example in which smoothing filter processing (smoothing processing) is performed after performing enlargement processing is presented in the invention of the prior application (Japanese Patent Application No. 8-237053).

  This will be specifically described. FIG. 36 is a diagram for explaining the smoothing process (smoothing process). Here, FIG. 36A shows a binary image of the original size, and FIG. 36B shows a binary image obtained by reducing this. In FIG. 36, the object area is indicated by a black circle, and the background area is indicated by a white circle.

  In this example, in order to smooth the discontinuity in the diagonal direction caused by sampling conversion (enlargement / reduction conversion), each pixel (white circle) in the background area is centered on it. Then, the top, bottom, left and right pixels, that is, adjacent pixels are examined. If two or more pixels are pixels (black circles) in the object area, processing for including the pixels in the background area in the object area is performed.

  That is, two or more pixels are adjacent to the adjacent pixel as in the case where the pixel to be inspected that is one pixel in the background area is a pixel at a position indicated by a double circle in FIG. When there is an area pixel (black circle), the pixel at the position indicated by the double circle (that is, the pixel to be inspected) is changed to a black circle pixel to be a pixel in the object area. For example, assuming that the black circle is “1” and the white circle is “0”, the pixel at the position indicated by the double circle (pixel value “0”) is replaced with the pixel value “1”. . By such processing, the discontinuity in the oblique direction can be eliminated.

FIG. 37 shows another example of the smoothing filter (smoothing process filter). If the center pixel of the 3 × 3 pixel mask shown in FIG. 37 is C, the upper left pixel is TL, the upper right pixel is TR, the lower left pixel is BL, and the lower right pixel is BR. The filtered value of the pixel C is obtained by the following equation. Here, the value of the object is represented by “1” and the background value is represented by “0”.

  That is, this filter calculation process checks whether the value obtained by adding the pixels TL, TR, BL, BR is greater than “2” if the pixel C is “0”, and if it is greater, the value of the pixel C If the pixel C is not “0”, the sum of the pixels TL, TR, BL, BR is less than “2”. If not, the value of the pixel C is set to “1”, and if not, the pixel C is set to “0”.

  According to this filter, since the value of the pixel C is corrected in consideration of the change in the pixel value located in the diagonal direction with respect to the target pixel C, the discontinuity in the diagonal direction is eliminated. The configuration of the smoothing filter is not limited to the above example, and a non-linear filter such as a median filter may be used.

  FIG. 58A is a diagram illustrating a process of enlarging the horizontal and vertical by a factor of two by bilinear interpolation. In FIG. 58A, the pixels of the reduced block are represented by white circles, that is, “O” marks, the pixels to be interpolated are represented by crosses, ie, “x” marks, and each pixel value is “0”. Alternatively, it is either “1”.

In this case, when the value of the pixel (reproduced pixel) to be interpolated with the logical expression of FIG.
Ip1 = Ia, Ip2 = Ib, Ip3 = Ic, Ip4 = Id
It becomes.
Accordingly, since the pixel values of the four interpolated pixels around the pixel A are all Ia, the enlarged image has the same value of 2 × 2 pixels, and the smoothness is lost. Therefore, the above problem can be solved by changing the weight of bilinear interpolation as follows.

Ip1: if (2 * Ia + Ib + Ic + Id> 2) then “1” else “0”
Ip2: if (Ia + 2 * Ib + Ic + Id> 2) then “1” else “0”
Ip3: if (Ia + Ib + 2 * Ic + Id> 2) then “1” else “0”
Ip4: if (Ia + Ib + Ic + 2 * Id> 2) then “1” else “0”
Here, Pi (i = 1, 2, 3, 4) is a pixel to be interpolated corresponding to the “x” position shown in FIG. 58 (a), and Ipi (i = 1, 2, 3, 4). ) Is the pixel value (“1” or “0”) of the pixel Pi (i = 1, 2, 3, 4). A, B, C, and D are the pixels of the reduced block, and Ia, Ib, Ic, and Id are the pixel values of these pixels A, B, C, and D.

“Ip1: if (2 * Ia + Ib + Ic + Id> 2) then“ 1 ”else“ 0 ”” means that the pixel value Ip1 of the pixel p1 is the double value of Ia, Ib, Ic, "1" if the sum of Id is greater than 2, otherwise "0".
“Ip2: if (Ia + 2 * Ib + Ic + Id> 2) then“ 1 ”else“ 0 ”” means
The pixel value Ip2 of the pixel p2 is “1” if the sum of the double value of Ib and Ia, Ic, and Id is greater than 2, otherwise it is “0”. Shows,
“Ip3: if (Ia + Ib + 2 * Ic + Id> 2) then“ 1 ”else“ 0 ”” means
The pixel value Ip3 of the pixel p3 is “1” if the sum of the double value of Ic, Ia, Ib, and Id is greater than 2, and “0” otherwise. Shows,
“Ip4: if (Ia + Ib + Ic + 2 * Id> 2) then“ 1 ”else“ 0 ”” means
The pixel value Ip4 of the pixel p4 is “1” if the sum of the double value of Id, Ia, Ib, and Ic is 2 or more, and “0” otherwise. Show.
In addition, when enlarging 4 times horizontally and vertically, the above process may be repeated twice.

  The above is an example of performing enlargement processing by computation, but enlargement processing can be performed without using computation. An example of this will be described next.

<Example of enlargement processing without calculation>
Here, a table is prepared and held in a memory, and pixels are uniquely replaced according to this table.

This will be specifically described. First, for example, the memory address is 4 bits, and Ip1, Ip2, Ip3, Ip4 obtained by the pattern of Ia, Ib, Ic, Id are assigned to the addresses obtained by arranging Ia, Ib, Ic, Id in the table below. Record in advance.

  In this example, when “Ia, Ib, Ic, Id” is “0, 0, 0, 0”, “Ip1, Ip2, Ip3, Ip4” is “0, 0, 0, 0”. , “Ia, Ib, Ic, Id” is “0, 0, 0, 1”, “Ip1, Ip2, Ip3, Ip4” is “0, 0, 0, 0”. , Ib, Ic, Id ”is“ 0, 0, 1, 1 ”,“ Ip1, Ip2, Ip3, Ip4 ”is“ 0, 0, 1, 1 ”. When “Ic, Id” is “0, 1, 0, 1”, “Ip1, Ip2, Ip3, Ip4” is “0, 1, 0, 1” and “Ia, Ib, Ic” , Id ”is“ 0, 1, 1, 0 ”,“ Ip1, Ip2, Ip3, Ip4 ”is“ 0, 1, 1, 0 ”, and so on. If the combination of the contents of Ic and Id is determined, Ip1, Ip2, Ip3, and Ip4 are unambiguous. It represents a table that is determined by.

  If such a table is set and held in the memory so that the contents of Ia, Ib, Ic, and Id are addresses and the stored data at those addresses are Ip1, Ip2, Ip3, and Ip4, An interpolation value can be obtained simply by inputting the addresses Ia, Ib, Ic, and Id into the memory and reading out the corresponding Ip1, Ip2, Ip3, and Ip4. Here, the sequence of Ip1, Ip2, Ip3, and Ip4 is a binary number. If a numerical value obtained by converting this into a decimal number is called a context, this technique is based on the context determined by Ip1, Ip2, Ip3, and Ip4. It can be said that the interpolation values Ip1, Ip2, Ip3, and Ip4 are determined.

The context is
Context = 2 * 2 * 2 * Ia + 2 * 2 * Ib + 2 * Ic + Id
It is obtained by.

  In the above example, the method for obtaining the interpolated value depending on the context is referred to as four pixels. However, the number of pixels to be referred to is not limited to this, and can be realized in any number of cases such as 12 described below.

  Also, with respect to the arrangement of the reference pixels and the interpolation pixels, for example, as shown in FIG. 64A, for the interpolation pixel P1, nine pixels in a region surrounded by a dotted line are used as contexts (= 0 to 511). ) And determining the interpolated pixel P1 to be “0” or “1” depending on the context.

  Also for the interpolated pixels P2, P3, and P4, the dotted lines in FIGS. 64B, 64C, and 64D correspond to the positions of the respective interpolated positions and the four reference pixels closest to the interpolated positions. A pixel surrounded by is used as a reference pixel.

  At this time, the reference pixels A to I are arranged so that the relative positions of the reference pixels A to I with respect to the interpolation pixel P are the same (for example, (b) is an arrangement obtained by rotating (a) 90 degrees clockwise). If the same pixel pattern is rotated, the same index is used, so that a memory having a table of contents common to the interpolation pixels P1 to P4 can be used.

Next, an example in which a macroblock of 4 × 4 pixels is enlarged to 16 × 16 pixels will be described.
To expand a 4 × 4 pixel macroblock to 16 × 16 pixels, first, the 4 × 4 size macroblock MB is expanded to 8 × 8 size, and then the 8 × 8 size macroblock MB. The procedure for enlarging the macroblock MB to a 16 × 16 macroblock MB is taken.

  FIG. 65A shows external reference pixels AT and AL when the macro block MB of 4 × 4 size is expanded to a size of 8 × 8, and when the macro block MB of 8 × 8 size is expanded. External reference pixels BT and BL are shown in FIG.

  The external reference pixel AT is arranged in 2 rows and 8 columns, the external reference pixel AL is arranged in 4 rows and 2 columns, the external reference pixel BT is arranged in 2 rows and 12 columns, and the external reference pixel BL is arranged in 8 rows and 2 columns.

  As described with reference to FIG. 42, the values of these external reference pixels need to be obtained as an average value of pixels at predetermined positions of the already reproduced macroblock. However, in the binary image encoding circuit 1700 and the binary image decoding circuit 2800, when encoding with the size of 4 × 4 pixels, the external reference pixels AT and AL are referred to and the size of 8 × 8 pixels is used. When encoding, reference is made to the external reference pixels BT and BL, and if this is diverted, it is possible to avoid the waste of obtaining the external reference pixel only for the enlargement process.

Here, the external references BT and BL can be described in C language, which is a language for computer programming, from the external reference pixels AT and AL without obtaining an average value.

However, the average value calculation can be omitted although the result is somewhat different from the case of obtaining the average value.

  In the above-described processing, for example, the pixel p1 is copied to the pixel p2 and the pixel p3, and the pixel p4 is copied to the pixel p5 and the pixel p6. By doing so, BT and BL are generated. Note that “[] []” in the C language program indicates an array, and the numbers in [] indicate decimal numbers.

  The specific method of enlargement processing has been described above. Next, a specific method of the reduction process will be described.

[Reduction processing]
FIG. 38 is an example of a reduction process for reducing a block (macroblock) to a size of “½” in length and breadth. In this example, assuming that each area within the dotted line frame is a unit reduced block area, the average for every 2 × 2 pixels in the unit reduced block area (a total of 4 pixels indicated by “O” in each dotted line frame). The value (“×” in FIG. 38) is the pixel value in the unit reduced block area. That is, when a macroblock is reduced to a size of “¼” in the vertical and horizontal directions, an average value for every 4 × 4 pixels in the unit reduced block area is obtained for each unit reduced block area. The pixel value may be used.

  Further, as shown in FIG. 39, when a certain unit reduced block area composed of pixels A, B, C, and D is considered, the value of the pixel X of the unit reduced block area is obtained and set as the value of the unit reduced block area. In this case, the average values of the pixels A to D are not calculated, but the pixels E, F, G, H, I, J, K, L, M, N, O, and P in a wider range of pixels are included. Thus, an average value of these A to P may be obtained. In other words, among the pixels in the adjacent unit reduced block area, the average value of these pixels including the pixels adjacent to the pixels A to D is adopted.

The above is an example in which the average value of the existing pixel values is used as the pixel value of the unit reduced block area, thereby reducing the number of pixels in the unit reduced block area and reducing the size of the macroblock. It was. Without such calculation, the macroblock can be reduced by mechanical thinning processing.
An example of this will be described next. First, the process closed in the macro block will be described.

  FIG. 40 shows an example of reduction processing by pixel thinning. A macro block MB having a 16 × 16 pixel configuration is shown as being reduced to a block having an 8 × 8 pixel configuration as a result of reduction processing by pixel thinning. That is, the pixels indicated by the dotted white circles in the figure are thinned out pixels, and the pixels indicated by the solid white circles are the pixel values of the unit reduced block area (“×” in FIG. 40). . In this case, the reduction ratio (CR) represents the ratio of pixels to be thinned out. Note that the pixel thinning method is not limited to the method described with reference to FIG. 40. For example, the pixels may be thinned in a five-cell lattice shape. In this case, the enlargement process corresponds to interpolating the thinned pixels.

  The specific methods for various reduction processes have been described above. Next, the enlargement process will be described.

  FIG. 41 is a diagram illustrating the enlargement process. In the figure, a solid square frame indicates a macro block, and each square of the portion displayed by a dotted frame indicates a unit reduced block area. White circles in the unit reduced block area represent existing pixels, and the number of pixels is increased by interpolation to enlarge the macroblock. The interpolated pixel is represented by “x”, and after the interpolation, the “O” pixel becomes an unnecessary pixel.

When interpolating the pixels at the macroblock boundary, pixel values outside the macroblock are required. In this case, as indicated by the arrows in the figure, the nearest pixel value in the macroblock may be assigned.
That is, when performing pixel interpolation of a certain unit reduced block area, a total of 9 pixel values of the pixel values of the surrounding 8 blocks which are each of the unit reduced block areas adjacent to itself are necessary for processing. However, if the certain unit reduced block area in the macroblock is located at the macroblock boundary part, some of the surrounding 8 blocks belong to the macroblocks other than their own. This means that it is necessary to separately obtain pixel values outside the self-affiliated macroblock. In this case, as indicated by the arrows in the figure, the nearest pixel value in its own macroblock is assigned and used as the pixel value in the neighboring unit reduced block area necessary for the processing for convenience. , You can use.

Note that the reduction / enlargement processing for each macroblock does not need to be closed within the macroblock, and as shown in FIG. 42, the reproduction values near the block (blocks adjacent to the left, upper, upper left, and upper right) May be used.
This will be specifically described.
In FIG. 42, a solid-line rectangular frame indicates a certain block (macro block), and an “x” mark indicates each pixel of an image (standard magnification image) when the magnification is 1 ×. A macroblock usually has a 16 × 16 pixel configuration, and when the frame is compressed to 1/2, the pixels in the block have an 8 × 8 pixel configuration, and the previous 16 × 16 pixel configuration has 2 × 2 pixels each. 42 is represented by one pixel. In this case, the image of 2 × 2 pixels is represented by “O” in FIG. 42 in which the representative point information is represented in the one-pixel representation format. The inside of the frame indicated by the dotted line is a unit reduced block area, which shows a 4-pixel configuration (2 × 2) area in the standard magnification image. It is expressed by one pixel.

When restoring a 1/2 reduced image to the original image size (when restoring to a standard magnification image), the 1 pixel area of the 1/2 reduced image is restored to the 4-pixel configuration, but it is not closed within the macroblock. To do this by interpolation:
For example, in FIG. 42, consider a case where a pixel 1 and a pixel 2 in a unit reduced block region (dotted frame region) of a macroblock are reproduced by interpolation. In this case, the pixel information of “O” is present at the time of processing. Accordingly, since the existing pixels around the pixel 1 and the pixel 2 to be interpolated are the pixels 3, 4, 5, and 6, bilinear interpolation is performed using these pixels 3, 4, 5, and 6. However, “pixel 3” and “pixel 4” are pixels belonging to an adjacent block (adjacent macroblock), and are pixels before enlargement (pixels of a 1/2 reduced image), and the adjacent blocks are temporally related. Since the block is in the position to be processed at the previous time point, it is unnecessary data after the macroblock enlargement process, so it may be discarded from the viewpoint of saving memory resources. is there.

  In the case of such a system, for example, in the case of “pixel 3”, an average value of pixels 7, 8, 9, and 10 that are existing pixels that have been interpolated in the macroblock is obtained, and the discarding is performed. Although it is one method to set the value of the “pixel 3” that has been reduced, if it is desired to reduce the calculation for obtaining the average value, out of the total of four pixels of the pixels 7, 8, 9, and 10, The average value of the pixels 9 and 10 close to the pixels 1 and 2 to be inserted may be used as the value of the discarded “pixel 3”.

  Further, if the pixel 10 is used as it is to obtain the value of “pixel 3”, further calculation can be omitted. Similarly, “pixel 4” is substituted with a pixel value in the vicinity thereof. In the same case, the interpolation of “pixel 11” includes, for example, an average value of pixels 14 and 15 instead of “pixel 12”, and instead of pixel 16 and pixel 18 instead of “pixel 13”. Pixel 17 is used.

Since the frame image Pf is usually encoded by dividing the minimum rectangular range mainly including the object portion, for example, the coding area CA shown in FIG. 59A into blocks (macroblocks), the coding area In the block located at the boundary portion, the blocks adjacent to the left, upper, upper left, and upper right of the block may be located outside the coding area CA.
In this case, as shown in FIG. 42, even when the reproduced pixel value (reproduced pixel value) in the vicinity of the block is used, if the macroblock is located outside the coding area CA, the reproduced binary value is set. Without reference, the nearest pixel value in the self-affiliated macroblock may be assigned and used for convenience as shown in FIG.

  Further, when data is exchanged on a transmission path that is affected by an error, in order to make it less susceptible to an error, the unit is smaller than the coding area CA (this is called a “synchronization recovery unit”). The encoding process may be closed.

  Thereby, it becomes possible to cut off the influence of the error by this “synchronization recovery unit”, and it becomes difficult to be affected by the error. Here, the “synchronization recovery unit” refers to each area surrounded by a dotted line indicated by the symbol Un in FIG. 59 (b). The “synchronization recovery unit” is an area obtained by dividing the coding area CA into small parts, but it is not different from being composed of a plurality of macroblocks.

  In this method, since the encoding process is closed within the “synchronization recovery unit”, even if the data used in a certain “synchronization recovery unit” includes transmission errors, refer to the erroneous data. Processing is performed only within a certain “synchronization recovery unit”, and adjacent “synchronization recovery unit” is not processed with reference to the erroneous data, so that a transmission error is not easily propagated. That is why.

  Even in this case, as shown in FIG. 42, when the reproduction pixel values near the block are used, the reproduction pixel values of the macroblocks belonging to the synchronization recovery unit other than the “synchronization recovery unit” including the block are not referred to. As shown in FIG. 41, the nearest pixel value in the macroblock is assigned.

  As described above, “whether or not to refer to the outside of the macroblock or the synchronization recovery unit” indicates that a switching bit is prepared in the code, and if switching is performed using this bit, the frequency of transmission errors and allowable operations It is possible to deal with various situations such as the amount and the amount of memory.

  By the way, as described above, in bilinear interpolation, processing is performed using only the surrounding four pixels. For this reason, discontinuity in the oblique direction occurs particularly in the image of the reproduced pixels, and the tendency for visual deterioration to occur is unavoidable. To avoid this, for example, taking the example of interpolation of the interpolation target pixel in FIG. 41, pixels included in an enlarged reference range wider than the reference range of bilinear interpolation at the time of the interpolation. Interpolate using.

  In other words, by interpolating using pixels included in the expanded reference range that is wider than the bilinear interpolation reference range, the tendency of pixels in a wider range is reflected, and the problem of discontinuity is avoided. Is done. In addition, the odd number of pixels such as “9 pixels” can obtain the majority effect and the effect of avoiding discontinuity than the even number of pixels used for the interpolation such as “4 pixels”. It may be obtained even more significantly.

FIG. 58B shows an example in which interpolation is performed using 12 pixels. Using the notation of the embodiment described above with reference to FIG. 58 (a), reproduced pixels at certain positions in a macroblock in a 1/2 reduced image of a macroblock are represented by p1, p2, p3, p4, and When those values (pixel values) are Ip1, Ip2, Ip3, Ip4, these Ip1, Ip2, Ip3, Ip4 are:
Ip1: if (4 * Ia + 2 * (Ib + Ic + Id) + Ie + If + Ig + Ih + Ii + Ij + Ik + Il)> 8 then “1” else “0”
Ip2: if (4 * Ib + 2 * (Ia + Ic + Id) + Ie + If + Ig + Ih + Ii + Ij + Ik + Il)> 8 then “1” else “0”
Ip3: if (4 * Ic + 2 * (Ib + Ia + Id) + Ie + If + Ig + Ih + Ii + Ij + Ik + Il)> 8 then “1” else “0”
Ip4: if (4 * Id + 2 * (Ib + Ic + Ia) + Ie + If + Ig + Ih + Ii + Ij + Ik + Il)> 8 then “1” else “0”
It can be expressed as However, a is pixel A, b is pixel B, c is pixel C, d is pixel D, e is pixel E, f is pixel F, g is pixel G, h is pixel H, i is pixel I, and j is pixel J and k are pixels K, and l is a pixel L.

  Further, the “1/4” size reduction / enlargement processing may be realized by performing the “l / 2” size reduction / enlargement processing twice.

  Next, an example of a combination with enlargement / reduction processing in units of frames will be described as a fifth specific example.

(Fifth example)
The prior art disclosed by the present inventors in Japanese Patent Application No. 8-237053 includes an embodiment in which rate control is performed by reducing / enlarging in units of frames (actually, a square area including an object), and a block And an embodiment in which rate control is performed by performing reduction and enlargement in units of small areas. Further, in the first to third specific examples described above, examples in which rate control is performed by performing reduction / enlargement in units of more specific small areas are shown.

  The specific example described here shows an example in which the reduction / enlargement processing in units of frames and the reduction / enlargement processing in units of small areas are used in combination.

<Alpha map encoding device>
FIG. 25 is a diagram illustrating an alpha map encoding apparatus according to this specific example. This apparatus includes reduction circuits 5300, 5210, 5260, binary image encoding circuit 5220, enlargement circuits 5230, 5400, motion compensation prediction circuit 5250, and multiplexing circuits 5240, 5500.

  In such a configuration, the binarized alpha map image supplied via the alpha map signal input line 20 is reduced in frame units in the reduction circuit 5300 based on the reduction / enlargement ratio CR. The signal reduced in frame units is supplied to the alpha map encoding circuit 5200 via the signal line 5020, and is divided into small areas and then encoded.

  Here, the alpha map encoding circuit 5200 is equivalent to the alpha map encoding circuit 200 of FIG. 12, and the components 5210 to 5260 of the alpha map encoding circuit 5200 are the same as those of the alpha map encoding circuit 200 of FIG. Since it has the same function as the components 210 to 260, the description of the alpha map encoding circuit 5200 is omitted here. Further, the configuration of the alpha map encoding circuit 200 in FIG. 12 is a simplified representation of the configuration of the alpha map encoding circuit 200 in FIG. A configuration equivalent to the alpha map encoding circuit 200 may be used.

  In the encoded information encoded by the alpha map encoding circuit 5200, the reduction / enlargement ratio CRb for each small region is multiplexed and supplied to the multiplexing circuit 5500 via the signal line 5030, and is supplied in frame units. The data is multiplexed with the encoded information of the reduction / enlargement ratio CR and output via the signal line 30.

  Further, the reproduced image of the alpha map encoding circuit 5200 is supplied to the enlargement circuit 5400 via the signal line 5040, is enlarged based on the reduction / enlargement ratio CR in frame units, and then output via the signal line 40. .

<Alpha map decoding device>
FIG. 26 is a diagram for explaining a decoding device according to this example. This alpha map decoding apparatus includes separation circuits 6500 and 6430, a binary image decoding circuit 6410, a reduction circuit 6450, a motion compensation prediction circuit 6440, and an expansion circuit 6420 and 6600.

  In such a configuration, the encoded information supplied via the signal line 80 is separated into a frame unit reduction / enlargement ratio CR and small region unit encoded information by the separation circuit 6500. Then, the small area unit encoding information is supplied to the alpha map decoding circuit 6400 via the signal line 6080, and the small area unit reproduction signal is supplied to the enlargement circuit 6600 via the signal line 6090.

  Since the alpha map decoding circuit 6400 is equivalent to the alpha map decoding circuit 400 of FIG. 13, description thereof is omitted here. Further, since the configuration of the alpha map decoding circuit 400 in FIG. 13 is a simplified representation of the configuration of the alpha map decoding circuit 400 in FIG. 2, the configuration of the alpha map decoding circuit 6400 is as shown in FIG. A configuration equivalent to the alpha map decoding circuit 400 may be used.

  The enlargement circuit 6600 enlarges the reproduction signal supplied via the signal line 6090 based on the information on the reduction rate CR in units of frames, and outputs it from the signal line 90.

  As described above, the alpha map signal is reduced / enlarged in units of frames and reduced / enlarged in units of small areas. By combining the reduction / enlargement in units of frames and the reduction / enlargement in units of small areas, side information such as reduction / enlargement ratio information is reduced, so encoding is performed at a particularly low encoding rate. Effective in cases.

Next, let's touch on the frame memory.
Although not explicitly shown in FIG. 25 and FIG. 26, both the encoding device and the decoding device require a frame memory for storing a reproduced image. FIG. 52 is a diagram illustrating an example of the resolution for each frame. In the present invention, since motion compensation prediction is used, for example, when encoding a frame at time n, the resolution of the frame at time n-1 is changed to the resolution at the frame at time n (in this case, reduced / enlarged). Rate). Here, as shown in FIG. 52, the reproduced image to be stored in the frame memory has a frame unit resolution (in this example, CR = 1/2 if it is a frame at time n (FIG. 52). (A)), the frame at time n is stored with CR = 1 (FIG. 52 (c))) and the original resolution (always the reduction / enlargement ratio CR = 1 regardless of the time). Two cases of accumulation are conceivable.

  In the former, the frame memory stores a reproduction image having a resolution in units of frames supplied via the signal line 5040 and the signal line 6090, and the frame memory in the latter is stored via the signal line 40 and the signal line 90. The reproduced image having the original resolution to be supplied is accumulated.

  Therefore, when the frame memory is clearly shown in the encoding device of FIG. 25 and the decoding device of FIG. 26, the former frame memory (this is expressed as FM1 (for the encoding device) and FM3 (for the decoding device)). 53 and 54, the latter frame memory (referred to as FM2 (for the encoding device) and FM4 (for the decoding device)) is shown in FIG. 55 and FIG. 56.

  That is, the encoding apparatus of FIG. 53 is configured to hold the information of the reduction / enlargement ratio CR and the output information of the enlargement circuit 5230 in the frame memory of FM1, and give this hold information to the MC (motion compensation prediction circuit) 5250 54, the decoding apparatus of FIG. 54 is configured to store the CR information and the output from the expansion circuit 6420 in a frame memory such as FM3, and to provide the stored output to the motion compensation prediction circuit 6440. .

  55 is configured to hold the information of the reduction / enlargement ratio CR and the output information of the enlargement circuit 5400 in a frame memory such as FM2, and supply this hold information to the MC (motion compensation prediction circuit) 5250. The decoding apparatus of FIG. 56 is configured to store the CR information and the output from the expansion circuit 6600 and to provide the stored output to the motion compensation prediction circuit 6440.

  A specific configuration example of the frame memories FM1 and FM3 is as shown in FIG. 57 (a), and a specific configuration example of the frame memories FM2 and FM4 is as shown in FIG. 57 (b). Become.

  As shown in FIG. 57 (a), the frame memories FM1 and FM3 reduce the frame memory m11 that stores the image of the current frame and the image held in the frame memory m11 to the information corresponding to the enlargement / reduction ratio CR separately given. A reduction / enlargement circuit m12 for enlargement processing and a frame memory m13 for saving the output reduced / enlarged by the reduction / enlargement circuit m12. The frame memories FM2 and FM4 are shown in FIG. 57 (b). As described above, the frame memory m21 that stores the image of the current frame, the reduction circuit m22 that reduces the image held in the frame memory m21 in correspondence with the information of the enlargement / reduction ratio CR given separately, and the reduction processing by the reduction circuit m22 And a frame memory m12 for storing the output output.

The operation of the frame memory having such a configuration will be described.
First, in the case of the frame memories FM1 and FM3, a reproduction image having a resolution corresponding to the frame is supplied to each of the frames via the signal line 5040 and the signal line 6090 and stored in the frame memory m11 for storing the current frame. . In the frame memory m11, all the reproduced images of the current frame are stored when the encoding of the current frame (for example, time n) is completed.

  Next, the reduction / enlargement circuit m12 reads the reproduced image of the frame at the time n from the frame memory m11 when the encoding of the frame at the time n + 1 is started, and reduces / enlarges the frame at the time n + 1 in units of frames. Reduction / enlargement processing (resolution conversion) is performed so as to match the rate CR.

  In the example of FIG. 52, the frame reduction / enlargement ratio CR at time n is “1/2” as shown in (a), and the frame at time n + 1, which is the next frame, as shown in (b). Since the reduction / enlargement ratio CR is “1”, the reduction / enlargement circuit m12 performs processing for converting the reduction / enlargement ratio CR from “1/2” to “1”.

  The reconstructed image at time n subjected to resolution conversion by the reduction / enlargement circuit m12 is stored in the frame memory m13 of the previous frame, and becomes a reference image for motion compensation prediction of the frame at time n + 1.

  In the case of the frame memories FM2 and FM4, a reproduced image having the original resolution (CR = 1) is supplied to the frame memories FM2 and FM4 via the signal line 40 and the signal line 90, and stored in the frame memory m21 of the current frame. The frame memory m21 stores all the reproduced images of the current frame at the time when the encoding of the current frame (for example, time n) is completed.

  Next, at the time when the encoding of the frame at time n + 1 is started, the reduction circuit m22 inserts the reproduced image of the frame at time n from the frame memory m21, and reduces or enlarges the frame at time n + 1 in frame units. Reduction processing (resolution conversion) is performed so as to match the rate CR.

  Since the reproduced image stored in the frame memory m21 always has the reduction / enlargement ratio CR = 1, in the example of FIG. 52, the reduction / enlargement ratio CR at time n + 1 is “1”. In this case, the reduction circuit m22 Then, resolution conversion processing is not performed. In the case of the example in FIG. 52, when the current frame is time n + 1, the resolution of the frame at time n + 2 is “1/2”. Therefore, in the reduction circuit m22, the reduction / enlargement ratio CR is changed from “l” to “1 /”. The resolution conversion process to 2 "is performed.

  The playback image of the frame at time n, whose resolution has been converted by the reduction circuit m22, is stored in the frame memory m33 of the previous frame, and becomes a reference image for motion compensation prediction of the frame at time n + 1.

  The specific configuration and operation of the frame memories FM1, FM2, FM3, and FM4 have been described above. However, the frame memories FM1 and FM3 are reconstructed images with resolution in units of frames supplied via the signal line 5040 and the signal line 6090. In addition, the frame memories FM2 and FM4 are characterized in that a reproduced image having a resolution in units of frames supplied via the signal line 40 and the signal line 90 is accumulated. In addition, various other configurations can be considered.

(Sixth specific example)
Next, an example of a method of encoding macroblock attribute information will be described as a sixth specific example. First, the proposed method will be described in Japanese Patent Application No. 8-237053, which is a prior art.

FIG. 29 shows an example of attribute information of a certain macroblock at time n and time n-1. However, the attribute information here is “allW” (all the constituent pixels of the macroblock are white), “allB” (the constituent pixels of the macroblock are all black), “Multi” (the constituent pixels of the macroblock) Is information indicating the contents of the contents of the macroblock, such as black and white.
For example, “allW” is labeled “0”, “allB” is labeled “3”, and “Multi” is labeled “1”.

  Focusing on the smallest rectangular area including the object part in the frame and setting a rectangular area so that the upper left of the rectangular area touches the boundary of the area, distribution of attribute information (label) of each macroblock included in the set rectangular area For example, as shown in FIG.

Then, a distribution example of the constituent pixel attribute information of the macroblock in the frame at time n shown in (a) in FIG. 29 and the constituent pixel attribute information of the macroblock in the frame at time n-1 shown in (b). As in the distribution example of FIG. 5, very similar labeling is performed between alpha maps of frames close in time.
Therefore, in such a case, since the correlation of labels between frames is high, encoding efficiency is greatly improved by encoding the label of the current frame using the label of the already encoded frame. Will be.

In general, the encoding area in the frame at time n (minimum rectangular range mainly including the object portion, for example, the encoding area CA shown in FIG. 59A) and the encoding in the frame at time n−1. The region size may be different. In this case, as an example, the encoding area size in the frame at time n−1 is matched with the size in the frame at time n by the procedure shown in FIG. For example, when the coding area size in the frame at time n is one row longer and the column is shorter by one column than the coding area size in the frame at time n−1, as shown in FIG. Next, for the encoding region in the frame at the time n−1 with a short row, the macro block column for one column at the right end in the region is cut, and then the macro block column for one row at the bottom is copied below it. Increase the line.
This state is shown in FIG.

  In addition, when the coding area size in the frame at time n−1 is shorter by one column for the column and longer by one line than the coding area size in the frame at time n, The macroblock sequence for one row is cut, and then the macroblock sequence for the rightmost one column in the coding area is copied adjacently to increase it by one column.

  If the size does not match, adjust the size in this way. Note that the method of adjusting the size is not limited to the above method. Finally, as shown in FIG. 30 (b), the label in the frame at time n-1 adjusted to the size in the frame at time n is referred to as the label at time n-1 ′ for convenience. This will be used for the following explanation.

  FIG. 31A shows the difference between the above-described macroblock attribute information at time n and the above-described macroblock attribute information at time n−1 ′, that is, the difference between each label at each macroblock position. The result taken by the thing of the same position is shown.

  Here, “S” indicates “labels match”, and “D” indicates “labels do not match”.

  On the other hand, FIG. 31 (b) shows the result of taking the difference between the labels of adjacent pixel positions in the above-described macroblock attribute information at time n. Here, the label at the left end takes a difference from the label at the pixel position at the right end on one line, and the label at the pixel position at the upper left end takes a difference from “0”. Hereinafter, for convenience, FIG. 31A is referred to as interframe coding, and FIG. 31B is referred to as intraframe coding.

  As shown in FIG. 31, since the inter-frame coding has a higher ratio of “S” than the intra-frame coding and the inter-frame coding is more predictive, the amount of codes can be reduced.

  In addition, when the correlation between frames is extremely small, there is a possibility that the coding efficiency may be reduced as compared with intra-frame coding. In this case, it is possible to switch between intra-frame encoding with 1-bit code or inter-frame encoding so that encoding can be performed with intra-frame encoding. As a matter of course, since the frame to be encoded first has no reference label, intra-frame encoding is performed. At this time, a code for switching between frames / within frames is not necessary.

An example of switching between several types of prediction methods will be described more specifically.
In the above-described example, the case where intra-frame encoding is performed when the correlation between frames is small is shown. However, in addition to this, for example, when the present invention is used for image transmission, and when transmission errors become a problem, intra-frame coding is effective.

  For example, when a transmission error occurs and the previous frame is not reproduced correctly, if the inter-frame coding is used, the current frame is not reproduced correctly, but if the intra-frame coding is used, it can be reproduced correctly. .

  Even with intra-frame coding, if many macroblocks are referenced, they are vulnerable to transmission errors. In other words, if the number of referenced macroblocks is increased, the amount of code can be reduced, but if the number of referenced macroblocks is increased, the possibility of referring to macroblocks containing transmission errors increases. If the number of macroblocks to be referred to is reduced, the amount of code increases. It can be said that it will become stronger.

  Therefore, it is necessary to devise a technique that is resistant to transmission errors and reduces the amount of codes. You can do this as follows:

As an example, in order to be resistant to transmission errors and reduce the amount of codes, it is effective to prepare several prediction modes and switch between them.
As a prediction mode, for example,
(A) Inter-frame coding mode (B) Intra-frame coding mode (C) Synchronization recovery intra-unit coding mode (D) There is a mode without prediction.

  Here, as already described, the “synchronization recovery unit” means that the rectangular area of the object is further divided, that is, the rectangular coding area CA is further divided in a required macroblock unit, For example, it is divided so that the code amount of each synchronization recovery unit becomes equal, or a predetermined number of macroblocks are combined into a synchronization recovery unit.

  In “Synchronous recovery unit coding mode”, the reference block (the macroblock to be referenced, that is, the adjacent macroblock of its own macroblock) is outside the “synchronous recovery unit” even in the frame. In some cases, for example, a predetermined label is used as a predicted value without reference.

  As a result, even if a transmission error occurs in the frame, if it is outside the “synchronization recovery unit”, the “synchronization recovery unit” can be correctly reproduced.

  “No prediction” means that the label of the macroblock is encoded without referring to other macroblocks at all, and this mode is strongest against errors.

  These multiple modes are prepared, and one of the optimum modes is selected and used depending on the frequency of occurrence of errors. The switching may be performed for each “synchronization unit”, for each frame, or for each sequence. Information on which mode is used for encoding is sent from the encoding device to the decoding device.

  As another mode, there is a method of switching the encoding table depending on the occupied position in the frame of the target region to be encoded.

That is, as a general tendency of an image, for example, as shown in FIG. 20, there is a high probability that an object exists at the center of the frame, and a high probability that no object exists at the end of the frame.
Focusing on this, if a macroblock that touches the end of the frame uses a table that assigns a short code to “allW”, and other macroblocks use a table that assigns a short code to “allB”. The amount of code can be reduced without using prediction. This is the no prediction mode.

  There is also a method of preparing a plurality of encoding tables more simply and switching the encoding tables for use. The switching information is encoded, for example, for each “synchronization recovery unit”, for each frame, or for each sequence.

  FIG. 32 is a block diagram showing a system configuration of this example for realizing the above-described processing, and the flow of processing will be described with reference to this block diagram.

  In the configuration of FIG. 32, a portion surrounded by a broken line is a portion related to this specific example for realizing the processing described above. FIG. 32A shows an alpha map encoding apparatus, which includes an object area detection circuit 3100, a blocking circuit 3110, a labeling circuit 3120, a block encoding circuit 3130, a label memory 3140, a size changing circuit 3150, and a label encoding circuit 3160. , And a multiplexing circuit (MUX) 3170.

  Among these, the object area detection circuit 3100 detects a square area of a portion including the object in the alpha map signal based on the input alpha map signal, and includes the information about the size of the square area. This outputs an alpha map signal of a rectangular area. The blocking circuit 3110 is a circuit that converts the alpha map signal in the rectangular region into a macro block, and the labeling circuit 3120 applies the macro block converted alpha map signal for each block in the macro block. The attributes (allW (white only), Multi (mixed white and black), allB (black only)) of the alpha map signal contents are determined, and labels ("0", "1", "3") corresponding to the respective attributes ).

  The block encoding circuit 3130 is a circuit that encodes an alpha map signal in a macroblock having a label of “1” (Multi), and the label memory 3140 is supplied from the labeling circuit 3120. A memory for storing label information and area size information given from the object area detection circuit 3100 via the label memory output line 3020 and supplying the accumulated label information and size information to the size changing circuit 3150 together. is there.

  The size changing circuit 3150 uses the label information and size information of the frame at time n−1 supplied from the label memory 3140 and the size information of the frame at time n given from the object area detection circuit 3100 to generate a time n−. 1 is a circuit that changes the size of the label information so as to correspond to the size of the time n, and the label encoding circuit 3160 is supplied from the labeling circuit 3120 with the size-changed label information as a predicted value. It is a circuit that encodes label information.

  The multiplexing circuit 3170 multiplexes the encoding information obtained by the label encoding circuit 3160, the encoding information supplied from the block encoding circuit 3130, and the size information given from the object area detection circuit 3100. It is a circuit to output.

  In the encoding apparatus having such a configuration, the alpha map signal supplied via the signal line 3010 is supplied to the object area detection circuit 3100, and the object area detection circuit 3100 includes a square including an object from the alpha map signal. Detect areas. Information regarding the size of the rectangular area is output via a signal line 3020, and the alpha map signal inside the area is supplied to the blocking circuit 3110.

  The blocking circuit 3110 converts the alpha map signal inside this area into a macro block. The alpha block signal that has been macroblocked is supplied to a labeling circuit 3120 and a block coding circuit 3130.

  The labeling circuit 3120 determines attributes (“allW”, “Muti”, “allB”) for each macroblock, and assigns labels (“0”, “1”, “3”) corresponding to the attributes. The assigned label information is supplied to a block encoding circuit 3130, a label memory 3140, and a label encoding circuit 3160.

  In the block encoding circuit 3130, when the label is “1” (Multi), the alpha map signal in the macroblock is encoded, and the encoded information is supplied to the multiplexing circuit 3170. In the label memory 3140, label information supplied from the labeling circuit 3120 and area size information via the label memory output line 3020 are accumulated, and the label information and the size information are combined from the label memory output line 3030 to the size changing circuit 3150. Supplied to.

  In the size changing circuit 3150, the time information is obtained from the label information and size information of the frame at time n−1 supplied through the label memory output line 3030 and the size information at time n supplied through the signal line 3020. The label information whose size is changed so that the label information of n−1 corresponds to the size of time n is supplied to the label encoding circuit 3160.

  The label encoding circuit 3160 encodes the label information supplied to the labeling circuit 3120 using the label information supplied from the size changing circuit 3150 as a predicted value, and the encoded information is supplied to the multiplexing circuit 3170. In the multiplexing circuit 3170, the encoding information supplied from the block encoding circuit 3130, the label encoding circuit 3130, and the label encoding circuit 3160 and the size information supplied via the label memory output line 3020 are multiplexed. Thereafter, the signal is output via the signal line 3040.

  The above is the configuration and operation of the encoding apparatus. Next, the configuration and operation of the decoding apparatus will be described.

  The alpha map encoding apparatus shown in FIG. 32B includes a separation circuit (DMUX) 3200, a label composite circuit 3210, a size change circuit 3220, a label memory 3230, and a block composite circuit 3240.

  Among these, the separation circuit 3200 is a circuit that separates encoded information supplied via the signal line 3050, and the label composite circuit 3210 is supplied from the size changing circuit 3220 at the time n-1. This is a circuit that reproduces label information at time n using information obtained by changing the size of label information as a predicted value.

  The size changing circuit 3220 is a circuit that functions in the same manner as the size changing circuit 3150, and is supplied from the label memory 3230 and includes label information and size information in the frame at time n−1, and a separation circuit 3200. Is a circuit that changes the size of the label information in the frame at time n-1 so as to correspond to the size at time n from the size information of the frame at time n given separately from the label n. This is a circuit having the same function as that of the label memory 3140. The label information decoded and supplied from the label combination circuit 3210 and the size information of the area given from the separation circuit 3200 are accumulated and stored. This is a memory for supplying label information and size information to the size changing circuit 3220 together.

  The block decoding circuit 3240 is a circuit that reproduces an alpha map signal for each block in accordance with the reproduced label information supplied from the label decoding circuit 3210.

The operation of the decoding apparatus having such a configuration will be described.
The demultiplexing circuit 3200 demultiplexes the encoded information supplied via the signal line 3050, supplies the encoded information to the block decoding circuit 3240 and the label decoding circuit 3210, and outputs size information via the signal line 3060. . The label decoding circuit 3210 reproduces the label information in the frame at time n using the information obtained by changing the size of the label information in the frame at time n−1 supplied from the size changing circuit 3220 as a predicted value.

  The reproduced label information is supplied to the block decoding circuit 3240 and the label memory 3230. The block decoding circuit 3240 reproduces the alpha map signal for each block according to the reproduced label information supplied from the label decoding circuit 3210. The size changing circuit 3220 and the label memory 3230 operate in the same manner as the size changing circuit 3150 and the label memory 3140, respectively, and thus will not be described in detail here.

  As described above, it is combined with an encoding device that labels the alpha map in units of macro blocks and uses the macro block label of the already encoded frame to encode the macro block label of the current frame. An example of an apparatus has been described. The macroblocks are labeled very similar between alphamaps in near temporal frames. Therefore, in such a case, since the correlation between the labels is high between the frames, encoding of the label of the current frame by using the label of the already encoded frame greatly improves the encoding efficiency. Will be able to.

  By the way, in the invention as such a prior art, the VLC (variable length coding) table is switched with reference to adjacent one block (one macro block) within a frame or between frames. In this case, when the inter-frame correlation is high, the “adjacent block between frames” is referred to, and when the inter-frame correlation is low, the “adjacent block in the frame” is referred to switch the VLC table. . However, in actual applications, it is often better to use both interframe correlation and intraframe correlation.

  Therefore, if the mode at a certain pixel position is “M (h, v, t)” (h, v, and t represent coordinate axes in the horizontal, vertical, and time directions, respectively), the mode “M (x, y, n ) ", For example," M (x-1, y, n) "," M (x, y-1, n) "," M (x, y, n-1) " The VLC table is selected by referring to it. Here, when the number of modes is three as shown in FIG. 29, if the number of reference blocks is three (3 macroblocks), the number of VLC tables is 3 to the third power (= 27). It is also possible to increase the number of reference blocks further (for example, “M (x−1, y−1, n)”, “M (x, y, n−2)”).

  In this case, not only the number of VLC tables increases, but also the inter-block correlation with newly added reference blocks decreases, so that the coding efficiency does not improve much even if the number of reference blocks is increased. Therefore, it is necessary to make a trade-off between the number of VLC tables and encoding efficiency.

  Next, a specific example of another method for encoding block attribute information will be described.

  Here, a method of encoding block attribute information using the label of the previous frame for prediction will be described.

<Specific Example of Encoding Device that Uses Previous Frame Label for Prediction>
FIG. 43 is a block diagram of an encoding circuit as a specific example of the present invention. As shown in the figure, this encoding circuit includes an object area detection circuit 502, a blocking circuit 504, a labeling circuit 506, a label encoding circuit 508, a label memory 509, a reference block determination circuit 510, and a prediction circuit 512. The

  Among these, the object area detection circuit 502 sets an area represented by a multiple of the block size including the object from the alpha map signal 501 as an encoding area, and cuts out the alpha map signal 503 of the encoding area. A blocking circuit 504 divides (blocks) the extracted alpha map signal 503 into block units (macroblock units) having a 16 × 16 pixel configuration, and outputs a labeling circuit 506. In this case, a predetermined label corresponding to the degree of inclusion of the object is assigned to the blocked alpha map signal 505 and output as label information 507.

  The label encoding circuit 508 encodes and outputs the label information 507 by switching the encoding table according to the given predicted value 514, and the label memory 509 is the above-described label assigned to each block by the labeling circuit 506. Label information 507 is stored, and the reference block determination circuit 510 performs processing such as determining a block at the same position as the encoded block in the previous frame as the reference block 511, and the prediction circuit 512 includes a label memory. The label 513 of the previous frame held in 509 is referenced, the label at the position of the reference block 511 is predicted, and this is sent as a predicted value 514 to the label encoding circuit 508.

In the encoding apparatus having such a configuration, the alpha map signal 501 is input to the object region detection circuit 502. In the object area detection circuit 502, an area including an object and represented by a multiple of the block size is set as an encoding area, and an alpha map 503 cut out in the encoding area is sent to the blocking circuit 504. In the blocking circuit 504, the alpha map 503 is divided into block units (macro block units) having a 16 × 16 pixel configuration, and the blocked alpha map 505 is sent to the labeling circuit 506. In the labeling circuit 506, for example,
-Object is not included in the block: “Label 0”
-The object is included in a part of the block: "Label 1"
-Everything in the block is an object: "Label 3"
The label information 507 (mode information) is given for each block. The label information 507 is sent to the label encoding circuit 508 and stored in the label memory 509. The label memory 509 stores labels that have been encoded so far.

  On the other hand, in the reference block determination circuit 510, for example, a block at the same position as the encoded block in the previous frame is determined as the reference block 511 and sent to the prediction circuit 512. The prediction circuit 512 also receives the label 513 of the previous frame from the label memory 509 and sends the label at the position of the reference block 511 to the label encoding circuit 508 as the prediction value 514. In the label encoding circuit 508, the encoding table is switched by the predicted value 514, the label information 507 is encoded, and the code 515 is output.

  Here, when the coding area is always equal to the frame, one reference block is determined. However, if the coding area is smaller than the frame and the position of the coding frame is different between the previous frame and the current frame, use the coordinate axis with the corner of the frame as the origin, or use the corner of the coding area as the origin. The reference block differs depending on the coordinate axis.

  The handling of the coordinate axes will be described in detail.

  44 (a) and 44 (b) show examples of mode information MD of frame images Fn-1 and Fn at time n-1 and time n and each macroblock of the coding area CA in the respective frames Fn-1 and Fn. It is.

  In the invention of the prior application (Japanese Patent Application No. 8-237053), as an example, the origin Vc0 of the coding area in the current frame (time n) and the origin Vp0 of the coding area in the previous frame (time n-1). A specific example of determining a block to be used as a reference when coding mode information of the block is presented. In this method, the blocks are associated based on the coordinate axes of the coding area.

  In this case, as shown in FIG. 45A, the right end or lower end of the encoding area of the previous frame is “cut” or “copyed” to match the encoding area size of the current frame.

  In the example of FIG. 44, the left end and the upper end of the encoding area are changed. In such a case, the blocks corresponding to the mode information of the current frame do not match the shaded blocks (21 blocks) as shown in FIG. 45 (a). There is a risk that the conversion efficiency will decrease.

  In the example as shown in FIG. 44, it is better to match the origin Fc0 of the current frame with the origin Fp0 of the previous frame and use the block at the closest block position on the coordinate axis of the frame as the reference block.

  In the case of FIG. 44, when the reference block is obtained based on the coordinate axis of the frame, it is as shown in FIG. That is, in the example of FIG. 44, there is a change between the left end and the upper end, so that the left end and the upper end are “cut” or “copy” as shown in FIG. 45B to match the size of the current frame coding area. I will let you. In this case, as shown in FIG. 45B, the blocks corresponding to the mode information of the current frame are only the hatched blocks (three) as shown in FIG.

  That is, the coding efficiency can be improved by switching whether the label of the previous frame is changed based on the coordinate axis of the coding area or the coordinate axis of the frame depending on the situation. As a method for determining the coordinate axis, the optimum information may be selected on the encoding device side and switching information may be sent, or both the encoding device and the decoding device may use known information.

  FIG. 46 is an example in which it is better to use the coordinate axes of the coding area. This example shows that the frame has changed significantly from the frame Fn-1 in (a) to the frame Fn in (b), as in the case where the camera is shaken to the right. In the case of this example, as apparent from the figure, the position of the coding area in the frame is greatly shifted, so it is not a good idea to determine the reference block based on the coordinate axis of the frame.

  That is, if the position of the coding area CA in the frame is greatly deviated between the current frame (frame Fn at time n) and the previous frame (frame Fn-1 at time n-1), the coding area It is better to use the coordinate axis of the frame, and when the position of the coding area in the frame is not so shifted, it is better to use the coordinate axis of the frame.

Whether or not the position of the coding area in the frame between the current frame Fn and the previous frame Fn-1 is greatly deviated depends on information (vector) prev_refscurr_ref indicating the position of the coding area in the frame and the size of the coding area. You can judge from that.
That is, the encoded data structure in the moving image encoding apparatus that uses the alpha map is as shown in FIG. 47 according to the standard. That is, the encoded data includes an encoding area layer, a macroblock MB layer, and a binary shape layer. The encoding area layer includes encoding area size information, encoding area position information, and encoding area reduction / enlargement. Includes rate information. The MB layer is composed of binary shape information, texture MV information, multi-value shape information, and texture information. The binary shape information is mode information, motion vector information, reduction / enlargement ratio information, scan direction information, binary. Contains encoding information.

  Among these, the coding area size information indicates information indicating the size (vertical and horizontal size) of the encoding area, and the coding area position information indicates information indicating the position of the encoding area (the positions of Vp0 and Vc0). The encoding area reduction / enlargement ratio information refers to reduction / enlargement ratio (CR) information of a binary image in units of encoding areas.

  The MB encoded information indicates information for reproducing an object in the MB. The binary shape information in the MB layer indicates information indicating whether each pixel in the MB is inside the object, and the texture MV information is motion vector information for motion compensation prediction of the luminance signal and color difference signal in the MB. The multi-value shape information indicates weighting information when the object is combined with another object, and the texture information indicates the encoding information of the luminance signal and color difference signal in the MB.

  The mode information in the binary shape layer indicates information representing the attribute of the binary image in the MB, and the motion vector information indicates motion vector information for motion compensation prediction of the binary image in the MB. The enlargement rate information indicates the reduction / enlargement rate (CR) information of the binary image in MB units, and the scan direction information indicates information indicating whether the encoding order is the horizontal direction or the vertical direction. Indicates encoding information of a binary image.

  Information indicating the position of the coding area (positions of Vp0 and Vc0) is stored in the coding area position information. Therefore, the position of the coding area (positions of Vp0 and Vc0) is known by using this information. be able to. The frame Fn-1 and the frame Fn are compared using this information. This comparison is performed by obtaining vectors from the home position of the frame to the home position of the coding area.

  As a result, for example, as shown in FIG. 46, when the difference between “prev_ref and curr_ref” is large and there is almost no change in the size of the encoding region between the current frame and the previous frame, the difference is based on the coordinate axis of the encoding region. You can see that it is better. Here, “prev_ref and curr_ref” and the information on the size of the coding area are coded prior to coding of the coding area, and are known information on the decoding device side, so which coordinate axis is used. No additional information for identifying

  In addition, as shown in FIG. 48A, when the encoding area 531 is set as a part of the frame 530, the label is not determined outside the encoding area 531. However, when encoding the next frame, the outside of the encoding area 531 may be a reference block, so it is necessary to insert some label.

  FIG. 48 (d) shows an example in which a predetermined value, in this case “0”, is inserted into the unlabeled portion in FIG. 48 (a). FIG. 48 (c) is an example in which the label undetermined portion in FIG. 48 (a) is extrapolated from the encoding area. This is because the movement of the object is large or the shape changes drastically. This is effective when there is a high probability that an object will appear in the next frame in a part where there was no object.

  FIG. 48B shows an example in which only the portion of the coding area 532 of the next frame is extrapolated and the other portions are not overwritten in the memory space of the label memory 509 with respect to the label undetermined portion of FIG. Yes, in this way, the labels of two or more previous frames can be used for prediction.

  When it is desired to use the label of the previous frame for prediction, there is another method of updating only the coding area in the memory space without performing extrapolation or insertion of a predetermined value at all.

Next, the label prediction method when the frame reduction / enlargement ratio (CR) described with reference to FIG. 52 is switched for each frame will be described.
FIG. 60 shows an example of reduction processing, in which the previous frame is “CR = 1” and the current frame is “CR = 1/2”. In this case, the target of the current frame to be obtained by reduction As shown in the figure, there are four blocks of MB2 to MB5 corresponding to the block, for example, the macroblock MB1 (see FIG. 61). That is, the macroblocks MB2, MB3, MB4, and MB5 of the previous frame become the reduced macroblock MB1.

  Now, assuming that the address of the macroblock MB1 of the current frame is (x, y), the addresses of the blocks MB2 to MB5 of the previous frame are (2x, 2y), (2x + 1, 2y), (2x, 2y + 1), (2x + 1, 2y + 1).

  The coefficient “2” here is given as a ratio of the value of the reduction / enlargement ratio CR between the previous frame and the current frame.

In order to predict the encoding of the label of the block MB1, it is appropriate to use any of the labels of the blocks MB2, MB3, MB4, and MB5, but there are several methods for determining the label.
First, the simplest and the least amount of calculation is a method using a label of a block at a predetermined position (for example, upper left) among the four. Alternatively, when there are four labels that are the same, if the label with the largest number is used as the predicted value, the probability that the prediction is successful will increase.

  If the number of labels with the same content is equal, that is, if they are divided into two sets, order the labels in advance in descending order of appearance frequency, and select the upper label as the predicted value in that order. To do. When a rectangular coding area CA is set in a frame, using a coordinate axis whose origin is the corner of the coding area CA, the boundary of the block (macroblock to be coded) of the current frame is as shown in FIG. Overlaps the macroblock boundary of the reference frame.

  However, if the region to be encoded can be set in steps smaller than the width of the macroblock and the coordinate axis with the frame corner as the origin is used, generally the block boundaries do not overlap as shown in FIG. A total of nine macroblocks of ~ MB14 are referenced.

  In this case, the label of the macro block MB10 that is referred to as a whole is used.

  FIG. 63 shows an example in which the previous frame has a reduction / enlargement ratio CR = 1/2 and the current frame has a reduction / enlargement ratio CR = 1. In this case, the macroblock MB19 has Will refer to the part. At this time, the label of the block MB15 is used as a predicted value, or since the reference portion is also close to the blocks MB16 to MB18, the predicted value is determined by, for example, majority voting as described above in consideration of the labels of these blocks. You may decide.

<Configuration Example of Decoding Device Using Label for Prediction>
FIG. 49 is a block diagram showing a configuration example of the decoding apparatus according to the present invention in which labels are used for prediction.

  This decoding apparatus includes a label decoding circuit 516, a label memory 517, a reference block determination circuit 518, and a prediction circuit 520.

  Among these, the label decoding circuit 516 decodes the label data from the input code data to be decoded, and the label memory 517 stores the decoded label data. The reference block determination circuit 518 performs processing such as determining a block at the same position as the encoded block in the previous frame as the reference block 519.

  Further, the prediction circuit 520 has a function of obtaining a prediction value 522 from the label 521 and the reference block 519 of the previous frame, and giving the prediction value 522 to the label decoding circuit 516.

In the decoding apparatus having such a configuration, the encoded data stream 515 to be decoded is input to the label decoding circuit 516, and the label is decoded.
On the other hand, the label memory 517 stores labels that have been decoded so far. In the reference block determination circuit 518, the reference block 519 is determined and sent to the prediction circuit 520 in the same manner as described in the encoding apparatus. Also in the prediction circuit 520, the prediction value 522 is obtained from the label 521 and the reference block 519 of the previous frame, and sent to the label decoding circuit 516, similarly to the encoding device. The label decoding circuit 516 decodes and outputs the label 523 by switching the decoding table according to the predicted value 522.

  Further, when encoding a mode “M (h, v, t)” (h, v, and t respectively represent horizontal, vertical, and time coordinate axes) of a certain block, for example, “M (x− 1, y, n) ”,“ M (x, y−1, n) ”,“ M (x, y, n−1) ”, and the like are used to switch the encoding table. A mode set shown below (hereinafter referred to as mode set A) including a part of motion vector information used for motion compensation can also be used in the mode.

[Mode Set A]
(1) “a11W”
(2) “a11B”
(3) “copy (motion vector == 0)”
(4) “copy (motion vector! = 0)”
(5) “coded”
Here, (3) and (4) of mode set A are both copy modes, but (3) of mode set A means that the motion vector is zero, and (4) of mode set A Means that the motion vector is non-zero. In the case of mode set A (4), it is necessary to separately encode the value of the motion vector. In the case of mode set A (3), it is not necessary to encode the motion vector. When mode set A is used when the probability that the motion vector is zero is high, the sum of the code amount of the mode and the code amount of the motion vector can be reduced.

In this example, when all the pixels of the block obtained as “copy (motion vector == 0)” are, for example, black, the same reproduced image can be obtained in both modes (2) and (3). That is, there is no need to separate these two modes. Similarly, when all the pixels of the block obtained as “copy (motion vector == 0)” are white, it is not necessary to separate (1) and (3). Therefore,
[Mode Set B]
(1) “a11W”
(2) “a11B”
(3) “copy (motion vector! = 0)”
(4) “coded”
As
Step A1: If the motion compensated prediction image obtained when “copy (motion vector == 0)” is all black, the process proceeds to step A3. Otherwise, the process proceeds to step A2.

  Step A2: If the motion compensated prediction image obtained when “copy (motion vector == 0)” is all white, the process proceeds to Step A4. Otherwise, the process proceeds to Step A5.

  Step A3: If “M (*, *, *)” to be referred to is “copy (motion vector == 0)”, “M (*, *, *)” is replaced with “a11B”. Proceed to step A6.

  Step A4: When “M (*, *, *)” to be referred to is “copy (motion vector == 0)”, “M (*, *, *)” is replaced with “a11W”. Proceed to step A6.

  Step A5: “M (x, y, n)” is encoded using the encoding table of mode set A. End of encoding.

  Step A6: “M (x, ysn)” is encoded using the encoding table of mode set A. End of encoding.

  By using the algorithm (FIG. 50) that takes the above steps A0 to A6, there is no waste that a plurality of modes are prepared for the same result, and the code amount of block attribute information can be reduced. This is because the average code length of a code that switches four modes (mode set B) can be shorter than the average code length of a code that switches five modes (mode set A). However, the method of switching only the mode set A on a block-by-block basis is used when the increase in the amount of calculation and memory increases slightly compared to the case of using only the mode set A, and this increase is not a problem.

  In the decoding process, the coding table is determined to be either for mode set A or mode set B with the same algorithm as in the flowchart of FIG. 50, and decoding is performed using that table.

FIG. 51 shows another algorithm that can obtain the same effect as the above-described algorithm. here,
[Mode Set C]
(1) “a11W”
(2) “copy (motion vector == 0)”
(3) “copy (motion vector! = 0)”
(4) “coded”
[Mode set D]
(1) “a11B”
(2) “copy (motion vector == 0)”
(3) “copy (motion vector! = 0)”
(4) “coded”
It is.

  The flowchart of FIG. 51 will be described.

  Step B1: If the motion compensated prediction image obtained when “copy (motion vector == 0)” is all black, the process proceeds to Step B3. Otherwise, the process proceeds to Step B2.

  Step B2: If the motion compensated prediction image obtained when “copy (motion vector == 0)” is all white, the process proceeds to Step B4. Otherwise, the process proceeds to Step B5.

  Step B3: When “M (*, *, *)” to be referred to is “allB”, “M (*, *, *)” is replaced with “copy (motion vector == 0)”. Proceed to B6.

  Step B4: When “M (*, *, *)” to be referred to is “a11W”, “M (*, *, *)” is replaced with ““ copy (motion vector == 0) ”. Proceed to

  Step B5: “M (x.y.n)” is encoded using the encoding table of mode set A. End of encoding.

  Step B6: “M (xsysn)” is encoded using the encoding table of mode set C. End of encoding.

  Step B7: “M (x.y.n)” is encoded using the encoding table of the mode set D. End of encoding.

  As the block attributes, coding parameters such as block size, block reduction rate, coding scan direction, motion vector value, and the like can be included as necessary. An example including the scan direction is shown.

[Mode set E]
(1) “a11W”
(2) “a11B”
(3) “copy (motion vector == 0)”
(4) “copy (motion vector! = 0)”
(5) “Coded and horizontal scan”
(6) “Coded and vertical scan”
“==” indicates that the left side is equal to the value on the right side, and “! =” Indicates that the left side is not equal to the value on the right side.

  As described above, in the present invention, the reference block of the previous frame is used for prediction. However, instead of the label of this reference block, the alpha map itself may be used for prediction. In other words, the alpha map is stored in the memory, and each time the attribute of each block is encoded, the attribute of the reference block (“allW”, “Multi”, “allB”, etc.) is determined, and Switch coding table.

  In this way, the block used when the previous frame was encoded can be a reference block even if its position is shifted by several pixels. That is, the reference block does not have to overlap with the block used when the previous frame is encoded, and prediction with higher accuracy is possible.

It is also possible to combine prediction using an alpha map of a reference block and prediction using a label.
For example, first, using the alpha map of the reference block, the encoding table is switched by “allW”, “Multi”, “allB”, and for “Multi”, encoding is performed using the reference block label. Switch between tables.

  There is also a method of using a motion vector given for each block for a part to be referred to in the previous frame. That is, the part indicated by the motion vector of the macroblock adjacent to the encoded block that has already been encoded is cut out from the previous frame, and whether it is “allW”, “Multi”, or “allB”. Switch coding table.

(Seventh specific example)
The present invention uses an alpha map signal representing an object shape and a position in a screen in order to separate the background and the object in the method of encoding the image by dividing the screen into a background and an object when encoding the image. To do. Then, this alpha map signal is encoded with the encoded information of the image and converted into a bit stream, which is transmitted and stored. The former is for broadcasting and personal computer communication, the latter is for music CD, etc. It will be traded as a product containing the content.

  Then, when providing moving image content recorded as a product on a storage medium, a storage medium is stored as a bitstream that combines the encoded information of the image and the compression encoded version of the alpha map signal. An example of a playback system for a storage medium that stores a compression-encoded bitstream including an alpha map image, while allowing movies to be watched by storing content over a long period of time on the medium. This will be described as a seventh specific example.

<Specific examples of a medium for storing information according to the present invention>
This specific example will be described with reference to FIGS. FIG. 27 shows code sequences of mode information b0, motion vector information b1, reduction / enlargement ratio information b2, scan direction information b3, and binary image encoding information b4 multiplexed in the VLC / multiplexing circuit 1800 of FIG. An example format is shown. In the present invention, when decoding the binary image encoded information b4, information from b0 to b3 needs to be decoded. Further, if the mode information b0 is not decoded prior to all other information bl to b4, the other information cannot be decoded. Therefore, each information b0 to b4 must have a configuration in which mode information is arranged at the beginning and binary plane image coding information is arranged at the end as shown in FIG.

  FIG. 28 is a diagram showing a system for reproducing an image signal using the recording medium 8100 in which the code string shown in FIG. 27 is stored. The recording medium 8100 stores a code string including the code string shown in FIG. Reference numeral 8200 denotes a decoder for reproducing an image signal from a code string stored in the storage medium 8100, and 8300 is an image information output device for outputting a reproduced image.

  In this system having such a configuration, a code string having a format as shown in FIG. The decoder 8300 reproduces an image signal from the code string stored in the storage medium 8100. That is, the decoder 8200 reads a code string from the storage medium 8100 via the signal line 8010, and generates a reproduced image by the procedure shown in FIGS. FIG. 34 is a flowchart of the “binary image reproduction” step (S5) in FIG.

  The contents of the processing in the decoder 8200 will be described with reference to FIGS. That is, first, mode information is decoded (step S1), and it is checked whether or not the decoded mode information corresponds to any of “allW”, “allB”, and “copy” (steps S2, S3, S4). ).

  As a result, if “allW”, all the pixel values in the macroblock are set to white (step S6), and if “allW” is not “allW”, all the pixel values in the macroblock are all processed. The processing is finished in black (step S7), and if it is not “allW”, “allB”, or “copy”, the motion vector is decoded (step S8), and motion compensation prediction is performed (step S8). In step S9), the macro block is copied with the calculated motion compensation prediction value (step S10). Then, the process ends.

  On the other hand, if it is not “allW”, “allB”, or “copy” in steps S2, S3, S4, the process proceeds to binary image reproduction processing (step S5).

  The process in S5 is as shown in FIG. 34. First, it is checked whether or not the "inter" encoding is performed (step S21). As a result, if it is “inter” encoding, motion vector information is decoded (step S25), motion compensation prediction is performed (step S26), then reduction / enlargement ratio information is decoded (step S22), and scan direction information is obtained. Decoding is performed (step S23). Then, the binary encoded information is decoded (step S24), and the process ends.

  On the other hand, if the result of determination in step S21 is not “inter” encoding, the reduction / enlargement ratio information is decoded (step S22), and the scan direction information is decoded (step S23). Then, the binary encoded information is decoded (step S24), and the process ends.

  In this way, the decoder 8200 reproduces an image, and supplies the reproduced plane image to the image information output device 8300 via the signal line 8020, whereby the reproduction plane image is displayed on the image information output device 8300. .

  Here, the image information output device refers to, for example, a display or a printer. As in the previous specific example, in the case of an encoding / decoding device that combines reduction / enlargement in units of frames and reduction / enlargement in units of small areas, information on the reduction / enlargement ratio of frame units is shown in FIG. Therefore, the code of the frame unit reduction / enlargement ratio is positioned before the code sequence of all the small region units in the frame.

  As described above, when a moving image and an alpha map signal as contents are compressed and encoded and stored as a bit stream in the storage medium, a playback system for the storage medium can be provided.

(Eighth specific example)
A specific example regarding motion vector predictive coding will be described as an eighth specific example.

  21 and 22 are diagrams showing the framework of the present invention. In FIG. 21, the motion vector detected by the motion vector detection circuit 1780 is supplied to the MV encoding circuit 1790 via the signal line 1070, encoded, and then supplied to the VLC / multiplexing circuit 1800. It is multiplexed with the encoded information and output via line 30.

  In FIG. 22, the motion vector information b 1 separated from the encoded information supplied via the signal line 80 by the VLD / separation circuit 2100 is reproduced as a motion vector signal by the motion vector reproduction circuit 2900.

  This specific example relates to the MV encoding circuit 1790 and the motion vector reproduction circuit 2900 described above.

  In general, since the correlation of a motion vector signal is strong between adjacent blocks, a motion vector is encoded by predictive encoding in order to remove this correlation.

FIG. 66 is a diagram for explaining an example of motion vector predictive coding.
In FIG. 66, each square frame indicates a macroblock, and a square frame whose background is indicated by a dot pattern is a block to be encoded. If the motion vector of the encoding target block is MVs, the prediction vector MVPs for the MVs of the encoding target block can be obtained by the macroblock immediately before the encoding target block (next to the left of the encoding target block in FIG. 66). ), The motion vector MVs2 of the block immediately above the current block to be encoded (next to the top of the current block in FIG. 66), and the block immediately to the right of the immediately preceding macroblock Motion vectors MVs are used.

  As described above, the prediction vector MVPs for the motion vector MVs of the encoding target block is generally obtained using the motion vectors MVs1, MVs2, and MVs3 of the blocks around the encoding target block.

  For example, horizontal and vertical components MVPs_h and MVPs_v of MVPs are obtained as follows.

MVPs_h = Median (MVs1_h, MVs2_h, MVs3_h)
MVPs_v = Median (MVs1_v, MVs2_v, MVs3_v)
Here, “Median ()” is a process for obtaining the median of the values in “()”, and the horizontal and vertical components of the motion vector MVsn (n = 1, 2, 3) are respectively
MVsn_h, MVsn_v
It is written.

  As another example of obtaining the prediction vector MVPs, it is checked whether or not there is a motion vector in each block in the order of MVs1, MVs2, and MVs3, and the motion vector of the block in which the motion vector exists first is set as MVPs. There is also a method.

  FIGS. 67A and 67B are examples of frame images Fn−1 and Fn at time n−1 and time n and encoding regions CAn−1 and CAn in the respective frames. Here, when the block around the encoding target block is a block that does not include an object, there is no motion vector in the block. Also, in the case of a block that has been intra-coded, there is no motion vector in that block.

For example, when all of MVs1, MVs2, and MVs3 do not exist, a default value (vector) is used as the prediction vector MVPs.
When the motion of the object is small, there is no problem even if this default value is set to “zero vector”, which is a motion vector that is zero, but as in the transition from the frame of FIG. 67 (a) to the frame of FIG. 67 (b), When the movement of the position of the object in the frame is large, the motion vector is not predicted and the encoding efficiency is lowered.

  In the present invention, as this default value, the difference vector “offset” between the vector “prev_ref” shown in FIG. 67A and the vector “curr_ref” shown in FIG. It is characterized by being switched adaptively.

  Here, “offset” is obtained by the following equation.

offset = prev_ref-curr_ref
The switching between the default value “offset” and “zero vector” is obtained, for example, based on the error value of the object at time n and the object at time n−1 obtained based on the coordinate axis of the frame and the coordinate axis of the encoding area. When the error value of the object at time n and the object at time n−1 is compared, the default value is “offset” when the former is large, and “zero vector” when the latter is large. It may be used as.

  In this case, it is necessary to send 1-bit side information as switching information. In addition, switching between “offset” and “zero vector” as the default value of the predicted value of the motion vector can be similarly applied to the prediction coding of the motion vector of the texture information.

  Next, a specific configuration example of the MV encoding circuit 1790 illustrated in FIG. 21 and a specific configuration example of the MV reproduction circuit 2900 illustrated in FIG. 22 will be described as a ninth specific example.

(Ninth example)
[Configuration of MV Encoding Circuit 1790]
68 is a block diagram showing a specific example of the MV encoding circuit 1790 and its auxiliary circuit in the system shown in FIG. 21, where (a) is a default value arithmetic circuit as an auxiliary circuit, and (b) is MV encoding. Circuit 1790.

  The default value calculation circuit as an auxiliary circuit shown in FIG. 68A is the object position of the current frame as viewed from the previous time point from the current processing frame and the previous processing frame for the object area. The movement is obtained as a motion vector value between frames. In FIG. 68 (a), e10 is an encoding area detection circuit, e11 is a default value determination circuit, e12 is an area information memory, and e13 is an offset calculation. A circuit, e14, is a selector.

  Further, e1 is a signal line for inputting the current frame data and corresponds to the signal line 20 of the system in FIG. 21, and receives the current frame data input from the signal line 20 as an input. belongs to. Further, e2 in FIG. 68A is a signal line for supplying the previous time point frame data held in the frame memory 1300, and the frame data of the previous time point is sent from the frame memory 1300 via this signal line e2. Receive supply. Further, e3 is a signal line for outputting flag information from the default value determination circuit e11, e4 is a signal line for supplying information on the coding area CAn from the coding area detection circuit e10, and e6 is area information. This is a signal line for supplying position information of the coding area CAn-1 read from the memory e12.

  The coding area detection circuit e10 detects the size and position information VC0 of the coding area CAn based on the image signal of the current frame Fn-1 supplied via the signal line e1. The detection result is supplied to the default value determination circuit e11, the area information memory e12, and the offset calculation circuit e13 via the signal line e4.

  The area information memory e12 is a memory for accumulating the size and position information of the coding area CAn-1, and after the encoding of the frame at time n is completed, the size and position information of the coding area CAn. Will be accumulated.

  The offset calculation circuit e13 obtains the vector value “offset” using the position information of the coding area CAn supplied via the signal line e4 and the coding area CAn−1 supplied via the signal line e6. , It is given to the selector e14.

  Further, the selector e14 receives “zero vector” which is a zero motion vector value and “offset” given from the offset calculation circuit e13, and either one of them is set as a default value determination circuit e11 via the signal line e3. Is a circuit that selects the vector value selected by the selector e14 as a default value to the selector 1793 of the MV encoding circuit 1790 via the signal line e5.

The above is the configuration of the default value arithmetic circuit.
Next, the configuration of the MV encoding circuit 1790 will be described.

  As shown in FIG. 68B, the MV encoding circuit 1790 includes an MV memory 1791, an MV prediction circuit 1792, a selector 1793, and a difference circuit 1794.

  Among these, the MV memory 1791 is a memory for storing the motion vector information from the motion vector detection circuit 1780 supplied via the signal line 1070 in FIG. 21, and the motion vector MVsn (n−) around the encoding target block. 1, 2, 3) are held in the MV memory 1790.

  The MV prediction circuit 1792 is a circuit for obtaining a prediction vector MVPs based on the motion vector MVsn (n−1, 2, 3) around the encoding target block supplied from the MV memory 1791. Here, when MVsn (n-1, 2, 3) does not exist, the prediction vector MVPs is not normally obtained, and therefore the MV prediction circuit 1792 identifies whether or not the prediction vector MVPs has been normally obtained. The identification signal is supplied to the selector 1793 through the signal line 1795.

  The selector 1793 receives the MVPs supplied from the MV prediction circuit 1792 and the default value given via the signal line e5, and selects one of them according to the signal supplied via the signal line 1795. Then, it is supplied to the difference circuit 1794.

  The difference circuit 1794 is a circuit for obtaining a motion vector prediction error signal. The difference circuit 1794 receives the motion vector information from the motion vector detection circuit 1780 supplied via the signal line 1070 and the MVPs or default value supplied via the selector 1793. And the result is output from the MV encoding circuit 1790 as motion vector information b1.

Next, the operation of the coding circuit having such a configuration will be described.
In FIG. 68A, the image signal of the frame Fn which is the image signal at the previous time point (frame data of the frame at the previous time point) stored in the frame memory 1300 is supplied to the signal line e1, and the signal line e2 is supplied to the signal line e2. Is supplied with the image signal of the frame Fn−1, which is the current image signal (frame data of the current frame).

  Then, the image signal of the frame Fn is input to the default value determining circuit e11, and the image signal of the frame Fn-1 is input to the default value determining circuit e11 and the coding area detecting circuit e10.

  In the coding area detection circuit e10, the size and position information VC0 of the coding area CAn is detected based on the image signal of the frame Fn-1, and the detection result is a default value determination circuit e11 via the signal line e4. Are supplied to the area information memory e12 and the offset calculation circuit e13.

  On the other hand, in the default value determination circuit e11, the information on the encoding area CAn from the encoding area detection circuit e10 supplied via the signal line e4 and the encoding supplied from the area information memory e12 via the signal line e6. Using the size and position information of region CAn-1, the error amount between frame Fn and frame Fn-1 is compared with the error amount between coding regions CAn and CAn-1. Then, as a result of the comparison, if the former is large, it is decided to use “offset” as the default value, otherwise use the zero vector as the default value, and use “offset” as the default value, or zero Information on a flag for identifying whether to use a vector is output via a signal line e3.

  The flag information output from the default value determination circuit e11 via the signal line e3 is shown in FIG. 47 together with the size and position information of the coding area CAn output from the coding area detection circuit e10 via the signal line e4. Are multiplexed in the layer of the coding area in the data format. Then, it is used for transmission or storage in a recording medium.

  On the other hand, the area information memory e12 is a memory for storing the size and position information of the coding area CAn-1, and after the encoding at time n is completed, the size and position information of the coding area CAn is stored. .

  The offset calculation circuit e13 obtains the vector value “offset” using the position information of the coding area CAn supplied via the signal line e4 and the coding area CAn−1 supplied via the signal line e6. Is supplied to the selector e14.

  In the selector e14, either “offset” or zero vector is selected according to the flag supplied from the default value determination circuit e11 via the signal line e3, and this is set as the default value. This default value is output to the selector 1793 of the MV encoding circuit 1790 via the signal line e5.

  The motion vector MVs of the block to be encoded is supplied to the MV encoding circuit 1790 in FIG. 68B via the signal line 1070 and is supplied to the MV memory 1791 and the difference circuit 1794.

  The MV prediction circuit 1792 is supplied with the motion vector MVsn (n−1, 2, 3) around the encoding target block from the MV memory 1791, and the prediction vector MVPs is obtained. Here, when MVsn (n-1, 2, 3) does not exist, the prediction vector MVPs is not normally obtained. Therefore, a signal for identifying whether or not the prediction vector MVPs has been normally obtained is generated. Is supplied to the selector 1793 through the signal line 1795.

  The selector 1793 selects MVPs supplied from the MV prediction circuit 1792 or a default value output from the signal line e5 in accordance with the identification signal supplied via the signal line 1795, and supplies the selected value to the difference circuit 1794. The

  The difference circuit 1794 calculates a motion vector prediction error signal, and the calculation result is output from the MV encoding circuit 1790 as motion vector information b1.

  The above is the processing content of motion vector encoding. Next, the decoding process of the motion vector encoded in this way will be described.

[Decoding circuit]
69 is a block diagram showing a specific example of a decoding circuit for realizing the present invention, and is a block diagram showing a main configuration for reproducing MV-encoded data. It is a block diagram which shows the specific example of the MV reproduction | regeneration circuit 2900 and its incidental circuit in the system shown in FIG. In FIG. 69, (a) is a default value arithmetic circuit as an auxiliary circuit, and (b) is an MV reproduction circuit 2900.

  The default value arithmetic circuit as an auxiliary circuit shown in FIG. 69 (a) includes an area information memory d10, an offset calculation circuit d11, and a selector d12. Further, d1 is a signal line for giving information on a flag for identifying a default value selection included in an upper layer of encoded data transmitted or stored in a storage medium and read out, and d2 is a code. These are signal lines that give position information of the area CAn included in the upper layer of the data that has been converted and transmitted, and these correspond to e3 and e4 on the encoding circuit side.

  Further, d3 is a signal line that gives position information of the area CAn-1, and d4 is a signal line that outputs a default value.

  The area information memory d10 is a memory for accumulating position information of the area CAn-1, and the offset calculation circuit d11 is supplied via the position information of the area CAn supplied via the signal line d2 and the signal line d3. The vector value “offset” is obtained using the position information of the area CAn−1, and the obtained vector value “offset” is supplied to the selector d12.

  In addition, the selector d12 uses one of the zero motion vector value given in advance and the vector value “offset” given from the offset calculation circuit d11 as a default value selection identification flag given via the signal line e1. The output of the selector d12 is output to the signal line d4 as a default value and is supplied to the MV reproduction circuit 2900.

The above is the configuration of the default value arithmetic circuit on the decoding side.
Next, the configuration of the MV playback circuit 2900 will be described.

As shown in FIG. 69B, the MV reproduction circuit 2900 includes an adder circuit 2901, a selector 2902, an MV prediction circuit 2903, and an MV memory 2904.
Among these, the addition circuit 2901 is a circuit that receives the motion vector information b1 which is a prediction error signal of the motion vector of the decoding target block and the default value given through the selector 2902, and adds and outputs them. The added output is output to the MV memory 2904 and the signal line 2030 in the configuration of FIG.

  The MV memory 2904 is for holding the addition output of the addition circuit 2910 and supplying a motion vector MVsn (n = 1, 2, 3) around the block to be decoded. The circuit 2903 obtains a prediction vector MVPs from the motion vectors MVsn (n = 1, 2, 3) around the decoding target block supplied from the MV memory 2904 and supplies the prediction vector MVPs to the selector 2902. When MVsn (n = 1, 2, 3) does not exist, the prediction vector MVPs is not normally obtained, so the MV prediction circuit 2903 identifies whether or not the prediction vector MVPs has been normally obtained. A function for generating a signal is provided, and this identification signal is supplied to the selector 2902 through a signal line 2905.

  The selector 2902 receives the default value given via the signal line e4 and the prediction vector MVPs given from the MV prediction circuit 2903, and selects one of them according to the identification signal supplied via the signal line 2905. This is a circuit to be given to the adding circuit 2901.

Next, the operation of the decoding side system having such a configuration will be described.
In FIG. 69A, a flag for identifying whether “offset” or zero vector is used as a default value is supplied to the signal line d1, and position information of the region CAn is supplied to the signal line d2.

  The position information of the area CAn supplied via the signal line d2 is supplied to the area information memory d10 and the offset calculation circuit d11. The area information memory d10 is a memory that accumulates position information of the area CAn-1, and accumulates position information of the area CAn after decoding at time n.

  In the offset calculation circuit d11, the vector value “offset” is obtained using the position information of the area CAn supplied via the signal line d2 and the position information of the area CAn−1 supplied via the signal line d3. Thereafter, the vector value “offset” is supplied to the selector d12.

  In the selector e12, either “offset” or “zero vector” is selected in accordance with the flag supplied via the signal line d1, and the default value is output to the MV regeneration circuit 2900 via the signal line e4. To do.

  Next, in the MV reproduction circuit 2900 of FIG. 69B, “motion vector information b1”, which is a prediction error signal of the motion vector of the decoding target block, is supplied to the addition circuit 2901.

  Also, the motion vector MVsn (n = 1, 2, 3) around the decoding target block is supplied from the MV memory 2904 to the MV prediction circuit 2903, and the prediction vector MVPs is obtained. Here, when MVsn (n = 1, 2, 3) does not exist, the prediction vector MVPs is not normally obtained. Therefore, a signal for identifying whether or not the prediction vector MVPs has been normally obtained is displayed on the signal line 2905. To the selector 2902.

  The selector 2902 selects either MVPs supplied from the MV prediction circuit 2903 or a default value output from the signal line d4 in accordance with the identification signal supplied via the signal line 2905, and sends it to the adder circuit 2901. Supplied.

  The addition circuit 2901 adds the motion vector prediction error signal (“motion vector information b1”) and the prediction signal MVPs, thereby reproducing the motion vector MVs of the decoding target block. The motion vector MVs of the block to be decoded is output from the MV playback circuit 2900 via the signal line 2030 and stored in the MV memory 2904.

  In this manner, the necessary MV encoding process with the configuration shown in FIG. 21 and the necessary MV reproduction process with the configuration shown in FIG. 22 can be realized.

  Although various embodiments have been described above, according to the present invention, it is possible to efficiently encode and decode the alpha map information, which is sub-image information representing the shape of the object, the position in the screen, and the like. Thus, an image encoding device and a decoding device that can perform the above are obtained.

  In addition, according to the present invention, since the amount of alpha map code can be reduced, it is possible to separate each object separately without significant reduction in coding efficiency as compared with the conventional coding method in which coding is performed in units of frames. It becomes possible to encode.

  The present invention is not limited to the specific examples described above, and can be implemented with various modifications.

1 is a block diagram showing an overall schematic configuration of an encoding apparatus as a premise to which the present invention is applied. The block diagram which shows the schematic structure of the whole decoding apparatus used as the premise to which this invention is applied. The block diagram which shows the structure of the alpha map encoding circuit in the encoding apparatus used as the premise to which this invention is applied. The block diagram which shows the structure of the alpha map decoding circuit used as the premise to which this invention is applied. The block diagram which shows one specific example of the encoding circuit concerning the 1st specific example of this invention. The block diagram which shows one specific example of the decoding circuit concerning the 1st specific example of this invention. It is a figure for demonstrating the 1st specific example of this invention, Comprising: The figure which shows the example of the mode around the change pixel b1. It is a figure for demonstrating the 1st specific example of this invention, Comprising: The figure which shows the example of a reference pixel. It is a figure for demonstrating the 1st specific example of this invention, Comprising: The figure for demonstrating the determination method of a context number. The block diagram which shows the example of an encoding circuit concerning the 2nd specific example of this invention. The block diagram which shows the specific example of the decoding circuit concerning the 2nd specific example of this invention. The block diagram which shows the specific example of the encoding circuit concerning the 3rd specific example of this invention. The block diagram which shows the specific example of the decoding circuit concerning the 3rd specific example of this invention. It is a figure for demonstrating the 3rd example of this invention, Comprising: The figure explaining the detection means of the change pixel in inter-frame coding. The figure explaining scan direction switching. The figure explaining an alpha map. It is a figure for demonstrating a prior art, Comprising: The figure which shows the example of a VLC table (variable length encoding table). It is a figure for demonstrating a prior art, Comprising: The figure showing the relationship of the change pixel at the time of encoding in a block unit, and the figure showing the reference area for detecting b1 (the relationship of the change pixel of block-based encoding) A diagram showing a reference area). It is a figure for demonstrating a prior art, Comprising: The flowchart in the case of encoding MMR on a block basis. The figure which shows a mode that the screen of the alpha map used by this invention was divided | segmented into the macroblock (MB) unit by predetermined multiple pixel structure. The block diagram which shows the structural example of the alpha map encoding circuit in the 4th example of this invention. The block diagram which shows the structural example of the alpha map decoding circuit in the 4th example of this invention. The figure for demonstrating Markov model encoding. The block diagram which shows the structural example of the binary image encoding circuit and binary image decoding circuit which switch and use several binary surface image encoding methods. The block diagram which shows the structure of the alpha map encoding circuit which combined reduction / enlargement of the frame unit and reduction / enlargement of the small area unit in the 5th example of this invention. The block diagram which shows the structure of the alpha map decoding circuit which combined reduction / enlargement of the frame unit and reduction / enlargement of the small area unit in the 5th example of this invention. The figure explaining the arrangement | sequence order of the bit sequence output from the alpha map encoding circuit of this invention. The block diagram which shows the system structural example of the 6th specific example of this invention. The figure for demonstrating the prior art in the 6th specific example of this invention. The figure for demonstrating the prior art in the 6th specific example of this invention. The figure for demonstrating the prior art in the 6th specific example of this invention. The figure for demonstrating the prior art in the 6th specific example of this invention. The flowchart for demonstrating the process sequence in the 6th specific example of this invention. The flowchart for demonstrating the process sequence in the 6th specific example of this invention. The figure for demonstrating the bilinear interpolation utilized for a reduction / enlargement process. The figure for demonstrating a smoothing process (smoothing process). The figure for demonstrating another example of the smoothing filter (smoothing process filter) used by this invention. The figure for demonstrating an example of the reduction process which reduces a block (macroblock) to the size of length and width "1/2". The figure for demonstrating the example of the method of calculating | requiring the pixel value of a reduction | decrease block. The figure for demonstrating the example of the reduction process by pixel thinning. The figure for demonstrating the example of the expansion process used by this invention. The figure for demonstrating another example of the reduction / enlargement process for every block. The block diagram which shows the structural example of the encoding circuit in the 6th specific example of this invention. The figure for demonstrating the example of mode information MD of each macroblock of the encoding area CA in the frame images Fn-1 and Fn in each of the time n-1 and the time n, and each frame Fn-1. The figure for demonstrating the mode of the change of the position of the block corresponding to the change of an encoding area | region, and mode information. The figure for demonstrating the example of the mode information MD of each macroblock of the encoding area CA in the frame images Fn-1 and Fn and the each frame Fn-1 and Fn in the time n-1 and the time n of this invention. The figure for demonstrating the encoding data structure in the moving image encoder which uses an alpha map together. The figure for demonstrating the coping method of the label undetermined part produced when the object part which an encoding area occupies is a part of frame. The block diagram which shows the structural example of the decoding apparatus of this invention made to use a label for prediction. The flowchart which shows the example of the encoding process procedure used with the encoding apparatus of this invention. The flowchart which shows another example of the encoding process procedure used with the encoding apparatus of this invention. The figure showing the example of the resolution for every flame | frame. The block diagram which shows the structural example of an encoding apparatus at the time of expressing explicitly including the frame memory required for the encoding apparatus of FIG. The block diagram which shows the structural example of a decoding apparatus at the time of explicitly including the frame memory required for the decoding apparatus of FIG. The block diagram which shows another structural example of an encoding apparatus at the time of expressing explicitly including the frame memory required for the encoding apparatus of FIG. The block diagram which shows the structural example of another decoding apparatus at the time of explicitly including the frame memory required for the decoding apparatus of FIG. The block diagram which shows the structural example of the frame memory used with the encoding apparatus and decoding apparatus of this invention. The figure for demonstrating the example of the process expanded to 2 times both horizontal and vertical by bilinear interpolation. The figure for demonstrating the processing content expanded twice horizontally and vertically by the bilinear interpolation process used by this invention. The figure for demonstrating the example of the reduction process used by this invention. The figure for demonstrating the reduction process used by this invention. The figure for demonstrating the label prediction method used by this invention. The figure for demonstrating the process which decompress | restores a flame | frame by the expansion process from the reduction | decrease frame used by this invention. The figure explaining the utilization range of the interpolation pixel position and reference pixel in the expansion process applied by this invention. The figure explaining the example of the addition process of the reference pixel in the expansion process applied by this invention. The figure for demonstrating an example of the prediction encoding of the motion vector used by this invention. The figure for demonstrating the inconvenience of the prediction of a motion vector in case the motion of the position of the object in a flame | frame is large. The block diagram which shows the specific structural example of the MV encoding circuit 1790 and its peripheral circuit in this invention system. The block diagram which shows the specific structural example of the MV reproduction circuit 2900 and its peripheral circuit in this invention system.

Explanation of symbols

DESCRIPTION OF SYMBOLS 2 ... a1 detection circuit 3,14 ... Memory 5 ... Mode determination circuit 8 ... Encoding circuit 1-10 ... Table determination circuit 13 ... Decoding circuit 16 ... Table determination circuit 17, 26 ... Table 19 ... a1 reproduction circuit 22, 27 ... Counter 23, 28 ... Huffman table generation circuit 100 ... Difference circuit 110, 350 ... Motion compensation prediction circuit 120 ... Orthogonal transformation circuit 130 ... Quantization circuit 140 ... Variable length encoding circuit 150, 320 ... Inverse quantization circuit 160, 330 ... inverse orthogonal transform circuit 170, 340 ... adder circuit 180, 240 ... multiplexing circuit 200 ... alpha map encoding circuit 210, 230, 260, 420 ... resolution conversion circuit 220 ... binary image encoding circuit 300, 430, 520 ... Separation circuit 310... Variable length decoding circuit 400... Alpha map decoding circuit 410. Decoding circuit 502 ... Object region detection circuit 504 ... Blocking circuit 506 ... Labeling circuit 508 ... Label encoding circuit 509 ... Label memory 510 ... Reference block determination circuit 512 ... Prediction circuit 516 ... Label decoding circuit 517 ... Label memory 518 Reference block determination circuit 520 Prediction circuit 1100 Mode determination circuit 1110 CR (reduction / enlargement ratio) determination circuit 1200, 2300 Selector 1300 Frame memory 1400, 1500 ... In-block pixel value setting circuit 1600 Motion compensation prediction circuit DESCRIPTION OF SYMBOLS 1700 ... Binary image encoding circuit 1710, 1730, 1740, 2830, 2840 ... Reduction circuit 1720 ... Expansion circuit 1750, 1760, 2820, 2850 ... Transposition circuit 1770 ... Scan type (ST) determination circuit 1780 ... Motion Vector detection circuit (MVE)
1791 ... MV memory 1792 ... MV prediction circuit 1793 ... selector 1794 ... difference circuit 1800 ... VLC (variable length coding) / multiplexing circuit 1900 ... motion vector reproduction circuit 2100 ... VLC (variable length coding) / separation circuit 2400, 2500 ... Pixel value setting circuit in macroblock 2600 ... Motion compensation prediction circuit 2700 ... Frame memory 2800 ... Binary image decoding circuit 2810 ... Enlargement circuit 2800 ... Binary image decoding circuit 2901 ... Addition circuit 2902 ... Selector 2903 ... MV prediction Circuit 2904 ... MV memory 8100 ... Storage medium 8200 ... Decoder 8300 ... Image information output device e10 ... Encoding area detection circuit e11 ... Default value determination circuit e12 ... Area information memory e13 ... Offset calculation circuit e14 ... Selector d10 ... Area information memory d11: Offset calculation circuit d12: Selector.

Claims (4)

In a moving image encoding apparatus that encodes a plurality of frames of moving image signals obtained as time-series data in units of arbitrary shapes,
Means for dividing the rectangular region including the object into blocks each composed of M × N pixels (M: number of pixels in the horizontal direction, N: number of pixels in the vertical direction);
Reduction processing means for reducing each block to 1 / 2N (N = 1, 2, 3,...) In both the horizontal and vertical directions for an alpha map signal that is a binary image representing the shape of the object;
A memory for holding neighborhood reproduction values of the block;
Means for obtaining a reference pixel value by reducing the reproduction value held in the memory to 1 / 2N;
An enlargement processing means for enlarging to the original size by repeating the process of enlarging twice in the horizontal and vertical directions N times;
Configured with
The enlarging processing means includes an enlarging circuit for enlarging the reduced block so as to always use the reference pixel value reduced to 1 / 2N, and the encoding process is performed for rate control in encoding. In an image encoding device that selectively uses a block subjected to enlargement and a block subjected to enlargement processing,
The expansion circuit is
A memory for holding neighborhood reproduction values of the block;
Means for obtaining a reference pixel value by reducing the reproduction value held in the memory to 1 / 2N according to the reduction ratio of the block;
An enlargement processing means for enlarging to the original size by repeating the process of enlarging twice in the horizontal and vertical directions N times;
Configured with
An image encoding apparatus characterized in that the enlargement processing means always uses a reference pixel value reduced to 1 / 2N. An enlargement circuit for enlarging a block of a binary image reduced to 1 / 2N (N = 1, 2, 3,...) Both horizontally and vertically,
A memory for holding a reproduction value near the block;
Means for obtaining a reference pixel value by reducing the reproduction value held in the memory to 1 / 2N according to the reduction ratio of the block;
Enlarging to the original size by repeating the process of enlarging both horizontally and vertically N times N times, and in this enlarging process, an enlarging processing means that always uses a reference pixel value reduced to 1 / 2N;
An image encoding apparatus comprising: A moving image encoding apparatus that encodes a plurality of frames of moving image signals obtained as time-series data for each object having an arbitrary shape, wherein a rectangular region including the object is represented by M × N pixels (M: number of pixels in the horizontal direction, N: the number of pixels in the vertical direction), and an image encoding device that sequentially encodes the inside of the rectangular area according to a certain rule for each of the blocks obtained by the division. And
Setting means for setting an area represented by a multiple of the block size including the object in the frame, a dividing means for dividing the area set by the setting means into blocks, and a motion in the divided block In an image coding apparatus having means for predictively coding a motion vector necessary for compensated prediction,
A memory that holds a first position vector that represents the position in the frame of the region in the reference frame;
Encoding means for encoding a second position vector representing a position in the frame of the reference frame region;
A motion vector motion vector memory that holds a motion vector of a reproduced block near the encoding target block;
Means for predicting the motion vector of the block to be encoded using the motion vector stored in the motion vector memory,
When the motion vector used in the prediction means does not exist in the motion vector memory, a default motion vector is used as a predicted value, and the default motion vector is a difference vector between the first position vector and the second position vector. And an image encoding device characterized in that the zero vector is switched and used. A moving image decoding apparatus that decodes a plurality of frames of moving image signals obtained as time series data for each object having an arbitrary shape, and includes M × N pixels (M: number of pixels in the horizontal direction, N: vertical) An image decoding apparatus that sequentially decodes a rectangular area according to a certain rule for each block composed of the number of pixels in the direction), and is represented by a multiple of the block size including the object in the frame In an image decoding apparatus that reproduces an area for each block,
Means for decoding a motion vector encoded by predictive coding necessary for motion compensation prediction in the block;
Means for decoding a prediction-encoded motion vector required for motion compensated prediction in a reference frame;
A memory that holds a first position vector that represents the position of the region in the reference frame in the frame;
Means for decoding a second position vector representing a position within a frame of an area within the frame;
A motion vector memory that holds a motion vector of a corrected block near the decoding target block;
Predicting means for predicting a motion vector of a decoding target block using a motion vector held in the motion vector memory,
If a motion vector to be used by the prediction means does not exist in the motion vector memory, a default motion vector is used as a predicted value, and the default motion vector is a combination of the first position vector and the second position vector. An image decoding apparatus characterized in that a difference vector or a zero vector is used.

Images