JP2004320808A - Image coding apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an apparatus fore efficiently coding the shape information of an object. <P>SOLUTION: A binary image coding apparatus includes a means for dividing a rectangular region including the object at each block (BL) constituted of M×N pixels, and a means 220 for sequentially coding the BL according to a predetermined rule in the rectangular region, in a coding circuit of an alpha map displaying the shape of the object in a dynamic image coding apparatus for coding a plurality of frames of dynamic image signals obtained as time series data at each object of an arbitrary shape. The relative address coding is applied to the entirely or the part of the BL. This binary image coding apparatus includes a means for storing the regenerated value near the BL, a regenerated signal holding means FM for storing the regenerated signal of already coded frame (FM), a predicting means 250 for generating a motion compensation predicted value using the regenerated signal in the holding means, and a means for detecting a changed pixel including the regenerated value near the BL. The referred changed pixel of relative address coding is obtained not from the pixel value in the BL but from the motion compensation predicted signal. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to an image encoding apparatus, an image decoding apparatus, and a recording medium for recording image encoded data for encoding, transmitting, and storing an image signal with high efficiency and decoding.

  Since an image signal has an enormous amount of information, it is generally compressed and encoded when used for transmission or storage. To encode an image signal with high efficiency, an image in a frame unit is converted into a required number of pixels (for example, M × N pixels (M: the number of pixels in the horizontal direction, N: the number of pixels in the vertical direction)). The image is divided into blocks, and an orthogonal transform is performed for each of the divided blocks to separate the spatial frequency of the image into frequency components, which are obtained as transform coefficients and encoded.

  By the way, as one of the image coding, in “JYAWang et.al.“ Applying Mid-level Vision Techniques for Video Data Compression and Manipulation ”, MITMedia Lab.Tech.Report No.263, Feb.1994,” An image coding method belonging to a category called mid-level coding has been proposed.

  In this method, for example, assuming that there is an image composed of a background and a subject (hereinafter, the subject is referred to as an object) as shown in FIG. Are encoded separately.

  As described above, in order to separately encode the background (FIG. 16C) and the object (FIG. 16B), for example, binary sub-image information indicating the shape of the object and the position in the screen is used. An alpha map signal (see FIG. 16D, in which white pixels indicate the pixels of the object) is required. The background alpha map signal (FIG. 16E) is uniquely obtained from the object alpha map signal.

  By the way, as a method of efficiently encoding the alpha map signal, a binary image encoding method (for example, MMR (Modified Modified READ) encoding or the like) or a line figure encoding method (chain encoding or the like). Is used.

  In order to further reduce the code amount of the alpha map, a method of approximating the outline of the shape by polygon and smoothing it with a spline curve (J. Ostermann, "Object-based analysis-synthesis coding based on the souece model of moving rigid 3D objects ”, Signal Process.:Image Comm.Vol.6 No.2 pp.143-161,1994) and a method of approximating a curve when reducing and encoding an alpha map and enlarging it (Japanese Patent Application No. Hei 5- 297133).

  Here, an example of a method of approximating a curve when reducing, encoding, and enlarging an alpha map will be described.

<Specific Example of MMR Encoding Circuit>
(Specific example of block-based coding)
FIG. 18A shows the relationship between changed pixels when encoding is performed in block units (for example, in a block unit having an M × N pixel configuration (M: the number of pixels in the horizontal direction, N: the number of pixels in the vertical direction)). FIG. FIG. 18B is a diagram illustrating a reference area for detecting the reference change pixel b1.
In the block-based encoding, the encoding of the changed pixel may be simplified and encoded as follows. The following processing may be performed by switching the order of scanning, or may be applied to reduced blocks.

The coding of the simplified changed pixel is performed as follows.
Now, for the changed pixel ai (i = 0 to 1) and the reference changed pixel b1, the addresses (or pixel order) obtained in raster order from the upper left of the screen are abs_ai (i = 0 to 1) and abs_b1, respectively. When the line to which the changed pixel a0 belongs is denoted as a0_line, the values of a0_line, r_ai (i = 0 to 1), and r_b1 are obtained by the following equations.

a0_line = (int) ((abs_a0 + WIDTH) / WIDTH) -1
r_a0 = abs_a0-a0_line * WIDTH
r_a1 = abs_a1-a0_line * WIDTH
r_b1 = abs_b1- (a0_line-1) * WIDTH
In the above equation, * indicates multiplication, (int) (X) indicates truncation of X below the decimal point, and WIDTH indicates the number of pixels in the block in the horizontal direction.

  Then, a reproduced value is obtained by encoding the value of the relative address “r_a1−r_b1” or “r_a1−r_a0” of the changed pixel. This is the above-mentioned “simplified coding of changed pixels”.

  FIG. 19 is a flowchart showing the flow of processing when MMR encoding, which is a binary image encoding method, is performed on a block basis. Hereinafter, the encoding process will be described with reference to the flowchart. In this process, first, the position of the starting point change pixel is initialized (S501), and the pixel value at the initial position (upper left pixel of the block) is encoded with one bit (S502). The pixel b1 is detected (S503).

  Here, when the reference change pixel b1 is not detected, the vertical mode cannot be used because no change pixel exists in the reference area. Therefore, the state of the vertical pass mode is set to “TRUE” (true), and b1 is detected. In this case, since the vertical mode can be used, the state of the vertical pass mode is set to “FALSE” (false).

Thus, the setting in the initial state is completed, and the process proceeds to the processing of the encoding loop.
First, the changing pixel a1 is detected (S505), and it is determined whether or not the changing pixel a1 is detected (S506). If the changing pixel a1 is not detected, since there is no changing pixel, the encoding is performed. An encoding end code (EOMB; End of MB) indicating the end of the encoding is encoded (S507).

  Also, as a result of the determination in S506, when the changed pixel a1 is detected, the state of the vertical pass mode (Vertical pass mode) is determined (S508). Here, if the state of the vertical path mode is “TRUE”, the encoding processing of the vertical path mode (S516) is performed, and if the state of the vertical path mode is “FALSE”, b1 is detected (S509).

  Next, it is determined whether or not b1 is detected (S510). If the reference change pixel b1 is not detected, the process proceeds to the horizontal mode step (S513), and if the reference change pixel b1 is detected, Determines whether the absolute value of “r_a1−r_b1” is greater than a threshold value (VTH) (S511). As a result, if the absolute value is less than the threshold value (VTH), Proceeding to S512), if it is larger than the threshold value (VTH), proceeding to the horizontal mode step (S513).

  In the horizontal mode step (S513), the value of "r_a1-r_a0" is encoded. Here, it is determined whether the value of “r_a1−r_a0” is smaller than “WIDTH” (S514). As a result, if the value is “WIDTH” or more, the state of the vertical path mode is set to “TRUE” (true). ) (S515), and proceeds to the vertical pass mode step (S516). When the vertical pass mode step (S516) ends, the state of the vertical pass mode is set to “FALSE” (false).

  As described above, after any of the vertical mode, the horizontal mode, and the vertical pass mode is completed (after encoding up to al is completed), the position of a1 is set as a new position of a0 (S518), and the process returns to S505.

FIG. 17 is an example of a VLC table (variable length code table).
Here, when the state of the vertical path mode is “TRUE”, since only three types of codes occur: V0 (mode V0), H (horizontal mode), and EOMB (encoding process end code), VLC can be switched according to the state of the pass mode. When the state of the vertical path mode is “TRUE”, EOMB occurs only when a0 is at the upper left position (initial position) of the block. Therefore, in this case, the code in “0” in FIG. 17 is used.
JYA Wang et. Al. “Applying Mid-level Vision Techniques for Video Data Compression and Manipulation”, MIT Media Lab. Tech. Report No. 263, Feb. 1994 J. Ostermann, “Object-based analysis-synthesis coding based on the souece model of moving rigid 3D objects”, Signal Process .: Image Comm. Vol. 6 No. 2 pp. 143-161, 1994

  When encoding an image, there is a method of encoding by dividing the screen into a background and an object. In this case, in order to separate the background from the object, an alpha map signal indicating the shape of the object and the position in the screen is used. Is required. The information of the alpha map is also encoded together with the encoded information of the image into a bit stream, which is transmitted and stored.

  However, in the method of encoding by dividing the screen into the background and the object, the code amount increases due to the alpha map, compared to encoding the entire screen as in the conventional encoding method. This causes a problem, and a decrease in coding efficiency due to an increase in the code amount of the alpha map becomes a problem.

  Therefore, an object of the present invention is to provide an image code which can efficiently encode information of an alpha map, which is sub-image information representing the shape of an object, a position in a screen, and the like, and can decode the information. It is an object of the present invention to provide a chemical conversion device.

  To achieve the above object, the present invention is configured as follows.

First, in an image encoding apparatus that encodes an image together with an alpha map that is information for distinguishing between an object area and a background area of the image, and the alpha map is encoded using relative address encoding.
Means for coding a changed pixel which has already been coded as a reference changed pixel, and coding the reference changed pixel and a symbol representing the relative position of the changed pixel to be coded next using a variable length coding table; Means for holding two or more long encoding tables and switching the variable length encoding table in accordance with the already encoded alpha map pattern.

Second, a decoding device for decoding an encoded bit stream obtained by encoding by the encoding device,
Means for decoding the symbol using a variable length coding table, and means for holding two or more variable length coding tables and switching the variable length coding table according to the already decoded alpha map pattern. It is characterized by having.

  Further, the means for switching the variable length coding table switches according to a pattern near a reference change pixel.

  The present apparatus having such a configuration can perform a plurality of types of variable encoding / decoding in which a symbol specifying a position of a changing pixel is encoded using a variable length encoding table to reduce the code amount. According to the present invention, a long encoding table is prepared and the variable length encoding table is switched according to the already encoded alpha map pattern. The effect that the code amount can be further reduced is obtained.

Secondly, the present invention provides an alpha map representing the shape of an object in a moving image encoding apparatus which encodes a plurality of frames (image frames) of moving image signals obtained as time-series data for each object having an arbitrary shape. Means for dividing a rectangular area including an object into blocks each including M × N pixels (M: the number of pixels in the horizontal direction, N: the number of pixels in the vertical direction), Means for sequentially encoding the blocks obtained by the division according to a certain rule in the rectangular area, and a binary image code for applying relative address encoding to all or a part of the blocks. In the gasifier,
Using a reproduction value storage unit for storing a reproduction value in the vicinity of the block, an image holding unit (frame memory) for storing a reproduction signal of an already encoded frame (image frame), and a reproduction signal in the image holding unit (frame memory) A motion compensation prediction circuit that generates a motion compensation prediction value, and a unit that refers to the reproduction value storage unit and detects a change pixel including a reproduction value near a block. The pixels are obtained not from the pixel values in the block but from the motion compensation prediction signal.

  Further, in the alpha map decoding circuit, for each block composed of M × N pixels, a means for sequentially decoding a rectangular area including an object according to a certain rule, a means for storing a reproduction value in the vicinity of the block, An image holding means (frame memory) for storing a reproduced signal of the converted frame (image frame), a motion compensation prediction circuit for generating a motion compensation predicted value using the reproduced signal in the image holding means (frame memory), Means for detecting a changed pixel including a reproduction value in the vicinity of a block, and a reference changed pixel for relative address coding is obtained not from a pixel value in the block but from a motion compensation prediction signal. is there.

  This makes it possible to efficiently encode the information of the alpha map, which is the sub-image information indicating the shape of the object and the position in the screen, and to decode the information.

  Also, a means for storing a reproduction value near a block, an image holding means (frame memory) for storing a reproduction signal of an already encoded frame (image frame), and a reproduction signal in the image holding means (frame memory) are used. A motion compensation prediction circuit for generating a motion compensation prediction value, means for detecting a change pixel including a reproduction value in the vicinity of a block, and a reference change pixel for relative address encoding obtained from a reproduction pixel value in the block. And means for switching the reference change pixel of the relative address encoding obtained from the motion compensation prediction signal, and encodes the relative address encoded information together with the switching information.

  Further, in the alpha map decoding circuit, for each block composed of M × N pixels, a means for sequentially decoding a rectangular area including an object according to a certain rule, a means for storing a reproduction value in the vicinity of the block, An image holding means (frame memory) for storing a reproduced signal of the converted frame (image frame), a motion compensation prediction circuit for generating a motion compensation predicted value using the reproduced signal in the image holding means (frame memory), Means for detecting a changed pixel including a reproduced value in the vicinity of the block, and obtained from a reproduced pixel value in the block, a reference changed pixel of relative address encoding, and obtained from a motion compensation prediction signal, There is provided a means for switching the reference change pixel in the relative address encoding, and the reference change pixel is obtained according to the switching information.

  In this case, in the relative address encoding, whether to detect the reference change pixel b1 from the "current block" which is the block of the image currently being processed or from the "compensated block" which is the block of the image processed last time Can be processed by switching on a block basis, and the encoding side also encodes the information for this switching, and the decoding side decodes this, and converts it to the information for switching at the time of decoding processing. Based on this, it is possible to switch between detecting the reference change pixel b1 from within the “current block” and detecting from the “compensated block” on a block basis. Optimal processing can be performed based on the image content, and more efficient encoding can be performed.

The present invention also relates to a moving picture coding apparatus that codes moving picture signals of a plurality of frames obtained as time-series data for each object having an arbitrary shape, wherein a rectangular area including the object is M × N pixels (M: horizontal (The number of pixels in the vertical direction, N: the number of pixels in the vertical direction), and an image to be sequentially coded according to a certain rule in the rectangular area for each of the blocks obtained by the division. In the encoding device,
For an alpha map signal representing the shape of an object, a frame memory for storing a playback signal of the frame including a playback signal in the vicinity of the block and a playback signal of a frame for which encoding has been completed;
Means for replacing all pixel values in the block with one of two values;
A motion compensation prediction unit that generates a motion compensation prediction value using a reproduction signal of a frame that has already been encoded in the frame memory;
Means for reducing / enlarging the binary image for each block;
Means for encoding the reduction / enlargement ratio as side information;
An alpha map encoding means including a binary image encoding means for encoding a binary image reduced for each block is provided.

  Then, the alpha map encoding means reduces and enlarges the reproduced image of the block by reproducing and replacing the reproduced value of the block with any one of two values, a motion compensation predicted value, and each block. The selection is made from any of the obtained reproduction values. Therefore, encoding of the alpha map signal can be performed in a high-quality and efficient form, and high-quality image can be maintained and encoding can be performed at a high compression rate.

The moving picture decoding apparatus decodes a moving picture signal of a plurality of frames obtained as time-series data for each object having an arbitrary shape, and includes M × N pixels (M: the number of pixels in a horizontal direction, N : The number of pixels in the vertical direction).
A reproduction signal of the frame including a reproduction signal in the vicinity of the block, a frame memory for storing a reproduction signal of an encoded frame,
Means for replacing all pixel values in the block with one of two values;
A motion compensation prediction unit that generates a motion compensation prediction value using a reproduction signal of a frame that has already been encoded in the frame memory;
Means for reducing / enlarging the binary image for each block;
An alpha map decoding means comprising a binary image decoding means for decoding a binary image reduced for each block is provided.

  Then, the alpha map decoding means reduces and enlarges the reproduced image of the block, a reproduced value obtained by replacing the entire block with any one of two values, a motion compensation prediction value, and each block. The reproduction value is selected from any of the reproduction values obtained in the above. Therefore, a high-quality image can be reproduced.

Further, when encoding the alpha map for each block, a method of encoding attributes for each block, a means for setting an encoding area including an object and represented by a multiple of the block size,
Means for dividing the area into blocks;
Labeling means for assigning a unique label to each attribute to each block; storage means for holding the label for each frame;
Determining means for determining a reference block of the previous frame corresponding to the encoded block of the current frame;
Prediction means for determining a predicted value by at least the label of the previous frame and the reference block held in the storage means,
An encoding means for encoding the label information of the encoded block using the predicted value is provided.

In addition, as a decoding device that reproduces the attribute of each block of the alpha map,
Storage means for holding the decrypted label for each frame;
Determining means for determining a reference block of the previous frame corresponding to the decoded block of the current frame;
Prediction means for determining a prediction value based on at least the label of the previous frame stored in the storage means and the reference block;
A decoding unit for decoding the label information of the decoded block using the predicted value is provided.

  Thus, when the alpha map is encoded for each macroblock (for example, a divided unit image block in a case where an image is divided into a predetermined multiple pixel configuration such as 16 × 16 pixels), an attribute unique to the attribute of each block is used. By attaching a label and encoding the label and reproducing the label to reproduce the original alpha map data, efficient encoding can be performed.

Further, the present invention encodes an image together with an alpha map which is information for distinguishing between an object area and a background area of the image.When encoding an alpha map for each block, an attribute of each block is changed. An image encoding device of a scheme for encoding,
Means for setting an encoding area represented by a multiple of the block size including objects, means for dividing the area into blocks, and labeling means for assigning a unique label to each attribute to each block Memory means for holding the label or the alpha map for each frame; determining means for determining a reference block of a previous frame corresponding to an encoded block of a current frame; and a preceding frame stored in at least the storage means. , A predictor for determining a predicted value by the reference block, and an encoder for encoding the label information of the coded block using the predicted value.

  Further, storage means for holding the reduction / enlargement ratio in units of frames is provided, and the encoding unit has a variable reduction / expansion ratio of the frame in units of frames, and performs encoding corresponding to the reduction / expansion ratio. Means for determining the size of the previous frame corresponding to the coding block of the current frame by using the reduction / enlargement rate of the current frame and the reduction / enlargement rate of the previous frame obtained from the storage means. Means for determining a reference block are provided.

  Alternatively, storage means for holding the reduction / enlargement ratio in units of frames is provided, and the encoding unit has a variable reduction / expansion ratio of the frame in units of frames, and performs encoding corresponding to the reduction / expansion ratio. Means for determining a reference to a previous frame corresponding to an encoded block of the current frame by using the reduction / enlargement rate of the current frame and the reduction / enlargement rate of the previous frame obtained from the storage means. When there are a plurality of reference blocks, the prediction means includes means for determining a label having a large number of labels among the labels of the plurality of reference blocks as a predicted value.

  Alternatively, a storage unit for holding the reduction / enlargement ratio in units of frames is provided, and the encoding unit has a variable reduction / expansion ratio of the frame in units of frames, and performs encoding corresponding to the reduction / expansion ratio. Means for encoding a coded block using one variable-length coding table selected from a plurality of types according to the reduction / enlargement ratio of the previous frame, the current frame, or both. And determining the reference block of the previous frame corresponding to the encoded block of the current frame using the reduction / enlargement ratio of the current frame and the reduction / expansion ratio of the previous frame obtained from the storage unit. Means for determining.

Further, in a decoding device for reproducing an attribute of each block of the alpha map,
Storage means for holding a decoded label or alpha map for each frame; determining means for determining a reference block of the previous frame corresponding to the decoded block of the current frame; and a label of the previous frame held at least in the storage means Alternatively, there is provided a prediction means for deciding a predicted value using the alpha map and the reference block, and a decoding means for decoding the label information of the decoded block using the predicted value.

Further, further, means for changing the reduction / enlargement ratio of the frame in frame units and decoding the reduction / enlargement ratio information,
Storage means for holding the reduction / enlargement information,
The determining means has a function of determining the reference block of the previous frame corresponding to the decoded block of the current frame using the reduction / enlargement rate of the current frame and the reduction / enlargement rate of the previous frame read from the storage means. It is characterized by having.

Alternatively, a frame reduction / enlargement ratio is variable for each frame, and means for decoding the reduction / enlargement ratio information,
Storage means for holding the reduction / enlargement information,
The determining means has a function of determining the reference block of the previous frame corresponding to the decoded block of the current frame using the reduction / enlargement rate of the current frame and the reduction / enlargement rate of the previous frame read from the storage means. The prediction means is characterized in that, when there are a plurality of reference blocks, among the labels of the plurality of reference blocks, a label having a large number is used as a prediction value.

Further, as an enlargement circuit for enlarging a block of a binary image reduced to 1 / 2N (N = 1, 2, 3,...) In both the horizontal and vertical directions,
A memory for holding the reproduction value near the block, means for obtaining a reference pixel value by reducing the reproduction value stored in the memory to 1 / 2N according to the reduction ratio of the block, Means for enlarging the original size by repeating the processing for enlarging N times N times, wherein the enlarging means always uses a reference pixel value reduced to 1 / 2N.

Also, the present invention is a moving picture coding apparatus that codes moving picture signals of a plurality of frames obtained as time-series data for each object having an arbitrary shape, wherein a rectangular area including an object is represented by M × N pixels (M: horizontal pixels). Number, N: the number of pixels in the vertical direction), and an image coding apparatus that sequentially codes the rectangular area according to a certain rule for each of the blocks obtained by the division. And
Setting means for setting an area represented by a multiple of the block size including the object in the frame, dividing means for dividing the area set by the setting means into blocks, and moving in the divided block. Means for predictive coding of a motion vector required for compensated prediction,
A memory for holding a first position vector representing the position of the region in the reference frame in the frame, and an encoding means for encoding a second position vector representing the position of the region in the reference frame in the frame A motion vector memory that holds a motion vector of a reproduced block near the encoding target block, and a unit that predicts a motion vector of the encoding target block using the motion vector stored in the motion vector memory. Equipped with
If the motion vector used in the prediction means does not exist in the motion vector memory, a default motion vector is used as a predicted value, and the default motion vector is a difference between the first position vector and the second position vector. It is characterized in that the vector and the zero vector are switched and used.

The moving picture decoding apparatus decodes a moving picture signal of a plurality of frames obtained as time-series data for each object having an arbitrary shape, and includes M × N pixels (M: the number of pixels in a horizontal direction, N The number of pixels in the vertical direction is an image decoding device that sequentially decodes a rectangular area in accordance with a predetermined rule for each block, which is expressed in multiples of a block size including an object in the frame. In an image decoding apparatus for reproducing an area to be reproduced block by block,
Means for decoding a predictively encoded motion vector required for motion compensated prediction in the block, and means for decoding a predicted encoded motion vector required for motion compensated prediction in a reference frame A memory for holding a first position vector representing the position of the region in the reference frame in the frame, and a means for decoding a second position vector representing the position of the region in the frame in the frame A motion vector memory that stores a motion vector of a corrected block in the vicinity of the decoding target block, and a prediction unit that predicts a motion vector of the decoding target block using the motion vector stored in the motion vector memory. Have
If a motion vector to be used by the prediction means does not exist in the motion vector memory, a default motion vector is used as a predicted value, and the default motion vector is a first position vector and a second position vector. And a zero vector.

  According to the present invention, since the code amount of an alpha map can be reduced, encoding can be performed separately for each object without a significant decrease in encoding efficiency, as compared with a conventional encoding method in which encoding is performed in frame units. Will be able to

  Hereinafter, specific examples of the present invention will be described with reference to the drawings. First, an outline of an image encoding and image decoding apparatus to which the present invention is applied will be described.

(Image coding and image decoding apparatus to which the present invention is applied)
FIG. 1 is a block diagram of an image encoding apparatus to which the present invention is applied, which is a method of encoding by dividing the inside of a screen into a background and an object when encoding an image. As shown in FIG. 1, the image coding apparatus according to the present invention includes a difference circuit 100, a motion compensation prediction circuit (MC) 110, an orthogonal transformation circuit 120, a quantization circuit 130, a variable length coding circuit (VLC) 140, It comprises a quantization circuit (IQ) 150, an inverse orthogonal transformation circuit 160, an addition circuit 170, a multiplexing circuit 180, and an alpha map encoding circuit 200.

  The alpha map encoding circuit 200 encodes the input alpha map, outputs the encoded signal to the multiplexing circuit 180 as an alpha map signal, and decodes the alpha map signal to generate a local decoded signal. It has a function to output.

  In particular, when encoding the input alpha map, the present alpha map encoding circuit 200 performs a process of reducing the resolution at a given reduction ratio (magnification), and encodes the resolution reduced process. At the same time, it has a function of multiplexing the coded data and reduction ratio information (magnification information) and outputting the multiplexed data to the multiplexing circuit 180 as an alpha map signal. Then, as the local decoded signal, a signal obtained by performing a process of returning the resolution-reduced signal to the original resolution is used.

  The difference circuit 100 calculates a difference signal between the motion compensation prediction signal supplied from the motion compensation prediction circuit 110 and the input image signal, and the orthogonal transformation circuit 120 converts the difference signal supplied from the difference circuit 100 into According to the information of the alpha map, it is converted into orthogonal transform coefficients and output.

  The quantization circuit 130 is a circuit for quantizing the orthogonal transform coefficients obtained by the orthogonal transform circuit 120, and the variable length encoding circuit 140 encodes and outputs the output of the quantization circuit 130. The multiplexing circuit 180 multiplexes and multiplexes the signal coded by the variable length coding circuit 140 and the alpha map signal together with side information such as motion vector information, and outputs the multiplexed signal as a bit stream.

  The inverse quantization circuit 150 is for inversely quantizing the output of the quantization circuit 130, and the inverse orthogonal transform circuit 160 is for inversely orthogonally transforming the output of the inverse quantization circuit 150 based on the alpha map. The addition circuit 170 adds the output of the inverse orthogonal transform circuit 160 and the prediction signal (motion compensation prediction signal) given from the motion compensation prediction circuit 110 and outputs the result to the difference circuit 100.

  The motion compensation prediction circuit 110 has a frame memory, and has a function of operating based on a local decoded signal provided from the alpha map decoding circuit 200 and accumulating a signal of an object area and a signal of a background area. Further, the motion compensation prediction circuit 110 predicts a motion compensation value from the accumulated image of the object area and outputs the predicted value, and also predicts a motion compensation value from the accumulated image of the background area and outputs the predicted value. Having.

  The operation of the present device having such a configuration will be described. This apparatus receives an image signal and an alpha map of the image signal.

  Of these, the image signal is divided into blocks of a predetermined pixel size (for example, M × N pixels (M: the number of pixels in the horizontal direction, N: the number of pixels in the vertical direction)) for each frame, and The signals are supplied to the difference circuit 100 via the signal line 10 in the position order. Then, the difference circuit 100 calculates a difference signal between the input (image signal) and the prediction signal (the output of the motion compensation prediction signal from the object prediction circuit 110), and supplies the difference signal to the orthogonal transformation circuit 120.

  The orthogonal transform circuit 120 converts the supplied difference signal into an orthogonal transform coefficient according to the information of the alpha map supplied from the alpha map encoding circuit 200 via the signal line 40, and then supplies the orthogonal signal to the quantization circuit 130. I do. And it is quantized here. The transform coefficients obtained by the quantization by the quantization circuit 130 are encoded by the variable length encoding circuit 140 and supplied to the inverse quantization circuit 150.

  The transform coefficient supplied to the inverse quantization circuit 150 is inversely quantized here and then inversely transformed by the inverse orthogonal transform circuit 160. Then, the sum is added to the motion compensation prediction value supplied from the motion compensation prediction circuit 110 in the addition circuit 170, output as a locally decoded image, and input to the motion compensation prediction circuit 110 again.

  The locally decoded image output from the adding circuit 170 is stored in a frame memory in the motion compensation prediction circuit 110.

  On the other hand, the motion compensation prediction circuit 110 outputs the “object motion compensation prediction value” at the processing timing of the block of the object area based on the local decoded signal given from the alpha map decoding circuit 200, At the timing, “the motion compensation prediction value of the background portion” is output and given to the difference circuit 100.

  That is, in the motion compensation prediction circuit 110, whether the image signal of the block corresponding portion of the object is currently input to the difference circuit 100 from the local decoded signal of the alpha map signal, or the image signal of the block corresponding portion of the background portion is 100, the motion compensation prediction signal of the object is input during the image signal input period of the block corresponding portion of the object, and the image signal is input during the image signal input period of the block corresponding portion of the background portion. For example, a background motion compensation prediction signal is provided to the difference circuit 100.

  The difference circuit 100 calculates the difference between the input image signal and the prediction signal corresponding to the area of the image. As a result, if the input image is in the area corresponding to the object, the corresponding position of the object is determined. In the case where the difference signal from the predicted value in the above is an input image of a background area, a difference signal from the predicted value corresponding to the background position is calculated and supplied to the orthogonal transformation circuit 120.

  The orthogonal transform circuit 120 converts the supplied difference signal into an orthogonal transform coefficient according to the information of the alpha map supplied via the signal line 40, and then supplies the orthogonal transform coefficient to the quantization circuit 130. Then, the orthogonal transform coefficients are quantized by the quantization circuit 130.

  The transform coefficients quantized by the quantization circuit 130 are encoded by the variable length encoding circuit 140 and supplied to the inverse quantization circuit 150. Then, the transform coefficient supplied to the inverse quantization circuit 150 is inversely quantized here, and then inversely transformed in the inverse orthogonal transformation circuit 160 and supplied to the addition circuit 170. Then, it is added to the predicted value supplied to the adding circuit 170 via the predicted value switching circuit 500.

  The signal of the locally decoded image that is the output of the addition circuit 170 is supplied to the motion compensation prediction circuit 110. Then, in the motion compensation prediction circuit 110, whether the signal corresponding to the block of the object or the signal corresponding to the block of the background portion is currently output from the addition circuit 170 from the local decoded signal of the alpha map signal When the signal corresponding to the block of the object is being output, the frame memory for the object is output. When the signal corresponding to the background block is being output, the signal is applied to the background memory. And store it in the corresponding memory.

  As a result, only the object image is obtained in the object frame memory, and only the background image is obtained in the background memory. Thus, the motion compensation prediction circuit 110 can obtain the predicted value of the object image using the object image, and can obtain the predicted value of the background image using the image of the background portion.

  As described above, the alpha map encoding circuit 200 encodes the input alpha map, and supplies the encoded alpha map signal to the multiplexing circuit 180 via the signal line 30.

  Further, the multiplexing circuit 180 is supplied with the transform coefficient output from the variable length coding circuit 140 via a line 40. Then, the multiplexing circuit 180 multiplexes the supplied alpha map signal and the encoded value of the transform coefficient with side information such as motion vector information, and then outputs the multiplexed signal via the signal line 50 to output the main image code. Bit stream as the final output of the encoding device.

  The above is the configuration and operation of the encoding apparatus. In obtaining an error signal of an image, the current block position of the image being processed according to the alpha map is set in the object area in order to perform motion compensation prediction using the object and background images. If the current block position of the image being processed is the object region position, the prediction value obtained from the image for the object is used while determining whether the position is the position or the background region position. The difference is obtained using the predicted value obtained from the image for use.

  Then, in the prediction for the object and the background, the motion compensation prediction circuit causes the image obtained from the difference to hold the image of the corresponding area portion in accordance with the alpha map, and to use for the prediction. As a result, optimal motion compensation prediction can be performed for each of the object and the background, and high-quality image compression encoding and decoding can be performed.

  FIG. 2 is a block diagram of a decoding apparatus to which the present invention is applied. As shown in FIG. 2, the decoding device includes a demultiplexing circuit 300, a variable length decoding circuit 310, an inverse quantization circuit 320, an inverse orthogonal transform circuit 330, an addition circuit 340, a motion compensation prediction circuit 350, and alpha map decoding. And a circuit 400.

  The separation circuit 300 is a circuit that separates an input coded bit stream to obtain an alpha map signal and an image coded signal, and the like. The alpha map decoding circuit 400 is separated by the separation circuit 300. This is a circuit for decoding the alpha map signal and reproducing the alpha map.

  The variable length decoding circuit 310 decodes the coded signal of the image separated by the separation circuit 300, and the inverse quantization circuit 320 inversely quantizes the decoded signal to return to the original coefficient. The inverse orthogonal transform circuit 330 performs an inverse orthogonal transform of the coefficient according to the alpha map and returns the result to a prediction error signal, and the addition circuit 340 applies the motion compensation from the motion compensation prediction circuit 350 to the prediction error signal. The prediction value is added and output as a reproduced image signal. This reproduced image signal is the final output of the decoding device.

  The motion compensation prediction circuit 350 obtains an object image and a background image by accumulating the reproduced image signal output from the addition circuit 340 in the frame memory according to the alpha map, and obtains an object image from the accumulated image. The motion compensation prediction signal and the background motion compensation prediction are obtained.

  In the decoding device having such a configuration, the coded bit stream is supplied to the demultiplexing circuit 300 via the line 70, and is separated for each piece of information in the demultiplexing circuit 300, so that the coded bit stream is related to the alpha map signal. Codes and variable-length codes of image signals.

  Then, the code relating to the alpha map signal is supplied to the alpha map decoding circuit 400 via the signal line 80, and the variable length code of the image signal is supplied to the variable length decoding circuit 310, respectively.

  The code related to the alpha map signal is reproduced into an alpha map signal in the alpha map decoding circuit 400 and output to the inverse orthogonal transform circuit 330 and the motion compensation prediction circuit 350 via the signal line 90.

  On the other hand, the variable length decoding circuit 310 decodes the code supplied from the demultiplexing circuit 300 and supplies it to the dequantizing circuit 320, where it is dequantized. The inversely quantized transform coefficients are inversely transformed by the inverse orthogonal transform circuit 330 according to the alpha map supplied via the line 90, and supplied to the adding circuit 340. The addition circuit 340 adds the signal subjected to the inverse orthogonal transformation from the inverse orthogonal transformation circuit 330 and the motion compensation prediction signal supplied from the motion compensation prediction circuit 350 to obtain a reproduced image.

  The above is the outline of the image encoding device and the image decoding device to which the present invention is applied. The present invention relates to an alpha map encoding circuit 200 which is a component of the encoding device shown in FIG. 1 and an alpha map decoding circuit 400 which is a component of the decoding device shown in FIG. 1 shows an embodiment.

  Before describing the details of the main part of the present invention, an alpha map encoding circuit 200 as a prior art will be mentioned. FIG. 3 is a block diagram showing a configuration of an alpha map encoding circuit 200 according to the prior art (Japanese Patent Application No. 8-237053). As shown in the figure, the alpha map encoding circuit 200 in the prior art includes a resolution conversion circuit (reduction circuit) 210, an (enlargement circuit) 230, a binary image encoding circuit (block-based MMR encoder) 220, and a multiplexing circuit. 240.

  Among these, the resolution conversion circuit 210 is a conversion circuit for resolution reduction conversion, encodes an alpha map at a reduction rate according to a given enlargement rate, and the resolution conversion circuit 230 is a conversion circuit for resolution reduction / enlargement conversion. And has a function of encoding an alpha map at an enlargement ratio according to a given enlargement ratio.

  The resolution conversion circuit 230 is provided in order to return the size reduced by the resolution conversion circuit 210 to the original size. The alpha map returned to the original size by the resolution conversion circuit 230 is transmitted via the signal line 40. It becomes an alpha map local decoded signal provided to the orthogonal transform circuit 120 and the inverse orthogonal transform circuit 160.

  The binary image encoding circuit 220 encodes and outputs the resolution-reduced and converted alpha map signal output from the resolution conversion circuit 210, and the multiplexing circuit 240 outputs the binary image encoded output and the signal. It is to multiplex and output the information of the enlargement ratio given via the line 60.

  In the alpha map encoding circuit 200 having such a configuration, the alpha map input via the alpha map signal input line 20 is reduced-encoded by the resolution conversion circuit 210 at a specified enlargement ratio, and the encoded alpha map is encoded. A map signal is output via a signal line 30, and a locally decoded signal obtained by decoding the reduced-encoded alpha map signal to the original resolution by a resolution conversion circuit 230 is output via a signal line 40 to an orthogonal transform circuit. 120, and outputs the result to the inverse orthogonal transform circuit 160.

  That is, by supplying setting information of a desired reduction / enlargement ratio to the alpha map encoding circuit 200 via the signal line 60, the above-mentioned trade-off can be achieved.

  The setting information signal of the reduction / enlargement ratio supplied via the signal line 60 is supplied to the resolution conversion circuits 210 and 230 and the binary image encoding circuit 220 to control the generated code amount of the alpha map signal. It becomes. The code (setting information signal) of the reduction / enlargement rate supplied via the signal line 60 is multiplexed with the encoded alpha map signal by the multiplexing circuit 240 and output via the signal line 30. Then, it is supplied to the multiplexing circuit 180 which is the final output stage of the image encoding device as an encoded signal of the alpha map.

  Next, the alpha map decoding circuit 400 as the prior art will be described. FIG. 4 shows a specific alpha map decoding circuit 400 in the prior art (Japanese Patent Application No. 8-237053).

  As shown in the figure, the alpha map decoding circuit 400 includes a binary image decoding circuit 410 (block-based MMR decoder), a resolution conversion circuit 420, and a separation circuit 430.

  The separating circuit 430 converts the sign of the alpha map signal and the reduction / enlargement rate from the alpha map signal separated by the separating circuit 300 in the image decoding apparatus shown in FIG. 2 and input to the alpha map decoding circuit 400. The binary image decoding circuit 410 converts the code of the alpha map signal into a code of the reduction / enlargement ratio which is given separately from the separation circuit 430. Therefore, this is a circuit for returning to a binary image, and the resolution conversion circuit 420 performs resolution expansion conversion on this binary image in accordance with the sign of the reduction / enlargement ratio given separately from the separation circuit 430, and outputs it.

  In FIG. 4, the code supplied to the alpha map decoding circuit 400 via the signal line 80 is separated into the code of the alpha map signal and the code of the reduction / enlargement ratio by the separation circuit 430, and the signal line 81 and the signal Output via line 82.

  The binary image decoding circuit 410 reproduces the reduced alpha map signal from the code of the alpha map signal supplied via the signal line 81 and the code of the reduction / enlargement rate supplied via the signal line 82. , To the resolution conversion circuit 420 via the signal line 83. The resolution conversion circuit 420 expands the reduced alpha map signal to the original size from the sign of the reduction / enlargement rate supplied via the signal line 82 to reproduce the alpha map signal, and then transmits the signal via the signal line 90. Output.

  The above is the outline of the encoding circuit and the decoding circuit to which the present invention is applied.

  Next, details of the present invention applied to such an encoding circuit and a decoding circuit will be described. First, when encoding a symbol specifying the position of a changing pixel using a variable length encoding table, a variable length encoding table (VLC table) is created so that a short code is assigned to a symbol having a high frequency of appearance. Thus, when the total code amount is reduced, the code amount is further reduced by adaptively switching the variable length coding table (VLC table) according to the pixel pattern of the already coded / decoded alpha map. A specific example will be described.

(First specific example)
According to the present invention shown in the first specific example, a symbol specifying a position of a changed pixel is encoded using a variable length encoding table, and the variable length encoding table is switched according to the already encoded alpha map pattern. It is characterized by the following.

  A first specific example of the present invention will be described.

  The binary image encoding circuit 220 in FIG. 3 encodes the symbol (Mode) representing the position of the changing pixel using the variable length encoding table (VLC table) shown in FIG. For example, when the vertical pass mode is “FALSE” and “V0 (r_a1−r_b1 = 0 in vertical mode)”, “1” is output as a code. Alternatively, when “V1 (| r_a1−r_b1 | = 1 in vertical mode)”, “01s” (“0” or “1” depending on whether s: r_a1−r_b1 is positive or negative) is output, and “EOMB (encoding end) )), "0001" is output.

  On the other hand, the binary image decoding circuit 410 of FIG. 4 uses the same VLC table of FIG. 17 used for encoding and, for example, when the input code is “1”, “resets V0”. If the input code is "0001", "EOMB" is reproduced.

  In this case, if a variable length coding table (VLC table) is created so that a short code is assigned to a symbol having a high appearance frequency, the total code amount can be reduced.

  According to the present invention, the code amount is further reduced by adaptively switching a variable length coding table (VLC table) according to a pixel pattern of an already encoded / decoded alpha map.

<Configuration of Encoding Circuit of First Specific Example>
FIG. 5 is a block diagram showing a specific example of the present invention, showing the alpha map encoding circuit 200 of FIG. 1 or the binary image encoding circuit 220 of FIG. 3 in more detail.

  The alpha map signal 1 is input to an a1 detection circuit 2 and a memory 3 that holds an encoded alpha map. The a1 detection circuit 2 detects the position 4 of the changed pixel a1 described with reference to FIG. At the same time, the memory 3 sends the position 6 of the reference change pixel b1 to the mode determination circuit 5.

  The mode determination circuit 5 determines a mode according to the algorithm described with reference to FIG. 19, and sends the mode to the encoding circuit 8 as a symbol 7 to be encoded.

  From the memory 3, the pattern 9 around the encoded reference change pixel b1 is sent to the table determination circuit 1-10. In the table determination circuit 1-10, one of the plurality of variable length coding tables is selected and output.

  Here, for example, as shown in FIG. 7, if there is an edge from the upper right to the lower left above the reference change pixel b1, the same edge often extends linearly below the reference change pixel b1. , Among pixels x1, x2, and x3, there is a high probability that x1 has a1.

  Therefore, when the pattern above the reference change pixel b1 has such a pattern, a table in which a short code is assigned to VL1 (r_a1−r_b1 = −1) is used.

<A more detailed example of the table determination method>
8 and 9 show more detailed specific examples of the table determination method. Here, attention is paid to c0 to c5 in the upper two lines of the reference change pixel b1 shown in FIG. Assuming that these pixels have the same value as the reference change pixel b1 as "1" and different values as "0", "0" and "1" are arranged in the order of c0 to c5 as shown in FIG.
The binary number converted to a decimal number is called a context number.

And, corresponding to each context number, for example,
[When context number = 0]
V0 1
VL1 010
VR1 011
VL2 000010
VR2 000011
EOMB 0001
H 001
[When context number = 1]
V0 010
VL1 1
VR1 000010
VL2 011
VR2 000011
EOMB 0001
H 001
[When context number = 2]
A variable-length encoding table is prepared so as to be omitted below.

In this table,
VL1 represents r_a1−r_b1 = −1,
VL2 is r_a1-r_b1 = -2,
VR1 is r_a1−r_b1 = 1,
VR2 represents r_a1−r_b1 = 2.

  In FIG. 7, since the context number = 1, a table in which the upper VL1 can be encoded with one bit is selected.

  Returning to FIG. 5 again, the description will be continued. In the encoding circuit 8, the code 12 is determined using the selected table 11 sent from the table determination circuit 1-10. Then, the determined code 12 is output.

  FIG. 6 shows an alpha map decoding circuit 400 as a component of the decoding device shown in FIG. 2 or a binary image decoding circuit 410 as a component of the decoding device shown in FIG. 4 in more detail. FIG. This is for decoding the code 12 generated in the specific example of FIG.

  The code 12 is input to the decoding circuit 13.

  The memory 14 holds the alpha map decoded so far, and the pattern 15 around the reference change pixel b1 is sent to the table determination circuit 16.

  In the table determination circuit 16, one of the plurality of variable length coding tables is sent to the decoding circuit 13 as a selected table 17. The algorithm of the table determination is the same as that of the table determination circuit 1-10 in FIG.

  The symbol 18 is decoded by the table 17 and sent to the a1 reproducing circuit 19. The a1 reproducing circuit 19 finds the position of a1 from the symbol 18 and the position 1-20 of b1 sent from the memory 14, and reproduces the alpha map 1-21 up to a1.

  The reproduced alpha map 1-21 is output and held in the memory 14 for future decoding.

  As described above, the first specific example is for switching a plurality of predetermined variable length coding tables. FIG. 10 shows a specific example of dynamically modifying the table according to the frequency of actually generated symbols. Is shown as a second specific example.

(Second specific example)
<Configuration of Encoding Device of Second Specific Example>
In the first specific example described above, a plurality of predetermined variable length coding tables are switched. FIG. 10 shows a specific example in which the table is dynamically modified according to the frequency of actually generated symbols. . This is a configuration in which a counter 22 and a Huffman table generation circuit 23 are added to the configuration of FIG. 5 which is the configuration of the first specific example.

  The counter 22 receives the symbol 7 from the mode determination circuit 5 and the context number 24 from the table determination circuit 1-10. The counter 22 holds the number of occurrences of each symbol for each context number.

  After a certain period of time has elapsed, the number of occurrences 25 of each symbol is sent to the Huffman table generation circuit 23 for each context number.

In the Huffman table generation circuit 23, the coding table 26 is generated by Huffman coding (Fujita "Basic Information Theory", Shokodo, pp. 52-53, 1987). The table 26 is sent to the table determination circuit 1-10, and the table of the corresponding context number is replaced with the table 26. The generation and replacement of the Huffman table are performed for all context numbers.

<Configuration of Decoding Device of Second Specific Example>
FIG. 11 shows a decoding device for decoding the code generated in the specific example of FIG. The decoding device as the first specific example shown in FIG. 11 also has a configuration in which a counter 27 and a Huffman table generation circuit 28 are added to FIG.

  The operations of the counter 27 and the Huffman table generation circuit 28 are the same as those in FIG. As described above, in the first and second specific examples, the encoding / decoding in which the code amount is reduced by encoding the symbol specifying the position of the changing pixel using the variable length encoding table. In the present invention, a plurality of types of variable length coding tables are prepared, and the variable length coding tables are switched according to the already coded alpha map pattern. According to the above, the effect that the code amount of the alpha map can be further reduced can be obtained.

  Next, the reference change pixel of the relative address coding is calculated not from the pixel value in the block of M × N pixels (M: the number of pixels in the horizontal direction, N: the number of pixels in the vertical direction) but from the motion compensation prediction signal. A specific example to be obtained will be described as a third specific example.

(Third specific example)
The third specific example is characterized in that the reference change pixel of the relative address coding is obtained not only from the pixel value in the block but also from the motion compensation prediction signal.

  FIG. 12 is a block diagram illustrating an alpha map encoding circuit as a third specific example. FIG. 13 is a block diagram illustrating an alpha map decoding circuit as a third specific example.

  The alpha map encoding circuit 200 and the alpha map decoding circuit 400 according to the present invention will be described with reference to FIG. 12, FIG. 13 and FIG.

  In the third specific example, the alpha map encoding circuit 200 shown in FIG. 3 is configured as shown in FIG. 12, and the alpha map decoding circuit 400 shown in FIG. 4 is configured as shown in FIG.

  As shown in FIG. 12, the alpha map encoding circuit 200 of this specific example includes a resolution conversion circuit (reduction processing circuit) 210, a resolution conversion circuit (enlargement processing circuit) 230, and a binary image encoding circuit (block- based MMR encoder) 220, a multiplexing circuit 240, and a motion compensation prediction circuit 250 and a reduction circuit 260.

  Among them, the resolution conversion circuit 210 is a conversion circuit for resolution reduction conversion, and encodes an alpha map at a reduction rate according to a given reduction / enlargement rate setting information signal. A conversion circuit for enlargement conversion, which has a function of encoding an alpha map at an enlargement ratio according to a given enlargement ratio.

  The resolution conversion circuit 230 is provided in order to return the size reduced by the resolution conversion circuit 210 to the original size. The alpha map returned to the original size by the resolution conversion circuit 230 is transmitted via the signal line 40. It becomes an alpha map local decoded signal provided to the orthogonal transform circuit 120 and the inverse orthogonal transform circuit 160.

  The binary image encoding circuit 220 encodes the resolution-reduced and converted alpha map signal output from the resolution conversion circuit 210 into a binary image and outputs the binary image. The details will be described later. The encoding is performed by using the motion compensated prediction signal of the resolution-converted alpha map supplied from the circuit 260 through the signal line 42. The multiplexing circuit 240 multiplexes the binary image coded output and the information of the given enlargement ratio and outputs the multiplexed output.

  The configuration of the encoding circuit in the third specific example is different from the above-described configuration of the prior art (the circuit in FIG. 3) in that a motion compensation prediction circuit 250 and a resolution conversion circuit 260 for reduction processing are provided. The motion compensation prediction circuit 250 includes a frame memory for storing a reproduced image of a previously encoded frame, and has a configuration in which the reproduction signal supplied from the enlargement circuit 230 can be stored. Further, a motion vector signal (not shown) is supplied to the motion compensation prediction circuit 250, a motion compensation prediction signal is generated according to the motion vector signal, and a resolution conversion circuit 260 for reduction processing is generated via the signal line 41. It is configured to supply to.

  The resolution conversion circuit 260 for reduction processing converts the motion compensation signal supplied from the motion compensation prediction circuit 250 supplied via the signal line 41 in accordance with the reduction / enlargement ratio setting information signal supplied via the signal line 60. Then, the image data is output to the binary image encoding circuit 220 via the signal line 42.

  When configured as the binary image encoding circuit 220, the resolution-reduced and converted alpha map signal from the resolution conversion circuit 210 supplied via the signal line 21 is encoded into a binary image and output.

  In the alpha map encoding circuit 200 having such a configuration, the setting information signal of the reduction / enlargement ratio supplied through the signal line 60 is supplied to the resolution conversion circuits 210, 230, 260 and the binary image encoding circuit 220. To control the generated code amount of the alpha map signal. The code (setting information signal) of the reduction / enlargement rate supplied via the signal line 60 is multiplexed with the encoded alpha map signal by the multiplexing circuit 240 and output via the signal line 30. Then, it is supplied to the multiplexing circuit 180 shown in FIG. 1, which is the final output stage of the image encoding device, as an encoded signal of the alpha map.

  In the present apparatus, the resolution conversion circuit 210 performs reduction coding on an alpha map input via the alpha map signal input line 20 in accordance with setting information of a desired reduction / enlargement ratio provided via the signal line 60, and performs binary coding. This is given to the encoding circuit 220.

  The binary image encoding circuit 220 converts the resolution-reduced converted alpha map signal obtained from the resolution conversion circuit 210 from the resolution-reduced converted alpha map supplied by the signal line 42 from the resolution conversion circuit 260 for reduction processing. , And is supplied to the multiplexing circuit 240 and the resolution conversion circuit 230 as a binary image encoded output. Then, the multiplexing circuit 240 multiplexes the coded alpha map signal, which is the coded output of the binary image, with the information of the enlargement rate given via the signal line 60, and outputs the multiplexed signal to the signal line 30.

  On the other hand, in the resolution conversion circuit 230, the reduced / encoded alpha map signal (binary image encoded output) given from the binary image encoding circuit 220 is reduced / enlarged via the signal line 60. In accordance with the rate setting information signal, decoding is performed to the original resolution to obtain a local decoded signal, and the obtained local decoded signal is transmitted via the signal line 40 to the motion compensation prediction circuit 250 and the orthogonal transform circuit 120 of FIG. Output to the circuit 160.

  On the other hand, the motion compensation prediction circuit 250 includes a frame memory for storing a reproduced image of the previously encoded frame, and stores a reproduced signal supplied from the resolution conversion circuit 230 for enlargement processing. Then, the motion compensation prediction circuit 250 generates a motion compensation prediction signal of an alpha map according to a separately supplied motion vector signal, and supplies the motion compensation prediction signal to the resolution conversion circuit 260 for reduction processing via the signal line 41. The resolution conversion circuit 260 performs resolution reduction conversion of the supplied motion compensation prediction signal in accordance with a reduction / enlargement ratio setting information signal obtained via the signal line 60, and supplies the signal to the binary image encoding circuit 220.

  The binary image encoding circuit 220 uses the motion-compensated prediction signal of the resolution-reduced and converted alpha map supplied from the resolution conversion circuit 260 for reduction processing, and uses the resolution-reduced converted alpha obtained from the resolution conversion circuit 210. Encode the map signal.

  The above is the outline of the alpha map encoding circuit of the third specific example. The alpha map decoding circuit is as follows.

  As shown in FIG. 13, the alpha map decoding circuit 400 of this specific example includes a binary image decoding circuit 410 (block-based MMR decoder), a resolution conversion circuit (enlargement processing circuit) 420, a demultiplexing circuit 430, Further, a motion compensation prediction circuit 440 and a resolution conversion circuit (reduction processing circuit) 450 are provided.

  Among them, the separating circuit 430 is separated by the separating circuit 300 in the image decoding apparatus shown in FIG. 2 and the sign and the reduction of the alpha map signal are converted from the alpha map signal input to the alpha map decoding circuit 400. The binary image decoding circuit 410 separates the code of the alpha map signal from the code of the reduction / enlargement ratio (setting of the reduction / enlargement ratio) provided by the separation circuit 430. A signal that returns the resolution-converted alpha-map motion compensation prediction signal supplied from the resolution conversion circuit 450 for reduction processing via the signal line 92, which will be described in detail later. It is used for decryption.

  The resolution conversion circuit 420 for the enlargement process separates the binary image, which is the code of the alpha map signal from the binary image decoding circuit 410, from the demultiplexing circuit 430 and gives the code of the reduction / enlargement ratio (reduction / enlargement). This is a function of performing resolution enlargement conversion in accordance with an enlargement ratio setting information signal) and outputting the result.

  The configuration of the decoding circuit in the third specific example is different from the above-described prior art configuration (the circuit in FIG. 4) in that the decoding circuit includes a motion compensation prediction circuit 440 and a resolution conversion circuit 450 for reduction processing. is there. The motion compensation prediction circuit 440 has a frame memory for storing a reproduced image of a previously encoded frame, stores a reproduction signal supplied from the resolution conversion circuit 420 for enlargement processing, and A signal (not shown) is supplied, a motion compensation prediction signal is generated in accordance with the motion vector signal, and supplied to the resolution conversion circuit 450 for reduction processing via the signal line 91.

  The resolution conversion circuit 450 reduces the motion compensation prediction signal in accordance with the reduction / enlargement ratio setting information signal supplied via the signal line 82, and then sends the signal to the binary image decoding circuit 410 via the signal line 92. Output.

  In the alpha map decoding circuit 400 having such a configuration, the code supplied to the alpha map decoding circuit 400 via the signal line 80 is converted by the demultiplexing circuit 430 into the code of the alpha map signal and the code of the reduction / enlargement ratio. And output via signal lines 81 and 82, respectively.

  Although the details will be described later, the binary image decoding circuit 410 uses the motion compensated prediction signal of the resolution-converted alpha map supplied from the resolution conversion circuit 450 for reduction processing via the signal line 92 to generate a signal line. Decoding processing for returning to a binary image in accordance with the code of the alpha map signal supplied via 81 and the code of the reduction / enlargement ratio (reduction / enlargement ratio setting information signal) supplied via the signal line 82 Thus, the reduced alpha map signal is reproduced and supplied to the resolution conversion circuit 420 via the signal line 83.

  The resolution conversion circuit 420 enlarges the reduced alpha map signal reproduced by the binary image decoding circuit 410 to the original size based on the sign of the reduction / enlargement rate supplied via the signal line 82. After reproducing the alpha map signal, it is output via a signal line 90.

  The binary image encoding circuit 220 converts the resolution-reduced converted alpha map signal obtained from the resolution conversion circuit 210 from the resolution-reduced converted alpha map supplied by the signal line 42 from the resolution conversion circuit 260 for reduction processing. , And is supplied to the multiplexing circuit 240 and the resolution conversion circuit 230 as a binary image encoded output. Then, the multiplexing circuit 240 multiplexes the coded alpha map signal, which is the coded output of the binary image, with the information of the enlargement rate given via the signal line 60, and outputs the multiplexed signal to the signal line 30.

  On the other hand, in the resolution conversion circuit 420, the reduced / encoded alpha map signal (binary image encoded output) provided from the binary image decoding circuit 410 is reduced / enlarged via the signal line 82. It decodes to the original resolution according to the rate setting information signal to obtain a local decoded signal, and outputs the obtained local decoded signal to the motion compensation prediction circuit 440.

  On the other hand, the motion compensation prediction circuit 440 includes a frame memory for storing a reproduced image of a previously encoded frame, and stores a reproduction signal supplied from a resolution conversion circuit 420 for enlargement processing. Then, the motion compensation prediction circuit 420 generates a motion compensation prediction signal of an alpha map according to a separately supplied motion vector signal, and supplies the motion compensation prediction signal to the resolution conversion circuit 450 for reduction processing via the signal line 91. The resolution conversion circuit 450 performs resolution reduction conversion of the supplied motion compensation prediction signal in accordance with the reduction / enlargement ratio setting information signal obtained via the signal line 82, and supplies it to the binary image decoding circuit 410.

  The binary image decoding circuit 410 uses the motion compensation prediction signal of the resolution-converted alpha map given from the resolution conversion circuit 450 for reduction processing, and sets the reduction / enlargement ratio from the separation circuit 430. According to the signal, the alpha map signal from the separation circuit 430 is decoded.

  The above is the outline of the decoding circuit to which the present invention is applied.

  As already described, the configuration of the encoding circuit in the third specific example to which the present invention is applied is that the configuration of the prior art (the circuit of FIG. ), And the configuration of the decoding circuit is different from the configuration of the prior art (the circuit of FIG. 4) in that a motion compensation prediction circuit 440 and a reduction circuit 450 are provided.

  The motion compensation prediction circuit 250 or 440 includes a frame memory for storing a reproduced image of a previously encoded frame, and stores a reproduction signal supplied from the enlargement circuit 230 or 420. Further, a motion vector signal (not shown) is supplied to the motion compensation prediction circuit 250 or 440, a motion compensation prediction signal is generated according to the motion vector signal, and the motion compensation prediction signal is generated via the signal line 41 and the signal line 91. It is supplied to the reduction circuit 260 or 450.

  Here, as the motion vector signal, the motion vector signal used in the motion compensation prediction circuit 110 or 350 provided in the apparatus of FIGS. 1 and 2 may be used, or in the alpha map encoding circuit 200, By providing a motion vector detecting circuit for an alpha map, a motion vector signal for an alpha map may be obtained.

  That is, various methods for obtaining the motion vector signal to be supplied to the motion compensation prediction circuit 250 or 440 are known, and are not related to the present invention.

  In the reduction circuit 260 or 450, the motion compensation signal supplied through the signal line 41 and the signal line 91 is changed according to the reduction / enlargement ratio setting information signal supplied through the signal line 60 and the signal line 82. After the reduction, the signal is output via the signal line 42 and the signal line 92.

  When configured as the binary image encoding circuit 220, the resolution-reduced and converted alpha map signal supplied via the signal line 21 is encoded into a binary image and output.

Here, the point that the binary image encoding circuit 220 according to this example is fundamentally different from the above-described prior art is that the motion compensation prediction signal of the resolution-reduced and converted alpha map supplied via the signal line 42 is provided. In that it has a function of encoding using
This will be described in detail.

  FIG. 14 is a diagram for explaining a method of encoding using a motion compensated prediction signal, and shows one of the divided image blocks of the N × M pixel configuration in the image of the frame image unit.

  In FIG. 14, “current block” is a processing target block, which is a block of an input current processed image. “Compensated block” is a compensation block, which is a block of an image that has been processed last time.

  In the prior art which is a premise of application of the present invention, the reference change pixel b1 on the corresponding block of the alpha map corresponding to the block of the current processing target image is detected in the same "current block" as the change pixels a0 and a1. Was.

  On the other hand, in the present invention, the reference change pixel b1 is detected from within the "compensated block" which is a motion compensation prediction signal, and this point is a new concept. That is, the reference change pixel b1 on the corresponding block of the alpha map corresponding to the block of the current processing target image is detected from within the “compensated block” that is the motion compensation prediction signal.

  The present invention is the same as the prior art in that encoding and decoding are performed using relative addresses of a0, a1 and b1 only with a different means for detecting the reference change pixel b1.

  In FIG. 14, a0 is a starting-point changing pixel, and encoding has been completed up to the starting-point changing pixel a0. Also, a1 is the next change pixel after the starting point change pixel a0, and b0 is a pixel at the same position as a0 (not necessarily a change pixel) in the "compensated block". When the line to which a0 (b0) belongs is denoted as "a0-line", the reference change pixel b1 is defined as follows.

  Here, abs_X is the address of the pixel X when the block is scanned in raster order from the upper left pixel of the block. Note that the address of the upper left pixel of the block is “0”.

  abs_b0 <abs_b1, and the pixel indicated by the symbol "x" is a changed pixel. When the changed pixel x is on "a0-line", the first change of the color opposite to "a0" is performed. The pixel is set as a reference change pixel b1. If the change pixel is not on the "a0-line", the first change pixel on the line is set as a reference change pixel b1.

  FIG. 14A shows a case where the changed pixel is not on “a0_line”. In this case, the first changed pixel on the next line is “b1”.

  FIG. 14B shows a case where the changed pixel is on “a0−line”. Since this changed pixel × is not the opposite color to “a”, the changed pixel is not changed to “b1”. The first change pixel of the line is “b1”.

  The values of "a0-line", "r_ai (i = 0 to 1)" and "r_b1" are obtained by the following equations.

a0-line = (int) ((abs-a0 + WIDTH) / WIDTH) -1
r-a0 = abs-a0-a0-line * WIDTH
r-a1 = abs-a1-a0-line * WIDTH
r-b1 = abs-b1-a0-line * WIDTH
In the above equation, * indicates multiplication, (int) (X) indicates truncation of X below the decimal point, and WIDTH indicates the number of pixels in the block in the horizontal direction.

  In the present invention, since the definition of the reference change pixel b1 is different from that of the prior art, the definition of "r_b1" is also changed as in the above equation.

  The example described with reference to FIG. 14 is an example of a unit for obtaining the reference change pixel b1 from within the “compensated block”, and the detection of the reference change pixel b1 can be variously modified.

  Further, in the binary image decoding circuit, a binary image encoding circuit 220 using a motion compensated prediction signal (“compensated block”) of the resolution-converted alpha map supplied through the signal line 92 is used. The reference change pixel b1 is detected by the same procedure as that described above.

  Furthermore, whether to detect the reference change pixel b1 from within the “current block” or from within the “compensated block” can be switched, for example, in block units. At this time, the binary image encoding circuit 220 also encodes the switching information, and the binary image decoding circuit 410 also decodes the switching information. On the basis of the information, whether to detect the reference change pixel b1 from within the “current block” or from within the “compensated block” is switched, for example, in block units.

  This makes it possible to perform optimal processing based on the image content in block units, and to achieve more efficient encoding.

  Also, as in the prior art, a means for switching the scan order is provided to switch the scan order to a horizontal scan as shown in FIG. 15A, or to a vertical scan as shown in FIG. By performing the switching, the number of changed pixels is reduced, and the code amount is further reduced, which also leads to more efficient coding.

As described above, the third specific example is a code for encoding an alpha map representing the shape of an object in a moving image encoding device that encodes a plurality of frames of moving image signals obtained as time-series data for each object having an arbitrary shape. Means for dividing a rectangular area including an object into blocks each including M × N pixels (M: the number of pixels in the horizontal direction, N: the number of pixels in the vertical direction). Means for sequentially encoding the blocks in the rectangular area according to a predetermined rule, wherein the binary image encoding apparatus applies relative address encoding to all or a part of the blocks.
Means for storing a reproduction value of a block vicinity, a frame memory for storing a reproduction signal of an already encoded frame, a motion compensation prediction circuit for generating a motion compensation prediction value using the reproduction signal in the frame memory, And a means for detecting a changed pixel including the reproduced value of the reference address. The reference changed pixel for relative address coding is obtained not from the pixel value in the block but from the motion compensation prediction signal.

  Further, in the alpha map decoding circuit, for each block composed of M × N pixels, a means for sequentially decoding a rectangular area including an object according to a certain rule, a means for storing a reproduction value in the vicinity of the block, Frame memory for storing a reproduced signal of a converted frame, a motion compensation prediction circuit for generating a motion compensation predicted value using the reproduced signal in the frame memory, and means for detecting a change pixel including a reproduced value near a block And a reference change pixel for relative address encoding is obtained not from a pixel value in the block but from a motion compensation prediction signal.

  This makes it possible to efficiently encode the information of the alpha map, which is the sub-image information indicating the shape of the object and the position in the screen, and to decode the information.

  Also, a reproduction value accumulating means for storing a reproduction value near the block, a frame memory for storing a reproduction signal of an already encoded frame, and a motion compensation prediction for generating a motion compensation prediction value using the reproduction signal in the frame memory. A circuit, means for referring to the information of the reproduction value stored in the reproduction value accumulation means, and detecting a changed pixel including a reproduction value in the vicinity of the block; and a relative address encoding obtained from the reproduction pixel value in the block. And a means for switching the reference change pixel of the relative address encoding obtained from the motion compensation prediction signal, and encodes the relative address encoded information together with the switch information.

  Further, in the alpha map decoding circuit, for each block composed of M × N pixels, means for sequentially decoding a rectangular area including an object in accordance with a certain rule, and reproduction value accumulation means for storing reproduction values near the block. A frame memory for storing a reproduction signal of an already encoded frame, a motion compensation prediction circuit for generating a motion compensation prediction value using the reproduction signal in the frame memory, and a change pixel including a reproduction value near a block. Means for detecting a reference change pixel of relative address encoding obtained from a reproduced pixel value in the block and a reference change pixel of relative address encoding obtained from a motion compensation prediction signal. And a reference change pixel is obtained according to the switching information.

  In this case, in the relative address encoding, whether to detect the reference change pixel b1 from within the "current block" or from within the "compensated block" can be switched and processed on a block basis. The information for switching is also coded, and the decoding side decodes the information. At the time of decoding the coded alpha map, the reference change pixel b1 is set to “current” based on the information for switching. Whether to detect from within "block" or from within "compensated block" can be switched on a block-by-block basis, which enables optimal processing based on the image content in block units , And more efficient encoding becomes possible.

  As described above, according to the present invention, an image encoding apparatus capable of efficiently encoding information of an alpha map, which is sub-image information representing the shape of an object, a position in a screen, and the like, and capable of decoding the information. And an image decoding device.

  The above-described specific example describes an example in which the configuration of the alpha map encoding circuit 200 uses MMR (Modified Modified READ) encoding. However, the present invention is not limited to the MMR coding, and can be realized using any other binary image coding circuit. Therefore, such an example will be described below.

(Fourth specific example)
The specific configurations of the alpha map encoding circuit 200 and the alpha map decoding circuit 400 will be described with reference to FIGS. 21, 22, and 20.

  FIG. 20 is a diagram in which the screen of the alpha map is divided into macroblock (MB) units having a predetermined multiple pixel configuration of, for example, 16 × 16 pixels, and the squares indicate division boundaries. Each cell is a macroblock (MB).

  In the case of an alpha map represented by a binary value (there may be represented by a multi-value including a weight coefficient when combining objects), the shape information of the object is represented by either white or black for each pixel. Is done. Therefore, as shown in FIG. 20, the state of the contents of each macro block (MB) on the alpha map screen is “allW” (all white), “allB” (all black), and “Multi” (other). It is classified into one of three types.

  In the case of a screen as shown in FIG. 20 which is an alpha map of a person image, the background is “white” and the person part is “black”, so that the macro block (MB) including the boundary part of the object classified as “Multi” ) May be binary-coded. Also, among the blocks (MB) classified as “Multi”, when the motion compensation prediction error value is equal to or smaller than a set value (threshold), the block (MB) is copied with the motion compensation prediction value. As described above, when the mode of the macro block (MB) to be copied is expressed as “copy” and the mode of the macro block (MB) to be binary-plane image encoded is expressed as “coded”, the code of the macro block (MB) is expressed. The following four modes are available.

(1) "allW"
(2) "allB"
(3) "copy"
(4) "coded"
The encoding or decoding method in each mode will be described in the alpha map encoding circuit 200 and the alpha map decoding circuit 400.

<Configuration Example of Alpha Map Encoding Circuit in Fourth Specific Example>
FIG. 21 is a detailed configuration diagram of the alpha map encoding circuit 200. In the configuration of FIG. 21, the mode determination circuit 1100, the CR (reduction / enlargement ratio) determination circuit 1110, the selector 1200, the intra-block pixel value setting circuits 1400 and 1500, the motion compensation prediction circuit 1600, the binary image encoding circuit 1700, Reduction circuits 1710, 1730, 1740, enlargement circuit 1720, frame memory 1300, transposition circuits 1750, 1760, scan type (ST) determination circuit 1770, motion vector detection circuit (MVE) 1780, MV encoding circuit 1790, VLC (variable length) Encoding) / multiplexing circuit 1800.

  Among these, the intra-block pixel value setting circuit 1400 is a circuit that generates pixel data that makes all the pixel values in the macroblock white, and the intra-block pixel value setting circuit 1500 sets all the pixel values in the macroblock to black. This is a circuit that generates pixel data.

  The CR (reduction / enlargement ratio) determination circuit 1110 analyzes the alpha map signal supplied via the alpha map signal input line 20 and determines at what reduction / enlargement ratio the alpha map image for one frame should be processed. And outputs the result of the determination as reduction / enlargement ratio information b2. Further, the reduction circuit 1710 converts the alpha map signal supplied via the alpha map signal input line 20 into the entire frame, and the scan type (ST) determination circuit 1770 outputs the encoded output from the binary image encoding circuit 1700. The scan direction is determined based on the scan direction information b4.

  In addition, the transposition circuit 1750 transposes the position of each macro block for one frame of the alpha map signal reduced by the reduction circuit 1710 based on the scan direction information b4 output from the scan type (ST) determination circuit 1770. The transposition circuit 1760 transposes the outputs of the reduction circuits 1710, 1720, and 1730 based on the scan direction information b4 output from the scan type determination circuit 1770, and outputs the transposed data. The encoding circuit 1700 encodes and outputs the reduced alpha map signal provided through these transposition circuits 1750 and 1760.

  The enlargement circuit 1720 is a circuit that performs an enlargement process on the alpha map signal supplied via the reduction circuit 1710 at the reduction / enlargement ratio output from the CR determination circuit 1110. The motion compensation prediction circuit 1600 is stored in the frame memory 1300. A motion compensation prediction signal is generated using the reproduced image of the reference frame, and is output to the mode determination circuit 1100 and the reduction circuit 1740. The reduction circuit 1740 outputs the motion compensation prediction signal to the output of the CR determination circuit 1110. The reduction circuit 1730 is a circuit for reducing the reproduction image of the reference frame stored in the frame memory 1300 at the reduction / enlargement ratio output from the CR determination circuit 1110. is there.

  The selector 1200 also controls the reproduction signal m0 from the intra-block pixel value setting circuit 1400, the reproduction signal m1 from the intra-block pixel value setting circuit 1500, and the motion compensation from the motion compensation prediction circuit 1600 according to the classification information output from the mode determination circuit 1100. The circuit selects and outputs a required signal from the prediction signal m2 and the reproduction signal m3 from the enlargement circuit 1720. The frame memory 1300 is a memory for storing the output of the selector 1200 in frame units.

  Further, the mode determination circuit 1100 is supplied via the alpha map signal input line 20 with reference to the motion compensation prediction signal from the motion compensation prediction circuit 1600 and the encoded signal b4 from the binary image encoding circuit 1700. The alpha map signal is analyzed to determine whether the macro block is classified into “allW”, “allB”, “copy”, or “coded”, and to output the determination result as mode information b0.

  The motion vector detection circuit (MVE) 1780 detects a motion vector from an alpha map signal supplied via the alpha map signal input line 20, and the MV encoding circuit 1790 generates a motion vector detection circuit (MVE) 1780. Is encoded, and the encoding result is output as motion vector information b1. For example, when predictive coding is applied to the MV coding circuit 1970, the prediction error signal is output as motion vector information b1.

  The VLC (variable-length coding) / multiplexing circuit 1800 includes mode information b0 from the mode determining circuit 1100, motion vector information b1 from the MV coding circuit 1790, and reduction / encoding from the CR (reduction / enlargement ratio) determination circuit 1110. Receiving the enlargement ratio information b2, the scan direction information b3 from the scan type (ST) determination circuit 1770, and the binary coded information b4 from the binary image coding circuit 1700, these are variable-length coded, multiplexed, and multiplexed. 30 is output.

  In such a configuration, the alpha map signal to be encoded is supplied to the alpha map encoding circuit 200 via the alpha map signal input line 20. Then, in the alpha map encoding circuit 200 receiving this, the mode determination circuit 1100 analyzes the alpha map signal and classifies the macro block into any of “allW”, “allB”, “copy”, and “coded”. Is determined. Here, for example, the number of mismatched pixels is used as a classification evaluation / valuation standard.

Specifically, it is as follows.
The mode determination circuit 1100 first calculates the number of mismatched pixels when all the signals in the input macroblock are replaced with white, and classifies a macroblock in which this number is equal to or less than the threshold value to “allW”. Similarly, the mode determination circuit 1100 classifies the macro block as “allW” when the entire macro block is replaced with black. Similarly, a macroblock in which the number of mismatched pixels when the entire macroblock is replaced with black is equal to or less than the threshold is classified as “allB”.
Next, the mode determination circuit 1100 calculates the number of mismatched pixels with the motion compensation predicted value supplied via the signal line 1010 for a macroblock that is not classified as “allW” or “allB”. Blocks whose number is equal to or smaller than the threshold are classified as “copy”.
Here, a macroblock that is not classified as “allW”, “allB” or “copy” is classified as “coded”.

  The classification information b0 from the mode determination circuit 1100 is supplied to the selector 1200 via the signal line 1020. In the selector 1200, when the mode of the block is “allW”, the in-block pixel value setting circuit 1400 outputs the in-block pixel value. Selects the reproduced signal m0, which is all white, supplies it to the frame memory 1300 via the signal line 40, stores it in the storage area of the frame, and outputs it as the output of the alpha map encoding circuit 200.

  Similarly, when the mode of the macroblock is “allB”, the selector 1200 selects the reproduction signal m1 in which the pixel values in the macroblock are all black in the pixel value setting circuit 1500 in the macroblock. When the mode of the block is “copy”, the motion compensation prediction signal m2 generated by the motion compensation prediction circuit 1600 supplied via the signal line 1010 is selected, and the mode of the macroblock is “coded”. In this case, the reproduction signal m3 supplied through the reduction circuit 1710 and the enlargement circuit 1720 is selected, supplied to the frame memory 1300 via the signal line 40, stored in the storage area of the frame, and It is output as an output signal of the encoding circuit 200.

  The pixel value of the macroblock classified as “coded” by the mode determination circuit 1110 is reduced by the reduction circuit 1710 and then encoded by the binary image encoding circuit 1700. Here, the setting information of the reduction / enlargement ratio (CR (Conversion Ratio)) used in the reduction circuit 1710 is obtained in the CR determination circuit 1110. For example, when the reduction ratio is set to three types of “1 (no reduction)”, “1/2 (1/2 for both horizontal and vertical)”, and “1/4 (1/4 for both horizontal and vertical)”, The CR determination circuit 1110 calculates a CR (reduction / enlargement ratio) in the next step.

      (1) The number of mismatches between the reproduced signal when the macroblock is reduced to “１ ／” and the number of mismatches with the macroblock is calculated. If this number is equal to or smaller than the threshold, the reduction ratio is set to “1/1”. 4 ".

      (2) When the number of mismatches is larger than the threshold value in the above (1), the number of mismatches between the reproduction signal when the macroblock is reduced to “１ ／” and the macroblock is calculated, If the number is equal to or smaller than the threshold value, the reduction ratio is set to “１ ／”.

      (3) If the number of mismatches is larger than the threshold value in (2), the reduction rate of the macroblock is set to “1”.

  The value of CR (reduction / enlargement ratio) thus obtained is supplied to reduction circuits 1710, 1730, 1740, an enlargement circuit 1720, and a binary image encoding circuit 1700 via a signal line 1030, and VLC (Variable length encoding)-After being supplied to the multiplexing circuit 1800 and encoded, it is multiplexed with another code. In the transposition circuit 1750, the position of the reduced signal of the block supplied from the reduction circuit 1710 is transposed (the horizontal address and the vertical address are exchanged).

  As a result, it is possible to switch between performing the coding order in the horizontal scanning order and performing the coding order in the vertical direction. The transposition circuit 1760 transposes the position of each signal between the reproduced pixel value near the macroblock reduced by the reduction circuit 1730 and the motion compensation prediction signal reduced by the reduction circuit 1740.

  In the transposition circuits 1750 and 1760, whether or not to perform transposition processing is determined, for example, by performing encoding in a horizontal scan and a vertical scan in a binary image encoding circuit 1700, and outputting a code output via a signal line 1040. By supplying the conversion information b4 to the ST (Scan Type) determination circuit 1770, the ST determination circuit 1770 determines the scan direction in which the code amount is reduced.

  In the binary image encoding circuit 1700, the signal of the macroblock supplied from the transposition circuit 1750 is encoded using the reference signal supplied from the transposition circuit 1760.

  As an example of a specific binary image encoding method, the method used in the above-described third specific example can be cited, but is not limited thereto, and the binary image encoding circuit in the fourth specific example is not limited to this. At 1700, another binary image encoding can be applied.

  The blocks classified as “coded” include an “intra” coding mode using a reference pixel in a frame and an “inter” coding mode using a motion compensation prediction signal. Here, for switching between intra and inter, for example, the coding information from the binary image coding circuit 1700 supplied via the line 1040 is supplied to the mode determination circuit 1100, and the mode in which the code amount is reduced is selected. Just fine. Here, information on the selected encoding mode (intra / inter) is supplied to the binary image encoding circuit 1700 via the signal line 1050.

  The encoded information of the binary image encoding circuit 1700 encoded in the optimal mode selected by the above means is supplied to the VLC / multiplexing circuit 1800 via the signal line 1040, and together with other codes. Multiplexed. Also, information on the optimal mode is supplied from the mode determination circuit 1100 to the VLC / multiplexing circuit 1800 via the signal line 1060, and after being encoded, is multiplexed with other codes. Further, the motion vector detection circuit (MVE) 1780 detects an optimal motion vector. Here, there are various methods for detecting a motion vector, and the method for detecting a motion vector is not a main part of the present invention, and therefore will not be described here. The detected motion vector is supplied to the motion compensation prediction circuit 1600 via the signal line 1070, and after being encoded by the MV encoding circuit 1790, is supplied to the VLC / multiplexing circuit 1800 to be encoded. After that, it is multiplexed with other codes.

  The multiplexed code is output via the signal line 30. The motion compensation prediction circuit 1600 generates a motion compensation prediction signal using the reproduced image of the reference frame stored in the frame memory 1300 based on the motion vector signal supplied via the signal line 1070, and sets the signal line 1010 to The signal is output to the mode determination circuit 1100 and the reduction circuit 1740 via the output terminal.

  As a result of the above processing, it is possible to perform image encoding with good image quality and at a high compression rate.

  Next, decoding will be described.

<Configuration Example of Alpha Map Decoding Circuit>
FIG. 22 is a detailed configuration diagram of the alpha map decoding circuit 400.
As shown in the figure, the alpha map decoding circuit 400 includes a VLC (variable length coding) / separation circuit 2100, a mode reproduction circuit 2200, a selector 2300, pixel value setting circuits 2400 and 2500 in a macro block, and a motion compensation prediction circuit. 2600, a frame memory 2700, a binary image decoding circuit 2800, an enlargement circuit 2810, transposition circuits 2820 and 2850, reduction circuits 2830 and 2840, a binary image decoding circuit 2800, and a motion vector reproduction circuit 2900.

  Among these, the VLC (variable length coding) / separation circuit 2100 decodes the coded bit stream of the multiplexed and transmitted alpha map, and outputs the mode information b0, the motion vector information b1, the reduction / enlargement ratio. The mode reproduction circuit 2200 receives the separated mode information b0 and separates it into information b2, scan direction information b3, and binary image coding information b4. The mode reproducing circuit 2200 receives the separated mode information b0, and outputs “allW”, “allB”, “copy”, This is a circuit for reproducing which of the four modes “coded”.

  The binary image decoding circuit 2800 converts the binary image coded information b4 separated by the VLC (variable length coding) / separation circuit 2100 from the separated reduction / enlargement ratio information b2 and the mode reproduction circuit 2200. And a circuit for decoding and outputting a binary image using the reproduced mode information and the information of the transposition circuit 2850. The enlargement circuit 2810 performs the decoding using the information of the separated reduction / enlargement ratio information b2. The transposition circuit 2820 transposes the enlarged image according to the separated scan direction information b3, and outputs the reproduced signal as a reproduction signal m3.

  The macro-block pixel value setting circuit 2400 is a circuit that generates a reproduction signal m0 in which all the macro-block pixel values are whitened. This is a circuit for generating a reproduced signal m1 that is made black.

  The motion vector reproducing circuit 2900 is a circuit for reproducing a macroblock motion vector using the motion vector information b1 separated by the VLC (variable length coding) / separating circuit 2100, and the motion compensation prediction circuit 2600 A circuit for generating a motion-compensated predicted value m2 from a reproduced image using the reproduced reference frame in the frame memory 2700 using the reproduced motion vector. The reduction circuit 2840 uses the information of the separated reduction / enlargement ratio information b2. The reduction circuit 2830 performs reduction processing on the motion compensation prediction value m2, and performs reduction processing on the image of the reference frame stored in the frame memory 2700 using the information of the separated reduction / enlargement ratio information b2. , A transposition circuit 2850 is a scheme separated by the VLC (variable length coding) / separation circuit 2100. Based on the emission direction information b3, a circuit for outputting a reduced image permutation to the binary image decoding circuit 2800 outputs to the reduction circuit 2830,2840.

  The selector 2300 also controls the reproduction signals m0 and m1 of the intra-macroblock pixel value setting circuits 2400 and 2500, the motion compensation prediction value m2 of the motion compensation prediction circuit 2600, and the transposition circuit according to the mode information output from the mode reproduction circuit 2200. One of the reproduced signals m3 of 2820 is selected and output, and the frame memory 2700 is a memory for holding the image signal output from the selector 2300 in frame units.

  The alpha map decoding circuit 400 having such a configuration is supplied with an encoded bit stream of the alpha map via the signal line 80. This bit stream is supplied to a VLD (variable length decoding) / separation circuit 2100.

  Then, the VLD (variable length decoding) / separation circuit 2100 decodes this bit stream, and outputs the mode information b0, the motion vector information b1, the reduction / enlargement ratio information b2, the scan direction information b3, and the binary image code. Separation into the conversion information b4. These separated information are managed in macroblock units.

  Then, among the separated information, the mode information b0 is supplied to the mode reproduction loop 2200, and classifies which of the following modes the macroblock belongs to.

(1) "allW"
(2) "allB"
(3) "copy"
(4) "coded"
Here, “coded” includes the “intra” mode and the “inter” mode as described in the specific example of the encoding circuit in FIG. That is, the "intra" coding mode uses the reference pixels in the frame, and the "inter" coding mode uses the motion compensation prediction signal.

  In the selector 2300, when the mode of the macroblock is “allW” in accordance with the above-described encoding mode supplied via the signal line 2010, all the pixel values in the macroblock are set in the intra-block pixel value setting circuit 2400. The whitened reproduction signal m0 is selected, supplied to the frame memory 2700 via the signal line 90, stored in the storage area of the image frame to which the macroblock belongs in the frame memory 2700, and the alpha map decoding circuit 400 Is output as a decoded alpha map image.

  Similarly, when the mode of the macroblock is “allB”, the reproduction signal m1 in which the pixel values in the block are all black is selected in the macroblock pixel value setting circuit 2500, and the mode of the block is changed to “allB”. In the case of “copy”, the motion compensation prediction signal m2 generated by the motion compensation prediction circuit 2600 supplied via the signal line 2020 is selected. When the mode of the block is ゛ “coded”, The reproduction signal m3 supplied through the enlargement circuit 2810 and the transposition circuit 2820 is selected, supplied to the frame memory 2700 via the signal line 90, and stored in the storage area of the image memory to which the macro block belongs in the frame memory 2700. Output from the alpha map decoding circuit 400 as a decoded alpha map image. That.

  On the other hand, the separated motion vector information b1 is supplied to the motion vector reproduction circuit 2900, and the motion vector of the macro block is reproduced. Here, the reproduced motion vector is supplied to a motion compensation prediction circuit 2600 via a signal line 2030. The motion compensation prediction circuit 2600 reproduces a reference frame stored in a frame memory 2700 based on the motion vector. A motion compensation prediction value is generated from the surface image. Then, the signal is output to the reduction circuit 2840 and the selector 2300 via the signal line 2020.

  Further, the separated reduction / enlargement ratio information b2 is supplied to a reduction circuit 2830 and an enlargement circuit 2810, and is also supplied to a binary image decoding circuit 2800, and the scan direction information b3 is supplied to transposition circuits 2820 and 2850. Is done.

  Then, in the transposition circuit 2820 that has received the scan direction information b3, the position of the reproduction signal of the macro block that has been returned to the original size and supplied from the enlargement circuit 2810 is transposed (the horizontal address and the vertical direction). Addresses are swapped). Further, in the transposition circuit 2850 that has received the scan direction information b3, the position of each signal is determined by the reproduced pixel value in the vicinity of the block reduced by the reduction circuit 2830 and the motion compensation prediction signal reduced by the reduction circuit 2840. Will be transposed.

  In the binary image decoding circuit 2800 that has received the reduction / enlargement ratio information b2, the binary image coding information b4 of the macroblock separated by the separation circuit 2100 is used by using the reference signal supplied from the transposition circuit 2850. To decrypt.

  By the way, as described in the fourth specific example of Japanese Patent Application No. 8-237053 proposed by the present inventors as prior art, the outputs of the enlargement circuits 1720 and 2810 have image quality due to discontinuity in the oblique direction. Deterioration may occur, but in order to solve this problem, the enlargement circuits 1720 and 2810 may be provided with a filter for suppressing discontinuity in oblique directions.

  Note that, similarly to the binary image encoding circuit 1700, the configuration of the binary image decoding circuit 2800 to be used is not limited to the configuration example shown in the third specific example.

<Specific example of switching a plurality of binary image encoding methods>
In the above specific examples, the specific examples of the binary image encoding circuit 1700 and the binary image decoding circuit 2800 are not limited to the third specific example. Therefore, other examples will be described.

As another binary image coding method, for example, there is a Markov model coding method (see: Television Society, “Image Information Compression”, pp. 171 to 176). FIG. 23 is a view for explaining an example of the Markov model coding method. A pixel x in the figure is a target pixel to be encoded, and pixels a to f are reference pixels referred to when encoding the pixel x.
Here, it is assumed that the pixels a to f have already been coded when the pixel x is coded. Depending on the state of the reference pixels a to f, the target pixel x is adaptively a variable length code table when VLC (variable length coding) is used and a probability table when arithmetic code is used. It is switched and encoded.

  When this specific example is applied to the present invention, “intra” coding uses a pixel coded earlier than a pixel of interest in a macroblock to be currently processed as a reference pixel, and “inter” coding uses In the macro block to be processed, not only the pixel encoded before the pixel of interest but also the pixel in the motion compensation prediction error signal may be used as the reference pixel.

In this specific example, for example,
[Method 1] Method of third specific example (encoding method based on MMR)
When,
[Method 2] A method of adaptively switching and encoding the Markov model coding is provided, and a method of adaptively selecting and using these is adopted.

  FIG. 24A is a block diagram of a binary image encoding circuit 1700 used in this specific example. As shown, the binary image encoding circuit 1700 includes a pair of selectors 4110 and 4140 and a pair of binary image encoding circuits. Among these, the selector 4101 is an input side selection circuit, and the selector 4102 is an output side selection circuit. The binary image encoding circuit 4120 is a first binary image encoding circuit for encoding according to the above [method 1], and the binary image encoding circuit 4130 is a second binary image encoding circuit for encoding according to the above [method 2]. This is a value image encoding circuit.

  In the binary image encoding circuit 1700, a selector 4110 that selectively switches a signal of a processing target macroblock supplied from the transposition circuit 1750 via the signal line 4101 in accordance with a switching signal supplied via the signal line 4103. In step 4140, the signal is adaptively distributed to the first binary image encoding circuit 4120 and the second binary image encoding circuit 4130, and the divided macroblock signal is encoded.

  The encoded information is output via a signal line 4102. In FIG. 24A, signal lines 1030 and 1050 for signals supplied to the binary image encoding circuit 1700 and signal lines from the transposition circuit 1760 are omitted.

  Here, the switching information supplied via the signal line 4103 may be a preset default value, or may be determined as appropriate switching information based on the image content, separately encoded as side information, and transmitted to the decoding side. You may make it.

  This makes it possible to perform optimal processing based on the image content, and to select and use an appropriate one from a plurality of encoding methods according to the application.

  Similarly, FIG. 24B is a block diagram of a binary image decoding circuit 2800 used in this example. As shown, the binary image decoding circuit 2800 includes a pair of selectors 4210 and 4240 and a pair of binary image decoding circuits 4220 and 4230. Among these, the selector 4210 is an input side selection circuit, and the selector 4240 is an output side selection circuit. The binary image decoding circuit 4220 is a first binary image decoding circuit for decoding an image encoded by the above [method 1], and the binary image decoding circuit 4230 is a first binary image decoding circuit. 2 is a second binary image decoding circuit that decodes a converted image.

  In the binary image decoding circuit 2800 having such a configuration, the binary image encoding information b4 of the processing target macroblock to be displayed via the signal line 4201 is converted into a switching signal supplied via the signal line 4203. In response, selectors 4210 and 4240 adaptively distribute and decode to first binary image decoding circuit 4220 and second binary image decoding circuit 4230.

  The decoded signal is output via signal line 4202. In FIG. 24B, a signal line for supplying a signal from the separation circuit 2100 to the binary image decoding circuit 2800, a signal line from the mode reproduction circuit 2200, and a signal line from the transposition circuit 2850 are omitted. .

  Here, the switching information supplied via the signal line 4203 may be a default value, or may be information sent from an encoder that obtains the switching information from a coded bit stream.

  Next, a specific example of a circuit for performing reduction / enlargement will be described.

<< Specific examples related to reduction / enlargement circuits >>
[Example of enlargement processing]
According to the present invention, rate control is performed by performing reduction / enlargement processing in units of coding regions or blocks (units of macroblocks). As an example of a technique used for the reduction / enlargement processing, there is “bilinear interpolation” disclosed by the present inventors in Japanese Patent Application No. 8-237053. Let me mention this technology. The bilinear interpolation is described as shown in FIG. 35, based on the reference document “Onoe: Image Processing Handbook, p. 630, Shokodo”.

  In FIG. 35A, Pex is a pixel position after conversion, and the Pex indicates a real pixel position as shown in FIG. 35A.

  Therefore, based on the distance relationship between the input signal and the integer pixel positions A, B, C, and D, the pixel values Ia to Id of A to D are divided into Pex by the logical formula shown in FIG. Find the pixel value Ip.

  Such a process is called “bilinear interpolation”, and the pixel value Ip of Pex can be easily obtained from the pixel values Ia to Id of A to D.

  By the way, since this bilinear interpolation processing is performed using only surrounding four pixels, a change in a wide range is not reflected, and discontinuity particularly in an oblique direction is likely to occur. As an example for solving this problem, an example in which a smoothing filter process (smoothing process) is performed after an enlargement process is performed is proposed in the prior invention (Japanese Patent Application No. 8-237053).

  This will be specifically described. FIG. 36 is a diagram for explaining the smoothing process (smoothing process). Here, FIG. 36A shows a binary image of the original size, and FIG. 36B shows a binary image obtained by reducing the size. In FIG. 36, the object area is indicated by a black circle, and the background (background) area is indicated by a white circle.

  In this example, in order to smooth the discontinuity in the oblique direction caused by the sampling conversion (enlargement / reduction conversion), each pixel (white circle) in the background area is centered on each pixel. Then, the upper, lower, left and right pixels, that is, adjacent pixels are examined. If two or more of the pixels are pixels in the object area (black circles), a process of including the pixels in the background area in the object area is performed.

  That is, as in the case where the pixel to be inspected, which is one pixel in the background area, is the pixel at the position indicated by the double circle in FIG. When there is a pixel (black circle) in the area, the pixel at the position indicated by the double circle (that is, the pixel to be inspected) is set as a pixel of the black circle to be a pixel in the object area. Assuming that a pixel indicated by a black circle is “1” and a pixel indicated by a white circle is “0”, a process of replacing a pixel (pixel value “0”) at a position indicated by a double circle with a pixel value “1” is performed. . By such processing, discontinuity in the oblique direction can be eliminated.

FIG. 37 shows another example of the smoothing filter (smoothing processing filter). Assuming that the pixel at the center of the mask of 3 × 3 pixels shown in FIG. 37 is C, the upper left pixel is C, the upper right pixel is TR, the lower left pixel is BL, and the lower right pixel is BR with respect to C. , The filtered value of pixel C is determined by: Here, the value of the object is represented by “1”, and the value of the background is represented by “0”.

  That is, in this filter operation processing, if the pixel C is “0”, it is checked whether or not the value obtained by adding the pixels TL, TR, BL, and BR is larger than “2”. Is set to “1”, and if not large, the pixel C is set to “0”. If the pixel C is not “0”, is the value obtained by adding the pixels TL, TR, BL, BR smaller than “2”? It is checked whether or not the value of the pixel C is "1" if the value is small, and "0" if the value is not large.

  According to this filter, the value of the pixel C is corrected in consideration of a change in the pixel value positioned diagonally with respect to the target pixel C, so that the discontinuity in the diagonal direction is eliminated. The configuration of the smoothing filter is not limited to the above example, and a non-linear filter such as a median filter may be used.

  FIG. 58A is a diagram illustrating a process of enlarging both horizontally and vertically two times by bilinear interpolation. In FIG. 58A, the pixels of the reduced block are represented by white circles, that is, “O” marks, the pixels to be interpolated are represented by cross marks, that is, “X” marks, and each pixel value is “0”. Alternatively, it is either “1”.

In this case, when the value of the pixel (reproduced pixel) to be interpolated is obtained by the logical expression of FIG.
Ip1 = Ia, Ip2 = Ib, Ip3 = Ic, Ip4 = Id
It becomes.
Therefore, since all the pixel values of the four interpolation pixels around the pixel A are Ia, 2 × 2 pixels of the enlarged image have the same value, and the smoothness is lost. Therefore, the above problem is solved by changing the weight of the bilinear interpolation as follows.

Ip1: if (2 * Ia + Ib + Ic + Id> 2) then "1" else "0"
Ip2: if (Ia + 2 * Ib + Ic + Id> 2) then "1" else "0"
Ip3: if (Ia + Ib + 2 * Ic + Id> 2) then "1" else "0"
Ip4: if (Ia + Ib + Ic + 2 * Id> 2) then "1" else "0"
Here, Pi (i = 1, 2, 3, 4) is an interpolated pixel corresponding to the “x” position shown in FIG. 58A, and Ipi (i = 1, 2, 3, 4) ) Is the pixel value (“1” or “0”) of the pixel Pi (i = 1, 2, 3, 4). A, B, C and D are the pixels of the reduced block, and Ia, Ib, Ic and Id are the pixel values of these pixels A, B, C and D.

"Ip1: if (2 * Ia + Ib + Ic + Id> 2) then" 1 "else" 0 "" means that the pixel value Ip1 of the pixel p1 is twice the value of Ia, Ib, Ic, If the sum with Id is greater than 2, it is set to "1", otherwise it is set to "0",
"Ip2: if (Ia + 2 * Ib + Ic + Id> 2) then" 1 "else" 0 "means
The pixel value Ip2 of the pixel p2 is "1" if the sum of Ib, Ia, Ic and Id is greater than 2, and "0" otherwise. Shows,
"Ip3: if (Ia + Ib + 2 * Ic + Id> 2) then" 1 "else" 0 "means
The pixel value Ip3 of the pixel p3 is "1" if the sum of Ic and the sum of Ia, Ib and Id is greater than 2, and "0" otherwise. Shows,
"Ip4: if (Ia + Ib + Ic + 2 * Id> 2) then" 1 "else" 0 "means
The pixel value Ip4 of the pixel p4 is "1" if the sum of twice the value of Id and Ia, Ib, and Ic is 2 or more, and "0" otherwise. Is shown.
In the case where the image is enlarged four times both horizontally and vertically, the above processing may be repeated twice.

  The above is an example of performing the enlargement process by calculation, but the enlargement process can be performed without calculation. An example will be described below.

<Example of enlargement processing not based on calculation>
Here, a table is prepared and stored in a memory, and pixels are uniquely replaced according to the table.

This will be specifically described. First, for example, a memory address is set to 4 bits, and Ip1, Ip2, Ip3, Ip4 obtained by the patterns of Ia, Ib, Ic, Id are added to the address obtained by arranging Ia, Ib, Ic, Id in the following table. Beforehand.

  This example shows that when “Ia, Ib, Ic, Id” is “0, 0, 0, 0”, “Ip1, Ip2, Ip3, Ip4” is “0, 0, 0, 0”. When “Ia, Ib, Ic, Id” is “0, 0, 0, 1”, it means that “Ip1, Ip2, Ip3, Ip4” is “0, 0, 0, 0”. , Ib, Ic, Id "are" 0, 0, 1, 1 ", it means that" Ip1, Ip2, Ip3, Ip4 "is" 0, 0, 1, 1 ". When "Ic, Id" is "0, 1, 0, 1", "Ip1, Ip2, Ip3, Ip4" is "0, 1, 0, 1" and "Ia, Ib, Ic". , Id "is" 0, 1, 1, 0 ", it means that" Ip1, Ip2, Ip3, Ip4 "is" 0, 1, 1, 0 ". If the combination of the contents of Ic and Id is determined, Ip1, Ip2, Ip3 and Ip4 are unambiguous. It represents a table that is determined by.

  If such a table is set and held in the memory so that the contents of Ia, Ib, Ic, and Id are addresses and the data stored at the addresses are Ip1, Ip2, Ip3, and Ip4, when performing interpolation processing, Interpolated values can be obtained simply by inputting the addresses of Ia, Ib, Ic and Id into the memory and reading out the corresponding Ip1, Ip2, Ip3 and Ip4. Here, the sequence of Ip1, Ip2, Ip3, and Ip4 is a binary number. If a numerical value obtained by converting this into a decimal number is called a context, this method is based on the context obtained by Ip1, Ip2, Ip3, and Ip4. It can be said that this embodiment determines the interpolation values Ip1, Ip2, Ip3, and Ip4.

The context is
Context = 2 * 2 * 2 * Ia + 2 * 2 * Ib + 2 * Ic + Id
Is determined by

  In the above-described example, the number of pixels to be referred to is four in the method of obtaining an interpolated value according to a context. However, the present invention is not limited to this, and can be realized in any number of cases such as twelve described below.

  As for the arrangement of the reference pixel and the interpolation pixel, for example, as shown in FIG. 64A, nine pixels in a region surrounded by a dotted line with respect to the interpolation pixel P1 are defined as contexts (= 0 to 511). ) Is determined, and the interpolation pixel P1 is determined to be “0” or “1” depending on the context.

  For the interpolation pixels P2, P3, and P4, the dotted lines in FIGS. 64B, 64C, and 64D correspond to the respective interpolation positions and the positional relationship between the four nearest reference pixels surrounding the positions. The pixels surrounded by are defined as reference pixels.

  At this time, as shown in the figure, the reference pixels A to I are arranged such that the relative positions with respect to the interpolation pixels P are the same (for example, (b) is an arrangement obtained by rotating (a) clockwise by 90 degrees. If the same pixel pattern is simply rotated, the same context will be used, so that a memory having a table of contents common to the interpolation pixels P1 to P4 can be used.

Next, an example in which a macroblock of 4 × 4 pixels is enlarged to 16 × 16 pixels will be described.
To expand a macroblock of 4 × 4 pixels to 16 × 16 pixels, first expand a macroblock MB of 4 × 4 size to 8 × 8 size, and then expand a macroblock MB of 8 × 8 size. Is enlarged to a macroblock MB of 16 × 16 in size.

  FIG. 65A shows external reference pixels AT and AL when the macro block MB having the size of 4 × 4 is enlarged to the size of 8 × 8. FIG. 65A shows the external reference pixels AT and AL when the macro block MB having the size of 8 × 8 is enlarged. The external reference pixels BT and BL are shown in FIG.

  The external reference pixels AT are arranged in 2 rows and 8 columns, the external reference pixels AL are arranged in 4 rows and 2 columns, the external reference pixels BT are arranged in 2 rows and 12 columns, and the external reference pixels BL are arranged in 8 rows and 2 columns.

  As described with reference to FIG. 42, the values of these external reference pixels need to be obtained as the average values of the pixels at predetermined positions in the macro block that has already been reproduced. However, the binary image encoding circuit 1700 and the binary image decoding circuit 2800 refer to the external reference pixels AT and AL when encoding with a size of 4 × 4 pixels and have a size of 8 × 8 pixels. When encoding, since the external reference pixels BT and BL are referred to, the use of the external reference pixels BT and BL can avoid the waste that the external reference pixels are bothersomely obtained only for the enlargement processing.

Here, if the external reference BT and BL are described in C language, which is a language for computer programming, from the external reference pixels AT and AL without calculating the average value,

Although the result is slightly different from the case of obtaining the average value, the average value calculation can be omitted.

  In the above-described process, for example, the pixel p1 is copied to the pixel p2 and the pixel p3, and the pixel p4 is copied to the pixel p5 and the pixel p6. By doing so, BT and BL are generated. Note that “[] []” in the C language program indicates an array, and numbers in [] indicate decimal numbers.

  The specific method of the enlargement processing has been described above. Next, a specific method of the reduction processing will be described.

[Reduction processing]
FIG. 38 shows an example of a reduction process for reducing a block (macroblock) to a size of “1/2” in the vertical and horizontal directions. In this example, assuming that each area within the dotted frame is a unit reduced block area, the average of every 2 × 2 pixels (total 4 pixels indicated by “O” in each dotted frame) within the unit reduced block area The value (“×” in FIG. 38) is a pixel value in the unit reduced block area. That is, when a macroblock is reduced to a size of “1/4” in the vertical and horizontal directions, an average value for each 4 × 4 pixel in the unit reduced block area is obtained, and the average value in the unit reduced block area is calculated. That is, the pixel value may be used.

  Further, as shown in FIG. 39, when considering a certain unit reduced block area composed of pixels A, B, C, and D, the value of the pixel X of the unit reduced block area is obtained and set as the value of the unit reduced block area. Does not calculate the average value of the pixels A to D, but includes the pixels E, F, G, H, I, J, K, L, M, N, O, and P in a wider range of pixel positions. Thus, an average value of these A to P may be obtained. That is, of the pixels in the adjacent unit reduced block area, a method is adopted in which the average value of the pixels including the pixels adjacent to the pixels A to D is adopted.

The above is an example in which the average value of the existing pixel values is used as the pixel value of the unit reduced block area to reduce the number of pixels in the unit reduced block area and reduce the size of the macroblock. Was. A macroblock can be reduced by mechanical thinning-out processing without such calculation.
An example will be described below. First, a process closed in a macro block will be described.

  FIG. 40 shows an example of a reduction process by pixel thinning. A macroblock MB having a 16 × 16 pixel configuration is reduced to a block having an 8 × 8 pixel configuration as a result of performing reduction processing by pixel thinning. That is, the pixels indicated by the dotted white circles in the figure are the thinned pixels, and the pixels indicated by the solid white circles are the pixel values (“×” in FIG. 40) of the unit reduced block area. . In this case, the reduction ratio (CR) indicates the ratio of pixels to be thinned. The method of pixel thinning is not limited to the method described with reference to FIG. 40. For example, pixels may be thinned in a quincunx lattice. In this case, the enlargement processing corresponds to interpolating the thinned pixels.

  The specific methods of various reduction processes have been described above. Next, the enlargement processing will be described.

  FIG. 41 is a diagram illustrating the enlarging process. In the figure, a solid rectangular frame indicates a macroblock, and each cell in a portion indicated by a dotted frame indicates a unit reduced block area. White circles in the unit reduced block area represent existing pixels, and the number of pixels is increased by interpolation to enlarge the macro block. Pixels to be interpolated are represented by “x”, and after interpolation, “O” pixels are unnecessary pixels.

When interpolating pixels at the macroblock boundary, pixel values outside the macroblock are required. In this case, as shown by the arrow in the figure, the closest pixel value in the macro block may be assigned.
That is, when performing pixel interpolation of a certain unit reduced block area, a total of 9 pixel values of the pixel values of the surrounding blocks, which are the unit reduced block area and the unit reduced block area adjacent thereto, are necessary for the processing. However, if a certain unit reduced block area in a macroblock is located at a macroblock boundary, some of the surrounding eight blocks belong to a macroblock that does not belong to the own block. This means that it is necessary to separately obtain a pixel value outside the macro block belonging to itself. In this case, as indicated by the arrow in the figure, the nearest pixel value in the macroblock to which the own device belongs is assigned, and the pixel value in the neighboring unit reduced block area necessary for the processing is conveniently assigned. , Just use.

Note that the reduction / enlargement processing for each macroblock does not need to be closed within the macroblock, and as shown in FIG. 42, the reproduction values (blocks adjacent to the left, upper, upper left, upper right May be used.
This will be described specifically.
In FIG. 42, a solid rectangular frame indicates a certain block (macro block), and “x” marks indicate pixels of an image (standard magnification image) at a magnification of 1 ×. A macro block usually has a 16 × 16 pixel configuration. When a frame is compressed to １ ／, the pixels in the block have an 8 × 8 pixel configuration, and the previous 16 × 16 pixel configuration has a size of 2 × 2 pixels. Is represented by one pixel, the "O" mark in FIG. 42 represents an image of 2 × 2 pixels in this case in which the information of the representative point is represented by one pixel. The inside of the frame shown by the dotted line is the unit reduced block area, which is a four-pixel configuration (2 × 2) area in the standard magnification image. It is represented by one pixel.

When restoring the 1/2 reduced image to the original image size (restoring to the standard magnification image), the area of one pixel of the 1/2 reduced image is returned to the 4-pixel configuration, but is not closed within the macroblock. To do this by interpolation:
For example, in FIG. 42, consider a case where a pixel 1 or a pixel 2 in a certain unit reduced block area (an area within a dotted frame) of a certain macroblock is reproduced by interpolation. In this case, what exists at the time of the processing is the pixel information marked with “O”. Therefore, the existing pixels around the pixel 1 or the pixel 2 to be interpolated are the pixels 3, 4, 5, and 6, so that bilinear interpolation is performed using these pixels 3, 4, 5, and 6. However, “pixel 3” and “pixel 4” are pixels belonging to an adjacent block (adjacent macro block), and are pixels before enlargement (pixels of a 縮小 reduced image). Since this is a block at the position processed at the previous point in time, it is unnecessary data after this macroblock enlargement process, so it may have already been discarded from the viewpoint of saving memory resources. is there.

  In the case of such a system, for example, in the case of "pixel 3", the average value of pixels 7, 8, 9, and 10, which are the existing neighboring pixels in the macro block, is obtained, and the One of the methods is to use the value of “pixel 3” that has been obtained. However, if it is desired to reduce the calculation for calculating the average value, the number of pixels among pixels 7, 8, 9, and 10 out of a total of four pixels is used. The average value of the pixels 9 and 10 that are close to the pixels 1 and 2 to be inserted may be used as the value of the discarded “pixel 3”.

  Further, if the pixel 10 is diverted to the value of “pixel 3”, the calculation can be further omitted. Similarly, for “pixel 4”, a pixel value near the same is substituted. In a similar case, the interpolation of “pixel 11” includes, for example, the average value of pixels 14 and 15 instead of “pixel 12”, the pixel 16 instead of “pixel 13”, and the pixel 18 instead of pixel 18. The pixel 17 is used.

Usually, the frame image Pf is divided into blocks (macroblocks) in a minimum rectangular range mainly including an object portion, for example, the coding area CA shown in FIG. In some cases, blocks adjacent to the left, upper, upper left, and upper right sides of the block may be located outside the coding area CA.
In this case, as shown in FIG. 42, even when the reproduced pixel value near the block (the value of the reproduced pixel) is used, if the macroblock is located outside the coding area CA, the reproduced binary value is not changed. Without reference, the nearest pixel value in the macro block belonging to itself may be assigned and used for convenience as shown in FIG.

  Further, when data is transmitted / received on a transmission line affected by an error, in order to make the effect of the error harder, a unit smaller than the above-mentioned coding area CA (this is called a “synchronization recovery unit”) is used. The encoding process may be closed.

  As a result, the influence of the error can be cut off in this “synchronization recovery unit”, and the influence of the error is reduced. Here, the “synchronization recovery unit” refers to each area surrounded by a dotted line indicated by the reference symbol Un in FIG. 59B. The “synchronization recovery unit” is an area obtained by dividing the coding area CA into small pieces, but is still composed of a plurality of macroblocks.

  In this method, since the encoding process is closed within the "synchronization recovery unit", even if the data used in a certain "synchronization recovery unit" includes a transmission error, the errored data is referred to. The processing method is performed only within a certain "synchronization recovery unit", and the adjacent "synchronization recovery unit" does not process by referring to the erroneous data. That is,

  Even in this case, as shown in FIG. 42, when using the reproduction pixel values near the block, the reproduction pixel values of the macro blocks belonging to the synchronization recovery unit other than the “synchronization recovery unit” including the block are not referred to. , The closest pixel value in the macroblock is assigned as shown in FIG.

  As described above, "whether to refer to the outside of the macroblock or the synchronization recovery unit or not" is determined by preparing a switch bit for the code and using the switch bit to switch the frequency of the transmission error and the allowable operation. It is possible to cope with various situations such as an amount and a memory amount.

  By the way, as described above, in bilinear interpolation, processing is performed using only surrounding four pixels. Therefore, discontinuity in the oblique direction is particularly generated in the image formed by the reproduced pixels, and it is inevitable that visual degradation is likely to occur. To avoid this, for example, taking the example of interpolating the interpolation target pixel in FIG. 41, the pixel included in the enlarged reference range wider than the reference range of the bilinear interpolation at the time of the interpolation is taken. Interpolate using

  That is, by performing interpolation using pixels included in an enlarged reference range wider than the reference range of bilinear interpolation, the tendency of pixels in a wider range is reflected, and the problem of discontinuity is avoided. Is done. In addition, the majority decision effect is obtained when the number of pixels used for the interpolation is "odd number" such as "9 pixels", and the effect of avoiding discontinuity is smaller than that of "even pixels" such as "4 pixels". In some cases, it can be obtained more significantly.

FIG. 58B shows an example in which interpolation is performed using 12 pixels. Using the notation of the embodiment described above with reference to FIG. 58 (a), the reproduced pixels at a certain position in a macroblock in a 1/2 reduced image of a certain macroblock are p1, p2, p3, p4, and When those values (pixel values) are Ip1, Ip2, Ip3, and Ip4, these Ip1, Ip2, Ip3, and Ip4 are:
Ip1: if (4 * Ia + 2 * (Ib + Ic + Id) + Ie + If + Ig + Ih + Ii + Ij + Ik + Il)> 8 then "1" else "0"
Ip2: if (4 * Ib + 2 * (Ia + Ic + Id) + Ie + If + Ig + Ih + Ii + Ij + Ik + Il)> 8 then "1" else "0"
Ip3: if (4 * Ic + 2 * (Ib + Ia + Id) + Ie + If + Ig + Ih + Ii + Ij + Ik + Il)> 8 then "1" else "0"
Ip4: if (4 * Id + 2 * (Ib + Ic + Ia) + Ie + If + Ig + Ih + Ii + Ij + Ik + Il)> 8 then "1" else "0"
Can be represented by Where a is pixel A, b is pixel B, c is pixel C, d is pixel D, e is pixel E, f is pixel F, g is pixel G, h is pixel H, i is pixel I, and j is pixel J and k indicate the pixel K, and l indicates the pixel L.

  Further, the “１ ／” size reduction / enlargement process may be realized by performing the “l / 2” size reduction / enlargement process twice.

  Next, an example of a combination with the enlargement / reduction processing in frame units will be described as a fifth specific example.

(Fifth specific example)
The prior art shown by the present inventors in Japanese Patent Application No. 8-237053 includes an embodiment in which rate control is performed by performing reduction / enlargement on a frame basis (actually, a rectangular area including an object), and a block-like embodiment. And an embodiment in which rate control is performed by performing reduction / enlargement in units of small areas. Further, in the above-described first to third specific examples, an example has been described in which rate control is performed by performing reduction / enlargement in more specific small area units.

  In this specific example described here, an example is shown in which reduction / enlargement processing in units of frames and reduction / enlargement processing in units of small regions are used in combination.

<Alpha map encoding device>
FIG. 25 is a diagram for explaining the alpha map encoding device of this example. This device includes reduction circuits 5300, 5210, 5260, a binary image encoding circuit 5220, enlargement circuits 5230, 5400, a motion compensation prediction circuit 5250, and multiplexing circuits 5240, 5500.

  In such a configuration, the binarized alpha map image supplied via the alpha map signal input line 20 is reduced by the reduction circuit 5300 in frame units based on the reduction / enlargement ratio CR. The signal reduced in frame units is supplied to an alpha map encoding circuit 5200 via a signal line 5020, and is divided into small areas and then encoded.

  Here, the alpha map encoding circuit 5200 is equivalent to the alpha map encoding circuit 200 of FIG. 12, and the constituent elements 5210 to 5260 of the alpha map encoding circuit 5200 are respectively equivalent to those of the alpha map encoding circuit 200 of FIG. Since the components 210 to 260 have the same function, the description of the alpha map encoding circuit 5200 is omitted here. Further, since the configuration of the alpha map encoding circuit 200 in FIG. 12 is a simplified representation of the configuration of the alpha map encoding circuit 200 in FIG. 1, the configuration of the alpha map encoding circuit 5200 is as shown in FIG. A configuration equivalent to the alpha map encoding circuit 200 may be used.

  The coded information coded by the alpha map coding circuit 5200 is multiplexed with the reduction / enlargement rate CRb for each small area, and is supplied to the multiplexing circuit 5500 via the signal line 5030, and is transmitted in frame units. It is multiplexed with the encoded information of the reduction / enlargement rate CR and output via the signal line 30.

  Further, the reproduced image of the alpha map encoding circuit 5200 is supplied to the enlargement circuit 5400 via the signal line 5040, and is enlarged via the signal line 40 after being enlarged based on the reduction / enlargement ratio CR in frame units. .

<Alpha map decoding device>
FIG. 26 is a diagram for explaining the decoding device of this specific example. This alpha map decoding device includes separation circuits 6500 and 6430, a binary image decoding circuit 6410, a reduction circuit 6450, a motion compensation prediction circuit 6440, and enlargement circuits 6420 and 6600.

  In such a configuration, the coded information supplied via the signal line 80 is separated by the separation circuit 6500 into a reduction / enlargement ratio CR in frame units and coded information in small area units. Then, the coded information in small area units is supplied to the alpha map decoding circuit 6400 via the signal line 6080, and the reproduction signal in small area units is supplied to the enlargement circuit 6600 via the signal line 6090.

  The alpha map decoding circuit 6400 is equivalent to the alpha map decoding circuit 400 in FIG. Since the configuration of the alpha map decoding circuit 400 in FIG. 13 is a simplified representation of the configuration of the alpha map decoding circuit 400 in FIG. 2, the configuration of the alpha map decoding circuit 6400 in FIG. A configuration equivalent to the alpha map decoding circuit 400 may be used.

  The enlargement circuit 6600 enlarges the reproduction signal supplied via the signal line 6090 based on the information of the reduction ratio CR in frame units, and outputs the reproduced signal from the signal line 90.

  As described above, here, the alpha map signal is reduced / enlarged in frame units and reduced / expanded in small area units. By combining the reduction / expansion in frame units and the reduction / expansion in small area units, side information such as reduction / enlargement ratio information is reduced, so that encoding is performed at a particularly low encoding rate. It is effective in the case.

Next, the frame memory will be described.
Although not explicitly shown in FIGS. 25 and 26, both the encoding device and the decoding device require a frame memory for storing a reproduced image. FIG. 52 is a diagram illustrating an example of the resolution for each frame. In the present invention, since the motion compensation prediction is used, for example, when encoding the frame at the time n, the resolution of the frame at the time n-1 is changed to the resolution of the frame at the time n (in this case, reduction / enlargement). Rate). Here, as shown in FIG. 52, the reproduced image stored in the frame memory is converted by the alpha map encoding circuit 5200 in frame unit resolution (in this example, CR = １ ／ for a frame at time n (FIG. 52) (A)), if the frame is at time n, the frame is stored at CR = 1 ((c) in FIG. 52), and the frame is stored at the original resolution (reduction / enlargement ratio CR = 1 regardless of the time). There are two cases in which the data is accumulated.

  In the former, the frame memory accumulates a reproduced image with a resolution of a frame unit supplied through the signal line 5040 and the signal line 6090, and the frame memory in the latter stores through the signal line 40 and the signal line 90. The supplied reproduced image of the original resolution is stored.

  Therefore, when the frame memory is specified in the encoding device of FIG. 25 and the decoding device of FIG. 26, the former frame memory (this is represented as FM1 (for the encoding device) and FM3 (for the decoding device)) 53, FIG. 53 and FIG. 54. In the case of the latter frame memory (referred to as FM2 (for the encoding device) and FM4 (for the decoding device)), FIG. It looks like 56.

  53. That is, the encoding apparatus shown in FIG. 53 holds information on the reduction / enlargement ratio CR and output information from the enlargement circuit 5230 in a frame memory of FM1 and supplies the held information to an MC (motion compensation prediction circuit) 5250. The decoding device of FIG. 54 is configured to store the CR information and the output from the enlargement circuit 6420 in a frame memory of FM3, and to provide the stored output to the motion compensation prediction circuit 6440. .

  Further, the encoding apparatus shown in FIG. 55 stores information on the reduction / enlargement ratio CR and output information from the enlargement circuit 5400 in a frame memory of FM2, and supplies the held information to an MC (motion compensation prediction circuit) 5250. The decoding apparatus in FIG. 56 is configured to store the CR information and the output from the enlargement circuit 6600, and to provide the stored output to the motion compensation prediction circuit 6440.

  A specific configuration example of the frame memories FM1 and FM3 is as shown in FIG. 57A, and a specific configuration example of the frame memories FM2 and FM4 is as shown in FIG. Become.

  As shown in FIG. 57A, the frame memories FM1 and FM3 are reduced to correspond to the information of the frame memory m11 for storing the image of the current frame and the enlargement / reduction ratio CR to which the image held in the frame memory m11 is separately given. It comprises a reduction / enlargement circuit m12 for enlargement processing, and a frame memory m13 for storing the output reduced / enlarged by the reduction / enlargement circuit m12. The frame memories FM2 and FM4 are shown in FIG. As described above, the frame memory m21 for storing the image of the current frame, the reduction circuit m22 for reducing the image held in the frame memory m21 according to the information of the separately provided enlargement / reduction ratio CR, and the reduction processing by the reduction circuit m22 And a frame memory m12 for storing the output.

The operation of the frame memory having such a configuration will be described.
First, in the case of the frame memories FM1 and FM3, a reproduced image of a corresponding resolution of the frame is supplied to each of the frames via the signal line 5040 and the signal line 6090 and stored in the frame memory m11 for storing the current frame. . The frame memory m11 stores all reproduced images of the current frame when the encoding of the current frame (for example, time n) is completed.

  Next, at the time when the encoding of the frame at the time n + 1 is started, the reduction / enlargement circuit m12 reads the reproduced image of the frame at the time n from the frame memory m11 and reduces / enlarges the frame at the time n + 1 in frame units. Reduction / enlargement processing (resolution conversion) is performed so as to match the rate CR.

  In the example of FIG. 52, the reduction / enlargement rate CR of the frame at time n is “１ ／” as shown in FIG. 52A, and the frame at time n + 1 which is the next frame as shown in FIG. Since the reduction / enlargement ratio CR is “1”, in this case, the reduction / enlargement circuit m12 performs a process of converting the reduction / enlargement ratio CR from “１ ／” to “1”.

  The reproduced image at the time n whose resolution has been converted by the reduction / enlargement circuit m12 is stored in the frame memory m13 of the previous frame, and becomes a reference image for motion compensation prediction of the frame at the time n + 1.

  In the case of the frame memories FM2 and FM4, reproduced images of the original resolution (CR = 1) are supplied to these via the signal lines 40 and 90, and are stored in the frame memory m21 of the current frame. When the encoding of the current frame (for example, time n) is completed, all reproduced images of the current frame are stored in the frame memory m21.

  Next, in the reduction circuit m22, at the time when the encoding of the frame at the time n + 1 is started, the reproduced image of the frame at the time n is loaded from the frame memory m21, and the frame at the time n + 1 is reduced and enlarged in frame units. Reduction processing (resolution conversion) is performed so as to match the rate CR.

  Since the reproduced image stored in the frame memory m21 always has the reduction / enlargement ratio CR = 1, in the example of FIG. 52, the reduction / enlargement ratio CR at time n + 1 is “1”. In this case, the reduction circuit m22 Does not perform the resolution conversion process. In the case of the example of FIG. 52, when the current frame is at time n + 1, the resolution of the frame at time n + 2 is “１ ／”, so the reduction circuit m22 changes the reduction / enlargement ratio CR from “1” to “1/1”. 2 ".

  The reproduced image of the frame at time n whose resolution has been converted by the reduction circuit m22 is stored in the frame memory m33 of the previous frame, and serves as a reference image for motion compensation prediction of the frame at time n + 1.

  The specific configuration and operation of the frame memories FM1, FM2, FM3, and FM4 have been described above. The frame memories FM1 and FM3 are used to reproduce reproduced images of frame unit resolution supplied via the signal lines 5040 and 6090. Are stored in the frame memories FM2 and FM4, and the frame memories FM2 and FM4 store reproduced images of a resolution in frame units supplied via the signal lines 40 and 90. Various other configurations are conceivable.

(Sixth specific example)
Next, an example of a method for encoding attribute information of a macroblock will be described as a sixth specific example. First, the proposed method will be described in Japanese Patent Application No. 8-237053, which is a prior art.

FIG. 29 illustrates an example of attribute information of a certain macroblock at time n and time n-1. However, the attribute information referred to here is “allW” (all the constituent pixels of the macroblock are white), “allB” (all the constituent pixels of the macroblock are black), “Multi” (the constituent pixels of the macroblock) Is information indicating the state of the contents of the macro block.
For example, "allW" is labeled with a code "0", "allB" is labeled with a code "3", and "Multi" is labeled with a code "1".

  Focusing on the minimum rectangular area including the object part in the frame, and setting the rectangular area so that the upper left is in contact with the boundary of the area, the distribution (label) of the attribute information of each macroblock included in the set rectangular area Is, for example, as shown in FIG.

Then, a distribution example of the configuration pixel attribute information of the macro block in the frame at time n shown in FIG. 29A and the configuration pixel attribute information of the macro block in the frame at time n-1 shown in FIG. As shown in the distribution example, very similar labeling is performed between alpha maps of frames that are close in time.
Therefore, in such a case, since the label correlation between frames is high, the coding efficiency is greatly improved by coding the label of the current frame using the label of the already coded frame. Will be done.

Further, generally, the coding area in the frame at the time n (the minimum rectangular area mainly including the object portion, for example, the coding area CA shown in FIG. 59A) and the coding in the frame at the time n-1 The size of the area may be different. In this case, as an example, the size of the coding area in the frame at time n-1 is adjusted to the size in the frame at time n by the procedure shown in FIG. For example, when the coding area size in the frame at time n is one row longer and one column shorter than the coding area size in the frame at time n−1, as shown in FIG. Then, for the coding region in the frame at the short time n-1 in the row, the macroblock sequence for one column at the right end in the region is cut, and then the macroblock sequence for the lower one row is copied below it. More rows.
This state is shown in FIG.

  When the coding area size in the frame at time n-1 is shorter than the coding area size in the frame at time n by one column for the column and one row longer than the row, the lower end of the coding area is One row of the macroblock sequence is cut, and then the rightmost macroblock sequence in the coding area is copied next to the macroblock sequence to increase by one column.

  If the sizes do not match, adjust the size in this way. Note that the method of adjusting the size is not limited to the above method. Finally, as shown in FIG. 30B, the label in the frame at the time n-1 adjusted to the size in the frame at the time n is referred to as a label at the time n-1 'for convenience. Notation will be used in the following description.

  FIG. 31A shows the difference between the above-described macroblock attribute information at time n and the above-described macroblock attribute information at time n−1 ′, that is, the difference between each label at each macroblock position. The results obtained with the same position are shown.

  Here, “S” indicates “the labels match”, and “D” indicates “the labels do not match”.

  On the other hand, FIG. 31B shows a result obtained by calculating a difference between labels of adjacent pixel positions in the attribute information of the macroblock at time n. Here, the difference between the leftmost label and the label at the rightmost pixel position on one line is determined, and the difference between the label at the leftmost pixel position and “0” is determined. Hereinafter, for convenience, FIG. 31A will be referred to as inter-frame coding, and FIG. 31B will be referred to as intra-frame coding.

  As shown in FIG. 31, the ratio of “S” is larger in the inter-frame coding than in the intra-frame coding, and the prediction is more appropriate in the inter-frame coding, so that the code amount can be reduced.

  If the correlation between the frames is extremely small, the coding efficiency may be lower than that in the intra-frame coding. In this case, it is possible to switch between intra-frame encoding and inter-frame encoding with 1-bit code so that encoding can be performed by intra-frame encoding. As a matter of course, the frame to be encoded first has no label to refer to, so intra-frame encoding is performed. At this time, there is no need for a code for switching between frames or within a frame.

An example in which several types of prediction methods are switched will be described more specifically.
In the above example, the case where intra-frame encoding is performed when the correlation between frames is small has been described. However, besides this, intra-frame coding is also effective when transmission errors become a problem, for example, when the present invention is used for image transmission.

  For example, if a transmission error occurs and the previous frame is not correctly reproduced, the current frame is not correctly reproduced using inter-frame coding, but can be correctly reproduced using intra-frame coding. .

  Also, even in intra-frame coding, if many macroblocks are referred to, it is vulnerable to transmission errors. In other words, if the number of referenced macroblocks is large, the code amount can be reduced, but if the number of referenced macroblocks is large, the possibility of referring to a macroblock that includes a transmission error increases, and the referenced macroblock Is included in the processing result and is reflected in the processing result.Therefore, it can be said that transmission errors are weakened.Conversely, if the number of referenced macroblocks is reduced, the code amount increases. Can be said to be stronger.

  Therefore, it is necessary to devise a device that is resistant to transmission errors and reduces the code amount. You can do this as follows.

As an example, in order to be resistant to transmission errors and reduce the code amount, it is effective to prepare several prediction modes and switch between them.
As the prediction mode, for example,
(A) Inter-frame coding mode (B) Intra-frame coding mode (C) Intra-synchronization unit coding mode (D) There is a mode without prediction.

  Here, the “synchronization recovery unit” means, as already described, a rectangular area of an object further divided, that is, a rectangular encoded area CA further divided into required macroblock units. For example, the synchronization recovery unit is divided so that the code amount is equal, or a predetermined number of macroblocks are collectively used as a synchronization recovery unit.

  In the “synchronous recovery unit intra-coding mode”, a reference block (a macroblock to be referred to and a neighboring macroblock of its own belonging macroblock) is located outside the “synchronous recovery unit” even in a frame. In some cases, without reference, for example, a predetermined label is used as the predicted value.

  As a result, even if a transmission error occurs in a frame, if the transmission error is outside the "synchronization recovery unit", the "synchronization recovery unit" can be correctly reproduced.

  Also, "no prediction" means that the label of the macroblock is encoded without referring to any other macroblock, and this mode is the strongest for errors.

  A plurality of these modes are prepared, and one of the optimum ones is selectively switched and used depending on the frequency of occurrence of an error. The switching may be performed for each “synchronization recovery unit”, may be performed for each frame, or may be performed for each sequence. Information on which mode has been encoded is sent from the encoding device to the decoding device.

  As another mode, there is a method of switching an encoding table depending on an occupation position in a frame of an area to be encoded.

That is, as a general tendency of an image, for example, as shown in FIG. 20, there is a high probability that an object exists at the center of a frame, and there is a high probability that an object does not exist at the end of the frame.
Focusing on this, a table in which a short code is assigned to "allW" is used for a macroblock adjacent to the end of a frame, and a table in which a short code is assigned to "allB" is used for other macroblocks. , The code amount can be reduced without using prediction. This is the no prediction mode.

  There is also a simpler method in which a plurality of encoding tables are prepared and the encoding tables are switched and used. The switching information is encoded for each “synchronization recovery unit”, for each frame, or for each sequence.

  FIG. 32 is a block diagram showing a system configuration of this specific example for realizing the above-described processing, and the flow of the processing will be described with reference to this block diagram.

  In the configuration of FIG. 32, a portion surrounded by a broken line is a portion related to the present specific example that realizes the above-described processing. FIG. 32A shows an alpha map encoding device, which includes an object area detection circuit 3100, a blocking circuit 3110, a labeling circuit 3120, a block encoding circuit 3130, a label memory 3140, a size changing circuit 3150, and a label encoding circuit 3160. , A multiplexing circuit (MUX) 3170.

  Among these, the object area detection circuit 3100 detects a rectangular area of a portion including an object in the alpha map signal based on the input alpha map signal, and detects the rectangular area along with information on the size of the rectangular area. It outputs an alpha map signal for a rectangular area. The blocking circuit 3110 is a circuit for converting the alpha map signal of the rectangular area into a macro block, and the labeling circuit 3120 is adapted to convert the macro map into the alpha map signal for each block. The attributes (allW (only white), Multi (mix of white and black), allB (only black)) of the alpha map signal contents are determined, and the labels (“0”, “1”, “3”) corresponding to each attribute are determined. ).

  The block encoding circuit 3130 encodes an alpha map signal in a macroblock having a label of “1” (Multi), and the label memory 3140 is supplied from the labeling circuit 3120. A memory for storing the label information and the size information of the area given from the object area detection circuit 3100 via the label memory output line 3020, and supplying the stored label information and the size information together to the size changing circuit 3150. is there.

  The size changing circuit 3150 obtains the time n− based on the label information and the size information of the frame at the time n−1 supplied from the label memory 3140 and the size information of the frame at the time n supplied from the object area detection circuit 3100. This is a circuit that changes the size of the label information of No. 1 so as to correspond to the size of the time n. The label encoding circuit 3160 is supplied from the labeling circuit 3120 with the size-changed label information as a predicted value. This is a circuit for encoding label information.

  The multiplexing circuit 3170 multiplexes the encoded information obtained by the label encoding circuit 3160, the encoded information supplied from the block encoding circuit 3130, and the size information given from the object area detection circuit 3100. This is the output circuit.

  In the encoding device having such a configuration, the alpha map signal supplied via the signal line 3010 is supplied to the object region detection circuit 3100, and the object region detection circuit 3100 converts the alpha map signal into a square including an object. Detect the area. Information on the size of this rectangular area is output via a signal line 3020, and the alpha map signal inside the area is supplied to a blocking circuit 3110.

  Blocking circuit 3110 macroblocks the alpha map signal inside this area. The macro map-converted alpha map signal is supplied to a labeling circuit 3120 and a block encoding circuit 3130.

  The labeling circuit 3120 determines attributes (“allW”, “Multi”, “allB”) for each macroblock, and assigns labels (“0”, “1”, “3”) corresponding to each attribute. Then, the assigned label information is supplied to the block encoding circuit 3130, the label memory 3140, and the label encoding circuit 3160.

  When the label is “1” (Multi), the block encoding circuit 3130 encodes the alpha map signal in the macroblock, and supplies the encoded information to the multiplexing circuit 3170. The label memory 3140 stores the label information supplied from the labeling circuit 3120 and the size information of the area via the label memory output line 3020. Supplied to.

  The size changing circuit 3150 obtains the time from the label information and the size information of the frame at the time n−1 supplied through the label memory output line 3030 and the size information at the time n supplied through the signal line 3020. The label information obtained by changing the size of the n-1 label information so as to correspond to the size of the time n is supplied to the label encoding circuit 3160.

  The label encoding circuit 3160 encodes the label information supplied to the labeling circuit 3120 using the label information supplied from the size changing circuit 3150 as a prediction value, and supplies the encoded information to the multiplexing circuit 3170. The multiplexing circuit 3170 multiplexes the coded information supplied from the block coding circuit 3130, the label coding circuit 3130, and the size information supplied via the label memory output line 3020 with the coded information supplied from the label coding circuit 3160. Thereafter, the signal is output via a signal line 3040.

  The above is the configuration and operation of the encoding device. Next, the configuration and operation of the decoding device will be described.

  The alpha map encoding device shown in FIG. 32B includes a demultiplexing circuit (DMUX) 3200, a label demultiplexing circuit 3210, a size changing circuit 3220, a label memory 3230, and a block demultiplexing circuit 3240.

  Among them, the demultiplexing circuit 3200 is a circuit for separating the encoded information supplied via the signal line 3050, and the label demultiplexing circuit 3210 is supplied from the size changing circuit 3220 at the time n-1. This is a circuit for reproducing the label information at time n using the information obtained by changing the size of the label information as a prediction value.

  Further, the size changing circuit 3220 is a circuit having the same function as the size changing circuit 3150, and includes the label information and the size information in the frame at the time n−1 supplied from the label memory 3230, and the demultiplexing circuit 3200. Is a circuit for changing the size of the label information in the frame at the time n−1 from the size information of the frame at the time n, which is given separately from the size of the frame at the time n−1. This is a circuit having the same function as the label memory 3140. The label memory 3140 stores the label information decoded and supplied from the label demultiplexing circuit 3210 and the size information of the area provided from the demultiplexing circuit 3200. This is a memory for supplying both the label information and the size information to the size changing circuit 3220.

  The block decoding circuit 3240 is a circuit that reproduces an alpha map signal for each block according to the reproduced label information supplied from the label decoding circuit 3210.

The operation of the decoding device having such a configuration will be described.
The separating circuit 3200 separates the coded information supplied via the signal line 3050 and supplies the coded information to the block combining circuit 3240 and the label decoding circuit 3210, and outputs size information via the signal line 3060. . The label decoding circuit 3210 reproduces the label information in the frame at the time n using the information obtained by changing the size of the label information in the frame at the time n-1 supplied from the size changing circuit 3220 as a prediction value.

  The reproduced label information is supplied to the block decoding circuit 3240 and the label memory 3230. The block decoding circuit 3240 reproduces an alpha map signal for each block according to the reproduced label information supplied from the label decoding circuit 3210. The size changing circuit 3220 operates the same as the size changing circuit 3150, and the label memory 3230 operates the same as the label memory 3140.

  As described above, the encoding apparatus is configured to label the alpha map in macroblock units and use the macroblock labels of the already coded frames to encode the macroblock labels of the current frame. An example of the device has been described. Between the alpha maps of the frames that are close in time, the macroblock is labeled very similarly. Therefore, in such a case, since the label correlation between the frames is high, the label of the current frame is coded by using the label of the already coded frame, thereby greatly improving the coding efficiency. Will be able to do it.

  By the way, in the invention as the prior art, the VLC (variable length coding) table is switched by referring to one adjacent block (one macroblock) within a frame or between frames. In this case, when the inter-frame correlation is high, the VLC table is switched by referring to “adjacent blocks between frames”, and when the inter-frame correlation is low, the “adjacent blocks in frame” is referred to. . However, in actual applications, it is often better to use both inter-frame correlation and intra-frame correlation.

  Therefore, if the mode at a certain pixel position is “M (h, v, t)” (where h, v, and t represent the horizontal, vertical, and time coordinate axes, respectively), the mode “M (x, y, n)” )), For example, “M (x−1, y, n)”, “M (x, y−1, n)”, “M (x, y, n−1)” The VLC table will be selected by referring to the table. Here, when the number of modes is three as shown in FIG. 29, if the number of reference blocks is three (three macroblocks), the number of VLC tables is three to the third power (= 27). Further, the number of reference blocks can be further increased (for example, “M (x−1, y−1, n)”, “M (x, y, n−2)”).

  In this case, not only does the number of VLC tables increase, but also the inter-block correlation with newly added reference blocks decreases. Therefore, even if the number of reference blocks is increased, coding efficiency does not improve much. Therefore, it is necessary to make a trade-off between the number of VLC tables and the coding efficiency.

  Next, a specific example of another method of encoding the attribute information of the block will be described.

  Here, a method of encoding the attribute information of the block using the label of the previous frame for prediction will be described.

<Specific Example of Encoding Device Using Label of Previous Frame for Prediction>
FIG. 43 is a block diagram of an encoding circuit as one specific example of the present invention. As shown in the figure, the encoding circuit includes an object area detection circuit 502, a blocking circuit 504, a labeling circuit 506, a label encoding circuit 508, a label memory 509, a reference block determination circuit 510, and a prediction circuit 512. You.

  Among these, the object area detection circuit 502 sets the area including the object and represented by a multiple of the block size as the coding area from the signal 501 of the alpha map, and cuts out the alpha map signal 503 of the coding area. The block forming circuit 504 divides (cuts) the cut-out alpha map signal 503 into blocks of 16 × 16 pixels (units of macro blocks) and outputs the divided signals. Is to assign a predetermined label according to the degree of inclusion of the object to the block-formed alpha map signal 505 and output the label information 507.

  The label encoding circuit 508 switches the encoding table according to the given prediction value 514 to encode and output the label information 507, and the label memory 509 stores the above-described label assigned by the labeling circuit 506 for each block. The reference block determination circuit 510 stores label information 507. The reference block determination circuit 510 determines a block located at the same position as the encoded block in the previous frame as the reference block 511. The label at the position of the reference block 511 is predicted by referring to the label 513 of the previous frame held at 509 and sent to the label encoding circuit 508 as a predicted value 514.

In the encoding device having such a configuration, the signal 501 of the alpha map is input to the object area detection circuit 502. In the object region detection circuit 502, a region including the object and represented by a multiple of the block size is set as a coding region, and the alpha map 503 cut out from the coding region is sent to the blocking circuit 504. In the blocking circuit 504, the alpha map 503 is divided into 16 × 16 pixel block units (macroblock units), and the blocked alpha map 505 is sent to the labeling circuit 506. In the labeling circuit 506, for example,
-No object is included in the block: "Label 0"
-An object is included in a part of the block: "label 1"
-Everything in the block is an object: "Label 3"
Label information 507 (mode information) is provided for each block. The label information 507 is sent to the label encoding circuit 508 and stored in the label memory 509. The label memory 509 stores the labels coded up to that time.

  On the other hand, in the reference block determination circuit 510, for example, a block located at the same position as the encoded block in the previous frame is determined as the reference block 511 and sent to the prediction circuit 512. The label 513 of the previous frame is also input to the prediction circuit 512 from the label memory 509, and the label at the position of the reference block 511 is sent to the label encoding circuit 508 as a prediction value 514. The label encoding circuit 508 switches the encoding table according to the predicted value 514, encodes the label information 507, and outputs a code 515.

  Here, when the coding area is always equal to the frame, one reference block is determined. However, if the coding area is smaller than the frame and the position of the coding frame area is different between the previous frame and the current frame, use a coordinate axis whose origin is the corner of the frame or set the corner of the coding area to the origin. The reference block differs depending on which coordinate axis is used.

  The handling of the coordinate axes will be described in detail.

  FIGS. 44A and 44B are examples of frame images Fn-1 and Fn at time n-1 and time n, and mode information MD of each macroblock of the coding area CA in each frame Fn-1 and Fn. It is.

  In the invention of the prior application (Japanese Patent Application No. 8-237053), as an example, the origin Vc0 of the coding area in the current frame (time n) and the origin Vp0 of the coding area in the previous frame (time n-1) And a specific example of determining a block to be referred to when encoding the mode information of the block is presented. This is to associate blocks based on the coordinate axes of the coding area.

  In this case, as shown in FIG. 45 (a), the size of the coding area of the current frame is matched by “cut” or “copy” at the right end or the lower end of the coding area of the previous frame.

  In the example of FIG. 44, the left end and the upper end of the coding area are changed. In such a case, the block corresponding to the mode information of the current frame does not match the shaded block (21 blocks) as shown in FIG. 45 (a). The conversion efficiency may be reduced.

  In the example as shown in FIG. 44, it is preferable that the origin Fc0 of the current frame coincides with the origin Fp0 of the previous frame, and the block at the closest block position on the coordinate axis of the frame be the reference block.

  In the case of FIG. 44, when the reference block is obtained based on the coordinate axes of the frame, the result is as shown in FIG. 45B. That is, in the example of FIG. 44, since there is a change between the left end and the upper end, the size of the left end and the upper end are set to “cut” or “copy” as shown in FIG. Will be. In this case, the blocks corresponding to the mode information of the current frame are, as shown in FIG. 45B, the only blocks that do not match are the three hatched blocks.

  That is, encoding efficiency can be improved by switching between changing the label of the previous frame based on the coordinate axis of the coding area or changing the label based on the coordinate axis of the frame according to the situation. The method of determining the coordinate axes may be such that the encoding device selects the most appropriate one and sends the switching information, or the encoding device and the decoding device may determine the coordinate axes using known information.

  FIG. 46 is an example in which it is better to use the coordinate axes of the coding area. This example shows that the frame changes greatly from the frame Fn-1 in (a) to the frame Fn in (b), as when the camera is swung rightward. In the case of this example, as is apparent from the figure, since the position of the coding area in the frame is largely shifted, it is not advisable to determine the reference block based on the coordinate axes of the frame.

  That is, if the position of the coding area CA in the current frame (the frame Fn at the time n) and the previous frame (the frame Fn-1 at the time n-1) is greatly different from each other, the coding area It is better to use the coordinate axis of the frame, and when the position of the coding area in the frame is not so shifted, it is better to use the coordinate axis of the frame.

Whether or not the position of the coding area in the current frame Fn and the previous frame Fn-1 is largely shifted is determined by information (vector) prev_refscurr_ref indicating the position of the coding area in the frame and the size of the coding area. You can judge from it.
That is, the coded data configuration in the moving picture coding apparatus using the alpha map together is as shown in FIG. 47 according to the standard. In other words, the coded data includes a coding area layer, a macroblock MB layer, and a binary shape layer, and the coding area layer includes coding area size information, coding area position information, coding area reduction / enlargement. Includes rate information. The MB layer includes binary shape information, texture MV information, multi-valued shape information, and texture information. The binary shape information includes mode information, motion vector information, reduction / enlargement rate information, scan direction information, and binary information. Contains encoding information.

  Among these, the coding area size information indicates information indicating the size (vertical and horizontal size) of the coding area, and the coding area position information indicates information indicating the position of the coding area (Vp0, Vc0). The coding area reduction / enlargement rate information indicates reduction / enlargement rate (CR) information of a binary image in units of coding area.

  The MB encoding information indicates information for reproducing an object in the MB. The binary shape information in the MB layer indicates information indicating whether each pixel in the MB is inside the object, and the texture MV information indicates motion vector information for motion compensation prediction of a luminance signal or a color difference signal in the MB. The multi-valued shape information indicates weighting information when an object is combined with another object, and the texture information indicates encoding information of a luminance signal or a color difference signal in the MB.

  The mode information in the binary-shaped layer indicates information indicating an attribute of a binary image in the MB, and the motion vector information indicates motion vector information for performing motion compensation prediction on a binary image in the MB. The enlargement ratio information indicates reduction / enlargement ratio (CR) information of the binary image in MB units, and the scan direction information indicates information indicating whether the encoding order is horizontal or vertical. Indicates encoding information of a binary image.

  Information indicating the position of the coding region (the position of Vp0, Vc0) is contained in the coding region position information, and therefore, the position of the coding region (the position of Vp0, Vc0) is known by using this information. be able to. The frame Fn-1 is compared with the frame Fn using this information. This comparison is performed by obtaining vectors from the home position of the frame to the home position of the coding area.

  As a result, for example, as shown in FIG. 46, when the difference between “prev_ref and curr_ref” is large and the size of the coding area is hardly changed between the current frame and the previous frame, based on the coordinate axes of the coding area. It turns out that it is better. Here, “prev_ref and curr_ref” and the information on the size of the coding area are coded prior to coding of the coding area, and are known information on the decoding device side. No additional information for identifying the

  When the coding area 531 is set as a part of the frame 530 as shown in FIG. 48A, the label is not determined outside the coding area 531. However, at the time of encoding the next frame, a label may need to be inserted beforehand because the outside of the encoding area 531 may be a reference block.

  FIG. 48D shows an example in which a predetermined value, in this case, “0” is inserted into the part of the label whose content is undetermined in FIG. FIG. 48 (c) shows an example in which the undetermined label portion in FIG. 48 (a) is extrapolated from the coding area. This is because the motion of the object is large or the shape changes drastically. This is effective when there is a high probability that the object will appear in the next frame in a portion where there is no object.

  FIG. 48B shows an example of extrapolating only the coding area 532 of the next frame in the memory space of the label memory 509 with respect to the label undetermined part of FIG. In this case, the label of the frame two or more frames earlier can be used for prediction.

  When it is desired to use the label of the previous frame for prediction, there is another method of updating only the coding area in the memory space without performing any extrapolation or inserting a predetermined value.

Next, a description will be given of a label prediction method when the frame reduction / enlargement ratio (CR) is switched for each frame as described with reference to FIG.
FIG. 60 shows an example of reduction processing, in which the previous frame is “CR = 1” and the current frame is “CR =＝”. In this case, the target of the current frame to be reduced and obtained is As shown in the figure, there are four blocks of the previous frame corresponding to the macroblock MB1, which is a block, for example, MB2 to MB5 (see FIG. 61). That is, the macroblocks MB2, MB3, MB4, and MB5 of the previous frame become the reduced macroblock MB1.

  Now, assuming that the address of the macroblock MB1 of the current frame is (x, y), the addresses of the blocks MB2 to MB5 of the previous frame are (2x, 2y), (2x + 1, 2y), (2x, 2y + 1), (2x + 1, 2y + 1).

  Here, the coefficient “2” is given as a ratio between the value of the reduction / enlargement ratio CR between the previous frame and the current frame.

It is appropriate to use any one of the labels of the blocks MB2, MB3, MB4, and MB5 for predicting the encoding of the label of the block MB1, but there are several methods for determining the label.
First, the simplest method with the least amount of calculation is a method using a label of a block at a predetermined position (for example, upper left) among the four. Alternatively, when the same label is used for the four labels, the probability of the prediction being high increases if the label having the largest number is used as the predicted value.

  If the number of labels with the same content is equal, that is, if they are divided into two sets, the labels are ordered in advance in descending order of appearance frequency, and the higher label is selected as the predicted value in that order. I do. When a rectangular encoding area CA is set in a frame, using a coordinate axis whose origin is the corner of the encoding area CA, as shown in FIG. 61, the boundary of the block (the macroblock to be encoded) of the current frame becomes It overlaps the macroblock boundary of the reference frame.

  However, the region to be coded can be set in steps smaller than the width of the macroblock, and using a coordinate axis with the corner of the frame as the origin, the boundaries of the blocks generally do not overlap as shown in FIG. A total of nine macroblocks from MB14 to MB14 are referred to.

  In this case, the label of the macro block MB10 which is referred to as a whole is used.

  FIG. 63 shows an example in which the previous frame has the reduction / enlargement ratio CR = 1/2 and the current frame has the reduction / enlargement ratio CR = 1. In this case, the macroblock MB19 is located at the lower right of the macroblock MB15 in the reduced frame. Will refer to the part. At this time, the label of the block MB15 is used as the predicted value, or the reference portion is close to the blocks MB16 to MB18. You may decide.

<Configuration example of decoding device using label for prediction>
FIG. 49 is a block diagram showing a configuration example of a decoding apparatus according to the present invention in which labels are used for prediction.

  This decoding device includes a label decoding circuit 516, a label memory 517, a reference block determination circuit 518, and a prediction circuit 520.

  Among them, the label decoding circuit 516 decodes the label data from the input code data to be decoded, and the label memory 517 stores the decoded label data. The reference block determination circuit 518 performs a process of determining a block at the same position as the encoded block in the previous frame as the reference block 519.

  Further, the prediction circuit 520 has a function of obtaining a predicted value 522 from the label 521 of the previous frame and the reference block 519 and providing the predicted value 522 to the label decoding circuit 516.

In the decoding device having such a configuration, the stream 515 of the encoded data to be decoded is input to the label decoding circuit 516, and the label is decoded.
On the other hand, the label that has been decoded so far is stored in the label memory 517. In the reference block determination circuit 518, the reference block 519 is determined and sent to the prediction circuit 520 in the same manner as described in the coding device. The prediction circuit 520 also obtains a predicted value 522 from the label 521 of the previous frame and the reference block 519 in the same manner as the encoding device, and sends the predicted value 522 to the label decoding circuit 516. The label decoding circuit 516 switches the decoding table according to the predicted value 522 to decode and output the label 523.

  Further, when encoding a mode “M (h, v, t)” (h, v, t respectively represent horizontal, vertical, and time-direction coordinate axes) of a certain block, for example, “M (x− The coding table is switched with reference to "1, (y, n)", "M (x, y-1, n)", "M (x, y, n-1)", etc., which are used here. A mode may include a set of modes described below (this will be referred to as a mode set A) including a part of information of a motion vector used for motion compensation.

[Mode set A]
(1) “a11W”
(2) “a11B”
(3) “copy (motion vector == 0)”
(4) “copy (motion vector! = 0)”
(5) “coded”
Here, (3) and (4) of mode set A are both copy modes, but (3) of mode set A means that the motion vector is zero, and (4) of mode set A Means that the motion vector is non-zero. In the case of mode set A (4), it is necessary to separately encode the value of the motion vector, but in the case of mode set A (3), it is not necessary to encode the motion vector. When the probability that the motion vector is zero is high, using the mode set A can reduce the total of the code amount of the mode and the code amount of the motion vector.

In this example, when all the pixels of the block obtained as “copy (motion vector == 0)” are, for example, black, the same reproduced image is obtained in both modes (2) and (3). That is, there is no need to separate these two modes. Similarly, when all the pixels of the block obtained as “copy (motion vector = 0)” are white, it is not necessary to separate the above (1) and (3). Therefore,
[Mode set B]
(1) “a11W”
(2) “a11B”
(3) “copy (motion vector! = 0)”
(4) “coded”
As
Step A1: If the motion compensated prediction images obtained when “copy (motion vector = 0)” are all black, the process proceeds to step A3; otherwise, the process proceeds to step A2.

  Step A2: If all motion-compensated predicted images obtained when “copy (motion vector == 0)” are white, the process proceeds to step A4; otherwise, the process proceeds to step A5.

  Step A3: If “M (*, *, *)” to be referred to is “copy (motion vector == 0)”, “M (*, *, *)” is replaced with “a11B”. Proceed to step A6.

  Step A4: If “M (*, *, *)” to be referred to is “copy (motion vector == 0)”, “M (*, *, *) ゛” is replaced with “a11W”. Proceed to step A6.

  Step A5: “M (x, y, n)” is encoded using the encoding table of the mode set A. End of encoding.

  Step A6: “M (x, ysn)” is encoded using the encoding table of the mode set A. End of encoding.

  The use of the algorithm (FIG. 50) that follows the procedure from A0 to A6 eliminates the waste of preparing a plurality of modes for the same result, and reduces the code amount of the block attribute information. This is because the average code length of the code that switches between the four modes (mode set B) can be shorter than the average code length of the code that switches between the five modes (mode set A). However, the method of switching only the mode set A for each block slightly increases the amount of calculation and the amount of memory as compared with the case where only the mode set A is used.

  In the decoding processing, the encoding table is determined to be either for the mode set A or for the mode set B using exactly the same algorithm as in the flowchart of FIG. 50, and decoding is performed using the table.

FIG. 51 shows another algorithm that can obtain the same effect as the above-described algorithm. here,
[Mode set C]
(1) “a11W”
(2) “copy (motion vector == 0)”
(3) “copy (motion vector! = 0)”
(4) “coded”
[Mode set D]
(1) “a11B”
(2) “copy (motion vector == 0)”
(3) “copy (motion vector! = 0)”
(4) “coded”
It is.

  The flowchart of FIG. 51 will be described.

  Step B1: If the motion-compensated predicted images obtained when “copy (motion vector = 0)” are all black, the process proceeds to step B3; otherwise, the process proceeds to step B2.

  Step B2: If the motion-compensated predicted image obtained when “copy (motion vector == 0)” is all white, the process proceeds to step B4; otherwise, the process proceeds to step B5.

  Step B3: If “M (*, *, *)” to be referred to is “allB”, replace “M (*, *, *)” with “copy (motion vector == 0)”. Proceed to B6.

  Step B4: If “M (*, *, *)” to be referred to is “a11W”, replace “M (*, *, *)” with “copy (motion vector == 0).” Stepb7 Proceed to.

  Step B5: “M (xyn)” is encoded using the encoding table of the mode set A. End of encoding.

  Step B6: “M (xsysn)” is encoded using the encoding table of the mode set C. End of encoding.

  Step B7: “M (xyn)” is encoded using the encoding table of the mode set D. End of encoding.

  Further, as the attributes of the block, encoding parameters such as a block size and a block reduction ratio, an encoding scan direction, a motion vector value, and the like can be included as necessary. An example including a scan direction is shown.

[Mode set E]
(1) “a11W”
(2) “a11B”
(3) “copy (motion vector == 0)”
(4) “copy (motion vector! = 0)”
(5) “Coded and horizontal scan”
(6) “coded and vertical scan”
Note that “==” indicates that the left side is equal to the value of the right side, and “! =” Indicates that the left side is not equal to the value of the right side.

  As described above, in the present invention, the reference block of the previous frame is used for prediction. However, instead of the label of this reference block, the alpha map itself may be used for prediction. That is, the alpha map is stored in the memory, and each time the attribute of each block is encoded, the attribute (“allW”, “Multi”, “allB”, etc.) of the reference block is determined, and the attribute is determined based on the attribute. Switch the encoding table.

  In this way, a block whose position is shifted by several pixels from the block used when encoding the previous frame can also be used as a reference block. That is, the reference block does not have to exactly overlap with the block used when encoding the previous frame, and more accurate prediction is possible.

Further, it is also possible to combine the prediction using the alpha map of the reference block and the prediction using the label.
For example, first, the encoding table is switched by “allW”, “Multi”, and “allB” using the alpha map of the reference block, and the encoding table of “Multi” is encoded using the label of the reference block. Try to switch tables.

  For a part to be referred to in the previous frame, there is a method using a motion vector given for each block. That is, a portion pointed by the motion vector of the macroblock adjacent to the coded block, which has already been coded, is cut out from the previous frame, and it is determined whether it is “allW”, “Multi”, or “allB”. Switch the encoding table.

(Seventh specific example)
The present invention uses an alpha map signal representing an object shape and a position in a screen in order to separate a background from an object in a method of coding by dividing the screen into a background and an object when encoding an image. I do. Then, the alpha map signal is encoded together with the encoding information of the image to form a bit stream, which is transmitted or stored. The former is used for broadcasting and personal computer communication, and the latter is used for music CDs and the like. It will be traded as a product with content.

  Then, when providing moving image content recorded on a storage medium as a product, the encoded information of the image and the compressed and encoded version of the alpha map signal are accumulated in a storage medium as a bit stream, and one The following describes an example of a playback system that targets a storage medium storing a compression-encoded bit stream including an image of an alpha map. This will be described as a seventh specific example.

<Specific example of medium for storing information according to the present invention>
This specific example will be described with reference to FIGS. FIG. 27 is a code sequence of mode information b0, motion vector information b1, reduction / enlargement ratio information b2, scan direction information b3, and binary image coding information b4 multiplexed in the VLC / multiplexing circuit 1800 of FIG. Is shown. In the present invention, when decoding the binary image encoding information b4, the information b0 to b3 needs to be decoded. If the mode information b0 has not been decoded prior to all other information bl to b4, other information cannot be decoded. Therefore, the information b0 to b4 must have a configuration in which mode information is arranged at the beginning and binary plane image encoding information is arranged at the end as shown in FIG.

  FIG. 28 is a diagram illustrating a system that reproduces an image signal using the recording medium 8100 in which the code string illustrated in FIG. 27 is stored. A code sequence including the code sequence shown in FIG. 27 is stored in the recording medium 8100. Reference numeral 8200 denotes a decoder that reproduces an image signal from the code string stored in the storage medium 8100, and reference numeral 8300 denotes an image information output device that outputs a reproduced image.

  In this system having such a configuration, a code string having a format as shown in FIG. 27 is stored in the storage medium 8100. The decoder 8300 reproduces an image signal from the code string stored in the storage medium 8100. That is, the decoder 8200 reads a code string from the storage medium 8100 via the signal line 8010, and generates a reproduced image according to the procedures shown in FIGS. Here, FIG. 34 is a flowchart of the step (S5) of “binary image reproduction” in FIG.

  The processing in the decoder 8200 will be described in detail with reference to FIGS. That is, first, the mode information is decoded (step S1), and it is checked whether or not the decoded mode information corresponds to any of “allW”, “allB”, and “copy” (steps S2, S3, S4). ).

  As a result, if “allW”, all the pixel values in the macroblock are set to white, and the processing is completed (step S6). If “allB” instead of “allW”, all the pixel values in the macroblock are changed. If the processing is finished in black (step S7), and if it is neither "allW" nor "allB" but "copy", the motion vector is decoded with information (step S8), and motion compensation prediction is performed (step S8). In step S9), the inside of the macro block is copied with the obtained motion compensation prediction value (step S10). Then, the process ends.

  On the other hand, if it is neither "allW" nor "allB" or "copy" in steps S2, S3 and S4, the process proceeds to the binary image reproduction process (step S5).

  The process in S5 is as shown in FIG. 34. First, it is checked whether or not the coding is “inter” coding (step S21). As a result, if the coding is “inter”, the motion vector information is decoded (step S25), the motion compensation prediction is performed (step S26), and the reduction / enlargement ratio information is decoded (step S22). The decryption is performed (step S23). Then, the binary coded information is decoded (step S24), and the process ends.

  On the other hand, if the result of determination in step S21 is that the encoding is not "inter", the reduction / enlargement ratio information is decoded (step S22), and the scan direction information is decoded (step S23). Then, the binary coded information is decoded (step S24), and the process ends.

  In this way, the decoder 8200 reproduces an image and supplies the reproduced plane image to the image information output device 8300 via the signal line 8020, so that the reproduced plane image is displayed on the image information output device 8300. .

  Here, the image information output device refers to, for example, a display or a printer. As described in the specific example above, in the case of an encoding / decoding device that combines reduction / enlargement in units of frames and reduction / enlargement in units of small regions, information on the frame unit reduction / enlargement ratio is shown in FIG. Must be reproduced prior to the small area unit code string shown in (1), the code of the frame unit reduction / enlargement ratio is located before all the small area unit code strings in the frame.

  As described above, when a moving image as a content and an alpha map signal are compression-encoded and stored as a bit stream in a storage medium, a playback system for the storage medium can be provided.

(Eighth specific example)
A specific example relating to predictive coding of a motion vector will be described as an eighth specific example.

  FIG. 21 and FIG. 22 are diagrams showing the framework of the present invention. In FIG. 21, the motion vector detected by the motion vector detection circuit 1780 is supplied to an MV encoding circuit 1790 via a signal line 1070, encoded, and then supplied to a VLC / multiplexing circuit 1800. Multiplexed with the encoded information of

  In FIG. 22, the motion vector information b1 separated from the encoded information supplied via the signal line 80 by the VLD / separation circuit 2100 is reproduced by the motion vector reproduction circuit 2900 as a motion vector signal.

  This specific example relates to the MV encoding circuit 1790 and the motion vector reproducing circuit 2900.

  Generally, since the correlation of a motion vector signal is strong between adjacent blocks, a motion vector is coded by predictive coding to remove the correlation.

FIG. 66 is a diagram for describing an example of predictive coding of a motion vector.
In FIG. 66, each square frame indicates a macroblock, and a square frame whose base is indicated by a dot pattern is an encoding target block. Assuming that the motion vector of the current block is MVs, in order to obtain the prediction vector MVPs for the MVs of the current block, a macro block immediately before the current block (the left side of the current block in FIG. 66) ), A motion vector MVs2 of a macroblock immediately above the current block to be coded (above the current block to be coded in FIG. 66), and a block to the right of the immediately above macroblock. Is used.

  As described above, the prediction vectors MVPs for the motion vector MVs of the current block are generally obtained using the motion vectors MVs1, MVs2, and MVs3 of the blocks around the current block.

  For example, the horizontal / vertical components MVPs_h and MVPs_v of MVPs are obtained as follows.

MVPs_h = Median (MVs1_h, MVs2_h, MVs3_h)
MVPs_v = Median (MVs1_v, MVs2_v, MVs3_v)
Here, “Median ()” is a process of calculating the median of the values in “()”, and the horizontal and vertical components of the motion vector MVsn (n = 1, 2, 3) are
MVsn_h, MVsn_v
It is written.

  As another example of obtaining the prediction vector MVPs, it is checked whether or not a motion vector exists in each block in the order of MVs1, MVs2, and MVs3, and the motion vector of the block in which the motion vector exists first is defined as MVPs. There are ways.

  FIGS. 67A and 67B show examples of frame images Fn-1 and Fn at time n-1 and time n, and coding areas CAn-1 and CAn in each frame. Here, when a block around the current block is a block that does not include an object, the block does not have a motion vector. Also, in the case of an intra-coded block, no motion vector exists in the block.

For example, when all of MVs1, MVs2, and MVs3 do not exist, a default value (vector) is used for the prediction vector MVPs.
When the motion of the object is small, there is no problem if this default value is set to “zero vector” which is a zero motion vector. However, as shown in the transition from the frame of FIG. 67 (a) to the frame of FIG. When the movement of the position of the object in the frame is large, the motion vector cannot be predicted, and the coding efficiency is reduced.

  In the present invention, as the default values, a difference vector “offset” between the vector “prev_ref” shown in FIG. 67A and the vector “curr_ref” shown in FIG. It is characterized by being adaptively switched for use.

  Here, “offset” is obtained by the following equation.

offset = prev_ref-curr_ref
Switching between the default values “offset” and “zero vector” is performed based on, for example, the error value of the object at time n and the object at time n−1 obtained based on the coordinate axes of the frame, and the coordinate axis of the coding area. The error value of the object at the time point n and the error value of the object at the time point n-1 are compared. If the former is larger, “offset” is used. If the latter is larger, “zero vector” is used. What is necessary is just to use it.

  In this case, it is necessary to send 1-bit side information as switching information. Switching between “offset” and “zero vector” as the default value of the predicted value of the motion vector can be similarly applied to the prediction encoding of the motion vector of the texture information.

  Next, a specific configuration example of the MV encoding circuit 1790 shown in FIG. 21 and a specific configuration example of the MV reproduction circuit 2900 shown in FIG. 22 will be described as a ninth specific example.

(Ninth specific example)
[Configuration of MV encoding circuit 1790]
FIG. 68 is a block diagram showing a specific example of the MV encoding circuit 1790 and its accompanying circuits in the system shown in FIG. 21. (a) is a default value calculation circuit as an ancillary circuit, and (b) is an MV encoding circuit. The circuit 1790.

  The default value calculation circuit as an additional circuit shown in FIG. 68 (a) calculates the position of the object in the current frame as viewed from the previous time from the processing frame at the current time and the processing frame at the previous time for the object area. The movement is obtained as a motion vector value between frames. In FIG. 68 (a), e10 is an encoding area detection circuit, e11 is a default value determination circuit, e12 is an area information memory, and e13 is an offset calculation. The circuit, e14, is a selector.

  In addition, e1 is a signal line for inputting the current frame data, which corresponds to the signal line 20 of the system in FIG. 21, and is for receiving the current frame data input from the signal line 20 as an input. belongs to. Further, e2 in FIG. 68 (a) is a signal line for supplying the previous frame data held in the frame memory 1300, and the signal of the previous frame data is transmitted from the frame memory 1300 via this signal line e2. Receive supply. Further, e3 is a signal line for outputting flag information from the default value determination circuit e11, e4 is a signal line for supplying information on the coding area CAn from the coding area detection circuit e10, and e6 is area information. This is a signal line for supplying position information of the coding area CAn-1 read from the memory e12.

  The coding area detection circuit e10 detects the size and position information VC0 of the coding area CAn based on the image signal of the current frame Fn-1 supplied via the signal line e1, The detection result is supplied to the default value determination circuit e11, the area information memory e12, and the offset calculation circuit e13 via the signal line e4.

  The area information memory e12 is a memory for storing information on the size and position of the coding area CAn-1, and after the coding of the frame at the time n is completed, the size and position information of the coding area CAn is obtained. Will be accumulated.

  The offset calculation circuit e13 obtains a vector value “offset” using the position information of the coding area CAn supplied via the signal line e4 and the coding area CAn-1 supplied via the signal line e6. , To the selector e14.

  The selector e14 receives the “zero vector”, which is a zero motion vector value, and the “offset” given from the offset calculation circuit e13, and inputs one of them via the signal line e3 to the default value determination circuit e11. And a circuit that outputs the vector value selected by the selector e14 as a default value to the selector 1793 of the MV encoding circuit 1790 via the signal line e5.

The above is the configuration of the default value calculation circuit.
Next, the configuration of the MV encoding circuit 1790 will be described.

  The MV encoding circuit 1790 includes an MV memory 1791, an MV prediction circuit 1792, a selector 1793, and a difference circuit 1794, as shown in FIG.

  Among them, the MV memory 1791 is a memory for storing the motion vector information from the motion vector detection circuit 1780 supplied via the signal line 1070 in FIG. 21, and the motion vector MVsn (n− 1, 2, 3) are stored in the MV memory 1790.

  The MV prediction circuit 1792 is a circuit that obtains prediction vectors MVPs based on the motion vectors MVsn (n-1, 2, 3) around the encoding target block supplied from the MV memory 1791. Here, if MVsn (n−1, 2, 3) does not exist, the prediction vector MVPs cannot be obtained normally, and the MV prediction circuit 1792 determines whether the prediction vector MVPs has been obtained normally. And a function of supplying the identification signal to the selector 1793 via the signal line 1795.

  The selector 1793 receives the MVPs supplied from the MV prediction circuit 1792 and a default value provided via the signal line e5 as inputs, and selects one of the two according to the signal supplied via the signal line 1795. Then, it is supplied to the difference circuit 1794.

  The difference circuit 1794 is a circuit for obtaining a prediction error signal of a motion vector. The difference circuit 1794 is provided with a motion vector information supplied from a motion vector detection circuit 1780 via a signal line 1070 and MVPs or a default value supplied via a selector 1793 , And outputs the result as motion vector information b1 from the MV encoding circuit 1790.

Next, the operation of the encoding circuit having such a configuration will be described.
In FIG. 68A, an image signal of a frame Fn which is an image signal at a previous time point (frame data of a frame at the previous time point) stored in the frame memory 1300 is supplied to a signal line e1, and is supplied to a signal line e2. Is supplied with an image signal of a frame Fn-1 which is a current image signal (frame data of the current frame).

  Then, the image signal of the frame Fn is input to the default value determination circuit e11, and the image signal of the frame Fn-1 is input to the default value determination circuit e11 and the coding area detection circuit e10.

  The coding area detection circuit e10 detects the size and position information VC0 of the coding area CAn based on the image signal of the frame Fn-1, and outputs the detection result via a signal line e4 to a default value determination circuit e11. Are supplied to the area information memory e12 and the offset calculation circuit e13.

  On the other hand, in the default value determination circuit e11, the information of the coding area CAn from the coding area detection circuit e10 supplied via the signal line e4 and the coding supplied from the area information memory e12 via the signal line e6. Using the size and position information of the area CAn-1, the error amount between the frame Fn and the frame Fn-1 is compared with the error amount between the coding area CAn and CAn-1. As a result of the comparison, if the former is large, it is determined that “offset” is used as a default value, otherwise, a zero vector is used as a default value, and “offset” is used as a default value, or The flag information for identifying whether to use the vector is output via a signal line e3.

  The flag information output from the default value determination circuit e11 via the signal line e3 is shown in FIG. 47 together with the size and position information of the coding area CAn output from the coding area detection circuit e10 via the signal line e4. Is multiplexed on the layer of the coding area in the data format of. Then, the data is transmitted or stored in a recording medium.

  On the other hand, the area information memory e12 is a memory for storing the size and position information of the coding area CAn-1, and stores the size and position information of the coding area CAn after the coding at the time n is completed. .

  The offset calculating circuit e13 obtains a vector value “offset” using the position information of the coding area CAn supplied via the signal line e4 and the coding area CAn-1 supplied via the signal line e6. After that, the data is supplied to the selector e14.

  In the selector e14, one of “offset” and the zero vector is selected according to a flag supplied from the default value determination circuit e11 via the signal line e3, and this is set as a default value. Then, this default value is output to the selector 1793 of the MV encoding circuit 1790 via the signal line e5.

  The motion vector MVs of the encoding target block is supplied to the MV encoding circuit 1790 of FIG. 68B via a signal line 1070, and is supplied to the MV memory 1791 and the difference circuit 1794.

  The MV prediction circuit 1792 is supplied with the motion vectors MVsn (n-1, 2, 3) around the encoding target block from the MV memory 1791, and obtains prediction vectors MVPs. Here, if MVsn (n−1, 2, 3) does not exist, the prediction vector MVPs cannot be obtained normally, and a signal for identifying whether or not the prediction vector MVPs has been obtained normally is generated. Is supplied to the selector 1793 via the signal line 1795.

  The selector 1793 selects either the MVPs supplied from the MV prediction circuit 1792 or the default value output from the signal line e5 according to the identification signal supplied via the signal line 1795, and supplies the selected value to the difference circuit 1794. You.

  The difference circuit 1794 calculates a prediction error signal of the motion vector, and the calculation result is output from the MV encoding circuit 1790 as motion vector information b1.

  The above is the processing content of the motion vector coding. Next, the decoding process of the motion vector encoded in this manner will be described.

[Decoding circuit]
FIG. 69 is a block diagram showing a specific example of a decoding circuit for realizing the present invention. FIG. 69 is a block diagram showing a main part configuration for reproducing MV encoded data. FIG. 21 is a block diagram showing a specific example of an MV reproduction circuit 2900 and its accompanying circuits in the system shown in FIG. 69A shows a default value calculation circuit as an additional circuit, and FIG. 69B shows an MV reproduction circuit 2900.

  The default value calculation circuit as an additional circuit shown in FIG. 69A includes a region information memory d10, an offset calculation circuit d11, and a selector d12. Further, d1 is a signal line for providing flag information for default value selection identification included in an upper layer of encoded data transmitted or data stored in a storage medium and read out, and d2 is a code line. These are signal lines that provide position information of the area CAn included in the upper layer of the data that has been converted and transmitted, and these correspond to e3 and e4 on the encoding circuit side.

  Further, d3 is a signal line for providing position information of the area CAn-1, and d4 is a signal line for outputting a default value.

  The area information memory d10 is a memory for storing the position information of the area CAn-1, and the offset calculating circuit d11 is provided with the position information of the area CAn supplied via the signal line d2 and the position information supplied via the signal line d3. A vector value "offset" is obtained using the position information of the area CAn-1 to be obtained, and the obtained vector value "offset" is supplied to the selector d12.

  The selector d12 sets one of the zero motion vector value given in advance and the vector value “offset” given from the offset calculation circuit d11 to a default value selection / identification flag given via the signal line e1. The output of the selector d12 is output as a default value to the signal line d4 and is provided to the MV reproducing circuit 2900.

The above is the configuration of the default value calculation circuit on the decoding side.
Next, the configuration of the MV reproduction circuit 2900 will be described.

The MV reproducing circuit 2900 includes an adding circuit 2901, a selector 2902, an MV prediction circuit 2903, and an MV memory 2904, as shown in FIG.
Among these, the addition circuit 2901 receives the motion vector information b1 which is the prediction error signal of the motion vector of the decoding target block and the default value given via the selector 2902, and adds and outputs both. The added output is output to the MV memory 2904 and the signal line 2030 in the configuration of FIG.

  The MV memory 2904 holds the addition output of the addition circuit 2910 and supplies the motion vector MVsn (n = 1, 2, 3) around the block to be decoded. The circuit 2903 obtains the prediction vector MVPs from the motion vector MVsn (n = 1, 2, 3) around the decoding target block supplied from the MV memory 2904 and supplies the prediction vector MVPs to the selector 2902. If MVsn (n = 1, 2, 3) does not exist, the prediction vector MVPs cannot be obtained normally, so the MV prediction circuit 2903 identifies whether the prediction vector MVPs has been obtained normally. A signal generation function is provided, and this identification signal is supplied to a selector 2902 via a signal line 2905.

  The selector 2902 receives the default value given via the signal line e4 and the prediction vector MVPs given from the MV prediction circuit 2903, and selects one of them according to the identification signal supplied via the signal line 2905. This is a circuit provided to the addition circuit 2901.

Next, the operation of the decoding side system having such a configuration will be described.
In FIG. 69A, a flag for identifying whether to use “offset” or a zero vector as a default value is supplied to the signal line d1, and positional information of the area CAn is supplied to the signal line d2.

  The position information of the area CAn supplied via the signal line d2 is supplied to the area information memory d10 and the offset calculation circuit d11. The area information memory d10 is a memory for storing the position information of the area CAn-1, and stores the position information of the area CAn after the decoding at the time n is completed.

  In the offset calculation circuit d11, the vector value “offset” is obtained using the position information of the region CAn supplied via the signal line d2 and the position information of the region CAn-1 supplied via the signal line d3. After that, the vector value “offset” is supplied to the selector d12.

  In the selector e12, one of “offset” and “zero vector” is selected in accordance with the flag supplied via the signal line d1, and is output to the MV reproducing circuit 2900 via the signal line e4 as a default value. I do.

  Next, in the MV reproducing circuit 2900 of FIG. 69B, “motion vector information b1” which is a prediction error signal of a motion vector of the decoding target block is supplied, and supplied to the adding circuit 2901.

  Further, the motion vector MVsn (n = 1, 2, 3) around the decoding target block is supplied from the MV memory 2904 to the MV prediction circuit 2903, and the prediction vector MVPs is obtained. Here, when MVsn (n = 1, 2, 3) does not exist, the predicted vector MVPs cannot be obtained normally, and a signal for identifying whether or not the predicted vector MVPs has been obtained normally is sent through the signal line 2905. The signal is supplied to the selector 2902 via the selector.

  In the selector 2902, either the MVPs supplied from the MV prediction circuit 2903 or the default value output from the signal line d4 is selected according to the identification signal supplied via the signal line 2905. Supplied.

  The adding circuit 2901 adds the motion vector prediction error signal (“motion vector information b1”) and the prediction signal MVPs, thereby reproducing the motion vector MVs of the decoding target block. Then, the motion vector MVs of the decoding target block is output from the MV reproducing circuit 2900 via the signal line 2030 and is stored in the MV memory 2904.

  In this way, the necessary MV encoding processing in the configuration shown in FIG. 21 and the required MV reproduction processing in the configuration shown in FIG. 22 can be realized.

  Although various embodiments have been described above, according to the present invention, it is possible to efficiently encode information of an alpha map, which is sub-image information representing the shape of an object, a position in a screen, and the like, and to decode the information. Thus, an image encoding device and a decoding device capable of performing the above are obtained.

  Also, according to the present invention, since the code amount of the alpha map can be reduced, compared to the conventional coding method of coding on a frame basis, it is possible to separately perform coding for each object without a significant reduction in coding efficiency. It can be encoded.

  The present invention can be implemented with various modifications without being limited to the specific examples described above.

FIG. 1 is a block diagram illustrating a schematic configuration of an entire encoding device to which the present invention is applied. FIG. 1 is a block diagram showing a schematic configuration of an entire decoding device to which the present invention is applied. FIG. 1 is a block diagram showing a configuration of an alpha map encoding circuit in an encoding device to which the present invention is applied. FIG. 2 is a block diagram showing a configuration of an alpha map decoding circuit to which the present invention is applied. FIG. 2 is a block diagram showing a specific example of an encoding circuit according to a first specific example of the present invention. FIG. 2 is a block diagram showing a specific example of a decoding circuit according to a first specific example of the present invention. FIG. 9 is a diagram for explaining the first specific example of the present invention, and is a diagram illustrating an example of a state around a change pixel b1. FIG. 4 is a diagram for describing a first specific example of the present invention, and is a diagram illustrating an example of a reference pixel. FIG. 6 is a diagram for explaining a first specific example of the present invention, and is a diagram for explaining a method of determining a context number. FIG. 9 is a block diagram showing an example of an encoding circuit according to a second specific example of the present invention. FIG. 14 is a block diagram showing a specific example of a decoding circuit according to a second specific example of the present invention. FIG. 13 is a block diagram showing a specific example of an encoding circuit according to a third specific example of the present invention. FIG. 14 is a block diagram showing a specific example of a decoding circuit according to a third specific example of the present invention. FIG. 10 is a diagram for explaining a third specific example of the present invention, and is a diagram for explaining a change pixel detecting unit in inter-frame coding. FIG. 3 is a diagram for explaining switching of a scanning direction. The figure explaining an alpha map. FIG. 9 is a diagram for explaining the prior art, and is a diagram illustrating an example of a VLC table (variable length encoding table). 7A and 7B are diagrams for explaining the prior art, showing a relationship between changed pixels when encoding is performed in block units and a diagram showing a reference region for detecting b1 (the relationship between the changed pixels in block-based encoding and Diagram showing a reference region). FIG. 9 is a diagram for explaining the prior art, and is a flowchart in a case where MMR is encoded on a block basis. FIG. 4 is a diagram showing a state in which a screen of an alpha map used in the present invention is divided into macroblock (MB) units having a predetermined plural pixel configuration. FIG. 14 is a block diagram illustrating a configuration example of an alpha map encoding circuit according to a fourth specific example of the present invention. FIG. 14 is a block diagram illustrating a configuration example of an alpha map decoding circuit according to a fourth specific example of the present invention. FIG. 3 is a diagram for explaining Markov model coding. FIG. 2 is a block diagram illustrating a configuration example of a binary image encoding circuit and a binary image decoding circuit that use a plurality of binary surface image encoding methods by switching. FIG. 18 is a block diagram showing the configuration of an alpha map encoding circuit in which reduction / enlargement in units of frames and reduction / expansion in units of small regions are combined in a fifth specific example of the present invention. FIG. 19 is a block diagram showing a configuration of an alpha map decoding circuit in which reduction / enlargement in units of frames and reduction / expansion in units of small regions are combined in a fifth specific example of the present invention. FIG. 7 is a diagram for explaining the arrangement order of bit strings output from the alpha map encoding circuit of the present invention. FIG. 14 is a block diagram showing a system configuration example of a sixth specific example of the present invention. FIG. 14 is a diagram for explaining a prior art in a sixth specific example of the present invention. FIG. 14 is a diagram for explaining a prior art in a sixth specific example of the present invention. FIG. 14 is a diagram for explaining a prior art in a sixth specific example of the present invention. FIG. 14 is a diagram for explaining a prior art in a sixth specific example of the present invention. 16 is a flowchart for explaining a processing procedure in a sixth specific example of the present invention. 16 is a flowchart for explaining a processing procedure in a sixth specific example of the present invention. FIG. 9 is a diagram for explaining bilinear interpolation used for reduction / enlargement processing. FIG. 4 is a diagram for explaining a smoothing process (smoothing process). FIG. 7 is a diagram for explaining another example of a smoothing filter (smoothing filter) used in the present invention. FIG. 7 is a diagram for explaining an example of a reduction process for reducing a block (macroblock) to a size of “1/2” in the vertical and horizontal directions. FIG. 7 is a diagram for explaining an example of a technique for obtaining a pixel value of a reduced block. FIG. 9 is a diagram for explaining an example of a reduction process by pixel thinning. FIG. 9 is a diagram for explaining an example of an enlargement process used in the present invention. FIG. 9 is a diagram for explaining another example of the reduction / enlargement processing for each block. FIG. 15 is a block diagram illustrating a configuration example of an encoding circuit according to a sixth specific example of the present invention. The figure for demonstrating the example of the mode image MD of the frame image Fn-1 at the time n-1 and the time n, Fn, and each macroblock of the coding area | region CA in each frame Fn-1 and Fn. FIG. 9 is a diagram for explaining a state of a change in a coding area and a change in position of a block corresponding to mode information. FIG. 8 is a diagram for explaining an example of frame images Fn-1 and Fn at time n-1 and time n and mode information MD of each macroblock of the coding area CA in each frame Fn-1 and Fn according to the present invention. FIG. 4 is a diagram for describing an encoded data configuration in a moving image encoding device that uses an alpha map together. FIG. 7 is a diagram for explaining a method of coping with an undetermined label portion that occurs when a target portion occupied by an encoding region is a part of a frame. FIG. 3 is a block diagram showing a configuration example of a decoding device of the present invention in which a label is used for prediction. 9 is a flowchart illustrating an example of an encoding processing procedure used in the encoding device of the present invention. 9 is a flowchart illustrating another example of the encoding processing procedure used in the encoding device of the present invention. The figure showing the example of the resolution for every frame. FIG. 26 is a block diagram illustrating a configuration example of an encoding device when a frame memory required for the encoding device in FIG. 25 is explicitly included and represented. FIG. 27 is a block diagram illustrating a configuration example of a decoding device when a frame memory required for the decoding device in FIG. 26 is explicitly included and represented. FIG. 26 is a block diagram illustrating another configuration example of the encoding device in a case where a frame memory required for the encoding device in FIG. 25 is explicitly included and represented. FIG. 27 is a block diagram illustrating a configuration example of another decoding device in a case where a frame memory required for the decoding device in FIG. 26 is explicitly included. FIG. 2 is a block diagram showing a configuration example of a frame memory used in the encoding device and the decoding device of the present invention. FIG. 9 is a diagram for describing an example of processing for enlarging both horizontally and vertically two times by bilinear interpolation. FIG. 4 is a diagram for explaining processing contents for enlarging both twice in the horizontal and vertical directions by bilinear interpolation processing used in the present invention. FIG. 4 is a diagram for explaining an example of a reduction process used in the present invention. FIG. 4 is a diagram for explaining a reduction process used in the present invention. FIG. 4 is a diagram for explaining a label prediction method used in the present invention. FIG. 9 is a diagram for describing a process of restoring a frame by a scaling process from a reduced frame used in the present invention. FIG. 4 is a diagram for explaining an interpolation pixel position and a use range of a reference pixel in an enlargement process applied in the present invention. FIG. 9 is a diagram illustrating an example of a reference pixel addition process in the enlargement process applied in the present invention. FIG. 4 is a diagram for explaining an example of predictive coding of a motion vector used in the present invention. FIG. 9 is a diagram for explaining inconvenience of predicting a motion vector when the movement of the position of an object in a frame is large. FIG. 13 is a block diagram showing a specific configuration example of an MV encoding circuit 1790 and its peripheral circuits in the system of the present invention. FIG. 13 is a block diagram showing a specific configuration example of an MV reproduction circuit 2900 and its peripheral circuits in the system of the present invention.

Explanation of reference numerals

2 a1 detecting circuit 3, 14 memory 5 mode determining circuit 8 encoding circuit 1-10 table determining circuit 13 decoding circuit 16 table determining circuit 17, 26 table 19 a1 reproducing circuit 22, 27 ... Counters 23 and 28 Huffman table generation circuit 100 Difference circuit 110 and 350 Motion compensation prediction circuit 120 Orthogonal transformation circuit 130 Quantization circuit 140 Variable length encoding circuit 150 and 320 Inverse quantization circuit 160 and 330 ... Inverse orthogonal transform circuits 170,340 ... Addition circuits 180,240 ... Multiplexing circuits 200 ... Alpha map encoding circuits 210,230,260,420 ... Resolution conversion circuits 220 ... Binary image encoding circuits 300,430,520 ... Separating circuit 310 ... Variable length decoding circuit 400 ... Alpha map decoding circuit 410 ... Binary image Decoding circuit 502 ... Object area detection circuit 504 ... Blocking circuit 506 ... Labeling circuit 508 ... Label encoding circuit 509 ... Label memory 510 ... Reference block determination circuit 512 ... Prediction circuit 516 ... Label decoding circuit 517 ... Label memory 518 ... Reference block determination circuit 520 ... Prediction circuit 1100 ... Mode determination circuit 1110 ... CR (reduction / enlargement ratio) determination circuit 1200,2300 ... Selector 1300 ... Frame memory 1400,1500 ... Pixel pixel value setting circuit 1600 ... Motion compensation prediction circuit 1700 binary image coding circuit 1710, 1730, 1740, 2830, 2840 reduction circuit 1720 expansion circuit 1750, 1760, 2820, 2850 transposition circuit 1770 scan type (ST) determination circuit 1780 Vector detection circuit (MVE)
1791 MV memory 1792 MV prediction circuit 1793 selector 1794 difference circuit 1800 VLC (variable length coding) / multiplexing circuit 1900 motion vector reproducing circuit 2100 VLC (variable length coding) / separating circuit 2400 2500: macroblock pixel value setting circuit 2600: motion compensation prediction circuit 2700: frame memory 2800 ... binary image decoding circuit 2810 ... enlargement circuit 2800 ... binary image decoding circuit 2901 ... addition circuit 2902 ... selector 2903 ... MV prediction Circuit 2904 MV memory 8100 Storage medium 8200 Decoder 8300 Image information output device e10 Encoding area detection circuit e11 Default value determination circuit e12 Area information memory e13 Offset calculation circuit e14 Selector d10 Area information memory d11: Offset calculation circuit d12: Selector

Claims (17)

When encoding an image together with an alpha map that is information for distinguishing between an object area and a background area of the image, when encoding the alpha map for each block, the attribute of each block is encoded. Image encoding apparatus,
Means for setting an encoding area, including the object, represented by a multiple of the block size;
Means for dividing the area into blocks;
Labeling means for assigning a unique label to each attribute for each block;
A storage means for holding the label or the alpha map for each frame;
Determining means for determining a reference block of the previous frame corresponding to the encoded block of the current frame;
At least a label or an alpha map of a previous frame held in the storage unit, and a prediction unit that determines a prediction value by the reference block;
Encoding means for encoding the label information of the encoded block using the predicted value,
An image encoding device comprising: The image encoding device according to claim 1,
The coding area is a part of a frame,
An image coding apparatus according to claim 1, wherein said storage means stores a label obtained by extrapolating from the label of the coding area as a label other than the coding area. The image encoding device according to claim 1,
The coding area is a part of a frame,
The storage means uses the label of the coding area as a holding target, and when the reference block in the prediction means is outside the coding area of the previous frame, uses the label of the coding area nearby as the prediction value. An image encoding device characterized by the above-mentioned. The image encoding device according to any one of claims 1 to 3,
The coding area is a part of a frame,
The image coding apparatus according to claim 1, wherein said determining means determines said reference block based on a coordinate axis of said coding area. The image encoding device according to any one of claims 1 to 3,
The coding area is a part of a frame,
The determining means selects one of a first reference block obtained based on a coordinate axis of the coding area and a second reference block obtained based on a coordinate axis of the frame to form the reference block. An image encoding device, comprising: The image encoding device according to claim 5,
The determining means is configured to select one of the first reference block and the second reference block that has a smaller code amount, and to encode information for identifying the selection result. Image encoding device. The image encoding device according to claim 5,
The determination means may be configured to select one of the first reference block and the second reference block according to the position of the coding area of the previous frame and the position of the coding area of the current frame. An image encoding device characterized by the above-mentioned. The image encoding device according to any one of claims 1 to 7,
Prepare multiple encoding tables,
An image coding apparatus according to the above-mentioned prediction means, wherein an index indicating which coding table is used is used as the prediction value. The image encoding device according to any one of claims 1 to 8,
An image coding apparatus according to claim 1, wherein said prediction means is configured to determine a prediction value also using a label of a block of a current frame which has already been coded. The image encoding device according to any one of claims 1 to 9,
As the attributes of the coded block, at least a first attribute indicating that the block is inside the object, a second attribute indicating that the block is outside the object, and the block are 2 A third attribute meaning that value image encoding is performed, and a fourth attribute meaning that the block copies a predetermined portion of the alpha map of the previous frame as it is,
When the predetermined portion of the alpha map of the previous frame straddles the inside and outside of the object of the previous frame, the first to fourth attributes are separately encoded, and the predetermined attribute of the alpha map of the previous frame is encoded. If the part is inside the object of the previous frame, the first attribute and the fourth attribute are identified and encoded, and the predetermined part of the alpha map of the previous frame is An image encoding apparatus characterized in that, when it is outside, the second attribute and the fourth attribute are identified and encoded. The image encoding device according to claim 1,
In addition to providing storage means for holding the reduction / enlargement ratio in frame units,
The encoding means includes a means for changing the reduction / enlargement ratio of a frame in frame units, and encoding the frame in accordance with the reduction / enlargement / reduction ratio.
The determining means determines the reference block of the previous frame corresponding to the coded block of the current frame using the reduction / enlargement rate of the current frame and the reduction / enlargement rate of the previous frame obtained from the storage means. An image encoding apparatus comprising: The image encoding device according to claim 1,
In addition to providing storage means for holding the reduction / enlargement ratio in frame units,
The encoding means has a variable frame reduction / enlargement ratio for each frame, and includes means for encoding corresponding to the reduction / enlargement / reduction ratio.
The determining means determines the reference block of the previous frame corresponding to the coded block of the current frame using the reduction / enlargement rate of the current frame and the reduction / enlargement rate of the previous frame obtained from the storage means. With means,
Further, the prediction means includes a means for setting, when there are a plurality of reference blocks, a label having a large number among labels of the plurality of reference blocks as a prediction value. The image encoding device according to claim 12,
An image coding apparatus according to claim 1, wherein, when there are a plurality of reference blocks, the prediction unit sets a label of a block at a predetermined position among the plurality of reference blocks as a prediction value. The image encoding device according to claim 1,
In addition to providing storage means for holding the reduction / enlargement ratio in frame units,
The encoding means includes a means for changing the reduction / enlargement ratio of a frame in frame units, and encoding the frame in accordance with the reduction / enlargement / reduction ratio.
The determining means determines the reference block of the previous frame corresponding to the coded block of the current frame using the reduction / enlargement rate of the current frame and the reduction / enlargement rate of the previous frame obtained from the storage means. With means,
Further, the prediction means may include a means for assigning priorities to the labels in advance and, when there are a plurality of reference blocks, among the labels of the plurality of reference blocks, a label having the higher priority as a predicted value. An image encoding device characterized by the following. The image encoding device according to claim 1,
In addition to providing storage means for holding the reduction / enlargement ratio in frame units,
The encoding means has a variable frame reduction / enlargement ratio for each frame, and encodes the frame in accordance with the reduction / enlargement / reduction ratio. Accordingly, means for encoding the encoded block using one variable-length encoding table selected from a plurality of types is provided,
The determining means determines the reference block of the previous frame corresponding to the coded block of the current frame using the reduction / enlargement rate of the current frame and the reduction / enlargement rate of the previous frame obtained from the storage means. An image encoding device comprising: means. The image encoding device according to claim 1,
Means for holding a motion vector given a motion vector for each block,
Means for determining a second reference block of the previous frame using the motion vector of the already coded block;
Means having a plurality of types of encoding tables, selecting one of these encoding tables by the second reference block, and subjecting the selected encoding table to encoding by the encoding means;
An image encoding device comprising: The image encoding device according to claim 1,
An image coding apparatus according to claim 1, wherein said prediction means selects and uses one of a plurality of types of prediction methods determined in advance.

Images