JP3387820B2 - Image prediction decoding apparatus and image prediction decoding method - Google Patents

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image predictive decoding device.
Image and image predictive decoding methods
The present invention relates to an image predictive decoding process in a strange case .

[0002]

2. Description of the Related Art In order to efficiently store or transmit digital images, compression coding is required. As a method for compression-encoding a digital image, JPEG or MPE
In addition to the discrete cosine transform (DCT) represented by G,
There are waveform coding methods such as subband, wear bullet, and fractal. Further, in order to remove redundant signals between images, inter-image prediction using motion compensation is performed, and the difference signal is waveform-coded.

Here, MPE based on motion compensation DCT
The G method will be described. First, try to encode 1
The input image of the frame is divided into a plurality of 16 × 16 pixel macroblocks and processed. One macroblock is further divided into four blocks each having a size of 8 × 8 pixels, and the DC for the block having a size of 8 × 8 pixels is divided.
Quantize after applying T. This is called intraframe coding.

On the other hand, in a motion detection method such as block matching, the most error of the target macroblock is detected from another frame temporally adjacent to the current frame containing the target macroblock to be quantized. A small prediction macroblock is detected, motion compensation from a past image is performed based on the detected motion, and an optimum prediction block is acquired. The signal indicating the motion to the prediction macroblock with the smallest error is the motion vector.
The image that is referenced to generate the prediction macroblock,
Hereinafter, it is referred to as a reference image. Next, the difference between the target block and the corresponding prediction block is calculated, and DC is calculated for the difference.
T is performed, the DCT transform coefficient is quantized, and the quantized output is transmitted or stored together with motion information. This is called interframe coding.

The inter-frame coding has a mode in which prediction is performed only from a preceding image in the display order and a mode in which prediction is performed from both a preceding image and a succeeding image.
The former is called forward prediction and the latter is called bidirectional prediction.

On the receiving side, after the quantized transform coefficient is restored to the original difference signal, a prediction block is obtained based on the motion vector based on the difference signal, and the prediction block and the difference signal are obtained. Add and play the image. In this conventional technique, it is premised that the reference image (the image referred to for generating the predicted image) and the target image have the same size.

Recently, in order to improve the compression efficiency and at the same time, it is possible to reproduce the object unit forming an image, and the object forming the image is separately compression-encoded and transmitted as an image of an arbitrary shape. There is. When encoding and decoding an image of such an arbitrary shape, the size of the image often changes. As an example, the ball may become smaller and disappear. In some cases, the size of the image (object) may be zero.

[0008]

In the usual case, the reference image is the reproduced image immediately before the target image. When the size of the reference image is zero, the reference image cannot be predictively encoded because nothing is defined in the reference image, that is, there is no significant image data used for the predictive encoding. Also in this case, if the conventional technique is applied, the only method is intraframe coding. However, when the intra-frame coding is performed, the code amount generally increases and the compression rate decreases. If the image frequently disappears (in this case, the image size becomes zero) or appears in the moving image sequence, the coding efficiency becomes very poor. For example, in an image in which a spotlight is flushed, if the light disappears or appears on an image-by-image basis, the images of all lights are intra-frame encoded.

The present invention has been made in view of the above problems , and forms an image in order to improve the compression efficiency.
When the objects are individually compression-coded and transmitted for each object
If the image size changes and the object disappears
The image is used as a reference image during predictive decoding of the image.
Residual signal (difference signal) that can be avoided
IMAGE PREDICTION DECODING DEVICE AND IMAGE PREDICTION DECODING FOR SUPPRESSION
Aim to get a way.

An image predictive decoding apparatus according to the present invention (Claim 1) is an image predictive decoding apparatus which decodes coded data and reproduces an image using a predicted image, wherein the coded data is decoded. decoding means for reduction, sign-coded data input to the decoding means immediately before the front Kifu No. data is chromatic
When the target image coded data is included, the significant
A target image encoded data to generate a predicted image as a reference image reproduced image obtained by decoding, sign-coded data input to the decoding means immediately before the front Kifu-coding data Yes
If it does not include any target image coded data, it is input to the decoding means before the immediately preceding coded data,
A predicted image generating unit that generates a predicted image using a reproduced image obtained by decoding encoded data including significant target image encoded data as a reference image; a predicted image generated by the predicted image generating unit; And an addition unit for generating a reproduced image by adding the image decoded by the decoding unit.

[0011] The present invention (Claim 2), in the image predictive decoding apparatus according to claim 1, the predictive image generation means, the previous SL immediately before input to the decoding means coded data
Judgment whether or not to include significant target image coded data ,
Is obtained is assumed to be executed based on the flag included in the immediately preceding coded data input before Symbol decoding means.

An image predictive decoding method according to the present invention (claim 3) is an image predictive decoding method of decoding coded data and reproducing an image using a predicted image, wherein the coded data is decoded. However, before if the decoded sign-encoded data immediately before the Kifu-coding data includes significant object image coded data, the reproduced image obtained by decoding the significant object image coded data the generated predicted image as a reference image, before if the decoded sign-encoded data immediately before the Kifu No. data does not include significant object image coded data, prior to the immediately preceding coded data Decrypted, significant
A predicted image is generated with the reproduced image obtained by decoding the encoded data including the target image encoded data as the reference image,
The reproduced image is generated by adding the generated predicted image to the decoded image.

[0013] The present invention (Claim 4), in the image predictive decoding method according to claim 3, wherein, the previous determination of whether decoding encoded data before Kijika includes significant object image coded data , and executes, based on the flag included previously decoded coded data prior Kijika.

[0014]

[0015]

[0016]

[0017]

[0018]

[0019]

[0020]

[0021]

[0022]

[0023]

[0024]

[0025]

[0026]

[0027]

[0028]

[0029]

[0030]

[0031]

[0032]

[0033]

[0034]

DETAILED DESCRIPTION OF THE INVENTION Embodiments of the present invention will be described below with reference to FIGS. (Embodiment 1) FIG. 1 is a flow chart of a predictive image generation method in an image predictive decoding method according to Embodiment 1 of the present invention. Before describing FIG. 1, an image prediction method in image predictive decoding according to the present invention will be described with reference to FIG. The image size of the input image used in the image predictive decoding method according to the first embodiment is variable, and the size may be zero in some cases.

FIG. 2A shows each of images 201 to 210 of moving images arranged in the display order. Image 201 is the first screen,
Next, images 202, ... Are sequentially displayed. This order is shown by # 1 to # 10. Image # 1 (201)
Is the first image and is intra-frame coded. In the first embodiment, an image (1 frame) is converted into a plurality of 8 × 8 images.
It is divided into blocks each having a pixel size, DCT is performed on each block having a size of 8 × 8 pixels, and quantization is performed. Variable-length coding is performed on the quantized coefficient. In the case of decoding, the coded data obtained by the variable length coding is subjected to variable length decoding, the quantized coefficient obtained by the decoding is inversely quantized, and then the inverse DCT transform is performed to obtain an image. Reproduce.
Next, the image # 2 (202) is replaced with the image # which has already been reproduced.
1 (201), interframe predictive coding is performed.

In the first embodiment, the motion detection method of block matching detects the prediction block having the smallest error with respect to the target block from the image # 1 (201). Based on the detected motion from the target block to the prediction block, the optimum prediction block is acquired from the reproduced image # 1 (201) by motion compensation of the target block. Next, the difference between the target block and the corresponding prediction block is obtained, the difference is subjected to DCT, the DCT transform coefficient is quantized, and the quantized output is transmitted or accumulated together with motion information. Here, the reproduced image # 1 (201) becomes a reference image of the image # 2 (202). This is called forward prediction. At the time of decoding, the prediction block is added to the difference obtained by the inverse quantization and the inverse DCT, and the image is reproduced.

Similarly, image # 3 (203) and image # 4
(204) is predictive coding from the reference image indicated by the arrow. Also, image # 6 (206) and image # 8 (20
8), like image # 10 (210), it is possible to make predictions from the image two images before. Also, image # 5 (205),
As with image # 7 (207) and image # 9 (209), in addition to forward prediction, images displayed after the image can also be referred to. Such prediction is referred to as backward prediction by referring to the image displayed after the image. When performing both forward prediction and backward prediction, this is called bidirectional prediction. The bidirectional prediction includes a forward prediction mode, a backward prediction mode, and an interpolation mode for averaging the forward and backward predictions.

FIG. 2B shows the transmission order of the images predicted in FIG. 2A, that is, the decoding order. Image # 1 (21
1) is first decoded and reproduced. Refer to it,
Image # 2 (212) is decoded. Image # 5 (216)
For such a bidirectional predicted image, it is necessary to first decode and reproduce the reference image used for prediction. Therefore, the image # 6 (215) is ahead of the image # 5 (216). Similarly, the image # 8 (217) becomes the image # 7.
Before (218), the image # 10 (219) is the image #
9 (220), transmitted, decoded, and reproduced.

When transmitting images of variable image size, the size must be transmitted for each image. In the first embodiment, the image size is described at the beginning of the encoded data of the image, and the horizontal and vertical sizes Hm, V are set.
Each m is represented by 20 bits. FIG. 7 shows the first embodiment.
In this coded data (VD), in addition to the horizontal and vertical sizes Hm and Vm indicating the image size data, a motion vector, a quantization width, and , DCT coefficients, etc. are included.

Next, the predictive image generation method in the predictive image decoding method according to the first embodiment of the present invention will be described with reference to the flowchart of FIG. When generating the predicted image, first, in step 102, the size of the immediately preceding reference image is input, and in step 103, it is checked whether the size of the reference image is zero.

Here, the reference image is always in front of the image which is the object of decoding (in the case of encoding, the object of encoding) in the decoding order shown in FIG. 2 (a). The reference image is an image reproduced immediately before in the image predictive decoding method according to the first embodiment. For example, Figure 2 (b)
Image # 4 (214) is a reference image
It becomes 3 (213). However, the image reproduced by bidirectional prediction is not used as a reference image because it is not used for prediction. Therefore, for example, the reference image of the image # 8 (217) is the image # 6 (215).

If the size of the reference image is not zero in the determination of step 103 in FIG. 1, the process proceeds to step 104, and in step 104, a predicted image is generated using the immediately preceding reference image. On the other hand, if it is determined in step 103 that the size of the reference image is zero, the process proceeds to step 105, and in step 105, the most recently played image whose image size is not zero is used as the reference image to calculate the predicted image. To generate. How to find a recently reproduced image whose image size is not zero will be described below with reference to FIG.

When the predicted image of the image # 4 (214) is generated, the image # 3 (2) before the image # 4 (214) is generated.
13) and the size of image # 2 (21
It is assumed that the size of 2) is not zero. In this case, the image #
2 (212), the predicted image of the image # 4 (214) is generated. Similarly, when the predicted image of the image # 6 (215) is generated, if the sizes of the image # 3 (213) and the image # 4 (214) are zero, the image #
2 (212), a predicted image is generated. Here, as the method of generating the predicted image, in the first embodiment,
Similar to MPEG1, a block-based motion compensation method is used.

FIG. 3 shows a block diagram of an image predictive decoding apparatus according to Embodiment 1 of the present invention. The image predictive decoding apparatus 300 according to the first embodiment is configured to receive image data obtained by compression-encoding an image having a variable image size by a predetermined method, and perform a predictive decoding process on the image data.

That is, this image predictive decoding device 300
Is a data analyzer 302 that analyzes the compression-coded image data and outputs the quantization width and DCT coefficient to line 312, the motion vector to line 318, and the image size to line 321; It has a decoder 303 for converting the data of the compressed block from 302 (compressed block) into a decompressed block by decompression processing, and an adder 306 for adding the decompressed block and the prediction block to generate a reproduced block. ing.

Further, the image predictive decoding device 300 is
Frame memory unit 309 for storing the reproduction block
And a prediction image generator 310 that generates an address for accessing the frame memory based on the motion vector and obtains a block corresponding to the address from the image in the frame memory as the prediction block. . Here, the predictive image generator 310 determines, based on the image size from the data analyzer 302, one recently reproduced image having significant image data to be referred to as a reference image. Is also supposed to be done. The reference image is determined by the controller 320 that controls the frame memory unit 309 based on the image size from the data analyzer 302, as shown by the dotted line in FIG.
And the controller 320 selects the most recently reproduced image in which the significant image data to be referenced is present as the reference image.
9 may be controlled.

Further, here, the decoder 303 is
Inverse quantizer 304 that performs inverse quantization processing on the compressed block from the data analyzer 302, and processing that converts the frequency domain signal into a spatial domain signal with respect to the output of the inverse quantizer 304 from line 313. And an inverse discrete cosine converter (IDCT) 305 for performing Also, reference numerals 301 and 307 in the figure respectively denote the main image predictive decoding device 3
00 is an input terminal and an output terminal.

The operation of the image predictive decoding apparatus according to the first embodiment configured as described above will be described below. Image data (encoded data) obtained by compression-encoding an image having a variable image size by a predetermined method is input to the input terminal 301.
To enter. In the first embodiment, compression encoding is performed by the same motion compensation DCT method as in MPEG1, and the encoded data includes the motion vector, the quantization width, the DCT coefficient, and the image size as described above. Data, is included.

Next, the data analyzer 302 analyzes the compression-encoded image data, and as the data of the compressed block, the quantization width and the DCT coefficient are calculated by the line 31.
It is output to the decoder 303 via 2 Further, the motion vector analyzed by the data analyzer 302 is sent to the predicted image generator 310 via the line 318, and the image size similarly analyzed by the data analyzer 302 is controlled via the line 321. Output to the device 320.

In the decoder 303, the inverse quantizer 304
Then, the inverse discrete cosine transformer (inverse DCT transformer) 305 decompresses the data of the compressed block, that is, the compressed block, and restores the expanded block. In the first embodiment, the inverse quantizer 304 inversely quantizes the compressed block, the inverse discrete cosine transform (IDCT) 305 transforms the frequency domain signal into the spatial domain signal, and the decompression block 31.
Get 4. The predictive image generator 310 generates an address 321 for accessing the frame memory unit 309 based on the motion vector sent via the line 318 and inputs this to the frame memory unit 309, A prediction block 317 is generated from the image stored in the unit 309 and is output. The prediction block 317,
That is, the reproduction block 315 is generated by inputting 319 and the expanded block 314 to the adder 306 and adding them. Then, the reproduction block 315 is
At the same time as outputting from the output terminal 307, the line 316
Through the frame memory unit 309. In addition,
Here, when the intra-frame coding is performed, the sample values of the prediction block are all zero.

The operation of the predictive image generator 310 is shown in FIG.
Is the same as that described using the flowchart of FIG. That is, first, the size of the reference image is input to the predicted image generator 310, and the predicted image generator 310 determines the reference image. The reference image may be determined by controlling the frame memory unit 309 via the controller 320 via the line 322 based on the information as to whether the size of the reference image is zero. It is possible.

FIG. 4 is a block diagram of a frame memory bank 406 which is an example of a frame memory unit in the image predictive decoding apparatus according to the first embodiment of the present invention.
In the frame memory bank 406, three frame memories 401 to 403 are provided. The reproduced image is stored in any of these frame memories 401 to 403. Also, when generating the predicted image,
The frame memories 401 to 403 are accessed.

In the first embodiment, the changeover switch 4
04 and 405. The switch 405 is
It is used to determine in which frame memory the reproduced image input via the line 408 (corresponding to the line 316 in FIG. 3) is stored, and is controlled by the controller 320, that is, the control signal. The frame memories 401 to 403 are sequentially switched according to 322. That is, after the first reproduced image is stored in the frame memory 401, the second reproduced image is stored in the frame memory 40.
Store in 2. The same applies hereinafter, but after the third reproduced image is stored in the frame memory 403, the frame memory 401 is switched to. The switch 404 connects to the predictive image generator 310 via line 407 (corresponding to line 317 in FIG. 3). The switch 404 is also switched in a predetermined order by the controller 320, that is, according to the control signal 322. However, the switching order is changed depending on the size of the reference image. For example, according to a predetermined order, even when connecting to the frame memory 402 and generating a predicted image, if the image size of the frame memory 402 is zero, the controller 320 causes the previous frame memory 401 (this Controls the switch 404 to connect to a non-zero image size). In this way, the predicted image can be generated from the reference image whose image size is not zero. The switch 404 may be simultaneously connected to a plurality of frame memories. In a device that resets the frame memory each time one image is played back, the controller 320 manages the frame memory so that the frame memory is not reset when the size of the image played back is zero. No, recently played images can be left in the frame memory. That is, it is possible not to update the frame memory.

Although the case where the block motion compensation discrete cosine transform method is used has been described in the first embodiment, the present invention is not limited to the prediction method (global motion compensation, arbitrary lattice block motion compensation, etc.). Can also be applied to the prediction method using. Further, although the case where one reproduction image is used as the reference image has been described, the same can be applied to a case where a predicted image is generated from a plurality of reference images.

According to the first embodiment as described above,
The size of the immediately preceding input reference image is detected, and when the size of the immediately preceding reference image is not 0, a predicted image is generated using the immediately previous reference image, and the size of the immediately previous reference image is 0. In this case, since the predicted image is generated using the image that has been recently reproduced and whose size is not 0, the objects that form the image are divided into object units in order to improve the compression efficiency.
An image whose image size changes and disappears when compression-encoded and transmitted separately is not used as a reference image in predictive decoding and prediction encoding of an image, and the residual error is eliminated. The effect that proper prediction decoding and prediction encoding which can suppress a signal (difference signal) can be performed is acquired.

(Embodiment 2) In addition, Embodiment 1 described above
In the above, the case where the size of the reference image is detected as zero and the reference image is determined by using the detected information is described. However, the fact that the image size is zero is another index ( For example, in the case of being indicated by a 1-bit flag F, etc.), the index can be used to perform the control, and the second embodiment is configured in this way.

That is, in the second embodiment, the encoded data of the target image is encoded as shown in FIG. 9 when the image size is zero, that is, the corresponding reference image is completely transparent. A 1-bit flag F indicating that there is no data is provided before the image data, that is, before the horizontal and vertical sizes Hm and Vm indicating the image size data (here Then, if the image size is zero, the flag F is set to "0".) In such a case, the method of generating the predicted image is controlled by using the flag F as shown in FIG. It's something that you do.

Next, a predictive image generation method in the predictive image decoding method according to the second embodiment of the present invention will be described with reference to the flowchart of FIG. When generating the predicted image, first, in step 802, the immediately preceding reference image is input, and in step 803, the flag F of the reference image is “1”.
Find out if If the flag F of the reference image is "1" in the determination of step 803, this means that the image size is not zero, in other words, the reference image is not completely transparent, and encoded data exists. Therefore, in step 804, a predicted image is generated using the immediately preceding reference image.

If the flag F of the reference image is not "1" in the determination at step 803 in FIG. 8, the process proceeds to step 805, and at step 805, the flag F that is not "0" is the most recently reproduced image. Is used as a reference image to generate a predicted image.

According to the second embodiment, as in the first embodiment, when the objects forming the image are separately compression-coded for each object and transmitted, the image size becomes smaller. An image that changes and disappears is not used as a reference image, and proper prediction decoding and prediction encoding that can suppress a residual signal (difference signal) can be performed. It is assumed that the coded data of the target image has a flag at the head thereof indicating whether or not significant coded data to be referenced exists in the immediately preceding reproduced image, and the reference image is detected by detecting this flag. Since the determination is made, it is possible to easily perform the calculation for determining the reference image.

(Embodiment 3) FIG. 5 is a flow chart of a predictive image generation method in an image predictive decoding apparatus according to Embodiment 3 of the present invention. The predictive image generation method in the third embodiment is almost the same as that in the first embodiment shown in FIG. 1, and the different processing is step 505 in FIG. 5, which replaces step 105 in FIG. Step 505
Then, when the reference image is zero or when the reference image is completely transparent (or the image flag F is "0").
Then, a prediction image having a predetermined value, that is, a prediction image having a predetermined value is generated as the prediction image.

In the third embodiment, this is gray, that is, the values of the luminance signal and the color difference signal are both 128. As a result, in the third embodiment, when encoding, the gray block is subtracted from the block to be encoded,
When decoding, the gray block is added to the block to be decoded. The predetermined value may be a variable value, which may be transmitted from the encoding unit to the decoding unit and used to generate the predicted image.

According to the third embodiment as described above, when the objects forming an image are separately compression-encoded for each object and transmitted, the image size changes and the image disappears. The image is not used as a reference image, proper prediction decoding and prediction encoding capable of suppressing a residual signal (difference signal) can be performed, and the size of the reference image is 0. When detected,
That is, when the reference image is completely transparent, the predicted image having the predetermined value is generated as the predicted image, so that the same effect as in the first embodiment can be obtained. The effect that can easily generate is obtained.

(Embodiment 4) FIG. 10 shows a flowchart of a predictive image generation method in an image predictive decoding apparatus according to Embodiment 4 of the present invention. The predictive image generation method in the fourth embodiment is almost the same as that in the second embodiment shown in FIG. 8, and the different processing is step 805 in FIG.
Is a step 1005 in FIG. In step 1005, when the flag F of the image of the reference image is "0", a prediction image having a predetermined value substituted, that is, a prediction image having a predetermined value is generated as the prediction image.

According to the fourth embodiment as described above, when the objects forming the image are separately compression-coded for each object and transmitted, the image size changes and the image disappears. The image is no longer used as a reference image, and proper predictive decoding and predictive coding capable of suppressing the residual signal (difference signal) can be performed, and the coded data of the target image is In addition, a flag indicating whether or not significant coded data to be referred to in the immediately preceding reproduced image is present is provided, and when it is detected that this flag is “0”, as a predicted image,
Since it is designed to generate a prediction image with a predetermined value,
It is possible to obtain the same effect as that in the above-described second embodiment, and further it is possible to obtain the effect that the predicted image can be easily generated.

(Embodiment 5) FIG. 6 is a flow chart of a predictive image generation method in an image predictive decoding method using bidirectional prediction according to Embodiment 5 of the present invention. Hereinafter, in the case of bidirectional prediction, which is a feature of the fifth embodiment,
The processing when the reference image size is zero, that is, when the reference image is completely transparent will be described.

That is, as shown in FIG. 6, first, in step 602, the sizes of the front and rear reference images (reference images in both directions) are input. Image # 5 in Figure 2 (a)
(205) is a bidirectional predicted image, the forward reference image is image # 4 (204), and the backward reference image is image # 6.
(206).

Then, step 603 and step 604
Thus, if both the sizes of the front and rear reference images are zero, in step 605, an image obtained by substituting a predetermined value, that is, an image having a predetermined value is set as a predicted image. If the size of the forward reference image is zero and the size of the backward reference image is not zero according to steps 603 and 604, the predicted image is generated using only the backward reference image in step 606.

Next, at step 603 and step 607, if the size of the forward reference image is not zero and the size of the backward reference image is zero, at step 608, the predicted image is obtained using only the forward reference image. To generate. When both the sizes of the front and rear reference images are not zero in step 603 and step 607, in step 609, the predictive image is generated using the reference images in both directions.

Then, in step 610, the generated predicted image is output, the coding unit subtracts the predicted image from the target image, and the decoding unit adds the predicted image to the difference between the target images. To do. In this way, the residual signal (difference signal) can be suppressed.

According to the fifth embodiment as described above, when the objects forming the image are separately compression-coded for each object and transmitted, the prediction image is generated using the bidirectional reference image. , Proper predictive decoding and predictive coding capable of suppressing a residual signal (difference signal) without using an image whose image size changes and disappearing as a reference image. In addition, since the predicted image having the predetermined value is generated as the predicted image, it is possible to obtain the effect that the predicted image can be easily generated.

(Embodiment 6) FIG. 11 is a flowchart of a predictive image generation method in an image predictive decoding method using bidirectional prediction according to Embodiment 6 of the present invention. Hereinafter, the sixth embodiment has the same relationship as the first and third embodiments with the second and fourth embodiments. That is, in the sixth embodiment, “is the size zero?” In steps 603, 604, and 607 of FIG. 6 in the fifth embodiment shown in FIG. 6 is changed to “is the flag F of“ 0 ”?”. Change to steps 1103, 1104 and 1107 of FIG. 11, respectively.
It is what

Therefore, according to the sixth embodiment, when the objects forming the image are separately compression-coded for each object and transmitted, the prediction image is generated using the bidirectional reference image. In this case, an image in which the image size changes and the image disappears is not used as a reference image, and proper prediction decoding and prediction encoding capable of suppressing the residual signal (difference signal) can be performed. In addition to being able to be performed, when it is detected that the flags F of both the front and rear reference images are 0, a prediction image having a predetermined value is generated, so that the image size changes,
It is possible to easily detect an image in which the image disappears, and further it is possible to obtain an effect that a predicted image can be easily generated. An image predictive coding apparatus will be described below as Embodiments 7 and 8 of the present invention.

(Embodiment 7) FIG. 12 is a block diagram of an image predictive coding apparatus according to Embodiment 7 of the present invention.
The image predictive coding apparatus 1000 includes a texture coding unit 1100 that performs predictive coding on a texture signal including a luminance signal and a color difference signal, and a shape coding unit 1200 that performs predictive coding on a shape signal. .

The texture encoding unit 1100 is
A blocker 1110 that divides a texture signal of a frame into macroblocks each having a size of 16 × 16 pixels, which is a unit of encoding processing, and outputs the target block that is an encoding processing target Subtractor 1160 for obtaining a difference from the prediction block, a compression encoder 1120 for compressing and encoding the difference, and a compression encoder 1120.
Local decoder 1130 for decompressing and decoding the output of Here, the compression encoder 1120 includes a discrete cosine transformer 1121 that performs DCT on the difference,
And a quantizer 1122 for quantizing the DCT transform coefficient. The local decoder 1130 also inversely quantizes the output of the quantizer 1122.
31 and an inverse discrete cosine transformer 1132 which performs an inverse DCT on the output of the inverse quantizer 1131 to transform a frequency domain signal into a spatial domain signal.

Further, the texture coding unit 1100.
Is an adder 1170 that adds a decompressed block that is the output of the inverse discrete cosine transformer 1132 and the prediction block to generate a reproduced block, a frame memory unit (FM1) 1140 that stores the reproduced block, A prediction image generator 1150 that acquires a prediction block corresponding to the target block by motion compensation from the image stored in the frame memory unit 1140 based on the motion information detected by the motion detection method.

The predictive image generator 1150 generates a predictive block (predictive image) from the reproduced image stored in the frame memory block 1140 based on the image size obtained from the output of the blocker 1110. An operation of setting a reference image to be referred to is also performed.

On the other hand, the shape coding unit 1200 processes the shape signal of one frame into 16 × 1 which is the unit of coding processing.
A blocker 1210 that divides and outputs each macroblock having a size of 6 pixels, a subtracter 1260 that obtains a difference between a target block that is a target of encoding processing, and a prediction block corresponding to the target block, and the difference And a shape decoder 1230 for decoding the output of the shape encoder 1220 by a decoding method corresponding to the above-mentioned predetermined encoding method. ing. Here, the shape encoder 1120 is configured to encode the output of the subtractor 1260 using a quadtree, a chain encoding method, or the like.

The shape coding unit 1200 adds the decoded block output from the shape decoder 1230 and the prediction block to generate a reproduced block, and outputs from the adder 1270. Frame memory unit (FM2) 1240 for storing decoded blocks
And a predictive image generator 1250 for acquiring a predictive block corresponding to the target block by motion compensation from the shape information stored in the frame memory unit 1240 based on the motion information detected by a predetermined motion detecting method. is doing. Here, the predicted image generator 1250 generates a predicted block (predicted image) from the reproduced image stored in the frame memory block 1240 based on the image size obtained from the output of the blocker 1210. An operation of determining a reference image to be referred to is also performed.

The above coding units 1100 and 120
The determination of the reference image at 0 is performed as shown by the dotted line in FIG.
A shape detector 1280 for performing shape detection based on the reproduction block is provided, and each frame memory unit (FM) is detected by the shape detection output output from the shape detector 1280.
1, FM2) 1140 and 1240 may be controlled. In this case, the control of the frame memory unit by the shape detection output is performed in exactly the same way as the control of the frame memory unit 309 by the controller 320 in the first embodiment. Further, in this case, the shape detection output is supplied to the variable length encoder 1010 and is transmitted together with the encoded data of the texture signal and the shape signal.

Further, the image predictive coding apparatus 1000 described above.
Is a variable-length coding that performs variable-length coding on the coded texture signal output from the texture coding unit 1100, the coded shape signal output from the shape coding unit 1200, and the shape detection output, and multiplexes and outputs. It has a container 1010. In the figure, 1001 is a texture signal input terminal, 1002 is a shape signal input terminal, and 1003 is an encoded data output terminal.

Next, the operation will be described. When a texture signal (luminance / color difference signal) and a shape signal are input to the image predictive coding apparatus 1000, the texture signal and the shape signal respectively correspond to the coding units 1100,
Blocks 1110 and 1210 in 1200 are divided into macroblocks that are units of coding processing, and predictive coding processing is performed for each macroblock.

In the texture signal coding unit 1100,
The difference between the target block and the prediction block is the subtractor 1160.
And the difference is calculated by the DCT unit 1121.
The DCT coefficient is converted into a T coefficient, and this DCT coefficient is further converted into a quantizer 1.
At 122, it is converted into a quantized coefficient. Then, this quantized coefficient is output to the variable length encoder 1010.

The quantized coefficient is inversely transformed into a DCT coefficient by the inverse quantizer 1131, and this DCT coefficient is I
By the process of converting the frequency domain data into the spatial domain data in the DCT unit 1130, it is converted into the decompressed block corresponding to the target block. Further, the decompressed block and the prediction block are added by the adder 1170 to generate a reproduced block. Then, this reproduction block is stored in the frame memory unit 1140. At this time, the predictive image generator 1150 generates a predictive block corresponding to the target block by motion compensation from the image stored in the frame memory unit 1140 based on the motion information detected by a predetermined motion detecting method. Processing is taking place. Further, in this predicted image generator 1150,
Based on the image size obtained from the output of the blocker 1110, refer to one recently reproduced image in which there is significant image data to be referenced from the reproduced images stored in the frame memory block 1140. Determine as an image. Note that, when the shape detector 1280 is provided, this reference image is determined by the shape detection output which is the output thereof, that is, information on whether or not the size of the reproduced image to be normally referred to is zero. It is also possible to control the frame memory unit 1140.

Also, the texture coding unit 1100.
In parallel with the above process, the shape coding unit 1200 performs the predictive coding process on the shape signal in substantially the same manner as the predictive coding process on the texture signal. That is, the difference between the target block and the prediction block is obtained by the subtracter 1260, and the difference is encoded by the shape encoder 1220 by a method such as a quadtree or a chain encoding method, and the variable length encoder is used. Is output to 1010. The shape coded signal from the shape encoder 1220 is the shape decoder 123.
0 is restored, and the restored block and the prediction block are added by the adder 1270 to generate a regenerated block.

The reproduced block output from the adder 1270 is stored in the frame memory unit 1240, and the predictive image generator 1250 uses the frame memory unit 12 based on the motion information detected by a predetermined motion detection method.
From the shape information stored in 40, a prediction block corresponding to the target block is generated by motion compensation. Further, in this predictive image generator 1250, based on the image size obtained from the output of the blocker 1210, there is significant image data to be referenced from the reproduced image stored in the frame memory block 1240. One reproduced image reproduced in step S1 is determined as a reference image.

When the shape detector 1280 is provided, the reference image is determined by determining whether the output of the shape detector, that is, the size of the reproduced image to be normally referred to is zero. It is also possible to control the frame memory unit 1240 according to the information. Further, in this case, the reproduction block is the shape detector 1.
It is input to 280, and shape detection is performed here. For example, when the shape signal is a binary signal, if there is only black data among white data and black data as shape data, there is no reproduction shape. At this time, there is no texture signal corresponding to the shape signal of this block. In this case, as described above, from the shape detector 1280, a flag indicating that there is no encoded data or data having an image size of zero is output as the shape detection output from each of the frame memory units 1140 and 124.
0 and the variable length encoder 1010. In each of the frame memory units 1140 and 1240, control similar to the control of the frame memory 309 by the controller 320 in the first embodiment is performed by the shape detection output.

As described above, according to the seventh embodiment, each of the coding units 1100 and 1200 has a blocker 1110,
Based on the image size obtained from the output of 1210, from the reproduced images stored in the frame memory blocks 1140 and 1210, a recently reproduced one reproduced image having significant image data to be referred to is referred to as a reference image. Therefore, when compressing and encoding the objects that make up the image separately for each object and transmitting the image, the image that changes in image size and becomes too large is referred to as the reference image in the predictive encoding of the image. Therefore, it is possible to obtain an effect that proper predictive coding can be performed. In addition, the coded data that is predictively coded by the image predictive coding apparatus according to the seventh embodiment can be correctly decoded by the image predictive decoding apparatus according to the second embodiment.

When the shape detector 1280 is provided, the shape encoding unit 1200 determines whether or not a reference image corresponding to the input target block exists, and the shape of the reproduced block of the shape signal is determined. By detecting
When there is no shape of the reproduction block, the texture encoding unit and the shape encoding unit generate the prediction block using the reproduction block having the recently reproduced shape instead of the reproduction block corresponding to the target block. When the objects that make up an image are compressed and encoded separately for each object and transmitted, an image that changes in image size and causes the image to be cut is used as a reference image in image predictive encoding. It is possible to obtain the effect that proper prediction encoding capable of suppressing the residual signal (difference signal) can be performed. In this case, the coded data that is predictively coded by the image predictive coding apparatus according to the seventh embodiment can be correctly decoded by the image predictive decoding apparatus according to the second embodiment. In other words, in the image predictive decoding device, the data analyzer 302 controls the frame memory block 309 based on the shape detection output, and thereby, when decoding the coded data that is predictively coded in object units. Therefore, it is possible to perform an appropriate predictive decoding without using an image whose image size is changed and the image is cut off as a reference image in the predictive decoding of the image.

In the seventh embodiment, the selection of the reproduced image as the reference image by the predicted image generators 1150 and 1250 or the control of the frame memory units 1140 and 1240 by the shape detection output is the same as in the first embodiment. The case where the predicted image generators 1150 and 1250 select the reproduced image as the reference image or the controller 320 controls the frame memory unit 309 in exactly the same manner has been described, but the present invention is not limited to this.

For example, when there is no image data to be referenced in the frame immediately before the target frame, a predicted image having a predetermined value may be generated as the predicted image as in the third embodiment. In this case, as the image predictive decoding device corresponding to the image predictive encoding device, the device that performs the image predictive decoding process according to the third embodiment can be used.

Further, the prediction processing in the seventh embodiment may be a bidirectional prediction processing as in the fifth embodiment.
In this case, as the image predictive decoding device corresponding to the image predictive encoding device, the device that performs the image predictive decoding process according to the fifth embodiment can be used.

(Embodiment 8) FIG. 13 is a block diagram of an image predictive coding apparatus according to Embodiment 8 of the present invention.
The image predictive decoding apparatus 1000a includes a texture encoding unit 1100a that performs predictive encoding on a texture signal composed of a luminance signal and a color difference signal, and a shape encoding unit 1200a that performs predictive encoding on a shape signal. .

Here, the texture coding unit 110 is used.
0a is the texture encoding unit 11 according to the seventh embodiment.
In addition to the configuration of 00, before the blocker 1110,
The texture signal is blocked by the control signal by the blocker 111.
It is configured to have a changeover switch 1190 that switches between 0 and ground and supplies it to one of them.

Further, the shape coding unit 1200a does not have the shape detector 1280 in the shape coding unit 1200 of the seventh embodiment, and a shape signal is transmitted to the block former 1210 by a control signal. Blocker 121
It differs from the seventh embodiment in that it has a changeover switch 1290 that switches between 0 and ground and supplies it to one of them.

Further, the image predictive decoding device 1000a.
Has a shape detector 1020 which receives the shape signal and outputs the shape detection output to the changeover switches 1190 and 1290 as the control signal. Here, as a result of the shape detection by the shape detector 1020, when the input shape signal does not have a shape, each of the changeover switches outputs the input texture signal and shape signal by the shape detection output. When the shape signal that is input to the ground side and that is input to the other side has a shape, the texture signal and the shape signal are input to the blockers 1110, 1 respectively.
These signals are switched so as to be input to 210. The shape detection output is output from the variable length encoder 1010 to the encoders 1100a and 1200a.
The variable length coding process is performed together with the coded data from.

Next, the operation of the image predictive decoding apparatus according to the eighth embodiment will be briefly described. In the image predictive decoding device 1000a according to the eighth embodiment having such a configuration, except for the control of the changeover switch by the shape detector 1020,
The same predictive decoding operation as in Embodiment 7 is performed.

That is, this image predictive decoding device 1000
In a, when the texture signal and the shape signal are input, the shape detector 1020 detects whether or not the input shape signal has a shape. As a result of this detection, when the shape signal has no shape, the changeover switches 1190 and 1290 are controlled by the output of the shape detector 1020, and the input texture signal and shape signal are supplied to the ground side. That is, at this time, the predictive coding process for the shape signal and the texture signal is not performed, and the shape detection output is supplied to the variable length encoder 1010.

On the other hand, as a result of the above detection, when the input shape signal has a shape, the changeover switches 1190 and 129 are output by the output of the shape detector 1020.
0 is controlled, the texture signal and the shape signal are input to the blockers 1110 and 1210, and the predictive coding process is performed on each of them. Then, each encoding unit 1100a
And 1200a, the shape detection output is output to the variable length encoder 1010.

As described above, in the eighth embodiment, the shape detector 1020 for determining whether or not the input shape signal has a shape is provided, and when the shape signal has a shape, , The texture signal and the shape signal are subjected to predictive coding, and when the shape signal does not have a shape, the texture signal and the shape signal are not subjected to predictive coding. When the compression encoding is separately performed for each unit and the image is transmitted, the image that the image size changes and the image becomes large is not used as a reference image in the predictive encoding of the image, and the residual signal ( It is possible to perform proper predictive coding capable of suppressing the differential signal).

Further, since the detection output from the shape detector is encoded and transmitted, the image predictive decoding apparatus uses this detection output as a synchronization signal, that is, the image size becomes small. While the image disappears, stop playing the encoded data corresponding to this image,
It is possible to properly perform the predictive decoding process for an image in which the image size changes and the image becomes large.

Further, a program for realizing the image predictive decoding method or apparatus and the image predictive coding method or apparatus configuration shown in each of the above embodiments is recorded on a data recording medium such as a floppy disk. By doing so, it becomes possible to easily carry out the processing shown in each of the above embodiments in an independent computer system.

FIG. 14 illustrates a case where the predictive decoding process or the predictive coding process in each of the above-described embodiments is carried out by a computer system using a floppy disk storing a program corresponding to these signal processes. FIG. FIG. 14 (a) shows the external appearance of the floppy disk FD as viewed from the front, the cross-sectional structure, and the floppy disk body which is the recording medium.
Shows an example of the physical format of the floppy disk body D. Floppy disk body D is case FC
A plurality of tracks Tr are formed concentrically from the outer circumference to the inner circumference on the surface of the disc body D, and each track is angularly divided into 16 sectors Se. Therefore, in the floppy disk body D storing the above program, the floppy disk body D
Data as the above-mentioned program is recorded in the area allocated above.

FIG. 14 (c) shows a structure for recording / reproducing the above program on / from the floppy disk FD. When recording the program on the floppy disk FD, the data as the program is written from the computer system Cs to the floppy disk FD via the floppy disk drive FDD. Also,
When the image transmission method or the image decoding apparatus is constructed in the computer system Cs by the program in the floppy disk FD, the program is read from the floppy disk FD by the floppy disk drive FDD and transferred to the computer system Cs.

In the above description, the data recording medium has stored therein the program for performing the predictive decoding process or the predictive coding process in each of the above-mentioned embodiments, but the data recording medium is the above. There is also one in which encoded data of an image in each embodiment is stored.

Further, in the above description, the image processing by the computer system using the floppy disk as the data recording medium has been described, but this image processing can be similarly performed by using the optical disk. The recording medium is not limited to this, but may be an IC card, a ROM cassette, or the like.
If the program can be recorded, the above image processing can be similarly performed.

[0107]

According to the present invention the image predictive decoding apparatus and the image predictive decoding method according to (Claim 1, 3) As described above, according to the present invention, when decoding the sign-data, before Kifu Goka If the decoded sign-encoded data immediately before the data includes significant object image coded data, a prediction image reproduced image obtained by decoding the significant object image encoded data as a reference picture produced, before if the decoded sign-encoded data immediately before the Kifu No. data does not include significant object image coded data than said immediately preceding encoded data is decoded before a significant Since the predicted image is generated using the reproduced image obtained by decoding the encoded data including the target image encoded data as the reference image, and the generated predicted image and the decoded image are added to generate the reproduced image, Compose images to improve compression efficiency The image data of the body, the object units,
It is possible to avoid using an image in which the image size changes and the object disappears when the image is separately compression-encoded and transmitted as a reference image when decoding the image data. As a result, it is possible to obtain a remarkable effect that an appropriate decoding process can be performed, that is, a decoding process corresponding to an efficient compression encoding process in which the code amount is suppressed can be performed.

In accordance with the invention (claim 2,4), image predictive decoding apparatus according to claim 1 and 3, wherein, in the image predictive decoding method, significant decoding encoded data before before Kijika
The determination of whether including the target image coded data, since executed based on a flag included in the decoded encoded data before Kijika <br/> before, in addition to the above effects, determining a reference picture It is possible to obtain the effect of easily performing the calculation.

[0109]

[0110]

[0111]

[0112]

[0113]

[0114]

[Brief description of drawings]

FIG. 1 is a flowchart showing a predictive image generation method in an image predictive decoding method according to Embodiment 1 of the present invention.

FIG. 2 is a schematic diagram showing a structure of image prediction in image predictive decoding according to the present invention.

FIG. 3 is a block diagram showing an image predictive decoding device according to the first embodiment of the present invention.

FIG. 4 is a block diagram showing a frame memory used in the image predictive decoding device according to the first embodiment of the present invention.

FIG. 5 is a flowchart showing a predictive image generation method in the image predictive decoding method according to the third embodiment of the present invention.

FIG. 6 is a flowchart showing a predictive image generation method in the image predictive decoding method according to the fourth embodiment of the present invention.

FIG. 7 is a diagram showing image data according to the first embodiment of the present invention.

FIG. 8 is a flowchart showing a predictive image generation method in the image predictive decoding method according to the second embodiment of the present invention.

FIG. 9 is a diagram showing image data according to the second embodiment of the present invention.

FIG. 10 is a flowchart showing a predictive image generation method in the image predictive decoding method according to the fifth embodiment of the present invention.

FIG. 11 is a flowchart showing a predictive image generation method in the image predictive decoding method according to the sixth embodiment of the present invention.

FIG. 12 is a block diagram showing an image predictive coding device according to a seventh embodiment of the present invention.

FIG. 13 is a block diagram showing an image predictive coding apparatus according to Embodiment 8 of the present invention.

FIGS. 14 (a) to 14 (c) are diagrams in which the coded data according to each embodiment of the present invention is stored, or the image predictive decoding method, the image predictive decoding apparatus, and the image predictive coding method are used. , Or a diagram for explaining a data recording medium for storing a program for realizing image processing by the image predictive coding apparatus by a computer.

[Explanation of symbols]

301,1001,1002 Input terminal 302 data analyzer 303, 1130, 1230 Decoder 304, 1131 inverse quantizer 305, 1132 Inverse Discrete Cosine Transform (IDCT) 306, 1170, 1270 adder 307 output terminal 309, 1140, 1240 Frame memory unit 310, 1150, 1250 Predictive image generator 320 controller 406 frame memory bank 401-403 frame memory 404,405 switch 407 and 408 lines 1000,1000a image predictive coding apparatus 1003 output terminal 1010 variable length encoder 1020,1280 Shape detector 1100, 1100a Texture coding unit 1200,1200a shape coding unit 1110,1210 blocker 1120, 1220 encoder 1121 Discrete Cosine Transform (DCT) 1122 quantizer 1160, 1260 Subtractor 1170, 1270 adder 1190, 1290 changeover switch # 1 (201) to # 10 (210) images # 1 (211) to # 9 (220) images Cs computer system D floppy disk body F flag FC floppy disk case FD floppy disk FDD floppy disk drive Hm, Vm vertical, horizontal data size

Claims (4)

(57) [Claims] 1. A decodes the coded data, an image predictive decoding apparatus for reproducing an image by using a prediction image, and decoding means for decoding the coded data, prior to Kifu-coding data is input to the decoding means immediately before
If sign-encoded data contains a significant target image encoded data to generate a prediction image reproduced image obtained by decoding the significant object image encoded data as a reference picture, before Kifu Goka is input to the decoding means immediately before the data
If sign-encoded data does not include significant object image coded data, decodes the encoded data including said than the immediately preceding coded data input to said decoding means before, a significant object image coded data A predicted image generating unit that generates a predicted image using the reproduced image obtained as a reference image, and a predicted image generated by the predicted image generating unit,
An image predictive decoding device comprising: an addition unit configured to add the image decoded by the decoding unit to generate a reproduced image. 2. An image predictive decoding apparatus according to claim 1, wherein the predicted image generating means, previously entered Symbol decoding means
Immediately before the immediately preceding coded data to determination of whether including significant object image coded data, which is input before Symbol decoding means
Executed based on the flag included in the encoded data of
An image predictive decoding device characterized by the above. Wherein the encoded data to decode, an image predictive decoding method for reproducing an image using the prediction image, decoding the encoded data, is decoded immediately before the front Kifu-coding data the mark-coding data
If it contains a significant object image coded data, said chromatic
Generating a predicted image as a reference image reproduced image obtained by decoding the meaning of the target image coded data, decoded sign-encoded data immediately before the front Kifu-coding data
If they do not contain significant object image coded data, decodes the encoded data including said is immediately before <br/> decoded before the encoding data, a significant object image <br/> image encoded data An image predictive decoding method characterized by generating a predicted image by using a reproduced image obtained as a reference image as a reference image, and adding the generated predicted image and a decoded image to generate a reproduced image. 4. The image predictive decoding method according to claim 3, the pre Kijika before whether decoding coded data includes significant object image coded data decision, before before Kijika An image predictive decoding method, which is executed based on a flag included in decoded encoded data.