JP2000013791A - Image encoding device, image encoding method, image decoding device, image decoding method, and providing medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To perform normal decoding even from halfway of a coded bit stream by encoding an image, including header of an upper layer in a header of a lower layer and outputting the coded bit stream. SOLUTION: A VLC(variable length coding) device 6 forms a coded bit stream by arranging information that should originally be arranged at respective headers of VS, VISO, VO, VOL, GOV and VOP and further arranging a variable length coded result of image data which is supplied from a quantizer 5 and outputs it to a transmission buffer 7. Also, the device 6 outputs the information arranged at each header of the VS, VISO, VO and VOL which are upper layers than the GOV to a buffer 16 and stores it in it. After that, the device 6 reads information of each header of the VS, VISO, VO and VOL which are upper layers than the GOV stored in the buffer 16 at the time of outputting a GOV header, inserts it into a prescribed position of the GOV header and outputs it.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

[0001] The present invention relates to an image encoding device and an image encoding method, an image decoding device and an image decoding method, and a providing medium. In particular, for example, moving image data is recorded on a recording medium such as a magneto-optical disk or a magnetic tape, and is reproduced and displayed on a display or the like, a video conference system, a video telephone system, a broadcasting device, a multimedia database. Suitable for transmitting moving image data from the transmitting side to the receiving side via a transmission path, such as a search system, and receiving and displaying the moving image data on the receiving side, or for editing and recording. The present invention relates to an image encoding device and an image encoding method, an image decoding device and an image decoding method, and a providing medium.

[0002]

2. Description of the Related Art For example, in a system for transmitting moving image data to a remote place, such as a video conference system or a video telephone system, image data is converted into a line correlation or a frame in order to use a transmission path efficiently. The compression encoding is performed using the inter-correlation.

A typical high-efficiency video coding scheme is Moving Picture Experts Group (MPEG).
(Moving picture coding for storage). This is ISO-I
Discussed in EC / JTC1 / SC2 / WG11,
It has been proposed as a standard, and employs a hybrid method combining motion compensation prediction coding and DCT (Discrete Cosine Transform) coding.

[0004] In MPEG, several profiles and levels are defined in order to support various applications and functions. The most basic is the main profile main level (MP @ ML (Main Profile
at Main Level)).

FIG. 24 is a diagram showing MP @ M in the MPEG system.
5 shows an exemplary configuration of an L encoder.

[0006] Image data to be encoded is input to a frame memory 31 and is temporarily stored. Then, the motion vector detector 32 reads out the image data stored in the frame memory 31 in units of macroblocks composed of, for example, 16 pixels × 16 pixels, and detects the motion vector.

Here, the motion vector detector 32 processes the image data of each frame as any one of an I picture, a P picture, and a B picture. It should be noted that it is determined in advance as to which of the I, P, and B pictures the image of each frame input sequentially is processed (for example, I, B,
P, B, P,..., B, P).

That is, the motion vector detector 32 refers to a predetermined reference frame in an image stored in the frame memory 31, and the reference frame and the current frame are to be encoded. 16 pixels of frame x 16
By performing pattern matching (block matching) with a small block (macro block) of the line, a motion vector of the macro block is detected.

[0009] Here, in MPEG, the image prediction modes include four modes: intra coding (intra-frame coding), forward prediction coding, backward prediction coding, and bidirectional prediction coding.
There are types, I pictures are intra-coded, P pictures are intra-coded or forward predicted coded, B pictures are intra-coded, forward predicted coded, backward predicted coded, or both methods predictive coded .

That is, the motion vector detector 32 sets the intra coding mode as the prediction mode for the I picture. In this case, the motion vector detector 32
The motion vector is not detected, and the prediction mode (intra prediction mode) is output to the VLC (variable length coding) unit 36 and the motion compensator 42.

The motion vector detector 32 performs forward prediction on a P picture and detects the motion vector. Further, the motion vector detector 32 compares a prediction error caused by performing forward prediction with, for example, a variance of a coding-target macroblock (a macroblock of a P picture), and the variance of the macroblock is more predictive. When the difference is smaller than the error, the intra coding mode is set as the prediction mode, and the prediction mode is output to the VLC unit 36 and the motion compensator 42. If the prediction error caused by performing the forward prediction is smaller, the motion vector detector 32 sets the forward prediction encoding mode as the prediction mode, and sets the VLC unit 36 and the motion compensator 42 together with the detected motion vector. Output to

Further, the motion vector detector 32 performs forward prediction, backward prediction, and bidirectional prediction on the B picture, and detects respective motion vectors. Then, the motion vector detector 32 performs forward prediction, backward prediction,
And a minimum prediction error of the bidirectional prediction (hereinafter, appropriately referred to as a minimum prediction error), and the minimum prediction error and the variance of the encoding target macroblock (the macroblock of the B picture), for example. Compare with As a result of the comparison, if the variance of the macroblock is smaller than the minimum prediction error, the motion vector detector 32 sets the intra coding mode as the prediction mode, and sets the VLC unit 3
6 and the motion compensator 42. If the minimum prediction error is smaller, the motion vector detector 32 sets the prediction mode in which the minimum prediction error is obtained as the prediction mode, and sets the VLC unit 3 together with the corresponding motion vector.
6 and the motion compensator 42.

The motion compensator 42 includes a motion vector detector 3
When both the prediction mode and the motion vector are received from 2,
According to the prediction mode and the motion vector, the coded and locally decoded image data stored in the frame memory 41 is read out and supplied to the computing units 33 and 40 as a predicted image.

The arithmetic unit 33 reads from the frame memory 31 the same macroblock as the image data read from the frame memory 31 by the motion vector detector 32,
The difference between the macro block and the predicted image from the motion compensator 42 is calculated. This difference value is supplied to the DCT unit 34.

On the other hand, when only the prediction mode is received from the motion vector detector 32, that is, when the prediction mode is the intra-coding mode, the motion compensator 42 does not output a predicted image. In this case, the arithmetic unit 33 (the same applies to the arithmetic unit 40 to be described later) does not perform any processing, and converts the macro block read from the frame memory 31 into the DC
Output to the T unit 34.

In the DCT unit 34, the output of the arithmetic unit 33 is subjected to DCT processing, and the resulting DCT coefficient is supplied to the quantizer 35. In the quantizer 35,
A quantization step (quantization scale) is set corresponding to the amount of data stored in the buffer 37 (the amount of data stored in the buffer 37) (buffer feedback), and the DCT from the DCT unit 34 is set in the quantization step. The coefficients are quantized. The quantized DCT coefficients (hereinafter, appropriately referred to as quantization coefficients) are supplied to the VLC unit 36 together with the set quantization steps.

In the VLC unit 36, the quantized coefficient supplied from the quantizer 35 is converted into a variable length code such as a Huffman code and output to a buffer 37. further,
The VLC unit 36 performs a quantization step from the quantizer 35,
The prediction mode (mode indicating which of intra coding (intra-picture predictive coding), forward predictive coding, backward predictive coding, and bidirectional predictive coding has been set) and motion from the motion vector detector 32 The vector is also variable-length coded and output to the buffer 37.

The buffer 37 temporarily stores data from the VLC unit 36 and smoothes the data amount.
The data is output to a transmission path (not shown) or recorded on a recording medium.

The buffer 37 outputs the data storage amount to the quantizer 35, and the quantizer 35 sets a quantization step according to the data storage amount from the buffer 37. That is, when the buffer 37 is about to overflow, the quantizer 35 increases the quantization step, thereby reducing the data amount of the quantization coefficient. When the buffer 37 is about to underflow, the quantizer 35 reduces the quantization step, thereby increasing the data amount of the quantization coefficient. Thus, the overflow and the underflow of the buffer 37 are prevented.

The quantization coefficient and the quantization step output from the quantizer 35 are determined not only by the VLC unit 36 but also by the inverse quantizer 3.
8 as well. Inverse quantizer 38
Then, the quantized coefficient from the quantizer 35 is inversely quantized in accordance with a quantization step from the quantizer 35, and is thereby converted into a DCT coefficient. The DCT coefficient is supplied to an IDCT unit (inverse DCT unit) 39. I
In the DCT unit 39, the DCT coefficient is subjected to an inverse DCT process, and is supplied to the arithmetic unit 40.

The arithmetic unit 40 receives the output of the IDCT unit 39 and, as described above, the motion compensator 42 and the arithmetic unit 33.
Are supplied with the same data as the prediction image supplied to the calculation unit 40. The arithmetic unit 40 adds the signal (prediction residual) from the IDCT unit 39 and the prediction image from the motion compensator 42, The original image is locally decoded (however, when the prediction mode is intra coding, the output of the IDCT unit 39 is supplied to the frame memory 41 through the arithmetic unit 40). This decoded image is the same as the decoded image obtained on the receiving side.

The decoded image (local decoded image) obtained by the arithmetic unit 40 is supplied to and stored in the frame memory 41, and then inter-coded (forward predictive coding, backward predictive coding, quantitative predictive coding). ) Is used as a reference image (reference frame) for the image to be processed.

Next, FIG. 25 shows an example of the configuration of an MPEG @ ML decoder in MPEG for decoding the encoded data output from the encoder shown in FIG.

The encoded data transmitted via the transmission path is received by a receiving device (not shown), or the encoded data recorded on the recording medium is reproduced by a reproducing device (not shown),
The data is supplied to the buffer 101 and stored.

An IVLC unit (inverse VLC unit) (variable length decoder) 102 reads out the coded data stored in the buffer 101 and performs variable length decoding to convert the coded data into a motion vector, a prediction mode, Separation into quantization steps and quantization coefficients. Among them, the motion vector and the prediction mode are supplied to the motion compensator 107, and the quantization step and the quantization coefficient are supplied to the inverse quantizer 103.

The inverse quantizer 103 inversely quantizes the quantized coefficient supplied from the IVLC unit 102 in accordance with the quantization step also supplied from the IVLC unit 102.
The resulting DCT coefficient is output to IDCT unit 104. The IDCT unit 104 performs an inverse DCT on the DCT coefficient from the inverse quantizer 103 and supplies the result to an arithmetic unit 105.

The output of the motion compensator 107 is supplied to the arithmetic unit 105 in addition to the output of the IDCT unit 104. That is, the motion compensator 107 reads the already decoded image stored in the frame memory 106 in accordance with the motion vector and the prediction mode from the IVLC unit 102 as in the case of the motion compensator 41 in FIG. ,
The prediction image is supplied to the arithmetic unit 105. Arithmetic unit 10
5 decodes the original image by adding the signal (prediction residual) from the IDCT unit 104 and the predicted image from the motion compensator 107. This decoded image is stored in the frame memory 1
06 and stored. Note that the IDCT device 104
If the output of is intra-coded,
The output passes through the arithmetic unit 105 and is supplied to and stored in the frame memory 106 as it is.

The decoded image stored in the frame memory 106 is used as a reference image for an image to be subsequently decoded, read out as appropriate, and supplied to, for example, a display (not shown) and displayed.

In MPEG1 and MPEG-2, B pictures are not used as reference pictures, so that the frame memory 41 is used in each of the encoder and the decoder.
(FIG. 24) or 106 (FIG. 25).

[0030]

FIG. 24 and FIG. 25 described above.
Are compliant with the MPEG1 / 2 standard, but currently, a VO (Video Ob) which is a sequence of objects such as an object constituting an image is used.
ject), the encoding is performed in units of ISO-IEC
MPEG / JTC1 / SC29 / WG11
(Moving Picture Experts Group) 4 is being standardized.

Since MPEG4 has been standardized mainly for use in the field of communications, the GOP (Group Of Picture) defined in MPEG1 / 2 has been defined in MPEG4. Therefore, when MPEG4 is used for storage media, it is expected that efficient random access will be difficult.

For this reason, the applicant of the present application has proposed the introduction of a GOV (Group Of VOP) layer corresponding to a GOP defined by MPEG1 / 2 in order to enable efficient random access. No.-80758, and this GOV layer was introduced in MPEG4.

By the way, for example, MPEG1, 2, 4,
H. An encoded bit stream obtained by encoding according to a standard such as H.263 has a hierarchical structure including a plurality of layers. Then, on the encoder side, information necessary for decoding is arranged in a header in each layer, and on the decoder side, necessary information is extracted from the header of each layer, and the encoded bit stream is decoded.

Therefore, according to MPEG1 / 2, when a GOP is randomly accessed, information of a header of an upper layer may be required to decode the GOP. It is a standard that allows information of the header of the upper layer to be retransmitted.

However, MPEG4 does not have a standard in which information of the header of the upper layer can be retransmitted after transmission of the upper layer as appropriate.
Even if efficient random access becomes possible due to the introduction of the V layer, information on the header of the upper layer necessary for decoding the GOV cannot be obtained, and thus a normal decoding result cannot be obtained. There is a risk.

Here, when an encoded bit stream of MPEG4 is recorded on a storage medium, it is possible to obtain information of a header of an upper layer by accessing the recording medium. When the encoded bit stream is broadcasted, the information of the header of the upper layer cannot be obtained unless the encoded bit stream is received from the beginning. If the decoding is started halfway, a normal decoding result may not be obtained.

The present invention has been made in view of such a situation, and is intended to enable normal decoding even in the middle of an encoded bit stream.

[0038]

An image coding apparatus according to the present invention comprises coding means for coding an image and outputting a coded bit stream including information of a header of an upper layer in a header of a lower layer. It is characterized by the following.

The image coding method of the present invention is characterized in that an image is coded and a coded bit stream is output including the information of the upper layer header in the lower layer header.

The image decoding apparatus of the present invention extracts information included in a lower layer header from an encoded bit stream including a lower layer header including information of an upper layer header, and extracts the information based on the information. It is characterized by comprising decoding means for decoding the encoded bit stream.

According to the image decoding method of the present invention, information contained in a lower layer header is extracted from an encoded bit stream including a lower layer header including information of an upper layer header, and based on the information, The method is characterized in that the encoded bit stream is decoded.

The providing medium of the present invention encodes an image,
It is characterized in that an encoded bit stream obtained by including information of an upper layer header in a lower layer header is provided.

In the image encoding apparatus of the present invention, the encoding means encodes the image and outputs an encoded bit stream including the information of the header of the upper layer in the header of the lower layer. .

In the image encoding method according to the present invention, an image is encoded, and an encoded bit stream is output, including the information of the upper layer header in the lower layer header.

In the image decoding apparatus according to the present invention, the decoding means extracts information contained in the lower layer header from the coded bit stream including the information of the upper layer header in the lower layer header. The encoded bit stream is decoded based on the information.

In the image decoding method according to the present invention, information contained in a lower layer header is extracted from an encoded bit stream including a lower layer header including information of an upper layer header, and the information is extracted based on the information. , Coded bit stream is decoded.

In the providing medium of the present invention, an image is encoded, and an encoded bit stream obtained by including information of an upper layer header in a lower layer header is provided.

[0048]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described below. Before that, an encoded bit stream specified in MPEG4 will be described. In this case, the FCD (Final Comitt
ee Draft) will be described.

FIG. 1 shows the structure of an encoded bit stream defined by the FCD.

As shown in the figure, the coded bit stream has a VS (Visual Object Sequence) layer and a VISO (V
isual Object) layer, VO (video Object) layer, VOL
(Video Object Layer), GOV (Group of VOP) layer, V
It has a hierarchical structure composed of a plurality of layers, such as an OP (Video Object Plane) layer (in FIG. 1, the higher the layer, the higher the layer).

That is, the coded bit stream is configured in units of VS. Here, VS is an image sequence, and corresponds to, for example, one program or movie.

Each VS is composed of one or more VISOs. Here, there are several types of VISO. That is, VISO includes, for example, a still texture object (Still Texture Object) as a still image and a face object (Face Objec) composed of a face image.
t), VO (Video Objec)
t) and so on. Therefore, when the encoded bit stream is that of a moving image, VISO is composed of VO.

VO is composed of one or more VOLs (Video Object L
ayer) (when the image is not hierarchized (hierarchical encoding), it is composed of one VOL, and when the image is hierarchized, it is composed of VOLs of the same number of layers).

The VOL is a required number of GOVs (Group of V
OP), and the GOV is composed of one or more VOPs (Video Obs).
ject plane). In addition, GOV
In this case, VOL is not less than one VOP
It will be composed of

VOP corresponds to a conventional frame.

The relationship between VS, VO, and VOP will be further described. As described above, VS is an image sequence and corresponds to, for example, one program. VO is the sequence of each object constituting the composite image when there is a sequence of the composite image, and VOP means VO at a certain time. That is, for example, if there is a composite image F3 composed of the images F1 and F2, the image F1 or F2 arranged in time series is VO, and the image F1 or F2 at a certain time is , Are VOPs. Therefore, the VO is the VOP of the same object at different times.
Can be called a set of

Here, VS,
Shows the syntax of VISO and VO. FIGS. 5 to 7 show VOL syntax, FIG. 8 shows GOV syntax, and FIGS. 9 to 11 show VOP syntax, respectively. Note that the semantics of the flags described in the syntax of each layer conform to the MPEG4 FCD standard (14
Please refer to it as described in 496-2).

VS, VISO, MPEG4 FCD standard
The information of the VO and VOL headers includes essential information necessary for decoding the encoded bit stream, and without such information, as described above, it is not possible to decode the encoded bit stream correctly. Have difficulty.

That is, for example, random access or F-code access to an encoded bit stream recorded on a recording medium is performed.
When performing special reproduction such as F / FR (fast forward / rewind), or when accessing a broadcast coded bit stream from the middle, decoding of the coded bit stream must be started first. , VS, VIS
It is necessary to decode the information of the O, VO, and VOL headers.

However, according to the MPEG4 FCD standard, VS,
Only transmission of VISO, VO, and VOL headers is permitted. In this case, it is particularly difficult to start decoding from the middle of a coded bit stream that is broadcast.

Further, for example, a flag for designating an encoding mode is described in the VOL header, including a quantization matrix. These encoding modes are set depending on the properties of the coded bit stream obtained from the picture to be coded, that is, to optimize the statistical properties of the picture to be coded. However, for a long-time image sequence, for example, the properties of an image may greatly change depending on the time, so that the value initially set in the VOL header is not always optimal. Nevertheless, the beginning of the image sequence (here, VO in which a flag specifying the encoding mode is described)
The fact that the flag for designating the encoding mode can be set only at the head of L) hinders efficient encoding.

FIG. 12 shows a configuration example of an embodiment of an encoder to which the present invention is applied. It should be noted that a frame memory 1, a motion vector detector 2, a computing unit 3, a DCT unit 4, a quantizer 5, and a VL
The C unit 6, the buffer 7, the inverse quantizer 8, the IDCT unit 9, the arithmetic unit 10, the frame memory 11, and the motion compensator 12 constitute a frame memory 3 constituting the encoder shown in FIG.
1, motion vector detector 32, arithmetic unit 33, DCT unit 3
4. Corresponds to the quantizer 35, VLC unit 36, buffer 37, inverse quantizer 38, IDCT unit 39, arithmetic unit 40, frame memory 41, and motion compensator 42, respectively. Therefore, in each of the frame memory 1 to the motion compensator 12, the same processing as the processing of the frame memory 31 to the motion compensator 42 may be performed, and the description of such the same processing will be appropriately omitted.

The VOPs constituting the digital image signal to be encoded are sequentially supplied to the frame memory 1 (receiving means), where they are received and temporarily stored. Further, the frame memory 1 is also supplied with a flag FSZ indicating the size of the VOP supplied thereto in a predetermined absolute coordinate system and a flag FPOS indicating the position. These flags FSZ and FPOS are also temporarily stored.

The VOP stored in the frame memory 1 is
The motion vector is read by the motion vector detector 2 in macroblock units. Then, the motion vector detecting circuit 2 performs each VOP according to a predetermined sequence set in advance.
To I (Intra) -VOP, P (Predictive) -VO
P or B (Biderectionally Predictive)-VOP
Process as Each VOP input sequentially
Is processed as any of VOPs of I, P, and B,
It is predetermined (for example, I, B, P, B, P,.
.. Processed as B and P).

The motion vector detector 2 performs motion compensation on a macroblock to be processed with reference to a predetermined reference image (VOP), and detects a motion vector of the macroblock.

Here, the motion compensation (inter-frame prediction) has three types of prediction modes: forward prediction, backward prediction, and bidirectional prediction. In P-VOP, motion compensation is performed only in forward prediction only. The vector detector 2 detects a motion vector that minimizes the prediction error. Also, B-VOP
Are subjected to motion compensation in three types: forward prediction, backward prediction, and bidirectional prediction, and the motion vector detector 2 detects a motion vector that minimizes the prediction error in each prediction mode. Further, the motion vector detector 2 selects one of the three prediction modes in which the minimum prediction error is obtained,
The motion vector in the prediction mode is also selected.

Then, the motion vector detector 2 compares the prediction error obtained as a result of the motion compensation with the variance of the macroblock to be coded. As a result, if the variance of the macroblock is smaller, the interframe prediction is not performed on the macroblock and the intraframe coding is performed. In this case, the prediction mode is intra-picture coding (intra), and such a prediction mode is supplied from the motion vector detector 2 to the VLC unit 6 and the motion compensator 12. On the other hand, when the prediction error is smaller, the prediction mode and the motion vector from which the prediction error was obtained are supplied from the motion vector detector 2 to the VLC unit 6 and the motion compensator 12. Note that the prediction mode for the I-VOP is
It is always intra-coded.

Here, the VOP sequences to be encoded may have different sizes and positions. Therefore, when detecting a motion vector, it is necessary to set a reference coordinate system and detect the motion vector in the coordinate system. Therefore, here, a certain absolute coordinate is assumed, and a motion vector at the absolute coordinate is calculated. That is, the motion vector detector 2 is supplied with a flag FSZ indicating the magnitude of the VOP in the absolute coordinate system and a flag FPOS indicating the position. And VOP to be processed based on the flag FPOS
And a VOP serving as a reference image are arranged in an absolute coordinate system,
The motion vector of (the macroblock of) the VOP to be processed is calculated.

On the other hand, the motion compensator 12 generates a predicted image by performing motion compensation on the VOP stored in the frame memory 11 based on the motion vector from the motion vector detector 2 and the prediction mode. I do. This predicted image is supplied to the arithmetic unit 3. The arithmetic unit 3 is also supplied from the frame memory 1 with the macroblock to be encoded read from the frame memory 1 by the motion vector detector 2. Then, the arithmetic unit 3 calculates a pixel value of each pixel constituting the macroblock to be encoded,
The difference between the pixel values of the pixels constituting the predicted image is calculated, and the difference signal is output to the DCT unit 4. When the encoding target macroblock is an intra macroblock, the arithmetic unit 3 outputs the encoding target macroblock to the DCT unit 4 as it is.

In the DCT unit 4, the output of the arithmetic unit 3 is
The data is subjected to a CT (discrete cosine transform) process and converted into DCT coefficients. This DCT coefficient is input to the quantizer 5, quantized in a quantization step corresponding to the data storage amount (buffer storage amount) of the transmission buffer 7, and then input to the VLC (variable length coding) device 6. You.

The VLC unit 6 converts the image data supplied from the quantizer 5 into a variable length code such as a Huffman code, and converts the resulting coded bit stream into
Output to the transmission buffer 7.

The VLC unit 6 has a quantization step (scale) set by the quantizer 5 and a prediction mode (intra-picture prediction, forward prediction, backward prediction, or bidirectional prediction) set by the motion vector detector 2. Mode that indicates whether the
And a motion vector are generated by a key signal encoder 13 described later.
The encoding result of the key signal is supplied. Further, the VLC unit 6 has a flag FSZ.
And FPOS have also been made available. The VLC unit 6 inserts (arranges) these pieces of information and further the information stored in the buffer 16 into a header of a predetermined layer of the coded bit stream configured as shown in FIG. .

The VLC unit 6 outputs the information arranged in the header of each layer to the buffer 16, and the buffer 16 stores the information supplied from the VLC unit 6. ing.

The transmission buffer 7 temporarily stores the coded bit stream from the VLC unit 36 and outputs a quantization control signal corresponding to the stored amount to the quantizer 5. That is, the transmission buffer 7 supplies the quantization control signal for increasing the quantization scale to the quantizer 5 when the accumulated amount increases to the allowable upper limit value, and increases the quantization scale to thereby increase the quantization scale. 5 reduces the amount of data output. When the remaining storage amount decreases to the permissible lower limit, the transmission buffer 7 supplies a quantization control signal for reducing the quantization scale to the quantizer 5 to reduce the quantization scale, thereby performing quantization. The amount of data output from the device 5 is increased. In this manner, overflow and underflow of the transmission buffer 7 are prevented.

The coded bit stream stored in the transmission buffer 7 is read at a predetermined timing and supplied to a recording medium 201 such as a magnetic tape, a magnetic disk, a magneto-optical disk, and a phase change disk. Or recorded on an analog public network, ISDN,
The data is transmitted via a transmission medium 202 such as a satellite line, a CATV network, and a terrestrial wave. Thereby, the encoded bit stream is provided to the decoder of FIG. 16 described below via the recording medium 201 and the transmission medium 202.

Here, as described above, VO is a sequence of each object constituting the composite image when a sequence of the composite image exists, and VOP means VO at a certain time. That is, for example, if there is a composite image F3 composed by combining the images F1 and F2, an image in which the images F1 or F2 are arranged in time series is
Each is a VO, and the image F1 or F2 at a certain time is a VOP. Therefore, for example, if the image F1 is set as the background and the image F2 is set as the foreground,
To obtain the composite image F3, it is necessary to combine the images F1 and F2 using a key signal for extracting the image F2. That is, in order to obtain the composite image F3, a key signal for removing the image F2 is required.

For this reason, a key signal for removing each VOP is supplied to the key signal encoder 13. In the key signal encoder 13, the key signal supplied thereto is, for example, It is encoded by a predetermined method such as DPCM. The result of encoding the key signal is supplied to the VLC unit 6 and the key signal decoder 14.

The key signal decoder 14 decodes the encoded result of the key signal from the key signal encoder 13 and outputs the result to the motion vector detector 2, DCT unit 4, IDCT unit 9, motion compensator 12, and pixel replacement. The processing is performed in these blocks using the decoding result of the key signal as necessary.

As described above, the motion vector detector 2 includes:
The key signal locally decoded by the key signal decoder 14 is also supplied. The key signal is used when the motion vector detector 2 calculates a prediction error of a macroblock.

That is, since the VOP is an image of a certain object at a certain time, its shape is basically an arbitrary shape. In this case, the macroblock to be coded has an image (a pixel constituting the object) There may be areas where there is no. In such a case, the motion vector detector 2 calculates a prediction error by excluding a pixel in which no image exists in the macroblock to be encoded, that is, predicting a pixel in which an image exists. Using only the error, calculate the prediction error of the macroblock to be encoded,
A motion vector that minimizes the motion vector is detected. For each pixel in the macro block to be encoded, local decoding of the macro block to be encoded is performed in order to recognize whether an image exists. The key signal is referred to.

Specifically, in the motion vector detector 2,
A pixel having a key signal of 0 is recognized as a pixel that does not have an image and belongs to a region outside the object (image object). It is recognized that the pixel is in the area inside the (image object). Then, the motion vector detector 2 does not calculate the difference between the pixel whose key signal is 0 and the reference image for obtaining the predicted image.

When the VOP has a rectangular shape, the key signal always has a value other than 0 (binary).
The key (hard key) is 1, gray scale (gray sca
le) The key (soft key) is 1 to 255)
Therefore, the prediction error is calculated using all the pixels of the macroblock.

On the other hand, the data output from the quantizer 5 is also supplied to an inverse quantizer 8, which outputs the data corresponding to the quantization step supplied from the quantizer 5. And inversely quantized to obtain DCT coefficients. This DCT coefficient is input to an IDCT (inverse DCT) unit 9 and the inverse DCT
After being processed, it is supplied to the arithmetic unit 10.

When the prediction mode is any of forward prediction, backward prediction, and bidirectional prediction, the arithmetic unit 10
In addition to the output of the IDCT unit 9, a predicted image output by the motion compensator 12 is also supplied. The arithmetic unit 10 decodes the image by adding the predicted image output by the motion compensator 12 to the output of the IDCT unit 9, and supplies the decoded image to the pixel replacement unit 15.

When the prediction mode is intra-picture coding, the arithmetic unit 10 supplies the output of the IDCT unit 9 to the pixel replacing unit 15 as it is.

In the pixel replacement unit 15, the output of the arithmetic unit 10 is subjected to padding processing described later, and is supplied to the frame memory 11. In the frame memory 11, the output of the pixel replacement unit 15 is stored, and the stored value, that is, the decoded image is used for motion compensation by the motion compensator 12. Note that the frame memory 11 is also supplied with the flags FSZ and FPOS, and the frame memory 11 stores these flags FSZ and FPOS.

Next, referring to the flowchart of FIG. 13, padding (paddin) performed by the pixel replacement unit 15 of FIG.
g) The processing will be described.

In the padding process, first, in step S1, one of the pixels constituting the macro block supplied from the arithmetic unit 10 to the pixel replacement unit 15 is set as a target pixel, and the key signal for the target pixel is set to 0. Is determined. In step S1, when it is determined that the key signal for the target pixel is not 0, that is,
If the pixel of interest is one that forms the inside of the image object, the process proceeds to step S2, where the pixel replacement unit 15 performs no processing on the pixel of interest and outputs it to the frame memory 11 as it is. Proceed to S4.

Here, when the VOP to be encoded has a rectangular shape, as described above, the key signal always takes a value other than 0. Therefore, the pixel replacement unit 15 outputs all the pixels in the VOP. Is not processed and is output as it is.

On the other hand, if it is determined in step S1 that the key signal for the pixel of interest is 0, that is,
If the pixel of interest constitutes the outside of the image object, the process proceeds to step S3, where the pixel value of the pixel of interest is
For example, it is set to 0, and the process proceeds to step S4. In step S4, it is determined whether or not the processing has been performed for all the pixels constituting the macroblock from the arithmetic unit 10.
If it is determined that the processing has not been performed for all the pixels, the process returns to step S1, and the same processing is repeated with a pixel that has not been set as the target pixel as a new target pixel.

In step S4, the operation unit 10
If it is determined that the processing has been performed for all the pixels from, the process proceeds to step S5, the horizontal line including the macro block from the arithmetic unit 10 is selected as the horizontal line of interest, and the process proceeds to step S6. In step S6, the pixel values of the pixels at both ends of the horizontal line of interest are determined.

That is, when attention is paid to a certain horizontal line of the macroblock after the processing of steps S1 to S4, the pixel values at both ends of the horizontal line of interest are all 0. (The case where the pixels at both ends are outside the image object), the case where the pixel value at one end is not 0 (the case where only the pixel at one end is inside the image object), and the case where the pixel values at both ends are not 0 (Cases where the pixels at both ends are inside the image object) occur in three cases. Step S
In 6, it is determined to which of these three cases the horizontal line of interest belongs.

In step S6, when it is determined that the pixel values at both ends of the horizontal line of interest are both 0,
The process proceeds to step S7, where 0 is set to the variable C secured for the horizontal line of interest, and the process proceeds to step S10. If it is determined in step S6 that the pixel values at both ends of the horizontal line of interest are not 0, the process proceeds to step S8, and the variable C secured for the horizontal line of interest is added to the pixels at both ends of the horizontal line of interest. The average value is set, and the process proceeds to step S10. Further, if it is determined in step S6 that only one of the pixel values at both ends of the horizontal line of interest is not 0, the process proceeds to step S9, and the variable C secured for the horizontal line of interest is added to the variable C of the horizontal line of interest. 0 of pixel values at both ends of
Is set, and the process proceeds to step S10.

In step S10, it is determined whether or not all horizontal lines of the macroblock from the arithmetic unit 10 have been processed as the horizontal line of interest. If all horizontal lines have not yet been processed as the horizontal line of interest. If determined, the process returns to step S5, and a horizontal line that has not been selected as the horizontal line of interest is selected as a new horizontal line of interest, and the same processing is repeated thereafter.

If it is determined in step S10 that all horizontal lines have been processed as the horizontal line of interest, the process proceeds to step S11.

In steps S11 to S16,
Steps S5 to S10 are performed on the vertical line, not the horizontal line, of the macro block from the arithmetic unit 10.
The same processing is performed as in the case of.

That is, in step S11, a vertical line having a macroblock from the arithmetic unit 10 is selected as a target vertical line, and the flow advances to step S12. Step S
In 12, the pixel values of the pixels at both ends of the vertical line of interest are determined.

That is, even if a certain vertical line of the macroblock after the processing of steps S1 to S4 is focused on, the pixel value at both ends of the focused vertical line is zero. (The case where the pixels at both ends are outside the image object), the case where the pixel value at one end is not 0 (the case where only the pixel at one end is inside the image object), and the case where the pixel values at both ends are not 0 (Cases where the pixels at both ends are inside the image object) occur in three cases. Step S
In 12, it is determined to which of these three cases the vertical line of interest belongs.

If it is determined in step S12 that the pixel values at both ends of the vertical line of interest are all 0, the process proceeds to step S13, where 0 is set to a variable B secured for the vertical line of interest. Step S1
Proceed to 6. If it is determined in step S12 that the pixel values at both ends of the vertical line of interest are not 0, the process proceeds to step S14, and the variables B secured for the vertical line of interest are added to the pixels at both ends of the vertical line of interest. The average value is set, and the process proceeds to step S16. Further, if it is determined in step S12 that only one of the pixel values at both ends of the target vertical line is not 0, the process proceeds to step S15, and the variable B secured for the target vertical line is added to the variable B for the target vertical line. Is set to the non-zero value of the pixel values at both ends of
Proceed to 16.

In step S16, it is determined whether or not all vertical lines of the macro block from the arithmetic unit 10 have been processed as the target vertical line. If it is determined, the process returns to step S11, and a vertical line that has not yet been selected as the target vertical line is selected as a new target vertical line, and the same processing is repeated thereafter.

If it is determined in step S16 that all vertical lines have been processed as the target vertical line, the process proceeds to step S17, and among the pixels constituting the macroblock from the arithmetic unit 10, the process proceeds to step S2. A certain pixel is selected as a target pixel from among the pixels except for the pixel output to the frame memory 11, and the process proceeds to step S18.

In step S18, the value of the set (B, C) of the variables B and C for each of the vertical and horizontal lines intersecting on the target pixel is determined.

In step S18, when the variable B is 0,
If it is determined that C is not 0, the process proceeds to step S19, where the value of the variable C is output to the frame memory 11 as the pixel value of the target pixel, and the process proceeds to step S23. Also,
In step S18, both variables B and C are set to 0
If it is determined that it is not, the process proceeds to step S20, where the average value of the variables B and C is output to the frame memory 11 as the pixel value of the target pixel, and the process proceeds to step S23. Further, when it is determined in step S18 that both the variables B and C are 0, the process proceeds to step S21, where the pixel value of the target pixel is kept at 0, and the process proceeds to step S2.
Proceed to 3.

On the other hand, if it is determined in step S18 that the variable B is not 0 and the variable C is 0, the process proceeds to step S18.
Proceeding to 19, the value of the variable B is set as the pixel value of the target pixel,
The data is output to the frame memory 11, and the process proceeds to step S23. In step S23, it is determined whether or not all of the pixels constituting the macroblock from the computing unit 10 have been processed except for the pixels output directly in step S2, and if it is determined that the processing has not been performed, Returning to step S17, a pixel that has not yet been set as the target pixel is newly selected as the target pixel, and the same processing is repeated thereafter.

In step S23, the operation unit 1
If it is determined that all the pixels constituting the macroblock starting from 0 have been processed except for the pixels output as they are in step S2, the process proceeds to step S24, and among the pixels already output to the frame memory 11, The pixel closest to each pixel that has not yet been output to the frame memory 11 (hereinafter, appropriately referred to as an unoutput pixel) is detected. Further, in step S24, the pixel value of the detected pixel is output to the frame memory 11 as a pixel value of a non-output pixel, and the padding process ends.
If two or more pixels that have been output to the frame memory 11 and are closest to the non-output pixel are detected, the average value of those pixel values is calculated as the pixel value of the non-output pixel. Is output as

By performing the above-described padding processing, pixels constituting the outside of the image object are interpolated, so to speak, so that mosquito noise can be reduced and motion compensation can be made more efficient.

Next, the processing of the VLC unit 6 (encoding means) in FIG. 1 will be further described.

The VLC unit 6 includes VS, VISO, VO, V
The information to be originally arranged is arranged in the header of each of the OL, GOV, and VOP, and the variable-length encoding result of the output of the quantizer 5 is arranged to form an encoded bit stream. Output to

The VLC unit 6 outputs information stored in the headers of VS, VISO, VO, and VOL, which are higher layers than GOV, to the buffer 16 for storage.

After that, when outputting the GOV header, the VLC unit 6 reads out the information of each header of VS, VISO, VO, and VOL of the hierarchy higher than GOV stored in the buffer 16, and reads out the predetermined information of the GOV header. Insert (include) at the position of and output. Therefore, in this case, the GOV header includes VS, VI in addition to the information to be originally arranged there.
Information of each header of SO, VO, and VOL is also arranged.

FIG. 14 shows a VLC performing the above processing.
3 shows the syntax of the GOV output from the device 6. Note that the shaded portions in FIG. 14 are different from the syntax in the FCD shown in FIG.

Group_VOP_start_code is a unique 32-bit code indicating the start position of the GOV. time_code
The (time information) is formed of 18 bits, and represents the display time of the VOP displayed first with a second precision in the GOV. this
time_code is `` time and control codes for video tape reco specified in IEC standardpublication 461.
rders ".

For closed_gop and broken_link, refer to the MPEG4VideoFCD standard (ISO / IEC 14496-2).

Is_extension (header information presence / absence flag)
Is a 1-bit flag introduced in the present embodiment.
Indicates whether the header includes information for initializing the decoder and other information of each header of VS, VISO, VO, and VOL. In the VLC unit 6, for example, when the flag is_extension is 1, VS, VISO, VO,
Information of each header of VOL (VisualObjectSequence (), Vi
sualObject (), VideoObject (), VideoObjectLayer ())
Is included in the GOV header. That is, the flag is_extensio
When n is 1, information of each header of VS, VISO, VO, and VOL is group_VOP_start_code, time_code, close
It is placed after d_gop, broken_link, and is_extension.

Further, when the flag is_extension is 1, the VLC unit 6 includes the information of each header of VS, VISO, VO, and VOL in the GOV header, and then includes the V
The information of each header of S, VISO, VO, and VOL is supplied to the buffer 16 and stored in place of the information stored up to now.

Note that the VLC unit 6 has VS, VISO, V
When the O and VOL headers are subsequently output, the information of the headers is supplied to the buffer 16 and stored.

Therefore, the buffer 16 always has the latest V
This means that the header information of S, VISO, VO, and VOL is stored.

Here, when the flag is_extension is 1, information of each header of VS, VISO, VO, and VOL is included in the GOV header, and the included VS and VIS are included.
The information of each header of O, VO, and VOL is supplied to and stored in the buffer 16 for the following reason.

That is, the VLC unit 6 includes VS and VIS included in the GOV header in order to improve the coding efficiency.
The information of each header of O, VO, and VOL can be changed. In this case, since the information after the change is the latest information, the VS and VI included in the GOV header are stored in the buffer 16 in order to store the latest information.
The information of each header of SO, VO and VOL is stored in the buffer 16.
To be stored in the memory.

On the other hand, if the flag is_extension is 0, V
In the LC unit 6, the information of each header of VS, VISO, VO, and VOL is not included in the GOV header.

The stored value in the buffer 16 can be changed from outside. That is, VS,
In some cases, it is desired to change part or all of the information of each header of VISO, VO, and VOL in the middle of an encoded bit stream. That is, for example, there is a case where it is desired to change the quantization matrix used for decoding during the decoding of the encoded bit stream. In such a case, the user:
VS, VISO, VO, stored in the buffer 16
The information of each header of the VOL can be appropriately changed to desired information. The information after this change is the flag is_exten
Since the output is placed in the GOV header whose sion is 1, the decoder performs decoding based on the changed information after receiving the GOV header.

Next, the processing of the VLC unit 6 for outputting the GOV of the syntax as shown in FIG. 14 will be described with reference to the flowchart of FIG.

As described above, the VLC unit 6 outputs VS, V
The information to be laid out is arranged in the header of each of ISO, VO, VOL, GOV, and VOP, and the result of variable-length encoding of the output of the quantizer 5 is arranged to form an encoded bit stream. , To the transmission buffer 7.

Further, the VLC unit 6 includes VS, VISO,
Each time the VO and VOL headers are output, the information arranged in each header is output to the buffer 16 and stored (overwritten).

When outputting the GOV header, the VLC unit 6 determines in step S31 that the GOV header
It is determined whether the flag is_extension for the header is 1. In step S1, the flag is_exten
If it is determined that sion is not 1 (it is 0), VLC
The device 6 arranges information to be originally arranged (information shown in FIG. 8) and a flag is_extension in the GOV header (V
The information of each header of S, VISO, VO, and VOL is not arranged), and the resulting GOV header is output. Then, the process waits until the next GOV header is output, and returns to step S31.

On the other hand, in step S31, the flag is
If it is determined that _extension is 1, step S3
2 to VS, VIS stored in the buffer 16
Read the latest information of each header of O, VO, VOL,
The latest information, the flag is_extension, and the information to be originally arranged are arranged and output in the GOV header.
Then, the process waits until the next GOV header is output, and returns to step S31.

The flag is_ext arranged in each GOV
The value of the extension is set in the VLC device in advance, for example, on the administrator side of the encoder.

FIG. 16 shows a configuration example of an embodiment of a decoder for decoding an encoded bit stream provided via the recording medium 201 or the transmission medium 202. Buffer 21, IVLC constituting this decoder
Unit 22, inverse quantizer 23, IDCT unit 24, arithmetic unit 2
5. The frame memory 26 and the motion compensator 27 are a buffer 101, an IVLC unit 102, an inverse quantizer 103, an IDCT unit 104, an arithmetic unit 105, a frame memory 106, and a motion compensator 107 which constitute the decoder shown in FIG. Respectively. Therefore, in each of the buffer 21 to the motion compensator 27, the same processing as the processing of each of the buffer 101 to the motion compensator 107 may be performed, and the description of such the same processing will be appropriately omitted.

The coded bit stream provided via the recording medium 201 or the transmission medium 202 is received by the reception buffer 21 (receiving means) and is temporarily stored. Then, the encoded bit stream stored in the reception buffer 21 is read out by an IVLC (variable length decoding) unit 22 as appropriate.

The IVLC unit 22 (decoding means) performs variable-length decoding on the coded bit stream read from the reception buffer 21, sends the motion vector and the prediction mode to the motion compensator 27, and inversely quantizes the quantization step. In the vessel 23,
In addition to outputting the image data, the image data (the quantized DCT coefficient) subjected to the variable length decoding is output to the inverse quantizer 23. Note that the IVLC unit 22 also includes information necessary for initializing parameters used for decoding by the decoder, which is included in the header of each layer, and other information (for example, whether to perform overlap motion compensation. To the necessary blocks (for example, a flag indicating whether or not to perform overlap motion compensation) to the motion compensator 27, and a quantization matrix to the inverse quantizer 23. , Each supplied)

Further, the IVLC unit 22 decodes the flag is_extension for the GOV header, and
If xtension is 1, ie, the GOV header
When the information of each header of S, VISO, VO, and VOL is included, the information is also included in VS, VISO, VO, and VO.
Variable-length decoding is performed in the same manner as each header of L, and the decoding result is
Supply necessary blocks. Specifically, for example, a motion vector, a prediction mode, a flag indicating whether to perform overlap motion compensation, and the like are provided to the motion compensator 27, and a quantization step and a quantization matrix are provided to the inverse quantizer 23.
, Respectively.

The IVLC unit 22 decodes the flags FSZ and FPOS included in the encoded bit stream, and supplies them to the frame memory 26, the motion compensator 27, and the key signal decoder 29. Further, the IVLC unit 22 extracts an encoded key signal (key signal bit stream) included in the encoded bit stream, and supplies the extracted key signal to the key signal decoder 29.

The key signal decoder 29 decodes the key signal bit stream supplied from the IVLC unit 22. The decoded key signal is sent to the IDCT unit 24 and the motion compensator 2
7 and the pixel replacement unit 28.

The inverse quantizer 23 inversely quantizes the image data supplied from the IVLC unit 22 in accordance with the quantization step also supplied from the IVLC unit 22, and the IDCT unit 24
Output to The IDCT unit 24 performs an inverse DCT process on the data (DCT coefficient) output from the inverse quantizer 23, and supplies the data to the arithmetic unit 25.

When the image data supplied from the IDCT unit 24 is I-VOP data, the arithmetic unit 25 converts the data into image data (P or B-
In order to generate a predicted image of VOP data), it is supplied to the frame memory 26 via the pixel replacement unit 28 and stored as it is.

Note that the pixel replacement unit 28 performs the same processing as the pixel replacement unit 15 in FIG.

On the other hand, the data supplied to the arithmetic unit 25 is
If the data is P or B-VOP data, the motion compensator 2
Reference numeral 7 reads a previously decoded image stored in the frame memory 26 in accordance with the motion vector and the prediction mode supplied from the IVLC unit 22 to generate a predicted image and output it to the calculator 25. In computing unit 25, ID
Image data (difference data) supplied from the CT unit 24,
The predicted image data supplied from the motion compensator 27 is added to obtain a decoded image. This decoded image is output to the pixel replacement unit 28
Is supplied to and stored in the frame memory 26 via the CPU, and is appropriately used as a reference image (image referred to for generating a predicted image) of an image to be decoded later. The decoded image stored in the frame memory 26 is not only used as a reference image as described above, but is also appropriately read and supplied to, for example, a display (not shown) and displayed.

Next, the processing performed by the IVLC unit 22 of FIG. 16 on the GOV header will be further described with reference to the flowchart of FIG.

When the IVLC unit 22 receives the GOV header, the IVLC unit 22 performs a process to be performed normally (a process to be performed when the GOV header shown in FIG. 8 is transmitted) with respect to the GOV header. , The flag is located in the GOV header (FIG. 14)
Determine whether _extension is 1 or not. Step S
41, if it is determined that the flag is_extension is not 1 (it is 0), that is, VS,
If the information of each header of VISO, VO, and VOL is not included, the process returns to step S41 after waiting for the next GOV header to be transmitted.

In step S41, the flag is
If it is determined that _extension is 1, ie, GOV
If the header includes the information of each header of VS, VISO, VO, and VOL, the process proceeds to step S42,
The LC unit 22 supplies the information of each header of the VS, VISO, VO, and VOL to a necessary block, and outputs the next GOV.
After the header is transmitted, the process returns to step S41.

Next, in addition to the syntax shown in FIG. 14, the GOV can be configured, for example, like the syntax shown in FIG.

That is, FIG. 18 shows another example of the syntax of GOV. In FIGS. 14 and 18, brok
The distance from the bottom of en_link to the top of next_start_code is different. Also, the shaded portions in FIG.
It is different from the syntax in the FCD shown in FIG.

In the embodiment shown in FIG. 14, the flag is_exten
With sion, it was possible to set only whether or not to transmit the information of each header of VS, VISO, VO, and VOL in the GOV header. However, in the embodiment of FIG. 18, by adopting the flag load_data_type, , VIS
It is possible to set whether or not to transmit all the information of each header of O, VO, and VOL in the GOV header as well as to transmit only a part of the information. That is, in the embodiment of FIG. 18, VS, VISO,
Only part of the information of each header of VO and VOL can be included in the GOV header, and the flag load_data_ty
According to pe, a part of information included in the GOV header can be identified.

Specifically, in FIG. 18, load_data_
The type is a variable-length code, and indicates immediately after whether or not information for initializing a decoder at the time of random access is to be transmitted and, if so, the type of information to be transmitted. That is, for example, as shown in FIG.
When the type is “1”, the information of the header in the higher layer than the GOV layer is not included in the GOV. Also, load_data_
When the type is '01', VS and VI are set in the same manner as in the case where the flag is_extension is 1 in the embodiment of FIG.
SO, VO, VOL header information (VisualObjectSe
quence (), VisualObject (), VideoObject (), VideoObje
ctLayer ()) are all included in the GOV. Further, when load_data_type is '001', VS, VIS
Among the information of the O, VO, and VOL headers, information of predetermined parameters is included in the GOV.

That is, in the embodiment of FIG.
Information included in the GOV when _data_type is '001' is defined as download_parameters ().

Here, in the present embodiment, download_par
ameters () is defined, for example, as shown in FIG.

In FIG. 20, the flag obmc_disable is
This is a 1-bit flag indicating whether to use overlap motion compensation. When this value is “1”, overlap motion compensation is not used, and when this value is “0”, overlap motion compensation is used. Flag quan
t_type is a 1-bit flag indicating a method of inverse quantization. If this value is '0', inverse quantization is performed using the inverse quantization method specified in H.263, and if '1', the inverse quantization specified in MPEG2 is used. Inverse quantization is performed using a quantization method. When the inverse quantization method specified in MPEG2 is used, a flag indicating whether to download the quantization matrix is further transmitted.
Also, when downloading the quantization matrix,
The quantization matrix to be downloaded is also transmitted.

In addition, download_parameters () in FIG.
Load_intra_quant_mat, intra_ specified in
quant_mat, load_nonintra_quant_mat, nonintra_quant
_mat, load_intra_quant_mat_grayscale, iontra_quant_
mat_grayscale, load_nonintra_quant_mat_grayscale,
The semantics of nonintra_quant_mat_grayscale is F
This is the same as the semantics of the flag of the same name defined in the VOL (FIGS. 5 to 7) on the CD.

In the embodiment shown in FIG. 19, the flag lo
Although only three cases are defined for ad_data_type, it is also possible to define four or more cases.
In this case, information of a combination different from the combination of information defined by download_parameters () in FIG.
It can be arranged in the OV header.

FIG. 21 shows a configuration example of an embodiment of an encoder that outputs the GOV header shown in FIG. In the figure, the same reference numerals are given to portions corresponding to the case in FIG. That is, the encoder of FIG. 21 has the same configuration as that of FIG. 12 except that a parser 17 (selection means) is newly provided.

The parser (flag discriminator) 17 refers to the flag load_data_type for the GOV header that the VLC unit 6 is going to output, and the flag load_data_type
, The information is read from the buffer 16 and VL
It is supplied to the C unit 6. In the VLC device 6, the information supplied from the parser 17 together with the flag load_data_type together with the GO
It is arranged and output at the predetermined position shown in FIG. 18 of the V header.

Next, the processing of the parser 17 for outputting the GOV of the syntax as shown in FIG. 18 to the VLC unit 6 will be described with reference to the flowchart of FIG.

At the timing when the GOV header is output, the VLC unit 6 sets the flag load_d for the GOV header.
The ata_type is supplied to the parser 17. The parser 17
The flag load_data_type is received from the LC unit 6, and the value is determined in step S51. Step S51
, When it is determined that the flag load_data_type is 1, the parser 17 does not output anything to the VLC unit 6, and the flag load_data_type placed in the next GOV header is output.
After waiting for the type to be transmitted from the VLC device 6, the process returns to step S51. In this case, in the VLC device 6, GO
Information to be originally placed and load_data_type in the V header
And outputs the resulting GOV header.

In step S51, the flag lo
When it is determined that the ad_data_type is 01, the process proceeds to step S52, where the parser 17
The latest information of each header of S, VISO, VO, and VOL is read and supplied to the VLC unit 6. And the next GOV
After waiting for the flag load_data_type arranged in the header to be transmitted from the VLC device 6, the process returns to step S51. Therefore, in this case, in the VLC device 6, in addition to the information to be originally arranged and the load_data_type,
Information of each header of VS, VISO, VO, and VOL is also arranged.

On the other hand, in step S51, the flag lo
When it is determined that the ad_data_type is 001, the process proceeds to step S53, and the parser 17 determines, among the information stored in the buffer 16, the download_paramet shown in FIG.
Select and read out what is included in ers (), and VLC unit 6
To supply. Then, the process returns to step S51 after waiting for the flag load_data_type arranged in the next GOV header to be transmitted from the VLC device 6. Therefore, in this case, in the VLC device 6, in addition to the information to be originally arranged and the load_data_type, the down stream shown in FIG.
load_parameters () is also placed.

Next, the coded bit stream provided from the encoder of FIG. 21 via the recording medium 201 or the transmission medium 202 can be decoded by the decoder having the configuration shown in FIG.

FIG. 23 shows that the IVLC unit 22 of the decoder having the configuration shown in FIG.
It is a flowchart for demonstrating the process performed about a V header.

Upon receiving the GOV header, the IVLC unit 22 performs a process to be performed normally (a process to be performed when the GOV header shown in FIG. 8 is transmitted) with respect to the GOV header. , Flag lo located in the GOV header (FIG. 18)
Determine the value of ad_data_type. If it is determined in step S61 that the flag load_data_type is 1, the process returns to step S61 after waiting for the next GOV header to be transmitted.

In step S61, the flag lo
If ad_data_type is determined to be 01, ie, G
If the OV header includes the information of each header of VS, VISO, VO, and VOL, the process proceeds to step S62,
The IVLC unit 22 determines, based on the flag load_data_type,
The information of each of the VS, VISO, VO, and VOL headers is extracted from the encoded bit stream and supplied to necessary blocks. That is, the information is subjected to variable-length decoding, and the resulting, for example, motion vector, prediction mode, and a flag indicating whether or not to perform overlap motion compensation are sent to the motion compensator 27, The quantization matrices are supplied to the inverse quantizers 23, respectively. Then, the process returns to step S61 after waiting for the next GOV header to be transmitted.

On the other hand, in step S61, the flag lo
When it is determined that ad_data_type is 001, that is,
If the GOV header includes download_parameters () (parameter update information), the process proceeds to step S63, where the IVLC unit 22 extracts the download_parameters () from the encoded bit stream based on the flag load_data_type, and Supply to the new block. That is,
The download_parameters () is subjected to variable-length decoding, and the resulting flag, for example, indicating whether or not to perform overlap motion compensation, is supplied to the motion compensator 27, and the quantization step and the quantization matrix are dequantized. Table 23
Respectively. Then, the process returns to step S61 after waiting for the next GOV header to be transmitted.

As described above, in the GOV header, all or part of the information of the headers of the VS, VISO, VO, and VOL in the upper layer (download_parameters () shown in FIG. 20 in this embodiment) Is included, so that random decoding or the like can be performed on the encoded bit stream, and normal decoding can be performed midway. Furthermore, it is possible to change the quantization step and the quantization matrix at the beginning of the GOV, and as a result,
Efficient encoding can be performed.

The case where the present invention is applied to an encoder / decoder that performs encoding / decoding based on MPEG4 has been described above.
However, the present invention is not limited to encoding / decoding based on.

In the present embodiment, download_param
Although the information shown in FIG. 20 is included in the GOV header as eters (), the information included in the GOV header as download_parameters () is not limited to the information shown in FIG. 20.

Further, the encoder shown in FIGS. 12 and 21 and the decoder shown in FIG. 16 can be realized by hardware, or can be realized by causing a computer or the like to execute a program. Is also possible.

In MPEG4, hierarchical coding for realizing scalability is possible, but the present invention is applicable regardless of whether or not hierarchical coding is performed.

[0166]

As described above, according to the image encoding apparatus and the image encoding method of the present invention, the upper layer header is added to the lower layer header in the encoded bit stream obtained by encoding the image. Header information is included. Therefore, efficient encoding becomes possible.

Further, according to the image decoding apparatus and the image decoding method of the present invention, information included in a lower layer header is extracted from an encoded bit stream including a lower layer header including information of an upper layer header. The encoded bit stream is decoded based on the information. Therefore,
Normal decoding can be performed even in the middle of the encoded bit stream.

Further, according to the providing medium of the present invention, an encoded bit stream obtained by encoding an image and including information of an upper layer header in a lower layer header is provided. Therefore, normal decoding can be performed even in the middle of the encoded bit stream.

[Brief description of the drawings]

FIG. 1 is a diagram illustrating a configuration of an encoded bit stream defined by the MPEG4 standard FCD.

FIG. 2 is a diagram illustrating the syntax of VS defined by the MPEG4 standard FCD.

FIG. 3 is a diagram showing the syntax of VISO defined by the MPEG4 standard FCD.

FIG. 4 is a diagram illustrating the syntax of a VO defined by the MPEG4 standard FCD.

FIG. 5 is a diagram showing the syntax of a VOL defined by the MPEG4 standard FCD.

FIG. 6 is a diagram illustrating the syntax of a VOL defined by the MPEG4 standard FCD.

FIG. 7 is a diagram showing the syntax of a VOL defined by the MPEG4 standard FCD.

FIG. 8 is a diagram showing the syntax of GOV defined by the MPEG4 standard FCD.

FIG. 9 is a diagram illustrating the syntax of a VOP defined by the MPEG4 standard FCD.

FIG. 10 is a diagram illustrating the syntax of a VOP defined by the MPEG4 standard FCD.

FIG. 11 is a diagram illustrating the syntax of a VOP defined by the MPEG4 standard FCD.

FIG. 12 is a block diagram illustrating a configuration example of an embodiment of an encoder to which the present invention has been applied.

FIG. 13 is a flowchart for explaining processing of the pixel replacement unit 15 in FIG. 12;

14 is a diagram illustrating the syntax of a GOV output by the VLC device 6 in FIG.

FIG. 15 is a flowchart for explaining processing of the VLC device 6 in FIG. 12;

FIG. 16 is a block diagram illustrating a configuration example of an embodiment of a decoder to which the present invention has been applied.

FIG. 17 is a flowchart for explaining processing of the IVLC unit 22 in FIG. 16;

18 is a diagram illustrating the syntax of a GOV output from the VLC device 6 in FIG. 21.

FIG. 19 is a diagram for describing load_data_type.

20 is a diagram illustrating the syntax of download_parameters () in FIG.

FIG. 21 is a block diagram illustrating a configuration example of another embodiment of an encoder to which the present invention has been applied.

FIG. 22 is a flowchart for explaining processing of the parser 17 of FIG. 21;

FIG. 23 is a flowchart for explaining processing of the IVLC unit 22 in FIG. 16;

FIG. 24 is a block diagram illustrating a configuration of an example of a conventional encoder.

FIG. 25 is a block diagram showing a configuration of an example of a conventional decoder.

[Explanation of symbols]

1 frame memory (receiving means), 2 motion vector detector, 3 operation unit, 4 DCT unit, 5 quantizer, 6 VLC unit (encoding means), 7 buffer,
8 inverse quantizer, 9 IDCT unit, 10 operation unit, 11 frame memory, 12 motion compensator,
13 key signal encoder, 14 key signal decoder,
15 pixel replacement unit, 16 buffers, 17 parser (selection unit), 21 buffers (reception unit), 22
IVLC unit (decoding means), 23 inverse quantizer, 24
IDCT unit, 25 arithmetic unit, 26 frame memory, 27 motion compensator, 28 pixel replacement unit, 29
Key signal decoder, 201 recording medium, 202 transmission medium

 ──────────────────────────────────────────────────続 き Continued on front page F term (reference) 5C053 FA21 FA23 FA27 GA11 GB19 GB21 GB26 GB29 GB32 GB38 KA04 LA14 5C059 KK03 MA00 MA04 MA05 MA23 MA31 MB01 MB11 MB22 MB27 MC11 MC14 MC38 ME02 NN01 NN28 PP05 PP06 PP07 RC04 RC24 RC38 SS01 SS07 SS11 UA02 UA05 UA33 UA34 UA39

Claims (23)

[Claims] 1. An image encoding apparatus that encodes an image and outputs an encoded bit stream having a hierarchical structure including a plurality of layers, comprising: a receiving unit that receives the image; And an encoding unit that outputs the encoded bit stream including the information of the header of the upper layer in the header of the image encoding apparatus. 2. The encoding unit according to claim 1, wherein the header of the lower layer includes information necessary for initializing a parameter for decoding the encoded bit stream, among information of the header of the upper layer. The image encoding device according to claim 1, wherein: 3. The encoding means arranges, in the lower layer header, a header information presence / absence flag indicating whether or not to include the information of the upper layer header, and the header information presence / absence flag is 2. The image encoding apparatus according to claim 1, wherein the information of the header of the upper layer is included in the header of the lower layer only when indicating that the information of the header of the layer is included. 4. The encoding means arranges, in the lower layer header, an identification flag for identifying information of the higher layer header included therein, and includes the lower layer header in the lower layer header. 2. The image encoding apparatus according to claim 1, further comprising a selection unit that selects information of a header of an upper layer according to the identification flag. 5. The lower hierarchy is a hierarchy for defining a group composed of one or more images, and its header contains time information on the display time of the image displayed first in the group. When included, the encoding unit arranges information of the header of the upper layer after the time information.
An image encoding device according to claim 1. 6. The image is a moving picture (MPEG).
When encoding is performed by a method conforming to the standards of Experts Group (GOV) 4, the encoding means may include a GOV (Group of VOP (Video Objec).
t Plane)) In the header of the hierarchy, VS (Visual Object Sequ
ence) hierarchy, VISO (Visual Object) hierarchy, VO (V
video object layer or VOL (video object layer)
The image encoding device according to claim 1, wherein r) includes information of one or more headers in a hierarchy. 7. The GOV hierarchy header includes a flag indicating whether to use overlap motion compensation, a flag indicating a method of inverse quantization, or a quantization matrix. The image encoding device according to claim 6. 8. An image encoding method for encoding an image and outputting an encoded bit stream having a hierarchical structure including a plurality of layers, comprising: receiving the image; encoding the image; Outputting the coded bit stream including information of a header of an upper layer. 9. Obtained by encoding an image,
An image decoding device that decodes an encoded bit stream having a hierarchical structure including a plurality of layers, a receiving unit that receives the encoded bit stream including information of an upper layer header in a lower layer header, An image decoding apparatus, comprising: decoding means for extracting information included in a lower layer header and decoding the encoded bit stream based on the information. 10. The header of the lower layer includes information of the header of the upper layer necessary for initializing a parameter for decoding the coded bit stream. Item 10. The image decoding device according to Item 9. 11. The header of the lower layer includes a header information presence / absence flag indicating whether or not to include information of the header of the upper layer. When indicating to include header information,
The image decoding apparatus according to claim 9, wherein information of the header of the upper layer included in the header of the lower layer is extracted. 12. The lower layer header includes an identification flag for identifying information of the upper layer header included therein, and the decoding unit determines the lower layer header based on the identification flag. 10. The image decoding apparatus according to claim 9, wherein information of the header of the upper layer included in the header is extracted. 13. The lower hierarchy is a hierarchy for defining a group composed of one or more images, and its header includes time information on the display time of the image displayed first in the group. The image decoding device according to claim 9, wherein when included, the information of the header of the upper layer is arranged after the time information. 14. The encoded bit stream may be MP
In the case where the image is encoded by a method based on the standard of EG (Moving Picture Experts Group) 4, the header of the GOV (Group of VOP (Video Object Plane)) layer includes a VS (Visual Object Sequence) layer. , VIS
The image decoding apparatus according to claim 9, wherein information of one or more headers of an O (Visual Object) layer, a VO (Video Object) layer, or a VOL (Video Object Layer) layer is included. 15. The method according to claim 14, wherein the GOV layer header includes a flag indicating whether to use overlap motion compensation, a flag indicating an inverse quantization method, or a quantization matrix. An image decoding apparatus according to claim 1. 16. An image decoding method for decoding an encoded bit stream having a hierarchical structure composed of a plurality of layers, which is obtained by encoding an image, wherein information of an upper layer header is added to a lower layer header. An image decoding method, comprising: receiving the encoded bitstream including the extracted bitstream; extracting information included in the header of the lower layer; and decoding the encoded bitstream based on the information. 17. A providing medium for providing a coded bit stream having a hierarchical structure composed of a plurality of layers, obtained by coding an image, wherein the image is coded and a header of a lower layer is provided in a header of a lower layer. A providing medium for providing the coded bit stream obtained by including information of a header of a layer. 18. The header of the lower layer includes information of the header of the upper layer necessary for initializing parameters for decoding the coded bit stream. Item 18. The providing medium according to Item 17. 19. The providing medium according to claim 17, wherein the lower layer header includes a header information presence / absence flag indicating whether to include information of the upper layer header. 20. The providing medium according to claim 17, wherein the lower layer header includes an identification flag for identifying information of the upper layer header included therein. 21. The lower hierarchy is a hierarchy for defining a group composed of one or more images, and its header contains time information on the display time of the image displayed first in the group. 18. The providing medium according to claim 17, wherein when included, the information of the header of the upper layer is arranged after the time information. 22. The coded bit stream has an MP
In the case where the image is encoded by a method based on the standard of EG (Moving Picture Experts Group) 4, the header of the GOV (Group of VOP (Video Object Plane)) layer includes a VS (Visual Object Sequence) layer. , VIS
18. The providing medium according to claim 17, wherein information of one or more headers of an O (Visual Object) layer, a VO (Video Object) layer, or a VOL (Video Object Layer) layer is included. 23. The method according to claim 22, wherein the GOV layer header includes a flag indicating whether to use overlap motion compensation, a flag indicating an inverse quantization method, or a quantization matrix. Provided medium as described.