JP3380981B2 - Image encoding device and image encoding method, image decoding device and image decoding method, and recording medium - Google Patents

Description

Detailed Description of the Invention

[0001]

TECHNICAL FIELD The present invention relates to an image coding apparatus and an image coding method, an image decoding apparatus and an image decoding method, and a recording medium . In particular, for example, moving image data is recorded on a recording medium such as a magneto-optical disk or a magnetic tape and is reproduced and displayed on a display or the like, or a video conference system, a video telephone system, broadcasting equipment, a multimedia database. Suitable for transmitting moving image data from the transmitting side to the receiving side via a transmission line, such as a search system, and for receiving and displaying this at the receiving side, or for editing and recording. Image encoding device and image encoding method, image decoding device and image decoding method, and recording medium .

[0002]

2. Description of the Related Art In a system for transmitting moving image data to a remote place, such as a video conference system or a video telephone system, for example, the image data is line-correlated or framed in order to use the transmission line efficiently. The compression coding is performed by using the inter-correlation.

A typical example of a high-efficiency coding method for moving images is MPEG (Moving Picture Experts Group).
There is a (video encoding for storage) system. This is ISO-I
Discussed in EC / JTC1 / SC2 / WG11,
It has been proposed as a standard proposal, and a hybrid method in which motion compensation predictive coding and DCT (Discrete Cosine Transform) coding are combined is adopted.

In MPEG, several profiles and levels are defined in order to support various applications and functions. The most basic is the main profile main level (MP @ ML (Main Profile
at Main Level)).

FIG. 24 shows MP @ M in the MPEG system.
The structure of an example of the encoder of L is shown.

Image data to be encoded is input to the frame memory 31 and temporarily stored. Then, the motion vector detector 32 reads out the image data stored in the frame memory 31, for example, in units of macroblocks composed of 16 pixels × 16 pixels, and detects the motion vector.

Here, the motion vector detector 32 processes the image data of each frame as one of an I picture, a P picture, and a B picture. It should be noted that which of I, P, and B pictures is to be processed as an image of each frame that is sequentially input is predetermined (for example, I, B, or B).
Processed as P, B, P, ... B, P).

That is, the motion vector detector 32 refers to a predetermined reference frame in the image stored in the frame memory 31, and the reference frame and the current encoding target. 16 pixels of frame x 16
By performing pattern matching (block matching) with a small block (macroblock) of a line, the motion vector of that macroblock is detected.

Here, in MPEG, there are four image prediction modes: intra coding (intra-frame coding), forward predictive coding, backward predictive coding, and bidirectional predictive coding.
There are types, I pictures are intra coded, P pictures are intra coded or forward predictive coded, and B pictures are intra coded, forward predictive coded, backward predictive coded, or both methods predictive coded. .

That is, the motion vector detector 32 sets the intra coding mode as the prediction mode for the I picture. In this case, the motion vector detector 32
The motion vector is not detected, and the prediction mode (intra prediction mode) is output to the VLC (variable length coding) unit 36 and the motion compensator 42.

The motion vector detector 32 also performs forward prediction for P pictures to detect the motion vector. Further, the motion vector detector 32 compares a prediction error caused by performing forward prediction with, for example, the variance of the macroblock to be encoded (macroblock of P picture), and the variance of the macroblock is predicted. If the difference is smaller than the error, the intra coding mode is set as the prediction mode and output to the VLC unit 36 and the motion compensator 42. Further, if the prediction error caused by performing the forward prediction is smaller, the motion vector detector 32 sets the forward prediction coding mode as the prediction mode, and the detected motion vector, the VLC unit 36, and the motion compensator 42 are set. Output to.

Further, the motion vector detector 32 performs forward prediction, backward prediction, and bidirectional prediction for the B picture, and detects each motion vector. Then, the motion vector detector 32 uses forward prediction, backward prediction,
And a minimum prediction error (hereinafter, appropriately referred to as a minimum prediction error) among the prediction errors for bidirectional prediction, and the minimum prediction error and, for example, the variance of the macroblock to be encoded (the macroblock of the B picture). Compare with. As a result of the comparison, when the variance of the macroblock is smaller than the minimum prediction error, the motion vector detector 32 sets the intra coding mode as the prediction mode, and the VLC unit 3
6 and the motion compensator 42. If the minimum prediction error is smaller, the motion vector detector 32 sets the prediction mode in which the minimum prediction error is obtained as the prediction mode, and the VLC unit 3 with the corresponding motion vector.
6 and the motion compensator 42.

The motion compensator 42 is a motion vector detector 3
When receiving both the prediction mode and the motion vector from 2,
According to the prediction mode and the motion vector, the encoded and already locally decoded image data stored in the frame memory 41 is read out, and this is supplied to the computing units 33 and 40 as a predicted image.

The calculator 33 reads from the frame memory 31 the same macroblock as the image data read from the frame memory 31 by the motion vector detector 32,
The difference between the macroblock and the predicted image from the motion compensator 42 is calculated. This difference value is supplied to the DCT device 34.

On the other hand, the motion compensator 42 does not output the predicted image when it receives only the prediction mode from the motion vector detector 32, that is, when the prediction mode is the intra coding mode. In this case, the arithmetic unit 33 (similarly to the arithmetic unit 40 to be described later) does not perform any processing, and the macroblock read from the frame memory 31 is directly processed by the DC
Output to the T unit 34.

In the DCT unit 34, the output of the arithmetic unit 33 is subjected to DCT processing, and the resulting DCT coefficient is supplied to the quantizer 35. In the quantizer 35,
A quantization step (quantization scale) is set corresponding to the amount of data accumulated in the buffer 37 (the amount of data stored in the buffer 37) (buffer feedback), and the DCT from the DCT unit 34 is set at the quantization step. The coefficients are quantized. The quantized DCT coefficient (hereinafter, appropriately referred to as a quantized coefficient) is supplied to the VLC unit 36 together with the set quantization step.

In the VLC unit 36, the quantized coefficient supplied from the quantizer 35 is converted into a variable length code such as Huffman code, and output to the buffer 37. further,
The VLC unit 36 uses the quantization step from the quantizer 35,
Prediction mode from motion vector detector 32 (mode indicating which of intra coding (intra-picture predictive coding), forward predictive coding, backward predictive coding, or bidirectional predictive coding) is set and motion The vector is also variable-length coded and output to the buffer 37.

The buffer 37 temporarily stores the data from the VLC device 36, smooths the data amount, and, for example,
The data is output to a transmission path (not shown) or recorded on a recording medium.

The buffer 37 also outputs the data storage amount to the quantizer 35, and the quantizer 35 sets the quantization step in accordance with the data storage amount from the buffer 37. That is, the quantizer 35 increases the quantization step when the buffer 37 is about to overflow, thereby reducing the data amount of the quantization coefficient. The quantizer 35 reduces the quantization step when the buffer 37 is likely to underflow, thereby increasing the data amount of the quantization coefficient. In this way, overflow and underflow of the buffer 37 are prevented.

The quantizing coefficient and the quantizing step output by the quantizer 35 are not limited to those of the VLC unit 36, but also of the inverse quantizer 3.
It is designed to be supplied to 8 as well. Inverse quantizer 38
Then, the quantized coefficient from the quantizer 35 is inversely quantized in accordance with the quantization step from the quantizer 35, and converted into the DCT coefficient. The DCT coefficient is supplied to the IDCT device (inverse DCT device) 39. I
In the DCT unit 39, the DCT coefficient is subjected to inverse DCT processing and supplied to the arithmetic unit 40.

In addition to the output of the IDCT device 39, the operation device 40 outputs the motion compensator 42 to the operation device 33 as described above.
Is supplied with the same data as the predicted image that is supplied to the arithmetic unit 40, and the arithmetic unit 40 adds the signal (prediction residual) from the IDCT unit 39 and the predicted image from the motion compensator 42, The original image is locally decoded (however, when the prediction mode is intra coding, the output of the IDCT unit 39 is passed through the arithmetic unit 40 and supplied to the frame memory 41). The decoded image is the same as the decoded image obtained on the receiving side.

The decoded image (locally decoded image) obtained by the arithmetic unit 40 is supplied to and stored in the frame memory 41, and then inter-coded (forward predictive coding, backward predictive coding, quantity direction predictive coding). ) Is used as a reference image (reference frame) for the image.

Next, FIG. 25 shows an example of the structure of an MP @ ML decoder in MPEG for decoding the encoded data output from the encoder of FIG.

The encoded data transmitted through the transmission path is received by a receiving device (not shown), or the encoded data recorded on the recording medium is reproduced by a reproducing device (not shown),
The data is supplied to and stored in the buffer 101.

The IVLC unit (inverse VLC unit) (variable length decoder) 102 reads the coded data stored in the buffer 101 and performs variable length decoding to convert the coded data into a motion vector, a prediction mode, The quantization step and the quantization coefficient are separated. Of these, the motion vector and the prediction mode are supplied to the motion compensator 107, and the quantization step and the quantization coefficient are supplied to the inverse quantizer 103.

The inverse quantizer 103 inversely quantizes the quantized coefficient supplied from the IVLC unit 102 in accordance with the quantization step also supplied from the IVLC unit 102,
The resulting DCT coefficient is output to the IDCT device 104. The IDCT unit 104 performs inverse DCT on the DCT coefficient from the inverse quantizer 103, and supplies it to the arithmetic unit 105.

The output of the IDCT unit 104 and the output of the motion compensator 107 are also supplied to the arithmetic unit 105. That is, the motion compensator 107 reads the already-decoded image stored in the frame memory 106 according to the motion vector and the prediction mode from the IVLC unit 102, as in the case of the motion compensator 41 of FIG. ,
The prediction image is supplied to the arithmetic unit 105. Calculator 10
5 decodes the original image by adding the signal (prediction residual) from the IDCT unit 104 and the predicted image from the motion compensator 107. This decoded image is stored in the frame memory 1
06 and stored. The IDCT device 104
If the output of is intra-coded, then
The output passes through the arithmetic unit 105, is supplied to the frame memory 106 as it is, and is stored therein.

The decoded image stored in the frame memory 106 is used as a reference image of an image to be subsequently decoded, and is appropriately read and supplied to, for example, a display not shown for display.

In MPEG1 and MPEG2, the B picture is not used as a reference image, so that the frame memory 41 is used in each of the encoder and the decoder.
(FIG. 24) or 106 (FIG. 25).

[0030]

[Problems to be Solved by the Invention]
The encoders and decoders shown in (1) are compliant with the MPEG1 / 2 standard, but are currently VO (Video Ob) which is a sequence of objects such as objects forming an image.
ISO-IEC for the method of encoding in units of
In / JTC1 / SC29 / WG11, MPEG
(Moving Picture Experts Group) 4 is in the process of standardization.

Meanwhile, with regard to MPEG4, since standardization work has been advanced mainly for use in the field of communication, GOP (Group Of Picture) defined in MPEG1 / 2 is defined in MPEG4. Therefore, if MPEG4 is used as a storage medium, it is expected that efficient random access will be difficult.

Therefore, in order to enable efficient random access, the applicant of the present invention has introduced a GOV (Group Of VOP) layer corresponding to GOP defined by MPEG1 / 2 in Japanese Patent Application No. -80758, previously proposed, and in MPEG4 this GOV layer was introduced.

By the way, for example, MPEG 1, 2, 4,
H. An encoded bit stream obtained by performing encoding in accordance with a standard such as H.263 has a hierarchical structure including a plurality of layers. Then, on the encoder side, the information necessary for decoding is arranged in the header in each layer, and on the decoder side, the necessary information is extracted from the header of each layer, and the encoded bit stream is decoded.

Therefore, in MPEG1 / 2, when the GOP is randomly accessed, the information of the header of the upper layer may be necessary for decoding the GOP. It is a standard that can retransmit the information of the header of the upper layer.

However, in MPEG4, there is no standard that it is possible to appropriately retransmit the information of the header of the upper layer after the transmission of the upper layer. Therefore, GO
Even if efficient random access becomes possible by introducing the V layer, the information of the upper layer header necessary for decoding the GOV cannot be obtained, and thus the normal decoding result cannot be obtained. There is a risk.

Here, when the MPEG4 coded bit stream is recorded in the storage medium, it is possible to obtain the information of the header of the upper layer by accessing the recording medium. When a coded bitstream is broadcast, etc., unless the coded bitstream is received from the beginning, the information of the upper layer header cannot be obtained. If it starts in the middle, there is a possibility that a normal decoding result cannot be obtained.

The present invention has been made in view of such a situation, and makes it possible to perform normal decoding even in the middle of an encoded bit stream.

[0038]

An image coding apparatus according to the present invention is arranged in a storage unit for storing header information of an upper layer of a coded bitstream in a storage unit and a header of a lower layer of the coded bitstream. The header information of the upper layer corresponding to the identification flag is read from the storage unit and arranged in the header of the lower layer, and the header information of the upper layer read from the storage unit by the arrangement unit is the lower layer. and output means for outputting disposed in the header encoded bit stream, the identification flag is small
Also transmits no header information of the upper layer, the upper layer
All header information of the
Of the header information, only certain header information is transmitted, 3
It is characterized by defining a street .

Overlap motion compensation is included in the header information.
Parameter indicating whether to use or dequantization
You can include parameters that represent the law.

The storage means stores the latest header information of the upper hierarchy.
Can be stored in the storage unit.

Upper layer header stored in the storage unit
Change means for changing the information can be further provided.

The image coding method of the present invention is responsive to the storage step of storing the header information of the upper layer of the coded bitstream in the storage section and the identification flag arranged in the header of the lower layer of the coded bitstream. The placement step of reading the header information of the upper layer from the storage unit and placing it in the header of the lower layer, and the header information of the upper layer read from the storage unit in the processing of the placement step is placed in the header of the lower layer. saw including an output step of outputting a coded bit stream, the identification flag is small
Also transmits no header information of the upper layer, the upper layer
All header information of the
Of the header information, only certain header information is transmitted, 3
It is characterized by defining a street .

In the image coding apparatus and method of the present invention, the header information of the upper layer of the coded bitstream is stored in the storage unit, and the header information of the lower layer of the coded bitstream is stored according to the identification flag. The upper-layer header information read from the storage unit is placed in the lower-layer header, and the upper-layer header information read from the storage unit is placed in the lower-layer header. Output and at least the identification flag
Also transmits no header information of the upper layer, the upper layer
All header information of the
Of the header information, only certain header information is transmitted, 3
The street is regulated .

The image decoding apparatus of the present invention comprises an extracting means for extracting the header information of the upper layer contained in the header of the lower layer of the encoded bitstream based on the identification flag,
Decoding means for decoding the coded bit stream based on the header information extracted by the extracting means ,
The flag transmits at least no upper layer header information
No, all upper layer header information is transmitted,
Is only the specified header information in the upper-layer header information
Is specified .

The header information can include a parameter indicating whether or not the overlap motion compensation is used, or a parameter indicating a dequantization method.

According to the image decoding method of the present invention, the extraction step of extracting the header information of the upper layer included in the header of the lower layer of the encoded bitstream based on the identification flag, and the header extracted by the processing of the extraction step. based on the information, see contains a decoding step of decoding the coded bit stream, the identification flag is at least, the upper layer header
Does not transmit any information, transmits all upper-layer header information
To send, or a predetermined number of upper-layer header information
It is characterized by defining three types of transmitting only header information .

In the image decoding apparatus and method of the present invention, the upper layer header information included in the lower layer header of the coded bitstream is extracted based on the identification flag, and based on the extracted header information, Encoding bit
Stream is decoded and the identification flag contains at least
The header information of the upper layer that does not transmit any header information of the upper layer
All header information is transmitted, or the upper layer header
Of the information, only the predetermined header information is transmitted.
Is specified .

[0048]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the present invention will be described below, but before that, a coded bit stream defined in MPEG4 will be described. In addition, here, FCD (Final Comitt) which is a Draft of MPEG4 standard is used.
An encoded bitstream in ee Draft) will be described.

FIG. 1 shows the structure of a coded bit stream specified by FCD.

As shown in the figure, the coded bit stream has a VS (Visual Object Sequence) layer, a VISO (V
isual Object) layer, VO (video Object) layer, VOL
(Video Object Layer), GOV (Group of VOP) layer, V
It has a hierarchical structure composed of a plurality of layers such as OP (Video Object Plane) layer (the upper layer in FIG. 1 constitutes a higher layer).

That is, the coded bit stream is configured in units of VS. Here, VS is an image sequence and corresponds to, for example, one program or movie.

Each VS is composed of one or more VISOs. Here, there are several types of VISO. That is, in the VISO, for example, a still texture object (Still Texture Object) that is a still image and a face object (Face Objec) composed of a face image are included.
t), VO (Video Objec) that is a moving image object
t) etc. Therefore, if the encoded bitstream is of a moving image, the VISO is composed of VOs.

VO is one or more VOLs (Video Object L
ayer) (composed of one VOL when the image is not layered (layered encoding), and is composed of the number of VOLs when the image is layered).

VOL is a required number of GOVs (Group of V
OP), and GOV is one or more VOP (Video Ob
ject Plane) sequence. In addition, GOV
May be omitted, in which case the VOL is one or more VOPs.
Will be composed of.

A VOP corresponds to a conventional frame.

The relationship between VS, VO, and VOP will be further described. VS is an image sequence as described above, and corresponds to, for example, one program. Then, VO is a sequence of each object forming a composite image when a sequence of the composite image exists, and VOP means VO at a certain time. That is, for example, when there is a composite image F3 configured by combining the images F1 and F2, the images F1 or F2 arranged in time series are VOs, respectively, and the image F1 or F2 at a certain time is , And each is a VOP. Therefore, VOs are VOPs of the same object at different times.
Can be said to be a set of.

Here, in each of FIGS. 2 to 4, VS,
The syntax of VISO and VO is shown. 5 to 7 show VOL syntax, FIG. 8 shows GOV syntax, and FIGS. 9 to 11 show VOP syntax. The flag semantics described in the syntax of each layer are defined in the MPEG4 FCD standard (14
496-2), refer to it.

VS, VISO, in the FCD standard of MPEG4,
The information of the VO and VOL headers includes essential information necessary for decoding the coded bitstream, and without this information, as described above, it is impossible to correctly decode the coded bitstream. Have difficulty.

That is, for example, random access to the coded bit stream recorded on the recording medium or F
When performing special playback such as F / FR (fast forward / rewind) or accessing a coded bit stream being broadcast from the middle, first of all, in order to start decoding of the coded bit stream, , VS, VIS
It is necessary to decode the information in the O, VO, and VOL headers.

However, according to the FCD standard of MPEG4, the VS,
Only VISO, VO, and VOL headers are allowed to be transmitted, and in this case, it is particularly difficult to start decoding from the middle of the coded bit stream that is broadcast.

Further, for example, the VOL header describes a quantization matrix and a flag designating a coding mode. These coding modes are set depending on the property of the coded bitstream obtained from the image to be coded, that is, the statistical property of the image to be coded is optimized. However, for example, with respect to a long-time image sequence or the like, the nature of the image may change greatly depending on the time, so that the value initially set in the VOL header is not always optimal. Nevertheless, at the beginning of the image sequence (here, the VO in which the flag designating the coding mode is described
The fact that the flag that specifies the encoding mode can be set only at the beginning of L) impedes efficient encoding.

Therefore, FIG. 12 shows a configuration example of an embodiment of an encoder to which the present invention is applied. A frame memory 1, a motion vector detector 2, an arithmetic unit 3, a DCT unit 4, a quantizer 5, and VL which constitute this encoder.
The C unit 6, the buffer 7, the inverse quantizer 8, the IDCT unit 9, the arithmetic unit 10, the frame memory 11, and the motion compensator 12 are the frame memory 3 which constitutes the encoder shown in FIG.
1, motion vector detector 32, calculator 33, DCT device 3
4, a quantizer 35, a VLC device 36, a buffer 37, an inverse quantizer 38, an IDCT device 39, a calculator 40, a frame memory 41, and a motion compensator 42, respectively. Therefore, the frame memory 1 to the motion compensator 12 may perform the same process as the frame memory 31 to the motion compensator 42, respectively, and the description of the same process will be appropriately omitted.

The VOPs forming the digital image signal to be coded are sequentially supplied to the frame memory 1 (reception means), where they are received and temporarily stored. Further, the frame memory 1 is also supplied with a flag FSZ indicating the size of the VOP supplied thereto in a predetermined absolute coordinate system and a flag FPOS indicating the position. These flags FSZ and FPOS are also temporarily stored.

The VOP stored in the frame memory 1 is
The motion vector detector 2 reads the data in macroblock units. Then, the motion vector detection circuit 2 follows each VOP according to a preset predetermined sequence.
, I (Intra) -VOP, P (Predictive) -VO
P or B (Biderectionally Predictive) -VOP
Process as. Each VOP input sequentially
Is processed as a VOP of I, P, or B,
Predetermined (for example, I, B, P, B, P, ...
.. are treated as B and P).

The motion vector detector 2 refers to a predetermined reference image (VOP) that is predetermined for the macroblock to be processed, performs motion compensation, and detects the motion vector of the macroblock.

Here, there are three types of prediction modes of motion compensation (inter-frame prediction): forward prediction, backward prediction, and bidirectional prediction. In P-VOP, motion compensation is performed only in forward prediction, and motion compensation is performed. The vector detector 2 detects a motion vector that minimizes the prediction error. Also, B-VOP
Is subjected to motion compensation in three types of prediction, forward prediction, and bidirectional prediction, and the motion vector detector 2 detects a motion vector that minimizes the prediction error in each prediction mode. Furthermore, the motion vector detector 2 selects one of the three prediction modes in which the minimum prediction error is obtained,
The motion vector in the prediction mode is also selected.

Then, the motion vector detector 2 compares the prediction error obtained as a result of motion compensation with the variance of the macroblock to be coded. As a result, if the variance of the macroblock is smaller, interframe prediction is not performed for that macroblock, and intraframe coding is performed. In this case, the prediction mode is intra-picture coding (intra), and such a prediction mode is supplied from the motion vector detector 2 to the VLC unit 6 and the motion compensator 12. On the other hand, when the prediction error is smaller, the prediction mode and the motion vector for which the prediction error is obtained are supplied from the motion vector detector 2 to the VLC unit 6 and the motion compensator 12. The prediction mode for I-VOP is
Be sure to use intra-picture coding.

Here, the VOP sequences to be encoded may differ in size and position. Therefore, when detecting a motion vector, it is necessary to set a reference coordinate system and detect the motion vector in the coordinate system. Therefore, here, one certain absolute coordinate is assumed, and the motion vector at the absolute coordinate is calculated. That is, the motion vector detector 2 is supplied with the flag FSZ indicating the size of the VOP in the absolute coordinate system and the flag FPOS indicating the position, and the motion vector detector 2 receives this flag FSZ. And VOP to be processed based on flag FPOS
And the VOP to be the reference image are arranged in the absolute coordinate system,
The motion vector of (the macroblock of) the VOP to be processed is calculated.

On the other hand, the motion compensator 12 generates a predicted image by performing motion compensation on the VOP stored in the frame memory 11 based on the motion vector from the motion vector detector 2 and the prediction mode. To do. This predicted image is supplied to the arithmetic unit 3. The encoding unit macroblock read from the frame memory 1 by the motion vector detector 2 is also supplied to the arithmetic unit 3 from the frame memory 1. Then, the computing unit 3 sets the pixel value of each pixel forming the encoding target macroblock,
The difference between the pixel values of the pixels forming the predicted image is calculated, and the difference signal is output to the DCT unit 4. If the macroblock to be encoded is an intra macroblock, the arithmetic unit 3 outputs the macroblock to be encoded as it is to the DCT unit 4.

In the DCT unit 4, D
CT (discrete cosine transform) processing is performed and converted into DCT coefficients. The DCT coefficient is input to the quantizer 5, quantized in a quantization step corresponding to the data storage amount (buffer storage amount) of the transmission buffer 7, and then input to the VLC (variable length coding) unit 6. It

The VLC device 6 converts the image data supplied from the quantizer 5 into a variable length code such as Huffman code, and the resulting encoded bit stream is
Output to the transmission buffer 7.

In the VLC unit 6, the quantizer 5 sets a quantization step (scale), and the motion vector detector 2 sets a prediction mode (intra-picture prediction, forward prediction, backward prediction, or bidirectional prediction). Mode),
And the motion vector is a key signal encoder 13 described later.
More key signal encoding results are supplied. Further, the VLC unit 6 has a flag FSZ.
And FPOS is also being supplied. The VLC unit 6 inserts (arranges) these pieces of information, and further, the information stored in the buffer 16 into the header of a predetermined layer of the encoded bitstream configured as shown in FIG. 1 and outputs it. .

The VLC unit 6 outputs the information arranged in the header of each layer to the buffer 16, and the buffer 16 stores the information supplied from the VLC unit 6. ing.

The transmission buffer 7 temporarily stores the coded bit stream from the VLC unit 36 and outputs a quantization control signal corresponding to the stored amount to the quantizer 5. That is, the transmission buffer 7 supplies the quantization control signal for increasing the quantization scale to the quantizer 5 when the accumulated amount increases to the allowable upper limit value, and increases the quantization scale, thereby increasing the quantization scale. 5 reduces the amount of data output. Further, when the remaining storage amount decreases to the allowable lower limit value, the transmission buffer 7 supplies the quantizer 5 with a quantization control signal that reduces the quantization scale, and reduces the quantization scale, thereby performing quantization. The amount of data output from the device 5 is increased. In this way, overflow and underflow of the transmission buffer 7 are prevented.

Then, the encoded bit stream accumulated in the transmission buffer 7 is read out at a predetermined timing and supplied to a recording medium 201 such as a magnetic tape, a magnetic disk, a magneto-optical disk, a phase change disk or the like. Recorded, or analog public network, ISDN,
It is transmitted via a transmission medium 202 such as a satellite line, a CATV network, or a terrestrial wave. As a result, the coded bitstream is provided to the decoder of FIG. 16 described later via the recording medium 201 and the transmission medium 202.

Here, as described above, VO is a sequence of each object forming a synthetic image when a sequence of the synthetic image exists, and VOP means VO at a certain time. That is, for example, when there is a composite image F3 that is composed by combining the images F1 and F2, the image F1 or F2 arranged in time series is
Each is a VO, and the image F1 or F2 at a certain time point is a VOP. Therefore, for example, if the image F1 is the background and the image F2 is the foreground,
In order to obtain the combined image F3, it is necessary to combine the images F1 and F2 using the key signal for removing the image F2. That is, in order to obtain the composite image F3, a key signal for extracting the image F2 is required.

Therefore, the key signal for removing each VOP is supplied to the key signal encoder 13. In the key signal encoder 13, the key signal supplied thereto is, for example, It is encoded by a predetermined method such as DPCM. The encoded result of the key signal is supplied to the VLC unit 6 and the key signal decoder 14.

The key signal decoder 14 decodes the encoded result of the key signal from the key signal encoder 13, and the motion vector detector 2, the DCT unit 4, the IDCT unit 9, the motion compensator 12, and the pixel replacement. The blocks are supplied to the device 15, and the blocks are processed by using the decryption result of the key signal as needed.

Thus, the motion vector detector 2 has
Although the key signal locally decoded by the key signal decoder 14 is also supplied, this key signal is used when the motion vector detector 2 calculates the prediction error of the macroblock.

That is, since the VOP is an image of a certain object at a certain time, its shape is basically an arbitrary shape. In this case, the image (pixels forming the object) is displayed in the macroblock to be encoded. May include areas where no. In such a case, the motion vector detector 2 is adapted to calculate a prediction error by excluding pixels in the macroblock to be encoded in which an image does not exist, that is, prediction of a pixel in which an image exists. Calculate the prediction error of the macroblock to be encoded using only the error,
It is designed to detect a motion vector that minimizes it, and for each pixel in the macroblock to be encoded, in order to recognize whether or not an image exists, local decoding of the macroblock to be encoded is performed. The generated key signal is referred to.

Specifically, in the motion vector detector 2,
A pixel having a key signal of 0 is recognized as a pixel belonging to a region outside the object (image object) in which an image does not exist, and a pixel having a key signal other than 0 indicates that an image exists. It is recognized as a pixel in the area inside the (image object). Then, the motion vector detector 2 does not calculate the difference between the pixel having the key signal of 0 and the reference image for obtaining the predicted image.

When the VOP has a rectangular shape, the key signal is always a value other than 0 (binary).
1 for the key (hard key), gray scale (gray sca
le) key (soft key) from 1 to 255)
Therefore, the prediction error is calculated using all the pixels of the macroblock.

On the other hand, the data output from the quantizer 5 is also supplied to the inverse quantizer 8, and in the inverse quantizer 8, the data corresponds to the quantization step supplied from the quantizer 5. Inversely quantized to obtain DCT coefficients. This DCT coefficient is input to the IDCT (inverse DCT) unit 9 and the inverse DCT
After being processed, it is supplied to the arithmetic unit 10.

When the prediction mode is any of forward prediction, backward prediction, and bidirectional prediction, the arithmetic unit 10
In addition to the output of the IDCT device 9, the predicted image output by the motion compensator 12 is also supplied. The arithmetic unit 10 adds the predicted image output from the motion compensator 12 to the output of the IDCT unit 9 to decode the image, and supplies the image to the pixel replacer 15.

When the prediction mode is intra-picture coding, the arithmetic unit 10 supplies the output of the IDCT unit 9 to the pixel replacing unit 15 as it is.

In the pixel replacer 15, the output of the calculator 10 is subjected to padding processing, which will be described later, and the result is supplied to the frame memory 11. The output of the pixel replacer 15 is stored in the frame memory 11, and the stored value, that is, the decoded image is used for motion compensation by the motion compensator 12. The flags FSZ and FPOS are also supplied to the frame memory 11, and the frame memory 11 also stores these flags FSZ and FPOS.

Next, referring to the flowchart of FIG. 13, the padding (paddin) performed by the pixel replacer 15 of FIG.
g) Describe the processing.

In the padding process, first, in step S1, one of the pixels forming the macroblock supplied from the calculator 10 to the pixel replacer 15 is set as a target pixel, and the key signal for the target pixel is set to 0. Is determined. When it is determined in step S1 that the key signal for the pixel of interest is not 0, that is,
If the pixel of interest constitutes the inside of the image object, the process proceeds to step S2, where the pixel replacer 15 does not perform any processing on the pixel of interest and outputs it to the frame memory 11 as it is. Proceed to S4.

Here, when the VOP to be encoded has a rectangular shape, the key signal always has a value other than 0 as described above, so that the pixel replacer 15 selects all the pixels in the VOP. Will be output without any processing.

On the other hand, if it is determined in step S1 that the key signal for the pixel of interest is 0, that is,
If the pixel of interest constitutes the outside of the image object, the process proceeds to step S3, and the pixel value of the pixel of interest is
For example, it is set to 0, and the process proceeds to step S4. In step S4, it is determined whether or not the processing has been performed for all the pixels forming the macroblock from the arithmetic unit 10, and
If it is determined that the processing has not been performed for all the pixels, the process returns to step S1 and the same processing is repeated with the pixel not yet set as the target pixel as a new target pixel.

In step S4, the arithmetic unit 10
If it is determined that the processing has been performed for all the pixels from, the process proceeds to step S5, the horizontal line having the macroblock from the arithmetic unit 10 is selected as the target horizontal line, and the process proceeds to step S6. In step S6, the pixel values of the pixels at both ends of the horizontal line of interest are determined.

That is, when attention is paid to a certain horizontal line of the macroblock after the processing of steps S1 to S4, the pixel values at both ends of the horizontal line of interest are both 0. (The pixels at both ends are outside the image object), the pixel value at one end is not 0 (only the pixels at one end are inside the image object), and the pixel values at both ends are not 0 There are three cases (cases where pixels at both ends are inside the image object). Step S
At 6, it is determined which of these three cases the horizontal line of interest belongs to.

If it is determined in step S6 that the pixel values at both ends of the horizontal line of interest are both 0,
The process proceeds to step S7, 0 is set to the variable C secured for the horizontal line of interest, and the process proceeds to step S10. If it is determined in step S6 that the pixel values at both ends of the target horizontal line are not 0, the process proceeds to step S8 and the variable C secured for the target horizontal line is set to the pixels at both ends of the target horizontal line. The average value is set, and the process proceeds to step S10. Further, when it is determined in step S6 that only one of the pixel values at both ends of the target horizontal line is not 0, the process proceeds to step S9, and the variable C secured for the target horizontal line is set to the target horizontal line. 0 of the pixel values at both ends of
The other value is set, and the process proceeds to step S10.

In step S10, it is determined whether or not all the horizontal lines of the macroblock from the arithmetic unit 10 have been processed as the noticeable horizontal lines, and all the horizontal lines have not been processed as the noticeable horizontal lines. When it is determined, the process returns to step S5, a horizontal line that is not yet selected as the horizontal line of interest is selected as a new horizontal line of interest, and the same processing is repeated thereafter.

If it is determined in step S10 that all the horizontal lines have been processed as the noticeable horizontal lines, the process proceeds to step S11.

In steps S11 to S16,
Steps S5 to S10 are performed on the vertical line, not the horizontal line of the macroblock from the arithmetic unit 10.
The same processing as in the case of is performed.

That is, in step S11, the vertical line with the macroblock from the arithmetic unit 10 is selected as the vertical line of interest, and the process proceeds to step S12. Step S
At 12, the pixel values of the pixels at both ends of the vertical line of interest are determined.

That is, even when attention is paid to a certain vertical line of the macroblock after the processing of steps S1 to S4, the pixel values at both ends of the vertical line of interest are both 0. (The pixels at both ends are outside the image object), the pixel value at one end is not 0 (only the pixels at one end are inside the image object), and the pixel values at both ends are not 0 There are three cases (cases where pixels at both ends are inside the image object). Step S
At 12, it is determined which of these three cases the vertical line of interest belongs to.

When it is determined in step S12 that the pixel values at both ends of the target vertical line are both 0, the process proceeds to step S13, and 0 is set to the variable B secured for the target vertical line. Step S1
Go to 6. If it is determined in step S12 that the pixel values at both ends of the vertical line of interest are not 0, the process proceeds to step S14 and the variable B secured for the vertical line of interest is set to the pixels at both ends of the vertical line of interest. The average value is set, and the process proceeds to step S16. Further, when it is determined in step S12 that only one of the pixel values at both ends of the target vertical line is not 0, the process proceeds to step S15, and the variable B secured for the target vertical line is set to the target vertical line. One of the pixel values at both ends of the non-zero value is set, and step S
Proceed to 16.

In step S16, it is determined whether or not all the vertical lines of the macroblock from the arithmetic unit 10 have been processed as the target vertical lines, and all the vertical lines have not been processed as the target vertical lines. If determined, the process returns to step S11, a vertical line that has not yet been selected as the vertical line of interest is selected as a new vertical line of interest, and the same processing is repeated thereafter.

If it is determined in step S16 that all the vertical lines have been processed as the vertical lines of interest, the process proceeds to step S17, and among the pixels forming the macro block from the arithmetic unit 10, the same as in step S2. A pixel is selected as a pixel of interest from the pixels excluding the pixel output to the frame memory 11, and the process proceeds to step S18.

In step S18, the value of the set (B, C) of variables B and C for each of the vertical line and the horizontal line intersecting on the target pixel is determined.

In step S18, the variable B is 0,
When it is determined that C is not 0, the process proceeds to step S19, the value of the variable C is output to the frame memory 11 as the pixel value of the pixel of interest, and the process proceeds to step S23. Also,
In step S18, both variables B and C are 0
If it is determined that it is not, the process proceeds to step S20, the average value of the values of the variables B and C is output to the frame memory 11 as the pixel value of the pixel of interest, and the process proceeds to step S23. Furthermore, when it is determined in step S18 that both the variables B and C are 0, the process proceeds to step S21, in which the pixel value of the target pixel remains 0, and step S2
Go to 3.

On the other hand, if it is determined in step S18 that the variable B is not 0 and C is 0, then step S
In step 19, the value of the variable B is the pixel value of the target pixel,
It is output to the frame memory 11, and the process proceeds to step S23. In step S23, it is determined whether or not all the pixels forming the macroblock from the arithmetic unit 10 except the pixels output as they are in step S2 have been processed. If it is determined that the processing has not been performed, Returning to step S17, a pixel that has not yet been set as a target pixel is newly selected as a target pixel, and the same processing is repeated thereafter.

In step S23, the arithmetic unit 1
When it is determined that all the pixels forming the macro block from 0 except the pixel output as it is in step S2 are processed, the process proceeds to step S24, and among the pixels already output to the frame memory 11. The pixel closest to each pixel that has not been output to the frame memory 11 (hereinafter, appropriately referred to as a non-output pixel) is detected. Further, in step S24, the pixel value of the detected pixel is output to the frame memory 11 as the pixel value of the non-output pixel, and the padding process ends.
In addition, when two or more pixels that are closest to the non-output pixel have already been detected among the pixels output to the frame memory 11, the average value of those pixel values is the pixel value of the non-output pixel. Is output as.

By performing the padding processing as described above, the pixels constituting the outside of the image object are interpolated, so to speak, whereby the mosquito noise can be reduced and the motion compensation efficiency can be improved.

Next, the processing of the VLC unit 6 (encoding means) of FIG. 1 will be further described.

The VLC device 6 has VS, VISO, VO, V
Information to be originally arranged is arranged in the headers of OL, GOV, and VOP, and further, the variable-length coding result of the output of the quantizer 5 is arranged to form a coded bit stream, and the transmission buffer 7 Output to.

Further, the VLC unit 6 outputs the information arranged in the headers of VS, VISO, VO, and VOL, which are higher layers than the GOV, to the buffer 16 and stores it therein.

After that, when outputting the GOV header, the VLC unit 6 reads the information of the VS, VISO, VO, and VOL headers of the hierarchy higher than the GOV, which is stored in the buffer 16, and determines the predetermined GOV header. Insert (include) in the position of and output. Therefore, in this case, in addition to the information that should be originally placed in the GOV header, VS, VI
Information of each header of SO, VO, and VOL is also arranged.

FIG. 14 shows a VLC which performs the above processing.
The syntax of GOV output by the device 6 is shown. The shaded area in FIG. 14 is different from the syntax in the FCD shown in FIG.

Group_VOP_start_code is a 32-bit unique code indicating the start position of GOV. time_code
The (time information) is composed of 18 bits and represents the display time with the second precision of the VOP first displayed in the GOV. this
time_code is `` time and control codes for video tape reco '' specified in IEC standard publication 461.
"rders".

For closed_gop and broken_link, refer to the MPEG4 Video FCD standard (ISO / IEC 14496-2).

Is_extension (header information presence / absence flag)
Is a 1-bit flag introduced in this embodiment.
It indicates whether or not the header includes information for initializing the decoder of the VS, VISO, VO, and VOL headers and other information. In the VLC device 6, for example, when the flag is_extension is 1, VS, VISO, VO,
Information of each VOL header (VisualObjectSequence (), Vi
sualObject (), VideoObject (), VideoObjectLayer ())
Is included in the GOV header. That is, the flag is_extensio
When n is 1, the information of each header of VS, VISO, VO, and VOL is group_VOP_start_code, time_code, close.
Placed after d_gop, broken_link, is_extension.

Further, when the flag is_extension is 1, the VLC unit 6 includes the information of each header of VS, VISO, VO, and VOL in the GOV header, and then includes the included V.
The information of each header of S, VISO, VO, and VOL is supplied to the buffer 16 and stored in place of the information stored so far.

The VLC device 6 is provided with VS, VISO, V
Even when the O and VOL headers are subsequently output, the information of the headers is supplied to the buffer 16 and stored therein.

Therefore, the buffer 16 always has the latest V.
The information on the headers of S, VISO, VO, and VOL is stored.

Here, when the flag is_extension is 1, after the information of each header of VS, VISO, VO, and VOL is included in the GOV header, the included VS, VIS is included.
The information of each header of O, VO, and VOL is supplied to the buffer 16 and stored therein for the following reason.

That is, the VLC unit 6 includes VS and VIS to be included in the GOV header in order to improve coding efficiency.
The information of each header of O, VO, and VOL can be changed. In this case, since the changed information is the latest information, the VS and VI included in the GOV header are stored in order to store the latest information in the buffer 16.
Information of each header of SO, VO, and VOL is stored in the buffer 16
It is designed to be supplied to and stored in.

On the other hand, when the flag is_extension is 0, V
In the LC device 6, the information of each VS, VISO, VO, and VOL header is not included in the GOV header.

The value stored in the buffer 16 can be changed externally. That is, VS,
There are cases where it is desired to change some or all of the information in each header of VISO, VO, and VOL in the middle of the encoded bitstream. That is, for example, there is a case where it is desired to change the quantization matrix used for decoding during the decoding of the encoded bitstream. In such cases, the user
VS, VISO, VO stored in the buffer 16
The information in each header of the VOL can be appropriately changed to desired information. The information after this change is the flag is_exten
Since the sion is placed in the GOV header and is output, the decoder receives the GOV header and then decodes it based on the changed information.

Next, the processing of the VLC unit 6 for outputting the GOV of the syntax shown in FIG. 14 will be described with reference to the flowchart of FIG.

As described above, the VLC device 6 has VS, V
An encoded bit stream is formed by arranging information to be originally arranged in the headers of ISO, VO, VOL, GOV, and VOP, and further arranging the variable-length coding result of the output of the quantizer 5. , To the transmission buffer 7.

Further, the VLC device 6 has VS, VISO,
Each time each header of VO and VOL is output, the information arranged in each header is output to the buffer 16 and stored (overwritten).

When outputting the GOV header, the VLC unit 6 determines the GOV header in step S31.
It is determined whether the flag is_extension for the header is 1. In step S1, the flag is_exten
If it is determined that sion is not 1 (0), VLC
The device 6 arranges the information that should be originally arranged (information shown in FIG. 8) and the flag is_extension in the GOV header (V
The information of each header of S, VISO, VO, and VOL is not arranged), and the GOV header obtained as a result is output. Then, after waiting for the timing of outputting the next GOV header, the process returns to step S31.

On the other hand, in step S31, the flag is
If it is determined that _extension is 1, step S3
2 proceeds to VS, VIS stored in the buffer 16
Read the latest information of each header of O, VO, VOL,
The latest information, the flag is_extension, and the information to be originally placed are placed in the GOV header and output.
Then, after waiting for the timing of outputting the next GOV header, the process returns to step S31.

The flag is_ext allocated to each GOV
The value of the tension is set in the VLC device in advance on the administrator side of the encoder, for example.

Next, FIG. 16 shows an example of the configuration of an embodiment of a decoder for decoding a coded bit stream provided via the recording medium 201 or the transmission medium 202. Buffer 21 and IVLC that constitute this decoder
Device 22, inverse quantizer 23, IDCT device 24, calculator 2
5, a frame memory 26, and a motion compensator 27 are a buffer 101, an IVLC unit 102, an inverse quantizer 103, an IDCT unit 104, an arithmetic unit 105, a frame memory 106, and a motion compensator 107 which constitute the decoder shown in FIG. It corresponds to each. Therefore, the buffer 21 to the motion compensator 27 may perform the same processing as the buffer 101 to the motion compensator 107, respectively, and the description of the same processing will be appropriately omitted.

The coded bit stream provided via the recording medium 201 or the transmission medium 202 is received by the reception buffer 21 (reception means) and temporarily stored. Then, the coded bit stream stored in the reception buffer 21 is read by the IVLC (variable length decoding) unit 22 as appropriate.

The IVLC unit 22 (decoding means) performs variable length decoding of the coded bit stream read from the reception buffer 21, the motion vector and the prediction mode to the motion compensator 27, and the quantization step to the inverse quantization. In the vessel 23,
The image data (quantized DCT coefficient) subjected to variable length decoding is output to the inverse quantizer 23 while being output. The IVLC unit 22 also determines other information necessary for initializing the parameters included in the header of each layer and used in the decoding process of the decoder, and other information (for example, whether to perform overlap motion compensation). A flag indicating a quantization matrix or the like is appropriately supplied to a necessary block (for example, a flag indicating whether or not to perform overlap motion compensation is supplied to the motion compensator 27, and a quantization matrix is supplied to the inverse quantizer 23). , Each supplied)

Furthermore, the IVLC unit 22 decodes the flag is_extension for the GOV header, and the flag is_e.
When xtension is 1, that is, in the GOV header, V
When the information of each header of S, VISO, VO, and VOL is included, the information is also VS, VISO, VO, and VO.
Variable length decoding is performed in the same way as for each header of L, and the decoding result is
Supply the required blocks. Specifically, for example, a motion vector, a prediction mode, a flag indicating whether or not to perform overlap motion compensation, and the like are provided to the motion compensator 27, and a quantization step, a quantization matrix, and the like are provided to the dequantizer 23.
, Respectively.

The IVLC unit 22 also decodes the flags FSZ and FPOS contained in the encoded bit stream and supplies them to the frame memory 26, motion compensator 27, and key signal decoder 29. Further, the IVLC unit 22 extracts the encoded key signal (key signal bit stream) included in the encoded bit stream and supplies it to the key signal decoder 29.

The key signal decoder 29 decodes the key signal bit stream supplied from the IVLC unit 22. This decoded key signal is used for the IDCT unit 24 and the motion compensator 2
7 and the pixel replacer 28.

The inverse quantizer 23 inversely quantizes the image data supplied from the IVLC unit 22 in accordance with the quantization step also supplied from the IVLC unit 22, and the IDCT unit 24
Output to. The IDCT device 24 performs inverse DCT processing on the data (DCT coefficient) output from the inverse quantizer 23, and supplies the data to the calculator 25.

When the image data supplied from the IDCT device 24 is I-VOP data, the calculator 25 converts the image data into image data (P or B-
In order to generate a predicted image of (VOP data), it is directly supplied to the frame memory 26 via the pixel replacer 28 and stored therein.

The pixel replacer 28 performs the same processing as the pixel replacer 15 of FIG.

On the other hand, the data supplied to the arithmetic unit 25 is
If it is P or B-VOP data, the motion compensator 2
7 reads out the already decoded image stored in the frame memory 26 according to the motion vector and the prediction mode supplied from the IVLC unit 22, thereby generating a predicted image and outputting it to the calculator 25. ID in the calculator 25
Image data (difference data) supplied from the CT device 24,
The predicted image data supplied from the motion compensator 27 is added to obtain a decoded image. This decoded image is used as the pixel replacer 28.
The image is supplied to and stored in the frame memory 26 via, and is appropriately used as a reference image (image referred to for generating a predicted image) of an image to be decoded later. The decoded image stored in the frame memory 26 is used as a reference image as described above, and is also read as appropriate and supplied to, for example, a display (not shown) for display.

Next, with reference to the flowchart of FIG. 17, the processing performed by the IVLC unit 22 of FIG. 16 with respect to the GOV header will be further described.

Upon receiving the GOV header, the IVLC unit 22 performs the processing that should normally be performed on the GOV header (the processing that should be performed when the GOV header shown in FIG. 8 is transmitted), and at step S41. , Flag is located in the GOV header (Fig. 14)
Determine whether _extension is 1. Step S
In 41, when it is determined that the flag is_extension is not 1 (is 0), that is, in the GOV header, VS,
When the information of each header of VISO, VO, and VOL is not included, it waits for the next GOV header to be transmitted, and returns to step S41.

In step S41, the flag is
When it is determined that _extension is 1, that is, GOV
When the header includes information of each header of VS, VISO, VO, and VOL, the process proceeds to step S42 and IV
The LC unit 22 supplies the information of the respective headers of VS, VISO, VO, and VOL to the necessary block, and the next GOV.
After waiting for the header to be transmitted, the process returns to step S41.

Next, the GOV can be configured as the syntax shown in FIG. 18, for example, in addition to the syntax shown in FIG.

That is, FIG. 18 shows another example of the syntax of GOV. In addition, in FIG. 14 and FIG.
There is a difference between the bottom of en_link and the top of next_start_code. In addition, the shaded portion in FIG.
This is a part different from the syntax in the FCD shown in FIG.

In the embodiment shown in FIG. 14, the flag is_exten is used.
It was possible to set only whether or not the information of each header of VS, VISO, VO, and VOL is transmitted in the GOV header by sion, but in the embodiment of FIG. 18, by adopting the flag load_data_type, VS , VIS
It is possible to set not only whether all the information in each of the O, VO, and VOL headers is transmitted in the GOV header, but also to transmit only a part of the information. That is, in the embodiment of FIG. 18, VS, VISO,
It is possible to include only a part of the information of each header of VO and VOL in the GOV header, and the flag load_data_ty
According to pe, some information included in the GOV header can be identified.

Specifically, in FIG. 18, load_data_
The type is a variable length code, and immediately after this, indicates whether or not to transmit information for initializing the decoder at the time of random access and, if so, the type of information to be transmitted. That is, for example, as shown in FIG. 19, load_data_
When the type is "1", the information of the header in the upper hierarchy than the GOV layer is not included in the GOV. Also, load_data_
When type is “01”, VS and VI are the same as when the flag is_extension is 1 in the embodiment of FIG.
Information of each header of SO, VO, VOL (VisualObjectSe
quence (), VisualObject (), VideoObject (), VideoObje
All of ctLayer ()) are included in the GOV. Furthermore, when load_data_type is '001', VS, VIS
Of the information of each header of O, VO, and VOL, information of a predetermined parameter that is determined in advance is included in the GOV.

That is, in the embodiment of FIG. 18, load
Information included in GOV when _data_type is “001” is defined as download_parameters ().

Here, in the present embodiment, download_par
ameters () is defined as shown in FIG. 20, for example.

In FIG. 20, the flag obmc_disable is
This is a 1-bit flag indicating whether to use overlap motion compensation. If this value is '1', overlap motion compensation is not used, and if it is '0', overlap motion compensation is used. Flag quan
t_type is a 1-bit flag indicating the method of inverse quantization. If this value is '0', inverse quantization is performed using the inverse quantization method specified in H.263, and if it is '1', the inverse quantization specified in MPEG2 is performed. Inverse quantization is performed using the quantization method. When using the inverse quantization method defined in MPEG2, a flag indicating whether or not to download the quantization matrix is further transmitted.
Also, when downloading the quantization matrix,
The downloaded quantization matrix is also transmitted.

In addition, download_parameters () shown in FIG.
Load_intra_quant_mat, intra_ specified in
quant_mat, load_nonintra_quant_mat, nonintra_quant
_mat, load_intra_quant_mat_grayscale, iontra_quant_
mat_grayscale, load_nonintra_quant_mat_grayscale,
The semantics of nonintra_quant_mat_grayscale are F
This is the same as the semantics of the flag of the same name defined in the VOL (FIGS. 5 to 7) on the CD.

In the embodiment of FIG. 19, the flag lo
Regarding ad_data_type, only three cases are specified, but it is also possible to specify four or more cases.
In this case, the information of the combination different from the combination of the information defined by download_parameters () in FIG.
It becomes possible to arrange in the OV header.

FIG. 21 shows a configuration example of an embodiment of an encoder for outputting the GOV header shown in FIG. In addition, in the figure, the same reference numerals are given to the portions corresponding to the case in FIG. That is, the encoder of FIG. 21 is configured similarly to the case of FIG. 12, except that the parser 17 (selection means) is newly provided.

The parser (flag discriminator) 17 refers to the flag load_data_type regarding the GOV header which the VLC unit 6 is about to output, and the flag load_data_type.
Information is read from the buffer 16 according to
It is supplied to the C unit 6. In the VLC unit 6, the information supplied from the parser 17 together with the flag load_data_type is GO.
The V header is arranged and output at a predetermined position shown in FIG.

Next, the processing of the parser 17 for outputting the GOV having the syntax shown in FIG. 18 to the VLC device 6 will be described with reference to the flowchart of FIG.

The VLC unit 6 outputs the GOV header flag load_d at the timing of outputting the GOV header.
The ata_type is supplied to the parser 17. Parser 17 is V
The flag load_data_type is received from the LC device 6, and its value is determined in step S51. Step S51
When it is determined that the flag load_data_type is 1, the parser 17 outputs nothing to the VLC unit 6 and the flag load_data_ arranged in the next GOV header.
After waiting for the type to be transmitted from the VLC unit 6, the process returns to step S51. In this case, in the VLC device 6, GO
Information that should be placed in the V header and load_data_type
And output the resulting GOV header.

In step S51, the flag lo
When it is determined that ad_data_type is 01, the process proceeds to step S52, where the parser 17 reads V from the buffer 16.
The latest information in each header of S, VISO, VO, and VOL is read and supplied to the VLC unit 6. And the next GOV
After waiting for the flag load_data_type arranged in the header to be transmitted from the VLC unit 6, the process returns to step S51. Therefore, in this case, in the VLC unit 6, in addition to the information and load_data_type that should be originally arranged in the GOV header,
Information on each header of VS, VISO, VO, and VOL is also arranged.

On the other hand, in step S51, the flag lo
If it is determined that ad_data_type is 001, the parser 17 proceeds to step S53, and the parser 17 download_paramet shown in FIG. 20 among the information stored in the buffer 16.
Select the one included in ers () and read it out.
Supply to. Then, after waiting for the flag load_data_type arranged in the next GOV header to be transmitted from the VLC unit 6, the process returns to step S51. Therefore, in this case, in the VLC unit 6, in addition to the information and load_data_type to be originally arranged in the GOV header, down shown in FIG.
load_parameters () is also placed.

Next, the encoded bit stream provided from the encoder of FIG. 21 via the recording medium 201 or the transmission medium 202 can be decoded by the decoder having the configuration shown in FIG.

FIG. 23 shows that the IVLC unit 22 of the decoder having the configuration shown in FIG. 16 has the syntax GO shown in FIG.
It is a flow chart for explaining processing performed about a V header.

Upon receiving the GOV header, the IVLC unit 22 performs the processing that should normally be performed on the GOV header (the processing that should be performed when the GOV header shown in FIG. 8 is transmitted), and at step S61. , The flag lo located in the GOV header (FIG. 18)
Determine the value of ad_data_type. When it is determined in step S61 that the flag load_data_type is 1, the process waits for the next GOV header to be transmitted, and the process returns to step S61.

Further, in step S61, the flag lo
When it is determined that ad_data_type is 01, that is, G
If the OV header includes the information of the VS, VISO, VO, and VOL headers, the process proceeds to step S62.
The IVLC unit 22 uses the flag load_data_type to
The information of the VS, VISO, VO, and VOL headers is extracted from the encoded bitstream and supplied to the necessary blocks. That is, the information is subjected to variable length decoding, and a flag obtained as a result thereof, for example, a motion vector, a prediction mode, and whether or not to perform overlap motion compensation is provided to the motion compensator 27, and the quantization step and the The respective quantization matrices are supplied to the inverse quantizer 23. Then, after waiting for the next GOV header to be transmitted, the process returns to step S61.

On the other hand, in step S61, the flag lo
When it is determined that ad_data_type is 001, that is,
If the GOV header includes download_parameters () (parameter update information), the IVLC unit 22 extracts the download_parameters () from the encoded bitstream based on the flag load_data_type, and proceeds to step S63. Supply to the block. That is,
The download_parameters () is subjected to variable length decoding, and the resulting flag, for example, indicating whether or not to perform overlap motion compensation is provided to the motion compensator 27, and the quantization step and the quantization matrix are inversely quantized. Bowl 23
, Respectively. Then, after waiting for the next GOV header to be transmitted, the process returns to step S61.

As described above, in the GOV header, all or part of the information of the VS, VISO, VO, and VOL headers of the upper layer, according to the header (download_parameters () shown in FIG. 20 in this embodiment). Since it has been included, it is possible to perform random access or the like to the encoded bit stream and perform normal decoding from the middle. Furthermore, it becomes possible to change the quantization step and the quantization matrix at the beginning of the GOV, and as a result,
It becomes possible to perform efficient encoding.

The case where the present invention is applied to the encoder / decoder which performs encoding / decoding based on MPEG4 has been described above. However, the applicable range of the present invention is MPEG4.
It is not limited to encoding / decoding based on

Further, in the present embodiment, download_param
The information shown in FIG. 20 as the eters () is included in the GOV header, but the information included in the GOV header as the download_parameters () is not limited to that shown in FIG.

Further, the encoder shown in FIGS. 12 and 21 and the decoder shown in FIG. 16 can be realized by hardware, or can be realized by causing a computer or the like to execute a program. Is also possible.

Also, with MPEG4, hierarchical coding for achieving scalability is possible, but the present invention is applicable regardless of whether hierarchical coding is performed or not.

[0166]

According to the first aspect of the present invention, efficient coding is possible.

According to the second aspect of the present invention , normal decoding can be performed even in the middle of the encoded bit stream.

[0168]

[Brief description of drawings]

FIG. 1 is a diagram showing a configuration of a coded bitstream defined by MPEG4 standard FCD.

[Fig. 2] Fig. 2 is a diagram illustrating the syntax of VS defined by the MPEG4 standard FCD.

[Fig. 3] Fig. 3 is a diagram illustrating the syntax of VISO defined by the MPEG4 standard FCD.

FIG. 4 is a diagram showing the syntax of VO defined by the MPEG4 standard FCD.

[Fig. 5] Fig. 5 is a diagram illustrating the syntax of a VOL defined by the MPEG4 standard FCD.

FIG. 6 is a diagram showing the syntax of a VOL defined by the MPEG4 standard FCD.

[Fig. 7] Fig. 7 is a diagram illustrating the syntax of a VOL defined by the MPEG4 standard FCD.

[Fig. 8] Fig. 8 is a diagram illustrating the syntax of GOV defined by the MPEG4 standard FCD.

[Fig. 9] Fig. 9 is a diagram illustrating the syntax of a VOP defined by the MPEG4 standard FCD.

[Fig. 10] Fig. 10 is a diagram illustrating the syntax of a VOP defined by the MPEG4 standard FCD.

FIG. 11 is a diagram showing the syntax of VOP defined by the MPEG4 standard FCD.

FIG. 12 is a block diagram illustrating a configuration example of an embodiment of an encoder to which the present invention has been applied.

FIG. 13 is a flowchart for explaining the process of the pixel replacer 15 in FIG.

14 is a diagram showing the syntax of GOV output by the VLC unit 6 of FIG.

FIG. 15 is a flowchart for explaining the processing of the VLC device 6 of FIG.

FIG. 16 is a block diagram showing a configuration example of an embodiment of a decoder to which the present invention has been applied.

FIG. 17 is a flowchart for explaining the processing of the IVLC device 22 of FIG.

FIG. 18 is a diagram showing the syntax of GOV output by the VLC unit 6 of FIG. 21.

FIG. 19 is a diagram for explaining load_data_type.

[Fig. 20] Fig. 20 is a diagram illustrating the syntax of download_parameters () in Fig. 18.

FIG. 21 is a block diagram showing a configuration example of another embodiment of an encoder to which the present invention has been applied.

22 is a flowchart for explaining the process of the parser 17 in FIG.

23 is a flow chart for explaining the processing of the IVLC device 22 of FIG.

FIG. 24 is a block diagram showing a configuration of an example of a conventional encoder.

FIG. 25 is a block diagram showing a configuration of an example of a conventional decoder.

[Explanation of symbols]

1 frame memory (reception means), 2 motion vector detectors, 3 arithmetic units, 4 DCT units, 5 quantizers, 6 VLC units (encoding means), 7 buffers,
8 inverse quantizer, 9 IDCT device, 10 arithmetic unit, 11 frame memory, 12 motion compensator,
13 key signal encoder, 14 key signal decoder,
15 pixel replacer, 16 buffer, 17 parser (selecting means), 21 buffer (receiving means), 22
IVLC device (decoding means), 23 inverse quantizer, 24
IDCT device, 25 arithmetic unit, 26 frame memory, 27 motion compensator, 28 pixel replacer, 29
Key signal decoder, 201 recording medium, 202 transmission medium

Continuation of the front page (56) References JP-A-8-125966 (JP, A) Underface, Japan, 1992, Vol. 18 No. 8, p. 124-146 Computer Science Magazine bit vol. 29 no. 6 Kyoritsu Publishing Co. (1997.6.1), Japan, p. 86-96 (58) Fields investigated (Int.Cl. ⁷ , DB name) H04N ^5/ 91-5/95 H04N ^7/ 24-7/68

Claims (8)

(57) [Claims] 1. An image encoding apparatus for encoding an image and outputting an encoded bitstream having a hierarchical structure composed of a plurality of layers, wherein header information of an upper layer of the encoded bitstream is stored in a storage unit. Storage means, and arrangement means for reading header information of the higher layer according to an identification flag arranged in the header of the lower layer of the encoded bitstream from the storage unit and arranging it in the header of the lower layer. , by the arrangement means, and output means for header information of the upper layer which is read from the storage unit to output the encoded bit stream which is disposed in the header of the lower layer, the identification flag is at least, Header information of the upper layer
No information is transmitted, all header information of the upper layer is transmitted.
Of the header information of the upper layer
An image coding apparatus characterized in that it defines three types of transmitting only predetermined header information . To wherein said header information, picture coding according to claim 1, characterized in that it includes a parameter representing the process parameters or inverse quantization, indicating whether to use overlapping motion compensation apparatus. 3. The image coding apparatus according to claim 1, wherein the storage unit stores the latest header information of the upper layer in the storage unit. 4. The image coding apparatus according to claim 1, further comprising a changing unit that changes the header information of the upper layer stored in the storage unit. 5. An image encoding method for an image encoding apparatus, which encodes an image and outputs an encoded bitstream having a hierarchical structure composed of a plurality of layers, wherein header information of an upper layer of the encoded bitstream is provided. A storage step of storing in the storage unit, read the header information of the upper layer according to the identification flag arranged in the header of the lower layer of the encoded bitstream from the storage unit, in the header of the lower layer seen including a placement step of placing, and an output step of header information of said placement step of processing the upper read from the storage unit in a hierarchical outputs the coded bit stream arranged in a header of the lower layer , The identification flag is at least the header information of the upper layer.
No information is transmitted, all header information of the upper layer is transmitted.
Of the header information of the upper layer
An image coding method characterized in that it defines three types of transmitting only predetermined header information . 6. The header information of the upper layer of the encoded bitstream read from the storage unit according to the identification flag arranged in the header of the lower layer of the encoded bitstream of the hierarchical structure is the lower layer. An image decoding apparatus for decoding the coded bitstream arranged in the header of the extracting unit, which extracts header information of the upper layer included in the header of the lower layer based on the identification flag; Decoding means for decoding the encoded bitstream based on the header information extracted by the means , wherein the identification flag is at least the header information of the upper layer.
No information is transmitted, all header information of the upper layer is transmitted.
Of the header information of the upper layer
An image decoding device characterized in that it defines three types of transmitting only predetermined header information . To wherein said header information, the image decoding apparatus according to claim 6, characterized in that it includes a parameter representing the process parameters or inverse quantization, indicating whether to use overlapping motion compensation . 8. The header information of the upper layer of the coded bitstream, which is read from the storage unit and corresponds to the identification flag arranged in the header of the lower layer of the coded bitstream of the hierarchical structure, is the lower layer. In the image decoding method of the image decoding device for decoding the coded bitstream arranged in the header, the extracting step of extracting header information of the higher layer included in the header of the lower layer based on the identification flag. If, on the basis of the header information extracted by the processing of the extracting step, seen including a decoding step of decoding the coded bit stream, the identification flag is at least, the header information of the upper layer
No information is transmitted, all header information of the upper layer is transmitted.
Of the header information of the upper layer
An image decoding method characterized by defining three types of transmitting only predetermined header information .