JP2824024B2 - Image decoding method and apparatus - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image decoding method and apparatus for decoding an encoded image signal.

[0002]

2. Description of the Related Art When digitally represented image data is transmitted or stored, encoding is performed to reduce the amount of data. As an encoding method, there is a method of reducing redundancy by utilizing temporal or spatial correlation of image information (image data).

As a method utilizing temporal correlation, there are a method of encoding a difference between two consecutive screens (frames) and a method of detecting a motion of an image to perform motion compensation. As a method using spatial correlation, an image is divided into blocks of a predetermined size (for example, 8 pixels each in the vertical direction and the horizontal direction), and the data in the blocks are orthogonally transformed. Some (for example, rearrange from low frequency components to high frequency components) and perform variable length coding. An image coding system (hereinafter, abbreviated as MPEG2), which is being standardized by the Moving Picture Experts Group (MPEG), uses the above two methods in combination.
The provisional recommendation for MPEG2 is “Generic Coding of Moving P
ISO / IE entitled “ictures and Associated Audio”
C13818-2.

[0004] The present invention is applicable to the processing of decoding any MPEG2 image, so the decoding processing will be described.

FIG. 5 is a block diagram showing a configuration example of an image decoding apparatus for decoding data encoded by such a method. In FIG. 5, a buffer control unit 1, a variable length decoder 2, a scan converter 3, an inverse quantizer 4, an inverse DCT
The decoding processing is executed by the unit 5 and the motion compensation image reproducing unit 6. Reference numeral 50 denotes a memory, which comprises a buffer memory 51 and frame memories (memory for three I, P, and B frames described later) 52, 53, and 54. Reference numeral 100 denotes an input bit stream representing an encoded image;
Indicates a reproduced image.

Next, the operation will be described. The input bit stream 100 is controlled by the buffer control unit 1
The data 40 is stored in the buffer memory 51. The data 41 read from the buffer memory 51 is subjected to variable-length decoding by the variable-length decoder 2.

Although not all data is variable-length coded, it is assumed that fixed-length codes are also decoded by the variable-length decoder 2. Next, after the order of the data is rearranged by the scan converter 3, the data is inversely quantized by the inverse quantizer 4. Next, the inverse DCT unit 5 performs an inverse discrete cosine transform. The motion-compensated image reproducing unit 6 performs reproduction in consideration of the motion of the image. In MPEG2, a temporally intermediate frame (here, a B frame) is predicted from both a temporally earlier frame (here, an I frame) and a temporally later frame (here, a P frame). Therefore, in order to reproduce the B frame, it is necessary to read out the predicted frame data 42 and 43 of the I frame and the P frame which have been decoded in advance from the frame memories 52 and 53 (MPE).
In G2, a later P frame is decoded prior to a B frame). In the prediction method, there is a field prediction in addition to the above-described frame prediction. Similar to the frame prediction, the field prediction is performed by using a temporally earlier field (here, an I field) and a temporally later field (here, a field). A temporally intermediate field (here, the B field) is predicted from both the P field and the P field. Predicted frame data or predicted field data 42, 43 and inverse DC
Depending on the prediction error output from the T unit 5, a B frame or B
The field is reproduced by the motion-compensated image reproducing unit 6 and written into the frame memory 54 as reproduced pixel data 44.
The I, P, and B frames in the frame memories 52, 53, and 54 are read from each memory in a predetermined order (the B frame data 45 is read in FIG. 5), and a reproduced image 200 is output.

The present invention is applicable to an apparatus for processing any MPEG2 image. As an example, consider the case of reproducing an NTSC image. One frame of the NTSC image is composed of 720 horizontal pixels and 480 vertical lines as shown in FIG. This is divided into 16 pixels both horizontally and vertically. A unit of one division is called a macroblock (hereinafter abbreviated as MB). The NTSC image is divided into 45 MB in width and 30 MB in height, a total of 1350 MB. In MPEG2, a set of macroblocks closed in one horizontal line is called a slice, and an NTSC image is divided into at least 30 slices.

In MPEG2, the types of macroblocks are roughly classified into two types: intra macroblocks and inter macroblocks. Intra macroblocks are macroblocks coded using only image data in the macroblock. On the other hand, an inter macroblock is a macroblock in which a difference between a reproduced picture of a previous or future picture is encoded. In addition to the difference signal, a motion vector representing a moving amount of an image from a previous or future reproduced image is added to the inter macroblock. In the case of field prediction, a predicted image is in units of fields, so that two motion vectors are required for one macroblock. In the field prediction of a B frame, a total of four motion vectors are required for one macroblock because prediction is performed from both a temporally previous field and a temporally subsequent field.

In predictive coding using a restored image as a reference image as in MPEG2, if the reference image cannot be correctly restored due to an error generated when decoding a digital storage medium, a communication channel and a bit stream. , A plurality of pictures referring to the restored image (P pictures and B pictures are
(Refer to the restored image of the picture or the P picture), and this error propagates, and the image cannot be restored properly. MPE
In G2, some measures are taken to deal with such an error. One of them is an intra motion vector (Intra_MV). As described above, a motion vector is added to an inter macroblock.
No motion vector is added to the intra macroblock. However, there is also a mode in which a motion vector is added to an intra macro block to prevent errors, and this mode is called an intra motion vector.

Next, the MPE necessary to explain the present invention will be described.
The characteristics of the G2 input bit stream 100 will be briefly described. MPEG2 input bit stream 100
Has a hierarchical structure, and the highest layer of the encoded bit stream is called a sequence layer, and the layers of a group of picture layer, a picture layer, a slice layer, a macro block layer, and a block layer are defined in this order. . Also, M
In an input bit stream 100 using a variable length coding technique such as PEG2, a data boundary is indicated by using a unique code that does not appear anywhere else on the bit stream, and M
In PEG2, this is called a start code. MPEG2
The start code includes a sequence start code (Sequence_start_code) indicating the start of a sequence and a picture start code (Picuture) indicating the start of a picture.
_start_code), a slice start code (Slice_start_code) indicating the start of a slice, and a sequence error code (Sequence_error_code) indicating the presence of an uncorrectable error that has occurred on the transmission path. The start code of the lowest layer defined by MPEG2 is a slice start code.

In variable length decoding, first, after reading out a bit stream from the buffer memory 51, first, a sequence start code is detected and decoding is started. Thereafter, decoding is sequentially performed. For example, if a sequence error code is present at the position of the bit stream of the macroblock 500 in FIG. 7, the decoder is forced to discard subsequent data until the next start code. That is, assuming that one horizontal row of macroblocks is composed of one slice in FIG. 7, stream data corresponding to 500 to 503 macroblocks is discarded. As a process for compensating for the image of the discarded data portion, for example, the decoder determines the macroblock of the slice immediately above (the macroblocks 501 to 502 vertically above macroblocks 500 to 503 in FIG. 7 in the vertical direction). A method of concealing a restored image by using a motion vector and using an image of a previously reproduced frame as a predicted image is conceivable. in this case,
If the macroblock immediately above is an inter macroblock, the motion vector of the macroblock may be referred to. I can do it. In FIG. 7, only two horizontal rows + α are shown.

Next, an example of error concealment using the intra motion vector of the intra macro block and the motion vector of the inter macro block will be described. Each time a motion vector is decoded, the motion vector is written to memory,
When a sequence error code is detected, the error is concealed using the motion vector of the macroblock immediately above the macroblock that cannot be decoded, for example, the motion vector value of the macroblock 45 times before in the case of NTSC. Also, the motion vector per macroblock is
In the case of field prediction and bidirectional prediction, there are four motion vectors as described above, and each motion vector is divided in the horizontal direction and the vertical direction. In order to express each motion vector, for example, in the case of the main level of the main profile of MPEG2, a bit width of 12 bits in the horizontal direction and a bit width of 9 bits in the vertical direction are required.

[0014]

As described above, an error in a variable length decoder or an error generated in a transmission path or the like is recovered using a motion vector of an intra macroblock and a motion vector of an inter macroblock. In this case, a memory for storing a motion vector for a value (the number of macroblocks in the horizontal direction) obtained by dividing the number of pixels in the horizontal direction of the display image by 16 pixels (for example, in the case of NTSC) is required for 45 macroblocks. Become.

In the conventional apparatus, the motion vectors for the number of macroblocks in the horizontal direction are stored using the memory as described above. For example, a decoder capable of decoding a bit stream of an NTSC image encoded at the main level of the main profile has been realized by using four RAMs each having a configuration of 45 words × 21 bits.

Further, in order to process a high-resolution image such as an HDTV or a UDTV, a memory having a larger capacity is required. For example, when such an image decoding device is realized in an LSI, the circuit scale is large. The increase in size leads to an increase in chip size, which is not economical.

An object of the present invention is to provide an image decoding method and apparatus which have solved the above-mentioned disadvantages of the prior art.

[0018]

According to the present invention, there is provided an image decoding method for a buffer memory which includes a start code of a previously decoded slice including a macroblock immediately above a macroblock which cannot be decoded due to an error. An address is read from a register, the bit stream of the buffer memory corresponding to this address is read again, and a motion vector of a macroblock corresponding to a macroblock immediately above a macroblock that cannot be restored due to an error is obtained again to obtain the macro that cannot be restored. It conceals blocks.

When the intra motion vector is in a mode in which no motion vector is added, decoding is performed with the motion vector set to zero.

Further, the address of the buffer memory is stored in the register only when the row of the macro block is changed.

Further, the address of the buffer memory of an arbitrary macro block in the row of the macro block is stored in a register.

The image decoding apparatus according to the present invention predicts an image frame from a plurality of prediction frames and decodes encoded image data by using a digital storage medium, an error generated in a communication channel, and a bit stream. In the image decoding device that performs concealment of an error with respect to an error that occurs when decoding the error by referring to a motion vector of a macroblock directly above a motion vector value in a vertical direction,
Means having a means for re-reading, from the buffer memory, the coded image data at the address of the buffer memory containing the start code of the already decoded slice including the macro block immediately above the macro block which cannot be decoded due to an error It is.

[0023]

According to the image decoding method and apparatus of the present invention, when error concealment is performed when a correct image cannot be decoded due to an error in a variable length decoder or an error in a transmission path indicated by a sequence error code, Reads from the register the address of the buffer memory that contains the start code of the previously decoded slice that contains the macroblock directly above the macroblock that cannot be decoded due to an error, and reads the buffer memory stream corresponding to this address again Thus, a motion vector of a macroblock corresponding to a position immediately above a macroblock that cannot be restored due to an error can be obtained by decoding again, and a macroblock that cannot be restored can be concealed. In addition, a memory having a large capacity, which is conventionally required, is not required, and the circuit scale can be reduced.

Further, even when an intra motion vector is not added, error concealment can be performed.

[0025]

【Example】

[Embodiment 1] FIG. 1 is a block diagram showing an embodiment of the present invention. In this figure, reference numeral 10 denotes a buffer control unit which includes registers 11 and 12 and is basically the same as the buffer control unit 1 of FIG. 5, but also performs control unique to the present invention as described later. Reference numeral 20 denotes a variable length decoder, which is basically the same as the variable length decoder 2 in FIG. 1, but also performs control unique to the present invention as described later. Reference numeral 150 denotes a control line for performing characteristic control of the present invention, which is input from the variable length decoder 20 to the buffer control unit 10 as described later. Reference numeral 300 denotes a digital storage medium which temporarily stores the digital data and outputs the input bit stream 100. 400 is a communication channel,
Directly without digital storage media 300,
An input bit stream 100 is input. Note that 151
Indicates data, and the same reference numerals as those in FIG. 5 indicate the same parts.

FIG. 2 is a block diagram showing the details of the configuration of the main part of the embodiment of FIG.
Reference numerals 202 and 203 denote the internal configuration of the variable-length decoder 20; 201, a slice start code decoder for decoding a slice start code; 202, a sequence error code decoder for decoding a sequence error code;
Reference numeral 3 denotes another code decoder for decoding other data. The slice start code decoder 201 decodes a slice start code. In the slice start code, the number of lines of the slice is encoded. If the number of lines is larger than the number of lines of the previous slice start code, the address of the buffer memory 51 containing the slice start code is stored in a register. Is transmitted to the buffer control unit 10, and the buffer control unit 10 stores the address of the buffer memory 51 in the register 11.

When the sequence error code decoder 202 decodes the sequence error code, it outputs an instruction signal 153 for concealing an error to the buffer control unit 10. Also, when an error occurs during decoding of other data, an instruction signal 154 for concealing the error is output from the other code decoder 203 to the buffer control unit 10. When an error concealment instruction signal is input from the sequence error code decoder 202 or another code decoder 203, the buffer control unit 10 detects the next start code, checks the range in which the error should be concealed, and performs concealment. In order to resume decoding later, the address of the buffer memory 51 storing the start code is stored in the register 1.
2 and the data from the slice start code stored in the register 11 is read from the buffer memory 51 again. The re-read data is input to the variable-length decoder 20 via the buffer control unit 10, and searches for a motion vector of a macroblock in which an error should be concealed.

This will be described with reference to FIGS. 3 (a) and 3 (b). FIG. 3A is a diagram for explaining a macroblock to be concealed and a range to be decoded again, and shows an example in which a macroblock in one horizontal row is composed of one slice.

When there is a sequence error code at the position of the stream data corresponding to the macroblock 600, first, the next start code is detected, and the range (macroblocks 600 to 603) where the error should be concealed is checked. Buffer memory 5 for slice start code preceding stream data corresponding to macro block 604
From address 1 (stored in register 11),
The stream data is read out again, and decoded again by the variable length decoder 20. The decoded data before the macroblock 601 is ignored, and only the motion vector values of the macroblocks 601 to 602 are used.
Hide the macroblock of. When the concealment up to the macro block 603 is completed, the stream data is read from the decoding restart address after the concealment stored in the register 12 and the process returns to the normal decoding process.

FIG. 3B is a diagram for explaining a unit of reading from the buffer memory 51, and shows an example in which one horizontal row of macroblocks is composed of one slice.

When the sequence error code is detected by the sequence error code decoder 202, the start code is detected until the next start code, and the read unit 30
2 as the decoding restart address after the concealment ends.
It is stored in the register 12. Next, stream data is sequentially read from the address of the read unit 300 stored in the register 11, and the start code does not always start from the beginning of the read unit. Hide errors as described above. After the concealment process is completed, the stream data is read from the address of the read unit 302 stored in the register 12, and after detecting the start code, the process returns to the normal decoding process.

When the motion vector of the macroblock to be concealed is obtained, the image concealed by the motion compensation image reproducing unit 6 is obtained by using the previous reproduced image as a reference image. [Embodiment 2] This embodiment is an embodiment for decoding a bit stream to which no intra motion vector is added. Processing other than the intra macroblock is the first embodiment.
But the intra macroblock has no motion vector since no intra motion vector is added.
As (zero), an image in which an error is concealed can be obtained as in the first embodiment. [Embodiment 3] In this embodiment, in addition to storing the address of the buffer memory containing the slice start code in the register 11, the same arbitrary position in each row, for example, the center macroblock is included. This is an embodiment in which an address of a buffer memory, a boundary position of a macroblock in a read unit, and a motion vector value of the macroblock are stored in a register.

The motion vector of the inter macro block temporally located next to the intra macro block encodes a motion vector corresponding to the motion of the reference image, but the motion vector of the inter macro block temporally located next to the inter macro block is encoded. Since only the difference between the motion vector of the macroblock and the motion vector of the previous inter macroblock is encoded, the motion vector of the macroblock also needs to be stored. Also, since the boundaries of the macroblocks in the stream are not indicated by unique data, it is necessary to store the positions of the boundaries of the macroblocks in the read unit.

FIG. 4 shows an example of the third embodiment. If the address of the buffer memory containing the macroblock 511 at the same arbitrary position in each row and the motion vector value of the macroblock are stored, the sequence error code 5
When the detection of 10 is to the right of the macroblock 511 at the arbitrary position, the number of macroblocks to be decoded again is smaller than in the first embodiment, so that the motion vector of the macroblock to be concealed is obtained at high speed.

[0035]

As described above, according to the image decoding method of the present invention, when decoding coded image data by predicting an image frame from a plurality of predicted frames, a digital storage medium or a communication channel is used. In an image decoding method for concealing an error generated in the above or an error generated when decoding a bit stream by referring to a motion vector of a macroblock immediately above in a vertical direction, decoding by an error Read out from the register the address of the buffer memory that contained the start code of the already decoded slice containing the macroblock immediately above the impossible macroblock, and read out the bit stream of the buffer memory corresponding to this address again. Macroblocks that cannot be restored due to errors Since the motion vector of the macroblock corresponding to the top is decoded again to conceal the macroblock that cannot be restored, a circuit that can reduce errors occurring when decoding digital storage media, communication channels, and bitstreams is reduced. Error concealment can be performed on a scale.

Even when decoding a bit stream in a mode to which no intra motion vector is added, error concealment can be performed by decoding the motion vector as zero.

The image decoding apparatus according to the present invention decodes an error and a bit stream generated in a digital storage medium, a communication channel, in order to predict an image frame from a plurality of prediction frames and decode encoded image data. In an image decoding apparatus that performs error concealment with respect to an error that occurs at the time of performing an error concealment by referring to a motion vector of a macroblock immediately above the motion vector value in the vertical direction, the encoded image data is read again from the buffer memory. Means, digital storage media,
There is an advantage that the error concealment can be performed with a small circuit scale for an error generated when decoding the communication channel and the bit stream.

[Brief description of the drawings]

FIG. 1 is a block diagram showing the configuration of an embodiment of the present invention.

FIG. 2 is a block diagram showing details of a configuration of a main part of the embodiment of FIG. 1;

FIG. 3 is a diagram illustrating a configuration of each code of an input bit stream.

FIG. 4 is a configuration diagram of a macro block showing another embodiment of the present invention.

FIG. 5 is a block diagram illustrating a configuration example of a conventional image decoding device.

FIG. 6 is a diagram illustrating a configuration of a macroblock of one frame of an NTSC image.

FIG. 7 is a diagram illustrating the relationship between a macroblock in which a sequence error has occurred and a predicted image.

[Explanation of symbols]

 Reference Signs List 3 scan converter 4 inverse quantizer 5 inverse DCT unit 6 motion compensation image reproducing unit 10 buffer control unit 20 variable length decoder 40 data 41 data 42 predicted frame data 43 predicted frame data 44 reproduced pixel data 45 data 50 memory 51 buffer Memory 52 Frame memory 53 Frame memory 54 Frame memory 100 Input bit stream 150 Control line 151 Data 152 Control signal 153 Instruction signal 154 Instruction signal 200 Playback image 201 Slice start code decoder 202 Sequence error code decoder 203 Other code decoder 500 Macro Block 501 Macro block 510 Sequence error code 511 Macro block

────────────────────────────────────────────────── ─── Continued on the front page (72) Inventor Yoshikazu Kawamura 4-36-19 Yoyogi, Shibuya-ku, Tokyo Inside Graphics Communication Laboratories Co., Ltd. (72) Inventor Takayuki Kobayashi, Shibuya-ku, Tokyo 4-36-19 Yoyogi Inside Graphics Communication Laboratories Co., Ltd. (72) Inventor Shigeru Komatsu 4-36-19 Yoyogi Shibuya-ku, Tokyo Graphics Communication Laboratories Co., Ltd. (72) Inventor Ryuji Saito 4-36-19 Yoyogi, Shibuya-ku, Tokyo Inside Graphics Communication Laboratories Co., Ltd. (72) Inventor Tomoyuki Shindo 4-36-19 Yoyogi, Shibuya-ku, Tokyo Graphics Communication Co., Ltd.・ Laboratories (56) References JP-A-5-292488 (JP, A) (58) Fields investigated (Int. Cl. ⁶ , DB name) H04N ^7/ 24-7/68

Claims (5)

(57) [Claims] 1. An image display apparatus comprising: a plurality of prediction frames for predicting an image frame and decoding encoded image data; an error occurring in a digital storage medium or a communication channel; or an error occurring in decoding a bit stream. On the other hand, in an image decoding method for concealing an error by referring to a motion vector of a macroblock directly above a motion vector value in a vertical direction, an image decoding method that includes a macroblock just above a macroblock that cannot be decoded due to an error has already been performed. The address of the buffer memory containing the decoded start code of the slice is read out from the register, and the bit stream of the buffer memory corresponding to this address is read out again and the macroblock corresponding to the macroblock immediately above the macroblock that cannot be restored due to an error is read out. Restore motion vector again And concealing the macroblock that cannot be restored. 2. The image decoding method according to claim 1, wherein the address of the buffer memory is stored in the register only when the row of the macroblock changes. 3. The image decoding method according to claim 2, wherein when decoding a bit stream in a mode to which no intra motion vector is added, the decoding is performed with the motion vector set to zero. 4. The image decoding method according to claim 3, wherein an address of a buffer memory of an arbitrary macro block in a row of the macro block is stored in a register. 5. A method for predicting an image frame from a plurality of predicted frames and decoding encoded image data, wherein an error occurred in a digital storage medium or a communication channel or an error occurred in decoding a bit stream. On the other hand, in an image decoding apparatus that conceals an error by referring to a motion vector of a macroblock directly above a motion vector value in the vertical direction, a macroblock just above a macroblock that cannot be decoded due to an error from a buffer memory is included. An image decoding apparatus, comprising: re-reading means for re-reading the coded image data at the address of the buffer memory in which the start code of the slice already decoded has been included.