JP2002077922A - Picture control system and picture decoder - Google Patents

Abstract

PROBLEM TO BE SOLVED: To improve the picture quality by an efficient error cover-up control considering the correlation of frames. SOLUTION: A coding means 11 codes a picture to generate an I-picture and a P-picture. A picture transmitting means 12 transmits a flag for identifying the I- and P-pictures and a frame train composed of the I- and P-pictures. A correlation value calculating means 21 calculates a correlation value of the preceding frame and the current frame for a received frame train. An error cover-up frame determining means 22 determines which frame is taken as an error cover-up frame for concealing an error between the preceding frame and the current frame, using at least either information of the identifying flag or the correlation value. An error cover-up process means 23 adaptively covers up the error to decode, using the determined error cover-up frame.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image control system and an image decoding apparatus, and more particularly to an image control system for encoding / decoding an image and an image decoding apparatus for decoding an image.

[0002]

2. Description of the Related Art Digital video coding systems are based on ISO.
/ IEC MPEG and ITU-T H.264. H.262 and the like are internationally standardized, and with the development of multimedia communication in recent years, higher quality is required.

As a technique for improving the quality of digital video coding, there is error concealment. This is because, when an image transmitted on the encoder side has an error during transmission, the pixel data of the block that cannot be decoded normally on the decoder side is replaced with the pixel data of the block that can be decoded normally. This is a technique for making image quality degradation less noticeable.

[0004]

However, if the above-described error concealment is performed between frames having low correlation, there has been a problem that the image quality is greatly deteriorated in the past.

FIG. 8 is a diagram showing a problem of the conventional error concealment. The decoder receives and decodes frame f1 at time (t-1), and then decodes frame t1 with error at block b2.
Is received. In this case, conventionally, in the process of decoding the frame f2, error concealment is realized by extracting a block b1 which is a block at the same position (or near) as the block b2 from the frame f1, and replacing the block b2 with the block b1. Was.

However, when a scene change occurs between the frames f1 and f2, the correlation between the frames f1 and f2 is low, and if the above-described error concealment is performed in such a situation, the image quality is greatly deteriorated. And quality and reliability are reduced.

As described above, in the error concealment control, the correlation of the frames is important, but in the conventional digital video coding technology, adaptive and efficient error concealment control based on the correlation of the frames is performed. I wasn't.

The present invention has been made in view of such a point, and an object of the present invention is to provide an image control system which performs efficient error concealment control in consideration of frame correlation to improve image quality. With the goal.

It is another object of the present invention to provide an image decoding apparatus which performs efficient error concealment control in consideration of frame correlation and improves image quality.

[0010]

According to the present invention, in order to solve the above-mentioned problems, an image control system 1 for encoding and decoding an image and transmitting the image as shown in FIG. Encoding means 11 for encoding to generate I and P pictures, image transmitting means 12 for transmitting an identification flag for identifying I and P pictures, and a frame sequence composed of I and P pictures
And a correlation value calculating unit 21 that calculates a correlation value between the previous frame and the current frame for the received frame sequence, information on the identification flag,
Error concealment frame determining means 22 for determining whether the error concealment frame used for error concealment is the previous frame or the current frame using at least one of the correlation value and the error concealment frame. Error concealment processing means 23 for adaptively performing error concealment and decoding using frames
, And an image decoding device 20 comprising:

Here, the encoding means 11 encodes an image to generate an I picture and a P picture. The image transmitting unit 12 transmits an identification flag for identifying an I picture and a P picture, and a frame sequence including the I picture and the P picture. The correlation value calculator 21 calculates a correlation value between the previous frame and the current frame for the received frame sequence. The error concealment frame determining means 22 includes:
Information of the identification flag, and at least one of the correlation value, using an error concealment frame used when error concealment,
Determine whether to use the previous frame or the current frame. The error concealment processing means 23 adaptively performs error concealment using the determined error concealment frame and decodes the error concealment frame.

[0012]

Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a principle diagram of the image control system of the present invention. The image control system 1 includes an image encoding device 10 and an image decoding device 20, and performs image encoding / decoding and image transmission.

The image encoding apparatus 10 is provided with an encoding unit 1
1 encodes an image to generate an I picture (intra-frame predicted image) and a P picture (inter-frame forward predicted image).

The image transmitting means 12 transmits an identification flag for identifying whether the picture is an I picture or a P picture, and a frame sequence including the I picture and the P picture to the image decoding apparatus 20. The frame sequence composed of the I picture and the P picture will be described later with reference to FIG.

In the image decoding apparatus 20, a correlation value calculating means 21 calculates a correlation value between a previous frame and a current frame for a received frame sequence. Here, the previous frame is a frame at time (t-1), and is a frame that has already been decoded. The current frame is the frame at time t and is the frame to be decoded.

The error concealment frame determining means 22 uses at least one of the information of the identification flag and the calculated correlation value to determine whether the error concealment frame used for error concealment is the previous frame or the current frame. Decide whether to use a frame.

Here, the error block B is added to the current frame Fb.
In the case where there is 2, when extracting and concealing the block B1 at the same position or in the vicinity of the error block B2 from the previous frame Fa, the previous frame Fa is an error concealment frame. Or, a block B2 near the error block B2
When concealment is performed using a, the current frame Fb becomes an error concealment frame. The details of the procedure for determining the frame for error concealment will be described later with reference to FIGS.

The error concealment processing means 23 adaptively performs error concealment using the determined error concealment frame and decodes the error concealment frame. That is, error concealment processing is performed by adaptively using either the previous frame or the current frame as the error concealment frame (replace the error block with a normal block extracted from the current frame or the previous frame).

Next, a frame sequence transmitted from the image encoding device 10 will be described. FIG. 2 is a diagram illustrating an example of a frame sequence. Here, the frame at the start of image encoding (at the start of communication) or at the time of a scene change is called a first frame, and N (two in the figure) consecutive frames after the first frame are referred to as a second frame. Call.

The first and second frames include:
The image encoding device 10 performs intra-frame prediction encoding to generate an I-picture frame. Here, the I picture of the first frame corresponds to the first I picture of the present invention, and the I picture of the second frame corresponds to the third I picture of the present invention.

After the I-picture is properly repeated, a P-picture frame is generated by performing forward inter-frame predictive coding (inter-frame predictive coding using the immediately preceding past frame). . A plurality of frames having such an arrangement are a frame sequence to be decoded by the image decoding device 20. It should be noted that I-pictures (= second I-pictures) even at the time other than the start of encoding and when a scene change occurs
Is sometimes sent.

Next, a procedure for determining an error concealment frame using only the information of the identification flag will be described. FIG. 3 is a diagram showing a state transition of the error concealment frame determination. (A) is a state transition diagram, and (B) is a table showing the state transition diagram of (A).

The state S1 is an error concealment frame.
The state in which the current frame is used, the state S2 is a state in which the previous frame is used as an error concealment frame, and the state S3 is a state in which the previous frame is used as an error concealment frame.

Here, in the state S1, if the received identification flag indicates an I picture, the state transits to a state S2, and if the identification flag indicates a P picture, the state transits to a state S3. In the state S2, if the received identification flag indicates an I picture, the state transits to the state S2 itself and the identification flag is set to P.
If a picture is indicated, the flow goes to the state S3.

In the state S3, the received identification flag is I
If it indicates a picture, the state transits to state S1, and if the identification flag indicates a P picture, it transits to state S3 itself.
FIG. 4 is a diagram showing an application example of the state transition shown in FIG.
It is assumed that a frame sequence including I-pictures and P-pictures arranged as shown in the figure is input to the image decoding device 20. Note that frames F1 and F7 are first I pictures, and frames F2, F3, F8, and F9 are third I pictures.

At the start of encoding, the received frame F1 is I
It is a picture (it can be determined to be an I picture from the identification flag), and the error concealment frame is set as the current frame in state S1. That is, when there is an error block in the frame F1, an error concealment process of replacing the error block with a neighboring block (the block of the frame F1 itself) is performed.

In the next frame F2, since the identification flag = I, a transition is made from the state S1 to the state S2, and the error concealment frame becomes the previous frame. That is, when there is an error block in the frame F2, a block at the same position or in the vicinity of the error block is extracted from the frame F1, and
An error concealment process to replace it is performed. The subsequent frame F3 also has the identification flag = I and is in the state S2, and the previous frame (frame F2) is used for error concealment.

Since frame F4 has the identification flag = P, the state transits from state S2 to state S3, and the error concealment frame becomes the previous frame. Thereafter, since the identification flag is P until frame F6, the state S3 continues, and the previous frame is used for error concealment.

When a scene change occurs, the frame F7
Since the identification flag becomes I, the state transits from the state S3 to the state S1, and the error concealment frame becomes the current frame. The frames F8 and F9 have the identification flag = I and the state S
2 and the error concealment frame becomes the previous frame.
Since frames F10 and F11 have the identification flag = P, the state transits to the state S3, and the error concealment frame becomes the previous frame.

Next, a procedure for determining an error concealment frame using both the information of the identification flag and the inter-frame correlation value will be described. FIG. 5 is a diagram showing the state transition of the error concealment frame determination. (A) is a state transition diagram, and (B) is a table showing the state transition diagram of (A).

The state S11 is a state in which the current frame is used as an error concealment frame, the state S21 is a state in which a previous frame is used as an error concealment frame, and the state S31 is a state in which the previous frame is used as an error concealment frame. In this state, a frame is used.

Here, in the state S11, the identification flag =
If I and the correlation value = small (the value of the inter-frame correlation is small), the state transits to the state S11 itself. If the identification flag = I and the correlation value = large (the value of the inter-frame correlation is large), Transition to the state S21. When the identification flag is P, the state transits to the state S31 regardless of the inter-frame correlation.

At the time of the state S21, if the identification flag = I and the correlation value = small, the processing transits to the state S11. If the identification flag = I and the correlation value = high, the processing transits to the state S21 itself. When the identification flag is P, the state transits to the state S31 regardless of the inter-frame correlation.

In the state S31, if the identification flag is "I", the state transits to the state S11 regardless of the inter-frame correlation. If the identification flag is P, the state transits to the state S31 itself regardless of the inter-frame correlation.

FIG. 6 is a diagram showing an application example of the state transition shown in FIG. It is assumed that a frame sequence including I-pictures and P-pictures arranged as shown in the figure is input to the image decoding device 20. The frames F21 and F29 are the first
, The frame F27 is the second I picture, and the frames F22, F23, and F28 are the third I picture.
It is a picture.

At the start of encoding, the received frame F21 is an I picture, and the state for error concealment is set to the current frame (frame F21 itself) in state S11. The next frame F22 has the identification flag = I, and the frame F
Since the correlation value with S = 21 is large, the state S11
21 and the error concealment frame becomes the previous frame (frame F21). Since the subsequent frame F23 has the identification flag = I and the correlation value with the frame F22 = large, the state is state S21, and the previous frame (frame F22) is used for error concealment.

In the frame F24, since the identification flag = P, the state transits from the state S21 to the state S31 (in this case,
The correlation value is ignored), and the error concealment frame is the previous frame. Thereafter, until the frame F26, the identification flag = P
Therefore, the state S31 continues, and the previous frame is used for error concealment.

In the frame F27, since the identification flag is I, the state transits from the state S31 to the state S11 (in this case,
The correlation value is ignored), and the error concealment frame becomes the current frame.

In the frame F28, since the identification flag = I and the correlation value with the frame F27 = large, the state transits from the state S11 to the state S21, and the previous frame is used for error concealment. When a scene change occurs, the frame F29 changes from the state S21 to the state S11 because the identification flag = I and the correlation value with the frame F28 = small, and the error concealment frame becomes the current frame. And the frames F30, F31
Transitions to state S31 with the identification flag = P (in this case, the correlation value is ignored), and the error concealment frame becomes the previous frame.

Next, the configuration in the case where the image decoding apparatus 20 of the present invention is applied to the decoding side apparatus of the hybrid coding system which performs frame prediction and orthogonal transformation will be described. FIG. 7 is a diagram showing a configuration of the image decoding device.

For the image decoding apparatus 20a, the entropy decoding means 24 performs entropy decoding (variable length decoding) of the received encoded image. Further, it detects an error block and transmits the error block information to the error concealment processing means 23. Further, the transmitted identification flag is transmitted to the error concealment frame determining means 22.

The inverse quantization means 25 returns the quantization level number generated by quantization on the encoder side to a quantization value. The inverse orthogonal transform unit 26 performs an inverse orthogonal transform on the quantized value to generate quantized difference data. The adder 27 adds the quantized difference data and the predicted image of the previous frame to generate a decoded image.

The switch 29 switches to the terminal a 2 in the case of inter-frame prediction, and transmits the previous frame image stored in the memory 28 to the adder 27. In the case of intra-frame prediction, switching to terminal a1 is performed.
The transmission of the signal to the adder 27 is not performed.

The correlation value calculating means 21 calculates a correlation value between the current frame and the previous frame stored in the memory 28. The error concealment frame determining means 22 includes a switch 22a and a determining means 22b.
2b determines from the identification flag and the correlation value whether to use the current frame or the previous frame as the error concealment frame.

The switch 22a switches to the terminal a3 if the error concealment frame is the previous frame, and switches to the terminal a4 if the error concealment frame is the current frame, based on the instruction from the judgment means 22b.

The error concealment processing means 23 performs error concealment processing based on the error block information and the error concealment frame, and outputs a final decoded image without error blocks. As described above, according to the present invention, a correlation value between a previous frame and a current frame is calculated from a frame sequence including an I picture and a P picture, and an error is calculated using at least one of the information of the identification flag and the correlation value. The error concealment frame used for concealment is determined to be the previous frame or the current frame, and the error concealment process is adaptively performed.

As a result, efficient error concealment control in consideration of frame correlation can be performed, and image quality and quality can be improved. In addition, even in an environment where transmission path errors are severe, such as a wireless environment, quality deterioration due to transmission path errors at the start of code (at the start of communication) and at the time of scene change can be efficiently suppressed. Less image transmission becomes possible.

[0048]

As described above, according to the image control system of the present invention, in the image encoding apparatus, a frame sequence composed of an I picture and a P picture and an identification flag for identifying the I picture and the P picture are used. Transmitting, the image decoding apparatus calculates a correlation value between the previous frame and the current frame, and determines at least one of the information of the identification flag and the correlation value to determine an error concealment frame to be used for error concealment. Thus, the configuration is such that the error concealment processing is adaptively performed. As a result, efficient error concealment control in consideration of frame correlation can be performed, and image quality and quality can be improved.

Further, the image decoding apparatus of the present invention calculates a correlation value between the previous frame and the current frame for the received frame sequence, and uses at least one of the information of the identification flag and the correlation value to generate an error. An error concealment frame used for concealment is determined, and error concealment processing is adaptively performed. As a result, efficient error concealment control in consideration of frame correlation can be performed, and image quality and quality can be improved.

[Brief description of the drawings]

FIG. 1 is a principle diagram of an image control system of the present invention.

FIG. 2 is a diagram illustrating an example of a frame sequence.

FIG. 3 is a diagram showing a state transition of determination of a frame for error concealment. (A) is a state transition diagram, and (B) is a table showing the state transition diagram of (A).

FIG. 4 is a diagram illustrating an application example of the state transition illustrated in FIG. 3;

FIG. 5 is a diagram showing a state transition of determination of a frame for error concealment. (A) is a state transition diagram, and (B) is a table showing the state transition diagram of (A).

FIG. 6 is a diagram illustrating an application example of the state transition illustrated in FIG. 5;

FIG. 7 is a diagram illustrating a configuration of an image decoding device.

FIG. 8 is a diagram showing a problem of the conventional error concealment.

[Explanation of symbols]

 REFERENCE SIGNS LIST 1 image control system 10 image encoding device 11 encoding means 12 image transmitting means 20 image decoding apparatus 21 correlation value calculating means 22 error concealment frame determining means 23 error concealment processing means

 ──────────────────────────────────────────────────続 き Continuing on the front page (72) Inventor Eiji Morimatsu 4-1-1, Kamiodanaka, Nakahara-ku, Kawasaki-shi, Kanagawa Prefecture Inside Fujitsu Limited (72) Inventor Atsushi Ichiki 4 Kamidadanaka, Nakahara-ku, Kawasaki-shi, Kanagawa Prefecture 1-chome No. 1 Fujitsu Limited F term (reference) 5C059 PP05 PP06 RF01 TA76 TA80 TB04 TC14 TD01 UA05 5K014 AA01 FA07 FA08 FA11

Claims (4)

[Claims] 1. An image control system for encoding and decoding an image and transmitting the image, comprising: an encoding unit that encodes the image to generate an I picture and a P picture; An image encoding device configured to transmit an identification flag for identifying a picture, and a frame sequence including the I picture and the P picture; A correlation value calculating unit that calculates a correlation value between the frame and the current frame; and at least one of the information of the identification flag and the correlation value, the error concealment frame used when performing error concealment, and The error concealment frame determining means for determining whether to use the frame as the current frame or the current frame; and performing the error concealment adaptively using the determined error concealment frame. Image control system comprising: the error concealment processing unit, and the image decoding device comprising, a to decode Te. 2. The error concealment frame determining means determines a first I picture generated at the start of image coding or a scene change and a second I picture at the time of switching from a P picture. The error concealment frame as a current frame, a third I picture following the first I picture or the second I picture,
2. The image control system according to claim 1, wherein the error concealment frame is a previous frame for a P picture. 3. An image decoding apparatus for decoding an image, comprising: a frame sequence including an I picture and a P picture;
Correlation value calculation means for calculating a correlation value between a previous frame and a current frame, an identification flag for identifying the I picture and the P picture, and at least one of the correlation value,
Error concealment frame determining means for determining whether the error concealment frame to be used for error concealment is the previous frame or the current frame; and using the determined error concealment frame, An image decoding apparatus, comprising: an error concealment processing unit for performing concealment and decoding. 4. The error concealment frame determining means determines whether a first I picture generated at the start of image encoding or a scene change occurs and a second I picture at the time of switching from a P picture. The error concealment frame as a current frame, a third I picture following the first I picture or the second I picture,
4. The image decoding apparatus according to claim 3, wherein the error concealment frame is a previous frame for a P picture.