JP2008042769A - Moving image decoder and moving image decoding method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To display appropriate reproduction images by switching whether to perform error concealment processing or stop decoding for moving image data in which errors are mixed. <P>SOLUTION: The moving image decoder includes an error detection part 1 for fetching an arriving moving image frame and detecting errors, a decoding part 3 for decoding the moving image frame, a concealment part 4 for restoring the error of the moving image frame, an image buffer 5 for storing image data decoded by the decoding part 3 or image data restored by the concealment part 4, and a control part 20. In the control part 20, when the results detected by the error detection part 1 indicate the error, restoration by the concealment part 4 or restoration stop until an I frame to arrive next is selected on the basis of an interval to the I frame to arrive immediately after the frame wherein the error is generated. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a moving picture decoding apparatus including a concealment unit that repairs a moving picture frame error, and a moving picture decoding method used in the apparatus.

  Conventionally, for example, the video compression standard H.264 defined by ITU-T. When moving image data encoded using H.264 is transmitted, an error may be mixed. In order to cope with this, error concealment for recovering moving image data including an error and obtaining appropriate image data is obtained. Processing is in progress.

  On the other hand, when a frame containing an error is referenced, error propagation occurs between the frames until the refreshed I frame is sent next. For this reason, when an error is mixed, a method of freezing the display image using the image data decoded immediately before is known.

  However, when the freeze method is used, if an error is mixed immediately after the arrival of the I frame, there is time until the refreshed I frame is sent next, and an image that has not changed for a while. There was a problem that the significance of video distribution was lost.

On the other hand, in Patent Document 1, the decoding information from the decoder and the number of error processing macroblocks are used. When the number of error processing blocks exceeds the threshold, a still image is output, and when the number of error processing blocks falls below the threshold, the decoder A method for resuming the output of decryption information from is disclosed.
JP 2003-116136 A

  In decoding of transmitted moving image data, a moving image decoding device that appropriately performs error concealment processing on moving image data mixed with an error or stops decoding to make a still image and this device Therefore, there is a need for a video decoding method used in Japanese.

  The present invention has been made as a solution to the above-described problem relating to the decoding of moving image data, and its purpose is to perform error concealment processing or stop decoding on moving image data mixed with errors. It is an object to provide a moving picture decoding apparatus capable of switching and displaying an appropriate reproduced image and a moving picture decoding method used in this apparatus.

  A video decoding device according to the present invention includes an error detection unit that takes in an incoming video frame and detects an error, a decoding unit that decodes a video frame, a concealment unit that repairs an error in the video frame, Immediately after the frame in which an error occurred when the image memory that stores the image data decoded by the decoding unit or the image data restored by the concealment unit and the detection result by the error detection unit indicate an error And a control unit that selects restoration by the concealment unit or restoration stop until the next I frame, based on the time until the I frame that arrives at.

  In the moving image decoding apparatus according to the present invention, when the control unit selects restoration stop until the next incoming I frame, the image data already stored in the image memory is used as reproduction image information. To do.

  The moving image decoding method according to the present invention is a moving image decoding method in which image data obtained by decoding a moving image frame is used as reproduced image information, or image data obtained by correcting an error in the moving image frame is used as reproduced image information. An error detection step of capturing an incoming moving image frame and detecting an error, and when the detection result of the error detection step indicates an error, based on the time from the frame in which the error has occurred to the immediately following I frame, And a control step for selecting whether to perform error correction or to stop error correction until the next incoming I frame.

  The moving image decoding method according to the present invention includes a step of storing decoded image data or restored image data in an image memory, and when the control step selects restoration stop until the next incoming I frame. Is characterized in that the image data already stored in the image memory is used as reproduced image information.

  In the present invention, when a moving image frame is captured and error detection is performed, and the detection result indicates an error, error correction is performed based on the time until the I frame that arrives after the frame in which the error has occurred, or the next arrival Since it is selected whether or not the error repair is stopped until the I frame to be performed, it is possible to appropriately perform the switching process between the error concealment process or the still image for the moving image data in which the error is mixed.

  Embodiments of a moving picture decoding apparatus and a moving picture decoding method according to the present invention will be described below with reference to the accompanying drawings. In each figure, the same components are denoted by the same reference numerals, and redundant description is omitted. In FIG. The structure of the access unit prescribed | regulated to H.264 is shown. The access unit is a collection of information in the bitstream in units of pictures, and is composed of one to a plurality of NALs. NAL units in the main picture are included in all access units. An AU limiter indicating the head of the access unit, an SPS in which the profile and level of the image are set, a PPS that is a header indicating the coding mode of the entire picture, and an SEI that is timing information of each picture are added.

  As shown in FIG. 2, the NAL unit has a data structure in which an RBSP generated by the VCL is added after the head NAL header. A nal_unit_type indicating the type of the NAL unit is added to the NAL header.

  The detection of the I frame is performed by specifying the encoding method based on the nal_unit_type. The time information of each frame is acquired based on BPSE (buffering period SEI) or PTSEI (picture timing SEI) or POC (Picture Order Count), or based on a time stamp added in a lower layer at the time of transmission. .

  In the case of MPEG-4 Simple Profile, there are two types: P-VOP (Video Object Plane) indicating inter-frame encoding and I-VOP indicating intra-frame encoding. In MPEG-4 or the like, time information is obtained by PTS (Presentation Time Stamp), and an encoding method is specified by a slice header.

  FIG. 3 shows the main part of an embodiment of the moving picture decoding apparatus that receives the moving picture data as described above and decodes it. As shown in the block diagram of FIG. 3, the main parts of the video decoding device are an error detection unit 1, a control unit 20, a decoding unit 3, a concealment unit 4, a decoded image buffer 5, a frame memory 6, and an image output unit 7. It has composition which comprises. The encoded video stream V has a data structure as shown in FIG.

  The error detection unit 1 detects whether an error is included in the moving image stream V. Note that error detection is performed when the encoded data cannot be decoded based on the table, or by using a decoder function that detects an error between parameters. The control unit 20 includes an encoding type detection unit 22, and the encoding type detection unit 22 checks nal_unit_type and recognizes an I frame. The control unit 20 includes an I frame period detection unit 21, and obtains a period (average period) in which the I frame is detected by the encoding type detection unit 22.

  Further, the control unit 20 is provided with a switching control unit 23. When the detection result by the error detection unit 1 indicates an error, the switching control unit 23 includes a time period from the frame to an I frame that arrives later. Based on the threshold value set in advance, the restoration by the concealment unit 4 or the restoration stop until the next I frame is selected.

  The decoding unit 3 decodes the slice data with reference to the image stored in the decoded picture buffer 5 (DPB: Decoded Picture Buffer) according to the encoding type, and stores the slice data in the decoded image buffer 5 again. When the detection result by the error detection unit 1 indicates an error, the concealment unit 4 performs concealment processing (restoration processing) with reference to the image stored in the decoded image buffer 5 based on the encoding type, Are stored in the decoded image buffer 5 again. Here, as the concealment process (restoration process) performed by the concealment unit 4, for example, a technique disclosed in JP-A-2005-311512 can be used.

The frame memory 6 stores the reproduced image in units of frames. The image output unit 7 reads out the frame image data from the frame memory 6 and performs frame-by-frame display to provide a moving image. The control unit 20 is connected to a threshold setting unit 31 and is configured to set a threshold T _th used for selection processing for restoration by the concealment unit 4 or restoration stop until the next I frame. .

In the moving picture decoding apparatus configured as described above, the control unit 20 operates according to the flowchart shown in FIG. 4, and the operation will be described based on this. Decoding is started by the arrival of the moving image data encoded, I-frame period detecting section 21 calculates the I-frame interval T _i (S11). This I frame period T _i is calculated at any time during the decoding process because I frames are not inserted at fixed intervals. As a calculation method, there are a method of taking an average at I-frame intervals of several cycles, a method of taking a majority decision, a method of taking a maximum value, and the like. If an error is detected during this detection, the error is excluded from the target for calculating the period T _i of the I frame.

  Following the process of step S11, a picture boundary is detected (S12). Here, H. The H.264 bit stream is taken as an example, and picture boundary detection is performed using nal_unit_type. Subsequent to step S12, based on the output of the error detection unit 1, it is detected whether or not the frame contains an error (S13). If the frame does not contain an error, the decoding unit 3 is instructed to perform decoding, and as long as the encoded moving image data arrives, the process returns to “start loop” and continues. As a result, the image data decoded by the decoding unit 3 is accumulated in the decoded image buffer 5, sent to the frame memory 6 and accumulated. The frame image data stored in the frame memory 6 is read by the image output unit 7 and a moving image is displayed by frame advance display.

  On the other hand, if it is detected in step S13 that the frame contains an error, it is detected whether or not the current process skips to the next I frame (S14). If it is detected in step S14 that skipping is in progress, the process returns to "start loop" and continues.

If it is detected in step S14 that the skip to the next I frame is not in progress, the time interval T from the frame in which the error is mixed to the next I frame is calculated (S15). Specifically, the time information t ₁ of the immediately preceding I frame (I frame in which no error is mixed) shown in FIG. 5 is detected and held by the control unit 20 from the time stamp. Further, the time information t _error of the current frame F _error in which the error is mixed is detected by the control unit 20 from the time stamp. The time interval between the I frame (predicted I frame) that arrives immediately after the current frame F _{error and} the immediately preceding I frame is the I frame period Ti calculated by the I frame period detection unit 21 in step S11. Therefore, the time interval T until the next I frame is obtained by the control unit 20 as T = T _i − (t _error −t ₁ ).

Subsequent to step S15, the switching control unit 23 compares the threshold T _th set by the threshold setting unit 31 with the time interval T to the next I frame obtained in step S15 (S16). As a result of this comparison, if the time interval T is larger than the threshold value T _th , the process branches to YES in step S16, and an instruction is given to the concealment unit 4 to perform the concealment process (S17). The result of the concealment process by the concealment unit 4 is sent to the decoded image buffer 5 and output to the frame memory 6 for use in moving image display. When the process of step S17 is completed, the process is continued by returning to “loop start” as long as the encoded moving image data arrives, and ends when no moving image data arrives.

If the time interval T is equal to or smaller than the threshold value T _th as a result of the comparison in step S16, the process branches to NO in step S16, and the switching control unit 23 gives a non-operation instruction to the decoding unit 3 and the concealment unit 4. Then, a process of skipping to the next I frame is performed (S18). In this skip process, the processes of the encoding unit 3 and the concealment unit 4 are stopped, and image display using the image data already stored in the frame memory 6 is performed. That is, a still image is displayed. When the process of step S17 is completed, the process is continued by returning to “loop start” as long as the encoded moving image data arrives, and ends when no moving image data arrives.

The threshold value T _th set by the threshold value setting unit 31 is time here, but a ratio (1/3, 1/2, etc.) to the I frame period Ti is set so that the control unit 20 converts it to the threshold value T _th. Anyway. Also, it may be determined whether or not to skip using the ratio itself as a threshold. In any case, when the user sets a threshold value by the threshold value setting unit 31, a moving image is displayed with an image quality desired by the user. That is, when displaying an image with smooth movement, the threshold value T _th is set to be large, and when displaying an image with clear image quality, the threshold value T _th is set to be small. Will do.
In the embodiment shown here, the example in which the time information is set as the threshold value is shown, but the same effect can be obtained by performing comparison based on the number (ratio) of decoded frames starting from the I frame.

H. is an example of a coding method of moving image data that arrives at the moving image decoding apparatus of the present invention. The figure which shows the structure of the access unit by H.264. H. is an example of a coding method of moving image data that arrives at the moving image decoding apparatus of the present invention. The figure which shows the structure of the NAL unit by H.264. The block diagram which shows the principal part of the Example in the moving image decoding apparatus of this invention. The flowchart for showing the operation | movement of the Example in the moving image decoding apparatus of this invention. The figure for demonstrating the time interval calculation from the error mixing frame to the I frame immediately after in the Example in the moving image decoding apparatus of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Error detection part 3 Decoding part 4 Concealment part 5 Decoded image buffer 6 Frame memory 7 Image output part 20 Control part 21 Encoding classification detection part 22 I frame period detection part 23 Switching control part 31 Threshold setting part

Claims (4)

An error detection unit that captures incoming video frames and detects errors;
A decoding unit for decoding a moving image frame;
A concealment unit that repairs an error in the video frame;
An image memory for storing the image data decoded by the decoding unit or the image data restored by the concealment unit;
When the detection result by the error detection unit indicates an error, the restoration by the concealment unit or the restoration stop until the next I frame arrives based on the interval from the frame in which the error has occurred to the I frame that comes immediately after A moving picture decoding apparatus comprising: a control unit that selects the moving picture decoding apparatus; 2. The moving image decoding according to claim 1, wherein the control unit uses the image data already stored in the image memory as reproduction image information when the restoration stop is selected until the next incoming I frame. apparatus. In the moving picture decoding method in which the image data obtained by decoding the moving image frame is used as the reproduction image information, or the image data obtained by correcting the error of the moving image frame is used as the reproduction image information
An error detection step for capturing an incoming video frame and detecting an error;
When the detection result of the error detection step indicates an error, the error is repaired or the error is repaired up to the next I frame based on the interval from the frame in which the error has occurred to the immediately following I frame. And a control step of selecting whether to stop the video decoding method. Storing the decoded image data or the restored image data in an image memory,
4. The moving image decoding according to claim 3, wherein when the restoration stop is selected until the next incoming I frame in the control step, the image data already stored in the image memory is used as reproduction image information. Method.

Images