JP2008035216A - Moving picture decoder - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To determine reliability with respect to auxiliary time information when reproduction time information needs correction and to perform compensation of the time information more appropriately. <P>SOLUTION: The decoder is equipped with: a separation part 11 for separating video information and reproduction time information from a stream where the video information and reproduction time information are multiplied by the packet, detecting transmission errors by the packet, and outputting a transmission system error detection signal; a moving picture signal decoding part 21 for decoding the video information and outputting the decoded video and the auxiliary time information; an auxiliary time reliability determining part 23 for determining reliability of the packet where the auxiliary time information is stored on the basis of the transmission system error detection signal; a reproduction time correction part 26 for correcting the reproduction time information on a video on the basis of the determination result by the auxiliary time reliability determining part 23 and the auxiliary time information; and a reproduction output part 25 for displaying the decoded video on the basis of the reproduction time information corrected by the reproduction time correction part 26. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a moving picture decoding apparatus that receives a stream in which video information and playback time information are multiplexed in units of packets, decodes the video information, and displays the decoded video based on the playback time information.

Conventionally, when performing reliability determination on playback time information (PTS) of decoded video, and when performing correction at the time of error on playback time information (PTS), the latest playback time information determined to be correct in the past A system that uses auxiliary time information in a bitstream is disclosed in Patent Document 1.
JP 2002-135777 A

  For example, when the conventional method described in the above-mentioned Patent Document 1 is adopted with respect to the encoding method of the H.264 recommendation by ITU-T adopted by MPEG-2 or the like, it becomes as follows. The example shown in FIG. 1 assumes an example at a variable frame rate of 1 PES (Packetized Elementary Stream) = nAU (Access Unit) (n is an integer of 1 or more). Here, PES is a packet-multiplexed video encoded data, and AU is a unit in which NALs (units of data delimiters in H.264) are grouped in units of pictures.

  The PES header stores a PTS (Presentation Time Stamp) that is a media playback time stamp. However, in a so-called error environment such as wireless communication, the PTS value may not be normal. In such a case, a certain time interval obtained from auxiliary time information in the bitstream is added to the latest PTS decoded in the past. Addition is performed to determine whether the value falls within the interval, thereby determining the reliability of the PTS. And when the determination result that it is unreliable arises, it correct | amends using auxiliary time information. The auxiliary time information is information used when the time stamp of the moving image cannot be acquired or when the reliability is low, and an offset from the reproduction time information correctly acquired before the determination (determination that it is not reliable) is used. It is set for the purpose of correcting the time information.

  By the way, in H.264, NAL called SEI (Supplemental Enhancement Information) is used as auxiliary time information. The SEI includes a buffering period SEI and a picture timing SEI. The buffering period SEI is information indicating a buffering period of an image starting from an IDR picture. In the example of FIG. 1, the buffering period is started with the IDR picture of frame 0 (frame 0). On the other hand, the picture timing SEI includes a syntax of cpb_removal_delay, and this value can be used as an offset from the buffering period.

  If the PTS can be acquired normally, the image is displayed according to the PTS. If it is detected that the PTS has an incorrect value, the display time is calculated using an offset starting from the IDR picture. Actually, cpb_removal_delay represents the extraction delay of the encoded stream from the buffer DPB (Decoded Picture Buffer), and in order to indicate the display time, it is necessary to consider the extraction time from the DPB. However, in a decoding device with display order = decoding order, dpb_output_delay (removal time from DPB) = 0 may generally be set, and the description will be made on the assumption of this case.

  When the technique disclosed in the cited document 1 is applied to the SEI reliability determination, it is regarded as an error if the reproduction time does not fall within a certain fixed time interval, but the frame interval varies at a variable frame rate. It does not always fit within a fixed interval. In the example of FIG. 1, when the offset value in the non-IDR picture of the frame N (frame N) is compared with the offset value (cpb_removal_delay) in the non-IDR picture of the frame (N-1) (frame N-1), the value is 26. → 32, it is considered an error because the increment of the time interval greatly exceeds the average increment so far. However, there are cases where the time exceeding the upper limit of the time interval is correct at the variable frame rate. In the example of FIG. 1, the degree of increment is not so steep, and frames 1 to (N−1) (frame 1, frame 2,..., Frame N−1) that have playback times within the interval are normal. In fact, there are cases where it is wrong. It is difficult to determine an appropriate correction fixed time interval for a varying frame rate.

  The technique disclosed in Patent Document 1 has an advantage that the reproduction time of an image at the time of PTS error can be corrected with a value close to the original reproduction time by using auxiliary time management information. The determination about the reliability of the auxiliary time information itself is not taken into consideration. For this reason, when the auxiliary time information includes an error, not only correction but also PTS reliability determination cannot be performed correctly.

  Conventionally, the problem to be solved by the present invention is that the reliability of the auxiliary time information itself is not considered, and if the auxiliary time information includes an error, not only the correction but also the reliability of the PTS. The present invention has been made in order to solve such a problem, and its purpose is to perform reliability determination on auxiliary time information when it is necessary to correct reproduction time information. And providing a moving picture decoding apparatus capable of correcting time information more appropriately.

  The moving picture decoding apparatus according to the present invention separates video information and reproduction time information from a stream in which video information and reproduction time information are multiplexed in units of packets, detects transmission errors in units of packets, and transmits transmission system error detection signals. A video signal decoding unit that decodes the video information and outputs decoded video and auxiliary time information, and a packet that stores the auxiliary time information based on the transmission system error detection information. An auxiliary time reliability determination unit that determines reliability, a reproduction time correction unit that corrects video reproduction time information based on a determination result by the auxiliary time reliability determination unit and the auxiliary time information, and the reproduction time correction And a reproduction output unit for displaying the decoded video based on the reproduction time information corrected by the unit.

  According to the present invention, the auxiliary time reliability determination unit and the reproduction time correction unit are provided, and the reliability of the packet storing the auxiliary time information is determined based on the transmission system error detection information. Since the playback time information of the video is corrected based on the time information, it becomes possible to determine the reliability of the auxiliary time information itself used for the correction when correcting the time when the playback time error is detected. The accuracy of the reproduction time correction can be improved.

  Embodiments of a video decoding device according to the present invention will be described below with reference to the accompanying drawings. FIG. 2 shows an embodiment of the moving picture decoding apparatus according to the present invention. The moving picture decoding apparatus according to the present invention includes, for example, a separation unit 10 that separates multiple streams captured via a wireless line. The multiplexed stream is obtained by packetizing an audio bit stream and a moving image bit stream for each frame or for each of a plurality of frames to form a PES packet, and further converting the PES packet into a fixed-length TS (Transport Stream) packet. . A time stamp (PTS) that is reproduction time information for reproduction output is added to the header of the PES packet. Further, reference time calibration information (generally, program time reference value PCR: Program Clock Reference) for synchronizing the time on the transmission side and the reception side is added to the header of this TS packet.

  FIG. 3 shows the TS packet format. As described above, a TS (Transport Stream) packet is a packet obtained by further packetizing a PES packet and used in the MPEG-2 system. The header portion is provided with a “transport_error_indicator” flag. If this “transport_error_indicator” flag is set, it indicates that an error exists in the packet. NAL (data delimitation unit in H.264) is divided and stored in a plurality of TS packets, but by holding the error flag value of each packet and creating a logical sum for each NAL delimiter, It is determined whether an error is included in NAL units.

  The separation unit 10 extracts video information, reproduction time information, a transmission system error detection signal, audio information, and a program time reference value PCR from the multiplexed stream and outputs them. The video information is sent to the moving picture signal decoding unit 21, the playback time information is sent to the playback time reliability determination unit 22 and the audio playback output unit 32, and the transmission system error detection signal is sent to the playback time reliability determination unit 22 and the auxiliary time. The program time reference value PCR is sent to the time management unit 24, and the audio information is sent to the audio signal decoding unit 31.

  The moving image signal decoding unit 21 decodes the video information into a decoded video signal and sends it to the reproduction output unit 25, and also extracts auxiliary time information SEI in the bitstream to extract the auxiliary time reliability determination unit 23 and the reproduction time correction. This is sent to the unit 26. The reproduction time reliability determination unit 22 determines the reliability of the reproduction time information based on the transmission system error detection signal, and sends the reproduction time reliability determination information as a result to the auxiliary time reliability determination unit 23. When it is determined that the reproduction time information is reliable, the reproduction time information is sent to the reproduction output unit 25 and the reproduction time correction unit 26.

  The auxiliary time reliability determination unit 23 determines the reliability of the auxiliary time information according to the reproduction time reliability determination information sent from the reproduction time reliability determination unit 22. That is, when the reproduction time reliability determination unit 22 determines that the reproduction time information is not reliable, the auxiliary time reliability determination unit 23 determines the reliability of the auxiliary time information based on the transmission system error detection signal. The reliability determination information, which is the determination result, is sent to the reproduction time correction unit 26.

  The reproduction time correction unit 26 performs time correction according to the reliability determination information sent from the auxiliary time reliability determination unit 23 and sends correction time information as a correction result to the reproduction output unit 25. That is, when the auxiliary time reliability determination unit 23 determines that the auxiliary time information is reliable, the reproduction time correction unit 26 creates a correction time based on the previous reproduction time information and the auxiliary time information SEI. On the other hand, when the auxiliary time reliability determination unit 23 determines that the auxiliary time information is not reliable, the auxiliary time information is not used and the reproduction time is corrected using the time interval obtained by the predetermined process. And output as corrected time information.

  The time management unit 24 calibrates the reference time information (STC) based on the program time reference value (PCR) extracted from the header information of the TS packet. This reference time information (STC) is sent to the audio reproduction output unit 32 and the reproduction output unit 25 and used as a reference for synchronous reproduction. The reproduction output unit 25, based on the reference time information sent from the time management unit 24, the reproduction time information extracted by the separation unit 11 when the reproduction time reliability determination unit 22 determines that the reproduction time information is reliable. When the playback time reliability determination unit 22 determines that the playback time information is not reliable, the decoded video signal is decoded at a time according to the correction time sent from the playback time correction unit 26. The video signal is transmitted to the display device 27.

  The audio reproduction output unit 32 sends the decoded audio signal to the audio output device 33 when the reproduction time information extracted and sent by the separation unit 11 matches the reference time information (STC) sent from the time management unit 24. .

  In the moving picture decoding apparatus configured as described above, each unit operates as follows. The separation unit 11 separates the video information and the reproduction time information from the multiplexed bit stream. The separated video information is decoded by the moving image signal decoding unit 21 and sent to the reproduction output unit 25. The reproduction time information separated by the separation unit 11 is sent to the reproduction time reliability determination unit 22, and reliability determination is performed by a transmission system error detection signal sent from the separation unit 11. Here, the reproduction time reliability determination unit 22, the auxiliary time reliability determination unit 23, and the reproduction time correction unit 26 perform processing based on the flowchart shown in FIG. 4, and the operation will be described with reference to FIG.

  Reproduction is performed every time the frame N (N is an integer) arrives, and the processing of the reproduction time reliability determination unit 22 or the auxiliary time reliability determination unit 23 is performed depending on whether the time information PTS of one frame can be extracted. The That is, since the playback time information PTS can be extracted from the first AU in 1 PES = 1 AU or 1 PES = nAU (n is an integer equal to or greater than 1), this is detected (S11), and the process branches to YES upon detection of the playback time information PTS. Then, the error detection information (“transport_error_indicator” flag) of the transmission system is extracted to determine the reliability of the reproduction time information PTS (S12). In step S12, the reproduction time reliability determination unit 22 decodes the reproduction time information PTS in the NAL format (creates a logical sum of “transport_error_indicator” flags), determines an error in the reproduction time information PTS, and reproduces the reproduction time reliability. Output sex determination information. With this process, it is not necessary to use the auxiliary time information SEI for the reliability determination of the reproduction time information PTS, and the reproduction time reliability can be independently determined regardless of whether the auxiliary time information SEI has an error.

  If it is determined in step S12 that the reproduction time information is reliable, the information is set as reproduction time information and sent to the reproduction output unit 25 (S13). On the other hand, if it is determined in step S12 that the reproduction time information is not reliable, the process proceeds to the process of the auxiliary time reliability determination unit 23 (step S14). The auxiliary time reliability determination unit 23 determines whether the auxiliary time information extracted by the video signal decoding unit 21 is a reliable value based on the transmission system error detection signal sent from the separation unit 11 (S14). Specifically, in step S14, an error included in the NAL of the SEI is detected by setting and resetting the “transport_error_indicator” flag. If the “transport_error_indicator” flag is not set and no error is detected, the SEI offset value (cpb_removal_delay) is added to the previous reproduction time information PTS to obtain correction time information and output it to the reproduction output unit 25 ( S15). In this case, as shown in FIG. 5, frames 0 to N (frame 0, frame 1, frame 2,..., Frame N−1, frame N) arrive and a frame (N−1) Even if the interval (cpb_removal_delay) between (frame N-1) and frame N (frame N) is larger than the increment of the previous interval (cpb_removal_delay), the value may be trusted.

  In step S14, when it is detected that the “transport_error_indicator” flag is set and it is determined that the flag is not reliable, the image is represented by a representative value such as an average frame interval without using an offset value (cpb_removal_delay). Is corrected (S16). Here, the average value of the average frame interval or the like may be used by obtaining an average of the frame intervals for a predetermined frame immediately before being determined to be unreliable, or may be used when receiving the multiplexed stream. The average of frame intervals may be obtained and stored for a predetermined frame continuously or at an appropriate timing, and this may be used.

  Due to the processing of the reproduction time reliability determination unit 22, auxiliary time reliability determination unit 23, and reproduction time correction unit 26 as described above, the reproduction time information is trusted by the reproduction time reliability determination unit 22. If it is determined that the reproduction time information can be reproduced, the reproduction time information extracted by the separation unit 11 is sent. If the reproduction time reliability determination unit 22 determines that the reproduction time information is not reliable, the reproduction time information is transmitted. Correction time information is sent from the correction unit 26. Therefore, the reproduction output unit 25 reproduces the decoded video signal according to one of the reproduction time information or the correction time information and the reference time information sent from the time management unit 24 and transmits the decoded video signal to the display device 27.

  In the present embodiment, the MPEG-2 TS packet is described as an example of the transmission system, but the present invention is not limited to this.

The figure for demonstrating the procedure which determines the reliability of reproduction | regeneration time by the method of a prior art example in the system which transmits with the variable frame rate of the encoding system of a H.264 recommendation. The block diagram which shows the Example of the moving image decoding apparatus which concerns on this invention. The figure which shows the TS packet format transmitted in the moving image decoding apparatus which concerns on this invention. The flowchart which shows the process which determines the reliability of the reproduction time in the moving image decoding apparatus which concerns on this invention, and auxiliary time. The figure for demonstrating the procedure which determines the reliability of reproduction | regeneration time by the method of the Example which concerns on this invention in the system which transmits with the variable frame rate of the encoding system of a H.264 recommendation.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Separation part 21 Video signal decoding part 22 Reproduction time reliability determination part 23 Auxiliary time reliability determination part 24 Time management part 25 Reproduction output part 26 Reproduction time correction part 27 Display device

Claims (3)

Separating the video information and the reproduction time information from the stream in which the video information and the reproduction time information are multiplexed in units of packets, and detecting a transmission error in units of packets and outputting a transmission system error detection signal;
A video signal decoding unit that decodes the video information and outputs decoded video and auxiliary time information;
Based on the transmission system error detection information, an auxiliary time reliability determination unit that determines the reliability of the packet in which the auxiliary time information is stored;
A reproduction time correction unit for correcting the reproduction time information of the video based on the determination result by the auxiliary time reliability determination unit and the auxiliary time information;
A moving image decoding apparatus comprising: a reproduction output unit that displays the decoded video based on reproduction time information corrected by the reproduction time correction unit. When it is determined that the auxiliary time information is reliable based on the reliability determination result received from the auxiliary time reliability determination unit, the reproduction time correction unit is obtained from the previously acquired reproduction time information and the auxiliary time information. While correcting the reproduction time information of the decoded image based on the addition of the time interval information, when it is determined that the auxiliary time information is not reliable, the latest reproduction time information acquired in the past and the average frame interval information are added. The moving image decoding apparatus according to claim 1, wherein the reproduction time information of the decoded image is corrected based on the decoded image. The moving picture decoding apparatus according to claim 1, further comprising a reproduction time reliability determination unit that determines reliability of the reproduction time information based on the transmission system error detection information.

Images