JP4390666B2 - Method and apparatus for decoding and reproducing compressed video data and compressed audio data - Google Patents

Description

  The present invention relates to a method for decoding and reproducing compressed video data and compressed audio data, and more particularly to a method for performing synchronous reproduction of video data and audio data during intermediate reproduction.

  There is an MPEG1 system as a system for compressing and encoding digitized video data and audio data and multiplexing the data into a single data stream. The data structure of the MPEG1 system stream will be described with reference to FIG. A video data stream and an audio data stream that are compression-encoded according to the MPEG1 system are called elementary streams. These elementary streams are divided and multiplexed into units called packets. A packet header is added to the packet. The packet header includes a stream identification code for identifying a data stream, a PTS (Presentation Time Stamp) that is time information for performing synchronous reproduction, and, in the case of image data, data decoding. A DTS (Decoding Time Stamp), which is time information indicating the order, is described.

  Furthermore, a unit called a pack is formed by a set of an arbitrary number of packets. A pack header is added to the pack, and a pack start code, reference clock information called SCR (System Clock Reference), and the like are described in the pack header. In the MPEG1 system, a reference time called STC (System Time Clock) is defined in order to perform synchronized playback of video and audio, and the MPEG1 decoding / playback apparatus has a 90 kHz STC. The conventional MPEG1 decoding / playback apparatus determines the reference time required for synchronous playback of video and audio by reproducing the reference time at the time of encoding with STC by referring to the SCR in the pack header.

  Also, as shown in FIG. 3, a system header can be added after the pack header, and the bit rate of each elementary stream is described in the system header. In this manner, a stream obtained by multiplexing a plurality of elementary streams such as video and audio in units of packs is called an MPEG1 system stream.

  Next, the configuration of a conventional MPEG1 decoding / playback apparatus 40 that performs synchronized playback of video data and audio data from an MPEG1 system stream will be described with reference to FIG. When performing decoding and reproduction, the MPEG1 system stream output from the transfer device 41 is input to the DEMUX 42. Here, the transfer device 41 is a device that extracts and outputs an MPEG1 system stream from a storage medium such as a CD-ROM or a hard disk. Based on the stream identification code in the packet header, the DEMUX 42 converts the input MPEG1 system stream into a video data stream (video elementary stream, hereinafter referred to as video ES) and an audio data stream (audio elementary stream, hereinafter referred to as audio ES). The demultiplexers are separated into a video decoder 44 and an audio decoder 45, respectively.

  Further, the DEMUX 42 outputs the SCR value extracted from the pack header to the STC generation unit 43. The STC generation unit 43 performs adjustment so that the received SCR value matches its own STC, and distributes the STC to the video decoder 44 and the audio decoder 45. Specifically, the reference time at the time of encoding in the SCR is indicated by a counter value in units of 90 kHz, and the STC generation unit 43 matches the value of the STC counter included in the SCR with the received SCR. Reproduce the reference time for encoding.

  The video decoder 44 decodes the video ES received from the DEMUX 42 and outputs a video signal. The received video ES is temporarily stored in the input buffer 441. The DTS / PTS comparison unit 443 acquires the DTS and the PTS from the video ES stored in the input buffer 441 and compares the acquired DTS and PTS with the reference time STC distributed from the STC generation unit 43. Further, the DTS / PTS comparison unit 443 instructs the video decoding unit 442 to decode the picture included in the video ES at the timing when the DTS and STC match, and after decoding at the timing when the PTS and STC match. The video decoding unit 442 is instructed to output a video signal. The video decoding unit 442 acquires the video ES from the input buffer 441, performs decoding according to the timing instructed by the DTS / PTS comparison unit 443, and outputs the decoded video signal.

  The audio decoder 45 decodes the audio ES received from the DEMUX 42 and outputs an audio signal. The received audio ES is temporarily stored in the input buffer 451. The PTS comparison unit 453 acquires the PTS from the audio ES stored in the input buffer 451 and compares the acquired PTS with the STC distributed from the STC generation unit 43. Furthermore, the PTS comparison unit 453 instructs the audio decoding unit 452 to output the decoded audio signal at the timing when the PTS and STC match. The audio decoding unit 452 acquires the audio ES from the input buffer 451, performs decoding, and outputs a decoded video signal according to the timing indicated by the PTS comparison unit 453.

  As described above, the conventional MPEG1 decoding / playback apparatus 40 enables synchronized playback of video and audio by determining the playback output timing of video data and audio data according to the PTS included in the packet header. . Even when playback is performed from the middle of the MPEG1 system stream (hereinafter referred to as halfway playback), video and audio can be synchronized (hereinafter referred to as AV synchronization) by using PTS. In the following, AV synchronization processing using PTS during halfway playback will be described with reference to FIG. 5 and FIG.

  FIG. 5 is a timing chart showing AV synchronization during playback. The horizontal axis in FIG. 5 indicates the STC value. This STC value is determined and distributed by the STC generation unit 43 so as to match the SCR value acquired from the pack header. FIG. 5A shows the timing at which the video ES is input to the input buffer 441 included in the video decoder 44 via the transfer device 41 and the DEMUX 42 during halfway playback. In the figure, I1, P1, B1, etc. indicate frames of an I picture, a P picture, and a B picture, respectively. FIGS. 5B and 5C show the PTS value and the DTS value assigned to each picture of the video ES shown in FIG.

  Here, the I picture, the P picture, and the B picture indicate differences in the encoding method of the video frame (picture). In the MPEG1 system, the pictures constituting the video ES are the I picture, the P picture, and the picture. It is encoded into one of the B pictures. The I picture is encoded by an intra-frame encoding method that encodes using only information in its own frame (picture), and can be decoded without requiring information of other pictures. On the other hand, a P picture is obtained by encoding a difference by performing inter-frame prediction in the forward direction from a past I picture, and information on the past I picture is required for decoding. Further, the B picture is obtained by performing inter-picture predictive coding from two past and future pictures. In decoding, B picture information in addition to past I picture information is required. .

  FIG. 5F shows the timing at which the video ES is decoded in the video decoding unit 442 in FIG. 5A according to the DTS value in FIG. Here, the head of the picture to be decoded is the picture I1, and the pictures P1, B1, and B2 whose DTS values are 99 to 101 are not decoded. This is because the past I pictures necessary for decoding the pictures P1, B1, and B2 do not exist in the video ES that has been reproduced halfway, and these interframe predictive-encoded pictures cannot be decoded. Because. Therefore, decoding is started from a picture I1 that can be decoded independently. Note that the pictures P1, B1, and B2 that cannot be decoded are discarded from the input buffer 441 and the video decoding unit 442.

  FIG. 5G shows the timing when the video signal decoded according to the PTS value of FIG. 5B is output from the video decoding unit 442. As described above, since the pictures P1, B1, and B2 cannot be decoded, the video signal corresponding to I1 is output.

  On the other hand, FIG. 5D shows the timing at which the audio ES is input to the input buffer 451 included in the audio decoder 45, and FIG. 5E shows the PTS value corresponding to the audio ES in FIG. Is shown. FIG. 5 (h) shows the timing at which the audio decoding unit 442 decodes the audio ES of FIG. 5 (d) in accordance with the PTS value of FIG. 5 (e) and the decoded audio signal is output. is there. For the output of the audio signal, the output of A1 may be started from the point of time when the STC value is 100. Normally, the same PTS value as that of the video signal I1 is set from the point of time of the STC value 103 at which the video signal can be output. By starting the output of the audio signal A4, the reproduction of video and audio is started at the same time.

  FIG. 6 is a flowchart showing a procedure of AV synchronization processing using PTS. In step S1, the PTS comparison unit 453 included in the audio decoder 45 acquires the audio PTS attached to the head packet of the audio ES that is played back midway. In step S <b> 2, the video decoding unit 442 included in the video decoder 44 sequentially reads out and acquires pictures from the video ES stored in the input buffer 441. The video decoding unit 442 determines whether or not the read picture is an I picture (step S3). Since the picture that is not an I picture cannot be decoded as described above, the acquired picture is discarded and the next picture is discarded. Read. On the other hand, if the read picture is an I picture, the DTS / PTS comparison unit 443 obtains the PTS and DTS assigned to the I picture (step S4).

  In step S5, the PTS comparison unit 453 included in the audio decoder 45 compares the STC value distributed from the STC generation unit 43 with the PTS value acquired in step S1, and at the timing when both match, the audio decoding unit 452 is instructed to output the decoded audio signal. In step S6, the audio decoding unit 452 decodes the audio ES and outputs the audio signal according to the timing instructed by the PTS comparison unit 453.

  In step S7, the DTS / PTS comparison unit 443 compares the STC value distributed from the STC generation unit 43 with the PTS and DTS values acquired in step S4, and at the timing when both match, the video decoding unit Instruct 442 to decode the picture and output the decoded video signal. In step S8, the video decoding unit 442 decodes a picture and outputs a video signal according to the timing instructed by the DTS / PTS comparison unit 443.

As described above, in the conventional MPEG1 decoding / playback apparatus, the video PTS and the audio PTS start outputting video signals and audio signals at the timing when they match the values of the STC counter. Can also perform AV synchronization. Thus, for example, Patent Documents 1 and 2 disclose decoding and reproducing apparatuses that perform intermediate reproduction by establishing AV synchronization using PTS.
JP-A-7-170490 Japanese Patent Laid-Open No. 9-219838

  As described above, the AV synchronization processing performed in the conventional MPEG1 decoding / playback apparatus is performed using PTS. However, in the MPEG1 standard defined in ISO / IEC11172-1, it is not always necessary to assign a video PTS to all pictures. When a picture other than the picture located at the head of the MPEG1 system stream is packetized, the video PTS is used. It is also allowed not to give. If no video PTS is given, AV synchronization cannot be performed by the above-described method using the PTS. As described above, when the decoding and reproduction of the MPEG1 system stream to which the video PTS is not added is started, not only AV synchronization cannot be established but also the reproduction start timing of the video and audio cannot be synchronized. There is a problem that output cannot be started simultaneously. Specifically, since the reproduction timing cannot be determined based on the coincidence of PTS and STC, the output of the video signal starts from the video signal corresponding to the first acquired I picture, and the output of the audio signal is at the head of the audio ES. When starting from the corresponding audio signal, a time lag is generated in the output start time of both of them corresponding to the reproduction time of the picture discarded without being decoded before the leading I picture is acquired.

  The decoding / reproducing method according to the present invention includes a compressed video data stream and a compressed audio data stream that are composed of at least a first picture obtained by intra-coding an image frame and a second picture obtained by inter-frame predictive coding an image frame. A method for decoding and reproducing data and audio data, wherein the first picture is detected as viewed from the head of the compressed video data stream, and immediately before the first picture from the head of the compressed video data stream. Obtained by calculating a reproduction time when reproducing up to the second picture located, starting video output from video data obtained by decoding the first first picture, and decoding the compressed audio data stream From the beginning of the audio data stream, the audio data is output from the audio data after the playback time. It is intended to start.

  On the other hand, the decoding / reproducing apparatus according to the present invention decodes / reproduces video data from a compressed video data stream including at least a first picture obtained by intra-coding an image frame and a second picture obtained by inter-frame predictive coding an image frame. And a video decoder obtained by decoding the first picture as viewed from the beginning of the compressed video data stream. Video output is started, and the audio decoder corresponds to the playback time of all pictures located before the first first picture from the beginning of the audio data stream obtained by decoding the compressed audio data stream. Audio data is skipped and the playback time is Those configured to start a voice output from the voice data after the audio data.

  According to the decoding / reproducing method or the decoding / reproducing apparatus according to the present invention, the output start timing of the audio signal can be delayed by a time corresponding to the reproduction time of the picture located before the first first picture. Furthermore, since a picture positioned before the first first picture cannot be decoded, there is a time when there is no reproducible video signal at the start of reproduction of the audio signal. According to the decoding / reproducing method or the decoding / reproducing apparatus according to the present invention, reproduction can be started except for the audio signal corresponding to the time when the video signal does not exist. Thereby, even when the video PTS is not detected, it is possible to match the reproduction start timing of the audio signal with the reproduction start timing of the video signal.

  According to the present invention, even when playback time information such as PTS is not added to a compression-encoded video stream, the playback start timing of video and audio can be matched.

Embodiment 1 of the Invention
FIG. 1 shows a configuration of an MPEG1 decoding / playback apparatus 10 according to the first exemplary embodiment of the present invention. The MPEG1 decoding / playback apparatus 10 includes a picture counter 144 for counting the number of discarded pictures in the video decoder 14, and an offset calculating unit 154 that calculates an offset time at the time of outputting an audio signal from the number of discarded pictures counted in the picture counter 144. Is provided in the audio decoder 15. The functions of the transfer device 41, the DEMUX 42, and the STC generation unit 43 provided in the MPEG1 decoding / playback apparatus 10 are the same as those provided in the conventional MPEG1 decoding / playback apparatus 40, and thus description thereof is omitted.

  The video decoder 14 decodes the video ES received from the DEMUX 42 and outputs a video signal. Here, the functions of the input buffer 441 and the DTS / PTS comparison unit 443 are the same as those provided in the conventional MPEG1 decoding / playback apparatus 40 shown in FIG.

  Similar to the video decoding unit 442 provided in the conventional MPEG1 decoding / playback apparatus 40, the video decoding unit 142 obtains a picture from the input buffer 441, and decodes it according to the decoding order indicated by the DTS according to an instruction from the DTS / PTS comparison unit 443. And the decoded video signal can be output in accordance with the reproduction order indicated by PTS. In addition, the video decoding unit 142 is configured to sequentially reproduce from the first I picture when the reproduction timing instruction cannot be obtained from the DTS / PTS comparison unit 443 because the PTS cannot be acquired.

  The video decoding unit 142 is similar to the conventional video decoding unit 442 in that the P picture and the B picture are discarded until the I picture located at the head of the video ES is acquired. It is configured to notify that the picture has been discarded. The picture counter 144 is a counter that counts the number of pictures discarded until the video decoding unit 142 detects the I picture located at the head of the video ES, and receives a notification that the video decoding unit 142 has discarded the picture. The number of discarded pictures is counted sequentially.

  The audio decoder 15 decodes the audio ES received from the DEMUX 42 and outputs an audio signal. Here, the functions of the input buffer 451 and the PTS comparison unit 453 are equivalent to those provided in the conventional MPEG1 decoding / playback apparatus 40.

Offset calculation unit 154 refers to the number of discarded pictures picture counter 144 holds, calculates the reproduction time T _P corresponding to discards picture. The reproduction time _TP is the following formula (1),
T _P = PIC_CNT × T _F (1)
Calculated by Here, PIC_CNT is discarded number of pictures, _{T F} is the reproduction time per picture. The value of _TF is determined by the television signal format followed by the video ES, and is 1/30 seconds for the NTSC system and 1/24 seconds for the PAL system.

The audio decoding unit 152 obtains the audio ES from the input buffer 451 in the same manner as the audio decoding unit 452 provided in the conventional MPEG1 decoding / playback device 40, and after decoding according to the playback timing indicated by the PTS according to the instruction of the PTS comparison unit 453 The video signal can be output. In addition, the audio decoding unit 152, by discarding from the beginning of the audio signal reproduced audio data reproduction time T _P min that corresponds to the number of discarded picture calculated by the offset calculation unit 154, delays the output start timing Is configured to start output.

  With such a configuration, it is possible to match the playback start timing of video and audio even when no PTS is added to the video stream.

  Next, an AV synchronization and playback timing synchronization processing flow performed by the MPEG1 decoding / playback apparatus 10 according to the present embodiment will be described with reference to FIG. Note that the AV synchronization process when the video PTS is given to the picture and the video PTS can be acquired is the same as the conventional process described with reference to FIG. The same symbols as in FIG. 6 are given.

  First, steps S1 to S3 are the same as the conventional processing described with reference to FIG. In step S22, when the picture is discarded because the picture acquired in step S3 is a P picture or a B picture, the number of discarded pictures held by the picture counter 144 is counted up.

  In step S21, the DTS / PTS comparison unit 443 determines whether or not a video PTS is assigned to the first I picture acquired in step S3. When the video PTS is assigned, the processing from steps S4 to S8 is performed. Since these processing steps are the same as the conventional processing steps described with reference to FIG.

On the other hand, if it is determined in step S21 that the video PTS is not assigned to the first I picture, the processing from steps S23 to S25 is performed. In step S23, the offset calculation unit 154, calculates the reproduction time T _P corresponding to the number of discarded picture by the equation (1). Subsequently, in step S24, the audio decoder 152, the audio data reproduction time _{T P} min, discarded from the head of the audio ES. Finally, in step S25, the video decoding unit 142 starts outputting the video signal from the first I picture, and the audio decoding unit 152 starts outputting the audio signal from the first data after discarding the audio data.

  By processing in this way, the output start timing of the audio signal can be delayed by a time corresponding to the reproduction time of the picture discarded in the video decoder 14, and the audio signal of the portion where no video signal exists is further reduced. Excluding playback can be started. Thereby, even when the video PTS is not detected, the reproduction start timing of the video signal and the audio signal can be matched.

  In the above description, the decoding / playback apparatus that can match the playback start timing of the video signal and the audio signal when the video PTS cannot be obtained from the video ES when the MPEG1 system stream is decoded and played back has been described. The invention is not limited to the MPEG1 decoding / playback apparatus, and is also applicable to the MPEG2 decoding / playback apparatus that decodes / plays back an MPEG2 program stream when a PTS indicating the playback timing of a picture cannot be obtained from a PES (Packetized Elementary Stream) packet. It goes without saying that it is possible.

It is a block diagram of the MPEG1 decoding reproduction | regeneration apparatus concerning this invention. It is a flowchart which shows the AV synchronous process concerning this invention. It is a figure which shows the data structure of an MPEG1 system stream. It is a block diagram of the conventional MPEG1 decoding reproduction | regeneration apparatus. It is a timing chart for demonstrating AV synchronization. It is a flowchart which shows the conventional AV synchronous process.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 MPEG1 decoding reproduction apparatus 14 Video decoder 15 Audio decoder 144 Picture counter 154 Offset calculation part

Claims (5)

Video data and audio data are decoded and reproduced from a compressed video data stream and a compressed audio data stream which are composed of at least a first picture obtained by intra-coding an image frame and a second picture obtained by inter-frame predictive coding an image frame. A method,
Detecting the first first picture viewed from the beginning of the compressed video data stream;
Calculating a playback time when it is assumed that all pictures whose display order is positioned before the first first picture in the compressed video data stream are played back;
Starting video output from video data obtained by decoding the first first picture;
A decoding / reproducing method for starting audio output from audio data after the reproduction time from the head of an audio data stream obtained by decoding the compressed audio data stream. The reproduction time includes the number of pictures from a picture physically located at the head of the compressed video data stream to a picture physically located immediately before the first first picture, and a predetermined reproduction time per picture. The decoding reproduction | regenerating method of Claim 1 calculated based on. A video decoder that decodes and reproduces video data from a compressed video data stream composed of at least a first picture obtained by intra-coding an image frame and a second picture obtained by inter-frame predictive coding an image frame;
An audio decoder that decodes and reproduces audio data from the compressed audio data stream;
The video decoder starts video output from video data obtained by decoding the first picture as viewed from the beginning of the compressed video data stream;
The audio decoder skips audio data corresponding to playback times of all the pictures whose display order is located before the first first picture from the beginning of the audio data stream obtained by decoding the compressed audio data stream. A decoding / reproducing apparatus, wherein audio output is started from audio data after audio data corresponding to the reproduction time. Further comprising a calculating means for calculating the playback time,
4. The audio decoder according to claim 3, wherein the audio decoder starts audio output from audio data after the reproduction time calculated by the calculation means from the head of the audio data stream obtained by decoding the compressed audio data stream. Decoding and playback device. The calculating means counts the number of pictures physically located before the first picture as viewed from the head of the compressed video data stream, and the number of pictures obtained by counting and a predetermined picture The decoding / reproducing apparatus according to claim 4, wherein the reproduction time is calculated based on a winning reproduction time.

Images