JP2004023686A - Mpeg2 decoder - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To synchronously output image data and sound data in the intended timing with an encoder even when there are fluctuations in a PTS and an internal STC. <P>SOLUTION: A system decoder separating part 300 separates an encoded stream and PTS information from an input stream. A sound decoding part 301 decodes encoded sound data and an image decoding part 302 decodes encoded image data. A sound delay memory 303 delays the sound decoded data output, and an image delay memory 305 delays the image decoded data output. A PTS comparing part 304 compares phase differences of image/sound PTS. A voice PTS predicting part 306 predicts the PTS of the next frame from the imparted sound PTS, and an image PTS predicting part 307 predicts the PTS of the next frame from the imparted image PTS. A sound phase deviation detecting part 308 compares the sound PTS predictive value with the sound PTS, and an image phase deviation detecting part 309 compares the image PTS predictive value with the image PTS. An image/sound phase deviation detecting part 310 compares the phase deviation quantities of the image and the sound. <P>COPYRIGHT: (C)2004,JPO

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to a device that decodes encoded data and outputs the decoded data based on PTS (Presentation Time Stamp) information included in the data.
[0002]
[Prior art]
The image and audio compression technology represented by the international encoding standard MPEG2 has been widely adopted in the broadcasting field because of its high image quality and high compression efficiency. An outline of MPEG2 decoding will be described below.
In MPEG2, images and sounds are separately encoded. For this reason, PES (Packetized Elementary Stream) assigned in pack units to synchronize the two at the time of decoding, PTS assigned in packet units, and output designation time information called PTS, and STC (System Time Clock) which has as a reference time inside the decoder ) Is used.
(A time reference value PCR (Program Clock Reference) is also required, but is not described in this patent because it is not used in this patent.)
[0003]
FIG. 6 is a diagram showing an example of a data structure of encoded data (bit stream) according to MPEG2.
A pack, which is one unit of encoded data, includes a pack header 101 for storing information about the pack, an image PES (Packetized Elementary Stream) packet 102 for storing encoded image data, and an audio PES packet 103 for storing encoded audio data. Consists of The PES packet is obtained by packetizing image encoded data (elementary stream) and audio encoded data (elementary stream) with variable length. The image PES packet 102 and the audio PES packet 103 have PES headers 104 and 105, respectively, in which PTSs 106 and 107 are stored. One PES packet is composed of a plurality of TS (Transport Stream) packets 108 having the same PID (Packet Identification: 13-bit stream identification information indicating the attribute of an individual stream of the packet). The TS packet is a fixed-length packet of 188 bytes and includes a 4-byte TS header 109 and a payload 110.
[0004]
The PTS is information for displaying or outputting the PES packet (hereinafter simply referred to as output) when the STC in the MPEG system decoder matches the PTS. The accuracy of the PTS is represented by a 33-bit value measured with a clock of 90 kHz. The PTS is provided for the decoder when the encoder performs encoding. For example, when STC is T0, the decoder outputs decoded image data t1 seconds after the moment when the PES packet is received, and wants to output the decoded audio data t2 seconds after the moment the decoder receives the PES packet. In this case, the image PTS is “T0 + t1” and the audio PTS is “T0 + t2”. Actually, the encoder assigns t1 and t2 in consideration of the encoding and decoding processing times inside the encoder and the decoder.
[0005]
FIG. 7 shows a basic conceptual diagram of a video / audio synchronization system using PTS.
The system decoder separation unit 200 of the decoder separates each coded stream of audio and video from the input stream and separates each PTS. Thereafter, when decoding the respective streams, different decoding delays occur in the audio decoding unit 201 and the image decoding unit 202.
The PTS comparison unit 204 calculates the difference between the respective PTSs, and determines which phase of the image or audio the output should be adjusted to. One of the outputs is delayed according to the result of the determination to synchronize them. This delay is controlled including the difference between the delays separately generated in the delay memories 203 and 205.
[0006]
The audio data output / image data output speeds are output at different constant speeds.
[0007]
[Problems to be solved by the invention]
In order to output the image / audio data decoded by the MPEG2 decoder at the timing intended by the encoder as described above, there is no error between the PTS provided by the encoder and the STC reproduced on the decoder side. is necessary. Also, in the broadcasting field, when using the MPEG2 system, it is required that the synchronization between audio and images be synchronized with accuracy of several ms.
However, when the phase of the PTS added to the video / audio stream and the phase of the decoder internal STC fluctuate due to the state of the encoder or the state of the transmission path between the encoder and the decoder, the PTS is used. When the output control is performed, the output timing of the image data or the audio data does not become the correct timing, and the required accuracy may not be satisfied because the phases of the image data and the audio data are shifted.
[0008]
In that case, the MPEG2 standard configuration presupposes that it operates according to the encoder, so a PTS is added to each stream unit, and each decoder operates according to the PTS so that the image and the audio can be in phase. . In this case, there is no mechanism for monitoring the data output phase of the image and the sound and feeding it back. For this reason, it is not possible to detect how much the data output phase of the image / sound has fluctuated, and decoding and reproduction are performed independently of each other.
[0009]
SUMMARY OF THE INVENTION The present invention has been made to solve the above problems, and detects the output phase shift of the image / audio data even if the PTS stored in the PES packet and the internal STC fluctuate. It is an object of the present invention to provide an MPEG2 decoder which realizes synchronous output of image data and audio data at a timing intended by a converter.
[0010]
[Means for Solving the Problems]
An MPEG2 decoder according to the present invention decodes data (hereinafter, referred to as “encoded data”) that includes output designated time information (hereinafter, referred to as PTS information) and is encoded according to the MPEG2 system, and decodes the decoded data based on the PTS information. In the output MPEG2 decoder,
Means for predicting next PTS information from the PTS information, means for decoding the encoded data, means for outputting decoded image data and audio data, output timing of the decoded image data, and decoding Means for detecting a phase shift in the output timing of the output audio data.
[0011]
Further, the MPEG2 decoder according to the present invention is provided with means for adjusting the output timing of the decoded data, and always adjusts the output timing of the decoded data based on the value of the phase shift.
[0012]
Also, the MPEG2 decoder according to the present invention is provided with an adjusting means for adjusting the output timing of the decoded data and a means for storing a threshold value, wherein the adjusting means determines whether or not the phase shift value exceeds the threshold value. The output timing is adjusted.
[0013]
Further, the MPEG2 decoder according to the present invention is provided with means for externally inputting a threshold value.
[0014]
The MPEG2 decoder according to the present invention further comprises means for adjusting the output timing of the decoded data, and means for storing a monitoring cycle for detecting a phase shift,
In the monitoring cycle, the output timing of the decoded data is adjusted based on the value of the phase shift.
[0015]
Further, the MPEG2 decoder according to the present invention is provided with means for externally inputting a monitoring cycle.
[0016]
BEST MODE FOR CARRYING OUT THE INVENTION
Embodiment 1 FIG.
FIG. 1 is a schematic configuration diagram of an MPEG2 decoder according to Embodiment 1 of the present invention.
In FIG.
Reference numeral 300 denotes a system text / separator separating unit for separating an encoded stream, PTS, and other necessary information from an input stream.
Reference numeral 301 denotes an audio decoding unit that decodes encoded audio data.
Reference numeral 302 denotes an image decoding unit that decodes encoded image data.
Reference numeral 303 denotes an audio delay memory for delaying audio decoded data output.
A PTS comparison unit 304 compares the phase difference between the image and audio PTS.
An image delay memory 305 delays the output of the decoded image data.
An audio PTS prediction unit 306 predicts the PTS of the next frame from the supplied audio PTS.
An image PTS prediction unit 307 predicts the PTS of the next frame from the given image PTS.
An audio phase shift detecting unit 308 compares the audio PTS prediction value with the audio PTS.
An image phase shift detecting unit 309 compares the predicted image PTS value and the image PTS.
Reference numeral 310 denotes an image / sound phase shift detector that compares the amount of phase shift between the image and the sound. Reference numeral 311 denotes an address audio output control memory configured with a ring buffer for outputting audio.
Reference numeral 312 denotes an address image output control memory configured with a ring buffer for outputting an image.
313 is a data writing unit for writing the decoded audio data.
314 is a data writing unit for writing the decoded image data.
A data reading unit 315 reads out the decoded audio data written in the audio output control memory at the actual data output speed.
A data reading unit 316 reads out the decoded image data written in the image output control memory at the actual data output speed.
[0017]
In the MPEG2 decoder of FIG. 1, the system coder separator 300 separates each coded stream of audio / video from the input stream and separates each PTS. Thereafter, when decoding the respective streams, different decoding delays occur in the audio decoding unit 301 and the image decoding unit 302.
The PTS comparison unit 304 calculates each PTS difference, and determines which phase of the image or audio is to be matched with the output phase. The output is delayed according to the above determination result to synchronize the two.
This delay is controlled including the difference between the delays generated separately in the delay memories 303 and 305.
[0018]
At this time, as described above, the input decoded data is output at a certain constant speed, so that the time required for outputting one bit of data is fixed for each of the image and the sound.
[0019]
FIG. 2 is a schematic diagram of PTS prediction in this embodiment. FIG. 3 is a flowchart of the PTS prediction operation in this embodiment. The count timing signal and the predicted PTS acquisition timing signal are processed by another unit. The outline of the operation will be described below.
[0020]
First, it is checked whether the PTS prediction has been started (step S1). If the PTS prediction has not been started, it is checked whether the PTS has been received next. (Step S2) When the PTSn has been successfully received, the PTSn separated from the system text / coater separation unit 400 is set in the PTS prediction unit 408. (Step S3)
[0021]
Next, it is checked whether the acquisition of the encoded data has been completed (step S4). If the encoded data n can be received, the encoded data n separated by the system text / separator separating unit 400 is transferred to the decoding unit 401 (step S5). .
Thereafter, it is checked whether the decoding is completed (step S6). If the decoding is completed, the data writing unit 403 transfers the decoded data n to the output control memory (step S7). After the transfer is completed, a read instruction is issued to the data read unit 405 (step S8).
Thereafter, the decoded data written to the output control memory 404, which is a link buffer, by the data reading unit 405 which has received the read instruction is designated with an address, and is read at the speed at which each decoded data is reproduced.
The set PTSn is added by the PTS adder every reading speed unit time every time the decoded data is read (step S21). (Step S20)
[0022]
If the PTS prediction has been started in step s1, it is checked whether the information of PTSn + 1 on the next PES has been received (step S9), and the acquired PTSn + 1 is set in the phase shift detection unit 409. (Step S10)
Similarly, it is checked whether the acquisition of the encoded data n + 1 has been completed (step S11). If the encoded data has been received, the encoded data n + 1 separated by the system text / separator separation unit 400 is transferred to the decoding unit 401 (step S11). Step S12). Thereafter, it is checked whether the decoding has been completed (step S13). If the decoding has been completed, the decoded data n + 1 is transferred to the output control memory 404 (step S14).
[0023]
The write start address of the decryption data n + 1 written in the output control memory 404 is set in the designated address read start detecting unit 406 (step S15). The designated address read start detecting unit 406 monitors whether the read address matches the write start address of the decryption data n + 1. When the read addresses match, the designated address read start detection unit 406 notifies the PTS prediction unit 408 of a predicted PTS acquisition timing signal. The PTS predicting unit 408 notified of the predicted PTS acquisition timing signal acquires the PTS predicted value calculated by the PTS adding unit and notifies the phase shift detecting unit 409 of the PTS predicted value. (Step S16)
[0024]
The phase shift detector 409, which has been notified of the PTS predicted value, compares the set PTSn + 1 with the PTS predicted value to detect the phase shift time, and detects the current output data phase shift time (step S17). Also, PTSn + 1 is set in the PTS prediction unit 408 for the next prediction operation. (Step S18)
[0025]
With the above method, the audio PTS prediction unit 306 of FIG. 1 starts the next audio PTS prediction while outputting audio decoded data. The audio phase shift detector 308 compares the audio PTS prediction value passed from the audio PTS predictor with the audio PTS passed from the system coder separator 300 to detect the phase shift time from the start of audio decoding. Similarly, the image PTS prediction unit 307 compares the predicted value of the image PTS predicted by the image phase deviation detection unit 309 with the image PTS passed from the system text / separation unit 300, and detects the phase deviation time from the start of audio decoding. The phase shift time information of both is notified to the audio / image phase shift detecting unit 310, and it is possible to detect how much the image / audio output is currently shifted by comparing the phase shift times of both the audio and the image. .
[0026]
Embodiment 2 FIG.
In the second embodiment, the function of notifying the phase shift correction value from the audio / image phase shift detection unit 310 of the first embodiment to the video / audio delay memories 303 and 305 and the function of detecting the phase shift detection unit 310 A description will be given of an MPEG2 data coater that operates so that the phase of audio / video data is matched by automatically adjusting the delay of audio / video by adding a function of calculating a correction value for video / audio from the phase difference obtained. The system configuration is the same as that of the first embodiment, and a description thereof will be omitted.
[0027]
The data path of the MPEG2 data coater is the same as that of the first embodiment, and will not be described.
[0028]
The operation up to the detection of the phase shift of the image / sound data is the same as that of the first embodiment, so that the description is omitted.
[0029]
The image / sound phase shift detecting unit 310 detects the phase shift of the image / sound data, and calculates a correction value of the image / sound from the detected phase shift of the image / sound data. The calculated correction values are set as correction values in the image / audio delay memories 303 and 305, respectively, so that the operation is always performed so that the phase shift of the image / audio is eliminated, thereby preventing the occurrence of the phase difference of the image / audio data. it can.
[0030]
Embodiment 3 FIG.
In the third embodiment, the function of notifying the image / sound delay memories 303 and 305 of the phase shift correction value from the image / sound phase shift detection unit 310 of the first embodiment, and the allowable range of the image / sound phase shift time Of the user, the function of storing the allowable range, and the function of adjusting the delay amount according to the correction value calculated by the image / sound phase shift detecting unit 310 by extending the function of the user. A description will be given of an MPEG2 data recorder device that operates so that the phases of audio and video data match within the range.
FIG. 4 is a schematic configuration diagram of an MPEG2 decoder according to Embodiment 3 of the present invention. 4, the same reference numerals as those in FIG. 1 denote the same or corresponding parts. Reference numeral 317 denotes an input unit for externally inputting a permissible range of the image / sound phase shift time, and 318 denotes a permissible range storage unit.
[0031]
Since the data path of the MPEG2 data coater is the same as that of the first embodiment, the description is omitted.
[0032]
The operation up to the detection of the phase shift of the image / sound data is the same as that of the first embodiment, and the description is omitted.
[0033]
The allowable range of the phase shift is input in advance from the input unit 317 by the user, and is stored in the allowable range storage unit 318. It is checked whether the phase shift of the image / sound data detected by the image / sound phase shift detection unit 310 does not exceed the allowable range. If the phase shift exceeds the allowable range, a correction value for the image / audio is calculated. By setting the calculated correction values in the image / audio delay memories 303 and 305, it is possible to control the image / audio phase in accordance with the allowable range, and to generate a phase difference of the image / audio data exceeding the allowable range. Can be prevented.
[0034]
Further, since the user can input from the outside, the allowable value can be changed according to the state of the system, and a flexible response is possible.
[0035]
In the above example, the allowable value is input from the input unit 317. However, the present invention is not limited to this. For example, the allowable value may be downloaded from another system such as a server.
[0036]
Embodiment 4 FIG.
In the fourth embodiment, the function of notifying the image / sound delay memories 303 and 305 of the phase shift correction value from the image / sound phase shift detection unit 310 of the first embodiment, and the function of detecting the phase shift of the image / sound from outside by the user. The function of inputting the monitoring cycle of the above, the function of storing the monitoring cycle, and the function of adjusting the delay amount according to the correction value calculated by the image / sound phase shift detecting unit 310 are extended to provide a certain monitoring. A description will be given of an MPEG2 data recorder device which operates so that the phases of audio / video data coincide with each other in a cycle.
FIG. 5 is a schematic configuration diagram of an MPEG2 decoder according to Embodiment 4 of the present invention. 5, the same reference numerals as those in FIG. 1 denote the same or corresponding parts. Reference numeral 317 denotes an input unit for externally inputting a permissible range of the image / sound phase shift time, and 319 denotes a monitoring cycle storage unit. Here, it is assumed that the image / sound phase shift detection unit 310 has a timer.
[0037]
Since the data path of the MPEG2 data coater is the same as that of the first embodiment, the description is omitted.
[0038]
The operation up to the detection of the phase shift of the image / sound data is the same as that of the first embodiment, and the description is omitted.
[0039]
The monitoring cycle of the phase shift detection is input by the user from the input unit 317 in advance, and is stored in the monitoring cycle storage unit 319. Further, a timer (not shown) provided in the image / sound phase shift detecting unit 310 is initialized when the image / sound phase shift detecting unit 310 is activated, and starts counting up immediately thereafter. Then, the timer is initialized by a signal generated when the value of the timer coincides with the monitoring period, and the counting up is restarted immediately thereafter.
[0040]
The image / sound phase shift detection unit 310 checks whether the monitoring cycle time stored in the monitoring cycle storage unit 319 has elapsed, and if the monitoring cycle time has elapsed, the image / sound phase shift detection unit 310 detects it. The correction value of the image / audio is calculated from the phase shift of the image / audio data. By setting correction values in the image / audio delay memories 303 and 305, it is possible to prevent the occurrence of an image / audio phase difference in response to a user monitoring cycle request.
[0041]
Also, since the user can input from the outside, the monitoring cycle can be changed according to the state of the system, and a flexible response is possible.
[0042]
In the above example, the monitoring period is input from the input unit 317. However, the present invention is not limited to this. For example, the monitoring period may be downloaded from another system such as a server.
[Brief description of the drawings]
FIG. 1 is a block diagram illustrating a configuration of an MPEG2 decoder according to the present invention.
FIG. 2 is a block diagram illustrating a PTS prediction method / correction method.
FIG. 3 is a flowchart illustrating a PTS prediction method.
FIG. 4 is a schematic configuration diagram of an MPEG2 decoder according to a third embodiment of the present invention.
FIG. 5 is a schematic configuration diagram of an MPEG2 decoder according to Embodiment 4 of the present invention.
FIG. 6 is a diagram showing the relationship between bit streams in MPEG2.
FIG. 7 is a block diagram showing a configuration of a conventional MPEG2 decoder.
[Explanation of symbols]
101 Pack header 102 Image PES packet 103 Audio PES packet 104 Image PES header 105 Audio PES header 106 Image PTS
107 Voice PTS
108 TS packet 109 TS header, 110 TS payload 200 System decoder separation unit 201 Audio decoding unit 202 Image decoding unit 203 Audio delay memory 204 PTS comparison unit 205 Image delay memory 300 System decoder separation unit 301 Audio decoding unit 302 Image decoding unit 303 Audio Delay memory 304 PTS comparison unit 305 Image delay memory 306 Audio PTS prediction unit 307 Image PTS prediction unit 308 Audio phase shift detection unit 309 Image phase shift detection unit 310 Image / audio phase shift detection unit 311 Audio output control memory 312 Image output control memory 313 Data writing unit (voice)
314 Data writing unit (image)
315 Data readout unit (voice)
316 Data readout unit (image)
317 Input unit 318 Allowable value storage unit 319 Monitoring cycle storage unit 400 System decoder separation unit 401 Decoding unit 402 Delay memory 403 Data writing unit 404 Output control memory 405 Data reading unit 406 Designated decoding data reading start detection unit 407 PTS addition unit 408 PTS prediction unit 409 Phase shift detection unit

Claims (6)

An MPEG2 decoder that decodes data (hereinafter referred to as coded data) that includes output designated time information (hereinafter referred to as PTS information) and is encoded according to the MPEG2 system, and outputs decoded data based on the PTS information,
Means for predicting next PTS information from the PTS information, means for decoding the encoded data, means for outputting decoded image data and audio data, output timing of the decoded image data, and decoding An MPEG2 decoder comprising means for detecting a phase shift in output timing of the output audio data. 2. The MPEG2 decoder according to claim 1, further comprising means for adjusting the output timing of the decoded data, and always adjusting the output timing of the decoded data based on the value of the phase shift. Adjustment means for adjusting the output timing of the decoded data, and means for storing a threshold value are provided, and the adjustment means adjusts the output timing of the decoded data only when the value of the phase shift exceeds the threshold value. The MPEG2 decoder according to claim 1, wherein 4. The MPEG2 decoder according to claim 3, further comprising means for externally inputting a threshold value. Means for adjusting the output timing of the decoded data, and means for storing the monitoring cycle of the phase shift detection,
2. The MPEG2 decoder according to claim 1, wherein an output timing of decoded data is adjusted based on the value of the phase shift in the monitoring cycle. 6. The MPEG2 decoder according to claim 5, further comprising means for externally inputting a monitoring cycle.

Images