JP3133630B2 - MPEG system decoder - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

The present invention relates to an MPEG (Moving Pic)
This is related to a system decoder.

[0002]

2. Description of the Related Art The information handled by multimedia is enormous and diverse, and it is necessary to process such information at high speed in order to put multimedia into practical use. In order to process information at high speed, data compression / decompression technology is indispensable. As such data compression / decompression technology, the “MPEG” method is exemplified. This MPEG system is based on ISO (International Organizati
on for Standardization) / IEC (Intarnational El)
It is being standardized by the MPEG Committee (ISO / IEC JTC1 / SC29 / WG11) under the ectrotechnical Commission.

[0003] MPEG is composed of three parts. Part 1 “MPEG System Part” (ISO / IEC
IS 1172 Part 1: Systems) specifies a multiplex structure (multiplex structure) of video data and audio data and a synchronization method. Part 2 "MPE
In the "G video part" (ISO / IEC IS 1172 Part2: Video), a high-efficiency encoding method of video data and a format of the video data are specified. Part 3 "MPE
G Audio Part ”(ISO / IEC IS 1172 Part3: Audio
In), a high-efficiency encoding method of audio data and a format of audio data are defined.

At present, MPEG-1 and MPEG-1 are mainly used due to the difference in encoding rate.
-2. The video data handled by the MPEG video part relates to a moving image, and the moving image is composed of several tens (eg, 30) frames per second. Video data is stored in a sequence (Seq
uence), a GOP (Group Of Pictures), a picture, a slice (Slice), a macroblock (Macroblock), and a block in the order of six layers. In MPEG-1, a frame corresponds to a picture. In MPEG-2, a frame or a field can correspond to a picture. Two fields constitute one frame. The structure in which a frame corresponds to a picture is called a frame structure, and the structure in which a field corresponds to a picture is called a field structure.

[0005] MPEG uses a compression technique called inter-frame prediction. Inter-frame prediction compresses data between frames based on temporal correlation. In the inter-frame prediction, bidirectional prediction is performed. The bidirectional prediction is to use both forward prediction for predicting a current reproduced image from a past reproduced image (or picture) and backward prediction for predicting a current reproduced image from a future reproduced image. .

[0006] This bidirectional prediction is based on an I picture (Intra-Pi
), a P picture (Predictive-Picture), and a B picture (Bidirectionally predictive-Picture). The I picture is generated independently of a past or future reproduced image. The P picture is generated by forward prediction (prediction from a past I picture or P picture). B pictures are generated by bidirectional prediction. In bidirectional prediction, a B picture is generated by any one of the following three predictions. Prediction from past I or P pictures, prediction from future I or P pictures, past and future I or P pictures
Prediction from pictures. Then, these I, P, and B pictures are respectively encoded. That is, an I picture is generated even if there is no past or future picture. On the other hand, a P picture is not generated without a past picture,
A B picture is not generated without a past or future picture.

In the inter-frame prediction, first, an I-picture is periodically generated. Next, a frame several frames ahead of the I picture is generated as a P picture. This P
The picture is generated by one-way (forward) prediction from the past to the present. Subsequently, a frame located before the I picture and after the P picture is generated as a B picture. When generating this B picture, an optimal prediction method is selected from three of forward prediction, backward prediction and bidirectional prediction. In general, in a continuous moving image, a current image and images before and after the current image are very similar, and only a part thereof is different. Then, the previous frame (for example, I
(Picture) and the next frame (for example, P picture) are assumed to be the same, and if there is a change between both frames, only the difference (B picture) is extracted and compressed. Thus, data between frames can be compressed based on temporal correlation.

[0008] The data sequence (bit stream) of the video data encoded in accordance with the MPEG video part is called an MPEG video stream (hereinafter abbreviated as a video stream). A data string of audio data encoded in accordance with the MPEG audio part is called an MPEG audio stream (hereinafter, abbreviated as audio stream). And the video stream and the audio stream are MP
The data is time-division multiplexed in accordance with the EG system part and becomes an MPEG system stream (hereinafter abbreviated as a system stream) as one data string. System streams are also called multiplex streams. M
PEG-1 is mainly used for CD-ROM (Compact Disc-Read Onl
y Memory) and other storage media.
G-2 supports a wide range of applications, including MPEG-1.

The flow from the encoding to the decoding in the MPEG part is as follows. MPE
A G system encoder (hereinafter, abbreviated as a system encoder) separately encodes video data and audio data while maintaining coordination, and generates a video stream and an audio stream. Next, MP
A multiplexer (MUX) provided in the EG system encoder multiplexes a video stream and an audio stream so as to conform to a format of a transmission medium or a recording medium, and generates a system stream. The system stream is transmitted from the MUX via a transmission medium or recorded on a recording medium.

A demultiplexer (Demultiplexer) provided in an MPEG system decoder (hereinafter abbreviated as a system decoder).
A multiplexer (DMUX) separates a system stream into a video stream and an audio stream.
Next, the system decoder individually decodes each stream to generate a video decoded output (hereinafter, referred to as video output) and an audio decoded output (hereinafter, referred to as audio output). The video output is output to a display, and the audio output is output to a speaker via a D / A (Digital / Analog) converter and a low-frequency amplifier.

The system stream includes a plurality of packs (Pac
k), where each pack consists of multiple packets
It consists of. In each packet, there are a plurality of access units. The access unit is a unit for performing decoding and reproduction, and is 1 for a video stream.
One picture corresponds to one picture, and an audio stream corresponds to one audio frame.

The system encoder adds a pack header to the head of a pack and adds a packet header to the head of a packet. The pack header is SCR (System Clock R
eference), which includes reference information such as a reference time for synchronous reproduction. Here, the reproduction means the output of the video output and the audio output to the outside.

The packet header contains information for identifying whether the following data is video data or audio data, and information for managing a decoding reproduction time called a time stamp (hereinafter abbreviated as TS). Including. The packet length strongly depends on the transmission medium and application. For example, ATM (Asynchronous Transfer Mod
Some are as short as 53 bytes, as in e), and some are as long as 4096 bytes, such as CD-ROM. The upper limit of the packet length is set to 64 Kbytes.

For example, data recording on a CD-ROM is performed continuously in units of sectors, and the data is read out by a CD-ROM player every second.
It is performed at a constant speed of 75 sectors. In the CD-ROM, each sector corresponds to one pack, and the pack and the packet are the same.

[0015] The system encoder adds a TS corresponding to the access unit to the packet header when the packet has the head of the access unit, and does not add the TS when the packet does not have the head of the access unit. When two or more access units are at the head of a packet, the system encoder adds only a TS corresponding to the first access unit to the packet header.

The TS includes a PTS (Presentation Time St
amp) and DTS (Decoding Time Stamp). The MPEG system part is based on STD (System Targe
A virtual reference decoder called t Decoder) defines a decoding standard. The reference clock for the STD is a synchronization signal called STC (System Time Clock).

The PTS is information for managing the time of reproduction output. The precision of this PTS is represented by a 32-bit length value measured with a clock of 90 kHz. When the PTS matches the STC, the system decoder decodes the access unit to which the PTS is added, and generates a playback output.

As described above, since the MPEG video part uses the inter-frame prediction technique, the I picture and the P picture are transmitted as a video stream prior to the B picture. For this reason, upon receiving the video stream, the system decoder rearranges and decodes the picture order to the original order based on the picture header at the beginning of each picture of the video stream, and generates a video output. DTS is information for managing the decoding start time after rearranging the pictures. When the PTS and the DTS are different, the system encoder adds both to the packet header, and when they match, adds only the PTS. Specifically, in a video stream having a B picture, both a PTS and a DTS are added to a packet having an I picture and a P picture, and only a PTS is added to a packet having a B picture.
In a video stream without a B picture, P
Only TS is added.

The SCR is information for setting or calibrating the value of the STC to the value intended by the system encoder. The accuracy of this SCR is 90 kHz in MPEG-1.
The value measured with the clock of z is expressed as a 32-bit length, and MP
In EG-2, a value measured with a 27 kHz clock is represented by a 42-bit length. The SCR is transmitted in 5 bytes in MPEG-1 and 6 bytes in MPEG-2, and the system decoder sets the STC according to the value of the SCR at the moment of the arrival of the last byte.

FIG. 10A shows an example of a system stream. One pack is composed of a pack header H and each packet V1, V2, A1... V6, V7. The packets include packets V1 to V7 of video data and packets A1 to A3 of audio data. These packets are arranged in numerical order when viewing one of the video data and the audio data, but the video data packets and the audio data packets are mixed in the other party. For example, a packet V of video data
1, V2 is followed by an audio data packet A1, followed by video data packet V3, and further followed by audio data packets A2, A3. Here, the SCR is included in the pack header H, the PTS (V1) is included in the packet header of the packet V1, the PTS (A1) is included in the packet header of the packet A1, and the packet V
The PTS (V6) is added to the packet header of No. 6 respectively. Therefore, as shown in FIG. 10 (b), the access unit α in each of the packets V1 to V5, and as shown in FIG. 10 (c), the access unit β in each of the packets A1 to A3.
However, as shown in FIG. 10D, the access unit γ is composed of the packets V6 and V7. In this case, the access units α and γ each correspond to one picture, and the access unit β corresponds to one audio frame. Note that in FIGS. 10A to 10D, the DTS
Has been omitted.

FIG. 11 shows a conventional system decoder 111.
Is shown. The system decoder 111
It comprises a PEG audio decoder 112, an MPEG video decoder 113, and an audio video parser (AV parser) 114. A demultiplexer (DMUX) 115 is provided in the AV parser 114.
Is provided.

The AV parser 114 receives a system stream transferred from the outside. DMUX115
Is based on the system stream packet header,
Separate system stream into video stream and audio stream. That is, the system stream shown in FIG.
The video stream is composed of a video stream composed of V7 and an audio stream composed of packets A1 to A3 of audio data.

Further, the AV parser 114 converts the system stream from the SCR and audio PTS (hereinafter, PTS).
(A)) and video PTS (hereinafter referred to as PTS (V)). And the AV parser 114
Outputs the audio stream, SCR, PTS (A) to the audio decoder 112, and outputs the video stream, SCR, PTS (V) to the video decoder 1 respectively.
13 is output.

The audio decoder 112 decodes the audio stream according to the MPEG audio part and generates an audio output. Video decoder 1
13 decodes the video stream according to the MPEG video part and generates a video output. The video output is output to a display 116, and the audio output is output to a speaker 118 via an audio playback device 117 having a D / A converter and a low-frequency amplifier.

Here, the audio decoder 112 and the video decoder 113 synchronously reproduce the audio output and the video output based on the SCR and the PTS, respectively. That is, the audio decoder 112 uses the SCR and the PTS
(A) The reproduction time (reproduction timing) of the audio output is set based on (PTS (A1)), and the reproduction of the access unit γ is started at time t3 as shown in FIG. The video decoder 113 uses the SCR and PTS (V) (P
TS (V1), PTS (V6)), the reproduction time (reproduction timing) of the video output is set, and FIG.
As shown in 0 (c), reproduction of each access unit α, β is started at each time t1, t2. At this time, the setting of the playback time of the audio output in the audio decoder 112 and the setting of the playback time of the video output in the video decoder 113 are performed separately according to the respective PTS (A) and PTS (V).

[0026]

In the synchronous reproduction of the audio output and the video output, it is necessary to consider "lip sync". The lip sync means that the movement of the mouth of the person and the sound projected on the display are synchronized. Speech is faster than mouth movement,
On the other hand, if the lip sync is slow, it is said that the lip sync is out of sync. The lip-sync deviation is not a problem as long as it is below the detection limit of human hearing. However, if the detection limit is exceeded, the viewer will feel uncomfortable. Generally, it is said that the detection limit of the shift of the lip sync is about several milliseconds.

The conventional system decoder 11 shown in FIG.
In the case of 1, lip sync cannot be sufficiently achieved. This is because the decoding processing time of the STD (reference decoder), that is, the internal delay time of the STD is assumed to be zero. The actual decoding processing time of the audio decoder 112 and the video decoder 113 is extremely short but not zero. The decoding processing time (internal delay time) differs for each of the decoders 112 and 113 and also for the data amount of the access unit to be processed.
For example, the number of packets constituting each of the access units α to γ as shown in FIGS. 10B to 10D is usually different, and the packet length of each packet is not necessarily the same. Therefore, the data amount of each of the access units α to γ usually differs.

Therefore, in order to overcome the above-mentioned drawbacks, either the video output or the audio output is delayed according to the calculation result of the difference between PTS (V) and PTS (A), so that both are output. A method for achieving synchronization has been proposed. However, this method requires a delay memory to delay the video or audio output. This leads to an increase in circuit scale and cost. Further, accurate control of the delay memory is considered difficult. If the control is performed by the AV parser 114, the software load on the AV parser 114 is increased, which hinders the operation of the AV parser 114.

The present invention has been made to solve the above problems, and an object of the present invention is to provide an MPEG system decoder capable of sufficiently synchronizing audio output and video output.

[0030]

Means for Solving the Problems The invention according to claim 1
Is the MPEG system stream transferred from outside
System to the packet of the MPEG system stream.
MPEG system stream based on
Video stream and MPEG audio stream
Demultiplexer and MPEG system stream
From the SCR, audio timestamp and video
Separating means for separating the time stamps from each other;
Audio register and audio bit buffer and audio
Odecode core circuit and audio control circuit
MPEG audio decoder and video register
Video bit buffer, video decode core circuit and video
MPEG video decoder composed of
An MPEG system decoder provided with
The audio register stores the audio data transferred from the separation means.
Audio time stamps, and
The auto-buffer is the buffer transferred from the demultiplexer.
Audio streams are sequentially stored and the audio deco
The core circuit reads the audio read from the bit buffer.
Audio stream conforms to MPEG audio part
To decode and generate audio output,
The control circuit controls the data in the MPEG audio decoder.
Calculate the code processing time and calculate the decoding processing time and minutes
SCR transferred from remote means and read from register
Based on the time stamp of the audio output
Calculate the playback timing of the audio output, and
Controls the decode core circuit according to the
The register determines the time of the video transmitted from the separation means.
Sequentially accumulating stamps, said video bit buffer comprising:
Video stream transferred from demultiplexer
And the video decode core circuit stores the bits
MPEG stream of video stream read from buffer
Decodes according to the video part and generates video output
And the video control circuit is provided to the MPEG video decoder.
The decoding processing time in
And the SCR transferred from the separation means and the register
Based on the timestamp of the video read from
Calculate the playback timing of the video output and
Controls the decode core circuit according to the MPEG
The audio decoder uses the audio bit buffer
Delay time and internal delay of audio decode core circuit
Between the audio read from the audio register and
Generate a second timestamp based on the timestamp
The video control circuit provides a video time stamp
And video mapping
Circuit internal delay time and read from video register
Based on the video timestamp and the second timestamp
Therefore, the skip operation or
A repeat operation is performed, and the video signal generated by the video control circuit is generated.
Skip operation or repeat
Judge an error in the control signal for performing the
Equipped with skip judgment circuit or repeat judgment circuit to correct
That is the gist.

According to the second aspect of the present invention, an externally transmitted
MPEG system stream
Based on the packet header of the G system stream, MP
EG system stream as MPEG video stream
Demultiplexing to separate MPEG audio stream
And SCR from MPEG system stream
Dio timestamp and video timestamp
Separation means to separate each, audio register and audio
Audio bit buffer and audio decode core circuit
MPEG audio system consisting of audio and audio control circuits
Video decoder, video register and video bit buffer
A video decoder core circuit and a video control circuit.
MPEG system with MPEG video decoder
A stem decoder, wherein the audio register is F
In the IFO configuration, audio transferred from the separation means
Of the audio bits
The buffer consists of RAM with FIFO structure,
The audio stream transferred from the lexer
The audio decoding core circuit stores the bit
The audio stream read from the buffer
Decodes according to the G audio part, and
Generating an output, wherein the audio control circuit
Required for the audio stream to be read from the
Time and decode processing time in the decode core circuit
And decoding processing in the MPEG audio decoder
Calculation time, decode processing time, and
SCR transferred from the
Audio output based on audio timestamp
Calculate the force regeneration timing and follow the
Controlling the decode core circuit, and
Is the FIFO configuration, and the video transferred from the separation means.
The time stamp of the video bit
The buffer consists of a RAM with FIFO structure,
Accumulate video streams transferred from KUSA
The video decode core circuit is a bit buffer.
The video stream read from the
Decode in accordance with the protocol, generate a video output, and
The video control circuit converts the video stream from the bit buffer
Time required to read the data and the decoding core circuit
MPEG video decoder from decoding processing time
Calculated decoding time in its decoding process
Time, SCR transferred from separation means,
Based on the time stamp of the video read from the
To calculate the playback timing of the video output, and
Controlling the decode core circuit in accordance with the
The G audio decoder includes a delay time calculation circuit and an audio
E subtraction circuit, addition circuit, sampling frequency detection circuit
A time stamp generation circuit comprising:
The calculation circuit calculates the internal delay time of the audio bit buffer.
The audio subtraction circuit calculates an audio bit
Buffer internal delay time and audio decode core times
Path internal delay time and read from audio register
Audio based on the audio timestamp
The sum of each internal delay time was subtracted from the time stamp
A sampling frequency detection circuit.
Sampling audio data from the stream
The frequency is detected and the clock corresponding to the sampling frequency is detected.
Generating a lock, wherein the adding circuit comprises an audio subtracting circuit
The second clock is added by adding the value generated by
The video control circuit generates a write address.
Detection circuit, read address detection circuit, and picture header
Synchronization determination with detection circuit, mapping circuit, second register
Circuit, first and second comparison processing circuits, and first and second comparison processing circuits.
Video subtraction circuit, and the write address detection circuit
Path is used for externally transmitted video streams.
Packets with a video timestamp added
When writing to the video bit buffer,
Address in video bit buffer
The video register is provided to a write address detection circuit.
So the address and video timestamp
And sequentially accumulates the read address detection circuit.
Is the video stream read from the video bit buffer.
The address of the stream is detected and the picture header
The path is the video stream written to the video bit buffer.
Detects picture header at the beginning of each picture in the stream
Of the picture specified in the picture header.
And the first comparison processing circuit detects the video bit
Address of the video stream read from the
And the time of the video read from the video register
Compare the address corresponding to the stamp, and both addresses
Detecting whether they match, the mapping circuit
No. 1 comparison processing circuit and picture header detection circuit detection
Based on the results, the video time stamps and the picture
And the second register is a one-stage switch.
And a picture
Type of picture detected by the header detection circuit
Based on the video corresponding to the I-picture or P-picture.
Video stamp corresponding to B picture
The first video subtraction times
The path is between the internal delay time of the video decode core circuit and the external
From the specified first value and the second register
Video tie based on the video timestamp
From the internal time stamp and the externally specified
1 to generate a value obtained by subtracting the sum of
The arithmetic circuit is configured to generate the second time stamp generated by the time stamp generating circuit.
Generated by the first video subtraction circuit from the timestamp of
The second comparison processing circuit generates a value obtained by subtracting the value.
The second value specified from outside and the second video subtraction circuit
Comparing the generated values, the synchronization determination circuit performs mapping
The circuit can be used to match video timestamps and pictures.
When the tapping is performed, the comparison result of the second comparison processing circuit
The skip operation to the video decode core circuit based on the
Or to generate a control signal for performing the repeat operation,
In the video decode core circuit, the skip operation
The picture transferred from the video bit buffer
Are discarded, and the
Is not performed, and the video bit
The video output of the picture transferred from the
Video output from the video control circuit
Skip operation or repeat operation in Odecode core circuit
To determine and correct an error in the control signal
That a kipping judgment circuit or a repeat judgment circuit
This is the gist.

The third aspect of the present invention is the first aspect of the present invention.
3. The MPEG system decoder according to claim 2, wherein
Video decode core generated from the video control circuit
The control signal for causing the circuit to perform the skip operation is constant
If the control signal is generated more than once
That the first skip enabling means is provided.
Make a summary.

[0033] The invention described in claim 4 is the invention according to claim 1 or
3. The MPEG system decoder according to claim 2, wherein
Video decode core generated from the video control circuit
A control signal is generated to cause the circuit to perform a skip operation.
After a certain period of time, the second control signal is activated.
The gist of the present invention is that a skip enabling means is provided.

[0034] The invention according to claim 5 is the invention according to claim 1 or
3. The MPEG system decoder according to claim 2, wherein
Video decode core generated from the video control circuit
The control signal for causing the circuit to repeat
If the control signal is generated more than once
That the first repeat activating means is provided.
Make a summary.

The invention according to claim 6 is the invention according to claim 1 or
3. The MPEG system decoder according to claim 2, wherein
Video decode core generated from the video control circuit
A control signal is generated to cause the circuit to repeat.
After a certain period of time, the second control signal is activated.
The gist of the present invention is to provide the repeat validating means.

[0036] The invention described in claim 7 is based on claim 1 or
3. The MPEG system decoder according to claim 2, wherein
Video decode core generated from the video control circuit
The control signal for causing the circuit to perform the skip operation
Enables the control signal when it is generated more than once in a row
And the control signal is continuously generated a predetermined number of times.
After a certain period of time from the first generation,
It is necessary to provide a skip judgment circuit to validate the signal.
To the effect.

The invention described in claim 8 is the invention according to claim 1 or
3. The MPEG system decoder according to claim 2, wherein
Video decode core generated from the video control circuit
The control signal for causing the circuit to repeat
Enables the control signal when it is generated more than once in a row
And the control signal is continuously generated a predetermined number of times.
After a certain period of time from the first generation,
It is necessary to provide a repeat judgment circuit to validate the signal.
To the effect.

The ninth aspect of the present invention relates to the first to eighth aspects.
An MPEG system decoder according to any one of the preceding claims.
The skip operation of the video decode core circuit is B
The point is that kucha is performed with priority.

[0039]

According to the invention described in claim 1 or 2,
For example, the internal delay time of the MPEG audio decoder is
Audio bit buffer internal delay time and audio data
It is defined by the internal delay time of the code core circuit.
And each delay time and audio time stamp
A second time stamp is generated based on the second time stamp.

In the MPEG video decoder, the second
Time stamp and internal delay of MPEG video decoder
The skip operation to the video decode core circuit based on the
Operation or repeat operation. As a result, each
Even if the internal delay time of the
Can be taken.

According to the second aspect of the present invention, externally
By adjusting the specified first value, the audio
The output phase and the video output phase can be shifted arbitrarily.
it can. Also adjusts a second value specified externally
The accuracy of audio output and video output synchronization
Can be set arbitrarily. And the control signal error
And then correct them to further synchronize each output.
Can be taken accurately.

According to the third or fifth aspect of the present invention,
Control signal is not generated continuously more than a certain number of times
, The control signal is not activated. That is, the control signal
If the issue is continuously generated less than a certain number of times,
The control signal is determined to be incorrect and is corrected. The result
As a result, the outputs can be synchronized more accurately.

According to the invention described in claim 4 or 6,
If it is not a certain time after the control signal is generated,
The control signal is not activated. As a result, the same
The period can be taken more accurately.

According to the seventh or eighth aspect of the present invention,
If so, claim 4 and claim 5 or claim 6 and claim 7
By using them together, the synergistic effects of each invention are further effective.
Fruit can be enhanced.

According to the ninth aspect of the present invention, the priority
Lower B pictures take precedence over I and PP pictures
And skipping will result in the video being played
The number of dropped frames is reduced, and the motion of the video becomes smoother.
You.

[0046]

【Example】

(First Embodiment) An MPEG system decoder according to one embodiment of the present invention will be described with reference to the drawings. FIG. 1 shows a block circuit of an MPEG system decoder 1 according to the present embodiment.

The system decoder 1 includes an MPEG audio decoder 2, an MPEG video decoder 3, and an audio video parser (AV parser) 4. The AV parser 4 includes a demultiplexer (DMU)
X) 5 for inputting an MPEG system stream transferred from an external device (for example, a video CD player). The DMUX 5 separates the system stream into an MPEG video stream and an MPEG audio stream according to the packet header of the system stream. The AV parser 4 converts the SCR,
The audio PTS (hereinafter referred to as PTS (A)) and the video PTS (hereinafter referred to as PTS (V)) are separated. The audio stream, SCR, PTS (A)
The video stream, SCR, and PTS (V) are output to the video decoder 3, respectively.

The audio decoder 2 has a register 11,
It includes a bit buffer 12, a decode core circuit 13, and a control circuit 14. Register 11 is a FIFO (First-In
-First-Out), PTS (A) is sequentially accumulated. The bit buffer 12 is a random access memory (RAM) having a FIFO structure.
s Memory), and sequentially stores audio streams. The decode core circuit 13 decodes the audio stream supplied from the bit buffer 12 according to the MPEG audio part, and generates an audio output. The control circuit 14 calculates the reproduction time (reproduction timing) of the audio output based on the decoding processing time in the audio decoder 2, that is, the internal delay time of the audio decoder 2, and the SCR and PTS (A), and according to the calculation result. It controls the decode core circuit 13.

The video decoder 3 includes a register 21, a bit buffer 22, a decode core circuit 23, and a control circuit 24.
It has. The register 21 has a FIFO configuration and the PTS
(V) is sequentially accumulated. The bit buffer 22 is composed of a RAM having a FIFO structure, and sequentially stores video streams. The decode core circuit 23 decodes the video stream supplied from the bit buffer 22 according to the MPEG video part, and generates a video output. The control circuit 24 determines the decoding processing time in the video decoder 3;
That is, the reproduction time of the video output is calculated based on the internal delay time of the video decoder 3 and the SCR and PTS (V), and the decoding core circuit 23 is controlled according to the calculation result.

The video output is the display 25
The audio output is output to a speaker 27 via an audio reproducing device 26 having a D / A converter (not shown) and a low-frequency amplifier (not shown).

When the system stream having the configuration shown in FIG. 10A transferred from the outside is input, the AV parser 4 receives a video stream composed of packets V1 to V7 of video data and a packet A1 of audio data. To A3.

The audio decoder 2 has an internal delay time of the audio decoder 2, SCR and PTS (A) (P
TS (A1)), the reproduction time of the audio output is set, and the reproduction of the access unit γ is started at time t3 as shown in FIG. 10B. More specifically, the control circuit 14 reads the PTS (A1) from the register 11, reads the audio stream from the bit buffer 12, and transfers the audio stream to the decode core circuit 13. At this time, the control circuit 14 determines the internal delay time of the audio decoder 2 and the SC
The reproduction time of the audio output is calculated based on R and PTS (A1). The decode core circuit 13 decodes each packet A1 to A3 of the audio stream in accordance with the MPEG audio part and generates an audio output. The control circuit 14 controls the decode core circuit 13 so that the audio output is reproduced according to the calculated reproduction time (output time to the outside).

By the way, the internal delay time of the audio decoder 2 is the time required for reading the audio stream from the bit buffer 12 (the bit buffer 12
) And the decode processing time in the decode core circuit 13 (the internal delay time of the decode core circuit 13). The internal delay time of the bit buffer 12 changes according to the occupancy of the audio stream in the bit buffer 12, and the larger the occupancy, the larger the internal delay time. The internal delay time of the decode core circuit 13 is constant. The time required for reading the PTS (A) from the register 11 is smaller than the internal delay time of the bit buffer 12, and is negligible even when combined with the signal processing time in the control circuit 14.

The video decoder 3 has an internal delay time of the video decoder 3, SCR and PTS (V) (PTS (V
1), PTS (V6)), the video output playback time is set, and as shown in FIGS. 10B and 10C, the playback of each access unit α, β is started at each time t1, t2. More specifically, the control circuit 24 includes PTS (V1), PTS (V6)
From the register 21, read the video stream from the bit buffer 22, and transfer it to the decode core circuit 23. The control circuit 24 controls the video decoder 3
, SCR and each PTS (A1), PTS
Based on (V6), the playback time of the video output is calculated. The decode core circuit 23 decodes each packet V1 to V7 of the video stream according to the MPEG video part,
Generate video output. The control circuit 24 controls the decode core circuit 23 so that the video output is reproduced according to the calculated reproduction time (output time to the outside).

The internal delay time of the video decoder 3 includes the time required for reading a video stream from the bit buffer 22 (the internal delay time of the bit buffer 22) and the decoding processing time in the decode core circuit 23 (the decode core circuit). 23 internal delay time). The internal delay time of the bit buffer 22 changes depending on the occupation amount of the video stream in the bit buffer 22, and the larger the occupation amount, the larger the internal delay time. The internal delay time of the decode core circuit 23 is a fixed value. The control circuit 24 controls the register 21 so that the time required for reading the PTS (V) becomes equal to the internal delay time of the bit buffer 22.

The setting of the playback time of the audio output in the audio decoder 2 and the setting of the playback time of the video output in the video decoder 3 correspond to the respective PTS (A) and PTS.
Performed separately in accordance with (V).

As described above, in this embodiment, the SCR
The reproduction time of the audio output and the video output is set in consideration of the internal delay time of each of the decoders 2 and 3 as well as the PTS and the PTS. This allows the audio output and the video output to be sufficiently synchronized (lip sync). This eliminates the need for providing a delay memory for delaying one of the video output and the audio output, and makes it possible to avoid an increase in circuit size and cost due to the provision of the delay memory.

(Second Embodiment) FIG. 2 shows a second embodiment of the present invention.
This will be described with reference to FIG. In this embodiment, the first
The same components as those in the embodiment have the same reference numerals, and detailed description thereof will be omitted.

FIG. 2 shows a block circuit of the MPEG system decoder 31 of this embodiment. System decoder 31
Comprises an MPEG audio decoder 32, an MPEG video decoder 33, and an AV parser 4. The AV parser 4 includes a demultiplexer (DMUX) 5.

The AV parser 4 outputs the separated audio stream, SCR, PTS (A) to the audio decoder 32, and outputs the video stream, PTS (V) to the video decoder 33, respectively. Here, in the present embodiment, unlike the first embodiment, the AV parser 4 does not provide the video decoder 33 with the SCR.

The audio decoder 32 has a register 1
1, a bit buffer 12, a decode core circuit 13, a control circuit 14, and a time stamp generation circuit 41. The time stamp generation circuit 41, as described later,
Time stamp A ₂ -PTS (hereinafter abbreviated as A ₂ -PTS)
Generate The control circuit 14 controls the time stamp generation circuit 41 as well as the register 11, the bit buffer 12, and the decode core circuit 13. The control circuit 14
Calculates the reproduction time (reproduction timing) of the audio output based on the SCR and PTS (A), and does not consider the internal delay time of the audio decoder 32.

The video decoder 33 includes a register 21, a bit buffer 22, a decode core circuit 23, and a control circuit 42. The control circuit 42 calculates the playback time of the video output, and controls the decode core circuit 23 according to the calculation result. The playback time is based on the A ₂ -PTS generated from the time stamp generation circuit 41 and the decoding processing time of the video decoder 33, that is, the video decoder 3
3 is calculated based on the internal delay time (hereinafter referred to as video decode delay time) D (t) and PTS (V). The video decode delay time D (t) is the sum of the internal delay time VD of the bit buffer 22 and the internal delay time ΔV of the decode core circuit 23.

FIG. 3 shows a block circuit of the time stamp generation circuit 41. The time stamp generation circuit 41 includes a delay time calculation circuit 51, a subtraction circuit 52, a sampling frequency detection circuit 53, and an addition circuit. The delay time calculation circuit 51 calculates an internal delay time AD of the bit buffer 12. The internal delay time AD changes depending on the occupation amount of the audio stream in the bit buffer 12, and the larger the occupation amount, the larger the internal delay time AD. The subtraction circuit 52 outputs the P
From TS (A) to internal delay time AD and decode core circuit 1
3 is subtracted from the sum of the internal delay times ΔA to generate A ₁ -PTS. That is, A ₁ -PTS is generated according to the following equation.

A ₁ -PTS = PTS (A) -AD-ΔA Therefore, A ₁ -PTS is affected by the internal delay times AD and ΔA with respect to PTS (A). Internal delay time Δ
A is a constant value.

The sampling frequency detection circuit 53 detects a sampling frequency of audio data from the audio stream, and generates a clock signal CK corresponding to the sampling frequency. The sampling frequency is set to 44.1 kHz in the CD (Compact Disc) standard. The addition circuit 54 adds A ₁ -PTS and the clock CK to generate A ₂ -PTS. Here, adding the clock CK to A ₁ -PTS is performed in real time by A ₂ -PTS.
This is for generating S. As described above, PTS (A)
Is added to the packet header of the packet when the beginning of the audio frame (or access unit) is included in the packet. However, if the beginning of the audio frame is not included in the packet, PTS (A) is not added. If a packet has two or more audio frame heads, only the PTS (A) corresponding to the first audio frame is added to the packet header of the packet. Even if the beginning of an audio frame is included in a packet, P
TS (A) is not always added. Thus P
With the addition of TS (A), PTS (A) is read from register 11 only intermittently. Therefore, the time stamp generation circuit 41 outputs the PTS
When (A) is not read, the previously read PTS
It generates A _1-PTS from (A), to generate the A _2-PTS by adding the clock CK to the A _1-PTS. This allows
The time stamp generation circuit 41 outputs A ₂ -P in real time.
Generate TS. Each time a new PTS (A) is read from the register 11, the A ₂ -PTS is newly generated regardless of the previously generated A ₂ -PTS.

As described above, the time stamp generation circuit 41
Is the internal delay time of the audio decoder 32 (= AD +
And .DELTA.A), generates the A _2-PTS based on the clock CK corresponding to the sampling frequency of the audio data. Therefore, the effects of the internal delay times AD and ΔA and the clock CK are added to A ₂ -PTS with respect to PTS (A).

FIG. 4 shows a block circuit of the video decoder 33. The control circuit 42 includes a write address detection circuit 6
1, read address detection circuit 62, picture header detection circuit 63, mapping circuit 64, register 65, synchronization determination circuit 66, first and second comparison processing circuits 67 and 70, first and second subtraction circuits 68 and 69, and Each circuit 61-70
Is provided. The control core circuit 71 includes the bit buffer 22 and the decode core circuit 2.
3 is also controlled.

When a video stream is accumulated in the bit buffer 22, the write address detection circuit 61
The address Add of the packet to which S (V) has been added is detected. More specifically, the AV parser 4 separates PTS (V) from the video stream, the bit buffer 22 stores the video stream, and the register 21 stores the PTS (V).
Store TS (V). At this time, the write address detection circuit 61 determines that the video stream to which the PTS (V) has been added is written to the bit buffer 22 without separating the PTS (V), and determines the address of the packet to which the PTS (V) has been added. Add is detected. This means that the detected address Add corresponds to the address of PTS (V). The reason why the address Add of the packet can be made to correspond to the address of the PTS (V) is as follows. The data amount of the PTS (V) is sufficiently smaller than the data amount of the packet, and the address of the packet stored in the bit buffer 22 does not change even if the PTS (V) is included in the video stream.

The control core circuit 71 receives the detected address.
Add is sequentially stored in the register 21 in association with PTS (V).
I do. The register 21 is, for example, a stack of (n + 1) stages.
It is composed of Register 21 contains the video stream
(N + 1) PTS (V_m) ~
PTS (V_{m + n}) Is the corresponding address Add_m~ Add
_{m + n}And are accumulated in one set. Read address detection
The output circuit 62 outputs the video read from the bit buffer 22.
Detect the address of the video stream. Picture header
The detection circuit 63 detects the bit written in the bit buffer 22.
Picture header at the beginning of each picture in the video stream
The header specified in each picture header is detected.
Detect the type of the texture (I, P, B). System
The control core circuit 71 performs bit-backup according to the detection result.
Video for one picture at regular intervals from file 22
Read a stream.

The first comparison processing circuit 67 calculates the address of the video stream read from the bit buffer 22 and
The PTS (V) (PTS) read from the register 21
(V _m )) is compared with the address Add (Add _m ), and it is determined whether or not both addresses match. The mapping circuit 64 determines the PTS according to the determination result of the first comparison processing circuit 67 and the detection result of the picture header detection circuit 63.
(V) is mapped to a picture. This mapping will be described below.

The operation of each of the circuits 62 to 64, 67 will now be described.
5 (a), an example of the video stream shown in FIG.
Therefore, it will be described. As shown in FIG.
The ream is composed of two packets P and Q, and each packet
PTS (V)_m), PTS
(V_{m + 1}) Is added to each. 3 packets P
B pictures B1, B2, and B3. This B
The head of the structure B1 is not in the packet P. Packet
Q is B picture B3, I picture I1, and P picture
Key P1. The head of this B picture B3 is
Not in Ket Q. That is, PTS (V_m) Is B pic
PTS (V) corresponding to channels B2 and B3.
_{m + 1}) Is the P corresponding to the I picture I1 and the P picture P1.
TS (V). Then, each PTS (V_m), PTS (V
_{m + 1}) Indicates each address Add_m, Add _{m + 1}Correspond to each P
TS (V_m), PTS (V_{m + 1}) And each address Add_m,
Add _{m + 1}Are stored in the register 21. FIG. 5 (a)
The video stream shown in FIG.
As shown in FIG. 5B, each PTS (V_m), PTS
(V_{m + 1}) Is accumulated with the exclusion.

When the video stream is read from the bit buffer 22, the read address detection circuit 62 detects the address of the video stream, and the first comparison processing circuit 67 determines the address and the address Add _m stored in the register 21. Compare with The picture header detection circuit 63 detects a picture header attached to the head of the picture of the read video stream. When the first comparison processing circuit 67 determines that the two addresses match,
The mapping circuit 64 determines that the picture preceded by the detected picture header (B picture B2 in this case) corresponds to the address Add _m (that is, PTS (V _m )). Specifically, as shown in FIG.
The mapping circuit 64 determines that BTS corresponds to PTS (V _m ).
It is B picture B2 instead of picture B1, and PTS (V
It is determined that what corresponds to ( _{m + 1} ) is not the B picture B3 but the I picture I1. This determination operation is the mapping.

As described above, in this embodiment, each of the circuits 62 to 64 and 67 performs the same operation as that of calculating the internal delay time VD of the bit buffer 22. That is, when a video stream is provided from the bit buffer 22 to the decode core circuit, associating each picture with PTS (V) corresponds to the calculation of the internal delay time VD. Therefore, the PTS read from the register 21
(V) shows the PTS at the time of writing to the register 21.
(V), the internal delay time VD of the bit buffer 22
The effect is taken into account. The operations of the circuits 62 to 64 and 67 in the video decoder 33 correspond to the operations of the delay time calculation circuit 51 in the audio decoder 32.
The internal delay time VD changes depending on the occupation amount of the video stream in the bit buffer 22, and the larger the occupation amount, the larger the internal delay time VD.

The register 65 is formed of a one-stage stack, and performs an operation according to the inter-frame prediction technique. The operation is based on the PTS (V) corresponding to the I picture or the P picture and the PTS (V) corresponding to the B picture according to the picture type (I, P, B) detected by the picture header detection circuit 63. Replace with

The first subtraction circuit 68 calculates the internal delay time ΔV of the decode core circuit 23 from the PTS (V) read from the register 65 and the value x set by the external input device 43 shown in FIG. Is subtracted to generate V ₁ -PTS.
That is, V ₁ -PTS is generated according to the following equation.

V ₁ -PTS = PTS (V) -ΔV-x Here, PTS (V) read from the register 65 reflects the influence of the internal delay time VD. Therefore, the PT written in the register 21 is stored in V ₁ -PTS.
The video decoding delay time D (t) (= V
D + ΔV) and the value x. The internal delay time ΔV is a constant value. The value x is set by the user operating the input device 43.

The second subtraction circuit 69 subtracts V ₁ -PTS from A ₂ -PTS generated from the time stamp generation circuit 41 to generate V ₂ -PTS. That is, V ₂ -PTS is generated according to the following equation.

V ₂ -PTS = A ₂ -PTS-V ₁ -PTS = A
₂ -PTS-PTS (V) + ΔV + x Here, A ₂ -PTS is generated in real time.
Therefore, no matter what timing V ₁ -PTS is generated, V ₂ -PTS is generated reliably (or in real time).

The second comparison processing circuit 70 compares the value y set by the external input device 44 shown in FIG. 2 with V ₂ -PTS. The value y is set by the user operating the input device 44, and is set so as to be larger than half the time during which one picture is being reproduced. When the mapping between the PTS (V) and the picture is performed by the mapping circuit 64, the synchronization determination circuit 66
The control signals SS, Sn, SR are generated according to the comparison result of 0. The synchronization determination circuit 66 generates a control signal SS when V ₂ -PTS <-y. The synchronization determination circuit 66
When y ≦ V ₂ -PTS ≦ y (that is, | V ₂ -PTS | ≦ y), the control signal Sn is generated, and when y <V ₂ -PTS, the control signal SR is generated.

The synchronization determination circuit 66 determines whether A ₂ -PTS and V
_When the value y is sufficiently smaller than _1- PTS, the control signals SS, Sn and SR are generated in the following cases. The synchronization determination circuit 66 generates the control signal SS when A ₂ -PTS <V ₁ -PTS. The synchronization determination circuit 66 calculates A ₂ -PT
When S = V ₁ -PTS, a control signal Sn is generated and A ₂ -P
When TS> V ₁ -PTS, a control signal SR is generated. The control signals SS, Sn, SR are input to the decode core circuit 23 to control the decode core circuit 23.

The decode core circuit 23 decodes the video stream read from the bit buffer 22,
Generate a video output for each picture. Here, when the control signal SS is being generated, the decode core circuit 23 performs a skip operation. More specifically, while the control signal SS is being generated, the decode core circuit 23 discards the picture transferred from the bit buffer 22, and does not decode the discarded picture.
When the generation of the control signal SS is stopped, the decode core circuit 23 returns to the normal operation. As a result, on the display 25, skip reproduction in which the reproduction screen is skipped by several frames is performed.

When the control signal Sn is being generated, the decode core circuit 23 performs a normal operation, and
At 5, normal reproduction is performed. When the control signal SR is being generated, the decode core circuit 23 performs a repeat operation. More specifically, while the control signal SR is being generated, the decode core circuit 23 continues to output the video output of the picture transferred from the bit buffer 22 before the control signal SR is generated. Then, when the generation of the control signal SR is stopped, the decode core circuit 23 returns to the normal operation. As a result, on the display 25, the repeat playback in which the same playback screen continues is performed.

For example, A_Two-PTS and V₁-Compared to PTS
When all the values y are sufficiently small, the decode core circuit 23
Is A_Two-PTS <V₁Skip operation is performed for -PTS
A _Two-PTS = V₁-For PTS, perform normal operation
A, and A_Two-PTS> V₁-Repeat operation for PTS
Is performed respectively.

The reason why the value y can be designated by the input device 44 is that A ₂ -PTS and V ₁ -PTS rarely coincide completely. Decode core circuit 23, and A _2-PTS and V _1-PTS performs normal operation when perfectly matched _{(A 2 -PTS = V 1 -PTS} ). Therefore, the value y compared to A ₂ -PTS and V ₁ -PTS
Is sufficiently small, the decode core circuit 23 rarely performs a normal operation. Therefore, the viewer (or the user) appropriately sets the value y, so that A
_{Even if the 2-} PTS and V ₁ -PTS do not completely match, if they almost match, the decode core circuit 23 performs a normal operation. That is, the value y can be specified by the input device 44 so that the matching condition between A ₂ -PTS and V ₁ -PTS has an allowable range.

The value x can be specified by the input device 43 so that the decoding core circuit 23 can arbitrarily perform a skip operation or a repeat operation. As the viewer adjusts the value x, V ₁ -PTS changes. The decode core circuit 23 performs each operation according to the change of V ₁ -PTS.

As described above, in this embodiment, the control circuit 4
2 is the internal delay time of the audio decoder 32 (= AD
+ .DELTA.A) and A _2-PTS generated based on a video decode delay time D (t) (= VD + ΔV), P based on the values x
By correcting TS (V), V ₂ -PTS is generated. And
The control circuit 42 generates one of the control signals SS, Sn, SR in accordance with the result of comparison between V ₂ -PTS and the value y, and performs a decoding operation so as to perform any of a skip operation, a normal operation, and a repeat operation. The circuit 23 is controlled.

Therefore, the control circuit 42 controls not only the internal delay time of the video decoder 33 but also the audio decoder 3.
The decoding core circuit 23 is also controlled based on the internal delay time of 2. That is, the control circuit 42 controls the video decoder 33
The playback time of the video output is set in consideration of not only the internal delay time of the audio decoder 32 but also the internal delay time of the audio decoder 32. In other words, the playback timing of the video output is adjusted according to the playback timing of the audio output. This makes it possible to synchronize the audio output and the video output more reliably than in the first embodiment.

For example, when the playback timing of the video output is later than the playback timing of the audio output,
The control circuit 42 causes the decode core circuit 23 to perform a skip operation, and the display 25 performs skip reproduction. As a result, the playback timing of the video output catches up with the playback timing of the audio output. Conversely, when the playback timing of the video output is ahead of the playback timing of the audio output, the control circuit 42 causes the decode core circuit 23 to perform a repeat operation, and
Performs repeat playback. As a result, the playback timing of the video output matches the playback timing of the audio output.

The reason why the playback timing of the video output is made to match the playback timing of the audio output is as follows. Humans cannot detect the displacement of the moving image displayed on the display 25 even if the moving image is shifted by several frames. On the other hand, when the sound output from the speaker 27 is displaced, the human can detect the displacement as noise that is easily heard.

Adjusting the value y by the user is
Set the synchronization accuracy of the video output and video output arbitrarily
Enable. The larger the value y is set, the more A_Two-PTS
And V ₁-The permissible range of the matching condition with PTS has increased,
As a result, the accuracy of the synchronization between audio output and video output is low.
It becomes. In this way, the audio output and video output are
The period accuracy can be set in the system stream.
Some PTSs (PTS (A) and PTS (V)
 ) May not be added accurately.
For example, a so-called video CD currently on the market
Indicates that the PTS is not correctly added.
You. The value y is half the time during which one picture is playing
It is set to be larger than a minute.
The audio output and video output
This is because the degree does not change.

Adjusting the value x by the user allows the phase of the audio output and the phase of the video output to be deliberately shifted. This function is provided by using this embodiment on a CD-ROM.
This is suitable when applied to a system stream read from a storage medium such as a storage medium. For example, when a user reproduces a moving image at a higher speed than a normal reproduction speed, a synchronization deviation occurs between an audio output and a video output, and the function can be exhibited when correcting the synchronization deviation.
When a video is played at a higher speed than the normal playback speed, the user performs fast-forward playback to watch the video in a short time, or performs fast-forward playback or fast-forward reverse playback to search for the video to be viewed. In that case, the audio output is also reproduced.

(Third Embodiment) FIG. 6 shows a third embodiment of the present invention.
This will be described with reference to FIG. In the present embodiment, the same components as those in the second embodiment have the same reference numerals, and detailed description thereof will be omitted.

FIG. 6 shows a block circuit of the MPEG system decoder 81 of this embodiment. System decoder 81
Includes an audio decoder 32, an MPEG video decoder 82, and an AV parser 4. The AV parser 4 has a DMUX 5. The AV parser 4 outputs the audio stream, SCR, and PTS (A) to the audio decoder 31, respectively, and outputs the video stream and PTS (V) to the video decoder 82, respectively.

The video decoder 82 includes a register 21, a bit buffer 22, a decode core circuit 23, and a control circuit 83. The control circuit 83 includes A ₂ -PTS,
The decoding processing time of the video decoder 82 and the PTS (V)
, The playback time of the video output is calculated, and the calculation result is corrected. Then, it controls the decode core circuit 23 according to the correction result. The internal delay time of the video decoder 82 is the same as the internal delay time of the video decoder 33 of the second embodiment, and hereinafter, the video decode delay time D (t)
That.

FIG. 7 shows a block circuit of the video decoder 82. The control circuit 83 includes the write address detection circuit 6
1, read address detection circuit 62, picture header detection circuit 63, mapping circuit 64, register 65, synchronization determination circuit 66, first and second comparison processing circuits 67 and 70, first and second subtraction circuits 68 and 69, repeat Judgment circuit 8
4. Skip determining circuit 85, and circuits 61 to 70, 8
4 and 85 are provided. The control core circuit 72 also controls the bit buffer 22 and the decode core circuit 23.

The repeat determination circuit 84 is provided with the synchronization determination circuit 6
The control signal SRm is generated based on the control signals Sn and SR generated from the input device 6 and the values z1 and z2 set by the input devices 86 and 87 shown in FIG. Skip determination circuit 85
Are control signals Sn, generated from the synchronization determination circuit 66,
A control signal SSm is generated according to SS, values w1 and w2 set by the input devices 88 and 89 shown in FIG.

The decode core circuit 23 controls each control signal SS
, Sn, SR, not the control signals SSm, Sn, SRm
Works according to When the control signal SSm is being generated, the decode core circuit 23 performs a skip operation. When the control signal Sn is generated, the decode core circuit 23
Performs normal operation. Furthermore, when the control signal SRm is being generated, the decode core circuit 23 performs a repeat operation.

FIG. 8 shows a block circuit of the repeat determination circuit 84. The repeat determination circuit 84 includes a counter 91,
92, first to third comparison processing circuits 93 to 95, OR (O
R) circuit 96 is provided. The counter 91 outputs the control signal S
Every time R is generated, the count operation is performed while incrementing the count value. The first comparison processing circuit 93 compares the count value of the counter 91 with the value z1, and generates a control signal SRm when the count value is larger. The OR circuit 96 outputs a reset signal to the counter 91 when at least one of the control signals Sn and SRm is generated. The counter 91 resets the count value in response to the reset signal.

The second comparison processing circuit 94 outputs a count start signal to the counter 92 when the count value of the counter 91 is larger than zero. The counter 92 starts the count operation in response to the count start signal, and increments the count value at regular intervals. The third comparison processing circuit 95 compares the count value of the counter 92 with the value z2, and when the count value is larger, generates a control signal SRm and outputs a reset signal to the counter 92. The counter 92 resets the count value in response to the reset signal.

The first comparison processing circuit 93 generates the control signal SRm when the control signal SR is continuously generated by the counter value of the counter 91 by the number of times greater than the value z1. Therefore, unless the control signal SR is generated continuously more times than the value z1, the decode core circuit 23 does not perform the repeat operation. This is because even when the playback timing of the video output is not advanced from the playback timing (or playback timing) of the audio output, the synchronization determination circuit 66 erroneously generates the control signal SR and outputs the control signal SR. This is because 23 may perform a repeat operation. For example, when the PTS (A) or PTS (V) is incorrect, or when the moving image is reproduced at a higher speed than the normal reproduction speed, the synchronization determination circuit 66 may erroneously generate the control signal SR. is there.

Therefore, when the synchronization determination circuit 66 generates the control signal SR more than a certain number of times (= z1) continuously, the repeat determination circuit 84 controls the control signal Sr.
It determines that R is correct and generates a control signal SRm. The decode core circuit 23 performs a repeat operation according to the control signal SRm. The generation of the control signal SRm in this manner is based on the fact that the decode core circuit 23
Prevents a repeat operation.

When the control signal SR is generated once, the counter 92 starts a counting operation.
The third comparison processing circuit 95 generates the control signal SRm regardless of the count value of the counter 91 at that time.
This fixed time is determined by the increment speed of the counter 92 and the value z2. Therefore, the synchronization determination circuit 66
Generates a control signal SR after a certain period of time,
The decode core circuit 23 performs a repeat operation. This is because even if the control signal SR is correct, the control signal SR is not always generated more than a certain number of times (= z1). Therefore, when a certain period of time has elapsed since the generation of the control signal SR, the repeat determination circuit 84 determines that the control signal SR is a correct signal and generates the control signal SRm. Decode core circuit 23
Performs a repeat operation according to the control signal SRm. The operations of the second and third comparison processing circuits 94 and 95 and the counter 92 complement the operations of the counter 91, the first comparison processing circuit 93, and the OR circuit 96, and reliably generate the control signal SRm.

FIG. 9 shows a block circuit of the skip determination circuit 85. The skip determination circuit 85 includes a counter 10
1, 102; first to third comparison processing circuits 103 to 105;
OR (OR) circuit 106, B picture priority processing circuit 10
7 is provided.

Each time the control signal SS is generated, the counter 101 increments its count value. The first comparison processing circuit 103 compares the count value of the counter 101 with the value w1, and generates a control signal SSp when the count value is larger. The second comparison processing circuit 104 outputs a count start signal to the counter 102 when the count value of the counter 101 is larger than zero. Counter 1
02 starts the count operation in response to the count start signal, and increments the count value at regular intervals.
The third comparison processing circuit 105 compares the count value of the counter 102 with the value w2, and generates a control signal SSp when the count value is larger.

The B picture priority processing circuit 107 generates a control signal SSm according to the control signal SSp and the picture type (I, P, B) detected by the picture header detection circuit 63. The control signal SSm is a signal that causes the decode core circuit 23 to perform a skip operation by giving priority to a B picture over an I picture or a P picture. OR circuit 10
6 outputs a reset signal to the counter 101 when at least one of the control signals Sn and SSm is generated. The counter 101 resets the count value in response to the reset signal. The counter 102 resets the counter value when the control signal SSm is generated. The first comparison processing circuit 103 generates a control signal SSp when the control signal SS is continuously generated a number of times greater than the value w1 according to the counter value of the counter 101. The B picture priority processing circuit 107 generates a control signal SSm according to the control signal SSp and the picture type (I, P, B). Therefore, unless the control signal SS is continuously generated for a number of times greater than the value w1, the decode core circuit 23 does not perform the skip operation. This is because the synchronization determination circuit 66 may erroneously generate the control signal SS even when the playback timing of the video output is not ahead of the playback timing of the audio output. For example, when the PTS (A) or PTS (V) is incorrect, or when the moving image is reproduced at a higher speed than the normal reproduction speed, the synchronization determination circuit 66 may erroneously generate the control signal SS. is there.

Therefore, when the synchronization determination circuit 66 generates the control signal SS more than a certain number of consecutive times (= w1), the skip determination circuit 85 controls the control signal Ss.
S is determined to be correct, and a control signal SSp is generated.
The decode core circuit 23 performs a skip operation according to the control signal SSp. Generating the control signal SSp in this way is based on the erroneous control signal SS,
3 prevents the skip operation from being performed.

Incidentally, since the B picture is generated by bidirectional prediction, the data amount is small, and its importance is I.
It is lower than that of a picture or a P picture. Therefore, skip playback with priority given to B-pictures of low importance reduces dropped frames that occur in the reproduced moving image.

When the control signal SS is generated once, the counter 102 starts the counting operation, and after a predetermined time, the third comparison processing circuit 105 outputs the control signal regardless of the count value of the counter 101 at that time. Generate SSp. This fixed time is determined by the increment speed of the counter 102 and the value w2. Therefore, when a predetermined time has elapsed since the synchronization determination circuit 66 generated the control signal SS, the decode core circuit 23 performs a skip operation.
This is because even if the control signal SS is correct,
This is because the control signal SS is not always generated more than a certain number of times (= w1). Therefore, when a certain period of time has elapsed since the generation of the control signal SS, the skip determination circuit 85 determines that the control signal SS is a correct signal and generates the control signal SSp. The decode core circuit 23 performs a skip operation according to the control signal SSp. Thus, the second and third comparison processing circuits 104, 10
5 and the operation of the counter 102 complement the operations of the counter 101, the first comparison processing circuit 103, and the OR circuit 106, and reliably generate the control signal SSp.

In this embodiment, even if erroneous control signals SR and SS are generated for some reason,
Each of the determination circuits 84 and 85 corrects each of the control signals SR and SS to generate each of the control signals SRm and SSm. The decode core circuit 23 operates (skip operation, repeat operation) according to each control signal SRm, SSm. This makes it possible to reliably synchronize the audio output and the video output even when erroneous control signals SR and SS are generated.

The setting of the values z1, z2, w1, and w2 by the user using the input devices 86 to 89 enables the determination circuits 84 and 85 to adjust the degree of correction of the control signals SR and SS. . When skip playback is performed, skip playback of B-pictures of low importance is given priority over I-pictures and P-pictures, so that the number of dropped frames that occur in the played back moving image is reduced and the moving image moves smoothly. As a result, the image quality can be improved.

The above embodiments may be modified as follows. (1) In the second and third embodiments, the sampling frequency detection circuit 53 and the addition circuit 54 are omitted. In this case,
Since the PTS cannot be generated in real time, the generation accuracy of the #PTS decreases. However, even in this case, the synchronization between the audio output and the video output can be more reliably achieved than in the first embodiment. When many PTS (A) are added to the audio stream, the same performance as the second and third embodiments can be obtained.

(2) In the second and third embodiments, ΔPTS
Instead, the subtraction circuit 69 generates a value obtained by subtracting [V] PTS from PTS (A). In this case, PTS (A)
, PTS (V) is corrected based on the video decoding delay time D (t) and the value x, and #PTS is generated. In this case, the operation of the decode core circuit 23 cannot be controlled based on the internal delay time of the audio decoder 32, but the reproduction timing of the video output is controlled in accordance with the reproduction timing of the audio output. There is no. Therefore, when the internal delay time of the audio decoder 32 is small, the same performance as that of the second and third embodiments can be obtained. However, in this case, the control circuit 1
4 calculates the reproduction time (reproduction timing) of the audio output based on the internal delay time of the audio decoder 32 and the SCR and PTS (A) in the same manner as in the first embodiment, and decodes the decoding core circuit 13 according to the calculation result. Control.

(3) In the second and third embodiments, the value x is omitted. Further, the value y is fixed. In this case, each value x,
Other functions and effects are the same as those of the above embodiments, except that the function related to y is omitted.

(4) In the third embodiment, the comparison processing circuits 94 and 95 and the counter 92 are omitted from the repeat determination circuit 84. In this case, only the functions related to the circuits 94, 95, and 92 are omitted, and other functions and effects are the same as those of the above-described embodiment.

(5) In the third embodiment, the comparison processing circuits 104 and 105 and the counter 102 are omitted from the skip determination circuit 85. Also, when the skip determination circuit 85
The picture priority processing circuit 107 is omitted. In this case, only the functions related to the circuits 104, 105, 102, and 107 are omitted, and other functions and effects are the same as those in the above-described embodiment.

(6) In the second and third embodiments, the register 21 is constituted by a one-stage stack. In this case, as the number of stacks in the register 21 increases, the PTS that can be used is increased.
(V) also increases, but the capacity of the register 21 also increases. Therefore, the number of stacks in the register 21 may be appropriately set based on the circuit scale, cost, and required performance.

(7) In the third embodiment, when the count value of each of the counters 91 and 101 is larger than a predetermined value equal to or greater than zero, each of the comparison processing circuits 94 and 104
The counting operation of 2, 102 is started.

(8) PTS is replaced with DTS, and the operation is performed in the same manner as in the above embodiment. In this case, the same operation and effect as the above embodiment can be obtained. Although the embodiments have been described above, technical ideas other than the claims that can be grasped from the embodiments will be described below together with their effects.

(A) In the MPEG system decoder according to the third aspect, the second value specified from the outside is set so as to be larger than half of the time during which one picture is reproduced. MPEG system decoder.

In this way, the synchronization of each output can be adjusted optimally. (B) an MPEG system decoder according to claim 3, wherein said sampling frequency is 44.1 kHz;
G system decoder.

In this way, the audible frequency band can be sufficiently covered. By the way, in this specification, a member according to the configuration of the invention is defined as follows.

(A) It is assumed that the separating means comprises an audio video parser. (B) The first repeat validating means includes a counter 91, a comparison processing circuit 93, and an OR circuit 96.

(C) The second repeat validating means comprises a counter 92 and comparison processing circuits 94 and 95. (D) The first skip enabling means includes a counter 101,
It comprises a comparison processing circuit 103 and an OR circuit 106.

(E) The second skip enabling means comprises a counter 102 and comparison processing circuits 104 and 105. (F) The first value is a value x and the second value is a value y.

(G) The second register is the register 65, the first comparison processing circuit is the comparison processing circuit 67, the second comparison processing circuit is the comparison processing circuit 70, and the first subtraction circuit is the subtraction circuit 6.
8. The second subtraction circuit is a subtraction circuit 69.

(H) The time stamp includes not only the PTS but also the DTS.

[0127]

According to the present invention, it is possible to provide an MPEG system decoder capable of sufficiently synchronizing an audio output and a video output.

[Brief description of the drawings]

FIG. 1 is a block circuit diagram of a first embodiment.

FIG. 2 is a block circuit diagram of a second embodiment.

FIG. 3 is a main part block circuit diagram of a second embodiment.

FIG. 4 is a main part block circuit diagram of a second embodiment.

FIG. 5 is an explanatory diagram for explaining a second embodiment.

FIG. 6 is a block circuit diagram of a third embodiment.

FIG. 7 is a main part block circuit diagram of a third embodiment.

FIG. 8 is a main part block circuit diagram of a third embodiment.

FIG. 9 is a main part block circuit diagram of a third embodiment.

FIG. 10 is an explanatory diagram for explaining an MPEG system stream.

FIG. 11 is a block circuit diagram of a conventional example.

[Explanation of symbols]

 1,31 MPEG system decoder 2,32 MPEG audio decoder 3,33,82 MPEG video decoder 4 audio video parser 5 DMUX 11,21 register 12,22 bit buffer 13,23 decode core circuit 14,24,42,83 control circuit 41 Time stamp generation circuit 84 Repeat judgment circuit 85 Skip judgment circuit

──────────────────────────────────────────────────続 き Continuation of the front page (56) References JP-A-2-35886 (JP, A) JP-A-6-96574 (JP, A) JP-A-6-97927 (JP, A) JP-A-7-97 177479 (JP, A) JP-A-6-343065 (JP, A) JP-A-6-237437 (JP, A) JP-A-6-333341 (JP, A) JP-A-6-343065 (JP, A) (58) Field surveyed (Int. Cl. ⁷ , DB name) H03M 7/30

Claims (9)

(57) [Claims] 1. A demultiplexer for separating an MPEG system stream into an MPEG video stream and an MPEG audio stream based on a packet header of the MPEG system stream for an MPEG system stream transferred from the outside, an SCR and audio from the MPEG system stream. Separating means for separating the time stamp of the video signal from the time stamp of the video, an MPEG audio decoder comprising an audio register, an audio bit buffer, an audio decoding core circuit and an audio control circuit, a video register, a video bit buffer and a video MPEG comprising a decode core circuit and a video control circuit
An MPEG system decoder comprising a video decoder, wherein the audio register sequentially accumulates the time stamp of the audio transferred from the separating means, and the audio bit buffer sequentially stores the audio stream transferred from the demultiplexer. The audio decoding core circuit decodes the audio stream read from the bit buffer in accordance with the MPEG audio part, and generates an audio output. The audio control circuit determines a decoding processing time in the MPEG audio decoder. The playback timing of the audio output is calculated based on the decoding processing time, the SCR transferred from the separation means, and the time stamp of the audio read from the register. Controlling the decode core circuit according to the timing, the video register sequentially accumulates the time stamp of the video transferred from the separating means, the video bit buffer sequentially stores the video stream transferred from the demultiplexer, The video decoding core circuit decodes the video stream read from the bit buffer in accordance with the MPEG video part, generates a video output, the video control circuit calculates a decoding processing time in the MPEG video decoder, Based on the decoding processing time, the SCR transferred from the separating means, and the time stamp of the video read from the register, the playback timing of the video output is calculated, and the decode core circuit is controlled according to the playback timing. The MPEG audio A coder that generates a second time stamp based on an internal delay time of an audio bit buffer, an internal delay time of an audio decode core circuit, and a time stamp of audio read from an audio register; A video time stamp is mapped to a picture, and a skip operation is performed by the video decode core circuit based on the internal delay time of the video decode core circuit, the video time stamp read from the video register, and the second time stamp. Or a skip determination circuit or a repeat determination circuit for determining and correcting an error in a control signal for causing a video decoding core circuit generated from the video control circuit to perform a skip operation or a repeat operation. MPEG system decoder 2. A demultiplexer for separating an MPEG system stream into an MPEG video stream and an MPEG audio stream based on a packet header of the MPEG system stream for an MPEG system stream transferred from the outside, and an SCR and audio from the MPEG system stream. Separating means for separating the time stamp of the video signal from the time stamp of the video, an MPEG audio decoder comprising an audio register, an audio bit buffer, an audio decoding core circuit and an audio control circuit, a video register, a video bit buffer and a video MPEG comprising a decode core circuit and a video control circuit
An MPEG system decoder comprising a video decoder, wherein the audio register sequentially stores time stamps of audio transferred from the separating means in a FIFO configuration, and the audio bit buffer is a RAM in a FIFO configuration.
And sequentially accumulates the audio stream transferred from the demultiplexer. The audio decode core circuit decodes the audio stream read from the bit buffer in accordance with the MPEG audio part, and generates an audio output. The audio control circuit determines the MPEG time based on the time required for reading the audio stream from the bit buffer and the decode processing time in the decode core circuit.
A decoding processing time in the audio decoder is calculated, and a reproduction timing of the audio output is calculated based on the decoding processing time, the SCR transferred from the separating unit, and the time stamp of the audio read from the register. The video register controls the decode core circuit in accordance with the reproduction timing, the video register sequentially stores time stamps of the video transferred from the separating means in a FIFO configuration, the video bit buffer includes a RAM in a FIFO configuration, and a demultiplexer. The video stream core circuit sequentially accumulates the transferred video streams, the video decode core circuit decodes the video stream read from the bit buffer in accordance with the MPEG video part, and generates a video output. Bit buffer The decoding processing time in the MPEG video decoder is calculated from the time required for the video stream to be read from the video stream and the decoding processing time in the decoding core circuit, and the decoding processing time, the SCR transferred from the separating means,
The playback timing of the video output is calculated based on the time stamp of the video read from the register, and the decoding core circuit is controlled in accordance with the playback timing. The MPEG audio decoder adds a delay time calculation circuit, an audio subtraction circuit, and A time stamp generation circuit comprising a circuit and a sampling frequency detection circuit, wherein the delay time calculation circuit calculates an internal delay time of an audio bit buffer, and the audio subtraction circuit includes an internal delay time of the audio bit buffer and audio data. Based on the internal delay time of the decode core circuit and the audio time stamp read from the audio register, generate a value obtained by subtracting the sum of the internal delay times from the audio time stamp, and the sampling frequency detection circuit Audios Detecting a sampling frequency of audio data from the stream, generating a clock corresponding to the sampling frequency, the adding circuit adding a value generated by an audio subtracting circuit and the clock to generate a second time stamp, The video control circuit includes a write address detection circuit, a read address detection circuit, a picture header detection circuit, a mapping circuit, a second register, a synchronization determination circuit, first and second comparison processing circuits, and first and second video. A write address detection circuit, when a packet to which a video time stamp is added in an externally transferred video stream is written to a video bit buffer, an address of the packet in the video bit buffer. And the video register detects The address detected by the write address detection circuit and the time stamp of the video are sequentially stored in association with each other. The read address detection circuit detects the address of the video stream read from the video bit buffer, and The detection circuit detects a picture header attached to the head of each picture of the video stream written to the video bit buffer, detects a type of a picture defined in the picture header, and the first comparison processing circuit Comparing the address of the video stream read from the video bit buffer with the address corresponding to the time stamp of the video read from the video register to detect whether both addresses match, First comparison processing circuit and picture header detection Performing a mapping between a video time stamp and a picture based on the detection result of the output circuit, wherein the second register is configured by a one-stage stack and detected by a picture header detection circuit according to an inter-frame prediction technique. The first video subtraction circuit replaces a time stamp of a video corresponding to an I picture or a P picture with a time stamp of a video corresponding to a B picture, based on a type of the picture. From the video time stamp, based on the first time value externally specified and the video time stamp read from the second register, and the sum of the internal delay time and the first externally specified value The second video subtraction circuit generates a value obtained by subtracting the second Generating a value obtained by subtracting the value generated by the first video subtraction circuit from the imstamp; and the second comparison processing circuit outputs the second value specified by the external device and the value generated by the second video subtraction circuit When the mapping circuit performs mapping between the video time stamp and the picture, the synchronization determination circuit causes the video decoding core circuit to perform a skip operation or a repeat operation based on the comparison result of the second comparison processing circuit. In the video decoding core circuit, in the skip operation, the picture transferred from the video bit buffer is discarded, and the discarded picture is not decoded. The video output of the picture transferred from the video bit buffer is continuously output, and MPEG system decoder having a skip determining circuit or repeat determining circuit for correcting and determining an error of the control signal for causing a skip operation or repeat operation to the video decode core circuit generated from the video control circuit. 3. The MPE according to claim 1 or claim 2.
In the G system decoder, when a control signal for causing a video decode core circuit generated by the video control circuit to perform a skip operation is continuously generated for a certain number of times or more, a first signal for validating the control signal is provided. MPEG system decoder provided with skip enable means. 4. The MPE according to claim 1 or claim 2.
In the G system decoder, a control signal for enabling a video decode core circuit generated by the video control circuit to perform a skip operation is generated a predetermined time after the control signal is enabled.
MPEG system decoder provided with skip enable means. 5. The MPE according to claim 1 or claim 2.
In the G system decoder, when a control signal generated by the video control circuit for causing a video decode core circuit to perform a repeat operation is continuously generated for a certain number of times or more, a first signal that validates the control signal is provided. MPEG system decoder provided with a repeat validating means. 6. The MPE according to claim 1 or claim 2.
In the G system decoder, a control signal for enabling a video decode core circuit generated by the video control circuit to perform a repeat operation is generated a predetermined time after the control signal is enabled, and
MPEG system decoder provided with a repeat validating means. 7. The MPE according to claim 1 or claim 2.
In a G system decoder, when a control signal for causing a video decoding core circuit generated by the video control circuit to perform a skip operation is continuously generated for a predetermined number of times or more, the control signal is enabled, and the control signal is enabled. An MPEG system decoder provided with a skip determination circuit for validating a control signal a predetermined time after the first generation of the control signal, if the predetermined number has been continuously generated. 8. The MPE according to claim 1 or claim 2.
In a G system decoder, when a control signal for causing a video decode core circuit generated by the video control circuit to perform a repeat operation is continuously generated for a predetermined number of times or more, the control signal is enabled, and the control signal is enabled. An MPEG system decoder provided with a repeat determination circuit for validating a control signal a predetermined time after the first generation of the control signal, if the predetermined number has been continuously generated. 9. The M according to claim 1, wherein
In the PEG system decoder, the skip operation of the video decoding core circuit is preferentially performed on a B picture.