JP4997735B2 - Data processing apparatus, method thereof, and program - Google Patents

Description

The present invention relates to a data processing apparatus, method and program thereof, and outputs video and audio by decoding compressed video data and compressed audio data encoded based on, for example, MPEG2 (Moving Picture Experts Group Layer 2) system. It is suitable for application to a decoding device.
Compressed video data and compressed audio data are targeted for data that can be obtained from a recording medium such as a DVD / Blu-Ray Disc, digital broadcasting, and any digital media such as the Internet.

  In recent years, digital broadcast receivers are becoming popular as this type of decoding device. Such a digital broadcast receiver is configured to output video and audio according to a processing procedure RT1X as shown in FIG. 1 when a channel switching operation is performed by a user, for example.

As shown in FIG. 1, when the channel switching operation is performed by the user, the digital broadcast receiver moves to step SP1 and receives an NIT (Network Information Table). This NIT indicates physical information (in this case, satellite orbit, polarization, frequency for each transponder, etc.) regarding the transmission path.
Thus, the digital broadcast receiver can select a transponder corresponding to the requested channel requested by the channel switching operation based on the NIT, and as a result, can receive the transport stream corresponding to the requested channel. it can.

Next, the digital broadcast receiver moves to the process of step SP2 and receives a PAT (Program Association Table).
Then, the digital broadcast receiver moves to the processing of step SP3, and receives a PMT (Program Map Table) corresponding to the requested channel based on this PAT.

Subsequently, the digital broadcast receiver proceeds to the process of step SP4, and recognizes a PID (Program Identification) corresponding to the requested channel based on the received PMT. In this case, the PID corresponds to identification information for identifying a packet including compressed video data, compressed audio data, PCR (Program Clock Reference) data, and the like corresponding to the requested channel.
This digital broadcast receiver separates the packet corresponding to the recognized PID from the received transport stream, thereby streaming compressed video data, compressed audio data, PCR data, etc. corresponding to the requested channel. Can be obtained as

  Incidentally, the processing from step SP1 to SP4 described above is widely known as separation processing (DEMUX processing) (see Patent Document 1).

  By the way, the PCR data in the stream obtained in this way indicates the reference time on the encoding apparatus side where the compressed video data and the compressed audio data are encoded.

Therefore, when the digital broadcast receiver moves to the process of the next step SP5, the reference time (STC (System Time Clock)) counted in this digital broadcast receiver is set to the time indicated in the PCR data.
Thereby, the reference time measured in this digital broadcast receiver can be synchronized with the time on the encoding apparatus side.

In addition, in this stream, as shown in FIG. 2, for example, the time at which the video should be displayed for each compressed video data “Dpic (0)”, “Dpic (1)”,. Is associated with display time information (PTS (Presentation Time Stamp)).
Similarly, in this stream, for example, for each of compressed audio data “Dframe (0)”, “Dframe (1)”,. Time information (PTS) is associated.

  In this way, this digital broadcast receiver, for example, when the time reference time coincides with the display time information “pts (0)” associated with the compressed video data “Dpic (0)”, step SP6. Then, the video “Vpic (0)” based on the video data obtained as a result of decoding the compressed video data “Dpic (0)” is displayed on the display unit.

  In addition, the digital broadcast receiver performs the processing in step SP7 when the measured reference time matches the sound output time information “pts (0)” associated with the compressed audio data “Dframe (0)”. Then, the audio “Sframe (0)” based on the audio data obtained as a result of decoding the compressed audio data “Dframe (0)” is output via a speaker or the like.

  In this way, the digital broadcast receiver uses the video display timing for displaying the video “Vpic (0)”, and the audio “Sframe (0)” corresponding to the video “Vpic (0)”,. ,... Can be matched to some extent with the sound output timing.

  On the other hand, as a data reproducing apparatus for reproducing a disc as a recording medium on which moving images are digitized and recorded, a data reproducing apparatus corresponding to a variable rate as shown in FIG. 3 has been proposed (see Patent Document 2). ).

In this data reproducing apparatus, data recorded on the optical disc 1 is reproduced from the pickup 2. The pickup 2 irradiates the optical disc 1 with laser light, and reproduces data recorded on the optical disc 1 from the reflected light.
The reproduction signal output from the pickup 2 is input to the demodulation circuit 3 and demodulated. The data demodulated by the demodulating circuit 3 is input to the ECC circuit 5 via the sector detecting circuit 4 to detect and correct errors.

If the sector number (address assigned to the sector of the optical disk 1) is not normally detected in the sector detection circuit 4, a sector number abnormality signal is output to the track jump determination circuit 18.
The ECC circuit 5 outputs an error occurrence signal to the track jump determination circuit 18 when uncorrectable data occurs. The data subjected to the error correction is supplied from the ECC circuit 5 to the ring buffer memory 7 and stored therein.

  The ring buffer control circuit 6 reads an address for each sector from the output of the sector detection circuit 4 and designates a write address (write point (WP)) on the ring buffer memory 7 corresponding to the address. The ring buffer control circuit 6 designates the read address (reproduction point (RP)) of the data written in the ring buffer memory 7 based on the code request signal from the multiplexed data separation circuit 8 at the subsequent stage, and the reproduction point. Data is read from (RP) and supplied to the multiplexed data separation circuit 8.

The header separation circuit 9 of the multiplexed data separation circuit 8 separates the pack header and the packet header from the data supplied from the ring buffer memory 7 and supplies them to the separation circuit control circuit 11.
The separation circuit control circuit 11 sequentially connects the input terminal G and the output terminals (switched terminals) H1 and H2 of the switching circuit 10 according to the stream id information of the packet header supplied from the header separation circuit 9. Thus, the time-division multiplexed data is correctly separated and supplied to the corresponding code buffers 13 and 15.

  The video code buffer 13 generates a code request signal to the multiplexed data separation circuit 8 according to the remaining amount of the internal code buffer. Then, the received data is stored. It also receives a code request signal from the video decoder 14 and outputs internal data. The video decoder 14 reproduces a video signal from the supplied data and outputs it from the output terminal.

  The audio code buffer 15 generates a code request signal to the multiplexed data separation circuit 8 according to the remaining amount of the internal code buffer. Then, the received data is stored. It also receives a code request signal from the audio decoder 16 and outputs internal data. The audio decoder 16 reproduces an audio signal from the supplied data and outputs it from the output terminal.

Thus, the video decoder 14 requests data from the video code buffer 13, the video code buffer 13 issues a request to the multiplexed data separation circuit 8, and the multiplexed data separation circuit 8 requests the ring buffer control circuit 6. Put out.
At this time, data flows from the ring buffer memory 7 in the opposite direction to the request.

By the way, for example, when data processing relating to a simple screen continues and the data consumption amount per unit time of the video decoder 14 decreases, reading from the ring buffer memory 7 also decreases. In this case, the amount of data stored in the ring buffer memory 7 increases, which may cause overflow.
Therefore, the track jump determination circuit 18 calculates (detects) the amount of data currently stored in the ring buffer memory 7 based on the write point (WP) and the reproduction point (RP), and the data is set to a predetermined value set in advance. When the reference value is exceeded, it is determined that the ring buffer memory 7 may overflow, and a track jump command is output to the tracking servo circuit 17.

When the track jump determination circuit 18 detects a sector number abnormality signal from the sector detection circuit 4 or an error occurrence signal from the ECC circuit 5, the track jump determination circuit 18 stores the write address (WP) and read address (RP) in the ring buffer memory 7. While the remaining amount of data is obtained, reading from the ring buffer memory 7 to the multiplexed data separation circuit 8 is performed while the optical disk 1 makes one rotation from the current track position (while waiting for one rotation of the optical disk 1 ). Find the amount of data needed to guarantee.

  If the remaining amount of data in the ring buffer memory 7 is large, underflow does not occur in the ring buffer memory 7 even if data is read from the ring buffer memory 7 at the highest transfer rate. It is determined that the error can be recovered by reproducing the generated position again with the pickup 2, and a track jump command is output to the tracking servo circuit 17.

When the track jump determination circuit 18 outputs a track jump command, the tracking servo circuit 17 causes the playback position of the pickup 2 to jump from position A to position B on the inner circumference of one track as shown in FIG.
In the ring buffer control circuit 6, the playback position of the optical disk 1 is rotated once again until it reaches the position A from the position B, that is, the sector number obtained from the sector detection circuit 4 becomes the sector number at the time of track jump. Until this time, writing of new data to the ring buffer memory 7 is prohibited, and data already stored in the ring buffer memory 7 is transferred to the multiplexed data separation circuit 8 as necessary.

Further, even if the sector number obtained from the sector detection circuit 4 after the track jump matches the sector number at the time of the track jump, the amount of data stored in the ring buffer memory 7 exceeds a predetermined reference value, that is, If there is a possibility that the ring buffer memory 7 overflows, the writing of data to the ring buffer memory 7 is not resumed and a track jump is performed again.
In this manner, in this data reproducing apparatus, variable rate can correspondence by having a ring buffer memory 7, it is possible to further perform retry (retry) for errors.

  By the way, in the data reproducing apparatus described above, video data, audio data and subtitle data multiplexed at variable rates so as to comply with ISO11172 (MPEG1) or ISO13818 (MPEG2) are converted into video data, audio data, and audio data. If data and subtitle data are synchronized and played back, the synchronization error is corrected, an error occurs, and the functions of search operation, pause operation, and top feed operation can be realized, the usefulness as a data playback device is remarkably improved. It is thought that it can be improved.

In consideration of the above points, Patent Document 3 reproduces data in which video data, audio data, and caption data compressed at a variable rate are multiplexed with high accuracy and realizes various functions. Data reproducing devices and data recording media have been proposed.
JP 2003-274304 A JP-A-6-124168 JP 2004-362775 A

  By the way, the digital broadcast receiver described above actually generates a vertical synchronization signal for synchronizing the vertical operation of the scanning line used when displaying the video on the display unit, and synchronizes with this to generate the video. “Vpic (0)”,... Are sequentially displayed.

  Therefore, in the case shown in FIG. 2, for example, the voice “Sframe (0)” can be output at the sound output time “pts (0)” at which the voice “Sframe (0)” should be output. First, for the video “Vpic (0)” corresponding to this audio “Sframe (0)”, a vertical synchronization signal is generated at the display time “pts (0)” at which the video “Vpic (0)” should be displayed. Therefore, this video “Vpic (0)” cannot be displayed, and this video “Vpic (0)” is displayed when the subsequent vertical synchronizing signal is generated.

  As a result, in the digital broadcast receiver described above, the sound output timing of the sound “Sframe (0)”,... And the video “Vpic (0)” corresponding to the sound “Sframe (0)”,. Since there is a maximum difference of ± 1/2 frame from the video display timing, it is difficult to say that the video display timing and the sound output timing are accurately matched.

  As a technique to avoid this problem, it is also possible to adjust the vertical sync signal generated in the digital broadcast receiver to match the display time information before displaying the video “Vpic (0)”,. However, if this adjustment is made, the vertical sync signal will be disturbed and this will cause a shock to the displayed video “Vpic (0)”. It is not preferable.

  In addition, even if the video display timing and the sound output timing can be matched with high accuracy, the above-described digital broadcast receiver actually does not perform PID switching or video sequence (video sequence: number of horizontal / vertical pixels, frame rate). When the display delay time (dt) changes due to the switching of the PTS grid), the audio (Audio) skips or the high-precision video and audio (AV: Audio, Video) synchronize. Either it would remain.

  Further, in the data reproducing apparatus disclosed in Patent Document 3, as a method for avoiding this problem, before displaying the video “Vpic (0)”,... Since the vertical synchronization signal is disturbed, the video “Vpic (0)”, which is displayed by this, has a so-called shock.

  An object of the present invention is to provide a data processing apparatus and method capable of avoiding audio skipping and video output disturbance while maintaining high-precision AV synchronization even when the display delay time (dt) changes. And to provide a program.

According to a first aspect of the present invention, there is provided a data processing apparatus including a display unit for displaying a video based on video data, a counter unit for counting a reference time, and a video based on the video data when a video display timing signal is supplied. After the display time at which the video is to be displayed, a supply means for supplying the video display timing signal indicating the video display timing to the display means at a predetermined cycle; the display time at which the video based on the video data is to be displayed ; The calculation means for calculating the display delay time from the reference time until the video is displayed by the display means, and the display delay time calculated by the calculation means from the sound output time when the sound based on the audio data should be output Sound output means capable of outputting sound based on the sound data, and the sound output according to the sound output state of the sound output means. And control means for controlling the sound output timing of the stages of speech, at least, on the basis of the display delay time and the sound output time information and the reference time information, detects the synchronization deviation of the video and the audio, the detected deviation Synchronization detection correcting means for correcting the synchronization shift according to the amount .

  Preferably, the control means starts outputting the sound by synchronizing the video and the sound in a state where no sound is output from the sound output means.

  Preferably, in a state where sound is output from the sound output means, the control means outputs the sound with the display delay time immediately before the change when the display delay time by the calculation means changes. continue.

  Preferably, the control means re-synchronizes the video and the sound with the latest display delay time by the calculation means when the sound output of the sound output means is resumed after being paused.

  Preferably, there is provided synchronization adjusting means for matching the sound output timing with the video display timing based on the sound output time information included in the audio data and the display delay time by the calculating means.

Preferably, a video decoding unit that decodes video data and supplies it to the display unit, a display delay that holds display delay time information that is a calculation result of the calculation unit, and updates the hold information at a timing that satisfies a predetermined condition The synchronization detection and correction means has no synchronization deviation from the display delay time as the calculation result of the calculation means and the display delay time held in the display delay time management means. To check whether or not there is a synchronization error between the video and audio. If the video is advanced compared to the audio, the video decoding means is instructed to stop the temporary operation. When the video is delayed compared to the audio, a skip command is sent to the video decoding means so as to skip .
Preferably, the audio decoding means for decoding the audio data and supplying it to the sound output means, and the display delay time information as the calculation result of the calculating means are held, and the holding information is updated at a timing satisfying a predetermined condition. Display delay time management means, and calculation means for supplying difference information between the sound output time and the reference time to the synchronization detection correction means. The synchronization detection correction means is a calculation result of the calculation means. From the difference information and the display delay time stored in the display delay time management means, it is confirmed that there is no synchronization deviation in the audio, whether there is a synchronization deviation between the video and the audio, If it is ahead of the video, a pause command is sent to the audio decoding means to stop the temporary operation, and if the audio is behind the video, the audio decoding means is read. If then so skip to send a command.

A data processing method according to a second aspect of the present invention includes a display step of displaying a video based on video data on a display means when a video display timing signal is supplied, a counting step of counting a reference time, and the video data A supply step of supplying the video display timing signal indicating the timing for displaying the video at a predetermined cycle after the display time at which the video based on the video is to be displayed; the display time at which the video based on the video data is to be displayed; and the reference a calculation step of calculating a display delay time until the video by the display step from is displayed, when the display delay time calculated in the calculation step from the sound output time to be output the sound based on the sound data has elapsed A sound output step for outputting sound based on the sound data; and the sound output step in the sound output step In response to a force condition, a control step for controlling the sound output timing of the sound output step, at least, on the basis of the display delay time and the sound output time information and the reference time information, the video and the audio sync And a synchronization detection correcting step for correcting the synchronization shift according to the detected shift amount .

A program according to a third aspect of the present invention is based on a display process for displaying a video based on video data on a display means , a count process for counting a reference time, and the video data when a video display timing signal is supplied. From the display time at which video is to be displayed, supply processing for supplying the video display timing signal indicating the timing at which video is to be displayed at a predetermined cycle, the display time at which video based on the video data is to be displayed, and the reference time when passed a calculating process for calculating a display delay time until the video is displayed by the display processing, the display delay time calculated in the calculation processing from the sound output time to be output the sound based on the sound data, the a sound output process for outputting audio based on the audio data, in accordance with the output state of the sound in the sound output processing, sound output of the sound output processing A control process for controlling the timing, at least, on the basis of the display delay time and the sound output time information and the reference time information, detects the synchronization deviation of the video and the audio, synchronization deviation in accordance with the detected shift amount A data processing program for causing a computer to execute data processing including synchronization detection correction processing for correcting data.

  According to the present invention, even if the display delay time (dt) changes due to switching of PID or switching of video sequence (number of horizontal and vertical pixels, frame rate, PTS grid), etc., while maintaining high-precision AV synchronization There is an advantage that audio sound skipping and video output disturbance can be avoided.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

FIG. 5 is a block diagram showing a configuration example of the data processing apparatus according to the embodiment of the present invention.
FIG. 6 is a time chart showing basic video display timing and sound output timing of the data processing apparatus of FIG.
FIG. 7 is a time chart showing video display timing and sound output timing related to the update of the display delay time of the data processing apparatus of FIG.
The data processing apparatus 100 of this embodiment can be applied to, for example, a digital broadcast receiver and a stored data reproduction apparatus.

  As shown in FIG. 5, the data processing apparatus 100 includes a digital tuner 101, a storage medium 102, a stream buffer 103, a separation processing unit (DEMUX processing unit) 104, a source oscillation clock generation unit 109, a reference time (STC (System Time Clock) )) Counter unit 110, video ES buffer 111, caption ES buffer 112, audio ES buffer 113, display time information extraction unit 114, video decoder unit 115, character output time information extraction unit 116, caption decoder unit 117, sound output time information Extraction unit 118, audio decoder unit 119, first comparison circuit 120, second comparison circuit 121, third comparison circuit 122, horizontal / vertical synchronizing signal generation circuit 123, first latch circuit 124, first arithmetic circuit 125, adder 126 , Synchronization signal generation circuit 127, display delay time management unit 128, display processing unit 129, display unit 1 30, a sound output processing unit 131, a sound output unit 132, a second latch circuit 133, a second arithmetic circuit 134, a third arithmetic circuit 135, a synchronization shift detection unit 136, and a synchronization shift correction unit 137.

The DEMUX processing unit 104 according to the present embodiment includes a NIT acquisition unit 105, a PAT acquisition unit 106, a PMT acquisition unit 107, and a filter unit 108.
The filter unit 108 includes a PCR (Program Clock Reference) filter 108A, a video filter 108B, a caption filter 108C, and an audio filter 108D.

  In the data processing device 100, the digital tuner 101 supplies an input stream obtained from a television broadcast wave received via an antenna or the like to the NIT acquisition unit 105 in the DEMUX processing unit 104.

The NIT acquisition unit 105 acquires a network information table (NIT) from the input stream supplied from the digital tuner 101.
Accordingly, the digital tuner 101 can select a transponder corresponding to the requested channel requested by the user's channel switching operation based on the acquired NIT, and as a result, receives the transport stream corresponding to the requested channel. can do.

Thereafter, the PAT acquisition unit 106 provided in the DEMUX processing unit 104 acquires a PAT (Program Association Table).
Next, a PMT acquisition unit 107 provided in the DEMUX processing unit 104, based on the PAT acquired by the PAT acquisition unit 106, a PMT (Program Map Table) corresponding to the requested channel requested by the user channel switching operation. To get.

  Then, the PMT acquisition unit 107 recognizes a PID (Program Identification) corresponding to the requested channel based on the acquired PMT, and recognizes the recognized PID in the PCR filter 108A, the video filter 108B, the subtitle filter 108C in the filter unit 108, and Set for each of the audio filters 108D.

  Further, from the storage medium (such as DISK) 102, the stored data is put into the stream buffer 103, and the stream in the stream buffer 103 is supplied to the DEMUX processing unit 104.

The PCR filter 108A, the video filter 108B, the subtitle filter 108C, and the audio filter 108D in the filter unit 108 of the DEMUX processing unit 104 correspond to the request channel from the received transport stream based on the PID set for each. PCR, video packet, audio packet, and subtitle packet including PCR data, compressed video data, subtitle data, and compressed audio data can be extracted.
Each extracted data is supplied to the STC counter unit 110, the video ES buffer 111, the caption ES buffer 112, and the audio ES buffer 113, respectively.

The source oscillation clock generation unit 109 performs negative feedback control such that the signal output from the STC counter unit 110 is phase-synchronized with the PCR data in the PCR packet extracted by the PCR filter 108A as a reference signal.
Thereby, the source oscillation clock generation unit 109 can generate the operation clock signal S101 synchronized with the reference clock signal on the encoder device side, and the obtained operation clock signal S101 is converted into the horizontal / vertical synchronization signal generation circuit 123, the synchronization signal. This is supplied to the generation circuit 127 and the STC counter unit 110.

The horizontal / vertical synchronization signal generation circuit 123 is configured to synchronize with the vertical synchronization signal S102 for synchronizing the vertical operation of the scanning line based on the operation clock signal S101 supplied from the source oscillation clock generation unit 109, A horizontal synchronization signal S103 for synchronizing the operation is generated.
The horizontal / vertical synchronization signal generation circuit 123 supplies the generated vertical synchronization signal S102 to the first latch circuit 124 and the display processing unit 129.
The horizontal / vertical synchronization signal generation circuit 123 supplies the operation clock signal S101 supplied from the source oscillation clock generation unit 109 and the generated vertical synchronization signal S102 and horizontal synchronization signal S103 to the display processing unit 129. .
That is, the horizontal / vertical synchronizing signal generation circuit 123 functions as a part of the supply unit.

Thereby, the display processing unit 129 is supplied from the digital video signal S104 supplied from the video decoder unit 115 and the subtitle decoder unit 117 in accordance with the supplied operation clock signal S101, vertical synchronization signal S102, and horizontal synchronization signal S103. The subtitle signal S110 is supplied to the display unit 130 at the subsequent stage.
As a result, a video based on the digital video signal S104 and the caption signal S110 is displayed on the display unit 130.

The synchronization signal generation circuit 127 generates an audio frame output synchronization signal S105 based on the operation clock signal S101 supplied from the source oscillation clock generation unit 109.
The synchronization signal generation circuit 127 supplies the generated synchronization signal S105 and the operation clock signal S101 supplied from the source oscillation clock generation unit 109 to the sound output processing unit 131.
In this way, the sound output processing unit 131 converts the digital audio signal S106 supplied from the audio decoder unit 119 into the output sound of the subsequent stage according to the supplied operation clock signal S101 and audio frame output synchronization signal S105. Supplied to the unit 132.
As a result, sound based on the digital sound signal S106 is output via the sound output unit 132.
The synchronization signal generation circuit 127 supplies the generated audio frame output synchronization signal S105 to the second latch circuit 133.

On the other hand, the STC counter unit 110 measures a reference time (STC (System Time Clock)) in the streams from the digital tuner 101 and the storage medium 102 based on the operation clock signal S101 from the source oscillation clock generation unit 109.
When the PCR packet is supplied from the PCR filter 108A, the STC counter unit 110 adjusts the time reference time to the time indicated in the PCR data in the supplied PCR packet.
Thereby, the reference time measured in the streams from the digital tuner 101 and the storage medium 102 can be synchronized with the time on the encoding apparatus side.
Then, the STC counter unit 110 converts the reference time information I101 indicating the reference time being measured into the source oscillation clock generation unit 109, the first latch circuit 124, the second latch circuit 133, the first comparison circuit 120, the second This is supplied to the comparison circuit 121 and the third comparison circuit 122.

  The display time information extraction unit 114 extracts display time information (PTS (Presentation Time Stamp)) I102 associated with the compressed video data for each frame from the video ES buffer 111, and extracts this from the first comparison circuit 120. , To the first arithmetic circuit 125 and the second arithmetic circuit 134.

The first comparison circuit 120 compares the reference time information I101 sequentially supplied from the STC counter unit 110 with the display time information I102 supplied from the display time information extraction unit 114.
Then, as shown in FIG. 6, for example, the first comparison circuit 120 decodes when the reference time indicated in the reference time information I101 coincides with, for example, the display time “PTSV (0)” indicated in the display time information I102. A decode processing start command signal S107 for starting the processing is supplied to the video decoder unit 115.

  When the decoding process start command signal S107 is supplied from the first comparison circuit 120, the video decoder 115 decodes the corresponding compressed video data “Dpic (0)” for one frame obtained from the video ES buffer 111. As a result, a digital video signal S104 is generated and supplied to the display processing unit 129.

In this case, however, the display processing unit 129 does not receive the vertical synchronization signal S102 from the horizontal / vertical synchronization signal generation circuit 123 at the reference time “PTSV (0)” shown in FIG. At the time “vt (n)” when the video signal is supplied, the digital video signal S104 for one frame supplied from the video decoder unit 115 is supplied to the display unit 130.
Accordingly, the timing for displaying the video “Vpic (0)” based on the digital video signal S104 is a predetermined time dt () than the display time “PTSV (0)” at which the video “Vpic (0)” is to be displayed. 0) (hereinafter, this predetermined time is referred to as a display delay time).
When the video frame rate or the number of horizontal and vertical pixels changes, the period setting of the vertical synchronization signal may be changed as necessary. Since the video display delay time changes with the occurrence of such an event, the video display delay time is hereinafter referred to as dt [latest value].
On the other hand, the display delay time used for adjusting the audio output delay time is called dt.

By the way, the first latch circuit 124 of the present embodiment, for example, at the timing T1 when the vertical synchronization signal S102 is supplied from the horizontal / vertical synchronization signal generation circuit 123 in advance, the reference time “vt ( n-1) "is latched.
Thus, the reference time information I101 indicating the reference time “vt (n−1)” is supplied from the first latch circuit 124 to the arithmetic circuit 125.

  When the display time information I102 indicating the display time “PTSV (0)” is supplied from the display time information extraction unit 114, the first arithmetic circuit 125 calculates the display delay time dt by calculating the following equation. Calculate [latest value].

(Equation 1)
dt [latest value] = Vt-(PTSV-STC [vt]) mod Vt ... Formula (1)
The formula (1) will be described in detail. After obtaining the display time “PTSV (0)” at which the video “Vpic (0)” based on the compressed video data “Dpic (0)” should be displayed, 11, after subtracting the STC time “STC [vt]” when the vertical synchronization signal S102 is supplied thereafter, the result is divided by the cycle time Vt of the vertical synchronization signal S102. . Then, the display delay time dt [latest value] is calculated by subtracting the remainder resulting from the division from the cycle time Vt of the vertical synchronization signal S102.

The first arithmetic circuit 125 supplies the display delay time information I103 indicating the display delay time dt [latest value] calculated in this way to the display delay time management unit 128.
The display delay time management unit 128 holds the supplied display delay time dt [latest value].
The first arithmetic circuit 125, the display delay time management unit 128, and the like function as a part of calculation means.

  The display delay time management unit 128 updates the display delay time dt at a timing that satisfies the condition of step SP301 or SP303, as shown in the flowchart of FIG. Information necessary for the condition determination of SP301 or SP303 is given by the audio synchronization deviation detection information I108 from the audio decoder unit 119.

  The display delay time management unit 128 supplies information I105 indicating the display delay time dt to the adder 126.

  On the other hand, the sound output time information extraction unit 118 extracts sound output time information (PTS) I104 associated with the compressed sound data for each frame from the sound ES buffer 113, and this is extracted into the third arithmetic circuit. 135 is supplied to the adder 126.

  The adder 126 displays the display delay time supplied from the display delay time management unit 128 with respect to, for example, the sound output time “PTSA (0)” indicated by the sound output time information I104 supplied from the sound output time information extraction unit 118. The display delay time dt indicated by the information I105 is added, and the corrected sound output time information I106 indicating the addition result “vt (n)” is supplied to the second comparison circuit 121.

The second comparison circuit 121 compares the reference time information I101 sequentially supplied from the STC counter unit 110 with the modified sound output time information I106 supplied from the adder 126.
The second comparison circuit 121 starts decoding processing when the reference time indicated in the reference time information I101 matches the sound output time “vt (n)” indicated in the modified sound output time information I106. The decoding process start command signal S108 is supplied to the audio decoder unit 119.
In this embodiment, the adder 126, the second comparison circuit 121, and the like constitute a synchronization adjustment unit.

When the decoding processing start command signal S108 is supplied from the second comparison circuit 121, the audio decoder unit 119 receives the corresponding compressed audio data “Dframe (0) for one frame obtained from the audio ES buffer 113 in response thereto. ”Is generated, and a digital audio signal S106 is generated and supplied to the sound output unit 132 via the sound output processing unit 131.
As a result, the audio “Sframe (0)” based on the compressed audio data “Dframe (0)” is output at the time “vt (n)” delayed by the display delay time dt from the time “PTSA (0)”.

  The character output time information extraction unit 116 extracts character output time information (PTS (Presentation Time Stamp)) I107 associated with the compressed video data from the caption ES buffer 112, and this is extracted from the third comparison circuit 122 and the third comparison circuit 122. 1 is supplied to the arithmetic circuit 125.

The third comparison circuit 122 compares the reference time information I101 sequentially supplied from the STC counter unit 110 with the character output time information I107 supplied from the character output time information extraction unit 116.
Then, the third comparison circuit 122, when the reference time indicated in the reference time information I101 coincides with the character output time indicated in the character output time information I107, decode processing start instruction signal S109 for starting the decoding process. Are supplied to the caption decoder unit 117.

  When the decoding process start command signal S109 is supplied from the third comparison circuit 122, the caption decoder unit 117 generates a caption signal S110 by decoding the compressed caption data obtained from the caption ES buffer 113, and generates the caption signal S110. Supply to the display processing unit 129.

  In this way, the data processing apparatus 100 shown in this example, as shown in FIG. 6, the video display timing at which the video “Vpic (0)” is displayed and the audio corresponding to the video “Vpic (0)”. The sound output timing at which “Sframe (0)” is output can be matched with high accuracy.

Thereafter, in this example, the data processing apparatus 100 also performs the same operation from the display delay time management unit 128 to the sound output time indicated by the sound output time information I104 supplied from the sound output time information extraction unit 118. The display delay time dt indicated by the supplied display delay time information I105 is continuously added.
As a result, the video display timing at which the video “Vpic (1)”,... Is displayed and the sound output timing at which the corresponding audio “Sframe (1)”,. The accuracy can be matched.

<Comparative Example 1>
On the other hand, for example, in the stored data reproducing apparatus disclosed in Patent Document 3, as shown in FIG. 8, before the video “Vpic (0)”,. Was adjusted to match the display time information, the vertical sync signal was disturbed, and the video “Vpic (0)”,.
Specifically, the playback apparatus of the stored data at a timing T2 in FIG. 8, modulate to match the vertical synchronizing signal being generated in the display time information PTSV (0). Therefore, the vertical synchronization signal (VSYNC) is disturbed, and a shock is applied to the video Vpic (0) displayed immediately after that.

<Comparative example 2>
Also, in a digital broadcast receiver that does not employ the configuration according to the embodiment of the present invention, high-accuracy AV synchronization is accompanied by PID switching, video sequence (horizontal and vertical pixel count, frame rate, PTS grid) switching, etc. If the delay value (dt) for guaranteeing changes, the audio skips or the high-precision AV synchronization cannot be maintained.
Specifically, as shown in FIG. 9, the display time delay changes as shown by the display time delays dt (0) and dt (1) in the figure by switching the PID or the video sequence. In that case, if dt (0) <dt (1), the output time of the next Sframe (1) is PTSA (1) + dt (1) even if the output of Sframe (0) ends. The output time is not yet reached. Therefore, sound skip occurs.
Further, if dt (0)> dt (1), the output time of Sframe (1) will be reached before the output of Sframe (0) is completed, resulting in a synchronization shift.

The data processing apparatus 100 according to the embodiment of the present invention is configured in consideration of the above points. As shown in FIG. 6 described above, PID switching and video sequence (number of horizontal and vertical pixels, frame rate, PTS) Even if the delay value (dt) for ensuring high-accuracy AV synchronization changes due to switching of the grid), audio skipping and disturbance of video output can be avoided while maintaining high-accuracy AV synchronization.
Specifically, as shown in FIGS. 6 and 7, even when the latest value of the display delay time is changed from dt (0) to dt (1), even when Sframe (1) is output. By making the display delay time dt to be the same as dt at the time of Sframe (0), audio skipping and video output disturbance can be avoided.

  Hereinafter, the state transition, the display delay time dt update process, and the synchronization shift detection process in the data processing apparatus 100 according to the present embodiment will be described in order with reference to the flowcharts of FIGS.

First, state transition will be described.
FIG. 10 is a flowchart for explaining state transition of the data processing apparatus according to the present embodiment.

  In the data processing apparatus 100, the initial setting state is set in step SP101 and the stop state is set in step SP102, and in step ST103, it is determined whether it is broadcast reception viewing or storage media playback.

For example, in the case of broadcast reception viewing, a viewing preparation state is set as in step SP104.
Then, NIT is acquired in step SP105 by broadcast reception, PAT is acquired in step SP106, PMT is acquired in step SP110, and each PID is acquired.
Next, in step SP115, using the PID recognized based on this PMT, the PCR packet, the video packet, and the audio packet are separated from the transport stream.
In step SP116, decoding of each separated ES is started. Next, in step SP117, a reference clock (STC) is set based on the reference time measured by the STC counter unit 110 based on the synchronization reference selected in step SP111 or step SP113.

  In the subsequent step SP118, the reference time information I101 ("vt (n-1)") supplied from the STC counter unit 110 is supplied to the first latch circuit at the timing T1 shown in FIG. 6 when the vertical synchronization signal S102 is supplied. 124 is latched.

  Then, the process proceeds to step SP119, for example, whether or not the display time information I102 (“PTSV (0)”) of the compressed video data “Dpic (0)” to be decoded first can be extracted within a predetermined time by the display time information extraction unit 114. Determine whether.

  If a positive result is obtained in step SP119, this means that the display time information I102 (“PTSV (0)”) corresponding to the compressed video data “Dpic (0)” has been extracted. Proceed to the next Step SP121.

  In step SP121, the above expression is used by using the extracted display time information I102 (“PTSV (0)”) and the reference time information I101 (“vt (n−1)”) latched by the first latch circuit 124. By causing the first arithmetic circuit 125 to calculate (1), the display delay time dt [latest value] is calculated.

Next, the process proceeds to step SP122, and the display delay time dt [latest value] of the video “Vpic (0)” to be displayed can be obtained in advance.
Accordingly, in step SP124, the audio “Sframe (0)” corresponding to the video “Vpic (0)” can be output with a delay of the display delay time dt.
In addition, display of video and subtitles is started when VSync after the display time coincides with STC is supplied in steps SP123 and SP125.

If a negative result is obtained in step SP119, this is because the display time information I102 (“PTSV (0)”) corresponding to the compressed video data “Dpic (0)” is missing. This means that the display time information I102 (“PTSV (0)”) could not be extracted. At this time, the process proceeds to step SP120 instead of step SP121 for calculating the display delay time dt [latest value].
In step SP120, the value of the display delay time dt [latest value] is set to, for example, “0”, and the process proceeds to subsequent steps SP123, SP124, and SP125.

In step SP126, when the viewing / playback steady state is reached, synchronization deviation is detected and corrected.
Details will be described later in association with the flowcharts of FIGS.
Further, for each display time information (PTSV), a display delay time dt [latest value] is calculated as described later in association with FIG. Furthermore, as will be described with reference to FIG. 12, the display delay time dt is updated as needed.

  In the case of stored media playback, for example, in response to a user input or an instruction from an external interface, the playback steady state transitions to the search state in step SP127, the pause state in step SP128, and the frame advance state in step SP129.

  In the search state in step SP127, only the I picture data that appears at regular intervals in the video data is reproduced, and the P picture or B picture between the I picture and the I picture is skipped without being reproduced, thereby storing the storage medium. This is an operation of reproducing the video data recorded in 102 in a time shorter than the time required for normal reproduction.

In step SP128, when the state is shifted to the pause state, the audio decoder unit pauses the decoding.
In addition, the video decoder unit 115 and the caption decoder unit 117 wait for the vertical synchronization signal S102 from the horizontal / vertical synchronization signal generation circuit 123, and after detecting the vertical synchronization signal, the video decoder unit 115 and the caption decoder unit 117 temporarily stop decoding. Continue to display the last decoded screen.
The STC counter unit 110 stops the STC automatic counter up.

In step SP129, when the frame advance state is entered, the video decoder unit 115 performs decoding for one frame.
That is, it waits for the vertical synchronizing signal S102 from the horizontal / vertical synchronizing signal generation circuit 123, and the video decoder unit 115 performs decoding for one frame by the next vertical synchronizing signal S102, and the decoding is temporarily performed by the next vertical synchronizing signal S102. Stop.
Next, the STC is advanced by one frame. That is, the display time of one frame is added to the STC of the STC counter unit 110, and the value is set in the STC counter unit 110 again. The above processing is repeated until the frame advance is canceled.

  When returning to the playback state or stopping from any of the search state at step SP127, the pause state at step SP128, and the frame advance state at step 129, display of the video currently displayed at step SP130 The display time information is already extracted with the time as the current time.

The data processing apparatus 100 according to the present embodiment is configured to include the following control methods in the synchronization error detection method and the synchronization error correction method.
When audio is not being output, high-precision AV synchronization is taken and audio output is started.
Even if the display delay time (dt [latest value]) changes due to switching of VideoSequence (frame rate, number of horizontal and vertical pixels) while audio is output, the audio output is the display delay time (dt) before the change. continue.
When audio resumes output after audio is paused due to underflow, PTS gap, playback mode transition, etc., resynchronization is performed with the latest display delay time (dt [latest value]).
This ensures high accuracy even when the display delay time (dt) changes due to switching of PID or video sequence (horizontal / vertical pixel count, frame rate, PTS grid) while performing AV synchronization with high accuracy. While maintaining AV synchronization, audio skipping and video output disturbance can be avoided.

Next, the display delay time dt [latest value] calculation process will be described.
FIG. 11 is a flowchart for explaining the calculation of the display delay time dt [latest value] of the data processing apparatus 100 according to the present embodiment.

  First, in step SP201, the video decoder unit 115 determines whether or not the video frame rate or the number of horizontal and vertical pixels has changed. If there is a change, the process proceeds to step SP202.

  In step SP202, the period of the vertical synchronization signal is set as necessary. That is, the horizontal / vertical synchronization signal generating circuit 123 changes the cycle setting according to the vertical synchronization signal setting information I109 from the video decoder 115. Thereafter, the process proceeds to step SP203.

  In step SP203, the cycle time Vt of the vertical synchronizing signal is calculated.

  In step SP204, the display time information I102 indicating the display time PTSV is sent from the display time information extraction unit 114 to the arithmetic circuit 125.

  On the other hand, the first latch circuit 124 has reference time information I101 (STC [vt] indicating the reference time supplied from the STC counter unit 110 at the timing when the vertical synchronization signal S102 is supplied from the horizontal / vertical synchronization signal generation circuit 123 in advance. ]) Is latched. As a result, the reference time information I101 indicating the reference time is supplied from the first latch circuit 124 to the arithmetic circuit 125 (steps SP205 and SP206).

  Since information necessary for calculating dt [latest value] has been prepared, dt [latest value] is calculated by performing calculation of equation (1) in step SP207.

  In step SP208, the information I103 of dt [latest value] is sent from the arithmetic circuit 125, and the display delay time management unit 126 stores dt [latest value].

Next, an audio output delay time dt update process will be described.
FIG. 12 is a flowchart for explaining an audio output delay time dt update process of the data processing apparatus according to the present embodiment.

If the determination result in step SP301 or step SP303 is affirmative, the output delay time dt is updated to dt [latest value] in step SP302 or step SP304.
Information necessary for the condition determination in step SP301 or step SP303 is given by the audio synchronization deviation detection information I108 from the audio decoder unit 119.
If the determination result is negative, the output delay time dt does not change.
In addition, the time chart of FIG. 7 shows a situation in which the output delay time dt is updated when an audio gap or silence occurs as the condition of step SP303.

Next, detection and correction of synchronization deviation between video and audio will be described.
When both the video decoder unit 115 and the audio decoder unit 119 are in a decoding state, a means for detecting and correcting a synchronization shift between the display image and the output audio called lip sync is required.

First, detection and correction of video synchronization deviation will be described.
FIG. 13 is a flowchart for explaining detection / correction of video synchronization deviation in the data processing apparatus according to the present embodiment.

In step SP401, the display time information extraction unit 114 determines whether or not the PTSV output by the next vertical synchronization signal S102 has been obtained.
If the result is positive, the process proceeds to step SP402.

  In step SP402, the arithmetic circuit 125 obtains a new PTSV from the display time information extraction unit 114.

  The first latch circuit 124 latches the reference time information I101 (STC [vt]) indicating the reference time supplied from the STC counter unit 110 at the timing when the vertical synchronization signal S102 is supplied (steps SP403 and SP404). .

Moving to step SP405, the first comparison circuit 120 calculates PTSV-STC [vt] and sends the result to the synchronization shift detection unit 136. Further, the arithmetic circuit 125 sends the calculated dt [latest value] to the synchronization deviation detection unit 136.

In step SP406, the synchronization deviation detection unit 136 confirms that the sum of the calculation result of PTSV-STC [vt] and the display delay time dt [latest value] becomes zero. That is, it is confirmed that there is no video synchronization error.
If the result is affirmative, the process proceeds to step SP408. If the result is negative, the process proceeds to step SP407.

  In step SP408 and step SP409, the deviation detection method and the deviation correction method are confirmed. If at least one of the results is negative, no deviation detection or deviation correction is performed. If both are positive, the process proceeds to step SP410.

  In step SP410, the synchronization error detection unit 136 checks whether or not there is a synchronization error between Video and Audio. If there is no gap, do nothing. If a synchronization error has occurred, a synchronization error detection signal S111 is sent to the synchronization error correction unit 137.

In step SP411, the synchronization error correction unit 137 calculates the amount of synchronization error.
When (PTSV-STC [vt]) − (PTSA-STC [at]) is larger than (dt−dt [latest value]), video is advanced as compared with audio, so a pause command is issued to the video decoder unit. 115 (step SP413).
Further, when (PTSV-STC [vt]) − (PTSA-STC [at]) is smaller than (dt−dt [latest value]), the video is delayed compared to Auido, and therefore the video decoder unit 115. Is sent a skip command (step SP414).

If the determination result in step SP406 is negative, the absolute value of the difference value between PTSV-STC [vt] and dt [latest value] is compared with the constant value Tn (step SP407).
If the absolute value of the difference value is less than Tn, the deviation is examined (step SP412), and then the deviation is corrected (steps SP413 and SP414).

  If the determination result in step SP407 is negative, that is, if the absolute value of the difference value is greater than or equal to a certain value Tn, it is determined that a serious synchronization error has occurred, the buffer is cleared, and a warning is displayed. Transition to the state (steps SP415 and SP416).

Next, detection and correction of audio synchronization deviation will be described.
FIG. 14 is a flowchart for explaining detection / correction of audio synchronization loss in the data processing apparatus according to the present embodiment.

  In step SP501, it is determined whether or not the sound output time information extraction unit 118 has obtained the PTSA output by the next audio frame output synchronization signal S105. If the result is affirmative, the process proceeds to step SP502.

  In step SP502, the third arithmetic circuit 135 obtains a new PTSA from the sound output time information extraction unit 118.

  The second latch circuit 133 latches the reference time information I101 (STC [at]) indicating the reference time supplied from the STC counter unit 110 at the timing when the audio frame output synchronization signal S105 is supplied from the synchronization signal generation circuit 127. (Steps SP403 and SP404).

  In step SP505, the third arithmetic circuit 135 calculates PTSA-STC [at] and sends the result to the synchronization shift detection unit 136. Further, the display delay time management unit 128 sends dt to the synchronization shift detection unit 136.

In step SP506, the synchronization shift detection unit 136 confirms that the sum of the calculation result of PTSA-STC [at] and the display delay time dt is zero. That is, it is confirmed that there is no audio synchronization loss.
If the result is affirmative, the process proceeds to step SP508. If the result is negative, the process proceeds to step SP507.

  In step SP508 and step SP509, the deviation detection method and the deviation correction method are confirmed. If at least one of the results is negative, no deviation detection or deviation correction is performed. If both are positive, the process proceeds to step SP510.

  In step SP510, the synchronization error detection unit 136 checks whether or not there is a synchronization error between Video and Audio. If there is no gap, do nothing. If a synchronization error has occurred, a synchronization error detection signal S111 is sent to the synchronization error correction unit 137.

In step SP511, the synchronization error correction unit 137 calculates the amount of synchronization error.
When (PTSA-STC [at])-(PTSV-STC [vt]) is larger than (dt [latest value] dt), the audio is advanced as compared with the video, so the pause command is sent to the audio decoder unit 119. (Step SP513).
If (PTSA-STC [at])-(PTSV-STC [vt]) is smaller than (dt [latest value] dt), the audio is delayed compared to the video, and therefore the audio decoder unit 119 A skip reading command is sent (step SP514).

If the determination result in step SP506 is negative, the absolute value of the difference value between PTSA-STC [at] and dt is compared with a constant value Tn (step SP507).
If the absolute value of the difference value is less than Tn, the deviation is checked (step SP512), and then the deviation is corrected (steps SP513 and SP514).

  If the determination result in step SP507 is negative, that is, if the absolute value of the difference value is greater than or equal to a certain value Tn, it is determined that a serious synchronization error has occurred, the buffer is cleared, a warning is displayed, and then the operation is stopped. Transition to the state (steps SP515 and SP516).

As described above, in the data processing apparatus 100 according to the present embodiment, in the state where no sound is output from the sound output processing unit 131 and the sound output unit 132, the output of the sound is performed by synchronizing the video and the sound. When the display delay time by the arithmetic circuit 125 changes while the sound is output from the sound output processing unit 131 and the sound output unit 132, the sound output is performed with the display delay time immediately before the change. When the audio output is resumed after the audio output is temporarily stopped, since the video and audio are synchronized again with the latest display delay time by the arithmetic circuit 125, the following effects can be obtained. .

Even if the display delay time (dt) changes, it is possible to avoid audio skipping and disturbance of video output while maintaining high-precision AV synchronization.
Further, for example, the present invention can be used for a decoding apparatus that outputs video and audio by decoding compressed video data and compressed audio data encoded based on the MPEG method or the like.
The decoding device here includes a broadcast receiving / viewing device, a reproducing device for storage media (DVD, Blu-ray Disk, HDD, etc.), etc., and the present invention can also be used for these.
And the following effects are acquired in these apparatuses.

1) High accuracy AV synchronization can be guaranteed.
2) Even if the display delay time (dt) changes due to switching of PID or switching of video sequence (number of horizontal / vertical pixels, frame rate, PTS grid), high-accuracy AV synchronization can be maintained.
3) Even if the display delay time (dt) changes due to PID switching or video sequence (horizontal / vertical pixel count, frame rate, PTS grid) switching, etc., audio sound is maintained while maintaining high-precision AV synchronization. It is possible to avoid flying and disturbance of video output.
4) When the display unit 130 and the sound output unit 132 are connected to a recorder or encoder (for example, in the case of a system for recompression / encoding such as an HDD recorder, a DVD recorder, etc.) Disappears.

It is a flowchart which shows the general honor audio | voice output processing procedure of a digital broadcast receiver. It is a time chart which shows video display timing and sound output timing. It is a figure which shows the structural example of a data reproduction apparatus. FIG. 4 is a diagram showing a state of track jumping in the data reproducing apparatus in FIG. 3. It is a block diagram which shows the structural example of the data processor which concerns on embodiment of this invention. 6 is a time chart showing basic video display timing and sound output timing of the data processing apparatus of FIG. 5. 6 is a time chart showing video display timing and sound output timing related to update of display delay time of the data processing device of FIG. 5. It is a figure shown as a comparative example with respect to this embodiment, Comprising: It is a time chart which shows the video display timing and sound output timing of the reproducing | regenerating apparatus of the accumulation data disclosed by patent document 3. FIG. It is a figure shown as a comparative example with respect to this embodiment, Comprising: It is a time chart which shows the video display timing and sound output timing of a digital broadcast receiver when not having the structure of this embodiment. It is a flowchart for demonstrating the state transition of the data processor which concerns on this embodiment. It is a flowchart for demonstrating calculation of the display delay time dt [latest value] of the data processor 100 concerning this embodiment. It is a flowchart for demonstrating the update process of the display delay time dt of the data processor which concerns on this embodiment. It is a flowchart for demonstrating detection and correction | amendment of the synchronization loss of the video (Video) in the data processor which concerns on this embodiment. 6 is a flowchart for explaining detection and correction of audio synchronization loss in the data processing apparatus according to the embodiment.

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 ... Data processing apparatus, 101 ... Digital tuner, 102 ... Storage medium, 103 ... Stream buffer, 104 ... Separation processing part (DEMUX processing part), 105 ... NIT acquisition part, 106 ... PAT acquisition unit, 107 ... PMT acquisition unit, 108 ... filter unit, 108A ... PCR filter, 108B ... video filter, 108C ... subtitle filter, 108D ... audio filter 10 9 ... Source oscillation clock generator, 110 ... STC counter, 111 ... Video ES buffer, 112 ... Subtitle ES buffer, 113 ... Audio ES buffer, 114 ... Display time Information extraction unit, 115 ... Video decoder unit, 116 ... Output time information extraction unit, 117 ... Subtitle decoder unit, 118 ... During sound output Information extraction unit, 119: Audio decoder unit, 120: First comparison circuit, 121: Second comparison circuit, 122: Third comparison circuit, 123: Horizontal / vertical synchronization signal generation circuit, 124 ... first latch circuit, 125 ... first arithmetic circuit, 126 ... adder, 127 ... synchronization signal generation circuit, 128 ... display delay time management unit, 129 ... display processing , 130 ... display unit, 131 ... sound output processing unit, 132 ... sound output unit, 133 ... second latch circuit, 134 ... second arithmetic circuit, 135 ... third Arithmetic circuit, 136... Synchronization deviation detection unit, 137... Synchronization deviation correction unit.

Claims (13)

Display means for displaying video based on video data when a video display timing signal is supplied;
Counter means for counting the reference time;
Supply means for supplying the video display timing signal indicating the timing for displaying video after a display time at which video based on the video data is to be displayed to the display means at a predetermined period;
Calculating means for calculating a display delay time from the display time when the video based on the video data is to be displayed and the reference time until the video is displayed by the display means;
Sound output means capable of outputting sound based on the sound data when the display delay time calculated by the calculation means has elapsed from the sound output time at which sound based on the sound data should be output;
Control means for controlling the sound output timing of the sound of the sound output means according to the output state of the sound in the sound output means ;
Synchronization detection correction means for detecting a synchronization shift between the video and the audio based on at least the display delay time, the sound output time information, and the reference time information, and correcting the synchronization shift according to the detected shift amount; A data processing apparatus. Video decoding means for decoding video data and supplying it to the display means;
Holding display delay time information as a calculation result of the calculation means, and having display delay time management means for updating the hold information at a timing satisfying a predetermined condition,
The synchronization detection correction means is
Confirm that there is no synchronization deviation from the video from the display delay time calculated by the calculation means and the display delay time held in the display delay time management means, and whether there is a synchronization deviation between the video and audio. Check whether or not
If the video is ahead of the audio, send a pause command to the video decoding means to stop the temporary operation,
2. The data processing apparatus according to claim 1 , wherein when the video is delayed compared to the audio, a skip instruction is transmitted so as to skip the video decoding means . Audio decoding means for decoding audio data and supplying it to the sound output means;
Display delay time management means for holding display delay time information which is a calculation result of the calculation means, and updating the hold information at a timing satisfying a predetermined condition;
Calculating means for supplying difference information between the sound output time and the reference time to the synchronization detection correcting means,
The synchronization detection correction means is
Confirm that there is no synchronization loss between the audio and the video based on the difference information as the calculation result of the calculation means and the display delay time held in the display delay time management means. Check
If the audio is ahead of the video, send a pause command to the audio decoding means to stop the temporary operation,
3. A data processing apparatus according to claim 1 , wherein when the audio is delayed from the video, a skip instruction is transmitted so as to skip the audio decoding means . 2. Synchronous adjustment means for adjusting timing so that the sound output timing and the video display timing coincide with each other based on sound output time information included in the audio data and the display delay time by the calculating means. 4. The data processing device according to any one of 3. The control means includes
The above in the state where the sound output sound from the means is not output, the data processing apparatus as claimed in any one of 4 to take the synchronization with the video and the audio to start output of the audio. The control means includes
In a state where the sound from the sound output means is output, if the display delay time by the calculation means is changed, either with a display delay time of the change immediately before the claims 1 to continue the output of the sound of 5 the data processing apparatus according to one or. The control means includes
The when the audio output of the sound output means outputs resumed after pausing, as claimed in any one of claims 1 to resynchronize with the video and audio in the latest of the display delay time 6 by the calculating means Data processing device. A display step of displaying a video based on the video data on the display means when the video display timing signal is supplied;
A counting step for counting a reference time;
A supply step of supplying the video display timing signal indicating a timing for displaying the video at a predetermined cycle after a display time when the video based on the video data is to be displayed;
A calculation step of calculating a display delay time from the display time when the video based on the video data is to be displayed and the reference time until the video is displayed by the display step;
A sound output step for outputting sound based on the sound data when the display delay time calculated in the calculation step has elapsed from the sound output time at which sound based on the sound data is to be output;
A control step for controlling the sound output timing of the sound output step in accordance with the output state of the sound in the sound output step ;
A synchronization detection correcting step for detecting a synchronization shift between the video and the audio based on at least the display delay time, the sound output time information, and the reference time information, and correcting the synchronization shift according to the detected shift amount; A data processing method. A video decoding step of decoding video data by the video decoding means and supplying the decoded video data to the display means;
A display delay time holding step of holding display delay time information as a calculation result of the calculation step and updating the hold information at a timing satisfying a predetermined condition;
In the synchronization detection correction step,
Confirm that there is no synchronization error between the video and the audio by confirming that there is no synchronization error from the display delay time calculated in the calculation step and the display delay time held in the display delay time step. Check
If the video is ahead of the audio, send a pause command to the video decoding means to stop the temporary operation,
9. The data processing method according to claim 8 , wherein when the video is delayed compared to the audio, a skip instruction is transmitted to the video decoding means so as to skip the video . A sound decoding step of decoding sound data by the sound decoding means and supplying the sound data to the sound output means;
A display delay time step that holds display delay time information that is a calculation result of the calculation step and updates the hold information at a timing that satisfies a predetermined condition;
A calculation step of calculating difference information between the sound output time and the reference time,
The synchronization detection correction step is
Check that there is no synchronization error in the audio from the difference information, which is the calculation result of the calculation step, and the display delay time held in the display delay time step. Check
If the audio is ahead of the video, send a pause command to the audio decoding means to stop the temporary operation,
The data processing method according to claim 8 or 9 , wherein when audio is delayed compared to video, a skip instruction is transmitted so as to skip to the audio decoding means . Display processing for displaying video based on video data on the display means when the video display timing signal is supplied;
A counting process for counting the reference time;
A supply process for supplying the video display timing signal indicating a timing for displaying the video at a predetermined cycle after a display time when the video based on the video data is to be displayed;
A calculating process for calculating a display delay time from the display time and the reference time to be displayed an image based on the video data until the image is displayed by the display process,
A sound output process for outputting sound based on the sound data when the display delay time calculated in the calculation process has elapsed from the sound output time at which sound based on the sound data should be output;
Depending on the output state of the sound in the sound output process, a control process for controlling the sound output timing of the sound output processing,
Synchronization detection correction processing for detecting a synchronization shift between the video and the audio based on at least the display delay time, the sound output time information, and the reference time information, and correcting the synchronization shift according to the detected shift amount;
A data processing program for causing a computer to execute data processing including: A video decoding process for decoding video data by the video decoding means and supplying the video data to the display means;
Holding display delay time information that is a calculation result of the calculation process, and updating display delay time information at a timing that satisfies a predetermined condition,
In the above sync detection correction process,
Check that there is no synchronization error between the video and the audio by confirming that there is no synchronization error from the display delay time that is the calculation result of the calculation process and the display delay time that is held by the display delay time process. Check
If the video is ahead of the audio, send a pause command to the video decoding means to stop the temporary operation,
12. The data processing program according to claim 11, which causes the computer to execute data processing for transmitting a skip instruction so as to skip the video decoding means when the video is delayed compared to the audio . An audio decoding process for decoding the audio data by the audio decoding means and supplying it to the sound output means;
Display delay time information that is a calculation result of the calculation process, and holds the display delay time information at a timing that satisfies a predetermined condition;
A calculation process for calculating difference information between the sound output time and the reference time,
The synchronization detection correction process is
Confirm that there is no synchronization error in the audio from the difference information that is the calculation result of the calculation process and the display delay time held in the display delay time process, and whether there is a synchronization error between the video and the audio. Check
If the audio is ahead of the video, send a pause command to the audio decoding means to stop the temporary operation,
The data processing program according to claim 11 or 12, which causes the computer to execute data processing for transmitting a skip instruction so as to skip to the audio decoding means when the audio is delayed compared to the video .

Images