JP2006148679A - Data processing device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To reproduce audio and video to be both synchronized as being intended initially about each output path. <P>SOLUTION: A data processing device has a receiving unit for receiving a data stream and for outputting as an original stream or as a copied stream after being copied, a separating unit for extracting first data, its first time information, second data, and its second time data from the original stream, and for extracting first data, its first time information, second data, and its second time information from the copied stream, a first decoding unit for outputting the first data based on the first time information after decoding with respect to each stream, and a second decoding unit for outputting the second data based on the second time information. For synchronous reproduction of the first and second data, the device further has a delay setting unit for holding delay quantity information of a reproducing time of each stream, and for correcting the second time information based on the information and delaying the reproducing time of the second data. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a technique for reproducing audio and video in synchronization when reproducing compression-coded audio data and video data.

  2. Description of the Related Art Conventionally, a data reproducing apparatus that reads a video signal and an audio signal recorded by compression encoding from a recording medium such as a CD-ROM, a DVD-ROM, and a hard disk, decodes (expands), and outputs the data is known. Video and audio reproduction is realized by sending the decompressed data to a monitor, an acoustic speaker, or the like connected to the data reproduction device.

  Many standards are defined for compression coding. For example, MPEG (Motion Picture Experts Group) is known as a standard for compressing and encoding both video signals and audio signals. In the MPEG standard, in order to ensure that the video represented by the video signal and the audio represented by the audio signal are reproduced synchronously, time information representing the time to be reproduced is added to each of the audio signal and the video signal. . Thereby, at the time of expansion, the audio signal and the video signal can be reproduced in synchronization with the system time base reference value of the system device by referring to the time information.

  A conventional reproducing method will be described below. Note that the applicant has recognized the signal reproduction device disclosed in Patent Document 1, and in order to clarify the problem of the present invention, the contents of the prior art will be briefly described as a conventional example. In this method, a delay unit is provided in each decoding for audio delay and image delay, and reproduction is realized while taking a lip sync by performing a predetermined delay.

  A conventional signal reproducing apparatus will be described below. FIG. 11 shows an implementation of a conventional signal reproduction device. Referring to FIG. 11, audio delay unit 187 outputs the audio signal input from audio decoder 186 for a time obtained by adding the offset width input from control unit 194 to a predetermined delay time common to video delay unit 190. After the delay, it is output to an audio digital-analog converter (abbreviated as ADAC) 188. The video delay unit 190 delays the video signal input from the video decoder 189 by a predetermined time, and then outputs the video signal to the mixer 191 and the video digital-analog converter (abbreviated as VDAC) 192. The control unit 194 receives a delay command or the like from the command input unit 193 and sets the offset width of the lip sync designated.

  When the audio playback and the video display timing are out of synchronization, the viewer feels uncomfortable. In particular, if audio reproduction precedes video reproduction for a certain time (approximately 30 milliseconds) or more, the movement of the mouth of the person in the video and the synchronization of the audio are conspicuous, giving the viewer a sense of discomfort. In addition, if processing that requires more time is performed in the processing of the connected external device, the video and audio playback timing will be further shifted. Therefore, the output timing is adjusted using the video delay unit 190 and the audio delay unit 187.

  A case where not only the first output unit but also the second output unit is added using the configuration of the conventional signal reproduction device will be described with reference to FIG. FIG. 12 shows a configuration of a conventional signal reproducing apparatus having a plurality of output paths. For example, the first output is connected to a high-definition image for a large-screen theater and an external audio amplifier, and the second output is connected to a recorder such as a video deck or a monitor output at hand with standard image quality. A system is assumed. In FIG. 15, the audio decoder 201 has the same function as the audio decoder 186. The video decoder 202 is equivalent to the video decoder 189. The audio delay unit A 203 is equivalent to the audio delay unit 187, the video delay unit 204 is equivalent to the video delay unit 190, and the ADAC 205 is equivalent to the ADAC 188. The 206 VDAC is equivalent to the VDAC192. Further, the audio delay unit B207 delays each of them for input to the encoder 209 and a video delay unit B208.

There is a difference in the delay setting between the image processing time for standard image quality in the video decoder 202 and the image processing time for high-definition image quality. Furthermore, delay settings suitable for the processing contents for each output, such as delay settings when performing IP conversion (interlace to progressive), delay settings when there is surround sound processing, and the like are necessary. Audio and video can be synchronized by setting each delay unit from the control unit. By providing a semiconductor memory as the delay unit, even if a plurality of output units are provided, this can be accommodated.
JP 2001-101803 A (page 3-5, FIGS. 1 and 2)

  When a semiconductor memory is used for the delay unit as in the conventional signal reproducing apparatus, the memory size of the semiconductor memory must be increased in order to increase the delay amount. However, generally, as the memory size increases, the cost of the semiconductor memory increases, and the price of the signal reproducing device is greatly affected. Therefore, it is necessary to make the memory size as small as possible.

  An object of the present invention is to reproduce audio and video in a synchronized manner as originally intended while suppressing an increase in memory cost as much as possible. In particular, even when there are multiple output paths for audio and video, and the delay amount of the output path differs due to differences in output destination externally connected devices, processing types, etc., each output path is as originally intended. The audio and video are played back synchronously.

  The data processing apparatus according to the present invention receives a data stream and outputs it as an original stream, copies the received data stream and outputs it as a duplicate stream, and outputs each of the original stream and the duplicate stream. The receiving part which provides the identification information which can identify is provided. The data stream includes first data relating to compression-encoded video content, first time information defining a reproduction time of the first data, second data relating to content different from the video content, and reproduction time of the second data. The second time information that defines The data processing apparatus extracts the first data, the first time information, the second data, and the second time information from the original stream, and extracts the first data, the first time from the duplicate stream. A separator that extracts information, the second data, and the second time information; a first decoder that decodes the first data and outputs the first stream based on the first time information with respect to the original stream; A second decoding unit that outputs the second data with respect to the original stream based on second time information; and a third decoding unit that decodes the first data with respect to the duplicate stream and outputs the decoded data based on the first time information. A decoding unit; a fourth decoding unit that outputs the second data based on the second time information with respect to the duplicate stream; and a first delay And a tough and second delay setting unit.

  In the case where the first time information and the second time information define a time for reproducing the first data and the second data in synchronization, the first delay setting unit is configured to reproduce the original stream. First delay information for defining a delay amount of the second data, and correcting the second time information based on the first delay information, delaying the reproduction time of the second data, and the second delay setting unit Holds second delay information that defines a delay amount of the reproduction time of the duplicate stream, corrects the second time information based on the second delay information, and delays the reproduction time of the second data Let

  The data processing apparatus may further include at least one data buffer storing each of the second data of the original stream and the second data of the duplicate stream. The second data of the original stream is compressed and encoded, and the at least one data buffer stores the compressed and encoded second data before the reproduction time defined by the modified second time information. You may take out and output to the said 2nd decoding part and the said 4th decoding part.

  The at least one data buffer may include a first buffer that stores second data of the original stream and a second buffer that stores second data of the duplicate stream. The first buffer takes out the second data of the original stream and outputs the second data to the second decoding unit before the reproduction time defined by the second time information corrected by the first delay setting unit, The second buffer may extract the second data of the duplicate stream and output it to the fourth decoding unit before the reproduction time defined by the second time information modified by the second delay setting unit. .

  The data processing apparatus according to the present invention has a receiving unit that receives a data stream and outputs it as an original stream. The data stream includes first data relating to compression-encoded video content, first time information defining a reproduction time of the first data, second data relating to content different from the video content, and reproduction time of the second data. The second time information that defines In addition, the data processing device includes a separation unit that extracts the first data, the first time information, the second data, and the second time information from the original stream, the first data, the first time information, A duplicating unit that duplicates the second data and the second time information; a first decoding unit that decodes the first data and outputs the original data based on the first time information; and the original stream A second decoding unit that outputs the second data based on second time information, a third decoding unit that decodes the copied first data and outputs the first data based on the copied first time information, and And a fourth decoding unit that outputs the duplicated second data based on the duplicated second time information, and a first delay setting unit and a second delay setting unit.In the case where the first time information and the second time information extracted from the original stream define a time for reproducing the first data and the second data in synchronization, the first delay setting unit is , Holding first delay information that defines a delay amount of the reproduction time of the original stream, correcting the second time information based on the first delay information, and delaying the reproduction time of the second data The second delay setting unit holds second delay information that defines a delay amount of the reproduction time of the copied first data, and corrects the second time information based on the second delay information. The reproduction time of the second data is delayed.

  The second data of the original stream is compression encoded. The data processing apparatus may further include at least one data buffer for storing each of the second data of the original stream and the duplicated second data. The at least one data buffer extracts the compression-encoded second data before the reproduction time defined by the modified second time information, and outputs the second data to the second decoding unit and the fourth decoding unit. .

  The at least one data buffer may include a first buffer that stores second data of the original stream and a second buffer that stores duplicated second data.

  According to the present invention, original data and copied data are created based on compression-encoded content data including input video and audio data. By holding the compressed and encoded data for delay output, synchronized playback is performed by independently setting the delay for each path for video and audio decoded separately based on each sync signal. There is an advantage that can be executed.

  Embodiments of a data processing apparatus according to the present invention will be described below with reference to the accompanying drawings.

(Embodiment 1)
First, the configuration of the data processing apparatus according to the first embodiment of the present invention will be described with reference mainly to FIG. FIG. 1 shows a functional block configuration of a data processing apparatus 100 according to the present embodiment. FIG. 1 also shows video / audio equipment 13-16 connected to the data processing device 100. The data processing apparatus 100 performs binarization processing on a data stream read from an optical disc such as a DVD-ROM via an optical pickup, and outputs the data stream as a digital data stream. Thereafter, the stream is separated into video data and audio data, each of which is decoded and output to the audiovisual apparatus.

  The data processing apparatus 100 is a DVD player, a DVD recorder, a PC, or the like, but is a hard disk recorder when a hard disk is used instead of the optical disk. Alternatively, when a digital broadcast data stream is received via an antenna or the like, the data processing apparatus 100 is realized as a set-top box, a digital satellite broadcast receiver, or the like. On the other hand, the audiovisual apparatus is an interface apparatus such as a monitor output apparatus such as a TV, an audio output amplifier, and an AV amplifier having an AV selector function. FIG. 1 shows an example in which there are a plurality of devices that display video, devices that output audio, and the like.

  Receiving unit 1 that receives data supplied from various encoding methods, an optical disk device that stores digital compression encoded audio / video signals such as digital broadcasts, and a digital broadcast tuner device, etc. It has a separation unit 2 that separates recorded audio / video signals, separates compressed video data and compressed audio data, and performs input analysis to extract video playback time information and audio playback time information.

  In MPEG, in order to realize synchronized playback of video and audio, the time of playback is managed for each decoding unit called video and audio access units (one frame for video and one audio frame for audio). A time stamp value is added. This time stamp value is called PTS (Presentation Time Stamp), and audio PTS (hereinafter referred to as “APTS”) for audio and video PTS (hereinafter referred to as “VPTS”) for video are defined. . In this specification, a state where audio and video are synchronized is referred to as “AV synchronization”.

  When the PTS matches the above-mentioned system time reference STC, the access unit to which the PTS is added is output for reproduction. Like the SCR, the accuracy of the PTS is a value measured with a 90 kHz clock and is represented by a 33-bit length. The reason for measuring at 90 kHz is that it is a common multiple of the frame frequency of both NTSC and PAL video systems, and to obtain higher accuracy than one sample period of audio.

  One of the features of the data processing apparatus 100 according to the present embodiment is that the output that precedes when the video and audio that should be reproduced in synchronization with each other are reproduced due to various arithmetic processes performed in the audiovisual apparatus. Is output with a delay. For example, when video data playback is delayed from audio playback in order to perform interlace / progressive conversion of video data in an audiovisual apparatus, the audio data preceding by the time corresponding to the delay amount of the playback time of the video data Is output with a delay. Thereby, synchronized reproduction of video data and audio data becomes possible. The delayed output of audio data is realized by correcting the PTS that defines the playback time of the audio data. The amount of delay is set based on the expected time, the time set by the user, the time required for video processing in the audiovisual apparatus that can identify and determine the audiovisual apparatus, and the like.

  The characteristics of the data processing apparatus 100 described above are mainly realized by the decoding unit 101 of the data processing apparatus 100. Hereinafter, the decoding unit 101 will be described in detail.

  FIG. 2 shows a detailed functional block configuration example of the decoding unit 101. The decoding unit 101 receives a video buffer unit (not shown), a video playback time information management unit A301, a system time reference unit 11, a compressed video buffer unit A17, and a video decoding unit A3 in response to an input from the separation unit 2. A compressed audio buffer unit A7, an audio reproduction time information management unit A401, an audio decoding unit A4, and an audio delay setting unit A9.

  The separation unit 2 separates and extracts the digital compression encoded audio and video data from the data stream, extracts the PTS added to the video pack as video playback time information, and adds the PTS added to the audio pack. Are extracted as audio playback time information.

  First, components related to video processing will be described. The compressed video buffer unit A17 primarily stores the video data separated by the separation unit 2. The video playback time information management unit A301 generates and manages a table that associates video data stored in the compressed video buffer unit A17 with video playback time information.

  The video decoding unit A3 analyzes the attribute information (video header information) of the compressed and encoded video data, and decodes the video data stored in the compressed video buffer unit A17 according to the video playback time information. The video decoding unit A3 is provided with a frame buffer unit 303 and a video output unit 304. The frame buffer unit 303 stores the decoded video data. The video output unit 304 outputs video data to the outside. This video data is further output to an audiovisual apparatus (for example, the external video apparatus 13) connected to the data processing apparatus 100.

  Next, components related to audio processing will be described. The compressed audio buffer unit A7 is a semiconductor memory that can store the audio data separated by the separation unit 2. The buffer size varies depending on the compression rate of the encoded audio data to be handled, and is, for example, about several kilobytes to several tens of kilobytes. The audio playback time information management unit A401 generates and manages a table associating audio data stored in the compressed audio buffer unit A7 with audio playback time information. This table will be described in detail later with reference to FIG.

  The delay setting unit A9 holds delay information that defines a delay amount of the audio playback time, corrects the audio playback time information based on the delay information, and delays the playback time. The delay setting unit A9 has, for example, a memory (not shown) that holds delay information, and a time that is assumed in view of the processing of the audiovisual equipment is set in advance. Further, it is possible to hold information on delay time set by the user in units of several milliseconds or video frames. Thereby, fine adjustment according to the processing delay generated in the audiovisual apparatus becomes possible.

  The audio decoding unit A4 analyzes the attribute information (audio header information) of the compression-coded audio data, and decodes the audio data stored in the compressed audio buffer unit A7 according to the audio reproduction time information. The audio decoding unit A4 is provided with a PCM buffer unit 403 and an audio output unit 404. The PCM buffer unit 403 stores the decoded audio data. The audio output unit 404 outputs audio data to the outside. The audio data is further output to a video / audio device (for example, the external audio device 14) connected to the data processing apparatus 100.

  Note that the video decoding unit A3 and the audio decoding unit A4 described above can be realized by separate semiconductor integrated circuits or the like and a control program thereof. However, as long as each function can be realized, it may be realized by one semiconductor integrated circuit and the control program.

  Next, a configuration for reproducing video data and audio data output from the video output unit 304 and the audio output unit 404 in synchronization will be described. The decoding timing and the reproduction timing are realized by the system time reference unit A11. The system time reference unit A11 generates a system time reference STC inside the data processing apparatus 100 of the MPEG system. In order to generate an STC on the data processing apparatus 100 side, an SCR (System Clock Reference) value defined in a program stream (PS) used in a DVD disk or the like is used. The SCR is described in the pack header. When the input data stream is a transport stream (TS) used in BS digital broadcasting, PCR (Program Clock Reference) is used.

  The system time reference unit A11 sets the SCR value to the same value as the system time reference STC when the last byte of the video data arrives. Thereby, a reference time can be set. Further, by combining this STC value setting and a PLL (phase locked loop) circuit (not shown), it is possible to have an STC whose frequency completely matches the reference time system clock of the decoding unit 60. The system clock frequency of the system time reference STC is 27 MHz. Each PTS (frequency: 90 kHz) is referred to by dividing the STC by a counter or the like.

  When the data processing apparatus 100 performs conventional processing, the video decoding unit A3 outputs and reproduces the access unit to which the PTS is added when the system time reference STC matches VPTS. The audio decoding unit A4 outputs and reproduces the access unit to which the PTS is added when the STC matches the APTS. Since the frequency of the PTS is 90 kHz, each of the video data and the audio data is decoded so that the STC can be synchronized with the VPTS and the APTS within this accuracy range. As a result, synchronized playback of video and audio can be realized.

  On the other hand, according to the processing according to the present embodiment, the data processing apparatus 100 realizes synchronized playback of video and audio in consideration of video processing delay in the audiovisual apparatus. Hereinafter, an outline thereof will be described with reference to FIG. 3, and thereafter, a processing procedure for realizing the operation will be described with reference to FIG.

  FIG. 3 shows playback timings for realizing synchronized playback of video and audio. The playback timing is defined based on video playback time information and audio playback time information. In FIG. 3, the timing indicated by the reproduction time information is described in four stages. The first level shows the timing of video playback time information (VPTS) defined in the data stream. An encoder (not shown) that has generated a data stream performs compression encoding with the intention of reproducing a video at times Tv1, Tv2,. The data stream also includes audio reproduction time information (APTS) to be reproduced at times Ta1, Ta2,. The encoder requests video and audio to be reproduced in synchronization with the system clock STC by attaching VPTS and APTS, respectively.

  Next, the second row in FIG. 3 shows the playback timing of the video delayed by the video processing in the audiovisual apparatus. For example, regarding the video displayed first, a video that should originally be displayed at time Tv1 is reproduced at time Tp1 after time Tv1 due to a delay caused by video processing. Let the time difference be Δt.

  Next, the fourth level shows the audio reproduction timing after the delay setting unit A9 of the present embodiment corrects the original reproduction time information (APTS). The audio delay setting unit 9 corrects the audio playback time information (APTS) indicating that it should be played back at the time Ta1 to time information (NAPTS) shifted by the video delay time (Δt), and plays back the audio. The time is delayed. Therefore, the video output is reproduced at the second stage timing (time Tp1, Tp2,...), And the audio output is output and reproduced at the fourth stage timing (time Tna1, Tna2,...). As a result, synchronized playback of video and audio can be realized.

  Note that “synchronous playback” of video and audio means that the video playback timing is within a certain time range (for example, between 50 ms ahead and 30 ms delay) relative to the audio playback timing. Say. It is not limited to the case where the video and audio PTS match.

  FIG. 4 shows a decoding processing procedure according to this embodiment. First, the separation unit 1 separates the encoded data stream into encoded video data and encoded audio data. In steps S101 and S102, the encoded frame data and reproduction time information are encoded from the encoded data (video / audio). Extract (VPTS / APTS). The encoded frame data of video and audio are respectively stored in the compressed video buffer unit A17 and the compressed audio buffer unit A7, and the reproduction time information (VPTS / APTS) is the video reproduction time information management unit A301 and the audio reproduction time information management unit. Sent to A401.

  FIG. 5 shows a configuration of a table generated in the audio reproduction time information management unit A401. This table associates audio data stored in the compressed audio buffer unit 7 with audio reproduction time information. First, the audio reproduction time information management unit A401 divides into units called slots for each APTS, for example. Each slot is associated with audio address information. The audio address information is stored in association with each slot as data referring to the head address of the audio data stored in the audio buffer unit A7. The relationship between the data arrangement of the stored audio data and the data position referred to by the audio address information is shown.

  Furthermore, the compressed audio buffer unit A7 has a write pointer for specifying the last position (address in the buffer) where the compressed audio data is written. The position indicated by the write pointer is updated to indicate the last position when data is written. When the compressed audio buffer A7 writes the data up to the final address of the buffer, the compressed audio buffer A7 returns the write position to the first address. That is, buffer address management is performed in a ring (wraparound) manner.

  The compressed audio buffer unit A7 has a read pointer for specifying the final position (final address in the buffer) from which the compressed audio data is read. When the compressed audio data is read by the audio decoding unit A4, the position indicated by the read pointer of the compressed audio data is updated. Therefore, subsequent data can be written up to the read position. The managed compressed audio data is read while managing so that the position of the write pointer does not overtake the position of the read pointer, and the subsequent compressed audio data separated by the separation unit 2 is written.

  Refer to FIG. 4 again. In step S103, the video decoding unit A3 decodes the encoded frame data of the video before the VPTS reaches the system time reference STC, and stores the decoded video data in the frame buffer unit 303.

  Next, in step S104 and step S105, the delay setting unit A9 adds the PTS value Δt corresponding to the preset delay time to the APTS value to obtain a new audio reproduction time information NAPTS value (NAPTS = APTS + Δt). . Through the above processing, new reproduction time information (NAPTS) obtained by correcting the reproduction time information (APTS) of audio is obtained. In the present embodiment, description will be made assuming that the delay time is preset by the user. The user can select an item to be set using an input button, a remote controller (not shown), or the like provided as the delay setting unit A9. For example, the time to be delayed is set to “60 ms”. The PTS value Δt can be obtained based on the set delay time.

  Although the setting of the delay time is necessary for realizing the AV synchronization, the reason why the delay time can be set in advance before actual reproduction is as follows. That is, in the video decoding unit A3, for example, the processing time for outputting a standard-definition image is substantially constant, and the processing time for outputting a high-definition image is also substantially constant (however, both processes are performed). Time may be different). Therefore, it can be said that how much audio should be delayed can be specified in advance according to the processing content of the video. As a result, the user can set a delay time before actual reproduction.

  In step S106, the audio decoding unit A4 decodes the encoded frame data of the audio before the NAPTS reaches the system time reference STC. In step S108, the audio decoding unit A4 stores the decoded audio data in the PCM buffer unit 403.

  In step S107, the video decoding unit A3 determines whether or not the system time reference STC has reached VPTS. “Achieved” indicates when the STC matches the VPTS or when the STC exceeds the VPTS. If it has reached, the video output unit 304 outputs the video data from the frame buffer in step S107. If not, wait until it reaches.

  Next, in step S108, the audio decoding unit A4 determines whether or not the STC has reached NAPTS. “Reached” here is the same as video, and indicates when STC matches NAPTS or when STC exceeds NAPTS. If it has reached, the audio output unit 404 outputs audio data from the PCM buffer. If not, wait until it reaches.

  In step S109, the video decoding unit A3 and the audio decoding unit A4 determine whether the decoding is finished. If it has been completed, the process is terminated. If not, the process returns to step S101 or step S102 to continue the process. Note that the processing is also ended when the user instructs the stop of reproduction. Note that the order of steps S101 and S102 is changed in a timely manner depending on which input is first.

  Depending on the device, there may be a case where the optical output terminal is output as a Dolby Digital stream. In such a case, the data is temporarily stored in a stream buffer (not shown), the STC is referred to, and the audio is output when the NAPTS matches or exceeds.

  Video decoding and audio decoding are performed so that VPTS and NAPTS are synchronized with the STC. In particular, if the VPTS is set to NAPTS between the time of 50 milliseconds preceding and the time of 30 milliseconds delay, the lip sync shift will not be noticed.

  Next, connection between the data processing apparatus 100 and a plurality of types of external video equipment, external audio equipment, and the like will be described. Content data encoded (compressed) by various encoding methods, digital compression encoded audio / video data such as digital broadcasts, and the like are decoded by the data processing apparatus 100, and are then used by external video equipment, external audio equipment, and the like. Played.

  First, the processing of the receiving unit 1 to which a digital compression encoded audio / video data stream is supplied will be described below. In the following description, it is assumed that the receiving unit 1 receives a transport stream (TS) including, for example, an audio data stream and a video data stream.

  An identification number (hereinafter abbreviated as ID) for identifying each stream is attached to the audio data stream and video data stream input to the receiving unit 1. When receiving a satellite broadcast or the like, the receiving unit 1 built in a conventional set-top box for satellite broadcasting uses audio / video data related to the program based on a program ID that identifies the program selected by the user. Specify the stream ID. Then, an audio / video data stream having the specified ID is extracted from the plurality of data streams and supplied to the separation unit 2.

  In addition to the above operation, the receiving unit 1 according to the present embodiment further creates a duplicate of the data stream of the extracted audio / video data stream and specifies the duplicated data stream when extracting the previous data stream An ID different from the ID is created and assigned to the duplicated data stream. That is, the ID of the original data stream to be duplicated is different from the ID of the duplicated data stream.

  Next, the separation unit 2 separates the video data stream and the audio data stream using the ID, and performs input analysis for separating the compressed video data and the compressed audio data and extracting the video reproduction time information and the audio reproduction time information.

  The separation unit 2 also handles a plurality of compressed audio data and a plurality of compressed video data, and has a function of supplying each data to a plurality of decoding units. The separation unit 2 supplies compressed data having a designated ID to each of a plurality of decoding units in the data processing device.

  Therefore, the following two can be processed without distinction with respect to the separation unit 2, and separate decoding processing is performed by a plurality of decoding units.

  (1) When a user designates reproduction of a plurality of programs at the same time, a plurality of program IDs are designated and a data stream of the plurality of programs is supplied.

  (2) When the user designates the ID of a single program, as a result, the receiving unit 1 is supplied with each data stream before duplication and after duplication.

  The data processing apparatus 100 according to the present embodiment can play back content while synchronizing AV in each external connection device even when output paths to a plurality of external connection devices are provided. For example, a high-definition broadcast of BS digital broadcasting can be reproduced and output from one terminal with high image quality, and can be reproduced and output for recording from the other terminal connected to a standard-definition video recorder. In this example, in order to create a high-definition broadcast screen (first screen), it is necessary to perform image processing for display with higher image quality, and accordingly video output processing takes time. On the other hand, in order to create an output video screen (second screen) to a standard-definition video recorder, it is necessary to perform image processing for converting the format of a high-definition video into a standard video. The image processing times of the first screen and the second screen are set in the form of connection to the external device, and need not be the same. Therefore, the above-described processing for delaying audio reproduction is performed, and after AV synchronization is achieved, video and audio signals are output.

  The feature of the data processing apparatus according to the present embodiment is that not only the data stream having the conventional designated ID is passed through the receiving unit 1 to the separating unit 2, but also a copy of the data stream with the designated ID is created and a different ID is assigned. By giving the duplicate data stream and notifying the separation unit 2 of different ID information, the post-processing by the separation unit 2 is exactly the same as the decoding processing of a plurality of programs.

  Further, when one output of the data processing device is connected to a recording processing device (built-in video device or the like) and the remaining output is prepared for an external connection device, the same configuration can be used. Also, when the data processing apparatus is a digital broadcast receiver and has an encoding apparatus for recording a DVD while reproducing the data stream in the digital broadcast transport stream format and encoding it in the program stream format for DVD recording. Can be handled with the same configuration.

  When recording on a medium such as a DVD with an encoding device, AV synchronization must be present. This is because the recorded medium is not always decoded and reproduced by the same device as the device that performed the encoding and recording. This is because if the VPTS for video and the APTS for audio are attached in a state where the AV synchronization is shifted, the playback side assumes that the shifted PTS is regular time information and plays back.

  As described above, the delay time set for output to an externally connected video device and audio device is different from the delay time for internal encoding. When a user uses an externally connected device, whether or not AV synchronization is realized is confirmed based on whether the announcer's video (mouth movement) matches the sound in a news program or the like. be able to. However, it is not possible to directly check whether AV synchronization is realized for the internal encoding video and audio unless the video and audio are output on a monitor. In particular, when receiving and recording a satellite digital broadcast, the original image quality of a data stream to be decoded may be changed due to the convenience of the broadcasting station during the program. In some cases, high-definition image quality (hereinafter, high-definition image quality) is changed to standard image quality, or vice versa.

  In such a case, the image processing time actually required for converting the image quality is different from the processing time originally assumed. If the same delay time is continuously set without taking this difference in image processing time into consideration, a state in which AV synchronization is not achieved continues from the time when the image quality during the program changes.

  Therefore, in the present embodiment, the delay information is changed when the delay information of the image processing is acquired, and the delay setting is performed again. As a result, AV synchronization can be achieved even after a change has occurred. Even in the path to be output to the external connection, if the change of the delay time is recognized instead of the content of the image processing, the delay time may be changed.

  In the above description, it is assumed that the PCM data is buffered after decoding the compression-encoded audio data. This corresponds to preparation processing of an output signal for output to the audio output unit. In order to delay the audio signal, the audio buffer unit A7 (FIG. 1) according to the present embodiment is compared with the configuration of the audio delay unit 187 (FIG. 11) that delays the decoded PCM data of the conventional example. Since the compressed and encoded audio data is held, the amount of data is small and the buffer size is small.

  The buffer capacity required for the audio buffer unit A7 is the amount of encoded audio data received before the reproduction time defined by the original APTS and the elapse of time corresponding to the reproduction time delay amount (Δt). Is equal to or greater than the sum of the amount of encoded audio data received. For example, consider a case where an additional buffer capable of additionally storing data of a maximum of 100 milliseconds is necessary as a buffer necessary for delay. This corresponds to a case where the delay time is a maximum of 100 milliseconds, and in video, it corresponds to a display time of about 3 frames of video at the time of NTSC format output. The capacity of the additional buffer required in this case is about 22 kilobytes in the AAC format of the audio compression format used in BS digital broadcasting (data amount for 6 channels is calculated as 288 kilobit seconds per channel). This capacity can be reduced to a fraction of the capacity for additionally storing PCM data after decoding.

  In the present embodiment, the video decoding unit A3 and the audio decoding unit A4 decode the compressed and encoded video data and audio data and store them in the frame buffer unit 303 and the PCM buffer unit 403. However, for example, when linear PCM data is included as audio data in a DVD, the decoding operation is not necessary. This is because the linear PCM data is encoded but not compressed. However, even if the audio data is linear PCM data, it is reproduced in synchronization with the video in accordance with the synchronization information included in the stream, so the above description can be applied as it is. Further, when the data processing apparatus 100 outputs an encoded stream such as Dolby Digital from the optical output terminal as it is, it is temporarily stored in a stream buffer (not shown), and the STC is referred to, and when the NAPTS reaches the STC. Thus, the output from the audio output unit 404 may be performed.

Next, a specific example of video processing for output to the external video equipment 13 that generates audio delay will be described. The expected video processing is
(1) Conversion from film material at 24 frames per second to video material at 30 frames per second,
(2) Conversion processing between interlace scanning and progressive scanning, conversion processing between standard definition (SD) D1 standard signal and high definition image quality (HD) D3 standard signal, between NTSC system and PAL system Conversion processing (3) video enlargement / reduction based on an instruction from the user, editing processing, and (4) synthesis processing of a plurality of videos. During normal playback that does not require the above processing, there is no need to output audio data with a delay. However, since a delay may occur due to on / off of the video processing, it is necessary to set a delay amount and output a delay of audio data based on the setting.

  In such a case, it is preferable to provide a plurality of pieces of delay information having different delay times in the delay setting unit A9 and associate them according to the type of processing to be performed. Furthermore, when the user inputs the type of image processing to be executed by the external video device 13, the delay time may be changed according to the type.

  FIG. 6 shows a functional block configuration of the video output unit 304 equipped with a processing circuit for performing video processing. The video output unit 304 receives the decoded frame data from the frame buffer unit 303 and outputs video data that has undergone predetermined video processing. The video output unit 304 includes an IP conversion unit 371, a format conversion unit 372, an enlargement / reduction unit 373, another screen composition unit 374, a GUI superimposition unit 375, and an output unit 376. In relation to these components, an external video input unit 377 and a GUI data generation unit 378 are separately provided in the video decoding unit A3.

  The IP conversion unit 371 performs a conversion process between interlace scanning and progressive scanning. The format conversion unit 372 performs a conversion process between the 480i video signal and the 1080i video signal and a conversion process between the NTSC system and the PAL system. The enlargement / reduction unit 373 performs enlargement, reduction, and editing processing of the video based on an instruction from the user. The other screen combining unit 374 combines the decoded video and the video input from the external video input unit 377. The GUI superimposing unit 375 superimposes the GUI data generated by the GUI data generating unit 378 on the video obtained by the processing so far. Then, the output unit 376 outputs the finally obtained video data. This output is sent to the external video equipment 13. The video decoding unit A3 can specify the delay time by acquiring the time when the video data is obtained based on the STC and comparing it with the VPTS of the video data.

  Furthermore, as shown in FIG. 7, the connection between the data processing apparatus 100 and the audiovisual equipment 120 and the types of data exchanged between them are shown. In the example of FIG. 7, data is exchanged according to the HDMI standard. First, the HDMI transmitter 480 transmits a control command for obtaining information (device-specific information) for specifying the audiovisual device 120 to the HDMI receiver 430 of the audiovisual device 120. Based on the control command, the HDMI receiver 430 transmits the device ID of its own device to the HDMI transmitter 480 as device specific information. The device ID is stored, for example, as manufacturer specific information (Vendor Specific Data) in the ROM 431 in the audiovisual device 120, and the manufacturer name, model name, and the like can be specified.

  The HDMI transmitter 480 receives the device ID. When the delay setting unit A9 receives the device ID, the delay setting unit A9 refers to a table defined in the internal memory or the external memory card. The table describes the correspondence between device IDs and delay settings. And based on apparatus ID and its table, the delay time resulting from the video processing which the audiovisual apparatus 120 performs can be obtained. Thereby, the delay time can be specified. If the acquired information is stored as a default setting value in the data processing apparatus, it is possible to always watch AV in the same state as long as the device connection is not changed. If there is a change in the connected device ID or the like, the setting may be changed by receiving information on the partner device each time.

  As these externally connected devices, monitor output devices such as a TV, interface devices such as audio output amplifiers and AV amplifiers having an AV selector function, portable output devices, in-vehicle AV playback devices, and the like are assumed.

  In the first embodiment, in order to synchronize the AV synchronization of each of the plurality of output paths, a buffer unit and a decoding unit before compression are provided. This is effective when the semiconductor cost is lower than when a buffer memory is provided for each output path after decoding. For example, it is assumed that the microprocessor has a high function and a plurality of decoding functions can be realized by the same semiconductor. Then, it is possible to reduce the cost of the entire apparatus by buffering in a compressed state before decoding rather than placing a buffer memory after decoding for each output path.

  In addition, a plurality of decoding units may be required for one data processing apparatus. For example, it is a case where a compression-encoded data stream of BS digital broadcasting is re-encoded and written on a hard disk or an optical disk, and the data stream is reproduced at a time later than the broadcasting time. The re-encoding process is a process in which a compression-encoded data stream is once decoded by a decoding unit and encoded (compression encoding) with a further increase in the compression rate, and includes the decoding process. The reproduction process is a process for decoding and outputting the compression-encoded data stream in the decoding unit. In the above example, two decoding units are required at the same time, and a delay setting for AV synchronization is required for each output. Therefore, in a data processing apparatus having a function of chasing and playing back a digital broadcast, a plurality of decoding units are required. Therefore, a special decoding unit is added to realize the delay setting for a plurality of outputs in the first embodiment. do not have to. The first effect is to reduce the cost by reducing the memory size for the delay buffer.

(Embodiment 2)
FIG. 8 is a block diagram showing the configuration of the data processing apparatus 102 according to the present embodiment. Hereinafter, the description will focus on the differences from the data processing apparatus 100 according to the first embodiment. In FIG. 8, components corresponding to the compressed video buffer units A and B in the first embodiment are not described, but this difference is for simplification and not essential.

  The receiving unit 21 duplicates only the video data stream and assigns an ID for identifying the data stream before duplication and the data stream after duplication. On the other hand, the audio data stream is not duplicated.

  Therefore, the separating unit 22 divides and supplies video stream data with different IDs to the video decoding unit A 23 and the video decoding unit B 26, and supplies the audio stream data to the compressed audio common buffer unit 27.

  The compressed audio common buffer unit 27 has a write pointer that moves the latest writing position to the final written point by writing the compressed audio data. The compressed audio common buffer unit 27 also has read pointers A and B. By reading the compressed audio data by the audio decoding unit A24, the read pointer A of the compressed audio data is updated. Further, the compressed audio data is read by the audio decoding unit B25, whereby the read pointer B of the compressed audio data is updated. The compressed audio common buffer unit 27 returns the writing position to the first address if writing is performed up to the last address. That is, buffer address management is performed in a ring (wraparound) manner. Accordingly, the next writing can be performed up to the read position, and the separation unit 2 prevents the write pointer from overtaking the read pointer of the two read pointers that has not been read yet. The compressed audio data writing is managed as a buffer. The functions of the write pointer and read pointer described above are the same as the write pointer and read pointer in the first embodiment.

  Compared to the first embodiment, the second embodiment requires about half the buffer capacity of the compressed audio data. In the first embodiment, each decoding unit has its own audio buffer. However, in the second embodiment, the data itself stored in the audio buffer is the same, and the common use of the buffer is achieved by controlling a plurality of read positions by the difference in setting of each delay time. .

(Embodiment 3)
FIG. 9 is a block diagram showing the configuration of the data processing apparatus 104 according to the present embodiment. Hereinafter, the difference from the first embodiment will be mainly described. In FIG. 9, components corresponding to the compressed video buffer units A and B in the first embodiment are not described, but this difference is for simplification and not essential.

  The receiving unit 41 acquires a video data stream and an audio data stream having an ID corresponding to the program designated by the user, and passes them to the separating unit 42. A data replication unit 53 is provided after the separation unit 42 to replicate the input data stream. The data before the data duplicating unit 53 is sent to the first decoder unit (video decoding unit A43 and compressed audio buffer unit A47), and the data stream duplicated by the data duplicating unit 53 is sent to the second decoder unit (video decoding unit B45 and To the compressed audio buffer B48).

  The subsequent processing is the same as in the first embodiment. This is the same as in the first embodiment even if the receiving unit 41, the separating unit 42, and the decoder unit are not modified when the data processing unit 104 is configured by a semiconductor by providing the data duplicating unit 53. The effect is obtained.

(Embodiment 4)
FIG. 10 is a block diagram showing the configuration of the data processing apparatus 106 according to the present embodiment. Hereinafter, the difference from the data processing apparatus 104 according to the third embodiment will be mainly described.

  The receiving unit 81 acquires a video data stream and an audio data stream having an ID corresponding to the program designated by the user, and passes them to the separating unit 82. A data duplicating unit 93 is provided after the separating unit 82 to duplicate the data stream of the input video data. The data before data duplication is supplied to the first decoder unit (video decoding unit A83), and the data stream duplicated by the data duplicating unit 93 is supplied to the second decoder unit (video decoding unit B85).

  Therefore, the separation unit 82 supplies the audio stream data to the compressed audio common buffer unit 87.

  The compressed audio common buffer unit 87 has a write pointer that moves the latest writing position to the last written point by writing the compressed audio data. Then, the compressed audio data is read by the audio decoding unit A84, whereby the read pointer A of the compressed audio data is updated. Further, the compressed audio data is read out by the audio decoding unit B86, whereby the read pointer B of the compressed audio data is updated. The compressed audio common buffer unit 87 is in a ring-shaped usage form in which the writing position returns to the first address if writing is performed up to the last address. Accordingly, the next writing can be performed up to the read position, and the separation unit 2 prevents the write pointer from overtaking the read pointer of the two read pointers that has not been read yet. The compressed audio data writing is managed as a buffer.

  Compared to the third embodiment, the fourth embodiment requires about half the buffer capacity of the compressed audio data. In the first embodiment, each decoding unit has its own audio buffer. However, in the fourth embodiment, the data itself stored in the audio buffer is the same, and the common use of the buffer is achieved by controlling a plurality of read positions by the difference in setting of each delay time. . The subsequent processing is the same as that of the data processing apparatus 100 according to the first embodiment.

  In each of the above-described embodiments, the input data has been described as data input from the outside and data input from an external recording medium, but may be data existing in the apparatus in advance.

  In the above-described embodiment, the description has been made assuming that the separation unit separates the video signal and the audio signal. However, it may be file data that is separated in advance. If the playback time information related to the compressed video data, the compressed audio data and the playback time information are input and the playback time information can be played back with the synchronization information, the data processing device according to each of the above embodiments is configured. can do. This is because, as a result of editing a signal photographed by a video camera or the like on a personal computer, playback time information associated with compressed video data, compressed audio, in both AV and data mixed files or AV data independent files. The present invention is applied to all cases where data, reproduction time information, and data information on a file are reproduced in synchronization with each other.

  Application examples of the data processing apparatus according to the above-described embodiment include a set-top box, a digital satellite broadcast receiver and its recording device, a DVD player or DVD recorder related device, a hard disk recorder, and a personal computer.

  The data processing apparatus according to the present invention has a plurality of output paths for display and other purposes based on the digitally encoded video signal and audio signal synchronization signal. By storing in the buffer memory and further setting the delay time by each signal processing independently, it can be used for the purpose of dynamically adjusting the AV synchronization of each path.

It is a figure which shows the structure of the functional block of the data processor 100 by Embodiment 1. FIG. FIG. 3 is a diagram illustrating a detailed functional block configuration example of a decoding unit 101. It is a figure which shows the reproduction | regeneration timing which implement | achieves the synchronous reproduction | regeneration of video and audio. 6 is a flowchart illustrating a decoding processing procedure according to the first embodiment. It is a figure which shows the structure of the table produced | generated in audio reproduction time information management part A401. It is a figure which shows the structure of the functional block of the video output part 304 carrying the processing circuit which performs video processing. It is a figure which shows the kind of data transmitted / received between the data processing apparatus 100 and the audiovisual equipment 120, and them. It is a block diagram which shows the structure of the data processor 102 by Embodiment 2. It is a block diagram which shows the structure of the data processor 104 by Embodiment 3. It is a block diagram which shows the structure of the data processor 106 by Embodiment 4. It is a block diagram which shows the implementation form of the conventional signal reproducing | regenerating apparatus. It is a block diagram which shows the structure of the conventional signal reproducing | regenerating apparatus which has several output paths.

Explanation of symbols

1 receiving unit 2 input unit 3 video decoding unit A
4 Audio decoding part A
5 Video decoding part B
6 Audio decoding part B
7 Compressed audio buffer A
8 Compressed audio buffer B
9 Delay setting part A
10 Delay setting part B
11 System time reference part A
12 System time reference part B
13, 16 External video equipment 14, 15 External audio equipment 17, 18 Compressed video buffer section 100, 102, 104, 106 Data processing apparatus 101 Decoding section

Claims (6)

A data stream is received and output as an original stream, and the received data stream is duplicated and output as a duplicate stream. At the time of output, identification information for identifying each of the original stream and the duplicate stream is given. The data stream includes: first data relating to compression-encoded video content; first time information defining a reproduction time of the first data; second data relating to content different from the video content; A receiving unit having second time information defining a reproduction time of the second data;
The first data, the first time information, the second data, and the second time information are extracted from the original stream, and the first data, the first time information, and the second data are extracted from the duplicate stream. And a separation unit for extracting the second time information,
A first decoding unit that decodes the first data and outputs the first stream based on the first time information with respect to the original stream;
A second decoding unit that outputs the second data based on second time information with respect to the original stream;
A third decoding unit that decodes the first data and outputs the decoded data based on the first time information;
A fourth decoding unit that outputs the second data based on the second time information with respect to the duplicate stream;
A data processing apparatus having a first delay setting unit and a second delay setting unit,
In the case where the first time information and the second time information define a time for reproducing the first data and the second data in synchronization,
The first delay setting unit holds first delay information that defines a delay amount of the reproduction time of the original stream, corrects the second time information based on the first delay information, and Delay the data playback time,
The second delay setting unit holds second delay information that defines a delay amount of the reproduction time of the duplicate stream, corrects the second time information based on the second delay information, and A data processing apparatus that delays the reproduction time of data. At least one data buffer storing each of the second data of the original stream and the second data of the duplicate stream;
The second data of the original stream is compression encoded;
The at least one data buffer extracts the compression-encoded second data before the reproduction time defined by the corrected second time information, and outputs the second data to the second decoding unit and the fourth decoding unit The data processing apparatus according to claim 1. The at least one data buffer includes a first buffer that stores second data of the original stream, and a second buffer that stores second data of the duplicate stream,
The first buffer takes out the second data of the original stream and outputs it to the second decoding unit before the reproduction time defined by the second time information corrected by the first delay setting unit,
The second buffer extracts the second data of the duplicate stream and outputs the second data to the fourth decoding unit before the reproduction time defined by the second time information corrected by the second delay setting unit. Item 3. A data processing apparatus according to Item 2. A receiving unit that receives a data stream and outputs the data stream as an original stream, wherein the data stream includes: first data relating to compression-encoded video content; first time information that defines a reproduction time of the first data; A receiving unit having second data relating to content different from video content and second time information defining a reproduction time of the second data;
A separation unit that extracts the first data, the first time information, the second data, and the second time information from the original stream;
A replication unit that replicates the first data, the first time information, the second data, and the second time information;
A first decoding unit that decodes the first data and outputs the first stream based on the first time information with respect to the original stream;
A second decoding unit that outputs the second data based on second time information with respect to the original stream;
A third decoding unit that decodes the replicated first data and outputs the decoded data based on the replicated first time information;
A fourth decoding unit for outputting the duplicated second data based on the duplicated second time information;
A data processing apparatus having a first delay setting unit and a second delay setting unit,
In the case where the first time information and the second time information extracted from the original stream define a time for reproducing the first data and the second data in synchronization,
The first delay setting unit holds first delay information that defines a delay amount of the reproduction time of the original stream, corrects the second time information based on the first delay information, and Delay the data playback time,
The second delay setting unit holds second delay information that defines a delay amount of the reproduction time of the replicated first data, and corrects the second time information based on the second delay information; A data processing device for delaying the reproduction time of the second data. The second data of the original stream is compression encoded;
The apparatus further comprises at least one data buffer storing each of the second data of the original stream and the duplicated second data, wherein the at least one data buffer is a reproduction defined by the modified second time information. The data processing apparatus according to claim 4, wherein the second data that has been compression-encoded before time is extracted and output to the second decoding unit and the fourth decoding unit. 6. The data processing apparatus according to claim 5, wherein the at least one data buffer includes a first buffer that stores second data of the original stream, and a second buffer that stores duplicated second data. .