JP2009272945A - Synchronous reproduction apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To synchronize video with sound, under a system configuration such that a video reproducing apparatus and an audio reproducing apparatus are connected separately to a network. <P>SOLUTION: The synchronous reproduction apparatus comprises a decoder 36, which is provided on an audio reproducing apparatus 3 side to decode the sound of content data; a synchronous control part 32 which transmits an audio side clock signal that controls the timing of decoding with the decoder 36 to a video reproduction apparatus 2 side through a network 4; a decoder 26 which is provided on a video reproducing apparatus 2 side to decode video of the content data; and a synchronous control part 22 which is provided on the video reproduction apparatus 2 side to adjust count values of decoding with the decoder 26 according to the audio side clock signal transmitted from the audio reproduction apparatus 3 side. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a synchronized playback apparatus that synchronizes video and audio when a stream in which video and audio are combined using separate video playback apparatuses and audio playback apparatuses on a network.

  In recent years, digital compression technology such as MPEG (Moving Picture Expert Group) has been used to store high-quality video and audio with a small amount of data, or to transmit via a network communication channel with a limited bandwidth. It is becoming possible.

Further, in general homes, with the spread of Ethernet (registered trademark) and the like, it has become an environment in which a network can be easily constructed without introducing special equipment.
From the background described above, usage such as viewing of video and music via a LAN (Local Area Network) is increasing.

  By the way, when trying to view content stored in the server via the LAN, generally, after the MPEG data from the server is stored in the reception buffer to some extent in the playback device, the MPEG is based on the decoder clock. The decoding and outputting of the decoding result are performed, and when the buffer becomes small, the reproduction data from the client side is taken to the server, and the reception buffer is replenished.

Conventionally, viewing via such a network is such that a stream in which video and audio are multiplexed is received and played back by a single playback device, or audio and video as represented by RTP (ReaI-timeTransport Protocol). The mainstream is a configuration in which a stream transmitted separately is received and played back by a single playback device that can simultaneously decode video and audio (for example, Patent Document 1).
JP 2005-102192 A

  However, in the conventional device disclosed in Patent Document 1 described above, since the video and audio are multiplexed streams, the video and audio are reproduced by a single playback device. Ability to decode both audio and audio is required. There is no problem with such a device as long as it is a video / audio encoded by a general encoding method such as MPEG2. However, in recent years, various compression schemes have been developed by improving the encoding technique. Particularly, regarding audio, in addition to MPEG2, MP3 (MPEG Audio Layer-3), AAC (Advanced Audio Coding), AC-3 ( There are many encoding methods such as Audio Code number 3) (registered trademark). Therefore, if a single playback device does not have the ability to decode audio encoded by the various encoding methods described above, there may be cases where video can be played back but audio cannot be played back. .

  In addition, even if a single playback device has the ability to decode audio, the user has an audio playback device that can achieve higher-quality audio playback in terms of analog performance after decoding. In such a case, there is a need to reproduce the sound with the sound reproducing apparatus.

As a technique for solving the above problems, audio data is supplied to an audio reproduction device via a network such as a LAN, and a single reproduction device (video reproduction device) reproduces video, and audio is audio. A method of reproducing with a reproducing apparatus is conceivable.
When such a method is realized, there arises a problem that video and audio must be synchronized.

  In other words, conventionally, when video and audio are received and played back by different devices via a network such as a LAN, the difference in delay time due to the difference in buffer amount at the time of reception and the delay time in the decoder Since it depends on the decoder, the synchronization relationship was not guaranteed at all even if it started simultaneously.

  Further, in the conventional apparatus, each reproduction apparatus decodes each according to each clock oscillator, so that the reproduction clock itself is not synchronized with the clock on the encoder side, and is gradually increased by the error of the clock oscillator in each decoder. However, there was a problem that the video and audio were misaligned.

  Therefore, the present invention solves the above-described problem, and enables synchronization playback of video and audio in a system configuration in which the video playback device and the audio playback device are separately connected to a network. An object is to provide a playback device.

Therefore, in order to solve the above problems, the present invention provides the following apparatus.
(1) When the video data and the audio data included in the content data are played back independently using the video playback device and the audio playback device arranged independently on the network, the video playback device performs the A synchronized playback device that synchronizes playback of video data and playback of audio data by the audio playback device,
An audio decoding means provided on the audio reproduction device side for decoding audio data of the content data;
Audio-side synchronization control that is provided on the audio reproduction device side and transmits a control value for timing to output an audio signal as a decoding result by the audio decoding means to the video reproduction device side through the network as an audio-side synchronization signal And
Video decoding means provided on the video reproduction device side for decoding the video data of the content data;
Provided on the video playback device side, receives the audio side synchronization signal transmitted from the audio playback device side, and in response to the audio side synchronization signal, the frequency of the clock used by the video decoding means, and the video decoding A video-side synchronization control unit that adjusts at least one of the frames reproduced by the means;
A synchronous playback device comprising:

(2) provided on the video playback device side;
A measurement signal for measuring a response time in the network is transmitted to the audio reproduction device side via the network, and a response signal returned from the audio reproduction device side according to the measurement signal is received. , Measuring a transmission delay time from transmission of the measurement signal to reception of the response signal, and correcting the value of the audio side synchronization signal received from the audio reproduction device side according to the measured transmission delay time The synchronized playback apparatus according to (1), further comprising a delay correction unit.

  According to the present invention, video and audio can be transmitted between a video playback device and an audio playback device installed separately under a system configuration in which the video playback device and the audio playback device are separately connected to a network. Can be played back synchronously.

  Therefore, it is possible to meet the needs of playing and enjoying the sound with a sound reproducing device capable of reproducing the sound with higher quality.

  Exemplary embodiments of a synchronized playback device according to the present invention will be described below in detail with reference to the accompanying drawings. FIG. 1 is a conceptual diagram showing the overall configuration of the synchronized playback apparatus according to the present embodiment.

(Overall configuration of the device)
The synchronized playback device has a function of synchronizing and reproducing the video and audio by using the video playback device 2 and the audio playback device 3 separately distributed separately on the network. ing. In the present embodiment, the AV server 1 connected to the network 4, the video playback device 2, and the audio playback device 3 (including 3 a to 3 c) are configured and connected to the network 4. The video and audio of the same content are transmitted from the AV server 1 to the video reproduction device 2 and the audio reproduction device 3, and reproduced and output in each device.

  In the present embodiment, the AV server 1 is a device that serves as a content server that distributes content data. For example, terrestrial broadcasting, satellite broadcasting, cable television broadcasting is received, or content data including audio / video is relayed by connecting to the content distribution server 5 on the Internet 42 via the router 44 and the LAN 4. , An apparatus having a function of accumulating and redistributing. Examples of the AV server 1 include a home server and a set top box.

  The video playback device 2 includes a communication interface connected to a network, and by connecting to the AV server 1, the video playback device 2 plays back video included in the content data relayed / distributed by the AV server 1, and outputs and displays it on the monitor 2a. is there. In this description, for convenience, the video playback device 2 is named, but the video playback device 2 may have an audio playback function. As this video reproduction device 2, there is a DVD player / recorder having a function of connecting to the network 4, for example.

  The present invention entrusts the audio reproduction function to the audio reproduction device 3 side capable of reproducing audio with higher quality under such a premise, and the video reproduction and audio reproduction device in the video reproduction device 2. 3 is characterized in that it is synchronized with the audio reproduction in step 3.

  The audio playback device 3 is configured separately from the video playback device 2, has a communication interface connected to the network, and is included in the content data relayed / distributed by the AV server 1 by connecting to the AV server 1. It is a device that reproduces sound and outputs it from speakers, headphones, or the like. Examples of the audio playback device 3 include an audio component, a portable silicon player 3a, a PDA 3b, and a mobile phone 3c that have a function of connecting to the network 4 and can reproduce high-quality audio.

  Note that the AV server 1 and the video playback device 2 may be a single device and may have both functions. In this case, while the video is played back by the AV server / video playback device, the audio of the same content is reproduced and output in synchronization with the audio playback device 3. Similarly, the AV server 1 may be configured to also serve as the audio playback device 3, and the content may be transmitted to the video playback device 2 connected to the network 4, and the audio and video may be played back and output in synchronization. Good. Further, these device operations may be performed by a controller connectable to the network 4.

  When each playback device plays back and outputs, the video played back by the video playback device 2 and the audio played back by the audio playback device 3 are synchronized so that the viewer can determine the timing of the video and audio. Control so that there is no sense of incongruity due to misalignment.

  It should be noted that the content stream may be transmitted separately on the AV server 1 side by separating it into a video stream and an audio stream having the same system clock.

FIG. 2 is a block diagram showing the internal configuration of the AV server 1, the video playback device 2, and the audio playback device 3.
As shown in the figure, the AV server 1 includes a tuner unit 12 that receives content via a television broadcast receiving antenna 43 and the like, and when the received content is an analog signal, the analog signal is digitized and the MPEG2 system or the like The encoder 14 converts the compressed content data into stream data (hereinafter referred to as PS data) in PS (program stream) format, and the PS format stream obtained by the encoder 14. When the data is stored or the content received by the tuner unit 12 is a digital signal, a storage 16 that directly stores stream data (hereinafter referred to as TS data) in the TS (Transport Stream) format that is the digital signal; For storage 16 Includes a storage management unit 13 for managing the input and output of the stream data, a network control unit 11 sends the stream data to the network 4, a stream buffer 15 is a stream data FlFO (First In First Out) buffer.

  The TS format is a format mainly used in communication media such as digital broadcasting and ATM (Asynchronous Transfer Mode) switching networks, and the PS format is mainly used in storage media such as a DVD (Digital Versatile Disc). Format.

  The stream data stored in the AV server 1 is read from the storage 16 in response to a request from the player. The read stream data is sent to the network control unit 11 while being stored in the stream buffer 15, and the network control unit 11 adds the header information of the network protocol (TCP / IP or UDP / IP) to the packet data. To the network 4.

  On the other hand, the video reproduction device 2 receives the packet data transmitted from the AV server 1 from the network 4 and temporarily extracts the stream data extracted by the network control unit 23 and the network control unit 23 that extracts the stream data from the packet data. Stream buffer 27, which is an FIFO buffer to be stored, and a TS packet timing reproduction unit 2001 that reads the stream data stored in the stream buffer 27 at a timing based on control from a synchronization control unit 22 described later and outputs the data to the decoder 26 A decoder 26 for decoding the stream data output from the TS packet timing unit 2001, an output interface 25 for displaying the decoded video on the display 24 and the like, and a TS packet timing reproducing unit 2 And timing of the stream data input and output in 01, and a synchronization control unit 22 for adjusting the timing of the decoding in the decoder 26. The TS packet timing 00 unit 21 is necessary when the stream data input to the video playback device 2 is TS data with a time stamp to be described later. In the case of normal TS data and PS data, unnecessary. Therefore, in the case of normal TS data and PS data, the stream data output from the stream buffer 27 is directly supplied to the decoder 26.

  The decoder 26 is a video decoding unit that decodes at least video data out of audio data and video data included in the stream data. As described above, TS data and PS data are input as stream data. , Have the ability to decode both of these data.

  In the video playback device 2 having such a configuration, the packet data received by the network control unit 23 is converted into stream data by removing the header of each protocol of TCP / IP or UDP / IP, and temporarily stored in the stream buffer 27. Are sequentially read from the stream buffer 37 by the TS packet timing reproduction unit 2001, supplied to the decoder 26, decoded, sent as a video signal through the output interface 25, and displayed on the display 24.

  On the other hand, the audio reproduction device 3 receives the packet data transmitted from the AV server 1 from the network 4 and temporarily extracts the stream data extracted by the network control unit 23 and the network control unit 33 that extracts the stream data from the packet data. A stream buffer 37 that accumulates the data, a TS packet timing reproduction unit 3001 that reads the stream data accumulated in the stream buffer 27 at a timing based on control from a synchronization control unit 32 to be described later, and outputs the data to the decoder 36; A decoder 36 that decodes stream data output from the packet timing reproduction unit 3001, an output interface 35 that outputs decoded audio from the speaker 34, and a stream in the TS packet timing reproduction unit 3001. The timing of the input and output of Mudeta, and a synchronization control unit 22 for adjusting the timing of the decoding in the decoder 36. Note that the TS packet timing reproduction unit 3001 is necessary in the case of TS data with a time stamp, as in the case of the video reproduction device 2.

  In the audio reproducing device 3 having such a configuration, the packet data received by the network control unit 33 is converted into stream data from which the header of each protocol of TCP / IP or UDP / IP is removed, and is temporarily stored in the stream buffer 37. After being accumulated, the TS packet timing reproduction unit 3001 sequentially reads out from the stream buffer 37, supplies it to the decoder 36, decodes it, sends it out as an audio signal through the output interface 35, and outputs it through the speaker 34. .

  For example, when the network used in the system as described above is composed of packet communication represented by Ethernet (registered trademark) and best effort by CSMA / CD (Carrier Sense Multiple Access with Collision Detection), it is associated so that it is output simultaneously. Since the reference clock for synchronizing the recorded video data and audio data cannot be directly transmitted to the video reproduction apparatus 2 and the audio reproduction apparatus 3, reproduction synchronization between these apparatuses is required. Special means are required.

In the following, an embodiment of the means for obtaining the reproduction synchronization according to the present application will be described with reference to FIGS.
FIG. 3 is a block diagram showing the internal configuration of the synchronization control unit 32 on the audio playback device 3 side, and FIG. 4 is a block diagram showing the internal configuration of the synchronization control unit 22 on the video playback device 2 side.

(Voice side)
First, the synchronization control unit 32 on the audio reproduction device 3 side will be described with reference to FIG.

  The synchronization control unit 32 includes a comparator 301 as a phase-locked loop circuit (PLL circuit), a DAC (Digital Analog Converter) 302 that converts a digital signal into an analog signal, and an LPF that is a filter that passes a low-frequency. (Low Pass Filter) 303, VCXO (Voltage Controlled Xtal Oscillator) 304 which is an oscillation circuit for controlling the oscillation frequency, and STC counter 305 are provided.

  In addition, the synchronization control unit 32 in this embodiment includes a VCXO 306, a timing control block 307, and a switch 308 in order to support reproduction of TS data with a time stamp.

  Note that the time stamp in this description refers to the TS format with time stamp described in the second volume of ARIB STD-B24 “Data broadcasting encoding method in digital broadcasting”, and is a PTS (Presentation Time) in the MPEG standard. (Stamp) and DTS (Decoding Time Stamp). “TS with time stamp” is used to record a partial TS, in which only a target program is extracted from an MPEG-TS in which multiple programs are multiplexed, such as broadcasting, on a storage medium such as a storage medium. A 4-byte time stamp indicating the arrival timing of a packet is added so that recovery is possible. The TS with a time stamp indicates a 192-byte MPEG-TS packet obtained by adding these 4 bytes to an 188-byte NPEG-TS packet.

  However, even with this time-stamped TS data, if the size of the stream buffers 27 and 37 in each device is different, the delay time until the stream data is input to the decoders 26 and 36 is different. It is completely out of sync. Further, since the delay times in the decoders 26 and 36 are not usually the same, the synchronization is also shifted.

  The internal operation of the synchronization control unit 32 configured as described above will be described below. When the input data is TS data with a time stamp, this data is supplied to the timing control block. When the input data is normal TS data and PS data without a time stamp, the PCR and SCR (hereinafter referred to as “PCR” and “SCR” respectively) extracted from these data. Both are collectively referred to as a clock reference) and supplied to the B-side terminal of the switch 308.

(For TS data and PS data)
First, the case where the input data is normal TS data and PS data will be described. In this case, 308 is connected to the terminal on the B side.
The clock reference extracted from the stream data received from the AV server 1 is supplied to the STC counter 305 via the B side terminal of the switch 308.

  The STC counter 305 is set to an initial value when the clock reference first arrives, and generates an STC by counting the clocks output from the VCXO 304 constituting the PLL. Then, this STC is supplied to the comparator 301.

The comparator 301 compares the supplied clock reference with the STC value at the time when the clock reference is supplied, calculates an error value, and supplies the error value to the DAC 302.
The DAC 302 generates an error voltage corresponding to the supplied error value and supplies it to the LPF 303.

  The LPF 303 removes the high frequency component from the supplied error voltage and supplies the low frequency component to the VCXO 304 as a control voltage.

  The VCXO 304 outputs a clock whose oscillation center frequency is variably controlled according to the supplied control voltage. The clock output from the VCXO 304 is fed back to the STC counter 305. The clock output from the VCXO 304 is used as a system clock for the decoder 36.

  Then, the decoder 36 decodes the stream data using the system clock generated as described above. At this time, the decoder 36 transmits a PTS (Presentation Time Stamp) associated with the frame at the moment when the audio frame is output to the video reproduction apparatus 2 side through the network 4 as an audio side synchronization signal. In the present embodiment, the audio side synchronization signal is packetized with a PTS value in a UDP (User Datagram Protocol) packet in consideration of a small overhead, and is transmitted as a PTS notification packet.

  Here, the PTS on the audio playback device 3 side is notified by the PTS notification packet when an audio frame associated with the PTS is output when the value of the STC counter matches the PTS on the audio playback device 3 side. This is because it is equivalent to notifying the value of the STC counter on the audio playback device 3 side to the video playback device 2 side. Instead of the PTS on the audio reproduction device 3 side, the STC counter value at the moment when the audio frame is output may be read and notified to the video reproduction device 2 side by a notification packet.

(For TS data with time stamp)
Next, a case where the input data is TS data with a time stamp will be described. In this case, the switch 308 is connected to the terminal on the A side.
The TS data with time stamp is supplied to the timing control block 307.

  The timing control block 307 is a circuit for controlling the timing of fetching this TS data based on the clock signal from the VCXO 306 and supplying this TS data to the decoder when reproducing TS data with a time stamp.

The timing control block 307 extracts the clock reference (PCR) from the TS data together with the above operation, and inputs it to the PLL circuit (comparator 301, DAC 302, LPF 303, VCXO 304, and STC counter 305) described above. The subsequent operation of the PLL circuit is the same as in the case of normal TS data and PS data.
The VCXO 306 is an oscillator that generates a clock used in the timing control block 307.

(Video side)
Next, the synchronization control unit 22 of the video reproduction device 3 will be described with reference to FIG.
The synchronization control unit 22 includes a comparator 205 as a phase-locked loop circuit (PLL circuit), a DAC 207 that converts a digital signal into an analog signal, an LPF 208 that passes a low frequency, a VCXO 209 that controls an oscillation frequency, and an STC counter 212.

  In addition, the synchronization control unit 22 in this embodiment includes a VCXO 201, a timing control block 202, and switches 203 and 204 in order to support reproduction of TS data with a time stamp.

(For TS data and PS data)
Similar to the explanation on the voice side, first, the case where the input data is normal TS data and PS data will be described. In this case, the switch 204 is connected to the terminal on the B side.

  The clock reference extracted from the stream data received from the AV server 1 is input to the comparator 205 via the B side terminal of the switch 204. However, when it first arrives, it is supplied to the STC counter 212 for initialization.

  Subsequent operations of the comparator 205, the DAC 207, the LPF 208, the VCXO 209, and the STC counter 212 in the PLL circuit are almost the same as those of the audio reproduction device 3, and thus the description thereof is omitted, and synchronization with the audio reproduction device 3 is achieved. Therefore, only the configuration added to the video playback device 2 side will be described.

  The synchronization control unit 22 of the video playback device 2 includes a delay correction block 213, an STC difference detection block 211, a comparator 210, and an adder 206 in addition to the components of the PLL circuit.

  The delay correction block 213 transmits a measurement signal for measuring the response time to the audio reproduction device 3 side through the network 4 and receives a response signal returned from the audio reproduction device 3 side according to the measurement signal. This is a module that measures the transmission delay time from the transmission of the measurement signal to the reception of the response signal, and corrects the audio side synchronization signal received from the audio reproduction device 3 side according to the measured transmission delay time.

  Specifically, as shown in FIG. 5, the delay correction block 213 transmits a response time measurement packet as a measurement signal a plurality of times from the video playback device 2 to the audio playback device 3 before starting the playback of the content. After that, the time Tr until receiving the response packet from the audio reproduction device 3 which is a response signal to the response signal is measured. Then, half of this Tr, that is, the average time of Tr / 2 is defined as the transmission delay time. The delay time of the PTS notification packet measured in this way is used for correction as an STC count value converted by the decoder clock frequency.

  Here, in the audio reproduction device 3, there is a possibility that a response delay time Tdl from the reception of the response time measurement packet from the video reproduction device 2 to the transmission of the response packet may occur. This response delay time Tdl is Since the time is considered to be sufficiently smaller than the amount of lip sync that can be detected by humans, a packet for notifying the PTS from the video playback device 2 from the moment when the audio playback device 3 outputs an audio frame with the average time of Tr / 2. There is no problem as time to receive. It is known that the detection limit of this human lip sync gap is about 45 ms for sound advance, about 125 ms for sound delay, and about 90 ms for sound advance and about 185 ms for sound delay in the evaluation experiment in the announcement. In the case of an evaluation experiment using a percussion instrument, it is also known that the sound advance is 23 ms, the sound delay is 56 ms, and the allowable limit is a sound advance of 56 ms and a sound delay of 130 ms.

  The comparator 210 compares the audio-side STC count value (that is, the PTS value) corrected by the delay correction block 213 with the STC count value of the decoder 26 to calculate these differences, and the STC difference detection block 211 performs the comparison. This is a module for detecting whether the difference obtained by the device 210 is smaller than the frame period of the Video signal, whether it is a positive value or a negative value.

  Specifically, the comparator 210 is a module that detects the difference between the corrected STC count value (PTS value) on the audio playback device 3 side and the STC count value on the video playback device 2 side. The STC difference detection block 211 instructs the decoder to skip a frame when the detected difference is a positive value and is larger than the count value corresponding to the video frame period. Control is performed from the STC difference detection block 211 so that frames are continuously output, so that a large time gap between video and audio can be corrected to a synchronized state in a short time.

  In the case of skipping frames, if I and P frames are skipped in MPEG, these frames are a reference source for decoding the subsequent frames, which affects the reproduced image. Therefore, in this embodiment, when skipping, the next B frame is skipped.

  When such frame skipping or continuous output is performed, when the next PCR arrives, the value is set in the STC counter 212 on the video reproduction apparatus 2 side so that the skipped frame is followed. In addition, the value of the STC counter is updated.

(For TS data with time stamp)
Next, a case where the input data is TS data with a time stamp will be described. In this case, the switch 204 is connected to the terminal on the A side. The TS data with time stamp is supplied to the timing control block 202.

  The timing control block 202 is a circuit for controlling the timing of fetching this TS data based on the clock signal from the VCXO 201 and supplying this TS data to the decoder when reproducing TS data with a time stamp.

  The timing control block 202 extracts the clock reference (PCR) from the TS data together with the above operation, and inputs it to the PLL circuit (comparator 205, adder 206, DAC 207, LPF 208, VCXO 209, and STC counter 212) described above. . The subsequent operation of the PLL circuit is the same as in the case of normal TS data and PS data.

  The VCXO 201 is an oscillator that generates a clock used in the timing control block 307.

  The switch 203 is a switch for selectively switching between an error voltage output from the LPF 208 and a fixed value voltage and inputting the voltage to the VCXO 201. When synchronizing with the external audio reproduction apparatus 3 side, the switch 203 Is connected to the C-side terminal and the error voltage from the LPF 208 is input to the VCXO 201. On the other hand, when synchronization is not performed, a switch is connected to the D-side terminal and a fixed value voltage is input to the VCXO 201. The clock of 202 is set to a fixed frequency, and the output of the STC difference detection block 211 is set to a fixed value output of “no STC count error”.

(Process flow)
Next, the flow of processing for realizing synchronized reproduction of video and audio with the above configuration will be described with reference to the sequence diagram of FIG.

A content data distribution request is transmitted from the audio reproduction device 3 or the video reproduction device 2 to the AV server 1 (S101).
When receiving the distribution request, the AV server 1 starts distributing content data (S105).

  In the present embodiment, the distribution of the content data will be described using an example in which data having the same content is transmitted to both the audio reproduction device 3 and the video reproduction device 2, but each of the audio reproduction device 3 and the video reproduction device 2 is provided. On the other hand, each of the video data and the audio data may be transmitted.

  Next, the transmission delay is measured and calculated for delay correction (S102 and S103). This step may be performed in advance before the content data distribution request.

  Next, in the audio playback device 3 and the video playback device 2, content data is received from the network 4 by the network control units 23 and 33, respectively (S104).

Next, the audio reproduction device 3 decodes the audio and transmits an audio side synchronization signal (PTS) extracted at the time of decoding to the video reproduction device 2 at the output timing of the audio.
The video reproduction device 2 receives the audio side synchronization signal (PTS) transmitted from the audio reproduction device 3 (S107).

Next, the video reproduction apparatus 2 side corrects the delay time calculated in S103 (S108).
In this delay correction, the delay time of the PTS notification packet measured and calculated in steps S102 and S103 is added as an STC count value converted by the decoder clock frequency. Then, the comparator 210 detects the difference between the corrected STC count value (PTS value) on the audio playback device 3 side and the STC count value on the video playback device 2 side.

  When the detected difference is larger than the count value corresponding to the video frame period (“Y” in step S109), control is performed from the STC difference detection block 211 to perform frame skipping or continuous frame output to the decoder. Thus, a large time shift between the video and audio is corrected and the synchronization state is maintained (S110).

On the other hand, if the detected difference is smaller than the count value corresponding to the video frame period (“N” in step S109), the processing is continued as it is.
Furthermore, when the content data to be transmitted is TS data with a time stamp, the VCXO 201 generating the clock of the timing control block 202 on the video playback device 2 side also receives the signal from the LPF 208 used for clock control of the decoder. Using the output, the clock of the timing control block 202 is made to follow the clock control of the decoder, the stream data flow rate and the operation clock of the decoder are matched, and the decoding operation is controlled without failure.

Through the processing as described above, on the video reproduction device 2 side, the synchronization control unit 22 operates so as to eliminate the deviation from the audio side synchronization signal (PTS) on the audio reproduction device 3 side, and the video and audio are synchronized. It becomes like this. Further, when the amount of deviation exceeds one frame period, the video playback is operated to follow the audio playback device 3 side by frame skip or continuous frame output as described above.
Therefore, it is possible to eliminate a sense of incongruity due to a difference between the video output timing and the audio output timing.

(Example of change)
The present invention is not limited to the embodiments described above, and various modifications can be made. For example, a plurality of decoders on the video reproduction device side having different sizes, processing capabilities, and decoding methods may be provided, and the decoder may be selected according to the degree of transmission delay detected in the delay correction block.

  Also, the user interface such as the operation panel, operation buttons, remote control, and liquid crystal display provided in the AV server 1, the audio playback device 3, and the video playback device 2 shown in FIG. The user operation signals can be transmitted and received mutually, and the above-mentioned content data distribution request and playback control such as playback / stop can be performed from any of the user interfaces connected to the network 4. These operations can be used as triggers for starting or stopping the lip sync process.

(Action / Effect)
According to the present embodiment described above, in the system configuration including the video reproduction device 2 and the audio reproduction device 3 separately distributed separately on the network 4, the audio side that controls the decoding on the audio reproduction device 3 side. The synchronization signal is acquired on the video reproduction device 2 side, and the video reproduction device 2 side is synchronized with the acquired audio side synchronization signal so that the video reproduction is synchronized with the audio reproduction, and lip synchronization is taken. it can.

  Further, in the present embodiment, the transmission delay time is measured at the time of synchronous reproduction, and the audio side clock signal received from the audio reproduction apparatus side is corrected according to the measured transmission delay time. Transmission delay can be eliminated, and lip sync can be taken with higher accuracy.

It is a conceptual diagram which shows the whole structure of the synchronous reproduction apparatus which concerns on embodiment. FIG. 2 is a block diagram showing an internal configuration of an AV server 1, a video playback device 2, and an audio playback device 3 according to the embodiment. It is a block diagram which shows the internal structure of the audio | voice reproduction apparatus side synchronous control part 32 which concerns on embodiment. It is a block diagram which shows the internal structure of the video reproduction apparatus side synchronous control part 22 which concerns on embodiment. It is explanatory drawing of the transmission delay measuring method which concerns on embodiment. It is a sequence diagram which shows operation | movement of the synchronous reproduction apparatus which concerns on embodiment.

Explanation of symbols

Tdl ... response delay time Tr ... time 1 ... AV server 2 ... video playback device 2a ... monitor 3 ... audio playback device (audio component)
3a ... Silicone player 3b ... PDA
3c ... Cellular phone 4 ... Network 5 ... Content distribution server 11 ... Network control unit 12 ... Tuner unit 13 ... Storage management unit 14 ... Encoder 15 ... Stream buffer 16 ... Storage 2001, 3001 ... TS packet timing reproduction unit 22, 32 ... Synchronization Control unit 23, 33 ... Network control unit 24 ... Display 25 ... Output interface 26, 36 ... Decoder 27, 37 ... Stream buffer 34 ... Speaker 35 ... Output interface 41 ... Wireless LAN router 42 ... Internet 43 ... TV broadcast receiving antenna 44 ... Router 201,306 ... VCXO
202, 307 ... Timing control block 203, 204, 308 ... Switch 205, 301 ... Comparator 206 ... Adder 207, 302 ... DAC
208,303 ... LPF
209, 304 ... VCXO
210 ... Comparator 211 ... STC difference detection block 212,305 ... STC counter 213 ... Delay correction block

Claims (2)

When the video data and the audio data included in the content data are played back independently using the video playback device and the audio playback device that are independently arranged on the network, the video data by the video playback device A synchronized playback device that synchronizes playback and playback of audio data by the audio playback device,
An audio decoding means provided on the audio reproduction device side for decoding audio data of the content data;
Audio-side synchronization control that is provided on the audio reproduction device side and transmits a control value for timing to output an audio signal as a decoding result by the audio decoding means to the video reproduction device side through the network as an audio-side synchronization signal And
Video decoding means provided on the video reproduction device side for decoding the video data of the content data;
Provided on the video playback device side, receives the audio side synchronization signal transmitted from the audio playback device side, and in response to the audio side synchronization signal, the frequency of the clock used by the video decoding means, and the video decoding A video-side synchronization control unit that adjusts at least one of the frames reproduced by the means;
A synchronous playback device comprising: Provided on the video playback device side,
A measurement signal for measuring a response time in the network is transmitted to the audio reproduction device side via the network, and a response signal returned from the audio reproduction device side according to the measurement signal is received. , Measuring a transmission delay time from transmission of the measurement signal to reception of the response signal, and correcting the value of the audio side synchronization signal received from the audio reproduction device side according to the measured transmission delay time The synchronized playback apparatus according to claim 1, further comprising a delay correction unit.