JP2009049919A - Video sound reproduction method and video sound reproducing system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To always synchronize video reproduction and sound reproduction in a video sound reproducing system of a radio communication system, equipped with a server, a video-receiving and reproducing apparatus, and a sound-receiving and reproducing apparatus. <P>SOLUTION: A video-receiving and reproducing apparatus 30 receives encoded video data transmitted from a server 10, writes it in a buffer, and reads it from the buffer and decodes the same. A sound-receiving and reproducing apparatus 50 receives the encoded sound data transmitted from the server 10, writes it in a buffer, and reads it from the buffer and decodes the same. The video-receiving and reproducing apparatus 30 and the sound-receiving and reproducing apparatus 50, respectively detect delay time in the self apparatus buffer and transmit a detection result to the server 10. The server 10 calculates, from its detection result, increase or decrease in the fraction of the delay time in the buffer in the sound-receiving and reproducing apparatus 50 or the target value, and instructs the sound-receiving and reproducing apparatus 50 to adjust the buffer delay time. The sound-receiving and reproducing apparatus 50 adjusts the buffer delay time according to its instruction. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a system for decoding encoded video data and audio data to reproduce video and audio, and a video / audio reproduction method in the system.

  Video data and audio data encoded by an encoding method such as MPEG (Moving Picture Experts Group) 2 recorded on a recording medium or transmitted from a transmission source, such as a DVD reproduction system or a digital television system When video and audio are reproduced by decoding the video, it is necessary to synchronize the reproduction of the video and the reproduction of the audio, that is, the video and the audio are reproduced with a predetermined timing relationship.

  Japanese Patent Application Laid-Open No. 2005-286742 discloses a time stamp recorded on a DVD together with encoded video and audio signals in a DVD player in a system in which a DVD player and a receiving device are connected by a cable. Based on the information, etc., time data for synchronizing the video signal and the audio signal on the receiving side is generated and added to the video signal and audio signal read from the DVD and decoded, and the video signal is encoded. The audio signal is transmitted to the reception device without being encoded, and the reception device decodes the encoded video signal and, based on the time data, the decoded video signal and the received audio. Even if the encoding process before transmission and the decoding process after reception are performed only on the video signal by synchronizing the signal It has been shown to eliminate the time difference between the video signal and the audio signal at the.

  Patent Document 2 (Japanese Patent Laid-Open No. 2007-28261) discloses a PES (Packetized Elementary Stream) including a DTS (Decoding Time Stamp) indicating data decoding timing and a PTS (Presentation Time Stamp) indicating data output (reproduction) timing. ) When decoding an MPEG video / audio stream composed of video data and PES audio data including DTS and PTS, and reproducing the video and audio, when the video is subjected to processing such as resolution conversion after decoding, Rewrite the DTS and PTS included in the PES audio data so as to delay the audio decoding timing and output timing by the processing time delay, and MPEG video and audio By outputting the stream to the MPEG decoder, it has been shown to synchronize the reproduction of the video playback and voice.

The prior art documents listed above are as follows.
JP 2005-286742 A JP 2007-28261 A

  For example, in a conventional home theater system, a video / audio signal source such as a DVD player, a projector (projection type display device) as a video reproduction device, and an audio reproduction device that reproduces multi-channel audio are connected by wire. There was a need to run a cable in the room.

  In contrast, recently, a video / audio signal source such as a DVD player is wirelessly connected to a projector and an audio reproduction device by wireless communication using the IEEE 802.11 standard, Bluetooth (registered trademark), or UWB (Ultra Wide Band). Thus, the home theater system can be constructed as a wireless communication system.

  When the home theater system is constructed as a wireless communication system in this way, it is not necessary to run cables in the room, and individual devices can be arranged flexibly.

  A digital television system can also be constructed as a wireless communication system including a television receiver (tuner unit), a video reproduction device, and an audio reproduction device.

  In this case, a video / audio signal source such as a DVD player or a television receiver transmits the encoded video data and the encoded audio data wirelessly as a server, and a video reproduction device such as a projector serves as a video reception / reproduction device as a server. It is assumed that the encoded video data transmitted from is received and decoded, and the audio reproduction device receives and decodes the encoded audio data transmitted from the server as an audio reception and reproduction device.

  However, when the video / audio reproduction system is constructed as a wireless communication system in this way, the video packet is transmitted from the server to the video reception / playback apparatus by radio waves, and the audio packet is transmitted from the server to the audio reception / playback apparatus. When an obstacle appears between the video reception / playback device and the audio reception / playback device, the radio wave directly coming to the video reception / playback device or audio reception / playback device is blocked, or the radio wave reflected from the wall or ceiling is directly received. The transmission path characteristics between the server, the video reception / playback apparatus, and the audio reception / playback apparatus are always changed by being superimposed on the video.

  Therefore, in the video / audio reproduction system of the above wireless communication system, when a packet cannot be transmitted due to a transmission error due to degradation of transmission path characteristics, the packet is retransmitted, and a temporary decrease in data transmission speed due to the packet retransmission, In order to absorb the change in the delay time until the packet arrives, the video reception / playback device and the audio reception / playback device are each provided with a buffer for temporarily storing the received data. In the video reception / playback device, the code received from the server is provided. The encoded video data is received and written in the buffer, read out from the buffer with a certain time delay and decoded, and the audio reception / playback apparatus also receives the encoded audio data transmitted from the server and writes it in the buffer for a certain period of time. Read and decode from the buffer with a delay.

  Therefore, video playback and audio playback are not synchronized, and in many cases, the delay time (data stay time) in the buffer of the video reception / playback apparatus is longer than the delay time (data stay time) in the buffer of the audio reception / playback apparatus. As a result, the audio is reproduced before the video in terms of timing.

  The method disclosed in Patent Document 1 is a video signal in which a DVD player, which is a transmission device, and a reception device are connected by wire, and an encoded video signal and a decoded audio signal are transmitted to the same reception device. The time difference between the video signal and the audio signal is constant, and the time difference between the video signal and the audio signal is changed on the receiving side due to the change in the transmission path characteristics as in the video / audio reproduction system of the wireless communication system described above. It cannot be applied if it changes.

  Also, the method disclosed in Patent Document 2 does not transmit encoded video data and audio data to separate reception / playback apparatuses by radio, and a time delay with respect to video audio by processing such as resolution conversion is known. It is not possible to apply to the video / audio reproduction system of the wireless communication system.

  Therefore, the present invention is configured to always synchronize the reproduction of the video and the reproduction of the audio in the video / audio reproduction system of the wireless communication system.

The video / audio reproduction method of the present invention comprises:
A server that wirelessly transmits encoded video data and encoded audio data, a video reception / playback device that receives the encoded video data transmitted from the server, writes it in a buffer, reads out the decoded data from the buffer, and reproduces the video And audio / video in a video / audio reproduction system of a wireless communication system including an audio reception / reproduction device that receives and decodes encoded audio data transmitted from the server, writes it in a buffer, reads out the decoded audio from the buffer, and reproduces audio. A playback method,
The video reception / playback device and the audio reception / playback device each detecting a delay time in its own buffer;
Based on the detection result in this detection step, the server, the video reception / playback apparatus, or the audio reception / playback apparatus may increase or decrease the delay time in the buffer of the video reception / playback apparatus or the audio reception / playback apparatus or the target. Calculating a time at which video reproduction at the video reception / playback apparatus and voice reproduction at the audio reception / playback apparatus are synchronized as a value;
Based on the calculation result in this calculation step, the video reception / playback device or the audio reception / playback device adjusts the delay time in its own buffer;
It is characterized by providing.

  According to the present invention, video reproduction and audio reproduction are always synchronized regardless of changes in transmission path characteristics between the server and the video reception / reproduction device and audio reception / reproduction device.

[1. Example of video / audio reproduction system: FIGS. 1 to 4]
(1-1. Overview of the whole system: Fig. 1)
FIG. 1 shows an example of a video / audio reproduction system according to the present invention, in which the video / audio reproduction system is configured as a home theater system.

  Specifically, in the video / audio reproduction system of this example, the server 10 is a DVD player, the video reception / reproduction device 30 is a projector, and the audio reception / reproduction device 50 is a device that reproduces 5.1 channel surround sound.

  The server 10, the video reception / playback device 30, and the audio reception / playback device 50 can perform two-way wireless communication with each other by the IEEE 802.11 standard, Bluetooth (registered trademark), or UWB (Ultra Wide Band). Configure as a thing.

  As a DVD player, the server 10 reproduces video data and audio data encoded by the MPEG2 encoding method from a DVD loaded therein, and transmits the encoded video data to the video reception / reproduction device 30 by streaming. The streamed audio data is streamed to the audio reception / playback apparatus 50.

  The video reception / playback device 30 receives the encoded video data transmitted from the server 10 and writes it in a buffer, reads it from the buffer, decodes it, converts it into an analog video signal, and converts it into an analog video signal as a projector. The video is displayed on the display) and projected on the screen 3 in front of the user 1.

  The audio receiving / reproducing apparatus 50 receives the encoded audio data of each channel transmitted from the server 10, writes it in the buffer, reads it from the buffer, decodes it, and converts it into an analog audio signal.

  The audio signals of the respective channels, that is, left, center, right, rear left (surround left), rear right (surround right) and subwoofer are supplied to corresponding speakers 71, 72, 73, 74, 75 and 76. .

  The connection between the audio reception / playback device 50 and the speakers 71 to 76 is such that all channels are wired by cable, some channels are wired by cable, and other channels are wireless by infrared communication or the like, or All channels are wirelessly connected by infrared communication or the like.

  The server 10, the video reception / playback device 30, and the audio reception / playback device 50 are controlled so that their clocks coincide with each other as the system clock.

(1-2. Server example: FIG. 2)
FIG. 2 shows an example of a DVD player as the server 10.

  In the server 10 of this example, the DVD 11 is rotationally driven by the spindle motor 12, and the encoded video data and the encoded data are read from the DVD 11 by the optical head (optical pickup) 13 which is fed in the radial direction of the DVD 11 by a head feed mechanism not shown in the figure. A video / audio stream signal including audio data is read.

  The read video / audio stream signal is demodulated by the RF amplifier 14 and then separated into encoded video data and encoded audio data by the demultiplexer 15.

  Further, in the wireless communication unit 16, the device address of the video reception / playback apparatus 30 is added as transmission destination identification information to the encoded video data, and the encoded reception data of the audio reception / playback apparatus 50 is added to the encoded audio data. The encoded video data and the encoded audio data are modulated for radio transmission, frequency-converted, and transmitted from the antenna 17 after the device address and the like are added as transmission destination identification information.

  Further, in the server 10, the CPU 21, the ROM 22 in which various programs to be executed by the CPU 21 and necessary fixed data are written, and the RAM 23 in which the programs and data are expanded are connected to the bus 24 to constitute the control unit 20. .

  The control unit 20 controls each unit of the server 10 and executes processing related to adjustment of a buffer delay time as described below.

  The video receiving / reproducing apparatus 30 receives and decodes only the encoded video data and discards the encoded audio data. The audio receiving and reproducing apparatus 50 receives and decodes only the encoded audio data and discards the encoded video data. The server 10 does not need to transmit the encoded video data and the encoded audio data separately as described above.

(1-3. Example of Video Reception / Reproduction Device: FIG. 3)
FIG. 3 shows an example of a projector as the video reception / playback apparatus 30.

  The encoded video data transmitted from the server 10 to the video reception / playback device 30 as described above is received by the antenna 31, frequency-converted by the wireless communication unit 32, demodulated, and temporarily stored in the buffer 33.

  Further, the encoded video data is read from the buffer 33, decoded by the video decoder 34, and converted into an analog video signal by the D / A converter 35.

  The video signal from the D / A converter 35 is supplied to the LCD 37 in the projector optical system 36, the video is displayed on the LCD 37, and the video is projected on the screen 3 as shown in FIG.

  Further, in the video reception / playback apparatus 30, the CPU 41, the ROM 42 in which various programs to be executed by the CPU 41 and necessary fixed data are written, and the RAM 43 in which the programs and data are expanded are connected to the bus 44, and the control unit 40 Composed.

  The control unit 40 controls each unit of the video reception / playback device 30 and executes processing related to adjustment of a buffer delay time as described below.

(1-4. Example of Audio Reception / Reproduction Device: FIG. 4)
FIG. 4 shows an example of the audio reception / playback apparatus 50.

  The encoded audio data of each channel transmitted from the server 10 to the audio reception / reproduction device 50 as described above is received by the antenna 51, frequency-converted and demodulated by the radio communication unit 52, and then temporarily stored in the buffer 53. Is done.

  Further, the encoded audio data of each channel is read from the buffer 53, decoded by the audio decoder 54, and converted into analog audio signals by the D / A converters 551 to 556.

  The audio signals of the respective channels from the D / A converters 551 to 556 are amplified by the amplifier circuits 561 to 566, led out to the output terminals 571 to 576, and supplied to the speakers 71 to 76 as shown in FIG. .

  Further, in the audio receiving / reproducing apparatus 50, the CPU 61, the ROM 62 in which various programs to be executed by the CPU 61 and necessary fixed data are written, and the RAM 63 in which the programs and data are expanded are connected to the bus 64, and the control unit 60 Composed.

  The control unit 60 controls each unit of the audio reception / reproduction device 50 and executes processing related to adjustment of a buffer delay time as described later.

[2. Video and audio playback method: FIGS.
Video packets are sequentially output from the demultiplexer 15 of the server 10 as encoded video data, as shown at the top of FIG. 5A, and as encoded audio data at the top of FIG. 5B. As shown, voice packets are sequentially output.

  Then, the video packet is delayed by the time V1 by the wireless communication unit 16 of the server 10 and transmitted from the server 10 to the video reception / playback device 30 and further received by the wireless communication unit 32 of the video reception / playback device 30. In the process of being written into the buffer 33 and read out from the buffer 33, it is delayed by a time V 2 and input to the video decoder 34.

  Further, the final analog video signal is a process in which the encoded video data input to the video decoder 34 is decoded by the video decoder 34 and further converted into an analog video signal by the D / A converter 35. Is delayed by a time V3 with respect to the encoded video data input to.

Therefore, the delay time Vt from when the video packet is output from the demultiplexer 15 of the server 10 until the video is reproduced and displayed by the video reception / reproduction device 30 is
Vt = V1 + V2 + V3 (1)
It is represented by

  On the other hand, the voice packet is delayed by the time A1 by the wireless communication unit 16 of the server 10 and transmitted from the server 10 to the voice reception / playback device 50, and further received by the wireless communication unit 52 of the voice reception / playback device 50. In the process of being written into the buffer 53 and read out from the buffer 53, it is delayed by a time A 2 and input to the audio decoder 54.

  Further, the final analog audio signal is processed in a process in which the encoded audio data input to the audio decoder 54 is decoded by the audio decoder 54 and further converted into analog audio signals by the D / A converters 551 to 556. The encoded audio data input to the decoder 54 is delayed by a time A3.

Therefore, the delay time At from when the audio packet is output from the demultiplexer 15 of the server 10 to when the audio is reproduced and output by the audio receiving / reproducing apparatus 50 and the speakers 71 to 76 is
At = A1 + A2 + A3 (2)
It is represented by

  Here, the delay times V1 and A1 do not change depending on the transmission path characteristics and the like, and are substantially equal to each other.

  In addition, time stamp information indicating data decoding timing and data output (reproduction) timing, such as DTS (Decoding Time Stamp) and PTS (Presentation Time Stamp), is embedded in the video packet and the audio packet, respectively. Since each of the playback device 30 and the audio reception / playback device 50 compares the time stamp information with the system clock, decodes the encoded video data and the encoded audio data, and outputs an analog video signal and an analog audio signal. If the system clock is accurate, the delay times V3 and A3 are also approximately equal to each other.

  Furthermore, the time until the video packet is transmitted from the server 10 to the video reception / playback device 30, received by the wireless communication unit 32 of the video reception / playback device 30 and input to the buffer 33, and the voice packet is received / played back from the server 10. The time from transmission to the apparatus 50, reception by the wireless communication unit 52 of the audio reception / playback apparatus 50 and input to the buffer 53 is substantially equal to each other.

Therefore, the time from the time when the video data is written to the buffer 33 to the time when the same video data is read from the buffer 33, that is, the delay time (data stay time) in the buffer 33 of the video data is Vb, and the audio data is buffered. Assuming that Ab is the delay time (data stay time) in the buffer 53 of the audio data from the time when the same audio data is read from the buffer 53, that is, the total delay time Vt for the video signal. And Δt, the total delay time At for the audio signal,
Δt = Vt−At = Vb−Ab (3)
It is represented by

  Therefore, if the delay time Vb in the video data buffer 33 and the delay time Ab in the audio data buffer 53 are set to be equal to each other, Δt = 0, and the video reception / audio reception in the video reception / playback apparatus 30 is achieved. The audio playback in the playback device 50 is synchronized.

  FIG. 6 shows the relationship between the data transmission rate (communication rate) and the buffer delay time. The buffer 5 is the buffer 33 of the video reception / reproduction device 30 or the buffer 53 of the audio reception / reproduction device 50.

  When the reception / playback apparatus (video reception / playback apparatus 30 or audio reception / playback apparatus 50) starts receiving data (encoded video data or encoded audio data), the received data is stored in the buffer 5 and the data occupation area of the buffer 5 5a increases and free space 5b decreases.

  As shown in FIG. 6A, the state when the data occupation area 5a reaches a predetermined amount is defined as an initial state.

  When the data transmission speed (corresponding to the data input speed to the buffer 5) and the decoding processing speed (video or audio reproduction processing speed, corresponding to the data output speed from the buffer 5) are the same, this initial state Will be maintained.

  On the other hand, if the transmission path characteristics deteriorate, the number of retransmissions increases, and the data transmission rate decreases, the data inflow amount to the buffer 5 becomes smaller than the data outflow amount from the buffer 5 to the decoder. As shown in FIG. 6B, the state changes to a state in which the data occupation area 5a decreases and the free area 5b increases.

  Conversely, when the data transmission rate is higher than the decoding processing rate, the buffer 5 shifts to a state where the data occupation area 5a increases and the free area 5b decreases as shown in FIG. 6C.

In this case, assuming that the change in the data transmission rate due to the change in the transmission path characteristics is temporary, the delay time Xb in the buffer 5 is obtained by changing the received data amount Dr in the data occupation area 5a by the data average transmission rate Sd. Divided
Xb = Dr / Sd (4)
The received data amount Dr in the buffer 5 and the delay time Xb in the buffer 5 have a proportional relationship.

  Therefore, in the video / audio reproduction system of the above example, the delay time in the buffer 33 of the video reception / playback apparatus 30 is equal to the delay time in the buffer 53 of the audio reception / playback apparatus 50, and Δt = 0. The received data amount in the buffer 33 or the received data amount in the buffer 53 is adjusted, and the delay time in the buffer 33 or the delay time in the buffer 53 is adjusted.

  For this purpose, it is necessary to detect the delay time in the buffer 33 and the delay time in the buffer 53 at that time, and this is performed by the following method.

  In one method, when the video packet is input to the buffer 33, the control unit 40 of the video reception / playback device 30 adds information indicating the time at the end of the video packet, and the video packet is read from the buffer 33. When output, the difference between the time at that time and the time indicated by the time information added to the video packet is calculated as a delay time in the buffer 33, and the control unit 60 of the audio reception / playback device 50 is also the same. In addition, the difference between the time when the voice packet is output from the buffer 53 and the time when the voice packet is input to the buffer 53 is calculated as a delay time in the buffer 53.

  As another method, the control unit 40 of the video reception / playback apparatus 30 and the control unit 60 of the audio reception / playback apparatus 50 measure the delay time for each byte stored in the buffer 33 and the buffer 53, respectively. Also good.

  In the video / audio reproduction system of the above example, unless the buffer delay time is adjusted, in many cases, the delay time in the buffer 33 of the video reception / playback apparatus 30 is the delay time in the buffer 53 of the audio reception / playback apparatus 50. Since it becomes longer, the delay time in the buffer 53 of the audio reception / playback device 50 is adjusted to be equal to the delay time in the buffer 33 of the video reception / playback device 30 as follows.

  In the first method, the server 10 instructs the audio reception / playback device 50 to adjust the delay time in the buffer 53 by indicating the increase / decrease in the buffer delay time or the target value.

  FIG. 7 shows an example of a series of processes relating to the adjustment of the buffer delay time executed by the control unit 20 of the server 10, the control unit 40 of the video reception / playback device 30, and the control unit 60 of the audio reception / playback device 50 in this case. Show.

  In this example, after starting the reproduction of video and audio, the control unit 40 of the video reception / reproduction device 30 detects the delay time in the buffer 33 of its own device in step 131, and sets the delay time in step 132. The information shown is transmitted to the server 10 as data of a specific packet format, and the control unit 60 of the audio reception / reproduction device 50 detects the delay time in the buffer 53 of the own device in step 151, and in step 152 Information indicating the delay time is transmitted to the server 10 as data of a specific packet format.

  In step 111, the control unit 20 of the server 10 receives the delay time information from the video reception / playback device 30 and the audio reception / playback device 50, and sets the buffer delay time indicated by the delay time information from the video reception / playback device 30. When the buffer delay time indicated by the delay time information from Vb and the audio reception / playback apparatus 50 is Ab, Δt = Vb−Ab is calculated as an adjustment amount of the buffer delay time in step 113, and further in step 115. Then, an adjustment command for instructing that the buffer delay time should be increased or decreased by an amount corresponding to the adjustment amount is transmitted to the audio reception / playback apparatus 50.

  The control unit 60 of the audio reception / reproduction device 50 receives the adjustment command in step 157, and further increases or decreases the delay time in the buffer 53 of the own device by the adjustment amount indicated by the adjustment command in step 158. Let

  That is, when Δt is positive, the amount of received data stored in the buffer 53 is increased, and the delay time in the buffer 53 is increased by Δt. When Δt is negative, the amount of received data stored in the buffer 53 is increased. The delay time in the buffer 53 is decreased by | Δt |. As a result, the delay time in the buffer 53 of the audio reception / playback device 50 becomes equal to the delay time in the buffer 33 of the video reception / playback device 30.

  Thereafter, in steps 119, 139, and 159, the control units 20, 40, and 60 determine whether or not to end the reproduction of the video and audio, and reproduce the video and audio by the user's end operation on the server 10 or the like. When the process ends, the process related to the adjustment of the buffer delay time is also completed. Otherwise, the process related to the adjustment of the buffer delay time as described above is repeatedly executed at a predetermined time interval.

  Therefore, even if the transmission path characteristics change and the amount of received data in the buffers 33 and 53 changes as shown in FIG. 6, video and audio are always reproduced in synchronization.

  The control unit 20 of the server 10 transmits an adjustment command to the audio reception / playback device 50 using the delay time in the buffer 33 of the video reception / playback device 30 as a target value of the delay time in the buffer 53 of the audio reception / playback device 50. The system may be configured such that the control unit 60 of the audio reception / playback device 50 adjusts the delay time in the buffer 53 so as to match the target value. In this case, it is not necessary to transmit information indicating the delay time in the buffer 53 from the audio reception / playback apparatus 50 to the server 10.

  In the second method, the server 10 is not involved in adjusting the buffer delay time, and the video reception / playback device 30 indicates to the audio reception / playback device 50 the increase / decrease in the buffer delay time or the target value, and the delay in the buffer 53 is performed. Instruct the time adjustment.

  FIG. 8 shows an example of a series of processes relating to the adjustment of the buffer delay time executed by the control unit 40 of the video reception / playback device 30 and the control unit 60 of the audio reception / playback device 50 in this case.

  In this example, after starting the reproduction of video and audio, the control unit 40 of the video reception / reproduction device 30 detects the delay time in the buffer 33 of the own device in step 131 and controls the control unit of the audio reception / reproduction device 50. 60 detects the delay time in its own buffer 53 in step 151, and further transmits information indicating the delay time to the video reception / playback device 30 in step 153.

  In step 134, the control unit 40 of the video reception / playback apparatus 30 receives the delay time information from the audio reception / playback apparatus 50, and the delay time in the buffer 33 of the own apparatus detected in step 131 is Vb. When the buffer delay time indicated by the delay time information from the device 50 is Ab, in step 135, Δt = Vb−Ab is calculated as an adjustment amount of the buffer delay time, and in step 136, the adjustment amount An adjustment command for instructing that the buffer delay time should be increased or decreased is transmitted to the audio reception / reproduction device 50.

  The control unit 60 of the audio reception / reproduction device 50 receives the adjustment command in step 157, and further, in step 158, similarly to the first method shown in FIG. Increase or decrease by the amount of adjustment indicated by the adjustment command.

  Thereafter, in steps 139 and 159, the control units 40 and 60 determine whether or not to end the reproduction of the video and audio, and when the reproduction of the video and audio is terminated by the user's termination operation or the like at the server 10, The process related to the adjustment of the buffer delay time is also completed. If not, the process related to the adjustment of the buffer delay time as described above is repeatedly executed at predetermined time intervals.

  Therefore, even in the second method, video and audio are always reproduced in synchronization. Moreover, in the second method, the buffer delay time can be adjusted more easily without involving the server 10.

  The control unit 40 of the video reception / playback device 30 transmits an adjustment command to the audio reception / playback device 50 using the delay time in the buffer 33 of the own device as the target value of the delay time in the buffer 53 of the audio reception / playback device 50. The system may be configured such that the control unit 60 of the audio reception / playback device 50 adjusts the delay time in the buffer 53 so as to match the target value. In this case, it is not necessary to transmit information indicating the delay time in the buffer 53 from the audio reception / reproduction device 50 to the video reception / reproduction device 30.

  In the third method, the server 10 does not participate in the adjustment of the buffer delay time, and the audio reception / playback device 50 adjusts the delay time in the buffer 53 regardless of an instruction from the video reception / playback device 30.

  FIG. 9 shows an example of a series of processes related to the adjustment of the buffer delay time executed by the control unit 40 of the video reception / playback device 30 and the control unit 60 of the audio reception / playback device 50 in this case.

  In this example, after starting the reproduction of video and audio, the control unit 60 of the audio reception / reproduction device 50 detects the delay time in the buffer 53 of the own device in step 151, and controls the control unit of the video reception / reproduction device 30. 40 detects the delay time in the buffer 33 of its own device in step 131, and further transmits information indicating the delay time to the audio reception / playback device 50 in step 133.

  In step 154, the control unit 60 of the audio reception / reproduction device 50 receives the delay time information from the video reception / reproduction device 30, and sets the buffer delay time indicated by the delay time information to Vb and the self-detection detected in step 151. When the delay time in the buffer 53 of the apparatus is Ab, in step 155, Δt = Vb−Ab is calculated as an adjustment amount of the buffer delay time, and in step 158, the first method shown in FIG. Similarly, the delay time in the buffer 53 is increased or decreased by the amount of adjustment.

  Thereafter, in steps 139 and 159, the control units 40 and 60 determine whether or not to end the reproduction of the video and audio, and when the reproduction of the video and audio is terminated by the user's termination operation or the like at the server 10, The process related to the adjustment of the buffer delay time is also completed. If not, the process related to the adjustment of the buffer delay time as described above is repeatedly executed at predetermined time intervals.

  Therefore, even in the third method, video and audio are always reproduced in synchronization. Moreover, in the third method, the buffer delay time can be adjusted more easily than in the second method.

[3. Other Embodiments]
In the embodiment described above, the video / audio reproduction system is a home theater system in which the server 10 is a DVD player, the video reception / reproduction device 30 is a projector, and the audio reception / reproduction device 50 is a device that reproduces 5.1 channel surround sound. However, the present invention is not limited to this, and the server receives video / audio data of digital TV broadcasting or video / audio data on the Internet, or reproduces video / audio data recorded on a recording medium such as a hard disk. Then, a device that transmits the encoded video data and the encoded audio data, a video reception / playback device that receives the encoded video data and reproduces the video, and an audio reception / playback device that receives the encoded audio data Therefore, it is suitable for a video / audio playback system that is a device for playing back audio of one channel or multiple channels. It can be.

  In this case, when the video / audio signal is obtained as an analog signal by the server as the video / audio signal source, the server encodes the video / audio signal by the MPEG2 encoding method or the like, and encodes the encoded video data and the encoded audio. The data is transmitted to the video reception / playback apparatus and the audio reception / playback apparatus.

  The present invention can also be applied to the case where the same video / audio reproduction system is used by switching to a different application, and the buffer amounts of the video reception / reproduction apparatus and the audio reception / reproduction apparatus are changed according to the application.

  For example, there is a case where the same video / audio reproduction system is used by switching between movie viewing and video conferencing. In this case, in order to give priority to image quality when used for movie appreciation, the video reception / playback apparatus and audio reception / playback apparatus have a larger buffer, and when used for video conference use, priority is given to responsiveness. Therefore, the buffers of the video reception / playback apparatus and the audio reception / playback apparatus are set to be small. In this case as well, by adjusting the buffer delay time as described above, video playback and audio playback are always performed regardless of whether they are used for movie viewing or video conferencing. Can be synchronized.

  Further, the present invention comprises a plurality of video reception / playback devices or audio reception / playback devices, and each of the video reception / playback devices or audio reception / playback devices receives encoded video data or encoded audio data transmitted from a server. The present invention can also be applied to a video / audio reproduction system in which data is written into a buffer, read from the buffer, and decoded.

1 is a diagram illustrating an example of a video / audio reproduction system according to the present invention. It is a figure which shows an example of a server. It is a figure which shows an example of an image | video receiving / reproducing apparatus. It is a figure which shows an example of an audio | voice receiving / reproducing apparatus. It is a figure where it uses for description of the delay of a video signal and an audio | voice signal. It is a figure where it uses for description of buffer delay time. It is a figure which shows an example of a series of processes of the 1st method which concerns on adjustment of buffer delay time. It is a figure which shows an example of a series of processes of the 2nd method which concerns on adjustment of buffer delay time. It is a figure which shows an example of a series of processes of the 3rd method which concerns on adjustment of buffer delay time.

Explanation of symbols

  Since all the main parts are described in the figure, they are omitted here.

Claims (11)

A server that wirelessly transmits encoded video data and encoded audio data, a video reception / playback device that receives the encoded video data transmitted from the server, writes it in a buffer, reads out the decoded data from the buffer, and reproduces the video And audio / video in a video / audio reproduction system of a wireless communication system including an audio reception / reproduction device that receives and decodes encoded audio data transmitted from the server, writes it in a buffer, reads out the decoded audio from the buffer, and reproduces audio. A playback method,
The video reception / playback device and the audio reception / playback device each detecting a delay time in its own buffer;
Based on the detection result in this detection step, the server, the video reception / playback apparatus, or the audio reception / playback apparatus may increase or decrease the delay time in the buffer of the video reception / playback apparatus or the audio reception / playback apparatus or the target. Calculating a time at which video reproduction at the video reception / playback apparatus and voice reproduction at the audio reception / playback apparatus are synchronized as a value;
Based on the calculation result in this calculation step, the video reception / playback device or the audio reception / playback device adjusts the delay time in its own buffer;
A video / audio reproduction method comprising: The video / audio reproduction method according to claim 1,
The calculation step is executed by the server, and the server indicates the calculation result and instructs the video reception / playback apparatus or the audio reception / playback apparatus to adjust the buffer delay time. Method. The video / audio reproduction method according to claim 1,
The calculation step is executed by one of the video reception / playback device and the audio reception / playback device, and the one device indicates the calculation result and instructs the other device to adjust the buffer delay time. A video / audio reproduction method comprising: The video / audio reproduction method according to claim 1,
The calculation step is performed by one of the video reception / playback device and the audio reception / playback device, and the one device executes the adjustment step. The video / audio reproduction method according to claim 1,
The encoded audio data is audio data of a plurality of channels, and the audio receiving / reproducing apparatus reproduces audio of a plurality of channels. A server that wirelessly transmits encoded video data and encoded audio data, a video reception / reproduction device that receives the encoded video data transmitted from the server, writes it in a buffer, reads out the decoded data from the buffer, and reproduces the video And a video / audio reproduction system of a wireless communication system including an audio reception / reproduction device that receives encoded audio data transmitted from the server, writes the encoded audio data in a buffer, reads out and decodes the buffer from the buffer, and reproduces audio. ,
The video reception / playback device and the audio reception / playback device each include means for detecting a delay time in the buffer of the device itself,
The server, the video reception / playback device, or the audio reception / playback device, based on a detection result by the detection means, an increase / decrease amount of a delay time or a target value in the buffer of the video reception / playback device or the audio reception / playback device And means for calculating a time for synchronizing the video playback in the video reception / playback apparatus and the audio playback in the voice reception / playback apparatus,
The video reception / playback apparatus or the audio reception / playback apparatus includes means for adjusting a delay time in a buffer of the own apparatus based on a calculation result by the calculation means.
A video / audio reproduction system characterized by that. The video / audio reproduction system according to claim 6,
The calculation means is provided in the server, and the server indicates the calculation result and instructs the video reception / playback apparatus or the audio reception / playback apparatus to adjust the buffer delay time. . The video / audio reproduction system according to claim 6,
The calculation means is provided in one of the video reception / playback device and the audio reception / playback device, and the one device indicates the calculation result and instructs the other device to adjust the buffer delay time. A video / audio reproduction system characterized by that. The video / audio reproduction system according to claim 6,
The video / audio reproduction system, wherein the calculating means and the adjusting means are provided in one of the video reception / playback apparatus and the audio reception / playback apparatus. The video / audio reproduction system according to claim 6,
The encoded audio data is audio data of a plurality of channels, and the audio receiving / reproducing apparatus reproduces audio of a plurality of channels. A server that wirelessly transmits encoded video data and encoded audio data, a video reception / playback device that receives the encoded video data transmitted from the server, writes it in a buffer, reads out the decoded data from the buffer, and reproduces the video And a video / audio reproduction system of a wireless communication system including an audio reception / reproduction device that receives and decodes encoded audio data transmitted from the server, writes it in a buffer, reads out the decoded data from the buffer, and reproduces audio. Computer
Means for detecting a delay time in a buffer of the video reception / playback device and the audio reception / playback device;
Based on the detection result, as the increase / decrease in the delay time in the buffer of the video reception / playback apparatus or the audio reception / playback apparatus or the target value, the video playback at the video reception / playback apparatus and the audio reception / playback apparatus Means for calculating a time for synchronization with audio reproduction; and
Means for adjusting a delay time in the buffer of the video reception / playback device or the audio reception / playback device based on the calculation result;
A program for video / audio reproduction, characterized in that it functions as a program.

Images