JP2008099209A - Content reproducing apparatus and reproduction timing synchronizing method thereof - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To correct out-of-clock synchronization between transmitting and receiving sides while keeping excellent reproduction quality of content data even when a receiving-side clock velocity is lower than a transmitting-side clock velocity. <P>SOLUTION: When a buffering time quantity of video and audio RTP packets in a buffer memory exceeds a threshold value, at mobile terminals MS1-MSm, a packet is detected from among video and audio RTP packets received thereafter, the packet containing a video frame which is positioned just before an I frame and of which the frame size is below a threshold value, and a silent audio frame of which a reproduction time is corresponding to that of the relevant video frame. When video and audio RTP packets meeting these conditions are selected, these packets are discarded from the buffer memory. Furthermore, at the same time, switching processing is performed to sequentially carry time stamps of following video and audio RTP packets. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a content reproduction apparatus that receives and reproduces content data including at least one of a video frame and an audio frame that are packetized and transmitted from a content server, for example, and a reproduction timing synchronization method thereof.

  2. Description of the Related Art In recent years, with the remarkable development of communication technology, a service for distributing content data including video data and audio data from a content server to a content reproduction device via a communication network is provided. By using such a service, the user can receive and play back desired contents such as movies and monitoring data using his / her personal computer, portable terminal, and the like.

  By the way, the personal computer and the portable terminal used by the user as a content reproduction apparatus generally operate with a unique clock, and therefore are not always synchronized with the content server on the transmission side. For this reason, when content is played back for a long time in the content playback device, the phase shift of the clock between the transmission side and the reception side increases, and as a result, the buffering amount of the video frame and the audio frame increases and decreases and the playback delay increases Or buffer underflow.

On the other hand, when the synchronization between data contents, for example, video data and audio data, is out of sync due to fluctuations in the reception timing of data packets, on the other hand, one data is delayed to re-establish between them. Means for achieving synchronization have been proposed (see, for example, Patent Document 1).
JP 2003-204492 A

  However, in the conventional proposal, it is possible to resynchronize the synchronization deviation between the video data and the audio data, but the transmission timing of the content data in the transmission side device and the content data in the content reproduction device on the reception side. The deviation from the reproduction timing cannot be corrected. In addition, as a resynchronization means, a method of delaying one of the data is shown, but in this method, resynchronization is possible when the clock speed of the receiving apparatus is faster than the clock speed of the transmitting apparatus, In addition, resynchronization is difficult when the clock speed of the receiving apparatus is slower than the clock speed of the transmitting apparatus.

  The present invention has been made paying attention to the above circumstances, and the object of the present invention is to reduce the clock synchronization deviation between transmission and reception even when the clock speed of the receiving apparatus is slower than the clock speed of the transmitting apparatus. It is an object of the present invention to provide a content playback apparatus and a playback timing synchronization method thereof that can be corrected while maintaining good playback quality.

  In order to achieve the above object, an aspect of the present invention provides a content reproduction apparatus that receives and reproduces content data including at least one of a video frame and an audio frame that are packetized and transmitted from a transmission side apparatus. A delay time with respect to a reference value of the reproduction timing of the video frame or audio frame is detected. When the detected delay time exceeds a preset value, the information frame is located immediately before the intra-coded frame from the video frames or audio frames received thereafter. Is detected as an adjustment frame, and when this adjustment frame is detected, the adjustment frame is discarded and the playback time of the subsequent frame is advanced. It is a thing.

  Therefore, when the clock speed of the content playback device on the reception side is slower than the clock speed of the transmission side device, if the delay time with respect to the reference value of the playback timing of the video frame or audio frame on the reception side exceeds a preset value, Thereafter, an adjustment frame is selected from video frames or audio frames received thereafter, the adjustment frame is discarded, and the reproduction time of the subsequent frame is advanced. For this reason, the reproduction delay due to the clock delay of the content reproduction apparatus is reduced.

  In addition, as an adjustment frame, a video frame or an audio frame that is located immediately before the intra-coded frame and whose information amount is equal to or less than a threshold value is selected. For this reason, even if the adjustment frame is discarded, the occurrence of discontinuity such as interruption or unnatural change in the reproduced video or reproduced sound is suppressed to a low level, thereby preventing the reproduction quality from being deteriorated.

  In other words, according to the present invention, even when the clock speed of the receiving apparatus is slower than the clock speed of the transmitting apparatus, the content reproduction that can correct the clock synchronization deviation between transmission and reception while maintaining the reproduction quality of the content data is good. An apparatus and a reproduction timing synchronization method thereof can be provided.

Embodiments of the present invention will be described below with reference to the drawings.
(First embodiment)
In the first embodiment of the present invention, in a mobile terminal as a content playback device, the received video RTP packet and audio RTP packet are located immediately before the intraframe-encoded key frame and the frame size is reduced. A packet including a video frame below the threshold and a silent audio frame corresponding to the video frame and the playback time is detected, the detected packet is discarded, and the time stamps of the subsequent video RTP packet and audio RTP packet are recorded. It is made to move up sequentially.

  FIG. 1 is a schematic configuration diagram of a mobile communication system using a content reproduction apparatus according to the first embodiment of the present invention. This system includes a communication network NW having a mobile communication exchange function and a plurality of base stations BS1 to BSn distributed in a service area. Then, by connecting the mobile terminals MS1 to MSm as the content playback devices to the content server SV via the base stations BS1 to BSn and the communication network NW, the content data distributed by the content server SV is transmitted to the mobile terminals MS1 to MSm. It can be received and played back. For example, RTP (Real-time Transport Protocol) is used as a communication protocol used when distributing content data from the content server SV to the mobile terminals MS1 to MSm. RTP is described in detail in the mastering TCP / IP RTP edition by Colin Perkins, translated by Otsumu Odori.

  By the way, the portable terminals MS1 to MSm are configured as follows. FIG. 2 is a block diagram showing the functional configuration. That is, the mobile terminals MS <b> 1 to MSm include a wireless unit 1, a baseband unit 2, a user interface unit 3, and a power supply unit 4.

  In the figure, a radio signal arriving from base stations BS1 to BSn via a radio channel is received by an antenna 11 and then input to a receiving circuit (RX) 13 via an antenna duplexer 12. The reception circuit 13 includes a high frequency amplifier, a frequency converter, and a demodulator. Then, after the radio frequency signal is amplified with a low noise amplifier, the frequency converter mixes the received local oscillation signal generated from the frequency synthesizer (SYN) 14 with the received signal and converts the frequency into a received baseband signal. The signal is digitally demodulated with a demodulator. As the demodulation method, for example, an orthogonal demodulation method and a spectrum despreading method using a spreading code are used. The reception local oscillation signal frequency generated from the frequency synthesizer 14 is instructed from the control module 21 provided in the baseband unit 2.

  The demodulated signal output from the demodulator of the receiving circuit 13 is input to the baseband unit 2. The demodulated signal is separated into a control packet and a data packet by the demultiplexing unit 22. Of these, the data packet is input to the packet reverse conversion unit 28. The control packet includes, for example, a transmission source number, location information of the own terminal and the partner communication terminal, and these are used in the control module 21 for connection control and the like.

  The packet reverse conversion unit 28 separates and extracts video data and audio data from the input data packet, converts these data into information formats that can be decoded, and inputs them to the audio decoder 26 and the video decoder 25. To do. The audio decoder 26 decodes the input audio data by a decoding method corresponding to the designated audio encoding method, and outputs the decoded audio signal from the speaker 34 of the user interface unit 3 as a loud sound. For example, AMR (Adaptive Multi-Rate) is adopted as the speech encoding method. AMR prescribes small-sized data indicating that the encoded speech is silent, separately from speech, so that speech and silence can be distinguished from the size of the encoded speech. It is possible to adopt AAC (Advanced Audio Coder) or MP3 (MPEG-1 Audio Layer 3) instead of AMR. In this case, however, it is silent or the volume is low enough to correspond to silence. Therefore, it is necessary to detect from the content of the data rather than the size of the encoded audio data. The video decoder 25 decodes the input video data in accordance with, for example, MPEG4 (Moving Picture Coding Experts Group 4) system, and the decoded video signal is displayed on a liquid crystal display (LCD) 33 via a display control unit (not shown). To be displayed.

  The camera (CAM) 31 and the microphone 32 of the user interface unit 3 and the video encoder 23 and the audio encoder 24 of the baseband unit 2 are used when videophone communication is performed between the mobile terminals MS1 to MSm. The transmission circuit (TX) 15 of the wireless unit 1 converts the transmission packet generated by the baseband unit 2 into a wireless signal and transmits the wireless signal from the antenna 11 to the base stations BS1 to BSn. As a modulation method, a QPSK (Quadriphase Phase Shift Keying) method or a QAM (Quadrature Amplitude Modulation) method and a spread spectrum method using a spread code are used.

  The power supply unit 4 is provided with a battery 41 such as a lithium ion battery, a charging circuit 42 for charging the battery 41 based on a commercial power output (AC 100 V), and a voltage generating circuit (PS) 43. Yes. The voltage generation circuit 43 is composed of, for example, a DC / DC converter, and generates a predetermined power supply voltage Vcc based on the output voltage of the battery 41.

  By the way, the baseband unit 2 includes a control module 21 that comprehensively controls the operation of the entire terminal, and a memory (MEM) 29 using, for example, a NAND flash memory. The memory 29 stores transmission / reception data such as transmission / reception mail and answering record data, telephone book data, outgoing / incoming call history, and various control data, and further stores content data distributed from the content server SV. The

  The control module 21 includes a central processing unit (CPU) that uses a microprocessor, a ROM that stores application programs, and a buffer memory that temporarily stores transmission and reception packets. As a control function according to the present invention, a delay time detection function 211, an adjustment frame selection function 212, and a reproduction timing adjustment function 213 are provided. All of these functions are realized by causing the CPU to execute an application program.

  The delay time detection function 211 detects a delay time with respect to the reference value of the reproduction timing of the received video frame and audio frame. Specifically, the buffering time amount with respect to the reference value of the video packet and the audio packet in the buffer memory in the control module 21 is detected.

  The adjustment frame selection function 212 is an image received after that when the delay time (buffering time amount) with respect to the reference value detected by the delay time detection function 211 exceeds a preset threshold value. Among the RTP packet and the audio RTP packet, a video frame that is located immediately before the intra-coded video frame (key frame (I frame)) and has a frame size equal to or smaller than a threshold value, the video frame, and the playback time Detects a packet containing a silent voice frame corresponding to. When video RTP packets and audio RTP packets satisfying these conditions are detected, video frames and audio frames included in these packets are selected as playback timing adjustment frames.

  When the adjustment frame selection function 212 selects the adjustment frame, the reproduction timing adjustment function 213 discards the packet including the adjustment frame from the buffer memory. At the same time, a replacement process is performed so that the time stamps (PTS) of subsequent video RTP packets and audio RTP packets are sequentially advanced.

  Next, the content reproduction operation by the mobile terminals MS1 to MSm configured as described above will be described. FIG. 3 is a flowchart showing the control procedure and control contents. Here, an example will be described in which mobile terminal MS1 receives and reproduces content data including video data and audio data distributed from content server SV.

  In the mobile terminal MS1, the video RTP packet and the audio RTP packet received and demodulated by the wireless unit 1 are temporarily stored in the buffer memory in the baseband unit 2. Then, after being read out at the appropriate time adjusted timing, it is depacketized by the packet inverse conversion unit 28, further decoded by the video decoder 25 and the audio decoder 26, and reproduced and output by the LCD 33 and the speaker 35.

  By the way, if the clock speed of the portable terminal MS1 is slower than the clock speed of the content server SV, the buffering time amount in the buffer memory gradually increases from the reference value as shown in FIG. It will exceed. Therefore, the control module 21 monitors the buffering time amount in the buffer memory while performing control for receiving the video RTP packet and audio RTP packet in step S31 as shown in FIG. Then, it is determined in step S32 whether or not the buffering time amount has increased beyond a threshold value. If the result of this determination is that the buffering time amount has not reached the threshold value, the control module 21 decodes the video RTP packet and audio RTP packet to the video decoder 25 and audio decoder 26 in steps S37 and S38. To do.

Then, every time decoding / reproduction processing of one video RTP packet and audio RTP packet is completed, the control module 21 determines in step S39 whether reproduction of all received RTP packets is completed. If an unreproduced RTP packet remains, the process returns to step S31, and RTP packet reception / reproduction control in steps S31 to S38 described above is repeatedly executed.
Therefore, in the above state, the received video RTP packet and audio RTP packet are decoded and reproduced in the order of reception according to the timing indicated by the time stamp.

  On the other hand, it is assumed that the buffering time amount has increased beyond a threshold value. Then, the control module 21 proceeds to step S33 and performs adjustment frame detection processing as follows. That is, each time a video RTP packet and an audio RTP packet are received thereafter, the video frame and the audio frame inserted in the packet are analyzed, and whether or not the frame has a condition as an adjustment frame is determined. judge.

At this time, as a condition,
(1) The received video frame is located immediately before a so-called key frame (I frame) that is intra-coded, and the frame size is equal to or smaller than a threshold value. The threshold value may be set to a value that can be regarded as having few color components, for example.
(2) The audio frame corresponding to the video frame and the playback time is a silent frame.
There are two.

  When a video frame and an audio frame satisfying both of the above two conditions are detected, the control module 21 proceeds from step S34 to step S35 and discards the detected video frame and audio frame as an adjustment frame. To do. At the same time, in step S36, a replacement process is performed so that the time stamps (PTS) of subsequent video RTP packets and audio RTP packets are sequentially advanced.

  For example, it is assumed that the video frame included in the video RTP packet V3 shown in FIG. 5B is an I frame, and the frame size of the video frame included in the video RTP packet V2 immediately before it is less than a certain value. Also, as shown in FIG. 5A, it is assumed that the audio frames included in the audio RTP packets A3, A4, and A5 temporally corresponding to the video RTP packet V2 are all silent frames. Then, the control module 21 selects video frames and audio frames included in the video RTP packet V2 and audio RTP packets A3, A4, and A5 as adjustment target frames, and discards them from the buffer memory. Then, the time stamps tV3, tV4,... Of the subsequent video RTP packets V3, V4,... Are sequentially moved up to tV2, tV3,. At the same time, as shown in FIG. 5A, the time stamps tA6, tA7, tA8 of the subsequent voice RTP packets A6, A7, A8 are moved up to tA3, tA4, tA5, respectively.

  When the adjustment frame discarding and time stamp replacement processing ends, the control module 21 proceeds to steps S37 and S38, and performs decoding and reproduction control of the video frame and audio frame after the adjustment of the reproduction timing. Therefore, the reproduction timing of the video frame and the audio frame is advanced, and as a result, the occurrence of the reproduction delay of the video and audio is prevented.

  Note that, between the video RTP packet V3 positioned immediately after the discarded video RTP packet V2 and the audio RTP packet A6 positioned immediately after the discarded audio RTP packets A3, A4, A5, FIG. In some cases, a time difference ΔT may occur as shown in FIG. In this case, if the packet is discarded and the playback time is simply moved forward, the video RTP packet V3 is scheduled to be played back before the audio RTP packet A6 before the playback timing adjustment. End up.

  Therefore, when the control module 21 decodes and reproduces the video frame included in the video RTP packet V3 and the audio frame included in the audio RTP packet A6, the control module 21 matches the difference between the playback start timings of both frames with the original playback start timing. Therefore, for example, as shown in FIG. 5C, the reproduction timing of the audio frame included in the audio RTP packet A6 is delayed by ΔT from the reproduction timing tV3 of the video RTP packet V3 after the adjustment of the reproduction timing, and the timing t′A3. And Similarly, with respect to each audio frame after the audio RTP packet A7, the reproduction timing is sequentially delayed by ΔT to be t′A4 and t′A5 as shown in FIG. By doing so, the synchronization shift between the video frame and the audio frame is corrected, and the video and audio can be reproduced with higher quality.

  In the period until the video frame and the audio frame satisfying both of the two conditions are detected in step S34, the control module 21 proceeds to steps S37 and S38, and the video RTP packet stored in the buffer memory and Audio RTP packets are decoded and reproduced in the order received.

  As described above, in the first embodiment, in mobile terminals MS1 to MSm, when the buffering time amount of video and audio RTP packets in the buffer memory exceeds a threshold value, video received after that is received. From the RTP packet and the audio RTP packet, a packet including a video frame located immediately before the I frame and having a frame size equal to or smaller than a threshold and a silent audio frame corresponding to the video frame and the reproduction time is detected. When a video RTP packet and an audio RTP packet that satisfy these conditions are selected, these packets are discarded from the buffer memory. At the same time, the replacement process is performed so that the time stamps of the subsequent video RTP packet and audio RTP packet are sequentially advanced.

  Therefore, even when the clock speed of the mobile terminals MS1 to MSm on the receiving side is slower than the clock speed of the content server SV on the transmitting side, there is a possibility that reproduction delay of the received video frame and audio frame may occur. Occurrence of the reproduction delay can be prevented by discarding and time stamp advance processing.

  In addition, as a frame for adjusting the reproduction timing, a video frame and a silent audio frame that are located immediately before the I frame and whose frame size is equal to or smaller than a threshold value are selected. For this reason, even if the adjustment frame is discarded, the interruption and unnatural change of the reproduced video and reproduced audio can be suppressed, and thereby the continuous reproduction of high quality video and audio can be maintained.

(Second Embodiment)
In the second embodiment of the present invention, in the content server on the transmission side, an I frame and a video frame for adjusting a playback timing whose frame size is equal to or smaller than a certain value are periodically inserted in the video RTP packet stream. At the same time, a silent audio frame is inserted at a position temporally corresponding to the adjustment video frame in the audio RTP packet stream. Then, in the mobile terminal on the receiving side, the packet including the adjustment video frame and the silent audio frame is detected from the received video RTP packet and audio RTP packet streams, and the detected packet is discarded. At the same time, the time stamps of subsequent video RTP packets and audio RTP packets are sequentially advanced.

FIG. 6 is a block diagram showing the configuration of the content server SV having a function for realizing the reproduction timing synchronization method according to the second embodiment of the present invention.
The content server SV includes a central processing unit (CPU) 51. The CPU 51 is connected to a program memory 53 and a content server 54 via a bus 52, and further to an encoder 55, a communication interface (communication I / F) 56, and an input interface (input I / F) 57. ing.

  The encoder 55 performs processing for encoding video data and audio data to be distributed under the control of the CPU 51. Under the control of the CPU 51, the communication I / F 56 transmits the video RTP packet and the audio RTP packet to the communication network NW toward the distribution destination mobile terminals MS1 to MSm. A camera 58 and a microphone 59 are connected to the input I / F 57. The input I / F 57 captures a video signal from the camera 58 and a sound signal from the microphone 59 under the control of the CPU 51.

  The content memory 54 stores content data transferred from a content creation source terminal (not shown) and content data read from a storage medium (not shown). Further, if necessary, the encoded data of the video signal and audio signal captured by the input I / F 57 is stored as content data.

The program memory 53 stores a content distribution control program 531 and an adjustment frame insertion control program 532 as control programs for realizing the functions according to the present invention.
The content distribution control program 531 causes the encoder 55 to encode the video signal and the audio signal captured by the input I / F 57, and further converts the video frame and the audio frame generated by the encoding process into an RTP packet. After that, the CPU 51 is caused to execute control to transmit from the communication I / F 56 to the mobile terminals MS1 to MSm as distribution destinations. Further, the CPU 51 is caused to execute control for reading the content data stored in the content memory 54 and converting the content data into an RTP packet and transmitting it.

  The adjustment frame insertion control program 532 causes the CPU 51 to execute the following processing. That is, when generating the video RTP packet and the audio RTP packet, I-frames that are intra-frame encoded are inserted into the video packet stream at regular intervals, and the frame size is set immediately before the I-frame. Insert a video frame for adjusting the playback timing below a certain size. In addition, a silent audio frame is inserted at a position temporally corresponding to the video frame for adjustment in the audio packet stream.

  It should be noted that in general, the place where silence can be inserted is determined by the content of the stream, and therefore video frames and audio frames that satisfy the above conditions may not be inserted in the time interval. In that case, the reproduction timing of a silent audio frame may be detected, and an I-frame encoded within the frame based on the timing may be inserted as appropriate regardless of the time interval.

  Note that the clock accuracy of the mobile terminals MS1 to MSm is about 50 ppm. For this reason, for example, if one frame is an audio frame of 20 msec, it takes 400 seconds (about 7 minutes) until the synchronization loss of one frame occurs, and if one frame is a video frame of 66 msec, it is one frame. It takes about 22 minutes for synchronization to occur. Therefore, the Tc may be set to about 7 minutes or 22 minutes.

  With such a configuration, when the content server SV distributes the content data, for example, as shown in FIG. 7, I frames V2, Vi, Vj,... .. Are further inserted immediately before these I frames V2, Vi, Vj,..., And adjustment video frames V1, Vi-1, Vj-1,. In addition, silent audio frames A1, A2, A3, Am-3, Am are located at positions corresponding to the video frames V1, Vi-1, Vj-1,... For adjustment in the audio RTP packet stream. -2, Am-1, An-3, An-2, and An-1 are respectively inserted.

  On the other hand, in the mobile terminals MS1 to MSm, it is monitored whether or not the buffering time amount has increased beyond the threshold during the reception and reproduction of the video RTP packet and the audio RTP packet. When the amount of buffering time increases beyond the threshold value, an adjustment video frame and a silent audio frame located immediately before the I frame are respectively extracted from the video RTP packet and audio RTP packet streams received thereafter. The containing packet is detected. Then, a process of discarding the detected packet and a process of sequentially incrementing time stamps of the subsequent video RTP packet and audio RTP packet are performed. Note that the procedure and content of the content synchronized playback control by the mobile terminals MS1 to MSm are the same as those shown in FIG. 3 in the first embodiment.

  As described above, in the second embodiment, in the content server SV on the transmission side, the I frame in the video RTP packet stream and the video frame for adjusting the playback timing whose frame size is a certain value or less immediately before the I frame are fixed. In addition to being inserted at a time interval Tc, a silent audio frame is inserted at a position temporally corresponding to the video frame for adjustment in the audio RTP packet stream and transmitted. In the mobile terminals MS1 to MSm on the receiving side, the packet including the adjustment video frame and the silent audio frame is detected from the received video RTP packet and audio RTP packet streams, and the detected packet is detected. And the time stamps of subsequent video RTP packets and audio RTP packets are sequentially incremented, and then decoded and reproduced.

  For this reason, in the mobile terminals MS1 to MSm on the receiving side, when the amount of buffering time increases beyond the threshold, synchronization processing is performed by discarding the adjustment packet and raising the time stamp within the time interval Tc at the longest. Is called. For this reason, more stable reproduction timing synchronization control can be performed without causing a reproduction delay exceeding the allowable amount.

(Other embodiments)
In the first embodiment, the threshold value for determining an increase in the buffering time amount is a fixed value. However, the threshold value may be increased as the difference between the buffering time amount and the reference value increases. . That is, depending on the stream, there may be a case where a video frame having a size lower than the threshold value and a silent audio frame do not appear even though the difference between the buffering time amount and the reference value is large. In such a case, the threshold value is increased as the difference between the buffering time amount and the reference value increases. In this way, it is possible to increase the probability that the size of the video frame exceeds the threshold value, thereby increasing the frequency of adjusting the reproduction timing. As an algorithm for increasing the threshold value, a packet discarding method for avoiding congestion in the packet network can be applied.

  In addition, the configuration of the content server on the transmitting side, the procedure and content of adjustment frame insertion control, the type and configuration of the content playback device, the procedure and content of playback timing synchronization control, and the increase in the amount of buffering time are determined. The specific value of the threshold value, the specific value of the period Tc for inserting the adjustment frame in the transmission side apparatus, and the like can be variously modified and implemented without departing from the gist of the present invention.

  In short, the present invention is not limited to the above-described embodiments as they are, and can be embodied by modifying the components without departing from the scope of the invention in the implementation stage. Various inventions can be formed by appropriately combining a plurality of constituent elements disclosed in the above embodiments. For example, some components may be deleted from all the components shown in each embodiment. Furthermore, you may combine suitably the component covering different embodiment.

It is a figure which shows the structure of the mobile communication system using the content reproduction apparatus concerning 1st Embodiment of this invention. It is a block diagram which shows the structure of the content reproduction apparatus concerning the 1st Embodiment of this invention. 3 is a flowchart showing a procedure and details of content reproduction control by the content reproduction apparatus shown in FIG. 2. It is a figure which shows an example of the time change of the buffering amount in the content reproduction apparatus shown in FIG. FIG. 4 is a timing chart used for explaining the operation of content reproduction control shown in FIG. 3. It is a block diagram which shows the structure of the content server provided with the function which implements the reproduction | regeneration timing synchronization method concerning 2nd Embodiment of this invention. It is a timing diagram for demonstrating the reproduction | regeneration timing synchronization method concerning the 2nd Embodiment of this invention.

Explanation of symbols

  MS1 to MSm ... mobile terminal, NW ... communication network, BS1 to BSn ... base station, SV ... content server, 1 ... wireless unit, 2 ... baseband unit, 3 ... user interface unit, 4 ... power supply unit, 11 ... antenna, DESCRIPTION OF SYMBOLS 12 ... Antenna duplexer (DUP), 13 ... Reception circuit (RX), 14 ... Frequency synthesizer (SYN), 15 ... Transmission circuit (TX), 21 ... Control module, 211 ... Delay time detection function, 212 ... Adjustment frame Selection function, 213 ... reproduction timing adjustment function, 22 ... demultiplexing unit, 23 ... video encoder, 24 ... audio encoder, 25 ... video decoder, 26 ... audio decoder, 27 ... packet conversion unit, 28 ... packet reverse conversion unit, 29 ... Memory (MEM), 31 ... Camera (CAM), 32 ... Microphone 33 ... Liquid crystal display (LCD) 34 ... Speaker 35 ... Input device 41 ... Battery 42 ... Charging circuit (CHG) 43 ... Voltage generation circuit (PS) 51 ... CPU 52 ... Bus 53 ... Program memory, 531 ... Content distribution control program, 532 ... Adjustment frame insertion control program, 54 ... Content memory, 55 ... Encoder, 56 ... Communication interface, 57 ... Input interface, 58 ... Camera, 59 ... Microphone.

Claims (5)

In a content reproduction apparatus that receives and reproduces content data including at least one of a video frame and an audio frame that are packetized and transmitted from a transmission-side apparatus,
Means for detecting a delay time with respect to a reference value of the reproduction timing of the received video frame or audio frame;
When the detected delay time exceeds a preset value, it is located immediately before the intra-coded frame from the video frame or audio frame received thereafter and the amount of information is reduced. An adjustment frame detecting means for detecting a video frame or an audio frame below the threshold as an adjustment frame;
A content playback apparatus comprising: playback timing adjustment means for performing processing for discarding the adjustment frame and advancing the playback time of the subsequent frame when the adjustment frame is detected. When the content data includes a video frame and an audio frame whose playback times correspond to each other,
When the detected delay time exceeds a preset value, the adjustment frame detection means detects a frame immediately before an intra-coded frame from video and audio frames received thereafter. 2. The content according to claim 1, wherein both of a video frame located in a position and having an information amount equal to or smaller than a threshold and a silent audio frame corresponding to the video frame and a reproduction time are detected as adjustment frames. Playback device.   3. The content reproduction apparatus according to claim 1, wherein the adjustment frame detection means includes means for increasing the threshold value in accordance with an increase in the delay time. A playback timing synchronization method for a content playback device that receives and plays back content data including at least one of a video frame and an audio frame that are packetized and transmitted from a transmission side device,
In the transmission side device, when the content data is generated, a process of periodically inserting a video frame or an audio frame whose information amount is equal to or less than a threshold value immediately before the intra-coded frame;
In the content reproduction device, a video that is located immediately before an intra-coded frame and has an information amount equal to or less than a threshold value from among video frames or audio frames included in content data transmitted from the transmission-side device Detecting a frame or audio frame as an adjustment frame;
A reproduction timing synchronization method comprising: a step of performing a process of discarding the adjustment frame and advancing the reproduction time of a subsequent frame when the adjustment frame is detected in the content reproduction apparatus.   When the content data is generated, the transmission side apparatus periodically inserts a video frame whose information amount is equal to or less than a threshold value immediately before the intra-coded frame in the video stream, and the video 5. The reproduction timing synchronization method according to claim 4, wherein a silent audio frame corresponding to the frame and the reproduction time is inserted into the audio stream.

Images