JP2003249922A - Data receiver, method for processing received data and computer program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To enable high quality data reproduction by synchronizing a system clock of a data receiving side (decoding side) with a system clock of a data transmitting side. <P>SOLUTION: In a receiver for a packet attached with a time stamp, for example, a packet following a data communication protocol such as an RTP (real-time transport protocol), a difference of the time stamp attached to the packet is compared with a difference of a counter value of the system clock of a data receiver (decoder), and the system clock of the data receiver is controlled according to a parameter based on the comparison data. By this configuration, it is possible to synchronize the system clock of the data receiving side (decoding side) with the system clock of the data transmitting side even when a delay fluctuation in a network occurs and also in a configuration for performing processing after storing data in a buffer. <P>COPYRIGHT: (C)2003,JPO

Description

Detailed Description of the Invention

[0001]

The present invention relates to a data receiving device,
And a received data processing method, and a computer program. More specifically, in a received packet process based on a time stamp, a data receiving device, a received data processing method, and a computer, which enable a process in which clocks of a data transmitting side and a receiving side are synchronized
Regarding the program.

[0002]

2. Description of the Related Art In recent years, data transfer of images, voice data, and the like has been actively performed through various communication media such as Internet communication. In particular, in recent years, in data transfer on the Internet, in addition to the download type transmission system that has been conventionally used, services based on the stream type transmission system have been increasing. In the download type transmission method, when transmitting multimedia data such as a video file and an audio file, the data file is once downloaded from the distribution server to the storage medium of the receiving side terminal and then reproduced from the storage medium. Therefore, in this method, the file cannot be reproduced until the transfer is completely completed, and it is not suitable for long-time reproduction or real-time reproduction.

On the other hand, in the latter stream type transmission system,
While data is being transferred from the sender to the receiver,
This is to execute the playback processing of the received data in parallel,
It is used for Internet services such as Internet telephony, remote video conferencing, and video on demand.

In the stream type transmission system, for example, an MPEG stream generated by MPEG compression processing of image data is stored in an IP (Internet Protocol) packet and transferred over the Internet, and each communication of PC, PDA, mobile phone, etc. It is used in systems that receive data at terminals and is being developed. Such technology is effective in video-on-demand, streaming distribution of live video, or real-time communication such as video conference and videophone.

When an audio / video image is reproduced in real time by streaming in a video conference or video-on-demand using an IP network, TCP (Tran) is generally used for discarding a packet or retransmitting when a data error occurs.
A transport protocol such as a transport control protocol) cannot be used because the real-time property cannot be maintained. Further, TCP is a protocol for one-to-one communication, and it is not possible to transmit voice / moving image data to a plurality of partners. Therefore, in general, UDP (Us
er Datagram Protocol) is used.

UDP is designed to allow an application process to transfer data to another application process on a remote machine with minimal overhead. Therefore, the information entered in the UDP header is only the source port number, the destination port number, the data length, and the checksum, and the TCP header does not have a field for entering a packet sequence number that exists. If the order of the packets is out of order on the network, processing such as correctly changing the order cannot be performed. Further, neither TCP nor UDP has a field for entering time information such as a time stamp at the time of transmission.

The structure of the TCP header is shown in FIG. The TCP header includes a source port number, a destination port number, a sequence number indicating the data sequence in bytes indicating the number of bytes at the beginning of the data packet from the beginning of the data transmission, and the data sent next from the other party. Response confirmation number indicating transmission sequence number, header length, flag information consisting of code bits such as TCP segment processing method, window size indicating remaining receivable bytes of data,
Checksum as a reliability guarantee value of TCP packet,
It has an urgent pointer indicating a data item that requires urgent processing.

The details of the UDP header are shown in FIG. UDP
Is a protocol that provides connectionless services and has a simple header structure. As shown in FIG. 2, the UDP header has a source port number, a destination port number, a header as a data length, a segment length indicating the total number of bytes of the data length, and a checksum as a reliability guarantee value of the UDP packet. . Since UDP has such a simple configuration, control is simplified.

As described above, in the data transfer to which the UDP protocol is applied, when the order of the packets is out of order in the network, it is impossible to perform the processing such as the correct ordering of the order. Further, neither TCP nor UDP has a field for entering time information such as a time stamp at the time of transmission.

Therefore, RTP (Real-time T
report protocol) was devised. RTP is a transport protocol for transmitting and receiving voice and moving images in real time in an IP network such as the Internet (see RFC1889). Generally, RTP is used over UDP. RTP
Since the header sends a sequence number and a time stamp in the header, there are various merits such as that the receiving side can correct the order of the packets and the delay fluctuation in the network can be absorbed. For example, VoIP (Vo
It is used in the ice over IP) technology and the like.

FIG. 3 shows details of the RTP header in the structure of an IP packet in which data such as an MPEG-2 transport stream (TS) is packetized by RTP, UDP and IP. The RTP header includes a version number (v), padding (P), extension header (X),
Each field of the number of CRSC (Contributing Source), marker information (M), payload type, sequence number, RTP time stamp, synchronous transmission source (SSRC) identifier, and contribution transmission source (CSRC) identifier is provided. The processing time is controlled when the RTP packet is expanded by the time stamp added to the RTP header, and it becomes possible to control the reproduction of the real-time image or sound. Note that, for example, an MPEG transport stream as compressed data has a plurality of MPEs in an IP packet.
It is stored as a G-2-TS packet.

Thus, the protocol: RTP (Realtime
Transport Protocol) is a protocol suitable for a stream type transmission method and is defined by IETF RFC1889.
In data transfer according to RTP, a time stamp is added to a packet as time information, and the time relationship between the transmitting side and the receiving side is grasped by referring to the time stamp, and the data receiving side has a delay fluctuation (jitter) of packet transfer. ) It is possible to play in sync without being affected by.

Also, MPEG2 (Moving Picture Exp), which is an international standard for image compression encoding, is used.
erts Group-2) is a moving image compression method that can be used universally for all applications such as broadcasting, communication, and storage, and is used for DVD and digital television broadcasting.

In the MPEG2 system, MPEG Vi
An encoded bit stream (ES; Elementary Str) such as deo or MPEG Audio
It is possible to integrate a plurality of EAMs and multiplex them into one stream. Further, in the MPEG2 system, as shown in FIG. 4A, MPEG2-PS (P
program stream (one program constitutes one program), and as shown in FIG. 4 (b), M assuming a communication line in which a transmission bit error is expected to occur.
PEG2-TS (Transport Stream)
Two types of formats (one program can configure multiple programs) are defined. ATM and Ethernet
Since the occurrence of transmission errors is unavoidable in networks such as t, MPEG2-TS is generally used for such applications.

The packet structure of MPEG2-TS is shown in FIG. The MPEG2-TS packet is a fixed-length packet composed of a 4-byte header and a 184-byte payload for storing data. The TS header has a 1-byte sync byte (0x47), a flag indicating the attribute of the TS packet, and a 13-bit PID (Packet).
t ID), a scramble control identifier, an adaptation field identifier, and a 4-bit cyclic counter used to check the continuity of packets. Of these, the PID is the audio that constitutes the MPEG2-TS.
In order to distinguish the ES of Video or Video, MPEG2-
It is an identifier attached to each TS packet.

Further, as shown in FIG. 6, a field called an adaptation field can be transmitted in the TS packet as the entire data field portion of the payload or before the data. In this adaptation field, private data and PC
R (Program Clock Reference)
e) etc. can be stored. This PCR
Base 33bit is a counter of 90KHz, extended 9bi
t is a counter that further reduces it to 1/300, and is a time stamp that realizes 27 MHz in total. P
A TS packet having CR in the adaptation field is a P packet different from a video or audio stream.
Have an ID.

The PCR is used by the receiving side of the MPEG2 data to synchronize the system clock of the decoding circuit with that of the transmitting side. This mechanism will be described below.

When the clock frequency of the decoder completely matches the encoder side, the decoding / display of video and audio follows the transmitting side, and the end-to-end delay remains constant over time. However, in reality, they do not match perfectly. Therefore, if communication is continued, the receiving side can
The phenomenon that the display is gradually delayed occurs, which is fatal especially in applications such as videophones.

Therefore, on the decoding side, the system time clock (STC: System) of the decoder is based on the value of the PCR which is the sampled system clock of the encoder.
Time Clock) to follow that of the encoder.
As this method, for example, the PLL (Phas) shown in FIG. 7 is used.
e Locked Loop) is used.

In the PLL, STC is set to the value of the PCR when new MPEG data arrives at the decoder. Then, each time the PCR arrives at the decoder, the value of the PCR is compared with the current STC value output from the counter 104 in the subtractor 101, and the difference is passed to the low pass filter and the gain 102 stage. The output from the output is a voltage-controlled oscillator (VCO).
A control signal for controlling the instantaneous value of the scintillator 103. VCO 10 controlled by it
The output from 3 is used as the new system clock.

By repeating this loop, the system clock on the decoder side eventually coincides with the encoder side. (This state is called locking.) Incidentally, MPEG
In 2-TS, in order to stabilize the operation of this PLL synchronization, it is determined that the PCR transmission interval is 100 msec or less. (700 ms for MPEG2-PS
ec)

As a standard for transmitting MPEG in the IP network using the above-mentioned RTP, "R" is used.
TP Payload Format for MPE
G1 / MPEG2 Video "(RFC2250) exists as a standardized proposed protocol. This RFC describes an RTP packet transmission method and a packet encapsulation method. When MPEG-TS is used as data to be put in the payload part of RTP, 1 is set.
A plurality of MPEG2-TS packets can be put in one RTP packet and carried. Also,
As the RTP time stamp, the time with the same frequency of 90 kHz as the PCR is inserted. That is, as shown in FIG.
The TP time stamp must be synchronized with the PCR base.

As shown in FIG. 9A, MPEG2-T
When S is transmitted using the network 201 having almost no delay fluctuation due to a network such as ATM, the MPEG2-TS data is directly input to the decoder 202. Since the PCR samples and arrives at the encoding system clock, the PLL can be used to synchronize the encoding-side and decoding-side system clocks 203. However, when a large delay fluctuation occurs as in the case of transmission on the IP network such as the Internet as shown in FIG.
In order to remove the delay fluctuation of 1, the reception side must be buffered once using the delay fluctuation absorbing buffer 212 using the RTP. In this case, the system clock on the encoder side is the system clock on the decoder 213 side. 214
However, there is a problem that the method using the PLL cannot be used.

When the transmission data rate is completely constant, as shown in FIG.
Even if the reception side uses the delay fluctuation absorption buffer 222 once to remove the delay fluctuation of 1, the data amount monitoring section 225 looks at the data amount in the buffer and the decoder 223 at the timing of the transmission side. The system clock on the encoder side is changed to the system clock 2 on the decoder 223 side by executing control so that data is input to
Although it is possible to reflect it in H.24, there is a problem that it cannot be applied when the coding rate is dynamically changed on the transmitting side.

[0025]

SUMMARY OF THE INVENTION The present invention has been made in view of the above-mentioned problems, and in the data communication processing for which real-time reproduction is desired, such as video-on-demand or remote video conference, RTP is used. Synchronize the decoder system clock to the encoder side even when the receiver side buffers once to eliminate network delay fluctuations and when the rate changes dynamically. An object of the present invention is to provide a data receiving device, a received data processing method, and a computer program capable of performing the above.

[0026]

The first aspect of the present invention is as follows.
A data receiving device that receives a packet to which a time stamp is added and executes processing of packet stored data,
Time difference data of time stamps added to received packets before and after a predetermined measurement time interval: ΔT
_R and the difference data of the counter value that generates the reference clock timing of the system clock of its own device: ΔT_C, a difference analysis unit that executes a comparison process, and the time difference data of the time stamp acquired by the difference analysis unit: ΔT_R
And a counter value difference data: ΔT_C, a clock control unit that generates a control parameter of the system clock based on the difference, and a clock that is generated based on the control information of the clock control unit in the packet storage data. And a system clock supplied to packet processing means for executing processing.

Further, in an embodiment of the data receiving apparatus of the present invention, the packet processing means comprises a decoder for executing the decoding processing of the packet stored data, and a buffer means for executing the buffering of the packet stored data. It is characterized in that the configuration includes ,.

Furthermore, in one embodiment of the data receiving apparatus of the present invention, the clock control unit is configured to measure a measurement time interval:
When ΔT, based on the difference between the time difference data: ΔT_R of the time stamp acquired by the difference analysis unit and the difference data: ΔT_C of the counter value, D = (ΔT_
It is characterized in that a value: D calculated by R-ΔT_C) / ΔT is generated as a control parameter of the system clock.

Furthermore, in one embodiment of the data receiving apparatus of the present invention, the clock control unit is configured to measure a measurement time interval:
When ΔT, based on the difference between the time difference data: ΔT_R of the time stamp acquired by the difference analysis unit and the difference data: ΔT_C of the counter value, D = (ΔT_
R-ΔT_C) / ΔT, a value: D is generated as a control parameter of the system clock, and D>
When it is 0, control information for accelerating the counter operation and accelerating the system clock is supplied to the system clock,
In the case of D <0, it is characterized in that the counter operation is delayed and a process of supplying control information for delaying the system clock to the system clock is executed.

Further, in one embodiment of the data receiving apparatus of the present invention, the packet with the time stamp is a packet having an RTP header, and the time stamp is stored in the RTP header based on the clock of the packet transmitting side. It is characterized by being stored data.

Further, in an embodiment of the data receiving apparatus of the present invention, the packet with the time stamp is a packet storing an MPEG2 transport stream, and the decoder performs a decoding process of the MPEG2 transport stream. Is executed under the control of the clock signal supplied by the system clock.

Further, in one embodiment of the data receiving apparatus of the present invention, the clock control unit sets the time difference data of the time stamp: ΔT_R and the difference data of the counter value at each predetermined difference measurement time interval. : ΔT_
It is characterized in that the system clock control parameter calculation processing based on the difference from C is repeatedly executed.

Further, a second aspect of the present invention is a received data processing method for receiving a packet with a time stamp and executing processing of packet stored data, which is performed before and after a predetermined measurement time interval. Time difference data of the time stamp added to the packet: ΔT_R
And a difference analysis step of performing a comparison process with a difference data: ΔT_C before and after the measurement time interval of the value of the counter that generates the reference clock timing of the system clock of the own device, and the time stamp acquired in the difference analysis step. Based on the difference between the time difference data: ΔT_R and the difference value of the counter value: ΔT_C, based on the difference, the control parameter generation step of generating the control parameter of the system clock, and the control information generated in the control parameter generation step. A reception comprising: a system clock supplying step of supplying the generated clock to a packet processing means for executing processing of packet stored data; and a packet processing step of executing packet processing based on the clock. Data processing method A.

Further, in one embodiment of the received data processing method of the present invention, the packet processing means is a decoder for executing the decoding processing of the packet stored data, and a buffer means for executing the buffering of the packet stored data. And the buffer means executes output processing of the packet to the decoder based on the clock supplied in the system clock supply step, and the decoder based on the clock supplied in the system clock supply step. It is characterized in that the decoding processing of the packet stored data is executed.

Furthermore, in an embodiment of the received data processing method of the present invention, the control parameter generating step is: when the measurement time interval is ΔT, the time difference data of the time stamps acquired by the difference analysis unit: ΔT_R
And a value of D calculated by D = (ΔT_R−ΔT_C) / ΔT based on the difference between the difference data of the counter and ΔT_C: as a control parameter of the system clock.

Further, in an embodiment of the received data processing method of the present invention, the control parameter generating step is: when the measurement time interval is ΔT, the time difference data of the time stamps acquired by the difference analysis unit: ΔT_R
And a value D calculated by D = (ΔT_R−ΔT_C) / ΔT based on the difference between the difference data of the counter and ΔT_C: D is generated as the control parameter of the system clock, and when D> 0, , Supplying control information to the system clock, which accelerates the counter operation and accelerates the system clock, and when D <0, delays the counter operation and supplies control information to the system clock, which delays the system clock It is characterized by doing.

Further, in one embodiment of the received data processing method of the present invention, the packet with the time stamp is a packet storing an MPEG2 transport stream, and the decoder decodes the MPEG2 transport stream. The processing is executed under the control of a clock signal supplied by the system clock.

Furthermore, in one embodiment of the received data processing method of the present invention, the control parameter generating step includes time difference data of a time stamp: ΔT_R and a counter value for each predetermined difference measurement time interval. The configuration is characterized in that the control parameter calculation processing of the system clock based on the difference from the difference data: ΔT_C is repeatedly executed.

Further, a third aspect of the present invention is a computer program for receiving a packet to which a time stamp is added and executing processing of packet stored data, which is received before and after a predetermined measurement time interval. Time difference data of the time stamp added to the packet:
A difference analysis step of performing a comparison process between ΔT_R and difference data: ΔT_C before and after the measurement time interval of the value of the counter that generates the reference clock timing of the system clock of the own device; and the time stamp acquired in the difference analysis step. Time difference data: ΔT_
A control parameter generation step of generating a control parameter of the system clock based on the difference between R and the difference data of the counter value: ΔT_C, and a clock generated based on the control information generated in the control parameter generation step. To a packet processing means for executing processing of packet stored data, and a packet processing step for executing processing of packets based on the clock. is there.

The computer program of the present invention is, for example, a storage medium or communication medium provided in a computer-readable format for a general-purpose computer system capable of executing various program codes, such as a CD or FD. , MO, etc., or a computer program that can be provided by a communication medium such as a network. By providing such a program in a computer-readable format, processing according to the program is realized on the computer system.

Further objects, features and advantages of the present invention are as follows.
It will be clarified by a more detailed description based on embodiments of the present invention described below and the accompanying drawings. In this specification, the system is a logical set configuration of a plurality of devices, and is not limited to a device in which each configuration is provided in the same housing.

[0042]

BEST MODE FOR CARRYING OUT THE INVENTION According to the present invention, in data communication using a packet with a time stamp, for example, RTP, when buffering is once performed on the receiving side in order to remove delay fluctuation of the network, the rate is dynamically changed. The present invention realizes a data receiving device and a received data processing method having a configuration in which the system clock of the decoder is synchronized with the encoder side even if is changed.

The time stamp added as the RTP header information is, on average, a PCR representing the system clock of the transmitting side even if the delay fluctuation of the network is large.
Is in sync with. Therefore, the timing of this arrival is RT
By observing immediately before the P delay fluctuation absorbing buffer, the system clock on the transmitting side can be sampled. Based on this observation data, two samples on the time axis are sampled, the increment of the RTP time stamp and the increment of the counter due to the system clock on the decoding side are compared, and R
The difference between the system clock on the TP packet transmitting side and the system clock on the RTP packet receiving side is determined.

First, with reference to FIG. 11, the configuration and processing outline of a data receiving side device having a configuration for synchronizing the system clock of the decoder with the encoder side will be described.

For example, data such as MPEG2-TS is transferred to RT.
An example will be given of a configuration in which the packet is stored in a P packet and is transmitted using a network 301 such as the Internet where delay fluctuations occur.

A data packet has a data packet as shown in FIG.
An MPEG over IP packet which stores MPEG2-TS and has an RTP header, a UDP header, and an IP header is assumed. The RTP header stores a time stamp set based on the system clock of the data transmission source.

Returning to FIG. 11, the configuration and processing outline of the data receiving side device will be described. The IP packet storing the MPEG2-TS data received via the network is stored in the buffer 302.
2, the RTP time stamp data acquisition unit 311 acquires time stamp data from the RTP header of each received packet.

RTP time stamp data acquisition unit 311
The time stamp data acquired by is input to the difference analysis unit 312 each time each packet is received. Difference analysis unit 312
Inputs the time stamp data acquired by the RTP time stamp data acquisition unit 311 and also inputs the counter data for generating the reference clock of the system clock 304. FIG. 13 shows an example of the time stamp data acquired by the RTP time stamp data acquisition unit 311 input to the difference analysis unit 312 and the counter data value of the system clock 304. Time stamp data [RTPT_S0] input at each time T_0 to n
[RTPT_S0] and the counter value [CNTR_R_0] of the system clock 304 at the corresponding time
[CNTR_R_n] are sequentially stored in the memory in the difference analysis unit 312.

The difference analysis unit 312 uses a fixed time interval,
That is, after a predetermined difference detection time has elapsed, R
Calculate each difference data of the difference [ΔT_R] between the two time stamp data before and after the difference detection time interval acquired by the TP time stamp data acquisition unit 311 and the difference [ΔT_C] of the counter of the system clock 304 before and after the same time interval. To do.

The difference [ΔT_R] of the time stamp data calculated by the difference analysis unit 312 and the difference [ΔT] of the counter.
Each difference data of _C] is output to the clock control unit 313.

The clock control unit 313 includes a difference analysis unit 31.
Difference of time stamp data calculated by 2 [ΔT_R]
And a control parameter is generated based on each difference data of the difference [ΔT_C] of the counter, and the counter is corrected to generate the reference clock of the system clock based on the generated control parameter.

The formula for generating the control parameter [D] is obtained by the following formula to which each difference data of the difference [ΔT_R] of the time stamp data and the difference [ΔT_C] of the counter is applied. D = (ΔT_R−ΔT_C) / ΔT (Equation 1)

If D = 0, the difference data of the time stamp data [ΔT_R] and the difference data of the counter difference [ΔT_C] are exactly the same, and the system clock of the packet transmitting side and the system clock of the receiving side are synchronized. And
There is no need to adjust the counter. If D> 0, it indicates that the difference [ΔT_R] of the time stamp data is larger than the difference [ΔT_C] of the counter, and the counter of the system clock 304 on the packet receiving side compares with the system clock of the packet transmitting side. Therefore, there is a delay, and the control is executed so as to accelerate the counter of the system clock 304 according to the value of the control parameter D.

If D <0, it indicates that the difference [ΔT_R] of the time stamp data is smaller than the difference [ΔT_C] of the counter, and the counter of the system clock 304 on the packet receiving side indicates the system on the packet transmitting side. This indicates that the clock is faster than the clock, and the control is executed so as to delay the counter of the system clock 304 corresponding to the value of the control parameter D.

Based on the counter modified by the control in the clock control unit 313, the system clock 304 is a clock (27 MH) for the decoder 303.
z) and also provides a clock (90 KHz) for the puffer 302. The RTP time stamp includes the time of the same 90 kHz frequency as the PCR, and each packet is decoded by the decoder 303 by the corrected 90 KHz clock supplied from the system clock 304.
Output to the decoder 303 and the same corrected 2
By executing the decoding process with the 7 MHz clock, it is possible to perform the reproduction process in synchronization with the system clock of the encoder (data transmission side).

Packet transmission / reception between the encoder on the data transmitting side and the decoder on the data receiving side and the counter correction processing on the receiving side will be described with reference to FIG.

FIG. 14 shows the transmitting side for transmitting the packet having the RTP header storing the time stamp data in the upper part, and the receiving side for receiving the packet in a bold manner, and shows the elapsed time (t) from left to right. It is a time sequence diagram shown in FIG.

At the time of packet transmission, the device on the data transmission side sets the RTP counter values [R_0] to [R_N] based on the system clock on the encoder side as the RTP time stamp as the transmission time of each packet. The data is sent in the RTP header (see FIG. 12).

The packet arriving at the receiving side receives RTP counter [C_0] based on the system clock at the decoder side.
It is processed based on [C_N]. That is, [R_0]
-[R_N] are encoder side clocks, and [C_0]-[C_0]
_N] is a value that reflects the decoder side clock,
By comparing the "difference per unit time (time increment)" of these values on the receiving side, it is determined which clock is faster or slower, and based on the difference, the data receiving side Performs clock adjustment (on the decoding side).

By adjusting the clock on the decoding side so as to match the clock on the data transmitting side (encoding side), the synchronization of the clocks on the encoder side and the decoder side is realized. For example, in real-time reproduction processing of moving image data, etc. It is possible to prevent occurrence of reproduction disturbance such as temporary stop of image data.

When there is no delay fluctuation of the RTP packet on the communication network, (R_1-R) in FIG.
_0) = (C_1-C_0) holds, and good reproduction processing of the received packet can be performed without adjusting the clock. However, when there is delay fluctuation of the RTP packet on the communication network, The packet reception interval is different from the time stamp interval given to the packet on the transmission side.

For example, when a delay occurs on the network, a difference per unit time (time increment), that is, an error between the difference between (R_1-R_0) and (C_1-C_0) becomes large. Therefore, in order to reduce the influence of fluctuation, each difference (time increment) is compared by a certain time interval such as a time stamp (R
_0) packet arrival, the time stamp is (R_
It is necessary to set the time interval until the packet arrival of N) as the difference detection time interval before performing the packet detection.

The processing on the data receiving side (decoder side) will be described in detail with reference to FIG. Similar to the configuration described above, the processing packet is assumed to be an MPEG over IP packet which stores MPEG2-TS as shown in FIG. 12 described above and has an RTP header, a UDP header, and an IP header. An example will be given of a configuration in which a packet having an RTP header with a time stamp is transmitted using a network 401 such as the Internet in which delay fluctuation occurs. The RTP header stores a time stamp set based on the system clock of the data transmission source.

For packets received via the network, the RTP time stamp data acquisition unit 411 acquires time stamp data from the RTP header of each received packet before being stored in the buffer 402.

RTP time stamp data acquisition unit 411
The time stamp data acquired by is input to the difference analysis unit 412 every time each packet is received. Difference analysis unit 412
Inputs the time stamp data acquired by the RTP time stamp data acquisition unit 411 and also inputs the counter data for generating the reference clock of the variable frequency oscillator 414 of the system clock 404.

The difference analysis unit 412 sets a certain time interval, for example, the time interval from the arrival of the packet having the time stamp (R_S) to the arrival of the packet having the time stamp (R_E) as the difference detection time interval. ,
The difference between (R_S-R_E) and (C_S-C_E) is detected.

R input to the difference analysis unit 412 in FIG.
An example of the time stamp data acquired by the TP time stamp data acquisition unit 411 and the counter data value for generating the reference clock of the variable frequency oscillator 414 of the system clock 404 is shown. Of the RTP packets having the same RTP time stamp, the moment when the first packet arrives is set as the sample point. At two sample points T_S and T_E (T_S <T_E) on this time axis, reception R
Time stamps RTPT_S and RTPT of TP packet
_E and counter values CNTR_S and CNTR_E according to the system clock on the decoder side.

That is, each time [T_S] and time [T_S]
_S], time stamp data [RTPT] at time [T_E] after the preset difference detection time interval has elapsed.
_S], [RTPT_E], and the counter data [CNTR_S], [CNTR_E], the difference between the times [T_S] and [T_E] is calculated according to the following equation.

RTP time stamp difference (time increment) ΔT_R = RTPT_E-RTPT_S Counter difference (counter value increment) ΔT_C = CNTR_E-CNTR_S

The difference analysis unit 412 calculates each difference data of the difference [ΔT_R] of the time stamp data and the difference [ΔT_C] of the counter of the system clock 304 before and after the same time interval according to the above formula.

The difference [ΔT_R] of the time stamp data calculated by the difference analysis unit 412 and the difference [ΔT] of the counter.
_C] difference data is output to the clock control unit 413, and the clock control unit 413 converts the time stamp data difference [ΔT_R] calculated by the difference analysis unit 312 into the counter difference [ΔT_C] difference data. On the basis of,
Of the counter that generates the control parameter based on the above-mentioned formula 1, that is, D = (ΔT_R−ΔT_C) / ΔT, and generates the reference clock of the variable frequency oscillator 414 of the system clock 404 based on the generated control parameter. Perform correction.

For counter control, if D = 0, no adjustment is made. If D> 0, control to accelerate the counter. If D <0, control to slow down the counter. Is executed.

The variable frequency oscillator 414 of the system clock 404 provides the clock (27 MHz) to the decoder 303 based on the counter modified by the control in the clock controller 413. Further, the frequency divider 415 executes frequency division processing based on the input 27 MHZ, and the 90 KHz clock is input to the buffer 402. RTP time stamp is the same as PCR 90kH
The time according to the frequency of z is inserted, and each packet is output to the decoder 303 by the corrected 90 KHz clock supplied from the system clock 304, and the decoder 30
In the same manner, by performing the decoding process with the same corrected 27 MHz clock in 3, the encoder (data transmission side)
It is possible to perform a synchronized reproduction process using the system clock of.

Next, the processing procedure on the decoder side as the data receiving apparatus will be described with reference to the flowchart of FIG.

The data receiving apparatus waits for the arrival of the RTP packet in step S101. Step S102
At, the time stamp of the arriving RTP packet is set in RTPT_S. Also, a counter value having the same cycle (90 KHz in the case of MPEG2-TS) as the RTP time stamp counted up by the system clock of the own device is set in CNTR_S. Further, the arrival time is set to T_S.

In step S103, the process stands by until the predetermined difference detection time elapses. This waiting process is a process because if the difference detection time interval is too short, it becomes difficult to obtain accuracy due to the influence of delay fluctuation. Note that, depending on how the data transmission side attaches the RTP time stamp, the RTP time stamp having the same value may be added subsequently to different consecutive packets in synchronization with the PCR, so this packet is the head of the RTP time stamp group. If not, wait for reception of next RTP packet (S1
05), and if it is confirmed that it is the head of the RTP time stamp group (S104: Yes), step S106.
Proceed to. .

In step S106, the RTP time stamp of the arrived packet is set in RTPT_E.
Further, a counter value having the same cycle as the RTP time stamp counted up by the system clock on the decoder side is set in CNTR_E. Furthermore, the time is T_E
Set to.

In step S107, the control parameter value [D] is obtained from the above-mentioned [Equation 1], and in step S108, the variable frequency oscillator of the system clock is controlled based on the control parameter value [D].

In step S109, each data update process is executed. That is, RTPT_E is added to RTPT_S.
, CNTR_S to CNTR_E and T_S to T_
Substitute E and proceed to step S103 to repeat the same control.

By repeatedly executing this control processing in sequence, even when delay fluctuations occur in the network, and also when processing is performed after the data is stored in the buffer on the data receiving side, the data receiving side (decoding side) It is possible to synchronize the system clock of 1) with the system clock of the data transmitting side, that is, the time data defined by the time stamp, and for example, real-time reproduction of image data is surely executed. In the above embodiment, the MPEG2-TS stream is given as an example of data stored in the RTP packet.
The process of controlling the system clock by the method described above is performed by M
The same effect is obtained not only in the processing of the stream of PEG2-TS, but also in the processing of transferring other data such as other images and sounds.

FIG. 18 shows an example of the system configuration of a data receiving apparatus that executes the series of processes described in the above embodiment.
The data transmitted / received by the system of the present invention is encoded data, and the data transmitting apparatus executes the encoding (encoding) processing and the data receiving apparatus executes the decoding (decoding) processing. The encoded data is transmitted and received as a packet through the network. Therefore, on the data transmission side, packet generation (packetizing processing) is executed,
The data receiving side executes packet expansion (depacketizing process).

The data receiving apparatus (ex.P shown in FIG.
C) 850 is shown as a data transmission / reception device that also has a data transmission processing function. Therefore encoding
It has a codec 851 that executes processing, decoding (decoding) processing, and packet generation and expansion processing.

Further, the network interface 85 functioning as an interface with the communication network.
2, an input / output interface 853 with an input device such as a mouse 837 and a keyboard 836, an AV interface 85 for performing data input / output with an AV data input / output device such as a video camera 833, a microphone 834, a speaker 835, and the like.
4, a display interface 855 as a data output interface for the display 832, a data input / output interface, a data transfer control between the codec 851 and the network interface 852, and a CPU 856 that executes various program controls, a CP
Storage of various programs controlled and executed by U856,
A memory 857 including a RAM and a ROM that functions as a work area of the CPU 856 for storing data, a HDD 858 as a storage medium for storing data and programs.
And are connected to the PCI bus 859, respectively, and have a configuration capable of mutual data transmission / reception.

As shown in FIG. 18, the codec 851 inputs, for example, image data from the video camera 833 and audio data from the microphone 834, executes coding processing, packet generation processing (packetizing), and finally A packet having encoded data as a payload is generated. The generated packet is output to the PCI bus 859, output to the network via the network interface 852, and delivered to the destination address set in the header of the packet.

Under the control of the CPU 856 according to the software encoding program stored in the HDD 858 or the memory 857, the image data from the video camera 833 and the audio data from the microphone 834 are encoded and output to the network via the network interface 852. It may be configured to execute processing.

On the other hand, IP input via the network
The packetized data is output to the bus 856 via the network interface 852, buffered in the memory 857, and then input to the codec 851. Data (packet) output processing from the memory 857 to the codec 851, and the codec 85
In the decoding process of No. 1, control is executed by the clock signal supplied from the system clock 859 adjusted by the above process.

The codec 851 executes packet expansion processing (depacketizing) of input data, extracts encoded data stored as a payload, executes decoding processing, and reproduces and outputs it on the display 832 and the speaker 835. The output to the codec 851 from the memory 857 as the buffer in this configuration and the process in the codec 851 are controlled by the clock subjected to the adjustment process for synchronizing with the clock on the data transmission side by the clock control described above, and for example, MPEG2-
In the moving image reproduction process based on the TS packet, real-time reproduction synchronized with the data transmission side can be executed without error, and high-quality data reproduction can be performed.

The CPU 856 executes not only the program stored in the memory such as the ROM but also the processing control in Sota. It is also possible to load a program stored in various hard disks, a program transferred from a satellite or a network, received and installed, and the like into a memory such as a RAM (Random Access Memory) to be executed.

In this specification, the program may be processed by one computer or may be processed by a plurality of computers in a distributed manner. Further, the program may be transferred to a remote computer and executed.

The present invention has been described in detail above with reference to the specific embodiments. However, it is obvious that those skilled in the art can modify or substitute the embodiments without departing from the scope of the present invention. That is, the present invention has been disclosed in the form of exemplification, and should not be limitedly interpreted. In order to determine the gist of the present invention, the section of the claims described at the beginning should be taken into consideration.

[0091]

As described above, according to the data receiving device and the received data processing method of the present invention, in the packet receiving process in accordance with the data communication protocol such as RTP with the time stamp, the network receiving process is performed. Even when delay fluctuations occur, or when processing is performed after the data is stored in the buffer on the data receiving side, the system clock on the data receiving side (decoding side) can be synchronized with the system clock on the data transmitting side. For example, real-time reproduction of image data can be reliably performed, and highly reliable data transmission and high-quality data reproduction are possible.

[Brief description of drawings]

FIG. 1 is a diagram showing a TCP header structure.

FIG. 2 is a diagram showing a UDP header structure.

FIG. 3 is a diagram showing an RTP header structure.

FIG. 4 is a diagram illustrating MPEG2-TS (transport stream).

FIG. 5 is a diagram illustrating a transport stream (TS) header structure.

FIG. 6 is a diagram illustrating an adaptation field in a transport stream (TS) packet.

FIG. 7 is a diagram illustrating a control configuration using a PLL.

[Fig. 8] Fig. 8 is a diagram for describing correspondence between RTP time stamps and PCRs in MPEG-TS packets.

FIG. 9 is a diagram illustrating the influence of buffering in packet processing.

FIG. 10 is a diagram illustrating a configuration example for eliminating the influence of buffering in packet processing.

FIG. 11 is a diagram showing a configuration of a data receiving device of the present invention.

FIG. 12 is a diagram showing a packet configuration to be processed in the data receiving apparatus of the present invention.

FIG. 13 is a diagram showing an example of data acquired by a difference analysis unit in the data receiving apparatus of the present invention.

FIG. 14 is a diagram illustrating a processing sequence in the data receiving device of the present invention.

FIG. 15 is a diagram showing a configuration of a data receiving device of the present invention.

FIG. 16 is a diagram showing an example of data acquired by a difference analysis unit in the data receiving apparatus of the present invention.

FIG. 17 is a flowchart illustrating a processing procedure in the data receiving device according to the present invention.

FIG. 18 is a diagram showing a system configuration example of a data receiving apparatus of the present invention.

[Explanation of symbols] 101 subtractor 102 Low-pass filter & gain 103 Voltage Controlled Oscillator (VCO) 104 counter 201 network 202 decoder 203 system clock 211 network 212 buffer 213 decoder 214 system clock 221 network 222 buffer 223 decoder 224 system clock 225 Data amount monitoring unit 301 network 302 buffer 303 decoder 304 system clock 311 RTP time stamp acquisition unit 312 Difference analysis unit 313 Clock control unit 401 network 402 buffer 403 Decoder 404 system clock 411 RTP time stamp acquisition unit 412 Difference analysis unit 413 Clock control unit 414 Variable frequency oscillator 809 PCI bus 832 display 833 video camera 834 microphone 835 speaker 837 mouse 838 keyboard 850 data transceiver 851 codec 852 network interface 853 I / O interface 854 AV interface 855 display interface 856 CPU 857 memory 858 HDD 859 system clock

Continued front page    F-term (reference) 5C059 MA00 PP04 RB02 RB10 RC04                       SS07 SS08 TA00 TC00 TD05                       TD07 TD11 TD17 UA05 UA09                       UA12 UA32                 5K030 GA12 HA08 HB01 HB02 KA21                       LA15                 5K047 AA01 DD01 DD02 MM24 MM46                       MM56

Claims (14)

[Claims] 1. A data receiving device that receives a packet with a time stamp and executes processing of packet storage data, wherein a time difference between time stamps of the received packets before and after a predetermined measurement time interval. Data: ΔT
_R and the difference data of the counter value that generates the reference clock timing of the system clock of its own device: ΔT_C, a difference analysis unit that executes a comparison process, and the time difference data of the time stamp acquired by the difference analysis unit: ΔT_R , Counter value difference data: ΔT_
A clock control unit that generates a control parameter of the system clock based on a difference from C, and a clock processing unit that executes a process of packet stored data with a clock generated based on control information of the clock control unit. And a system clock supplied to the data receiving apparatus. 2. The packet processing means comprises a decoder for executing the decoding processing of the packet stored data, and a buffer means for executing the buffering of the packet stored data. Item 1. The data receiving device according to item 1. 3. The clock control unit is based on a difference between time difference data: ΔT_R of a time stamp acquired by the difference analysis unit and difference data: ΔT_C of a counter value when the measurement time interval is ΔT. The data receiving apparatus according to claim 1, wherein a value: D calculated by D = (ΔT_R−ΔT_C) / ΔT is generated as a control parameter of the system clock. 4. The clock control unit is based on a difference between time difference data: ΔT_R of a time stamp acquired by the difference analysis unit and difference data: ΔT_C of a counter value when the measurement time interval is ΔT. D = (ΔT_R−ΔT_C) / ΔT: a value D is generated as the control parameter of the system clock, and if D> 0, the control information that accelerates the counter operation and accelerates the system clock is generated. 2. The system clock is supplied, and when D <0, the counter operation is delayed, and a process of supplying control information for delaying the system clock to the system clock is executed. Data receiving device. 5. The packet to which the time stamp is added is a packet having an RTP header, and the time stamp is data stored in the RTP header based on the clock of the packet transmitting side. Item 1. The data receiving device according to item 1. 6. The time-stamped packet is a packet storing an MPEG2 transport stream, and the decoder controls the decoding process of the MPEG2 transport stream by a clock signal supplied by the system clock. The data receiving apparatus according to claim 1, wherein the data receiving apparatus is configured to be executed based on the above. 7. The control parameter of the system clock based on the difference between the time difference data of the time stamp: ΔT_R and the difference data of the counter value: ΔT_C, for each predetermined difference measurement time interval. The data receiving apparatus according to claim 1, wherein the data receiving apparatus is configured to repeatedly execute the calculation process. 8. A received data processing method for receiving a packet provided with a time stamp and processing the packet storage data, wherein the time of the time stamp given to the received packet before and after a predetermined measurement time interval. Difference data: ΔT
_R and a difference analysis step of performing a comparison process of difference data before and after the measurement time interval of the value of the counter that generates the reference clock timing of the system clock of the own device: ΔT_C; and a time stamp acquired in the difference analysis step. Based on the difference between the time difference data of ΔT_R and the difference data of the counter value: ΔT_C, and a control parameter generating step of generating a control parameter of the system clock, based on the control information generated in the control parameter generating step. A system clock supply step of supplying a clock generated by the above to a packet processing means for executing processing of packet stored data, and a packet processing step of executing processing of a packet based on the clock. Received data processing Reasoning method. 9. The packet processing means includes a decoder for performing decoding processing of the packet stored data and a buffer means for performing buffering of the packet stored data, and the buffer means supplies the system clock. A step of outputting a packet to the decoder based on the clock supplied in the step, wherein the decoder executes a decoding process of the packet stored data based on the clock supplied in the system clock supplying step. The received data processing method according to claim 8. 10. The control parameter generating step determines the difference between the time difference data: ΔT_R of the time stamp acquired by the difference analysis unit and the difference data: ΔT_C of the counter value when the measurement time interval: ΔT. 9. Based on the above, the value: D calculated by D = (ΔT_R−ΔT_C) / ΔT is generated as the control parameter of the system clock.
The received data processing method described in. 11. The control parameter generating step determines a difference between time difference data: ΔT_R of a time stamp acquired by the difference analysis unit and difference data: ΔT_C of a counter value when the measurement time interval is ΔT. Based on: D = (ΔT_R−ΔT_C) / ΔT: D is generated as a control parameter for the system clock, and if D> 0, control information for accelerating the counter operation and accelerating the system clock is generated. 9. The received data according to claim 8, wherein the system clock is supplied, and when D <0, a process of delaying the counter operation and supplying control information for delaying the system clock to the system clock is executed. Processing method. 12. The packet to which the time stamp is added is a packet storing an MPEG2 transport stream, and the decoder controls the decoding process of the MPEG2 transport stream by a clock signal supplied by the system clock. 9. The received data processing method according to claim 8, wherein the method is executed based on the above. 13. The control parameter generating step controls a system clock based on a difference between time difference data of a time stamp: ΔT_R and difference data of a counter value: ΔT_C at each predetermined difference measurement time interval. The received data processing method according to claim 8, wherein the parameter calculation process is repeatedly executed. 14. A computer program for receiving a packet to which a time stamp is added and executing processing of packet storage data, the time of the time stamp added to the received packet before and after a predetermined measurement time interval. Difference data: ΔT
_R and a difference analysis step of performing a comparison process of difference data before and after the measurement time interval of the value of the counter that generates the reference clock timing of the system clock of the own device: ΔT_C; and a time stamp acquired in the difference analysis step. Based on the difference between the time difference data of ΔT_R and the difference data of the counter value: ΔT_C, and a control parameter generating step of generating a control parameter of the system clock, based on the control information generated in the control parameter generating step. A system clock supply step of supplying a clock generated by the above to a packet processing means for executing processing of packet stored data; and a packet processing step of executing processing of a packet based on the clock. Computer Data program.