JP2008141633A - Data communication system, data-receiving apparatus and method, and data transmitting apparatus and method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To prevent waste of redundant transmission of a retransmission packet, without increasing the load of a network and a data communication system. <P>SOLUTION: A data transmission apparatus transmits a retransmission packet, when a retransmission request information received is related to a first retransmission request for a lost packet (ST13 and ST14). When the information is related to a retransmission request for a second time or after, the data transmission apparatus obtains the transmission time of the latest received packet from a transmission information storage part, based on the latest received packet information contained in the retransmission request information (ST17), and calculates the latest delay time taking the difference between the transmission time and the current time (ST17). Furthermore, the data transmitting apparatus transmits a retransmission packet, only when a time longer than the latest delay time has passed from the latest retransmission time of the lost packet to the current time (ST18 and ST14). When the time that has elapsed, starting from the latest retransmission time of the lost packet to the current time, is shorter than the latest delay time, there is the possibility is large that the received retransmission request information has been transmitted, before the arrival of the retransmission packet that has not been lost by the data-receiving apparatus, based on the transmission round-trip delay times which are measured regularly. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a data communication system, a data reception device, a data reception method, a data transmission device, and a data transmission method. Specifically, in the present invention, the data receiving apparatus transmits retransmission request information including lost packet information and latest received packet information to the data transmitting apparatus, and the data transmitting apparatus transmits the latest delay time (transmission based on the latest received packet information). A data communication system that prevents unnecessary redundant transmission of retransmission packets by not retransmitting lost packets when the latest delay time or more has not elapsed from the latest retransmission time of the lost packet to the current time. Etc.

  When video or audio data is transmitted over an IP (Internet Protocol) network, a streaming transmission method is used in which data is reproduced while being received. In particular, when playback time control or real-time performance is required, a streaming method in which data packetized by RTP (Realtime Transport Protocol) is transmitted by UDP (User Datagram Protocol) is suitable. Since this method does not have TCP (Transmission Control Protocol) rate control or lost packet recovery control as it is, rate control using RTP control information, retransmission control to recover lost packets, redundant code control, etc. have been developed. Is being used.

In retransmission control, the basic operation is to detect a lost packet at the receiving device, notify the transmitting device of the packet information (sequence number) as a retransmission request, and retransmit the requested packet. . Furthermore, since there is a possibility that the retransmitted packet may be lost again, and in the streaming method, the data needs to arrive by the timing of data reproduction processing. Therefore, the network delay is measured separately and calculated to be in time for the reproduction time. Further, a device for requesting and transmitting a retransmission packet has been devised (see Patent Document 1, Patent Document 2, and Patent Document 3).
JP 2002-84338 A JP 2003-169040 A JP 2005-136547 A

  In the prior art described in Patent Documents 1 to 3, the network delay is periodically measured, and retransmission is determined by the reception device or the transmission device using the measured value. However, in the situation where packet loss occurs, congestion and deterioration of link quality such as radio have occurred, and the actual delay value was measured during normal times due to increased buffering delay of each device in the network. It has been observed that the value is much larger than the value.

  For example, due to buffering delay due to congestion, the delay, which was normally 10 milliseconds, becomes 20-30 milliseconds, or the link state of a wireless LAN (Local Area Network) deteriorates temporarily. May be. In such a case, in the prior art, in order to determine retransmission based on a small measurement value before fluctuation, a retransmission packet that is not in time for the reproduction time of the reception device, or a duplicate transmission before the retransmission packet transmitted before is processed by the reception device. The retransmitted packet is transmitted as a redundant packet, and there is a problem that the network condition is further deteriorated.

  In order to avoid the transmission of retransmission packets that become such redundant packets, it is sufficient to reduce the delay measurement interval in the prior art as well, but the amount of measurement data required separately increases. There is a problem that the load on the network and the data communication system is increased.

  An object of the present invention is to prevent useless duplicate transmission of retransmission packets without increasing the load on the network and the data communication system.

The concept of this invention is
A data communication system comprising a data transmitting device and a data receiving device and transmitting packetized streaming data,
The data receiving device is:
A packet receiving unit that receives the packet of the streaming data transmitted from the data transmitting device;
A depacketizer for analyzing the packet of the streaming data received by the packet receiver and reconstructing the streaming data;
A packet loss detection unit that detects a lost packet based on the packet of the streaming data received by the packet reception unit;
A retransmission request generator for generating retransmission request information including information on a lost packet detected by the packet loss detector and information on a latest received packet received by the packet receiver;
A retransmission request transmission unit that transmits the retransmission request information generated by the retransmission request generation unit to the transmission device;
The data transmission device is
A packetizer for packetizing the streaming data;
A packet transmission unit that transmits the packet of the streaming data generated by the packetizer to the data reception device;
A transmission information storage unit for storing transmission information including at least the transmission time of each packet transmitted by the packet transmission unit;
A retransmission request receiving unit that receives the retransmission request information transmitted from the data receiving device;
Based on the latest received packet information included in the retransmission request information received by the retransmission request receiving unit, the transmission time of the latest received packet is acquired from the transmission information storage unit, and the difference between the transmission time and the current time A delay time calculation unit that calculates the latest delay time by taking
A packet retransmission control unit that performs retransmission control of the loss packet,
The packet retransmission control unit
When the retransmission request information received by the retransmission request receiving unit relates to the first retransmission request of the lost packet indicated by the lost packet information included in the retransmission request information, from the transmission time of the lost packet Regardless of the time until the current time, the packet transmission unit is controlled to transmit the lost packet to the data receiving device,
Included in the retransmission request information when the retransmission request information received by the retransmission request receiving unit relates to a second or subsequent retransmission request of the lost packet indicated by the lost packet information included in the retransmission request information. Based on the lost packet information, the latest retransmission time of the lost packet is acquired from the transmission information storage unit, and the latest delay time calculated by the delay time calculation unit has elapsed from the latest retransmission time to the current time. In the data communication system, the packet transmission unit is controlled to transmit the loss packet to the data receiving device.

  In the present invention, the data transmitting apparatus packetizes streaming data such as image data and audio data, and then transmits the packetized data to the data receiving apparatus via the network. The data receiving apparatus analyzes the received packet and reconstructs the streaming data. The data transmission device stores transmission information including at least the transmission time of each packet to be transmitted to the data reception device in the transmission information storage unit.

  Based on the received packet, the data receiving device detects a lost packet from, for example, missing sequence numbers. The data receiving apparatus generates retransmission request information including information on the lost packet (for example, sequence number) and information on the latest received packet (for example, sequence number), and transmits the retransmission request information to the data transmitting apparatus.

  In this case, the data reception device may generate retransmission request information when the lost packet reproduction time is a time after the time after the transmission round-trip delay time from the current time. When the reproduction time of the lost packet is a time before the time after the transmission round-trip delay time from the current time, the retransmission packet sent from the data transmission device in response to the retransmission request information is from the reproduction time of the packet. This is because there is a high possibility that the data arrives at a later time and is wasted.

  Further, in this case, when the data receiving apparatus cannot receive the predetermined loss packet within the transmission round-trip delay time from the time when the retransmission request information of the predetermined loss packet is transmitted, the data reception device displays the retransmission request information of the predetermined loss packet. It may be generated again. This is because, when a predetermined loss packet cannot be received within the transmission round trip delay time, there is a possibility that the predetermined loss packet sent from the data transmission apparatus according to the retransmission request information is lost again.

  The data transmitting apparatus receives retransmission request information transmitted from the data receiving apparatus. The data transmitting apparatus determines whether or not the received retransmission request information relates to the first retransmission request for the lost packet indicated by the lost packet information included therein. In the case of the first retransmission request, the data transmitting apparatus retransmits the lost packet to the data receiving apparatus regardless of the time from the transmission time of the lost packet to the current time.

  In the case of the second or subsequent retransmission request, the data transmitting apparatus acquires the transmission time of the latest received packet from the transmission information storage unit based on the latest received packet information included in the retransmission request information, and this transmission time The latest delay time is calculated by taking the difference between the current time and the current time. This latest delay time corresponds to the time required for round-trip transmission of the latest received packet. The data transmitting apparatus retransmits the lost packet to the data receiving apparatus when the latest delay time or more has elapsed from the latest retransmission time of the lost packet to the current time. When the latest delay time or more has not elapsed from the latest retransmission time to the current time of the lost packet, the received retransmission request information is based on the above-described transmission round-trip delay time before the arrival of the retransmission packet not lost by the data reception device. This is because there is a high possibility that it was transmitted.

  According to this invention, the data receiving apparatus transmits retransmission request information including lost packet information and latest received packet information to the data transmitting apparatus, and the data transmitting apparatus obtains the latest delay time based on the latest received packet information, and Since the lost packet is not retransmitted when the latest delay time or more has not elapsed from the latest retransmission time of the packet to the current time, it is possible to prevent unnecessary duplicate transmission of the retransmission packet.

  Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 shows a configuration of a data communication system 100 as an embodiment. The data communication system 100 includes a data transmission device 110 and a data reception device 120. The data transmission device 110 and the data reception device 120 are connected via a network (IP network) 130.

  The data transmission device 110 packetizes streaming data such as image data and audio data, and transmits an RTP packet to the data reception device 120. When there is a lost packet in the received RTP packet, the data reception device 120 transmits a retransmission request packet (retransmission request information) to the data transmission device 110 to request retransmission of the lost packet.

  FIG. 2 shows the configuration of the data transmission device 110. The data transmission device 110 includes a packetizer 111, a transmission buffer 112, an RTP data communication unit 113, a transmission information storage unit 114, a control data communication unit 115, and a retransmission request processing unit 116. The data source outputs streaming data as transfer data. The data source is, for example, a video camera, and outputs image data and audio data. The transfer data may be reproduction data such as a CD (Compact Disc) or DVD (Digital Versatile Disc), or data received from an external network, a broadcasting satellite, or the like. In the following description, the description will focus on the configuration for transferring image data from a data source from the data transmission device 110 to the data reception device 120.

  The packetizer 111 packetizes image data from the data source. The image data is encoded data that has been encoded by, for example, MPEG compression processing. The packetizer 111 generates a data packet according to RTP (hereinafter simply referred to as “packet”). This RTP is defined by IETF RFC1889. In data transfer according to RTP, a type stamp is added to a packet as time information, the time relationship between the transmitting side and the receiving side is grasped by referring to the time stamp, and the packet transfer delay fluctuation ( Synchronized playback is possible without being affected by jitter).

  The packetizer 111 executes a process for generating a packet using the encoded data as a payload. A packet is formed by adding an RTP packet header to the payload data. FIG. 3 shows the format of the RTP packet. RTP header includes version (V), padding (P), presence / absence of extension bit (X), number of transmission sources (Counter), marker information (marker_bit), payload type (Payloadtype), sequence number, time stamp, synchronization source Fields of (source) identifier (SSRC) and contribution source (source) identifier (CSRC) are provided.

  On the data receiving side, the processing time is controlled when the RTP packet is expanded based on the time stamp added to the RTP header, thereby enabling real-time image or audio reproduction control. In the RTP packet storing the encoded data of the image data, a common time stamp is set for a plurality of RTP packets belonging to one image frame, and the end packet constituting each frame is the end. An identification flag is stored in the RTP header.

  The transmission buffer 112 temporarily accumulates the RTP packets generated by the packetizer 111 and sequentially transmits each RTP packet to the IP network through the RTP data communication unit 113. The transmission buffer 112 retransmits the designated packet to the IP network through the RTP data communication unit 113 in response to the retransmission request from the retransmission request processing unit 116. In addition, the transmission buffer 112 stores the transmission packet and transmission information of the transmission packet in the transmission information storage 114. The transmission information includes at least the transmission time of the transmission packet.

  In this embodiment, the transmission information of the transmission packet includes the transmission time and the number of retransmissions. FIG. 4 shows an example of transmission information stored in the transmission information storage unit 114. The transmission time and the number of retransmissions are stored in association with the sequence number of the transmission packet. The transmission time indicates the time when the transmission packet is transmitted to the IP network, and the number of retransmissions indicates the number of retransmissions according to the retransmission request. When the transmission packet is retransmitted, the transmission time and the number of retransmissions are updated. For example, in FIG. 4, when the transmission packet of sequence number 5 is transmitted at time t5 and then the first retransmission is performed at time t5 ′, the transmission time is changed from “t5” to “t5 ′”. And the number of retransmissions is updated from “0” to “1”.

  The RTP data communication unit 113 transmits the RTP packet supplied from the transmission buffer 112 to the IP network after adding an IP header. The transmission buffer 112 and the RTP data communication unit 113 described above constitute a packet transmission unit.

  FIG. 5 shows the format of the IP header. Version indicating IPv4, IPv6, etc., header length, TOS (Type of Service) field storing priority information, packet length, packet identifier, control information on data division (fragment) at IP layer Flag, fragment offset indicating the location of the divided (fragmented) data, TTL (Time to Live) indicating time information until the data is discarded, protocol (4: IP, TCP: 7, UDP: 17...), Header checksum, source IP address, destination IP address.

  The control data communication unit 115 performs RTCP (Real-time Transport Control Protocol) packet communication with the data reception device 120. This RTCP is defined by IETF RFC1889. The control data communication unit 115 receives, for example, a NACK (Negative ACKnowledge) -RTCP packet sent from the data receiving device 120. This NACK-RTCP packet has retransmission request information. The control data communication unit 115 constitutes a retransmission request receiving unit.

  FIG. 6 shows a format of the NACK-RTCP packet. The RTCP header includes fields for version (V), padding (P), payload type (PT), packet length (length), synchronization source (source) identifier (SSRC), and name (Application name). It has been. In the case of a NACK-RTCP packet, the payload type (PT) is 204. The name is the name of an application program that handles NACK-RTCP packets. Retransmission request information (NACK) is arranged in the NACK payload.

  FIG. 7 shows the format of the NACK payload in this embodiment. The first 32 bits are provided with the transmission source identifier (Data source's SSRC) of the RTP packet requesting retransmission, and the subsequent 16 bits are provided with the sequence number (Seq_lost) of the lost packet (lost RTP packet). In 16 bits, the sequence number (Seq_last_recv) of the latest received packet (the latest received RTP packet at the time when the retransmission request information is created) is provided.

  In the example of FIG. 7, only one pair of the lost packet sequence number (Seq_lost) and the latest received packet sequence number (Seq_last_recv) is shown. However, when a plurality of lost RTP packets are requested collectively. Are provided with a plurality of pairs.

  When the control data communication unit 115 receives a NACK-RTCP packet, the retransmission request processing unit 116 determines whether or not to retransmit the RTP packet requested to be retransmitted in the retransmission request information, and determines that it should be retransmitted. At this time, the RTP packet retransmission request is sent to the transmission buffer 112. The retransmission request processing unit 116 constitutes a packet retransmission control unit.

  When the NACK-RTCP packet is received by the control data communication unit 115, the retransmission request processing unit 116 relates to the first retransmission request of the RTP packet for which retransmission is requested by this retransmission request information. Judge whether there is. When making the determination, the retransmission request processing unit 116 uses the sequence number (Seq_lost) of the lost packet included in the retransmission request information, and is stored from the transmission information storage unit 114 corresponding to the sequence number (Seq_lost). To determine the number of resends

  When the retransmission request processing unit 116 determines that the number of retransmissions is 0 and is related to the first retransmission request, the retransmission request processing unit 116 immediately retransmits the RTP packet with the sequence number (Seq_lost) included in the retransmission request information. Then, a retransmission request is sent to the transmission buffer 112. The retransmission request processing unit 116 performs the following process when determining that the number of retransmissions is 1 or more and is related to the second and subsequent retransmission requests.

  The retransmission request processing unit 116 uses the sequence number (Seq_last_recv) of the latest received packet included in the retransmission request information, and acquires the transmission time stored in correspondence with the sequence number (Seq_last_recv) from the transmission information storage unit 114 Then, taking the difference between the transmission time and the current time, the latest delay time (transmission round-trip delay time) is calculated. In this case, retransmission request processing section 116 constitutes a delay time calculating section.

  Then, the retransmission request processing unit 116 uses the sequence number (Seq_lost) of the lost packet included in the retransmission request information, and the transmission time stored in correspondence with the sequence number (Seq_lost) from the transmission information storage unit 114 ( Latest retransmission time) is acquired, and it is determined whether or not the latest delay time or more has elapsed from the latest retransmission time to the current time. The retransmission request processing unit 116 requests the transmission buffer 112 to retransmit the RTP packet with the sequence number (Seq_lost) included in the retransmission request information when the latest delay time or more has elapsed from the latest retransmission time to the current time. Send.

  The flowchart in FIG. 8 shows a retransmission control processing procedure in the retransmission request processing unit 116.

  When the control data communication unit 115 receives a NACK-RTCP packet requesting retransmission of a lost packet, the retransmission request processing unit 116 starts retransmission control processing in step ST11. In step ST12, retransmission request processing section 116 acquires lost packet information, that is, the sequence number Seq_lost of the lost packet, from retransmission request information (NACK) (see FIG. 7).

  Next, in step ST13, retransmission request processing section 116 determines whether or not the received retransmission request information relates to the first retransmission request for the lost packet indicated by the lost packet information included therein. To do. In this case, the number of retransmissions of the lost packet is acquired from the transmission information storage unit 114 using the sequence number Seq_lost of the lost packet acquired in step ST12 as an identifier.

  When the retransmission request processing unit 116 determines in step ST13 that the request is related to the first retransmission request, in step ST14, the retransmission request processing unit 116 transmits a retransmission request for the lost packet to the transmission buffer 112, and the lost packet is transmitted to the data receiving device. 120. In step ST15, the retransmission request processing unit 116 updates the information on the transmission time and the number of retransmissions of the loss packet in the transmission information storage unit 114, and then ends the retransmission control process in step ST16.

  When the retransmission request processing unit 116 determines in step ST13 that the request is related to the second and subsequent retransmission requests, the retransmission request processing unit 116 proceeds to step ST17. In step ST17, the retransmission request processing unit 116 acquires the latest received packet information from the retransmission request information (NACK) (see FIG. 7), that is, the sequence number Seq_last_recv of the latest received packet, and transmits the sequence number Seq_last_recv as an identifier. The transmission time of the latest received packet is acquired from the information storage unit 114, and the latest delay time (transmission round-trip delay time) is calculated by taking the difference between this transmission time and the current time.

  In step ST18, the retransmission request processing unit 116 acquires the transmission time (latest retransmission time) of the lost packet from the transmission information storage unit 114 using the sequence number Seq_lost of the lost packet acquired in step ST12 as an identifier. It is determined whether the latest delay time or more has elapsed from the latest retransmission time to the current time.

  When the latest delay time or more has not elapsed from the latest retransmission time to the current time, the retransmission request processing unit 116 moves to step ST16 and immediately ends the retransmission control process. That is, in this case, retransmission of the lost packet is not performed.

  If the latest delay time or more has elapsed from the latest retransmission time to the current time, the retransmission request processing unit 116 moves to step ST14, and in the transmission buffer 112, as in the case of the first retransmission request described above. The retransmission request for the lost packet is sent, and the lost packet is transmitted to the data receiving device 120. In step ST15, the retransmission request processing unit 116 updates the information on the transmission time and the number of retransmissions of the loss packet in the transmission information storage unit 114, and then ends the retransmission control process in step ST16.

  The retransmission request processing unit 116 performs a retransmission control process according to the flowchart of FIG. 8, so that the first time retransmission request does not depend on the time from the transmission time of the lost packet to the current time. Is retransmitted to the data receiving apparatus 120, and for the second and subsequent retransmission requests, the lost packet is retransmitted to the data receiving apparatus 120 only when the latest delay time or more has elapsed from the latest retransmission time of the lost packet to the current time. To do.

  FIG. 9 shows the configuration of the data receiving device 120. The data receiving apparatus 120 includes an RTP data communication unit 121, a reception buffer 122, a depacketizer 123, a retransmission request generation unit 124, a retransmission request list storage unit 125, and a control data communication unit 126.

  The RTP data communication unit 121 receives an RTP packet sent from the data transmission device 110 via the network (IP network) 130. The reception buffer 122 temporarily stores RTP packets received by the RTP data communication unit 121. The RTP data communication unit 121 and the reception buffer 122 constitute a packet reception unit. The depacketizer 123 analyzes the RTP packet accumulated in the reception buffer 122. The depacketizer 123 analyzes the header and payload in the RTP packet, and reconstructs the streaming data before packetization. Streaming data (encoded data) output from the depacketizer 123 is decoded by a decoder (not shown) to display an image.

  The control data communication unit 126 performs RTCP packet communication with the data transmission device 110. The control data communication unit 126 transmits the NACK-RTCP packet (see FIGS. 6 and 7) generated and transmitted by the reproduction request generation unit 124 to the data transmission device 110 as will be described later. In this case, the control data communication unit 126 constitutes a retransmission request transmission unit.

  In addition, the control data communication unit 126 transmits an RTCP packet for delay measurement to the data transmission device 110 in order to measure a transmission round trip delay time (RTT: Round Trip Time) by the retransmission request generation unit 124 and transmits the data. The RTCP packet for delay measurement sent from the device 110 is received. Although illustration is omitted, in the RTCP packet for delay measurement, the time when the data receiving apparatus 120 transmits the RTCP packet is arranged in the payload.

  The data transmitting apparatus 110 immediately sends back the RTCP packet for delay measurement sent from the data receiving apparatus 110 to the data receiving apparatus 120 without changing the transmission time. Therefore, the retransmission request generation unit 124 can measure the transmission round trip delay time by subtracting the current time from the transmission time in the delay measurement RTCP packet received by the control data receiving unit 126.

  The data receiving device 120 periodically measures such a transmission round trip delay time. However, in order to measure the transmission round-trip delay time, as described above, it is necessary to communicate RTCP packets for delay measurement between the data receiving apparatus 120 and the data transmitting apparatus 110, which is a control overhead. Therefore, this transmission round-trip delay time is performed at long intervals of, for example, several hundred milliseconds to several seconds. Note that the transmission round trip delay time is not measured by the data receiving device 120 as described above, but by the data sending device 110, and the data receiving device 120 notifies the data sending device 110 of the information on the transmission round trip delay time. You may be asked to do it.

  The retransmission request generation unit 124 detects a lost RTP packet based on the RTP packet received by the RTP data communication unit 126 and accumulated in the reception buffer 122. In this case, the retransmission request generation unit 124 monitors the sequence number of the received RTP packet, and when there is a missing sequence number, determines that the missing RTP packet with the missing sequence number has been lost. In this case, the retransmission request generation unit 124 constitutes a packet loss detection unit.

  The retransmission request generator 124 generates a NACK-RTCP packet (see FIGS. 6 and 7) requesting retransmission of a lost packet (lost RTP packet), and sends it to the control data communication unit 126. In addition, the retransmission request generation unit 124 stores the retransmission request list in the retransmission request list storage unit 125. FIG. 10 shows an example of the retransmission request list. In association with the sequence number of the loss packet for which a retransmission request has been made, a retransmission packet arrival time and a reproduction (decode) time are stored.

  The scheduled arrival time of the retransmission packet is a time obtained by adding the above-described transmission round trip delay time to the transmission time of the NACK-RTCP packet requesting retransmission. When a new NACK-RTCP packet is transmitted for a certain lost packet, the retransmission packet arrival scheduled time changes. The reproduction (decoding) time is obtained from time stamps of RTP packets located before and after the loss packet.

  When the retransmission request generation unit 124 generates a NACK-RTCP packet for a predetermined loss packet and sends it to the control data communication unit 126, the retransmission request generation unit 124 cannot receive the predetermined loss packet even if the transmission round-trip delay time elapses. The NACK-RTCP packet is generated again and sent to the control data communication unit 126. This is because when a predetermined loss packet cannot be received within the transmission round-trip delay time, the packet transmitted from the data transmission device 110 in response to the retransmission request information (NACK) may be lost again.

  In addition, the retransmission request generation unit 124, when the loss packet reproduction (decoding) time is a time after the above-described transmission round-trip delay time (scheduled packet arrival time) after the current time, A NACK-RTCP packet (see FIGS. 6 and 7) requesting retransmission is generated and sent to the control data communication unit 126. Therefore, the retransmission request generation unit 124 does not create a NACK-RTCP packet that requests retransmission of the lost packet when the reproduction time of the lost packet is a time before the retransmission packet arrival scheduled time from the current time.

  When the reproduction time of the lost packet is earlier than the scheduled arrival time of the retransmission packet, the lost packet sent from the data transmitting apparatus 110 corresponding to the retransmission request information (NACK) is later than the reproduction time of the packet. This is because there is a high possibility that the data will be received by the data receiving apparatus 110.

  The flowchart in FIG. 11 shows a processing procedure for retransmission request control in the retransmission request generator 124.

  When the RTP data communication unit 121 receives an RTP packet, the retransmission request generation unit 124 starts retransmission request control processing in step ST21. Then, retransmission request generation section 124 determines whether or not there is a packet loss in step ST22. In this case, the retransmission request generation unit 124 monitors the sequence number of the received RTP packet, and determines that there is a packet loss when there is a missing sequence number.

  When determining that there is no packet loss, the retransmission request generation unit 124 immediately moves to step ST27 and ends the retransmission request control process. If retransmission request generating section 124 determines that there is a packet loss, it proceeds to step ST23. In step ST23, the retransmission request generation unit 124 acquires the sequence number Seq_lost and the reproduction (decoding) time of the lost packet (lost RTP packet). In this case, the retransmission request generation unit 124 acquires the sequence number Seq_lost of the lost packet from the sequence number of the RTP packet before and after the lost packet, and reproduces the lost packet from the time stamp of the RTP packet before and after the lost packet ( Decode) Get the time.

  Next, in step ST24, retransmission request generation section 124 determines whether or not retransmission of the lost packet is in time for the reproduction (decoding) time. In this case, the retransmission request generator 124 retransmits the lost packet when the lost packet reproduction time is a time after the transmission round-trip delay time (RTT) from the current time (scheduled arrival time of the retransmission packet). Is determined to be in time for the playback time.

  If retransmission request generating section 124 determines that retransmission of the lost packet is not in time, it immediately moves to step ST27 and ends the retransmission request control processing. That is, in this case, a NACK-RTCP packet that requests retransmission of the loss packet is not generated. On the other hand, if retransmission request generating section 124 determines that the retransmission of the lost packet is in time, it moves to step ST25. In step ST25, the retransmission request generation unit 124 acquires the sequence number Seq_last_recv of the latest received packet (the latest RTP packet among the RTP packets received by the RTP data communication unit 121).

  In step ST <b> 26, the retransmission request generation unit 124 generates a NACK-RTCP packet that requests retransmission of the lost packet, and sends the NACK-RTCP packet to the data transmission device 110 through the control data communication unit 126. In the payload of the NACK-RTCP packet, as shown in FIG. 7, the sequence number Seq_lost of the lost packet and the sequence number Seq_last_recv of the latest received packet are arranged. In step ST26, the retransmission request generation unit 124 registers information related to the retransmission request (loss packet sequence number, retransmission packet arrival time, reproduction (decoding) time) in the retransmission request list (FIG. 10). reference).

  The retransmission request generation unit 124 moves to step ST27 after the process of step ST26 and ends the retransmission request control process.

  The retransmission request generation unit 124 performs a retransmission request control process according to the flowchart of FIG. 11 to detect a packet loss every time an RTP packet is received. When there is a packet loss, the retransmission request retransmission is performed. Only when the estimated packet arrival time is in time for the reproduction (decoding) time of the lost packet, a NACK-RTCP packet that requests retransmission of the lost packet is generated and sent to the data transmitting apparatus 110.

  The flowchart in FIG. 12 shows a processing procedure for retransmission request list control in the retransmission request generator 124.

  When the RTP data communication unit 121 receives an RTP packet, the retransmission request generation unit 124 starts a retransmission request list control process in step ST31. In step ST32, the retransmission request generation unit 124 updates the retransmission request list based on the received packet (received RTP packet) information. That is, when the received packet is a retransmission packet, retransmission request generating section 124 deletes information related to the received packet from the retransmission request list.

  Next, in step ST <b> 33, retransmission request generation section 124 acquires a lost packet whose current time exceeds the retransmission packet arrival scheduled time from the retransmission request list stored in retransmission request list storage section 125. Then, in step ST34, the retransmission request generation unit 124 has a lost packet that exceeds the estimated retransmission packet arrival time, and the estimated packet arrival time related to the retransmission of the lost packet becomes the reproduction (decoding) time of the lost packet. Judge whether it is in time.

  When there is no lost packet that exceeds the scheduled arrival time of the retransmission packet, or there is a lost packet that exceeds the scheduled arrival time of the retransmission packet, the retransmission request generation unit 124 If the time is not in time for the lost packet playback time, the process immediately proceeds to step ST37, and the retransmission request list control process is terminated.

  If there is a lost packet that exceeds the scheduled retransmission packet arrival time and the scheduled packet arrival time related to the retransmission of the lost packet is in time for the reproduction time of the lost packet, retransmission request generating section 124 proceeds to step ST35. In step ST35, the retransmission request generation unit 124 acquires the sequence number Seq_last_recv of the latest received packet (the latest RTP packet among the RTP packets received by the RTP data communication unit 121).

  In step ST <b> 36, the retransmission request generation unit 124 generates a NACK-RTCP packet that requests retransmission of the lost packet, and sends the NACK-RTCP packet to the data transmission device 110 through the control data communication unit 126. In addition, retransmission request generation section 124 updates the retransmission request list in step ST36. In this case, the retransmission packet arrival scheduled time corresponding to the loss packet is measured by using the above-described delay measurement RTCP packet at the transmission time of the NACK-RTCP packet that requests retransmission of the loss packet. The time is obtained by adding the round trip delay time.

  The retransmission request generation unit 124 moves to step ST37 after the process of step ST36 and ends the retransmission request control process.

  When retransmission request list generation unit 124 performs retransmission request list control processing according to the flowchart of FIG. 12, when a received packet is a retransmission packet, information related to the received packet is deleted from the retransmission request list. In addition, the retransmission request generation unit 124 performs a retransmission request list control process according to the flowchart of FIG. 12, so that every time an RTP packet is received, a lost packet that exceeds the retransmission packet arrival scheduled time from the retransmission request list. The NACK-RTCP packet for requesting retransmission of the lost packet is generated and the data transmitting apparatus 110 only when the estimated packet arrival time related to the retransmission of the lost packet is in time for the reproduction (decoding) time of the lost packet. Send to.

  The operation of the data communication system 100 shown in FIG. 1 will be described.

  Encoded image data is supplied from the data source to the packetizer 111 of the data transmission device 110 (see FIG. 2). In the packetizer 111, image data is packetized, and a data packet (RTP packet) according to RTP is generated. The RTP packet generated by the packetizer 111 is temporarily stored in the transmission buffer 112. Each RTP packet is sequentially extracted from the transmission buffer 112, sent to the RTP data communication unit 113, and transmitted to the data receiving device 120 (see FIG. 9) via the network (IP network) 130.

  The RTP data communication unit 121 of the data receiving apparatus 120 receives RTP packets sent from the data transmitting apparatus 110 via the network (IP network) 130. The RTP packet received by the RTP data communication unit 121 is temporarily stored in the reception buffer 122. The depacketizer 123 analyzes the RTP packet stored in the reception buffer 122 and reconstructs encoded image data (streaming data) before packetization. The encoded image data output from the depacketizer 123 is decoded by a decoder (not shown) to display an image.

  The retransmission request generation unit 124 of the data reception device 120 detects packet loss based on the RTP packet received by the RTP data communication unit 126 and accumulated in the reception buffer 122. When there is a packet loss, the retransmission request generator 124 generates a NACK-RTCP packet (see FIGS. 6 and 7) that requests retransmission of a lost packet (lost RTP packet). The NACK-RTCP packet generated by the retransmission request generation unit 124 is transmitted to the control data communication unit 126 and transmitted to the data transmission device 110 via the network (IP network) 130. In this case, information related to the retransmission request is registered in the retransmission request list (see FIG. 10).

  Note that the retransmission request generator 124 generates a NACK-RTCP packet that requests retransmission of the lost packet only when the scheduled arrival time of the lost packet is in time for the reproduction (decoding) time of the lost packet (see FIG. 11).

  In addition, every time an RTP packet is received by the RTP data communication 121, the retransmission request generation unit 124 confirms the presence of a lost packet that exceeds the scheduled retransmission packet arrival time from the retransmission request list, and the lost packet exists. Then, a NACK-RTCP packet requesting retransmission of the loss packet is generated (see FIG. 12).

  Also in this case, the retransmission request generation unit 124 requests NACK-RTCP to request retransmission of the lost packet only when the packet arrival scheduled time related to the retransmission of the lost packet is in time for the reproduction (decoding) time of the lost packet. A packet is generated.

  The control data communication unit 115 of the data receiving apparatus 110 receives a NACK-RTCP packet sent from the data receiving apparatus 120 via the network (IP network) 130. When the received NACK-RTCP packet relates to the first retransmission request, the retransmission request is sent from the retransmission request processing unit 116 to the transmission buffer 112 regardless of the time from the transmission time of the lost packet to the current time. The loss packet is retransmitted to the data receiving device 120 through the RTP data communication unit 113 (see FIG. 8).

  In addition, when the received NACK-RTCP packet is related to the second and subsequent retransmission requests, only when the latest delay time or more has elapsed from the latest retransmission time to the current time of the lost packet, the retransmission request processing unit 116 A retransmission request is sent to the transmission buffer 112, and the lost packet is retransmitted to the data reception device 120 through the RTP data communication unit 113 (see FIG. 8).

  A specific example of the loss packet retransmission operation will be described with reference to FIG. Assume that the normal transmission round-trip delay time (RTT) is 10 milliseconds, packet loss occurs due to congestion, and the delay time is 30 milliseconds due to an increase in buffering delay. Here, the normal transmission round trip delay time (RTT) corresponds to the transmission round trip delay time measured using the above-described RTCP packet for delay measurement.

  The data reception device 120 detects a packet loss according to the control flow of FIG. 11, transmits a retransmission request NACK1 (NACK-RTCP packet) to the data transmission device 110, and retransmits 10 milliseconds after the RTT as an estimated arrival time. Register in the request list. After 10 milliseconds, the data reception device 120 transmits a retransmission request NACK2 to the data transmission device 110 because the retransmission packet has not arrived by the scheduled arrival time according to the control flow of FIG. Hereinafter, if the estimated arrival time is before the reproduction (decoding) time, the data reception device 120 transmits a retransmission request NACK until a retransmission packet arrives.

  The data transmission apparatus 110 processes the retransmission request NACK1 according to the control flow of FIG. 8, and immediately transmits the retransmission packet because it is the first retransmission request of the RTP packet having the sequence number Seq_lost. Thereafter, the data transmission apparatus 110 receives the retransmission requests NACK2 and NACK3, but does not transmit the retransmission packet since the latest delay time of 30 milliseconds has not elapsed since the transmission of the corresponding retransmission packet.

  In the data reception device 120, when the retransmission packet arrives without loss 30 milliseconds after the retransmission request NACK1, the retransmission packets for the retransmission requests NACK2 and NACK3 are wasted. As described above, since the data transmission apparatus 110 does not transmit retransmission packets in response to the retransmission requests NACK2 and NACK3, the retransmission packets can be effectively reduced.

  If the retransmission packet is lost, the data reception device 120 transmits a retransmission request NACK4, and the data transmission device 110 determines the second retransmission packet transmission based on the latest delay time. After this, unless the delay information of the data receiving apparatus 120 is updated, retransmission requests NACK5 and NACK6 are transmitted every 10 milliseconds, and the data transmitting apparatus 110 is controlled so as not to send useless retransmission packets. In FIG. 13, the retransmission requests NACK4, NACK5 and NACK6 are not shown.

  In FIG. 13, it is assumed that the numbers in parentheses represent the sequence numbers of RTP packets, and the RTP packet with the sequence number of 5 is a lost RTP packet. In this case, the sequence number of the latest received packet when the retransmission request NACK1 is generated is 6. That is, in the retransmission request NACK1, the sequence number Seq_lost of the lost packet is 5, and the sequence number Seq_last_recv of the latest received packet is 6.

  In FIG. 13, for simplicity of illustration, illustration of RTP packets with sequence numbers of 7 and later is omitted. In retransmission requests NACK2 and NACK3, the sequence number Seq_lost of the lost packet is 5 as in the retransmission request NACK1, but the sequence number Seq_last_recv of the latest received packet is different from the retransmission request NACK1.

  According to the data communication system 100 illustrated in FIG. 1, the data reception device 120 transmits a NACK-RTCP packet including the sequence number Seq_lost of the lost packet and the sequence number Seq_last_recv of the latest reception packet to the data transmission device 110. Determines the latest delay time based on the sequence number Seq_last_recv of the latest received packet, and does not retransmit the lost packet unless the latest delay time has elapsed from the latest retransmission time of the lost packet to the current time. Unnecessary duplicate transmission can be prevented.

  Further, according to the data communication system 100 shown in FIG. 1, only 2-byte information (sequence number Seq_last_recv) is added to the NACK-RTCP packet requesting retransmission of the lost packet (see FIG. 7), which is used for delay measurement. Since the latest delay time is obtained only when retransmission is performed, additional overhead for the network 130, the data transmission device 110, and the data reception device 120 is extremely small.

  In the above-described embodiment, the lost packet information is the sequence number Seq_lost of the lost packet and the latest received packet information is the sequence number Seq_last_recv of the latest received packet. However, each is limited to the sequence number. Any information that can identify the lost packet and the latest received packet may be used.

  The present invention is effective in recovering packet loss due to congestion or deterioration of link quality such as radio, and contributes to efficient use of network bandwidth. Data communication for transmitting streaming data Applicable to the system.

It is a block diagram which shows the structure of the data communication system as embodiment. It is a block diagram which shows the structure of a data transmitter. It is a figure which shows the format of a RTP packet. It is a figure for demonstrating the transmission information of a transmission packet. It is a figure which shows the format of an IP header. It is a figure which shows the format of a NACK-RTCP packet. It is a figure which shows the format of a NACK payload. It is a flowchart which shows the process sequence of the retransmission control in a data transmitter. It is a block diagram which shows the structure of a data receiver. It is a figure for demonstrating a resending request list. It is a flowchart which shows the process sequence of the resending request control in a data receiver. It is a flowchart which shows the process sequence of the resending request list control in a data receiver. It is a sequence diagram for demonstrating the specific example of resending operation | movement of a loss packet.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 100 ... Data communication system, 110 ... Data transmission apparatus, 111 ... Packetizer, 112 ... Transmission buffer, 113 ... RTP data communication part, 114 ... Transmission information storage part, 115 ... Control data communication unit 116 ... Retransmission request processing unit 120 ... Data reception device 121 ... RTP data communication unit 122 ... Reception buffer 123 ... Depacketizer 124 ... Retransmission Request generation unit, 125 ... retransmission request list storage unit, 126 ... control data communication unit, 130 ... network (IP network)

Claims (7)

A data communication system comprising a data transmitting device and a data receiving device and transmitting packetized streaming data,
The data receiving device is:
A packet receiving unit that receives the packet of the streaming data transmitted from the data transmitting device;
A depacketizer for analyzing the packet of the streaming data received by the packet receiver and reconstructing the streaming data;
A packet loss detection unit that detects a lost packet based on the packet of the streaming data received by the packet reception unit;
A retransmission request generator for generating retransmission request information including information on a lost packet detected by the packet loss detector and information on a latest received packet received by the packet receiver;
A retransmission request transmission unit that transmits the retransmission request information generated by the retransmission request generation unit to the transmission device;
The data transmission device is
A packetizer for packetizing the streaming data;
A packet transmission unit that transmits the packet of the streaming data generated by the packetizer to the data reception device;
A transmission information storage unit for storing transmission information including at least the transmission time of each packet transmitted by the packet transmission unit;
A retransmission request receiving unit that receives the retransmission request information transmitted from the data receiving device;
Based on the latest received packet information included in the retransmission request information received by the retransmission request receiving unit, the transmission time of the latest received packet is acquired from the transmission information storage unit, and the difference between the transmission time and the current time A delay time calculation unit that calculates the latest delay time by taking
A packet retransmission control unit that performs retransmission control of the loss packet,
The packet retransmission control unit
When the retransmission request information received by the retransmission request receiving unit relates to the first retransmission request of the lost packet indicated by the lost packet information included in the retransmission request information, from the transmission time of the lost packet Regardless of the time until the current time, the packet transmission unit is controlled to transmit the lost packet to the data receiving device,
Included in the retransmission request information when the retransmission request information received by the retransmission request receiving unit relates to a second or subsequent retransmission request of the lost packet indicated by the lost packet information included in the retransmission request information. Based on the lost packet information, the latest retransmission time of the lost packet is acquired from the transmission information storage unit, and the latest delay time calculated by the delay time calculation unit has elapsed from the latest retransmission time to the current time. If there is, the data transmission system controls the packet transmission unit to transmit the loss packet to the data receiving device. A data receiving device for receiving packetized streaming data,
A packet receiving unit that receives the packet of the streaming data transmitted from the data transmitting device;
A depacketizer for analyzing the packet of the streaming data received by the packet receiver and reconstructing the streaming data;
A packet loss detection unit that detects a lost packet based on the packet of the streaming data received by the packet reception unit;
A retransmission request generator for generating retransmission request information including information on a lost packet detected by the packet loss detector and information on a latest received packet received by the packet receiver;
An information transmission unit comprising: an information transmission unit that transmits the retransmission request information generated by the retransmission request generation unit to the data transmission device. By further communication with the data transmission device, further comprising a delay time acquisition unit that periodically acquires a transmission round-trip delay time with the data transmission device,
The retransmission request generation unit
The retransmission request information is generated when the reproduction time of the lost packet detected by the packet loss detecting unit is a time after the transmission round-trip delay time acquired by the delay time acquiring unit from the current time. The data receiving device according to claim 2, wherein: By further communication with the data transmission device, further comprising a delay time acquisition unit that periodically acquires a transmission round-trip delay time with the data transmission device,
The retransmission request generation unit
When the predetermined loss packet is not received by the packet reception unit within the transmission round-trip delay time acquired by the delay time acquisition unit from the time when the retransmission request information of the predetermined loss packet is transmitted from the information transmission unit, The data receiving apparatus according to claim 2, wherein the retransmission request information of the predetermined loss packet is generated again. A receiving method for receiving packetized streaming data,
A packet reception step for receiving the packet of the streaming data transmitted from the transmission side;
A packet analyzing step of analyzing the packet of the streaming data received in the packet receiving step and reconstructing the streaming data;
A packet loss detection step for detecting a lost packet based on the packet of the streaming data received in the packet reception step;
A retransmission request generating step for generating retransmission request information including information on the lost packet detected in the packet loss detecting step and information on the latest received packet received in the packet receiving step;
A retransmission request transmission step of transmitting the retransmission request information generated in the retransmission request generation step to the transmission side. A data transmission device for transmitting packetized streaming data,
A packetizer for packetizing the streaming data;
A packet transmission unit for transmitting the packet of the streaming data generated by the packetizer to a data reception device;
A transmission information storage unit for storing transmission information including at least the transmission time of each packet transmitted by the packet transmission unit;
A retransmission request receiving unit for receiving retransmission request information including information on a lost packet and information on the latest received packet sent from the data receiving device;
Based on the latest received packet information included in the retransmission request information received by the retransmission request receiving unit, the transmission time of the latest received packet is acquired from the transmission information storage unit, and the difference between the transmission time and the current time A delay time calculation unit that calculates the latest delay time by taking
A packet retransmission control unit that performs retransmission control of the loss packet,
The packet retransmission control unit
When the retransmission request information received by the retransmission request receiving unit relates to the first retransmission request of the lost packet indicated by the lost packet information included in the retransmission request information, from the transmission time of the lost packet Regardless of the time until the current time, the packet transmission unit is controlled to transmit the lost packet to the data receiving device,
Included in the retransmission request information when the retransmission request information received by the retransmission request receiving unit relates to a second or subsequent retransmission request of the lost packet indicated by the lost packet information included in the retransmission request information. Based on the lost packet information, the latest retransmission time of the lost packet is acquired from the transmission information storage unit, and the latest delay time calculated by the delay time calculation unit has elapsed from the latest retransmission time to the current time. If there is, the data transmission device controls the packet transmission unit to transmit the loss packet to the data reception device. A transmission method for transmitting packetized streaming data,
A packet generation step for packetizing the streaming data;
A packet transmission step of transmitting the packet of the streaming data generated in the packet generation step to a reception side;
A transmission information storage step for storing transmission information including at least the transmission time of each packet transmitted in the packet transmission step in a transmission information storage unit;
A retransmission request receiving step for receiving retransmission request information including information on a lost packet and information on the latest received packet sent from the receiving side;
Based on the latest received packet information included in the retransmission request information received in the retransmission request receiving step, the transmission time of the latest received packet is acquired from the transmission information storage unit, and the difference between the transmission time and the current time A delay time calculating step for calculating the latest delay time by taking
A packet retransmission control step for performing retransmission control of the loss packet,
In the packet retransmission control step,
When the retransmission request information received in the retransmission request reception step relates to a first retransmission request of the lost packet indicated by the lost packet information included in the retransmission request information, the transmission time of the lost packet is Regardless of the time until the current time, the packet transmission step is controlled so as to transmit the lost packet to the receiving side,
Included in the retransmission request information when the retransmission request information received in the retransmission request receiving step relates to a second or subsequent retransmission request of the lost packet indicated by the lost packet information included in the retransmission request information. The latest retransmission time of the lost packet is acquired from the transmission information storage unit based on the lost packet information, and the latest delay time calculated in the delay time calculation step has elapsed from the latest retransmission time to the current time. If there is, the packet transmission step is controlled so that the loss packet is transmitted to the receiving side.

Images