JP2010187292A - Transmission apparatus and transmission control method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a transmission apparatus in which the transmission of a packet is controlled. <P>SOLUTION: The transmission apparatus includes: a packet transmitting unit 11 for packetizing a video frame for every video frame and transmitting it to a receiving device; an authentication receiving unit 12 for receiving an authentication for each packet from the receiving device; and a transmission control unit 10 for transferring a video frame received from an external device to the packet transmitting unit 11, recording a transmission time indicative of a time at which a video frame is transmitted to the receiving device, and also receiving an authentication from the authentication receiving unit 12 and managing the situation of receiving the video frame transmitted to the receiving device. When all authentications corresponding to the transmitted video frames are not received, the transmission control unit 10 determines a difference between a transmission time of the latest video frame, among the video frames of which the authentications are not received, and the present time and controls transmission of video frames on the basis of results of comparing the difference with a predetermined threshold. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a transmission apparatus that controls transmission of a packet.

  Normally, when data is transmitted using IP (Internet Protocol), UDP (User Datagram Protocol) is used when real time property is required, and TCP (Transmission Control Protocol) is used when real time property is not required. TCP provides reliability by retransmitting lost packets, but real-time performance is not guaranteed due to the retransmission function. On the other hand, UDP is not reliable because it does not have a retransmission function, but real-time performance is improved.

  When IP communication is performed using HSDPA (High Speed Downlink Packet Access) developed for mobile communication, loss of IP packets does not occur due to the HSDPA retransmission function. In this case, since the retransmission is performed even if UDP is used, the real-time property is impaired. In a situation where retransmission is performed, the IP packet to be retransmitted is stagnant in the network. When an IP packet is transmitted from the transmission side in such a situation, the number of IP packets stagnating in the network increases, the transmission delay increases, and real-time performance is further impaired.

  In order to improve the real-time property, it is necessary to reduce IP packets stagnating in the network. Japanese Patent Application Laid-Open No. 2004-228561 discloses a technique of using RTT (Round Trip Time) as a method for recognizing the degree of IP packet stagnation in the network and controlling the transmission rate based on the magnitude of RTT.

JP 2006-014329 A

  However, according to the above conventional technique, the transmission rate is controlled by the size of the RTT, but IP packets are still transmitted into the network even if the transmission rate is lowered. For this reason, there is a problem that the number of IP packets stagnating in the network may increase.

  Further, when the RTT cannot be measured, the transmission rate cannot be temporarily controlled. Therefore, there is a problem that the number of IP packets stagnating in the network may increase.

  Also, adjusting the transmission rate does not guarantee a delay time. For this reason, there has been a problem that real-time performance in consideration of the maximum delay time cannot be realized.

  The present invention has been made in view of the above, and an object of the present invention is to obtain a transmission apparatus capable of defining the maximum delay time while improving the real-time property.

  In order to solve the above-described problems and achieve the object, the present invention provides packet transmission means for packetizing continuous video frames for each video frame and transmitting the packets to the reception device, and delivery confirmation for each packet from the reception device. And a confirmation receiving means for receiving the image data, and transferring the continuous video frames received from an external device to the packet transmitting means, while recording a transmission time indicating a time at which the video frames are transmitted to the receiving device, Transmission control means for receiving a delivery confirmation from the confirmation receiving means and managing the reception status of the video frame transmitted to the receiving device, the transmission control means corresponding to the video frame transmitted by the packet transmitting means If not all acknowledgments have been received, the transmission time of the oldest video frame among the video frames that have not received the acknowledgment Obtains a difference between the current time, it controls the transmission of the video frame based on a comparison result between the difference and a predetermined threshold value, characterized in that.

  According to the present invention, there is an effect that the maximum delay time can be defined while improving the real-time property.

FIG. 1 is a diagram illustrating a configuration example of a communication system. FIG. 2 is a diagram illustrating a configuration example of a transmission apparatus. FIG. 3 is a flowchart illustrating a transmission control process of the transmission apparatus. FIG. 4 is a sequence diagram illustrating a transmission control process of the transmission apparatus. FIG. 5 is a sequence diagram illustrating a transmission control process of the transmission apparatus. FIG. 6 is a diagram illustrating a configuration example of a transmission apparatus.

  Embodiments of a transmission apparatus according to the present invention will be described below in detail with reference to the drawings. Note that the present invention is not limited to the embodiments.

Embodiment 1 FIG.
FIG. 1 is a diagram illustrating a configuration example of a communication system including a transmission apparatus according to the present invention. The communication system includes a transmission device 1, a reception device 2, and a network 3. The transmission device 1 transmits continuous video frame data. The receiving device 2 receives the video frame data. The network 3 mediates data transmission / reception between the transmission device 1 and the reception device 2. The transmission device 1 transmits video frame data via the network 3 while performing packet communication with the reception device 2. It is also possible to directly connect the transmission device 1 and the reception device 2 without going through the network 3.

  FIG. 2 is a diagram illustrating a configuration example of the transmission device 1. The transmission device 1 includes a transmission control unit 10, a packet transmission unit 11, and a confirmation reception unit 12. The transmission control unit 10 inputs video frame data from the outside, and determines whether or not the video frame data can be transmitted sequentially. The packet transmission unit 11 packetizes and transmits the video frame data to be transmitted. The confirmation receiving unit 12 receives delivery confirmation data.

  Next, the transmission operation of the transmission device 1 will be described. The transmission control unit 10 transfers continuous video frame data captured from an external device (not shown) to the packet transmission unit 11 in units of frames. At this time, video frames to be transmitted are managed using video frame numbers and the like. It also manages the time information stamped in the video frame and the transmission time. The packet transmission unit 11 that has received the video frame data frame by frame packetizes the video data in units of one frame of video data and transmits the packet data to the reception device 2 via the network 3.

  The confirmation reception unit 12 receives delivery confirmation data corresponding to the video packet transmitted by the packet transmission unit 11. The delivery confirmation data indicates a video frame number and the like, and the confirmation receiving unit 12 can recognize up to which video frame the reception device 2 has received. The confirmation receiving unit 12 notifies the transmission control unit 10 of information (for example, video frame number) of the video frame received by the receiving device 2.

  The transmission control unit 10 can recognize the status of the video frame whose delivery has not been confirmed from the status of the video frame transferred to the packet transmission unit 11 and the status of the video frame whose delivery has been confirmed notified from the confirmation receiving unit 12. For example, when a video frame is transmitted at a fixed frame rate, it is recognized that delivery confirmation of a video frame five video frames before the video frame to be transmitted has not been confirmed. In this case, when the difference in the number of video frames between a video frame that is about to be transmitted and a video frame that has not been confirmed for delivery exceeds a preset value (threshold value T), the transmission control unit 10 The transfer of the video frame data to the packet transmission unit 11 is interrupted. The transmission control unit 10 realizes low-delay video data transmission by suppressing the difference in the number of video frames from a video frame for which delivery confirmation has not been obtained.

  Next, transmission control of video frames based on the reception status of delivery confirmation data in the transmission device 1 will be described. FIG. 3 is a flowchart showing a transmission control process of the transmission apparatus 1. The transmission control unit 10 confirms the type of the received signal (step S1). When a video frame for transmission is received (step S1: video frame), a difference between the transmission time of the oldest unconfirmed video frame and the current time is obtained (step S2). However, when there is no unconfirmed delivery, the difference is calculated as 0. Next, the transmission control unit 10 checks whether or not the obtained difference is larger than the threshold value T (step S3).

  When the difference is larger than the threshold value T (step S3: Yes), the transmission control unit 10 does not transfer the received video frame to the packet transmission unit 11 and does not transmit the video frame (step S4). In this case, the transmitted video frame number is not updated. When the difference is equal to or smaller than the threshold value T (step S3: No), the transmission control unit 10 transfers the video frame to the packet transmission unit 11, and the packet transmission unit 11 receives the video packet via the network 3. It transmits to the apparatus 2 (step S5). Thereafter, the transmission control unit 10 updates the record of the transmitted video frame number (step S6).

  When the confirmation receiving unit 12 receives the delivery confirmation data, the confirmation receiving unit 12 transfers the delivery confirmation data to the transmission control unit 10. Alternatively, the transmission control unit 10 is notified of the video frame number. When the transmission control unit 10 receives the delivery confirmation data (step S1: delivery confirmation), the video frame number corresponding to the received delivery confirmation data is deleted from the previously recorded transmitted video frame numbers (step S7). ). As a result, only the transmitted video frame numbers that have not been confirmed for delivery (video frame numbers for which delivery has not been confirmed) are recorded.

  The transmission control unit 10 repeats transmission of video frames (steps S5 and S6) and reception of delivery confirmation data (steps S7 and S8), thereby sequentially updating recording of unconfirmed video frame numbers. When transmission is performed at a fixed frame rate, the difference obtained in step S2 can be obtained by “number of frames × frame period”. Therefore, the “difference” can be easily obtained from the number assigned to the video frame. When transmission is performed at a variable frame rate, the difference can be obtained using the time stamp given to the video frame as the transmission time.

  As described above, the transmission control unit 10 manages undelivered video frames, and when the difference between the transmission time of the oldest unacknowledged video frame and the current time is larger than the threshold T, transmission without transmitting the video frame. Take control. Thereby, it is possible to avoid a situation in which transmission further deteriorates in a situation where data stagnation occurs in the network. Further, since transmission of video data can be started quickly when stagnation in the network is alleviated, transmission with relatively low delay becomes possible.

  Next, specific transmission control performed by the transmission apparatus 1 will be described. FIG. 4 is a sequence diagram illustrating a transmission control process. Here, video frame data is generated at a fixed frame rate, and the transmission apparatus 1 transmits video packets at a constant period. The transmission control unit 10 sets the difference between the transmission time and the current time as 3 frame times, and does not transmit the video packet when the delivery confirmation data is not received for a video packet that is more than 3 frame times before.

  The transmission device 1 can transmit a video packet unconditionally up to the third frame. The video packet reaches the receiving device 2 with a delay. The receiving device 2 transmits delivery confirmation data corresponding to the received video packet to the transmitting device 1. When transmitting the fourth frame, the transmission device 1 can receive the second frame delivery confirmation data two frames before, and therefore can transmit the fourth frame. Furthermore, video packets can be transmitted up to the seventh frame because the delivery confirmation data three frames before is received.

  The transmission device 1 has not received the delivery confirmation data three frames before (the delivery confirmation data of the fifth frame) due to a stagnation of data in the network at the transmission timing of the eighth frame. Therefore, the video packet of the eighth frame is not transmitted. Thereafter, the transmission confirmation data of 3 frame times before is not received until the transmission timing of the 11th frame, so the video packet is not transmitted. Since the delivery confirmation data of the seventh frame has been received, there is no undelivered transmission of the transmitted video packet, and therefore the transmission device 1 resumes transmission of the video packet from the 12th frame.

  When the transmission device 1 has not received the delivery confirmation data corresponding to the video packet that is more than a predetermined frame time ago, the transmission device 1 does not transmit the video packet, thereby promoting low-latency transmission without promoting network congestion. it can.

  Note that when the video frame data input from the external device is being encoded within the frame, the transmission control unit 10 discards the video frame data not to be transmitted. For example, if each video frame is JPEG (Joint Photographic Experts Group) encoded, it is independent of other video frames, so even if some video frames are discarded, the decoding of each video frame is affected. Not give. Therefore, data can be discarded.

  On the other hand, when general moving picture encoding is used, encoding is performed using information of previous and subsequent video frames, and discarding data of one video frame also affects other video frames. In this case, the transmission control unit 10 requires processing such as limiting frames that can be discarded or re-encoding a video frame after a video frame in which data is discarded.

  If the transmission control unit 10 does not transmit a video packet, the transmission control unit 10 can easily limit the amount of transmission data and realize low-delay transmission by discarding the data of the video frame.

  Here, the setting of the threshold value T to be compared with the difference will be described. The time from transmission of a video packet to reception of delivery confirmation data varies due to fixed transmission delay in the network, fluctuating transmission fluctuation, data stagnation due to congestion, and the like. The presence or absence of data stagnation in the network can be estimated by comparing with a fixed transmission delay time. Therefore, as a method for calculating a fixed transmission delay, the time from transmission of a video packet to reception of delivery confirmation data is measured for a predetermined period, and the minimum value is selected. This is because the case where there is no transmission fluctuation or stagnation in the network is a fixed transmission delay, and the time at this time is the shortest. As described above, the comparison time for estimating the stagnation of data in the network is the minimum confirmation time until reception confirmation data is received.

  Since the time until receiving the acknowledgment data includes transmission fluctuations, taking this into account, the data stays in the network for N times (N> 1) the previous minimum confirmation time. The threshold value T is used to determine whether or not As a result, it is possible to detect and infer the presence or absence of data stagnation in the network without reacting to transmission fluctuations. Further, by performing transmission control so as to avoid stagnation, transmission with low delay becomes possible.

  In addition, since the minimum confirmation time until receiving the delivery confirmation data may change depending on the situation in the network, the communication path, the communication partner, etc., the minimum confirmation time for setting the threshold value T is cyclic. Update to For example, the minimum confirmation time is selected in the X time period, and the threshold value T is set using the minimum confirmation time at that time. The transmission control unit 10 performs transmission control using the threshold value T updated every X hours, so that the value of the threshold value T can be set according to the change in the transmission status, and optimal transmission control is performed. , Low-delay transmission is possible.

  In the transmission control unit 10, if transmission control is actually performed using only the difference from the unconfirmed delivery video frame, the transmission interval of the video frame may be biased. For example, in a situation where the transmission band of the network can ensure only half of the transmission rate that is originally desired to be transmitted, if transmission is performed every other video frame, the frame interval becomes constant and the received video is stabilized. Considering the above, the transmission rate is controlled in accordance with the reception interval of the delivery confirmation data. That is, when the transmission band of the network is smaller than the transmission rate, the reception interval of the acknowledgment data is proportional to the transmission rate of the network. Therefore, the transmission control unit 10 controls the transmission rate according to the reception interval of the delivery confirmation data. As a result, transmission that does not exceed the transmittable bandwidth of the network becomes possible.

  FIG. 5 is a sequence diagram illustrating a transmission control process. As in FIG. 4, video frame data is generated at a fixed frame rate, and the transmission apparatus 1 transmits video packets at a constant period. The transmission control unit 10 does not newly transmit the video packet when the delivery confirmation data is not received for the video packet three frames or more before.

  The transmission device 1 can transmit a video packet unconditionally up to the third frame. The video packet reaches the receiving device 2 with a delay. The receiving device 2 transmits delivery confirmation data corresponding to the received video packet to the transmitting device 1. When transmitting the fourth frame, the transmission device 1 can receive the first frame delivery confirmation data three frames before, and therefore can transmit the fourth frame.

  The transmission device 1 recognizes that the reception timing is late when receiving the delivery confirmation data of the second frame. Since the reception interval of the delivery confirmation data is nearly twice as large as the transmission interval of the video packet of the transmission device 1, the transmission device 1 lowers the transmission rate to ½. Since the delivery confirmation data of the second frame three frames before is received at the transmission timing of the fifth frame, it is possible to transmit the video packet of the fifth frame, but in order to reduce the transmission rate, 5 Do not send the video bucket of the frame. Since the delivery confirmation data of the third frame three frames before the reception timing of the sixth frame has not been received, the video packet of the sixth frame cannot be transmitted. Thereafter, at the transmission timing up to the eighth frame, since the delivery confirmation data three frames before is not received, the transmission device 1 cannot transmit the video packet until the eighth frame.

  Since there is no unacknowledged video packet at the transmission timing of the ninth frame, the transmission device 1 transmits the video packet. Originally, 3 frames can be transmitted continuously, but the 10th frame is not transmitted in order to halve the transmission rate. Thereafter, by transmitting every other frame, video frames can be transmitted with low delay, and the transmission frame interval is stabilized.

  In addition, the transmission device 1 always measures the time until receiving the delivery confirmation data, and when the transmission band increases, the time until reception decreases, so the transmission rate may be increased again according to the band. Is possible. As described above, the transmission device 1 can transmit the video to the reception device 2 at a stable frame rate by adjusting the transmission rate of the video packet based on the reception interval of the delivery confirmation data.

  On the other hand, when the data stagnates in the network but is resolved thereafter, the transmission control method transmits a new video packet after receiving all the delivery confirmation data corresponding to the transmitted video packet. At this time, if the transmission of the video packet can be resumed while the data stagnating in the network is decreasing smoothly, the transmission start timing can be advanced.

  An example will be described with reference to FIG. Here, the transmitting apparatus 1 cannot transmit the next video packet until it receives the delivery confirmation data of the seventh frame, which is the last transmitted video packet. This is because at the transmission timing of the 11th frame, transmission is not possible unless the delivery confirmation data of the 8th frame is received. However, there is only one video frame whose delivery has not been confirmed at this timing. Here, if it can be estimated that the delivery confirmation data of the seventh frame can be received immediately, the transmission device 1 can transmit the eleventh frame.

  That is, for all the transmitted video frames, a state in which all of the delivery confirmation data is likely to be received, for example, the preset number (M) of the remaining unconfirmed video frames such as 1 frame or less and 2 frames or less In the following cases, the transmission device 1 can assume that the delivery confirmation data has already been received, and advance the transmission restart timing.

  Although the case where transmission is resumed when the number of unconfirmed video frames is M or less has been described, the present invention is not limited to this. For example, it is also possible to set the condition that the difference between the transmission times of the oldest video packet and the newest video packet among the unconfirmed video frames is m hours or less.

  Further, when the video packet or the delivery confirmation data cannot be received due to an error or when a loss occurs in the network, the transmission device 1 cannot receive the delivery confirmation data. In order to deal with such a situation, the transmission control unit 10 sets a timer for receiving the delivery confirmation data, and assumes that the delivery confirmation data has been received when a preset time has elapsed (timeout). . As a result, the information on the unacknowledged video packet can be updated, and a new video packet can be transmitted.

  When the transmission device 1 cannot receive the delivery confirmation data corresponding to the transmitted video packet for a predetermined time or longer, the transmission device 1 regards the delivery confirmation data as received and performs subsequent transmission control, thereby causing a packet loss or a packet error. Even in this case, stable and low-delay transmission processing can be continued.

  As described above, in this embodiment, the difference between the transmission time of the oldest unconfirmed video frame and the current time is obtained, and if the difference is larger than a preset threshold time, a new video is obtained. The frame is not transmitted. As a result, it is possible to realize low-delay transmission without deteriorating data stagnation in the network while defining the delay time.

Embodiment 2. FIG.
In this embodiment, the transmission time of the video frame is managed based on a clock provided in the own device. A different part from Embodiment 1 is demonstrated.

  FIG. 6 is a diagram illustrating a configuration example of the transmission device 1a. The transmission device 1a includes a transmission control unit 10a, a packet transmission unit 11, and a confirmation reception unit 12. The transmission control unit 10 a includes a clock unit 13. The transmission control unit 10a inputs video frame data from the outside, and sequentially determines whether or not video frame data can be transmitted. Further, the transmission time of the video frame data is managed based on the time information from the clock unit 13. The clock unit 13 measures the transmission time when the video frame data is transferred to the packet transmission unit 11.

  Next, the transmission operation of the transmission device 1a will be described. The transmission control unit 10a transfers continuous video frame data fetched from an external device (not shown) to the packet transmission unit 11 in units of frames. At this time, information on the transmission time at the time of transfer from the clock unit 13 is read, and the video frame to be transmitted is managed using the transmission time information and the video frame number. The packet transmission unit 11 that has received the video frame data frame by frame packetizes the video data in units of one frame of video data and transmits the packet data to the reception device 2 via the network 3.

  The confirmation reception unit 12 receives delivery confirmation data corresponding to the video packet transmitted by the packet transmission unit 11. The delivery confirmation data indicates a video frame number and the like, and the confirmation receiving unit 12 can recognize up to which video frame the reception device 2 has received. The confirmation receiving unit 12 notifies the transmission control unit 10a of information (for example, video frame number) of the video frame received by the receiving device 2.

  The transmission control unit 10a determines the transmission time of the oldest video frame whose delivery has not been confirmed from the status of the video frame transferred to the packet transmission unit 11 and the status of the delivery-confirmed video frame notified from the confirmation reception unit 12. From the difference from the current time that can be read from the clock unit 13, it is possible to recognize the state of the video frame for which delivery has not been confirmed. For example, when the obtained difference exceeds a preset value (threshold value T), the transmission control unit 10a interrupts the transfer of the video frame data to the packet transmission unit 11. The transmission control unit 10a realizes low-delay video data transmission by suppressing a difference in transmission time from a video frame for which delivery confirmation has not been obtained to a certain level or less.

  As described above, according to the present embodiment, it is possible to manage the transmission time of a video frame to be transmitted using a clock provided in the transmission control unit even when a time stamp is not added to the video frame in advance. Thereby, the effect similar to Embodiment 1 can be acquired.

  As described above, the transmission apparatus according to the present invention is useful for packet communication, and is particularly suitable for communication that requires real-time performance.

1, 1a Transmitting device 2 Receiving device 3 Network 10, 10a Transmission control unit 11 Packet transmitting unit 12 Confirmation receiving unit 13 Clock unit

Claims (11)

Packet transmission means for packetizing successive video frames for each video frame and transmitting the packets to the receiving device;
Confirmation receiving means for receiving a delivery confirmation for each packet from the receiving device;
The continuous video frames received from the external device are transferred to the packet transmission means, while the transmission time indicating the time when the video frames are transmitted to the reception device is recorded, and the delivery confirmation is received from the confirmation reception means. A transmission control means for receiving and managing the reception status of the video frame transmitted to the receiving device;
With
When the transmission control means has not received all the delivery confirmations corresponding to the video frames transmitted by the packet transmission means, the transmission time of the oldest video frame among the video frames that have not received the delivery confirmation and Find the difference with the current time, and control the transmission of the video frame based on the comparison result between the difference and a predetermined threshold,
A transmission apparatus characterized by the above.   The transmission apparatus according to claim 1, wherein the transmission time is a time stamp given to the video frame by an external apparatus. The transmission control means further includes
Timer means for measuring the time when the video frame is transferred to the packet transmission means;
With
2. The transmission apparatus according to claim 1, wherein the transmission time is a transfer time measured by the timer means. The transmission control means includes
4. The transmission device according to claim 1, wherein when the difference is larger than a predetermined threshold value, control is performed so that a video frame received from the external device is not transmitted to the reception device. The transmission control means includes
5. The transmission apparatus according to claim 4, wherein when it is determined that control is performed so as not to transmit a video frame based on the comparison result, data of the video frame is discarded. The transmission control means includes
Measure the response period from sending the video packet to receiving the delivery confirmation corresponding to the video packet,
6. The transmission according to claim 1, wherein the predetermined threshold value is set to N times (N> 1) a shortest response period among the measured response periods. apparatus. The transmission control means includes
The transmission apparatus according to claim 6, wherein the shortest response period is periodically updated. The transmission control means includes
8. The transmission apparatus according to claim 1, wherein a transmission rate of the video frame is adjusted based on a reception interval of delivery confirmation. The transmission control means includes
Even if it is determined to control not to transmit a video frame based on the comparison result, if the number of transmitted video frames that have not received a delivery confirmation is less than a predetermined number, Control for transmitting is performed, The transmission apparatus as described in any one of Claims 1-8 characterized by the above-mentioned. The transmission control means includes
The transmission apparatus according to claim 1, wherein when a delivery confirmation is not received over a predetermined period, transmission control is performed assuming that the delivery confirmation is received. A packet transmission step of packetizing continuous video frames received from an external device for each video frame and transmitting the packets to the reception device;
A transmission time recording step for recording the transmission time of the video frame;
A delivery confirmation step of receiving a delivery confirmation for each packet from the receiving device;
A difference calculating step for obtaining a difference between the transmission time of the oldest video frame and the current time among the video frames not receiving the delivery confirmation when not receiving all the delivery confirmations corresponding to the transmitted video frames; ,
A transmission control step for controlling transmission of a video frame based on a comparison result between the difference and a predetermined threshold;
Including a transmission control method.

Images