WO2016058279A1

WO2016058279A1 - Multi-path data transmission method based on quality evaluation

Info

Publication number: WO2016058279A1
Application number: PCT/CN2015/000221
Authority: WO
Inventors: 钱晓炯; 陈兴桃; 刘谦
Original assignee: 钱晓炯
Priority date: 2014-10-16
Filing date: 2015-03-31
Publication date: 2016-04-21
Also published as: CN104410509A; CN104410509B

Abstract

A multi-path data transmission method based on quality evaluation, comprising: firstly, different clients are connected to different media servers according to a principle of proximity, then when a calling client initiates an audio/video communication to a called client, the calling client and the called client are connected with each other via a 0^th channel, the audio/video data is forwarded to a calling-end media server, and the calling-end media server forwards the called audio/video data to the calling client; and when the calling client and the called client are connected with each other via the 0^th channel, the calling client and the called client are also connected with each other by starting another three channels, and choosing the channel with the highest score of network path quality for audio/video data transmission. Compared with the prior art, the problem of get-through speed between two clients is solved; and meanwhile, the network path quality of different paths is dynamically managed, and audio/video data is transmitted via the optimum network path.

Description

Multi-path data transmission method based on quality evaluation

Technical field

The invention relates to a multi-path data transmission method based on quality evaluation.

Background technique

Real-time audio and video transmission over IP networks can use a variety of paths, including forwarding through media servers, peer-to-peer transmission, and more. The advantage of forwarding through the media server is that complicated operations such as point-to-point penetration are not required, and the audio and video data at both ends of the communication can be directly intercommunicated, the success rate of the connection can be almost 100%, and the connection speed is fast; the disadvantage of forwarding through the media server is communication. The overall traffic bottleneck is constrained by the bandwidth of the media server accessing the network, and the media server side also has the traffic cost. Point-to-point transmission is just the opposite. The advantage is that the overall communication traffic does not depend on the bandwidth of the media server accessing the network, and the cost of the media server is low. The disadvantage is that the peer-to-peer operation is required, the connection speed is slow, and it may not be able to connect, and the point-to-point is simple. The transmission method has a success rate of approximately 90%.

Regarding the transmission quality of media server forwarding and peer-to-peer transmission, sometimes this is good sometimes. For example, when two communication clients are connected in the same network service provider environment, the peer-to-peer transmission effect is often better than that of the media server. In particular, the two clients are in the same LAN, and the peer-to-peer effect is mostly Ok. For example, when two communication clients access the Internet without using the carrier network respectively, if the media server is deployed in a multi-line computer room (that is, the same computer room accesses different carrier networks at the same time), the media server forwarding effect is often better than the point-to-point transmission. . However, these analyses can only be roughly predicted, and the actual network quality should be based on actual testing.

FIG. 1 shows a structure of a media server forwarding mode. Client 1 and client 2 respectively access the carrier network through a router, and the media server is deployed on the public network; the data of the two clients is directly forwarded to the other party through the media server.

Figure 2 shows a peer-to-peer transmission mechanism, which is the simplest peer-to-peer network penetration method. The STUN server has a public network address, and the client accessed through the router can access it through the public network address. When terminal 1 attempts to send a probe packet to the STUN server, the NAT function of the router translates the intranet address of client 1 into a public network IP address plus port, and the data sent by other network elements on the public network to this address and port will be transferred. The client 1, the STUN server will send the public IP address plus the port to the client 2, so if the two clients send data to the public network address and port at the same time, most of the cases are able to pass. As described in this process, the point-to-point transmission mechanism needs to go through a detection process, and the connection time is inevitably long. At the same time, due to the difference of routers, the above mechanism cannot penetrate 100% of the various routers on the market.

Based on the current state of the art, there is a technical standard ICE (Interactive Conncctivity Establishment, RFC5245) that can combine the advantages of the above two transmission modes, and can try to perform point-to-point transmission. If the point-to-point transmission fails, server forwarding is performed to ensure cost and penetration. ICE actually combines various transmission modes such as point-to-point transmission and media server forwarding. It is not a completely new solution. The main problem is that the connection speed is slow, and there is no quality management for various transmission channels. Unable to get the best communication quality experience.

Summary of the invention

The technical problem to be solved by the present invention is to provide a quality evaluation-based multipath data transmission method with fast connection speed, high connection success rate, and effective consideration of network cost.

The technical solution adopted by the present invention to solve the above technical problem is: a multi-path data transmission method based on quality evaluation, which is characterized in that:

First, different clients connect to different media servers according to the principle of proximity, and then when the calling client initiates audio and video communication to the called client, the calling client and the called client first pass through the 0th channel. The nearest media server connected to the calling client is called the calling media server, and the nearest media server connected to the called client is called the called media server. The data transmission path of the 0th channel is: The calling client sends the calling audio and video data to the calling media server, and the calling media server forwards the calling audio and video data to the called media server, and the called media server forwards the calling audio and video data. To the called client; likewise, the called client sends the called audio and video data to the called media server, and the called media server forwards the called audio and video data to the calling media server, the calling end The media server forwards the called audio and video data to the calling client;

Then, while the calling client and the called client are connected through the 0th channel, the following channels are also started to be connected between the calling client and the called client:

In the first channel, the data transmission path of the first channel is: the calling client sends the calling audio and video data to the calling media server, and then the calling media server forwards the calling audio and video data to the called client; Similarly, the called client sends the called audio and video data to the called media server, and then the called media server forwards the called audio and video data to the calling client;

The data transmission path of the second channel and the second channel is: the calling client sends the calling audio and video data to the called media server, and then the called media server forwards the calling audio and video data to the called client: Similarly, the called client sends the called audio and video data to the calling media server, and then the calling media server forwards the called audio and video data to the calling client;

The data transmission path of the third channel and the third channel is: a point-to-point transmission mechanism between the calling client and the called client;

Finally, when the four transmission paths of the 0th channel, the 1st channel, the 2nd channel, and the 3rd channel are established, the calling client or/and the called client simultaneously send the network quality detection packet to the above four channels. Therefore, the network quality of the above four channels is detected, and then the network path quality scores of the above four channels are respectively obtained, and finally the main intersection The channel with the highest network path quality score is selected between the client and the called client for audiovisual data transmission.

Therefore, the method provided by the present invention can support the transmission of media data in multiple channels at the same time. This mode is applicable to the path bottleneck not being on the access side of the terminal, and in the middle of the path, the transmission throughput of a single path can be increased.

As an improvement, the network quality detection packet of the currently applied channel between the calling client and the called client is a custom data packet header added before the calling audio and video data or the called audio and video data, and the customized data packet header includes The header type, channel number, serial number, timestamp, and data stream type; and the network quality probe packets sent by the other three channels include the aforementioned custom packet header plus IP/UDP packet header.

Further improved, the network quality probe packets sent by the calling client or/and the called client to the four channels are simultaneously transmitted, and the total size of the four network quality probe packets simultaneously transmitted to the four channels does not exceed the calling party. 5% of audio and video data or called audio and video data.

Preferably, the size of the aforementioned custom packet header is 5 bytes, and the IP/UDP packet header is 55 bytes.

Further improved, the calling client or/and the called client are separated by a fixed time to send network quality detection packets to the above four channels, thereby re-detecting the network quality of the above four channels, and respectively obtaining the above four channels. The new network path quality score, the main network client and the called client select the channel with the highest network path quality score for the transmission of audiovisual data.

Further improvement, the network path quality score is obtained by the following formula:

Score=factor(path)*(clip3(0,100,Yd)*clip3(0,100,Y1))/100

Where score is the network path quality score, clip3 is a finite function, that is, to ensure that the output value is between 0 and 100, factor (path) is the weighting factor of the connection mode, depending on the server cost, the factor (path) of the different connection mode Set in advance;

And Yd=-19.58ln(delay)+149.58, where delay is a relative one-way path delay under different channels, and delay=Td(n)-Td_min+rtt_min/2, n=0, 1, 2, 3;

Td(0) represents the unidirectional path delay of the 0th channel, Td(0)=T0-t0-deltaT, deltaT is the time difference between the calling client and the called client, and T0 is sent by the calling client through the 0th channel. The time of the calling audio and video data, t0 is the time when the called client receives the calling audio and video data at the moment;

Td(1) represents the unidirectional path delay of the first channel, Td(1)=T1-t1-deltaT, deltaT is the time difference between the calling client and the called client, and T1 is sent by the calling client through the first channel. The moment when the calling audio and video data is called, t1 is the time when the called client receives the calling audio and video data at the moment.

Td(2) represents the unidirectional path delay of the second channel, Td(2)=T2-t2-deltaT, deltaT is the time difference between the calling client and the called client, and T2 is sent by the calling client through the second channel. The moment when the calling audio and video data is called, t2 is the moment when the called client receives the calling audio and video data at the moment;

Td(3) indicates the unidirectional path delay of the third channel, Td(3)=T3-t3-deltaT, deltaT is the time difference between the calling client and the called client, and T3 is sent by the calling client through the third channel. The moment of calling audio and video data, t3 The moment when the called client receives the calling audio and video data at the moment:

Td_min=min(Td(0), Td(1), Td(2), Td(3));

Rtt_min=min(rtt(0), rtt(1), rtt(2), rtt(3)), and rtt(0) represents the round-trip path delay of channel 0; rtt(1) represents the round-trip of channel 1 Path delay; rtt(2) represents the round-trip path delay of the second channel; rtt(3) represents the round-trip path delay of the third channel;

Yl=-29.43ln(PLR)-29.637; PLR is the data packet loss rate under different channels.

If the cost factor is not considered, the factor (path) is always equal to 1. If the forwarding cost of the media server is considered, the weighting factor of the media server forwarding mode can be set to 1, the calling media server forwarding mode and the called media server. The weighting factor of the forwarding mode is set to 2, and the weighting factor of the point-to-point mode is set to 4.

Compared with the prior art, the present invention has the advantages that the method provided by the present invention solves the problem of the connection speed between two clients, and also dynamically manages the network path quality of different paths to optimize the network path. The transmission of audiovisual data is performed.

DRAWINGS

1 is a structural block diagram of a media server forwarding mode in the prior art;

2 is a structural block diagram of a mechanism of point-to-point transmission in the prior art;

3 is a block diagram showing the connection of the 0th channel in the embodiment of the present invention;

4 is a block diagram showing the connection of the first channel in the embodiment of the present invention;

Figure 5 is a block diagram showing the connection of the second channel in the embodiment of the present invention;

Figure 6 is a block diagram showing the connection of the third channel in the embodiment of the present invention;

FIG. 7 is a structural diagram of a custom data packet header according to an embodiment of the present invention; FIG.

FIG. 8 is a block diagram of calculation of a relative one-way path delay according to an embodiment of the present invention.

detailed description

The invention will be further described in detail below with reference to the embodiments of the drawings.

The multi-path data transmission method based on quality evaluation provided by the invention first solves the problem of the connection speed, that is, different clients are connected to different media servers according to the principle of proximity, and then the calling client initiates to the called client. In the audio and video communication, the calling client and the called client first connect through the 0th channel;

The nearest media server connected to the calling client is called the calling media server, and the nearest media server connected to the called client is called the called media server;

As shown in FIG. 3, the data transmission path of the 0th channel is: the calling client sends the calling audio and video data to the calling media server, and the calling media server forwards the calling audio and video data to the called media. The server, the called media server forwards the calling audio and video data to the called client;

Similarly, the called client sends the called audio and video data to the called media server, and the called media server forwards the called audio and video data to the calling media server, and the calling media server will be called. The video data is forwarded to the calling client;

As shown in Figure 3, this method achieves the fastest connection between the calling client and the called client.

Then, when the calling client and the called client are connected through the 0th channel, the first channel, the second channel, and the third channel can also be connected between the calling client and the called client:

Wherein, as shown in FIG. 4, the data transmission path of the first channel is: the calling client sends the calling audio and video data to the calling end media server, and then the calling media server forwards the calling audio and video data to the called party. Calling the client; likewise, the called client sends the called audio and video data to the called media server, and then the called media server forwards the called audio and video data to the calling client;

As shown in FIG. 5, the data transmission path of the second channel is: the calling client sends the calling audio and video data to the called media server, and then the called media server forwards the calling audio and video data to the called client. Similarly, the called client sends the called audio and video data to the calling media server, and then the calling media server forwards the called audio and video data to the calling client;

As shown in FIG. 6, the data transmission path of the third channel is: a point-to-point transmission mechanism between the calling client and the called client;

Finally, when the four transmission paths of the 0th channel, the 1st channel, the 2nd channel, and the 3rd channel are established, the calling client or/and the called client simultaneously send the network quality detection packet to the above four channels. Therefore, the network quality of the above four channels is detected, and the network path quality scores of the above four channels are respectively obtained, and finally the channel with the highest network path quality score is selected between the primary client and the called client to perform audiovisual data. Transmission.

After the four transmission paths of the 0th channel, the 1st channel, the 2nd channel, and the 3rd channel are established, the network quality detection packet of the currently applied channel between the calling client and the called client is added to the master. Calling audio and video data or a 5-byte custom packet header before the called audio and video data, as shown in Figure 7, the custom packet header contains the header type, channel number Path, sequence number Seqno, timestamp Timestamp, The data stream type Stream type; and the network quality probe packets sent by the other three channels include the foregoing custom packet header plus IP/UDP packet header. The definition of the IP/UDP packet header is a conventional technique, and will not be described in detail here. The network quality probe packet sent by the channel is 60 bytes, and the network quality probe packets sent by the calling client or/and the called client to the four channels are sent simultaneously, and four times are sent simultaneously to the four channels. The total size of the network quality detection packet does not exceed 5% of the calling audio and video data or the called audio and video data. This method can achieve very fast quality status update frequency, which is easy to understand. A small amount of data, the network path will inevitably reduce the frequency of monitoring, the network sensitivity to changes in the signal quality decreases.

Traditional RTP/RTCP does not consider continuity and data correlation for network quality monitoring. The independence of monitoring functions is high, resulting in low quality update frequency, especially for pure audio communication. If the quality of RTCP is performed separately for each channel. Statistics, while satisfying the constraint of not exceeding 5% of the total flow, in the case of ensuring statistical accuracy The worst case update period is up to about 34 seconds. The method provided in this embodiment sends a network quality probe packet to four channels at the same time, and the total size of the four network quality probe packets does not exceed the calling audio and video data or is Calling 5% of audio and video data can increase the update cycle to about 11 times efficiency. That is, in the case of the same traffic and the same accuracy, the worst case update cycle takes only about 3 seconds to send a larger audio and video bit rate. In this case, the present invention performs quality status update at a frequency of up to 1 time per second, at which time the amount of redundancy of the probe packet is much less than 5%.

In addition, the calling client or/and the called client are simultaneously sent to the four channels to send a network quality detection packet at a fixed time, thereby re-detecting the network quality of the above four channels, thereby obtaining the above four channels respectively. The network path quality score, the main network client and the called client select the channel with the highest network path quality score for the transmission of audiovisual data.

Finally, the present invention also solves the problem of delay estimation in a single direction of four channels. The traditional RTCP method can only calculate the round-trip time delay rtt (Round-Trip Time) of the path, and the calculation method is a conventional method in the prior art. The round-trip path delay rtt of the path includes the sum of delays in both directions of the sender and the receiver, so the delay in a single direction cannot be accurately evaluated. The present invention can obtain the relative delay in a single direction on the four channels, and obtain 4 pieces. The quality status of the path in a single direction is used to make the best choice. Specifically, the network path quality score is obtained by the following formula.

Score=factor(path)*(clip3(0,100,Yd)*clip3(0,100,Y1))/100

See Figure 8, where score is the network path quality score, clip3 is a finite function, that is, to ensure that the output value is between 0 and 100, factor (path) is the weighting factor of the connection mode, depending on the server cost, different connections The mode factor (path) of the mode is set in advance;

Td(1) represents the unidirectional path delay of the first channel, Td(1)=T1-t1-deltaT, deltaT is the time difference between the calling client and the called client, and T1 is sent by the calling client through the first channel. The moment when the calling audio and video data is called, t1 is the time when the called client receives the calling audio and video data at the moment;

Td(3) indicates the unidirectional path delay of the third channel, Td(3)=T3-t3-deltaT, deltaT is the time difference between the calling client and the called client, and T3 is sent by the calling client through the third channel. The moment when the calling audio and video data is called, t3 is the moment when the called client receives the calling audio and video data at the moment;

Td_min=min(Td(0), Td(1), Td(2), Td(3));

Rtt_min=min(rtt(0), rtt(1), rtt(2), rtt(3)), and rtt(0) represents the round-trip path delay of channel 0; rtt(1) Indicates the round-trip path delay of the first channel; rtt(2) represents the round-trip path delay of the second channel; rtt(3) represents the round-trip path delay of the third channel;

The invention eliminates the time difference deltaT of two clients, obtains a relative one-way delay delay of four paths, and Td_min and rtt_min are iteratively updated through the statistical window, and another advantage of the present invention is that only according to each channel With one piece of data, the result can be obtained, which greatly saves the flow of the probe packet;

If the cost is not considered, the factor (path) is always equal to 1, and simply consider the forwarding cost of the server. You can set the weighting factor of the 0th channel to 1, and the weighting factor of the 1st channel and the 2nd channel to 2, the 3rd channel. The weighting factor is 4. In short, according to the size of the server cost, the importance of serving customers, dynamic adjustment of the factor (path).

Claims

A multi-path data transmission method based on quality evaluation, characterized in that;

First, different clients connect to different media servers according to the principle of proximity, and then when the calling client initiates audio and video communication to the called client, the calling client and the called client first pass through the 0th channel. connection;

The nearest media server connected to the calling client is called the calling media server, and the nearest media server connected to the called client is called the called media server;

The data transmission path of the 0th channel is: the calling client sends the calling audio and video data to the calling media server, and the calling media server forwards the calling audio and video data to the called media server, and the called media The server then forwards the calling audio and video data to the called client.

Similarly, the called client sends the called audio and video data to the called media server, and the called media server forwards the called audio and video data to the calling media server, and the calling media server will be called. The video data is forwarded to the calling client;

Then, when the calling client and the called client are connected through the 0th channel, the following channels are also started to be connected between the calling client and the called client;

In the first channel, the data transmission path of the first channel is: the calling client sends the calling audio and video data to the calling media server, and then the calling media server forwards the calling audio and video data to the called client; Similarly, the called client sends the called audio and video data to the called media server, and then the called media server forwards the called audio and video data to the calling client;

The data transmission path of the second channel and the second channel is: the calling client sends the calling audio and video data to the called media server, and then the called media server forwards the calling audio and video data to the called client; Similarly, the called client sends the called audio and video data to the calling media server, and then the calling media server forwards the called audio and video data to the calling client;

The data transmission path of the third channel and the third channel is: a point-to-point transmission mechanism between the calling client and the called client;

Finally, when the four transmission paths of the 0th channel, the 1st channel, the 2nd channel, and the 3rd channel are established, the calling client or/and the called client simultaneously send the network quality detection packet to the above four channels. Therefore, the network quality of the above four channels is detected, and the network path quality scores of the above four channels are respectively obtained, and finally the channel with the highest network path quality score is selected between the primary client and the called client to perform audiovisual data. Transmission.
The multi-path data transmission method based on quality evaluation according to claim 1, wherein the network quality detection packet of the currently applied channel between the calling client and the called client is in the calling audio and video data or is A custom data packet header that is added before the audio and video data. The custom data packet header includes a header type, a channel number, a sequence number, a timestamp, and a data stream type: the network quality probe packet sent by the other three channels includes the foregoing custom data. Header plus IP/UDP packet header.
The multi-path data transmission method based on quality evaluation according to claim 2, wherein the network quality detection packet sent by the calling client or/and the called client to the four channels is simultaneously transmitted, and is sent to the fourth The total size of the four network quality probe packets sent simultaneously by the channel does not exceed 5% of the calling audio and video data or the called audio and video data.
The multi-path data transmission method based on quality evaluation according to claim 2, wherein the size of the custom data packet header is 5 bytes, and the IP/UDP data packet header is 55 bytes.
The multi-path data transmission method based on quality evaluation according to claim 1, wherein the calling client or/and the called client send a network quality detection packet to the four channels at a fixed time interval, thereby The network quality of the above four channels is re-detected, and the new network path quality scores of the above four channels are respectively obtained, and the channel with the highest network path quality score is selected between the main client and the called client for audio-visual The transmission of data.
The multi-path data transmission method based on quality evaluation according to claim 1, wherein the network path quality score is obtained by the following formula:

Score=factor(path)*(clip3(0,100,Yd)*clip3(0,100,Yl))/100

Where score is the network path quality score, clip3 is a finite function, that is, to ensure that the output value is between 0 and 100, factor (path) is the weighting factor of the connection mode, depending on the server cost, the factor (path) of the different connection mode Set in advance;

And Yd=-19.58ln(delay)+149.58, where delay is a relative one-way path delay under different channels, and delay=Td(n)-Td_min+rtt_min/2, n=0, 1, 2, 3;

Td(0) represents the unidirectional path delay of the 0th channel, Td(0)=T0-t0-deltaT, deltaT is the time difference between the calling client and the called client, and T0 is sent by the calling client through the 0th channel. The time of the calling audio and video data, t0 is the time when the called client receives the calling audio and video data at the moment;

Td(1) represents the unidirectional path delay of the first channel, Td(1)=T1-t1-deltaT, deltaT is the time difference between the calling client and the called client, and T1 is sent by the calling client through the first channel. The moment when the calling audio and video data is called, t1 is the time when the called client receives the calling audio and video data at the moment;

Td(2) represents the unidirectional path delay of the second channel, Td(2)=T2-t2-deltaT, deltaT is the time difference between the calling client and the called client, and T2 is sent by the calling client through the second channel. The moment when the calling audio and video data is called, t2 is the moment when the called client receives the calling audio and video data at the moment;

Td(3) indicates the unidirectional path delay of the third channel, Td(3)=T3-t3-deltaT, deltaT is the time difference between the calling client and the called client, and T3 is sent by the calling client through the third channel. The moment when the calling audio and video data is called, t3 is the moment when the called client receives the calling audio and video data at the moment;

Td_min=min(Td(0), Td(1), Td(2), Td(3));

Rtt_min=min(rtt(0), rtt(1), rtt(2), rtt(3)), and rtt(0) represents the round-trip path delay of channel 0; rtt(1) represents the round-trip of channel 1 Path delay; rtt(2) indicates the round-trip path delay of the second channel; rtt(3) indicates the third channel Round trip path delay;

Yl=-29.43ln(PLR)-29.637; PLR is the data packet loss rate under different channels.
The multi-path data transmission method based on quality evaluation according to claim 6, wherein the weighting factor of the media server forwarding mode is 1, the weight of the calling media server forwarding mode and the forwarding mode of the called media server. The factor is 2, and the weighting factor of the point-to-point mode is 4.