CN117880604A

CN117880604A - Data transmission method, device and system based on interactive streaming media

Info

Publication number: CN117880604A
Application number: CN202410102410.0A
Authority: CN
Inventors: 禹俊磊
Original assignee: Shanghai Mihoyo Tianming Technology Co Ltd
Current assignee: Shanghai Mihoyo Tianming Technology Co Ltd
Priority date: 2024-01-24
Filing date: 2024-01-24
Publication date: 2024-04-12

Abstract

The disclosure relates to the field of data processing, and particularly discloses a data transmission method, device and system based on interactive streaming media. The method comprises the following steps: in the process of transmitting stream media data between a stream media server and a stream media client through a current transmission path, dynamically acquiring path transmission parameters of the current transmission path; and under the condition that the dynamic transmission quality of the current transmission path meets the switching condition according to the path transmission parameters, selecting a target transmission path from a plurality of available transmission paths, and switching the target transmission path into the current transmission path so as to transmit the streaming media data between the streaming media server and the streaming media client through the current transmission path. The method can dynamically acquire the path transmission parameters of the current transmission path in the process of transmitting the interactive streaming media data so as to realize dynamic switching among a plurality of available transmission paths, thereby being capable of adapting to the change conditions of various network states and improving the speed and the reliability of data transmission.

Description

Data transmission method, device and system based on interactive streaming media

Technical Field

The embodiment of the disclosure relates to the technical field of data processing, in particular to a data transmission method, device and system.

Background

Conventional streaming media forms, such as watching a movie on line, watching a television show, simply play non-real-time audio and video pictures, and the user cannot operate and get any feedback. In interactive streaming media, the video stream can be changed in real time according to the user's operation to generate feedback. For example, a cloud game dynamically transmits a real-time image of the game to a user according to the operation of the game by the user.

In interactive streaming media, higher requirements are put on the transmission speed and the transmission reliability between the client and the server. In the conventional manner, the transmission path between the client and the server is fixed, but the manner cannot adapt to the dynamically changing requirement of the network.

Disclosure of Invention

In view of the foregoing, the present disclosure is directed to providing a method, apparatus, and system for interactive streaming media-based data transmission that overcomes or at least partially solves the foregoing problems.

According to an aspect of the embodiments of the present disclosure, there is provided a data transmission method, including:

dynamically acquiring path transmission parameters of a current transmission path in the process of transmitting streaming media data between a streaming media server and a streaming media client through the current transmission path; the current transmission path is obtained by screening a plurality of available transmission paths between the streaming media server and the streaming media client, and the available transmission paths correspond to a plurality of extranet addresses of the streaming media server;

And under the condition that the dynamic transmission quality of the current transmission path meets the switching condition according to the path transmission parameters, selecting a target transmission path from the available transmission paths, and switching the target transmission path into the current transmission path so as to transmit the streaming media data between the streaming media server and the streaming media client through the current transmission path.

In an optional implementation manner, the streaming media server has M external network addresses, and the number of the available transmission paths is N;

before dynamically acquiring the path transmission parameters of the current transmission path in the process of transmitting the streaming media data between the streaming media server and the streaming media client through the current transmission path, the method further comprises the following steps:

transmitting a detection data packet through M candidate transmission paths corresponding to the M external network addresses respectively, and acquiring path transmission parameters of each candidate transmission path in the process of transmitting the detection data packet through each candidate transmission path so as to determine the initial transmission quality of each candidate transmission path;

according to the initial transmission quality of the M candidate transmission paths, screening N available transmission paths from the M candidate transmission paths; n, M are natural numbers, and N is smaller than M;

The data packet format and/or the data packet data amount of the detection data packet are determined according to the data type of the streaming media data to be transmitted.

In an alternative implementation, the path transmission parameters include: a transmission duration parameter and a packet loss rate parameter;

the determining the dynamic transmission quality of the current transmission path according to the path transmission parameters includes:

acquiring a plurality of parameter values corresponding to transmission duration parameters of a plurality of data packets, and determining a preset operation result of the plurality of parameter values; wherein, the preset operation result comprises at least one of the following: an expected operation result, a variance operation result, a maximum value operation result, and an average value operation result;

acquiring a time length weight value corresponding to the transmission time length parameter and a packet loss rate weight value corresponding to the packet loss rate parameter;

and determining the dynamic transmission quality of the current transmission path according to the duration weight value, the packet loss rate weight value, the parameter value of the packet loss rate parameter and the preset operation result.

In an alternative implementation manner, the duration weight value corresponding to the transmission duration parameter and the packet loss rate weight value corresponding to the packet loss rate parameter are determined by the following ways:

Determining a result distribution interval to which a preset operation result of the plurality of parameter values belongs, and determining a first weight value corresponding to the result distribution interval as the duration weight value according to a preset first interval weight mapping table; the first interval weight mapping table is used for storing mapping relations between a plurality of result distribution intervals and corresponding first weight values;

determining a numerical distribution interval to which a parameter value of the packet loss rate parameter belongs, and determining a second weight value corresponding to the numerical distribution interval as the packet loss rate weight value according to a preset second interval weight mapping table; the second interval weight mapping table is used for storing mapping relations between a plurality of numerical distribution intervals and corresponding second weight values.

In an alternative implementation, the streaming media data is data in a cloud game application;

and when the game type of the cloud game application corresponding to the streaming media data is the first type, the duration weight value is larger than the packet loss rate weight value;

and under the condition that the game type of the cloud game application corresponding to the streaming media data is the second type, the duration weight value is smaller than the packet loss rate weight value.

In an optional implementation manner, after the switching the target transmission path to the new current transmission path, the method further includes: setting a polled mark for the current transmission path and the target transmission path, and recording the dynamic transmission quality of the current transmission path;

the selecting a target transmission path from the plurality of available transmission paths comprises:

in the case that the available transmission paths contain the available transmission paths without the poll mark, selecting a target transmission path from the available transmission paths without the poll mark according to the initial transmission quality of the available transmission paths without the poll mark;

and when the available transmission paths are all available transmission paths provided with the polled marks, selecting a target transmission path from the available transmission paths provided with the polled marks according to the dynamic transmission quality of each available transmission path provided with the polled marks.

In an optional implementation manner, the setting the polled flag for the current transmission path and the target transmission path, and after recording the dynamic transmission quality of the current transmission path, further includes:

And deleting the polled mark of the transmission path after detecting that the setting time of the polled mark of any transmission path exceeds the preset validity period, and emptying the recorded dynamic transmission quality of the transmission path.

In an alternative implementation manner, the dynamically acquiring the path transmission parameter of the current transmission path includes:

and acquiring path transmission parameters of the current transmission path in the current time period every preset time period.

According to still another aspect of the embodiments of the present disclosure, there is provided a data transmission apparatus based on interactive streaming media, including:

the acquisition module is suitable for dynamically acquiring path transmission parameters of the current transmission path in the process of transmitting streaming media data between the streaming media server and the streaming media client through the current transmission path; the current transmission path is obtained by screening a plurality of available transmission paths between the streaming media server and the streaming media client, and the available transmission paths correspond to a plurality of extranet addresses of the streaming media server;

and the switching module is suitable for screening a target transmission path from the available transmission paths under the condition that the dynamic transmission quality of the current transmission path meets the switching condition according to the path transmission parameters, and switching the target transmission path into the current transmission path so as to transmit the streaming media data between the streaming media server and the streaming media client through the current transmission path.

the acquisition module is further adapted to: transmitting a detection data packet through M candidate transmission paths corresponding to the M external network addresses respectively, and acquiring path transmission parameters of each candidate transmission path in the process of transmitting the detection data packet through each candidate transmission path so as to determine the initial transmission quality of each candidate transmission path;

the switching module is specifically adapted to:

In an alternative implementation, the switching module is further adapted to:

According to still another aspect of the present disclosure, there is provided an electronic apparatus including: the device comprises a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface complete communication with each other through the communication bus;

the memory is configured to store at least one executable instruction, where the executable instruction causes the processor to perform the data transmission method as described above.

According to yet another aspect of the present disclosure, there is provided a computer storage medium having stored therein at least one executable instruction for causing a processor to perform a data transmission method as described above.

In the embodiment of the disclosure, the path transmission parameters of the current transmission path can be dynamically obtained in the transmission process of the interactive streaming media data; and under the condition that the dynamic transmission quality of the current transmission path meets the switching condition according to the path transmission parameters, selecting a target transmission path from a plurality of available transmission paths, and switching the target transmission path into the current transmission path so as to transmit the streaming media data between the streaming media server and the streaming media client through the switched current transmission path. Therefore, the method can dynamically acquire the path transmission parameters of the current transmission path in the process of transmitting the interactive streaming media data so as to realize dynamic switching among a plurality of available transmission paths, thereby being capable of adapting to the change conditions of various network states and improving the speed and the reliability of data transmission.

The foregoing description is merely an overview of the technical solutions of the present disclosure, and may be implemented according to the content of the specification in order to make the technical means of the present disclosure more clearly understood, and in order to make the above and other objects, features and advantages of the present disclosure more clearly understood, the following specific embodiments of the present disclosure are specifically described.

Drawings

Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are only for purposes of illustrating the preferred embodiments and are not to be construed as limiting the disclosure. Also, like reference numerals are used to designate like parts throughout the figures. In the drawings:

FIG. 1 illustrates a flow chart of a method of data transmission provided by one embodiment of the present disclosure;

FIG. 2 illustrates a flow chart of a method of data transmission provided by one specific example of the present disclosure;

fig. 3 is a schematic diagram of a data transmission device according to another embodiment of the disclosure;

fig. 4 is a schematic structural diagram of an electronic device according to another embodiment of the present disclosure.

Detailed Description

Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

Fig. 1 shows a flowchart of a data transmission method based on interactive streaming media according to an embodiment of the present disclosure. As shown in fig. 1, the method specifically includes:

step S110: in the process of transmitting stream media data between a stream media server and a stream media client through a current transmission path, dynamically acquiring path transmission parameters of the current transmission path; the current transmission path is obtained by screening a plurality of available transmission paths between the streaming media server and the streaming media client, and the available transmission paths correspond to a plurality of extranet addresses of the streaming media server.

Wherein the interactive streaming media application comprises: cloud gaming applications, remote desktop applications, virtual exhibition hall applications, and the like. In interactive streaming media, streaming media data (e.g., video streams) can be fed back based on real-time user-triggered changes in interactive operations.

Wherein, in order to facilitate providing reliable data transmission services for various types of clients, the streaming media server has a plurality of extranet addresses. Wherein each computer on the internet has an independent IP address that uniquely identifies a computer on the internet. The external network address is public network IP, which means that the IP address except the reserved IP address can be accessed with other computers on the Internet at will. Therefore, the streaming media server in this embodiment is a multi-host. By multi-homing host is meant: a computer (called a host) has a plurality of network connections and external network addresses. Wherein the plurality of external network addresses may correspond to different operators, respectively, so as to provide communication services for clients of the respective operators. It can be seen that each external network address of the streaming media server corresponds to at least one transmission path between the streaming media server and the streaming media client, and thus, there are multiple transmission paths between the streaming media server and the streaming media client.

The current transmission path is one of a plurality of available transmission paths between the streaming media server and the streaming media client. The number of available transmission paths may be equal to the number of transmission paths between the streaming server and the streaming client (i.e., all transmission paths are available), or the number of available transmission paths may be smaller than the number of transmission paths between the streaming server and the streaming client (i.e., part of transmission paths are unavailable). Specifically, the manner of determining whether the transmission path is available may be flexibly set by those skilled in the art, which is not limited in this application. In addition, in the initial stage, the manner of screening the current transmission path from the multiple available transmission paths can be flexibly set by those skilled in the art, which is not defined in the application.

In order to dynamically evaluate the quality of the current transmission path, path transmission parameters of the current transmission path are dynamically acquired in the transmission process of streaming media data. The path transmission parameters may be one or more, and are used to characterize the transmission quality of the network path, for example, may be various types such as delay transmission parameters, packet loss transmission parameters, jitter transmission parameters, and the like.

Step S120: and under the condition that the dynamic transmission quality of the current transmission path meets the switching condition according to the path transmission parameters, selecting a target transmission path from a plurality of available transmission paths, and switching the target transmission path into the current transmission path so as to transmit the streaming media data between the streaming media server and the streaming media client through the current transmission path.

Wherein, according to the path transmission parameter, the dynamic transmission quality of the current transmission path can be determined. The dynamic transmission quality means: the transmission quality of the current transmission path is dynamically determined, and the transmission quality obtained for different time periods is typically different. The dynamic transmission quality may be obtained by a preset operation, for example, various operations such as an average value taking operation and a maximum value taking operation may be performed on the parameter values of the path transmission parameters, so as to obtain the dynamic transmission quality. In summary, the present application does not limit the manner of determining the dynamic transmission quality, as long as the current transmission state of the transmission path can be characterized.

The switching condition can be flexibly set, so long as the transmission state of the current transmission path can be identified to be not ideal. For example, a handover threshold may be set, and when a comparison result between the score value of the dynamic transmission quality and the handover threshold is a preset result, it is determined that the dynamic transmission quality satisfies the handover condition. In the case where the dynamic transmission quality of the current transmission path satisfies the switching condition, the target transmission path is selected from among the plurality of available transmission paths, the target transmission path is switched to a new current transmission path, and the operation in step S110 is repeatedly performed.

Therefore, the method can dynamically acquire the path transmission parameters of the current transmission path in the process of transmitting the interactive streaming media data so as to realize dynamic switching among a plurality of available transmission paths, thereby being capable of adapting to the change conditions of various network states and improving the speed and the reliability of data transmission. In addition, because the transmission process of the interactive streaming media data is an actual service process, the path transmission parameters acquired in the process are consistent with the actual service process, and the method has higher accuracy.

Various modifications and alterations to the above embodiments will also occur to those skilled in the art:

in an alternative implementation, to ensure that the quality of the transmission paths meets the requirements, several available transmission paths may be screened in advance. The streaming media server has M external network addresses, and the number of available transmission paths is N. Correspondingly, before the path transmission parameters of the current transmission path are dynamically acquired in the process of transmitting the streaming media data between the streaming media server and the streaming media client through the current transmission path, the following operations are further executed: and sending the detection data packets through M candidate transmission paths corresponding to the M external network addresses respectively. In the process of transmitting the probe data packet through each candidate transmission path, the path transmission parameter of each candidate transmission path is acquired to determine the initial transmission quality of each candidate transmission path. The path transmission parameters of the candidate transmission paths are determined according to the transmission states of the candidate transmission paths when the probe data packets are transmitted. The initial transmission quality of the candidate transmission path is used for characterizing the transmission quality of the candidate transmission path in an initial state (i.e. before actually transmitting the streaming media data), and can be calculated by various operation modes. And screening N available transmission paths from the M candidate transmission paths according to the initial transmission quality of the M candidate transmission paths. The initial transmission quality of the M candidate transmission paths may be ranked, and the N available transmission paths may be screened according to the ranking result. Wherein N, M is a natural number, and N is smaller than M. The method can screen available transmission paths through the detection data packets, so that paths with unqualified transmission quality are removed, and the efficiency of subsequent path switching is improved.

In order to make the initial transmission quality closer to the actual traffic transmission state, the packet format and/or the packet data amount of the probe packet may be determined according to the data type of the streaming media data to be transmitted. For example, the data amount of the probe packet may be made consistent with the data type of the streaming media data to be transmitted, so that the probe packet can simulate a real streaming media packet. Therefore, by improving the format of the detection data packet, the detection data packet can be ensured to be more close to the on-line actual service data, so that the detection accuracy is improved.

In an alternative implementation, the path transmission parameters include: a transmission duration parameter, and a packet loss rate parameter. Accordingly, when determining the dynamic transmission quality of the current transmission path according to the path transmission parameters, the following manner may be implemented:

firstly, a plurality of parameter values corresponding to transmission time length parameters of a plurality of data packets are obtained, and a preset operation result of the plurality of parameter values is determined. The transmission duration parameter may specifically be a round trip duration parameter, and accordingly, a parameter value of the round trip duration parameter is used to characterize a total duration consumed by a round trip of the data packet. Wherein, the preset operation result comprises at least one of the following: the desired operation result, the variance operation result, the maximum operation result, and the average operation result. Considering that the sensitivity of the game to time delay is higher, the transmission quality can be measured more accurately by calculating the expected, variance or maximum value of the round trip time length of a plurality of data packets. In the conventional averaging method, the distribution balance of the round trip time of a plurality of data packets is not considered, in an extreme case, the round trip time of some data packets is very large, and the round trip time of other data packets is very small, so that the exception cannot be seen according to the average value of the round trip time, but the serious overtime situation of the time delay of individual data packets cannot be tolerated in the game service scene, so that the embodiment can effectively identify the data packets with serious overtime by means of the operation modes such as expectation, variance, maximum value and the like.

Then, a time length weight value corresponding to the transmission time length parameter and a packet loss rate weight value corresponding to the packet loss rate parameter are obtained. Different weights can be set for the transmission duration parameter and the packet loss rate parameter, so that the influence of the corresponding parameter value in the final calculation result can be flexibly adjusted.

And finally, determining the dynamic transmission quality of the current transmission path according to the time length weight value, the packet loss rate weight value, the parameter value of the packet loss rate parameter and a preset operation result. The parameter value of the packet loss rate parameter is used for representing the packet loss condition, the preset operation result is used for representing the transmission delay condition, and the parameter value of the packet loss rate parameter and the preset operation result are subjected to weight adjustment (such as weighting) through the duration weight value and the packet loss rate weight value, so that the final dynamic transmission quality is obtained.

In an alternative implementation, the duration weight value corresponding to the transmission duration parameter and the packet loss rate weight value corresponding to the packet loss rate parameter may be determined by:

determining a result distribution interval to which a preset operation result of a plurality of parameter values belongs, and determining a first weight value corresponding to the result distribution interval as a duration weight value according to a preset first interval weight mapping table; the first interval weight mapping table is used for storing mapping relations between a plurality of result distribution intervals and corresponding first weight values. In the first interval weight mapping table, the smaller the result distribution interval is, the smaller the first weight value is; the larger the result distribution interval, the larger the first weight value. Namely: the first weight value is proportional to the result distribution interval.

Determining a numerical distribution interval to which a parameter value of the packet loss rate parameter belongs, and determining a second weight value corresponding to the numerical distribution interval as a packet loss rate weight value according to a preset second interval weight mapping table; the second interval weight mapping table is used for storing mapping relations between a plurality of numerical distribution intervals and corresponding second weight values. In the second interval weight mapping table, the smaller the numerical distribution interval is, the smaller the second weight value is; the larger the numerical distribution interval, the larger the second weight value. Namely: the second weight value is proportional to the value distribution interval.

It can be seen that the first weight value and the second weight value may also be referred to as step factors, the values of which differ with the change of the distribution interval to which the values belong. In view of the fact that a larger distribution interval corresponds to a poor transmission quality, the first weight value and the second weight value are set in a stepwise fashion in order to promote the differentiation of specific scores of transmission quality.

In an alternative implementation, the streaming media data is data in a cloud gaming application. And in the case that the game type of the cloud game application corresponding to the streaming media data is the first type, the duration weight value is larger than the packet loss rate weight value; and under the condition that the game type of the cloud game application corresponding to the streaming media data is the second type, the duration weight value is smaller than the packet loss rate weight value. Therefore, the calculation mode of the transmission quality can be different according to different game types, so that the characteristics of different types of games can be flexibly adapted. Wherein the first type comprises: the games with high real-time interaction requirements, such as games with interactive operation types, such as open world adventure games, multi-player online competitive games and the like, are more sensitive to transmission delay, so the time length weight value is larger. The second type includes: games with high requirements on pictures and lower real-time requirements, such as Xiaoqie and other single games, are more sensitive to picture quality, but the requirements on time delay are slightly lower because online interaction with other people is not needed, so that the weight value of the packet loss rate is larger for the games of the second type.

In an alternative implementation, after switching the target transmission path to the current transmission path, the following operations are further performed: the polled flag is set for the current transmission path and the target transmission path, and the dynamic transmission quality of the current transmission path is recorded. Accordingly, when the target transmission path is selected from the plurality of available transmission paths, in the case where the available transmission paths for which the polled flag is not set are included in the plurality of available transmission paths, the target transmission path is selected from the available transmission paths for which the polled flag is not set according to the initial transmission quality of the available transmission paths for which the polled flag is not set; in the case where the plurality of available transmission paths are all available transmission paths provided with the polled flag, a target transmission path is selected from among the available transmission paths provided with the polled flag according to the dynamic transmission quality of each of the available transmission paths provided with the polled flag. Therefore, by setting the polled mark and recording the dynamic transmission quality, a better path can be preferentially selected from the non-polled available transmission paths as the current transmission path, so that all the available transmission paths can be ensured to be selected.

In an alternative implementation, the validity period may be set for the dynamic transmission quality of the path, considering that the specific value of the transmission quality will dynamically change with network changes, and therefore the transmission quality earlier in time will no longer have a reference value. Accordingly, after setting the polled flag for the current transmission path and the target transmission path and recording the dynamic transmission quality of the current transmission path, the following operations are further performed: after detecting that the setting time of the polled mark of any transmission path exceeds the preset validity period, deleting the polled mark of the transmission path, and clearing the recorded dynamic transmission quality of the transmission path.

In an alternative implementation, the path transmission parameters of the current transmission path are dynamically acquired by: and acquiring path transmission parameters of the current transmission path in the current time period every preset time period. The length of the time period can be flexibly adjusted according to the requirements in specific service scenes. And repeating the step of dynamically acquiring the path transmission parameters of the current transmission path and the subsequent steps in the process of transmitting the streaming media data between the streaming media server and the streaming media client through the switched current transmission path. Therefore, in the embodiment, the path transmission parameters of the current transmission path can be periodically acquired in the transmission process of the streaming media data, so that the dynamic switching of the transmission path is continuously realized, and the good transmission efficiency in the whole streaming media data transmission process is ensured.

For ease of understanding, the implementation of the above embodiments is described below by taking an example as an example. The example is mainly applied to the field of cloud games, wherein the cloud games are a game mode in which games are run and rendered on a remote server, and users receive video streams in real time through a network and send control signals. The cloud game has the advantages that the performance requirement of the user equipment is reduced, and a user adopting equipment with lower performance can obtain high-quality game experience. However, since cloud games deploy the running and rendering processes of the games on remote servers, the cloud games have very high demands on network latency and bandwidth, thereby ensuring a good game experience.

The following is an explanation of some concepts involved in this example:

cloud gaming: the cloud game is a game mode based on cloud computing, and in the running mode of the cloud game, all games run at a server side, and rendered game pictures are compressed and then transmitted to a user through a network. At the client, the user's game device does not need any high-end processor and graphics card, but only needs basic video decompression capability.

Audio video streaming: and synchronously transmitting the audio and video pictures on the equipment A to the equipment B for playing.

WebRTC: is a real-time communication technology that allows network applications or sites to establish Peer-to-Peer (Peer-to-Peer) connections between browsers without the aid of intermediaries, enabling the transmission of video streams and/or audio streams or any other data.

External network address: namely, public network IP is an IP address except for a reserved IP address, and can be randomly accessed to other computers on the Internet. The IP address is referred to as the public network IP. Each computer on the internet has an independent IP address that uniquely identifies a computer on the internet.

A multi-host: a computer (called a host) has a plurality of network connections and external network addresses.

RTT: round trip time, the network requests the time required to reach destination B from origin a and return to origin a again.

Packet loss rate: during the process of transmitting and receiving network data, the data packet is not transmitted to the application program for various reasons and is discarded. The number of discarded packets divided by the total number of transmission packets is the packet loss rate.

In application scenes such as cloud games and remote desktops, transmitted data is large in flow, session duration is long, and tolerance to transmission delay is low. Therefore, this type of traffic scenario has a higher demand for the quality of network transmission. In order to provide better cloud game service, the server side can configure a plurality of external network addresses and belongs to different operators, and correspondingly, the server side can select the optimal external network address to communicate with according to the different network operators to which the user belongs. In the existing audio and video streaming mode, webRTC is mostly adopted as an audio and video transmission technology of a bottom layer, when the WebRTC processes the selection of multi-host source IP, connectivity of a heavier path is better seen, and under the condition of the same transmission protocol, the best path is selected by comparing RTTs of all paths.

In the related art, when WebRTC selects an "optimal" path among a plurality of external network addresses of a server, there are at least the following problems:

(1) The consideration is single and not comprehensive enough: and judging according to RTT values of different paths only, and not considering the packet loss rate. However, the packet loss rate also has a great influence on the quality of transmission.

(2) The accuracy of the measurement results is not high: in the related art, before transmitting game data, the transmission quality of each path is measured by transmitting a probe packet. However, since the probe packet traffic is very small, the data traffic when RTT is measured is very small. The inventors have found that the magnitude of the load traffic on the path affects the transmission quality of the path to some extent. Therefore, in the case where the line load in the measurement phase is greatly different from the line load in the actual transmission phase, the measured value cannot represent the actual value at the time of transmitting the audio/video data.

(3) The dynamic change of the network environment cannot be adapted: in the related art, once the "best" path is selected, other paths are not considered. However, the network link may change all the time, and a certain path is better at the initial stage of the session, and may not be maintained all the time in the subsequent time. Therefore, the above manner cannot adapt to dynamically changing business scenarios in cloud games.

In order to solve the above-mentioned problems, the present example proposes a data transmission method based on interactive streaming media, and fig. 2 shows a flow chart of the method, and as shown in fig. 2, the method includes the following steps:

step S201: the probe packet is transmitted through a plurality of candidate transmission paths, and an initial transmission quality of each candidate transmission path is calculated.

Specifically, the probe data packet is sent through M candidate transmission paths corresponding to M external network addresses, and in the process of transmitting the probe data packet through each candidate transmission path, the path transmission parameter of each candidate transmission path is obtained, so as to determine the initial transmission quality of each candidate transmission path.

The path transmission parameters of the candidate transmission paths include: the initial transmission quality of the candidate transmission path is also called network transmission quality, the initial transmission quality can be recorded as q, the RTT and the packet loss rate are counted in real time after each path is established, the network transmission quality of the path is estimated according to the RTT and the packet loss rate every second, and the network transmission quality is recorded as q.

Wherein packet loss rate lossrate=the total number of pongs received in the counting period/the total number of pings sent in the counting period.

RTT = reception time of Pong-transmission time of Ping.

q＝Avg(RTT)*(1+200*LossRate)；

Wherein Avg represents an average value of RTTs of a plurality of data packets, and 200 is a customizable constant.

Step S202: and selecting a plurality of available transmission paths from the plurality of candidate transmission paths according to the initial transmission quality of the plurality of candidate transmission paths, and selecting a current transmission path from the plurality of available transmission paths.

For example, N available transmission paths are screened from the M candidate transmission paths according to initial transmission quality of the M candidate transmission paths.

In the early stage of the session, a path with the smallest q value can be selected as a current transmission path according to the q value of each path, so as to be used for transmitting the audio and video data stream. Then, other paths are screened, and paths with small differences from the minimum q value are marked as alternative paths (namely available transmission paths).

Wherein the gap is not as large as can be measured by the following formula:

q≤q _min *(1+0.2)+10

wherein q _min The initial transmission quality of the path with the smallest q value.

Step S203: in the process of transmitting stream media data between a stream media server and a stream media client through a current transmission path, path transmission parameters of the current transmission path are dynamically acquired.

The path transmission parameters mainly refer to RTT parameters and packet loss rate parameters. Because the line load condition and the network real-time state of the current transmission path when transmitting the streaming media data are different from those of the transmission of the detection data packet, the path transmission parameters of the current transmission path need to be dynamically acquired in real time.

Step S204: and judging whether the dynamic transmission quality of the current transmission path meets the switching condition according to the path transmission parameters.

Specifically, after the selected current transmission path transmits a data stream for a period of time, the dynamic transmission quality of the current transmission path, which is also called network transmission quality, is estimated according to RTT and packet loss rate, and the current transmission path is marked as a polled path (i.e. a polled mark is set for the current transmission path).

Where the requirement of the best network is typically Avg (RTT) <=40 ms and LossRate < =1%.

Q= (c×avg (RTT)) (k×lossrate), where C and K are step factors.

When Avg (RTT) <=40 ms, the value of C is minimum (e.g., 0.5). The larger the Avg (RTT) value, the larger the C value. When Avg (RTT) > = 200ms, the C value reaches a maximum (e.g. 4.0).

When LossRate < = 1%, the value of K is minimum (0.1). The larger the LossRate value, the larger the K value. When LossRate > =20%, the K value reaches a maximum (e.g. 3.0).

The switching condition may be that the Q value is greater than a preset value. Since the larger the Q value is, the worse the transmission quality is, it is determined that the dynamic transmission quality of the current transmission path satisfies the switching condition in the case that the Q value is larger than the preset value. For example, if the Q value is small (e.g., Q < =23), the steps S203 and S204 are continued. Step S203 may be a step performed periodically, and the execution period may be 1 minute. For another example, if the Q value is large (e.g., Q > 23), it is determined that the dynamic transmission quality of the current transmission path satisfies the switching condition.

Step S205: and under the condition that the dynamic transmission quality of the current transmission path meets the switching condition according to the path transmission parameters, selecting a target transmission path from a plurality of available transmission paths, and switching the target transmission path into the current transmission path.

If there is an available transmission path which is not marked as a polled path, selecting a path with the smallest q value from the available transmission paths which are not marked as the polled paths as a target transmission path, and circularly executing the steps. If all available transmission paths are marked as polled paths, selecting a path with the minimum Q value from all polled paths as a target transmission path, and circularly executing the steps.

Step S206: the polled mark is set for the current transmission path after switching, and the dynamic transmission quality of the current transmission path is recorded.

Step S207: after detecting that the setting time of the polled mark of any transmission path exceeds the preset validity period, deleting the polled mark of the transmission path, and clearing the recorded dynamic transmission quality of the transmission path.

The Q value of each transmission path is time-efficient, and after a certain period of time (e.g., 3 min), the corresponding Q value is cleared, and the polled flag of the polled path is deleted.

In summary, this example has at least the following advantages:

(1) In this example, the operation action of optimal path selection and "optimal sorting selection" is not a one-time action, but the sorting action is periodically repeated and the whole session process is traversed, so that the change of the network can be better dealt with.

(2) The ordering logic between the multiple paths is optimized, dividing the ordering into two phases. The first stage obtains a preliminary sequencing result by adopting a traditional detection packet mode, and screens available transmission paths meeting the conditions according to the preliminary sequencing result to enter the next stage. In the second stage, the selected available transmission paths are sequentially polled to become selected paths for transmitting real data (i.e. streaming media data). And after a short period of time, evaluating the transmission performance of the path load, and switching the next path to transmit data. And finally, selecting an optimal path according to the transmission performance of the load, and more conforming to the actual use condition compared with the transmission performance of the path which is only measured when the path is empty.

In a word, the method improves the stability of cloud game services, and can automatically switch to the next available external network address when an external network address of a service end machine is abnormally unavailable in the middle of a session. In addition, the network transmission quality is improved slightly, and especially when the operator used by the user client is a non-mainstream small operator (such as a non-mobile, communication and telecom operator), the operator to which the external network address of the cloud service end belongs cannot be successfully matched with the operator used by the client, so that the transmission path cannot be selected by directly using the matching mode of the operator, and by adopting the scheme in the example, the route selection can be better realized in the scene.

Fig. 3 is a schematic diagram of a data transmission device according to another embodiment of the present disclosure, where the data transmission device is suitable for a data transmitting end in an interactive streaming media application, and the device specifically includes:

the obtaining module 31 is adapted to dynamically obtain path transmission parameters of a current transmission path in a process of transmitting streaming media data between a streaming media server and a streaming media client through the current transmission path; the current transmission path is obtained by screening a plurality of available transmission paths between the streaming media server and the streaming media client, and the available transmission paths correspond to a plurality of extranet addresses of the streaming media server;

and the switching module 32 is adapted to screen a target transmission path from the available transmission paths and switch the target transmission path to the current transmission path, so as to transmit the streaming media data between the streaming media server and the streaming media client through the current transmission path, if the dynamic transmission quality of the current transmission path meets the switching condition according to the path transmission parameter.

the switching module is specifically adapted to:

In an alternative implementation, the switching module is further adapted to:

Fig. 4 shows a schematic structural diagram of an electronic device according to another embodiment of the disclosure, and the specific embodiment of the disclosure is not limited to the specific implementation of the electronic device.

As shown in fig. 4, the electronic device may include: a processor 502, a communication interface (Communications Interface) 504, a memory 506, and a communication bus 508.

Wherein:

processor 502, communication interface 504, and memory 506 communicate with each other via communication bus 508.

A communication interface 504 for communicating with network elements of other devices, such as clients or other servers.

The processor 502 is configured to execute the program 510, and may specifically perform relevant steps in the above-described embodiment of the method for detecting a video image.

In particular, program 510 may include program code including computer-operating instructions.

The processor 502 may be a central processing unit CPU, or a specific integrated circuit ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement embodiments of the present disclosure. The one or more processors comprised by the electronic device may be the same type of processor, such as one or more CPUs. But may also be different types of processors such as one or more CPUs and one or more ASICs.

A memory 506 for storing a program 510. Memory 506 may comprise high-speed RAM memory or may also include non-volatile memory (non-volatile memory), such as at least one disk memory.

The program 510 may be specifically configured to cause the processor 502 to perform the respective operations corresponding to the above-described embodiments of the video image detection method.

The algorithms and displays presented herein are not inherently related to any particular computer, virtual machine, or other apparatus. Various general purpose devices may also be used with the teachings herein. The required structure for the construction of such devices is apparent from the description above. In addition, the present disclosure is not directed to any particular programming language. It will be appreciated that the disclosure described herein may be implemented in a variety of programming languages, and the above description of specific languages is provided for disclosure of enablement and best mode of the present disclosure.

In the description provided herein, numerous specific details are set forth. However, it is understood that embodiments of the disclosure may be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure an understanding of this description.

Similarly, it should be appreciated that in the above description of exemplary embodiments of the disclosure, various features of the disclosure are sometimes grouped together in a single embodiment, figure, or description thereof for the purpose of streamlining the disclosure and aiding in the understanding of one or more of the various disclosed aspects. However, the disclosed method should not be construed as reflecting the intention that: i.e., the claimed disclosure requires more features than are expressly recited in each claim.

Those skilled in the art will appreciate that the modules in the apparatus of the embodiments may be adaptively changed and disposed in one or more apparatuses different from the embodiments. The modules or units or components of the embodiments may be combined into one module or unit or component and, furthermore, they may be divided into a plurality of sub-modules or sub-units or sub-components. Any combination of all features disclosed in this specification (including any accompanying claims, abstract and drawings), and all of the processes or units of any method or apparatus so disclosed, may be used in combination, except insofar as at least some of such features and/or processes or units are mutually exclusive. Each feature disclosed in this specification (including any accompanying claims, abstract and drawings), may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise.

Furthermore, those skilled in the art will appreciate that while some embodiments herein include some features but not others included in other embodiments, combinations of features of different embodiments are meant to be within the scope of the disclosure and form different embodiments. For example, any of the claimed embodiments can be used in any combination.

Various component embodiments of the present disclosure may be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. Those skilled in the art will appreciate that some or all of the functions of some or all of the components in an apparatus according to embodiments of the present disclosure may be implemented in practice using a microprocessor or Digital Signal Processor (DSP). The present disclosure may also be embodied as a device or apparatus program (e.g., computer program and computer program product) for performing a portion or all of the methods described herein. Such a program embodying the present disclosure may be stored on a computer readable medium, or may have the form of one or more signals. Such signals may be downloaded from an internet website, provided on a carrier signal, or provided in any other form.

It should be noted that the above-mentioned embodiments illustrate rather than limit the disclosure, and that those skilled in the art will be able to design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The disclosure may be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by one and the same item of hardware. The use of the words first, second, third, etc. do not denote any order. These words may be interpreted as names.

Claims

1. A data transmission method based on interactive streaming media comprises the following steps:

2. The method of claim 1, wherein the streaming server has M external network addresses, and the number of available transmission paths is N;

3. The method of claim 1 or 2, wherein the path transmission parameters include: a transmission duration parameter and a packet loss rate parameter;

4. The method of claim 3, wherein the duration weight value corresponding to the transmission duration parameter and the packet loss rate weight value corresponding to the packet loss rate parameter are determined by:

5. The method of claim 3 or 4, wherein the streaming media data is data in a cloud gaming application;

6. The method according to any one of claims 1-5, wherein after the switching the target transmission path to a new current transmission path, further comprising: setting a polled mark for the current transmission path and the target transmission path, and recording the dynamic transmission quality of the current transmission path;

7. The method of claim 6, wherein the setting of the polled flag for the current transmission path and the target transmission path, and after recording the dynamic transmission quality of the current transmission path, further comprises:

8. The method according to any one of claims 1-7, wherein the dynamically acquiring path transmission parameters of the current transmission path comprises:

9. A data transmission apparatus based on interactive streaming media, comprising:

10. An electronic device, comprising: the device comprises a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface complete communication with each other through the communication bus;

the memory is configured to store at least one executable instruction that causes the processor to perform the data transmission method according to any one of claims 1-8.

11. A computer storage medium having stored therein at least one executable instruction for causing a processor to perform the data transmission method of any one of claims 1-8.