CN110830821A

CN110830821A - RTP (real-time transport protocol) stream transmission method and device based on slices, terminal and server

Info

Publication number: CN110830821A
Application number: CN201810923636.1A
Authority: CN
Inventors: 洪家明; 李涛
Original assignee: Hytera Communications Corp Ltd
Current assignee: Hytera Communications Corp Ltd
Priority date: 2018-08-14
Filing date: 2018-08-14
Publication date: 2020-02-21

Abstract

The application provides a slice-based RTP stream transmission method, a slice-based RTP stream transmission device, a terminal and a server, wherein the server stores slice-based RTP data with different code rate specifications aiming at the same RTP stream file. And when the client detects that the current network condition changes, determining a new target code rate specification matched with the changed network condition. And then, requesting RTP slice data corresponding to the new target code rate specification from the server. Thus, RTP data with corresponding code rate specification can be transmitted in a self-adaptive manner according to the change of the network condition. When the network condition is better, the RTP data with higher code rate specification is transmitted, and when the network condition is worse, the RTP data with lower code rate specification can be switched to in time, so that the transmission process of the RTP stream file is smooth, and the user experience is improved.

Description

RTP (real-time transport protocol) stream transmission method and device based on slices, terminal and server

Technical Field

The invention belongs to the technical field of streaming media, and particularly relates to a slice-based RTP streaming transmission method, a slice-based RTP streaming transmission device, a slice-based RTP streaming transmission terminal and a slice-based RTP streaming transmission server.

Background

Real-time Transport Protocol (RTP) is a Real-time streaming Protocol. The RTP protocol is widely used for streaming media related communications and entertainment including telephony, video conferencing, video surveillance, television and network-based push-to-talk services (walkie-talkie-like calls).

At present, the RTP stream transmission is performed by taking the whole RTP stream as a unit, a server tries to send and a client tries to receive, and after determining the RTP stream to be transmitted, parameters of the RTP stream, such as a code rate, cannot be changed according to a network condition. Taking the transmission of the video file as an example, when the transmission network is degraded, the RTP stream may not be transmitted normally, and the client plays the video file, a pause phenomenon may occur, which seriously affects the viewing experience of the user.

Disclosure of Invention

In view of the above, the present invention provides a slice-based RTP streaming method, apparatus, terminal and server to solve the technical problem that the current RTP streaming method cannot adapt to network status transmission. The technical scheme is as follows:

in a first aspect, the present application provides a slice-based RTP streaming method, applied to a client, including:

sending a playing request for playing an RTP stream file based on RTP streams to a server, wherein the playing request comprises a file identifier of a target RTP stream file, the RTP stream file comprises at least one RTP stream, each RTP stream comprises a plurality of RTP slices, and each RTP slice has a unique slice identifier;

receiving all code rate specifications of the target RTP stream file returned by the server, wherein the code rate specifications are obtained by the server from the server according to the file identification;

determining a target code rate specification matched with the current network condition, and requesting content data corresponding to the target code rate specification from the server;

when the current network condition is determined to be changed, determining a new target code rate specification matched with the changed network condition;

and requesting the RTP data corresponding to the new target code rate specification from the server according to the slice identifier of the RTP slice data being received.

Optionally, requesting, from the server, file content data corresponding to the target bitrate includes:

requesting a slice identification list and slice time information of an RTP slice contained in the RTP data matched with the target code rate specification from the server;

according to the slice time information, determining a target slice identifier corresponding to a target time from the slice identifier list;

and requesting RTP slice data corresponding to the target slice identification from the server.

Optionally, requesting, from the server, RTP data corresponding to the new target bitrate specification according to a slice identifier of RTP slice data being received, where the requesting includes:

generating a code rate switching request according to the slice identifier of the RTP slice data being received and the new target code rate specification;

sending the code rate switching request to the server so that the server can switch the RTP slice data corresponding to the new target code rate specification;

and receiving RTP slice data corresponding to the new target code rate specification returned by the server.

In a second aspect, the present application further provides a slice-based RTP streaming method, applied in a server, including:

receiving a playing request sent by a client for playing a target RTP stream file based on RTP streams, wherein the playing request comprises a file identifier of the target RTP stream file, the RTP stream file comprises at least one RTP stream, each RTP stream comprises at least one RTP slice, and each RTP slice has a slice identifier for positioning the RTP slice;

analyzing the playing request to obtain a file identifier of a target RTP stream file;

acquiring all code rate specifications of the target RTP stream file stored in the server, and returning all the code rate specifications to the client;

receiving a first request which is sent by a client and contains a target code rate specification, wherein the target code rate specification is determined by the client according to the current network condition of a network where the client is located;

according to the first request, returning RTP data of a target code rate specification corresponding to a target RTP stream file to the client;

and after receiving a second request which is sent by the client and contains a new target code rate specification, returning RTP data corresponding to the new target code rate specification to the client, wherein the new target code rate specification is determined by the client according to the change condition of a network where the client is located.

Optionally, the first request further includes a target time requested by the client;

returning the RTP data of the target code rate specification corresponding to the target RTP stream file to the client, wherein the RTP data comprises:

analyzing the first request to obtain the target time;

searching a target slice identifier of an RTP slice corresponding to the target moment from the RTP data corresponding to the target code rate specification;

reading target RTP slice data containing the target slice identification, and returning the target RTP slice data to the client.

Optionally, the second request includes a target slice identifier and a new target bitrate specification of RTP slice data currently being received by the client;

after receiving a second request containing a new target bitrate specification sent by the client, the method returns RTP data corresponding to the new target bitrate specification to the client, and includes:

analyzing the second request to obtain a target slice identifier and a new target code rate specification of the RTP slice data currently received by the client;

analyzing the target slice identifier of the RTP slice data currently being received to obtain the serial number of the RTP slice data currently being received in the RTP data with the target code rate specification;

determining that a next sequence number of the sequence numbers is a target sequence number;

searching target RTP slice data corresponding to the target serial number from the RTP data corresponding to the new target code rate specification;

and sending the target RTP slice data to the client.

reading target RTP slice data corresponding to the sequence number from the RTP data corresponding to the new target code rate specification;

determining an unreceived portion of the RTP slice data currently being received by the client, and sending a portion of the target RTP slice data corresponding to the unreceived portion to the client.

In a third aspect, the present application provides another slice-based RTP streaming method, applied in a server, including:

receiving a playing request sent by a client, wherein the playing request comprises a file identifier of an RTP stream file;

determining all code rate specifications of the RTP stream file stored in the server according to the file identifier, and determining a target code rate specification;

returning RTP slice data corresponding to the target code rate specification to the client;

judging whether the network condition between the server and the client changes or not;

when the network condition of the network between the server and the client is determined to be changed, determining a new target code rate specification matched with the changed network condition;

determining a new target slice identifier of RTP slice data to be sent from the RTP slice data with the new target code rate specification according to the receiving result of the previous RTP slice data and the incidence relation among the RTP slices;

and reading the RTP slice data corresponding to the new target slice identification, and returning the read RTP slice data to the client.

Optionally, the determining whether the network condition between the server and the client changes includes:

receiving feedback receiving information returned by the client;

determining the packet loss rate of the data received by the client according to the receiving feedback information;

determining the network condition between the server and the client according to the packet loss rate;

when the packet loss rate is higher than a first threshold value, determining that the network condition between the server and the client side is poor;

and when the packet loss rate is lower than a second threshold, determining that the network condition between the server and the client side becomes good, wherein the first threshold is higher than the second threshold.

Optionally, the method further comprises: and when the packet loss rate is lower than a second threshold value, retransmitting the packet loss.

In a fourth aspect, the present application provides a slice-based RTP streaming apparatus, applied to a client, including:

a first request sending module, configured to send a play request for playing an RTP stream file based on an RTP stream to a server, where the play request includes a file identifier of a target RTP stream file, where the RTP stream file includes at least one RTP stream, each RTP stream includes a plurality of RTP slices, and each RTP slice has a unique slice identifier;

the first receiving module is used for receiving all code rate specifications of the target RTP stream file returned by the server, and the code rate specifications are obtained by the server from the server according to the file identification;

the first code rate determining module is used for determining a target code rate specification matched with the current network condition;

the first data acquisition module is used for requesting the server for the content data corresponding to the target code rate specification;

the second code rate determining module is used for determining a new target code rate specification matched with the changed network condition when the current network condition is determined to be changed;

and the second data acquisition module is used for requesting the server for the RTP data corresponding to the new target code rate specification according to the slice identifier of the RTP slice data being received.

In a fifth aspect, the present application provides a slice-based RTP streaming apparatus, applied to a server, including:

a first receiving module, configured to receive a play request sent by a client for playing a target RTP stream file based on RTP streams, where the play request includes a file identifier of the target RTP stream file, the RTP stream file includes at least one RTP stream, each RTP stream includes at least one RTP slice, and each RTP slice has a slice identifier for locating the RTP slice;

the analysis module is used for analyzing the playing request to obtain the file identifier of the target RTP stream file;

the code rate acquisition module is used for acquiring all code rate specifications of a target RTP stream file stored in the server and returning all the code rate specifications to the client;

the second receiving module is used for receiving a first request which is sent by the client and contains a target code rate specification, wherein the target code rate specification is determined by the client according to the current network condition of the network where the client is located;

a first data returning module, configured to return, to the client, RTP data of a target bitrate specification corresponding to the target RTP stream file according to the first request;

and the second data return module is used for returning RTP data corresponding to the new target code rate specification to the client after receiving a second request which is sent by the client and contains the new target code rate specification, wherein the new target code rate specification is determined by the client according to the change condition of a network where the client is positioned.

In a sixth aspect, the present application further provides a slice-based RTP streaming apparatus, applied in a server, including:

a first receiving module, configured to receive a play request sent by a client, where the play request includes a file identifier of an RTP stream file;

a first code rate determining module, configured to determine all code rate specifications of the RTP stream file stored in the server according to the file identifier, and determine a target code rate specification;

a first data returning module, configured to return RTP slice data corresponding to the target bitrate specification to the client;

the judging module is used for judging whether the network condition between the server and the client changes or not;

the second code rate determining module is used for determining a new target code rate specification matched with the changed network condition after determining that the network condition of the network between the server and the client is changed;

the slice identifier determining module is used for determining a new target slice identifier of the RTP slice data to be sent from the RTP slice data with the new target code rate specification according to the receiving result of the previous RTP slice data and the incidence relation among the RTP slices;

and the second data returning module is used for reading the RTP slice data corresponding to the new target slice identifier and returning the read RTP slice data to the client.

In a seventh aspect, the present application further provides a terminal device, including a memory and a processor, where the memory stores a program executable on the processor, and the processor executes the program to implement the method described in any one of the possible implementation manners of the first aspect.

In an eighth aspect, the present application further provides a server, including a memory and a processor, where the memory stores a program executable on the processor, and the processor executes the program to implement the method described in any one of the possible implementation manners of the second aspect.

In the RTP streaming method based on slices provided in this embodiment, the server stores the RTP data based on slices with different bitrate specifications for the same RTP streaming file. When the server transmits the RTP stream based on the slice to the client, the server can switch and transmit the RTP data of the code rate specification matched with the current network condition according to the change condition of the network condition. The client determines a target code rate specification matched with the current network condition and requests RTP data corresponding to the target code rate specification from a server; and when the client detects that the current network condition changes, determining a new target code rate specification matched with the changed network condition. And then, according to the slice identifier of the RTP slice data which is being received, requesting the RTP slice data corresponding to the new target code rate specification from the server. Thus, RTP data with corresponding code rate specification can be transmitted in a self-adaptive manner according to the change of the network condition. When the network condition is better, the RTP data with higher code rate specification is transmitted, and when the network condition is worse, the RTP data with lower code rate specification can be switched to in time, so that the transmission process of the RTP stream file is smooth, and the user experience is improved.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to these drawings without creative efforts.

Fig. 1 is a schematic diagram illustrating a format of a slice-based RTP composite stream according to an embodiment of the present application;

fig. 2 is a schematic flowchart of a slice-based RTP streaming method according to an embodiment of the present application;

fig. 3 is a flowchart of a process of acquiring RTP data with a target bitrate specification according to an embodiment of the present application;

fig. 4 is a flowchart of a process of acquiring RTP slice data with a new target bitrate specification according to an embodiment of the present application;

fig. 5 is a schematic diagram of adaptive transmission of an RTP stream based on slicing according to an embodiment of the present application;

fig. 6 is a schematic flowchart of another slice-based RTP streaming method according to an embodiment of the present application;

fig. 7 is a flowchart illustrating a process of determining a network condition by a server according to an embodiment of the present application;

fig. 8 is a block diagram of a slice-based RTP streaming apparatus according to an embodiment of the present application;

fig. 9 is a block diagram of another slice-based RTP transmission apparatus according to an embodiment of the present application;

fig. 10 is a block diagram of another slice-based RTP streaming apparatus according to an embodiment of the present application.

Detailed Description

In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

Referring to fig. 1, a schematic format diagram of a slice-based RTP composite stream according to an embodiment of the present application is shown, in which an RTP streaming media is stored in a server in a slice manner.

As shown in fig. 1, each RTP stream is concatenated by a plurality of independent slices (referred to herein as RTP slices), each RTP slice being independently decoded.

Each RTP slice includes a description area and one or more RTP independent decoding units, wherein each RTP independent decoding unit includes one or more RTP data packets, the one or more RTP independent decoding units being RTP slice data; the description area is an independent index packet attached in front of the RTP slice data, having a specific RTP payload type. The description area comprises a flow association description area, a slice association description area and a slice attribute description area;

the stream association description area stores stream association information used for describing which association type exists between the current RTP stream and the RTP streams, and the association type comprises composition, synchronization and self-adaptation; the adaptive association refers to an association relationship between RTP streams based on RTP slices with different code rates corresponding to the same RTP stream file.

The slice association description area stores slice association information used for describing 1-2 precursor slices and 1 subsequent slice of the current RTP slice;

the slice attribute description area is used for describing slice attribute information of a current RTP slice, and comprises the following steps: slice size, slice name, slice type, slice sequence number, timestamp, encryption information, etc. The transmission process requires that the data packets are as compact as possible, so the information in the description area is as compact as possible.

In one embodiment of the present application, the timestamp of an RTP slice is implemented to be uniformly defined, which may be a relative time or an absolute time. Wherein the absolute time and the relative time are both based on the same time reference to ensure that the timestamps of the associated RTP streams remain consistent.

The slice related information connects a series of RTP slices in series to form RTP streaming media based on slices, and each RTP slice is independently decoded, namely, the RTP slices with different media formats, decoding parameters and the like can exist in the same RTP stream, and the transmission and the playing are not influenced.

The stream association information associates a plurality of RTP streams based on the RTP slices to form a composite RTP stream, and can associate and transmit, associate and store, associate and position, associate and play, associate and encrypt and the like the plurality of RTP streams with a certain relationship. For example, the synchronization association of the same source RTP stream may define lip-sync, primary and secondary stream sync, etc.

The slice-based RTP streaming transmission method provided by the present application mainly realizes the adaptive transmission of RTP streams according to the network conditions on the basis of the RTP streaming media format shown in fig. 1.

Referring to fig. 2, a flow chart of a slice-based RTP streaming method according to an embodiment of the present application is shown, where the method is applied to a system including a client and a server. And as for the same RTP stream file, RTP data with a plurality of code rate specifications are stored in the server.

In this embodiment, the client may actively pull the RTP stream file from the server. As shown in fig. 2, the method may include the steps of:

s110, the client sends a playing request for playing the RTP stream file based on the RTP stream to the server.

The playback request includes a file identifier of the RTP stream file. Wherein the RTP stream file includes at least one RTP stream, each RTP stream including a plurality of RTP slices, each RTP slice having a slice identification for locating the RTP slice.

S120, the server analyzes the playing request sent by the client to obtain the target file identification of the target RTP stream file.

S130, the server obtains the target code rate specification of the target RTP stream file stored by the server and sends the target code rate specification to the client.

S140, the client determines a target code rate specification matched with the current network condition and sends a first request containing the target code rate specification to the server.

The first request is used for requesting the server for the RTP data of the target code rate specification corresponding to the target RTP stream file.

S150, the server obtains the RTP data with the target code rate specification corresponding to the target RTP stream file according to the first request and sends the RTP data to the client.

In an embodiment of the present application, the first request includes a target time requested by the client, where the target time may be a time at which the client determines to play an RTP stream file according to a user operation, or may be a system default play start time (which may be a start time or a last viewing deadline).

Usually, an RTP stream file has a certain playing duration, for example, a video playing duration of a video file, and the starting playing time refers to which time in the RTP stream file the playing starts. For example, if the playing time of a video file is 3min, and the user specifies that the starting time is 1 minute and 20 seconds, the video file is played from the 1 minute and 20 seconds of the video file.

As shown in fig. 3, the process of the server acquiring the RTP data with the target bitrate specification corresponding to the target RTP stream file may include S151 to S153:

s151, the server analyzes the first request to obtain the target time.

S152, the server searches the target slice identification of the RTP slice corresponding to the target moment from the RTP data of the target code rate specification corresponding to the target RTP stream file.

In one embodiment of the present application, each RTP slice data includes therein time offset information of the RTP slice, i.e., a time difference between the RTP slice data and the start of the first RTP slice data. And determining the playing time of the RTP slice in the whole RTP stream file according to the time offset information.

And S153, the server reads the target RTP slice data corresponding to the target slice identifier and sends the target RTP slice data to the client.

And when the client starts playing or drags the playing progress control to a new position for playing, the client requests according to the playing time. In the subsequent continuous playing process, the server returns a slice identifier list and slice time information of the RTP slice to the client, wherein the slice time information includes time offset information of the RTP slice. And the client determines a target slice identifier from the slice identifier list according to the slice time information and requests the server for RTP slice data corresponding to the target slice identifier. Therefore, the server does not need to store the relation between the RTP slice and the corresponding playing time, and can directly inquire the corresponding RTP slice according to the slice identifier sent by the client, so that the inquiry efficiency of the server is improved.

S160, when the client detects that the current network status changes, determining a new target bitrate specification matching the changed network status.

The client can sense the network condition of the network where the client is located in real time. And if the current network condition changes, determining a new target code rate specification matched with the changed network condition from all code rate specifications corresponding to the target RTP stream file.

And S170, the client generates a second request according to the slice identifier of the RTP slice data being received and the new target code rate specification, and sends the second request to the server.

The second request is used for requesting the server to switch the target RTP stream file from the RTP data with the target bitrate specification to the RTP data with the new target bitrate specification.

And the second request generated by the client comprises the slice identifier of the RTP slice data currently received by the client and the determined new target bitrate specification.

And S180, the server acquires the RTP data of the new target code rate specification corresponding to the target RTP stream file according to the second request and sends the RTP data to the client.

And the server determines the slice identifier of the RTP slice data with the new target code rate specification according to the slice identifier of the RTP slice data currently received by the client.

The RTP slice identifier generally includes code rate information and sequence number information of the RTP slice, where the sequence numbers of RTP slice data of different code rate specifications at the same playing time of the same RTP stream file are the same.

In one embodiment of the present application, the switch to the new target bitrate specification is started from the next slice of RTP slice data currently being received by the client. As shown in fig. 4, S180 may include:

and S181, analyzing the second request to obtain the target slice identifier and the new target bitrate specification of the RTP slice data currently being received by the client.

The slice id of an RTP slice usually contains two parts, one part is the bitrate specification information and the other part is the slice sequence number. The serial numbers of RTP slices at the same time in RTP streams with different code rate specifications corresponding to the same RTP stream file are the same.

S182, analyzing the target slice identification of the RTP slice data currently being received, and obtaining the serial number of the RTP slice data currently being received in the RTP data with the target code rate specification.

And S183, determining the next sequence number of the sequence numbers as the target sequence number.

S184, searching the RTP data corresponding to the target serial number from the RTP data corresponding to the new target code rate specification.

And S185, sending the target RTP slice data to the client.

Referring to fig. 5, a schematic diagram of adaptive transport of an RTP stream based on slices according to an embodiment of the present application is shown, as shown in fig. 5, an audio stream and a video stream are associated in a synchronous manner, and the video stream includes RTP streams with multiple bitrate specifications (e.g., 500K, 1M, and 3M).

Wherein, an RTP slice in an RTP stream with a code rate of 500K can be represented as: 500K slice 1, 500K slice 2, 500K slice 3, … …, 500K slice n;

an RTP slice within an RTP stream with a code rate of 1M may be represented as: 1M slice 1, 1M slice 2, 1M slice 3, … …, 1M slice n;

an RTP slice within an RTP stream with a code rate of 3M may be represented as: 3M slice 1, 3M slice 2, 3M slice 3, … …, 3M slice n.

When the network condition of the client is good, the server can be requested to transmit the RTP stream with the code rate of 3M, so that when the network bandwidth is sufficient, the RTP stream with the higher code rate is transmitted, and a user can watch a clearer video. When the network condition of the client side is poor, the RTP stream with the lower code rate of 1M or 500K is requested to be transmitted according to the transmission result of the previous slice and the incidence relation between the slices, so that the phenomenon of video playing blockage can not occur, and the video watching process of a user is smoother. The matching code rate may be determined based on network conditions.

When the network condition of the client side is improved, the client side can request to transmit the 1M or 3M RTP stream with higher code rate according to the transmission result of the previous slice and the incidence relation between the slices, and the definition of the video is improved on the premise of ensuring the smoothness of the video playing process.

In the method for transmitting a slice-based RTP stream according to this embodiment, the server stores slice-based RTP data with different bitrate specifications for a same RTP stream file, and in this case, when the server transmits the slice-based RTP stream to the client, the server can switch to transmit the RTP data with the bitrate specification matching the current network condition according to the change condition of the network condition. The client determines a target code rate specification matched with the current network condition and requests RTP data corresponding to the target code rate specification from a server; and when the client detects that the current network condition changes, determining a new target code rate specification matched with the changed network condition. And then, according to the slice identifier of the RTP slice data which is being received, requesting the RTP slice data corresponding to the new target code rate specification from the server. Thus, the RTP data of the corresponding code rate specification can be transmitted in a self-adaptive way according to the change of the network condition. When the network condition is better, the RTP data with higher code rate specification is transmitted, and when the network condition is worse, the RTP data with lower code rate specification can be switched to in time, so that the transmission process of the RTP stream file is smooth, and the user experience is improved.

Referring to fig. 6, a flow chart of another slice-based RTP streaming method according to an embodiment of the present application is shown; in this embodiment, the server actively sends an RTP stream matched with the changed bandwidth to the client according to the bandwidth change condition of the network in which the server is located. As shown in fig. 6, the method may include the steps of:

s210, the client sends a playing request to the server.

The play request contains the file identification of the target RTP stream file.

S220, the server obtains all code rate specifications of the target RTP stream file stored in the server according to the file identification, and determines the target code rate specification.

The server determines that the default code rate specification is a target code rate specification, wherein the default code rate specification can be the code rate specification of the RTP stream file transmitted to the client last time, or a certain default fixed code rate specification of the system.

The server can also determine a matched code rate specification as a target code rate specification of the transmission according to the network condition between the server and the client.

And S230, the server returns the RTP slice data corresponding to the target code rate specification to the client.

S240, the server judges whether the network condition of the network where the server is located changes; if the change occurs, S250 is performed; if no change occurs, go to S290;

and the server judges the network condition of the network in which the server is positioned according to the receiving feedback information fed back by the client. As shown in fig. 7, the process of the server determining the network condition of the network where the server is located may include: s241 to S245:

and S241, the server receives the feedback receiving information returned by the client.

After receiving the RTP data sent by the server, the client reports the situation of receiving the RTP data to the server periodically by receiving feedback information (such as RTCP and NACK). The received feedback information includes data packet loss information.

And S242, the server determines the packet loss rate of the data received by the client according to the receiving feedback information.

The server can calculate the packet loss rate of the data received by the client according to the data packet loss information contained in the receiving feedback information fed back by the client.

S243, the server judges the network condition between the server and the client according to the packet loss rate; if the packet loss rate is higher than the first threshold, performing S244; if the packet loss rate is lower than the second threshold, S245 is executed.

S244, it is determined that the network condition between the server and the client is deteriorated.

If the packet loss rate of the client receiving the RTP data is high and the packet is densely lost, it indicates that the network condition between the server and the client is not good, and at this time, the code rate specification of the RTP stream to be transmitted is to be reduced, and the server may actively select the RTP data which is lower by one level than the currently transmitted code rate specification.

S245, it is determined that the network condition between the server and the client becomes good.

If the packet loss rate of the client receiving the RTP data is low and the packet is lost occasionally, it indicates that the network condition between the service and the client is good, and a packet loss retransmission policy can be started to ensure that the client receives complete RTP data.

S250, the server determines a new target code rate specification matched with the changed network condition.

S260, the server determines a new target slice identifier from the RTP data of the new target code rate specification according to the receiving result of the previous RTP slice data and the incidence relation among the RTP slices.

In an embodiment of the present application, if the previous RTP slice data client fails to receive the previous RTP slice data, a new target slice identifier corresponding to the previous RTP slice data in the RTP data with the new target bitrate specification is determined.

In another embodiment of the present application, if the previous RTP slice data client receives successfully, a new target slice identifier corresponding to the next RTP slice data adjacent to the previous RTP slice data in the RTP slice data of the new target bitrate specification is determined.

S270, the server reads RTP slice data corresponding to the new target slice identification.

S280, the server sends the RTP slice data to the client.

S290, the server continues to transmit the RTP slice data corresponding to the current target code rate specification.

In the RTP streaming method based on slices provided in this embodiment, the server stores the RTP data based on slices with different bitrate specifications for the same RTP streaming file. The server actively sends RTP data to the client, and when the server detects that the network condition between the server and the client changes, a new target code rate specification is determined according to the changed network condition. And determining a new target slice identifier from the RTP data corresponding to the new target specification according to the receiving result of the previous RTP slice data and the slice association relation. And reading RTP slice data corresponding to the new target slice identifier and sending the RTP slice data to the client. The method realizes the self-adaptive transmission of the RTP data with the corresponding code rate specification according to the change of the network condition. When the network condition is better, the RTP data with higher code rate specification is transmitted, and when the network condition is worse, the RTP data with lower code rate specification can be switched to in time, so that the transmission process of the RTP stream file is smooth, and the user experience is improved.

Corresponding to the above method embodiments, the present application further provides slice-based RTP streaming apparatus embodiments.

Referring to fig. 8, a block diagram of a slice-based RTP streaming apparatus applied in a client according to an embodiment of the present application is shown. In this embodiment, the client actively pulls the RTP stream file from the server.

As shown in fig. 8, the apparatus includes: the first request sending module 110, the first receiving module 120, the first code rate determining module 130, the first data obtaining module 140, the second code rate determining module 150, and the second data obtaining module 160.

A first request sending module 110, configured to send a playing request for playing an RTP stream file based on an RTP stream to a server.

The playback request includes a file identification of a target RTP stream file, wherein the RTP stream file includes at least one RTP stream, each RTP stream includes a plurality of RTP slices, and each RTP slice has a unique slice identification.

A first receiving module 120, configured to receive all code rate specifications of the target RTP stream file returned by the server, where the code rate specifications are obtained by the server from the server according to the file identifier.

A first bitrate determining module 130, configured to determine a target bitrate specification matching a current network condition.

The first data obtaining module 140 is configured to request the server for content data corresponding to the target bitrate specification.

When the client starts playing or drags the playing progress control to a new position for playing, the client requests according to the playing time. For example, a playing request sent by the client to the server includes a target time, and the server determines an RTP slice corresponding to the target time and returns content data of the RTP slice to the client; meanwhile, the server returns a slice identification list and slice time information to the client, wherein the slice time information comprises the time offset information of the RTP slice. And in the subsequent continuous playing process, the client determines a target slice identifier corresponding to the current playing time from the slice identifier list according to the slice time information, and requests the server for content data corresponding to the target slice identifier. Therefore, the server does not need to store the relation between the RTP slice and the corresponding playing time, and can directly inquire the corresponding RTP slice according to the slice identifier sent by the client, so that the inquiry efficiency of the server is improved.

And a second code rate determining module 150, configured to determine a new target code rate specification matching the changed network condition when it is determined that the current network condition changes.

The second data obtaining module 160 is configured to request, from the server, RTP data corresponding to the new target bitrate specification according to the slice identifier of the RTP slice data being received.

In an embodiment of the present application, the second data obtaining module 160 is specifically configured to: generating a code rate switching request according to the slice identification of the RTP slice data being received and the new target code rate specification; sending a code rate switching request to a server so that the server can switch from the RTP slice data corresponding to the new target code rate specification; and receiving RTP slice data corresponding to the new target code rate specification returned by the server.

In the apparatus for transmitting RTP streams based on slices provided in this embodiment, the server stores the RTP data based on slices with different bitrate specifications for the same RTP stream file, and in this case, when the server transmits the RTP streams based on slices to the client, the server can switch to transmit the RTP data with bitrate specifications matched with the current network condition according to the change condition of the network condition. The client determines a target code rate specification matched with the current network condition and requests RTP data corresponding to the target code rate specification from a server; and when the client detects that the current network condition changes, determining a new target code rate specification matched with the changed network condition. And then, according to the slice identifier of the RTP slice data which is being received, requesting the RTP slice data corresponding to the new target code rate specification from the server. Thus, the RTP data of the corresponding code rate specification can be transmitted in a self-adaptive way according to the change of the network condition. When the network condition is better, the RTP data with higher code rate specification is transmitted, and when the network condition is worse, the RTP data with lower code rate specification can be switched to in time, so that the transmission process of the RTP stream file is smooth, and the user experience is improved.

Referring to fig. 9, a block diagram of another slice-based RTP transmission apparatus according to an embodiment of the present application is shown, where the apparatus is applied to a server, and in this embodiment, a client actively pulls an RTP stream file from the server.

As shown in fig. 9, the server may include a first receiving module 210, a parsing module 220, a code rate obtaining module 230, a second receiving module 240, a first data returning module 250, and a second data returning module 260.

A first receiving module 210, configured to receive a playing request sent by a client to play a target RTP stream file based on an RTP stream.

The playing request comprises a file identifier of a target RTP stream file; the RTP stream file includes at least one RTP stream, each RTP stream including at least one RTP slice, each RTP slice having a slice identification for locating the RTP slice.

The parsing module 220 is configured to parse the play request to obtain a file identifier of the target RTP stream file.

A code rate obtaining module 230, configured to obtain all code rate specifications of the target RTP stream file stored in the server, and return all code rate specifications to the client;

the second receiving module 240 is configured to receive a first request containing the target bitrate specification sent by the client.

The target code rate specification is determined by the client according to the current network condition of the network where the client is located.

A first data returning module 250, configured to return, to the client, the RTP data in the target bitrate specification corresponding to the target RTP stream file according to the first request.

In other embodiments, the first request further includes a target time of the client request;

the first data return module is specifically configured to: analyzing the first request to obtain target time; searching a target slice identifier of an RTP slice corresponding to the target moment from RTP data corresponding to the target code rate specification; and reading the target RTP slice data containing the target slice identifier, and returning the target RTP slice data to the client.

And a second data returning module 260, configured to return, to the client, RTP data corresponding to the new target bitrate specification after receiving a second request that is sent by the client and includes the new target bitrate specification.

The new target code rate specification is determined by the client according to the change condition of the network where the client is located.

The second request comprises a target slice identifier of RTP slice data currently received by the client and a new target code rate specification;

in one embodiment, the server switches to the new target bitrate specification starting from the next RTP slice to the RTP slice currently being sent. In this application scenario, the second data returning module 260 is specifically configured to:

analyzing the second request to obtain a target slice identifier and a new target code rate specification of RTP slice data currently received by the client; analyzing a target slice identifier of the RTP slice data currently being received to obtain a serial number of the RTP slice data currently being received in the RTP data with the target code rate specification; determining the next sequence number of the sequence number as a target sequence number; searching target RTP slice data corresponding to the target serial number from the RTP data corresponding to the new target code rate specification; and sending the target RTP slice data to the client.

In another embodiment, the server switches to the new target bitrate specification starting from the unsent part of the RTP slice currently being transmitted. In this application scenario, the second data returning module 260 is specifically configured to:

analyzing the second request to obtain a target slice identifier and a new target code rate specification of the RTP slice data currently received by the client; analyzing a target slice identifier of the RTP slice data currently being received to obtain a serial number of the RTP slice data currently being received in the RTP data with the target code rate specification; reading target RTP slice data corresponding to the serial number from RTP data corresponding to the new target code rate specification; determining an unreceived portion of RTP slice data currently being received by a client, and sending a portion of target RTP slice data corresponding to the unreceived portion to the client.

The slice-based RTP stream transmission apparatus provided in this embodiment implements adaptive transmission of RTP data with a corresponding bitrate specification according to a change in a network condition. When the network condition is better, the RTP data with higher code rate specification is transmitted, and when the network condition is worse, the RTP data with lower code rate specification can be switched to in time, so that the transmission process of the RTP stream file is smooth, and the user experience is improved.

Referring to fig. 10, a block diagram of another slice-based RTP streaming apparatus according to an embodiment of the present application is shown, where the apparatus is applied in a server. In this embodiment, the server actively pushes the RTP stream file to the client.

As shown in fig. 10, the apparatus may include: a first receiving module 310, a first code rate determining module 320, a first data returning module 330, a judging module 340, a second code rate determining module 350, a slice identification determining module 360 and a second data returning module 370.

The first receiving module 310 is configured to receive a play request sent by a client. The playback request contains a file identifier of the RTP stream file.

The first code rate determining module 320 is configured to determine all code rate specifications of the RTP stream file stored in the server according to the file identifier, and determine a target code rate specification.

The first data returning module 330 is configured to return RTP slice data corresponding to the target bitrate specification to the client.

The determining module 340 is configured to determine whether a network status between the server and the client changes.

In one embodiment, the determining module 340 is specifically configured to:

receiving feedback receiving information returned by a client, and determining the packet loss rate of data received by the client according to the feedback receiving information; determining the network condition between the server and the client according to the packet loss rate; when the packet loss rate is higher than a first threshold value, determining that the network condition between the server and the client side is poor; and when the packet loss rate is lower than a second threshold, determining that the network condition between the server and the client side becomes good, wherein the first threshold is higher than the second threshold. When the packet loss rate is lower than the second threshold, the packet loss is retransmitted, and the code rate of the transmitted RTP slice may not be switched.

And a second code rate determining module 350, configured to determine, when it is determined that the network condition of the network between the server and the client changes, a new target code rate specification matching the changed network condition.

And a slice identifier determining module 360, configured to determine, according to the receiving result of the previous RTP slice data and the association relationship between RTP slices, a new target slice identifier of the RTP slice data to be sent from the RTP slice data with the new target bitrate specification.

And a second data returning module 370, configured to read the RTP slice data corresponding to the new target slice identifier, and return the read RTP slice data to the client.

In the slice-based RTP stream transmission apparatus provided in this embodiment, the server stores slice-based RTP data with different bitrate specifications for the same RTP stream file. The server actively sends RTP data to the client, and when the server detects that the network condition between the server and the client changes, a new target code rate specification is determined according to the changed network condition. And determining a new target slice identifier from the RTP data corresponding to the new target specification according to the receiving result of the previous RTP slice data and the slice association relation. And reading RTP slice data corresponding to the new target slice identifier and sending the RTP slice data to the client. The device is used for realizing the self-adaptive transmission of the RTP data with the corresponding code rate specification according to the change of the network condition. When the network condition is better, the RTP data with higher code rate specification is transmitted, and when the network condition is worse, the RTP data with lower code rate specification can be switched to in time, so that the transmission process of the RTP stream file is smooth, and the user experience is improved.

The slice-based RTP streaming apparatus provided in the above embodiments includes a processor and a memory, where the functional modules included in the apparatus are all stored in the memory as program modules, and the processor executes the program modules stored in the memory to implement corresponding functions.

The processor comprises a kernel, and the kernel calls the corresponding program unit from the memory. The kernel can be set to be one or more than one, and the RTP data of the corresponding code rate specification can be transmitted in a self-adaptive mode according to the change of the network condition by adjusting the kernel parameters. When the network condition is better, the RTP data with higher code rate specification is transmitted, and when the network condition is worse, the RTP data with lower code rate specification can be switched to in time, so that the transmission process of the RTP stream file is smooth, and the user experience is improved.

The processor in this document may be a CPU, or an MCU, or a combination of a CPU and an MCU.

The memory may include volatile memory in a computer readable medium, Random Access Memory (RAM) and/or nonvolatile memory such as Read Only Memory (ROM) or flash memory (flash RAM), and the memory includes at least one memory chip.

The application provides a terminal device, which comprises a processor and a memory, wherein the memory is stored with a program which can run on the processor. The processor implements the slice-based RTP streaming method applied to the client described above when running the program stored in the storage.

The present application also provides a storage medium executable by a terminal device, the storage medium storing a program, the program implementing the above-mentioned slice-based RTP streaming method when executed by the terminal device.

The application also provides a server comprising a processor and a memory, the memory having stored therein a program executable on the processor. The processor implements the slice-based RTP streaming method applied to the server described above when running the program stored in the storage.

The present application also provides a storage medium executable by a server, the storage medium storing a program, the program when executed by the server implementing the above-mentioned slice-based RTP streaming method.

While, for purposes of simplicity of explanation, the foregoing method embodiments have been described as a series of acts or combination of acts, it will be appreciated by those skilled in the art that the present invention is not limited by the illustrated ordering of acts, as some steps may occur in other orders or concurrently with other steps in accordance with the invention. Further, those skilled in the art should also appreciate that the embodiments described in the specification are preferred embodiments and that the acts and modules referred to are not necessarily required by the invention.

It should be noted that, in the present specification, the embodiments are all described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments may be referred to each other. For the device-like embodiment, since it is basically similar to the method embodiment, the description is simple, and for the relevant points, reference may be made to the partial description of the method embodiment.

The steps in the method of the embodiments of the present application may be sequentially adjusted, combined, and deleted according to actual needs.

The modules and sub-modules in the device and the terminal in the embodiments of the application can be combined, divided and deleted according to actual needs.

In the several embodiments provided in the present application, it should be understood that the disclosed terminal, apparatus and method may be implemented in other manners. For example, the above-described terminal embodiments are merely illustrative, and for example, the division of a module or a sub-module is only one logical division, and there may be other divisions when the terminal is actually implemented, for example, a plurality of sub-modules or modules may be combined or integrated into another module, or some features may be omitted or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or modules, and may be in an electrical, mechanical or other form.

The modules or sub-modules described as separate parts may or may not be physically separate, and parts that are modules or sub-modules may or may not be physical modules or sub-modules, may be located in one place, or may be distributed over a plurality of network modules or sub-modules. Some or all of the modules or sub-modules can be selected according to actual needs to achieve the purpose of the solution of the present embodiment.

In addition, each functional module or sub-module in the embodiments of the present application may be integrated into one processing module, or each module or sub-module may exist alone physically, or two or more modules or sub-modules may be integrated into one module. The integrated modules or sub-modules may be implemented in the form of hardware, or may be implemented in the form of software functional modules or sub-modules.

Finally, it should also be noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.

The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

The foregoing is only a preferred embodiment of the present invention, and it should be noted that, for those skilled in the art, various modifications and decorations can be made without departing from the principle of the present invention, and these modifications and decorations should also be regarded as the protection scope of the present invention.

Claims

1. A slice-based RTP streaming method is applied to a client, and is characterized by comprising the following steps:

2. The method of claim 1, wherein requesting the server for the file content data corresponding to the target bitrate comprises:

3. The method of claim 1, wherein requesting the server for the RTP data corresponding to the new target bitrate specification according to the slice identifier of the RTP slice data being received comprises:

4. A slice-based RTP streaming method applied to a server is characterized by comprising the following steps:

5. The method of claim 4, wherein the first request further comprises a target time of the client request;

analyzing the first request to obtain the target time;

6. The method of claim 4, wherein the second request comprises a target slice identifier and a new target bitrate specification for RTP slice data currently being received by the client;

and sending the target RTP slice data to the client.

7. The method of claim 4, wherein the second request comprises a target slice identifier and a new target bitrate specification for RTP slice data currently being received by the client;

8. A slice-based RTP streaming method applied to a server is characterized by comprising the following steps:

9. The method of claim 8, wherein determining whether a network condition between the server and the client changes comprises:

receiving feedback receiving information returned by the client;

10. The method of claim 9, further comprising: and when the packet loss rate is lower than a second threshold value, retransmitting the packet loss.

11. A slice-based RTP streaming device applied to a client, comprising:

12. A slice-based RTP streaming device applied to a server is characterized by comprising:

13. A slice-based RTP streaming apparatus applied in a server, comprising:

14. A terminal device comprising a memory and a processor, the memory having stored therein a program executable on the processor, the processor implementing the method of any one of claims 1-3 when executing the program.

15. A server, comprising a memory and a processor, the memory having stored therein a program operable on the processor, the processor when executing the program implementing the method of any of claims 4-10.