CN116827920A - Weak network transmission method, system, equipment and medium of streaming media server - Google Patents

Weak network transmission method, system, equipment and medium of streaming media server Download PDF

Info

Publication number
CN116827920A
CN116827920A CN202310948333.6A CN202310948333A CN116827920A CN 116827920 A CN116827920 A CN 116827920A CN 202310948333 A CN202310948333 A CN 202310948333A CN 116827920 A CN116827920 A CN 116827920A
Authority
CN
China
Prior art keywords
transmission
gop sequence
network
frames
media server
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202310948333.6A
Other languages
Chinese (zh)
Inventor
刘笑笑
蔡晓晰
徐子潇
李威
瞿婷
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
CETC 52 Research Institute
Original Assignee
CETC 52 Research Institute
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by CETC 52 Research Institute filed Critical CETC 52 Research Institute
Priority to CN202310948333.6A priority Critical patent/CN116827920A/en
Publication of CN116827920A publication Critical patent/CN116827920A/en
Pending legal-status Critical Current

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/60—Network streaming of media packets
    • H04L65/75—Media network packet handling
    • H04L65/752—Media network packet handling adapting media to network capabilities
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/60—Network streaming of media packets
    • H04L65/65—Network streaming protocols, e.g. real-time transport protocol [RTP] or real-time control protocol [RTCP]

Abstract

The invention relates to the field of data transmission, in particular to a weak network transmission method, a weak network transmission system, weak network transmission equipment and weak network transmission media of a streaming media server, wherein the method comprises the following steps: reading media data to be sent in a streaming media server, dividing the media data into a plurality of GOP sequences, and packaging each frame of the media data into RTP data packets; transmitting RTP data packets in the current GOP sequence to a playing end, and counting the transmission state of the RTP data packets; and calculating available network bandwidth according to the transmission state and determining the number of frames to be transmitted in the next GOP sequence. The invention calculates the available network bandwidth independently by the streaming media server, overcomes the dependence on the feedback of the code stream information of the playing end, determines the number of frames to be sent by the next GOP sequence according to the calculated available network bandwidth, realizes the real-time adjustment of the transmission code rate of the media data, and ensures the adaptability of the streaming media server to the transmission in the weak network environment without the need of code stream retransmission.

Description

Weak network transmission method, system, equipment and medium of streaming media server
Technical Field
The invention relates to the field of data transmission, in particular to a weak network transmission method of a streaming media server.
Background
The audio/video streaming media server plays an important role in the whole video monitoring system as a core part for forwarding audio/video streaming media. The actual deployment network environment of the video monitoring system often has the conditions of limited bandwidth, more access devices, more playing ends and the like, so that the streaming media server is required to have the capability of forwarding in a weak network environment to a certain extent, namely the weak network transmission capability. The weak network transmission strategy has different realization in various open source streaming media frames, such as a strategy of re-transmitting again by adjusting the code rate of the coding end after calculating the available bandwidth and the available frame rate based on the feedback of the playing end of NACK (Negative Acknowledgment); for example, the RTP data packet is buffered and sent, and when the playing end detects the packet loss, the playing end re-requests the strategy of retransmitting the data packet by the server. In general, the core transmission policy of the existing streaming media server against the weak network environment can be summarized as follows: based on the packet loss feedback of the playing end, the server re-transmits the strategy of the data packet again after readjusting the code rate.
However, the existing weak network transmission strategy of the streaming media server has two technical problems: 1. the network condition is calculated by depending on the feedback of the playing end on the code stream information, and the technical problem that the playing end possibly loses packets or delays too much when the playing end feeds the received code stream information (media data information) back to the server under the weak network condition is ignored, so that the server cannot accurately acquire the receiving information of the playing end in real time under the condition that the network is not good; 2. the method of code stream retransmission is adopted to improve the transmission adaptability of the streaming media server in the weak network environment, namely, the streaming media server retransmits the missing media data to the playing end, but the code stream retransmission needs the streaming media server to carry out a large amount of buffering on the media data before transmission, so that a large amount of memory resources of the streaming media server are consumed, and the worse the network condition is, the more buffering is, the larger the memory capacity consumption of the server is.
Disclosure of Invention
The present invention provides a method, a system, a device and a medium for transmitting a weak network of a streaming media server, which aims to solve the technical problems.
The invention adopts the following technical scheme: a weak network transmission method of a streaming media server comprises the following steps:
step 101, reading media data to be sent in a streaming media server, dividing the media data to be sent into a plurality of groups of video frames, marking each group of video frames as a GOP sequence, slicing each frame of the media data and packaging the frames into RTP data packets;
step 102, sending RTP data packets in the current GOP sequence to a playing end, and counting the sending state of the RTP data packets;
step 103, calculating available network bandwidth according to the sending state;
and 104, determining the number of frames to be transmitted in the next GOP sequence according to the available network bandwidth, and transmitting the next GOP sequence according to the number of frames to be transmitted.
GOP (Group of Pictures) is a concept in video coding that defines the structure and coding scheme of a video sequence. In video coding, video frames are divided into different groups, each group being referred to as a GOP. RTP (Real-time Transport Protocol) is a network protocol for Real-time transmission of audio and video data and is widely used in Real-time communication applications such as VoIP (Voice over IP) and Real-time video streaming. The RTP protocol provides a standardized way to segment, transport, and reassemble audio and video data, which is an end-to-end protocol for transporting real-time data between a sender and a receiver. The main function of RTP is to split data, i.e. RTP splits an audio or video data stream into small packets, called RTP packets.
The technical conception of the invention is as follows: in the process that a streaming media server sends needed media data to a playing end and the playing end plays the needed media data in real time in a weak network environment, firstly, the media data to be sent are read from source media data of the streaming media server, the media data to be sent are divided into a plurality of groups of video frames, each group of video frames is recorded as a GOP sequence, each frame of the media data is sliced and packaged into RTP data packets, namely, one GOP sequence can be split into a plurality of RTP data packets for transmission, the RTP data packets are used as transmission units for packaging the media data, and the GOP sequence is used as a statistical unit for the sending state of the media data; secondly, taking delay of media data encapsulation transmission and the like to be transmitted into consideration, adopting a mode of transmitting RTP data packets at regular time, namely setting a streaming media server to try to transmit RTP data packets to a playing end for a plurality of times every second, transmitting one RTP data packet each time, and counting the transmission state of the RTP data packets; then, calculating available network bandwidth according to the sending state of RTP data packets of the streaming media server, and taking the available network bandwidth as index data for representing the current network condition, thereby completely overcoming the dependence of the calculation of the network condition on the feedback of code stream information of the playing end; determining the number of frames to be transmitted in the next GOP sequence according to the calculated current available network bandwidth, namely the code rate to be transmitted in the next GOP sequence, and realizing the real-time adjustment of the transmission code rate of the media data, thereby improving the transmission adaptability of the streaming media server in the weak network environment without code stream retransmission; and stopping the weak network transmission operation of the streaming media server when the playing end stops playing the media data in the streaming media server.
The invention creatively provides a weak network transmission method of a streaming media server, wherein the streaming media server independently calculates the available network bandwidth, uses the available network bandwidth as index data for representing the current network condition, completely overcomes the dependence of calculation of the network condition on the feedback of code stream information of a playing end, thereby solving the technical problem that the playing end possibly has excessive packet loss or delay when feeding back the code stream information under the weak network condition, determining the number of frames to be transmitted in the next GOP sequence according to the calculated available network bandwidth, realizing the real-time adjustment of the transmission code rate of media data, ensuring the transmission adaptability of the streaming media server under the weak network environment without code stream retransmission, and reducing the consumption of memory resources of the streaming media server.
Preferably, step 102 specifically includes:
transmitting RTP data packets in the current GOP sequence to a playing end, and screening effective transmission in the process of transmitting the RTP data packets;
and counting the transmission state of the RTP data packet in the effective transmission, wherein the transmission state comprises the times of successful transmission of the RTP data packet and the times of failure transmission of the RTP data packet.
When an RTP packet is transmitted to a playback end, most of the transmission packets contain media data, which is called active transmission, but there is a case where there is no media data in the RTP packet, which is called inactive transmission. The invalid transmission may occur because a batch of RTP packets has been transmitted, but there is no actual media data in the last few or last RTP packets, but a null packet. The null packets will not contain the actual media data but will still carry the RTP header and other control information to preserve the structural integrity of the RTP packets. In addition, in the weak network environment, due to the instability of the network condition, not every transmission can make the playing end successfully receive the RTP data packet transmitted by the streaming media server, which is called transmission failure.
By considering the empty packet phenomenon in the transmission process of the streaming media server and the transmission failure phenomenon in the weak network environment, the accuracy of independently calculating the available network bandwidth of the subsequent streaming media server side is increased to a certain extent.
Preferably, in step 103, the calculation formula of the available network bandwidth is:
wherein B is 0 Representing available network bandwidth after an effective GOP sequence has been transmitted, T 0 Representing the time taken to effectively transmit all RTP packets in a GOP sequence, M 0 Representing T 0 Total number of successful transmission times of RTP data packet in time period, M 1 Representing T 0 RTP data packet in time periodTotal number of transmission failures.
Preferably, step 104 specifically includes:
determining a current network condition according to the available network bandwidth;
determining the frame loss proportion of the next GOP sequence according to the current network condition;
and calculating the number of frames to be transmitted in the next GOP sequence according to the frame loss ratio, and transmitting the next GOP sequence according to the calculated number of frames.
Preferably, the method for determining the current network condition according to the available network bandwidth comprises the following steps:
dividing the available network bandwidth into a plurality of threshold intervals according to the bandwidth value, wherein the threshold intervals are used for representing different network conditions;
And comparing the calculated available network bandwidth with a threshold interval, determining a network condition matched with the calculated available network bandwidth according to the threshold interval in which the calculated available network bandwidth falls, and recording the network condition as the current network condition.
The network status refers to the quality and stability of the data transmitted in the network, and the available network bandwidth refers to the bandwidth available for data transmission in the network. The availability of network conditions may affect the amount of available network bandwidth. If the network condition is poor, such as network congestion, signal interference, packet loss and other problems, the quality of data transmission is reduced, and the available network bandwidth is affected. Thus, the present invention takes available network bandwidth as index data representing the current network conditions.
Preferably, the network conditions include a normal condition, a weak network condition and a limit condition, and the method for determining the frame loss ratio of the next GOP sequence according to the current network condition includes:
establishing a mapping table which reflects the mapping relation between different network conditions and the frame loss proportion of the next GOP sequence, wherein if the network conditions are normal conditions, the frame loss proportion of the next GOP sequence is determined to be zero, and if the network conditions are weak network conditions, the next frame loss proportion matched with the weak network conditions is determined according to the mapping table The frame loss ratio of one GOP sequence is determined as the frame loss ratio of the next GOP sequence if the network condition is the limit conditionWhere N represents the number of frames of the current GOP sequence;
and matching the current network condition with the mapping table to obtain the frame loss proportion of the next GOP sequence.
Preferably, the method for calculating the number of frames to be transmitted in the next GOP sequence according to the frame loss ratio includes:
and (3) subtracting the frame loss ratio from 1, and multiplying the frame loss ratio by the frame number of the current GOP sequence to obtain a value which is the number of frames to be transmitted in the next GOP sequence.
By establishing a plurality of objective and clear threshold intervals representing different network conditions and the frame loss proportion of the next GOP sequence corresponding to the threshold intervals, and calculating the number of frames to be transmitted in the next GOP sequence according to the frame loss proportion, namely the code rate to be transmitted by the next GOP sequence, the real-time adjustment of the transmission code rate of media data is realized without code stream retransmission, so that the consumption of memory resources of a streaming media server is reduced.
A weak network transmission system of a streaming media server, comprising:
the data acquisition module is used for reading media data to be sent in the streaming media server;
the preprocessing module is used for dividing the media data to be sent into a plurality of groups of video frames, marking each group of video frames as a GOP sequence, slicing each frame of the media data and packaging the sliced frame into RTP data packets;
The network sending module is used for sending RTP data packets in the current GOP sequence to the playing end;
a sending state statistics module, configured to count a sending state of the RTP packet;
a network status calculation module for calculating an available network bandwidth according to the transmission status;
and the frame loss processing module is used for determining the number of frames to be transmitted in the next GOP sequence according to the available network bandwidth and transmitting the next GOP sequence according to the number of frames to be transmitted.
A computer device, comprising:
a processor;
a memory for storing executable instructions of the processor;
wherein the processor is configured to perform a method of weak network transmission of a streaming server as described above via execution of the executable instructions.
A computer-readable storage medium comprising a memory, a storage medium, and a memory,
the computer readable storage medium stores a computer program which when executed by a processor implements a method of weak network transmission of a streaming server as described above.
The beneficial technical effects of the invention include: the method, the system, the equipment and the medium for transmitting the weak network of the streaming media server are creatively provided, the streaming media server side independently calculates the available network bandwidth, and the available network bandwidth is used as index data for representing the current network condition, so that the dependence of calculation of the network condition on the feedback of the code stream information of the playing end is completely overcome, and the technical problem that the possible packet loss or overlarge delay exists when the playing end feeds back the code stream information under the weak network condition is solved; by establishing a plurality of objective and clear threshold intervals representing different network conditions and the frame loss proportion of the next GOP sequence corresponding to the threshold intervals, and calculating the number of frames to be transmitted in the next GOP sequence according to the frame loss proportion, namely the code rate to be transmitted by the next GOP sequence, the real-time adjustment of the transmission code rate of media data is realized, the transmission adaptability of a streaming media server in a weak network environment is ensured without code stream retransmission, and the consumption of the memory resources of the streaming media server is reduced.
Other features and advantages of the present invention will be disclosed in the following detailed description of the invention and the accompanying drawings.
Drawings
The invention is further described with reference to the accompanying drawings:
fig. 1 is a flowchart of a weak network transmission method of a streaming media server according to an embodiment of the present invention.
Fig. 2 is a flowchart of a method for determining a frame loss ratio and transmitting a next GOP sequence according to an embodiment of the invention.
Fig. 3 is a schematic structural diagram of a weak network transmission system of a streaming media server according to an embodiment of the present invention.
Fig. 4 is a schematic structural diagram of a computer device according to an embodiment of the present invention.
Wherein: 1. the system comprises a data acquisition module, a preprocessing module, a network transmission module, a transmission state statistics module, a network condition calculation module, a frame loss processing module, a processor, a memory and a data transmission state statistics module.
Detailed Description
The technical solutions of the embodiments of the present invention will be explained and illustrated below with reference to the drawings of the embodiments of the present invention, but the following embodiments are only preferred embodiments of the present invention, and not all embodiments. Based on the examples in the implementation manner, other examples obtained by a person skilled in the art without making creative efforts fall within the protection scope of the present invention.
In the following description, directional or positional relationships such as the terms "inner", "outer", "upper", "lower", "left", "right", etc., are presented for convenience in describing the embodiments and simplifying the description, and do not indicate or imply that the devices or elements referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore should not be construed as limiting the application.
The embodiment of the application provides a weak network transmission method of a streaming media server, which is applied to a process that the streaming media server sends required media data to a playing end and the playing end plays the media data in real time under a weak network environment, and referring to fig. 1, the method comprises the following steps:
step 101, reading media data to be sent in a streaming media server, dividing the media data to be sent into a plurality of groups of video frames, marking each group of video frames as a GOP sequence, slicing each frame of the media data, and packaging the frames into RTP data packets.
GOP (Group of Pictures) is a concept in video coding that defines the structure and coding scheme of a video sequence. In video coding, video frames are divided into different groups, each group being referred to as a GOP. RTP (Real-time Transport Protocol) is a network protocol for Real-time transmission of audio and video data and is widely used in Real-time communication applications such as VoIP (Voice over IP) and Real-time video streaming. The RTP protocol provides a standardized way to segment, transport, and reassemble audio and video data, which is an end-to-end protocol for transporting real-time data between a sender and a receiver. The main function of RTP is to split data, i.e. RTP splits an audio or video data stream into small packets, called RTP packets. In this embodiment, each frame of media data is sliced into equal length and encapsulated into RTP packets.
On the other hand, in the present embodiment, the implementation manner of step 101 includes, but is not limited to, the following two. One implementation is as follows: the method comprises the steps of directly reading partial media data serving as media data to be transmitted from source media data of a streaming media server, dividing the media data to be transmitted into a plurality of groups of video frames, marking each group of video frames as a GOP sequence, slicing each frame of the media data and packaging the frames into RTP data packets, and placing the RTP data packets in a queue to be transmitted of the streaming media server to wait for transmission. Another implementation is: the streaming media server sets a buffer area with a fixed size for each play end, the buffer area is used for storing part of media data serving as media data to be sent in source media data, the media data to be sent is read from the buffer area, the media data to be sent is divided into a plurality of groups of video frames, each group of video frames is recorded as a GOP sequence, the number of frames of each GOP sequence is the same, each frame of the media data to be sent is sliced and packaged into RTP data packets, and the RTP data packets are placed in a queue to be sent in the buffer area to wait to be sent. The latter implementation mode finishes sending the media data in the previous section of buffer zone, stores the next section of media data to be sent, and then continues sending, so that the streaming media server does not need to carry out a large amount of buffering on RTP data packets before sending, and provides buffering for the streaming media server to send the media data, thereby reducing the occupation of memory resources of the streaming media server to a certain extent.
Step 102, sending RTP data packets in the current GOP sequence to the playing end, and counting the sending state of the RTP data packets.
In this embodiment, considering delay of media data encapsulation transmission to be sent, an RTP packet is sent at regular time, that is, the streaming media server is set to attempt to send RTP packets to the playing end several times per second, and one RTP packet in the current GOP sequence is sent each time. Illustratively, the timer is set to transmit k times per second, each time one RTP packet is transmitted at the head of the queue to be transmitted.
And step 103, calculating the available network bandwidth according to the transmission state.
Step 104, determining the number of frames to be transmitted in the next GOP sequence according to the available network bandwidth, and transmitting the next GOP sequence according to the number of frames to be transmitted.
The playing end, namely the client end, receives and plays the equipment or the application program of the streaming media content by communicating with the streaming media server, and controls the receiving and playing process according to the transmission strategy and the feedback mechanism of the server. In this embodiment, the playing end may be a computer, a smart phone, a tablet computer or other similar terminal devices.
The technical conception of the invention is as follows: in the process that a streaming media server sends needed media data to a playing end and the playing end plays the needed media data in real time in a weak network environment, firstly, the media data to be sent are read from source media data of the streaming media server, the media data to be sent are divided into a plurality of groups of video frames, each group of video frames are recorded as a GOP sequence, the number of frames of each GOP sequence is the same, each frame of the media data is sliced and packaged into RTP data packets, namely, one GOP sequence can be split into a plurality of RTP data packets for transmission, the RTP data packets are used as transmission units for packaging the media data, and the GOP sequence is used as a statistical unit for the sending state of the media data; secondly, taking delay of media data encapsulation transmission and the like to be transmitted into consideration, adopting a mode of transmitting RTP data packets at regular time, namely setting a streaming media server to try to transmit RTP data packets to a playing end for a plurality of times every second, transmitting one RTP data packet each time, and counting the transmission state of the RTP data packets; then, calculating available network bandwidth according to the sending state of RTP data packets of the streaming media server, and taking the available network bandwidth as index data for representing the current network condition, thereby completely overcoming the dependence of the calculation of the network condition on the feedback of code stream information of the playing end; determining the number of frames to be transmitted in the next GOP sequence according to the calculated current available network bandwidth, namely the code rate to be transmitted in the next GOP sequence, and realizing the real-time adjustment of the transmission code rate of the media data, thereby improving the transmission adaptability of the streaming media server in the weak network environment without code stream retransmission; and stopping the weak network transmission operation of the streaming media server when the playing end stops playing the media data in the streaming media server.
Compared with the prior art, when the streaming media server calculates the available network bandwidth, the playing end is required to count the received media data information in real time and transmit the media data information back to the streaming media server, and the feedback data has the packet loss condition under the condition of poor network condition, so that the calculation method which depends on the feedback of the playing end has certain defects. Therefore, the invention creatively provides a weak network transmission method of the streaming media server, wherein the streaming media server independently calculates the available network bandwidth, uses the available network bandwidth as index data for representing the current network condition, completely overcomes the dependence of calculation of the network condition on feedback of the code stream information of the playing end, thereby solving the technical problem that the playing end possibly has overlarge packet loss or delay when feeding back the code stream information under the weak network condition, determining the number of frames to be transmitted in the next GOP sequence according to the calculated available network bandwidth, realizing real-time adjustment of the transmission code rate of the media data, ensuring the transmission adaptability of the streaming media server under the weak network environment without code stream retransmission, and reducing the consumption of memory resources of the streaming media server.
On the other hand, in this embodiment, step 102 specifically includes:
Transmitting RTP data packets in the current GOP sequence to a playing end, and screening effective transmission in the process of transmitting the RTP data packets;
and counting the transmission state of the RTP data packet in the effective transmission, wherein the transmission state comprises the times of successful transmission of the RTP data packet and the times of failure transmission of the RTP data packet.
When an RTP packet is transmitted to a playback end, most of the transmission packets contain media data, which is called active transmission, but there is a case where there is no media data in the RTP packet, which is called inactive transmission. The invalid transmission may occur because a batch of RTP packets has been transmitted, but there is no actual media data in the last few or last RTP packets, but a null packet. Specifically, a "blank" may occur in several cases:
1) The encoder works: in the encoder, the media data is sliced and encapsulated into RTP packets. When all media data is sliced and encapsulated, a null packet situation occurs if the last few RTP packets do not have enough data to fill.
2) End of data flow: null packets may occur if the source media data stream has ended, i.e. no more media data is available for encapsulation into RTP packets.
3) The media data duration is insufficient: some media data may have a very short duration and may not fill one RTP packet. In this case, the last or last few RTP packets may have null packets.
While the null packets will not contain the actual media data, the RTP header and other control information will still be carried to preserve the structural integrity of the RTP packets. When receiving these empty packets, the playing end may process according to the media type and other information, for example, continue playing the media data before playing or mute, which will not be described in detail in this embodiment. In addition, in the weak network environment, due to the instability of the network condition, not every transmission can make the playing end successfully receive the RTP data packet transmitted by the streaming media server, which is called transmission failure. Therefore, the present embodiment records whether transmission succeeded or failed every time an RTP packet was transmitted. In a weak network environment, because network delay increases, an RTP data packet may generate a larger delay in the transmission process, the streaming media server may determine and record the transmission state of each RTP data packet by monitoring the transmission time of the RTP data packet, and if the transmission time exceeds a set threshold value, may determine that the RTP data packet fails to be transmitted.
According to the method and the device, by considering the empty packet phenomenon in the transmission process of the streaming media server and the transmission failure phenomenon in the weak network environment, the accuracy of independently calculating the available network bandwidth by the streaming media server in the weak network environment is improved to a certain extent.
On the other hand, in the embodiment, in step 103, the calculation formula of the available network bandwidth is:
wherein B is 0 Representing available network bandwidth after an effective GOP sequence has been transmitted, T 0 Representing the time taken to effectively transmit all RTP packets in a GOP sequence, M 0 Representing T 0 Total number of successful transmission times of RTP data packet in time period, M 1 Representing T 0 The total number of times the RTP packet transmission fails in the period of time, L, represents the length of each frame of media data after slicing.
The calculation formula for obtaining the available network bandwidth after simplification is as follows:
in this embodiment, the GOP sequence is used as a statistic unit for the sending state of media data, and the available network bandwidth after sending one GOP sequence is calculated in real time according to the number of times of successful sending, the number of times of failed sending and the sending time of media data in one GOP sequence, so that the sending size and sending frequency of media data can be better controlled, and better bandwidth utilization and more stable data transmission are realized. In addition, in streaming media application, real-time performance is very critical, because data needs to be transmitted and played according to a certain time interval, sending time of media data can be better controlled to a certain extent by counting with GOP sequence as a unit, the media data is ensured to arrive at a playing end on time, and continuity and smoothness are maintained during playing, so that quality and reliability of streaming media transmission are improved.
In this embodiment, referring to fig. 2, in step 104, the method for determining the number of frames to be transmitted in the next GOP sequence according to the available network bandwidth and transmitting the next GOP sequence according to the number of frames to be transmitted specifically includes:
step 201, determining the current network condition according to the available network bandwidth;
step 202, determining the frame loss ratio of the next GOP sequence according to the current network condition;
step 203, the frame number to be transmitted in the next GOP sequence is calculated according to the frame loss ratio, and the next GOP sequence is transmitted according to the calculated frame number.
On the other hand, in the present embodiment, in step 201, the method for determining the current network status according to the available network bandwidth includes:
dividing the available network bandwidth into a plurality of threshold intervals according to the bandwidth value, wherein the threshold intervals are used for representing different network conditions;
and comparing the calculated available network bandwidth with a threshold interval, determining a network condition matched with the calculated available network bandwidth according to the threshold interval in which the calculated available network bandwidth falls, and recording the network condition as the current network condition.
The network status refers to the quality and stability of the data transmitted in the network, and the available network bandwidth refers to the bandwidth available for data transmission in the network. The availability of network conditions may affect the amount of available network bandwidth. If the network condition is poor, such as network congestion, signal interference, packet loss and other problems, the quality of data transmission is reduced, and the available network bandwidth is affected. In practice, the worse the network condition is, the smaller the available network bandwidth calculated by the method proposed by the present embodiment is, whereas the larger the available network bandwidth calculated is, and the available network bandwidth calculated under the normal network condition is basically stable to a certain value. Therefore, the present embodiment uses the calculated available network bandwidth as the index data representing the current network condition, so that the accuracy of independently determining the current network condition by the streaming media server side can be ensured.
On the other hand, in this embodiment, the network conditions include a normal condition, a weak network condition, and a limit condition, and in step 202, the method for determining the frame loss ratio of the next GOP sequence according to the current network condition includes:
establishing a mapping table which reflects the mapping relation between different network conditions and the frame loss proportion of the next GOP sequence, wherein if the network conditions are normal conditions, the frame loss proportion of the next GOP sequence is determined to be zero, if the network conditions are weak net conditions, the frame loss proportion of the next GOP sequence matched with the weak net conditions is determined according to the mapping table, and if the network conditions are limit conditions, the frame loss proportion of the next GOP sequence is determined to be zeroWhere N represents the number of frames of the current GOP sequence;
and matching the current network condition with the mapping table to obtain the frame loss ratio of the next GOP sequence.
On the other hand, in the embodiment, in step 203, the method for calculating the number of frames to be transmitted in the next GOP sequence according to the frame loss ratio includes:
and (3) subtracting the frame loss ratio from 1, and multiplying the frame loss ratio by the frame number of the current GOP sequence to obtain a value which is the number of frames to be transmitted in the next GOP sequence.
In this embodiment, the number of frames to be transmitted in the next GOP sequence is determined according to the calculated available network bandwidth, and the implementation manner of transmitting the next GOP sequence according to the number of frames to be transmitted is as follows:
Firstly, setting a four-level threshold value of an available network bandwidth according to the bandwidth value, wherein the first threshold value is B41, the second threshold value is B42, the third threshold value is B43, and the fourth threshold value is B44, wherein B41< B42< B43< B44<1, 5 threshold value intervals representing network conditions are obtained according to the four-level threshold value, wherein (0, B41) represents a normal condition, (B41, B42) represents that the weak network condition is slight, (B42, B43) represents that the weak network condition is serious, (B43, B44) represents that the weak network condition is serious, and (B44, 1) represents a limit condition;
secondly, establishing a mapping table, wherein the mapping table reflects a mapping relation between 5 network conditions and the frame loss ratio of the next GOP sequence, and the frame loss ratio of the next GOP sequence corresponding to the 5 network conditions is respectively 0, P41, P42, P43 and P44, wherein 0< P41< P42< P43< P44<1;
then, the frame number N of one GOP sequence is obtained because the frame number of each GOP sequence is the same, and the calculated available network bandwidth B is calculated 0 In comparison with the interval of the threshold value,
if B 0 <B41, determining that the current network condition is normal, wherein the frame loss ratio of the next GOP sequence is 0, and the number of frames to be transmitted in the next GOP sequence is N (1-0) =N, namely, the frames in the next GOP sequence can be normally transmitted without frame loss;
If B41<B 0 B42 is not more than, determining that the current network condition is a slight weak network condition, determining that the network has frame loss, determining that the frame loss ratio is P41, and determining that the number of frames to be transmitted in the next GOP sequence is N (1-P41);
if B42<B 0 B43 is not more than, the current network condition is determined to be a serious weak network condition, the network frame loss is serious, the frame loss proportion is determined to be P42, and the number of frames to be transmitted in the next GOP sequence is N (1-P42);
if B43<B 0 B44 is not more than, the current network condition is determined to be a serious weak network condition, the network frame loss is serious, the frame loss proportion is determined to be P43, and the number of frames to be transmitted in the next GOP sequence is N (1-P43);
if B 0 >B44, determining the current network condition as the limit condition, in which case the network condition is very bad, thus setting the limit frame loss state, and determining the frame loss ratio asThe number of frames to be transmitted in the next GOP sequence isAnd the total data amount N of one GOP sequence is the sum of the data amounts of one I frame and the number of all P frames, so all P frames in the whole GOP sequence to be transmitted next are discarded and only the I frame is transmitted in the limit condition.
An I-frame (Intra-coded frame), also called a key frame, is an independent frame in a video sequence, and is decoded independent of other frames, and is usually the start of the video sequence, and contains complete image information, which can be used as a reference for other frames. A P frame (Predictive-coded frame) is a frame differentially encoded from a previous key frame or P frame, which contains only difference information from the previous frame, and an image can be restored by decoding the previous frame and restoring the difference information.
When the playing end decodes the received GOP sequence, the first P frame in the GOP sequence needs to refer to the previous I frame, and the subsequent P frame sequentially refers to the previous P frame, for example, P1 dependence I, P2 dependence P1, P3 dependence P2, and P4 dependence P3 … …. Therefore, the frame loss ratio is determined and then the frame is discarded from the last P frame in the next GOP sequence. For example, in a GOP sequence with a frame number of 50, when it is determined that the frame loss ratio is 0.16 after calculation, that is, 8 frames should be discarded, the frames of the next GOP sequence to be transmitted are the first 42 frames in the next GOP sequence.
In practice, the threshold B44 representing the limit condition has a good adaptive rate control effect within 30%, so the present embodiment preferably sets the threshold B44 representing the limit condition to 30%, and the corresponding other threshold values, such as B43, B42, and B41, are smaller than B44 and decrease in sequence, so that specific parameters can be set in the engineering.
In the embodiment, a set of frame loss strategies are designed to adapt to the change of the weak network transmission network environment while the streaming media server side independently calculates the available network bandwidth, a plurality of objective and clear threshold intervals representing different network conditions and the frame loss proportion of the number of transmittable frames in the next GOP sequence corresponding to the threshold intervals are established, and the transmission frame rate of the next GOP sequence of the streaming media server is reduced by discarding part of media data (P frames) in the GOP sequence to be transmitted in a certain proportion, so that the method for reducing the network transmission code rate of the media data is convenient and reliable to implement at the streaming media server side and can adapt to the fluctuation of the network conditions without code stream retransmission, thereby reducing the consumption of memory resources of the streaming media server.
On the other hand, the embodiment of the present application further provides a weak network transmission system of a streaming media server consistent with the foregoing concept of the weak network transmission method of a streaming media server, referring to fig. 3, including:
the data acquisition module 1 is used for reading media data to be sent in the streaming media server;
the preprocessing module 2 is used for dividing media data to be sent into a plurality of groups of video frames, marking each group of video frames as a GOP sequence, slicing each frame of the media data and packaging the frames into RTP data packets;
the network sending module 3 is used for sending RTP data packets in the current GOP sequence to the playing end;
a transmission state statistics module 4, configured to count a transmission state of the RTP packet;
a network status calculation module 5, configured to calculate an available network bandwidth according to the transmission status;
and the frame loss processing module 6 is configured to determine the number of frames to be sent in the next GOP sequence according to the available network bandwidth, and send the next GOP sequence according to the number of frames to be sent.
In another aspect, an embodiment of the present application further provides a computer device, referring to fig. 4, including:
a processor 7;
a memory 8 for storing executable instructions of the processor 7;
wherein the processor 7 is configured to perform a method of weak network transmission of a streaming server as described above via execution of executable instructions.
It should be noted that the description of the embodiment of the method for the computer device described above may also include other implementations. Specific implementation may refer to descriptions of related method embodiments, which are not described herein in detail.
The control system or the computer equipment of the air valve provided by the specification can also be applied to various data analysis processing systems. The computer device may be a separate server, or may include a server cluster, a system (including a distributed system), software (application), an actual operating device, a logic gate device, a quantum computer, or the like using the method or system of the embodiment of the present specification, and a terminal device that incorporates necessary implementation hardware.
The processor 7 may be a central processing unit (Central Processing Unit, CPU), and the processor 7 may be other general purpose processors, digital signal processors (Digital Signal Processor, DSP), application specific integrated circuits (Application Specific Integrated Circuit, ASIC), off-the-shelf programmable gate arrays (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or the like. The general purpose processor may be a microprocessor or may be any conventional processor.
Wherein the memory 8 stores program code executable by the processor 7 such that the processor 7 performs a deep learning based operation policy triggering method according to any of the above description. The memory 8 may in some embodiments be an internal storage unit of a computer device, such as a hard disk or a memory of the computer device. The memory 8 may in other embodiments also be an external storage device of the computer device, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card) or the like, which are provided on the computer device. Further, the memory 8 may also include both internal storage units and external storage devices of the computer device.
In another aspect, embodiments of the present application also provide a computer-readable storage medium,
the computer readable storage medium stores a computer program which, when executed by the processor 7, implements a method of weak network transmission of a streaming server as described above.
The computer readable storage medium of the present disclosure may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this disclosure, a computer-readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
In some implementations, the clients, servers may communicate using any currently known or future developed network protocol, such as HTTP (Hyper Text Transfer Protocol ), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), the internet (e.g., the internet), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future developed networks.
The computer readable storage medium may be contained in the computer device; or may exist alone without being fitted into the computer device.
The modules involved in the embodiments of the present application may be implemented in software or in hardware. The name of a module does not in some cases define the module itself.
The foregoing is merely illustrative of the preferred embodiments of the present disclosure and the technical principles applied thereto, and it will be understood by those skilled in the art that the scope of the disclosure is not limited to the specific combination of the technical features described above, but encompasses other technical solutions formed by any combination of the technical features described above or the equivalents thereof without departing from the spirit of the disclosure. Such as those described above, are mutually substituted with the technical features having similar functions disclosed in the present disclosure (but not limited thereto).
Moreover, although operations are depicted in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order. In certain circumstances, multitasking and parallel processing may be advantageous. Likewise, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of the present disclosure. Certain features that are described in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination.

Claims (10)

1. The weak network transmission method of the streaming media server is characterized by comprising the following steps:
step 101, reading media data to be sent in a streaming media server, dividing the media data to be sent into a plurality of groups of video frames, marking each group of video frames as a GOP sequence, slicing each frame of the media data and packaging the frames into RTP data packets;
step 102, sending RTP data packets in the current GOP sequence to a playing end, and counting the sending state of the RTP data packets;
Step 103, calculating available network bandwidth according to the sending state;
and 104, determining the number of frames to be transmitted in the next GOP sequence according to the available network bandwidth, and transmitting the next GOP sequence according to the number of frames to be transmitted.
2. The method for transmitting a stream media server according to claim 1, wherein,
step 102 specifically includes:
transmitting RTP data packets in the current GOP sequence to a playing end, and screening effective transmission in the process of transmitting the RTP data packets;
and counting the transmission state of the RTP data packet in the effective transmission, wherein the transmission state comprises the times of successful transmission of the RTP data packet and the times of failure transmission of the RTP data packet.
3. The method for transmitting a stream media server according to claim 2, wherein,
in step 103, the calculation formula of the available network bandwidth is:
wherein B is 0 Representing available network bandwidth after an effective GOP sequence has been transmitted, T 0 Representing the time taken to effectively transmit all RTP packets in a GOP sequence, M 0 Representing T 0 Total number of successful transmission times of RTP data packet in time period, M 1 Representing T 0 The total number of RTP packet transmission failures in a time period.
4. The method for transmitting a stream media server according to claim 1, wherein,
Step 104 specifically includes:
determining a current network condition according to the available network bandwidth;
determining the frame loss proportion of the next GOP sequence according to the current network condition;
and calculating the number of frames to be transmitted in the next GOP sequence according to the frame loss ratio, and transmitting the next GOP sequence according to the calculated number of frames.
5. The method for transmitting a stream media server according to claim 4, wherein,
the method for determining the current network condition according to the available network bandwidth comprises the following steps:
dividing the available network bandwidth into a plurality of threshold intervals according to the bandwidth value, wherein the threshold intervals are used for representing different network conditions;
and comparing the calculated available network bandwidth with a threshold interval, determining a network condition matched with the calculated available network bandwidth according to the threshold interval in which the calculated available network bandwidth falls, and recording the network condition as the current network condition.
6. The method for transmitting a stream media server according to claim 4, wherein,
the network conditions include a normal condition, a weak network condition and a limit condition, and the method for determining the frame loss ratio of the next GOP sequence according to the current network condition includes:
Establishing a mapping table, wherein the mapping table reflects mapping relations between different network conditions and frame loss ratios of a next GOP sequence, wherein if the network conditions are normal conditions, the frame loss ratio of the next GOP sequence is determined to be zero, if the network conditions are weak network conditions, the frame loss ratio of the next GOP sequence matched with the weak network conditions is determined according to the mapping table, and if the network conditions are limit conditions, the frame loss ratio of the next GOP sequence is determined to be zeroWhere N represents the number of frames of the current GOP sequence;
and matching the current network condition with the mapping table to obtain the frame loss proportion of the next GOP sequence.
7. The method for transmitting a stream media server according to claim 4, wherein,
the method for calculating the number of frames to be transmitted in the next GOP sequence according to the frame loss ratio comprises the following steps:
and (3) subtracting the frame loss ratio from 1, and multiplying the frame loss ratio by the frame number of the current GOP sequence to obtain a value which is the number of frames to be transmitted in the next GOP sequence.
8. A weak network transmission system of a streaming media server, comprising:
the data acquisition module is used for reading media data to be sent in the streaming media server;
The preprocessing module is used for dividing the media data to be sent into a plurality of groups of video frames, marking each group of video frames as a GOP sequence, slicing each frame of the media data and packaging the sliced frame into RTP data packets;
the network sending module is used for sending RTP data packets in the current GOP sequence to the playing end;
a sending state statistics module, configured to count a sending state of the RTP packet;
a network status calculation module for calculating an available network bandwidth according to the transmission status;
and the frame loss processing module is used for determining the number of frames to be transmitted in the next GOP sequence according to the available network bandwidth and transmitting the next GOP sequence according to the number of frames to be transmitted.
9. A computer device, comprising:
a processor;
a memory for storing executable instructions of the processor;
wherein the processor is configured to perform a method of weak network transmission of a streaming server according to any of claims 1 to 7 via execution of the executable instructions.
10. A computer-readable storage medium comprising,
the computer readable storage medium stores a computer program which when executed by a processor implements a method of weak network transmission of a streaming server according to any one of claims 1 to 7.
CN202310948333.6A 2023-07-31 2023-07-31 Weak network transmission method, system, equipment and medium of streaming media server Pending CN116827920A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310948333.6A CN116827920A (en) 2023-07-31 2023-07-31 Weak network transmission method, system, equipment and medium of streaming media server

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310948333.6A CN116827920A (en) 2023-07-31 2023-07-31 Weak network transmission method, system, equipment and medium of streaming media server

Publications (1)

Publication Number Publication Date
CN116827920A true CN116827920A (en) 2023-09-29

Family

ID=88124179

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310948333.6A Pending CN116827920A (en) 2023-07-31 2023-07-31 Weak network transmission method, system, equipment and medium of streaming media server

Country Status (1)

Country Link
CN (1) CN116827920A (en)

Similar Documents

Publication Publication Date Title
US7643480B2 (en) Method and system for reliably and efficiently transporting data over a network
JP4623616B2 (en) Data transmission method and apparatus
KR101046105B1 (en) Computer program manufacturing, resource demand adjustment methods, and end systems
US9450701B2 (en) Data flow control method and apparatus
US7315898B2 (en) Data communication system, data transmission apparatus, data reception apparatus, data communication method, and computer program
CN100375538C (en) Method for multimedia communication over packet channels
JP6023368B1 (en) Interactive real-time media transfer protocol
US9565482B1 (en) Adaptive profile switching system and method for media streaming over IP networks
US9628411B2 (en) Efficient packet processing at video receiver in multimedia communications over packet networks
CN113037440B (en) Data retransmission processing method and device, computer equipment and storage medium
EP1328096A2 (en) Multimedia data packet communication with data type identifiers
US20160323062A1 (en) Packet recovery in interactive real-time media protocol
WO2016045332A1 (en) Method and device for encoding parameter adjustment and feedback information processing
EP1301041A1 (en) Video data transmission method and apparatus
US10230651B2 (en) Effective intra-frame refresh in multimedia communications over packet networks
US9781488B2 (en) Controlled adaptive rate switching system and method for media streaming over IP networks
CN111740808A (en) Data transmission method and device
BRPI0808629A2 (en) PACKAGE LOSS EFFECTS REDUCTION IN VIDEO TRANSMISSIONS.
JP2004187286A (en) Mpeg-4 live unicast video streaming system in wireless network equipped with congestion control of end-to-end bit rate reference
CN108833930B (en) Live broadcast data transmission control method and device, live broadcast equipment and storage medium
Wu et al. Streaming high-definition real-time video to mobile devices with partially reliable transfer
Baldo et al. RTCP feedback based transmission rate control for 3G wireless multimedia streaming
US7203184B2 (en) Data transmitter, data receiver, and data transmitting/receiving method
US8127196B2 (en) Server and client for determining error restoration according to image data transmission, and method of determining error restoration according to image data transmission
Mehrotra et al. Optimizing FEC transmission strategy for minimizing delay in lossless sequential streaming

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination