CN111615006B

CN111615006B - Video code conversion transmission control system based on network state self-evaluation

Info

Publication number: CN111615006B
Application number: CN202010480017.7A
Authority: CN
Inventors: 高小翎; 王斌
Original assignee: Beijing Xunzhong Communication Technology Co ltd
Current assignee: Beijing Xunzhong Communication Technology Co ltd
Priority date: 2020-05-29
Filing date: 2020-05-29
Publication date: 2022-02-01
Anticipated expiration: 2040-05-29
Also published as: CN111615006A

Abstract

According to the video coding conversion transmission control system based on network state self-evaluation, aiming at the problems that a network evaluation algorithm only depends on packet loss rate to evaluate the network state, the evaluation division is too rough, the video transmission rate adjustment basis is large in limitation and the like, a method combining the packet loss rate, network delay and delay jitter as network state evaluation elements is provided, and the evaluation of the network state is more scientific and accurate; aiming at the problem that effective pre-control measures cannot be taken when the network is in an overload trend but is not overloaded, a network overload pre-control method is added, and the problem of network overload is ingeniously handled at opportunity; aiming at the problem that overload control cannot be performed by simply adjusting coding parameters when network overload is too serious, a non-key frame active discarding method is provided, so that the video quality is ensured, the transmission quality is ensured, a frame dropping strategy is performed before coding, the coding workload is reduced, and the method has robustness and high efficiency.

Description

Video code conversion transmission control system based on network state self-evaluation

Technical Field

The application relates to a video code conversion transmission control system, in particular to a video code conversion transmission control system based on network state self-evaluation, and belongs to the technical field of video code conversion transmission.

Background

With the rapid development of internet technology, the user has higher and higher requirements for the service quality of various network services, and the network service quality and performance guarantee has become the current research and development focus. In a real-time video streaming media transmission system based on the internet, it is critical to guarantee the quality of service in multimedia communication services, and in the design of a real-time video network communication system, the control of network overload is a key and difficult point, which can not only cause the waste of network resources, but also ensure that the packet loss rate and the time delay of video data are within a reasonable range, and can not affect the video watching effect of users.

Because the network service mechanism in the prior art cannot provide service quality guarantee in the multimedia transmission process and cannot meet the requirements of users, how to control the overload of the video network becomes a problem to be solved urgently. Channel overload in multimedia network communications results from network available resources failing to meet the data transfer volume in the actual channel, exceeding the currently available bandwidth. Overload easily causes severe jitter, data loss, time delay overtime, service quality and resource utilization rate reduction, and in severe cases, network communication system breakdown is also caused.

The video streaming media real-time transmission is mainly characterized by large demand on available resources of a network, high real-time requirement and capability of allowing a certain data volume to be lost. In order to improve the utilization rate of available bandwidth of a network and guarantee video transmission quality, a certain control strategy is required to be adopted to correctly distribute the bandwidth to prevent the occurrence of network overload, and if the occurrence cannot be avoided, the control is required to be carried out in time. The real-time video coding conversion network service has high requirements on bandwidth and delay, and video coding and decoding and playing quality are core technologies of the real-time video coding conversion network service. How to guarantee the service quality mainly solves the problems of delay overtime, network overload, overlarge packet loss rate and the like. In the condition that the available resources of the network are limited and the bandwidth of a channel fluctuates, what transmission control scheme is adopted to reduce the loss and the time delay of data is an important problem. The video transcoding is a technology for converting a video stream in an original format into a video stream in a target format, and can solve the problem of compatibility between a video transmitting end and a video receiving end. The video code conversion can complete the mutual conversion among video sampling rates, video compression standards, frame rates and resolutions with different formats, the conversion among code stream formats is sensitive and can almost meet various requirements of users, and the converted target code stream can be matched with the current network state and can meet the requirements of terminals and users.

In the video communication process, in order to ensure the video quality, the video sampling rate when the source code stream is compressed for the first time is generally higher, but when the network available resource condition can not meet the original video sampling rate, the source code stream needs to be compressed newly, and a lower target video sampling rate is set during compression, so that the problem of video sampling rate control in the code conversion process is also caused. In the real-time video transcoding transmission control technique, the prior art video sampling rate adjustment method is only suitable for a light network overload or a relatively small channel bandwidth condition.

The prior art has a network-centered transmission control method, a design method is similar to a virtual link thought established by a TCP protocol, but has the defect of poor expansibility, and provides some improved algorithms based on a random early detection queue management algorithm and calculation methods for modifying the length of a delay queue.

The prior art has a video sampling rate control technology, and provides an increasing, multiplying and decreasing algorithm, wherein the increasing, multiplying and decreasing algorithm is the combination of multiplication decreasing and addition increasing, a memory overload function is added, dynamic additive elements and multiplicative elements are adaptively changed according to a transmission rate, and the stability of a data stream is improved; however, in these methods, the evaluation division for the network only depends on the feedback of the packet loss rate, so the algorithm has a great limitation in real-time video transmission control.

The prior art has a frame dropping technology, and when a network is overloaded, a data packet with lower importance is selected to be dropped firstly. Although the method can reduce the load of the network and improve the performance of the network, the method has the disadvantages that one service flow is directly sacrificed, which is too costly for use, and some improved schemes can flexibly adjust the network load of the video data to be suitable for the network performance as far as possible under the condition of not changing the video image quality of a user, but the method has higher calculation complexity and is not commonly used in a real-time video transcoding service.

In summary, aiming at some defects existing in the prior art, the following problems are to be solved by the present application:

firstly, in the prior art, a network evaluation algorithm judges the network state only by means of the packet loss rate, the network state judgment division is too rough, the network state judgment division is used as the problems that the video transmission rate adjustment basis is large in limitation and the like, the provided judgment basis is unreliable, the judgment of the network state is not scientific and accurate enough, and a good basis cannot be laid for the classification decision of video coding conversion transmission control.

Secondly, the method in the prior art can not adopt effective pre-control measures under the condition that the network has an overload trend but is not overloaded yet, the multiplication and subtraction algorithm has no network overload pre-control method, the multiplication and subtraction element can not be adopted to increase the bit rate parameter when the network is underloaded, the multiplication and attenuation element can not be adopted to reduce the value of the bit rate parameter when the network is overloaded, the generation of overload is not pre-controlled, and the network overload problem can not be processed skillfully.

Thirdly, in the prior art, the overload of the network is too serious, the overload control cannot be performed by simply adjusting the coding parameters, the video sampling rate adjustment strategy cannot be combined, partial types of video frames cannot be selected to be abandoned according to the size of the network delay element, the video quality and the transmission quality cannot be guaranteed, and the coding workload is generally large.

Fourth, the algorithm complexity of the video sampling rate algorithm in the prior art is high, the algorithm is not easy to implement, the video sampling rate cannot be adjusted to be rapidly reduced or slowly increased, the image picture quality is not stable, the overload control speed is low, the data accumulation phenomenon easily occurs in a cache region, and the problems of overlong delay time, discontinuous pictures and the like cannot be avoided.

Fifthly, a self-evaluation self-adaptive transmission control system facing real-time video coding conversion is not realized in the prior art, the network transmission control capability and the video service quality guarantee mechanism are poor, the judgment of the network state is not accurate, the adjustment of the video data transmission rate cannot be matched with the current network bandwidth, and the transmission system has no robustness and high efficiency.

Disclosure of Invention

Aiming at the defects of the prior art, the video coding conversion transmission control system based on network state self-evaluation provided by the application judges the network state by only depending on the packet loss rate aiming at the problems that the network evaluation algorithm in the prior art is too rough in the judgment and division of the network state and has large limitation as the video transmission rate adjustment basis, and the like, provides a method combining the packet loss rate, the network delay and the delay jitter as the network state evaluation elements, provides more reliable judgment basis for the self-evaluation self-adaptive adjustment of the video data transmission rate, judges the network state more scientifically and accurately and lays a good foundation for the classification decision of video coding conversion transmission control; aiming at the problem that effective pre-control measures cannot be taken when the network in the prior art is in an overload trend but is not overloaded, a network overload pre-control method is added, and the problem of network overload is ingeniously processed at a certain moment; aiming at the problems that the network overload is too serious in the prior art and overload control cannot be performed by simply adjusting coding parameters, a non-key frame active abandoning method is provided, so that the video quality is ensured, the transmission quality is ensured, a frame dropping strategy is performed before coding, the coding workload is reduced, and the system has higher robustness and efficiency.

In order to achieve the technical effects, the technical scheme adopted by the application is as follows:

the video coding conversion transmission control system based on network state self-evaluation, a video sending end self-adaptively and dynamically adjusts the transmission rate and the transmission quantity of data, so that the transmitted video data quantity is matched with the bandwidth of a link channel; in the self-evaluation of the network state, a method combining packet loss rate, network delay and time delay jitter as network state evaluation elements is provided, the network state is divided into three basic states of underload, normal load and overload, and when the network is in normal load, the time delay jitter element is introduced to judge whether the current network has an overload trend; when the network is overloaded, a network delay element is introduced to judge the level of the overload degree of the network, in the adaptive transmission control, an adjustment algorithm based on the multiplication and subtraction video sampling rate is optimized, a network overload pre-control method is added to the multiplication and subtraction algorithm, when the video sampling rate parameter is adjusted to the set minimum value and the network is still overloaded, a non-key frame active discarding method is started, the discarded frame type is selected according to the evaluated network overload level, the transmitted data volume is reduced, and a non-key frame discarding active strategy is started before encoding;

the basic working flow of the application is as follows: after setting a basic code conversion requirement at a client, a user starts a real-time video code conversion communication session, the client firstly sends a common signaling to a server, the common signaling comprises a total path number, a code conversion platform, a code conversion sequence and code conversion mode information, after receiving the common signaling, the server clears a signaling storage cache and a code conversion task according to the last received common signaling, then creates a new signaling storage cache and initializes a cache space; then the client sends a request signaling, the request signaling comprises the format, frame rate, resolution and video sampling rate information of an original code stream and a target code stream, the server sends a request completion confirming signaling after receiving the request signaling, and then the server starts to send code stream data to perform video code conversion transmission work; and the server calculates packet loss rate, network delay and time delay jitter parameter values of the code stream, packages the parameter values into an SR report packet, and sends the SR report packet to the client to evaluate the network state and adjust the sending rate.

The method aims at the transmission of video data from a server side to a client side in a video coding conversion transmission control system, the server side is used as a sending end, the client side is used as a receiving end, the code stream data in the original format is sent to the client side for receiving and playing by adopting an RTP (real time protocol) protocol after being subjected to coding conversion at the sending end, the adjustment algorithm of the video sampling rate parameter is completed in a coding function of a coder converter, a non-key frame active abandon strategy is carried out before coding, and the workload of video coding is reduced;

the present application includes three subsystems: a network state self-evaluation subsystem, a self-adaptive video sampling rate adjustment subsystem and a non-key frame active abandoning subsystem.

The network state evaluation of the network state self-evaluation method judges the feedback information in a feedback mechanism through an RTCP real-time transport protocol, if the packet loss rate, the network delay and the time delay jitter parameter values in the feedback information are larger, namely the data packet delay is greatly changed, the network state is not good, and if the packet loss rate, the network delay and the time delay jitter parameter values are smaller, the network state is good;

the size of the packet loss rate reflects the current network condition, when the packet loss rate exceeds a reasonable critical value, the network bandwidth is seriously insufficient, the channel load exceeds the maximum load capacity, and the network overload occurs; the time delay jitter is used for evaluating whether the channel has a tendency of overload to be generated in a normal load working state; the method and the device use the time delay jitter to pre-control the network overload, adjust the output video sampling rate of the system in advance and avoid the network overload; judging the degree grade of network overload by using a network delay element, dividing the network overload into different grades, judging the network state more accurately, and adjusting the video data transmission rate more suitably to the current network bandwidth;

the network state self-evaluation method comprises a calculation method of each element of the network state self-evaluation method and a network state evaluation method.

Method for self-evaluating network state and its elementsIn the calculation, the network overload grade is divided according to the size of the network delay B, and before the network delay B is calculated, the long-term accumulated time delay jitter average value E is calculated_LAnd the average value E of the short-term accumulated delay jitter_dAccumulated delay jitter FC_kIs calculated as shown in equation 1, the average value E of the accumulated delay jitter_kThe calculation is as in equation 2,

FC_k＝C₁+C₂+…C_kformula 1

Long term accumulated delay jitter E_LCalculation of the mean value: counting from the first data sent until the current time, the jitter of the data used for transmission is calculated according to equation 3:

short term accumulated time delay jitter E_dThe average value of the two accumulated delay jitters is calculated, and the average value calculated by the two accumulated delay jitters is represented by formula 4:

if E_dGreater than E_LWhen the data transmission delay is rapidly increased and the network load is continuously increased, the network is gradually overloaded at the moment, and the transmission rate of the transmitting end is adjusted to increase the frequency when the overload is not serious enough in the initial stage;

network delay B is judged to be E_dGreater than E_LIs measured, and the buffer usage of the bottleneck link in the end-to-end transmission path of the video data is evaluated, and the data is sent between two adjacent SR report packets received by a sending partyThe time interval is a measuring period, the receiving end calculates the one-way transmission time delay D of a data packet every time the receiving end receives the data packet in the measuring period, then compares the delay sizes of all data in the measuring period, finds out the maximum and minimum data time delay values, and calculates the D of the ith RTP packet received in a measuring unit time_iThe value is compared with the one-way transmission time delay of all the data packets in the period, and the maximum value D of the data packets is found_maxAnd a minimum value D_minCalculating the maximum queue delay F_max：

F_max＝D_max-D_minFormula 5

Network delay of data packet F_iExpressed by equation 6, the average data network delay avgF in the period is expressed by equation 7, and n is the number of data packets:

F_i＝D_i-D_minformula 6

The calculation method of the network delay B is obtained from the equations 5 and 7 as equation 8,

that is, the network delay B is the average network delay of data in a unit period and the maximum queue delay F_maxThe value of the network delay B is in a value range of 0,1]Meanwhile, when the value is 0, it indicates that the buffer is empty, no data is queued up, and generally occurs at the beginning of transmission, and when the value is 1, it indicates that the buffer is full, and the network is heavily overloaded.

The network state evaluation method provides that the joint packet loss rate, the network delay and the time delay jitter are used as network state evaluation elements, the network state is divided in detail, and the specific method for evaluating the network state is as follows:

firstly, a network state is preliminarily judged, and if the packet loss rate A is larger than the maximum packet loss rate critical value G in m continuous test periods₂I.e. A>G₂If the network is always in the overload state, judging that the current network is in the overload state;

secondly, if the packet loss rate a is smaller than the minimum packet loss rate critical value G in n consecutive test periods₁I.e. A<G₁If the network load is constant, judging that the current network is under load;

thirdly, in addition to the above two cases, the packet loss rate is between two critical values, i.e. G₁≤A≤G₂The network is in a normal load state; for the evaluation of the normal load state, the method carries out further evaluation, and sets a time delay jitter critical value C_aAnd C is the measured delay jitter value:

when C is less than or equal to C_aWhen the network channel is not overloaded, the current network is relatively stable, and the state is judged to be stable and normal load;

when C is present>C_aWhen the network channel is overloaded, the channel bandwidth is in a fluctuating state, and the current network is judged to be unstable and constant load;

fourthly, when the network is in an overload state, two network delay critical values H are set in sequence₁And H₂And B is the measured network delay value, the network overload is divided into three levels, and the overload degree is further subdivided:

when B is less than or equal to H₁When the network is in the overload first stage, the network overload degree is relatively low;

when H is present₁<B<H₂In the second overload stage, the network overload degree is heavier;

when B is more than or equal to H₂In the third overload stage, the network overload degree is serious;

when the network state is judged for the first time, the network is evaluated by referring to the packet loss rate in two to three periods, and G is evaluated₁And G₂The value is selected, and when the difference between the values is large, the stability of the adaptive transmission control system is facilitated, but the packet loss rate is ensured to be G₂In time, the received video can be watched normally, so the packet loss rate is criticalThe value is determined by the receiving condition of the video watched by the user, and the time delay jitter critical value C_aIs chosen between 20ms and 50 ms.

In the adaptive video sampling rate adjustment subsystem, a sending end should have the capability of adaptively adjusting the data sending rate, and the capability of adaptively adjusting the data sending rate is described as follows: according to the amount of available bandwidth of the network channel, the transmission rate of the client of the data is increased or reduced, if the bandwidth of the network channel is increased, the transmission rate is increased, if the bandwidth of the network channel is reduced, the transmission rate is reduced, the load capacity in the network channel is controlled, and the effect of matching with the network bandwidth is achieved;

in a real-time video streaming transmission system, the rate of sending data by a sending end is more effectively adjusted by adjusting video coding parameters, and the coding compression of a video is realized by adopting an open source model X264 technology of the H.264 standard;

the adaptive video sampling rate adjustment method includes video sampling rate control of X264 and adaptive video sampling rate control.

The adaptive video sampling rate control is based on an increasing, multiplying and decreasing algorithm, when the network is overloaded, the multiplying type is adopted to decrease the output video sampling rate, and the requirement on the required channel bandwidth is timely reduced; when the network is under load, the sampling rate of the output video is increased in a sum mode, and the available resources of a channel are effectively utilized; when the channel load transmits data under normal conditions, whether the channel has an overload trend is evaluated according to the size of time delay jitter, if the channel does not have the overload trend, the output video sampling rate does not need to be reduced, if the channel has the overload trend, the output video sampling rate is finely adjusted to achieve the pre-control of network overload, and the specific method for controlling the self-adaptive video sampling rate comprises the following steps:

firstly, when the network obtained through evaluation is in an overload state, the multiplicative attenuation element b is used for reducing the sampling rate of the output video, and the current bit rate parameter is L_CAnd then:

L_C＝max{b×L_C，L_minformula 9 (0 < b < 1)

Secondly, when the network obtained through evaluation is in an underload state, the sum formula is usedGrowth factor L_inIncreasing output video sampling rate, with L for current bit rate parameter_CAnd then:

L_C＝min{L_C+L_in，L_min} (L_in＜＜L_C) Formula 10

Thirdly, when the network obtained through evaluation is in an unstable and constant load state, a subtractive element L is used_sdFine tuning the output video sampling rate, L for the current bit rate parameter_CAnd then:

L_C＝max{L_C-L_sd，L_min} (L_sd＜＜L_C) Formula 11

Fourthly, when the network obtained through evaluation is in a stable constant load state, the sampling rate of the output video is not required to be adjusted, and then L is_CThe change is not changed;

L_maxand L_minIs two critical values of the bit rate parameter, is the variable range of the bit rate parameter, when L_CLess than L_minWhen the video is compressed, the coding parameters are too small, and the compression is excessive, so that the quality of the coded video is seriously influenced; when L is_CIs equal to L_maxThen, the quality of the coded video meets the requirement, so the coding parameter does not need to be L_maxIt is also large, wasting network resources.

The non-key frame active abandoning is that in the network video communication process, if the channel bandwidth is seriously insufficient and the network available resources are seriously in short supply, a sending end side actively abandons part of video frames with lower importance, and the frame loss is to make the video sampling rate compressed and output by a video encoder to be suitable for the current channel bandwidth change as much as possible; the method for actively abandoning the non-key frames is independent of a video encoder, when the network is in a severe overload state, the effect of overload control cannot be achieved simply by adjusting encoding parameters, and at the moment, a sending end needs to take measures of actively choosing to abandon some non-key frames, so that the transmission capacity of network video data is reduced, and overload is controlled; the non-key frame active discarding method does not need to adjust the encoding parameters, only needs to determine the type of the discarded frame through a frame selection technology, and finally sends the video frame which is not discarded.

In the H.264 coding of the application, the video frame types comprise a P frame, an I frame and a B frame;

the method adopts the network delay element as the frame loss judgment basis in the frame loss strategy, when the network is evaluated and judged to be in an overload state, whether the bit rate parameter of the current video encoder reaches the minimum value is further judged, and if the bit rate parameter reaches the minimum L_C＝L_minIf the network overload reaches a certain severity, the network overload level is at least at the second level, and the sender decides to enable the non-key frame active discarding strategy according to the network overload level:

if the network overload is of the second level, the network delay satisfies H₁<B<H₂When the type of the current video frame is B frame, discarding the B frame before encoding, and when the type of the current video frame is not B frame, not discarding, and continuing to judge;

if the network overload is the third level, the network delay satisfies that B is more than or equal to H₂The overload degree of the network is very serious, and the current video frame type is a B frame or a P frame, the video frame type is discarded before coding, so that the data volume transmitted in a channel is reduced, and the pressure of the channel is reduced to the maximum extent;

if the network overload is the first level, the sender does not need to adopt a frame loss strategy, all data of the video is sent and transmitted, network resources are effectively utilized, and the quality of the transmitted video and the continuity of the played pictures are ensured.

Compared with the prior art, the application has the advantages and innovation points that:

the video coding conversion transmission control system based on network state self-evaluation, provided by the application, aims at the problems that a network evaluation algorithm in the prior art only judges the network state by means of packet loss rate, the judgment and division of the network state are too rough and the regulation of the video transmission rate has large limitation and the like, deeply analyzes network transmission quality evaluation indexes and methods, provides a method for combining the packet loss rate, network delay and delay jitter as network state evaluation elements, and divides the network state into detailed one, and further divides the network normal load state into normal load with an overload trend and normal load without the overload trend according to the delay jitter on the basis of the evaluation algorithm in the prior art; the overload is divided into three levels according to the network delay elements, the method provides more reliable judgment basis for self-evaluation self-adaptive adjustment of the video data transmission rate, the judgment of the network state is more scientific and accurate, and a good basis is laid for the classification decision of video coding conversion transmission control;

the video coding conversion transmission control system based on network state self-evaluation provided by the application aims at the problem that effective pre-control measures cannot be taken when the network is in an overload trend but is not overloaded by the method in the prior art, improves and optimizes the multiplication and subtraction algorithm, and adds a network overload pre-control method. The improved and optimized algorithm not only adopts a sum type increasing element to increase the bit rate parameter when the network is underloaded, but also adopts a multiplication type attenuation element to reduce the value of the bit rate parameter when the network is overloaded, and can also adopt a reduction type attenuation element to carry out small-amplitude adjustment when the network is in a normal load condition with an overload trend, thereby carrying out pre-control on the occurrence of overload, skillfully designing an overload pre-control scheme and skillfully processing the problem of network overload at the time.

Thirdly, the video coding conversion transmission control system based on network state self-evaluation provided by the application aims at the problems that the network overload is too serious in the prior art, and the overload control cannot be carried out by simply adjusting coding parameters.

Fourthly, the network state evaluation method of the video coding conversion transmission control system based on the network state self-evaluation has the advantages that: compared with the prior art, the adaptive transmission control algorithm in the prior art is more comprehensive, only the packet loss rate of video data is used as the judgment basis of the network state, the limitation is very large, the time delay jitter value is used as overload pre-control, the overload degree is expressed by using network delay, the network state evaluation is more accurate, and the network transmission control can adopt a corresponding adjusting method in time.

Fifthly, the self-adaptive video sampling rate adjustment algorithm of the video code conversion transmission control system based on the network state self-evaluation has the advantages that firstly, the algorithm complexity is low, the realization is easy, and the control effect is obvious; secondly, the video sampling rate is adjusted to be rapidly reduced and slowly increased, the image quality is relatively stable, and the overload control speed is high; thirdly, the buffer area can not generate data accumulation phenomenon, thereby avoiding the problems of overlong delay time, discontinuous pictures and the like; fourthly, overload precontrol can be carried out on the network, and robustness is better.

Sixth, the self-evaluation adaptive transmission control system for real-time video transcoding is developed by combining the proposed transmission control algorithm, and experiments and practices prove that the video transcoding transmission control system has good network transmission control capability and video service quality guarantee mechanism, the judgment of the network state is more accurate, the adjustment of the video data transmission rate is more suitable for the current network bandwidth, and the transmission system has higher robustness and efficiency.

Drawings

Fig. 1 is a schematic structural diagram of a video transcoding transmission control system according to the present application. .

Fig. 2 is a schematic diagram of a video quality of service guarantee mechanism framework according to the present application.

Fig. 3 is a schematic structural diagram of a network state evaluation method according to the present application.

FIG. 4 is a flow chart illustrating a method for adaptive video sampling rate control according to the present application.

FIG. 5 is a flow chart of the non-key frame active discard method of the present application.

Detailed Description

The technical solution of the video transcoding transmission control system based on network status self-evaluation provided in the present application is further described below with reference to the accompanying drawings, so that those skilled in the art can better understand the present application and can implement the present application.

According to the video coding conversion transmission control system based on network state self-evaluation, the transmission rate and the transmission quantity of data are dynamically adjusted in a self-adaptive mode by a video sending end under the condition of fluctuating network bandwidth, so that the transmitted video data quantity is matched with the link channel bandwidth, and the video transmission quality is guaranteed. According to the method and the device, the video data are transmitted by adopting an RTP (real-time transport protocol), a network evaluation scheme and a self-adaptive transmission rate control scheme are designed at a transmitting end, network overload is avoided in the real-time video coding conversion communication process, data loss is reduced, the time delay of the video data is controlled to be minimum, network resources are efficiently utilized, and the video quality is fully guaranteed.

The video coding conversion transmission control system based on network state self-evaluation provides a self-evaluation self-adaptive video coding conversion transmission control method aiming at the requirement of guaranteeing the quality of video coding conversion transmission service, and in the self-evaluation of the network state, a method combining packet loss rate, network delay and time delay jitter as network state evaluation elements is provided, the network state is divided into three basic states of underload, normal load and overload, and when the network is in normal load, the time delay jitter element is introduced to judge whether the current network has an overload trend; when the network is overloaded, a network delay element is introduced to judge the level of the overload degree of the network, in the adaptive transmission control, an adjustment algorithm based on the increasing, multiplying and subtracting video sampling rate is optimized, a network overload pre-control method is added to the increasing, multiplying and subtracting algorithm, when the video sampling rate parameter is adjusted to the set minimum value and the network is still overloaded, a non-key frame active discarding algorithm is started, the discarded frame type is selected according to the evaluated network overload level, the transmitted data volume is reduced, a non-key frame discarding active strategy is started before encoding, and the workload of video encoding is reduced.

Adaptive transmission control system framework

The method mainly aims at the transmission of video data from a server to a client in a video coding conversion transmission control system, the server is used as a sending end, the client is used as a receiving end, and code stream data in an original format is sent to the client to be received and played by an RTP (real-time transport protocol) protocol after being coded and converted at the sending end.

As shown in fig. 2, the overall scheme of the present application mainly includes three subsystems: a network state self-evaluation subsystem, a self-adaptive video sampling rate adjustment subsystem and a non-key frame active abandoning subsystem.

The basic workflow of the video coding conversion transmission control system based on network state self-evaluation provided by the application is as follows: after setting a basic code conversion requirement at a client, a user starts a real-time video code conversion communication session, the client firstly sends a common signaling to a server, the common signaling comprises a total path number, a code conversion platform, a code conversion sequence and code conversion mode information, after receiving the common signaling, the server clears a signaling storage cache and a code conversion task according to the last received common signaling, then creates a new signaling storage cache and initializes a cache space; then the client sends a request signaling, the request signaling comprises the format, frame rate, resolution and video sampling rate information of an original code stream and a target code stream, the server sends a request completion confirming signaling after receiving the request signaling, and then the server starts to send code stream data to perform video code conversion transmission work; and the server calculates packet loss rate, network delay and time delay jitter parameter values of the code stream, encapsulates the packet loss rate, the network delay and the time delay jitter parameter values into an SR report packet, and sends the SR report packet to the client so as to evaluate the network state and adjust the sending rate.

Second, network state self-evaluation method

The method aims at the problem that in the prior art, a network evaluation algorithm only judges the network state by depending on the packet loss rate, the network state evaluation division is too rough, and the video transmission rate adjustment basis is too limited. The network state evaluation judgment of the network state self-evaluation method is through feedback information in an RTCP real-time transport protocol feedback mechanism, if the packet loss rate, the network delay and the time delay jitter parameter values in the feedback information are large, namely the data packet delay is changed greatly, the network state is not good, and if the packet loss rate, the network delay and the time delay jitter parameter values are small, the network state is good.

The size of the packet loss rate reflects the current network condition, and when the packet loss rate exceeds a reasonable critical value, the network bandwidth is seriously insufficient, the channel load exceeds the maximum load capacity, and the network overload occurs. The delay jitter is used for evaluating whether the channel has a tendency of overload under a normal load working state. According to the method and the device, the network overload is pre-controlled by using the time delay jitter, the output video sampling rate of the system is adjusted in advance, and the network overload is avoided. The degree grade of the network overload is judged by using the network delay element, the network overload is divided into different grades, the judgment of the network state is more accurate, the adjustment of the video data transmission rate is more suitable for the current network bandwidth, and the transmission system has higher robustness and high efficiency, which is also an important innovation point of the application.

(I) calculation of each element of network state self-evaluation method

Queue management is an important technology for realizing overload control, a buffer area is distributed at a receiving end, video data which is sent and arrived is put in the buffer area to wait for reading, and in order to control excessive data of the buffer area and overlong queues when a network is overloaded, partial data needs to be selected to be discarded to shorten the length of the queues and adjust the occupancy rate of the buffer area. The queue management of the application adopts an overload pre-control mechanism and active queue management, the overload pre-control mechanism determines whether to discard the packet in advance according to the length of a data queue in a network link, if the queue length is longer, a part of data is selected to be discarded, and if the queue length is in a reasonable range, the packet is not discarded in advance. Active queue management first calculates an average queue length and compares it to two thresholds, does not discard video data when the calculated average queue length is below a minimum threshold, discards every arriving data packet when the calculated average queue length is above a maximum threshold, and selectively discards some data packets when the average queue length is between the two thresholds.

The network overload grade is divided according toThe network delay B is calculated by calculating the average value E of the long-term accumulated time delay jitter before calculating the network delay B_LAnd the average value E of the short-term accumulated delay jitter_dAccumulated delay jitter FC_kIs calculated as shown in equation 1, the average value E of the accumulated delay jitter_kCalculating formula 2, wherein the parameter k represents the data amount,

FC_k＝C₁+C₂+…C_kformula 1

(1) Long term accumulated delay jitter E_LCalculation of the mean value: counting the jitter of the transmitted data from the first data to the current time, and calculating the jitter of the transmitted data according to equation 3, where the parameter i represents the data amount at the current time:

(2) short term accumulated time delay jitter E_dCalculating the average value of (1); the method comprises the following steps of counting the accumulated time delay jitter calculated when a receiving end receives current data and the accumulated time delay jitter before the receiving end receives the data, wherein the average value calculated by the two accumulated time delay jitter is represented by formula 4, and a parameter i in the formula represents the data volume at the current moment:

if E_dGreater than E_LIt shows that the data transmission delay is rapidly increasing, the network load is continuously increasing, and at this time, the network is gradually overloaded. When the overload is not serious enough in the initial stage, the frequency is increased by adjusting the transmission rate of the transmitting end.

Network delay B is judged to be E_dGreater than E_LMeasured to evaluate the bottleneck chain of the video data in the end-to-end transmission pathThe buffer area using condition of the path takes the time interval between two adjacent SR report packets received by a sender as a measuring period, a receiving end calculates the one-way transmission delay D of a data packet every time the receiving end receives the data packet in the measuring period, then compares the delay of all data in the testing period, finds out the maximum and minimum data delay values, and calculates the D of the ith RTP packet received in a measuring unit time_iThe value is compared with the one-way transmission time delay of all the data packets in the period, and the maximum value D of the data packets is found_maxAnd a minimum value D_minCalculating the maximum queue delay F_max：

F_max＝D_max-D_minFormula 5

F_i＝D_i-D_minformula 6

(II) network state evaluation method

The application provides a method for combining packet loss rate, network delay and delay variation as network state evaluation elements, and divides the network state in detail, as shown in fig. 3, the specific method for evaluating the network state is as follows:

(1) the network state is preliminarily judged, and if the packet loss rate A is larger than the maximum packet loss rate critical value G in the continuous m test periods₂I.e. A>G₂If the network is always in the overload state, judging that the current network is in the overload state;

(2) if the packet loss rate A is smaller than the minimum packet loss rate critical value G in all the n continuous test periods₁I.e. A<G₁If the network load is constant, judging that the current network is under load;

(3) in addition to the above two cases, the packet loss rate is between two critical values, i.e. G₁≤A≤G₂The network is in a normal load state; for the evaluation of the normal load state, the method carries out further evaluation, and sets a time delay jitter critical value C_aAnd C is the measured delay jitter value:

(4) when the network is in an overload state, two network delay critical values H are set in sequence₁And H₂And B is the measured network delay value, the network overload is divided into three levels, and the overload degree is further subdivided:

when the network state is judged for the first time, the network can not be evaluated by simply referring to the packet loss rate value in one testing period and referring to the packet loss rates in two to three periods, so that the output video sampling of the client is avoidedThe rate appears to fluctuate in a saw-tooth fashion. For G₁And G₂The value is selected, and when the difference between the values is large, the stability of the adaptive transmission control system is facilitated, but the packet loss rate is ensured to be G₂In the process, the received video can be watched normally, so that the value of the packet loss rate critical value is determined by the receiving condition of the video watched by the user. Delay jitter critical value C_aIs selected between 20ms and 50ms, and the overload is subjected to pre-control processing. Network delay threshold H₁And H₂The method is a basis for dividing the overload degree of the network, is very important in a non-key frame active discarding algorithm, and selects the discarded frame type according to the overload level of the network. For real-time video stream data transmission, H₁And H₂The values of (d) can be set to 2.8% and 6.2%; in the real-time video communication transmission process, as long as the time delay jitter does not exceed 50ms, the video quality can be accepted by users, and the picture quality is in a suitable range.

The network state evaluation method has the advantages that: compared with the prior art, the adaptive transmission control algorithm in the prior art is more comprehensive, only the packet loss rate of video data is used as the judgment basis of the network state, the limitation is very large, the time delay jitter value is used as overload pre-control, the overload degree is expressed by using network delay, the network state evaluation is more accurate, and the network transmission control can adopt a corresponding adjusting method in time.

Adaptive video sampling rate adjusting method

When a video data packet is transmitted under the condition of network overload, a packet loss phenomenon can occur. With the change of the network state, the sending end should have the capability of adaptively adjusting the data sending rate, that is, changing the self-output video sampling rate. The ability to adaptively adjust the data transmission rate is described as: according to the amount of available bandwidth of the network channel, the transmission rate of the client of the data is increased or decreased, if the bandwidth of the network channel is increased, the transmission rate is increased, if the bandwidth of the network channel is decreased, the transmission rate is decreased, the load capacity in the network channel is controlled, and the effect of matching with the network bandwidth is achieved. The method has the greatest advantages of simplicity and easiness in operation, and the transmission quality of the video can be ensured only by changing the bit rate parameter of the video encoder without taking other measures.

In a real-time video streaming transmission system, it is more effective to adjust the rate of sending data by a sending end by adjusting video coding parameters, at present, coding compression technologies such as h.264 support dynamic adjustment of the sampling rate of output video, and the application adopts an open source model X264 technology of the h.264 standard to realize coding compression of video.

Video sampling rate control for X264

In the encoder library of X264, the adjustment of the bit rate parameters is based on libavcodec, and is suitable for video communication services with high real-time requirements. The video sampling rate control of X264 is mainly embodied in three aspects: QP, Bitrate and crf, when in use, only one of the three methods is selected, the video sampling rate control method of the QP is a quantization parameter in the fixed encoding process, and the size of a finally encoded file cannot be predicted; the sampling rate of the target video in the Bitrate fixed coding process is unchanged, and the size of the finally obtained coded file is predictable; crf achieves the quality requirement of video by dropping frames and discarding important frames, and the size of the obtained encoded file is unpredictable. The adaptive transmission strategy is suitable for selecting a method with a fixed bit rate to control the video sampling rate.

In an X264 encoder, a single-pass encoding mode is adopted for controlling the video sampling rate, the single-pass encoding mode can achieve the effect of secondary encoding, the single-pass encoding mode takes the characteristics of real-time video communication into consideration, and an average video sampling rate control algorithm is adopted

The average video sampling rate control algorithm only calculates corresponding average complexity for the coded video frame, then predicts the complexity of the current frame by using the average complexity value, and makes a proper QP selection, and the whole process does not need the participation of the uncoded video frame. The average video sample rate control algorithm does not take into account the non-proportional relationship of QP to output video sample rate, but only changes the encoding parameters at the video encoder. The average video sampling rate control algorithm compensates the excessive overflow of the data in the buffer area, so that the effect of secondary compression is approximately achieved by primary compression, the average video sampling rate control algorithm can control the size of the compressed data, and the video quality can not be reduced.

The X264 video coding and decoding algorithm has a correlation function of dynamically readjusting coding parameters, effectively utilizes network resources by adjusting the matching of bit stream parameters with fluctuating network bandwidth, and adjusts corresponding bit rate parameter values according to the estimated network state so as to enable the bit rate parameter values to be closer to the expected target video sampling rate.

(II) adaptive video sampling rate control

The adaptive video sampling rate control is based on an increasing, multiplying and decreasing algorithm, when the network is overloaded, the multiplying type is adopted to decrease the output video sampling rate, and the requirement on the required channel bandwidth is timely reduced; when the network is under load, the sampling rate of the output video is increased in a sum mode, and the available resources of a channel are effectively utilized; when the channel load transmits data under normal conditions, whether the channel has an overload trend is evaluated according to the time delay jitter, if the channel does not have the overload trend, the output video sampling rate does not need to be reduced, and if the channel has the overload trend, the output video sampling rate is finely adjusted to achieve the network overload pre-control. As shown in fig. 4, the specific method of adaptive video sampling rate control is as follows:

(1) when the network obtained through evaluation is in an overload state, the output video sampling rate is reduced by using a multiplicative attenuation element b, and the current bit rate parameter is L_CAnd then:

Lc＝max{b×Lc，L_minformula 9 (0 < b < 1)

(2) Using sum growth factor L when the network is under-loaded after evaluation_inIncreasing output video sampling rate, with L for current bit rate parameter_CAnd then:

Lc＝min{Lc+L_in，L_min} (L_inlc) formula 10

(3) When the network obtained through evaluation is in an unstable and constant load state, a subtractive element L is used_sdFine tuning the output video sampling rate, L for the current bit rate parameter_CAnd then:

Lc＝max{Lc-L_sd，L_min} (L_sdlc) formula 11

(4) When the network obtained through evaluation is in a stable constant load state, the output video sampling rate does not need to be adjusted, and then L_CAnd is not changed.

L_maxAnd L_minAre two critical values of the bit rate parameter and are the variable range of the bit rate parameter. When L is_CLess than L_minWhen the video is compressed, the coding parameters are too small, and the compression is excessive, so that the quality of the coded video is seriously influenced; when L is_CIs equal to L_maxThen, the quality of the coded video meets the requirement, so the coding parameter does not need to be L_maxIt is also large, wasting network resources. Three elements b and L in the algorithm_inAnd L_sdThe value of (A) is very important, b is too large, L_inIf the sampling rate is too small, the video sampling rate has too large variation amplitude, so that a serious oscillation phenomenon can be caused; b is too small, L_inToo large, the transmission control effect is not obvious; l is_sdToo large, the output video sampling rate will fluctuate, L_sdAnd if the value is too small, the effect of pre-control on network overload cannot be achieved.

The self-adaptive video sampling rate adjusting algorithm has the advantages that firstly, the algorithm is low in complexity, easy to realize and obvious in control effect; secondly, the video sampling rate is adjusted to be rapidly reduced and slowly increased, the image quality is relatively stable, and the overload control speed is high; thirdly, the buffer area can not generate data accumulation phenomenon, thereby avoiding the problems of overlong delay time, discontinuous pictures and the like; fourthly, overload precontrol can be carried out on the network, and robustness is better.

Active abandon method for non-key frame

Idea of algorithm (A)

Because the video sampling rate control method has obvious defects, the data output by video coding is certain, the transmission rate is adjusted, and the total video data volume is not changed, therefore, when the channel bandwidth is insufficient and the available resources of the network are in short supply, the transmission rate is reduced, but the data waits for reading and transmission in the buffer area, the transmission delay of the data packet is increased, and the real-time performance is deteriorated. In the subjective experience of the user, the video image received by the receiving end has the problems of jamming, discontinuous display and the like.

The non-key frame active abandoning is that in the network video communication process, if the channel bandwidth is seriously insufficient and the available resources of the network are seriously in short supply, a sending end side actively abandons part of video frames with lower importance, and the frame dropping is to make the video sampling rate compressed and output by a video encoder to be suitable for the current channel bandwidth change as much as possible. The method for actively abandoning non-key frames is independent of a video encoder, when the network is in a severe overload state, the effect of overload control cannot be achieved simply by adjusting encoding parameters, and at the moment, a sending end needs to take measures for actively choosing to abandon some non-key frames, so that the transmission quantity of network video data is reduced, and overload is controlled. The non-key frame active discarding method does not need to adjust the encoding parameters, only needs to determine the type of the discarded frame through a frame selection technology, and finally sends the video frame which is not discarded.

Video frame type division

When the network is in an overload state, a video sampling rate adjustment strategy is preferred, and when the video sampling rate output by a video encoder reaches a set minimum value, if a channel is still in the overload state, a non-key frame active abandon method is adopted to reduce the requirement on the network bandwidth.

In the h.264 coding of the embodiment of the present application, the video frame types include P frame, I frame, and B frame.

The P frame is an inter-prediction image, and is a forward prediction compressed frame with a higher compression rate by adopting an inter-prediction encoding mode, and the complete picture can be restored by considering the latest I frame or P frame in the front during decoding. P-frames have a drawback in decoding reconstructed images, and are relatively liable to cause diffusion of error information.

The I frame is an intra-frame image, adopts an intra-frame prediction coding mode, is an independent frame for coding and transmitting the whole frame of image data, and can reconstruct the whole frame of image by only using the information of the image picture of the frame of the I frame without adopting other frame of image data during decoding. The importance of the I frame is the highest, and the key frame in the whole video stream cannot be lost, because the key frame is the reference frame of the P frame and the B frame, if the I frame is lost, all the P frame and B frame images in the same group cannot be reconstructed.

The B frames are bidirectional predicted images, decoding and reconstruction of the images need to be finished by referring to image data of a front frame and image data of a rear frame, the importance of the B frames in the whole video stream is the lowest, and the importance of all the B frames in the same image group is the same.

(III) non-key frame active discard strategy

The method adopts the network delay element as the frame loss judgment basis in the frame loss strategy, and takes the counted network delay size as the reference standard for selecting the frame loss type.

The transmission control is carried out in the video coding conversion, the coding conversion is a full-decoding full-coding process, the video frame can store the related information of the frame in the decoding process, and the related information can be directly used in the coding process, so that repeated work is omitted. Therefore, in the frame loss scheme, some video frames need to refer to the coding information of the previous frame image when being coded, and at the moment, if the previous frame is abandoned, the information of the previous frame when being decoded can be selected, and the coding of the next frame cannot be influenced. Frame dropping is performed before video coding conversion, and the video frames needing to be dropped do not need to be coded, so that the workload of coding is reduced. This is also an advantage and innovation of the transmission control method of the present application.

As shown in fig. 5, when the network is evaluated and determined as an overload state, it is further determined whether the bit rate parameter of the current video encoder reaches the minimum value, and if the bit rate parameter reaches the minimum L_C＝L_minIf the network overload reaches a certain severity, the network overload level is at least at the second level, and the sender decides to enable the non-key frame active discarding strategy according to the network overload level:

if the network overload is of the second level, the network delay satisfies H₁<B<H₂And when the current video frame type is a B frame, discarding the B frame before encoding, and when the current video frame type is not the B frame, not discarding, and continuing to judge.

If the network overload is the third level, the network delay satisfies that B is more than or equal to H₂The overload degree of the network is very serious, and the current video frame type, whether a B frame or a P frame, is discarded before coding, so that the data volume transmitted in a channel is reducedAnd the channel pressure is reduced to the maximum extent.

In the non-key frame active discarding method, the network delay threshold value H₁And H₂The value of (A) is very important if H₁If the data transmission rate is too low, the data transmission rate is unnecessarily low, network resources are wasted, and the throughput is obviously reduced when video data are transmitted; if H is₁If the value is too large, the network is in a state of serious overload, but when a non-key frame active discarding strategy is started, wrong judgment occurs, the frame rate is not reduced enough, the network overload cannot be controlled, the overload degree is continuously increased, the utilization rate of the channel bandwidth is reduced, and even network channel collapse is generated. And a network delay threshold value H₂If the value is too small, and the network overload state does not reach a certain degree of severity, B frames and P frames in the video frame type are selected to be discarded, so that frames which are not necessary to be discarded are discarded, the video quality received by a receiving end is reduced, even interruption occurs, a user cannot tolerate the video, network resources are wasted, the rate is excessively reduced, and the network is possibly in an underload state; but if H₂If the value is too large, the overload degree of the network is evaluated to be lower than the actual overload state, the corresponding speed cannot be reduced, and the overload is more and more serious.

The method for actively abandoning the non-key frames has the main advantages of simple scheme design, capability of reducing the network transmission amount and the coding workload when the network is heavily overloaded, and no need of complex calculation.

The method and the device utilize the network delay elements to further divide the network overload level, so that the evaluation of the network state is more accurate. In the sending rate adjustment algorithm, a non-key frame active discarding method is added, a network delay element is used as an evaluation standard for frame loss selection, and the method is used under the condition that the network overload is serious. The combination of the video sampling rate control method and the non-key frame active abandoning method ensures that the transmission rate of the sending end is matched with the current network bandwidth in the control of the sending rate, thereby ensuring the service quality of video transcoding communication.

Claims

1. The video coding conversion transmission control system based on network state self-evaluation is characterized in that a video sending end self-adaptively and dynamically adjusts the transmission rate and the transmission quantity of data to enable the transmitted video data quantity to be matched with the bandwidth of a link channel; in the self-evaluation of the network state, a method combining packet loss rate, network delay and time delay jitter as network state evaluation elements is provided, the network state is divided into three basic states of underload, normal load and overload, and when the network is in normal load, the time delay jitter element is introduced to judge whether the current network has an overload trend; when the network is overloaded, a network delay element is introduced to judge the level of the overload degree of the network, in the adaptive transmission control, an adjustment algorithm based on the multiplication and subtraction video sampling rate is optimized, a network overload pre-control method is added to the multiplication and subtraction algorithm, when the video sampling rate parameter is adjusted to the set minimum value and the network is still overloaded, a non-key frame active discarding method is started, the discarded frame type is selected according to the evaluated network overload level, the transmitted data volume is reduced, and a non-key frame discarding active strategy is started before encoding;

the basic working flow is as follows: after setting a basic code conversion requirement at a client, a user starts a real-time video code conversion communication session, the client firstly sends a common signaling to a server, the common signaling comprises a total path number, a code conversion platform, a code conversion sequence and code conversion mode information, after receiving the common signaling, the server clears a signaling storage cache and a code conversion task according to the last received common signaling, then creates a new signaling storage cache and initializes a cache space; then the client sends a request signaling, the request signaling comprises the format, frame rate, resolution and video sampling rate information of an original code stream and a target code stream, the server sends a request completion confirming signaling after receiving the request signaling, and then the server starts to send code stream data to perform video code conversion transmission work; the server calculates packet loss rate, network delay and time delay jitter parameter values of the code stream, packages the parameter values into an SR report packet, and sends the SR report packet to the client to evaluate the network state and adjust the sending rate;

aiming at the transmission of video data from a server to a client in a video coding conversion transmission control system, the server is used as a sending end, the client is used as a receiving end, the code stream data in the original format is sent to the client for receiving and playing by adopting an RTP (real time protocol) protocol after being subjected to coding conversion at the sending end, an adjustment algorithm of video sampling rate parameters is completed in a coding function of a coder converter, a non-key frame active abandoning strategy is carried out before coding, and the workload of video coding is reduced;

the system comprises three subsystems: a network state self-evaluation subsystem, a self-adaptive video sampling rate adjustment subsystem and a non-key frame active abandoning subsystem;

the size of the packet loss rate reflects the current network condition, when the packet loss rate exceeds a reasonable critical value, the network bandwidth is seriously insufficient, the channel load exceeds the maximum load capacity, and the network overload occurs; the time delay jitter is used for evaluating whether the channel has a tendency of overload to be generated in a normal load working state; the network overload is pre-controlled by using the time delay jitter, the output video sampling rate of the system is adjusted in advance, and the network overload is avoided; judging the degree grade of network overload by using a network delay element, dividing the network overload into different grades, judging the network state more accurately, and adjusting the video data transmission rate more suitably to the current network bandwidth;

the network state self-evaluation method comprises a calculation method of each element of the network state self-evaluation method and a network state evaluation method;

in the calculation of each element of the network state self-evaluation method, the classification basis of the network overload grade is the size of the network delay B, and the network delay is calculatedBefore B, calculating the average value E of the long-term accumulated delay jitter_LAnd the average value E of the short-term accumulated delay jitter_dAccumulated delay jitter FC_kIs calculated as shown in equation 1, the average value E of the accumulated delay jitter_kThe calculation is as in equation 2,

FC_k＝C₁+C₂+…C_kformula 1

Average value E of long-term accumulated delay jitter_LThe calculation of (2): counting the jitter of all the transmitted data from the first data transmitted until the current time, and calculating the formula as 3:

average value of short-term accumulated delay jitter E_dThe calculation of (2) is to count the accumulated delay jitter calculated when the receiving end receives the current data and the accumulated delay jitter before the receiving end receives the data, and the average value calculated by the two accumulated delay jitters is represented by formula 4:

network delay B is judged to be E_dGreater than E_LThe method comprises the steps of (1) measuring, evaluating the buffer use condition of a bottleneck link in an end-to-end transmission path of video data, taking a time interval between two adjacent SR report packets received by a sending end as a measurement period, and enabling a receiving end to be at one endIn a measuring period, every time a data packet is received, calculating the one-way transmission time delay D of the data packet, then comparing the delay of all data in a testing period, finding out the maximum and minimum data time delay values, and for the ith RTP packet received in a measuring unit time, firstly calculating the D_iThe value is compared with the one-way transmission time delay of all the data packets in the period, and the maximum value D of the data packets is found_maxAnd a minimum value D_minCalculating the maximum queue delay F_max：

F_max＝D_max-D_minFormula 5

F_i＝D_i-D_minformula 6

that is, the network delay B is the average network delay of data in a unit period and the maximum queue delay F_maxThe value of the network delay B is in a value range of 0,1]When the value is 0, the buffer area is empty, no data is queued up, and the buffer area is full and the network is heavily overloaded, which generally occurs at the beginning of transmission;

first, the network state is processedThe preliminary judgment shows that if the packet loss rate A is larger than the maximum packet loss rate critical value G in the continuous m test periods₂I.e. A>G₂If the network is always in the overload state, judging that the current network is in the overload state;

thirdly, in addition to the above two cases, the packet loss rate is between two critical values, i.e. G₁≤A≤G₂The network is in a normal load state; for the evaluation of the normal load state, further evaluation is carried out, and a time delay jitter critical value C is set_aAnd C is the measured delay jitter value:

when the network state is judged for the first time, the network is evaluated by referring to the packet loss rate in two to three periods, and G is evaluated₁And G₂The value is selected, and when the difference between the values is large, the stability of the adaptive transmission control system is facilitated, but the packet loss rate is ensured to be G₂In time, the received video can be watched normally, so the value of the critical value of the packet loss rate is determined by the receiving condition of the video watched by the userTiming and delay jitter critical value C_aIs selected between 20ms and 50 ms;

the self-adaptive video sampling rate adjusting method comprises X264 video sampling rate control and self-adaptive video sampling rate control;

the self-adaptive video sampling rate control is based on an increasing, multiplying and decreasing algorithm, when the network is overloaded, the multiplying type is adopted to decrease the output video sampling rate, and the requirement on the required channel bandwidth is timely reduced; when the network is under load, the sampling rate of the output video is increased in a sum mode, and the available resources of a channel are effectively utilized; when the channel load transmits data under normal conditions, whether the channel has an overload trend is evaluated according to the size of time delay jitter, if the channel does not have the overload trend, the output video sampling rate does not need to be reduced, if the channel has the overload trend, the output video sampling rate is finely adjusted to achieve the pre-control of network overload, and the specific method for controlling the self-adaptive video sampling rate comprises the following steps:

L_c＝max{b×L_c,L_minformula 9 (0 < b < 1)

Secondly, when the network obtained through evaluation is in an underload state, a sum type growth factor L is used_inIncreasing output video sampling rate, with L for current bit rate parameter_CAnd then:

Lc＝min{L_c+L_in，L_min} (L_in＜＜L_c) Formula 10

Lc＝max{Lc-L_sd，L_min} (L_sd＜＜L_c) Formula 11

L_maxand L_minIs two critical values of the bit rate parameter, is the variable range of the bit rate parameter, when L_CLess than L_minWhen the video is compressed, the coding parameters are too small, and the compression is excessive, so that the quality of the coded video is seriously influenced; when L is_CIs equal to L_maxThen, the quality of the coded video meets the requirement, so the coding parameter does not need to be L_maxThe method is large and wastes network resources;

the non-key frame active abandoning is that in the network video communication process, if the channel bandwidth is seriously insufficient and the network available resources are seriously in short supply, a sending end side actively abandons part of video frames with lower importance, and the frame loss is to make the video sampling rate compressed and output by a video encoder to be suitable for the current channel bandwidth change as much as possible; the method for actively abandoning the non-key frames is independent of a video encoder, when the network is in a severe overload state, the effect of overload control cannot be achieved simply by adjusting encoding parameters, and at the moment, a sending end needs to take measures of actively choosing to abandon some non-key frames, so that the transmission capacity of network video data is reduced, and overload is controlled; the non-key frame active discarding method does not need to adjust coding parameters, only needs to determine the type of the discarded frame through a frame selection technology, and finally sends the video frame which is not discarded;

in H.264 coding, the video frame types comprise P frames, I frames and B frames;

using network delay element as frame loss strategyBased on the frame loss judgment, when the network is evaluated and judged to be in an overload state, whether the bit rate parameter of the current video encoder reaches the minimum value is further judged, and if the bit rate parameter reaches the minimum L_C＝L_minIf the network overload reaches a certain severity, the network overload level is at least at the second level, and the sender decides to enable the non-key frame active discarding strategy according to the network overload level: