CN115801683A

CN115801683A - Video processing method and device, electronic equipment and storage medium

Info

Publication number: CN115801683A
Application number: CN202211408416.8A
Authority: CN
Inventors: 李振; 林光亮; 杨水清
Original assignee: Yuanjingshengsheng Beijing Technology Co ltd
Current assignee: Yuanjingshengsheng Beijing Technology Co ltd
Priority date: 2022-11-10
Filing date: 2022-11-10
Publication date: 2023-03-14

Abstract

The application provides a video processing method. The method comprises the following steps: sending a video data stream, wherein the video data stream comprises at least one data packet; judging whether a data packet is lost in the video data stream, wherein if the data packet is lost, judging whether the attribute of a network for sending the video data stream is known, if the attribute of the network is unknown, reducing the data volume to be sent in a first preset time period to be less than a first data volume threshold value, detecting the change of round-trip delay in the first preset time period, identifying the attribute of the network according to the change of the round-trip delay, and continuously judging whether the data packet is lost in the video data stream.

Description

Video processing method and device, electronic equipment and storage medium

Technical Field

The present application relates to the field of network technologies, and in particular, to a video processing method and apparatus, an electronic device, and a storage medium.

Background

In the process of network video data stream transmission, the situation that some video data packets are lost and cannot reach a receiving end often occurs because of network reasons. If the lost data packet is not processed, the transmission data is lost, and the video is blocked. The reasons for packet loss are usually congestion packet loss due to network congestion and random packet loss due to non-network congestion. In some cases, data recovery is usually performed for all lost packets, but in case of network congestion, recovering data in large amounts may instead aggravate the network congestion, causing more packets to be dropped.

Disclosure of Invention

The embodiment of the application provides a video processing method, a video processing device, electronic equipment and a storage medium, so as to solve the problems in the related art.

In a first aspect, an embodiment of the present application provides a video processing method, where the method includes: sending a video data stream, wherein the video data stream comprises at least one data packet; and judging whether a data packet is lost in the video data stream, wherein if the data packet is lost, judging whether the attribute of a network for sending the video data stream is known, if the attribute of the network is unknown, reducing the data volume to be sent in a first preset time period to be less than a first data volume threshold value, detecting the change of round-trip delay in the first preset time period, identifying the attribute of the network according to the change of the round-trip delay, and continuously judging whether the data packet is lost in the video data stream.

In a second aspect, an embodiment of the present application provides a video processing method, including: sending a video data stream, wherein the video data stream comprises at least one data packet; judging whether a data packet is lost in the video data stream, wherein if the data packet is lost, judging whether the attribute of a network for sending the video data stream is known, if the attribute of the network is unknown, reducing the data volume to be sent in a first preset time period to be less than a first data volume threshold value, detecting the change of round-trip delay in the first preset time period, identifying the attribute of the network according to the change of the round-trip delay, and processing the lost data packet according to a preset strategy, wherein if the network is a congested network, the lost data packet is processed according to the first preset strategy; and if the network is not a congested network, processing the lost data packet according to a second predetermined policy.

In a third aspect, an embodiment of the present application provides an apparatus for video processing, including: the video transmitting unit is configured to transmit a video data stream, and the video data stream comprises at least one data packet; the processing unit is configured to judge whether a data packet is lost in the video data stream, wherein if the data packet is lost, the processing unit judges whether the attribute of a network for sending the video data stream is known, wherein if the attribute of the network is unknown, the processing unit reduces the data volume to be sent in a first preset time period to be less than a first data volume threshold value, detects the change of the round trip delay in the first preset time period, identifies the attribute of the network according to the change of the round trip delay, and continuously judges whether the data packet is lost in the video data stream.

In a fourth aspect, an embodiment of the present application provides an electronic device, including a memory, a processor, and a computer program stored on the memory, where the processor implements the method of any one of the above when executing the computer program.

In a fifth aspect, the present application provides a computer-readable storage medium, in which a computer program is stored, and the computer program, when executed by a processor, implements the method of any one of the above.

Compared with the prior art, the method has the following advantages:

according to the embodiment of the application, the reasons for video data packet loss can be classified, and corresponding strategies are adopted for processing according to the reasons for packet loss, so that the problem of packet loss generated under the conditions of congested and uncongested networks can be effectively solved, the video pause phenomenon is reduced, and better video watching experience is provided for users.

The foregoing description is only an overview of the technical solutions of the present application, and the following detailed description of the present application is given to enable the technical means of the present application to be more clearly understood and to enable the above and other objects, features, and advantages of the present application to be more clearly understood.

Drawings

In the drawings, like reference numerals refer to the same or similar parts or elements throughout the several views unless otherwise specified. The figures are not necessarily to scale. It is appreciated that these drawings depict only some embodiments in accordance with the disclosure and are not to be considered limiting of its scope.

Fig. 1 is a flow chart illustrating a video processing method according to an embodiment of the present application;

FIG. 2 is a flow chart illustrating a video processing method of another embodiment of the present application;

fig. 3 is a block diagram showing a configuration of a video processing apparatus according to an embodiment of the present application;

FIG. 4 is a block diagram illustrating an electronic device of an embodiment of the present application; and

fig. 5 is a graph illustrating a variation trend of round trip delay according to an embodiment of the present application.

Detailed Description

In the following, only certain exemplary embodiments are briefly described. As those skilled in the art will recognize, the described embodiments may be modified in various different ways, without departing from the spirit or scope of the present application. Accordingly, the drawings and description are to be regarded as illustrative in nature, and not as restrictive.

In order to facilitate understanding of the technical solutions of the embodiments of the present application, the following description is made of related art of the embodiments of the present application. The following related arts as alternatives can be arbitrarily combined with the technical solutions of the embodiments of the present application, and all of them belong to the scope of the embodiments of the present application.

An embodiment of the present application provides a video processing method 100, and the video processing method 100 of the present application will be described below with reference to fig. 1. Fig. 1 is a flow diagram illustrating a video processing method 100 according to an embodiment of the present application. As shown in fig. 1, the video processing method 100 may include: steps S101 to S107 are as follows. The following describes steps S101 to S107 in detail with reference to specific embodiments.

The process first proceeds to step S101. In step S101, a video data stream is transmitted, where the video data stream includes at least one data packet.

In the video processing method 100, it may be a server side to transmit a video data stream. In some embodiments, the server may be a cloud server. In other embodiments, the server may be a base station using mobile communication technology. The above is merely an example of the server, and those skilled in the art can apply the principles of the present application to the device that can transmit video data according to practical situations.

In some embodiments, the video is typically played starting a few seconds after the receiving end begins viewing the video data from the server, while the receiving end receives a subsequent portion of the video from the server. The Video data stream may be a Video data stream such as AVI (Audio Video Interleaved) Video, WMV (Windows Media Video format) Video, or MPEG (Moving Picture Experts Group) Video.

The packet may be a UDP (User Datagram Protocol) packet, which is composed of a header and a payload. The video data stream may include one or more data packets.

Although UDP packets can transport data in real time, the data cannot be guaranteed to arrive in a timely and orderly manner. A header of an RTP (Real-time Transport Protocol) packet includes fields such as a sequence number and a timestamp, and a receiver can determine whether the received packet is ordered or not and whether the packet is lost or not according to the sequence number. In some embodiments, the server encapsulates video data in an RTP (Real-time Transport Protocol) packet, and then encapsulates the RTP packet in a UDP packet, thereby implementing data transmission control.

In some embodiments, the server sends the video data stream to the receiver in the form of UDP combined with RTP, and the receiver can determine the sending order of the data packets and determine whether the data packets are lost through the sequence number in RTP.

Next, the process proceeds to step S102. In step S102, it is determined whether there is a packet loss in the video data stream.

Specifically, it is determined whether there is a packet loss in the current video segment in the video data stream, where the video segment is a component of the video data stream, for example, a 60-second video data stream can be divided into 6 10-second video segments, or 2 30-second video segments. The concrete can be adjusted according to the actual situation.

In some embodiments, whether the data packet sent by the server is correctly received by the receiver may be determined through the feedback information of the receiver. If the data packet is correctly received by the receiving end, the service end continues to send the video data stream without other processing.

The receiving end may enable the server to determine whether the transmitted data packet is lost by feeding back ACK (positive feedback) or NACK (Negative feedback) to the server. The ACK is that the receiving end informs the service end after receiving the data packet, and the NACK is that the receiving end informs the service end when not receiving the data packet.

In some embodiments, the server determines whether the data packet sent by the server is correctly received by the receiving end by setting a timeout. The server side starts a timer when sending the data packet, and if the timer is overtime and the server side still does not receive the feedback of the receiving end about the data packet, the server side considers that the packet is lost. The reason for the data packet loss may be network congestion packet loss or network signal interference packet loss.

The network congestion packet loss may be that a data packet needs to pass through a plurality of data switching nodes, such as routers, on a transmission path from a service end to a receiving end, the cache of the data switching nodes is exhausted, and the data switching nodes discard the data packet. Causes of network congestion typically include data switching node memory space limitations and data switching node processor performance limitations.

The network signal interference packet loss may be data packet loss caused by random noise, burst noise or radio frequency signal interference on a communication link.

In some embodiments, the server sends video data to the receiver in the form of UDP combined with RTP, and the receiver can determine the sending order of the data packets and determine whether the data packets are lost through sequence number in RTP.

If there is a packet loss, the process proceeds to step S103. In step S103, it is determined whether the attribute of the network used to transmit the video data stream is known.

The properties of the network can be divided into two cases, unknown and known. In case the network properties are unknown, the network properties need to be identified, which can be further classified as congested and uncongested in case the network properties are known.

The congestion network is a network in which network congestion packet loss occurs, and the non-congestion network is a network in which network signal interference packet loss occurs.

In some embodiments, the server may pre-record the attribute of the network used to transmit the video data stream, and further determine whether the attribute of the network is known. In other embodiments, the user may set the current network attribute to the server, so that the server determines whether the attribute of the current network is known.

If the attributes of the network are unknown, the process proceeds to step S104. In step S104, the amount of data to be sent in a first predetermined time period is reduced to be less than a first data amount threshold, a change of a round trip delay in the first predetermined time period is detected, an attribute of the network is identified according to the change of the round trip delay, and whether a data packet is lost in the video data stream is continuously determined.

In some embodiments, the amount of data to be transmitted may be the amount of data that has not been completely transmitted in the video data stream after the server detects the packet loss. In other embodiments, the data amount to be sent may be the data amount that the server needs to send in a predetermined time period before and after the data packet is lost; the data stream to be sent may also be the amount of data in the server sending buffer.

Round Trip Time (RTT) is the total Time delay experienced from the Time when the server sends data to the Time when the server receives an acknowledgement from the receiver. The round trip delay is generally composed of three parts, namely the propagation delay of a link, the processing delay of a receiving end and the queuing and processing delay of a router. For example, the server sends a data packet to the receiver at time T1, the receiver sends a feedback to the server after receiving the data packet, and the server receives the feedback at time T2, so that the round-trip delay is T2-T1. After identifying the attributes of the network, the server needs to continuously and repeatedly determine whether more data packets are lost in the video data stream, and process the lost data packets.

In some embodiments, the first predetermined time period is 100 milliseconds, the first data amount threshold is 100 bytes, the sending end may reduce the amount of data to be sent to be less than 100 bytes within 100 milliseconds, and detect the round trip delay change within the time period (i.e., within 100 milliseconds) to determine the attribute of the network. If the round trip delay is approximately normally distributed over time within the time period (the presented shape can be fitted to be normally distributed within a preset error range, for example, the distribution curve has multiple inflection points but the overall trend is normally distributed), or the presented shape is approximately satisfied that the ordinate corresponding to the abscissa starting point is smaller than the ordinate corresponding to the abscissa middle point, the network is identified as a congested network (i.e., the attribute of the network is congestion). And if the round trip delay is not generally in normal distribution over time in the time period and the presented shape generally does not meet the condition that the ordinate corresponding to the abscissa starting point is smaller than the ordinate corresponding to the abscissa intermediate point, identifying the network as a non-congestion network (i.e. the attribute of the network is non-congestion). The above description of the first predetermined time period and the first data amount threshold is only exemplary, and those skilled in the art may adaptively modify the first predetermined time period and the first data amount threshold according to actual needs as long as the principle of the embodiment can be implemented. In other embodiments, the first predetermined time period may be no greater than 1 second and the first data amount threshold may be no greater than 1024 bytes.

In some embodiments, with time as the independent variable and round trip delay as the dependent variable, equation (1) below is a probability density function of the probability distribution of the random variable x obeying the location parameter μ, the scale parameter σ. If the variation of the independent variable and the dependent variable substantially conforms to the following formula (1), or substantially conforms to the rule that the farther x is from μ, the smaller the function operation result, the network can be determined to be a congested network.

In formula (1), x represents time; μ represents the mean of time; σ represents the standard deviation; f (x) represents the operation result of the function.

In some embodiments, time is taken as the abscissa and round trip delay is taken as the ordinate. The abscissa corresponding to the maximum round-trip delay is the time when the data packet starts to be lost, after the data packet is lost, the server side reduces the data transmission amount, and the round-trip delay is reduced along with the increase of time. If the variation curve of the round trip delay conforms to the curve rule of fig. 5, the network can be determined to be a congested network.

The above description of the formula and the curve rule of the round trip delay variation with time is only exemplary, and those skilled in the art can select the round trip delay formula and the round trip delay curve rule according to the actual situation as long as the principle of the embodiment can be implemented.

After the network attribute is identified, the server needs to determine whether a next video segment in the video data stream has a data packet loss, where the next video segment may be a video segment in a next time sequence of a current video segment in the video data stream. For example, in a 60 second video data stream, the current video segment can be the 1 st to 10 th second video segments, and the next video segment can be the 10 th to 60 th second video segments with any length, or the other video frames except the 10 th second video frame to the 60 th second video segments with any length. If the attributes of the network are known, the process proceeds to step S105. In step S105, it is determined whether the network is a congested network.

In some embodiments, the server determines whether the network is a congested network by marking known network attributes, for example, marking a congested network as 0 and marking an uncongested network as 1.

If the network is a congested network, the process proceeds to step S106. In step S106, the lost data packet is processed according to a first predetermined policy.

The first predetermined policy is to reduce the amount of data to be transmitted in the second predetermined time period to less than a second data amount threshold.

In some embodiments, the second predetermined time period may be the same as the first predetermined time period, and the second data volume threshold may be the same as the first data volume threshold.

In an embodiment, the method 100 further comprises: and detecting the change of the round trip delay in the second preset time period at preset time intervals, and calculating the difference value between the round trip delay in the second preset time period and the round trip delay threshold value, wherein if the difference value is in a preset range, the data volume to be sent in the third preset time period is increased to be larger than the third data volume threshold value.

The predetermined time interval may be no more than one fifth of the second predetermined period of time, the round trip delay threshold may be an average of the round trip delays before a packet is lost, and the predetermined range may be an interval of minus 100 milliseconds to plus 100 milliseconds.

The third predetermined time period may be a time period from when the server recognizes that the network congestion has been relieved, for example, the time period may be 1 second.

The third data amount threshold may be a value not less than half of the maximum bandwidth from the server to the receiver, for example, the maximum bandwidth from the server to the receiver is 10M, and the third data amount threshold may be 6M.

In some embodiments, the second time period is 100 milliseconds, the second data volume threshold is 100 bytes, the predetermined time interval is 10 milliseconds, the round trip delay threshold is 60 milliseconds, the predetermined range is negative 10 milliseconds to positive 10 milliseconds, the third predetermined time period is 1 second, and the third data volume threshold is 1000 bytes. If data packet loss occurs in the congested network, the server can reduce the data volume to be sent to 100 bytes within 100 milliseconds, detect the round-trip delay every 10 milliseconds, and increase the data volume to be sent in the next 1 second to be more than 1000 bytes when the difference between the round-trip delay and the round-trip delay threshold falls in the interval from minus 10 milliseconds to plus 10 milliseconds. The above descriptions of the second predetermined time period, the second data amount threshold, the predetermined time interval, the round trip delay threshold, the predetermined range, the third predetermined time period, and the third data amount threshold are only exemplary, and those skilled in the art may adaptively modify the second predetermined time period, the second data amount threshold, the predetermined time interval, the round trip delay threshold, the predetermined range, the third predetermined time period, and the third data amount threshold according to actual needs, as long as the principle of the embodiment can be implemented

If the network is a non-congested network, the process proceeds to step S107. In step S107, the lost packets are processed according to a second predetermined policy.

The second predetermined policy may be to retransmit the lost data packet.

In some embodiments, the server numbers the data packets and sends the data packets to the receiver, the receiver feeds back the sequence number of the data packet that is not received to the server in a NACK manner, and the server resends the lost data packet to the receiver.

In some embodiments, the server numbers the data packet, sends the data packet to the receiving end and starts a timer, and if the timer times out, the server still does not receive the feedback of the receiving end to the data packet, and the server considers that the data packet is lost and resends the lost data packet to the receiving end.

The second predetermined policy may also be to forward error correct lost data packets.

In some embodiments, the server side encodes original data by using a Forward Error Correction (FEC) algorithm, so that redundant feature data is added to the original data, and the receiver side decodes received data by using the same algorithm, thereby correcting errors generated in a data transmission process and recovering a lost data packet. For example, after each n data packets are sent by the server, a redundancy coded data packet is sent, the redundancy coded data packet is obtained by xoring n initial data packets, the server sends n +1 data packets as a group, and if any one data packet is lost in the n +1 data packets, the receiver can reconstruct the lost data packet through the xoring operation.

Another embodiment of the present application provides a video processing method 200, and the video processing method 200 of the present application will be described with reference to fig. 2. Fig. 2 is a flow diagram illustrating a video processing method 200 according to another embodiment of the present application. As shown in fig. 2, the video processing method 200 may include: steps S201 to S206 are as follows. The following describes steps S201 to S206 in detail with reference to specific embodiments.

Steps S201 to S203 in the method 200 are the same as or similar to the corresponding contents of steps S101 to S103 in the method 100 above, respectively. For brevity of the description, detailed descriptions of steps S201 to S203 will be omitted here, and the relevant contents may refer to the description of the method 100 above.

In step S203, if it is determined that the attribute of the network through which the video data stream is transmitted is unknown, the process proceeds to step S204. In step S204, the amount of data to be transmitted in the first predetermined time period is reduced to be smaller than a first data amount threshold, a change of the round trip delay in the first predetermined time period is detected, and the attribute of the network is identified according to the change of the round trip delay.

In some embodiments, the amount of data to be transmitted may be the amount of data that has not been completely transmitted in the video data stream after the server detects the packet loss. In other embodiments, the data amount to be sent may be the data amount that the server needs to send in a predetermined time period before and after the data packet is lost; the data volume to be sent may also be the data volume in the server sending buffer.

Round-trip delay is the total delay experienced from the time the server sends data to the time the server receives an acknowledgement from the receiver. The round trip delay is generally composed of three parts, namely the propagation delay of a link, the processing delay of a receiving end and the queuing and processing delay of a router. For example, the server sends a data packet to the receiver at time T1, the receiver sends a feedback to the server after receiving the data packet, and the server receives the feedback at time T2, so that the round-trip delay is T2-T1.

In some embodiments, the first predetermined time period is 100 milliseconds, the first data amount threshold is 100 bytes, and the sender may reduce the amount of data to be sent to less than 100 bytes within 100 milliseconds, and detect the round trip delay variation within the time period (i.e., within 100 milliseconds) to determine the network property. If the round trip delay is generally normally distributed over time within the time period (the presented shape can be fitted to the normal distribution within a preset error range, for example, the distribution curve has multiple inflection points but the overall trend is normally distributed), or the presented shape generally satisfies that the ordinate corresponding to the abscissa starting point is smaller than the ordinate corresponding to the abscissa intermediate point, the network is identified as a congested network (i.e., the network attribute is congestion). And if the round trip delay is not in normal distribution along with the time variation in the time period and the presented shape generally does not satisfy that the ordinate corresponding to the abscissa starting point is smaller than the ordinate corresponding to the abscissa intermediate point, identifying the network as a non-congestion network (i.e. the network attribute is non-congestion). The above description of the first predetermined time period and the first data amount threshold is only exemplary, and those skilled in the art may adapt the first predetermined time period and the first data amount threshold according to actual needs as long as the principle of the embodiment can be implemented. In other embodiments, the first predetermined time period may be no greater than 1 second and the first data amount threshold may be no greater than 1024 bytes.

In some embodiments, time may be used as an independent variable and round trip delay as a dependent variable, and equation (1) below is a probability density function of a probability distribution of a random variable x obeying a location parameter μ and a scale parameter σ. If the variation of the independent variable and the dependent variable substantially conforms to the above formula (1), or substantially conforms to the rule that the farther x is from μ, the smaller the function operation result, the network can be determined to be a congested network.

In some embodiments, time is taken as the abscissa and round trip delay is taken as the ordinate. The abscissa corresponding to the maximum round-trip delay is the time when the data packet starts to be lost, after the data packet is lost, the server side reduces the data transmission amount, and the round-trip delay is reduced along with the increase of time. If the variation curve of the round trip delay conforms to the curve rule shown in fig. 5, the network can be determined to be a congested network.

If the network is identified as a congested network, the process proceeds to step S205.

If the network is identified as a non-congested network, the process proceeds to step S206.

In step S203, if it is determined that the attribute of the network through which the video data stream is transmitted is known, the process proceeds to step S207. In step S207, it is determined whether the network is a congested network, wherein if the network is a congested network, the process proceeds to step S205; if the network is determined to be a non-congested network, the process proceeds to step S206.

In the method 200, step S205 is the same as or similar to the content of step S106 in the method 100 above; step S206 is the same as or similar to step S107 in the method 100 above. Here, in order to keep the description concise, detailed description of step S205 and step S206 will be omitted.

Corresponding to the application scenario and method of the method provided in the embodiment of the present application, the embodiment of the present application further provides a video processing apparatus 300, and the video processing apparatus 300 of the present application will be described with reference to fig. 3. Fig. 3 is a block diagram showing a configuration of a video processing apparatus 300 according to an embodiment of the present application. As shown in fig. 3, the video processing apparatus 300 may include: a video transmission unit 301 and a processing unit 302.

The video transmitting unit 301 is configured to transmit a video data stream, where the video data stream includes at least one data packet.

The video transmitting unit 301 may be a component in a server, through which the server transmits a video data stream. The server can be a cloud server, a base station and the like.

For the description of the features such as video data streams and data packets, reference may be made to the above description of the video processing method 100, and a detailed description of such features will be omitted here for the sake of simplicity of the description.

A processing unit 302, configured to determine whether a data packet is lost in the video data stream, if so, the processing unit determines whether an attribute of a network used for sending the video data stream is known, and if not, the processing unit reduces a data amount to be sent within a first predetermined time period to be less than a first data amount threshold, detects a change of a round trip delay within the first predetermined time period, identifies the attribute of the network according to the change of the round trip delay, and continuously determines whether the data packet is lost in the video data stream.

The processing unit 302 may be a component in the server, through which the server processes the lost data packet.

In some embodiments, it may be determined whether the data packet sent by the server is correctly received by the receiving end through the feedback information of the receiving end, and if the data packet is correctly received by the receiving end, the server continues to send the video data stream without performing other processing.

In some embodiments, the server determines whether the data packet sent by the server is correctly received by the receiving end by setting a timeout. The server side starts a timer when sending the data packet, and if the timer is overtime and the server side still does not receive the feedback of the receiving end about the data packet, the server side considers that the packet is lost.

In some embodiments, the first predetermined time period is 100 milliseconds, the first data amount threshold is 100 bytes, and the sender may reduce the data amount to be sent to be less than 100 bytes within 100 milliseconds, and detect the round trip delay variation within the time period to determine the network attribute. If the round trip delay fluctuates with time during the time period, for example, the round trip delay is normally distributed with time, the network attribute is congested, otherwise the network attribute is uncongested. The above description of the first predetermined time period, the first data amount threshold, is merely exemplary and may be adapted by those skilled in the art.

In the embodiment described with reference to fig. 3, the above-described first predetermined period of time, first data amount threshold, first predetermined policy, and second predetermined policy are the same as or similar to the features in the above-described embodiment of the video processing method 100, and therefore, in order to make the description concise, a detailed description of the above-described features is omitted here.

The functions of each module in each device in the embodiment of the present application can be referred to the corresponding description in the above method, and have corresponding beneficial effects, which are not described herein again.

FIG. 4 is a block diagram of an electronic device used to implement embodiments of the present application. As shown in fig. 4, the electronic apparatus includes: a memory 401 and a processor 402, the memory 401 having stored therein computer programs executable on the processor 402. The processor 402, when executing the computer program, implements the method in the above embodiments. The number of the memory 401 and the processor 402 may be one or more.

The electronic device further includes:

and a communication interface 403, configured to communicate with an external device and perform data interactive transmission.

If the memory 401, the processor 402 and the communication interface 403 are implemented independently, the memory 401, the processor 402 and the communication interface 403 may be connected to each other through a bus and perform communication with each other. The bus may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, an Extended ISA (EISA) bus, or the like. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is shown in FIG. 4, but this does not indicate only one bus or one type of bus.

Optionally, in a specific implementation, if the memory 401, the processor 402, and the communication interface 403 are integrated on a chip, the memory 401, the processor 402, and the communication interface 403 may complete mutual communication through an internal interface.

Embodiments of the present application provide a computer-readable storage medium storing a computer program, which when executed by a processor implements the method provided in the embodiments of the present application.

The embodiment of the present application further provides a chip, where the chip includes a processor, and is configured to call and run an instruction stored in a memory from the memory, so that a communication device installed with the chip executes the method provided in the embodiment of the present application.

An embodiment of the present application further provides a chip, including: the system comprises an input interface, an output interface, a processor and a memory, wherein the input interface, the output interface, the processor and the memory are connected through an internal connection path, the processor is used for executing codes in the memory, and when the codes are executed, the processor is used for executing the method provided by the embodiment of the application.

It should be understood that the Processor may be a Central Processing Unit (CPU), other general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic, discrete hardware components, etc. A general purpose processor may be a microprocessor or any conventional processor or the like. It is noted that the processor may be a processor supporting Advanced reduced instruction set machine (ARM) architecture.

Further, optionally, the memory may include a read-only memory and a random access memory. The memory may be either volatile memory or nonvolatile memory, or may include both volatile and nonvolatile memory. The non-volatile Memory may include a Read-Only Memory (ROM), a Programmable ROM (PROM), an Erasable PROM (EPROM), an Electrically Erasable PROM (EEPROM), or a flash Memory. Volatile Memory can include Random Access Memory (RAM), which acts as external cache Memory. By way of example, and not limitation, many forms of RAM may be used. For example, static Random Access Memory (Static RAM, SRAM), dynamic Random Access Memory (DRAM), synchronous Dynamic Random Access Memory (SDRAM), double Data Rate Synchronous Dynamic Random Access Memory (DDR SDRAM), enhanced SDRAM (ESDRAM), SLDRAM (SLDRAM), and Direct Rambus RAM (DR RAM).

In the above embodiments, the implementation may be wholly or partially realized by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. The procedures or functions according to the present application are generated in whole or in part when the computer program instructions are loaded and executed on a computer. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium.

In the description herein, references to the description of the term "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the application. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, various embodiments or examples and features of different embodiments or examples described in this specification can be combined and combined by one skilled in the art without contradiction.

Furthermore, the terms "first", "second" and "first" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the feature. In the description of the present application, "a plurality" means two or more unless specifically limited otherwise.

Any process or method described in a flow diagram or otherwise herein may be understood as representing a module, segment, or portion of code, which includes one or more executable instructions for implementing specific logical functions or steps of the process. And the scope of the preferred embodiments of the present application includes other implementations in which functions may be performed out of the order shown or discussed, including substantially concurrently or in reverse order, depending on the functionality involved.

The logic and/or steps described in the flowcharts or otherwise described herein, such as an ordered listing of executable instructions that can be considered to implement logical functions, can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions.

It should be understood that portions of the present application may be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, the various steps or methods may be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. All or part of the steps of the method of the above embodiments may be implemented by hardware that is configured to be instructed to perform the relevant steps by a program, which may be stored in a computer-readable storage medium, and which, when executed, includes one or a combination of the steps of the method embodiments.

In addition, functional units in the embodiments of the present application may be integrated into one processing module, or each unit may exist alone physically, or two or more units are integrated into one module. The integrated module can be realized in a hardware mode, and can also be realized in a software functional module mode. The above-described integrated module, if implemented in the form of a software functional module and sold or used as a separate product, may also be stored in a computer-readable storage medium. The storage medium may be a read-only memory, a magnetic or optical disk, or the like.

The above description is only an exemplary embodiment of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily conceive various changes or substitutions within the technical scope described in the present application, and these should be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A video processing method, comprising:

sending a video data stream, wherein the video data stream comprises at least one data packet;

determining whether a packet is lost in the video data stream, wherein if a packet is lost, determining whether an attribute of a network used to transmit the video data stream is known, wherein,

if the attribute of the network is unknown, reducing the data volume to be sent in a first preset time period to be less than a first data volume threshold value, detecting the change of the round trip delay in the first preset time period, identifying the attribute of the network according to the change of the round trip delay, and continuously judging whether a data packet is lost in the video data stream.

2. The video processing method according to claim 1,

if the attributes of the network are known, determining if the network is a congested network, wherein,

processing the lost data packet according to a first predetermined policy if the network is a congested network; and

if the network is not a congested network, the lost data packets are processed according to a second predetermined policy.

3. The video processing method according to claim 2, wherein the first predetermined policy is to reduce the amount of data to be transmitted in a second predetermined time period to less than a second data amount threshold.

4. The video processing method of claim 3, further comprising:

detecting a change in round trip delay over the second predetermined period of time at predetermined time intervals;

calculating a difference between a round trip delay over the second predetermined period of time and a round trip delay threshold, wherein,

if the difference is within a predetermined range, increasing the amount of data to be transmitted within a third predetermined time period to be greater than a third data amount threshold.

5. The video processing method according to claim 2, wherein the second predetermined policy is to retransmit the lost data packet.

6. The video processing method according to claim 2, wherein the second predetermined policy is to forward error correct the lost data packets.

7. A video processing method, comprising:

determining whether a data packet is lost in the video data stream, wherein if a data packet is lost, determining whether an attribute of a network used for transmitting the video data stream is known, wherein,

if the attribute of the network is unknown, reducing the data volume to be sent in a first preset time period to be less than a first data volume threshold value, detecting the change of round trip delay in the first preset time period, identifying the attribute of the network according to the change of the round trip delay, and processing the lost data packet according to a preset strategy, wherein,

8. The video processing method according to claim 7,

9. The video processing method according to claim 7,

the first preset strategy is to reduce the data volume to be sent in a second preset time period to be less than a second data volume threshold value;

calculating a difference between a round trip delay and a round trip delay threshold for the second predetermined time period, wherein,

10. The video processing method according to claim 7, wherein the second predetermined policy is to retransmit the lost data packet.

11. The video processing method according to claim 7, wherein the second predetermined policy is to forward error correct the lost data packets.

12. An apparatus for video processing, comprising:

the video transmitting unit is configured to transmit a video data stream, and the video data stream comprises at least one data packet;

a processing unit configured to determine whether a packet is lost in the video data stream, wherein, if a packet is lost, the processing unit determines whether an attribute of a network used to transmit the video data stream is known, wherein,

if the attribute of the network is unknown, the processing unit reduces the data volume to be sent in a first preset time period to be less than a first data volume threshold value, detects the change of the round trip delay in the first preset time period, identifies the attribute of the network according to the change of the round trip delay, and continuously judges whether a data packet is lost in the video data stream.

13. An electronic device comprising a memory, a processor and a computer program stored on the memory, the processor implementing the method of any one of claims 1-11 when executing the computer program.

14. A computer-readable storage medium, having stored therein a computer program which, when executed by a processor, implements the method of any of claims 1-11.