CN118055260A

CN118055260A - Video processing method, device, equipment and computer storage medium

Info

Publication number: CN118055260A
Application number: CN202211430328.8A
Authority: CN
Inventors: 肖萃耀
Original assignee: Guangzhou Shiyuan Electronics Thecnology Co Ltd; Guangzhou Shirui Electronics Co Ltd
Current assignee: Guangzhou Shiyuan Electronics Thecnology Co Ltd; Guangzhou Shirui Electronics Co Ltd
Priority date: 2022-11-15
Filing date: 2022-11-15
Publication date: 2024-05-17

Abstract

The embodiment of the invention provides a video processing method, a device, equipment and a computer storage medium, wherein the method comprises the following steps: obtaining at least two queue length values of a video data cache queue based on a preset time interval, confirming a network delay state based on length relation information between the at least two queue length values, adjusting an initial code rate according to the network delay state to obtain a target code rate, encoding second video source data by using the target code rate to obtain target video encoding data corresponding to the target code rate, and outputting the target video encoding data. The network delay state is confirmed through the detection of the length of the video data cache queue, and the video data quantity is adjusted in real time according to the network delay state, so that the video playing stability is ensured, and the user watching experience is improved.

Description

Video processing method, device, equipment and computer storage medium

Technical Field

The present invention relates to the field of computer technologies, and in particular, to a video processing method, apparatus, device, and computer storage medium.

Background

With the rapid development of internet technology, video live services are increasingly widely used. Video live broadcast refers to live broadcast by using the internet and streaming media technology.

However, since the instability of the network signal may affect the uploading and downloading rates of the streaming media, especially in the mobile network environment, the playing effect is not ideal enough, and the user's look and feel is often poor due to the fact that the video is jammed and a screen is jumped, so a video processing method capable of improving the smoothness of video playing is needed to be proposed.

Disclosure of Invention

The main objective of the present disclosure is to provide a video processing method, apparatus, device and computer storage medium, which aim to ensure smoothness of video playing under the condition of network jitter. The technical scheme is as follows:

in a first aspect, an embodiment of the present invention provides a video processing method, including:

Acquiring at least two queue length values of a video data cache queue based on a preset time interval, wherein initial video coding data obtained by coding first video source data based on an initial code rate are stored in the video data cache queue;

Confirming a network delay state based on length relation information between at least two queue length values;

adjusting the initial code rate based on the network delay state to obtain a target code rate;

And encoding the second video source data by adopting the target code rate to obtain target video encoding data corresponding to the target code rate, and outputting the target video encoding data, wherein the second video source data is video source data obtained after the first video source data is obtained.

In a second aspect, an embodiment of the present invention provides a video processing apparatus, including:

The acquisition module is used for acquiring at least two queue length values of a video data cache queue based on a preset time interval, wherein initial video coding data obtained by coding first video source data based on an initial code rate is stored in the video data cache queue;

The network delay confirming module is used for confirming the network delay state based on the length relation information between at least two queue length values;

the code rate adjusting module is used for adjusting the initial code rate based on the network delay state so as to obtain a target code rate;

The encoding module is used for encoding the second video source data by adopting the target code rate, obtaining target video encoding data corresponding to the target code rate, and outputting the target video encoding data, wherein the second video source data is obtained after the first video source data is obtained.

In a third aspect, an embodiment of the present invention provides an electronic device, including: a memory, a processor and a computer program stored on the memory and executable on the processor, which when executed by the processor, performs the steps of the method as described above.

In a fourth aspect, embodiments of the present invention provide a computer storage medium having a computer program stored thereon, which when executed by a processor, implements the steps of the method as described above.

In the embodiment of the invention, at least two queue length values of a video data cache queue are obtained based on a preset time interval, a network delay state is confirmed based on length relation information between the at least two queue length values, then an initial code rate is adjusted according to the network delay state to obtain a target code rate, second video source data is encoded by adopting the target code rate, target video encoding data corresponding to the target code rate is obtained, and the target video encoding data is output. The network delay state is confirmed through the detection of the length of the video data cache queue, and the video data quantity is adjusted in real time according to the network delay state, so that the video playing stability is ensured, and the user watching experience is improved.

Drawings

In order to more clearly illustrate the embodiments of the invention or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present description, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

Fig. 1 is an exemplary schematic diagram of a video processing method according to an embodiment of the present invention;

fig. 2 is a schematic flow chart of a video processing method according to an embodiment of the present invention;

fig. 3 is a schematic diagram of a live broadcast process of a video processing method according to an embodiment of the present invention;

Fig. 4 is a schematic flow chart of a video processing method according to an embodiment of the present invention;

fig. 5 is a schematic diagram of an automatic packet loss flow of a video processing method according to an embodiment of the present invention;

Fig. 6 is a schematic diagram of a code rate adaptive flow chart of a video processing method according to an embodiment of the present invention;

Fig. 7 is a schematic structural diagram of a video processing apparatus according to an embodiment of the present invention;

fig. 8 is a schematic structural diagram of a video processing apparatus according to an embodiment of the present invention.

Detailed Description

The following description of the embodiments of the present invention will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are only some, but not all, of the embodiments of the present invention. All other embodiments, which can be made by one of ordinary skill in the art without undue burden from the present disclosure, are intended to be within the scope of the present disclosure.

The video processing device can be a terminal device such as a mobile phone, a computer, a tablet personal computer, intelligent home equipment, wearable equipment or vehicle-mounted equipment, and also can be a module for realizing a video processing method in the terminal device, the video processing device can acquire at least two queue length values of a video data cache queue based on a preset time interval, and initial video coding data obtained by coding first video source data based on an initial code rate is stored in the video data cache queue; confirming a network delay state based on length relation information between at least two queue length values; adjusting the initial code rate based on a network delay state to obtain a target code rate; and encoding the second video source data by adopting a target code rate to obtain target video encoding data corresponding to the target code rate, and outputting the target video encoding data, wherein the second video source data is video source data acquired after the first video source data is acquired.

Referring to fig. 1, an exemplary schematic diagram of a video processing method is provided for an embodiment of the present invention, and in particular, a hosting client is installed in a hosting device. The anchor client device can be a device with functions of acquisition, coding and data transmission, such as a computer and an intelligent camera device. Correspondingly, the audience client is installed in the audience device, and it should be noted that the anchor device may be the audience device at the same time, and the functions of the anchor client and the audience client may be integrated on the same client. The method comprises the steps that a main broadcasting end collects first video source data, initial video coding data are obtained by coding the first video source data based on an initial code rate, the initial video coding data are stored in a video data cache queue, then the initial video coding data in the video data cache queue are uploaded to a cloud end, and after the audience end is connected with the network, the initial video coding data are downloaded from the cloud end for viewing. Furthermore, the anchor end can also obtain at least two queue length values of the video data cache queue based on a preset time interval, confirm a network delay state based on length relation information between the at least two queue length values, adjust an initial code rate according to the network delay state to obtain a target code rate, encode second video source data acquired after the first video source data by adopting the target code rate to obtain target video encoded data, store the target video encoded data into the video data cache queue and upload the target video encoded data to the cloud.

It should be noted that, the cloud end is used as a data storage, and may be replaced by another type of server.

The video processing method provided in the present specification is described in detail below with reference to specific embodiments.

Referring to fig. 2, a flowchart of a video processing method is provided in an embodiment of the present invention. As shown in fig. 2, the method of the embodiment of the present invention may include the following steps S102 to S106.

S101, acquiring at least two queue length values of a video data cache queue based on a preset time interval, wherein initial video coding data obtained by coding first video source data based on an initial code rate is stored in the video data cache queue;

s102, confirming a network delay state based on length relation information between at least two queue length values;

s103, adjusting the initial code rate based on the network delay state to obtain a target code rate;

S104, encoding the second video source data by adopting the target code rate to obtain target video encoding data corresponding to the target code rate, and outputting the target video encoding data, wherein the second video source data is obtained after the first video source data is obtained.

The video processing method of the embodiment is used for video live broadcast services, such as instant live broadcast services and video on demand services. It can be appreciated that a live video service generally includes a video providing end, a server, and a video playing end, where the video providing end pushes a video to be watched by a user to the server, and the video playing end pulls a video to be played from the server. In the process of push and pull, data transmission is usually performed based on a network, and when the network shakes, the smoothness of video viewing is affected. Therefore, referring to fig. 3, fig. 3 is a schematic diagram of a live broadcast process of a video processing method according to an embodiment of the present invention, in which smoothness of video is generally ensured by adopting a delay manner, that is, video encoding is performed on video data to be played after video acquisition processing to obtain video encoded data, then the video encoded data is stored in a video data buffer queue to buffer data, and then the encoded data is sent to a server for user to pull and view. However, because the network signals are different in quality, the live time delay of the existing scheme is always increased, for example, the time delay of the original live broadcast for half a minute is 3 seconds, and the time delay is increased to 10 seconds, so that the situation can reduce the watching experience of video watching users for application scenes of online education, sports live broadcast, news live broadcast and the like which need to ensure real-time performance. Therefore, the video processing method aims at ensuring low delay, improving the fluency of live broadcast and improving user experience.

The following will explain each step in detail:

specifically, a preset time interval is set, and the queue length values of the video data cache queues are obtained, wherein the number of the queue length values is at least 2. It can be understood that the encoding is to store data (production data) into the video data buffer queue all the time, and meanwhile, the video data buffer queue always sends the encoded initial video encoded data in the queue to the cloud (consumption data), if the network is bad, the speed of sending the initial video encoded data with the same size from the video data buffer queue to the playing end is slow, and meanwhile, the data backlog in the video data buffer queue is caused, so that the video playing delay is increased. Therefore, based on the preset time interval, the queue length value of the video data cache queue is obtained, and the queue length value is used for representing the length of the video data cache queue, namely the size of the data volume which is not uploaded to the cloud.

Illustratively, the time span of the packets in the buffer queue is calculated as the queue length value by obtaining the time of the storing of each packet (video encoded data) in the video data buffer queue. Usually, the first video source data is collected and encoded according to seconds, and correspondingly, the preset time interval can be obtained for once per second to obtain the queue length value of the buffer queue, and the specific preset time interval can be set according to actual conditions. The video source data collected at present are called first video source data, and the first video source data are coded by adopting an initial code rate.

It will be appreciated that if the video source data is encoded at the same rate, the amount of data stored in the video data buffer queue per second is nearly uniform, and if the rate at which data is consumed in the buffer queue is slowed, the queue length value is increased. Therefore, the network delay state is judged by acquiring the queue length value of the video data cache queue. The length relation information may be an average value of the queue length values, a sum of the queue length values, a maximum value of the queue length values, and the like.

After at least two queue length values are obtained, namely when the number of the obtained queue length values reaches a preset statistical number, confirming length relation information among the queue length values. The smaller the statistics times, the faster the adjustment frequency of the code rate is, the specific number of the needed queue length values can be determined based on the application scene, for example, a teaching scene, and as the interaction requirement exists between a teacher and students, for example, the teacher asks the students to answer, the expected frequency can be adjusted quickly, so that the occurrence of live broadcast delay is reduced to the greatest extent, the smoothness of communication between the students and the teacher is ensured, and the teaching effect is further ensured.

In a possible implementation manner, a historical reference value of the queue length value corresponding to the current terminal may be obtained, for example, an average value of the currently calculated queue length values is compared with the historical reference average value, if the average value is greater than the historical reference value, the network is considered to have delay, and if the average value is less than the historical reference value, the network is considered to have good condition.

In yet another possible implementation, the network delay determination is performed based on the preset threshold and the length relationship information by setting a preset threshold. For example, if the maximum value in the queue length values is greater than the preset maximum value, the network is judged to have time delay, and if the minimum value in the queue length values is less than the preset minimum value, the network is judged to be good.

Specifically, the initial code rate is adjusted according to the network delay state, so that the target code rate is obtained. The initial code rate is the current code rate used by the encoder, the target code rate is because the acquired video source data is usually encoded to compress the video size for convenience in transmission, and the uploading speed is low and can not be changed when the network shakes, so that the data volume is required to be reduced, and the uploading time is shortened. Thus, the size of video source data acquired at the same time is adjusted by adjusting the code rate, which is the amount of data generated within 1 second of the encoder. If the network delay state is that the network delay is serious, the code rate is reduced, and if the network delay state is that the network is good, the code rate is properly adjusted to be higher, and the code rate can be specifically set according to the actual situation.

Specifically, after the target code rate is adjusted, encoding the second video source data acquired next to obtain target video encoding data, storing the target video encoding data into a video data buffer queue, and uploading the target video encoding data to a cloud for streaming at a playing end.

It can be understood that the above determination of the network delay condition can be repeatedly performed, so as to realize real-time monitoring of the network delay condition and timely adjustment of the code rate.

In this embodiment, at least two queue length values of a video data buffer queue are obtained based on a preset time interval, a network delay state is confirmed based on length relation information between the at least two queue length values, an initial code rate is adjusted according to the network delay state to obtain a target code rate, second video source data is encoded by adopting the target code rate, target video encoded data corresponding to the target code rate is obtained, and the target video encoded data is output. The network delay state is confirmed through the detection of the length of the video data cache queue, and the video data quantity is adjusted in real time according to the network delay state, so that the video playing stability is ensured, and the user watching experience is improved.

Referring to fig. 4, a flowchart of a video processing method is provided in an embodiment of the present invention. As shown in fig. 4, the method of the embodiment of the present invention may include the following steps S201 to S206.

S201, confirming whether the buffer capacity of the video data buffer queue is full;

It can be appreciated that since the capacity of the video data buffer queue is limited, new data cannot be stored if it is full, and thus the buffer queue is required to have an automatic packet loss function. Specifically, whether the buffer capacity of the video data buffer queue is full is confirmed, and if the buffer capacity is full, automatic packet loss is required.

S202, if yes, deleting the output cached video coding data in the video data cache queue;

And if the cache capacity of the video data cache queue is full, confirming the video coding data uploaded to the cloud in the video data cache queue and deleting the video coding data so as to store the newly acquired video source data. Referring to fig. 5, fig. 5 is a schematic diagram of an automatic packet loss flow of a video processing method according to an embodiment of the present invention, when encoded data is input into a video data buffer queue, whether a buffer (buffer capacity) is full is determined, and if yes, old data, that is, output buffered video encoded data is discarded; if not, the video coding data is built into a buffer queue. It should be noted that, the step of determining whether the buffer capacity is full may be performed at any time, or may be performed continuously, or may be performed at intervals of a preset time, by limiting the size of the buffer queue, when the buffer queue is full, old data packets are discarded, so that delay is avoided from becoming larger all the time.

S203, acquiring a time mark of the initial video coding data in the video data cache queue at intervals of preset time intervals;

In a possible implementation manner, time identifiers of initial video coding data in a video data buffer queue are obtained at intervals of preset time, wherein the time identifiers are times when the initial video coding data are coded and enter the buffer queue.

S204, calculating the time difference of the initial video coding data based on the time mark, and confirming the time difference as a queue length value of the video data cache queue;

because the first video source data is acquired in time sequence and coded and stored in the video data buffer queue, the time mark of the last packet is subtracted from the time mark of the first packet in the video data buffer queue, and the obtained time difference is used as a queue length value, and the unit is preferably millisecond.

S205, obtaining an average value of the queue length values and a difference value between a maximum value and a minimum value of at least two queue length values;

The method comprises the steps of calculating the queue length values of the video data cache queues for a plurality of times, and calculating the average value of the queue length values and the difference between the maximum value and the minimum value of the queue length values based on the queue length values. Such as calculating 1 queue length value per second, calculating 5 times in succession, obtaining 5 queue length values, calculating an average of the 5 queue length values, and the difference between the maximum and minimum values.

S206, taking the average value and/or the difference value as the length relation information, and confirming the network delay state based on the length relation information.

It can be understood that, since the amount of data generated by the encoder may change every second, there is a certain fluctuation, the data is smoothed by increasing the statistics number, that is, calculating the queue length value multiple times, so as to obtain a more accurate network condition judgment result. For example, the average queue length value is large within 5 seconds, which indicates that most of the 5 seconds are data backlog; for another example, if the difference between the maximum value and the minimum value of the queue length is large within 5 seconds, it indicates that the network fluctuates and the network is unstable within 5 seconds, and the network delay state is determined according to the maximum value in consideration of the network condition of the maximum value.

Further, based on the step S206, the determining the network delay state based on the length relation information, and adjusting the initial code rate based on the network delay state to obtain a target code rate, includes:

S301, if the difference value is larger than a first difference value threshold value, the network delay state is a first network delay state, and the initial code rate is reduced to a first optimized code rate based on a first reduction coefficient to serve as the target code rate;

s302, if the difference value is larger than a second difference value threshold and smaller than a first difference value threshold, or the average value is larger than a first average value threshold, the network delay state is a second network delay state, and the initial code rate is reduced to a second optimized code rate based on a second reduction coefficient to serve as the target code rate;

And S303, if the difference value is smaller than a third difference value threshold value and the average value is smaller than a second average value threshold value, the network delay state is a third network delay state, and the initial code rate value is improved to obtain a third optimized code rate based on a first expansion coefficient as the target code rate.

Specifically, the network delay condition is determined through the difference value and/or the average value. Illustratively, the first difference threshold is 2500kbps (kilobits per second), the second difference threshold is 1250kbps, the third difference threshold is 100kbps, the first average threshold is 4000kbps, and the second average threshold is 500 kbps.

If the difference is greater than 2500, the network is considered to be very bad, the first reduction coefficient is set to 0.85, the code rate is greatly reduced, and the target code rate=the initial code rate is 0.85. The lower the code rate is, the smaller the data quantity is, and the less clear the image quality is, so the minimum code rate is set, the code rate cannot be lower than the minimum code rate when the code rate is reduced, and the influence on watching caused by the too fuzzy image quality is avoided.

If 1250< difference <2500 or average >4000, the network is considered worse, the second reduction coefficient is set to 0.9, the code rate is reduced by a small margin, and the target code rate=initial code rate is 0.9.

If the difference is <100 and the average is <500, the network is considered to be good, the first expansion coefficient is set to 1.08, the code rate is increased by a small margin, and the target code rate=the initial code rate is 1.08.

It should be noted that, the first difference threshold, the second difference threshold, the first average threshold and the second average threshold are not limited, and may be set according to different application scenarios and network conditions.

Further, in an embodiment, the reducing the initial code rate to the first optimized code rate based on the first reduction coefficient includes:

S304, reducing the initial code rate to a first optimized code rate based on a first reduction coefficient, and selecting the largest code rate of the first optimized code rate and a preset minimum code rate as the target code rate;

It can be understood that the video code rate is related to the definition of the video, if the code rate is too low, the video watched by the user is too fuzzy, so that the preset minimum code rate is set, and when the code rate is reduced, the code rate cannot be lower than the preset minimum code rate, so that the influence on watching caused by too fuzzy image quality is avoided. And confirming the first optimized code rate, and taking the preset minimum code rate as a target code rate if the first optimized code rate is smaller than the preset minimum code rate. If the first optimized code rate is greater than the preset minimum code rate, the code rate is not reduced to the preset minimum code rate any more when the smooth playing of the video can be ensured only by adjusting the code rate to the first optimized code rate, so that the smooth playing of the video can be ensured at the same time under the condition of ensuring the video quality, and the target code rate=max (current code rate 0.85, minimum code rate) is continued to use the above example

The step of reducing the initial code rate to a second optimized code rate based on a second reduction coefficient, as the target code rate, includes:

S305, reducing the initial code rate to a second optimized code rate based on a second reduction coefficient, and selecting the largest code rate of the second optimized code rate and a preset minimum code rate as the target code rate;

And similarly, when the initial code rate is reduced based on the second reduction coefficient, comparing the second optimized code rate with a preset minimum code rate, and selecting the maximum code rate as a target code rate. Continuing the above example, the target code rate=max (current code rate 0.9, minimum code rate).

The step of improving the initial code rate value based on the first expansion coefficient to obtain a third optimized code rate, as the target code rate, includes:

S306, improving the third optimized code rate of the initial code rate value based on the first expansion coefficient, and selecting the smallest code rate of the third optimized code rate and a preset maximum code rate as the target code rate.

It can be understood that when the initial code rate is increased based on the first expansion coefficient, the occupation condition of the code rate to the bandwidth is considered, and it is generally considered that the third optimized code rate image quality obtained based on the expansion of the first expansion coefficient completely meets the use requirement, and the increase of the code rate is unnecessary, but the bandwidth is wasted, so that the maximum code rate value is set, and the continuous increase of the code rate is avoided. Continuing the above example, the target code rate=min (current code rate 1.08, maximum code rate).

Referring to fig. 6, fig. 6 is a schematic diagram of a code rate adaptive flow chart of a video processing method according to an embodiment of the present invention, in which the number of queue length values is five, a preset interval time is 1 second, a queue length value of a video data buffer queue is obtained once per second, after five times of obtaining, a difference value between an average value and a maximum value of the queue length values and a minimum value is calculated, and based on the average value and the difference value, network delay condition determination is performed, and meanwhile, an initial code rate is adjusted to obtain a target code rate.

In this embodiment, by acquiring the buffer capacity of the video data buffer queue, automatic packet loss is performed when the buffer capacity is full, thereby preventing delay caused by the buffer queue being full. And obtaining time marks of initial video coding data in a video data cache queue at intervals of preset time intervals, calculating a time difference based on the time marks, obtaining a queue length value of the video data cache queue based on the time difference, obtaining length relation information by calculating a difference value between an average value and a maximum value and a minimum value of at least two queue length values, confirming a network delay state based on the length relation information and a first difference value threshold, a second difference value threshold, a first average value threshold and a second average value threshold, and adjusting initial code rates based on corresponding first reduction coefficients, second reduction coefficients and first expansion coefficients under different network delay states to obtain a target code rate, thereby ensuring smoothness of watching videos of users under various network delay states. Further, a preset maximum code rate and a preset minimum code rate are set, video quality is considered and occupied bandwidth size is limited on the premise of ensuring fluency, and therefore user experience is improved.

The following describes in detail a video processing apparatus according to an embodiment of the present invention with reference to fig. 7. It should be noted that, the video processing apparatus in fig. 7 is used to execute the method of the embodiment shown in fig. 1 to 6 of the present specification, and for convenience of explanation, only the portion relevant to the embodiment of the present invention is shown, and specific technical details are not disclosed, please refer to the embodiment shown in fig. 1 to 6 of the present specification.

Referring to fig. 7, a schematic diagram of a video processing apparatus according to an exemplary embodiment of the present disclosure is shown. The video processing device may be implemented as all or part of the device by software, hardware, or a combination of both. The device 1 comprises an acquisition module 11, a network delay confirmation module 12, a code rate adjustment module 13 and a coding module 14.

The acquiring module 11 is configured to acquire at least two queue length values of a video data buffer queue based on a preset time interval, where initial video encoded data obtained by encoding first video source data based on an initial code rate is stored in the video data buffer queue;

a network delay confirmation module 12, configured to confirm a network delay state based on length relation information between at least two queue length values;

The code rate adjusting module 13 is configured to adjust the initial code rate based on the network delay state to obtain a target code rate;

The encoding module 14 is configured to encode the second video source data with the target code rate, obtain target video encoded data corresponding to the target code rate, and output the target video encoded data, where the second video source data is video source data obtained after the first video source data is obtained.

Optionally, the acquiring module 11 is specifically configured to acquire the time identifier of the initial video coding data in the video data buffer queue at intervals of a preset time interval;

and calculating the time difference of the initial video coding data based on the time mark, and confirming the time difference as a queue length value of the video data cache queue.

Optionally, the obtaining module 11 is further configured to confirm whether the buffer capacity of the video data buffer queue is full;

If yes, deleting the output cached video coding data in the video data cache queue. Optionally, the network delay confirmation module 12 is specifically configured to obtain an average value of the queue length values, and a difference value between a maximum value and a minimum value of at least two queue length values;

and taking the average value and/or the difference value as the length relation information, and confirming the network delay state based on the length relation information.

Optionally, the network delay confirmation module 12 is specifically configured to, if the difference is greater than a first difference threshold, determine that the network delay state is a first network delay state;

If the difference is greater than a second difference threshold and less than a first difference threshold, or the average is greater than a first average threshold, the network delay state is a second network delay state;

And if the difference value is smaller than a third difference value threshold value and the average value is smaller than a second average value threshold value, the network delay state is a third network delay state.

Optionally, the code rate adjustment module 13 is specifically configured to reduce, if the network delay state is a first network delay state, the initial code rate to a first optimized code rate based on a first reduction coefficient, and use the initial code rate as the target code rate;

If the network delay state is a second network delay state, the initial code rate is reduced to a second optimized code rate based on a second reduction coefficient, and the second optimized code rate is used as the target code rate;

and if the network delay state is a third network delay state, improving the third optimized code rate of the initial code rate value based on the first expansion coefficient, and taking the third optimized code rate as the target code rate.

Optionally, the code rate adjustment module 13 is specifically configured to reduce the initial code rate to a first optimized code rate based on a first reduction coefficient, and select a maximum code rate of the first optimized code rate and a preset minimum code rate as the target code rate;

Based on a second reduction coefficient, reducing the initial code rate to a second optimized code rate, and selecting the maximum code rate of the second optimized code rate and a preset minimum code rate as the target code rate;

And based on the first expansion coefficient, improving the third optimized code rate of the initial code rate value, and selecting the minimum code rate of the third optimized code rate and a preset maximum code rate as the target code rate.

It should be noted that, in the video processing apparatus provided in the foregoing embodiment, when the video processing method is executed, only the division of the foregoing functional modules is used as an example, in practical application, the foregoing functional allocation may be performed by different functional modules, that is, the internal structure of the device is divided into different functional modules, so as to complete all or part of the functions described above. In addition, the video processing apparatus and the video processing method embodiment provided in the foregoing embodiments belong to the same concept, which represents a detailed implementation process in the method embodiment, and are not described herein again.

The foregoing embodiment numbers of the present invention are merely for the purpose of description, and do not represent the advantages or disadvantages of the embodiments. In some cases, the actions or steps recited in the claims can be performed in a different order than in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

The embodiment of the present invention further provides a computer storage medium, on which a computer program is stored, where the computer program when executed by a processor implements the video processing method according to the embodiment shown in fig. 1 to fig. 6, and the specific execution process may refer to the specific description of the embodiment shown in fig. 1 to fig. 6, which is not repeated herein.

Referring to fig. 8, a schematic diagram of a video processing apparatus according to an exemplary embodiment of the present disclosure is shown. The video processing device in this specification may include one or more of the following: processor 110, memory 120, input device 130, output device 140, and bus 150. The processor 110, the memory 120, the input device 130, and the output device 140 may be connected by a bus 150.

Processor 110 may include one or more processing cores. The processor 110 connects various parts within the overall video processing device using various interfaces and lines, performs various functions of the terminal 100 and processes data by executing or executing instructions, programs, code sets, or instruction sets stored in the memory 120, and invoking data stored in the memory 120. Alternatively, the processor 110 may be implemented in at least one hardware form of digital signal Processing (DIGITAL SIGNAL Processing, DSP), field-Programmable gate array (Field-Programmable GATE ARRAY, FPGA), programmable logic array (Programmable Logic Array, PLA). The processor 110 may integrate one or a combination of several of a central processing unit (Central Processing Unit, CPU), an image processor (Graphics Processing Unit, GPU), a modem, etc. The CPU mainly processes an operating system, a user page, an application program and the like; the GPU is used for being responsible for rendering and drawing of display content; the modem is used to handle wireless communications. It will be appreciated that the modem may not be integrated into the processor 110 and may be implemented solely by a single communication chip.

The Memory 120 may include a random access Memory (Random Access Memory, RAM) or a Read-Only Memory (ROM). Optionally, the memory 120 includes a Non-transitory computer readable medium (Non-Transitory Computer-Readable Storage Medium). Memory 120 may be used to store instructions, programs, code, sets of codes, or sets of instructions. The memory 120 may include a stored program area and a stored data area, wherein the stored program area may store instructions for implementing an operating system, which may be an Android (Android) system, including an Android system-based deep development system, an IOS system developed by apple corporation, including an IOS system-based deep development system, or other systems, instructions for implementing at least one function (such as a touch function, a sound playing function, an image playing function, etc.), instructions for implementing the various method embodiments described above, and the like.

Memory 120 may be divided into an operating system space in which the operating system runs and a user space in which native and third party applications run. In order to ensure that different third party application programs can achieve better operation effects, the operating system allocates corresponding system resources for the different third party application programs. However, the requirements of different application scenarios in the same third party application program on system resources are different, for example, under the local resource loading scenario, the third party application program has higher requirement on the disk reading speed; in the animation rendering scene, the third party application program has higher requirements on the GPU performance. The operating system and the third party application program are mutually independent, and the operating system often cannot timely sense the current application scene of the third party application program, so that the operating system cannot perform targeted system resource adaptation according to the specific application scene of the third party application program.

In order to enable the operating system to distinguish specific application scenes of the third-party application program, data communication between the third-party application program and the operating system needs to be communicated, so that the operating system can acquire current scene information of the third-party application program at any time, and targeted system resource adaptation is performed based on the current scene.

The input device 130 is configured to receive input instructions or data, and the input device 130 includes, but is not limited to, a keyboard, a mouse, a camera, a microphone, or a touch device. The output device 140 is used to output instructions or data, and the output device 140 includes, but is not limited to, a display device, a speaker, and the like. In one example, the input device 130 and the output device 140 may be combined, and the input device 130 and the output device 140 are touch display screens.

The touch display screen may be designed as a full screen, a curved screen, or a contoured screen. The touch display screen may also be designed as a combination of a full screen and a curved screen, and the combination of a special-shaped screen and a curved screen, which is not limited in the embodiment of the present invention.

In addition, it will be appreciated by those skilled in the art that the configuration of the video processing apparatus shown in the above-described drawings does not constitute a limitation on the video processing apparatus, and the video processing apparatus may include more or less components than illustrated, or may combine some components, or may be arranged in different components. For example, the video processing device further includes components such as a radio frequency circuit, an input unit, a sensor, an audio circuit, a WiFi module, a power supply, and a bluetooth module, which are not described herein.

In the video processing device shown in fig. 8, the processor 110 may be configured to call a video processing application program stored in the memory 120, and specifically perform the following operations:

In one embodiment, the processor 110, when executing the obtaining at least two queue length values of the video data buffer queue based on the preset time interval, specifically performs the following operations:

Acquiring a time identifier of the initial video coding data in the video data cache queue at intervals of preset time intervals;

In one embodiment, before executing the acquiring the at least two queue length values of the video data buffer queue within the preset time interval, the processor 110 further executes the following operations:

Confirming whether the buffer capacity of the video data buffer queue is full;

if yes, deleting the output cached video coding data in the video data cache queue.

In one embodiment, the processor 110, when executing the length relation information between at least two queue length values, specifically performs the following operations when confirming a network delay state:

Obtaining an average value of the queue length values and a difference value between a maximum value and a minimum value of at least two queue length values;

In one embodiment, the processor 110, when executing the confirmation of the network delay state based on the length relation information, specifically performs the following operations:

if the difference is greater than a first difference threshold, the network delay state is a first network delay state;

In one embodiment, when the processor 110 performs the adjustment of the initial code rate based on the network delay state to obtain a target code rate, the following operations are specifically performed:

if the network delay state is a first network delay state, the initial code rate is reduced to a first optimized code rate based on a first reduction coefficient, and the initial code rate is used as the target code rate;

In one embodiment, when the processor 110 performs the reducing the initial code rate to the first optimized code rate based on the first reduction coefficient, the following operations are specifically performed as the target code rate:

based on a first reduction coefficient, reducing the initial code rate to a first optimized code rate, and selecting the maximum code rate of the first optimized code rate and a preset minimum code rate as the target code rate;

in one embodiment, when the processor 110 performs the step of reducing the initial code rate to a second optimal code rate based on the second reduction coefficient, the following operations are specifically performed:

In one embodiment, when executing the third optimized code rate for increasing the initial code rate value based on the first expansion coefficient, the processor 110 specifically executes the following operations as the target code rate:

In the embodiment of the invention, at least two queue length values of a video data cache queue are obtained based on a preset time interval, a network delay state is confirmed based on length relation information between the at least two queue length values, then an initial code rate is adjusted according to the network delay state to obtain a target code rate, second video source data is encoded by adopting the target code rate, target video encoding data corresponding to the target code rate is obtained, and the target video encoding data is output. The network delay state is confirmed through the detection of the length of the video data cache queue, and the video data quantity is adjusted in real time according to the network delay state, so that the video playing stability is ensured, and the user watching experience is improved. By acquiring the buffer capacity of the video data buffer queue, automatic packet loss is performed when the buffer capacity is full, so that delay caused by the full buffer queue is prevented. And obtaining time marks of initial video coding data in a video data cache queue at intervals of preset time intervals, calculating a time difference based on the time marks, obtaining a queue length value of the video data cache queue based on the time difference, obtaining length relation information by calculating a difference value between an average value and a maximum value and a minimum value of at least two queue length values, confirming a network delay state based on the length relation information and a first difference value threshold, a second difference value threshold, a first average value threshold and a second average value threshold, and adjusting initial code rates based on corresponding first reduction coefficients, second reduction coefficients and first expansion coefficients under different network delay states to obtain a target code rate, thereby ensuring smoothness of watching videos of users under various network delay states. Further, a preset maximum code rate and a preset minimum code rate are set, video quality is considered and occupied bandwidth size is limited on the premise of ensuring fluency, and therefore user experience is improved.

In addition, an embodiment of the present invention provides a computer program product, where the computer program product includes a computer program, and when the computer program is executed by a processor of an electronic device, makes the processor implement at least the video processing method provided in the foregoing embodiments shown in fig. 1 to 6.

Those skilled in the art will appreciate that implementing all or part of the above-described methods in accordance with the embodiments may be accomplished by way of a computer program stored on a computer readable storage medium, which when executed may comprise the steps of the embodiments of the methods described above. The storage medium may be a magnetic disk, an optical disk, a Read-Only Memory (ROM), a random-access Memory (Random Access Memory, RAM), or the like.

The foregoing disclosure is only illustrative of the preferred embodiments of the present invention and is not to be construed as limiting the scope of the claims, which follow the meaning of the claims of the present invention.

Claims

1. A method of video processing, the method comprising:

2. The method of claim 1, wherein the obtaining at least two queue length values for the video data buffer queue based on the preset time interval comprises:

3. The method of claim 1, wherein before the obtaining at least two queue length values of the video data buffer queue within the preset time interval, further comprises:

Confirming whether the buffer capacity of the video data buffer queue is full;

4. The method of claim 1, wherein said validating the network delay state based on the length relation information between at least two of the queue length values comprises:

5. The method of claim 4, wherein said validating a network delay state based on said length relationship information comprises:

6. The method of claim 5, wherein said adjusting the initial code rate based on the network delay state to obtain a target code rate comprises:

7. The method of claim 6, wherein the reducing the initial code rate to a first optimized code rate based on a first reduction coefficient as the target code rate comprises:

8. A video processing apparatus, the apparatus comprising:

The encoding module is used for encoding the second video source data by adopting the target code rate to obtain target video encoding data corresponding to the target code rate, and outputting the target video encoding data, wherein the second video source data is video source data obtained after the first video source data is obtained.

9. An electronic device, comprising: a processor and a memory; in which a memory stores a computer program adapted to be loaded by the processor and to perform the steps of the method according to any one of claims 1 to 7.

10. A computer storage medium, characterized in that it stores a computer program which, when executed by a processor, implements the steps of the method according to any one of claims 1 to 7.