WO2023279978A1

WO2023279978A1 - Video encoding method and apparatus, device, and storage medium

Info

Publication number: WO2023279978A1
Application number: PCT/CN2022/100805
Authority: WO
Inventors: 张凯明; 要瑞宵
Original assignee: 百果园技术(新加坡)有限公司; 张凯明
Priority date: 2021-07-09
Filing date: 2022-06-23
Publication date: 2023-01-12
Also published as: CN113573101A; CN113573101B

Abstract

A video encoding method and apparatus, a device, and a storage medium, relating to the technical field of computers. The method comprises: acquiring code rate change information corresponding to a current network condition, the code rate change information being used for indicating fluctuation of the current network condition relative to a past network condition (210); acquiring parameter change information corresponding to current video content, the parameter change information being used for indicating a content complexity degree of the current video content (220); determining a target resolution on the basis of the code rate change information and the parameter change information (230); and encoding a current video frame by using the target resolution (240). The fluctuation of network quality and the complexity degree of video content are considered while adjusting and controlling the resolution, such that when the network quality fluctuates too frequently, frequent switching of the resolution is effectively avoided, thereby facilitating reducing the breathing effect caused by frequent fluctuation of the network quality and improving video watching and call experience of users.

Description

Video coding method, device, equipment and storage medium

This application claims the priority of the Chinese patent application with the application number 202110779065.0 and the title of the invention "Video Coding Method, Device, Equipment, and Storage Medium" filed on July 09, 2021, the entire contents of which are incorporated by reference in this application .

technical field

The embodiments of the present application relate to the field of computer technologies, and in particular, to a video coding method, device, device, and storage medium.

Background technique

With the development of mobile Internet, instant messaging technology has penetrated into all aspects of people's daily life and work. The combination of instant messaging technology and video technology has created more diversified instant messaging scenarios, such as video calls, remote conferences, online live broadcasts, online education, and so on.

Usually, such instant messaging scenarios combined with video rely heavily on network quality. During video encoding, the encoder must adjust the video encoding strategy in real time based on the network quality, so that the bit rate consumed by video encoding conforms to the network bandwidth, so as to avoid playback freezes and playback failures caused by poor network quality. At present, the regulation of video coding strategies is mainly divided into two categories: coder internal code rate regulation and encoder external frame rate/resolution regulation. Encoder internal bit rate control refers to the bit rate adjustment within a certain range in the scene of a fixed frame rate/resolution; when the network quality fluctuates greatly, the external frame rate/resolution control of the encoder is usually performed to Consumed bitrate for smooth video encoding.

Contents of the invention

Embodiments of the present application provide a video encoding method, device, equipment, and storage medium. Described technical scheme is as follows:

On the one hand, the embodiment of the present application provides a video encoding method, the method comprising:

Obtain code rate change information corresponding to current network conditions, where the code rate change information is used to indicate fluctuations of the current network conditions relative to historical network conditions;

Acquire parameter change information corresponding to the current video content, where the parameter change information is used to indicate the content complexity of the current video content;

Determine a target resolution based on the code rate change information and the parameter change information;

Encoding the current video frame by using the target resolution.

On the other hand, an embodiment of the present application provides a video encoding device, the device comprising:

A code rate information acquisition module, configured to obtain code rate change information corresponding to current network conditions, where the code rate change information is used to indicate fluctuations of the current network conditions relative to historical network conditions;

A parameter information acquisition module, configured to acquire parameter change information corresponding to the current video content, where the parameter change information is used to indicate the content complexity of the current video content;

A resolution determination module, configured to determine a target resolution based on the code rate change information and the parameter change information;

A video coding module, configured to use the target resolution to code the current video frame.

In another aspect, an embodiment of the present application provides a computer device, the computer device includes a processor and a memory, and a computer program is stored in the memory, and the computer program is loaded and executed by the processor to realize the above-mentioned Video encoding method.

In yet another aspect, an embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the foregoing video encoding method is implemented.

In another aspect, an embodiment of the present application provides a computer program product, which, when the computer program product is run on a computer device, causes the computer device to execute the above video encoding method.

The technical solutions provided in the embodiments of the present application can bring the following beneficial effects:

During the video encoding process, the resolution used for encoding the current video frame is determined in combination with the fluctuation of the network quality and the complexity of the video content, so that the resolution can be adjusted more accurately. Since human eyes are more sensitive to changes in picture quality in simple scenes, this application combines the complexity of video content when adjusting the resolution, and considers the sensitivity of human eyes to different content complexity of video content, so that the resolution The regulation is more accurate and comprehensive. Moreover, this application considers the fluctuation of network quality and the complexity of video content when adjusting the resolution. When the network quality fluctuates too frequently, it effectively avoids frequent switching of resolutions, which helps to reduce the video frequency caused by frequent fluctuations in network quality. The resulting breathing effect improves the user's video viewing and calling experience.

Description of drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings that need to be used in the description of the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application. For those skilled in the art, other drawings can also be obtained based on these drawings without creative effort.

FIG. 1 is a schematic diagram of an instant messaging scenario provided by an embodiment of the present application;

FIG. 2 is a flowchart of a video coding method provided by an embodiment of the present application;

Fig. 3 is a schematic diagram of key block matching provided by an embodiment of the present application;

FIG. 4 is a flowchart of a video encoding method provided by another embodiment of the present application;

FIG. 5 is a block diagram of a video encoding device provided by an embodiment of the present application;

Fig. 6 is a block diagram of a video encoding device provided by another embodiment of the present application.

detailed description

In order to make the purpose, technical solution and advantages of the present application clearer, the implementation manners of the present application will be further described in detail below in conjunction with the accompanying drawings.

It can be seen from the introduction of the above background technology that the encoder internal bit rate control is applicable to the scene of fixed frame rate/resolution. When the network quality fluctuates greatly, the external frame rate/resolution control of the encoder is usually performed. In the following, the frame rate control and resolution control are introduced and explained respectively.

Frame rate control: The client automatically maintains a frame-level bit rate statistics queue, which is used to cache the actual consumed bit rate of recent encoded frames, and then calculates the frame-level average consumed bit rate of encoded frames, and enters encoding in the current frame Before the processor, by comparing the frame-level average consumption bit rate of the encoded frame with the target frame-level bit rate estimated by using the actual bandwidth, it is judged whether the current frame still has enough bit rate for encoding. If the remaining bit rate is insufficient, directly Discard the current frame and wait for the arrival of the next acquisition frame.

In one example, the frame-level average bitrate consumption calculation formula of encoded frames is as follows:

Then whether the current frame needs to be discarded is determined according to the following calculation formula:

Wherein, Rav is the frame-level average consumption code rate of the encoded frame, R _k is the actual consumption code rate of the kth frame, and n is the sequence number of the current frame; _TargteBits is the target frame-level code rate; FPS is the set frame rate; α is a threshold coefficient; if the discard flag is 1, it means that the current frame needs to be discarded; if the discard flag is 0, it means that the current frame does not need to be discarded. Based on the above formula, it can be seen that when the ratio between the frame-level average consumed bit rate of the encoded frame and the target frame-level bit rate is lower than a certain threshold, the current frame needs to be discarded to ensure a stable bit rate.

Resolution control: the code table solution. For videos with different resolutions, set various bit rates in sequence from small to large to encode and decode, get the reconstructed video, and calculate the PSNR (Peak Signal to Noise Ratio) under various bit rates. The PSNR value is higher Larger means better video quality. Through a large amount of experimental data, the most suitable bit rate range for a certain resolution can be obtained. For example, 270P (standard definition) video has the best video quality when the bit rate is 320kbps (kilobits per second) to 500kbps. A code table solution can be formed by integrating the appropriate bit rate ranges of various resolutions.

In one example, the code table scheme looks like this:

Among them, level represents different resolution levels, such as 180P (smooth), 270P (standard definition), 360P (high definition); level _mr refers to the default resolution level, such as the default resolution is 270P; uplevel and domnlevel represent level respectively The upper and lower bounds of the most suitable bit rate range for _mr , when the target frame level bit rate exceeds the bounds, it is necessary to switch the resolution according to the above code table scheme. For example, assuming that the above-mentioned level _mr is 270P, level _mr -1 is 180P, and domnlevel is 350kps, then when TargteBits fluctuates from 400kps to 300kbps, the resolution needs to be switched from 270P to 180P.

From the above introduction, it can be seen that the frame rate adjustment focuses more on ensuring the video quality, mainly at the expense of the smoothness of the video to obtain the stability of the bit rate. However, this type of adjustment will cause obvious video freezes, and even freezes in severe cases. Dead, very unfriendly to the user's subjective experience. In other words, the control of frame rate control is too strong, which is suitable for extreme situations. Resolution control focuses more on ensuring the fluency of the video, and adjusts the video resolution in real time based on the network quality to achieve the effect of balancing the video bit rate. Compared with frame rate control, the resolution control method is relatively gentler, more friendly to the user's subjective experience, and has a wider range of application scenarios.

However, in the case of resolution control, when the network quality fluctuates too frequently, the resolution switching will also be too frequent, which will produce a significant breathing effect. In addition, the human eye is more sensitive to changes in picture quality in simple scenes. At this time, resolution control needs to be more careful to avoid a sharp drop or rise in video picture quality due to excessive resolution control, which will affect users' viewing and viewing experience. call experience.

Based on this, an embodiment of the present application provides a video encoding method, which can more efficiently and flexibly determine the resolution used when encoding the current video frame during the video encoding process. In the following, the technical solutions provided by the present application will be described in conjunction with several embodiments.

It should be noted that the "instant messaging" mentioned in the following embodiments refers to instant messaging combined with video, such as video calls, teleconferencing, online live broadcast, online education, and so on. The embodiment of the present application does not limit the way of carrying out "instant messaging". Users can carry out instant messaging through applications, webpages, applets, etc. that have instant messaging functions. Optionally, the application program with the instant messaging function includes an instant messaging application program, a video playing application program, a live broadcast application program, a game application program, an educational application program, etc., which are not limited in this embodiment of the present application.

It should also be noted that the embodiment of this application is only for the convenience of explaining the process of video encoding and video decoding, and introduces video encoding and video decoding in the context of instant messaging, but this does not constitute a technical solution of this application. limit. In practical applications, the video encoding method provided by the embodiment of the present application can also be applied in non-instant messaging scenarios, for example, users share the captured video to personal social platforms, webpages and other scenarios, all of which should belong to the scope of protection of the present application within.

Please refer to FIG. 1 , which shows a schematic diagram of an instant messaging scenario provided by an embodiment of the present application. As shown in FIG. 1 , the instant messaging scenario includes: a video encoding end 110 and a video decoding end 120 .

The video encoding end 110 refers to the generation end of video content in the instant messaging scene, which is used to generate video content, encode the generated video content, and send the encoded video content to the video decoding end 120, etc. The video decoding end 120 refers to the receiving end of the video content in the instant messaging scene, and is used for decoding the received video content, playing the decoded video content, and so on. Optionally, the video encoding end 110 and the video decoding end 120 communicate with each other through a network. In one example, the video encoding end 110 and the video decoding end 120 may communicate directly, or communicate indirectly through a server (not shown in FIG. 1 ), for example, the video encoding end 110 sends the encoded video content to the server , and then the server sends the encoded video content to the video decoder 120 .

The embodiment of the present application does not limit the device type of the video encoding end 110 and/or the video decoding end 120. In one example, the video encoding end 110 and/or the video decoding end 120 is any of the following devices: computer equipment, terminal equipment, Servers, wearable devices, Bluetooth devices, in-vehicle devices, etc. In an example, the video encoding end 110 and the video decoding end 120 are of the same device type, or are of different device types. Of course, in an actual instant messaging scenario, the video encoding end 110 and the video decoding end 120 can be implemented as the same device, that is, the device has both the functions of the video encoding end 110 and the video decoding end 120 . FIG. 1 only illustrates that the video encoding end 110 and the video decoding end 120 are different devices for illustration, but this does not constitute a limitation to the technical solution of the present application.

It should be noted that in this embodiment of the application, the video encoding end 110 and the video decoding end 120 are distinguished only for the convenience of explaining the process of video encoding and video decoding, but this does not constitute a limitation to the technical solution of the application. In an actual instant messaging scenario, the video encoding end 110 can encode and send the video content, and can also receive and decode the video content; the video decoding end 120 can receive the video content and decode it, or Encode the video content and send it. In other words, in an actual instant messaging scenario, all parties involved in instant messaging can perform steps such as generating and encoding video content, decoding and playing video content generated by other parties, and so on.

Please refer to FIG. 2 , which shows a flowchart of a video encoding method provided by an embodiment of the present application. This method can be applied to the video encoding end 110 in the instant messaging scene shown in FIG. 1 . As shown in Fig. 2, the method includes the following steps (210-240).

In step 210, the code rate change information corresponding to the current network condition is acquired, and the code rate change information is used to indicate the fluctuation of the current network condition relative to the historical network condition.

Network quality has an important impact on the bit rate and resolution used in the video encoding process. In order to achieve better video encoding effects and ensure the clarity and fluency of video playback, it is necessary to consider the network quality on the video encoding. The influence of the parameters. From the above content, it can be seen that when the network quality fluctuates greatly and the bit rate control range is exceeded, the frame rate and/or resolution need to be regulated. However, in the embodiment of the present application, because the frame rate regulation is too strong, it is only applicable to extreme cases, so when the network quality fluctuates greatly, a more moderate resolution regulation method is adopted.

Therefore, in order to determine whether resolution adjustment is required, or in other words, in order to determine the target resolution adopted for the current video frame in the video encoding process, it is necessary to determine the fluctuation of network quality, that is, the difference between current network conditions and historical network conditions. fluctuation. Because the network quality is usually reflected by the network bandwidth, and the code rate is usually associated with the network bandwidth, therefore, in the embodiment of the present application, the code rate change information is used to indicate the fluctuation of the current network condition relative to the historical network condition. Wherein, the historical network condition may be the network quality from a certain time before the current time to the current time, or may be the network quality from the start time of encoding the video content to the current time. For other descriptions about bit rate change information, current network conditions, historical network conditions, etc., please refer to the following embodiments, and details will not be repeated here.

Step 220, acquiring parameter change information corresponding to the current video content, where the parameter change information is used to indicate the content complexity of the current video content.

Since the human eye is more sensitive to changes in picture quality in simple scenes, resolution control needs to be more careful when the video content is simple, so as to avoid a sharp drop or rise in video picture quality due to excessive resolution control. Compared with related technologies that only consider the impact of network quality on video coding, the video coding method provided by the embodiment of the present application also considers the sensitivity of human eyes to the complexity of different content of video content, so as to avoid drastic changes in video picture quality .

Therefore, in order to determine the target resolution adopted for the current video frame during the video encoding process, it is also necessary to determine the content complexity of the current video content. Usually, when the code rate is relatively stable, the higher the texture complexity of the video frame, the larger the actual quantization parameter (Quantization Parameter, QP) after encoding, and there is a positive correlation between the two. Therefore, the application implements In this example, the parameter change information of the quantization parameter is used to indicate the content complexity of the current video content. For other introductions such as parameter change information, content complexity, etc., please refer to the following embodiments, and details will not be repeated here.

Step 230: Determine the target resolution based on the code rate change information and the parameter change information.

After the code rate change information and the parameter change information are acquired, the target resolution used when encoding the current video frame is determined based on the two aspects of information. In an example, the video encoding end determines that resolution adjustment is not necessary based on the two aspects of information, and directly uses the original resolution of the current video frame as the target resolution. In another example, the video encoding end determines that resolution adjustment is required based on the two aspects of information, and then increases or decreases the original resolution of the current video frame to obtain the target resolution. For other descriptions such as determination of the target resolution, please refer to the following embodiments, and details will not be repeated here.

Step 240, encode the current video frame with the target resolution.

After the video encoding end determines the target resolution, it can use the target resolution to encode the current video frame, so that the video bit rate matches the network bandwidth, and avoids problems such as video playback freeze or playback failure. Optionally, the encoding of the current video frame by the video encoding end using the target resolution includes: performing scaling processing on the current video frame by the video encoding end based on the target resolution.

Exemplarily, when the target resolution is greater than the original resolution of the current video frame, encoding the current video frame with the target resolution includes: performing upsampling processing on the current video frame; When the frame resolution matches the target resolution, the target resolution is used to encode the upsampled current video frame.

Exemplarily, when the target resolution is smaller than the original resolution of the current video frame, encoding the current video frame with the target resolution includes: performing downsampling processing on the current video frame; When the frame resolution matches the target resolution, the target resolution is used to encode the downsampled current video frame.

Optionally, matching the resolution of the current video frame after the up-sampling process (or the current video frame after the down-sampling process) with the target resolution includes: the current video frame after the up-sampling process (or the current video frame after the down-sampling process) The resolution of the current video frame after upsampling) is equal to the target resolution, or the difference between the resolution of the current video frame after upsampling (or the current video frame after downsampling) and the target resolution is less than set the threshold.

To sum up, in the technical solution provided by the embodiment of the present application, by combining the fluctuation of the network quality and the complexity of the video content in the process of video encoding, the resolution adopted for encoding the current video frame is determined, so as to realize more accurate Adjust the resolution. Since human eyes are more sensitive to changes in picture quality in simple scenes, this application combines the complexity of video content when adjusting the resolution, and considers the sensitivity of human eyes to different content complexity of video content, so that the resolution The regulation is more accurate and comprehensive. Moreover, this application considers the fluctuation of network quality and the complexity of video content when adjusting the resolution. When the network quality fluctuates too frequently, it effectively avoids frequent switching of resolutions, which helps to reduce the video frequency caused by frequent fluctuations in network quality. The resulting breathing effect improves the user's video viewing and calling experience.

In the following, an introduction will be given to an exemplary implementation of the process of determining the code rate change information, the parameter change information, and the target resolution.

In an example, the above step 210 includes the following steps (212-216).

Step 212, determine the real-time target code rate, where the real-time target code rate is used to indicate the current network condition.

In order to enable the video frame encoded by the video encoding end to be successfully transmitted under the current network conditions, it is necessary to estimate the current network conditions to obtain the encoding bit rate that can be used as a reference for the video encoding end, that is, the real-time frame used to indicate the current network conditions Target bit rate. Optionally, the video encoding end uses a bandwidth estimation method to determine the real-time target code rate, and the bandwidth estimation method includes but not limited to: a link delay method, a packet detection bandwidth estimation method, and the like.

Step 214, obtaining the historical average code rate, which is used to indicate the historical network conditions.

In order to obtain the fluctuation of the current network condition relative to the historical network condition, the video encoding end also needs to determine the historical average bit rate used to indicate the historical network condition. Optionally, the historical average bit rate is the average bit rate from the start moment of video encoding to the current moment. For example, when the technical solution of the present application is applied to a video call scenario, the historical average bit rate may be when the video call is started The average code rate between the time and the current time; or, the historical average code rate is the average code rate in a period of time before the current time, for example, the historical average code rate is the average code rate within 5 seconds before the current time. Based on this, the video encoding end needs to maintain a code rate buffer, and the average value in the code rate buffer is the historical average code rate, which is used to indicate historical network conditions.

Step 216: Determine code rate change information based on the real-time target code rate and the historical average code rate.

After the video encoding end determines the real-time target bit rate and the historical average bit rate, it can compare the real-time target bit rate and the historical average bit rate to determine the bit rate change information. In one example, the above step 216 includes: determining a code rate difference between the real-time target code rate and the historical average code rate; and determining code rate change information based on the code rate difference. Optionally, determining the code rate change information based on the code rate difference includes: when the absolute value of the code rate difference is greater than the first threshold and the code rate difference is a positive number, determining that the code rate change information includes code The code rate changes drastically, and the code rate increases suddenly; when the absolute value of the code rate difference is greater than the first threshold, and the code rate difference is negative, the code rate change information includes that the code rate changes drastically, and the code rate sudden drop; when the absolute value of the code rate difference is smaller than the first threshold, determining that the code rate change information includes that the code rate does not change drastically. Optionally, the first threshold is a positive number.

Exemplarily, the real-time target code rate is R, and the historical average code rate is R _av , R _av =avg(buffer_R), wherein, buffer_R is the code rate cache maintained by the video encoding end, and avg(·) represents the averaging function; then The bit rate difference between the real-time target bit rate and the historical average bit rate is RR _av . Assuming that the first threshold is Th1, the code rate change information has the following situations:

In the case of |RR _av |<Th1, the code rate change information includes that the code rate has not changed drastically;

In the case of RR _av >Th1, the code rate change information includes a drastic change in the code rate, and a sudden increase in the code rate;

In the case of R _av -R>Th1, the code rate change information includes that the code rate changes drastically, and the code rate drops sharply.

It should be noted that the content of the code rate change information in the case of |RR _av |=Th1 is not limited in this embodiment of the application. In practical applications, it can be set that the code rate change information in this case includes code If the rate does not change drastically, it can also be set that the bit rate change information changes drastically in this case, and the bit rate is set to increase or decrease according to the positive or negative value of RR _av . It should be understood that all these should fall within the protection scope of the present application.

In an example, the above step 220 includes the following steps (222-226).

Step 222, acquiring a first quantization parameter value, where the first quantization parameter value refers to the quantization parameter value of the previous video frame of the current video frame.

Generally, when the current video content has no scene changes relative to the historical video content, the difference between the front and rear frames of the video is very small, and the content complexity of the video content can be considered to be consistent. Therefore, the current video frame can be used The content complexity of the previous video frame represents the content complexity of the current video frame. Therefore, in the embodiment of the present application, the video encoding end needs to obtain the quantization parameter value of the previous video frame of the current video frame, that is, the first quantization parameter value.

Step 224, acquiring a second quantization parameter value, where the second quantization parameter value refers to an average value of quantization parameter values of at least one historical video frame.

In order to obtain the content complexity of the current video content, the video encoding end also needs to determine the mean value of the quantization parameter values of at least one historical video frame, that is, the second quantization parameter value. Optionally, at least one historical video frame is all video frames from the start moment of video encoding to the current moment, for example, when the technical solution of the present application is applied to a video call scenario, at least one historical video frame may be a video call All video frames from the moment of opening to the current moment; or, at least one historical video frame is all video frames in a period of time before the current moment; or, at least one historical video frame is N video frames before the current video frame , N is a positive number, optionally, N is 5, 10, 15, etc. Based on this, the video encoder needs to maintain a quantization parameter value cache, and the average value in the quantization parameter value cache is the second quantization parameter value, which is used to indicate the content complexity of recent video content. Optionally, the video encoding end maintains a quantization parameter value cache with a length of N, where N is a positive number, for example, N is 10.

Step 226: Determine parameter change information based on the first quantization parameter value and the second quantization parameter value.

After determining the first quantization parameter value and the second quantization parameter value respectively, the video encoding end compares the first quantization parameter value and the second quantization parameter value to determine parameter change information. In an example, the above step 226 includes: determining a parameter difference between the first quantization parameter value and the second quantization parameter value; and determining parameter change information based on the parameter difference. Optionally, determining the parameter change information based on the parameter difference includes: when the absolute value of the parameter difference is greater than a second threshold, determining that the parameter change information indicates that the video content is complex; when the absolute value of the parameter difference is smaller than the second threshold In the case of the threshold, the determination parameter change information indicates that the video content is simple. Optionally, the second threshold is a positive number.

Exemplarily, the first quantization parameter value is QP, and the second quantization parameter value is QP _av , wherein, QP _av =avg(buffer_QP), buffer_QP is the quantization parameter value cache maintained by the video encoder, and avg(·) represents the average value function; then the parameter difference between the first quantization parameter value and the second quantization parameter value is QP-QP _av . Assuming that the second threshold is Th2, the parameter change information has the following situations:

In the case of |QP-QP _av |<Th2, the parameter change information indicates that the video content is simple;

In the case of |QP-QP _av |>Th2, the parameter change information indicates that the video content is complex.

It should be noted that the content indicated by the parameter change information in the case of |QP-QP _av |=Th2 is not limited in this embodiment of the present application. In practical applications, the parameter change information in this case can be set The content of the indicated video is simple, and it can also be set that in this case the parameter change information indicates that the content of the video is complex. It should be understood that all these should fall within the protection scope of the present application.

Based on the above introduction about code rate change information and parameter change information, the above step 230 includes any one of the following steps (232-238).

Step 232, when the code rate change information includes a sudden increase in the code rate, and the parameter change information indicates that the video content is simple, gradually increase the real-time consumption code rate within the target D frame interval; if the real-time consumption code rate exceeds the target D frame interval If it is greater than the real-time target bit rate, the original resolution of the current video frame is determined as the target resolution; if the real-time consumption bit rate is less than the real-time target bit rate after exceeding the target D frame interval, then increase the original resolution of the current video frame to get target resolution.

Since the human eye is more sensitive to changes in picture quality in simple scenes, when the parameter change information indicates that the video content is simple, and the bit rate change information includes a sudden increase in the bit rate, it should be avoided that the resolution control is too large to cause video distortion. Picture quality has risen dramatically. Therefore, in the embodiment of the present application, for the scene where the video content is simple and the code rate increases suddenly, the code rate control is first adopted, and then whether to adopt the resolution control is determined based on the result of the code rate control.

For scenes where the video content is simple and the bit rate increases suddenly, gradually increase the real-time consumption bit rate within the target D frame interval. Optionally, the target D frame is the next D frame, and the value of D may be 15. If the real-time consumption bit rate is greater than the real-time target bit rate after exceeding the target D frame interval, it means that the bit rate regulation can adapt to the current network conditions. At this time, the original resolution of the current video frame is determined as the target resolution. If the real-time consumption bit rate is lower than the real-time target bit rate after exceeding the target D frame interval, it means that the bit rate control still cannot adapt to the current network conditions, and then continue to use resolution control, that is, increase the original resolution of the current video frame to obtain the target resolution. The method of increasing the bit rate in the D frame interval improves the picture quality of the video, and can realize a relatively smooth transition.

In this example, since the code rate change information includes a sudden increase in the code rate, within the target D frame interval, the real-time target code rate is at a high level, and the real-time consumed code rate is at a low level. At this time, a waste of code rate may occur. In order to avoid waste of code rate, optionally, the video encoding end fills redundant error correction packets, for example, resends some data packets of key frames, so as to enhance the error correction capability of key frames.

Step 234, when the code rate change information includes a sudden drop in the code rate, and the parameter change information indicates that the video content is simple, gradually reduce the real-time consumption code rate within the target D frame interval; if the real-time consumption code rate exceeds the target D frame interval If it is smaller than the real-time target bit rate, the original resolution of the current video frame is determined as the target resolution; if the real-time consumption bit rate exceeds the target D frame interval and is greater than the real-time target bit rate, then the original resolution of the current video frame is reduced to obtain target resolution.

Since the human eye is more sensitive to changes in picture quality in simple scenes, when the parameter change information indicates that the video content is simple, and the bit rate change information includes a sudden drop in the bit rate, it should be avoided that the resolution control is too strong to cause video distortion. Picture quality drops drastically. Therefore, in the embodiment of the present application, for the scene where the video content is simple and the code rate drops suddenly, the code rate control is first adopted, and then whether to adopt the resolution control is determined based on the result of the code rate control.

For scenes where the video content is simple and the bit rate drops suddenly, gradually reduce the real-time consumption bit rate within the target D frame interval. Optionally, the target D frame is the next D frame, and the value of D may be 15. If the real-time consumption bit rate is lower than the real-time target bit rate after exceeding the target D frame interval, it means that the bit rate regulation can adapt to the current network conditions. At this time, the original resolution of the current video frame is determined as the target resolution. If the real-time consumption bit rate is greater than the real-time target bit rate after exceeding the target D frame interval, it means that the bit rate control still cannot adapt to the current network conditions, then continue to use resolution control, that is, reduce the original resolution of the current video frame to obtain the target resolution. The method of reducing the bit rate in the D frame interval reduces the picture quality of the video, and can achieve a relatively smooth transition.

Step 236, when the code rate change information includes a sudden increase in the code rate, and the parameter change information indicates that the video content is complex, gradually increase the real-time consumption code rate within the target D frame interval; if the real-time consumption code rate exceeds the target D frame interval If it is greater than the real-time target bit rate, the original resolution of the current video frame is determined as the target resolution; if the real-time consumption bit rate is less than the real-time target bit rate after exceeding the target D frame interval, then increase the original resolution of the current video frame to get target resolution.

When the parameter change information indicates that the video content is complex, the actual quantization parameter after encoding the historical video frame is relatively large; and because the bit rate change information includes a sudden increase in the bit rate, the real-time target bit rate is higher and the quantization parameter value is smaller. Therefore, in the embodiment of the present application, aiming at the scene where the video content is complex and the code rate increases rapidly, the code rate control is firstly used to improve the picture quality of the video frame by frame, and then it is determined whether to adopt the resolution control based on the result of the code rate control.

For scenes where the video content is complex and the bit rate increases suddenly, the real-time consumption bit rate is gradually increased within the target D frame interval. Optionally, the target D frame is the next D frame, and the value of D may be 15. If the real-time consumption bit rate is greater than the real-time target bit rate after exceeding the target D frame interval, it means that the bit rate regulation can adapt to the current network conditions. At this time, the original resolution of the current video frame is determined as the target resolution. If the real-time consumption bit rate is lower than the real-time target bit rate after exceeding the target D frame interval, it means that the bit rate control still cannot adapt to the current network conditions, and then continue to use resolution control, that is, increase the original resolution of the current video frame to obtain the target resolution. The method of increasing the bit rate in the D frame interval improves the picture quality of the video, and can realize a relatively smooth transition.

Step 238 , if the code rate change information includes a sudden drop in the code rate, and the parameter change information indicates that the video content is complex, reduce the original resolution of the current video frame to obtain the target resolution.

When the parameter change information indicates that the video content is complex, the actual quantization parameter after encoding the historical video frame is relatively large; and because the bit rate change information includes the bit rate drop, the real-time target bit rate is low and the quantization parameter value is large. Therefore, for scenarios where the video content is complex and the bit rate drops suddenly, it is difficult to adjust the bit rate to adapt to the current network conditions. The embodiment of the present application directly adjusts the resolution, that is, reduces the original resolution of the current video frame to obtain the target resolution to ensure the followability of the code rate.

Optionally, when the code rate change information includes that the code rate has not changed drastically, the internal code rate control scheme of the encoder can be directly adopted without resolution control, so that the video encoding end directly converts the original The resolution is determined as the target resolution.

To sum up, the technical solutions provided by the embodiments of the present application classify resolution adjustment into multiple types by combining fluctuations in network quality and complexity of video content. In addition, for scenes with simple video content, no matter how the network quality fluctuates, since the human eye is more sensitive to changes in picture quality in simple scenes, the bit rate control method is first used to smooth the picture quality changes of the video. When the network quality cannot be adapted, the resolution control method is adopted to effectively avoid the sharp change of the video picture quality. In addition, for scenes with complex video content, when the bit rate drops sharply, directly adjust the resolution to ensure the followability of the bit rate; Avoid drastic changes in picture quality. The embodiment of the present application improves the video playback effect or the video call effect through the above refined resolution control scheme, and brings better video playback or video call experience to the user.

In the resolution control scheme introduced in the above embodiments, the quantization parameter value of the previous video frame of the current video frame is used as the measure of the complexity of the content of the current video frame, which is suitable for the case where the difference between the front and back frames of the video is small Condition. Based on this, it is also necessary to determine whether the scene changes before adopting the resolution control solution introduced in the above embodiment.

Based on this, in an example, the above method further includes the following steps.

Step 250, detecting the scene change of the current video content relative to the historical video content.

In scenarios such as video playback or video calls, once the scene changes, the content complexity of the video content may change greatly, which may cause large bit rate fluctuations. program interferes. Therefore, in the embodiment of the present application, the scene change of the current video content relative to the historical video content can be detected first, so as to avoid errors and adverse effects caused by the scene change on the resolution adjustment. The embodiment of the present application does not limit the scene change detection method. Optionally, the scene change detection method includes: a key block matching algorithm, a full frame matching algorithm, a downsampling frame matching algorithm, and the like. In general, keyblock matching algorithms have low computational complexity. In the following, the process of applying the key block matching algorithm to detect scene changes will be described.

In an example, the above step 250 includes the following steps (252-256).

Step 252, determine at least one key block from the current video frame.

The embodiment of the present application does not limit the selected number, selected position, size, etc. of the key blocks. In an example, the video encoding end determines a key blocks from the current video frame, where a is a positive integer, such as 4, 5, 6, and so on. In an example, the size of at least one key block is the same, for example, the size is 16×16 pixels, or 8×8 pixels, or 32×32 pixels, or 4×4 pixels, or are 64×64 pixels; or, the size of at least one key block is not exactly the same, for example, the size of one key block is 16×16 pixels, and the size of another key block is 32×32 pixels, etc. In an example, at least one key block is evenly distributed in the current video frame, or at least one key block is concentrated in a certain area in the current video frame, for example, at least one key frame is concentrated in the middle area of the current video frame.

Exemplarily, the video encoding end determines 5 key blocks from the current video frame, the 5 key blocks are evenly distributed in the current video frame, and the size of the 5 key blocks are all 16×16 pixels. For example, as shown in Figure 3, a total of 5 squares with a size of 16×16 pixels are selected as key blocks at the center of the current video frame and its surroundings. The key block at the center of the current video frame is 310, and the key blocks around the center are The block is 320. The horizontal and vertical offsets of each key block 320 from the key block 310 are both 64 pixels, so that the five key blocks are evenly distributed in the current video frame as shown in FIG. 3 .

Step 254: For the first key block in the at least one key block, search for a matching block with the target radius as the search radius at the position corresponding to the first key block in the previous video frame of the current video frame.

After the key blocks are determined, the key blocks are matched. In the embodiment of the present application, key block matching is performed between the current video frame and the previous video frame of the current video frame, so that, for each key block in at least one key block, it is necessary to select from the previous video frame of the current video frame Do a matching block search. Exemplarily, for the first key block in at least one key block, it is first necessary to determine the corresponding position of the first key block in the previous video frame of the current video frame, and then use the target radius r as the search around the corresponding position Radius to search for matching blocks. The embodiment of the present application does not limit the size of the target radius. Optionally, the target radius is 1 pixel, or 2 pixels, or 4 pixels, or 8 pixels.

Step 256, if there is a block whose mean square error with the first key block is smaller than the target threshold among the searched blocks, determine that the first key block has a matching block in the previous video frame of the current video frame.

If the mean square error (Mean Square Error, MSE) between a block and the first key block can be searched within the search range of the previous video frame of the current video frame is less than the target threshold, then the block is determined as the first key block A matching block of a key block, that is, a matching block of the first key block exists in the previous video frame of the current video frame. Optionally, in addition to using the mean square error as the criterion for matching block search, the video encoding end can also use MAD (Mean Absolute Deviation, average absolute error), SAD (Sum of Absolute Difference, absolute error sum), etc. to perform matching block search Search, the embodiment of the present application does not limit the matching criteria used in the matching block search. Optionally, the size of the matching block and the key block must be the same. For example, assuming that the size of the first key block is 16×16 pixels, the size of the matching block of the first key block is also 16×16 pixels.

Exemplarily, the mean square error MSE between the searched blocks and key blocks within the search range is:

Among them, N is the size of the key block, such as N is 16; C _ij is the pixel value corresponding to the _i -th row and j-th column in the key block of the current video frame; The pixel value corresponding to column j. The smaller the MSE calculated by the above calculation formula, the higher the similarity between the searched block and the key block. In the embodiment of the present application, a target threshold Th ₀ is set, and when the MSE is smaller than Th ₀ , the searched block is considered to be a matching block of the key block.

In the embodiment of the present application, when at least one key block has a matching block in the previous video frame of the current video frame, it is considered that the current video frame is similar to the previous video frame, that is, it is determined that the current video frame is relatively similar to the historical video frame There is no scene change; and if there is a certain key block in at least one key block and there is no matching block in the previous video frame of the current video frame, it is determined that the current video frame has a scene change relative to the historical video frame.

Of course, in practical applications, it is also possible to determine that there is no scene change in the current video frame relative to the historical video frame when there are matching blocks in at least one key block of the target number of key blocks in the previous video frame of the current video frame . For example, if 5 key blocks are selected in the current video frame, and 4 key blocks in the 5 key blocks have matching blocks in the previous video frame of the current video frame, determine the current video frame relative to the history There is no scene change in the video frame. It should be understood that these solutions should also fall within the protection scope of the present application.

Based on the above scene change detection, in an example, the above step 230 is implemented as: determining the target resolution based on bit rate change information and parameter change information when the current video content has no scene change relative to the historical video content. That is, when it is detected that the scene does not change, the target resolution is determined by using the resolution control solution provided by the above embodiment, taking into account fluctuations in network quality and complexity of video content.

Based on the above scene change detection, in another example, after the above step 256, it also includes: when the current video content has a scene change relative to the historical video content, the resolution corresponding to the real-time target bit rate in the target correspondence , determined as the target resolution; wherein, the target correspondence includes at least one set of correspondences between the target bit rate and the resolution. That is, when a scene change is detected, the code table scheme is used to determine the target resolution, and the video encoding end determines the resolution corresponding to the real-time target bit rate from the target correspondence, and takes this resolution as the target resolution.

To sum up, the technical solution provided by the embodiment of the present application detects the scene change of the current video content relative to the historical video content, and combines the fluctuation of the network quality and the complexity of the video content when no scene change is detected. To determine the target resolution, effectively avoiding the impact of bit rate fluctuations on resolution adjustment due to scene changes, and improving the accuracy of resolution adjustment.

In the following, an exemplary embodiment is used to describe the video encoding method provided in this application.

Please refer to FIG. 4 , which shows a flowchart of a video encoding method provided by an embodiment of the present application. This method can be applied to the video encoding end 110 in the instant messaging scene shown in FIG. 1 . As shown in Fig. 4, the method includes the following steps (401 to 412).

Step 401, acquire video frame data. In the case of starting a video call or performing video playback (such as live video broadcasting), the video capture module at the video encoding end can call the camera module to capture video frames at the target frame rate, and transmit the captured video frame data to the video The audio and video processing module at the encoding end performs processing.

Step 402, acquiring network quality feedback. The audio and video processing module at the video encoding end can use the bandwidth estimation method to determine the real-time target code rate R, which is used to indicate the current network conditions. At the same time, this step will maintain a code rate buffer buffer_R, and the code rate buffer buffer_R can be a second-level code rate buffer, such as 5 seconds. The average value R _av of the code rate cache is the historical average code rate, which is used to indicate the historical network conditions.

Step 403, acquire video content feedback. When the bit rate is relatively stable, the higher the texture complexity of the video frame, the larger the actual quantization parameter value after encoding. Therefore, in this step, the quantitative parameter value is used to indicate the content complexity of the video content. The video encoding end obtains the quantization parameter value of the previous video frame of the current video frame, that is, the first quantization parameter value QP. At the same time, the video encoding end establishes and maintains a buffer_QP of quantization parameter values with a length of N. Optionally, the value of N is 10. The average value of the quantization parameter value buffer buffer_QP is the second quantization parameter value QP _av , which is used to indicate the content complexity of recent video content.

Step 404, detecting whether the scene changes. Once the scene changes, the content complexity of the video content may change greatly, which may cause large bit rate fluctuations. Therefore, the scene change of the current video content relative to the historical video content is detected first, so as to avoid errors and adverse effects caused by the scene change on the resolution adjustment. The video encoding end establishes a frame-level buffer buffer_pre, which is used to store the previous video frame data of the current video frame, so that each time the video encoding end receives the real-time current video frame, it performs key blocks between the current video frame and the previous video frame Match to detect whether the current video content has a scene change relative to the historical video content. If there is no scene change, execute the following step 405; if there is a scene change, execute the following step 411.

Step 405, detecting whether the network quality fluctuates. The video encoding end compares the real-time target bit rate R obtained in the above step 402 with the historical average bit rate R _av to determine whether the network quality fluctuates greatly. If the code rate difference between the real-time target code rate R and the historical average code rate R _av is greater than the first threshold Th1, it is determined that the network quality fluctuates greatly and the code rate changes sharply, and the following step 406 is executed. If the code rate difference between the real-time target code rate R and the historical average code rate R _av is less than or equal to the first threshold Th1, it is determined that the network quality fluctuation is small, and the original resolution of the current video frame is maintained, that is, the current The original resolution of the video frame is used as the target resolution, and the following step 412 is performed.

Step 406, determine the complexity of the content. The video encoder compares the first quantization parameter value QP and the second quantization parameter value QP _av obtained in step 403 above to determine the content complexity of the video content. If the absolute value of the parameter difference between the first quantization parameter value QP and the second quantization parameter value QP _av is greater than the second threshold Th2, it indicates that the video content is complex; if the first quantization parameter value QP and the second quantization parameter value QP If the absolute value of the parameter difference between _av is smaller than the second threshold Th2, it indicates that the video content is simple.

Step 407, execute the control scheme in the scenario where the video content is simple and the code rate increases suddenly. At this point, gradually increase the real-time consumption bit rate within the target D frame interval. If the real-time consumption bit rate is greater than the real-time target bit rate after exceeding the target D frame interval, it means that the bit rate regulation can adapt to the current network conditions. At this time, the original resolution of the current video frame is determined as the target resolution. If the real-time consumption bit rate is lower than the real-time target bit rate after exceeding the target D frame interval, it means that the bit rate control still cannot adapt to the current network conditions, and then continue to use resolution control, that is, increase the original resolution of the current video frame to obtain the target resolution.

Step 408, execute the control scheme in the scenario where the video content is simple and the code rate drops suddenly. At this time, gradually reduce the real-time consumption bit rate within the target D frame interval. If the real-time consumption bit rate is lower than the real-time target bit rate after exceeding the target D frame interval, it means that the bit rate regulation can adapt to the current network conditions. At this time, the original resolution of the current video frame is determined as the target resolution. If the real-time consumption bit rate is greater than the real-time target bit rate after exceeding the target D frame interval, it means that the bit rate control still cannot adapt to the current network conditions, then continue to use resolution control, that is, reduce the original resolution of the current video frame to obtain the target resolution.

Step 409, execute the control scheme in the scene where the video content is complex and the code rate increases suddenly. At this point, gradually increase the real-time consumption bit rate within the target D frame interval. If the real-time consumption bit rate is greater than the real-time target bit rate after exceeding the target D frame interval, it means that the bit rate regulation can adapt to the current network conditions. At this time, the original resolution of the current video frame is determined as the target resolution. If the real-time consumption bit rate is lower than the real-time target bit rate after exceeding the target D frame interval, it means that the bit rate control still cannot adapt to the current network conditions, and then continue to use resolution control, that is, increase the original resolution of the current video frame to obtain the target resolution.

Step 410, execute the control scheme in the scenario where the video content is complex and the code rate drops suddenly. At this point, the resolution is directly regulated, that is, the original resolution of the current video frame is reduced to obtain the target resolution.

Step 411, adopting the code table scheme to regulate the resolution. In the case of a scene change, the control schemes in steps 405 to 410 above are no longer applicable. At this time, the code table scheme is adopted to select the resolution corresponding to the real-time target bit rate as the target resolution. At the same time, this step clears the code rate buffer buffer_R maintained in the above step 402 and the quantization parameter value buffer buffer_QP maintained in the above step 403 to zero.

Step 412, encode the current video frame with the target resolution. The video encoding end scales the current video frame based on the target resolution, and after the resolution of the current video frame matches the target resolution, the current video frame is input to the encoder for video encoding. At the same time, the video encoding end updates the frame-level buffer buffer_pre maintained in step 404 above.

The following are device embodiments of the present application, which can be used to implement the method embodiments of the present application. For details not disclosed in the device embodiments of the present application, please refer to the method embodiments of the present application.

Please refer to FIG. 5 , which shows a block diagram of a video coding apparatus provided by an embodiment of the present application. The device has the function of realizing the above example of the video coding method, and the function may be realized by hardware, or may be realized by executing corresponding software by the hardware. The device may be the above-mentioned video coding end, or may be set in the above-mentioned video coding end. The apparatus 500 may include: a code rate information acquisition module 510 , a parameter information acquisition module 520 , a resolution determination module 530 and a video encoding module 540 .

The code rate information obtaining module 510 is configured to obtain code rate change information corresponding to the current network condition, and the code rate change information is used to indicate the fluctuation of the current network condition relative to the historical network condition.

The parameter information obtaining module 520 is configured to obtain parameter change information corresponding to the current video content, and the parameter change information is used to indicate the content complexity of the current video content.

A resolution determining module 530, configured to determine a target resolution based on the code rate change information and the parameter change information.

A video encoding module 540, configured to encode the current video frame by using the target resolution.

In an example, the code rate information acquisition module 510 is configured to: determine a real-time target code rate, the real-time target code rate is used to indicate the current network condition; acquire a historical average code rate, the historical average code rate It is used to indicate the historical network condition; and determine the code rate change information based on the real-time target code rate and the historical average code rate.

In an example, the determining the code rate change information based on the real-time target code rate and the historical average code rate includes: determining the code rate between the real-time target code rate and the historical average code rate Rate difference; when the absolute value of the code rate difference is greater than the first threshold and the code rate difference is a positive number, it is determined that the code rate change information includes a drastic change in the code rate, and the code rate Sudden increase; when the absolute value of the code rate difference is greater than the first threshold and the code rate difference is negative, determining that the code rate change information includes a sharp change in the code rate, and a sudden drop in the code rate ; In a case where the absolute value of the code rate difference is smaller than a first threshold, determining that the code rate change information includes that the code rate has not changed drastically.

In an example, the parameter information acquiring module 520 is configured to: acquire a first quantization parameter value, where the first quantization parameter value refers to the quantization parameter value of the previous video frame of the current video frame; acquire the second A quantization parameter value, the second quantization parameter value refers to the mean value of the quantization parameter value of at least one historical video frame; based on the first quantization parameter value and the second quantization parameter value, the parameter change information is determined.

In an example, the determining the parameter change information based on the first quantization parameter value and the second quantization parameter value includes: determining the difference between the first quantization parameter value and the second quantization parameter value The parameter difference between; in the case where the absolute value of the parameter difference is greater than the second threshold, it is determined that the parameter change information indicates that the video content is complex; in the case where the absolute value of the parameter difference is smaller than the second threshold , it is determined that the parameter change information indicates that the video content is simple.

In an example, the resolution determination module 530 is configured to: step by step within the target D frame interval when the code rate change information includes a sudden increase in code rate, and the parameter change information indicates that the video content is simple Increase the real-time consumption code rate; If the real-time consumption code rate is greater than the real-time target code rate after exceeding the target D frame interval, then determine the original resolution of the current video frame as the target resolution; if the The real-time consumption code rate is less than the real-time target code rate after exceeding the target D frame interval, then increasing the original resolution of the current video frame to obtain the target resolution; when the code rate change information includes a code rate drop, And when the parameter change information indicates that the video content is simple, gradually reduce the real-time consumption code rate within the target D frame interval; if the real-time consumption code rate is less than the real-time target code rate after exceeding the target D frame interval, then The original resolution of the current video frame is determined as the target resolution; if the real-time consumption bit rate is greater than the real-time target bit rate after exceeding the target D frame interval, then reduce the original resolution of the current video frame rate to obtain the target resolution; when the code rate change information includes a sudden increase in the code rate, and the parameter change information indicates that the video content is complex, gradually increase the real-time consumption code rate within the target D frame interval; if the The real-time consumption code rate is greater than the real-time target code rate after exceeding the target D frame interval, then the original resolution of the current video frame is determined as the target resolution; if the real-time consumption code rate exceeds the described After the target D frame interval is less than the real-time target bit rate, then increase the original resolution of the current video frame to obtain the target resolution; the bit rate change information includes a bit rate drop, and the parameter change information indicates video In the case of complex content, reduce the original resolution of the current video frame to obtain the target resolution.

In an example, the resolution determination module 530 is further configured to: determine the original resolution of the current video frame as the target when the code rate change information includes that the code rate has not changed drastically resolution.

In an example, the video encoding module 540 is configured to: perform upsampling processing on the current video frame when the target resolution is greater than the original resolution of the current video frame; When the resolution of the current video frame after the upsampling process matches the target resolution, use the target resolution to encode the current video frame after the upsampling process; When the target resolution is smaller than the original resolution of the current video frame, downsampling is performed on the current video frame; the resolution of the current video frame after the downsampling process is different from the target resolution In the case of rate matching, the target resolution is used to encode the current video frame after the downsampling process.

In an example, as shown in FIG. 6 , the apparatus 500 further includes: a scene change detection module 550, configured to detect a scene change of the current video content relative to historical video content; the resolution determination module 530, used The target resolution is determined based on the code rate change information and the parameter change information when the current video content has no scene change relative to the historical video content.

In one example, as shown in FIG. 6, the scene change detection module 550 is configured to: determine at least one key block from the current video frame; for the first key block in the at least one key block, The position corresponding to the first key block in the previous video frame of the current video frame is searched for matching blocks with the target radius as the search radius; if there is a distance between the searched block and the first key block If the mean square error is less than the target threshold, it is determined that the first key block has a matching block in the previous video frame of the current video frame; wherein, the at least one key block is in the previous video frame of the current video frame If there are matching blocks in a video frame, it is determined that a scene change occurs in the current video content relative to the historical video content.

In an example, the resolution determination module 530 is further configured to: in the case that the scene of the current video content changes relative to the historical video content, the resolution corresponding to the real-time target bit rate in the target correspondence rate, determined as the target resolution; wherein, the target correspondence includes at least one set of correspondences between target code rates and resolutions.

It should be noted that when the device provided by the above-mentioned embodiments implements its functions, it only uses the division of the above-mentioned functional modules as an example. In practical applications, the above-mentioned function allocation can be completed by different functional modules according to needs. The internal structure of the system is divided into different functional modules to complete all or part of the functions described above. In addition, the device and the method embodiment provided by the above embodiment belong to the same idea, and the specific implementation process thereof is detailed in the method embodiment, and will not be repeated here.

In an exemplary embodiment, there is also provided a computer device, the computer device includes a processor and a memory, and a computer program is stored in the memory, and the computer program is loaded and executed by the processor to realize the above-mentioned Video encoding method.

In an exemplary embodiment, there is also provided a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the above-mentioned video encoding method is implemented.

In an exemplary embodiment, a computer program product is also provided, which, when the computer program product is run on a computer device, causes the computer device to execute the above video encoding method.

The above are only exemplary embodiments of the application, and are not intended to limit the application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the application shall be included in the protection of the application. within range.

Claims

A video encoding method, characterized in that the method comprises:

Obtain code rate change information corresponding to current network conditions, where the code rate change information is used to indicate fluctuations of the current network conditions relative to historical network conditions;

Acquire parameter change information corresponding to the current video content, where the parameter change information is used to indicate the content complexity of the current video content;

Determine a target resolution based on the code rate change information and the parameter change information;

Encoding the current video frame by using the target resolution.
The method according to claim 1, wherein said obtaining code rate change information corresponding to current network conditions comprises:

determining a real-time target code rate, where the real-time target code rate is used to indicate the current network condition;

Obtaining a historical average code rate, where the historical average code rate is used to indicate the historical network condition;

Determine the code rate change information based on the real-time target code rate and the historical average code rate.
The method according to claim 2, wherein the determining the code rate change information based on the real-time target code rate and the historical average code rate includes:

determining the code rate difference between the real-time target code rate and the historical average code rate;

When the absolute value of the code rate difference is greater than a first threshold and the code rate difference is a positive number, determining that the code rate change information includes a drastic change in the code rate and a sudden increase in the code rate;

When the absolute value of the code rate difference is greater than a first threshold and the code rate difference is a negative number, determining that the code rate change information includes a drastic change in the code rate and a sudden drop in the code rate;

In a case where the absolute value of the code rate difference is smaller than the first threshold, it is determined that the code rate change information includes that the code rate does not change drastically.
The method according to claim 1, wherein said obtaining parameter change information corresponding to the current video content comprises:

Obtaining a first quantization parameter value, where the first quantization parameter value refers to the quantization parameter value of the previous video frame of the current video frame;

Obtaining a second quantization parameter value, where the second quantization parameter value refers to the mean value of the quantization parameter values of at least one historical video frame;

The parameter change information is determined based on the first quantization parameter value and the second quantization parameter value.
The method according to claim 4, wherein the determining the parameter change information based on the first quantization parameter value and the second quantization parameter value comprises:

determining a parameter difference between the first quantization parameter value and the second quantization parameter value;

When the absolute value of the parameter difference is greater than a second threshold, it is determined that the parameter change information indicates that the video content is complex;

In a case where the absolute value of the parameter difference is smaller than the second threshold, it is determined that the parameter change information indicates that the video content is simple.
The method according to claim 1, wherein the determining the target resolution based on the code rate change information and the parameter change information includes:

When the code rate change information includes a sudden increase in the code rate, and the parameter change information indicates that the video content is simple, gradually increase the real-time consumption code rate within the target D frame interval; if the real-time consumption code rate exceeds the specified After the target D frame interval is greater than the real-time target code rate, then the original resolution of the current video frame is determined as the target resolution; if the real-time consumption code rate is less than the real-time target after exceeding the target D frame interval code rate, then increase the original resolution of the current video frame to obtain the target resolution;

When the code rate change information includes a sudden drop in the code rate, and the parameter change information indicates that the video content is simple, gradually reduce the real-time consumption code rate within the target D frame interval; if the real-time consumption code rate exceeds the specified After the target D frame interval is less than the real-time target code rate, then the original resolution of the current video frame is determined as the target resolution; if the real-time consumption code rate is greater than the real-time target after exceeding the target D frame interval bit rate, then reduce the original resolution of the current video frame to obtain the target resolution;

When the code rate change information includes a sudden increase in the code rate, and the parameter change information indicates that the video content is complex, gradually increase the real-time consumption code rate within the target D frame interval; if the real-time consumption code rate exceeds the specified After the target D frame interval is greater than the real-time target code rate, then the original resolution of the current video frame is determined as the target resolution; if the real-time consumption code rate is less than the real-time target after exceeding the target D frame interval code rate, then increase the original resolution of the current video frame to obtain the target resolution;

If the code rate change information includes a sudden drop in code rate, and the parameter change information indicates complex video content, reduce the original resolution of the current video frame to obtain the target resolution.
The method according to claim 6, further comprising:

If the code rate change information includes that the code rate has not changed drastically, the original resolution of the current video frame is determined as the target resolution.
The method according to claim 1, wherein said encoding said current video frame using said target resolution comprises:

When the target resolution is greater than the original resolution of the current video frame, upsampling is performed on the current video frame; the resolution of the current video frame after the upsampling process is the same as the In the case where the target resolution matches, the target resolution is used to encode the current video frame after the upsampling process;

When the target resolution is smaller than the original resolution of the current video frame, downsampling is performed on the current video frame; the resolution of the current video frame after the downsampling process is the same as the If the target resolution matches the target resolution, the current video frame after the downsampling process is encoded by using the target resolution.
The method according to any one of claims 1 to 8, further comprising:

Detecting scene changes of the current video content relative to historical video content;

In a case where the current video content has no scene change relative to the historical video content, the target resolution is determined based on the code rate change information and the parameter change information.
The method according to claim 9, wherein the detecting the scene change of the current video content relative to the historical video content comprises:

determining at least one key block from the current video frame;

For the first key block in the at least one key block, at the position corresponding to the first key block in the previous video frame of the current video frame, perform a matching block search with the target radius as the search radius;

If there is a block whose mean square error with the first key block is smaller than the target threshold in the searched block, then it is determined that the first key block has a matching block in the previous video frame of the current video frame;

Wherein, in a case where the at least one key block has a matching block in the previous video frame of the current video frame, it is determined that a scene change occurs in the current video content relative to the historical video content.
The method according to claim 9, characterized in that the method further comprises:

In the case where the current video content has a scene change relative to the historical video content, the resolution corresponding to the real-time target bit rate in the target correspondence is determined as the target resolution;

Wherein, the target correspondence includes at least one set of correspondences between target code rates and resolutions.
A video encoding device, characterized in that the device comprises:

A code rate information acquisition module, configured to obtain code rate change information corresponding to current network conditions, where the code rate change information is used to indicate fluctuations of the current network conditions relative to historical network conditions;

A parameter information acquisition module, configured to acquire parameter change information corresponding to the current video content, where the parameter change information is used to indicate the content complexity of the current video content;

A resolution determination module, configured to determine a target resolution based on the code rate change information and the parameter change information;

A video coding module, configured to use the target resolution to code the current video frame.
A computer device, characterized in that the computer device includes a processor and a memory, and a computer program is stored in the memory, and the computer program is loaded and executed by the processor to implement any one of claims 1 to 11. The video coding method described in the item.
A computer-readable storage medium, on which a computer program is stored, wherein, when the computer program is executed by a processor, the video coding method according to any one of claims 1 to 11 is implemented.
A computer program product, characterized in that, when the computer program product is run on a computer device, the computer device is made to execute the video coding method according to any one of claims 1 to 11.