WO2023174216A1

WO2023174216A1 - Video resolution adaptive adjustment method, apparatus and device, and storage medium

Info

Publication number: WO2023174216A1
Application number: PCT/CN2023/081121
Authority: WO
Inventors: 陈俊奎; 谢澜; 陈新泽
Original assignee: 百果园技术(新加坡)有限公司; 陈俊奎
Priority date: 2022-03-18
Filing date: 2023-03-13
Publication date: 2023-09-21
Also published as: CN114827662A

Abstract

Provided in the embodiments of the present application are a video resolution adaptive adjustment method, apparatus and device, and a storage medium. The method comprises: acquiring the number of lost frames of a video image and consumed processing time for a single image frame; calculating a maximum processing frame rate according to the consumed processing time, and on the basis of the number of lost frames and the maximum processing frame rate, determining state information of a video frame, the state information comprising performance overload and performance excess; and if the state information is performance overload, reducing the resolution of the video image, and if the state information is performance excess, increasing the resolution of the video image. The present solution achieves adaptive adjustment of video resolution and weighs and optimizes a balance between the definition and smoothness during playing a video, so that high-quality video experience is provided for a user.

Description

Video resolution adaptive adjustment method, device, equipment and storage medium

This application claims priority to the Chinese patent application with application number 202210272513.2, which was submitted to the China Patent Office on March 18, 2022. The entire content of this application is incorporated into this application by reference.

Technical field

Embodiments of the present application relate to the field of video processing technology, and in particular, to a video resolution adaptive adjustment method, device, equipment and storage medium.

Background technique

With the development of computer and network technology, there are more and more various applications based on video images. For example, in a real-time live broadcast scenario, the anchor side collects video images through the camera, pre-processes the collected video images, and then renders and displays them on the screen of the anchor side. For the audience side, the anchor side encodes the pre-processed video images and packages them into protocol packets, and forwards them to the audience side through the backend server. The audience side decodes and renders them to the audience side's screen for display.

During the pre-processing and encoding of video images, a large number of processor operations and I/O operations are required. As user scenarios change, such as adjusting video special effects, beautification processing, etc., the device will perform single image processing. Frame processing capabilities change. For example, if special effects, beautification, or equipment heats up and frequency decreases as the duration of the live broadcast increases, the processor will take longer to process a single frame of the image, resulting in frame drops and freezes. When the special effects and beautification are reduced, the time taken by the processor to process a single frame of the image will be reduced, frame loss will be alleviated, and the smoothness of video playback will be improved. Therefore, how to effectively achieve the balance between video clarity and smoothness is an urgent problem that needs to be solved.

Contents of the invention

Embodiments of the present application provide a video resolution adaptive adjustment method, device, equipment and storage medium, which solves the problem of poor balance between clarity and fluency during video playback, realizes adaptive adjustment of video resolution, and provides Users are provided with a high-quality video experience.

In a first aspect, embodiments of the present application provide a video resolution adaptive adjustment method, which method includes:

Obtain the number of dropped frames of video images and the processing time of single frame images;

The maximum processing frame rate is calculated according to the processing time consumption, and the status information of the video frame is determined based on the number of lost frames and the maximum processing frame rate, where the status information includes performance overload and excess performance;

If the status information indicates performance overload, then lower the resolution of the video image;

If the status information indicates excess performance, the resolution of the video image is increased.

In a second aspect, embodiments of the present application also provide a video resolution adaptive adjustment device, including:

The data acquisition module is configured to obtain the number of dropped frames of the video image and the processing time of a single frame image;

A frame rate calculation module configured to calculate the maximum processing frame rate based on the processing time;

A status information determination module configured to determine status information of the video frame based on the number of lost frames and the maximum processing frame rate, where the status information includes performance overload and excess performance;

A resolution adjustment module configured to lower the resolution of the video image if the status information indicates performance overload, and to increase the resolution of the video image if the status information indicates excess performance.

In a third aspect, embodiments of the present application also provide a video resolution adaptive adjustment device, which includes:

one or more processors;

a storage device for storing one or more programs,

When the one or more programs are executed by the one or more processors, the one or more processors are caused to implement the video resolution adaptive adjustment method described in the embodiments of this application.

In a fourth aspect, embodiments of the present application also provide a storage medium that stores computer-executable instructions, which when executed by a computer processor are used to perform the video resolution adaptation described in the embodiments of the present application. Adjustment method.

In a fifth aspect, embodiments of the present application also provide a computer program product. The computer program product includes a computer program. The computer program is stored in a computer-readable storage medium. At least one processor of the device reads the computer program from the computer-readable storage medium. Obtain and execute the computer program, causing the device to execute the video resolution adaptive adjustment method described in the embodiments of this application.

In the embodiment of the present application, the number of dropped frames of the video image and the processing time of a single frame image are obtained. After calculating the maximum processing frame rate based on the processing time, the status information of the video frame is determined based on the number of dropped frames and the maximum processing frame rate. , the status information includes performance overload and excess performance. If the status information is performance overload, the resolution of the video image is adjusted downward. If the status information is excess performance, the resolution is adjusted upward. Adjusting the resolution of the video image achieves adaptive adjustment of the video resolution, weighs and optimizes the balance between clarity and smoothness during video playback, and provides users with a high-quality video experience.

Description of the drawings

Figure 1 is a flow chart of a video resolution adaptive adjustment method provided by an embodiment of the present application;

Figure 2 is a flow chart of a method for determining status information of a video frame provided by an embodiment of the present application;

Figure 3 is a flow chart of another video resolution adaptive adjustment method provided by an embodiment of the present application;

Figure 4 is a flow chart of another video resolution adaptive adjustment method provided by an embodiment of the present application;

Figure 5 is a structural block diagram of a video resolution adaptive adjustment device provided by an embodiment of the present application;

Figure 6 is a schematic structural diagram of a video resolution adaptive adjustment device provided by an embodiment of the present application.

Detailed ways

The embodiments of the present application will be further described in detail below with reference to the accompanying drawings and examples. It can be understood that the specific embodiments described here are only used to explain the embodiments of the present application, but are not intended to limit the embodiments of the present application. In addition, it should be noted that, for convenience of description, only some but not all structures related to the embodiments of the present application are shown in the drawings.

The terms "first", "second", etc. in the description and claims of this application are used to distinguish similar objects and are not used to describe a specific order or sequence. It is to be understood that the figures so used are interchangeable under appropriate circumstances so that the embodiments of the present application can be practiced in orders other than those illustrated or described herein, and that "first," "second," etc. are distinguished Objects are usually of one type, and the number of objects is not limited. For example, the first object can be one or multiple. In addition, "and/or" in the description and claims indicates at least one of the connected objects, and the character "/" generally indicates that the related objects are in an "or" relationship.

Figure 1 is a flow chart of a video resolution adaptive adjustment method provided by an embodiment of the present application, which can be used to adaptively adjust the resolution of video images. This method can be performed by computing devices such as servers, smart terminals, and laptops. , tablet computers, etc. to execute, including the following steps:

Step S101: Obtain the number of dropped frames of the video image and the processing time of a single frame image.

In one embodiment, the number of lost frames is the number of lost frames during the encoding process of the video image. Specifically, in the process of encoding each frame of the video image, the video image passes through After processing, it is sent to the encoding queue to be compressed and encoded by the encoder. The pre-processing process includes a series of processing processes such as data cleaning, skin color processing, skin grinding, noise reduction and face positioning for each frame of image collected by the camera. Usually the size of the encoding queue is a fixed length, that is, when the speed of encoding each frame of video image is less than the speed of the image input to the encoding queue, the excess image frames will overflow, causing frame loss and freezing.

In one embodiment, the number of lost frames can be obtained by counting the number of frames input to the encoding queue and the number of encoding processing frames. For example, the number of frames input to the encoding queue within a set fixed time minus the number of encoding processing frames can be obtained. The number of dropped frames.

In one embodiment, the processing time of the single frame image may be the processing time of each frame of the video image during the encoding process. It is calculated by the processing time of the acquired single frame image. Optionally, the calculation process includes: obtaining the processing time of a single frame image, and calculating the processing time of the single frame image based on the processing time and the set motion weighted exponential filtering formula. Specifically, the calculation formula is as follows:

Among them, t _i is the processing time consumed by the actual encoding process of the acquired i-th frame, is the processing time of a single frame image estimated last time, It is the calculated processing time of a single frame image, α is the motion weighting factor, and the exemplary value is 0.996. The estimation of the processing time of a single frame image calculated in this way is more accurate, which satisfies the characteristics of fast rise and slow decline, and avoids the problem of inaccurate calculation results of processing time due to jitter.

Step S102: Calculate the maximum processing frame rate according to the processing time, and determine the status information of the video frame based on the number of lost frames and the maximum processing frame rate. The status information includes performance overload and excess performance.

In one embodiment, the processing time is the processing time of each frame of image during the video image encoding process, and the maximum processing frame rate corresponds to the calculated maximum processing frame rate during the encoding process. Specifically, it can be: calculating the number of image frames processed per unit time according to the processing time to obtain the maximum processing frame rate. For example, the maximum processing frame rate is recorded as fps _{enc_capacity} , The calculated processing time of a single frame image is:

That is, the maximum processing frame rate is 20 frames per second.

In one embodiment, the status information of the video frame is determined based on the number of lost frames and the maximum processing frame rate obtained previously. The status information includes performance overload and excess performance during the encoding process. Among them, performance overload means that the current processing speed of video images cannot meet the speed of the input video images, that is, the image resolution needs to be reduced to ensure smooth playback of the video, while overperformance means that the current processing speed of video images is higher than the input speed. The speed of the video image can increase the image resolution to improve the clarity of the displayed video image.

In one embodiment, as shown in Figure 2, Figure 2 is a flow chart of a method for determining status information of a video frame provided by an embodiment of the present application, which specifically includes:

Step S1021: Determine whether the processing of video frames is performance overload based on the number of lost frames and the calculated frame loss threshold, as well as the maximum processing frame rate and the set standard frame rate threshold.

In one embodiment, the frame loss threshold is a dynamically determined threshold for comparison with the number of frames lost; the standard frame rate threshold is a set value. For example, the standard frame rate threshold can be 11, 15, or 20, etc., and can be flexibly set according to different video application scenarios.

In one embodiment, the step of calculating the frame loss threshold is also included. Optionally, the frame loss threshold is calculated based on different image resolutions. Specifically: the frame loss threshold is calculated based on the image resolution corresponding to the current gear and the lowered gear and the maximum processing frame rate, where the image resolution corresponding to the lowered gear is smaller than the image resolution corresponding to the current gear. Optionally, when adjusting the image resolution, multiple different image resolution gears are preset. Each image resolution gear corresponds to a fixed image resolution value. If there are six gears, the corresponding Exemplary image resolutions are 240P, 360P, 480P, 540P, 720P and 1080P respectively. The frame drop threshold is exemplarily recorded as Th _{enc_drop} , and the frame drop threshold is calculated as follows:

Among them, fps _{enc_capacity} is the maximum processing frame rate, i represents the image resolution level, the larger the value of i, the higher the corresponding image resolution. Among them, pixel _i represents the number of pixels under the image resolution corresponding to the i gear, where pixel _i = _wi * _hi , w _i and h _i are the width and height of the image respectively.

In one embodiment, the specific process of determining whether the processing of video frames is performance overload based on the number of lost frames and the calculated frame loss threshold, as well as the maximum processing frame rate and the set standard frame rate threshold may be: If the number of lost frames is greater than the calculated frame loss threshold, and the maximum processing frame rate is less than the set standard frame rate threshold, it is determined that the processing of video frames is performance overload. Code _coveruse represents performance overload, fps _{enc_capacity} is the maximum processing frame rate, Th _{enc_fps} is the set standard frame rate threshold, frames _{enc_drop} is the number of dropped frames, and Th _{enc_drop} is the calculated ’s frame loss threshold, the specific judgment conditions can be expressed as:

Step S1022: Determine whether the processing of the video frame has excessive performance based on the maximum processing frame rate and the calculated maximum frame rate threshold.

In one embodiment, when determining excess performance, the maximum processing frame rate is compared with the calculated maximum frame rate threshold. Specifically, the process of determining the maximum frame rate threshold before comparison includes: calculating the maximum frame rate threshold based on the image resolution corresponding to the current gear and the upward gear and the set maximum frame rate, where the upper gear The image resolution corresponding to the gear position is greater than the image resolution corresponding to the current gear position. Optionally, when adjusting the image resolution, multiple different image resolution gears are preset. Each image resolution gear corresponds to a fixed image resolution value. For example, if there are six gears, the gears are The corresponding image resolutions from low to high are, for example, 240P, 360P, 480P, 540P, 720P and 1080P respectively. For example, the set maximum frame rate is recorded as fps _max , and the exemplary value may be 24, where the maximum frame rate threshold is calculated based on the image resolution corresponding to the current gear and the upward gear and the set maximum frame rate. The process can be:

Among them, pixel _i represents the number of pixels at the image resolution corresponding to the i-th gear, where pixel _i = _wi * _hi , w _i and h _i are the width and height of the image respectively. The corresponding conditions for satisfying excess performance include: if the maximum processing frame rate is greater than the calculated maximum frame rate threshold, then the processing of the video frame is determined to be excess performance, and Codec _underuse represents excess performance. The corresponding judgment conditions can be expressed as follows:

Step S103: If the status information indicates performance overload, adjust the resolution of the video image downward; if the status information indicates excessive performance, adjust the resolution of the video image upward.

In one embodiment, after determining the status information, if the status information is performance overload, then Adjust the resolution of the video image. If the status information indicates excess performance, increase the resolution of the video image. Regarding the encoding processing stage of the video image, if it is determined that the performance is overloaded, the current encoding resolution is lowered, and if it is determined that the performance is excessive, the current encoding resolution is increased.

In one embodiment, multiple different image resolution gears are set for upward and downward adjustment of the image resolution, where each gear corresponds to a fixed resolution value. Optionally, when adjusting the resolution of the video image, adjust one gear at a time.

In one embodiment, if the determined status information is neither excessive performance nor performance overload, the corresponding determined status information is a normal status.

Optionally, by collecting status information, the status information is judged every fixed period T. If the same status information is triggered cumulatively N times in the period T, for example, the judgment of performance overload is triggered 6 times within a period of 10 seconds. If so, adjust the image resolution accordingly. If the above conditions are not met, the resolution will not be adjusted. Specifically, when the video image resolution in the encoding stage is increased, in principle it should not exceed the highest image resolution captured by the camera device. Of course, the downward adjustment process is also limited by the image resolution corresponding to the lowest gear set.

In one embodiment, before adjusting the video image resolution, especially the resolution of the camera image acquisition is limited by the model, it may only provide image acquisition at certain resolutions, and the pre-processing or encoding process is set as needed. When fixing several resolution types of width and height, before obtaining the number of dropped frames of the video image and the processing time of a single frame image, it also includes: cropping the video image collected by the camera device through the image cropping algorithm to obtain different gears video images at lower resolutions. That is, the image is cropped through the image cropping algorithm to obtain multiple images with different resolutions for adjusting the image resolution.

It can be seen from the above solution that by obtaining the number of dropped frames of the video image and the processing time of a single frame image, and calculating the maximum processing frame rate based on the processing time, the status information of the video frame is determined based on the number of dropped frames and the maximum processing frame rate. The status information includes performance overload and excess performance. If the status information indicates performance overload, the resolution of the video image is lowered. If the status information indicates excess performance, the resolution of the video image is raised. This is achieved The adaptive adjustment of video resolution balances and optimizes the balance between clarity and smoothness during video playback, providing users with a high-quality video experience.

In the specific implementation process, the processing time of a single frame image is calculated through the moving weighted exponential filtering formula, making the calculation of the processing time more accurate. By using the number of dropped frames, the processing time of a single frame image The processing time and the calculated maximum processing frame rate are used as the basis for judging the status information. The image resolution can be effectively adjusted based on this. During the specific adjustment process, dynamic calculation is used for the frame loss threshold and the maximum frame rate threshold. Obtained, the effect of resolution adaptive adjustment is greatly optimized.

Figure 3 is a flow chart of another video resolution adaptive adjustment method provided by an embodiment of the present application. It provides a process of video resolution adjustment in the encoding stage, as shown in Figure 3, which specifically includes:

Step S201: Obtain the processing time of a single frame image in the video image encoding processing stage, and calculate the processing time of the single frame image according to the processing time and the set motion weighted exponential filter formula.

Step S202: Obtain the number of lost frames in the video image encoding processing stage and the processing time of the single frame image, and calculate the number of image frames processed per unit time according to the processing time to obtain the maximum encoding processing frame rate.

Step S203: Calculate the frame loss threshold based on the image resolution corresponding to the current gear and the lowered gear and the maximum encoding frame rate, calculate the frame loss threshold based on the number of frames lost, the maximum encoding frame rate and the set standard The frame rate threshold determines whether the encoding process of video frames is a performance overload.

Step S204: Calculate the maximum frame rate threshold based on the image resolution corresponding to the current gear and the upward gear and the set maximum frame rate, and determine whether the video frame is encoded based on the maximum encoding processing frame rate and the calculated maximum frame rate threshold. For excess performance.

Step S205: If the number of lost frames is greater than the calculated frame loss threshold and the maximum encoding processing frame rate is less than the set standard frame rate threshold, it is determined that the encoding processing of the video frame is performance overload. If the maximum encoding processing frame rate is greater than the calculated frame rate If the maximum frame rate threshold is reached, it is determined that the encoding process of the video frame has excessive performance.

Figure 4 is a flow chart of another video resolution adaptive adjustment method provided by the embodiment of the present application, which provides the process of video resolution adjustment in the previous processing stage, as shown in Figure 4, which specifically includes:

Step S301: Obtain the number of dropped frames in the video image pre-processing stage and the processing time of a single frame image.

In one embodiment, a pre-processing queue is also maintained when pre-processing video images. Each frame of video image collected by the camera is input into the pre-processing queue in turn. During pre-processing, each frame of image in the pre-processing queue is processed in sequence. Taking the live broadcast scene as an example, it includes a series of processing processes such as data cleaning, skin color processing, skin resurfacing, noise reduction, and face positioning. If the speed of image pre-processing is lower than the speed of the input image, frame loss will occur.

In one embodiment, the number of lost frames in the pre-processing stage can be obtained by counting the number of frames input into the pre-processing queue and the number of frames processed in the pre-processing, such as subtracting the number of frames input into the pre-processing queue within a set fixed time. The specific number of frames lost can be obtained by removing the number of frames processed in pre-processing.

In one embodiment, the processing time of a single frame image in the pre-processing stage of the video image is the determined time-consuming of the single acquisition frame in the entire pre-processing stage. The calculation method can refer to the processing time of a single frame image in the encoding stage, which is also calculated based on the motion weighted exponential filtering formula and will not be described again here.

Step S302: Calculate the maximum pre-processing frame rate based on the processing time, and determine the status information of the video frame based on the number of lost frames and the maximum pre-processing frame rate.

Among them, the calculation method of obtaining the maximum pre-processing frame rate based on the processing time consumption is referred to the calculation method of obtaining the maximum encoding processing frame rate based on the processing time consumption. At the same time, the status information of the video frame is determined based on the number of dropped frames and the maximum pre-processing frame rate. The process can also refer to the process of determining the status information of the video frame based on the number of dropped frames and the maximum encoding processing frame rate at the same time, that is, the explanation part of step S102, which will not be described again here.

Step S303: If the status information indicates performance overload, lower the pre-processing resolution of the video image. If the status information indicates excessive performance, increase the pre-processing resolution of the video image.

For the specific resolution adjustment method, please refer to the explanation part of step S103 and will not be described again here.

On the basis of the above technical solutions, the processes of adaptive adjustment of image resolution for the pre-processing stage and encoding processing stage are respectively given. The two stages can adjust the image resolution independently, or use linked Adjust the image resolution in the form. Optionally, when the image resolution is lowered to reduce the image resolution in the pre-processing stage, the encoding processing stage also triggers a downward adjustment of the resolution to be consistent with the resolution lowered in the pre-processing; the current processing stage performs an upward adjustment of the image resolution. When increasing the image resolution, the encoding processing stage also triggers a resolution increase to be consistent with the resolution increase in pre-processing. It should be noted that the calculation formula of the above encoding processing and the description of the specific resolution adjustment method are also applicable to the pre-processing stage, and will not be described again here.

It can be seen from the above solution that by adaptively adjusting the image resolution in the pre-processing stage and encoding processing stage of the video image, the adaptive adjustment of the video resolution is achieved, and the balance of clarity and smoothness during video playback is achieved. Trade-off optimizations have been made to provide users with a high-quality video experience.

Figure 5 is a structural block diagram of a video resolution adaptive adjustment device provided by an embodiment of the present application. The device is used to execute the video resolution adaptive adjustment method provided by the above embodiment, and has functional modules and beneficial effects corresponding to the execution method. . As shown in Figure 5, the device specifically includes: data acquisition module 101, frame rate calculation module 102, status information determination module 103 and resolution adjustment module 104, wherein,

The data acquisition module 101 is configured to obtain the number of dropped frames of the video image and the processing time of the single frame image;

The frame rate calculation module 102 is configured to calculate the maximum processing frame rate based on the processing time;

The status information determination module 103 is configured to determine the status information of the video frame based on the number of lost frames and the maximum processing frame rate, where the status information includes performance overload and excess performance;

The resolution adjustment module 104 is configured to lower the resolution of the video image if the status information indicates performance overload, and to increase the resolution of the video image if the status information indicates excessive performance.

In a possible embodiment, the data acquisition module 101 is configured as:

Before obtaining the number of dropped frames of the video image and the processing time of a single frame image, obtain the processing time of a single frame image;

The device also includes a module for processing time-consuming calculations, configured to:

The processing time of a single frame image is calculated based on the processing time and the set motion weighted exponential filtering formula.

In a possible embodiment, the frame rate calculation module 102 is configured as:

Calculate the number of image frames processed per unit time based on the processing time to obtain the maximum processing frame rate.

In a possible embodiment, the status information determination module 103 is configured as:

Determine whether the processing of video frames is performance overload based on the number of lost frames and the calculated frame loss threshold, as well as the maximum processing frame rate and the set standard frame rate threshold;

Determine whether the processing of the video frame has excessive performance based on the maximum processing frame rate and the calculated maximum frame rate threshold.

In a possible embodiment, the status information determination module 103 is further configured to:

Before determining whether the processing of video frames is performance overload based on the number of lost frames and the set frame loss threshold, as well as the maximum processing frame rate and the calculated standard frame rate threshold, based on the current gear and the lowered gear corresponding to The frame loss threshold is calculated based on the image resolution and the maximum processing frame rate, where, the The image resolution corresponding to the lowered gear is smaller than the image resolution corresponding to the current gear.

If the number of lost frames is greater than the calculated frame loss threshold and the maximum processing frame rate is less than the set standard frame rate threshold, it is determined that the processing of the video frame is performance overloaded.

Before determining whether the processing of video frames is excessive performance based on the maximum processing frame rate and the calculated maximum frame rate threshold, the maximum frame is calculated based on the image resolution corresponding to the current gear and the upward gear and the set maximum frame rate. rate threshold, wherein the image resolution corresponding to the upward gear is greater than the image resolution corresponding to the current gear.

If the maximum processing frame rate is greater than the calculated maximum frame rate threshold, it is determined that the processing of the video frame has excessive performance.

In a possible embodiment, the data acquisition module 101 is configured as:

Obtain the number of lost frames in the pre-processing stage and/or encoding processing stage of the video image and the processing time of a single frame image;

The resolution adjustment module 104 is configured as:

The resolution of the video image in the pre-processing stage and/or the encoding process stage is reduced, and the resolution of the video image in the pre-processing stage and/or the encoding process stage is increased.

In a possible embodiment, the resolution adjustment module 104 is configured as:

Before obtaining the number of dropped frames of the video image and the processing time of a single frame image, the video image collected by the camera device is cropped through an image cropping algorithm to obtain video images with resolutions in different gears.

Figure 6 is a schematic structural diagram of a video resolution adaptive adjustment device provided by an embodiment of the present application. As shown in Figure 6, the device includes a processor 201, a memory 202, an input device 203 and an output device 204; the processor in the device The number of 201 may be one or more. In Figure 6, one processor 201 is taken as an example; the processor 201, memory 202, input device 203 and output device 204 in the device may be connected through a bus or other means. In Figure 6, For example, connect via a bus. As a computer-readable storage medium, the memory 202 can be used to store software programs, computer-executable programs and modules, such as program instructions/modules corresponding to the video resolution adaptive adjustment method in the embodiment of the present application. The processor 201 executes software programs, instructions and modules stored in the memory 202 to perform various functions of the device. Application and data processing, that is, to implement the above-mentioned video resolution adaptive adjustment method. The input device 203 may be used to receive input numeric or character information and generate key signal inputs related to user settings and functional control of the device. The output device 204 may include a display device such as a display screen.

Embodiments of the present application also provide a storage medium containing computer-executable instructions, which when executed by a computer processor are used to perform a video resolution adaptive adjustment method described in the above embodiments, wherein, include:

It is worth noting that in the above embodiments of the video resolution adaptive adjustment device, the various units and modules included are only divided according to functional logic, but are not limited to the above divisions, as long as they can realize the corresponding functions. ; In addition, the specific names of each functional unit are only for the convenience of distinguishing each other, and are not used to limit the protection scope of the embodiments of the present application.

In some possible implementations, various aspects of the method provided by this application can also be implemented in the form of a program product, which includes program code. When the program product is run on a computer device, the program code is used to The computer device is caused to execute the steps in the method according to the various exemplary embodiments of the present application described above. For example, the computer device may execute the video resolution adaptive adjustment method described in the embodiment of the present application. The program product may be implemented in any combination of one or more readable media.

Claims

Video resolution adaptive adjustment method, including:

Obtain the number of dropped frames of video images and the processing time of single frame images;

The maximum processing frame rate is calculated according to the processing time consumption, and the status information of the video frame is determined based on the number of lost frames and the maximum processing frame rate, where the status information includes performance overload and excess performance;

If the status information indicates performance overload, then lower the resolution of the video image;

If the status information indicates excess performance, the resolution of the video image is increased.
The video resolution adaptive adjustment method according to claim 1, wherein before obtaining the number of dropped frames of the video image and the processing time of the single frame image, it also includes:

The processing time to obtain a single frame image;

The processing time of a single frame image is calculated based on the processing time and the set motion weighted exponential filtering formula.
The video resolution adaptive adjustment method according to claim 1 or 2, wherein the maximum processing frame rate calculated according to the processing time consumption includes:

Calculate the number of image frames processed per unit time based on the processing time to obtain the maximum processing frame rate.
The video resolution adaptive adjustment method according to any one of claims 1-3, wherein the determining the status information of the video frame based on the number of dropped frames and the maximum processing frame rate includes:

Determine whether the processing of video frames is performance overload based on the number of lost frames and the calculated frame loss threshold, as well as the maximum processing frame rate and the set standard frame rate threshold;

Determine whether the processing of the video frame has excessive performance based on the maximum processing frame rate and the calculated maximum frame rate threshold.
The video resolution adaptive adjustment method according to claim 4, wherein the video frame is determined according to the number of lost frames and the set frame loss threshold, and the maximum processing frame rate and the calculated standard frame rate threshold. Before handling whether it is a performance overload, also include:

The frame loss threshold is calculated based on the image resolution corresponding to the current gear and the lowered gear, and the maximum processing frame rate, wherein the image resolution corresponding to the lowered gear is smaller than the image resolution corresponding to the current gear.
The video resolution adaptive adjustment method according to claim 4 or 5, wherein the determination is based on the number of lost frames and the calculated frame loss threshold, and the maximum processing frame rate and the set standard frame rate threshold. Whether the processing of video frames is performance overload, including:

If the number of lost frames is greater than the calculated frame loss threshold and the maximum processing frame rate is less than the set standard frame rate threshold, it is determined that the processing of the video frame is performance overloaded.
The video resolution adaptive adjustment method according to any one of claims 4 to 6, wherein before determining whether the processing of the video frame is excessive performance based on the maximum processing frame rate and the calculated maximum frame rate threshold, Also includes:

The maximum frame rate threshold is calculated based on the image resolution corresponding to the current gear and the upward gear, and the set maximum frame rate, wherein the image resolution corresponding to the upper gear is greater than the image resolution corresponding to the current gear.
The video resolution adaptive adjustment method according to any one of claims 4 to 7, wherein said determining whether the processing of video frames is excessive performance based on the maximum processing frame rate and the calculated maximum frame rate threshold, include:

If the maximum processing frame rate is greater than the calculated maximum frame rate threshold, it is determined that the processing of the video frame has excessive performance.
The video resolution adaptive adjustment method according to any one of claims 1 to 8, wherein the acquisition of the number of dropped frames of the video image and the processing time of a single frame image include:

Obtain the number of lost frames in the pre-processing stage and/or encoding processing stage of the video image and the processing time of a single frame image;

The downward adjustment of the resolution of the video image and the upward adjustment of the resolution of the video image include:

The resolution of the video image in the pre-processing stage and/or the encoding process stage is reduced, and the resolution of the video image in the pre-processing stage and/or the encoding process stage is increased.
The video resolution adaptive adjustment method according to any one of claims 1 to 9, wherein before obtaining the number of dropped frames of the video image and the processing time of the single frame image, it also includes:

The video images collected by the camera device are cropped using an image cropping algorithm to obtain video images with resolutions in different gears.
Video resolution adaptive adjustment device, including:

The data acquisition module is configured to obtain the number of dropped frames of the video image and the processing time of a single frame image;

A frame rate calculation module configured to calculate the maximum processing frame rate based on the processing time;

A status information determination module configured to determine status information of the video frame based on the number of lost frames and the maximum processing frame rate, where the status information includes performance overload and excess performance;

A resolution adjustment module configured to lower the resolution of the video image if the status information indicates performance overload, and to increase the resolution of the video image if the status information indicates excess performance.
A video resolution adaptive adjustment device, the device includes: one or more processors; a storage device for storing one or more programs, when the one or more programs are processed by the one or more The processor is executed, so that the one or more processors implement the video resolution adaptive adjustment method according to any one of claims 1 to 10.
A storage medium that stores computer-executable instructions, which when executed by a computer processor are used to perform the video resolution adaptive adjustment method according to any one of claims 1 to 10.
A computer program product includes a computer program, wherein when the computer program is executed by a processor, the video resolution adaptive adjustment method according to any one of claims 1 to 10 is implemented.