CN117094891A - Video self-adaptive superdivision control method, device, equipment, storage medium and product - Google Patents

Video self-adaptive superdivision control method, device, equipment, storage medium and product Download PDF

Info

Publication number
CN117094891A
CN117094891A CN202311056059.8A CN202311056059A CN117094891A CN 117094891 A CN117094891 A CN 117094891A CN 202311056059 A CN202311056059 A CN 202311056059A CN 117094891 A CN117094891 A CN 117094891A
Authority
CN
China
Prior art keywords
superdivision
super
image
division
algorithm
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202311056059.8A
Other languages
Chinese (zh)
Inventor
袁子逸
崔同兵
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Guangzhou Baiguoyuan Information Technology Co Ltd
Original Assignee
Guangzhou Baiguoyuan Information Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Guangzhou Baiguoyuan Information Technology Co Ltd filed Critical Guangzhou Baiguoyuan Information Technology Co Ltd
Priority to CN202311056059.8A priority Critical patent/CN117094891A/en
Publication of CN117094891A publication Critical patent/CN117094891A/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G06T3/4053—Scaling of whole images or parts thereof, e.g. expanding or contracting based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G06T3/4046—Scaling of whole images or parts thereof, e.g. expanding or contracting using neural networks
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • Biomedical Technology (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)

Abstract

The embodiment of the application provides a video self-adaptive superminute control method, a device, equipment, a storage medium and a product. According to the technical scheme provided by the embodiment of the application, when the self-adaptive superdivision condition is met, the image superdivision processing is carried out through the first superdivision algorithm based on the deep learning, and when the first switching condition is met according to the first time-consuming information of the image superdivision processing of the image to be processed through the first superdivision algorithm, the image superdivision processing is switched to the image superdivision processing through the second superdivision algorithm based on the traditional algorithm, and the time-consuming information of the image superdivision processing of the image to be processed through different superdivision algorithms is switched between the first superdivision algorithm and the second superdivision algorithm, so that the balance between the image superdivision quality and the image superdivision efficiency of the image superdivision processing is realized, the condition that the image superdivision quality is too low or the image superdivision efficiency is too low due to the single superdivision algorithm is reduced, the video superdivision processing effect is improved, and the video watching experience of users is improved.

Description

Video self-adaptive superdivision control method, device, equipment, storage medium and product
Technical Field
The embodiment of the application relates to the technical field of image processing, in particular to a video self-adaptive superminute control method, a device, equipment, a storage medium and a product.
Background
With the rapid development of intelligent devices such as computers and mobile phones and related technologies, internet and Streaming Media technologies (Streaming Media) have gradually penetrated into various aspects of human life. The streaming media technology is a technology for compressing a series of media data, and then transmitting the compressed media data in a segmented manner in a network, so as to realize real-time transmission of video and audio on the network for viewing. In the case of poor network conditions, the video stream received by the viewer is often a low resolution, low rate video, and the quality of the video is more limited than a high resolution, high rate video. Because of the limitation of network conditions, users cannot receive high-quality streams, and the low-resolution images can be super-divided into high-resolution images by utilizing the computing power of the mobile phone end of the audience, so that the image quality of the images is improved, and better watching experience is brought to the users.
Super-resolution, i.e., super-resolution, refers to inputting a low-resolution image, and processing the input low-resolution image to obtain a high-resolution image with the same content. At present, the super-division processing of the image is generally performed based on a traditional super-division algorithm or a super-division algorithm based on a deep learning network, the traditional algorithm is used for super-division at a user side, subjective watching experience obtained by a user is improved only a limited extent, but if the user side performance is not considered, the super-division algorithm based on the deep learning network with good fixing and selecting effects can cause serious heating of the user side, and because the super-division is too high in time consumption, the user can also cause blocking when watching the video, the watching experience is affected, and the video super-division effect is poor.
Disclosure of Invention
The embodiment of the application provides a video self-adaptive superminute control method, a device, equipment, a storage medium and a product, which are used for solving the technical problem that the video superminute processing effect is poor in a fixed superminute processing mode in the related technology and effectively improving the video superminute processing effect.
In a first aspect, an embodiment of the present application provides a video adaptive superminute control method, including:
under the condition that the self-adaptive superdivision condition is met, performing image superdivision processing through a first superdivision algorithm based on deep learning;
determining first time-consuming information for performing image superdivision processing on an image to be processed through the first superdivision algorithm;
and determining whether a first switching condition is met according to the first time-consuming information, and switching to image super-division processing through a second super-division algorithm based on a traditional algorithm under the condition that the first switching condition is met.
In a second aspect, an embodiment of the present application provides a video adaptive superdivision control device, including a first superdivision module, a first statistics module, and a first switching module, where:
the first superdivision module is configured to perform image superdivision processing through a first superdivision algorithm based on deep learning under the condition that the self-adaptive superdivision condition is met;
The first statistics module is configured to determine first time-consuming information for performing image superdivision processing on an image to be processed through the first superdivision algorithm;
the first switching module is configured to determine whether a first switching condition is met according to the first time-consuming information, and switch to image super-division processing through a second super-division algorithm based on a traditional algorithm under the condition that the first switching condition is met.
In a third aspect, an embodiment of the present application provides a video adaptive superminute control device, including: a memory and one or more processors;
the memory is used for storing one or more programs;
the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the video adaptive superdistribution control method as described in the first aspect.
In a fourth aspect, embodiments of the present application provide a non-volatile storage medium storing computer executable instructions which, when executed by a computer processor, are used to perform the video adaptive superdistribution control method according to the first aspect.
In a fifth aspect, embodiments of the present application provide a computer program product comprising a computer program stored in a computer readable storage medium, the at least one processor of the device reading and executing the computer program from the computer readable storage medium, causing the device to perform the video adaptive superdistribution control method as described in the first aspect.
According to the embodiment of the application, when the self-adaptive superdivision condition is met, the image superdivision processing is carried out through the first superdivision algorithm based on the deep learning, and when the first switching condition is met according to the first time-consuming information of the image superdivision processing carried out on the image to be processed through the first superdivision algorithm, the image superdivision processing is switched to the image superdivision processing carried out through the second superdivision algorithm based on the traditional algorithm, and the time-consuming information of the image superdivision processing carried out on the image to be processed according to different superdivision algorithms is switched between the first superdivision algorithm based on the deep learning and the second superdivision algorithm based on the traditional algorithm, so that the balance between the image superdivision quality and the image superdivision efficiency of the image superdivision processing is realized, the condition that the image superdivision quality is too low or the image superdivision efficiency is too low due to the single superdivision algorithm is reduced, the video superdivision processing effect is improved, and the video watching experience of users is improved.
Drawings
Fig. 1 is a flowchart of a video adaptive superminute control method according to an embodiment of the present application;
FIG. 2 is a schematic diagram of a first switching condition determining process according to an embodiment of the present application;
FIG. 3 is a schematic diagram of a switching flow of a super-division algorithm according to an embodiment of the present application;
Fig. 4 is a schematic diagram of a second switching condition judgment flow provided in an embodiment of the present application;
FIG. 5 is a schematic diagram of a adaptive policy disabling determination process according to an embodiment of the present application;
fig. 6 is a schematic structural diagram of a video adaptive superminute control device according to an embodiment of the present application;
fig. 7 is a schematic structural diagram of a video adaptive superminute control device according to an embodiment of the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the following detailed description of specific embodiments of the present application is given with reference to the accompanying drawings. It is to be understood that the specific embodiments described herein are merely illustrative of the application and are not limiting thereof. It should be further noted that, for convenience of description, only some, but not all of the matters related to the present application are shown in the accompanying drawings. Before discussing exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although a flowchart depicts operations (or steps) as a sequential process, many of the operations can be performed in parallel, concurrently, or at the same time. Furthermore, the order of the operations may be rearranged. The above-described process may be terminated when its operations are completed, but may have additional steps not included in the drawings. The processes described above may correspond to methods, functions, procedures, subroutines, and the like.
The video self-adaptive superminute control method provided by the application can be applied to video live scenes, short video watching scenes and the like, and aims to switch between a first superminute algorithm based on deep learning and a second superminute algorithm based on the traditional algorithm according to time-consuming information of image superminute processing of images to be processed according to different superminute algorithms, so that the balance between the image superminute quality and the image superminute efficiency of the image superminute processing is realized, the situations that the image superminute quality is too low or the image superminute efficiency is too low due to a single superminute algorithm are reduced, the video superminute processing effect is improved, and the video watching experience of users is improved. In the existing video superdivision scheme, the conventional superdivision algorithm or the superdivision algorithm based on the deep learning network is generally used, if the conventional algorithm is used for superdivision at the user side, subjective watching experience obtained by the user is improved only a limited extent, but if the performance of the user side is not considered, the deep learning-based algorithm with good fixed selection effect may cause serious heating of a mobile phone of the user, and because the superdivision is too high in time consumption, the user may be blocked when watching live broadcast, and watching experience is affected. Based on the above, the embodiment of the application provides a video self-adaptive super-division control method to solve the technical problem that the super-division processing effect of the conventional video super-division scheme is poor by using a fixed super-division algorithm.
Fig. 1 shows a flowchart of a video adaptive superminute control method according to an embodiment of the present application, where the video adaptive superminute control method according to the embodiment of the present application may be implemented by a video adaptive superminute control device, and the video adaptive superminute control device may be implemented by hardware and/or software and integrated in a video adaptive superminute control device (for example, a mobile phone, a tablet, etc. user end).
The following describes an example of a method for executing the video adaptive superminute control by the video adaptive superminute control device. Referring to fig. 1, the video adaptive super-division control method includes:
s110: and under the condition that the self-adaptive superdivision condition is met, performing image superdivision processing through a first superdivision algorithm based on deep learning.
The super-division algorithm configured in the video self-adaptive super-division control device comprises a first super-division algorithm based on deep learning and a second super-division algorithm based on a traditional algorithm, wherein the traditional algorithm can be bilinear interpolation algorithm (Bilinear Interpolation), bicubic interpolation algorithm (Bicubic Interpolation), nearest neighbor interpolation algorithm (Nearest Neighbor Interpolation), edge-directed interpolation algorithm (Edge-Directed Interpolation), edge reference algorithm (Edge Prior) and the like.
For example, when the super-division video to be processed is required to be processed, whether the video self-adaptive super-division control device meets the self-adaptive super-division condition is determined. The video to be super-divided comprises a plurality of frames of images to be processed, and the video to be super-divided can be video stream data (original video stream data provided by a main broadcasting end or video stream data transcoded by the server) provided by the server under a video live broadcast scene, or video data (original video data provided by a video publishing end or video data transcoded by the server) provided by the server under a segment video scene.
When the video self-adaptive superdivision control equipment meets the self-adaptive superdivision condition, determining that image superdivision processing is carried out through a first superdivision algorithm based on deep learning, namely carrying out image superdivision processing on the superdivision video frame by frame through the first superdivision algorithm based on deep learning, and obtaining a plurality of superdivision images. At this time, the resolution of the super-resolution image is higher than the resolution of the corresponding image to be super-resolution image, and a better display effect can be obtained when the video self-adaptive super-resolution control device displays the super-resolution image. Alternatively, the video playing may be performed by using the superdivision image obtained by the superdivision processing in the process of superdivision processing the superdivision video, or may be performed after finishing the superdivision processing of all the images to be processed of the superdivision video (there may be switching of the superdivision algorithm in the process).
In one possible embodiment, the video adaptive superdivision control method provided by the present solution further includes, before performing image superprocessing by a first superdivision algorithm based on deep learning under the condition that the adaptive superdivision condition is satisfied: and determining equipment configuration information and/or equipment state information, and determining whether the self-adaptive superdivision condition is met according to the equipment configuration information and/or the equipment state information.
When the superdivision video to be processed is required to be processed, whether the video self-adaptive superdivision control equipment meets self-adaptive superdivision conditions or not can be determined first so as to determine whether to enable self-adaptive control of a superdivision algorithm or not. For example, device configuration information and/or device state information of the video adaptive superminute control device are determined, and whether the video adaptive superminute control device meets the adaptive superminute condition is determined according to the device configuration information and the device state information.
The device configuration information provided by the scheme can be used for indicating the hardware performance of the video self-adaptive superminute control device, for example, the device configuration information can be one or a combination of a plurality of processor models, processor frequencies, running memory sizes and the like of the video self-adaptive superminute control device. The minimum configuration requirements meeting the requirements, such as the minimum processor model, the minimum processor frequency, the minimum running memory size, and the like, can be preset for different device configuration information, and when the processor model, the processor frequency, the running memory size respectively reach the minimum processor model, the minimum processor frequency, the minimum running memory size, the device configuration information is considered to meet the hardware performance requirements.
The device state information provided by the scheme can be used for indicating the current working state of the video self-adaptive super-division control device, for example, the device state information can be one or a combination of more of the residual capacity, the device temperature, the available running memory, the service life of the device and the like of the video self-adaptive super-division control device. The minimum state requirements meeting the requirements, such as the minimum remaining power, the minimum device temperature, the minimum available operation memory, the maximum device service life and the like, can be preset for different device state information, and when the requirements of the minimum remaining power, the minimum device temperature, the minimum available operation memory, the maximum device service life and the like are met, the device state information is considered to meet the operation state requirements.
Further, when the device configuration information meets the hardware performance requirement and/or the device state information meets the running state requirement, the video self-adaptive superdivision control device is determined to meet the self-adaptive superdivision condition. And when the equipment configuration information does not meet the hardware performance requirement and/or the equipment state information does not meet the running state requirement, determining that the video self-adaptive superdivision control equipment does not meet the self-adaptive superdivision condition. According to the scheme, whether the self-adaptive superdivision condition is met or not is accurately determined according to the equipment configuration information and/or the equipment state information of the video self-adaptive superdivision control equipment, whether the self-adaptive control of the superdivision algorithm is started or not is determined, the balance between the image superdivision quality and the image superdivision efficiency of the image superdivision processing is accurately realized, and the video watching experience is improved.
In one embodiment, when the video self-adaptive superdivision control device does not meet the self-adaptive superdivision condition or the device configuration information of the video self-adaptive superdivision control device does not meet the hardware performance requirement, the image superdivision processing can be directly performed through a second superdivision algorithm based on the traditional algorithm, so that the condition that the time consumption of superdivision processing is too large due to the fact that the first superdivision algorithm based on deep learning is started to perform the image superdivision processing, video playing is blocked seriously, and user experience is affected is reduced.
S120: and determining first time-consuming information for performing image super-division processing on the image to be processed through a first super-division algorithm.
For example, after determining that the image is super-divided by the first super-division algorithm based on the deep learning, first time-consuming information for performing the image super-division processing on the image to be processed by the first super-division algorithm is determined. Alternatively, the first time-consuming information may be time-consuming information that performs image super-division processing on each frame of to-be-processed image by using a first super-division algorithm, or may be average time-consuming information that performs image super-division processing on a set number (for example, 2 frames, 5 frames, 10 frames, etc.) of to-be-processed images by using the first super-division algorithm.
It is to be explained that when the video self-adaptive super-division control device meets the self-adaptive super-division condition, the image super-division processing is performed on the image to be processed in the super-division video through the first super-division algorithm, and the video self-adaptive super-division control device meets the self-adaptive super-division condition, so that the video self-adaptive super-division control device can be considered to have the potential of performing the image super-division processing by using the first super-division algorithm based on the deep learning, but the situation that the current state of the video self-adaptive super-division control device is not suitable for the first super-division algorithm may exist.
Based on the above, the method records the first time-consuming information of the image super-division processing of the image to be processed by the first super-division algorithm while the image super-division processing of the image to be processed is carried out by the first super-division algorithm. The first time-consuming information can be used for subsequently judging whether the current state of the video self-adaptive superminute control equipment is suitable for the first superminute algorithm or not, the fact that the first superminute algorithm is fixedly used for performing image superminute processing under the condition that the self-adaptive superminute condition is met is reduced, the time consumption of the image superminute processing is too long, the video superminute processing efficiency is affected, the condition of video watching experience of a user is reduced, the superminute algorithm suitable for the current state of the video self-adaptive superminute control equipment is selected, and video watching experience of the user is guaranteed.
S130: and determining whether a first switching condition is met according to the first time-consuming information, and switching to image super-division processing through a second super-division algorithm based on a traditional algorithm under the condition that the first switching condition is met.
Illustratively, it is determined whether the video adaptive superminute control device satisfies the first switching condition according to the first time-consuming information collected during the set time period. For example, it is determined whether the first time-consuming information collected during the set time period reaches a first set time threshold, or whether the proportion of the first time-consuming information collected during the set time period within the first set time threshold reaches a first set proportion threshold. If the video self-adaptive super-division control device reaches the first set time threshold or reaches the first set proportion threshold, the video self-adaptive super-division control device is considered to meet the first switching condition, and if the video self-adaptive super-division control device is within the first set time threshold or the first set proportion threshold, the video self-adaptive super-division control device is considered to not meet the first switching condition.
In one possible embodiment, as shown in a first switching condition determining flow chart provided in fig. 2, the video adaptive superminute control method provided in the present embodiment includes, when determining whether the first switching condition is satisfied according to the first time-consuming information:
s131: a first number of times that the first time-consuming information continuously reaches a set first time-consuming threshold is determined.
S132: in the case where the first time count reaches the set first time count threshold, it is determined whether the first switching condition is satisfied.
After performing image superfractionation processing on an image to be processed through a first superfractionation algorithm and determining first time consumption information, comparing the first time consumption information with a set first time consumption threshold value, determining whether the first time consumption information reaches the first time consumption threshold value according to a comparison result, and recording first times when the first time consumption information continuously reaches the first time consumption threshold value. For example, the first time count is incremented by one when the first time consumption information reaches the first time consumption threshold, and the first time count is cleared when the first time consumption information does not reach the first time consumption threshold.
When the first time number is increased, the first time number is compared with a set first time number threshold value, and whether the first switching condition is met is determined according to a comparison result. For example, when the first number reaches the first number threshold, it is determined that the first switching condition is satisfied, and when the first number does not reach the first number threshold, it is determined that the first switching condition is not satisfied. According to the scheme, whether the first switching condition is met or not is accurately determined according to the comparison condition of the first time when the first time consumption information continuously reaches the first time consumption threshold value and the first time consumption threshold value, the time for switching the super-division algorithm is timely judged, and video watching experience of a user is guaranteed.
When the video self-adaptive superdivision control equipment meets the first switching condition according to the first time-consuming information, the time consumed by the current image superdivision by using the first superdivision algorithm is considered longer, the video playing is easy to cause the blocking of video playing, the video watching experience of a user is influenced, and the currently adopted superdivision algorithm needs to be switched from the first superdivision algorithm to the second superdivision algorithm based on the traditional algorithm for image superdivision processing.
After the fact that the super-division algorithm is switched to the second super-division algorithm to perform image super-division processing is determined, continuing to perform image super-division processing on the subsequent images to be processed (namely, the remaining images to be processed which are not subjected to image super-division processing by the first super-division algorithm) frame by frame through the second super-division algorithm based on the traditional algorithm, so that a plurality of super-division images are obtained. At this time, the resolution of the super-resolution image is higher than the resolution of the corresponding image to be super-resolution image, and a better display effect can be obtained when the video self-adaptive super-resolution control device displays the super-resolution image. The display effect of the super-division image obtained by performing the image super-division processing through the second super-division algorithm may be lower than the display effect of the super-division image obtained by performing the image super-division processing through the first super-division algorithm, but the processing speed of the first super-division algorithm is faster, and when the time for performing the image super-division by using the first super-division algorithm is longer, the super-division processing is performed by switching to the first super-division algorithm, so that the clip of video playing is effectively reduced, and the video watching experience of a user is ensured.
In one embodiment, when the video self-adaptive super-division control device does not meet the first switching condition, continuing to perform image super-division processing through a first super-division algorithm based on deep learning, continuously determining first time-consuming information for performing image super-division processing on an image to be processed through the first super-division algorithm, determining whether the first switching condition is met according to the first time-consuming information, accurately determining the time for switching the super-division algorithm, and ensuring video watching experience of a user.
In a possible embodiment, as shown in a schematic diagram of a super-division algorithm switching flow provided in fig. 3, the video adaptive super-division control method provided in this embodiment further includes S140-S150 after switching to image super-processing by a second super-division algorithm based on a conventional algorithm when the first switching condition is satisfied:
s140: and determining second time-consuming information for performing image super-division processing on the image to be processed through a second super-division algorithm.
Illustratively, after switching to image superfractionation by a second superfractionation algorithm based on the conventional algorithm, second time-consuming information for image superfractionation of the image to be processed by the second superfractionation algorithm is determined. Alternatively, the second time-consuming information may be time-consuming information that performs image super-division processing on each frame of the image to be processed by using the second super-division algorithm, or may be average time-consuming information that performs image super-division processing on a set number (for example, 2 frames, 5 frames, 10 frames, etc.) of the images to be processed by using the second super-division algorithm.
It should be explained that, when the video adaptive superminute control device meets the first switching condition, the video adaptive superminute control device can be considered to have the potential of performing image superminute processing by using the first superminute algorithm based on the traditional algorithm, but the current state of the current video adaptive superminute control device is not suitable for the first superminute algorithm, the current state needs to be switched to the second superminute algorithm in time to perform image superminute processing, so that the time for image superminute is reduced, but the possibility that the state of the video adaptive superminute control device is restored to be suitable for the first superminute algorithm still exists later, in the process of performing image superminute processing on an image to be processed frame by frame, the switching from the first superminute algorithm based on deep learning to the second superminute algorithm based on the traditional algorithm is reversible, the image superminute processing of an adjacent image is not influenced, the superminute algorithm can be restored to the first superminute algorithm to perform image superminute processing, and the image superminute quality is improved.
Based on the image superdivision processing method, the image superdivision processing is carried out on the image to be processed through the second superdivision algorithm, and meanwhile second time-consuming information of the image superdivision processing is recorded on the image to be processed through the second superdivision algorithm. The second time-consuming information can be used for subsequently judging whether the current state of the video self-adaptive super-division control equipment is suitable for the first super-division algorithm or not, so that the condition that the image super-division quality is lower and the video watching experience of a user is influenced due to the fact that the second super-division algorithm is fixedly used for carrying out the image super-division processing under the condition that the first switching condition is met is reduced, the super-division algorithm suitable for the current state of the video self-adaptive super-division control equipment is selected, and the video watching experience of the user is guaranteed.
S150: and determining whether a second switching condition is met according to the second time-consuming information, and switching to image superdivision processing through a first superdivision algorithm based on deep learning under the condition that the second switching condition is met.
Illustratively, it is determined whether the video adaptive superminute control device satisfies the second switching condition according to second time-consuming information collected during the set time period. For example, it is determined whether the second time-consuming information collected over the set length of time is less than a second set time threshold, or whether the proportion of the second time-consuming information collected over the set length of time within the second set time threshold is less than a second set proportion threshold. If the video self-adaptive superminute control equipment is smaller than the second set time threshold or smaller than the second set proportion threshold, the video self-adaptive superminute control equipment is considered to meet the second switching condition, and if the video self-adaptive superminute control equipment reaches the second set time threshold or reaches the second set proportion threshold, the video self-adaptive superminute control equipment is considered to not meet the second switching condition. According to the scheme, after the super-division algorithm is switched to the second super-division algorithm to perform image super-division processing, whether the second switching condition for switching to the first super-division algorithm is met or not is determined according to the second time-consuming information, the time for recovering the state of the video self-adaptive super-division control equipment to be suitable for the first super-division algorithm is accurately determined, the super-division algorithm is timely recovered to the first super-division algorithm to perform image super-division processing, and the image super-division quality is improved.
In a possible embodiment, as shown in a second switching condition determining flow chart provided in fig. 4, the video adaptive superminute control method provided in the present embodiment includes, when determining whether the second switching condition is satisfied according to the second time-consuming information:
s151: determining that the second time consuming information is successively less than a second number of times that a second time consuming threshold is set.
S152: in the case where the second number of times reaches the second number of times threshold, it is determined whether a second switching condition is satisfied.
After each image superdivision processing is performed on the image to be processed through the second superdivision algorithm and the second time consumption information is determined, the second time consumption information is compared with a set second time consumption threshold value, whether the second time consumption information is smaller than the second time consumption threshold value or not is determined according to a comparison result, and a second time when the second time consumption information is continuously smaller than the second time consumption threshold value is recorded. For example, the second number of times is incremented by one when the second time-consuming information is less than the second time-consuming threshold, and the second number of times is cleared when the second time-consuming information is not less than the second time-consuming threshold.
And when the second times are increased, comparing the second times with a set second time threshold value, and determining whether the second switching condition is met according to the comparison result. For example, when the second number of times reaches the second number of times threshold, it is determined that the second switching condition is satisfied, and when the second number of times does not reach the second number of times threshold, it is determined that the second switching condition is not satisfied. According to the scheme, whether the second switching condition is met or not is accurately determined according to the comparison condition of the second time consumption information and the second time consumption threshold value, wherein the second time consumption information is continuously smaller than the second time consumption threshold value, the time for switching the super-score algorithm is timely judged, and video watching experience of a user is guaranteed.
When the video self-adaptive super-division control equipment meets the second switching condition according to the second time-consuming information, the state of the video self-adaptive super-division control equipment is considered to be restored to be suitable for the first super-division algorithm, and the currently adopted super-division algorithm can be switched from the second super-division algorithm to the first super-division algorithm to perform image super-division processing.
After the fact that the super-division algorithm is switched to the second super-division algorithm to perform image super-division processing is determined, continuing to perform image super-division processing on the subsequent images to be processed (namely, the remaining images to be processed which are not subjected to image super-division processing by the second super-division algorithm) frame by frame through the first super-division algorithm based on deep learning to obtain a plurality of super-division images.
In one embodiment, when the video self-adaptive super-division control device does not meet the second switching condition, continuing to perform image super-division processing through a second super-division algorithm based on the traditional algorithm, continuously determining second time-consuming information for performing image super-division processing on an image to be processed through the second super-division algorithm, determining whether the second switching condition is met according to the second time-consuming information, accurately determining the time for switching the super-division algorithm, and ensuring video watching experience of a user.
In a possible embodiment, as shown in a schematic diagram of an adaptive policy disabling determination flow provided in fig. 5, the video adaptive super-resolution control method provided in this embodiment, after switching to image super-resolution processing by a first super-resolution algorithm based on deep learning under the condition that a second switching condition is met, further includes:
s160: and determining the third times of image super-division processing through the first super-division algorithm and the fourth times of image super-division processing through the second super-division algorithm.
S170: and determining the duty ratio information of the image super-division processing through the first super-division algorithm based on the third times and the fourth times.
S180: it is determined whether to disable an adaptive strategy to the superdivision algorithm based on the duty cycle information.
In an exemplary process of performing image superdivision processing on a plurality of images to be processed in the superdivision video through the first superdivision algorithm or the second superdivision algorithm, a third time of performing image superdivision processing through the first superdivision algorithm and a fourth time of performing image superdivision processing through the second superdivision algorithm are recorded. The third times are consistent with the number of images subjected to image super-division processing by using the first super-division algorithm, and the fourth times are consistent with the number of images subjected to image super-division processing by using the second super-division algorithm.
And determining the duty ratio information of the image super-division processing through the first super-division algorithm according to the third times and the fourth times in the set time period in real time or according to the set time interval. Alternatively, the duty ratio information=third number/(third number+fourth number). Further, whether the self-adaptive strategy to the super-division algorithm is forbidden is determined according to the duty ratio information, and if the self-adaptive strategy to the super-division algorithm is forbidden, the second super-division algorithm based on the traditional algorithm is fixedly used for performing image super-division processing.
In one embodiment, in determining whether to disable the adaptive strategy to the superdistribution algorithm based on the duty cycle information, the adaptive strategy to the superdistribution algorithm may be disabled if the duty cycle information is less than a set duty cycle threshold. Illustratively, the duty ratio information is compared with a set duty ratio threshold, when the duty ratio information is smaller than the set duty ratio threshold, the performance and the state of the current video self-adaptive super-division control device are considered not to support deep learning-based image super-division for a long time, and the self-adaptive strategy of the super-division algorithm is disabled. If the duty ratio information is larger than or equal to the set duty ratio threshold value, the performance and the state of the current video self-adaptive super-division control equipment can be considered to support deep learning-based image super-division for a long time, and the self-adaptive strategy of the super-division algorithm can be continuously started. According to the method, according to the third times of image superdivision processing through the first superdivision algorithm and the fourth times of image superdivision processing through the second superdivision algorithm, the duty ratio information of the image superdivision processing through the first superdivision algorithm is determined, whether the self-adaptive strategy of the superdivision algorithm is forbidden or not is determined according to the duty ratio information, whether the performance and the state of the current video self-adaptive superdivision control equipment support deep learning-based image superdivision for a long time is accurately determined, whether the self-adaptive strategy of the superdivision algorithm is forbidden or not is determined, the situation that the performance and the state of the current video self-adaptive superdivision control equipment do not support deep learning-based image superdivision for a long time, and the superdivision algorithm is frequently switched between the first superdivision algorithm and the second superdivision algorithm, so that the quality of the image superdivision is unstable, and video watching experience of users is affected is provided for the users.
According to the method, when the self-adaptive superdivision condition is met, image superdivision processing is carried out through a first superdivision algorithm based on deep learning, when the first switching condition is met according to first time-consuming information of the image superdivision processing of the image to be processed through the first superdivision algorithm, the image superdivision processing is switched to the image superdivision processing through a second superdivision algorithm based on the traditional algorithm, the time-consuming information of the image superdivision processing of the image to be processed through different superdivision algorithms is switched between the first superdivision algorithm based on the deep learning and the second superdivision algorithm based on the traditional algorithm, balance between image superdivision quality and image superdivision efficiency of the image superdivision processing is achieved, the situation that the image superdivision quality is too low or the image superdivision efficiency is too low due to the single superdivision algorithm is reduced, video superdivision processing effect is improved, and video viewing experience of users is improved. Meanwhile, after the super-division algorithm is switched to the second super-division algorithm to perform image super-division processing, whether a second switching condition for switching to the first super-division algorithm is met or not is determined according to second time-consuming information, the time for recovering the state of the video self-adaptive super-division control equipment to be suitable for the first super-division algorithm is accurately determined, the super-division algorithm is timely recovered to the first super-division algorithm to perform image super-division processing, and image super-division quality is improved. And when the performance and the state of the video self-adaptive super-division control equipment do not support image super-division based on deep learning for a long time, the self-adaptive strategy of the super-division algorithm is forbidden, so that the condition that the quality of the image super-division is unstable and the video watching experience of a user is influenced due to frequent switching of the super-division algorithm is reduced.
Fig. 6 is a schematic structural diagram of a video adaptive superminute control device according to an embodiment of the present application. Referring to fig. 6, the video adaptive super-division control device includes a first super-division module 61, a first statistics module 62, and a first switching module 63.
Wherein, the first superdivision module 61 is configured to perform image superdivision processing through a first superdivision algorithm based on deep learning under the condition that the adaptive superdivision condition is satisfied; the first statistics module 62 is configured to determine first time-consuming information for performing image super-division processing on the image to be processed through the first super-division algorithm; the first switching module 63 is configured to determine whether a first switching condition is satisfied according to the first time-consuming information, and switch to performing image super-division processing by a second super-division algorithm based on a conventional algorithm if the first switching condition is satisfied.
According to the method, when the self-adaptive superdivision condition is met, image superdivision processing is carried out through a first superdivision algorithm based on deep learning, when the first switching condition is met according to first time-consuming information of the image superdivision processing of the image to be processed through the first superdivision algorithm, the image superdivision processing is switched to the image superdivision processing through a second superdivision algorithm based on the traditional algorithm, the time-consuming information of the image superdivision processing of the image to be processed through different superdivision algorithms is switched between the first superdivision algorithm based on the deep learning and the second superdivision algorithm based on the traditional algorithm, balance between image superdivision quality and image superdivision efficiency of the image superdivision processing is achieved, the situation that the image superdivision quality is too low or the image superdivision efficiency is too low due to the single superdivision algorithm is reduced, video superdivision processing effect is improved, and video viewing experience of users is improved.
In one possible embodiment, the video adaptive superminute control device further includes a second statistics module and a second switching module.
The second statistics module is configured to determine second time-consuming information for performing image superdivision processing on the image to be processed through the second superdivision algorithm;
the second switching module is configured to determine whether a second switching condition is met according to the second time-consuming information, and switch to image super-division processing through a first super-division algorithm based on deep learning under the condition that the second switching condition is met.
In one possible embodiment, the first switching module 63, when determining whether the first switching condition is satisfied according to the first time-consuming information, is configured to:
determining a first time when the first time consumption information continuously reaches a set first time consumption threshold;
and determining whether a first switching condition is satisfied in the case that the first time number reaches a set first time number threshold.
In one possible embodiment, the second switching module is configured to, when determining whether the second switching condition is met according to the second time-consuming information:
determining that the second time consuming information is continuously less than a second time consuming threshold value;
And determining whether a second switching condition is satisfied or not in the case that the second number reaches a second number threshold.
In one possible embodiment, the video adaptive superdivision control device further includes an adaptive superdivision condition judging module configured to determine device configuration information and/or device state information, and determine whether an adaptive superdivision condition is satisfied according to the device configuration information and/or the device state information.
In one possible embodiment, the video adaptive superminute control device further includes a frequency statistics module, a duty ratio determination module, and a policy disabling module, wherein:
the frequency counting module is configured to determine a third frequency of image super-division processing through the first super-division algorithm and a fourth frequency of image super-division processing through the second super-division algorithm;
the duty ratio determining module is configured to determine duty ratio information of image super-division processing through the first super-division algorithm based on the third times and the fourth times;
the policy disabling module is configured to determine whether to disable an adaptive policy to the superdivision algorithm based on the duty cycle information.
In one possible embodiment, the policy disabling module, when determining whether to disable the adaptive policy to the superdistribution algorithm based on the duty cycle information, is configured to disable the adaptive policy to the superdistribution algorithm if the duty cycle information is less than a set duty cycle threshold.
It should be noted that, in the embodiment of the video adaptive superminute control device, each unit and module included are only divided according to the functional logic, but not limited to the above-mentioned division, so long as the corresponding functions can be implemented; in addition, the specific names of the functional units are also only for distinguishing from each other, and are not used to limit the protection scope of the embodiments of the present application.
The embodiment of the application also provides video self-adaptive super-division control equipment which can integrate the video self-adaptive super-division control device provided by the embodiment of the application. Fig. 7 is a schematic structural diagram of a video adaptive superminute control device according to an embodiment of the present application. Referring to fig. 7, the video adaptive super division control apparatus includes: an input device 73, an output device 74, a memory 72, and one or more processors 71; a memory 72 for storing one or more programs; the one or more programs, when executed by the one or more processors 71, cause the one or more processors 71 to implement the video adaptive superminute control method as provided in the above embodiments. The video self-adaptive superminute control device, the video self-adaptive superminute control equipment and the video self-adaptive superminute control computer can be used for executing the video self-adaptive superminute control method provided by any embodiment, and have corresponding functions and beneficial effects.
Embodiments of the present application also provide a non-volatile storage medium storing computer-executable instructions that, when executed by a computer processor, are configured to perform a video adaptive superminute control method as provided by the above embodiments. Of course, the non-volatile storage medium storing the computer executable instructions provided in the embodiments of the present application is not limited to the video adaptive superminute control method provided above, and may also perform related operations in the video adaptive superminute control method provided in any embodiment of the present application. The video adaptive superminute control device, the device and the storage medium provided in the foregoing embodiments may perform the video adaptive superminute control method provided in any embodiment of the present application, and technical details not described in detail in the foregoing embodiments may be referred to the video adaptive superminute control method provided in any embodiment of the present application.
On the basis of the above embodiments, the embodiments of the present application further provide a computer program product, where the technical solution of the present application is essentially or a part contributing to the prior art or all or part of the technical solution may be embodied in the form of a software product, and the computer program product is stored in a storage medium, and includes several instructions to cause a computer device, a mobile terminal or a processor therein to execute all or part of the steps of the video adaptive superdistribution control method provided by the embodiments of the present application.

Claims (11)

1. The video self-adaptive superdivision control method is characterized by comprising the following steps of:
under the condition that the self-adaptive superdivision condition is met, performing image superdivision processing through a first superdivision algorithm based on deep learning;
determining first time-consuming information for performing image superdivision processing on an image to be processed through the first superdivision algorithm;
and determining whether a first switching condition is met according to the first time-consuming information, and switching to image super-division processing through a second super-division algorithm based on a traditional algorithm under the condition that the first switching condition is met.
2. The method according to claim 1, wherein when the first switching condition is satisfied, switching to image super-processing by a second super-division algorithm based on a conventional algorithm is performed, further comprising:
determining second time-consuming information for performing image super-division processing on the image to be processed through the second super-division algorithm;
and determining whether a second switching condition is met according to the second time-consuming information, and switching to image superdivision processing through a first superdivision algorithm based on deep learning under the condition that the second switching condition is met.
3. The method according to claim 1, wherein determining whether a first switching condition is satisfied according to the first time-consuming information comprises:
determining a first time when the first time consumption information continuously reaches a set first time consumption threshold;
and determining whether a first switching condition is satisfied in the case that the first time number reaches a set first time number threshold.
4. The method according to claim 2, wherein determining whether a second switching condition is satisfied according to the second time-consuming information comprises:
determining that the second time consuming information is continuously less than a second time consuming threshold value;
and determining whether a second switching condition is satisfied or not in the case that the second number reaches a second number threshold.
5. The method according to claim 1, wherein, in the case where the adaptive superdivision condition is satisfied, before performing the image superprocessing by the first superdivision algorithm based on the deep learning, further comprising:
and determining equipment configuration information and/or equipment state information, and determining whether the self-adaptive supersplit condition is met according to the equipment configuration information and/or the equipment state information.
6. The video adaptive super-division control method according to claim 2, wherein when the second switching condition is satisfied, switching to image super-division processing by a first super-division algorithm based on deep learning further comprises:
determining a third time of image super-division processing through the first super-division algorithm and a fourth time of image super-division processing through the second super-division algorithm;
determining duty ratio information of image super-division processing through the first super-division algorithm based on the third times and the fourth times;
and determining whether to disable an adaptive strategy for the super-division algorithm based on the duty ratio information.
7. The method of claim 6, wherein the determining whether to disable the adaptive strategy to the super-division algorithm based on the duty cycle information comprises:
and under the condition that the duty ratio information is smaller than a set duty ratio threshold value, disabling the self-adaptive strategy of the super-division algorithm.
8. The utility model provides a video self-adaptation superminute controlling means which characterized in that includes first superminute module, first statistics module and first switching module, wherein:
The first superdivision module is configured to perform image superdivision processing through a first superdivision algorithm based on deep learning under the condition that the self-adaptive superdivision condition is met;
the first statistics module is configured to determine first time-consuming information for performing image superdivision processing on an image to be processed through the first superdivision algorithm;
the first switching module is configured to determine whether a first switching condition is met according to the first time-consuming information, and switch to image super-division processing through a second super-division algorithm based on a traditional algorithm under the condition that the first switching condition is met.
9. A video adaptive superminute control device, comprising: a memory and one or more processors;
the memory is used for storing one or more programs;
the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the video adaptive superminute control method of any of claims 1-7.
10. A non-transitory storage medium storing computer executable instructions which, when executed by a computer processor, are for performing the video adaptive superdistribution control method according to any of claims 1-7.
11. A computer program product comprising a computer program, characterized in that the computer program, when executed by a processor, implements the video adaptive superdistribution control method according to any of claims 1-7.
CN202311056059.8A 2023-08-21 2023-08-21 Video self-adaptive superdivision control method, device, equipment, storage medium and product Pending CN117094891A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202311056059.8A CN117094891A (en) 2023-08-21 2023-08-21 Video self-adaptive superdivision control method, device, equipment, storage medium and product

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202311056059.8A CN117094891A (en) 2023-08-21 2023-08-21 Video self-adaptive superdivision control method, device, equipment, storage medium and product

Publications (1)

Publication Number Publication Date
CN117094891A true CN117094891A (en) 2023-11-21

Family

ID=88783063

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202311056059.8A Pending CN117094891A (en) 2023-08-21 2023-08-21 Video self-adaptive superdivision control method, device, equipment, storage medium and product

Country Status (1)

Country Link
CN (1) CN117094891A (en)

Similar Documents

Publication Publication Date Title
WO2021244341A1 (en) Picture coding method and apparatus, electronic device and computer readable storage medium
WO2021068598A1 (en) Encoding method and device for screen sharing, and storage medium and electronic equipment
WO2021175054A1 (en) Image data processing method, and related apparatus
WO2020107989A1 (en) Video processing method and apparatus, and electronic device and storage medium
CN110149555B (en) Video processing method and video receiving device
CN113055742B (en) Video display method, device, terminal and storage medium
WO2022199348A1 (en) Video encoding method and apparatus, video encoding device, and storage medium
CN114827662B (en) Video resolution adaptive adjustment method, device, equipment and storage medium
CN112399123B (en) Video definition adjusting method and device, electronic equipment and storage medium
CN102170552A (en) Video conference system and processing method used therein
WO2021047177A1 (en) Data encoding method and apparatus
CN113301342B (en) Video coding method, network live broadcasting method, device and terminal equipment
CN108600783A (en) A kind of method of frame rate adjusting, device and terminal device
CN112804527B (en) Image output method, image output device and computer-readable storage medium
CN112350998B (en) Video streaming transmission method based on edge calculation
CN102243856A (en) Method and device for dynamically switching screen data processing modes
WO2024061087A1 (en) Dynamic collection method and device for desktop image, and computer-readable storage medium
CN110572713B (en) Transcoding method and processing terminal for adaptive video bandwidth ratio
CN117094891A (en) Video self-adaptive superdivision control method, device, equipment, storage medium and product
US20140099039A1 (en) Image processing device, image processing method, and image processing system
CN114584831B (en) Video optimization processing method, device, equipment and storage medium for improving video definition
WO2020038071A1 (en) Video enhancement control method, device, electronic apparatus, and storage medium
CN111643901A (en) Method and device for intelligently rendering cloud game interface
CN105812923B (en) Play handling method and device based on video on demand
CN110858389A (en) Method and device for enhancing video image quality

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination