WO2021082171A1 - 一种增强视频画质的方法和装置 - Google Patents
一种增强视频画质的方法和装置 Download PDFInfo
- Publication number
- WO2021082171A1 WO2021082171A1 PCT/CN2019/123009 CN2019123009W WO2021082171A1 WO 2021082171 A1 WO2021082171 A1 WO 2021082171A1 CN 2019123009 W CN2019123009 W CN 2019123009W WO 2021082171 A1 WO2021082171 A1 WO 2021082171A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- image
- image quality
- video
- target
- content category
- Prior art date
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/234—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs
Definitions
- the present invention relates to the technical field of video processing, in particular to a method and device for enhancing video image quality.
- video quality enhancement has gradually become a hot topic in the current video field.
- video providers or video maintainers often need to perform image quality enhancement processing on video frames to optimize video content, improve image quality, and enrich video information.
- the video provider or the video maintainer may perform the image quality enhancement processing on the video file after obtaining the video file.
- image quality enhancement can include multiple processing such as image denoising, image de-artifacting, color enhancement, and image super-resolution.
- the video provider or video maintainer can manually select one or more image quality enhancements. Tool, through the image quality enhancement tool to sequentially perform a variety of image quality enhancement processing on the video file, so that the video file with enhanced image quality can be provided to the user.
- the video frame images in different video files are often processed the same indifferently, that is, the same image quality enhancement strategy is used for each video frame image of different screen content.
- This processing method cannot use the best image quality enhancement processing for different screen content, so the effect of image quality enhancement has limitations.
- the embodiments of the present invention provide a method and device for enhancing video quality.
- the technical solution is as follows:
- a method for enhancing video image quality including:
- an apparatus for enhancing video quality includes:
- the classification module is used to determine the image content category of the target video frame image of the video to be processed
- An evaluation module configured to evaluate the image quality of the target video frame image, and generate an image quality evaluation result
- the enhancement module is configured to perform image quality enhancement processing on the to-be-processed video according to the image content category and the image quality evaluation result.
- a network device in a third aspect, includes a processor and a memory.
- the memory stores at least one instruction, at least one program, code set, or instruction set, and the at least one instruction, the at least one instruction, and the at least one instruction set are stored in the memory.
- a piece of program, the code set or the instruction set is loaded and executed by the processor to realize the method for enhancing the video quality as described in the first aspect.
- a computer-readable storage medium stores at least one instruction, at least one program, code set, or instruction set, the at least one instruction, the at least one program, and the code
- the set or instruction set is loaded and executed by the processor to implement the method for enhancing video quality as described in the first aspect.
- the image content category of the target video frame image of the video to be processed is determined; the image quality of the target video frame image is evaluated to generate an image quality assessment result; according to the image content category and the image quality assessment result, the video to be processed is performed Picture quality enhancement processing.
- the video frame images are classified according to the screen content, and the image quality of the video frame images is evaluated, and then for each type of video frame image, the image quality enhancement processing is carried out with reference to the image quality evaluation results to ensure different screen content
- the image quality of the video frame image is correspondingly enhanced, so that the effect of the image quality enhancement is more prominent.
- FIG. 1 is a flowchart of a method for enhancing video quality according to an embodiment of the present invention
- FIG. 2 is a schematic diagram of a principle of enhancing video quality according to an embodiment of the present invention.
- FIG. 3 is a schematic diagram of a principle of enhancing video quality according to an embodiment of the present invention.
- FIG. 4 is a schematic diagram of a principle of enhancing video quality according to an embodiment of the present invention.
- FIG. 5 is a schematic structural diagram of an apparatus for enhancing video image quality according to an embodiment of the present invention.
- Fig. 6 is a schematic structural diagram of a network device provided by an embodiment of the present invention.
- the embodiment of the present invention provides a method for enhancing video image quality.
- the method is applicable to any network device with a video frame image processing function, and specifically may be a background server of a video provider or a video maintainer.
- the network device After the network device obtains any video, it can perform different types of image quality enhancement processing on the video frame images in it to provide external video data with higher image quality.
- the specific image quality enhancement processing here can be determined by the network equipment according to the image content of the video frame image, including but not limited to the following image quality enhancement processing: image denoising, image de-artifacting, color enhancement, image super-resolution .
- the network device can perform one or more image quality enhancement processing on the video frame image as needed, and for different video frame images in the same video, the network device can use the same Or different image quality enhancement processing. Further, for the same image quality enhancement processing, the network device can support multiple execution levels of image quality enhancement processing.
- Fig. 1 The processing flow shown in Fig. 1 will be described in detail below in conjunction with specific implementations.
- the principle can be referred to as shown in Fig. 2, and the content can be as follows:
- Step 101 Determine the image content category of the target video frame image of the video to be processed.
- the target video frame image may be a different video frame image in the to-be-processed video, as detailed in the subsequent description.
- the network device may first determine whether it is necessary to perform image quality enhancement processing on the video according to a preset standard. If necessary, the network device can select the target video frame image from all the video frame images of the video (which can be called the video to be processed), and then perform content recognition on the target video frame image to determine the image content category of the target video frame image .
- the image content category here can be determined by the network device according to the proportion of the video frame image, such as landscapes, portraits, animals, vehicles, and so on.
- the image content categories in this embodiment should be classified according to differences in image quality enhancement processing, that is, image quality enhancement processing performed on image data in different image content categories should have certain differences.
- the two image content categories should be combined into the same image content category. For example, if there are two image content categories "men” and "women”, if other image quality factors are the same, the image data under the two image content categories should be processed with the same image quality enhancement process. Therefore, " The two image content categories of "men” and “women” are merged into the image content category of "people".
- the network device can according to the image content category of multiple video frame images adjacent to the target video frame image in the video to be processed, Further content classification is performed on the target video frame image.
- the video frame images with similar content are classified into the same category, and the same image quality enhancement processing is subsequently used, avoiding the difference in image quality enhancement processing. This leads to the problem of excessive picture difference between adjacent video frames, which can improve the effect of video quality enhancement.
- an image classification model established based on a deep learning algorithm can be used to classify the video frame images.
- the method of model training can be: image data marked with category information in a preset training material set, and image data based on depth
- the image classification model established by the learning algorithm is trained; and the processing of step 101 may be as follows: input the target video frame image of the video to be processed into the trained image classification model to obtain the image content category of the target video frame image.
- the network device can collect a large number of image data of different image content categories in advance, and mark the category information of each image data, and then can aggregate all the image data to generate a training material set. After that, the network device may establish an image classification model based on a deep learning algorithm (for example, using a convolutional neural network), and then train the image classification model through the image data marked with category information in the training material set. In this way, after the training is completed, the network device can use the above-mentioned image classification model to classify the target video frame image, that is, the target video frame image of the video to be processed can be input into the trained image classification model to obtain the target video frame image Image content category.
- a deep learning algorithm for example, using a convolutional neural network
- the network device can also adjust the parameters of the image classification model. Specifically, it can input image data of different image content categories into the image classification model, and then according to the classification results and images output by the image classification model. For the error between the correct categories of the data, adjust the model parameters of the image classification model to enhance the accuracy of the image classification model for image classification. In this way, the use of deep learning technology to implement image classification processing can improve the convenience and accuracy of image classification.
- Step 102 Perform image quality evaluation on the target video frame image, and generate an image quality evaluation result.
- the network device may perform image quality evaluation processing on the target video frame image to evaluate the image quality of the target video frame image from multiple dimensions.
- the type of image quality evaluation processing can be determined according to the enhancement type of the image quality enhancement processing supported by the network device, such as image noise evaluation, image artifact evaluation, image color evaluation, image resolution evaluation, and so on.
- the network device can aggregate and generate multi-dimensional image quality evaluation results.
- the image quality evaluation results can be specifically used to indicate the enhancement types of various image quality enhancement processing.
- the image quality evaluation result can be: "Image denoising processing is required; Requires color enhancement processing; does not require image de-artifact processing".
- image quality evaluation processing can be carried out respectively.
- the processing of step 102 can be as follows: Use the image quality evaluation tool corresponding to the image content category to evaluate the target video frame The image is evaluated for image quality, and an image quality evaluation result is generated.
- each image quality evaluation tool may be dedicated to image quality assessment of video frame images under one image content category.
- the network device can use the image quality assessment tool corresponding to the image content category to perform image quality assessment on the target video frame image, thereby generating an image quality assessment result.
- the image quality evaluation process evaluates the image quality of the video frame image from multiple dimensions such as image noise evaluation, image artifact evaluation, image color evaluation, image resolution evaluation, etc.
- One image quality evaluation tool with the ability to evaluate all dimensions can be used, or multiple image quality evaluation tools corresponding to different dimensions, such as image noise evaluation tools, image artifact evaluation tools, etc., can be used.
- image noise evaluation tools such as image noise evaluation tools, image artifact evaluation tools, etc.
- the above-mentioned image quality evaluation tool may be an image quality evaluation model established based on a deep learning algorithm, that is, the processing of step 102 may specifically be: input the target video frame image into the image quality evaluation model corresponding to the image content category to obtain the image quality evaluation result.
- the training method of the image quality evaluation model can be as follows: obtain multiple high-quality images under the target image content category; perform degradation processing of different degradation types on each high-quality image, and obtain degradation marked with degradation type information Image: Through all the degraded images, the image quality evaluation model based on the deep learning algorithm is trained to generate the image quality evaluation model corresponding to the target image content category.
- the target image content category can be any image content category, and high-quality images can have image quality higher than the preset standard with a certain degree of degradation, such as high-definition or higher resolution, less than 10% noise, or image artifacts An image that occupies less than 10% of the screen.
- the network device can collect a large number of high-quality images in advance to generate a high-quality material library, and then the network device can extract multiple high-quality images from the high-quality material library, and perform different degradation types on each extracted high-quality image. Quality processing, and finally a degraded image marked with degraded type information is obtained.
- the degradation processing can correspond to the image quality enhancement processing. If the image quality enhancement processing is image denoising processing, the degradation processing is image noise enhancement processing; if the image quality enhancement processing is color enhancement processing, the degradation processing is Color weakening treatment.
- the network device may establish an image quality evaluation model based on a deep learning algorithm (such as using a convolutional neural network), and then train the image quality evaluation model through the aforementioned degraded images marked with degradation type information.
- a deep learning algorithm such as using a convolutional neural network
- the network device can use the above-mentioned image quality evaluation model to perform image quality evaluation processing on the target video frame image.
- the above image quality evaluation model has image quality evaluation capabilities in all dimensions, the degraded images under multiple types of degradation can be unified as the training material for the image quality evaluation model during training; and if the above image
- the quality evaluation model only has a single-dimensional image quality evaluation capability.
- Only degraded images under a single degraded type are used as training materials for the image quality evaluation model, and a corresponding training is generated for each dimension according to the above training process.
- Image quality evaluation model is based on a deep learning algorithm (such as using a convolutional neural network), and then train the image quality evaluation model through the aforementioned
- the network device can also adjust the parameters of the image quality evaluation model. Specifically, it can input the degraded image into the image quality evaluation model, and then output the image quality evaluation result according to the image quality evaluation model. For the degradation type corresponding to the degraded image, the model parameters of the image quality evaluation model are adjusted to enhance the accuracy of the image quality evaluation model for image quality evaluation.
- the network device needs to extract multiple high-quality images of the same graphic content category, and then perform subsequent processing, so that the image quality evaluation model corresponding to the graphic content category can be generated.
- the image quality evaluation model can also specifically evaluate the level of execution required when performing image quality enhancement processing. The better the original image quality of the image and the lighter the execution level, the weaker the effect of image quality enhancement. Accordingly, during the training process, when the network device performs degradation processing on high-quality images, it needs to perform degradation processing of different execution levels for the same degradation type, and finally obtain the information marked with degradation type information and execution level information. Degraded image. Next, the network device can train the image quality evaluation model through the degraded image marked with the degradation type information and the execution magnitude information, so that it can be trained to evaluate the execution magnitude of the image quality enhancement processing required for the image Image quality evaluation model.
- Step 103 Perform image quality enhancement processing on the video to be processed according to the image content category and the image quality evaluation result.
- the network device after the network device determines the image content category and image quality evaluation result of the target video frame image, it can use the image content category and image quality evaluation result as a reference to enhance the image quality of part of the video frame image to be processed. deal with. Specifically, the network device can perform different image quality enhancement processing on the video frame image of the video to be processed according to different image content categories and image quality evaluation results. Furthermore, there may be multiple image quality enhancement tools corresponding to different image content categories and image quality evaluation results on the network device. When performing image quality enhancement processing, the network device can be based on the image content category and image quality of the target video frame image. Evaluate the results, select the corresponding image quality enhancement tools, and then use these image quality enhancement tools to perform image quality enhancement processing on the video to be processed.
- the network device can determine the range of the video frame image to perform the image quality enhancement processing according to the selection rule of the target video frame image, and the specific content can refer to the subsequent description.
- the network device can also decide whether to perform image quality enhancement processing according to the image content category of the target video frame image. For example, for anchor videos, when the image content category is a category that is poorly related to the main content of the video, such as scenery or objects, you may not perform image quality enhancement processing, or choose to perform lighter image quality enhancement processing. , Which can ensure the main picture quality of the video while reducing the equipment resources consumed by the picture quality enhancement.
- the range of the video frame image for performing image quality enhancement processing is also different. Several situations are given as follows:
- the image quality enhancement process is performed on the target video frame image.
- the network device can completely perform the processing of steps 101 to 103 for each frame of the video to be processed, and the target video frame image can be any frame of the video to be processed. Therefore, the network device can directly perform image quality enhancement processing on the target video frame image according to the image content category and the image quality evaluation result of the target video frame image. In this way, the frame-by-frame content classification, image quality evaluation, and image quality enhancement processing of the video can effectively ensure that each video frame image receives targeted and effective image quality enhancement processing, thereby improving the effect of image quality enhancement. Further, the network device can only process each frame of the image in a specific segment of the video (such as the climax segment of the video). The climax segment of the video can be pre-marked manually or pre-detected by the network device. The specific detection method The content disclosed in the prior art can be used, and this embodiment will not repeat it.
- the target video frame image and the interval frame image corresponding to the target video frame image are subjected to image quality enhancement processing.
- the network device can select the target video frame image at preset intervals among all the video frame images of the video to be processed. For example, select a target video frame image every 5 video frame images, that is, the first and sixth video frames. One, 11th...5n+1th (n is a natural number) video frame image is the target video frame image. Correspondingly, all the video frame images between the two target video frame images are the interval frame images.
- the interval frame image corresponds to the previous target video frame image, that is, the 5n+2th to the 5n+5th
- the video frame image is an interval frame image corresponding to the 5n+1th target video frame image.
- the network device may perform image quality enhancement processing on the target video frame image and the interval frame image corresponding to the target video frame image according to the image content classification and image quality evaluation result of the target video frame image.
- image quality enhancement processing on the target video frame image and the interval frame image corresponding to the target video frame image according to the image content classification and image quality evaluation result of the target video frame image.
- the key frame image of the video to be processed is selected as the target video frame image, the key frame image of the video to be processed is subjected to image quality enhancement processing.
- the video frame can include I frame, B frame and P frame.
- the I frame is the key frame of the whole picture content of the self-contained image
- the P frame is recorded with the previous frame.
- B frame records the difference between the previous frame and the next frame.
- Enhancement which can ensure the overall image quality enhancement effect of the video to the greatest extent. Therefore, the network device can select the key frame image of the video to be processed as the target video frame image, and when performing image quality enhancement processing, only the key frame image of the video to be processed is subjected to image quality enhancement processing.
- the above-mentioned key frame image can also be manually defined or detected by the device, and meets the preset standard.
- a video frame image with a human face is a key frame image, or a specific timestamp
- the corresponding video frame image is the key frame image and so on.
- the network device may evaluate the enhancement type and execution level of the image quality enhancement processing required by the image, and then use the corresponding image quality enhancement tool to perform the image quality enhancement processing.
- the processing of step 103 may be as follows: use image quality enhancement tools corresponding to the image content category, enhancement type, and execution level to perform image quality enhancement processing on the video to be processed.
- the network device may be provided with an image quality enhancement tool for image quality enhancement for images of different image content categories.
- One image quality enhancement tool can be used to process a single enhancement type.
- Enhanced processing Based on the above settings, when the network device evaluates the image quality of the target video frame image, it can obtain an image quality evaluation result that includes the required multiple image quality enhancement processing enhancement types and execution levels, and then can use the target video frame
- the image content category of the image, as well as the above-mentioned enhancement type and execution level the image quality enhancement processing of the video to be processed. Refer to Figure 4 for the related process of the above content.
- the network device can choose to perform image quality enhancement processing corresponding to different execution levels on the key frame images and non-key frame images in the video.
- the video to be processed is selected
- the key frame image is used as the target video frame image
- the image quality enhancement processing of the video to be processed can be specifically as follows: use image content category, enhancement type, and execution level corresponding image quality enhancement tools to perform image quality enhancement processing on the key frame image; Use image content category, enhancement type, and execution level of lightweight corresponding image quality enhancement tools to perform image quality enhancement processing on non-key frame images associated with key frame images.
- the network device selects the key frame image of the video to be processed as the target video frame image, after obtaining the image content category and image quality evaluation result of the key frame image, it can first determine all the image content categories and images that meet the requirements.
- the image quality enhancement tool of the enhanced type in the quality evaluation result can be called tool A), and the light-weight image corresponding to the execution level Quality enhancement tool (may be called tool B), and then tool A can be used to perform image quality enhancement processing on key frame images, and tool B can be used to perform image quality enhancement processing on non-key frame images associated with the key frame image. It can be understood that the heavier the image quality enhancement processing is, the more equipment resources will be consumed.
- the heavyweight image quality enhancement processing on the key frame images can ensure the effect of image quality enhancement and improve the picture quality of the video.
- Lightweight image quality enhancement processing for non-key frame images has little effect on the overall image quality of the video, which can save equipment resources to a certain extent.
- the aforementioned image quality enhancement tool may be an image quality enhancement model established based on a deep learning algorithm, that is, the processing of step 103 may specifically be: using all image quality enhancement models corresponding to the image content category, enhancement type, and execution level, Perform image quality enhancement processing on the video to be processed.
- the training method of the image quality enhancement model can be as follows: obtain multiple high-quality images under the content category of the target image; each high-quality image is subjected to different execution levels of degradation processing corresponding to the target image quality enhancement processing to obtain different executions Degraded images of the order of magnitude; through the target degraded image of the target execution level and the high-quality image corresponding to the target degraded image, the image quality enhancement model established based on the deep learning algorithm is trained to generate the target image content category and target image quality The image quality enhancement model corresponding to the enhancement processing and the target execution level.
- the target image quality enhancement processing can be any enhancement type image quality enhancement processing supported by the network device, the target image content category can be any image content category, and the target execution magnitude can be any execution magnitude.
- the network device can collect a large number of high-quality images in advance, and classify the high-quality images according to the image content categories, and generate high-quality material libraries corresponding to different image content categories. After that, taking the target image content category as an example, the network device can extract multiple high-quality images from the high-quality material library corresponding to the target image content category, and perform the target image quality enhancement processing on each extracted high-quality image corresponding to the corresponding degradation Process to get a degraded image. Next, the network device can establish an image quality enhancement model based on a deep learning algorithm (such as using a convolutional neural network), and then train the image quality enhancement model through the aforementioned degraded images and high-quality images. In this way, after the training is completed, the network device can use the aforementioned image quality enhancement model to perform target image quality enhancement processing on the video frame images of the target image category.
- a deep learning algorithm such as using a convolutional neural network
- the network device can also adjust the parameters of the image quality enhancement model.
- the degraded image can be input into the image quality enhancement model, and then the enhanced image output by the image quality enhancement model and the original Adjust the model parameters of the image quality enhancement model to improve the image quality enhancement effect of the image quality enhancement model.
- network equipment can also train image quality enhancement models corresponding to different execution levels for the image quality enhancement processing of the same enhancement type.
- Each image quality enhancement model is dedicated to executing a single image Content type, single enhancement type, and single execution level of image quality enhancement processing, where the lighter the execution level, the lower the corresponding model complexity, the weaker the image quality enhancement effect, and the equipment resources consumed by performing image quality enhancement processing Less. Accordingly, during the training process, when the network device performs the degradation processing corresponding to the target image quality enhancement processing on the high-quality image, it can perform degradation processing of different execution levels, thereby obtaining degraded images of different execution levels.
- the network device can select the target degraded image of a single execution level (such as the target execution level) and the high-quality image corresponding to the target degraded image as training materials to train the image quality enhancement model, so that the target image can be trained
- the image quality enhancement model corresponding to the content category, target image quality enhancement processing, and target execution level.
- the image content category of the target video frame image of the video to be processed is determined; the image quality of the target video frame image is evaluated to generate an image quality assessment result; according to the image content category and the image quality assessment result, the video to be processed is performed Picture quality enhancement processing.
- the video frame images are classified according to the screen content, and the image quality of the video frame images is evaluated, and then for each type of video frame image, the image quality enhancement processing is carried out with reference to the image quality evaluation results to ensure different screen content
- the image quality of the video frame image is correspondingly enhanced, so that the effect of the image quality enhancement is more prominent.
- an embodiment of the present invention also provides an apparatus for enhancing video quality. As shown in FIG. 5, the apparatus includes:
- the classification module 501 is used to determine the image content category of the target video frame image of the video to be processed
- the evaluation module 502 is configured to evaluate the image quality of the target video frame image, and generate an image quality evaluation result
- the enhancement module 503 is configured to perform image quality enhancement processing on the to-be-processed video according to the image content category and the image quality evaluation result.
- the evaluation module 502 is specifically configured to:
- the image quality evaluation tool corresponding to the image content category is used to evaluate the image quality of the target video frame image to generate an image quality evaluation result.
- the image quality evaluation result includes enhancement types and execution levels of multiple image quality enhancement processing
- the enhancement module 503 is specifically used for:
- the image quality enhancement tool corresponding to the image content category, the enhancement type, and the execution level is used to perform image quality enhancement processing on the to-be-processed video.
- Fig. 6 is a schematic structural diagram of a network device provided by an embodiment of the present invention.
- the network device 600 may have relatively large differences due to different configurations or performances, and may include one or more central processing units 622 (for example, one or more processors) and a memory 632, and one or more storage application programs 642 or
- the storage medium 630 of the data 644 (for example, one or a storage device with a large amount of storage).
- the memory 632 and the storage medium 630 may be short-term storage or persistent storage.
- the program stored in the storage medium 630 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations on the network device 600.
- the central processing unit 622 may be configured to communicate with the storage medium 630, and execute a series of instruction operations in the storage medium 630 on the network device 600.
- the network device 600 may also include one or more power supplies 629, one or more wired or wireless network interfaces 650, one or more input and output interfaces 658, one or more keyboards 656, and/or, one or more operating systems 641, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, etc.
- the network device 600 may include a memory and one or more programs.
- One or more programs are stored in the memory and configured to be executed by one or more processors. The above instructions to enhance the video quality.
- the program can be stored in a computer-readable storage medium.
- the storage medium mentioned can be a read-only memory, a magnetic disk or an optical disk, etc.
Abstract
本发明公开了一种增强视频画质的方法和装置，属于视频处理技术领域。所述方法包括：确定待处理视频的目标视频帧图像的图像内容类别；对所述目标视频帧图像进行画质评估，生成画质评估结果；根据所述图像内容类别和所述画质评估结果，对所述待处理视频进行画质增强处理。采用本发明，可以保证不同画面内容的视频帧图像得到相适应的画质增强，从而使得画质增强的效果更加突出。
Description
本发明涉及视频处理技术领域，特别涉及一种增强视频画质的方法和装置。
随着计算机技术的不断发展，为了满足观众对视频画质日趋增高的需求，视频画质增强已逐渐成为当下视频领域中的热门话题。视频提供方或视频维护方在向用户提供视频前，往往需要对视频帧图像执行画质增强处理，以优化视频内容、改善图像质量、丰富视频信息量。
在现有的视频画质增强处理中，视频提供方或视频维护方可以在获取到视频文件后，对视频文件进行画质增强处理。具体而言，画质增强可包含图像去噪、图像去伪影、色彩增强、图像超分辨率在内的多种处理，视频提供方或视频维护方可以人工选择一种或多种画质增强工具，通过画质增强工具对视频文件依次执行多种画质增强处理，从而可以将画质增强后的视频文件提供给用户。
在实现本发明的过程中，发明人发现现有技术至少存在以下问题：
上述对视频文件进行画质增强的过程中，往往是对不同视频文件中的各视频帧图像进行相同无差别的处理，即对不同画面内容的各视频帧图像使用相同的画质增强策略，这种处理方式无法针对不同画面内容采用最优的画质增强处理，故而画质增强的效果具有局限性。
发明内容
为了解决现有技术的问题，本发明实施例提供了一种增强视频画质的方法和装置。所述技术方案如下：
第一方面，提供了一种增强视频画质的方法，所述方法包括：
确定待处理视频的目标视频帧图像的图像内容类别；
对所述目标视频帧图像进行画质评估，生成画质评估结果；
根据所述图像内容类别和所述画质评估结果，对所述待处理视频进行画质 增强处理。
第二方面，提供了一种增强视频画质的装置，所述装置包括：
分类模块，用于确定待处理视频的目标视频帧图像的图像内容类别；
评估模块，用于对所述目标视频帧图像进行画质评估，生成画质评估结果；
增强模块，用于根据所述图像内容类别和所述画质评估结果，对所述待处理视频进行画质增强处理。
第三方面，提供了一种网络设备，所述网络设备包括处理器和存储器，所述存储器中存储有至少一条指令、至少一段程序、代码集或指令集，所述至少一条指令、所述至少一段程序、所述代码集或指令集由所述处理器加载并执行以实现如第一方面所述的增强视频画质的方法。
第四方面，提供了一种计算机可读存储介质，所述存储介质中存储有至少一条指令、至少一段程序、代码集或指令集，所述至少一条指令、所述至少一段程序、所述代码集或指令集由处理器加载并执行以实现如第一方面所述的增强视频画质的方法。
本发明实施例提供的技术方案带来的有益效果是：
本发明实施例中，确定待处理视频的目标视频帧图像的图像内容类别；对目标视频帧图像进行画质评估，生成画质评估结果；根据图像内容类别和画质评估结果，对待处理视频进行画质增强处理。这样，将视频帧图像按照画面内容进行分类，并对视频帧图像的画质进行评估，然后对于每类视频帧图像，参考画质评估结果针对性地进行画质增强处理，以保证不同画面内容的视频帧图像得到相适应的画质增强，从而使得画质增强的效果更加突出。
为了更清楚地说明本发明实施例中的技术方案，下面将对实施例描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本发明的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下， 还可以根据这些附图获得其他的附图。
图1是本发明实施例提供的一种增强视频画质的方法流程图；
图2是本发明实施例提供的一种增强视频画质的原理示意图；
图3是本发明实施例提供的一种增强视频画质的原理示意图；
图4是本发明实施例提供的一种增强视频画质的原理示意图；
图5是本发明实施例提供的一种增强视频画质的装置结构示意图；
图6是本发明实施例提供的一种网络设备的结构示意图。
为使本发明的目的、技术方案和优点更加清楚，下面将结合附图对本发明实施方式作进一步地详细描述。
本发明实施例提供了一种增强视频画质的方法，所述方法适用于任意具备视频帧图像处理功能的网络设备，具体可以是视频提供方或视频维护方的后台服务器。网络设备在获取到任意视频后，可以对其中的视频帧图像进行不同种类的画质增强处理，以对外提供具备更高画质的视频数据。此处具体的画质增强处理可以由网络设备依据视频帧图像的图像内容来决定，包括但不仅限于以下几种画质增强处理：图像去噪、图像去伪影、色彩增强、图像超分辨率。在对一张视频帧图像进行画质增强处理时，网络设备可以按需要对视频帧图像进行一种或多种画质增强处理，且对于同一视频内的不同视频帧图像，网络设备可以采用相同或不同的画质增强处理。进一步的，针对同一种画质增强处理，网络设备可以支持多种执行量级的画质增强处理。
下面将结合具体实施方式，对图1所示的处理流程进行详细的说明，原理可参考图2所示，内容可以如下：
步骤101，确定待处理视频的目标视频帧图像的图像内容类别。
其中，在不同处理流程下，目标视频帧图像可以是待处理视频中不同的视频帧图像，具体可见后续说明。
在实施中，网络设备在获取到某一视频的视频文件后，可以先根据预设标准判断是否需要对该视频进行画质增强处理。如果需要，网络设备则可以从该视频(可称为待处理视频)的所有视频帧图像中选取目标视频帧图像，然后对目标视频帧图像进行内容识别，从而确定目标视频帧图像的图像内容类别。此 处的图像内容类别可以是网络设备根据视频帧图像的画面内容占比而确定的，如可分为风景、人像、动物、车辆等多种。此外，本实施例中的图像内容类别，应依据画质增强处理的差异而进行划分，即对不同图像内容类别下的图像数据进行的画质增强处理应具备一定的差异。从反面来讲，若两个图像内容类别下的图像数据适合采用相同的画质增强处理，则该两个图像内容类别应合为同一图像内容类别。例如，若存在“男人”和“女人”两个图像内容类别，在其它画质因素一致的情况下，应对两个图像内容类别下的图像数据采用相同的画质增强处理，故而，可以将“男人”和“女人”两个图像内容类别合并为“人物”这一图像内容类别。
值得一提的是，如果目标帧图像经识别后被确定为同时属于多种图像内容类别，网络设备则可以根据待处理视频中目标视频帧图像相邻的多个视频帧图像的图像内容类别，对目标视频帧图像进行进一步的内容分类。这样，由于视频中相邻的视频帧图像有很大概率内容相似，使得内容相似的视频帧图像被划分为同一类别，并且后续采用相同的画质增强处理，避免了因画质增强处理不同而导致相邻视频帧图像的画面差异过大的问题，从而可以提高视频画质增强的效果。
可选的，可以利用基于深度学习算法建立的图像分类模型实现对视频帧图像的画面分类，其中模型训练的方式可以为：通过预设的训练素材集中标记有类别信息的图像数据，对基于深度学习算法建立的图像分类模型进行训练；而步骤101的处理可以如下：将待处理视频的目标视频帧图像输入训练完成的图像分类模型，得到目标视频帧图像的图像内容类别。
在实施中，网络设备可以预先采集大量的不同图像内容类别的图像数据，并标记出每份图像数据的类别信息，进而可以将所有图像数据汇总生成训练素材集。之后，网络设备可以基于深度学习算法(如利用卷积神经网络)建立图像分类模型，再通过上述训练素材集中标记有类别信息的图像数据，对该图像分类模型进行训练。这样，在训练完成后，网络设备可以使用上述图像分类模型对目标视频帧图像进行分类处理，也即可以将待处理视频的目标视频帧图像输入训练完成的图像分类模型，得到目标视频帧图像的图像内容类别。此外，在对图像分类模型训练完成后，网络设备还可以对图像分类模型进行参数调整，具体可以是将不同图像内容类别的图像数据输入图像分类模型，然后根据图像 分类模型输出的分类结果和图像数据的正确类别间的误差，调整图像分类模型的模型参数，以增强图像分类模型对图像分类的精确性。这样，利用深度学习技术实现图像的分类处理，可以提高图像分类的便捷性和准确性。
步骤102，对目标视频帧图像进行画质评估，生成画质评估结果。
在实施中，网络设备在选取了目标视频帧图像之后，可以对目标视频帧图像进行画质评估处理，以从多个维度评估目标视频帧图像的画面质量。画质评估处理的类型可以根据网络设备所支持的画质增强处理的增强类型来确定，如可以包括图像噪声评估、图像伪影评估、图像色彩评估、图像分辨率评估等。之后，网络设备可以汇总生成多维度的画质评估结果，该画质评估结果可以具体用于指示各类画质增强处理的增强类型，例如画质评估结果可以是：“需要图像去噪处理；需要色彩增强处理；不需要图像去伪影处理”。
可选的，对于不同图像内容类别的视频帧图像，可以针对性地分别进行画质评估处理，相应的，步骤102的处理可以如下：利用图像内容类别对应的画质评估工具，对目标视频帧图像进行画质评估，生成画质评估结果。
在实施中，网络设备上可以设置有不同图像内容类别对应的不同画质评估工具，每个画质评估工具可以专用于对一个图像内容类别下的视频帧图像进行画质评估。这样，在确定了目标视频帧图像的图像内容类别后，网络设备可以利用该图像内容类别对应的画质评估工具，对目标视频帧图像进行画质评估，从而生成画质评估结果。可以理解，对不同图像内容类别的视频帧图像分开进行针对性的画质评估，可以提高画质评估的准确性。值得一提的是，画质评估处理是从图像噪声评估、图像伪影评估、图像色彩评估、图像分辨率评估等多维度评估视频帧图像的画质，相应的，在进行画质评估处理时可以采用具备所有维度评估能力的一个画质评估工具，也可以分别采用多个对应不同维度的画质评估工具，如图像噪声评估工具、图像伪影评估工具等。上述内容的相关流程可参考图3所示。
可选的，上述画质评估工具可以是基于深度学习算法建立的画质评估模型，即步骤102的处理具体可以为：将目标视频帧图像输入图像内容类别对应的画质评估模型，得到画质评估结果。
而其中画质评估模型的训练方式可以如下：获取目标图像内容类别下的多张优质图像；对每张优质图像分别进行不同降质类型的降质处理，得到标记有 降质类型信息的降质图像；通过所有降质图像对基于深度学习算法建立的画质评估模型进行训练，生成目标图像内容类别对应的画质评估模型。
其中，目标图像内容类别可以是任一图像内容类别，优质图像可以是画质高于预设标准、存在一定的降质空间，如清晰度为高清以上、噪声占比小于10％或图像伪影占画面10％以下的图像。
在实施中，网络设备可以预先采集大量的优质图像生成优质素材库，之后网络设备可以从优质素材库中抽取多张优质图像，并对抽取到的每张优质图像分别进行不同降质类型的降质处理，最后得到标记有降质类型信息的降质图像。此处，降质处理可以和画质增强处理对应，若画质增强处理为图像去噪处理，降质处理则为图像增噪处理；若画质增强处理为色彩增强处理，降质处理则为色彩弱化处理。接下来，网络设备可以基于深度学习算法(如利用卷积神经网络)建立画质评估模型，再通过上述标记有降质类型信息的降质图像，对该画质评估模型进行训练。这样，在训练完成后，网络设备则可以使用上述画质评估模型对目标视频帧图像进行画质评估处理。需要补充的是，如果上述画质评估模型具备所有维度的画质评估能力，则在训练时可以将多种降质类型下的降质图像统一作为画质评估模型的训练素材；而如果上述画质评估模型仅具备单一维度的画质评估能力，则在训练时仅将单一降质类型下的降质图像作为画质评估模型的训练素材，并且按照上述训练过程，针对各个维度训练生成一个对应的画质评估模型。
此外，在对画质评估模型训练完成后，网络设备还可以对画质评估模型进行参数调整，具体可以是将降质图像输入画质评估模型，然后根据画质评估模型输出的画质评估结果和降质图像对应的降质类型，调整画质评估模型的模型参数，以增强画质评估模型进行画质评估时的精确性。
以上述画质评估模型的训练过程为基础，还可以针对不同图像内容类别分别生成多个专用于评估单一图像内容类别的画质评估模型。在训练过程中，网络设备需要抽取同一图形内容类别的多张优质图像，再执行后续处理，从而可以生成该图形内容类别对应的画质评估模型。
进一步的，画质评估模型还可以具体评估出在进行画质增强处理时所需的执行量级，图像的原画质越好，执行量级越轻，画质增强的效果越弱。据此，在训练过程中，网络设备在对优质图像进行降质处理时，对于同一降质类型需 要执行不同执行量级的降质处理，最后得到标记有降质类型信息以及执行量级信息的降质图像。接下来，网络设备可以通过标记有降质类型信息以及执行量级信息的降质图像，对画质评估模型进行训练，从而可以训练得到能够评估出图像所需画质增强处理的执行量级的画质评估模型。
步骤103，根据图像内容类别和画质评估结果，对待处理视频进行画质增强处理。
在实施中，网络设备在确定了目标视频帧图像的图像内容类别和画质评估结果后，可以以该图像内容类别和画质评估结果为参考，对待处理视频的部分视频帧图像进行画质增强处理。具体来说，网络设备可以针对不同图像内容类别和画质评估结果，对待处理视频的视频帧图像执行不同的画质增强处理。更进一步的，网络设备上可以存在不同图像内容类别和画质评估结果对应的多个画质增强工具，在执行画质增强处理时，网络设备可以根据目标视频帧图像的图像内容类别和画质评估结果，挑选对应的画质增强工具，然后利用这些画质增强工具对待处理视频进行画质增强处理。
值得一提的是，网络设备可以根据目标视频帧图像的选取规则，决定执行画质增强处理的视频帧图像范围，具体内容可以参考后续说明。同时，网络设备还可以根据目标视频帧图像的图像内容类别，来决定是否执行画质增强处理。例如，对于主播类视频，当图像内容类别为风景或物体等与视频的主要内容关联性较差的类别时，则可以不进行画质增强处理，或选择执行量级更轻的画质增强处理，从而可以在保证视频的主要画面质量的同时，减少画质增强所消耗的设备资源。
可选的，对于目标视频帧图像的不同选取规则，执行画质增强处理的视频帧图像范围也不同，如下给出了几种情况：
其一，如果目标视频帧图像为待处理视频的任一帧图像，则对目标视频帧图像进行画质增强处理。
在实施中，网络设备可以针对待处理视频中的每一帧图像均完整的执行步骤101至103的处理，那么目标视频帧图像可以是待处理视频中的任一帧图像。故而，网络设备可以根据目标视频帧图像的图像内容类别和画质评估结果，直接对目标视频帧图像进行画质增强处理。这样，对视频进行逐帧内容分类、画质评估、画质增强的处理，可以有效保证每个视频帧图像得到针对性的、有效 的画质增强处理，从而可以提高画质增强的效果。进一步的，网络设备可以仅对视频特定段(如视频高潮段)内的每一帧图像进行处理，该视频高潮段可以是人工预先标记的，也可以是网络设备预先自行检测的，具体检测方式可以采用现有技术中已公开的内容，本实施例不再赘述。
其二，如果从待处理视频的所有视频帧图像中按照预设间隔选取目标视频帧图像，则对目标视频帧图像及目标视频帧图像对应的间隔帧图像进行画质增强处理。
在实施中，网络设备可以在待处理视频的所有视频帧图像中，按照预设间隔选取目标视频帧图像，如每隔5个视频帧图像选取一个目标视频帧图像，即第1个、第6个、第11个…第5n+1个(n为自然数)视频帧图像为目标视频帧图像。相应的，两个目标视频帧图像之间的所有视频帧图像即为间隔帧图像，此处可以定义间隔帧图像与前一目标视频帧图像对应，即第5n+2个至第5n+5个视频帧图像为第5n+1个目标视频帧图像对应的间隔帧图像。之后，网络设备可以根据目标视频帧图像的图像内容分类和画质评估结果，对目标视频帧图像以及目标视频帧图像对应的间隔帧图像进行画质增强处理。这样，由于视频中相邻的视频帧图像内容基本相似，故而既可以一定程度上保证对视频帧图像执行的画质增强处理的针对性和有效性，也可以大幅减少对视频帧图像进行画面分类及画质评估的处理，从而可以节省大量设备处理资源。
其三，如果选取待处理视频的关键帧图像作为目标视频帧图像，则对待处理视频的关键帧图像进行画质增强处理。
在实施中，基于现有的视频压缩编码技术，视频帧可以包含I帧、B帧和P帧，其中，I帧为自带图像全部画面内容的关键帧，P帧记录的是与前一帧的差别；B帧记录的是前一帧及后一帧的差别。相对来说，关键帧图像的画面内容将会影响其它非关键帧图像的画面内容，故而，在设备处理资源不足的情况下，可以仅选择对关键帧图像进行画面分类、画质评估和画质增强，从而可以最大程度保证视频整体的画质增强效果。因此，网络设备可以选择待处理视频的关键帧图像作为目标视频帧图像，并在执行画质增强处理时，仅对待处理视频的关键帧图像进行画质增强处理。
在另一场景下，上述关键帧图像也可以是人工定义或者设备自行检测的，符合预设标准的视频帧图像，如可以定义存在人脸的视频帧图像即为关键帧图 像，或者特定时间戳对应的视频帧图像即为关键帧图像等。
可选的，网络设备可以对图像所需的画质增强处理的增强类型和执行量级进行评估，进而利用对应的画质增强工具来进行画质增强处理。相应的，步骤103的处理可以如下：利用图像内容类别、增强类型和执行量级对应的画质增强工具，对待处理视频进行画质增强处理。
在实施中，网络设备上可以设置有针对不同图像内容类别的图像进行画质增强的画质增强工具。一个画质增强工具可以用于单一增强类型的处理，同一增强类型下可以存在多个不同执行量级对应的画质增强工具，每个画质增强工具用于执行一种执行量级的画质增强处理。基于上述设置，网络设备在对目标视频帧图像进行画质评估时，可以得到包含有所需的多种画质增强处理的增强类型和执行量级的画质评估结果，进而可以利用目标视频帧图像的图像内容类别，以及上述增强类型和执行量级，对待处理视频进行画质增强处理。上述内容的相关流程可参考图4所示。
可选的，基于上述存在执行量级的情况，网络设备可以选择对视频中的关键帧图像和非关键帧图像进行不同执行量级对应的画质增强处理，相应的，若选取待处理视频的关键帧图像作为目标视频帧图像，对待处理视频的画质增强处理则具体可以如下：利用图像内容类别、增强类型和执行量级对应的画质增强工具，对关键帧图像进行画质增强处理；利用图像内容类别、增强类型和执行量级的轻量级对应的画质增强工具，对关键帧图像关联的非关键帧图像进行画质增强处理。
在实施中，网络设备若选取待处理视频的关键帧图像作为目标视频帧图像，则在获取了关键帧图像的图像内容类别和画质评估结果后，可以先确定所有满足其图像内容类别和画质评估结果中增强类型的画质增强工具，然后从中选取画质评估结果中的执行量级对应的画质增强工具(可称为工具A)，以及该执行量级的轻量级对应的画质增强工具(可称为工具B)，进而可以使用工具A对关键帧图像进行画质增强处理，使用工具B对关键帧图像关联的非关键帧图像进行画质增强处理。可以理解，画质增强处理的量级越重，则消耗的设备资源越多，故而对关键帧图像进行重量级的画质增强处理，可以保证画质增强的效果，提高视频的画面质量，而对非关键帧图像进行较轻量级的画质增强处理，对视频整体的画面质量影响不大，可以一定程度上节省设备资源。
可选的，上述画质增强工具可以是基于深度学习算法建立的画质增强模型，即步骤103的处理具体可以为：利用图像内容类别、增强类型和执行量级对应的所有画质增强模型，对待处理视频进行画质增强处理。
而其中画质增强模型的训练方式可以如下：获取目标图像内容类别下的多张优质图像；对每张优质图像分别进行目标画质增强处理对应的不同执行量级的降质处理，得到不同执行量级的降质图像；通过目标执行量级的目标降质图像和目标降质图像对应的优质图像，对基于深度学习算法建立的画质增强模型进行训练，生成目标图像内容类别、目标画质增强处理和目标执行量级对应的画质增强模型。
其中，目标画质增强处理可以是网络设备支持的任一增强类型的画质增强处理，目标图像内容类别可以是任一图像内容类别，目标执行量级可以是任一执行量级。
在实施中，网络设备可以预先采集大量的优质图像，并按照图像内容类别对优质图像进行分类，生成不同图像内容类别对应的优质素材库。之后，以目标图像内容类别为例，网络设备可以从目标图像内容类别对应的优质素材库中抽取多张优质图像，并对抽取到的每张优质图像分别进行目标画质增强处理对应的降质处理，得到降质图像。接下来，网络设备可以基于深度学习算法(如利用卷积神经网络)建立画质增强模型，再通过上述降质图像和优质图像，对该画质增强模型进行训练。这样，在训练完成后，网络设备则可以使用上述画质增强模型对目标图像类别的视频帧图像进行目标画质增强处理。
此外，在对画质增强模型训练完成后，网络设备还可以对画质增强模型进行参数调整，具体可以是将降质图像输入画质增强模型，然后根据画质增强模型输出的增强图像和原先的优质图像，调整画质增强模型的模型参数，以提高画质增强模型的画质增强效果。
以上述画质增强模型的训练过程为基础，网络设备还可以针对同一增强类型的画质增强处理，训练出不同执行量级对应的画质增强模型，每个画质增强模型专用于执行单一图像内容类型、单一增强类型和单一执行量级的画质增强处理，其中，执行量级越轻，对应的模型复杂程度越低，画质增强效果越弱，执行画质增强处理所消耗的设备资源越少。据此，在训练过程中，网络设备在对优质图像进行目标画质增强处理对应的降质处理时，可以进行不同执行量级 的降质处理，从而得到不同执行量级的降质图像。接下来，网络设备可以选取单一执行量级(如目标执行量级)的目标降质图像和目标降质图像对应的优质图像，作为训练素材对画质增强模型进行训练，从而可以训练得到目标图像内容类别、目标画质增强处理和目标执行量级对应的画质增强模型。
本发明实施例中，确定待处理视频的目标视频帧图像的图像内容类别；对目标视频帧图像进行画质评估，生成画质评估结果；根据图像内容类别和画质评估结果，对待处理视频进行画质增强处理。这样，将视频帧图像按照画面内容进行分类，并对视频帧图像的画质进行评估，然后对于每类视频帧图像，参考画质评估结果针对性地进行画质增强处理，以保证不同画面内容的视频帧图像得到相适应的画质增强，从而使得画质增强的效果更加突出。
基于相同的技术构思，本发明实施例还提供了一种增强视频画质的装置，如图5所示，所述装置包括：
分类模块501，用于确定待处理视频的目标视频帧图像的图像内容类别；
评估模块502，用于对所述目标视频帧图像进行画质评估，生成画质评估结果；
增强模块503，用于根据所述图像内容类别和所述画质评估结果，对所述待处理视频进行画质增强处理。
可选的，所述评估模块502，具体用于：
利用所述图像内容类别对应的画质评估工具，对所述目标视频帧图像进行画质评估，生成画质评估结果。
可选的，所述画质评估结果包括多种画质增强处理的增强类型和执行量级；
所述增强模块503，具体用于：
利用所述图像内容类别、所述增强类型和所述执行量级对应的画质增强工具，对所述待处理视频进行画质增强处理。
图6是本发明实施例提供的网络设备的结构示意图。该网络设备600可因配置或性能不同而产生比较大的差异，可以包括一个或一个以上中央处理器622(例如，一个或一个以上处理器)和存储器632，一个或一个以上存储应用程序642或数据644的存储介质630(例如一个或一个以上海量存储设备)。其中， 存储器632和存储介质630可以是短暂存储或持久存储。存储在存储介质630的程序可以包括一个或一个以上模块(图示没标出)，每个模块可以包括对网络设备600中的一系列指令操作。更进一步地，中央处理器622可以设置为与存储介质630通信，在网络设备600上执行存储介质630中的一系列指令操作。
网络设备600还可以包括一个或一个以上电源629，一个或一个以上有线或无线网络接口650，一个或一个以上输入输出接口658，一个或一个以上键盘656，和/或，一个或一个以上操作系统641，例如Windows Server，Mac OS X，Unix，Linux，FreeBSD等等。
网络设备600可以包括有存储器，以及一个或者一个以上的程序，其中一个或者一个以上程序存储于存储器中，且经配置以由一个或者一个以上处理器执行所述一个或者一个以上程序包含用于进行上述增强视频画质的指令。
本领域普通技术人员可以理解实现上述实施例的全部或部分步骤可以通过硬件来完成，也可以通过程序来指令相关的硬件完成，所述的程序可以存储于一种计算机可读存储介质中，上述提到的存储介质可以是只读存储器，磁盘或光盘等。
以上所述仅为本发明的较佳实施例，并不用以限制本发明，凡在本发明的精神和原则之内，所作的任何修改、等同替换、改进等，均应包含在本发明的保护范围之内。
Claims (15)
- 一种增强视频画质的方法，其特征在于，所述方法包括：确定待处理视频的目标视频帧图像的图像内容类别；对所述目标视频帧图像进行画质评估，生成画质评估结果；根据所述图像内容类别和所述画质评估结果，对所述待处理视频进行画质增强处理。
- 根据权利要求1所述的方法，其特征在于，所述目标视频帧图像为所述待处理视频的任一帧图像；所述对所述待处理视频进行画质增强处理，包括：对所述目标视频帧图像进行画质增强处理。
- 根据权利要求1所述的方法，其特征在于，所述确定待处理视频的目标视频帧图像的图像内容类别之前，还包括：从所述待处理视频的所有视频帧图像中按照预设间隔选取目标视频帧图像；所述对所述待处理视频进行画质增强处理，包括：对所述目标视频帧图像及所述目标视频帧图像对应的间隔帧图像进行画质增强处理。
- 根据权利要求1所述的方法，其特征在于，所述确定待处理视频的目标视频帧图像的图像内容类别之前，还包括：选取所述待处理视频的关键帧图像作为目标视频帧图像；所述对所述待处理视频进行画质增强处理，包括：对所述待处理视频的关键帧图像进行画质增强处理。
- 根据权利要求1所述的方法，其特征在于，所述方法还包括：通过预设的训练素材集中标记有类别信息的图像数据，对基于深度学习算法建立的图像分类模型进行训练；所述确定待处理视频的目标视频帧图像的图像内容类别，包括：将所述待处理视频的目标视频帧图像输入训练完成的所述图像分类模型，得到所述目标视频帧图像的图像内容类别。
- 根据权利要求1所述的方法，其特征在于，所述对所述目标视频帧图像进行画质评估，生成画质评估结果，包括：利用所述图像内容类别对应的画质评估工具，对所述目标视频帧图像进行画质评估，生成画质评估结果。
- 根据权利要求6所述的方法，其特征在于，所述方法还包括：获取目标图像内容类别下的多张优质图像；对每张所述优质图像分别进行不同降质类型的降质处理，得到标记有降质类型信息的降质图像；通过所有所述降质图像对基于深度学习算法建立的画质评估模型进行训练，生成所述目标图像内容类别对应的画质评估模型；所述利用所述图像内容类别对应的画质评估工具，对所述目标视频帧图像进行画质评估，生成画质评估结果，包括：将所述目标视频帧图像输入所述图像内容类别对应的画质评估模型，得到画质评估结果。
- 根据权利要求1所述的方法，其特征在于，所述画质评估结果包括多种画质增强处理的增强类型和执行量级；所述根据所述图像内容类别和所述画质评估结果，对所述待处理视频进行画质增强处理，包括：利用所述图像内容类别、所述增强类型和所述执行量级对应的画质增强工具，对所述待处理视频进行画质增强处理。
- 根据权利要求8所述的方法，其特征在于，所述确定待处理视频的目标视频帧图像的图像内容类别之前，还包括：选取所述待处理视频的关键帧图像作为目标视频帧图像；所述利用所述图像内容类别、所述增强类型和所述执行量级对应的画质增 强工具，对所述待处理视频进行画质增强处理，包括：利用所述图像内容类别、所述增强类型和所述执行量级对应的画质增强工具，对所述关键帧图像进行画质增强处理；利用所述图像内容类别、所述增强类型和所述执行量级的轻量级对应的画质增强工具，对所述关键帧图像关联的非关键帧图像进行画质增强处理。
- 根据权利要求8所述的方法，其特征在于，所述方法还包括：获取目标图像内容类别下的多张优质图像；对每张所述优质图像分别进行目标画质增强处理对应的不同执行量级的降质处理，得到不同执行量级的降质图像；通过目标执行量级的目标降质图像和所述目标降质图像对应的优质图像，对基于深度学习算法建立的画质增强模型进行训练，生成所述目标图像内容类别、所述目标画质增强处理和所述目标执行量级对应的画质增强模型；所述利用所述图像内容类别、所述增强类型和所述执行量级对应的画质增强工具，对所述待处理视频进行画质增强处理，包括：利用所述图像内容类别、所述增强类型和所述执行量级对应的所有画质增强模型，对所述待处理视频进行画质增强处理。
- 一种增强视频画质的装置，其特征在于，所述装置包括：分类模块，用于确定待处理视频的目标视频帧图像的图像内容类别；评估模块，用于对所述目标视频帧图像进行画质评估，生成画质评估结果；增强模块，用于根据所述图像内容类别和所述画质评估结果，对所述待处理视频进行画质增强处理。
- 根据权利要求11所述的装置，其特征在于，所述评估模块，具体用于：利用所述图像内容类别对应的画质评估工具，对所述目标视频帧图像进行画质评估，生成画质评估结果。
- 根据权利要求11所述的装置，其特征在于，所述画质评估结果包括多种画质增强处理的增强类型和执行量级；所述增强模块，具体用于：利用所述图像内容类别、所述增强类型和所述执行量级对应的画质增强工具，对所述待处理视频进行画质增强处理。
- 一种网络设备，其特征在于，所述网络设备包括处理器和存储器，所述存储器中存储有至少一条指令、至少一段程序、代码集或指令集，所述至少一条指令、所述至少一段程序、所述代码集或指令集由所述处理器加载并执行以实现如权利要求1至10任一所述的增强视频画质的方法。
- 一种计算机可读存储介质，其特征在于，所述存储介质中存储有至少一条指令、至少一段程序、代码集或指令集，所述至少一条指令、所述至少一段程序、所述代码集或指令集由处理器加载并执行以实现如权利要求1至10任一所述的增强视频画质的方法。
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911031293.9 | 2019-10-28 | ||
| CN201911031293.9A CN111031346B (zh) | 2019-10-28 | 2019-10-28 | 一种增强视频画质的方法和装置 |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| WO2021082171A1 true WO2021082171A1 (zh) | 2021-05-06 |
Family
ID=70200123
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| PCT/CN2019/123009 WO2021082171A1 (zh) | 2019-10-28 | 2019-12-04 | 一种增强视频画质的方法和装置 |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN111031346B (zh) |
| WO (1) | WO2021082171A1 (zh) |
Families Citing this family (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113706393A (zh) * | 2020-05-20 | 2021-11-26 | 武汉Tcl集团工业研究院有限公司 | 视频增强方法、装置、设备及存储介质 |
| CN111970564B (zh) * | 2020-08-26 | 2023-03-24 | 展讯通信（上海）有限公司 | Hdr视频显示处理的优化方法及装置、存储介质、终端 |
| CN114363693B (zh) * | 2020-10-13 | 2023-05-12 | 华为技术有限公司 | 画质调整方法及装置 |
| CN112261383A (zh) * | 2020-10-27 | 2021-01-22 | 广西壮族自治区水产科学研究院 | 一种罗非鱼养殖水实时监测预警集成方法及集成系统 |
| CN113115075B (zh) * | 2021-03-23 | 2023-05-26 | 广州虎牙科技有限公司 | 一种视频画质增强的方法、装置、设备以及存储介质 |
| CN113194324B (zh) * | 2021-04-27 | 2022-07-29 | 广州虎牙科技有限公司 | 一种视频帧画质增强方法、直播服务端及电子设备 |
| CN113763296A (zh) * | 2021-04-28 | 2021-12-07 | 腾讯云计算（北京）有限责任公司 | 图像处理方法、设备以及介质 |
| CN113507598B (zh) * | 2021-07-09 | 2023-07-11 | Oppo广东移动通信有限公司 | 视频画面的显示方法、装置、终端及存储介质 |
| CN113628100A (zh) * | 2021-08-10 | 2021-11-09 | Oppo广东移动通信有限公司 | 视频增强方法、装置、终端及存储介质 |
| CN115914765A (zh) * | 2021-08-18 | 2023-04-04 | 北京字跳网络技术有限公司 | 一种画质调节方法、装置、设备及介质 |
| WO2023178648A1 (zh) * | 2022-03-25 | 2023-09-28 | 京东方科技集团股份有限公司 | 视频处理方法及装置、电子设备、计算机可读存储介质 |
| CN114463185B (zh) * | 2022-04-12 | 2022-11-01 | 山东百盟信息技术有限公司 | 一种短视频制作用图像信息处理方法 |
| CN115379259B (zh) * | 2022-08-18 | 2024-04-26 | 百度在线网络技术（北京）有限公司 | 视频处理方法、装置、电子设备和存储介质 |
| CN117437163A (zh) * | 2023-10-31 | 2024-01-23 | 奕行智能科技(广州)有限公司 | 一种图像增强方法、图像处理芯片及图像增强视频系统 |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20060092474A1 (en) * | 2000-02-22 | 2006-05-04 | Visualgold.Com, Inc. | System and method for image processing |
| CN104299196A (zh) * | 2014-10-11 | 2015-01-21 | 京东方科技集团股份有限公司 | 一种图像处理装置及方法、显示设备 |
| CN107481327A (zh) * | 2017-09-08 | 2017-12-15 | 腾讯科技（深圳）有限公司 | 关于增强现实场景的处理方法、装置、终端设备及系统 |
| CN108235117A (zh) * | 2018-01-29 | 2018-06-29 | 北京奇虎科技有限公司 | 一种视频调色方法和装置 |
| CN109934776A (zh) * | 2018-12-25 | 2019-06-25 | 北京奇艺世纪科技有限公司 | 模型生成方法、视频增强方法、装置及计算机可读存储介质 |
Family Cites Families (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7657111B2 (en) * | 2003-02-14 | 2010-02-02 | Fujifilm Corporation | Apparatus and program for image processing for obtaining processed compressed moving image data |
| US20150178896A1 (en) * | 2013-12-20 | 2015-06-25 | Zhigang Fan | Image processing and enhancement methods and associated display systems |
| CN106550244A (zh) * | 2015-09-16 | 2017-03-29 | 广州市动景计算机科技有限公司 | 视频图像的画质增强方法及装置 |
| CN106228518B (zh) * | 2016-07-18 | 2018-04-27 | 北京小米移动软件有限公司 | 可读性增强方法及装置 |
| CN107437096B (zh) * | 2017-07-28 | 2020-06-26 | 北京大学 | 基于参数高效的深度残差网络模型的图像分类方法 |
| CN107809559A (zh) * | 2017-09-21 | 2018-03-16 | 中国科学院长春光学精密机械与物理研究所 | 一种图像自学习增强方法和系统 |
| CN107730473A (zh) * | 2017-11-03 | 2018-02-23 | 中国矿业大学 | 一种基于深度神经网络的煤矿井下图像处理方法 |
| CN110278486A (zh) * | 2018-03-14 | 2019-09-24 | 阿里巴巴集团控股有限公司 | 图像处理、图像分辨率处理参数的生成方法及装置 |
| CN109064416B (zh) * | 2018-07-10 | 2021-06-15 | Oppo广东移动通信有限公司 | 图像处理方法、装置、存储介质及电子设备 |
| CN108810620B (zh) * | 2018-07-18 | 2021-08-17 | 腾讯科技（深圳）有限公司 | 识别视频中的关键时间点的方法、装置、设备及存储介质 |
| CN109525901B (zh) * | 2018-11-27 | 2020-08-25 | Oppo广东移动通信有限公司 | 视频处理方法、装置、电子设备及计算机可读介质 |
| CN110163810B (zh) * | 2019-04-08 | 2023-04-25 | 腾讯科技（深圳）有限公司 | 一种图像处理方法、装置以及终端 |
| CN110211119B (zh) * | 2019-06-04 | 2021-06-04 | 厦门美图之家科技有限公司 | 图像质量评估方法、装置、电子设备及可读存储介质 |
-
2019
- 2019-10-28 CN CN201911031293.9A patent/CN111031346B/zh active Active
- 2019-12-04 WO PCT/CN2019/123009 patent/WO2021082171A1/zh active Application Filing
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20060092474A1 (en) * | 2000-02-22 | 2006-05-04 | Visualgold.Com, Inc. | System and method for image processing |
| CN104299196A (zh) * | 2014-10-11 | 2015-01-21 | 京东方科技集团股份有限公司 | 一种图像处理装置及方法、显示设备 |
| CN107481327A (zh) * | 2017-09-08 | 2017-12-15 | 腾讯科技（深圳）有限公司 | 关于增强现实场景的处理方法、装置、终端设备及系统 |
| CN108235117A (zh) * | 2018-01-29 | 2018-06-29 | 北京奇虎科技有限公司 | 一种视频调色方法和装置 |
| CN109934776A (zh) * | 2018-12-25 | 2019-06-25 | 北京奇艺世纪科技有限公司 | 模型生成方法、视频增强方法、装置及计算机可读存储介质 |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111031346A (zh) | 2020-04-17 |
| CN111031346B (zh) | 2021-11-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2021082171A1 (zh) | 一种增强视频画质的方法和装置 | |
| US9934558B2 (en) | Automatic video quality enhancement with temporal smoothing and user override | |
| CN110832583A (zh) | 用于从多个图像帧生成概要故事板的系统和方法 | |
| US10614347B2 (en) | Identifying parameter image adjustments using image variation and sequential processing | |
| US11068746B2 (en) | Image realism predictor | |
| US10885718B2 (en) | Methods and systems for representing a pre-modeled object within virtual reality data | |
| JP2013507674A (ja) | デジタル画像を低ダイナミック・レンジ（ｌｄｒ）画像から高ダイナミック・レンジ（ｈｄｒ）画像に変換するための方法およびシステム | |
| US9799099B2 (en) | Systems and methods for automatic image editing | |
| Sinno et al. | Large scale subjective video quality study | |
| US10885308B2 (en) | Method and apparatus for evaluating illumination condition in face image | |
| CN110852965A (zh) | 一种基于生成对抗网络的视频光照增强方法及系统 | |
| CN103748613A (zh) | 用于图像去噪、压缩和插值的各向异性梯度正则化 | |
| KR102294817B1 (ko) | 동영상 분석 장치 및 방법 | |
| US11645579B2 (en) | Automated machine learning tagging and optimization of review procedures | |
| JP2013157964A (ja) | 画像処理プログラム、画像処理装置および画像処理方法 | |
| US20230117603A1 (en) | Quality estimation models for various signal characteristics | |
| CN112347990A (zh) | 基于多模态智能审稿系统及方法 | |
| US20240062545A1 (en) | Information processing device, information processing method, and recording medium | |
| US11042274B2 (en) | Extracting demonstrations from in-situ video content | |
| Gao et al. | Saliency-based parameter tuning for tone mapping | |
| US11527106B1 (en) | Automated video verification | |
| TWI820889B (zh) | 圖像處理方法及其裝置 | |
| US20230368490A1 (en) | Preview and capture of stroke outlines from images and video | |
| US20240005960A1 (en) | Scene-based edit suggestions for videos | |
| JP2021190807A (ja) | 画像処理装置、画像処理方法 |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| 121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 19951055 Country of ref document: EP Kind code of ref document: A1 |
|
| NENP | Non-entry into the national phase |
Ref country code: DE |
|
| 122 | Ep: pct application non-entry in european phase |
Ref document number: 19951055 Country of ref document: EP Kind code of ref document: A1 |