CN111932462B

CN111932462B - Training method and device for image degradation model, electronic equipment and storage medium

Info

Publication number: CN111932462B
Application number: CN202010830770.4A
Authority: CN
Inventors: 李兴龙
Original assignee: Oppo Chongqing Intelligent Technology Co Ltd
Current assignee: Oppo Chongqing Intelligent Technology Co Ltd
Priority date: 2020-08-18
Filing date: 2020-08-18
Publication date: 2023-01-03
Anticipated expiration: 2040-08-18
Also published as: CN111932462A

Abstract

The application relates to a training method and a device of an image degradation model, an electronic device and a computer readable storage medium, wherein the training method of the image degradation model comprises the following steps: acquiring a first-resolution image, and carrying out developing processing on the first-resolution image to obtain a developing image; shooting the developing image to obtain a second resolution image; the second resolution corresponding to the second resolution image is smaller than the first resolution corresponding to the first resolution image; performing resolution reduction processing on the first resolution image through an image degradation model to be trained to obtain a degradation image; and adjusting parameters of the image degradation model based on the difference between the degraded image and the second resolution image, and obtaining a trained image degradation model when a training stopping condition is met. By the scheme, the high-resolution image can be accurately converted into the low-resolution image so as to expand the training image.

Description

Training method and device for image degradation model, electronic equipment and storage medium

Technical Field

The present application relates to the field of computer technologies, and in particular, to a method and an apparatus for training an image degradation model, an electronic device, and a computer-readable storage medium.

Background

Super Resolution (SR) is a method for improving the Resolution of an image, and can change a low-Resolution image or video into a clear high-Resolution result without changing hardware equipment. When super-resolution reconstruction is carried out, a training data set is often required to be acquired, and a related algorithm of the super-resolution reconstruction is trained through the training data set so as to improve the performance of the super-resolution reconstruction.

However, the acquisition mode of the training data set of the conventional super-resolution reconstruction generally includes sampling an original high-resolution image twice and three times to obtain a corresponding low-resolution image, thereby obtaining the training data set. However, in practice the image will undergo many complex degradative processes such as motion blur, defocus blur, noise, etc., and the training data set obtained by just bi-cubic down-sampling is inaccurate.

Disclosure of Invention

The embodiment of the application provides a training method and device for an image degradation model, electronic equipment and a computer readable storage medium, and a low-resolution image corresponding to a high-resolution image can be accurately generated.

A training method of an image degradation model comprises the following steps:

acquiring a first-resolution image, and carrying out developing processing on the first-resolution image to obtain a developing image;

shooting the developing image to obtain a second resolution image; the second resolution corresponding to the second resolution image is smaller than the first resolution corresponding to the first resolution image;

performing resolution reduction processing on the first resolution image through an image degradation model to be trained to obtain a degradation image;

and adjusting parameters of the image degradation model based on the difference between the degraded image and the second resolution image, and obtaining a trained image degradation model when a training stopping condition is met.

An apparatus for training an image degradation model, comprising:

the acquisition module is used for acquiring a first resolution image and carrying out developing processing on the first resolution image to obtain a developing image;

the shooting module is used for shooting the developed image to obtain a second resolution image; the second resolution corresponding to the second resolution image is smaller than the first resolution corresponding to the first resolution image;

the training module is used for carrying out resolution reduction processing on the first resolution image through an image degradation model to be trained to obtain a degradation image;

and the adjusting module is used for adjusting the parameters of the image degradation model based on the difference between the degradation image and the second resolution image, and obtaining the trained image degradation model when the training stopping condition is met.

A computer device comprising a memory and a processor, the memory storing a computer program, the processor implementing the following steps when executing the computer program:

A computer-readable storage medium, on which a computer program is stored which, when executed by a processor, carries out the steps of:

According to the training method and device for the image degradation model, the electronic equipment and the computer readable storage medium, the image with the first resolution ratio is processed in a printing mode, the image with the first resolution ratio is shot, so that the degradation process of the image is simulated, and the image with the second resolution ratio after the resolution ratio is reduced is obtained. The first resolution image and the second resolution image are used for training an image degradation model, the second resolution image is used as a label, namely, the high resolution image and the corresponding low resolution image are used as a training image pair, and the low resolution image is used as a label, so that the obtained training image pair is more accurate and more conforms to the training image set obtained in the actual situation. And carrying out resolution reduction processing on the first resolution image according to the image degradation model to obtain a degradation image output by the model, namely obtaining a low resolution image. And adjusting model parameters according to the difference between the low-resolution image output by the image degradation model and the low-resolution image serving as a reference until the trained image degradation model is obtained when the training stopping condition is met. The trained image degradation model can accurately and quickly convert the input high-resolution image into a corresponding low-resolution image, so that a large number of high-resolution and low-resolution training image pairs can be quickly generated, and a training image data set is expanded.

Drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.

FIG. 1 is a diagram illustrating an exemplary environment in which a method for training an image degradation model may be implemented;

FIG. 2 is a schematic diagram illustrating a method for training an image degradation model according to an embodiment;

FIG. 3 is a flow diagram of a method for training an image degradation model in one embodiment;

FIG. 4 is a flow diagram of a process for registering a first resolution image based on a second resolution image in one embodiment;

FIG. 5 is a flow diagram of adjusting parameters of an image degradation model based on a difference between a degraded image and a second resolution image in one embodiment;

FIG. 6 is a flow diagram of resolution reduction processing of a first resolution image by an image degradation model to be trained in one embodiment;

FIG. 7 is a flow diagram of a process for processing a first resolution image in one embodiment;

FIG. 8 is a flowchart illustrating resolution reduction processing performed on an image to be processed by a trained image degradation model according to an embodiment;

FIG. 9 is a block diagram showing an example of an apparatus for a method of training an image degradation model;

FIG. 10 is a diagram illustrating an internal structure of an electronic device in one embodiment.

Detailed Description

In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.

It will be understood that, as used herein, the terms "first," "second," and the like may be used herein to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish one element from another. For example, the first resolution image may be referred to as a second resolution image, and similarly, the second resolution image may be referred to as a first resolution image, without departing from the scope of the present application. The first-resolution image and the second-resolution image are both images, but they are not images of the same resolution.

FIG. 1 is a diagram illustrating an application environment of a training method for an image degradation model according to an embodiment. As shown in fig. 1, the application environment includes an electronic device 110 and a server 120. The electronic device 110 may obtain the first resolution image from the server 120, and perform a printing process on the first resolution image to obtain a printed image. Then, the electronic device 110 shoots the print image to obtain a second resolution image; the second resolution corresponding to the second resolution image is less than the first resolution corresponding to the first resolution image. Then, the electronic device 110 performs resolution reduction processing on the first resolution image through the image quality degradation model to be trained, so as to obtain a quality degradation image. The electronic device 110 adjusts parameters of the image degradation model based on a difference between the degraded image and the second resolution image, resulting in a trained image degradation model when the training stop condition is satisfied. The trained image degradation model may be installed to run on the electronic device 110 or run on the server 120. The electronic device 110 may be, but is not limited to, various personal computers, notebook computers, smart phones, tablet computers, and portable wearable devices, and the server 120 may be implemented by an independent server or a server cluster formed by a plurality of servers.

FIG. 2 is a schematic diagram of a method for training an image degradation model in one embodiment. As shown in fig. 2, the electronic device may obtain a preset number of original images of a first resolution from the public high-definition image data set, where the original images of the first resolution are high-resolution images; the image sizes of the preset number of the original images of the first resolution may be different. The electronic equipment splices a preset number of original images with a first resolution ratio to obtain spliced images with the first resolution ratio. And then, the electronic equipment performs lossless cutting on the spliced image to obtain each first-resolution image with the same image size. The electronics then divide each first resolution image into regions, for example adding a grid to the first resolution image for subsequent registration. And then, the electronic equipment carries out printing processing on each first-resolution image through the printing equipment to obtain each printing image. The electronic device can shoot each developed image to obtain each second-resolution image. The second resolution corresponding to the second resolution image is less than the first resolution corresponding to the first resolution image. Further, the photographing magnification used when the electronic device photographs each of the developed images may be different. The electronic device may then use each first resolution image and the corresponding second resolution image to train the image degradation model. The trained image degradation model can accurately and quickly convert the input high-resolution images into corresponding low-resolution images, so that a large number of high-resolution and low-resolution training image pairs can be quickly generated.

FIG. 3 is a flow diagram of a method for training an image degradation model in one embodiment. The training method of the image degradation model in this embodiment is described by taking the electronic device in fig. 1 as an example. As shown in fig. 3, the training method of the image degradation model includes:

step 302, acquiring a first resolution image, and performing a printing process on the first resolution image to obtain a printed image.

Wherein, the process printing is the whole process of digital input, image processing and image output. It adopts color photographic method to expose digital image on color photographic paper and output color photo, and is a method for making digital photo with high speed, low cost and high quality. The digital input is that the traditional negative film, the reverse film and the finished photo are scanned into digital images by a scanning system of the digital developing machine and then are input into a computer connected with the developing machine. Digital printing not only prints images taken by digital cameras, but also prints conventional films and other digital images on various storage media.

The first-resolution image may be any one of an RGB (Red, green, blue) image, a RAW image, a grayscale image, a depth image, an image corresponding to the Y component in the YUV image, and the like. The RAW image is RAW data obtained by converting a captured light source signal into a digital signal by an image sensor. "Y" in YUV images represents brightness (Luma) and gray scale value, and "U" and "V" represent Chrominance (Chroma) and saturation, which are used to describe the color and saturation of the image and to specify the color of the pixel. Image resolution refers to the number of pixels in an image unit of inch.

Specifically, the electronic device may obtain the first resolution image from a local or other device or a network, or the electronic device may obtain the first resolution image by shooting a scene through a camera. The first resolution image is an electronic image. And then, the electronic equipment sends the acquired first resolution image to the developing equipment, and the first resolution image is developed into a photo or a postcard in a lossless manner through the developing equipment with ultrahigh pixel precision, wherein the developed image is the developing image.

In this embodiment, the printing resolution (DPI) of the printing apparatus is higher than 500 and has a high color fidelity to ensure that the resolution of the printed image is as close as possible to the electronic image of the first resolution.

In this embodiment, the acquiring the image of the first resolution includes: acquiring a preset number of original images with first resolution, and splicing the preset number of original images with the first resolution to obtain spliced images; and dividing the spliced image to obtain first resolution images with the same image size.

Specifically, the electronic device may obtain a preset number of original images of a first Resolution from the public High-definition image dataset, where the original images of the first Resolution are High Resolution (HR). The image sizes of the preset number of the original images of the first resolution may be different. The electronic equipment splices a preset number of original images with a first resolution ratio to obtain spliced images with the first resolution ratio. And then, performing lossless cropping on the spliced images to obtain each first-resolution image with the same image size. For example, 6 original images with different image sizes are subjected to stitching processing to obtain a stitched image with complete edges. Next, the stitched image was divided into nine parts to obtain 9 first-resolution images of the same image size.

Step 304, shooting the printing image to obtain a second resolution image; the second resolution corresponding to the second resolution image is less than the first resolution corresponding to the first resolution image.

Specifically, the electronic device captures the print image through the camera, and takes the captured image as the second-resolution image. The second Resolution image is referred to as a Low Resolution image (LR). The second resolution of the second resolution image is less than the first resolution of the first resolution image. The electronic device then takes the first resolution image and the corresponding second resolution image as a training image pair.

In this embodiment, the electronic device performs a printing process on the plurality of first-resolution images through the printing device to obtain a plurality of printed images. Then, the electronic device shoots the plurality of printed images through the camera to obtain a second resolution image corresponding to each printed image, so as to obtain a second resolution image corresponding to each first resolution image.

Optionally, the electronic device may capture a plurality of different print images with different capture magnifications to obtain each second-resolution image, where one print image corresponds to one second-resolution image. The electronic equipment can shoot each developed image through different shooting multiplying powers respectively to obtain a plurality of second resolution images corresponding to each developed image. The number of the sheets is at least two.

And step 306, performing resolution reduction processing on the first resolution image through the image degradation model to be trained to obtain a degradation image.

The resolution reduction processing refers to adding noise to an image, adding blurring to the image, or compressing the image to reduce the resolution of the image. The types of the resolution reduction processing include, but are not limited to, addition of gaussian noise, salt and pepper noise, poisson noise, gaussian blur, motion blur, image compression processing, and combination processing with each other.

Specifically, the first resolution image and the second resolution image are used as a training image pair to train the image degradation model. And the electronic equipment takes the second resolution image as a label, and inputs the first resolution image and the second resolution image into an image degradation model to be trained. The image degradation model to be trained performs resolution reduction processing on the first-resolution image, where the resolution reduction processing may be adding noise to the first-resolution image, adding blur to the first-resolution image, or performing compression processing on the first-resolution image, so as to output a degraded image after resolution reduction.

And 308, adjusting parameters of the image degradation model based on the difference between the degraded image and the second resolution image, and obtaining the trained image degradation model when the training stopping condition is met.

Specifically, the degraded image is a low-resolution image output as an image degradation model, and the second-resolution image is also a low-resolution image. The electronic device compares the degraded image with the second resolution image as a label, determines a difference between the degraded image and the second resolution image, and adjusts a parameter of the image degradation model according to the difference between the degraded image and the second resolution image. And continuing training the adjusted image degradation model through the image of the degradation image and the image of the second resolution ratio until the training stopping condition is met, and obtaining the trained image degradation model.

In this embodiment, the training stop condition is that the difference between the degraded image and the corresponding second-resolution image is smaller than a preset difference. Or the training stopping condition is that the loss error output by the image degradation model is smaller than a loss threshold value. Or the training stop condition may be that the number of times of training reaches a preset number of times.

And when the difference between the degraded image and the corresponding second resolution image is smaller than the preset difference or the loss error output by the image degraded model is smaller than the loss threshold, stopping training to obtain the trained image degraded model.

In this embodiment, the first resolution image is processed by printing, and the printed image is photographed to simulate the degradation process of the image, so as to obtain the second resolution image with the reduced resolution. The first resolution image and the second resolution image are used for training an image degradation model, the second resolution image is used as a label, namely, the high resolution image and the corresponding low resolution image are used as a training image pair, and the low resolution image is used as a label, so that the obtained training image pair is more accurate and more conforms to the training image set obtained in the actual situation. And carrying out resolution reduction processing on the first resolution image according to the image degradation model to obtain a degradation image output by the model, namely obtaining a low resolution image. And adjusting model parameters according to the difference between the low-resolution image output by the image degradation model and the low-resolution image serving as a reference until the trained image degradation model is obtained when the training stopping condition is met. The trained image degradation model can accurately and quickly convert the input high-resolution image into a corresponding low-resolution image, so that a large number of high-resolution and low-resolution training image pairs can be quickly generated, and a training image data set is expanded.

In one embodiment, capturing the print image to obtain the second resolution image comprises:

determining a first shooting magnification corresponding to the first resolution image; adjusting the shooting magnification of the electronic equipment to a second shooting magnification, and shooting the developed image through the second shooting magnification to obtain a second resolution image; the second shooting magnification is larger than the first shooting magnification.

Specifically, the electronic device acquires a first shooting magnification corresponding to the first resolution image, and determines a current shooting magnification of the electronic device. When the current shooting magnification of the electronic equipment is smaller than or equal to the first shooting magnification, the current shooting magnification is amplified to a second shooting magnification, and the second shooting magnification is larger than the first shooting magnification. And when the current shooting magnification of the electronic equipment is larger than the first shooting magnification, taking the current shooting magnification as a second shooting magnification. Then, the electronic device captures the developed image at a second capture magnification of the camera, and takes the captured image as a second resolution image. The second resolution image is referred to as a low resolution image.

In this embodiment, the electronic device may capture different first-resolution images using different second capture magnifications to obtain second-resolution images of different capture magnifications.

For example, the electronic device may capture 3 print images at 3 different capture magnifications, each capture magnification being used to capture one print image, resulting in 3 second resolution images, one print image corresponding to one second resolution image. Or the electronic device may respectively capture 3 printed images at 3 different capture magnifications, where each capture magnification captures the 3 printed images to obtain 9 second-resolution images, and 1 printed image corresponds to 3 second-resolution images, where the three second-resolution images are second-resolution images obtained at different capture magnifications.

In the embodiment, the developed image is shot by amplifying the shooting magnification, the resolution of the shot image is smaller than the first resolution, the high-resolution image can be directly shot by different shooting magnifications, and the low-resolution image can be accurately obtained. The real image degradation process is simulated to obtain the low-resolution image, and the low-resolution image is obtained more in line with the actual situation, so that the acquisition of the high-resolution image dataset and the low-resolution image dataset is more accurate.

In one embodiment, adjusting the shooting magnification of the electronic device to a second shooting magnification, and shooting the print image by the second shooting magnification to obtain a second resolution image includes:

determining an image size of the first resolution image; adjusting the shooting magnification of the electronic equipment to a second shooting magnification, and adjusting the distance between the electronic equipment and the printed image to shoot the printed image to obtain a second resolution image; the image size of the second resolution image is the same as the image size of the first resolution image.

Specifically, the electronic device may determine an image size of the first resolution image, and determine an image scene captured in the camera of the electronic device after adjusting the shooting magnification of the electronic device to the second shooting magnification. When the acquired image scene does not completely contain the image scene in the developed image, the distance between the camera of the electronic equipment and the developed image is larger, and the distance between the camera of the electronic equipment and the developed image is shortened, so that the image scene acquired by the camera is completely consistent with the image scene in the developed image. And the driving equipment drives the electronic equipment to move towards the developed image, and after the distance between the electronic equipment and the developed image is shortened, the developed image is shot through a second shooting magnification to obtain a second resolution image. It can be understood that, the image scene collected by the camera is completely consistent with the image scene in the print image, and the image size of the second resolution image captured by the camera is the same as the image size of the first resolution image corresponding to the print image.

In this embodiment, in order to avoid the influence of settings such as automatic exposure of the mobile phone on the shot image in the shooting process, parameters such as ISO sensitivity, exposure time, white balance and the like of the camera can be adjusted to eliminate the influence of overexposure and the like.

In this embodiment, the shooting magnification is adjusted and the distance between the electronic device and the print image is adjusted to ensure that the image scene acquired by the electronic device is completely consistent with the image scene in the print image, so that the image content and the image size of the shot second resolution image and the shot first resolution image (i.e., the low resolution image and the corresponding high resolution image) are completely consistent, and other parameters except for different resolutions of the high and low resolution images used for training the image degradation model are ensured to be the same, thereby improving the accuracy of the image degradation model.

In one embodiment, after the capturing the print image and obtaining the second resolution image, the method further comprises: carrying out registration processing on the first resolution image based on the second resolution image to obtain a registration image of the first resolution;

this carry out resolution ratio reduction processing to first resolution ratio image through waiting the image degradation model of training, obtain the degradation image, include: and carrying out resolution reduction processing on the registration image with the first resolution ratio through the image degradation model to be trained to obtain a degraded image.

Specifically, the electronic device determines a mapping relationship between the first resolution image and the second resolution image. The mapping relationship may map the first-resolution image to an image space of the second-resolution image, or map the second-resolution image to an image space of the first-resolution image, so that the first-resolution image and the second-resolution image are in the same image space with the same view angle, so as to accurately realize registration of the first-resolution image, and obtain a registration image of the first resolution.

The electronic device then takes the registered image of the first resolution and the image of the second resolution as a training image pair to train the image degradation model. Further, the electronic device uses the second resolution image as a label, and inputs the registration image of the first resolution and the second resolution image into the image degradation model to be trained. And the image degradation model to be trained carries out resolution reduction processing on the registration image with the first resolution, and outputs the image with the reduced resolution, namely the degraded image.

In this embodiment, the first-resolution image is registered with the second-resolution image, so that the first-resolution image and the second-resolution image are in the same image space, and it is ensured that parameters of the first-resolution registered image and the second-resolution image (i.e., the high-resolution image and the low-resolution image) as training images, other than the resolution, are kept as consistent as possible, thereby improving the accuracy of the image degradation model training.

In one embodiment, the registration process for the first resolution image may include correspondence of image content, alignment of color space, and the like. The alignment of the image content can adopt a mode of detecting and matching image feature points and calculating a homography matrix, and can also adopt a mode of matching image blocks. Aiming at the problem of mismatching of brightness and color space between the first resolution image and the second resolution image, some photos of a standard color card can be introduced in the shooting process, and through the difference between the first resolution image and the second resolution image of the standard color card, an alignment model of the color space and the brightness is modeled and analyzed, and histogram matching and other operations are carried out, so that the alignment of the brightness and the color space is realized.

In one embodiment, as shown in fig. 4, performing a registration process on the first-resolution image based on the second-resolution image to obtain a registration image of the first resolution includes:

step 402, acquiring feature points of the first resolution image and feature points of the second resolution image.

The feature point refers to a point where the image gray value changes drastically or a point where the curvature is large on the edge of the image (i.e., the intersection of two edges). Characteristic points such as eyes, nose tip, mouth corner, moles, center of object, etc., are not limited thereto.

Specifically, the electronic device detects the gray value of each pixel point in the first resolution image, and when the difference value of the gray values of the adjacent pixel points is greater than the threshold value, the region where the adjacent pixel points are located can be used as the feature point. And detecting the gray value of each pixel point in each frame of second-resolution image, and when the difference value of the gray values of the adjacent pixel points is greater than a threshold value, taking the area where the adjacent pixel points are located as the characteristic point.

In one embodiment, the electronic device may extract corresponding feature points from each frame of the second resolution image based on the feature points in the first resolution image. In another embodiment, the gray values of the pixels in the second resolution image may also be detected, and when the difference between the gray values of the adjacent pixels is greater than the threshold, the region where the adjacent pixels are located may be used as the feature point, and the corresponding feature point may be extracted from the remaining second resolution image and the first resolution image.

And step 404, determining a matching point pair between the first resolution image and the second resolution image according to the characteristic points of the first resolution image and the characteristic points of the second resolution image.

Specifically, the electronic device combines the feature points extracted from the first resolution image and the corresponding feature points of the second resolution image into a matching point pair.

In one embodiment, the first resolution image may be divided into a plurality of regions, for example, a nine-square grid is added to the first resolution image to divide the first resolution image into nine regions, and each point of the nine-square grid is taken as a feature point. And determining the characteristic points of the second resolution image according to the characteristic points in the first resolution image to form matching point pairs.

Step 406, a homography matrix between the first resolution image and the second resolution image is determined according to the matching point pairs.

Homography (Homography) is a concept in projective geometry, also called projective transformation. It maps points (three-dimensional homogeneous vectors) on one projective plane onto another projective plane and maps straight lines into straight lines, having line-preserving properties. The homography matrix is then a mapping of points and points, and the exact position of the corresponding point of an image point on another image can be found using the homography matrix. The homography matrix may be used to characterize a mapping relationship between the first resolution image and the second resolution image.

Specifically, the electronic device may calculate a homography matrix between the first resolution image and the second resolution image based on determining pairs of matching points between the first resolution image and the second resolution image. The second resolution image can be mapped into the same image space as the first resolution image by means of the homography matrix. Or the first resolution image can be mapped to the same image space as the second resolution image by means of a homography.

And step 408, performing registration processing on the first-resolution image based on the homography matrix to obtain a registration image of the first resolution.

Specifically, the electronic device performs registration processing on the first-resolution image according to the homography matrix between the first-resolution image and the second-resolution image, so as to obtain a registered image of the registered first resolution. Further, the first resolution image is mapped to the same image space as the second resolution image according to the homography matrix, and the offset amount between the first resolution image and the second resolution image is determined according to the matching point pairs. And then, the electronic equipment moves each feature point in the first resolution image by the offset to obtain a moved image, wherein the moved image is the registration image of the first resolution.

In one embodiment, the first resolution image may also be subjected to a registration process by a convolutional neural network.

In this embodiment, according to the matching point pair between the first resolution image and the second resolution image, the homography matrix between the first resolution image and the second resolution image is determined, so that the first resolution image can be mapped to the image space of the second resolution image, the second resolution image and the first resolution image are in the same view angle, and the first resolution image is accurately registered.

In one embodiment, as shown in fig. 5, adjusting parameters of the image degradation model based on a difference between the degraded image and the second-resolution image, when a training stop condition is satisfied, to obtain a trained degraded model, includes:

step 502, determining at least one of a peak signal-to-noise ratio, single level structure similarity, and multi-level structure similarity between the degraded image and the second resolution image.

The Peak Signal to Noise Ratio (PSNR) is a Ratio of energy of a Peak Signal to average energy of Noise. The PSNR can range from [0,100]. Single-layer Structural Similarity (SSIM), i.e., structural Similarity. The value range of SSIM can be [0,1], and the larger the value is, the smaller the image distortion is. (Multi-scale structural similarity, MS-SSIM for short), namely, multi-level structural similarity and multi-scale structural similarity. The value range of MS-SSIM can be [0,1], and the larger the value is, the smaller the image distortion is.

Specifically, the electronic device calculates at least one of SSIM, MS-SSIM, PSNR between the degraded image output by the image degradation model and the second-resolution image as a tag. For example, the electronic device calculates the PSNR, or SSIM, or MS-SSIM, between the degraded image and the second resolution image, or PSNR and SSIM, or PSNR and MS-SSIM, or SSIM and MS-SSIM, or PSNR and SSIM, and MS-SSIM, between the degraded image and the second resolution image.

Step 504, determining a target similarity between the degraded image and the second resolution image according to at least one of the peak signal-to-noise ratio, the single-level structure similarity, and the multi-level structure similarity.

Specifically, when the electronic device calculates any one of SSIM, MS-SSIM, PSNR between the degraded image and the second-resolution image, the calculated value is directly used as the target similarity between the degraded image and the second-resolution image.

When the electronic equipment calculates at least two of SSIM, MS-SSIM and PSNR between the degraded image and the second resolution image, the electronic equipment performs weighted summation on at least two calculated values to obtain the target similarity.

For example, the electronic device calculates a peak signal-to-noise ratio, a single-level structure similarity, and a multi-level structure similarity between the degraded image and the second resolution image, and then the electronic device performs weighted summation processing on the obtained peak signal-to-noise ratio, the single-level structure similarity, and the multi-level structure similarity to obtain the target similarity.

Step 506, adjusting parameters of the image degradation model according to the difference between the target similarity and the similarity threshold, continuing training, and obtaining the trained image degradation model when the training stopping condition is met.

Specifically, the electronic device obtains a similarity threshold, and compares the target similarity with the similarity threshold respectively. And when the target similarity is smaller than the similarity threshold, adjusting the parameters of the image degradation model and continuing training. And stopping training when the target similarity is greater than or equal to the similarity threshold value to obtain the trained image degradation model. In other embodiments, the training stopping condition may be that a preset training number is reached, and when the preset training number is reached, the training is stopped to obtain the trained image degradation model.

In the present embodiment, the similarity between the degraded image and the second-resolution image as a reference is determined from the objective parameters by calculating at least one of the peak signal-to-noise ratio between the degraded image and the second-resolution image, the single-level structure similarity, and the multi-level structure similarity. And adjusting model parameters according to the similarity and training to improve the precision of the image degradation model. And moreover, the trained image degradation images can be obtained by accurately converting the high-resolution images into the low-resolution images.

In one embodiment, as shown in fig. 6, performing resolution reduction processing on the first resolution image through an image degradation model to be trained to obtain a degraded image includes:

step 602, performing resolution reduction processing on the first resolution image respectively based on each degradation type in the image degradation model to be trained, so as to obtain a degradation image corresponding to each degradation type respectively.

The degradation type refers to a processing type for reducing resolution, and includes but is not limited to adding gaussian noise, salt and pepper noise, poisson noise, gaussian blur, motion blur and the like to an image, compressing the image, and any combination of noise, any combination of blur, noise and blur, mutual combination of noise and image compression, mutual combination of blur and image compression and the like.

Specifically, the first resolution image and the second resolution image are used as a training image pair to train the image degradation model. Further, the electronic device uses the second resolution image as a label, and inputs the first resolution image and the second resolution image into an image degradation model to be trained.

The image degradation model to be trained acquires each degradation type, and the resolution reduction processing is performed on the first-resolution image through each degradation type, where the resolution reduction processing may be, but is not limited to, adding noise to the first-resolution image, adding blur, or performing compression processing on the first-resolution image. And the electronic equipment acquires the degraded images which are output by the image degraded model and respectively correspond to the degraded types.

Adjusting parameters of the image degradation model based on a difference between the degraded image and the second resolution image, comprising: step 604, determining the similarity between the second resolution image and the degraded images corresponding to the respective degraded types.

Specifically, the quality-degraded images respectively corresponding to the quality-degraded types are low-resolution images, and the electronic device compares the quality-degraded images respectively corresponding to the quality-degraded types with the second-resolution images serving as labels, and determines the similarity between the quality-degraded images respectively corresponding to the quality-degraded types and the second-resolution images to obtain the similarities.

And 606, adjusting parameters of the image degradation model according to the similarity.

Specifically, the electronic device determines a similarity corresponding to each degradation type. And the electronic equipment acquires a similarity threshold value and compares the similarity corresponding to each degradation type with the similarity threshold value respectively. And when at least one similarity is smaller than the similarity threshold value, adjusting the parameters of the image degradation model and continuing training. And stopping training when the similarity corresponding to each degradation type is greater than or equal to the similarity threshold value, so as to obtain the trained image degradation model.

In this embodiment, adjusting the parameters of the image degradation model according to the similarity includes: and taking the degradation parameter corresponding to the degradation type corresponding to the degradation image with the highest image similarity of the second resolution as the parameter of the image degradation model.

Specifically, after determining the similarity corresponding to each degraded type, the electronic device determines the degraded type corresponding to the degraded image with the highest similarity. Then, the electronic device may use the degradation parameter of the degradation type corresponding to the highest similarity as a parameter of the image degradation model. The degradation parameter may be how much noise is added to the image, how much blur is added, or the degree of image compression.

In this embodiment, resolution reduction processing is performed on the first resolution image according to different degradation types in the image degradation model to obtain low resolution images (degradation images) of different degradation types, so that the image degradation model can be adjusted according to a similarity difference between the low resolution images of different degradation types and the low resolution image serving as a reference (i.e., the second resolution image), so as to improve the accuracy of the image degradation model. And moreover, the image degraded images can accurately convert the high-resolution images into the low-resolution images through different degradation types, so that a large number of training images are obtained.

In one implementation, the method further comprises: acquiring an image to be processed; inputting the image to be processed into the trained image degradation model, and performing resolution reduction processing on the image to be processed through the trained image degradation model to obtain a target image; the resolution of the target image is less than the resolution of the image to be processed.

Specifically, the image to be processed is a high-resolution image, and the electronic device inputs the acquired high-resolution image into a trained image degradation model. And the trained image degradation model performs resolution reduction processing on the image to be processed to obtain a corresponding low-resolution image, namely a target image. The trained image degradation model can quickly and accurately generate a large number of low-resolution images corresponding to the high-resolution images, so that a training image set can be expanded.

In one embodiment, as shown in fig. 7, acquiring a first resolution image, and performing a printing process on the first resolution image to obtain a printed image, includes:

step 702, acquiring an image in the first-resolution video to obtain a first-resolution image of each frame.

Specifically, the electronic device may obtain the first resolution video from a local or other device or a network, or the electronic device may capture the first resolution video through a camera. Then, the electronic device acquires each frame of image from the first resolution video to obtain each frame of first resolution image.

Step 704, performing a printing process on each frame of the first resolution image to obtain each frame of the printing image.

Specifically, the electronic device sends the acquired first resolution images of the frames to the developing device. And carrying out lossless printing on each frame of the first-resolution image into a corresponding printing image through ultrahigh-pixel-precision printing equipment to obtain each frame of the printing image.

This to the photographic printing image shoots, obtains second resolution ratio image, includes:

and 706, performing video recording on each frame of the developed image to obtain each frame of second-resolution image in the recorded video.

Specifically, the electronic device records the video of each frame of the development image to obtain a second-resolution image of each frame in the recorded video. A frame of the second resolution image corresponds to a frame of the first resolution image.

In this embodiment, the electronic device may capture the developed image of each frame by using the camera, so as to obtain the second-resolution image of each frame. Furthermore, the electronic device can shoot the developed images of each frame through different shooting magnifications to obtain the second-resolution images of each frame.

In this embodiment, a high-resolution image is obtained from a high-resolution video, a low-resolution image is obtained from a low-resolution video (recorded video), and the trained image degradation model can accurately and quickly convert the high-resolution video into the low-resolution video. And taking the high-resolution video and the low-resolution video as training samples of the video reconstruction model, so that the trained video reconstruction model can perform super-resolution reconstruction on the low-resolution video to obtain the high-resolution video.

In this embodiment, a plurality of frames of first-resolution images can be acquired from a high-resolution video by acquiring the plurality of frames of first-resolution images from the first-resolution video. And carrying out printing processing on the plurality of frames of first-resolution images to obtain a plurality of frames of printing images. The multi-frame image developing images are shot to obtain multi-frame second-resolution images, so that the multi-frame high-resolution images can be accurately converted into low-resolution images.

In one embodiment, each frame of the developed image is captured to obtain a second resolution image for each frame. And arranging the second-resolution images of each frame according to the time sequence of the corresponding first-resolution images in the first-resolution video to generate the second-resolution video.

In one embodiment, the electronic device may acquire images in the first resolution video to obtain each frame of the first resolution image. Then, the electronic device determines the shooting magnification of the first resolution video, and adjusts the shooting magnification of the electronic device to a second shooting magnification. The second shooting magnification is larger than the first shooting magnification. And the electronic equipment shoots the first-resolution video through the second shooting magnification to obtain a second-resolution video. Then, the electronic device acquires the second resolution image of each frame from the second resolution video to obtain the second resolution image corresponding to each first resolution image.

It can be understood that, in the present solution, a training image pair composed of a high resolution image and a low resolution image generated by a trained image degradation model, and a training video pair composed of a high resolution video and a low resolution video can be used as training data for any task that requires conversion from a high resolution to a low resolution. Such tasks include, but are not limited to: image deblurring, denoising, video deblurring, denoising, image restoration, video restoration, image super-resolution reconstruction and video super-resolution reconstruction.

In one embodiment, as shown in fig. 8, the method further comprises:

step 802, acquiring an image to be processed.

The image to be processed is a high-resolution image, and the image to be processed needs to be converted into a low-resolution image. The image to be processed can be any one of an RGB image, a RAW image, a grayscale image, a depth image, an image corresponding to the Y component in the YUV image, and the like.

Specifically, the electronic device may obtain the image to be processed from a local device or other devices or a network, or the electronic device may obtain the image to be processed by shooting a scene with a camera.

Step 804, inputting the image to be processed into the trained image degradation model, and performing resolution reduction processing on the image to be processed through the trained image degradation model to obtain a target image; the resolution of the target image is less than the resolution of the image to be processed.

Specifically, the electronic device inputs the image to be processed into a trained image degradation model. And the trained image degradation model performs degradation processing on the image to be processed, namely performs resolution reduction processing on the image to be processed to obtain a target image. The target image is a low-resolution image, and the resolution of the target image is smaller than that of the image to be processed.

It will be appreciated that the high and low resolution of the image are relative. In this embodiment, the resolution of the image to be processed is higher than that of the target image, and the image to be processed is regarded as a high resolution image and the target image is regarded as a low resolution image.

Step 806, determining the image to be processed and the corresponding target image as a training image pair, and training an image reconstruction model based on the training image pair to obtain a trained image reconstruction model; the trained image reconstruction model is used for performing super-resolution reconstruction on the image.

The image reconstruction model is used for performing super-resolution reconstruction on the low-resolution image to obtain a high-resolution image. Super-resolution reconstruction refers to the reconstruction of a low-resolution image or image sequence to obtain a high-resolution image.

Specifically, the electronic device determines the image to be processed and the corresponding target image as a training image pair. The electronic equipment inputs the training image into an image reconstruction model, a high-resolution image (namely an image to be processed) is used as a label, and the low-resolution image (namely a target image) is subjected to super-resolution reconstruction processing through the image reconstruction model to be trained to obtain the high-resolution image output by the image reconstruction model. And then, the electronic equipment compares the high-resolution image output by the image reconstruction model with the high-resolution image serving as the label, adjusts the parameters of the image reconstruction model according to the difference between the high-resolution image and the label, and continues training until the training stopping condition is met, so that the trained image reconstruction model is obtained. The trained image reconstruction model can carry out super-resolution reconstruction on a blurred low-resolution image or a low-resolution image with noise, and the noise or the blur of the image is removed to obtain a high-resolution image.

In this embodiment, a large number of low-resolution images corresponding to high-resolution images can be quickly and accurately generated by the trained image degradation model, so that the training image set can be expanded, and the obtained training images are more accurate. The generated training image pair is used for training the image reconstruction model, and the accuracy and precision of super-resolution reconstruction of the image reconstruction model can be effectively improved.

In one embodiment, after acquiring the image to be processed, the method further comprises:

classifying the image to be processed;

inputting an image to be processed into a trained image degradation model, and performing resolution reduction processing on the image to be processed through the trained image degradation model to obtain a target image, wherein the method comprises the following steps:

and inputting the classified images to be processed into the trained image degradation model, and performing image resolution reduction processing on the classified images to be processed based on each degradation type in the trained image degradation model to obtain a target image.

Specifically, the electronic device may obtain a first preset number of images to be processed, and classify the first preset number of images to be processed. Further, the electronic device may randomly divide the first preset number of images to be processed into a second preset number of categories. For example, 100 images to be processed are randomly divided into 10 classes. Or, the electronic device may classify the first preset number of images to be processed according to preset categories to obtain images to be processed corresponding to the categories. For example, 100 images to be processed are divided according to preset categories such as landscapes, people, animals and the like, and the images to be processed in each preset category are obtained.

And then, the electronic equipment inputs the classified images to be processed into the trained image degradation model. And acquiring each degradation type by the trained image degradation model, and respectively carrying out resolution reduction processing on the classified images to be processed according to each degradation type so as to obtain target images which are output by the image degradation model and respectively correspond to each degradation type, namely obtaining the target images which correspond to each image to be processed and have the same quantity as the degradation types. The target image is taken as a low resolution image. The resolution of the target image is less than the resolution of the image to be processed.

In this embodiment, the trained image degradation model obtains each degradation type, and performs resolution reduction processing on a type of to-be-processed image according to one degradation type, so as to obtain a target image corresponding to each to-be-processed image output by the image degradation model.

In this embodiment, the trained image degradation model obtains each degradation type, and can randomly allocate the degradation type to compare various images to be processed, and one type of image to be processed is allocated with at least one degradation type.

In this embodiment, the electronic device may determine the number of degradation types in the image degradation model, and divide the image to be processed into the same number of categories as the degradation types. And then, the electronic equipment inputs the classified images to be processed into the trained image degradation model. The image degradation model is a degradation type of each category of images to be processed respectively, so that resolution reduction processing is carried out on the images to be processed to obtain target images.

In this embodiment, by classifying the images to be processed, the trained image degradation model can simultaneously and rapidly convert a large number of high-resolution images into low-resolution images according to each degradation type, and can output more low-resolution images than the high-resolution images to expand the training image data.

In one embodiment, the resolution of each degradation type may be different, for example, the resolution of the image to be processed may be reduced by 5 times, 10 times, etc. by different degradation types. Further, different degradation types may correspond to a degradation range, for example, one degradation type may randomly reduce the resolution of the image to be processed by 5-10 times, another degradation type may randomly reduce the resolution of the image to be processed by 10-15 times, or 15-20 times, etc., but is not limited thereto.

In one embodiment, a method for training an image degradation model is provided, which includes:

the electronic equipment acquires the first resolution image, and carries out printing processing on the first resolution image to obtain a printing image.

Then, the electronic device determines a first shooting magnification corresponding to the first-resolution image, and determines an image size of the first-resolution image.

Further, the shooting magnification of the electronic equipment is adjusted to be a second shooting magnification, and the distance between the electronic equipment and the printed image is adjusted to shoot the printed image to obtain a second-resolution image; the image size of the second resolution image is the same as the image size of the first resolution image; the second shooting magnification is greater than the first shooting magnification, and the second resolution corresponding to the second resolution image is less than the first resolution corresponding to the first resolution image.

Then, the electronic equipment acquires the characteristic points of the first resolution image and the characteristic points of the second resolution image;

then, the electronic device determines a matching point pair between the first-resolution image and the second-resolution image according to the feature points of the first-resolution image and the feature points of the second-resolution image.

Further, the electronic device determines a homography matrix between the first resolution image and the second resolution image from the pairs of matching points.

Then, the electronic device performs registration processing on the first-resolution image based on the homography matrix to obtain a registration image of the first resolution.

And then, the electronic equipment performs resolution reduction processing on the registration image with the first resolution through the image degradation model to be trained to obtain a degraded image.

Next, the electronics determine at least one of a peak signal-to-noise ratio, a single level structural similarity, and a multi-level structural similarity between the degraded image and the second resolution image.

Next, the electronic device determines a target similarity between the degraded image and the second resolution image based on at least one of the peak signal-to-noise ratio, the structural similarity, and the multi-level structural similarity.

Further, the electronic equipment adjusts parameters of the image degradation model according to the difference between the target similarity and the similarity threshold, continues training, and obtains the trained image degradation model when the training stopping condition is met.

Optionally, the electronic device acquires an image to be processed; and classifying the image to be processed.

Secondly, the electronic equipment inputs the classified images to be processed into a trained image degradation model, and image resolution reduction processing is carried out on the classified images to be processed based on each degradation type in the trained image degradation model to obtain target images; the resolution of the target image is less than the resolution of the image to be processed.

Further, the electronic equipment determines the image to be processed and the corresponding target image as a training image pair, and trains the image reconstruction model based on the training image to obtain a trained image reconstruction model; the trained image reconstruction model is used for performing super-resolution reconstruction on the image.

In this embodiment, the first-resolution image is processed by printing, and the printed image is photographed to simulate the degradation process of the image, so as to obtain the second-resolution image with the reduced resolution. By adjusting the shooting magnification and adjusting the distance between the electronic equipment and the developing image, the image scene acquired by the electronic equipment is completely consistent with the image scene in the developing image, so that the high-resolution image and the low-resolution image used for training the image degradation model have the same parameters except for different resolutions.

And carrying out registration treatment on the first-resolution image to the second-resolution image so as to ensure that other parameters except the resolution of the high-resolution image and the low-resolution image which are taken as training images are kept consistent, thereby improving the accuracy of the training of the image degradation model.

The trained image degradation model can quickly and accurately generate a large number of low-resolution images corresponding to the high-resolution images, so that the training image set can be expanded, and the obtained training images are more accurate. The generated training image pair is used for training the image reconstruction model, so that the accuracy and precision of super-resolution reconstruction of the image reconstruction model can be effectively improved.

It should be understood that although the various steps in the flowcharts of fig. 2-8 are shown in order as indicated by the arrows, the steps are not necessarily performed in order as indicated by the arrows. The steps are not limited to being performed in the exact order illustrated and, unless explicitly stated herein, may be performed in other orders. Moreover, at least some of the steps in fig. 2-8 may include multiple sub-steps or multiple stages that are not necessarily performed at the same time, but may be performed at different times, and the order of performing the sub-steps or stages is not necessarily sequential, but may be performed alternately or alternately with other steps or at least some of the sub-steps or stages of other steps.

Fig. 9 is a block diagram illustrating a structure of a training apparatus for an image degradation model according to an embodiment. As shown in fig. 9, the apparatus includes: an acquisition module 902, a capture module 904, a training module 906, and an adjustment module 908. Wherein the content of the first and second substances,

an obtaining module 902 is configured to obtain a first resolution image, and perform a printing process on the first resolution image to obtain a printed image.

A shooting module 904, configured to shoot the print image to obtain a second resolution image; the second resolution corresponding to the second resolution image is smaller than the first resolution corresponding to the first resolution image.

The training module 906 is configured to perform resolution reduction processing on the first resolution image through an image degradation model to be trained, so as to obtain a degradation image.

An adjusting module 908 for adjusting parameters of the image degradation model based on a difference between the degraded image and the second resolution image, resulting in a trained image degradation model when a training stop condition is met.

In one embodiment, the obtaining module 902 is further configured to: determining a first shooting magnification corresponding to the first resolution image; adjusting the shooting magnification of the electronic equipment to a second shooting magnification, and shooting the developed image through the second shooting magnification to obtain a second resolution image; the second shooting magnification is larger than the first shooting magnification.

In the embodiment, the developed image is shot by amplifying the shooting magnification, the resolution of the shot image is smaller than the first resolution, the high-resolution image can be directly shot by different shooting magnifications, and the low-resolution image can be accurately obtained. The real image degradation process is simulated to obtain the low-resolution image, and the low-resolution image is obtained more in line with the actual situation, so that the acquisition of the high-resolution image data set and the low-resolution image data set is more accurate.

In one embodiment, the obtaining module 902 is further configured to: determining an image size of the first resolution image;

adjusting the shooting magnification of the electronic equipment to a second shooting magnification, and adjusting the distance between the electronic equipment and the printed image to shoot the printed image to obtain a second resolution image; the image size of the second resolution image is the same as the image size of the first resolution image.

In one embodiment, the apparatus further comprises: and a registration module. The registration module is to: after the printing image is shot to obtain a second resolution image, carrying out registration processing on the first resolution image based on the second resolution image to obtain a registration image of the first resolution;

the training module 906 is further configured to: and carrying out resolution reduction processing on the registration image with the first resolution ratio through the image degradation model to be trained to obtain a degraded image.

In one embodiment, the registration module is further to: acquiring characteristic points of a first resolution image and characteristic points of a second resolution image; determining a matching point pair between the first resolution image and the second resolution image according to the characteristic points of the first resolution image and the characteristic points of the second resolution image; determining a homography matrix between the first resolution image and the second resolution image according to the matching point pairs; and carrying out registration processing on the first-resolution image based on the homography matrix to obtain a registration image of the first resolution.

In this embodiment, the homography matrix between the first resolution image and the second resolution image is determined according to the matching point pairs between the first resolution image and the second resolution image, so that the first resolution image can be mapped to the image space of the second resolution image, the second resolution image and the first resolution image are in the same view angle, and the first resolution image is accurately registered.

In one embodiment, the adjustment module 908 is further configured to: determining at least one of a peak signal-to-noise ratio, single level structure similarity, and multi-level structure similarity between the degraded image and the second resolution image; determining a target similarity between the degraded image and the second resolution image according to at least one of the peak signal-to-noise ratio, the single-level structure similarity and the multi-level structure similarity; and adjusting parameters of the image degradation model according to the difference between the target similarity and the similarity threshold, continuing training, and obtaining the trained image degradation model when the training stopping condition is met.

In this embodiment, the similarity between the degraded image and the second resolution image as a reference is determined from the objective parameters by calculating at least one of the peak signal-to-noise ratio between the degraded image and the second resolution image, the single-level structure similarity, and the multi-level structure similarity. And adjusting model parameters according to the similarity and training to improve the precision of the image degradation model. And moreover, the trained image degradation images can be obtained by accurately converting the high-resolution images into the low-resolution images.

In one embodiment, the training module 906 is further configured to: respectively carrying out resolution reduction processing on the first resolution ratio image based on each degradation type in the image degradation model to be trained to obtain a degradation image corresponding to each degradation type;

the adjustment module 908 is further configured to: determining the similarity between the second resolution image and the degraded images corresponding to the degraded types respectively; and adjusting parameters of the image degradation model according to the similarity.

In one embodiment, the apparatus further comprises: and generating a module. The generation module is to: acquiring an image to be processed; inputting the image to be processed into the trained image degradation model, and performing resolution reduction processing on the image to be processed through the trained image degradation model to obtain a target image; the resolution of the target image is smaller than that of the image to be processed; determining an image to be processed and a corresponding target image as a training image pair, and training an image reconstruction model based on the training image to obtain a trained image reconstruction model; the trained image reconstruction model is used for performing super-resolution reconstruction on the image.

In this embodiment, a large number of low-resolution images corresponding to high-resolution images can be quickly and accurately generated by the trained image degradation model, so that the training image set can be expanded, and the obtained training images are more accurate. The generated training image pair is used for training the image reconstruction model, so that the accuracy and precision of super-resolution reconstruction of the image reconstruction model can be effectively improved.

In one embodiment, the generation module is further configured to: after the image to be processed is obtained, classifying the image to be processed; and inputting the classified images to be processed into the trained image degradation model, and performing image resolution reduction processing on the classified images to be processed based on each degradation type in the trained image degradation model to obtain a target image.

In one embodiment, the obtaining module 902 is further configured to: acquiring an image in a first-resolution video to obtain a first-resolution image of each frame; processing each frame of the first-resolution image to obtain each frame of processed image;

the photographing module 904 is further configured to: and carrying out video recording on each frame of the developed image to obtain each frame of second-resolution image in the recorded video.

The division of each module in the training apparatus for the image degradation model is only used for illustration, and in other embodiments, the training apparatus for the image degradation model may be divided into different modules as needed to complete all or part of the functions of the training apparatus for the image degradation model.

For specific limitations of the training apparatus for the image degradation model, reference may be made to the above limitations of the training method for the image degradation model, and details are not repeated here. The modules in the training device of the image degradation model can be wholly or partially realized by software, hardware and a combination thereof. The modules can be embedded in a hardware form or independent from a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.

Fig. 10 is a schematic diagram of an internal structure of an electronic device in one embodiment. As shown in fig. 10, the electronic device includes a processor and a memory connected by a system bus. Wherein, the processor is used for providing calculation and control capability and supporting the operation of the whole electronic equipment. The memory may include a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The computer program can be executed by a processor for implementing a training method of an image degradation model provided in the following embodiments. The internal memory provides a cached execution environment for the operating system computer programs in the non-volatile storage medium. The electronic device may be any terminal device such as a mobile phone, a tablet computer, a PDA (Personal Digital Assistant), a POS (Point of Sales), a vehicle-mounted computer, and a wearable device.

The implementation of each module in the training apparatus of the image degradation model provided in the embodiment of the present application may be in the form of a computer program. The computer program may be run on a terminal or a server. Program modules constituted by such computer programs may be stored on the memory of the electronic device. Which when executed by a processor, performs the steps of the method described in the embodiments of the present application.

The embodiment of the application also provides a computer readable storage medium. One or more non-transitory computer-readable storage media containing computer-executable instructions that, when executed by one or more processors, cause the processors to perform the steps of a method of training an image degradation model.

A computer program product comprising instructions which, when run on a computer, cause the computer to perform a method of training an image degradation model.

Any reference to memory, storage, database, or other medium used herein may include non-volatile and/or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically Programmable ROM (EPROM), electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM), which acts as external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous Link DRAM (SLDRAM), rambus (Rambus) direct RAM (RDRAM), direct bused dynamic RAM (DRDRAM), and Rambus Dynamic RAM (RDRAM).

The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the present application. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent application shall be subject to the appended claims.

Claims

1. A training method of an image degradation model is characterized by comprising the following steps:

determining at least one of a peak signal-to-noise ratio, single level structure similarity, and multi-level structure similarity between the degraded image and the second resolution image;

determining a target similarity between the degraded image and the second resolution image according to at least one of the peak signal-to-noise ratio, the structural similarity and the multi-level structural similarity;

and adjusting parameters of the image degradation model according to the difference between the target similarity and the similarity threshold, and obtaining the trained image degradation model when the training stopping condition is met.

2. The method of claim 1, wherein said capturing said print image to obtain a second resolution image comprises:

determining a first shooting magnification corresponding to the first resolution image;

adjusting the shooting magnification of the electronic equipment to a second shooting magnification, and shooting the developed image through the second shooting magnification to obtain a second resolution image; the second shooting magnification is larger than the first shooting magnification.

3. The method of claim 2, wherein adjusting the capture magnification of the electronic device to a second capture magnification by which the print image is captured to obtain a second resolution image comprises:

determining an image size of the first resolution image;

adjusting the shooting magnification of the electronic equipment to a second shooting magnification, and adjusting the distance between the electronic equipment and the developed image so as to shoot the developed image to obtain a second resolution image; the image size of the second resolution image is the same as the image size of the first resolution image.

4. The method of claim 1, further comprising, after said capturing said print image to obtain a second resolution image:

carrying out registration processing on the first resolution image based on the second resolution image to obtain a registration image of the first resolution;

the processing of resolution reduction is carried out on the first resolution ratio image through the image degradation model to be trained to obtain a degradation image, and the processing comprises the following steps:

and carrying out resolution reduction processing on the registration image of the first resolution through an image degradation model to be trained to obtain a degradation image.

5. The method of claim 4, wherein the registering the first-resolution image based on the second-resolution image to obtain a first-resolution registered image comprises:

acquiring characteristic points of the first resolution image and characteristic points of the second resolution image;

determining a matching point pair between the first resolution image and the second resolution image according to the characteristic points of the first resolution image and the characteristic points of the second resolution image;

determining a homography matrix between the first resolution image and the second resolution image according to the matching point pairs;

and carrying out registration processing on the first-resolution image based on the homography matrix to obtain a registration image of the first resolution.

6. The method according to claim 1, wherein the performing resolution reduction processing on the first resolution image through the image degradation model to be trained to obtain a degraded image comprises:

respectively carrying out resolution reduction processing on the first resolution image based on each degradation type in an image degradation model to be trained to obtain a degradation image corresponding to each degradation type;

the determining at least one of a peak signal-to-noise ratio, single-level structure similarity, and multi-level structure similarity between the degraded image and the second resolution image comprises:

determining at least one of a peak signal-to-noise ratio, single-level structure similarity and multi-level structure similarity between the second resolution image and the degraded images corresponding to the degraded types respectively;

the determining a target similarity between the degraded image and the second resolution image according to at least one of the peak signal-to-noise ratio, the structural similarity, and the multi-level structural similarity includes:

determining target similarity between the degraded images of the degradation types and the second resolution image according to at least one of the peak signal-to-noise ratio, the structural similarity and the multi-level structural similarity;

the adjusting parameters of the image degradation model according to the difference between the target similarity and the similarity threshold comprises:

and adjusting parameters of the image degradation model according to the difference between each target similarity and the similarity threshold.

7. The method of claim 1, wherein said acquiring a first resolution image and processing said first resolution image to obtain a processed image comprises:

acquiring an image in a first-resolution video to obtain a first-resolution image of each frame;

processing the first resolution ratio image of each frame to obtain a processing image of each frame;

the step of shooting the developing image to obtain a second resolution image comprises:

and carrying out video recording on the developed images of each frame to obtain second-resolution images of each frame in the recorded video.

8. The method according to any one of claims 1 to 7, further comprising:

acquiring an image to be processed;

inputting the image to be processed into the trained image degradation model, and performing resolution reduction processing on the image to be processed through the trained image degradation model to obtain a target image; the resolution of the target image is smaller than that of the image to be processed;

determining the image to be processed and the corresponding target image as a training image pair, and training an image reconstruction model based on the training image pair to obtain a trained image reconstruction model; the trained image reconstruction model is used for performing super-resolution reconstruction on the image.

9. The method of claim 8, further comprising, after said acquiring the image to be processed:

classifying the image to be processed;

the inputting the image to be processed into the trained image degradation model, and performing resolution reduction processing on the image to be processed through the trained image degradation model to obtain a target image, includes:

10. An apparatus for training an image degradation model, comprising:

the shooting module is used for shooting the developing image to obtain a second resolution image; the second resolution corresponding to the second resolution image is smaller than the first resolution corresponding to the first resolution image;

an adjustment module for determining at least one of a peak signal-to-noise ratio, a single-level structural similarity, and a multi-level structural similarity between the degraded image and the second resolution image; determining a target similarity between the degraded image and the second resolution image according to at least one of the peak signal-to-noise ratio, the structural similarity and the multi-level structural similarity; and adjusting parameters of the image degradation model according to the difference between the target similarity and the similarity threshold, and obtaining the trained image degradation model when the training stopping condition is met.

11. An electronic device comprising a memory and a processor, the memory having stored therein a computer program that, when executed by the processor, causes the processor to perform the steps of the method according to any one of claims 1 to 9.

12. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method according to any one of claims 1 to 9.