WO2023005818A1

WO2023005818A1 - Noise image generation method and apparatus, electronic device, and storage medium

Info

Publication number: WO2023005818A1
Application number: PCT/CN2022/107258
Authority: WO
Inventors: 郭桦
Original assignee: 维沃移动通信有限公司
Priority date: 2021-07-28
Filing date: 2022-07-22
Publication date: 2023-02-02
Also published as: CN113538223A

Abstract

A noise image generation method and apparatus, an electronic device, and a storage medium, relating to the technical field of image processing. The noise image generation method comprises: obtaining a first image and a second image (110), wherein the first image is a noiseless image, and the second image is a noisy image acquired by using a target image sensor; determining a noise indicator value corresponding to the target image sensor (120); adding noise to the first image according to the noise indicator value to obtain a third image (130); and adjusting the noise distribution of the third image according to the second image to obtain a noise image corresponding to the first image (140).

Description

Noise image generation method, device, electronic equipment and storage medium

Cross References to Related Applications

This application claims priority to the Chinese patent application 202110854815.6 entitled "Noise Image Generation Method, Device, Electronic Equipment, and Storage Medium" filed on July 28, 2021, the entire content of which is incorporated herein by reference.

technical field

The present application belongs to the technical field of image processing, and in particular relates to a noise image generation method, device, electronic equipment and storage medium.

Background technique

With the improvement of users' requirements on image quality, images containing noise captured by electronic devices cannot meet the needs of users more and more. Therefore, it is necessary to perform further noise reduction processing on the captured images.

In the process of image denoising using traditional learning and deep learning and other related artificial intelligence algorithms, it is usually necessary to obtain or construct noise-noise-free sample image pairs to use these sample image pairs to train image denoising models.

At present, when constructing training samples, the noise in the noise image usually uses random noise, so that the noise image obtained by noise synthesis cannot reflect the image sensor in the real electronic device, and the noise generated during the image capture process, Therefore, the synthesized noise image is not realistic enough, thereby reducing the accuracy of the subsequent image denoising model training process.

Contents of the invention

The purpose of the embodiments of the present application is to provide a noise image generation method, device, electronic equipment, and storage medium, which can solve the problem that the synthesized noise image in the prior art is not realistic enough, thereby reducing the accuracy of the subsequent image noise reduction model training process question.

In the first aspect, the embodiment of the present application provides a noise image generation method, the method comprising:

Acquiring a first image and a second image; wherein, the first image is a noise-free image, and the second image is a noisy image collected using a target image sensor;

determining a noise index value corresponding to the target image sensor;

adding noise to the first image according to the noise index value to obtain a third image;

The noise distribution of the third image is adjusted according to the second image to obtain a noise image corresponding to the first image.

In the second aspect, the embodiment of the present application provides a noise image generation device, the device includes:

An acquisition module, configured to acquire a first image and a second image; wherein, the first image is a noise-free image, and the second image is a noisy image collected using a target image sensor;

A determining module, configured to determine a noise index value corresponding to the target image sensor;

A noise adding module, for adding noise to the first image according to the noise index value, to obtain the third image;

The adjustment module is configured to adjust the noise distribution of the third image according to the second image to obtain a noise image corresponding to the first image.

In a third aspect, an embodiment of the present application provides an electronic device, the electronic device includes a processor, a memory, and a program or instruction stored in the memory and operable on the processor, and the program or instruction is The processor implements the steps of the method described in the first aspect when executed.

In a fourth aspect, an embodiment of the present application provides a readable storage medium, on which a program or an instruction is stored, and when the program or instruction is executed by a processor, the steps of the method described in the first aspect are implemented .

In the fifth aspect, the embodiment of the present application provides a chip, the chip includes a processor and a communication interface, the communication interface is coupled to the processor, and the processor is used to run programs or instructions, so as to implement the first aspect The steps of the method.

In the embodiment of the present application, the third image is obtained by acquiring a noise-free first image and a noisy second image collected by the target image sensor, and then adding noise to the first image according to the noise index value corresponding to the target image sensor , and then adjust the noise distribution of the third image according to the second image to obtain the noise image corresponding to the first image, because the third image is a noisy image generated for the noise index value corresponding to the target image sensor, and using the target image sensor The collected real noise image, that is, the second image, is used to optimize the third image. Therefore, the final noise image can be more pertinent to the target image sensor, and the noise in the generated noise image is also closer to The noise of the image actually collected by the target image sensor can improve the accuracy of the subsequent training process of the image noise reduction model for the target sensor.

Description of drawings

Fig. 1 is one of the flowcharts of a noise image generation method shown according to an exemplary embodiment;

Fig. 2 is a workflow diagram of an optimized discriminant network shown according to an exemplary embodiment;

Fig. 3 is the second flowchart of a method for generating a noise image according to an exemplary embodiment;

Fig. 4 is the third flowchart of a method for generating a noise image according to an exemplary embodiment;

Fig. 5 is a structural block diagram of a device for generating noise images according to an exemplary embodiment;

Fig. 6 is a structural block diagram of an electronic device according to an exemplary embodiment;

FIG. 7 is a schematic diagram of a hardware structure of an electronic device implementing an embodiment of the present application.

Detailed ways

The following will clearly describe the technical solutions in the embodiments of the present application with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, but not all of them. All other embodiments obtained by persons of ordinary skill in the art based on the embodiments in this application belong to the protection scope of this application.

The terms "first", "second" and the like in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific sequence or sequence. It should be understood that the terms so used are interchangeable under appropriate circumstances such that the embodiments of the application can be practiced in sequences other than those illustrated or described herein, and that references to "first," "second," etc. distinguish Objects are generally of one type, and the number of objects is not limited. For example, there may be one or more first objects. In addition, "and/or" in the specification and claims means at least one of the connected objects, and the character "/" generally means that the related objects are an "or" relationship.

The noise image generating method, device, electronic device, and storage medium provided by the embodiments of the present application will be described in detail below through specific embodiments and application scenarios with reference to the accompanying drawings.

Because the images straight out of the camera generally contain noise, and the higher the camera’s sensitivity (International Standardization Organization, ISO), the more serious the noise in the captured image, therefore, the image needs to be denoised. Here, ISO is a measure of The international uniform index of traditional camera photosensitive speed.

In the process of image denoising using traditional learning and deep learning and other related artificial intelligence algorithms, it is necessary to obtain or construct noise-noise-free sample image pairs to use these sample image pairs to train image denoising models.

The present application provides a noise image generation method, which can be applied to the scene of constructing noise images.

In addition, the noise image generation method provided in the embodiment of the present application may be executed by a noise image generation device, or a control module used for the noise image generation method in the noise image generation device. In the embodiment of the present application, the noise image generating method performed by the noise image generating device is taken as an example to illustrate the noise image generating method provided in the embodiment of the present application.

Fig. 1 is a flow chart showing a method for generating a noise image according to an exemplary embodiment.

As shown in FIG. 1 , the method for generating a noise image may include steps 110 to 140, specifically as follows.

Step 110, acquiring the first image and the second image.

In this embodiment of the present application, the first image may be a noise-free image, for example, a high-definition color (Red, Green, Blue, RGB) image. The second image may be a noisy image collected using the target image sensor in the target electronic device, where the target image sensor may be a type of sensor that is referred to when generating a noise image in the embodiment of the present application, or may be a sensor using the noise This second image can be used to optimize the initial noisy image for the type of sensor the denoising model is trained on. Both the first image and the second image can be obtained from a public dataset or a dataset captured by a target image sensor. During the acquisition process, the data can also be cleaned, and a large number of clear RGB images can be retained as the first image. In addition, you can also Using the electronic device with the target image sensor to shoot a plurality of RGB images with different luminances, and selecting one of them as the second image.

Step 120, determining a noise index value corresponding to the target image sensor.

Here, since different image sensors have different noise index values, corresponding noise index values need to be determined for different image sensors. Noise index value can be the value that can measure target image sensor noise degree, can represent with noiseVariance=Ax+B, wherein noiseVariance can be noise index value, x can be each in the original (RAWImage Format, RAW) image corresponding to the first image The pixel value of a pixel point, A and B can be parameters, A can be a parameter of one noise, and B can be a parameter of another noise. Therefore, by determining A and B, the noise index value that needs to be added for each pixel in the first image can be determined.

Step 130, adding noise to the first image according to the noise index value to obtain a third image.

Here, the third image can be an initial noise image. In order to better add noise to the first image, noise can be added to the RAW image corresponding to the first image. Specifically, the pixel value x of each pixel in the RAW image can be input In the above noise model noiseVariance=Ax+B, the output is the noise index value that needs to be added for each pixel, so as to obtain the third image, that is, the initial noise image.

Step 140, adjusting the noise distribution of the third image according to the second image to obtain a noise image corresponding to the first image.

Here, the noise image corresponding to the first image may be a noise image closer to an actual scene, and the noise image may be used for training a noise reduction model.

Exemplarily, the noise distribution of the second image can be determined first, and then the noise distribution of the third image can be adjusted according to the noise distribution of the second image, so that the noise distribution of the finally obtained noise image is the same or similar to that of the second image, thereby achieving better The purpose of being close to the noise characteristics of the actual image.

In an optional implementation, in order to further improve the authenticity of the generated noise image, the neural network model can be used to learn the noise distribution of the second image, and then the neural network model can be used to adjust the noise distribution of the third image to achieve The optimization process of the third image. Specifically, the above-mentioned third image can be optimized using an optimized discriminant network, and the function of the optimized discriminant network can be to perform domain confrontation generation between the third image and the second image, so that the noise distribution of the third image and the second image Keep as consistent as possible or similarity greater than a preset threshold in order to improve the authenticity of the generated noisy image.

Based on this, the above-mentioned step 140 may specifically include:

Inputting the third image to P consecutive convolutional layers in the first network to obtain P feature information output by the P convolutional layers;

The feature information output by the Pth convolutional layer is input to the consecutive P deconvolutional layers in the first network, and a noise image corresponding to the first image is outputted.

Here, the first network can be trained according to the second image, P can be a positive integer, P≥2, P convolution layers and P deconvolution layers can be in one-to-one correspondence, and the input information of the first deconvolution layer can be For the first feature information output by the first convolution layer and the second feature information output by the second deconvolution layer, the first convolution layer can be any convolution layer in the P convolution layers, and the first deconvolution The convolution layer may be a deconvolution layer corresponding to the first convolution layer among the P deconvolution layers, and the first deconvolution layer may be the next deconvolution layer of the second deconvolution layer.

Specifically, the third image can be input into consecutive P convolutional layers in the first network, and the target image undergoes matrix operations in each convolution operation Feature _n =w _n (w _n-1 (...(w ₁ x+ b ₁ ))+b _n-1 )+b _n can output the corresponding feature vector, where Feature _n represents the nth convolution module, that is, the convolution layer, the output feature vector, and w is the weight. After the feature vector output by the previous deconvolution layer and the feature vector output by the convolution layer corresponding to the previous deconvolution layer are connected through the skip structure, they are input to the next deconvolution layer, and after each deconvolution operation The matrix operation TransFeature _tn =

Finally, the output is the optimized third image, that is, the noise image corresponding to the first image, where TransFeature _tn represents the feature vector output by the deconvolution layer, w is the weight, and is connected to the previous deconvolution through a skip structure The feature vector output by the convolution layer and the feature vector output by the convolution layer corresponding to the previous deconvolution layer can fully retain the detailed information of the image.

In a specific example, the workflow of the optimized network, that is, the first network, can be shown in FIG. 2 , and the optimized network 220 can include an input convolution layer 221, four convolution modules 2221-2224, and four deconvolution modules 2231 - 2234, output deconvolutional layer 222 and 4 skip connections 22311-22341. Wherein, the input convolution layer 221 corresponds to the output deconvolution layer 222, the first convolution module 2221 corresponds to the fourth deconvolution module 2234, and the second convolution module 2222 corresponds to the third deconvolution module 2233 , the third convolution module 2223 corresponds to the second deconvolution module 2232 , and the fourth convolution module 2224 corresponds to the first deconvolution module 2231 . That is, if the first convolution layer is the input convolution layer 221, then the first deconvolution layer is the output deconvolution layer 222, and the second deconvolution layer is the fourth deconvolution module 2234; if the first convolution The stacked layer is the first convolution module 2221, the first deconvolution layer is the fourth deconvolution module 2234, the second deconvolution layer is the third deconvolution module 2233, and so on, and will not be repeated here .

Exemplarily, the synthetic domain image 210, that is, the third image, is input to four consecutive convolutional layers in the optimization network 220 through the input convolutional layer 221, and four feature information output by the four convolutional layers are obtained, specifically Yes, the first feature vector is obtained through the first convolution module 2221, the second feature vector is obtained through the second convolution module 2222, the third feature vector is obtained through the third convolution module 2223, and the fourth convolution module is obtained 2224, get the fourth eigenvector. Input the first feature information to four consecutive deconvolution layers in the optimization network 220 to obtain the noise image 230, specifically, pass the fourth feature vector and the fifth feature vector obtained through the first deconvolution module 2231 through the first After a skip structure 22311 is connected, it is input to the second deconvolution module 2232 to obtain the sixth feature vector, and after the third feature vector and the sixth feature vector are connected through the second skip structure 22321, they are input to the third deconvolution module 2233, obtain the seventh eigenvector, connect the second eigenvector and the seventh eigenvector through the third skip structure 22331, input it to the fourth deconvolution module 2234, obtain the eighth eigenvector, and combine the first eigenvector and the first eigenvector After the eight feature vectors are connected through the fourth skip structure 22341 , they are input to the output deconvolution layer 222 to obtain the noise image 230 .

In this way, through the continuous P convolutional layers and deconvolutional layers in the first network, and the skip structure, the third image can be optimized to obtain a noise image that is closer to the actual scene than the third image, and to improve the generated noise image. authenticity.

In addition, since there may be multiple noise index values corresponding to the target image sensor, noise may be respectively added to the first image according to the multiple noise index values to obtain multiple third images, for example, M third images.

Regarding the training process of the above-mentioned first network, in an optional implementation manner, when the number of third images is M, step 140 may specifically include:

input the target image to the first network, and output the sixth image;

acquiring a first noise distribution feature corresponding to the sixth image, and a second noise distribution feature corresponding to the second image;

Inputting the first noise distribution feature and the second noise distribution feature to the second network, and outputting the similarity value between the first noise distribution feature and the second noise distribution feature;

When the similarity value is less than the preset threshold, adjust the network parameters of the first network according to the similarity value and the corresponding loss value until the first network converges to obtain a trained first network.

Here, the target image may be any image in the M third images, M may be a positive integer, and M≥2. The sixth image may be an image obtained after the target image is optimized through the first network. The M noise images corresponding to the first image may be noise images closer to real noise distribution than the target image. The first network and the second network can form a generative confrontation network. The second image may be an image randomly selected from multiple RGB images with different brightnesses captured by the target image sensor, and the second image may be randomly selected multiple times during the training process.

Exemplarily, the above-mentioned optimized discriminant network may be a two-stage network model, wherein the first-stage network may be an optimized network, that is, the first network; the second-stage network may be a discriminant network, that is, the second network. The first network can be used to optimize the target image to generate a noise image closer to the real noise distribution, and the second network can be used to determine the difference between the first noise distribution feature corresponding to the sixth image and the second noise distribution feature corresponding to the second image. similarity value between them. Here, the first network can be trained, and when the similarity value is less than the preset threshold, the network parameters of the first network can be adjusted until the first network converges to obtain a trained first network. The first network can be used to generate a noisy image that is more suitable for the actual scene.

In addition, the sixth image output by the optimized network still belongs to the synthetic domain image, and its noise probability distribution can be expressed by Px; the noise probability distribution of the real domain image, that is, the second image, can be expressed by Py. Input the two types of images into the discriminant network, output the feature vector (feature map) through the continuous convolutional layer, and finally input the feature vector into the continuous three-layer fully connected layer, and finally output a probability value in the interval [0, 1] , the probability value represents the similarity between the synthetic domain and the real domain image. When the probability value is closer to 1, it means that the noise distribution of the two images is more similar. When the probability value is closer to 0, it means that the noise distribution of the two images is different. larger.

In addition, it can be set in the discriminant network that when the probability value is lower than 0.5, it is considered that there is a large gap between the noise distribution between the sixth image and the second image, and the probability value is fed back to the optimization network in the first stage, and the optimization network is The weight coefficients (weights) in the network will be adjusted to regenerate the optimized sixth image, and then the regenerated sixth image will be input into the discriminant network for similarity discrimination between the second image and the sixth image. Specifically, the weight adjustment can be determined by solving the partial derivative of the probability loss to the weight, for example, by updating w^new=Δw+w^old to perform weight adjustment and training until the training converges to obtain the final model, that is, after training first network.

In a specific example, the workflow of the discriminant network, that is, the second network, can be shown in FIG. 2. First, the synthetic domain image 210, that is, the target image, is input to the optimization network 220, and the initial noise image 230 is obtained as an output. That is the sixth image. The first noise distribution feature 2301 corresponding to the noise image 230 and the second noise distribution feature 2501 corresponding to the real domain image 250, that is, the second image are obtained through the continuous convolution layer 241 in the discriminant network 240, and the first The noise distribution feature 2301 and the second noise distribution feature 2501 are input to the continuous three-layer fully connected layer 242 in the discriminant network 240, and the output is to obtain the similarity value between the first noise distribution feature 2301 and the second noise distribution feature 2501, and according to The similarity is used to train the optimization network 220 to obtain a trained optimization network, so as to generate noise images that are more suitable for actual scenes.

In this way, during the process of training the first network, the second network is used to judge the similarity between the sixth image and the second image, and then adjust the network parameters of the first network according to the result of the judgment, so that the first network can be It has the ability to optimize the image noise distribution, further improving the authenticity of the noise image generated after the first network optimization.

In addition, in an optional implementation manner, in the case where the first image is a noise-free RGB image, before step 130, the noise image generation method may also include:

Convert the first image from an RGB image to an original image file;

Based on this, the above step 130 may include:

adding noise to the original image file according to the noise index value to obtain a third image;

Convert the third image from the raw image file to an RGB image.

Here, the original image file may be a RAW image. The noise distribution in the RGB image is complex and difficult to deal with. In order to better handle the noise distribution, noise can be added to the RAW image.

Exemplarily, when acquiring the first image, a high-definition RGB image can be acquired, and then the first image is converted from the RGB image to a RAW image through an inverse image signal processing (Image Signal Processing, ISP) operation. Specifically, for example, the RAW image can be obtained by methods such as inverse tone mapping, inverse gamma correction, and inversion of digital gain. Here, the ISP may include processes such as black level compensation, color interpolation (demosaicing), denoising, automatic white balance, and color correction. Among them, inverse tone mapping is a technology used to convert a standard dynamic range (Standard Dynamic Range, SDR) source signal into a high dynamic range (High Dynamic Range, HDR) source signal, which can be applied to production or terminal equipment. To a certain extent, HDR "restoration" and upward compatibility of existing SDR programs can be realized; Gamma anti-correction can be a method of editing the gamma curve of the image to perform nonlinear tone editing on the image, and can detect Gamma correction is the reverse operation of gamma correction.

Based on this, adding noise to the first image according to the noise index value may be adding noise to the RAW image, that is, the original image file, according to the noise index value. The third image thus obtained may also be a RAW image, that is, an original image file. Therefore, before adjusting the noise distribution of the third image, the third image may also be converted from the original image file to an RGB image.

In this way, since the RAW image can better reflect the noise distribution, by converting the first image from an RGB image to a RAW image, and then adding noise, the effect of noise addition can be better, and it is convenient to extract the noise distribution characteristics of the image .

Thus, by acquiring the noise-free first image and the noisy second image collected by the target image sensor, and then adding noise to the first image according to the noise index value corresponding to the target image sensor, the third image is obtained, and then according to the second The second image adjusts the noise distribution of the third image to obtain the noise image corresponding to the first image, because the third image is a noisy image generated for the noise index value corresponding to the target image sensor, and the real noise collected by the target image sensor is used image, that is, the second image, to optimize the third image, so that the final noise image can be more pertinent to the target image sensor, and the noise in the generated noise image is also closer to the target image sensor The noise of the actually collected image can improve the accuracy of the subsequent training process of the image noise reduction model for the target sensor.

Based on the above steps 110-140, in a possible embodiment, as shown in FIG. 3, step 120 may specifically include: steps 1201-1202, wherein:

Step 1201, determine a target Poisson noise index value and a target Gaussian noise index value corresponding to the target image sensor.

Here, regarding the noise index value noiseVariance=Ax+B, A can be the target Poisson noise index value corresponding to the target image sensor, B can be the target Gaussian noise index value corresponding to the target image sensor, and x can be the first The pixel value of each pixel in the image.

Exemplarily, in the case of determining the target Poisson noise index value A and the target Gaussian noise index value B, the noise index value noiseVariance of the target image sensor corresponding to each pixel can be determined.

Based on this, in an optional implementation manner, step 1201 may specifically include:

Acquire N fourth images;

Traversing the N fourth images, respectively calculating the first pixel average value and the first pixel variance value of the pixels contained in each fourth image;

Dividing the first pixel variance value by the first pixel average value to obtain a Poisson noise index value corresponding to each fourth image;

According to the N photosensitivity and N Poisson noise index values corresponding to the N fourth images, determine a first mapping relationship between the Poisson noise index value and the photosensitivity;

According to the first mapping relationship, M Poisson noise index values corresponding to the M target sensitivities are determined as target Poisson noise index values corresponding to the target image sensor.

Here, the fourth image may be an image of a standard color card collected at different sensitivities by using the target image sensor, N may be a positive integer, and N≧2. The M target sensitivities may be M sensitivities determined from the corresponding sensitivity range of the target image sensor, M may be a positive integer, and M≥2.

In a specific example, different shooting devices have different ISO segments. In order to make the generated noise image closer to the real scene, the ISO value can be randomly selected or equally spaced within the ISO segment corresponding to the target image sensor. Select, ISO can be calculated from the analog gain and digital gain set by the target image sensor. Shoot 24-color card images under different ISO conditions, calculate the average value and variance value of the pixels contained in each image in the 24-color card image, divide each variance value by the average value, and get the same as 10 24-color card images The 10 Poisson noise index values corresponding to the image are based on the maximum likelihood estimation algorithm, and the Poisson noise index value and the photosensitive The mapping relationship A=a0·ISO+a1 between degrees, wherein, a0 and a1 can be the parameters of the target image sensor, according to the mapping relationship, can determine the 5 Poissons corresponding to the 5 randomly selected target sensitivities Noise index values, as five target Poisson noise index values A corresponding to the target image sensor.

In this way, by determining the first mapping relationship between the Poisson noise index value and the sensitivity, a plurality of target Poisson noise index values can be obtained, thereby generating a plurality of third images that can basically cover the sensitivity range corresponding to the target image sensor , so that multiple noise images corresponding to the first image are finally obtained, which can fully simulate the noise environment of the target image sensor with respect to Poisson noise in different scenarios.

In addition, in an optional implementation manner, step 1201 may specifically include:

Acquire K fifth images;

Traverse the K fifth images, respectively calculate the second pixel variance values of the pixels contained in each image, and use the second pixel variance values corresponding to the K fifth images as corresponding to the K fifth images K Gaussian noise index values;

Based on the maximum likelihood estimation algorithm, according to K sensitivities and K Gaussian noise index values corresponding to the K fifth images, determine a second mapping relationship between the Gaussian noise index value and the sensitivity;

According to the second mapping relationship, M Gaussian noise index values corresponding to the M target sensitivities are determined as target Gaussian noise index values corresponding to the target image sensor.

Here, the fifth image may be an image collected at different sensitivities using the target image sensor, the image may be a black image, K may be a positive integer, and K≧2. Specifically, the fifth image may be acquired by using the target image sensor to capture a black image, or may be obtained by capturing an image while blocking a lens of the target image sensor.

In a specific example, 20 all-black images can be taken under different ISO conditions, and the variance value of the pixels contained in each image in the 20 all-black images can be calculated, and the 20 variance values can be compared with the 20 all-black images For the 20 Gaussian noise index values corresponding to the image, based on the maximum likelihood estimation algorithm, determine the mapping relationship between the Gaussian noise index value and the sensitivity B=b0·ISO·ISO+b1·ISO, where both b0 and b1 can be For the parameters of the target image sensor, according to the mapping relationship, 10 Gaussian noise index values corresponding to 10 target sensitivities can be determined as the 10 Gaussian noise index values B corresponding to the target image sensor.

In this way, by determining the second mapping relationship between the Gaussian noise index value and the sensitivity, a plurality of target Gaussian noise index values can be obtained, thereby generating a plurality of third images that can basically cover the sensitivity range corresponding to the target image sensor, so that Finally, multiple noise images corresponding to the first image are obtained, which can fully simulate the noise environment of the target image sensor with respect to Gaussian noise in different scenes.

Step 1202, according to the target Poisson noise index value and the target Gaussian noise index value, calculate the noise index value corresponding to the target image sensor.

In a specific example, according to multiple target Poisson noise index values A and multiple target Gaussian noise index values B, the noise index value noiseVariance=Ax+B corresponding to the target image sensor can be calculated.

In this way, through the above process, the noise of the Gaussian distribution and the noise of the Poisson distribution can be synthesized at the same time, making the noise distribution more diverse and further improving the authenticity of the noise image.

In order to better describe the whole solution, based on the foregoing embodiments, a specific example is given, as shown in FIG. 4 , the method for generating a noise image may include steps 410-450, which will be explained in detail below.

Step 410, acquiring the first image and the second image.

Here, a high-definition RGB image, that is, the first image, and a noisy RGB image obtained by using the target image sensor, that is, the second image may be acquired.

Step 420, converting the first image into a RAW image.

Here, the noise distribution in the RGB image is complex and difficult to deal with, so noise can be added on the basis of the RAW image, but it is difficult to obtain a high-definition RAW image, so you can first obtain the RGB image, and then convert the RGB image to RAW image, the specific conversion method will not be repeated here.

Step 430, perform noise index calibration on the sensor.

Here, since different sensors have different noise intensities, it is necessary to calibrate their noise index values for different sensors. The noise model can be represented by noiseVariance=Ax+B, where A and B are the noise index values to be calibrated, A can be the target Poisson noise index value, B can be the target Gaussian noise index value, and x can be the RAW image The pixel value of each pixel, noiseVariance may be a noise variance value corresponding to each pixel for the target image sensor, that is, a noise index value. The specific calibration process will not be repeated here.

Step 440, generating a third image based on the RAW image.

Here, adding noise based on the RAW image makes it easier to handle the noise distribution.

Step 450, input the third image and the second image into the optimization discriminant network.

Here, the initial noise image, that is, the third image, and the second image are input into the optimization discriminant network for multiple optimization and discrimination, and a noise image closer to the actual scene can be obtained. Of course, the first image and one or more noise images output by the optimized discriminant network can be used to form a noise-noise-free sample image pair for training an image denoising model corresponding to the target image sensor.

Based on the same inventive concept, the present application also provides a noise image generation device. The device for generating a noise image provided by the embodiment of the present application will be described in detail below with reference to FIG. 5 .

Fig. 5 is a structural block diagram of an apparatus for generating a noise image according to an exemplary embodiment.

As shown in Figure 5, the noise image generation device 500 may include:

An acquisition module 501, configured to acquire a first image and a second image; wherein, the first image is a noise-free image, and the second image is a noisy image collected using a target image sensor;

A determining module 502, configured to determine a noise index value corresponding to the target image sensor;

Noise adding module 503, for adding noise to the first image according to the noise index value, to obtain the third image;

The adjusting module 504 is configured to adjust the noise distribution of the third image according to the second image to obtain a noise image corresponding to the first image.

The above noise image generation device 500 will be described in detail below, specifically as follows:

In one of the embodiments, the determining module 502 may include:

A determination sub-module is used to determine a target Poisson noise index value and a target Gaussian noise index value corresponding to the target image sensor;

The calculation sub-module is used to calculate the noise index value corresponding to the target image sensor according to the target Poisson noise index value and the target Gaussian noise index value.

In one of the embodiments, the determining submodule may include:

The first acquisition unit is configured to acquire N fourth images; wherein, the fourth image is an image of a standard color card collected at different sensitivities using the target image sensor, N is a positive integer, and N≥2;

The first calculation unit is configured to traverse the N fourth images, and respectively calculate the first pixel average value and the first pixel variance value of the pixels contained in each fourth image;

The second calculation unit is used to divide the first pixel variance value by the first pixel average value to obtain the Poisson noise index value corresponding to each fourth image;

The first relationship determination unit is configured to determine a first mapping relationship between the Poisson noise index value and the sensitivity according to the N sensitivities corresponding to the N fourth images and the N Poisson noise index values;

The first index determination unit is configured to determine M Poisson noise index values corresponding to M target sensitivities according to the first mapping relationship as target Poisson noise index values corresponding to the target image sensor; wherein, the M targets The photosensitivity is M photosensitivity determined from the photosensitivity range corresponding to the target image sensor, M is a positive integer, and M≥2.

In one of the embodiments, the determining submodule may also include:

The second acquiring unit is used to acquire K fifth images; wherein, the fifth images are images acquired by using the target image sensor at different sensitivities, K is a positive integer, and K≥2;

The third calculation unit is used to traverse the K fifth images, respectively calculate the second pixel variance values of the pixels contained in each fifth image, and use the second pixel variance values corresponding to the K fifth images respectively K Gaussian noise index values corresponding to the K fifth images;

The second relationship determination unit is configured to determine a second mapping relationship between the Gaussian noise index value and the sensitivity according to K sensitivities and K Gaussian noise index values corresponding to the K fifth images;

The second index determining unit is configured to determine M Gaussian noise index values corresponding to the M target sensitivities according to the second mapping relationship as target Gaussian noise index values corresponding to the target image sensor.

In one of the embodiments, the adjustment module 504 includes:

The image input sub-module is used to input the target image to consecutive P convolutional layers to obtain P feature information output by the P convolutional layers;

The feature input submodule is used to input the feature information output by the Pth convolutional layer to the continuous P deconvolutional layers, and output the sixth image;

Among them, P is a positive integer, P≥2, P convolutional layers correspond to P deconvolutional layers one by one, and the input information of the first deconvolutional layer is the first feature information and the first feature information output by the first convolutional layer. The second feature information output by the two deconvolution layers, the first convolution layer is any convolution layer in the P convolution layers, and the first deconvolution layer is the P deconvolution layer and the first convolution layer The deconvolution layer corresponding to the layer, the first deconvolution layer is the next deconvolution layer of the second deconvolution layer.

In one of the embodiments, when the number of third images is M, the adjustment module 504 may further include:

The target image processing sub-module is used to input the target image to the first network before inputting the third image to the P consecutive convolutional layers in the first network to obtain the P feature information output by the P convolutional layers, The sixth image is outputted; wherein, the target image is any image in the M third images;

An acquisition submodule, configured to acquire a first noise distribution feature corresponding to the sixth image, and a second noise distribution feature corresponding to the second image;

The feature processing sub-module is used to input the first noise distribution feature and the second noise distribution feature to the second network, and output the similarity value between the first noise distribution feature and the second noise distribution feature;

The adjustment sub-module is used to adjust the network parameters of the first network according to the similarity value and its corresponding loss value when the similarity value is less than the preset threshold value until the first network converges to obtain the trained first network .

In one of the embodiments, the first image is a noise-free RGB image;

The noise image generation device 500 may also include:

A conversion module 505, configured to convert the first image from an RGB image to an original image file before adding noise to the first image according to the noise index value to obtain a third image;

Noise adding module 503 may include:

The noise adding submodule is used to add noise to the original image file according to the noise index value to obtain the third image;

The conversion sub-module is used to convert the third image from the original image file to an RGB image.

The apparatus for generating a noise image in the embodiment of the present application may be a device, or may be a component, an integrated circuit, or a chip in a terminal. The device may be a mobile electronic device or a non-mobile electronic device. Exemplarily, the mobile electronic device may be a mobile phone, a tablet computer, a notebook computer, a handheld computer, a vehicle electronic device, a wearable device, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook or a personal digital assistant (personal digital assistant). assistant, PDA), etc., non-mobile electronic devices can be servers, network attached storage (Network Attached Storage, NAS), personal computer (personal computer, PC), television (television, TV), teller machine or self-service machine, etc., this application Examples are not specifically limited.

The noise image generation device in the embodiment of the present application may be a device with an operating system. The operating system may be an Android (Android) operating system, an ios operating system, or other possible operating systems, which are not specifically limited in this embodiment of the present application.

The noise image generating device provided in the embodiment of the present application can realize various processes realized by the method embodiments in FIG. 1 to FIG. 4 , and details are not repeated here to avoid repetition.

Optionally, as shown in FIG. 6 , the embodiment of the present application further provides an electronic device 600, including a processor 601, a memory 602, and programs or instructions stored in the memory 602 and operable on the processor 601, When the program or instruction is executed by the processor 601, each process of the above noise image generating method embodiment can be realized, and the same technical effect can be achieved, so in order to avoid repetition, details are not repeated here.

It should be noted that the electronic devices in the embodiments of the present application include the above-mentioned mobile electronic devices and non-mobile electronic devices.

The electronic device 700 includes, but is not limited to: a radio frequency unit 701, a network module 702, an audio output unit 703, an input unit 704, a sensor 705, a display unit 706, a user input unit 707, an interface unit 708, a memory 709, and a processor 710, etc. part.

Those skilled in the art can understand that the electronic device 700 can also include a power supply (such as a battery) for supplying power to various components, and the power supply can be logically connected to the processor 710 through the power management system, so that the management of charging, discharging, and function can be realized through the power management system. Consumption management and other functions. The structure of the electronic device shown in FIG. 7 does not constitute a limitation to the electronic device. The electronic device may include more or fewer components than shown in the figure, or combine some components, or arrange different components, and details will not be repeated here. .

Wherein, the input unit 704 is configured to acquire a first image and a second image; wherein, the first image is a noise-free image, and the second image is a noisy image collected using a target image sensor;

The processor 710 is configured to determine a noise index value corresponding to the target image sensor; add noise to the first image according to the noise index value to obtain a third image; adjust the noise distribution of the third image according to the second image to obtain a noise distribution similar to the first image corresponding noise image.

Thus, by acquiring the noise-free first image and the noisy second image collected by the target image sensor, and then adding noise to the first image according to the noise index value corresponding to the target image sensor, the third image is obtained, and then according to the second The second image adjusts the noise distribution of the third image to obtain the noise image corresponding to the first image, because the third image is a noisy image generated for the noise index value corresponding to the target image sensor, and the real noise collected by the target image sensor is used image, that is, the second image, to optimize the third image, so that the final noise image can be more pertinent to the target image sensor, and the generated noise image is more realistic, which can improve subsequent image noise reduction. The accuracy of the model training process.

Optionally, the processor 710 is configured to determine a target Poisson noise index value and a target Gaussian noise index value corresponding to the target image sensor;

According to the target Poisson noise index value and the target Gaussian noise index value, the noise index value corresponding to the target image sensor is calculated.

Optionally, the input unit 704 is specifically configured to acquire N fourth images; wherein, the fourth images are images of standard color cards collected at different sensitivities using the target image sensor, N is a positive integer, and N≥2 ;

The processor 710 is specifically configured to traverse the N fourth images, respectively calculate the first pixel average value and the first pixel variance value of the pixels contained in each fourth image; divide the first pixel variance value by the first pixel variance value The average value of one pixel is used to obtain the Poisson noise index value corresponding to each fourth image; according to the N photosensitivity and N Poisson noise index values corresponding to the N fourth images, the Poisson noise index value and the photosensitivity index value are determined. The first mapping relationship between degrees; according to the first mapping relationship, determine M Poisson noise index values corresponding to M target sensitivities, as the target Poisson noise index value corresponding to the target image sensor; wherein, M The target sensitivity is M sensitivity determined from the sensitivity range corresponding to the target image sensor, M is a positive integer, and M≥2.

Optionally, the input unit 704 is also specifically configured to acquire K fifth images; wherein, the fifth images are images collected at different sensitivities using the target image sensor, K is a positive integer, and K≥2;

The processor 710 is specifically further configured to traverse the K fifth images, respectively calculate the second pixel variance values of the pixels contained in each fifth image, and calculate the second pixel variance values corresponding to the K fifth images respectively value as the K Gaussian noise index values corresponding to the K fifth images; according to the K sensitivities and K Gaussian noise index values corresponding to the K fifth images, determine the Gaussian noise index value and the sensitivity between the first Two mapping relationships: according to the second mapping relationship, determine M Gaussian noise index values corresponding to M target sensitivities as target Gaussian noise index values corresponding to the target image sensor.

Optionally, the processor 710 is also specifically configured to input the third image to P consecutive convolutional layers in the first network to obtain P feature information output by the P convolutional layers; wherein, the first network is based on The second image is trained; the feature information output by the Pth convolutional layer is input to the continuous P deconvolution layers in the first network, and the output is a noise image corresponding to the first image; wherein, P is a positive integer, P≥2, P convolutional layers correspond to P deconvolutional layers one by one, the input information of the first deconvolutional layer is the first feature information output by the first convolutional layer and the output of the second deconvolutional layer The second feature information, the first convolution layer is any convolution layer in the P convolution layers, and the first deconvolution layer is the deconvolution layer corresponding to the first convolution layer in the P deconvolution layers , the first deconvolution layer is the next deconvolution layer of the second deconvolution layer.

Optionally, the processor 710 is specifically further configured to input the target image to the first network when the number of the third images is M, and output the sixth image; wherein, the target image is M third images Arbitrary image in; Obtain the first noise distribution feature corresponding to the sixth image, and the second noise distribution feature corresponding to the second image; Input the first noise distribution feature and the second noise distribution feature to the second network, output Obtaining the similarity value between the first noise distribution feature and the second noise distribution feature; in the case that the similarity value is less than a preset threshold, adjust the network parameters of the first network according to the similarity value and its corresponding loss value, Until the first network converges, the trained first network is obtained.

Optionally, the processor 710 is also specifically configured to convert the first image from an RGB image to an original image file; add noise to the original image file according to the noise index value to obtain a third image; convert the third image from the original image file for an RGB image.

Therefore, by synthesizing the noise of Gaussian distribution and the noise of Poisson distribution at the same time, the noise distribution is made more diverse, and the authenticity of the noise image can be further improved.

It should be understood that, in the embodiment of the present application, the input unit 704 may include a graphics processor (Graphics Processing Unit, GPU) 14041 and a microphone 7042, and the graphics processor 7041 is used for the image capture device ( Such as the image data of the still picture or video obtained by the camera) for processing. The display unit 706 may include a display panel 7061, and the display panel 7061 may be configured in the form of a liquid crystal display, an organic light emitting diode, or the like. The user input unit 707 includes a touch panel 7071 and other input devices 7072 . The touch panel 7071 is also called a touch screen. The touch panel 7071 may include two parts, a touch detection device and a touch controller. Other input devices 7072 may include, but are not limited to, physical keyboards, function keys (such as volume control buttons, switch buttons, etc.), trackballs, mice, and joysticks, which will not be repeated here. Memory 709 may be used to store software programs as well as various data, including but not limited to application programs and operating systems. The processor 710 may integrate an application processor and a modem processor, wherein the application processor mainly processes an operating system, user interface, application program, etc., and the modem processor mainly processes wireless communication. It can be understood that the foregoing modem processor may not be integrated into the processor 710 .

The embodiment of the present application also provides a readable storage medium, the readable storage medium stores a program or an instruction, and when the program or instruction is executed by the processor, each process of the above embodiment of the method for generating a noise image is realized, and can achieve The same technical effects are not repeated here to avoid repetition.

Wherein, the processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer readable storage medium, such as computer read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk, etc.

The embodiment of the present application further provides a chip, the chip includes a processor and a communication interface, the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the above embodiment of the noise image generation method Each process, and can achieve the same technical effect, in order to avoid repetition, will not repeat them here.

It should be understood that the chips mentioned in the embodiments of the present application may also be called system-on-chip, system-on-chip, system-on-a-chip, or system-on-a-chip.

It should be noted that, in this document, the term "comprising", "comprising" or any other variation thereof is intended to cover a non-exclusive inclusion such that a process, method, article or apparatus comprising a set of elements includes not only those elements, It also includes other elements not expressly listed, or elements inherent in the process, method, article, or device. Without further limitations, an element defined by the phrase "comprising a ..." does not preclude the presence of additional identical elements in the process, method, article, or apparatus comprising that element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved. Functions are performed, for example, the described methods may be performed in an order different from that described, and various steps may also be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

Through the description of the above embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus a necessary general-purpose hardware platform, and of course also by hardware, but in many cases the former is better implementation. Based on such an understanding, the technical solution of the present application can be embodied in the form of computer software products, which are stored in a storage medium (such as ROM/RAM, magnetic disk, etc.) , optical disc), including several instructions to enable a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in various embodiments of the present application.

The embodiments of the present application have been described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementations. The above-mentioned specific implementations are only illustrative and not restrictive. Those of ordinary skill in the art will Under the inspiration of this application, without departing from the purpose of this application and the scope of protection of the claims, many forms can also be made, all of which belong to the protection of this application.

Claims

A noise image generation method, comprising:

Acquiring a first image and a second image; wherein, the first image is a noise-free image, and the second image is a noisy image collected using a target image sensor;

determining a noise index value corresponding to the target image sensor;

adding noise to the first image according to the noise index value to obtain a third image;

adjusting the noise distribution of the third image according to the second image to obtain a noise image corresponding to the first image.
The method according to claim 1, wherein said determining the noise index value corresponding to the target image sensor comprises:

determining a target Poisson noise index value and a target Gaussian noise index value corresponding to the target image sensor;

A noise index value corresponding to the target image sensor is calculated according to the target Poisson noise index value and the target Gaussian noise index value.
The method according to claim 2, wherein said determining the target Poisson noise index value corresponding to the target image sensor comprises:

Acquiring N fourth images; wherein, the fourth image is an image of a standard color card collected at different sensitivities using the target image sensor, N is a positive integer, and N≥2;

Traversing the N fourth images, respectively calculating the first pixel average value and the first pixel variance value of the pixels contained in each fourth image;

Dividing the first pixel variance value by the first pixel average value to obtain a Poisson noise index value corresponding to each fourth image;

determining a first mapping relationship between the Poisson noise index value and the sensitivity according to the N sensitivities and N Poisson noise index values corresponding to the N fourth images;

According to the first mapping relationship, determine M Poisson noise index values corresponding to M target sensitivities as target Poisson noise index values corresponding to the target image sensor; wherein, the M target sensitivities are M photosensitivity determined from the photosensitivity range corresponding to the target image sensor, M is a positive integer, and M≥2.
The method according to claim 2, wherein said determination of a target Gaussian noise index value corresponding to said target image sensor comprises:

Acquiring K fifth images; wherein, the fifth images are images collected at different sensitivities using the target image sensor, K is a positive integer, and K≥2;

Traverse the K fifth images, respectively calculate the second pixel variance values of the pixels contained in each fifth image, and use the second pixel variance values corresponding to the K fifth images as the K Gaussian noise index values corresponding to the K fifth images;

determining a second mapping relationship between the Gaussian noise index value and the sensitivity according to the K sensitivities corresponding to the K fifth images and the K Gaussian noise index values;

According to the second mapping relationship, M number of Gaussian noise index values corresponding to M target sensitivities are determined as target Gaussian noise index values corresponding to the target image sensor.
The method according to claim 1, wherein said adjusting the noise distribution of the third image according to the second image to obtain a noise image corresponding to the first image comprises:

The third image is input to P consecutive convolutional layers in the first network to obtain P feature information output by the P convolutional layers; wherein, the first network is obtained according to the training of the second image;

The feature information output by the Pth convolutional layer is input to the continuous P deconvolutional layers in the first network, and the noise image corresponding to the first image is obtained by outputting;

Wherein, P is a positive integer, P≥2, and the P convolution layers correspond to the P deconvolution layers one by one, and the input information of the first deconvolution layer is the first output of the first convolution layer. Feature information and the second feature information output by the second deconvolution layer, the first convolution layer is any convolution layer in the P convolution layers, and the first deconvolution layer is the A deconvolution layer corresponding to the first deconvolution layer among the P deconvolution layers, the first deconvolution layer being the next deconvolution layer of the second deconvolution layer.
The method according to claim 5, wherein, when the number of the third images is M, when the third images are input to the P consecutive convolutional layers in the first network, P Before the P feature information output by the convolutional layer, the method also includes:

Inputting the target image into the first network, and outputting a sixth image; wherein, the target image is any image in the M third images;

acquiring a first noise distribution feature corresponding to the sixth image, and a second noise distribution feature corresponding to the second image;

inputting the first noise distribution feature and the second noise distribution feature into a second network, and outputting a similarity value between the first noise distribution feature and the second noise distribution feature;

When the similarity value is less than a preset threshold, according to the similarity value and its corresponding loss value, adjust the network parameters of the first network until the first network converges to obtain the trained first network. a network.
A noise image generation device, comprising:

An acquisition module, configured to acquire a first image and a second image; wherein, the first image is a noise-free image, and the second image is a noisy image collected using a target image sensor;

A determining module, configured to determine a noise index value corresponding to the target image sensor;

A noise adding module, configured to add noise to the first image according to the noise index value to obtain a third image;

An adjustment module, configured to adjust the noise distribution of the third image according to the second image to obtain a noise image corresponding to the first image.
The device according to claim 7, wherein the determining module comprises:

A determining submodule, configured to determine a target Poisson noise index value and a target Gaussian noise index value corresponding to the target image sensor;

The calculation sub-module is used to calculate the noise index value corresponding to the target image sensor according to the target Poisson noise index value and the target Gaussian noise index value.
The device according to claim 8, wherein the determination submodule comprises:

The first acquisition unit is configured to acquire N fourth images; wherein, the fourth images are images of standard color cards collected at different sensitivities using the target image sensor, N is a positive integer, and N≥2 ;

The first calculation unit is configured to traverse the N fourth images, and respectively calculate the first pixel average value and the first pixel variance value of the pixels contained in each fourth image;

A second calculation unit, configured to divide the first pixel variance value by the first pixel average value to obtain a Poisson noise index value corresponding to each fourth image;

The first relationship determination unit is configured to determine the first relationship between the Poisson noise index value and the sensitivity according to the N sensitivities and N Poisson noise index values corresponding to the N fourth images. Mapping relations;

The first index determination unit is configured to determine M Poisson noise index values corresponding to M target sensitivities according to the first mapping relationship, as target Poisson noise index values corresponding to the target image sensor; wherein , the M target sensitivities are M sensitivities determined from the sensitivity range corresponding to the target image sensor, M is a positive integer, and M≥2.
The device according to claim 8, wherein the determination submodule comprises:

The second acquisition unit is configured to acquire K fifth images; wherein, the fifth images are images acquired by using the target image sensor at different sensitivities, K is a positive integer, and K≥2;

The third calculation unit is configured to traverse the K fifth images, respectively calculate the second pixel variance values of the pixels contained in each fifth image, and use the second pixel variance values corresponding to the K fifth images respectively The pixel variance value is used as K Gaussian noise index values corresponding to the K fifth images;

The second relationship determination unit is configured to determine the relationship between the Gaussian noise index value and the K sensitivities corresponding to the K fifth images and the K Gaussian noise index values based on a maximum likelihood estimation algorithm. a second mapping relationship between sensitivities;

The second index determination unit is configured to determine M Gaussian noise index values corresponding to the M target sensitivities according to the second mapping relationship as target Gaussian noise index values corresponding to the target image sensor.
The device according to claim 7, wherein the adjustment module comprises:

An image input sub-module, configured to input the target image to consecutive P convolutional layers to obtain P feature information output by the P convolutional layers;

The feature input submodule is used to input the feature information output by the Pth convolutional layer to the continuous P deconvolutional layers, and output the sixth image;

Wherein, P is a positive integer, P≥2, and the P convolution layers correspond to the P deconvolution layers one by one, and the input information of the first deconvolution layer is the first output of the first convolution layer. Feature information and the second feature information output by the second deconvolution layer, the first convolution layer is any convolution layer in the P convolution layers, and the first deconvolution layer is the A deconvolution layer corresponding to the first deconvolution layer among the P deconvolution layers, the first deconvolution layer being the next deconvolution layer of the second deconvolution layer.
The device according to claim 11, wherein, when the number of the third images is M, the adjustment module further includes:

The target image processing sub-module is used to input the target image to the P consecutive convolutional layers in the first network before obtaining the P feature information output by the P convolutional layers. The first network is output to obtain a sixth image; wherein, the target image is any image in the M third images;

An acquisition submodule, configured to acquire a first noise distribution feature corresponding to the sixth image, and a second noise distribution feature corresponding to the second image;

A feature processing sub-module, configured to input the first noise distribution feature and the second noise distribution feature to a second network, and output the similarity between the first noise distribution feature and the second noise distribution feature degree value;

An adjustment submodule, configured to adjust network parameters of the first network according to the similarity value and its corresponding loss value when the similarity value is less than a preset threshold until the first network converges , to obtain the trained first network.
An electronic device, comprising a processor, a memory, and a program or instruction stored on the memory and operable on the processor, when the program or instruction is executed by the processor, claims 1-6 are realized The steps of any one of the noisy image generating methods.
A readable storage medium, storing programs or instructions on the readable storage medium, and implementing the steps of the method for generating noise images according to any one of claims 1-6 when the programs or instructions are executed by a processor.
An electronic device configured to execute the steps of the method for generating a noise image according to any one of claims 1-6.
A computer program product, the computer program product is executed by a processor to implement the steps of the method for generating a noise image according to any one of claims 1-6.
A chip, the chip includes a processor and a communication interface, the communication interface is coupled to the processor, the processor is used to run a program or an instruction, and realize the noise as described in any one of claims 1-6 Steps of the image generation method.