CN114937194A

CN114937194A - Training method of image model, image denoising method, device, equipment and medium

Info

Publication number: CN114937194A
Application number: CN202210513484.4A
Authority: CN
Inventors: 干逸显; 王洋; 黄英仁; 吕中厚; 张华正; 田伟娟
Original assignee: Beijing Baidu Netcom Science and Technology Co Ltd
Current assignee: Beijing Baidu Netcom Science and Technology Co Ltd
Priority date: 2022-05-11
Filing date: 2022-05-11
Publication date: 2022-08-23

Abstract

The disclosure provides a training method of an image model, an image denoising method, a device, equipment and a medium, and relates to the field of artificial intelligence, in particular to the field of computer vision and deep learning. The specific implementation scheme is as follows: inputting a counterattack sample into a counterattack model to obtain a sample counterattack image, and denoising the counterattack sample by adopting the sample counterattack image to obtain a sample denoising image; and respectively extracting the characteristics of the sample noise reduction image and the sample original image, and training the anti-disturbance model according to the characteristic extraction result. The present disclosure may improve defense against the resistant sample.

Description

Training method of image model, image denoising method, device, equipment and medium

Technical Field

The disclosure relates to the technical field of artificial intelligence, in particular to the field of computer vision and deep learning, and specifically discloses an image model training method, an image denoising device, image denoising equipment and a medium.

Background

A challenge sample is an input sample formed by deliberately adding subtle perturbations to the dataset that cause the model to give an erroneous output with high confidence such that the model cannot effectively identify the challenge sample.

With the development of artificial intelligence, confrontational samples are beginning to be paid attention to gradually. Recently, various attack algorithms aiming at a target detection model appear, and the detection result of the model on an input picture can be disturbed by adding a small disturbance to an original picture, for example, a certain object in the picture disappears, or the model is wrongly classified to the certain object, but the recognition of human eyes on the picture is not influenced.

How to enable the target detection model to effectively identify and process the small disturbances becomes an urgent problem to be solved.

BRIEF SUMMARY OF THE PRESENT DISCLOSURE

The disclosure provides a training method of an image model, an image denoising method, an image denoising device, equipment and a medium.

According to an aspect of the present disclosure, there is provided a training method of an image model, including:

inputting a counterattack sample into a counterattack model to obtain a sample counterattack image, and denoising the counterattack sample by adopting the sample counterattack image to obtain a sample denoising image;

and respectively extracting the characteristics of the sample noise reduction image and the sample original image, and training the anti-disturbance model according to the characteristic extraction result.

According to still another aspect of the present disclosure, there is provided an image noise reduction method including:

inputting the image to be processed into the trained anti-disturbance model to obtain a target anti-disturbance image;

and denoising the image to be processed by adopting the target anti-disturbance image to obtain a target denoising image.

According to still another aspect of the present disclosure, there is provided an image model training apparatus including:

the sample denoising module is used for inputting the countermeasure sample into the inverse disturbance model to obtain a sample inverse disturbance image, and denoising the countermeasure sample by adopting the sample inverse disturbance image to obtain a sample denoising image;

and the anti-disturbance model training module is used for respectively extracting the characteristics of the sample noise reduction image and the sample original image and training the anti-disturbance model according to the characteristic extraction result.

According to still another aspect of the present disclosure, there is provided an image noise reduction device including:

the anti-disturbance image acquisition module is used for inputting the image to be processed into the trained anti-disturbance model to obtain a target anti-disturbance image;

and the noise reduction image acquisition module is used for carrying out noise reduction on the image to be processed by adopting the target anti-disturbance image to obtain a target noise reduction image.

According to still another aspect of the present disclosure, there is provided an electronic apparatus including:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform a method of image model training or image noise reduction provided by any embodiment of the present disclosure.

According to yet another aspect of the present disclosure, there is provided a non-transitory computer readable storage medium having stored thereon computer instructions for causing a computer to execute a training method of an image model or an image denoising method provided by any of the embodiments of the present disclosure.

According to yet another aspect of the present disclosure, a computer program product is provided, comprising a computer program which, when executed by a processor, implements a method of training an image model or a method of image denoising as provided by any of the embodiments of the present disclosure.

It should be understood that the statements in this section are not intended to identify key or critical features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description.

Drawings

The drawings are included to provide a better understanding of the present solution and are not to be construed as limiting the present disclosure. Wherein:

FIG. 1 is a schematic diagram of a method for training an image model according to an embodiment of the present disclosure;

FIG. 2A is a schematic diagram of a method for training an image model according to another embodiment of the present disclosure;

FIG. 2B is a schematic diagram of an anti-perturbation unit structure according to another embodiment of the disclosure;

FIG. 3A is a schematic diagram of an image denoising method according to another embodiment of the present disclosure;

FIG. 3B is a schematic illustration of a challenge sample test according to yet another embodiment of the present disclosure;

FIG. 3C is a schematic diagram illustrating detection of a denoised challenge sample according to yet another embodiment of the present disclosure;

FIG. 3D is a schematic illustration of the anti-perturbation of a challenge sample according to yet another embodiment of the present disclosure;

FIG. 4 is a diagram illustrating an image denoising method according to another embodiment of the present disclosure;

FIG. 5 is a schematic diagram of an image model training apparatus according to yet another embodiment of the present disclosure;

fig. 6 is a schematic diagram of an image noise reduction apparatus provided according to yet another embodiment of the present disclosure;

FIG. 7 is a block diagram of an electronic device used to implement an embodiment of the disclosure.

Detailed Description

Exemplary embodiments of the present disclosure are described below with reference to the accompanying drawings, in which various details of the embodiments of the disclosure are included to assist understanding, and which are to be considered as merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present disclosure. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.

Other methods for detecting the challenge samples exist in the related art, such as conventional image processing methods, e.g., Joint Photographic Experts Group (JPEG) compression, or other filtering algorithms, can be used to perform noise reduction on the input image. Or simultaneously carrying out noise reduction on the input image by multiple means, and judging whether the input image is a countermeasure sample or not by analyzing the difference between the input image and the original image after noise reduction. However, in practical applications, these conventional image processing means are not ideal for detecting the countermeasure sample, and other existing detectors can only determine whether the input sample is the countermeasure sample, and if it is true, alarm is given, and then the input image is manually processed. However, in a specific application scenario of the target detection model, a condition for taking over at any time by a human is not provided in some cases, and therefore it is not sufficient to merely judge whether an input image is a countermeasure sample, but the model is required to still give a more accurate output result even if an immediate input image is a countermeasure sample.

In the related art, the influence of malicious disturbance noise in the countermeasure sample on the model identification result can be diluted by preprocessing the countermeasure sample such as transformation and compression. However, the conventional preprocessing method does not aim at the challenge sample at the beginning of design, so that the defense effect on the challenge sample is very limited, or the influence on the non-malicious sample is large. The robustness of the model itself to malicious attacks can also be increased by introducing countervailing samples when training the model. However, this defense method needs to change the parameters of the model itself, and cannot be applied to the trained model.

Fig. 1 is a flowchart of a training method for an image model, which is provided according to an embodiment of the present disclosure, and this embodiment is applicable to a case where an anti-disturbance model is trained using a challenge sample, and this method may be performed by an image model training apparatus, which may be implemented in software and/or hardware. The device can be configured in an electronic device with corresponding data processing capability, and the method specifically comprises the following steps:

s110, inputting the countermeasure sample into an anti-disturbance model to obtain a sample anti-disturbance image, and denoising the countermeasure sample by adopting the sample anti-disturbance image to obtain a sample denoising image;

and S120, respectively extracting the characteristics of the sample noise reduction image and the sample original image, and training the anti-disturbance model according to the characteristic extraction result.

The training sample pair of the anti-disturbance model comprises a sample original image and an anti-sample of the sample original image, and the anti-sample is obtained by injecting malicious disturbance into the sample original image. The sample anti-disturbance image comprises anti-disturbance opposite to the malicious disturbance characteristic, and the sample noise reduction image is obtained by counteracting the malicious disturbance in the countersample by using the anti-disturbance in the anti-disturbance image.

Specifically, the counterattack sample injected with the malicious disturbance is input into an anti-disturbance model, the malicious disturbance is learned and amplified through an anti-disturbance unit in the anti-disturbance model, the anti-disturbance opposite to the malicious disturbance is generated, and the anti-disturbance model generates and outputs a sample anti-disturbance image according to the anti-disturbance. And carrying out image fusion on the sample noise reduction image and the countermeasure sample to enable opposite malicious disturbance and inverse disturbance to be mutually offset, so that the malicious disturbance in the countermeasure sample is eliminated, and the sample noise reduction image is obtained.

In this embodiment, feature extraction may be performed on the denoised sample denoising image and the sample original image through a task model, and a loss function of the anti-disturbance model is constructed according to feature extraction results of the denoised sample denoising image and the sample original image; and calculating the gradient of the parameters in the inverse disturbance model to the loss function in a reverse mode, and updating the parameters to be trained in the inverse disturbance model according to the gradient. The task model is different from the anti-disturbance model, and the task model is used for executing preset tasks on the noise reduction image, such as target detection, image classification and other tasks. And respectively carrying out feature extraction on the sample noise reduction image and the sample original image through the task model to obtain a sample noise reduction feature and a sample original feature which are used as feature extraction results of the sample noise reduction feature and the sample original feature. The task model can carry out nonlinear amplification processing on the sample noise reduction image and the sample original image, namely the characteristic difference between the sample noise reduction characteristic and the sample original characteristic is a nonlinear amplification result of the image difference between the sample noise reduction image and the sample original image, so that compared with the method of directly adopting the image difference between the sample noise reduction image and the sample original image to construct a loss function, the task model can more accurately learn the counteracting result of the anti-disturbance on the malicious disturbance; in addition, the matching degree of the anti-disturbance model and the task model can be improved, and therefore the accuracy of task processing on the noise-reduced image in the follow-up process is improved.

According to the anti-disturbance model generation method and device, the anti-disturbance image is generated through the anti-disturbance model to offset malicious noise artificially added in the anti-disturbance sample, the anti-disturbance model is trained according to the noise reduction result and the feature extraction result of the original image, the training effect of the anti-disturbance model is effectively improved, and the defense capability of the anti-disturbance model on the anti-disturbance sample is guaranteed under the condition of no artificial supervision.

In an alternative embodiment, the anti-disturbance model comprises at least two anti-disturbance units; and each anti-disturbance unit is respectively associated with different feature extraction units in the target detection model and is used for training the anti-disturbance units by adopting the features extracted by the associated feature extraction units.

The anti-disturbance unit is used for carrying out feature extraction on the malicious disturbance and generating the anti-disturbance opposite to the malicious disturbance. The target detection model is provided with a feature extraction unit which is used for extracting features of the sample noise reduction image and the sample original image, judging a counteracting result of malicious disturbance and anti-disturbance in the sample noise reduction image according to the feature extraction result, wherein each feature extraction unit in the target detection model is associated with one anti-disturbance unit in the anti-disturbance model, and the feature extraction unit and the anti-disturbance unit are respectively used for extracting features and generating anti-disturbance of the same type of malicious disturbance.

Specifically, the malicious disturbance may exist in multiple dimensions and angles, and a single anti-disturbance unit may only have a good recognition and anti-disturbance generation effect on some types of malicious disturbances, but cannot meet the processing requirements on various types of malicious disturbances. At least two anti-disturbance units are deployed in the anti-disturbance model and are respectively used as noise reducers for different types of malicious disturbances, so that the anti-disturbance model can effectively process various types of malicious disturbances. Meanwhile, a target detection model serving as a task model has a feature extraction unit associated with each anti-disturbance unit, and the feature extraction unit can extract malicious disturbance features and anti-disturbance features corresponding to the associated anti-disturbance units, so that the anti-disturbance generation effect of the associated anti-disturbance units is determined according to the malicious disturbance features and/or the anti-disturbance features in the extracted sample noise reduction images and sample original images, and the anti-disturbance units are fed back and trained based on the anti-disturbance generation effect. A plurality of anti-disturbance units are deployed in the anti-disturbance model, and each anti-disturbance unit is associated with a feature extraction unit in the target detection model, so that the anti-disturbance generation effect of the anti-disturbance model on various types of malicious disturbance is improved, and the applicability and the conformity of the anti-disturbance model and the target detection model are ensured.

Fig. 2A is a schematic diagram of a training method of an image model according to yet another embodiment of the present disclosure, which is optimized based on the foregoing embodiments. Referring to fig. 2A, the method includes:

s210, inputting the confrontation sample into at least two anti-disturbance units in the anti-disturbance model to obtain at least two sample anti-disturbance images, and denoising the confrontation sample by adopting the at least two sample anti-disturbance images to obtain at least two sample denoising images.

S221, respectively inputting the ith sample noise reduction image and the sample original image into the target detection model to obtain the ith sample noise reduction feature and the ith sample original feature output by the jth feature extraction unit in the target detection model. S222, training the ith anti-disturbance unit by adopting the noise reduction characteristic of the ith sample and the original characteristic of the ith sample;

the method comprises the steps that an ith sample noise reduction image is obtained by carrying out noise reduction on a countermeasure sample by adopting an ith sample anti-disturbance image; i and j are natural numbers, and the ith anti-disturbance unit is associated with the jth feature extraction unit; the ith sample anti-disturbance image is obtained by an ith anti-disturbance unit in the anti-disturbance model; .

The original image characteristics of the sample original characteristic characterization image are not injected with malicious disturbance, and the sample noise reduction characteristics characterize the image characteristics of the image after the malicious disturbance in the image is offset by the anti-disturbance.

Specifically, according to a first feature extraction unit j associated with an ith anti-disturbance unit i for generating the anti-disturbance, an ith sample noise reduction image i and a sample original image i are respectively input into a target detection model, a sample noise reduction feature i and a sample original feature i of the sample noise reduction image i and the sample original image i are extracted by using the feature extraction unit j in the target detection model, and parameters to be trained in the ith anti-disturbance unit i are updated according to a comparison result of the sample original feature i and the sample noise reduction feature i. The method comprises the steps of obtaining a characteristic extraction result of a sample original characteristic and a sample noise reduction characteristic through a correlation characteristic extraction unit, and updating a parameter to be trained of a correlation anti-disturbance unit based on the characteristic extraction result so as to improve the targeted training effect of each anti-disturbance unit.

Illustratively, in the case where the anti-disturbance units a1 and a2 exist in the anti-disturbance model, three feature extraction units B1, B2 and B3 exist in the target detection model, the feature extraction unit B1 is associated with the anti-disturbance unit a1, and the feature extraction unit B2 is associated with the anti-disturbance unit a2, the loss function of the anti-disturbance unit a1 is constructed by the features extracted by the feature extraction unit B1, and the loss function of the anti-disturbance unit a2 is constructed by the features extracted by the feature extraction unit B2. Because each feature extraction unit can extract features of the noise-reduced sample image and the original sample image to different degrees, namely, nonlinearly amplify the image to different degrees, each anti-disturbance unit is used for identifying disturbances with different scales and different types, and the training effect of each anti-disturbance unit can be improved.

S231, fusing the sample anti-disturbance images generated by at least two anti-disturbance units in the anti-disturbance model through a fusion model to obtain a sample fusion image;

s232, denoising the confrontation sample by adopting the sample fusion image to obtain a sample denoising result;

s233, respectively inputting the sample denoising result and the sample original image into the target detection model to obtain a sample denoising detection value and a sample original detection value;

and S234, training the fusion model according to the sample denoising detection value and the sample original detection value.

The network structure of the fusion model includes an anti-disturbance fusion layer (which may be a full connection layer, for example) for fusing the anti-disturbance images generated by the anti-disturbance units, so as to generate a sample fusion image including multiple anti-disturbances.

Specifically, besides training the anti-disturbance unit, training is also needed for a fusion model fusing a plurality of sample anti-disturbance images, so as to improve the fusion capability of the fusion model on the plurality of sample anti-disturbance images. And (3) counteracting and denoising the malicious disturbance in the resisting sample by adopting a sample fusion image obtained by fusing at least two sample anti-disturbance images by a fusion model to obtain a sample denoising result. And respectively inputting the sample denoising result and the sample original image into the target detection model to obtain a sample denoising detection value and a sample original detection value, taking the difference between the sample denoising detection value and the sample original detection value as a loss function in the fusion model training process, and updating the parameters to be trained in the fusion layer based on the loss function in the fusion model training process. The fusion model is trained by obtaining the sample noise reduction detection value and the sample original detection value, so that the loss function in the training process of the fusion model is effectively reduced, and the sample anti-disturbance image fusion effect of the fusion model is improved.

Optionally, the inverse perturbation unit includes at least two feature extraction layers and at least two inverse perturbation layers; the input of the kth reverse disturbance layer is the output of the kth feature extraction layer and the output of the (k + 1) th reverse disturbance layer; k is a natural number.

Specifically, the reverse disturbance layer may be connected to the feature extraction layer through a U-shaped tube structure, and is configured to generate reverse disturbance according to the malicious disturbance feature extracted by the feature extraction layer. The feature extraction layer is used as an upper path for extracting features and is composed of five basic convolution modules, each basic convolution module is composed of two batches of normalization convolution layers, and each batch of normalization convolution layers can be composed of one convolution layer, one normalization layer and one activation layer. The reverse disturbance layer is used as a lower path for generating reverse noise according to the extracted characteristics and consists of four similar basic convolution modules, however, a down-sampling layer using a bilinear interpolation method is added in front of each module, and finally after passing through the lower path, the reverse disturbance layer can output reverse disturbance which can be used for offsetting malicious disturbance noise on an anti-sample. In addition, at least two sets of reverse disturbance layers and feature extraction layers are arranged in the reverse disturbance unit and are respectively used for extracting image features of different depth levels and generating corresponding reverse disturbance. When the inverse perturbation of the next depth level is generated, in addition to the image feature of the current depth level, the inverse perturbation generated by the previous depth level is also referred to, that is, the input of the kth inverse perturbation layer is the output of the kth feature extraction layer and the output of the (k + 1) th inverse perturbation layer. The anti-disturbance is generated by combining the anti-disturbance of the previous level and the image characteristics of the current level, so that the anti-disturbance effectively covers malicious disturbance in the countermeasure sample, and the counteracting effect of the anti-disturbance on the malicious disturbance is improved.

For example, fig. 2B is a schematic structural diagram of an anti-disturbance unit provided in the embodiment of the present disclosure. Wherein, X1 on the graph represents an input confrontation sample (disturbance image), malicious noise extraction is performed through a series of feature extraction layers in the anti-disturbance unit, the anti-disturbance layer generates anti-disturbance, the final output of the anti-disturbance unit is-X1, and the model represents the learned anti-noise-anti-disturbance. The inverse perturbation will then be added to the input challenge samples resulting in a noise reduced input image, denoted X2. X represents an original image without malicious disturbance, X2 and X after noise reduction processing are simultaneously input into a target detection model, and the difference between the outputs of X2 and X after prediction of the target detection model is used as a loss function in the training process of the noise reducer. It should be noted that, the K + N (N is a natural number greater than 1) th group of feature inverse perturbation layers and feature extraction layers are shown here only for the purpose of illustrating that the inverse perturbation unit in the present disclosure may not be limited to two groups of inverse perturbation layers and feature extraction layers, and if there are more groups of inverse perturbation layers and feature extraction layers, the same structural connection manner may be used.

According to the embodiment of the invention, the associated feature extraction unit is used for obtaining the feature extraction results of the original features of the sample and the noise reduction features of the sample, and the parameters to be trained of the associated anti-disturbance units are updated based on the feature extraction results, so that the targeted training effect of each anti-disturbance unit is improved; the fusion model is trained by obtaining the sample noise reduction detection value and the sample original detection value, so that the loss function in the training process of the fusion model is effectively reduced, and the sample anti-disturbance image fusion effect of the fusion model is improved.

Fig. 3 is a flowchart of an image denoising method according to yet another embodiment of the present disclosure, which may be applied to a case where an image is denoised by using an anti-disturbance model, and the method may be performed by an image denoising device, which may be implemented in software and/or hardware. The device can be configured in an electronic device with corresponding data processing capability, and the method specifically comprises the following steps:

s310, inputting the image to be processed into the trained anti-disturbance model to obtain a target anti-disturbance image;

and S320, denoising the image to be processed by adopting the target anti-disturbance image to obtain a target denoising image.

Specifically, an image to be processed of the task model is input into the anti-disturbance model, the anti-disturbance model can be obtained by training through the image model training method in the disclosed embodiment, after the anti-disturbance image output by the anti-disturbance model is obtained, the anti-disturbance image and the image to be processed are fused to achieve noise reduction of the image to be processed, and then the target noise reduction image is obtained.

According to the embodiment of the disclosure, the anti-disturbance image is generated through the anti-disturbance model, so that the malicious noise in the image to be processed is counteracted, the defense capability for the countermeasure sample under the condition of no manual supervision is improved, the target detection model does not need to be reconstructed or trained, and the workload is reduced.

For example, fig. 3B is a schematic diagram of detection of a challenge sample, fig. 3C is a schematic diagram of a noise-reduced challenge sample, and fig. 3D is a schematic diagram of anti-perturbation of the challenge sample. Referring to fig. 3B-3D, denoising a countersample (as shown in fig. 3B) with a de-perturbation (as shown in fig. 3D) may result in a sample denoised image (as shown in fig. 3C). The countercheck sample is injected with the malicious disturbance, so that the target detection result of the countercheck sample falsely detects the cattle as the horse, and the noise reduction method provided by the disclosure can be used for inputting the noise-reduced image into the target detection model after the malicious disturbance in the countercheck sample is counteracted through the anti-disturbance, so that the target detection model can correctly identify the cattle. Therefore, through the anti-disturbance generated by the anti-disturbance model, the malicious disturbance noise added in the countermeasure sample can be eliminated, so that the countermeasure sample subjected to noise reduction can be normally processed by the target detection model, the countermeasure sample is not mistakenly identified by the target detection model, and the countermeasure sample does not need to be discarded.

Fig. 4 is a schematic diagram of an image denoising method according to another embodiment of the present disclosure, which is optimized based on the foregoing embodiments. Referring to fig. 4, the method includes:

s410, respectively inputting the images to be processed into at least two anti-disturbance units in the anti-disturbance model to obtain at least two target anti-disturbance images.

Specifically, when a plurality of malicious disturbance noises exist in the image to be processed, the anti-disturbance model generates corresponding anti-disturbance to each malicious disturbance noise, so that a plurality of target anti-disturbance images are obtained.

And S421, fusing the at least two target anti-disturbance images through the trained fusion model to obtain a target fusion image.

S422, denoising the image to be processed by adopting the target fusion image to obtain the target denoising image.

Specifically, a target fusion image is obtained by fusing a plurality of target anti-disturbance images through a fusion model, each malicious disturbance noise in the image to be processed is offset based on the target fusion image, the plurality of target anti-disturbance images are not required to be subjected to malicious disturbance offset noise reduction with the image to be processed one by one, and the noise reduction efficiency of the image to be processed under the complex malicious disturbance condition is improved.

According to the method and the device, the noise of the image to be processed is reduced through the target fusion image based on the target anti-disturbance images, and the noise reduction efficiency of the image to be processed in the complex malicious disturbance is improved.

Fig. 5 is a schematic diagram of an image model training apparatus according to yet another embodiment of the present disclosure, where the embodiment of the present disclosure is applied to a case where an anti-disturbance model is trained using a challenge sample, and the apparatus is configured in an electronic device with corresponding data processing capability, and can implement the image model training method according to any embodiment of the present disclosure.

The sample denoising module 510 is configured to input the countermeasure sample into an inverse perturbation model to obtain a sample inverse perturbation image, and perform denoising on the countermeasure sample by using the sample inverse perturbation image to obtain a sample denoising image;

and the inverse disturbance model training module 520 is configured to perform feature extraction on the sample noise reduction image and the sample original image respectively, and train the inverse disturbance model according to a feature extraction result.

The device and the module can execute the training method of the image model provided by any embodiment of the disclosure, and have corresponding functional modules and beneficial effects of the execution method.

Optionally, the anti-disturbance model includes at least two anti-disturbance units; and each anti-disturbance unit is respectively associated with different feature extraction units in the target detection model and is used for training the anti-disturbance units by adopting the features extracted by the associated feature extraction units.

Optionally, the inverse perturbation model training module 520 includes:

the characteristic obtaining unit is used for respectively inputting the ith sample noise reduction image and the sample original image into the target detection model to obtain the ith sample noise reduction characteristic and the ith sample original characteristic output by the jth characteristic extracting unit in the target detection model; the ith sample noise reduction image is obtained by reducing the noise of the confrontation sample by adopting an ith sample anti-disturbance image; the ith sample anti-disturbance image is obtained by an ith anti-disturbance unit in the anti-disturbance model; the anti-disturbance training unit is used for training the ith anti-disturbance unit by adopting the noise reduction characteristic of the ith sample and the original characteristic of the ith sample; wherein i and j are natural numbers, and the ith anti-disturbance unit is associated with the jth feature extraction unit.

Optionally, the apparatus further comprises:

the image fusion module is used for fusing the sample anti-disturbance images generated by at least two anti-disturbance units in the anti-disturbance model through a fusion model to obtain a sample fusion image;

the de-noising result acquisition module is used for de-noising the confrontation sample by adopting the sample fusion image to obtain a sample de-noising result;

a detection value acquisition module, configured to input the sample denoising result and the sample original image into the target detection model respectively, so as to obtain a sample denoising detection value and a sample original detection value;

and the fusion model training module is used for training the fusion model according to the sample denoising detection value and the sample original detection value.

The further described devices, modules and units can execute the training method of the image model provided by any embodiment of the disclosure, and have corresponding functional modules and beneficial effects of the execution method.

Fig. 6 is a schematic diagram of an image denoising apparatus according to still another embodiment of the present disclosure, where the embodiment of the present disclosure is applicable to a case of performing image denoising using an anti-disturbance model, and the apparatus is configured in an electronic device with corresponding data processing capability, and can implement the image denoising method according to any embodiment of the present disclosure.

The anti-disturbance image obtaining module 610 is configured to input the image to be processed into the trained anti-disturbance model to obtain a target anti-disturbance image;

and a noise-reduced image obtaining module 620, configured to perform noise reduction on the image to be processed by using the target anti-disturbance image to obtain a target noise-reduced image.

The device and the module can execute the image noise reduction method provided by any embodiment of the disclosure, and have corresponding functional modules and beneficial effects of the execution method.

Optionally, the anti-disturbance image obtaining module 610 includes:

the anti-disturbance image acquisition unit is used for respectively inputting the images to be processed into at least two anti-disturbance units in the anti-disturbance model to obtain at least two target anti-disturbance images;

the noise-reduced image obtaining module 620 includes:

the anti-disturbance image fusion unit is used for fusing the at least two target anti-disturbance images through the trained fusion model to obtain a target fusion image;

and the noise reduction image acquisition unit is used for carrying out noise reduction on the image to be processed by adopting the target fusion image to obtain the target noise reduction image.

The device, module and unit described further above can execute the image noise reduction method provided by any embodiment of the present disclosure, and have corresponding functional modules and beneficial effects of the execution method.

The present disclosure also provides an electronic device, a readable storage medium, and a computer program product according to embodiments of the present disclosure.

FIG. 7 illustrates a schematic block diagram of an example electronic device 700 that can be used to implement embodiments of the present disclosure. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the disclosure described and/or claimed herein.

As shown in fig. 7, the device 700 comprises a computing unit 701 which may perform various suitable actions and processes according to a computer program stored in a Read Only Memory (ROM)702 or a computer program loaded from a storage unit 708 into a Random Access Memory (RAM) 703. In the RAM703, various programs and data required for the operation of the device 700 can also be stored. The computing unit 701, the ROM 702, and the RAM703 are connected to each other by a bus 704. An input/output (I/O) interface 705 is also connected to bus 704.

Various components in the device 700 are connected to the I/O interface 705, including: an input unit 706 such as a keyboard, a mouse, or the like; an output unit 707 such as various types of displays, speakers, and the like; a storage unit 708 such as a magnetic disk, optical disk, or the like; and a communication unit 709 such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 709 allows the device 700 to exchange information/data with other devices via a computer network, such as the internet, and/or various telecommunication networks.

Computing unit 701 may be a variety of general purpose and/or special purpose processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various dedicated Artificial Intelligence (AI) computing chips, various computing units running machine learning model algorithms, a Digital Signal Processor (DSP), and any suitable processor, controller, microcontroller, and so forth. The calculation unit 701 performs the respective methods and processes described above, such as training of an image model or an image noise reduction method. For example, in some embodiments, the training of the image model or the image denoising method may be implemented as a computer software program tangibly embodied in a machine-readable medium, such as storage unit 708. In some embodiments, part or all of a computer program may be loaded onto and/or installed onto device 700 via ROM 702 and/or communications unit 709. When the computer program is loaded into the RAM703 and executed by the computing unit 701, one or more steps of the training of the image model or the image denoising method described above may be performed. Alternatively, in other embodiments, the computing unit 701 may be configured by any other suitable means (e.g., by means of firmware) to perform a training of an image model or an image denoising method.

Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuitry, Field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs), system on a chip (SOCs), Complex Programmable Logic Devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, receiving data and instructions from, and transmitting data and instructions to, a storage system, at least one input device, and at least one output device.

Program code for implementing the methods of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the program codes, when executed by the processor or controller, cause the functions/operations specified in the flowchart and/or block diagram to be performed. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package partly on the machine and partly on a remote machine or entirely on the remote machine or server.

In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which a user may provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic, speech, or tactile input.

The systems and techniques described here can be implemented in a computing system that includes a back-end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), Wide Area Networks (WANs), blockchain networks, and the internet.

The computer system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also called a cloud computing server or a cloud host, and is a host product in a cloud computing service system, so that the defects of high management difficulty and weak service expansibility in the traditional physical host and VPS service are overcome.

It should be understood that various forms of the flows shown above may be used, with steps reordered, added, or deleted. For example, the steps described in the present disclosure may be executed in parallel, sequentially, or in different orders, as long as the desired results of the technical solutions disclosed in the present disclosure can be achieved, and the present disclosure is not limited herein.

The above detailed description should not be construed as limiting the scope of the disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may be made in accordance with design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present disclosure should be included in the protection scope of the present disclosure.

Claims

1. A method of training an image model, comprising:

2. The method of claim 1, wherein the anti-perturbation model comprises at least two anti-perturbation cells; and each anti-disturbance unit is respectively associated with different feature extraction units in the target detection model and is used for training the anti-disturbance units by adopting the features extracted by the associated feature extraction units.

3. The method of claim 2, wherein the performing feature extraction on the sample noise-reduced image and the sample original image respectively, and training the anti-disturbance model according to the feature extraction result comprises:

respectively inputting the ith sample noise reduction image and the sample original image into a target detection model to obtain the ith sample noise reduction characteristic and the ith sample original characteristic output by a jth characteristic extraction unit in the target detection model; the ith sample noise reduction image is obtained by reducing the noise of the countermeasure sample by adopting an ith sample anti-disturbance image; the ith sample anti-disturbance image is obtained by an ith anti-disturbance unit in the anti-disturbance model;

training the ith anti-disturbance unit by adopting the ith sample noise reduction characteristic and the ith sample original characteristic;

wherein i and j are natural numbers, and the ith anti-disturbance unit is associated with the jth feature extraction unit.

4. The method of claim 2, further comprising:

fusing sample anti-disturbance images generated by at least two anti-disturbance units in the anti-disturbance model through a fusion model to obtain a sample fusion image;

denoising the confrontation sample by adopting the sample fusion image to obtain a sample denoising result;

respectively inputting the sample denoising result and the sample original image into the target detection model to obtain a sample denoising detection value and a sample original detection value;

and training the fusion model according to the sample noise reduction detection value and the sample original detection value.

5. The method of claim 4, wherein the inverse perturbation unit comprises at least two feature extraction layers and at least two inverse perturbation layers;

the input of the kth reverse disturbance layer is the output of the kth feature extraction layer and the output of the (k + 1) th reverse disturbance layer; k is a natural number.

6. An image denoising method, comprising:

7. The method of claim 6, wherein the inputting the image to be processed into the trained anti-disturbance model to obtain a target anti-disturbance image comprises:

respectively inputting the images to be processed into at least two anti-disturbance units in the anti-disturbance model to obtain at least two target anti-disturbance images;

the denoising of the image to be processed by adopting the target anti-disturbance image to obtain a target denoising image comprises:

fusing the at least two target anti-disturbance images through the trained fusion model to obtain a target fusion image;

and denoising the image to be processed by adopting the target fusion image to obtain the target denoising image.

8. An apparatus for training an image model, comprising:

9. The apparatus of claim 8, wherein the anti-perturbation model comprises at least two anti-perturbation units; and each anti-disturbance unit is respectively associated with different feature extraction units in the target detection model and is used for training the anti-disturbance units by adopting the features extracted by the associated feature extraction units.

10. The apparatus of claim 9, wherein the inverse perturbation model training module comprises:

the characteristic obtaining unit is used for respectively inputting the ith sample noise reduction image and the sample original image into the target detection model to obtain the ith sample noise reduction characteristic and the ith sample original characteristic output by the jth characteristic extracting unit in the target detection model; the ith sample noise reduction image is obtained by reducing the noise of the confrontation sample by adopting an ith sample anti-disturbance image; the ith sample anti-disturbance image is obtained by an ith anti-disturbance unit in the anti-disturbance model;

the anti-disturbance training unit is used for training the ith anti-disturbance unit by adopting the noise reduction characteristic of the ith sample and the original characteristic of the ith sample;

11. The apparatus of claim 9, wherein the apparatus further comprises:

12. The apparatus according to claim 11, wherein the inverse perturbation unit comprises at least two feature extraction layers and at least two inverse perturbation layers;

13. An image noise reduction apparatus comprising:

14. The apparatus of claim 13, wherein the anti-disturbance image acquisition module comprises:

the noise reduction image acquisition module includes:

15. An electronic device, comprising:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,

the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-7.

16. A non-transitory computer readable storage medium having stored thereon computer instructions for causing the computer to perform the method of any one of claims 1-7.

17. A computer program product comprising a computer program which, when executed by a processor, implements the method according to any one of claims 1-7.