CN117788906A - A large model generated image identification method and system - Google Patents

Abstract

The invention provides a large model generation image identification method and a large model generation image identification system. The method comprises the following steps: inputting the generated image into a first processing module based on residual filtering to obtain original characteristics; inputting the original features into a second processing module based on a self-attention mechanism and a residual error structure to obtain classification features; and inputting the classification characteristic into a classification network, and outputting a result of only true or false. The scheme provided by the invention solves the problems that the prior art cannot utilize shallow texture information of an input picture, and the loss function is simple and cannot dynamically change along with input data.

Description

A large model generated image identification method and system

Technical field

The invention belongs to the field of image identification, and in particular relates to a large model generated image identification method and system.

Background technique

With the development of artificial intelligence technology, large models have gradually matured and gradually played their respective roles in people's lives. Among them, AI-generated content (AIGC) is a popular large model direction. Large-scale image generation models that use a large number of diffusion models have entered people's field of vision. For example, large models such as StableDiffusion, Dreambooth, and Midjourney only require the user to enter a prompt word (prompt), and the large model service that allows the large model to automatically generate relevant images also allows various people who are not good at drawing to map their thoughts to on the image.

However, generating a large image model itself is also a double-sided sword, which will bring many disadvantages. For example, in general commercial activities, investors or buyers often want to purchase images designed and drawn by the artist himself rather than images generated by large models. Random use of large models to generate images will also lead to copyright disputes. For example, a large model that generates images can produce very detailed paintings based on prompt words, which may be used by people with evil intentions to spread rumors, causing incalculable consequences.

At this time, all sectors of society are urgently in need of technology that can distinguish between images generated by large models and real-life images.

Although previous studies have done some research on how to identify real images and fake images, in the field of identifying computer-generated images and real images, existing research tends to focus more on generative adversarial networks (GAN) or variational autoencoding. The method of distinguishing between images generated by traditional neural networks such as GAN and VAE and real-life images is proposed from various aspects such as spatial domain and frequency domain.

However, there is a big gap between the principles of large image generation models that use diffusion models to generate images and previous image generation networks. It is difficult to directly apply existing technology to the task of forgery identification of images generated by large models. Some scholars use existing image identification technology to identify large model images and real images. As a result, the model performance is very unsatisfactory and cannot meet people's current needs and expectations. With the rapid development of large models for diffusion image generation, it will become increasingly difficult for existing identification technologies to distinguish images generated by large models from real images.

current technology

DIRE technology comes from the paper DIRE for Diffusion-Generated Image Detection, which is the abbreviation of DIffusion Reconstruction Error. DIRE measures the error between the input image and its reconstruction through a pre-trained diffusion model. The authors of the paper found that images generated by the diffusion model are easier to reconstruct by the pre-trained diffusion model than real images, which are difficult to reconstruct due to various complex factors in reality. The input image is reconstructed through DDIM, and then the difference between the reconstructed image and the original image is calculated. Finally, the difference is used as a feature for binary classification to determine whether the image is a picture forged by a large model.

Shortcomings of existing technology

The first disadvantage of the existing method DIRE is that using the original image and the reconstructed image to make a difference will lose the shallow texture features of the original image and fail to extract enough information from the original image.

The second shortcoming of the existing method DIRE is that it does not pay attention to the relationship between pixels in the image generated by the large model.

The third shortcoming of the existing method DIRE is that the loss function is too simple and cannot dynamically adjust the learning step according to the different input data.

Summary of the invention

In order to solve the above technical problems, the present invention proposes a technical solution of a large model generated image identification method to solve the above technical problems.

A first aspect of the present invention discloses a large model generated image identification method. The method includes:

Step S1: Input the generated image into the first processing module based on residual filtering to obtain original features;

Step S2: Input the original features into the second processing module based on the self-attention mechanism and residual structure to obtain classification features;

Step S3: input the classification features into a binary classification network, and output only "true" or "false" results.

According to the method of the first aspect of the present invention, in step S1, the generated image is input into the first processing module based on residual filtering, and the method of obtaining original features includes:

The generated images are input into the residual filter and convolution kernel respectively, then the processing results of the residual filter and convolution kernel are merged, and finally the merged results are input into the first convolution pooling layer to obtain the original features.

According to the method of the first aspect of the present invention, in step S1, there are seventeen residual filters; there are eight convolution kernels; the values of the seventeen residual filters are fixed and will not change. Changed during learning; and the parameters of the eight convolution kernels are learned during training.

According to the method of the first aspect of the present invention, in step S1, the first convolution pooling layer is a convolution pooling layer with a convolution layer and a pooling layer applying a residual mechanism.

According to the method of the first aspect of the present invention, in the step S2, the method of inputting the original features into the second processing module based on the self-attention mechanism and the residual structure to obtain the classification features includes:

The original features are input into the second convolutional pooling layer to obtain the processing features, the processing features are input into the self-attention operation, and the results of the self-attention operation and the processing features are numerically added to obtain the classification features.

According to the method of the first aspect of the present invention, in the step S2, the V of the attention operation is the processing feature; through the calculation of Q and K, the softmax layer is used to obtain the weight assigned to the V , and then the weighted average is obtained to obtain the shallow texture features captured by the self-attention mechanism.

According to the method of the first aspect of the present invention, in step S3, a high-dimensional spherical boundary objective function is used in the two-classification network to optimize classification.

The second aspect of the present invention discloses a large model generated image identification system, the system comprising:

The first processing module is configured to input the generated image into the first processing module based on residual filtering to obtain original features;

The second processing module is configured to input the original features into the second processing module based on the self-attention mechanism and the residual structure to obtain the classification features;

The third processing module is configured to input the classification features into a two-classification network and output only a "true" or "false" result.

A third aspect of the invention discloses an electronic device. The electronic device includes a memory and a processor. The memory stores a computer program. When the processor executes the computer program, it implements the steps in the large model generation image identification method of any one of the first aspects of the present disclosure.

A fourth aspect of the present invention discloses a computer-readable storage medium. A computer program is stored on the computer-readable storage medium. When the computer program is executed by the processor, the steps in the large model generation image identification method of any one of the first aspects of the present disclosure are implemented.

In summary, the solution proposed by the present invention includes a network structure of a self-attention mechanism and a residual structure. Combined with the self-attention mechanism, it enhances the network's ability to extract and analyze shallow texture features invisible to the naked eye, and supplements the lost features through the residual. The information further enriches the basis for identification. In the classification based on the high-dimensional spherical boundary objective function, the training method of increasing the intra-group similarity training stride can help the model pay more attention to the intra-group similarity threshold. This application solves the problem that the existing technology cannot utilize the shallow texture information of the input image, and the loss function is simple and cannot dynamically change with the input data.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

Figure 1 is a flow chart of a large model generation image identification method according to an embodiment of the present invention;

Figure 2 is a module flow chart according to an embodiment of the present invention;

Figure 3 is a diagram of a residual filtering module according to an embodiment of the present invention;

Figure 4 shows the specific initialization values of seventeen residual filters according to the embodiment of the present invention;

Figure 5 is a self-attention module diagram according to an embodiment of the present invention;

Figure 6 is a structural diagram of a large model generated image identification system according to an embodiment of the present invention;

Figure 7 is a structural diagram of an electronic device according to an embodiment of the present invention.

Detailed ways

In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

A first aspect of the present invention discloses a large model generated image identification method. Figure 1 is a flow chart of a large model generated image identification method according to an embodiment of the present invention. As shown in Figure 1, the method includes:

Step S1: Input the generated image into the first processing module based on residual filtering to obtain original features;

Step S2: inputting the original features into a second processing module based on a self-attention mechanism and a residual structure to obtain classification features;

Step S3: Input the classification features into the binary classification network, and output only "true" or "false" results.

In step S1, as shown in Figure 2, the generated image is input into the first processing module based on residual filtering (the residual filtering module in Figure 2) to obtain the original features.

In some embodiments, in step S1, the generated image is input into the first processing module based on residual filtering, and the method for obtaining original features includes:

As shown in Figure 3, the generated images are input into the residual filter and convolution kernel respectively, then the processing results of the residual filter and convolution kernel are merged, and finally the merged results are input into the first convolution pooling layer , get the original features.

There are seventeen residual filters; there are eight convolution kernels; the values of the seventeen residual filters are fixed and will not be changed during learning; and the parameters of the eight convolution kernels are during training. Learned.

The first convolutional pooling layer is a convolutional pooling layer with a residual mechanism applied to the convolutional layer and the pooling layer.

Specifically, the convolutional pooling layer here includes a 3*3 convolutional layer, a regularization layer, a ReLU layer, and a maximum pooling layer.

As shown in Figure 4, the specific initialization values of the seventeen residual filters are as follows. These filters can efficiently extract the residual information of the image.

In step S2, as shown in Figure 2, the original features are input into the second processing module (self-attention module in Figure 2) based on the self-attention mechanism and residual structure to obtain classification features. Combined with the self-attention mechanism, the network's ability to extract and analyze shallow texture features invisible to the naked eye is enhanced, and the residual feature information is supplemented to further enrich the basis for identification.

In some embodiments, in step S2, the method of inputting the original features into a second processing module based on a self-attention mechanism and a residual structure to obtain classification features includes:

As shown in Figure 5, the original features are input into the second convolutional pooling layer to obtain the processing features, the processing features are input into the self-attention operation, and the results of the self-attention operation are numerically added to the processing features. , get the classification features. Through the residual mechanism, the information lost due to operation is supplemented and the identification effect is enhanced.

The V of the attention operation is the processed feature; through the calculation of Q and K, the softmax layer is used to obtain the weight assigned to the V, and then the weighted average is used to obtain the shallow texture feature captured by the self-attention mechanism.

Specifically, the convolutional pooling layer includes a 3*3 convolution layer, a regularization layer, a ReLU layer, a 3*3 convolution layer, a regularization layer, a ReLU layer, and a maximum pooling layer in the following order layer.

These processed features use a self-attention mechanism to capture the correlation of shallow texture features at the pixel level.

Before performing self-attention, the data needs to be adjusted so that it is in the form of (number of pixels * number of information channels). Then, the self-attention operation is performed on the data.

In step S3, the classification features are input into the binary classification network (spherical loss classification module in Figure 2), and only results of "true" or "false" are output.

In some embodiments, in step S3, a high-dimensional spherical boundary objective function is used to optimize classification in the two-classification network.

Specifically, the image counterfeiting problem can be regarded as a binary classification problem, and the output result is only "true" or "false". In order to improve the accuracy and effectiveness of classification, the present invention introduces a high-dimensional spherical boundary objective function in the classification layer. The high-dimensional spherical boundary objective function mainly revolves around the similarity step within the group and the similarity step between the groups. This invention pays more attention to the similarity within the group and designs a special weight w for the similarity within the group.

The high-dimensional spherical boundary objective function is an objective function that adaptively changes the optimization stride during the training process. Its stride update rules are as follows:

Initially, separate thresholds are set for intra-group similarity and inter-group similarity.

When a sample is input, the intra-group similarity and inter-group similarity of the current sample are first calculated. Then calculate the within-group similarity stride and the between-group similarity stride. The intra-group similarity step is the product of the weight w and the difference between the intra-group similarity threshold minus the intra-group similarity calculation value. The similarity step between groups is the difference between the calculated similarity value between groups minus the similarity threshold between groups. If these two strides are less than 0, then set them to 0, and always keep these two strides non-negative.

When calculating the metrics by group, the similarity loss within the group and the similarity loss between the groups should be multiplied by their respective strides before statistics are calculated. In this way, different strides can be updated for different data when solving the gradient descent problem.

After training the model using a high-dimensional spherical boundary objective function, the model can perform forgery identification on the input image to identify whether it was generated by a large model.

In summary, the solution proposed by this invention, the network structure of the self-attention mechanism and the residual structure, combines the self-attention mechanism to enhance the network's ability to extract and analyze shallow texture features invisible to the naked eye, and supplement the lost feature information through the residual Further enrich the basis for identification.

In the classification based on the high-dimensional spherical boundary objective function, the training method of increasing the intra-group similarity training stride can help the model pay more attention to the intra-group similarity threshold. It solves the problem that the existing technology cannot utilize the shallow texture information of the input image, and the loss function is simple and cannot dynamically change with the input data.

A second aspect of the present invention discloses a large model generation image identification system. Figure 6 is a structural diagram of a large model generated image identification system according to an embodiment of the present invention; as shown in Figure 6, the system 100 includes:

The first processing module 101 is configured to input the generated image into the first processing module based on residual filtering to obtain the original features;

The second processing module 102 is configured to input the original features into a second processing module based on a self-attention mechanism and a residual structure to obtain classification features;

The third processing module 103 is configured to input the classification features into a two-classification network and output only “true” or “false” results.

According to the system of the second aspect of the present invention, the first processing module 101 is specifically configured to input the generated image to the first processing module based on residual filtering, and the method for obtaining the original features includes:

As shown in Figure 3, the generated image is input into the residual filter and the convolution kernel respectively, and then the processing results of the residual filter and the convolution kernel are merged, and finally the merged result is input into the first convolution pooling layer to obtain the original features.

There are seventeen residual filters; there are eight convolution kernels; the values of the seventeen residual filters are fixed and will not be changed during learning; and the parameters of the eight convolution kernels are during training. Learned.

The first convolutional pooling layer is a convolutional pooling layer with a convolutional layer and a pooling layer applying a residual mechanism.

Specifically, the convolution pooling layer here includes a 3*3 convolution layer, a regularization layer, a ReLU layer, and a maximum pooling layer.

As shown in Figure 4, the specific initialization values of the seventeen residual filters are as follows. These filters can efficiently extract the residual information of the image.

According to the system of the second aspect of the present invention, the second processing module 102 is specifically configured to input the original features to the second processing module based on the self-attention mechanism and the residual structure, and the method of obtaining the classification features includes: :

As shown in Figure 5, the original features are input into the second convolutional pooling layer to obtain processed features, the processed features are input into the self-attention operation, and the result of the self-attention operation is numerically added to the processed features to obtain classification features. Through the residual mechanism, the information lost due to the operation is supplemented to enhance the identification effect.

The V of the attention operation is the processing feature; through the calculation of Q and K, the softmax layer is used to obtain the weight assigned to the V, and then the weighted average is obtained to obtain the shallow texture features captured by the self-attention mechanism .

Specifically, the convolutional pooling layer includes a 3*3 convolution layer, a regularization layer, a ReLU layer, a 3*3 convolution layer, a regularization layer, a ReLU layer, and a maximum pooling layer in the following order layer.

These processed features use a self-attention mechanism to capture the correlation of shallow texture features at the pixel level.

Before performing self-attention, the form of the data needs to be adjusted so that the data forms the form of (number of pixels * number of information channels). Then, self-attention operations are performed on the data.

According to the system of the second aspect of the present invention, the third processing module 103 is specifically configured to optimize classification using a high-dimensional spherical boundary objective function in the two-classification network.

Specifically, the image counterfeiting problem can be regarded as a binary classification problem, and the output result is only "true" or "false". In order to improve the accuracy and effectiveness of classification, the present invention introduces a high-dimensional spherical boundary objective function in the classification layer. The high-dimensional spherical boundary objective function mainly revolves around the similarity step within the group and the similarity step between the groups. This invention pays more attention to the similarity within the group and designs a special weight w for the similarity within the group.

The high-dimensional spherical boundary objective function is an objective function that adaptively changes the optimization stride during the training process. Its stride update rules are as follows:

At the beginning, separate thresholds are set for intra-group similarity and inter-group similarity.

When a sample is input, the intra-group similarity and inter-group similarity of the current sample are first calculated. Then calculate the within-group similarity stride and the between-group similarity stride. The intra-group similarity step is the product of the weight w and the difference between the intra-group similarity threshold minus the intra-group similarity calculation value. The similarity step between groups is the difference between the calculated similarity value between groups minus the similarity threshold between groups. If these two strides are less than 0, then set them to 0, and always keep these two strides non-negative.

When calculating metric indicators in groups, the similarity loss within the group and the similarity loss between groups must be multiplied by their respective strides before statistics are made. In this way, when solving gradient descent, different strides can be updated for different data.

After training the model using a high-dimensional spherical boundary objective function, the model can identify forgeries of input images and determine whether they are generated by a large model.

A third aspect of the invention discloses an electronic device. The electronic device includes a memory and a processor. The memory stores a computer program. When the processor executes the computer program, it implements the steps in the large model generation image identification method according to any one of the first aspects disclosed in the present invention.

Figure 7 is a structural diagram of an electronic device according to an embodiment of the present invention. As shown in Figure 7, the electronic device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. Among them, the processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes non-volatile storage media and internal memory. The non-volatile storage medium stores operating systems and computer programs. This internal memory provides an environment for the execution of operating systems and computer programs in non-volatile storage media. The communication interface of the electronic device is used for wired or wireless communication with external terminals. The wireless mode can be implemented through WIFI, operator network, near field communication (NFC) or other technologies. The display screen of the electronic device may be a liquid crystal display or an electronic ink display. The input device of the electronic device may be a touch layer covered on the display screen, or may be a button, trackball or touch pad provided on the housing of the electronic device. , it can also be an external keyboard, trackpad or mouse, etc.

Those skilled in the art can understand that the structure shown in Figure 7 is only a structural diagram of the part related to the technical solution of the present disclosure, and does not constitute a limitation on the electronic equipment to which the solution of the present application is applied. Specific electronic devices Devices may include more or fewer components than shown in the figures, or some combinations of components, or have different arrangements of components.

A fourth aspect of the present invention discloses a computer-readable storage medium. A computer program is stored on the computer-readable storage medium. When the computer program is executed by the processor, the steps in the large model generation image identification method according to any one of the first aspects disclosed in the present invention are implemented.

Please note that the technical features of the above embodiments can be combined in any way. To simplify the description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features , should be considered to be within the scope of this manual. The above embodiments only express several implementation modes of the present application, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the invention patent. It should be noted that, for those of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present application, and these all fall within the protection scope of the present application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims (10)

1. A method for large model generation image authentication, the method comprising:

s1, inputting a generated image into a first processing module based on residual filtering to obtain original characteristics;

s2, inputting the original features into a second processing module based on a self-attention mechanism and a residual error structure to obtain classification features;

and S3, inputting the classification characteristics into a classification network, and outputting a result with only true or false.

2. The method for discriminating a large model generated image according to claim 1 wherein in step S1, the method for inputting the generated image to a first processing module based on residual filtering to obtain an original feature includes:

and respectively inputting the generated image into a residual filter and a convolution kernel, combining the processing results of the residual filter and the convolution kernel, and finally inputting the combined result into a first convolution pooling layer to obtain the original characteristics.

3. The large model generation image discrimination method according to claim 2, wherein in said step S1, there are seventeen of said residual filters; the convolution kernel has eight; the values of seventeen residual filters are fixed and not changed in learning; whereas the parameters of the eight convolution kernels are learned during training.

4. The large model generation image discrimination method according to claim 2, wherein in said step S1, said first convolution pooling layer is a convolution pooling layer with a residual mechanism of convolution layer and pooling layer.

5. The method of claim 1, wherein in step S2, the step of inputting the original features into a second processing module based on a self-attention mechanism and a residual structure to obtain classification features comprises:

and inputting the original features into a second convolution pooling layer to obtain processing features, inputting the processing features into self-attention operation, and carrying out numerical addition on the self-attention operation result and the processing features to obtain classification features.

6. The large model generation image discrimination method according to claim 5, wherein in said step S2, V of said attention operation is said processing feature; the weights assigned to the V are obtained by calculation of Q and K using a softmax layer, and then the weighted average results in shallow texture features captured from the attention mechanism.

7. A large model generation image discrimination method according to claim 1, wherein in said step S3, classification is optimized using a high-dimensional spherical boundary objective function in said two-classification network.

8. A system for large model generation image authentication, the system comprising:

the first processing module is configured to input the generated image into the first processing module based on residual filtering to obtain original characteristics;

the second processing module is configured to input the original features into the second processing module based on a self-attention mechanism and a residual error structure to obtain classification features;

and the third processing module is configured to input the classification characteristic into a classification network and output a result of only true or false.

9. An electronic device comprising a memory storing a computer program and a processor implementing the steps of a large model generation image authentication method according to any one of claims 1 to 7 when the computer program is executed by the processor.

10. A computer readable storage medium, characterized in that the computer readable storage medium has stored thereon a computer program which, when executed by a processor, implements the steps of a large model generation image authentication method according to any of claims 1 to 7.