CN114612411A - Image tampering detection method, device, device and storage medium - Google Patents

Abstract

The application discloses an image tampering detection method, device, equipment and storage medium. The method comprises the following steps: acquiring an image to be detected; carrying out noise characteristic identification on the image to be detected to obtain spatial domain characteristic information of photoresponse non-uniform noise and frequency domain characteristic information of the photoresponse non-uniform noise in the image to be detected; and inputting the spatial domain characteristic information and the frequency domain characteristic information into an image tampering detection network to carry out image tampering detection, so as to obtain image detection information corresponding to the image to be detected. According to the method and the device, on the scene of image tampering detection of the media content, the spatial domain characteristic and the frequency domain characteristic of the photoresponse non-uniform noise in the image are combined, and the accuracy of a tampering detection result is improved.

Description

Image tampering detection method, device, device and storage medium

technical field

The present application relates to the technical field of artificial intelligence, and in particular, to an image tampering detection method, device, device and storage medium.

Background technique

In recent years, with the development of digital technology and network communication technology, the modification methods of digital images have become diversified, such as: using PhotoShop (image processing software), CrazyTalk (face animation production software) and various mobile phone editing APPs, etc. As image tampering techniques become more and more advanced, and the concealment of tampering is getting better and better, the difficulty of image tampering detection continues to increase.

Although there are many image tampering detection methods, most of these methods only consider image tampering detection through a single feature, such as: using the spatial features of the image's PRNU (Photo Response Non-Uniformity, light response non-uniformity) noise. Tampering detection, when the content of the image scene is complex and changeable, the tampering detection of a single feature cannot guarantee the accuracy of the detection results. Therefore, a more accurate technical solution needs to be provided.

SUMMARY OF THE INVENTION

The present application provides an image tampering detection method, device, equipment and storage medium, which combine the spatial and frequency domain characteristics of light response non-uniformity noise in an image to improve the accuracy of tampering detection results. The technical solution of the present application is as follows :

In one aspect, an image tampering detection method is provided, the method comprising:

Obtain the image to be detected;

Performing noise feature recognition on the image to be detected, to obtain spatial feature information of the non-uniform light response noise and frequency domain feature information of the non-uniform light response noise in the image to be detected;

Inputting the air domain feature information and the frequency domain feature information into an image tampering detection network for image tampering detection, and obtaining image detection information corresponding to the image to be detected.

In another aspect, an image tampering detection device is provided, the method comprising:

a to-be-detected image acquisition module for acquiring the to-be-detected image;

A noise feature identification module, configured to perform noise feature identification on the image to be detected, and obtain spatial feature information of light response non-uniformity noise and frequency domain feature information of the light response non-uniformity noise in the to-be-detected image;

The image tampering detection module is configured to input the air domain feature information and the frequency domain feature information into an image tampering detection network for image tampering detection, and obtain image detection information corresponding to the image to be detected.

In another aspect, there is provided an image tampering detection device, the device includes a processor and a memory, the memory stores at least one instruction or at least a piece of program, the at least one instruction or the at least one piece of program is generated by the The processor loads and executes to implement the image tampering detection method as described in the first aspect.

In another aspect, a computer-readable storage medium is provided, wherein the storage medium stores at least one instruction or at least one piece of program, and the at least one instruction or at least one piece of program is loaded and executed by a processor to achieve the first The image tampering detection method described in one aspect.

In another aspect, a computer program product or computer program is provided, the computer program product or computer program comprising computer instructions stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the image tampering detection method described in the first aspect.

An image tampering detection method, device, equipment and storage medium provided by this application have the following technical effects:

In the scene of performing tampering detection on images, the present application obtains the image to be detected, and then performs noise feature recognition on the image to be detected, so as to obtain the spatial characteristic information of the non-uniformity noise of light response in the image to be detected and all The frequency domain feature information of the light response non-uniformity noise; then, the spatial domain feature information and the frequency domain feature information are input into an image tampering detection network for image tampering detection, and the image detection information corresponding to the to-be-detected image is obtained, By increasing the feature dimension and using the dual features of the spatial domain and the frequency domain of the light response non-uniformity noise for image tampering detection, the accuracy of image tampering detection can be greatly improved.

Description of drawings

In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present application or in the prior art, the following briefly introduces the accompanying drawings used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description The drawings are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can also be obtained based on these drawings without any creative effort.

1 is a schematic diagram of an application environment provided by an embodiment of the present application;

2 is a schematic flowchart of an image tampering detection method provided by an embodiment of the present application;

FIG. 3 is a schematic flow chart of obtaining an image to be detected provided by an embodiment of the present application;

FIG. 4 is a schematic flowchart of performing noise feature recognition on an image to be detected to obtain spatial feature information of light response non-uniformity noise and frequency domain feature information of light response non-uniformity noise in the to-be-detected image provided by an embodiment of the present application;

FIG. 5 is a schematic flowchart of performing spatial feature extraction of light response non-uniformity noise on a to-be-detected image provided by an embodiment of the present application to obtain spatial feature information;

6 is a schematic flowchart of inputting air domain feature information and frequency domain feature information into an image tampering detection network to perform image tampering detection, and obtaining image detection information corresponding to an image to be detected, provided by an embodiment of the present application;

7 is a schematic diagram of an image tampering detection network provided by an embodiment of the present application;

8 is a schematic structural diagram of an image tampering detection network provided by an embodiment of the present application;

9 is a schematic flowchart of a network training method provided by an embodiment of the present application;

FIG. 10 is a block diagram of an image tampering detection device provided by an embodiment of the present application;

FIG. 11 is a schematic structural diagram of an image tampering detection device provided by an embodiment of the present application.

Detailed ways

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work shall fall within the protection scope of the present application.

It should be noted that the terms "comprising" and "having" in the description and claims of the present application and the above-mentioned drawings, as well as any modifications thereof, are intended to cover non-exclusive inclusion, for example, including a series of steps or units A process, method, system, product or server is not necessarily limited to those steps or units expressly listed, but may include other steps or units not expressly listed or inherent to such process, method, product or device.

It can be understood that, in the specific implementation of this application, related data such as user information is involved. When the above embodiments of this application are applied to specific products or technologies, the user's permission or consent needs to be obtained, and the collection of relevant data. , use and processing need to comply with relevant laws, regulations and standards of relevant countries and regions.

Please refer to FIG. 1. FIG. 1 is a schematic diagram of an application environment provided by an embodiment of the present application. The application environment may include a client 10 and a server 20, and the client 10 and the server 20 may communicate with each other through wired or wireless communication. connected directly or indirectly. The user may send an image tampering detection request to the server 20 through the client 10 . The server side 20 determines the corresponding image to be detected based on the image tampering detection request, and then performs noise feature recognition on the image to be detected to obtain the spatial feature information of the light response non-uniformity noise and the frequency domain feature information of the light response non-uniformity noise in the to-be-detected image , and then input the air domain feature information and frequency domain feature information into the image tampering detection network for image tampering detection, obtain image detection information corresponding to the image to be detected, and return the image detection information to the client 10 . It should be noted that FIG. 1 is only an example.

The client can be a smart phone, computer (such as desktop computer, tablet computer, laptop computer), digital assistant, intelligent voice interaction device (such as smart speaker), smart wearable device and other types of physical devices, or it can be running on physical devices. software, such as computer programs. The operating system corresponding to the client may be an Android system (Android system), an iOS system (a mobile operating system developed by Apple), a Linux system (an operating system), a Microsoft Windows system (Microsoft Windows operating system), and the like.

The server side can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or it can provide cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, Cloud servers for basic cloud computing services such as middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. The server may include a network communication unit, a processor, a memory, and the like. The server side can provide background services for the corresponding clients.

The above-mentioned client 10 and server 20 can be used to construct a system related to image tampering detection, and the system can be a distributed system. Taking the distributed system as the blockchain system as an example, it is formed by multiple nodes (any form of computing devices in the access network, such as servers, user terminals) and clients, and a peer-to-peer (P2P, PeerTo) node is formed between the nodes. Peer) network, P2P protocol is an application layer protocol running on top of Transmission Control Protocol (TCP, Transmission Control Protocol). In a distributed system, any machine such as a server and a terminal can join to become a node, and a node includes a hardware layer, a middle layer, an operating system layer and an application layer.

The functions of each node in the above blockchain system include:

1) Routing, a basic function that a node has to support communication between nodes.

In addition to the routing function, a node can also have the following functions:

2) Application, used to deploy in the blockchain, implement specific business according to actual business needs, record data related to the realization of functions to form record data, carry a digital signature in the record data to indicate the source of the task data, and send the record data To other nodes in the blockchain system, for other nodes to add the record data to the temporary block when verifying the source and integrity of the record data successfully.

3) Blockchain, including a series of blocks (Blocks) that follow each other in chronological order. Once a new block is added to the blockchain, it will not be removed. The block records the blockchain system. The record data submitted by the middle node.

A specific embodiment of an image tampering detection method provided by the present application is introduced below. FIG. 2 is a schematic flowchart of an image tampering detection method provided by an embodiment of the present application. The present application provides the method described in the embodiment or the flowchart. operational steps, but may include more or fewer operational steps based on routine or non-creative work. The sequence of steps enumerated in the embodiments is only one of the execution sequences of many steps, and does not represent the only execution sequence. When an actual system or product is executed, the methods shown in the embodiments or the accompanying drawings may be executed sequentially or in parallel (for example, a parallel processor or a multi-threaded processing environment). Specifically, as shown in Figure 2, the method may include:

S201, an image to be detected is acquired.

In the embodiment of this specification, the above-mentioned image to be detected may be any image that needs to be tampered with.

In practical applications, the size data of images from different sources may be different. In order to ensure the accuracy of the image tampering detection results, the initial to-be-detected image is usually preprocessed to obtain a preset size of the to-be-detected image. Specifically, the preset The size can be set in combination with the accuracy requirements of image tampering detection in practical applications. In order to avoid the damage to the inter-pixel features of the image that may be caused when the image is zoomed, in the embodiment of the present specification, the image to be detected may be acquired by performing clipping preprocessing on the initial image to be detected.

In some embodiments, as shown in FIG. 3 , FIG. 3 is a schematic flowchart of obtaining an image to be detected provided by the embodiments of this specification. Specifically, it may include:

S301, acquiring an initial image to be detected.

Specifically, the size data of the initial to-be-detected image is generally not smaller than the above-mentioned preset size.

S302, performing center cropping processing on the initial image to be detected to obtain an image to be detected.

In a specific embodiment, the center cropping process may be to perform center cropping on different channel data of the image according to the image channel data, and the specific cropping area may include:

startXpos=(srcWidth–dstWidth)/2;

startYpos=(srcHeight–dstHeight)/2;

endXpos=startXpos+dstWidth;

endYpos=startYpos+dstHeight;

Among them, (srcWidth, srcHeight) is the width and height of the initial image to be detected, (dstWidth, dstHeight) is the width and height of the image to be detected, (startXpos, endXpos) is the horizontal start position and end position of the cropping area, ( startYpos, endYpos) are the vertical start and end positions of the clipping area.

It can be seen from the above embodiments that the initial image to be detected is centrally cropped to obtain the image to be detected, which can ensure the consistency of the detected image size, avoid the damage to the features between image pixels that may be caused by image processing methods such as scaling processing, and improve the noise feature. Extraction accuracy, thereby improving the accuracy of tampering detection results.

S202 , performing noise feature recognition on the image to be detected, to obtain spatial feature information of the non-uniform light response noise and frequency domain feature information of the non-uniform light response noise in the image to be detected.

In practical applications, photoresponse non-uniformity noise is a kind of noise interference introduced into digital images due to hardware manufacturing defects of a camera sensor CCD (Charge Coupled Device, Charge Coupled Device). At present, the CCDs manufactured in the industry all have process problems, resulting in the inconsistent thickness of the semiconductor silicon material of each pixel of the CCD. When the camera is exposed to light, there will be slight differences between different pixels. This slight difference is brought into each pixel of the digital image. A relatively weak noise is formed. Therefore, in the embodiment of the present specification, by performing the feature identification of the non-uniform light response noise on the image to be detected, the spatial domain feature information and frequency domain feature information of the light response non-uniformity noise are obtained, so that the spatial domain feature information and the frequency domain feature information are based on the spatial domain feature information and frequency domain feature information. information for image tampering detection of the image to be detected.

Specifically, the spatial domain feature information can represent the spatial domain feature of the light response non-uniformity noise, and the frequency domain feature information can represent the frequency domain feature of the light response non-uniformity noise.

In the embodiment of this specification, as shown in 4, the above-mentioned noise feature recognition of the image to be detected, and obtaining the spatial domain feature information of the photoresponse non-uniformity noise and the frequency domain feature information of the photoresponse non-uniformity noise in the to-be-detected image may include:

S401, perform spatial feature extraction of light response non-uniformity noise on the image to be detected to obtain spatial feature information.

In the embodiment of this specification, the representation form of the airspace feature information may be an airspace feature image.

In a specific embodiment, as shown in FIG. 5 , the above-mentioned spatial feature extraction of non-uniform light response noise on the image to be detected, and obtaining spatial feature information may include:

S501 , extract high-frequency components from multiple color channels of the image to be detected, respectively, to obtain high-frequency component data of multiple color channels.

Specifically, the high-frequency component data may include: horizontal high-frequency components, vertical high-frequency components, and diagonal high-frequency components.

In a specific embodiment, performing high-frequency component extraction on multiple color channels of the image to be detected respectively, and obtaining high-frequency component data of the multiple color channels may include: respectively performing multi-scale wavelet transform on multiple color channels of the image to be detected After processing, wavelet domain high-frequency component data of multiple scales corresponding to each color channel are obtained. Optionally, the multi-scale wavelet transform processing may be a fourth-order wavelet transform, and the wavelet base may be db4 (fourth-order Dobessie wavelet).

S502: Perform noise intensity analysis on the high-frequency component data of multiple color channels, respectively, to obtain noise intensity data.

Specifically, the local noise variance analysis is performed on the horizontal high-frequency components, the vertical high-frequency components and the diagonal high-frequency components of each color channel, respectively, to obtain the target variance data of the horizontal high-frequency components and the vertical high-frequency components. The target variance data and the target variance data of the high frequency components in the diagonal direction are taken as the above noise strength data.

In an optional embodiment, the local noise variance analysis may include performing noise variance analysis of multiple window sizes on each high-frequency component data, obtaining variance data corresponding to the multiple window sizes, and selecting the variance corresponding to the multiple window sizes. The minimum value in the data is used as the target variance data for each high-frequency component data. Preferably, the multiple windows may include: 3, 5, 7, and 9, and the initial standard deviation in the local noise variance analysis process may be 5.

S503 , based on the noise intensity data, filter the high-frequency component data of the multiple color channels respectively, to obtain initial noise information of the multiple color channels.

In a specific embodiment, when the noise intensity data is the target variance data of the high-frequency components of each color channel, the above-mentioned filtering processing is performed on the high-frequency component data of the multiple color channels based on the noise intensity data. , obtaining the initial noise information of the multiple color channels may include: filtering the high frequency components of each color channel by using the target variance data of the high frequency components of each color channel to obtain the initial noise information of each color channel.

Optionally, the filters used in the filtering process may include, but are not limited to, Wiener filters, Kalman filters, and the like.

In a specific embodiment, in the case that the above-mentioned high-frequency components are extracted by multi-scale wavelet transform processing, the above-mentioned initial noise information

S504: Perform reconstruction processing on the initial noise information of multiple color channels to obtain initial spatial feature information.

In a specific embodiment, when the above-mentioned high-frequency components are extracted by multi-scale wavelet transform processing, the above-mentioned initial noise information may be wavelet domain noise information, and correspondingly, the initial noise information of multiple color channels is reconstructed The processing to obtain the initial spatial domain feature information may include: performing inverse wavelet transform processing on the wavelet domain noise information of multiple color channels to obtain the spatial domain noise information of the image to be detected, and using the spatial domain noise information as the initial spatial domain of the light response non-uniformity noise characteristic information.

S505 , performing noise enhancement processing on the initial airspace feature information to obtain airspace feature information.

In a specific embodiment, performing noise enhancement processing on the initial spatial feature information to obtain the spatial feature information may include:

1) Perform zero-average filtering on the initial airspace feature information to obtain the initial airspace feature information after filtering.

Specifically, the interference information in the initial spatial domain feature information is removed by zero-average filtering processing, so as to enhance the non-uniformity noise of the optical response.

2) Perform Fourier peak suppression processing on the initial spatial domain feature information after filtering to obtain spatial domain feature information.

In practical applications, the non-uniformity noise of the light response can be enhanced by removing the dazzling interfering pixels in the image from the Fourier frequency domain dimension through Fourier peak suppression processing.

It can be seen from the above embodiments that the initial noise information is obtained by filtering the high-frequency component data of the image to be detected, and the initial noise information is reconstructed to obtain the initial spatial feature information, and then noise enhancement processing is performed on the initial spatial feature information. , to obtain airspace feature information, which can effectively remove interference and improve the accuracy of airspace feature extraction.

S402 , performing space-frequency transform processing on the spatial-domain feature information to obtain frequency-domain feature information.

In the embodiment of this specification, the representation form of the frequency domain feature information may be a frequency domain feature image.

In a specific embodiment, the above-mentioned space-frequency transform processing on the spatial-domain feature information to obtain the frequency-domain feature information may include:

1) Discrete Fourier transform is performed on the spatial domain feature information to obtain the initial frequency domain feature information.

2) Perform spectral centering processing on the initial frequency domain feature information to obtain frequency domain feature information.

Specifically, in the case where the initial frequency domain feature information is the initial frequency domain feature image, the DC component in the initial frequency domain feature image is usually located in the four corner areas of the image, and the AC component in the initial frequency domain feature image is usually located in the image. However, in the embodiment of this specification, in order to facilitate subsequent feature extraction, spectral centering processing may be performed on the initial frequency domain feature information, and the DC component may be converted to the image center.

In an optional embodiment, the initial frequency domain feature image is divided based on the horizontal symmetry axis and the vertical symmetry axis of the image to obtain 4 regions, and the two diagonal regions of the image are exchanged respectively, so that the DC component is divided into four regions. Transform to the center of the image.

It can be seen from the above embodiments that discrete Fourier transform processing is performed on the spatial domain feature information to obtain initial frequency domain feature information, and frequency domain feature information is obtained by performing spectrum centering processing on the initial frequency domain feature information, which can effectively improve the frequency domain feature information. The accuracy of extraction improves the accuracy of detection results.

S203: Input the air domain feature information and the frequency domain feature information into an image tampering detection network to perform image tampering detection, and obtain image detection information corresponding to the image to be detected.

In the embodiment of this specification, the above-mentioned image tampering detection network may be obtained by performing image tampering detection training on a preset image tampering detection network based on sample air domain feature information and sample frequency domain feature information. Specifically, the image tampering detection network may include: : spatial domain feature aggregation layer, frequency domain feature aggregation layer, feature fusion layer and tamper detection layer.

In a specific embodiment, as shown in FIG. 6 , the above-mentioned inputting the air domain feature information and the frequency domain feature information into the image tampering detection network for image tampering detection, and obtaining the image detection information corresponding to the image to be detected may include:

S601 , input airspace feature information into an airspace feature aggregation layer to perform airspace feature aggregation processing to obtain target airspace feature information.

Specifically, the target spatial domain feature information can represent the spatial domain aggregation features of the non-uniformity noise of the light response. The representation form of the target airspace feature information can be a target airspace feature image.

In an optional embodiment, the spatial feature aggregation layer may include: a first convolution layer, a first batch of normalization layers, a first activation layer and at least one layer of MBConv (Mobile Inverted Bottleneck Convolution, Mobile Inverted Bottleneck Convolution). )Floor.

S602: Input the frequency domain feature information into the frequency domain feature aggregation layer to perform frequency domain feature aggregation processing to obtain target frequency domain feature information.

Specifically, the target frequency domain feature information can represent the frequency domain aggregation features of the non-uniformity noise of the light response. The representation form of the target frequency domain feature information may be a target frequency domain feature image.

In an optional embodiment, the frequency domain feature aggregation layer may include: a second convolution layer, a second batch of normalization layers, a second activation layer, and at least one layer of MBConv (Mobile Inverted Bottleneck Convolution, Mobile Inverted Bottleneck Convolution). accumulation) layer.

In practical applications, the structure of the spatial feature aggregation layer and the frequency domain feature aggregation layer can be the same or different, but the size of the target spatial feature image output by the spatial feature aggregation layer and the target frequency domain feature image output by the frequency domain feature aggregation layer. The size should be kept consistent to facilitate subsequent feature fusion.

S603, the target air domain feature information and the target frequency domain feature information are input into the feature fusion layer to perform feature fusion processing to obtain target feature fusion information.

Specifically, the target frequency domain feature information can represent the spatial domain aggregation features and frequency domain aggregation features of the non-uniformity noise of the light response.

In an optional embodiment, the feature fusion layer may be a channel fusion layer, and the above-mentioned inputting the target airspace feature information and the target frequency domain feature information into the feature fusion layer for feature fusion processing, and obtaining the target feature fusion information may include: The feature information and the target frequency domain feature information are input to the channel fusion layer, and the target air domain feature information and the target frequency domain feature information are superimposed and fused according to the image channel to obtain the target feature fusion information.

Specifically, the channel fusion layer can superimpose and fuse the target spatial domain feature information and the target frequency domain feature information from the dimension of the image channel. For example, the target spatial feature information is a spatial feature map with a size of 7×7×512, and the target frequency domain feature information is a frequency domain feature map with a size of 7×7×512, where 7×7 is the spatial resolution and 512 is the Correspondingly, superimposing and merging the target air domain feature information and the target frequency domain feature information from the dimension of the image channel may include: superimposing the air domain feature map and the frequency domain feature map according to the image channel to obtain a 7×7×1024 The feature map is fused, and the fused feature map is used as the target feature fusion information.

In another optional embodiment, the feature fusion layer may include: a feature vector generation layer and a feature vector fusion layer, wherein the target air domain feature information and target frequency domain feature information are input into the feature fusion layer for feature fusion processing to obtain target features. The fusion information may include: inputting the target spatial domain feature information into the feature vector generation layer to perform spatial domain feature vector generation processing to obtain a spatial domain feature vector; inputting the target frequency domain feature information into the eigenvector generation layer for frequency domain feature vector generation processing to obtain frequency domain features vector; input the air domain feature vector and the frequency domain feature vector into the feature vector fusion layer for feature vector fusion to obtain the target feature vector; take the target feature vector as the target feature fusion information.

Specifically, the feature vector generation layer can be a global average pooling layer or a fully connected layer. The feature vector generation layer can perform feature compression on the target spatial feature information and the target frequency domain feature information, respectively, to obtain the spatial domain feature vector and the frequency domain feature vector. ; Through the feature vector fusion layer, the air domain feature vector and the frequency domain feature vector can be subjected to point-by-point addition or point-by-point multiplication processing from the dimension of the feature vector to obtain the target feature vector.

It can be seen from the above embodiments that a variety of feature fusion methods are provided. For different types of images to be detected, image channel fusion or feature fusion methods that fuse the corresponding feature vectors of the images can be selected, thereby improving the target feature fusion information. The noise feature of the image to be detected. representation accuracy.

S604, input the target feature fusion information into the tampering detection layer to perform image tampering detection, and obtain image detection information.

In the embodiment of this specification, the image detection information may be used to represent whether the image to be detected has been tampered with. Specifically, the image detection information may be a detection label corresponding to the image to be detected, wherein the detection label may be a real image label or a tampering detection label.

In a specific embodiment, the tamper detection layer may include a third convolutional layer, a global average pooling layer, a fully connected layer, and an output layer.

Specifically, the third convolution layer can perform convolution processing on the input target feature fusion information to realize feature extraction of the target feature fusion information.

Specifically, the global average pooling layer can perform down-sampling operation on the output of the previous layer, that is, return the maximum value in the sampling window as the output of down-sampling. On the one hand, it can make the image smaller and simplify the computational complexity; on the other hand, it can perform feature compression to extract the main features.

Specifically, the fully connected layer can be used as a connection layer between the nodes of the upper and lower layers, and establishes a connection relationship between the data of each node obtained by the upper and lower layers. The fully connected layer can perform feature compression processing on the target feature fusion information to obtain the feature information to be detected.

Specifically, the classification layer can perform image tampering detection on the feature information to be detected, and output corresponding detection labels. In a specific embodiment, the classification layer can use an activation function to output the target knowledge point label. In an optional embodiment, the activation function can be a Softmax function, and the Softmax function includes a nonlinear classifier for processing Detect feature information for image tampering detection.

In addition, it should be noted that the image tampering detection network described in the embodiments of this specification is not limited to the above-mentioned preset image tampering detection network. In practical applications, it may also include other machine learning networks, such as decision tree machine learning networks, etc. The embodiments of the present application are not limited to the above-mentioned preset image tampering detection network.

In a specific embodiment, as shown in FIG. 7 , an image tampering detection network including the above-mentioned spatial domain feature aggregation layer, the above-mentioned frequency domain feature aggregation layer, the above-mentioned feature fusion layer and the above-mentioned tampering detection layer is established, and the light in the image to be detected is The spatial domain feature information and frequency domain feature information in response to the non-uniform noise are input into the image tampering detection network for image tampering detection, and the image detection information corresponding to the image to be detected is obtained.

In a specific embodiment, as shown in FIG. 8 , FIG. 8 is a schematic structural diagram of an image tampering detection network provided by an embodiment of the present application. Specifically, the image tampering detection network may include: a spatial domain feature aggregation layer, a frequency domain Feature aggregation layer, feature fusion layer and tampering detection layer; the spatial domain feature aggregation layer may include: the first convolution layer, the first batch normalization layer, the first activation layer and the 5-layer MBConv layer; the frequency domain feature aggregation layer may include : the second convolution layer, the second batch normalization layer, the second activation layer and the 5-layer MBConv layer; the tamper detection layer can include: the third convolution layer, the global average pooling layer, the fully connected layer and the classification layer.

In addition, it should be noted that the hierarchical structures of the spatial feature aggregation layer and the frequency domain feature aggregation layer in the embodiments of the present application are not fixed and can be arbitrarily designed, but the target spatial feature image and frequency domain feature aggregation layer output by the spatial feature aggregation layer The size of the output target frequency domain feature images should be consistent to facilitate subsequent feature fusion.

It can be seen from the above embodiments that using the spatial domain feature aggregation layer, the frequency domain feature aggregation layer, the feature fusion layer and the tampering detection layer to perform image tampering detection can improve the adaptability of tampering detection to different images, and thus can greatly improve the image tampering detection. 's accuracy.

In the embodiment of this specification, a preset image tampering detection network can be trained by using sample detection images to obtain the above-mentioned image tampering detection network.

In a specific embodiment, as shown in FIG. 9 , FIG. 9 is a schematic flowchart of a network training method provided by an embodiment of the present application. Specifically, it may include:

S901: Acquire a sample detection image and preset image detection information corresponding to the sample detection image.

In practical applications, training data may be determined before network training. Specifically, in this embodiment of the present application, a sample detection image containing preset image detection information may be obtained as training data.

Specifically, the preset image detection information may be a preset detection label pre-marked on the sample detection image. In the embodiment of this specification, the preset detection label may include a real image label or a tampered image label, and the sample detection image may include a sample real image and a sample tampered image. Correspondingly, the preset detection label of the sample real image may be a real image label , the preset detection label of the sample tampered image can be the tampered image label. Generally, the ratio of the sample real image and the sample tampered image in the training data can be preset according to the actual requirements of the network training accuracy.

In a specific embodiment, obtaining the sample detection image above may include:

1) Obtain the initial sample detection image.

2) Perform center cropping processing on the initial sample detection image to obtain a sample detection image.

In practical applications, by cropping and preprocessing the initial sample detection image, it can ensure that the image data input to the preset image tampering detection network has the same size, and at the same time avoid destroying the characteristics of the non-uniformity noise of light response between image pixels. , in addition, it can further reduce the time-consuming of training prediction.

S902: Perform noise feature recognition on the sample detection image to obtain sample spatial domain feature information of the sample light response non-uniformity noise and sample frequency domain feature information of the sample light response non-uniformity noise in the sample detection image.

S903: Input the sample air domain feature information and the sample frequency domain feature information into a preset image tampering detection network to perform image tampering detection, and obtain sample image detection information corresponding to the sample detection image.

S904, based on the preset image detection information and the sample image detection information, determine target loss information.

S905 , based on the target loss information, train a preset image tampering detection network to obtain an image tampering detection network.

In an optional embodiment, the above-mentioned sample image detection information may include a sample detection label of the sample detection image, and correspondingly, the above-mentioned target loss information may include detection label loss;

Correspondingly, the above-mentioned determination of target loss information based on the preset image detection information and the sample image detection information may include:

According to the preset detection label and the sample detection label, the detection label loss is determined.

In a specific embodiment, determining the detection label loss according to the preset detection label and the sample detection label may include determining the detection label loss between the preset detection label and the sample detection label based on a preset loss function.

In a specific embodiment, the detection label loss can represent the difference between the preset detection label and the sample detection label.

In a specific embodiment, the preset loss function may include, but is not limited to, a cross-entropy loss function, a logistic loss function, an exponential loss function, and the like.

In an optional embodiment, training a preset image tampering detection network based on the target loss information, and obtaining the image tampering detection network may include:

S9051, based on the target loss information, update network parameters of a preset image tampering detection network;

S9052, based on the updated preset image tampering detection network, repeatedly perform the image tampering detection training iterative operation including steps S903, S904 and S9051 until the image tampering detection convergence condition is reached;

S9053, use the preset image tampering detection network obtained when the image tampering detection convergence condition is reached as the image tampering detection network.

In an optional embodiment, the above-mentioned condition for reaching the convergence of image tampering detection may be that the number of training iteration operations reaches a preset number of training times. Optionally, reaching the convergence condition of image tampering detection may also be that the target loss information is less than a specified threshold. In the embodiment of this specification, the preset training times and the specified threshold may be preset in combination with the training speed and accuracy of the network in practical applications.

In a specific embodiment, the above-mentioned preset image tampering detection network may include a preset spatial domain feature aggregation layer, a preset frequency domain feature aggregation layer, a preset feature fusion layer, and a preset tampering detection layer. The air domain feature information and the sample frequency domain feature information are input into a preset image tampering detection network to perform image tampering detection, and the sample image detection information corresponding to the sample detection image obtained may include:

Input the sample spatial feature information into the preset spatial feature aggregation layer for spatial feature aggregation processing to obtain sample target spatial feature information; input the sample frequency domain feature information into the preset frequency domain feature aggregation layer to perform frequency domain feature aggregation processing to obtain the sample target frequency domain feature information; input the sample target air domain feature information and sample target frequency domain feature information into the preset feature fusion layer for feature fusion processing to obtain sample target feature fusion information; input the sample target feature fusion information into the preset tamper detection layer for image tampering Detect to obtain sample image detection information.

It can be seen from the above embodiments that, on the one hand, the machine learning training based on sample detection images and corresponding preset detection labels improves the generalization ability and robustness of the image tampering detection network, so as to better improve the network's ability to detect image tampering. accuracy.

It can be seen from the technical solutions provided by the above embodiments of the present application that, in the scenario of performing tampering detection on images, on the one hand, the initial image to be detected is centrally cropped to obtain the image to be detected, which can ensure the consistency of the detected image size. Avoid the damage to the features between image pixels that may be caused by image processing methods such as scaling processing, and improve the accuracy of noise feature recognition; on the other hand, by filtering the high-frequency component data of the image to be detected, the initial noise information is obtained. The initial noise information is reconstructed to obtain initial airspace feature information, and then noise enhancement processing is performed on the initial airspace feature information to obtain airspace feature information, which can effectively remove interference and improve the accuracy of airspace feature extraction. The information is subjected to discrete Fourier transform processing to obtain initial frequency domain feature information, and spectrum centralization is performed on the initial frequency domain feature information to obtain frequency domain feature information, which can effectively improve the accuracy of frequency domain feature extraction; on the other hand, Using the spatial feature aggregation layer, frequency domain feature aggregation layer, feature fusion layer and tampering detection layer for image tampering detection can improve the adaptability of tampering detection to different images, which can greatly improve the accuracy of image tampering detection; another On the one hand, machine learning training based on sample detection images and corresponding preset detection labels improves the generalization ability and robustness of the image forgery detection network, which can better improve the accuracy of the network for image forgery detection.

The embodiment of the present application further provides an image tampering detection device, as shown in FIG. 10 , the image tampering detection device may include:

a to-be-detected image acquisition module 1010, configured to acquire the to-be-detected image;

The noise feature identification module 1020 is configured to perform noise feature identification on the image to be detected, and obtain the spatial feature information of the non-uniform light response noise and the frequency domain feature information of the non-uniform light response noise in the image to be detected;

The image tampering detection module 1030 is configured to input the air domain feature information and the frequency domain feature information into the image tampering detection network for image tampering detection, and obtain image detection information corresponding to the image to be detected.

In some embodiments, the above-mentioned to-be-detected image acquisition module 1010 may include:

an initial to-be-detected image acquisition unit, configured to acquire an initial to-be-detected image;

The center cropping processing unit is used to perform center cropping processing on the initial to-be-detected image to obtain the to-be-detected image.

In the embodiment of this specification, the above-mentioned noise feature identification module 1020 may include:

The spatial feature extraction unit is used to perform spatial feature extraction of light response non-uniformity noise on the image to be detected to obtain spatial feature information;

The space-frequency transform processing unit is used for performing space-frequency transform processing on the spatial-domain feature information to obtain frequency-domain feature information.

In a specific embodiment, the above-mentioned airspace feature extraction unit may include:

a high-frequency component extraction unit, which is used for extracting high-frequency components from multiple color channels of the image to be detected, to obtain high-frequency component data of multiple color channels;

The noise intensity analysis unit is used for separately performing noise intensity analysis on the high-frequency component data of multiple color channels to obtain noise intensity data;

a filtering processing unit, configured to perform filtering processing on the high-frequency component data of multiple color channels based on the noise intensity data, to obtain initial noise information of multiple color channels;

a reconstruction processing unit, configured to perform reconstruction processing on the initial noise information of multiple color channels to obtain initial spatial feature information;

The noise enhancement processing unit is used for performing noise enhancement processing on the initial spatial feature information to obtain the spatial feature information.

In a specific embodiment, the above-mentioned image tampering detection module 1030 may include:

The airspace feature aggregation processing unit is used to input the airspace feature information into the airspace feature aggregation layer for airspace feature aggregation processing to obtain the target airspace feature information;

a frequency-domain feature aggregation processing unit, used for inputting the frequency-domain feature information into the frequency-domain feature aggregation layer for frequency-domain feature aggregation processing to obtain target frequency-domain feature information;

The feature fusion unit is used for inputting the target air domain feature information and the target frequency domain feature information into the feature fusion layer for feature fusion processing to obtain target feature fusion information;

The image tampering detection unit is used to input the target feature fusion information into the tampering detection layer for image tampering detection, and obtain image detection information.

In an optional embodiment, the above-mentioned feature fusion unit may include:

The stacking fusion unit is used to input the target airspace feature information and the target frequency domain feature information into the channel fusion layer, and stack and fuse the target airspace feature information and the target frequency domain feature information according to the image channel to obtain the target feature fusion information.

In a specific embodiment, the above device may further include:

The sample acquisition module is used to acquire the sample detection image and the preset image detection information corresponding to the sample detection image;

The sample noise feature identification module is used to identify the noise feature of the sample detection image, and obtain the sample spatial domain feature information of the sample photoresponse non-uniformity noise and the sample frequency domain feature information of the sample photoresponse non-uniformity noise in the sample detection image;

The sample image detection information module is used to input the sample air domain feature information and the sample frequency domain feature information into a preset image tampering detection network for image tampering detection, and obtain sample image detection information corresponding to the sample detection image;

The target loss information determination module is used to determine the target loss information based on the preset image detection information and the sample image detection information;

The network training module is used to train a preset image tampering detection network based on the target loss information to obtain an image tampering detection network.

It should be noted that the apparatus and method embodiments in the apparatus embodiments are based on the same inventive concept.

An embodiment of the present application provides an image tampering detection device. The image tampering detection device includes a processor and a memory, and the memory stores at least one instruction or at least one program, and the at least one instruction or the at least one program is executed by the processor. Load and execute to implement the image tampering detection method provided by the above method embodiments.

Further, FIG. 11 shows a schematic diagram of the hardware structure of an image tampering detection device for implementing the image tampering detection method provided by the embodiment of the present application. Provided image tampering detection device. As shown in FIG. 11 , the image tampering detection device 110 may include one or more processors 1102 (illustrated by 1102a, 1102b, . A processing device such as a programmable logic device FPGA), a memory 1104 for storing data, and a transmission device 1106 for a communication function. In addition, may also include: display, input/output interface (I/O interface), universal serial bus (USB) port (may be included as one of the ports of the I/O interface), network interface, power supply and/or camera. Those of ordinary skill in the art can understand that the structure shown in FIG. 11 is only a schematic diagram, which does not limit the structure of the above-mentioned electronic device. For example, the image tampering detection device 110 may also include more or fewer components than shown in FIG. 11 , or have a different configuration than that shown in FIG. 11 .

It should be noted that the one or more processors 1102 and/or other data processing circuits described above may generally be referred to herein as "data processing circuits." The data processing circuit may be embodied in whole or in part as software, hardware, firmware or any other combination. Additionally, the data processing circuitry may be a single, stand-alone processing module, or incorporated in whole or in part into any of the other elements in the image tampering detection device 110 (or mobile device). As referred to in the embodiments of the present application, the data processing circuit acts as a kind of processor control (eg, selection of a variable resistance termination path connected to an interface).

The memory 1104 can be used to store software programs and modules of the application software, such as a program instruction/data storage device corresponding to the image tampering detection method described in the embodiments of the present application, the processor 1102 runs the software programs and modules stored in the memory 1104 by running the software programs and modules. , so as to perform various functional applications and data processing, that is, to implement the above-mentioned image tampering detection method. Memory 1104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 1104 may further include memory located remotely from the processor 1102, and these remote memories may be connected to the image tampering detection device 110 via a network. Examples of such networks include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

Transmission means 1106 is used to receive or transmit data via a network. The specific example of the above-mentioned network may include a wireless network provided by the communication provider of the image tampering detection device 110 . In one example, the transmission device 1106 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through the base station so as to communicate with the Internet. In one embodiment, the transmission device 1106 may be a radio frequency (Radio Frequency, RF) module, which is used to communicate with the Internet in a wireless manner.

The display may be, for example, a touch screen-style liquid crystal display (LCD) that enables a user to interact with the user interface of the image tamper detection device 110 (or mobile device).

Embodiments of the present application further provide a computer-readable storage medium, which can be set in an image tampering detection device to store at least one instruction or at least a section for implementing the image tampering detection method in the method embodiment. A program, the at least one instruction or the at least one segment of the program is loaded and executed by the processor to implement the image tampering detection method provided by the above method embodiments.

Optionally, in this embodiment, the above-mentioned storage medium may be located in at least one network server among multiple network servers of a computer network. Optionally, in this embodiment, the above-mentioned storage medium may include but is not limited to: a U disk, a read-only memory (Read-Only Memory, ROM), a random access memory (Random Access Memory, RAM), a mobile hard disk, a magnetic Various media that can store program codes, such as a disc or an optical disc.

Embodiments of the present application also provide a computer program product or computer program, where the computer program product or computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the image tampering detection method provided by the method embodiment.

It should be noted that: the above-mentioned order of the embodiments of the present application is only for description, and does not represent the advantages and disadvantages of the embodiments. And the above describes specific embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that in the embodiments and still achieve desirable results. Additionally, the processes depicted in the figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

Each embodiment in this application is described in a progressive manner, and the same and similar parts between the various embodiments may be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the apparatus and device embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and reference may be made to some descriptions of the method embodiments for related parts.

Those of ordinary skill in the art can understand that all or part of the steps of implementing the above embodiments can be completed by hardware, or can be completed by instructing relevant hardware through a program, and the program can be stored in a computer-readable storage medium. The storage medium mentioned may be a read-only memory, a magnetic disk or an optical disk, etc.

The above descriptions are only preferred embodiments of the present application, and are not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application shall be included in the protection of the present application. within the range.

Claims (12)

1. an image tampering detection method, is characterized in that, described method comprises: Get the image to be detected; Performing noise feature recognition on the image to be detected, to obtain spatial feature information of the non-uniform light response noise and frequency domain feature information of the non-uniform light response noise in the image to be detected; Inputting the air domain feature information and the frequency domain feature information into an image tampering detection network for image tampering detection, and obtaining image detection information corresponding to the image to be detected. 2. The method according to claim 1, wherein the image tampering detection network comprises a spatial feature aggregation layer, a frequency domain feature aggregation layer, a feature fusion layer and a tampering detection layer, and the spatial feature information and the tampering detection layer are combined. The frequency domain feature information is input into an image tampering detection network for image tampering detection, and the obtained image detection information corresponding to the image to be detected includes: Inputting the airspace feature information into the airspace feature aggregation layer to perform airspace feature aggregation processing to obtain target airspace feature information; Inputting the frequency domain feature information into the frequency domain feature aggregation layer to perform frequency domain feature aggregation processing to obtain target frequency domain feature information; Inputting the target air domain feature information and the target frequency domain feature information into the feature fusion layer for feature fusion processing to obtain target feature fusion information; Inputting the target feature fusion information into the tampering detection layer to perform image tampering detection to obtain the image detection information. 3 . The method according to claim 1 , wherein the noise feature identification is performed on the image to be detected to obtain spatial feature information of non-uniformity noise of light response in the image to be detected and the light response. 4 . The frequency domain characteristic information of non-uniform noise includes: extracting the spatial feature of the light response non-uniformity noise on the to-be-detected image to obtain the spatial feature information; Space-frequency transform processing is performed on the spatial-domain feature information to obtain the frequency-domain feature information. 4 . The method according to claim 2 , wherein the feature fusion layer is a channel fusion layer, and the target air domain feature information and the target frequency domain feature information are input into the feature fusion layer for feature information 4 . Fusion processing, the obtained target feature fusion information includes: The target airspace feature information and the target frequency domain feature information are input into the channel fusion layer, and the target airspace feature information and the target frequency domain feature information are stacked and fused according to image channels to obtain the target feature fusion. information. 5 . The method according to claim 3 , wherein, performing spatial feature extraction of the light response non-uniformity noise on the to-be-detected image, and obtaining the spatial feature information comprises: 6 . Extracting high-frequency components for the multiple color channels of the image to be detected, respectively, to obtain high-frequency component data of the multiple color channels; Perform noise intensity analysis on the high-frequency component data of the multiple color channels, respectively, to obtain noise intensity data; Based on the noise intensity data, filtering processing is performed on the high-frequency component data of the multiple color channels, respectively, to obtain initial noise information of the multiple color channels; performing reconstruction processing on the initial noise information of the multiple color channels to obtain initial spatial feature information; Noise enhancement processing is performed on the initial airspace feature information to obtain the airspace feature information. 6. The method according to any one of claims 1 to 5, wherein the acquiring an image to be detected comprises: Obtain the initial image to be detected; Perform a center cropping process on the initial to-be-detected image to obtain the to-be-detected image. 7. The method according to any one of claims 1 to 5, wherein the method further comprises: obtaining the sample detection image and preset image detection information corresponding to the sample detection image; Perform noise feature recognition on the sample detection image to obtain sample spatial domain feature information of the sample light response non-uniformity noise and sample frequency domain feature information of the sample light response non-uniformity noise in the sample detection image; Inputting the sample air domain feature information and the sample frequency domain feature information into a preset image tampering detection network for image tampering detection, and obtaining sample image detection information corresponding to the sample detection image; determining target loss information based on the preset image detection information and the sample image detection information; Based on the target loss information, the preset image tampering detection network is trained to obtain the image tampering detection network. 8. The method according to claim 7, wherein the preset image tampering detection network comprises a preset spatial domain feature aggregation layer, a preset frequency domain feature aggregation layer, a preset feature fusion layer and a preset tamper detection layer , the inputting the sample air domain feature information and the sample frequency domain feature information into a preset image tampering detection network for image tampering detection, and obtaining the sample image detection information corresponding to the sample detection image includes: Inputting the sample airspace feature information into the preset airspace feature aggregation layer to perform airspace feature aggregation processing to obtain sample target airspace feature information; Inputting the sample frequency domain feature information into the preset frequency domain feature aggregation layer to perform frequency domain feature aggregation processing to obtain sample target frequency domain feature information; Inputting the sample target air domain feature information and the sample target frequency domain feature information into the preset feature fusion layer to perform feature fusion processing to obtain sample target feature fusion information; Inputting the sample target feature fusion information into the preset tampering detection layer to perform image tampering detection to obtain the sample image detection information. 9. An image tampering detection device, characterized in that the device comprises: a to-be-detected image acquisition module for acquiring the to-be-detected image; A noise feature identification module, configured to perform noise feature identification on the image to be detected, and obtain spatial feature information of light response non-uniformity noise and frequency domain feature information of the light response non-uniformity noise in the to-be-detected image; The image tampering detection module is configured to input the air domain feature information and the frequency domain feature information into an image tampering detection network for image tampering detection, and obtain image detection information corresponding to the image to be detected. 10. An image tampering detection device, characterized in that the device comprises a processor and a memory, and the memory stores at least one instruction or at least one program, and the at least one instruction or the at least one program is executed by the The processor loads and executes to implement the image tampering detection method as claimed in any one of claims 1 to 8. 11. A computer-readable storage medium, characterized in that, the storage medium stores at least one instruction or at least one program, and the at least one instruction or the at least one program is loaded and executed by a processor to implement the claim The image tampering detection method described in any one of 1 to 8 is required. 12. A computer program product, characterized in that, the computer program product comprises at least one instruction or at least one piece of program, and the at least one instruction or at least one piece of program is loaded and executed by a processor to implement the method according to claim 1 to 8. Any of the described image tampering detection methods.

Images