CN114612411A

CN114612411A - Image tampering detection method, device, equipment and storage medium

Info

Publication number: CN114612411A
Application number: CN202210214273.0A
Authority: CN
Inventors: 韩周; 董志强; 张壮
Original assignee: Tencent Technology Shenzhen Co Ltd
Current assignee: Tencent Technology Shenzhen Co Ltd
Priority date: 2022-03-04
Filing date: 2022-03-04
Publication date: 2022-06-10

Abstract

The application discloses an image tampering detection method, device, equipment and storage medium. The method comprises the following steps: acquiring an image to be detected; carrying out noise characteristic identification on the image to be detected to obtain spatial domain characteristic information of photoresponse non-uniform noise and frequency domain characteristic information of the photoresponse non-uniform noise in the image to be detected; and inputting the spatial domain characteristic information and the frequency domain characteristic information into an image tampering detection network to carry out image tampering detection, so as to obtain image detection information corresponding to the image to be detected. According to the method and the device, on the scene of image tampering detection of the media content, the spatial domain characteristic and the frequency domain characteristic of the photoresponse non-uniform noise in the image are combined, and the accuracy of a tampering detection result is improved.

Description

Image tampering detection method, device, equipment and storage medium

Technical Field

The present application relates to the field of artificial intelligence technologies, and in particular, to a method, an apparatus, a device, and a storage medium for detecting image tampering.

Background

In recent years, with the development of digital technology and network communication technology, the modification modes of digital images become diversified, such as: photoshop (image processing software), CrazyTalk (facial animation making software), various mobile phones are used for editing APP and the like, and the difficulty of image tampering detection is increased continuously due to the fact that image tampering methods are advanced and tampering concealment is good.

Although many image tamper detection methods are available, most of them only consider image tamper detection by a single feature, such as: tamper detection is performed through spatial domain characteristics of PRNU (Photo Response Non-Uniformity) noise of an image, and when the content of an image scene is complex and variable, the accuracy of a detection result is difficult to guarantee through the tamper detection of a single characteristic. Therefore, it is desirable to provide a more accurate solution.

Disclosure of Invention

The application provides an image tampering detection method, device, equipment and storage medium, wherein the spatial domain characteristics and the frequency domain characteristics of photoresponse non-uniform noise in an image are combined, so that the accuracy of a tampering detection result is improved, and the technical scheme of the application is as follows:

in one aspect, an image tampering detection method is provided, where the method includes:

acquiring an image to be detected;

performing noise characteristic identification on the image to be detected to obtain spatial domain characteristic information of photoresponse non-uniform noise and frequency domain characteristic information of the photoresponse non-uniform noise in the image to be detected;

and inputting the spatial domain characteristic information and the frequency domain characteristic information into an image tampering detection network to carry out image tampering detection, so as to obtain image detection information corresponding to the image to be detected.

In another aspect, an image tampering detection apparatus is provided, the method including:

the image acquisition module to be detected is used for acquiring an image to be detected;

the noise characteristic identification module is used for carrying out noise characteristic identification on the image to be detected to obtain the space domain characteristic information of the photoresponse non-uniform noise and the frequency domain characteristic information of the photoresponse non-uniform noise in the image to be detected;

and the image tampering detection module is used for inputting the airspace characteristic information and the frequency domain characteristic information into an image tampering detection network to carry out image tampering detection so as to obtain image detection information corresponding to the image to be detected.

In another aspect, an image tampering detection device is provided, the device includes a processor and a memory, the memory stores at least one instruction or at least one program, and the at least one instruction or the at least one program is loaded and executed by the processor to implement the image tampering detection method according to the first aspect.

In another aspect, a computer-readable storage medium is provided, in which at least one instruction or at least one program is stored, and the at least one instruction or the at least one program is loaded and executed by a processor to implement the image tampering detection method according to the first aspect.

In another aspect, a computer program product or computer program is provided, the computer program product or computer program comprising computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the image tampering detection method according to the first aspect.

The image tampering detection method, device, equipment and storage medium provided by the application have the following technical effects:

the method comprises the steps of acquiring an image to be detected in a scene of tampering detection of the image, and then performing noise characteristic identification on the image to be detected to obtain airspace characteristic information of photoresponse non-uniformity noise and frequency domain characteristic information of the photoresponse non-uniformity noise in the image to be detected; and then, inputting the spatial domain characteristic information and the frequency domain characteristic information into an image tampering detection network to perform image tampering detection to obtain image detection information corresponding to the image to be detected, and performing image tampering detection by increasing characteristic dimensions and utilizing dual characteristics of the spatial domain and the frequency domain of the light response non-uniform noise, so that the accuracy of tampering detection on the image can be greatly improved.

Drawings

In order to more clearly illustrate the technical solutions and advantages of the embodiments of the present application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative efforts.

FIG. 1 is a schematic diagram of an application environment provided by an embodiment of the present application;

fig. 2 is a schematic flowchart of an image tampering detection method provided in an embodiment of the present application;

FIG. 3 is a schematic flowchart of a process for acquiring an image to be detected according to an embodiment of the present disclosure;

fig. 4 is a schematic flow chart illustrating that noise characteristic identification is performed on an image to be detected to obtain spatial domain characteristic information of photoresponse non-uniform noise and frequency domain characteristic information of photoresponse non-uniform noise in the image to be detected according to the embodiment of the present application;

fig. 5 is a schematic flow chart illustrating a spatial domain feature extracting method for performing photoresponse non-uniform noise on an image to be detected to obtain spatial domain feature information according to an embodiment of the present application;

fig. 6 is a schematic flowchart of a process of inputting spatial domain characteristic information and frequency domain characteristic information into an image tampering detection network for image tampering detection to obtain image detection information corresponding to an image to be detected according to an embodiment of the present application;

FIG. 7 is a schematic diagram of an image tamper detection network according to an embodiment of the present disclosure;

fig. 8 is a schematic structural diagram of an image tampering detection network according to an embodiment of the present application;

fig. 9 is a schematic flowchart of a network training method according to an embodiment of the present application;

fig. 10 is a block diagram illustrating an image tampering detection apparatus according to an embodiment of the present disclosure;

fig. 11 is a schematic structural diagram of an image tampering detection device according to an embodiment of the present application.

Detailed Description

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

It should be noted that the terms "comprises" and "comprising," and any variations thereof, in the description and claims of this application and the above-described drawings, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or server that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.

It is understood that in the specific implementation of the present application, related data such as user information, when the above embodiments of the present application are applied to specific products or technologies, user permission or consent needs to be obtained, and the collection, use and processing of related data need to comply with related laws and regulations and standards of related countries and regions.

Referring to fig. 1, fig. 1 is a schematic diagram of an application environment provided by an embodiment of the present application, where the application environment may include a client 10 and a server 20, and the client 10 and the server 20 may be connected directly or indirectly through wired or wireless communication. The user can send an image tampering detection request to the server 20 through the client 10. The server 20 determines a corresponding image to be detected based on the image tampering detection request, then performs noise characteristic identification on the image to be detected to obtain spatial domain characteristic information of photoresponse non-uniform noise and frequency domain characteristic information of the photoresponse non-uniform noise in the image to be detected, inputs the spatial domain characteristic information and the frequency domain characteristic information into an image tampering detection network to perform image tampering detection to obtain image detection information corresponding to the image to be detected, and returns the image detection information to the client 10. It should be noted that fig. 1 is only an example.

The client may be a smart phone, a computer (e.g., a desktop computer, a tablet computer, a notebook computer), a digital assistant, an intelligent voice interaction device (e.g., a smart speaker), an intelligent wearable device, or other types of physical devices, or may be software running in the physical devices, such as a computer program. The operating system corresponding to the client may be an Android system (Android system), an iOS system (mobile operating system developed by apple inc.), a Linux system (one operating system), a Microsoft Windows system (Microsoft Windows operating system), and the like.

The server side may be an independent physical server, a server cluster or a distributed system formed by a plurality of physical servers, or a cloud server providing basic cloud computing services such as a cloud service, a cloud database, cloud computing, a cloud function, cloud storage, a Network service, cloud communication, a middleware service, a domain name service, a security service, a CDN (Content Delivery Network), a big data and artificial intelligence platform, and the like. Which may include a network communication unit, a processor, and memory, among others. The server side can provide background services for the corresponding client side.

The client 10 and the server 20 may be used to construct a system related to image tamper detection, which may be a distributed system. Taking a distributed system as an example of a blockchain system, the blockchain system is formed by a plurality of nodes (computing devices in any form in an access network, such as servers and user terminals) and a client, a Peer-To-Peer (P2P, Peer To Peer) network is formed between the nodes, and the P2P Protocol is an application layer Protocol operating on top of a Transmission Control Protocol (TCP). In a distributed system, any machine, such as a server or a terminal, can join to become a node, and the node comprises a hardware layer, a middle layer, an operating system layer and an application layer.

The functions of each node in the above-mentioned blockchain system include:

1) routing, a basic function that a node has, is used to support communication between nodes.

Besides the routing function, the node may also have the following functions:

2) the application is used for being deployed in a block chain, realizing specific services according to actual service requirements, recording data related to the realization functions to form recording data, carrying a digital signature in the recording data to represent a source of task data, and sending the recording data to other nodes in the block chain system, so that the other nodes add the recording data to a temporary block when the source and integrity of the recording data are verified successfully.

3) And the Block chain comprises a series of blocks (blocks) which are mutually connected according to the generated chronological order, new blocks cannot be removed once being added into the Block chain, and recorded data submitted by nodes in the Block chain system are recorded in the blocks.

The following describes a specific embodiment of an image tampering detection method provided by the present application, and fig. 2 is a schematic flowchart of an image tampering detection method provided by the embodiment of the present application, and the present application provides the method operation steps described in the embodiment or the flowchart, but may include more or less operation steps based on conventional or non-inventive labor. The order of steps recited in the embodiments is merely one manner of performing the steps in a multitude of sequences, and does not represent a unique order of performance. In actual system or product execution, sequential execution or parallel execution (e.g., parallel processor or multi-threaded environment) may be possible according to the embodiments or methods shown in the figures. Specifically, as shown in fig. 2, the method may include:

s201, acquiring an image to be detected.

In the embodiment of the present specification, the image to be detected may be any image that needs to be subjected to tamper detection.

In practical application, size data of images from different sources may be different, and in order to ensure accuracy of an image tampering detection result, an initial image to be detected is usually preprocessed to obtain an image to be detected with a preset size, and specifically, the preset size may be set in combination with precision requirements of image tampering detection in practical application. In order to avoid damage to features between pixels of an image, which may be caused when the image is subjected to scaling processing, in the embodiment of the present specification, an image to be detected may be acquired by performing cropping preprocessing on an initial image to be detected.

In some embodiments, as shown in fig. 3, fig. 3 is a schematic flowchart of a process for acquiring an image to be detected according to an embodiment of the present disclosure, which may specifically include:

s301, acquiring an initial image to be detected.

Specifically, the size data of the initial image to be detected is generally not smaller than the preset size.

S302, performing center cutting processing on the initial image to be detected to obtain the image to be detected.

In a specific embodiment, the center cropping processing may be center cropping of different channel data of the image according to the channel data of the image, and the specific cropping area may include:

startXpos＝(srcWidth–dstWidth)/2；

startYpos＝(srcHeight–dstHeight)/2；

endXpos＝startXpos+dstWidth；

endYpos＝startYpos+dstHeight；

wherein, (srcpidth, srcpight) is the width and height of the initial image to be detected, (dscpidth, dstHeight) is the width and height of the image to be detected, (startXpos, endXpos) is the starting position and the ending position in the horizontal direction of the cutting area, and (startYpos, endYpos) is the starting position and the ending position in the vertical direction of the cutting area.

The embodiment shows that the center cutting processing is carried out on the initial image to be detected to obtain the image to be detected, the size consistency of the detected image can be ensured, the damage to the characteristics among the pixels of the image possibly caused by image processing modes such as zooming processing and the like can be avoided, the accuracy of noise characteristic extraction is improved, and the accuracy of tampering the detection result is improved.

S202, performing noise characteristic identification on the image to be detected to obtain spatial domain characteristic information of the photoresponse non-uniform noise and frequency domain characteristic information of the photoresponse non-uniform noise in the image to be detected.

In practical applications, the photoresponse non-uniformity noise is a noise disturbance introduced into the digital image due to hardware manufacturing defects of the camera sensor CCD (Charge Coupled Device). The existing CCD manufactured in the industry has process problems, so that the thickness of a semiconductor silicon material of each pixel of the CCD is inconsistent, small difference can be generated among different pixels when a camera senses light, and the small difference is brought into each pixel of a digital image to form weak noise. Therefore, in the embodiment of the present specification, the spatial domain feature information and the frequency domain feature information of the photoresponse non-uniform noise are obtained by performing the feature identification of the photoresponse non-uniform noise on the image to be detected, so that the image tampering detection of the image to be detected is performed based on the spatial domain feature information and the frequency domain feature information.

Specifically, the spatial domain characteristic information may represent a spatial domain characteristic of the photo-response non-uniform noise, and the frequency domain characteristic information may represent a frequency domain characteristic of the photo-response non-uniform noise.

In this embodiment of the present specification, as shown in fig. 4, the performing noise feature identification on the image to be detected to obtain spatial domain feature information of the photoresponse non-uniform noise and frequency domain feature information of the photoresponse non-uniform noise in the image to be detected may include:

s401, performing spatial domain feature extraction of the photoresponse non-uniform noise on the image to be detected to obtain spatial domain feature information.

In the embodiment of the present specification, the spatial domain feature information may be represented in the form of a spatial domain feature image.

In a specific embodiment, as shown in fig. 5, the performing spatial domain feature extraction of the photoresponse non-uniform noise on the image to be detected to obtain spatial domain feature information may include:

s501, respectively extracting high-frequency components of a plurality of color channels of an image to be detected to obtain high-frequency component data of the plurality of color channels.

Specifically, the high-frequency component data may include: a horizontal direction high frequency component, a vertical direction high frequency component, and a diagonal direction high frequency component.

In a specific embodiment, the extracting high-frequency components of a plurality of color channels of an image to be detected respectively to obtain high-frequency component data of the plurality of color channels may include: and respectively carrying out multi-scale wavelet transformation processing on a plurality of color channels of the image to be detected to obtain wavelet domain high-frequency component data of a plurality of scales corresponding to each color channel. Alternatively, the multi-scale wavelet transform process may be a 4-level wavelet transform, and the wavelet basis may be db4 (a 4 th order multi-bayesian wavelet).

And S502, respectively carrying out noise intensity analysis on the high-frequency component data of the multiple color channels to obtain noise intensity data.

Specifically, local noise variance analysis is respectively carried out on the horizontal direction high-frequency component, the vertical direction high-frequency component and the diagonal direction high-frequency component of each color channel to obtain target variance data of the horizontal direction high-frequency component, target variance data of the vertical direction high-frequency component and target variance data of the diagonal direction high-frequency component; the target variance data of the horizontal direction high frequency component, the target variance data of the vertical direction high frequency component, and the target variance data of the diagonal direction high frequency component are taken as the noise intensity data.

In an optional embodiment, the local noise variance analysis may include performing noise variance analysis for a plurality of window sizes on each type of high-frequency component data, to obtain variance data corresponding to the plurality of window sizes, and selecting a minimum value of the variance data corresponding to the plurality of window sizes as target variance data of each type of high-frequency component data. Preferably, the plurality of windows may include: 3. 5, 7, 9, the initial standard deviation in the local noise variance analysis process may be 5.

And S503, respectively carrying out filtering processing on the high-frequency component data of the multiple color channels based on the noise intensity data to obtain initial noise information of the multiple color channels.

In a specific embodiment, in the case that the noise intensity data is target variance data of a high frequency component of each color channel, the filtering, based on the noise intensity data, the high frequency component data of a plurality of color channels, respectively, to obtain initial noise information of the plurality of color channels may include: and respectively carrying out filtering processing on the high-frequency component of each color channel by using the target variance data of the high-frequency component of each color channel to obtain the initial noise information of each color channel.

Optionally, the filter used in the filtering process may include, but is not limited to: wiener filters, kalman filters, and the like.

In a specific embodiment, in the case where the high frequency component extraction is a multi-scale wavelet transform process, the initial noise information is

S504, the initial noise information of the multiple color channels is reconstructed, and initial spatial domain feature information is obtained.

In a specific embodiment, in a case that the extracting of the high-frequency component is multi-scale wavelet transform processing, the initial noise information may be wavelet domain noise information, and correspondingly, reconstructing the initial noise information of a plurality of color channels to obtain initial spatial domain feature information may include: and performing inverse wavelet transform processing on the wavelet domain noise information of the multiple color channels to obtain spatial domain noise information of the image to be detected, and taking the spatial domain noise information as initial spatial domain characteristic information of the optical response non-uniform noise.

And S505, performing noise enhancement processing on the initial spatial domain characteristic information to obtain spatial domain characteristic information.

In a specific embodiment, the performing noise enhancement processing on the initial spatial domain feature information may obtain the spatial domain feature information, and the obtaining the spatial domain feature information may include:

1) and carrying out zero-averaging filtering processing on the initial airspace characteristic information to obtain the filtered initial airspace characteristic information.

Specifically, the interference information in the initial spatial domain feature information is removed through zero-averaging filtering processing, so as to enhance the photoresponse non-uniform noise.

2) And performing Fourier peak suppression processing on the initial spatial domain characteristic information after filtering processing to obtain spatial domain characteristic information.

In practical applications, the light response non-uniformity noise can be enhanced by removing the relatively harsh interference pixels in the image from the dimension of the fourier frequency domain through the fourier peak suppression process.

According to the embodiments, the initial noise information is obtained by filtering the high-frequency component data of the image to be detected, the initial noise information is reconstructed to obtain the initial airspace characteristic information, and then the noise enhancement processing is performed on the initial airspace characteristic information to obtain the airspace characteristic information, so that the interference can be effectively removed, and the accuracy of airspace characteristic extraction is improved.

S402, performing space-frequency transformation processing on the space-domain characteristic information to obtain frequency-domain characteristic information.

In the embodiment of the present specification, the representation of the frequency domain feature information may be a frequency domain feature image.

In a specific embodiment, the performing space-frequency transform on the spatial domain characteristic information to obtain frequency domain characteristic information may include:

1) and performing discrete Fourier transform processing on the spatial domain characteristic information to obtain initial frequency domain characteristic information.

2) And carrying out frequency spectrum centralization processing on the initial frequency domain characteristic information to obtain frequency domain characteristic information.

Specifically, in the case that the initial frequency domain feature information is an initial frequency domain feature image, the direct current components in the initial frequency domain feature image are usually located in four vertex angle regions of the image, and the alternating current components in the initial frequency domain feature image are usually located in the remaining region of the image.

In an alternative embodiment, the initial frequency domain feature image is divided based on the horizontal symmetry axis and the vertical symmetry axis of the image to obtain 4 regions, and the two regions at opposite corners of the image are interchanged respectively, so as to convert the dc component to the center of the image.

According to the embodiments, the discrete Fourier transform processing is performed on the spatial domain characteristic information to obtain the initial frequency domain characteristic information, and the frequency spectrum centralization processing is performed on the initial frequency domain characteristic information to obtain the frequency domain characteristic information, so that the accuracy of frequency domain characteristic extraction can be effectively improved, and the accuracy of the detection result is improved.

S203, inputting the spatial domain characteristic information and the frequency domain characteristic information into an image tampering detection network to carry out image tampering detection, and obtaining image detection information corresponding to the image to be detected.

In this embodiment of the present description, the image tampering detection network may be obtained by performing image tampering detection training on a preset image tampering detection network based on sample spatial domain characteristic information and sample frequency domain characteristic information, and specifically, the image tampering detection network may include: the system comprises a spatial domain characteristic aggregation layer, a frequency domain characteristic aggregation layer, a characteristic fusion layer and a tampering detection layer.

In a specific embodiment, as shown in fig. 6, the inputting the spatial domain characteristic information and the frequency domain characteristic information into an image tampering detection network for performing image tampering detection to obtain image detection information corresponding to an image to be detected may include:

s601, inputting the spatial domain characteristic information into a spatial domain characteristic aggregation layer to carry out spatial domain characteristic aggregation processing, and obtaining target spatial domain characteristic information.

Specifically, the target spatial domain feature information may characterize a spatial domain aggregation feature of the photoresponse non-uniformity noise. The expression form of the target spatial domain characteristic information can be a target spatial domain characteristic image.

In an alternative embodiment, the spatial signature aggregation layer may include: a first Convolution layer, a first batch normalization layer, a first activation layer and at least one MBConv (Mobile Inverted bottle Convolution) layer.

And S602, inputting the frequency domain characteristic information into a frequency domain characteristic aggregation layer to perform frequency domain characteristic aggregation processing, so as to obtain target frequency domain characteristic information.

Specifically, the target frequency domain characteristic information may characterize a frequency domain aggregation characteristic of the optical response nonuniformity noise. The representation form of the target frequency domain characteristic information can be a target frequency domain characteristic image.

In an alternative embodiment, the frequency domain feature aggregation layer may include: a second convolutional layer, a second batch normalization layer, a second active layer, and at least one MBConv (Mobile invoked bottle convolutional) layer.

In practical application, the spatial domain feature aggregation layer and the frequency domain feature aggregation layer may have the same or different structures, but the sizes of the target spatial domain feature image output by the spatial domain feature aggregation layer and the target frequency domain feature image output by the frequency domain feature aggregation layer should be kept consistent, so as to facilitate subsequent feature fusion.

And S603, inputting the target spatial domain characteristic information and the target frequency domain characteristic information into the characteristic fusion layer for characteristic fusion processing to obtain target characteristic fusion information.

Specifically, the target frequency domain characteristic information may characterize a spatial domain aggregation characteristic and a frequency domain aggregation characteristic of the photoresponse non-uniformity noise.

In an optional embodiment, the feature fusion layer may be a channel fusion layer, and the inputting the target spatial domain feature information and the target frequency domain feature information into the feature fusion layer for feature fusion processing to obtain the target feature fusion information may include: and inputting the target spatial domain characteristic information and the target frequency domain characteristic information into a channel fusion layer, and overlapping and fusing the target spatial domain characteristic information and the target frequency domain characteristic information according to an image channel to obtain target characteristic fusion information.

Specifically, the channel fusion layer may perform superposition fusion on the target spatial domain feature information and the target frequency domain feature information from the dimension of the image channel. For example, the target spatial domain feature information is a spatial domain feature map with a size of 7 × 7 × 512, and the target frequency domain feature information is a frequency domain feature map with a size of 7 × 7 × 512, where 7 × 7 is spatial resolution and 512 is the number of channels; correspondingly, the overlaying and fusing the target spatial domain characteristic information and the target frequency domain characteristic information from the dimension of the image channel may include: and superposing the space domain characteristic diagram and the frequency domain characteristic diagram according to image channels to obtain a 7 multiplied by 1024 fusion characteristic diagram, and taking the fusion characteristic diagram as target characteristic fusion information.

In another alternative embodiment, the feature fusion layer may include: the above-mentioned inputting the target spatial domain feature information and the target frequency domain feature information into the feature fusion layer for feature fusion processing to obtain the target feature fusion information may include: inputting the target airspace feature information into a feature vector generation layer to perform airspace feature vector generation processing to obtain an airspace feature vector; inputting the target frequency domain feature information into a feature vector generation layer to perform frequency domain feature vector generation processing to obtain a frequency domain feature vector; inputting the space domain feature vector and the frequency domain feature vector into a feature vector fusion layer to perform feature vector fusion to obtain a target feature vector; and taking the target feature vector as target feature fusion information.

Specifically, the feature vector generation layer may be a global average pooling layer or a full-link layer, and feature compression may be performed on the target airspace feature information and the target frequency domain feature information through the feature vector generation layer, so as to obtain an airspace feature vector and a frequency domain feature vector; through the feature vector fusion layer, the space domain feature vector and the frequency domain feature vector can be subjected to point-by-point addition or point-by-point multiplication from the dimension of the feature vector to obtain a target feature vector.

The embodiment can provide various feature fusion modes, and can select a feature fusion method for fusing image channels or fusing images corresponding to feature vectors aiming at different types of images to be detected, so that the characterization accuracy of the noise features of the images to be detected by the target feature fusion information is improved.

S604, inputting the target feature fusion information into a tampering detection layer to carry out image tampering detection, and obtaining image detection information.

In the embodiment of the present specification, the image detection information may be used to characterize whether the image to be detected is tampered. Specifically, the image detection information may be a detection label corresponding to the image to be detected, where the detection label may be a real image label or a tamper detection label.

In a particular embodiment, the tamper detection layer may include a third convolutional layer, a global average pooling layer, a fully-connected layer, and an output layer.

Specifically, the third convolution layer may perform convolution processing on the input target feature fusion information, so as to implement feature extraction on the target feature fusion information.

Specifically, the global average pooling layer may perform a down-sampling operation on the output of the previous layer, that is, return the maximum value in the sampling window as the down-sampled output. On one hand, the image can be reduced, and the calculation complexity is simplified; on the other hand, feature compression can be carried out to extract main features.

Specifically, the full connection layer may be used as a connection layer between nodes of the upper layer and the lower layer, and a connection relationship is established between data of each node obtained by the upper layer and the lower layer. The full connection layer can perform feature compression processing on the target feature fusion information to obtain feature information to be detected.

Specifically, the classification layer may perform image tampering detection on the feature information to be detected, and output a corresponding detection label. In a specific embodiment, the classification layer may output the target knowledge point tag by using an activation function, and in an optional embodiment, the activation function may be a Softmax function, and the Softmax function includes a nonlinear classifier for performing image tampering detection on the feature information to be detected.

In addition, it should be noted that the image tampering detection network described in the embodiment of this specification is not limited to the preset image tampering detection network, and in practical applications, the image tampering detection network may further include other machine learning networks, such as a decision tree machine learning network, for example, and the embodiment of this application is not limited to the preset image tampering detection network.

In a specific embodiment, as shown in fig. 7, an image tampering detection network is established that includes the spatial domain feature aggregation layer, the frequency domain feature aggregation layer, the feature fusion layer, and the tampering detection layer, and spatial domain feature information and frequency domain feature information of photoresponse non-uniform noise in the image to be detected are input into the image tampering detection network for image tampering detection, so as to obtain image detection information corresponding to the image to be detected.

In a specific embodiment, as shown in fig. 8, fig. 8 is a schematic structural diagram of an image tampering detection network provided in the embodiment of the present application, specifically, the image tampering detection network may include: the system comprises a spatial domain characteristic aggregation layer, a frequency domain characteristic aggregation layer, a characteristic fusion layer and a tampering detection layer; the spatial domain feature aggregation layer may include: a first convolution layer, a first batch normalization layer, a first activation layer and 5 MBConv layers; the frequency domain feature aggregation layer may include: a second convolution layer, a second batch normalization layer, a second active layer and 5 MBConv layers; the tamper detection layer may include: a third convolutional layer, a global average pooling layer, a fully connected layer, and a classification layer.

In addition, it should be noted that, in the embodiment of the present application, the hierarchical structures of the spatial domain feature aggregation layer and the frequency domain feature aggregation layer are not fixed, and may be designed arbitrarily, but the sizes of the target spatial domain feature image output by the spatial domain feature aggregation layer and the target frequency domain feature image output by the frequency domain feature aggregation layer should be kept consistent, so as to facilitate subsequent feature fusion.

According to the embodiment, the image tampering detection is performed by utilizing the spatial domain characteristic aggregation layer, the frequency domain characteristic aggregation layer, the characteristic fusion layer and the tampering detection layer, so that the tampering detection adaptability to different images can be improved, and the accuracy of the tampering detection on the images can be greatly improved.

In the embodiment of the description, the preset image tampering detection network can be trained through the sample detection image, so that the image tampering detection network is obtained.

In a specific embodiment, as shown in fig. 9, fig. 9 is a schematic flow chart of a network training method provided in the embodiment of the present application, which specifically includes:

s901, obtaining a sample detection image and preset image detection information corresponding to the sample detection image.

In practical application, before network training, training data may be determined, and specifically, in the embodiment of the present application, a sample detection image including preset image detection information may be acquired as the training data.

Specifically, the preset image detection information may be a preset detection label pre-labeled to the sample detection image. In this embodiment of the present specification, the preset detection tag may include a real image tag or a tampered image tag, the sample detection image may include a sample real image and a sample tampered image, and accordingly, the preset detection tag of the sample real image may be the real image tag, and the preset detection tag of the sample tampered image may be the tampered image tag. Generally, the proportion of the real image of the sample and the tampered image of the sample in the training data can be preset in combination with the actual requirement of the network training precision.

In a specific embodiment, the acquiring the sample detection image may include:

1) an initial sample test image is acquired.

2) And performing center cutting processing on the initial sample detection image to obtain a sample detection image.

In practical application, by performing cutting preprocessing on the initial sample detection image, the characteristics of photoresponse non-uniformity noise among image pixels can be prevented from being damaged while ensuring that the image data of a subsequently input preset image tampering detection network has the same size, and in addition, the time consumed by training and predicting can be further reduced.

S902, carrying out noise feature identification on the sample detection image to obtain sample space domain feature information of the sample photoresponse non-uniform noise and sample frequency domain feature information of the sample photoresponse non-uniform noise in the sample detection image.

And S903, inputting the sample spatial domain characteristic information and the sample frequency domain characteristic information into a preset image tampering detection network to carry out image tampering detection, and obtaining sample image detection information corresponding to the sample detection image.

And S904, determining target loss information based on the preset image detection information and the sample image detection information.

S905, training a preset image tampering detection network based on the target loss information to obtain the image tampering detection network.

In an alternative embodiment, the sample image detection information may include a sample detection label of the sample detection image, and correspondingly, the target loss information may include a detection label loss;

accordingly, the determining the target loss information based on the preset image detection information and the sample image detection information may include:

and determining the loss of the detection label according to the preset detection label and the sample detection label.

In a specific embodiment, the determining of the loss of the detection label according to the preset detection label and the sample detection label may include determining the loss of the detection label between the preset detection label and the sample detection label based on a preset loss function.

In a particular embodiment, the loss of the detection tag can be indicative of a difference between the predetermined detection tag and the sample detection tag.

In a particular embodiment, the pre-set penalty function may include, but is not limited to, a cross-entropy penalty function, a logic penalty function, an exponential penalty function, and the like.

In an optional embodiment, training a preset image tampering detection network based on the target loss information, and obtaining the image tampering detection network may include:

s9051, updating network parameters of the preset image tampering detection network based on the target loss information;

s9052, based on the updated preset image tampering detection network, repeatedly executing image tampering detection training iteration operations including the steps S903, S904 and S9051 until an image tampering detection convergence condition is reached;

and S9053, taking the preset image tampering detection network obtained under the condition that the convergence condition of the image tampering detection is reached as the image tampering detection network.

In an alternative embodiment, the reaching of the convergence condition for image tampering detection may be that the number of training iterations reaches a preset number of training. Optionally, the convergence condition of the image tamper detection may be reached by setting the target loss information to be smaller than a specified threshold. In the embodiment of the present specification, the preset training times and the specified threshold may be preset in combination with the training speed and accuracy of the network in practical application.

In a specific embodiment, the preset image tampering detection network may include a preset airspace feature aggregation layer, a preset frequency domain feature aggregation layer, a preset feature fusion layer, and a preset tampering detection layer, and correspondingly, the inputting the sample airspace feature information and the sample frequency domain feature information into the preset image tampering detection network to perform image tampering detection to obtain sample image detection information corresponding to the sample detection image may include:

inputting the sample airspace characteristic information into a preset airspace characteristic aggregation layer to carry out airspace characteristic aggregation processing to obtain sample target airspace characteristic information; inputting the sample frequency domain characteristic information into a preset frequency domain characteristic aggregation layer to carry out frequency domain characteristic aggregation processing to obtain sample target frequency domain characteristic information; inputting the sample target airspace characteristic information and the sample target frequency domain characteristic information into a preset characteristic fusion layer for characteristic fusion processing to obtain sample target characteristic fusion information; and inputting the sample target feature fusion information into a preset tampering detection layer to carry out image tampering detection, so as to obtain sample image detection information.

As can be seen from the above embodiments, on one hand, based on machine learning training of the sample detection image and the corresponding preset detection label, the generalization capability and robustness of the image tampering detection network are improved, so that the accuracy of the network in detecting image tampering can be better improved.

According to the technical scheme provided by the embodiment of the application, on the one hand, in the scene of tampering detection of the image, the center cutting processing is carried out on the initial image to be detected to obtain the image to be detected, so that the consistency of the sizes of the detected image can be ensured, the damage to the characteristics among the pixels of the image possibly caused by image processing modes such as zooming processing and the like can be avoided, and the accuracy of noise characteristic identification can be improved; on the other hand, initial noise information is obtained by filtering high-frequency component data of an image to be detected, reconstruction processing is carried out on the initial noise information to obtain initial airspace characteristic information, and then noise enhancement processing is carried out on the initial airspace characteristic information to obtain airspace characteristic information, so that interference can be effectively removed, and the accuracy of airspace characteristic extraction is improved; on the other hand, the space domain characteristic information is subjected to discrete Fourier transform processing to obtain initial frequency domain characteristic information, and the initial frequency domain characteristic information is subjected to frequency spectrum centralization processing to obtain frequency domain characteristic information, so that the accuracy of frequency domain characteristic extraction can be effectively improved; on the other hand, the image tampering detection is carried out by utilizing the airspace characteristic aggregation layer, the frequency domain characteristic aggregation layer, the characteristic fusion layer and the tampering detection layer, so that the tampering detection adaptability to different images can be improved, and the accuracy of the tampering detection on the images can be greatly improved; on the other hand, based on machine learning training of the sample detection image and the corresponding preset detection label, the generalization capability and robustness of the image tampering detection network are improved, and therefore the accuracy of the network in image tampering detection can be better improved.

An embodiment of the present application further provides an image tampering detection device, as shown in fig. 10, the image tampering detection device may include:

the to-be-detected image acquisition module 1010 is used for acquiring an image to be detected;

the noise feature identification module 1020 is configured to perform noise feature identification on the image to be detected to obtain spatial domain feature information of the photoresponse non-uniform noise and frequency domain feature information of the photoresponse non-uniform noise in the image to be detected;

and the image tampering detection module 1030 is configured to input the spatial domain feature information and the frequency domain feature information into an image tampering detection network to perform image tampering detection, so as to obtain image detection information corresponding to the image to be detected.

In some embodiments, the image acquiring module 1010 to be detected may include:

the initial image acquisition unit to be detected is used for acquiring an initial image to be detected;

and the center cutting processing unit is used for performing center cutting processing on the initial image to be detected to obtain the image to be detected.

In this embodiment, the noise characteristic identification module 1020 may include:

the spatial domain feature extraction unit is used for performing spatial domain feature extraction of the photoresponse non-uniform noise on the image to be detected to obtain spatial domain feature information;

and the space-frequency transformation processing unit is used for carrying out space-frequency transformation processing on the space-domain characteristic information to obtain frequency-domain characteristic information.

In a specific embodiment, the spatial domain feature extraction unit may include:

the high-frequency component extraction unit is used for respectively extracting high-frequency components of a plurality of color channels of an image to be detected to obtain high-frequency component data of the plurality of color channels;

the noise intensity analysis unit is used for respectively carrying out noise intensity analysis on the high-frequency component data of the multiple color channels to obtain noise intensity data;

the filtering processing unit is used for respectively filtering the high-frequency component data of the multiple color channels based on the noise intensity data to obtain initial noise information of the multiple color channels;

the reconstruction processing unit is used for reconstructing the initial noise information of the multiple color channels to obtain initial airspace characteristic information;

and the noise enhancement processing unit is used for carrying out noise enhancement processing on the initial airspace characteristic information to obtain the airspace characteristic information.

In a specific embodiment, the image tampering detection module 1030 may include:

the spatial domain characteristic aggregation processing unit is used for inputting the spatial domain characteristic information into a spatial domain characteristic aggregation layer to carry out spatial domain characteristic aggregation processing so as to obtain target spatial domain characteristic information;

the frequency domain characteristic aggregation processing unit is used for inputting the frequency domain characteristic information into the frequency domain characteristic aggregation layer to carry out frequency domain characteristic aggregation processing to obtain target frequency domain characteristic information;

the characteristic fusion unit is used for inputting the target airspace characteristic information and the target frequency domain characteristic information into the characteristic fusion layer to carry out characteristic fusion processing to obtain target characteristic fusion information;

and the image tampering detection unit is used for inputting the target characteristic fusion information into the tampering detection layer to carry out image tampering detection so as to obtain image detection information.

In an alternative embodiment, the feature fusion unit may include:

and the stacking fusion unit is used for inputting the target airspace characteristic information and the target frequency domain characteristic information into the channel fusion layer, and stacking and fusing the target airspace characteristic information and the target frequency domain characteristic information according to the image channel to obtain target characteristic fusion information.

In a specific embodiment, the apparatus may further include:

the sample acquisition module is used for acquiring a sample detection image and preset image detection information corresponding to the sample detection image;

the sample noise characteristic identification module is used for carrying out noise characteristic identification on the sample detection image to obtain sample space domain characteristic information of the sample photoresponse non-uniform noise and sample frequency domain characteristic information of the sample photoresponse non-uniform noise in the sample detection image;

the sample image detection information module is used for inputting the sample airspace characteristic information and the sample frequency domain characteristic information into a preset image tampering detection network to carry out image tampering detection so as to obtain sample image detection information corresponding to the sample detection image;

the target loss information determining module is used for determining target loss information based on preset image detection information and sample image detection information;

and the network training module is used for training a preset image tampering detection network based on the target loss information to obtain the image tampering detection network.

It should be noted that the device and method embodiments in the device embodiment are based on the same inventive concept.

The embodiment of the application provides an image tampering detection device, which comprises a processor and a memory, wherein at least one instruction or at least one program is stored in the memory, and the at least one instruction or the at least one program is loaded and executed by the processor to implement the image tampering detection method provided by the above method embodiment.

Further, fig. 11 shows a schematic hardware structure of an image tampering detection device for implementing the image tampering detection method provided in the embodiment of the present application, where the image tampering detection device may participate in constituting or including the image tampering detection apparatus provided in the embodiment of the present application. As shown in fig. 11, image tamper detection device 110 may include one or more (shown as 1102a, 1102b, … …, 1102 n) processors 1102 (processor 1102 may include, but is not limited to, a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 1104 for storing data, and a transmission device 1106 for communication functions. In addition, the method can also comprise the following steps: a display, an input/output interface (I/O interface), a Universal Serial Bus (USB) port (which may be included as one of the ports of the I/O interface), a network interface, a power source, and/or a camera. It will be understood by those skilled in the art that the structure shown in fig. 11 is only an illustration and is not intended to limit the structure of the electronic device. For example, the image tamper detection device 110 may also include more or fewer components than shown in FIG. 11, or have a different configuration than shown in FIG. 11.

It should be noted that the one or more processors 1102 and/or other data processing circuitry described above may be referred to generally herein as "data processing circuitry". The data processing circuitry may be embodied in whole or in part in software, hardware, firmware, or any combination thereof. Further, the data processing circuit may be a single, stand-alone processing module, or incorporated in whole or in part into any of the other elements in the image tamper detection device 110 (or mobile device). As referred to in the embodiments of the application, the data processing circuit acts as a processor control (e.g. selection of a variable resistance termination path connected to the interface).

The memory 1104 can be used for storing software programs and modules of application software, such as program instructions/data storage devices corresponding to the image tampering detection method described in the embodiment of the present application, and the processor 1102 executes various functional applications and data processing by running the software programs and modules stored in the memory 1104, so as to implement the image tampering detection method described above. The memory 1104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 1104 may further include a memory remotely located from the processor 1102, and such remote memory may be connected to the image tamper detection device 110 over a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.

The transmission device 1106 is used for receiving or transmitting data via a network. Specific examples of the network described above may include a wireless network provided by a communication provider of the image tampering detection device 110. In one example, the transmission device 1106 includes a network adapter (NIC) that can be connected to other network devices through a base station to communicate with the internet. In one embodiment, the transmission device 1106 may be a Radio Frequency (RF) module, which is used for communicating with the internet in a wireless manner.

The display may be, for example, a touch screen type Liquid Crystal Display (LCD) that may enable a user to interact with a user interface of the image tamper detection device 110 (or mobile device).

Embodiments of the present application further provide a computer-readable storage medium, where the storage medium may be disposed in an image tampering detection apparatus to store at least one instruction or at least one program for implementing an image tampering detection method in the method embodiments, where the at least one instruction or the at least one program is loaded and executed by the processor to implement the image tampering detection method provided in the method embodiments.

Alternatively, in this embodiment, the storage medium may be located in at least one network server of a plurality of network servers of a computer network. Optionally, in this embodiment, the storage medium may include, but is not limited to: various media capable of storing program codes, such as a usb disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a removable hard disk, a magnetic disk, or an optical disk.

Embodiments of the present application also provide a computer program product or computer program comprising computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions to cause the computer device to perform the image tampering detection method as provided by the method embodiments.

It should be noted that: the sequence of the embodiments of the present application is only for description, and does not represent the advantages and disadvantages of the embodiments. And specific embodiments thereof have been described above. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims may be performed in a different order than in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing may also be possible or may be advantageous.

The embodiments in the present application are described in a progressive manner, and the same and similar parts among the embodiments can be referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, for the apparatus and device embodiments, since they are substantially similar to the method embodiments, the description is relatively simple, and reference may be made to some descriptions of the method embodiments for relevant points.

It will be understood by those skilled in the art that all or part of the steps for implementing the above embodiments may be implemented by hardware, or may be implemented by a program instructing relevant hardware, where the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a magnetic disk or an optical disk, etc.

The above description is only exemplary of the present application and should not be taken as limiting the present application, as any modification, equivalent replacement, or improvement made within the spirit and principle of the present application should be included in the protection scope of the present application.

Claims

1. An image tampering detection method, characterized in that the method comprises:

acquiring an image to be detected;

2. The method according to claim 1, wherein the image tampering detection network comprises a spatial domain feature aggregation layer, a frequency domain feature aggregation layer, a feature fusion layer and a tampering detection layer, and the step of inputting the spatial domain feature information and the frequency domain feature information into the image tampering detection network for image tampering detection to obtain the image detection information corresponding to the image to be detected comprises:

inputting the airspace characteristic information into the airspace characteristic aggregation layer to carry out airspace characteristic aggregation processing to obtain target airspace characteristic information;

inputting the frequency domain characteristic information into the frequency domain characteristic aggregation layer to carry out frequency domain characteristic aggregation processing to obtain target frequency domain characteristic information;

inputting the target airspace characteristic information and the target frequency domain characteristic information into the characteristic fusion layer for characteristic fusion processing to obtain target characteristic fusion information;

and inputting the target feature fusion information into the tampering detection layer to carry out image tampering detection, so as to obtain the image detection information.

3. The method according to claim 1, wherein the performing noise feature recognition on the image to be detected to obtain spatial domain feature information of photoresponse non-uniformity noise and frequency domain feature information of the photoresponse non-uniformity noise in the image to be detected comprises:

performing airspace feature extraction of the photoresponse non-uniform noise on the image to be detected to obtain the airspace feature information;

and performing space-frequency transformation processing on the space domain characteristic information to obtain the frequency domain characteristic information.

4. The method according to claim 2, wherein the feature fusion layer is a channel fusion layer, and the inputting the target spatial domain feature information and the target frequency domain feature information into the feature fusion layer for feature fusion processing to obtain target feature fusion information comprises:

inputting the target airspace characteristic information and the target frequency domain characteristic information into the channel fusion layer, and stacking and fusing the target airspace characteristic information and the target frequency domain characteristic information according to image channels to obtain the target characteristic fusion information.

5. The method according to claim 3, wherein the spatial domain feature extraction of the photoresponse non-uniform noise on the image to be detected to obtain the spatial domain feature information comprises:

respectively extracting high-frequency components of a plurality of color channels of the image to be detected to obtain high-frequency component data of the plurality of color channels;

respectively analyzing the noise intensity of the high-frequency component data of the multiple color channels to obtain noise intensity data;

based on the noise intensity data, respectively carrying out filtering processing on the high-frequency component data of the multiple color channels to obtain initial noise information of the multiple color channels;

reconstructing the initial noise information of the multiple color channels to obtain initial airspace characteristic information;

and carrying out noise enhancement processing on the initial airspace characteristic information to obtain the airspace characteristic information.

6. The method according to any one of claims 1 to 5, wherein the acquiring the image to be detected comprises:

acquiring an initial image to be detected;

and performing center cutting processing on the initial image to be detected to obtain the image to be detected.

7. The method of any of claims 1 to 5, further comprising:

acquiring a sample detection image and preset image detection information corresponding to the sample detection image;

carrying out noise feature identification on the sample detection image to obtain sample space domain feature information of sample photoresponse non-uniform noise and sample frequency domain feature information of the sample photoresponse non-uniform noise in the sample detection image;

inputting the sample airspace characteristic information and the sample frequency domain characteristic information into a preset image tampering detection network for image tampering detection to obtain sample image detection information corresponding to the sample detection image;

determining target loss information based on the preset image detection information and the sample image detection information;

and training the preset image tampering detection network based on the target loss information to obtain the image tampering detection network.

8. The method according to claim 7, wherein the preset image tampering detection network includes a preset spatial domain feature aggregation layer, a preset frequency domain feature aggregation layer, a preset feature fusion layer, and a preset tampering detection layer, and the inputting the sample spatial domain feature information and the sample frequency domain feature information into the preset image tampering detection network for image tampering detection to obtain sample image detection information corresponding to the sample detection image includes:

inputting the sample airspace feature information into the preset airspace feature aggregation layer to carry out airspace feature aggregation processing to obtain sample target airspace feature information;

inputting the sample frequency domain characteristic information into the preset frequency domain characteristic aggregation layer to carry out frequency domain characteristic aggregation processing to obtain sample target frequency domain characteristic information;

inputting the sample target airspace characteristic information and the sample target frequency domain characteristic information into the preset characteristic fusion layer for characteristic fusion processing to obtain sample target characteristic fusion information;

and inputting the sample target feature fusion information into the preset tampering detection layer for image tampering detection to obtain the sample image detection information.

9. An image tamper detection apparatus, characterized in that the apparatus comprises:

10. An image tamper detection device, characterized in that the device comprises a processor and a memory, wherein at least one instruction or at least one program is stored in the memory, and the at least one instruction or the at least one program is loaded and executed by the processor to implement the image tamper detection method according to any one of claims 1 to 8.

11. A computer-readable storage medium, wherein at least one instruction or at least one program is stored in the storage medium, and the at least one instruction or the at least one program is loaded and executed by a processor to implement the image tampering detection method according to any one of claims 1 to 8.

12. A computer program product comprising at least one instruction or at least one program, the at least one instruction or the at least one program being loaded and executed by a processor to implement the image tamper detection method according to any one of claims 1 to 8.