CN115393698A

CN115393698A - Digital image tampering detection method based on improved DPN network

Info

Publication number: CN115393698A
Application number: CN202210979561.5A
Authority: CN
Inventors: 郭颖; 李海虎; 李晋宏
Original assignee: North China University of Technology
Current assignee: North China University of Technology
Priority date: 2022-08-16
Filing date: 2022-08-16
Publication date: 2022-11-25

Abstract

The invention discloses a digital image tampering detection method based on an improved DPN network, and relates to the field of digital image security. Training is carried out through an improved DPN neural network, a trained convolutional neural network is used for extracting tampering feature vectors, the probability of tampering the image is obtained through a full connection layer of the network, the type of the image to be detected is further judged, and finally, a tampering detection network model is evaluated. The invention combines the advantages of the traditional image processing method and the neural network, improves the existing deep learning detection method, and ensures that the network characteristic extraction is more sufficient, the network convergence is faster and the network prediction precision is higher.

Description

Digital image tampering detection method based on improved DPN network

Technical Field

The invention relates to the field of digital image security, in particular to a digital image tampering detection method based on an improved DPN network.

Background

Due to the popularity of the internet and the rapid development of multimedia technology, social media is becoming the current main information dissemination platform with its low cost and high profit, wherein digital images play an indispensable role in many fields with its characteristic of intuitively conveying information, but also bring about many troubles and challenges. On one hand, the popularity of image editing software and the maturity of digital image processing technology, the tampered image is difficult to be identified simply by human eyes by modifying the image content and performing post-processing on the tampered trace, and secondly, the development of multimedia technology and network communication is caused, the false image shows a trend of rising year by year, the potential safety hazard of the digital image is increasingly serious, the piracy problem and the information potential safety hazard caused by the potential safety hazard are social problems, and therefore, the problem of how to effectively identify the authenticity of the image becomes the problem which needs to be solved urgently at present.

For tampered images, there are two main categories: splicing, copying-pasting and deleting three types of image content modification, the tampering mode has larger misleading, and other modification modes such as blurring, compression and filtering mostly have smaller harmfulness of post-processing operation for covering tampering marks. Research shows that although some visual clues may not be left by image tampering, inherent statistical information of an image is changed in the tampering process, and the digital image tampering detection technology based on image content tampering feature extraction mainly analyzes inherent statistical information features of a digital image to identify authenticity and integrity of the image. At present, digital image tampering detection technologies based on image content can be mainly divided into two major categories, namely a passive evidence obtaining technology based on a traditional digital image processing method, and an image passive evidence obtaining technology based on deep learning. The traditional image tampering detection technology usually performs feature extraction by analyzing the uniqueness of tampering types and manually designing feature vectors to further perform image classification, but the accuracy of tampering detection is limited to a certain extent by the separation of the feature extraction and classification tasks while extra huge workload is brought by artificial design features. With the development of deep learning, the convolutional neural network makes great progress in the field of digital images. The digital image tampering detection technology based on deep learning enables a network model to automatically learn and extract effective features under the support of a large number of data sets by utilizing the adaptivity of a deep learning method. Early related scholars implemented end-to-end image tamper detection classification using convolutional neural networks. However, due to the particularity of the tampering features, image tampering classification needs to pay more attention to edge feature information of a network lower layer rather than semantic feature information of a network higher layer, and a traditional image classification network model cannot effectively solve the problems.

In the prior art, the traditional method artificially designs feature extraction, so that the workload of extracting features is large, and the separation detection accuracy of extracting and classifying tasks is low. Due to the particularity of the tampering features, the traditional deep learning classification network cannot effectively extract the tampering features, and the accuracy needs to be improved. The deep learning classification task focuses more on semantic information, and ignores low-level tampering edge characteristic information.

Therefore, those skilled in the art are devoted to developing a digital image tamper detection method based on an improved DPN network. The advantages of the traditional digital image processing method and the advantages of the deep learning method are combined, the problem of artificial feature extraction is effectively solved, and the image tampering detection network is more efficient and accurate.

Disclosure of Invention

In view of the above defects in the prior art, the technical problem to be solved by the invention is to liberate the artificial design of feature extraction features, realize end-to-end image self-adaptive feature extraction and tamper detection; aiming at the particularity of the tampering characteristics, the tampering characteristics are effectively and accurately extracted by combining deep learning with a traditional method; the accuracy of the image tampering detection task is improved by improving the DPN network to pay attention to the low-layer tampering characteristic information.

In order to achieve the above purpose, the present invention provides a digital image tamper detection method based on an improved DPN network, which includes the following steps:

step 1, preparing and preprocessing a data set;

step 2, building a tampering detection model, and building an improved DPN network model, wherein the DPN network model comprises a feature extraction link and a classification link;

step 3, training and adjusting parameters of the tampering detection model, loading a training set and a verification set, and initializing hyper-parameters;

and 4, testing and evaluating the tampering detection model, testing on the test set by loading the training weight, and finally evaluating the performance of the tampering detection model.

Further, in step 1, the data set is divided into a training set, a verification set and a test set.

Further, in step 1, the preprocessing includes fixing the image size, randomly turning horizontally, normalizing, and normalizing.

Further, in the step 2, a characteristic extraction link extracts the tampered characteristics.

Further, in the step 2, the classification link converts the tampering classification characteristics into corresponding class probabilities for discrimination.

Further, in step 2, information of the correlation change of neighboring pixels of the tampered edge is captured by a digital image processing technology SRM filter by using the fine boundary artifacts around the tampered region.

Further, in the step 3, the hyper-parameters include a learning rate, an optimizer, a loss function, batch-size, dropout, epoch, and a learning attenuation rate.

Further, step 1, fix the picture size to 256 × 256.

Further, step 1, converting the image into a tentor format.

Further, in step 1, the pictures are normalized by using normaize.

In a preferred embodiment of the invention, the invention provides a deep learning digital image tampering detection method based on improved DPN, which combines the advantages of the traditional digital image processing technology and deep learning, detects the image tampering type by preprocessing and automatically extracting tampering features, and has more sufficient model feature extraction and higher model detection accuracy.

According to the design scheme provided by the invention, a deep learning digital image tampering detection method based on an improved DPN network is provided, and comprises the following steps:

1. data set preparation and preprocessing: collecting the existing image tampering public data sets such as CASIA1.0 and CASIA2.0; dividing the training set, the verification set and the test set into a training set, a verification set and a test set according to a certain proportion; carrying out custom training on the public data set to make a tag and a corresponding numerical index; a common deep learning training data set processing mode is used in the model training data set loading process: the image size is fixed, and a series of processes such as random horizontal inversion, normalization, standardization and the like are performed, so that the data diversity is improved, and the convergence speed of the network is increased.

2. And (3) tamper detection model building: in the previous investigation, the detection and classification task of false image tampering can adopt a paradigm of network feature extraction and classifier. Both classical Resnet and Densenet are valid network references. The concrete points are as follows: resnet is essentially a multiplexing of extracted features in previous hierarchies, and tampering edge feature information extracted earlier is effectively reserved and extracted by adding Resnet branches. And new characteristics can be effectively explored for dense connection Densenet through dense connection paths. However, depending on the specificity of the task, not only the high-level semantic information of the tampered image is focused, but also information that the tampering of a certain pixel causes a change in the correlation between adjacent pixels needs to be captured. The invention combines the advantages of the two and is inspired by the DPN network, and an improved DPN network model is built, so that the model can effectively extract new features while multiplexing low-level edge feature information in the training process. As shown in fig. 2: the model framework comprises a feature extraction link and a classification link, wherein the feature extraction link mainly extracts the tampering features, and the classification link converts the tampering classification features into corresponding class probabilities for discrimination. The specific description is as follows:

by utilizing the fine boundary artifacts around the tampered region, the information of the correlation change of the adjacent pixels of the tampered edge is captured by a traditional digital image processing technology SRM filter, so that the noise information of the tampered image is also considered while the semantic information is considered in the model learning process. The concrete points are as follows: the method comprises the steps of preprocessing the SRM (-) by using a traditional image processing mode, updating weights in network training by packaging a learnable SRM convolution kernel, eliminating interference of image contents on steganographic feature information, restraining semantic information of an image so as to highlight falsification feature information of the image, and performing add operation with information of a subsequent RGB domain so that falsification feature capture is more sufficient.

In the feature extraction module, the improved DPN network model may be divided into five block modules, with the output of the previous block as the input of the next block. Each block is composed of two branches, respet and densenert, multiple units as shown in fig. 3. Through mutual learning and interaction among different modules of different branches in the model, the model is more sufficient and efficient in the feature extraction process. The method is mainly characterized in that the block consists of a series of convolution kernels with the convolution kernel size of 1x1,3x3, and new features are generated by carrying out dimension transformation and feature learning with different dimensions: fea _i ＝act(W _f *fea _i-1 +b _f ) Therein fea _i-1 For the output of the last convolution operation, W _f ,b _f Act is the characteristic weight, bias, and activation function of the convolution kernel, respectively.

For ResNet branch, sequentially performing dimension reduction, convolution operation and dimension increasing operation on input, and finally performing add operation on the obtained residual error characteristic output and input for multiple times. And (3) for the densenet branch, repeating the operation for multiple times by carrying out a series of Bach-Normalization, reLU, conv and other nonlinear transformations and finally carrying out cat operation on feature maps of different layers. The detailed parameters of each block of the model structure are shown in the following table:

in the classification link: the method comprises the steps of carrying out average pooling corresponding operation on tampering feature matrix information extracted based on an improved DPN network, utilizing a full connection layer of a neural network to obtain the probability that an image to be detected belongs to a tampering image, setting dropout to effectively relieve the occurrence of over simulation, achieving the regularization effect to a certain extent, improving the generalization capability of a network model, and finally judging the tampering type by utilizing probability scores.

3. Model training and parameter adjustment: loading a training set and a verification set, and initializing hyper-parameters: learning rate, optimizer, loss function, batch-size, dropout, epoch, learning attenuation rate and other series of hyper-parameters, and based on transfer learning, fine-tuning is performed on the task data set, so that the key tampering information features of the model are effectively and fully extracted, each layer of convolution weight is updated through forward transmission and backward gradient in the network training process until network training converges, the hyper-parameters are adjusted through verifying set training effect until the model training specifies the epoch, and training weight information reaching ideal effect is stored for subsequent testing.

4. Model testing and evaluation: and testing on the test set by loading the training weight, and finally evaluating the performance of the tampered model by corresponding data. Further shown in: for the classification module full-connection layer of the improved model, through a softmax function:

and calculating the fractional probability of the tampered and non-tampered images of the image to be detected, and reasonably evaluating the tampering detection model through corresponding evaluation indexes such as auc, acc and a confusion matrix.

The invention comprehensively utilizes the traditional digital image processing technology and the neural learning technology to carry out digital image tampering detection, effectively breaks through the limitation of manually designing feature extraction characteristics in the traditional detection to realize an end-to-end image tampering detection network, simultaneously avoids the phenomenon that the traditional classification neural network loses the low-level feature information of the local edge of the tampering characteristics due to much paying attention to the high-level semantic information, extracts more sufficient feature information based on the improved DPN neural network, pays attention to the image tampering region information, accelerates the convergence speed of the network, improves the detection classification capability of the network, and has better application prospect.

Compared with the prior art, the invention has the following obvious substantive characteristics and obvious advantages:

1. the traditional image processing method is combined with deep learning, and an end-to-end image tampering detection classification system is realized.

2. The method is improved on the basis of the DPN neural network, improves the network training efficiency, improves the detection and classification capability of the system, and has better application prospect.

The conception, the specific structure and the technical effects of the present invention will be further described with reference to the accompanying drawings to fully understand the objects, the features and the effects of the present invention.

Drawings

FIG. 1 is a schematic flow chart of a preferred embodiment of the present invention;

FIG. 2 is a diagram of an improved DPN deep learning convolutional neural network architecture in accordance with a preferred embodiment of the present invention;

fig. 3 is a schematic internal diagram of sub-module convx of the improved DPN network according to a preferred embodiment of the present invention.

Detailed Description

The technical contents of the preferred embodiments of the present invention will be more clearly and easily understood by referring to the drawings attached to the specification. The present invention may be embodied in many different forms of embodiments and the scope of the invention is not limited to the embodiments set forth herein.

In the drawings, structurally identical elements are represented by like reference numerals, and structurally or functionally similar elements are represented by like reference numerals throughout the several views. The size and thickness of each component shown in the drawings are arbitrarily illustrated, and the present invention is not limited to the size and thickness of each component. The thickness of the components has been exaggerated in some places in the drawings where appropriate for clarity of illustration.

The invention relates to a digital image tampering detection method, in particular to a DPN (dual path network) based improved deep learning digital image tampering detection method, which can be applied to the rapid and accurate identification of false tampered images and has a wide application prospect in the aspect of digital image security. Comprises the following steps: performing custom training on an existing public image tampering detection image dataset; an improved network model is constructed based on the design idea of the DPN neural network, different levels of feature information are fully utilized, training is carried out based on transfer learning, and key tampering feature information is obtained. And acquiring classification probability by using a full connection layer in the convolutional neural network, judging the tampering type of the image to be detected, and finally evaluating the model through a series of parameters.

Tamper detection network architecture: on the basis of a DPN network structure, the network is improved by combining the particularity of a tampering task, and the accuracy of tampering detection is effectively improved.

As shown in fig. 1, an improved DPN network-based deep learning end-to-end detection method is provided, which includes performing data enhancement on a tampered public data set, increasing data volume and diversity of training samples, and improving robustness of a model; processing the tampered image by using the traditional digital image processing technology to inhibit semantic information of the tampered image; and improving based on a DPN neural network, and realizing mutual interaction and mutual learning among different modules according to the particularity of the tampering characteristics. Training is carried out through initialization of a series of parameters of an initialized and improved DPN neural network, a trained convolutional neural network is used for extracting tampering feature vectors, and the probability of tampering images is obtained through a full connection layer of the network so as to judge the type of the images to be detected. And finally, evaluating the tampering detection network model by using the corresponding predicted data through the evaluation indexes such as auc (Area Under dark), accuracy, average accuracy and the like. The invention combines the advantages of the traditional image processing method and the neural network, improves the existing deep learning detection method on the existing basis, and ensures that the network feature extraction is more sufficient, the network convergence is quicker and the network prediction precision is higher. The specific implementation mode is as follows:

step 1: in the experiment, a distorted image detection network model is constructed by adopting a pytorch, all programs are written by using a python language, and the experiment is carried out in the environment of GTX3080Ti 12G GPU and 32GB RAM.

Step 2: the collection of the image tampering public data set is specifically represented by:

1) Collection of image-tampered common data set this example is a case 2.0 data set containing, among others, real images 7489, tampered images 5123, with a picture size of 640 x 480.

2) Invalid data in a data set are screened, and the data are filtered according to the following steps of 8:1:1 scale divides the data set into a training set, a validation set, and a test set.

3) And (3) making a custom training data set, generating a category name and a corresponding numerical index, and dividing the image label of the data set into two categories of 0 (tampered) and 1 (non-tampered) according to the task for subsequent processing.

And 3, step 3: loading the training set verification set data in the training model process and performing a series of preprocessing operations are specifically represented as follows:

1) Fix the picture size to 256 × 256 size;

2) The data diversity is improved by expanding the data set through random horizontal turning, and the robustness of the model is improved.

3) The data is normalized to convert the image into a tensor format, so that the subsequent training is facilitated;

4) The convergence speed of the network is increased using the normalization process.

And 4, step 4: the initialization of the hyper-parameters of the model training is specifically represented as follows:

1) The invention uses a crossEntropyLoss loss function;

2) Updating corresponding weight for the reverse gradient in the model training process by using an SGD optimizer to prevent partial optimization in the model training process, and respectively initializing corresponding parameters as follows: the learning rate is 0.001, the momentum =0.9, the weight attenuation weight =0.005, and the learning rate is linearly decreased, so that the learning speed of the network in the early stage of training is high, the learning speed in the later stage is low, and the overfitting phenomenon in the training process is inhibited.

3) Setting a network training period epoch =200, dropout =0.5, and adjusting the hyper-parameters according to the effect of the verification set in the training process.

And 5: the method comprises the steps of constructing an improved DPN image tampering detection model, wherein the improved DPN image tampering detection model comprises a feature extraction module and a classification module, the feature extraction module is composed of five blocks, each block comprises a respet branch and a densenet branch, the blocks are constructed by using a pitorch according to parameters in a table and a model structure diagram, activating functions all use Relu functions, and the classification module is composed of a full connection layer and an average pooling layer and the like, wherein the full connection layer and the average pooling layer are used for setting dropout, so that the generalization capability of the model is improved.

And loading pre-training model parameters into a model based on transfer learning, sending the training sample set into an improved DPN (distributed DPN network) model for training, performing add operation on the feature vector of an rgb domain and a high-frequency domain processed by an srm (self-defined convolution kernel) at a block1, and then sequentially extracting through a block module, so that the model can effectively extract tampered feature information.

And further, calculating the probability of tampering the detected image with each category through a softmax function in the full connection layer processing of the feature vector flattening network in the last step, and classifying the images to be detected in sequence. Continuously updating each convolution kernel parameter through the loss function back propagation in the training process until the network training is converged, and storing the corresponding model training weight.

Step 6: further, the weight of the training model in the above steps is loaded into the test set for verification, a confusion matrix is drawn to obtain the accuracy of model prediction, a classification evaluation index auc, and the network model is evaluated by corresponding average accuracy. The auc curve is defined as the area enclosed by the coordinate axis under the ROC curve, the classifier model has better effect when the numerical value is larger, acc is the prediction accuracy, and p _ ap and n _ ap are the average accuracy of positive and negative samples respectively. As shown in table 2:

model (model)	acc	auc	n_ap	p_ap
					DensetNet121	0.9605	0.9822	0.9756	0.9739
ResNet50	0.9589	0.985	0.9776	0.9539
					DPNet+	0.9645	0.994	0.9955	0.9856

The results of the steps are evaluated and analyzed, so that an improved digital image tampering detection network of the DPN network can be found, an end-to-end image tampering detection method is realized, the image to be detected can be quickly identified, the feature extraction is more efficient, and the identification precision is improved.

The foregoing detailed description of the preferred embodiments of the invention has been presented. It should be understood that numerous modifications and variations could be devised by those skilled in the art in light of the present teachings without departing from the inventive concepts. Therefore, the technical solutions that can be obtained by a person skilled in the art through logical analysis, reasoning or limited experiments based on the prior art according to the concepts of the present invention should be within the scope of protection determined by the claims.

Claims

1. A digital image tampering detection method based on an improved DPN network is characterized by comprising the following steps:

step 1, data set preparation and pretreatment;

and 4, testing and evaluating the tampering detection model, testing on a test set by loading the training weight, and finally evaluating the performance of the tampering detection model.

2. The digital image tampering detection method based on the improved DPN network as claimed in claim 1, wherein in step 1, the data set is divided into a training set, a verification set and a test set.

3. The digital image tamper detection method based on the improved DPN network of claim 1, wherein in the step 1, the preprocessing comprises fixing the image size, randomly turning the level, normalizing and standardizing.

4. The digital image tampering detection method based on the improved DPN network as claimed in claim 1, wherein in the step 2, the characteristic extraction step is used for extracting the tampering characteristics.

5. The digital image tampering detection method based on the improved DPN network as claimed in claim 1, wherein in the step 2, the classification step converts the tampering classification features into corresponding class probabilities for discrimination.

6. The digital image tampering detection method based on the improved DPN network as claimed in claim 1, wherein said step 2, using the subtle boundary artifacts around the tampered region, captures the information of the change of correlation of the neighboring pixels of the tampered edge by digital image processing technology (SRM) filter.

7. The digital image tampering detection method based on the improved DPN network, as claimed in claim 1, wherein the hyper-parameters in step 3 comprise learning rate, optimizer, loss function, batch-size, dropout, epoch, learning decay rate.

8. The improved DPN network-based digital image tamper detection method of claim 1, wherein step 1 fixes a picture size to 256 × 256.

9. The digital image tamper detection method based on the improved DPN network as claimed in claim 1, wherein step 1, the image is converted into a tentor format.

10. The digital image tampering detection method based on the improved DPN network as claimed in claim 1, wherein in step 1, the pictures are normalized by normaize.