KR20200046182A - Deep-running-based image correction detection system and method for providing non-correction detection service using the same - Google Patents

Abstract

According to an embodiment, a deep learning-based image correction detection system is a deep learning-based image correction detection system for discriminating forgery and alteration in an input image. The deep learning-based image correction detection system comprises: a specialized convolutional neural network for detecting correction feature information in the pixel domain of the input image; a Markov statistics based network that detects whether the input image is compressed after the input image is converted into a frequency domain and correction feature information in the compressed image; an integrated feature refinement unit that combines the correction feature information of the specialized convolutional neural network with the correction feature information of the Markov statistics based network; and a correction discrimination unit for discriminating forgery and alteration of the input image based on the correction feature information combined by the integrated feature refinement unit.

Description

Deep-running-based image correction detection system and method for providing non-correction detection service using the same}

The present invention relates to a deep learning-based image correction detection system and a method for providing a correction-free detection service using the same. More specifically, the present invention relates to a correction detection system that accurately determines whether an image is corrected based on deep learning, and a method of providing a service for detecting whether there is no correction using such a system.

With the development of digital cameras and mobile phones, anyone can create high-quality digital images, and in particular, images can be distributed to others by uploading and sharing images under the influence of social network services (SNS).

In addition, sophisticated image editing software such as Adobe PhotoShop, social network service's own image editing function, and the spread of multiple image editing applications have made it easy to modify the original image. In particular, image retouching techniques such as blurring, median filtering, Gaussian noise, resampling, cropping, and color modulation hide specific elements different from actual images, or Image forgery can occur, such as adding new elements by copy-move or splicing, and it is difficult to understand the integrity of the image without the help of experts. Serious cases can arise where the forged image can be distributed like the original image and the forged image can be recognized as court evidence by omitting the forgery trail.

With the rapid spread of image editing software, anyone can easily produce and distribute images, especially forgery and alteration of images. Techniques for detecting such forgery and alteration have been continuously studied from the past, but there are disadvantages of operating only in very limited environments such as a specific file format, manipulation, and compression quality.

Image forensic techniques have been studied to detect such forgery and alteration.

Conventional techniques have continuously studied techniques for detecting color modulation by analyzing pattern changes, and techniques for analyzing traces of resampling of images.

However, these techniques have limitations that are difficult to apply to all of the various manipulations and formats because they operate only in different operations, file formats, and specific environments.

Meanwhile, among the deep learning technologies, the convolutional neural network showed excellent performance in solving computer vision problems, and studies were conducted to detect various image manipulations using the convolutional neural network.

In detail, the convolutional neural network (CNN) showed excellent performance in solving computer vision problems such as object recognition and distinction.

However, if the convolutional neural network is applied to the image forensic as it is, it does not show good learning performance, and prior art 1 has identified the need to perform a pre-processing process before using the convolutional neural network.

In particular, Prior Art 1 designed a layer specialized at the front of the convolutional neural network, showing an overwhelming image manipulation detection performance compared to the existing prior arts.

1 is a deep learning neural network structure of prior art 1, characterized by including a bayar filter specialized in an image forensic technique at the very first layer of a convolutional neural network. This Bayard filter improves image manipulation detection performance by allowing the convolutional neural network to learn more adaptively to image changes with only the layer structure without any pre-processing or feature extraction. Ordered.

In detail, the Bayar filter, as in Equation 1 above, in this filter, the central weight value is fixed to -1, the sum of neighbor weight values is forced to be 1, and learning proceeds. Here, l and m denote pixel coordinates.

The neural network of the prior art 1 using this Bayard filter very well detected changes such as blur, noise, median, and resizing in the uncompressed image.

However, the prior art 1 has a disadvantage in that it is difficult to trace the manipulated traces in the compressed image due to data loss caused by the compression process.

In addition, the prior art 1 is specialized in a grayscale image, and there is a disadvantage in that it is difficult to detect a manipulated trace in a color image.

In other words, prior art 1 targets only uncompressed images and does not take into account the compression that is essentially experienced in the process of generating and distributing images, and that most of the images are compressed, and most digital images are color images. Considering the point, there is a difficult problem in using the actual technology.

(Prior Art 1) Belhassen Bayar and Matthew C Stamm. “A deep learning approach to universal image manipulation detection using a new convolutional layer. In Proceedings of the 4th ACM Workshop on Information Hiding and Multimedia Security ”

Currently, most digital images are stored in a compressed format such as the JPEG format, and in the compressed image, manipulation marks are difficult to track due to data loss during compression.

In particular, image forgery is recompressed, that is, double JPEG compression is inevitably caused because the image stored in the JPAK compression format is elaborated in software such as Photoshop and then re-save the image. However, there is a problem in that the manipulated traces are more lost.

Accordingly, an object of the present invention is to provide a deep learning-based image correction detection system capable of detecting image manipulation in a compressed environment.

In addition, an object of the present invention is to provide a deep learning-based image correction detection system capable of accurately detecting image manipulation on a color image.

In addition, an object of the present invention is to provide a method for providing an uncorrected detection service using the precise deep learning-based image correction detection system.

A deep learning-based image correction detection system according to an embodiment includes: a deep learning-based image correction detection system for determining forgery and alteration in an input image, a specialized convolutional neural network for detecting correction feature information in the pixel domain of the input image; A Markov statistics-based network that detects whether the input image is compressed in the frequency domain and whether it is compressed and correction characteristic information in the compressed image; An integrated feature refiner that combines the correction feature information of the specialized convolutional neural network with the correction feature information of the Markov characteristic based network; And a correction discrimination unit for discriminating forgery and alteration of the input image based on the correction feature information combined by the integrated feature refinement unit.

In this case, an image block unit for blocking the input image to a standard size may be further included.

In addition, the specialized convolutional neural network may include a correction feature pre-processing unit, a correction feature extraction unit, and a first feature purification unit.

In addition, the correction feature extraction unit includes a plurality of layers, and each layer may include a convolutional layer, a batch normalization, and a rectified linear unit (ReLU) function.

In this case, the size of all the convolutional layers included in the plurality of layers may be 5 × 5 or less, and the stride may be 1.

In addition, the Markov characteristic-based network may include a domain transform unit, a pixel difference calculator, a threshold function, a matrix transform unit, and a second feature refiner.

In this case, the domain transform unit may process the image block blocked by the image block unit by discrete cosine transform (DCT) to process data in the frequency domain.

In addition, the integrated feature refiner comprises a fully connected layer, a vector representing correction characteristic information of the specialized convolutional neural network, and a vector representing correction feature information of the Markov characteristic based network. It can have a structure that combines and delivers to a classifier.

In addition, the correction discrimination unit may detect pixels with correction possibility, calculate a correction probability value of the pixels, and output a forgery confirmation map displaying pixels with high probability of forgery in different colors according to the probability value.

A method of providing an uncorrected detection service according to an embodiment includes a method of providing an uncorrected detection service using the deep learning-based image correction detection system, comprising: receiving an input image for determining forgery and alteration from a user; Generating an image block by blocking the input image to a certain standard; Inputting the image block into the deep learning based image correction detection system, and outputting a forgery confirmation map from the deep learning based image correction detection system; And deep-learning the forgery confirmation map and the input image to output whether the forgery is falsified with a label of yes or no.

At this time, when it is determined whether the forgery is falsified, the method may further include generating an input image highlighting a region having high possibility of forgery and providing the generated input image to the user.

In addition, when it is determined that the falsification is No, the method may further include synthesizing a confirmation stamp that there is no correction in the input image and providing it to the user.

Deep learning based image correction detection according to an embodiment. The image manipulation can be well detected in a compressed environment by combining a convolutional neural network that detects image manipulation in various compression environments and a neural network based on Markov's characteristics considering compression.

In addition, the deep learning-based image correction detection according to the embodiment. In addition to being able to detect image manipulation occurring in various compression environments, it is possible to quickly and strongly detect forgery and alteration of color images.

In addition, the method for providing the uncorrected detection service according to the embodiment can accurately determine whether to correct the color image even if it is a compressed image, and provide the determined result so that the user can intuitively recognize it. There is an advantage.

1 shows the convolutional neural network structure of prior art 1.
2 shows a structural diagram of a deep learning-based image correction detection system according to a first embodiment of the present invention.
3 is a detailed structural diagram of a specialized convolutional neural network according to a first embodiment of the present invention.
4 is a detailed structural diagram of a Markov network according to a first embodiment of the present invention.
5 shows an experimental result of detecting a forged image by a deep learning-based image correction detection system according to a first embodiment of the present invention.
6A to 6D are graphs comparing the detection accuracy of the deep learning based image correction detection system and the detection accuracy of the prior art 1 according to the first embodiment of the present invention in different environments.
7 is a block diagram of a deep learning-based image correction detection system according to a second embodiment of the present invention.
8 is a detailed structural diagram of a deep learning-based image correction detection system according to a second embodiment of the present invention.
9A to 9C are graphs comparing image manipulation detection accuracy of different neural network systems for images of different resizing ratios.
10 is a flowchart for explaining a method of providing an uncorrected detection service according to an embodiment of the present invention.
11A illustrates a service in which an uncompensated detection service according to an embodiment of the present invention takes an uncompensated confirmation stamp on an image when there is no correction of an image.
FIG. 11B illustrates a service in which an uncorrected detection service according to an embodiment of the present invention highlights a suspected correction area when a correction of an image is detected.

The present invention can be applied to various transformations and can have various embodiments, and specific embodiments will be illustrated in the drawings and described in detail in the detailed description. Effects and features of the present invention and methods for achieving them will be clarified with reference to embodiments described below in detail with reference to the drawings. However, the present invention is not limited to the embodiments disclosed below, but may be implemented in various forms. In the following examples, terms such as first and second are not used in a limiting sense, but for the purpose of distinguishing one component from other components. In addition, a singular expression includes a plural expression unless the context clearly indicates otherwise. In addition, terms such as include or have means that a feature or component described in the specification exists, and does not preclude the possibility of adding one or more other features or components in advance. In addition, in the drawings, the size of components may be exaggerated or reduced for convenience of description. For example, since the size and thickness of each component shown in the drawings are arbitrarily shown for convenience of description, the present invention is not necessarily limited to what is shown.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings, and the same or corresponding components will be denoted by the same reference numerals when describing with reference to the drawings, and redundant description thereof will be omitted. .

<Terminal>

The neural network structure of the deep learning-based image correction detection system according to the embodiment is created through a computer language. Accordingly, the terminal is installed in the memory, and the processor reads the computer language through RAM, so that deep learning-based image correction detection can be performed through the processor. Likewise, a method for providing a correction-free detection service according to an embodiment may also be implemented through execution of a correction-free detection program in a processor.

Therefore, the subject driving the deep learning-based image correction detection system is the processor of the terminal.

The terminal of the deep learning-based image correction detection system according to the embodiment may include a processor that processes data and a memory that stores an image deep learning driving program for performing image deep learning, and the processor is the image deep By reading the running drive program, image deep learning described below is performed according to the established neural network system.

According to an embodiment, the deep learning-based image correction detection system is configured to include a main processor that controls all units and a graphic processing unit (GPU) that processes calculations required when driving a neural network according to image deep learning. Can be.

And the terminal according to the embodiment, the server computer, computer, smart phone, digital broadcasting terminal, the mobile phone, PDA (personal digital assistants), PMP (portable multimedia player), navigation, tablet PC with the deep learning-based image correction detection program installed (tablet PC), a wearable device may be included.

A program for providing a deep learning-based image correction detection program and an uncorrected detection service according to an embodiment is provided through a network, and a final service may be provided through the terminal through data exchange between the terminal and the server.

< First embodiment-Deep learning based image correction detection system>

2 shows a structural diagram of a deep learning-based image correction detection system according to a first embodiment of the present invention.

Referring to FIG. 2, the deep learning-based image correction detection system 10 according to the first embodiment has a dual stream network structure to detect each operation for image manipulations occurring in various compression environments. The convolutional neural network 100 specialized in the image forensic technique, the Markov characteristics-based neural network 200 considering whether or not the image is compressed, and the image correction characteristics extracted from the neural network are refined and determined to be manipulated. It includes a correction feature refining unit 310 and the correction discrimination unit 320.

One. Dual neural network network structure and input data

The structure of the two stream neural network of the deep learning based image correction detection system 10 according to the embodiment is shown in FIG. 2.

In detail, the deep learning-based image correction detection system 10 includes a specialized convolutional neural network 100 for detecting image manipulation and a neural network 200 based on Markov statistics considering compression. It is a dual network structure combining.

This dual neural network structure can consider whether the image is manipulated in the frequency domain, unlike the existing research that was considered only in the pixel domain, in order to solve problems that were difficult to directly apply the existing convolutional neural network to the digital image forensic technique. , It can be specialized in detecting whether or not the compressed image is manipulated at least once.

In an embodiment, an input image target to detect image manipulation is to distinguish and detect a normal image block and a forged image block in a JPEG compressed image. The actual image forgery and alteration occurs when the JPEG compressed image is loaded through the editing software and then locally stored in the image for re-storage and distribution. Especially, when re-storing the image, the JPEG is also compressed, so the manipulated area and normal people The regions are all double compressed, so it is necessary to distinguish the two regions. Among the types of image forgery mentioned above, copy-paste and splicing inevitably perform blurring or median filtering to hide traces during manipulation, and resampling also occurs for the naturalness of pasted objects. Therefore, hereinafter, it is limited to the object of detecting an image manipulation in an image that is compressed by JPEG.

2. Specialized Convolutional Neural Network (100)

2 to 3, the specialized convolutional neural network 100 according to an embodiment includes an image block unit 110, a correction feature pre-processing unit 120, a correction feature extraction unit 130, and a first feature refinement It may include a portion 140.

In detail, the image block unit 110 may block the input image to a size suitable for inputting the image in the convolutional neural network. For example, the image block unit 110 may crop the image so that the image block is 64x64.

Next, the blocked image block is input to the correction feature preprocessing unit 120.

In detail, the correction feature preprocessing unit 120 includes a specialized constrained convolution layer. The specialized convolution layer may operate as shown in Equation 1 in the same manner as the Bayard filter of Prior Art 1.

Next, the pre-processed image processed and output by the correction feature pre-processing unit 120 may be input to the correction feature extraction unit 130.

The correction feature extracting unit 130 includes a plurality of layers, and each layer includes a convolutional layer, a batch normalization, a rectified linear unit (ReLU) function, and max pooling layer).

In detail, referring to FIG. 3, including the first layer 131, the second layer 132, and the third layer 133, the convolution layer 135, the batch normalization 136, and the rectified linear, respectively It can be seen that the unit function 137 is included, and the first layer 131 and the second layer 132 further include a max pooling layer 138.

That is, the feature extraction unit 130 includes a plurality of convolutional layers. In order to stack the plurality of layers, it is preferable that all of the convolution layers included in the feature extraction unit 130 are 5 × 5 or less. .

In particular, it is preferable that the convolution layer accumulated after the second is 3 × 3 or less. In addition, the stride of each convolution layer is fixed to 1 because the trace of manipulation can be missed, and only the pooling layer can be set to an integer of 2 or more.

In detail, the reason for using a small filter size for the convolution layer is because it is possible to use several rectified linear unit (ReLU) functions first. This makes it possible to replace multiple layers using small filters instead of one layer using large filters. Second, it reduces the number of weights to be learned. As in the prior art 1, it is clear that the number of weights is less when using 3 3 × 3 convolutional layers as in the embodiment rather than using one 7 × 7 convolutional layer. It can have a big advantage.

In addition, we used batch normalization to prevent the proposed network from overfitting.

The correction feature extracted through the feature extraction unit 130 may be transmitted to the first feature purification unit 140.

The first feature refiner 140 may be trained to refine and distinguish the extracted image correction features, including at least one fully connected layer.

In an embodiment, the first feature refining unit 140 may be composed of two fully connected layers 141 and 142 and rectified linear unit functions at the output terminals of each fully connected layer.

The revised feature refined by the first feature refining unit 140 is transmitted to the integrated feature refining unit 310 together with the revised feature extracted from the Markov characteristic-based network 200.

3. Network based on Markov statistics

The Markov characteristic-based network 200 according to an embodiment may distinguish between single-JPEG compression and double-JPEG compression, and detect whether an image is deformed when compressing a double-JPEG. have.

The Markov characteristic-based network 200 can effectively detect image deformation in a J-pack format compressed image in consideration of whether or not the image is manipulated in the frequency domain.

2 and 4, the Markov characteristic-based network 200 according to the embodiment includes a domain converter 210, a pixel difference calculator 220, a threshold function 230, and a matrix converter 240 ) And the second feature refining unit 250.

In detail, the domain converter 210 performs discrete cosine transform (DCT) on the image block blocked by the image block unit 110 to process the data in the frequency domain. In detail, the domain converter 210 performs a discrete cosine transform (DCT) on the input image block every 8 × 8 blocks to process the data in the frequency domain.

In addition, the pixel difference calculator 220 may obtain the arrays Bh and Bv in which the difference between neighboring pixels is obtained in the horizontal and vertical directions, respectively, as shown in Equation 2 below for blocks Bx and y subjected to discrete cosine transformation.

Next, the threshold function 230 maps the array values into the threshold range [-4, 4].

Next, the matrix transform unit 240 obtains a 9 × 9 transition probability matrix (TPM) of the horizontal block Bh as the value of the corresponding block, as shown in Equation 3 below, and calculates two transition probability matrices for horizontal and vertical [ 1, 9 × 9 × 2] Combined into a one-dimensional vector.

Thereafter, the combined one-dimensional vector is delivered to the second feature refinement unit 250, and the second feature delivery unit may be configured as a fully connected layer.

The Markov characteristic-based network 200 is an image forensic technique that is transformed into a network suitable for a dual network, transforms and analyzes single and double J-pack compressed images into a frequency domain, thereby transforming images in the compressed image. Can be accurately detected.

4. Integrated feature refining unit 310 and correction discrimination unit 320

Finally, each correction feature information output from the specialized convolutional neural network 100 and the Markov statistics based network is transmitted to the integrated feature refiner 310.

The integrated feature refining unit 310 is composed of the complete connection layers 311 and 312, and combines the feature information to finally try to learn for classification.

The integrated feature refinement unit 310 according to the embodiment has a structure in which vectors of the two complete layers are combined and transferred to a classifier rather than a method of performing an average to prevent data loss of minute characteristics for an image forensic.

The correction feature information combined by the integrated feature refining unit 310 is transmitted to the correction discrimination unit 320.

The correction discrimination unit 320 may detect pixels with correction possibility, calculate a correction probability value of the pixels, and extract them as a forgery confirmation map. That is, as shown in FIG. 5, a forgery confirmation map displaying pixels with high probability of forgery in different colors according to a probability value may be output.

The correction discrimination unit 320, a softmax function, and an Adam optimizer capable of escaping from a local minima faster than a stochastic gradient descent that has been widely used in the past. It includes.

In detail, referring to FIG. 5, the deep learning-based image correction detection system 10 according to the embodiment, when a forged image (Figure (a)) is input from the original image, the forged region is shown in Figure (b). Similarly, the modulation probability value for each pixel can also be displayed to output the detected result.

Deep learning based image correction detection designed in this way. The image manipulation can be well detected even in a compressed environment by combining the convolutional neural network that detects image manipulation in various compression environments and the neural network 200 based on the Markov characteristic considering compression.

In detail, Table 1 is a table showing the deep learning network (bayas) of the prior art 1 and the detection rate of each image forgery of the network of the first embodiment.

As can be seen from Table 1, the two networks conducted experiments to detect the deformation in a double compressed image for a total of four image transformation types (Gaussian blurring, Gaussian noise, median filtering, and resampling).

, It can be confirmed through Table 1 that the network of Example 1 recorded all advantages over the Bayar technique of Prior Art 1 for all variations and various compression qualities (Q1 = 70, 90 / Q2 = 60, 70, 80, 90). have.

Also, referring to FIGS. 6A to 6D, it can be confirmed through a detection accuracy graph that the network of Example 1 always records a higher detection rate than Prior Art 1 for each learning section for each image transformation type.

< Second embodiment-Deep learning based image correction detection system>

Hereinafter, the deep learning-based image correction detection system 20 according to the second embodiment will be described, and a description overlapping with the first embodiment may be omitted.

Referring to FIG. 7, the deep learning-based image correction detection system 20 according to the second embodiment includes an image blocking unit 410, a correction feature pre-processing unit 420, a correction feature extraction unit 430, and a correction feature refinement It includes a section 441 and a correction discrimination section 442.

In detail, referring to FIGS. 7 to 8, the image blocker 410 may adjust the size of the input image to a size suitable for input to a convolutional neural network.

The deep learning-based image correction detection system 20 according to the present embodiment is suitable for analyzing a color image, and the input image is a color image and may be composed of a plurality of layers by differently configuring layers for each color.

In an embodiment, the image blocker 410 may crop the image so that the image block is 256x256x3.

Next, the blocked image may be transferred to the correction feature pre-processing unit 420.

The correction feature pre-processing unit 420 may enlarge the correction feature of the image using a high-pass filter.

In detail, the correction feature pre-processing unit 420 may include a high-pass filter to highlight a resize trace feature.

The pre-processing unit for the correction feature is considered to be a method for finding hidden stego information in an image, similar to an image forensic technique.

The image block in which the resize feature is highlighted may be input to the image correction feature extraction unit 430.

The correction feature extraction unit 430 may extract an image correction feature using a pre-trained deep learning convolutional neural network model.

In detail, the correction feature extraction unit 430 may include a plurality of convolutional layers, a batch normalization, a rectified linear unit (ReLU) function, and a plurality of max pooling layers. have.

The correction feature extraction unit 430 according to the embodiment may be a pre-trained VGG19 convolutional neural network model. For example, the correction feature extraction unit 430 may have a structure in which two convolution layers, a pooling layer, two convolution layers, a pooling layer, four convolution layers, a pooling layer, and two convolution layers are sequentially stacked. .

In this case, the correction feature extracting unit 430 may be a pre-trained convolutional neural network model and a model in which the weight is changed for each input image.

In detail, the weight (w) of the pre-trained VGG19 convolutional neural network according to the embodiment may be modified for each image to be trained to extract the forgery-feature feature (VGG feature).

In addition, since the plurality of convolution layers included in the correction feature extraction unit 430 are stacked as a plurality of layers, it is preferable that all of the convolution layers have a size of 3 × 3 or less. In addition, the stride of each convolution layer is fixed to 1 because the trace of manipulation can be missed, and only the pooling layer can be set to an integer of 2 or more.

Then, the correction feature information extracted by the correction feature extraction unit 430 is input to the feature refinement unit.

The feature refiner 441 may be trained to refine and distinguish the image correction features extracted using a fully-connected layer.

In detail, the feature refiner 441 may be trained to refine and distinguish the extracted image correction features, including at least one fully connected layer.

In an embodiment, the feature refining unit 441 may be composed of two fully connected layers and a rectified linear unit function at the output terminals of each fully connected layer.

The correction feature information combined in the feature refining portion 441 is transmitted to the correction discriminating portion 442.

The correction discrimination unit 442 may detect pixels with correction possibility, and calculate a correction probability value of the pixels.

In addition, the correction discrimination unit 442 may indicate whether the image block is corrected through a softmax function with a probability between 0 and 1.

In detail, the correction discrimination unit 442 is a softmax function and an Adam optimizer that can escape from a local minima faster than a conventional stochastic gradient descent. optimizer).

Deep learning based image correction detection designed in this way. In addition to being able to detect image manipulation occurring in various compression environments, it is possible to quickly and strongly detect forgery and alteration of color images.

Hereinafter, in order to check the performance of the network according to the present embodiment, an image having a different resizing ratio from prior art 1 was input, and forgery extraction accuracy was calculated.

9A to 9C, it can be seen that the network according to the present embodiment is learned and operated faster than the network of the prior art 1 at all resizing ratios, and its accuracy is also high.

<Non-compensation detection service>

Hereinafter, a method of providing an uncorrected detection service using the deep learning-based image correction detection systems 10 and 20 according to the above-described embodiment will be described in detail.

Such a service may be provided through a user's terminal, and correction detection may be performed by a server for providing a correction-free detection service.

That is, the user may be provided with a service by accessing the service providing server through a terminal, uploading an image to be checked, and receiving the verified image from the service providing server.

Referring to FIG. 10, first, a user may input an input image I suspected of forgery and alteration through a terminal.

In detail, the user can request forgery verification by transmitting the image stored in the terminal to the service providing server.

The service providing server may block the input image received, generate it as an image block (IB), and input the generated image block (IB) to the deep learning-based image correction detection systems 10 and 20.

The deep learning-based image correction detection system 10 and 20 according to an embodiment deep-learns an input image block (IB), and a forgery confirmation map displaying a pixel for which correction is suspected in an image and a forgery probability value of the pixel (PM) can be extracted.

The service providing server may deep-learn the forgery confirmation map (PM) again through the convolutional neural network 30, and output whether the forgery or alteration is a yes / no label.

In this case, if it is determined that the forgery is falsified, the convolutional network may generate an input image highlighting the type of forgery and the area with high probability of forgery from the positions and arrangements of pixels having a high probability of forgery.

In more detail, referring to FIG. 11A, if it is determined that the forgery or alteration is NO, the service providing server may synthesize and provide a confirmation stamp Y that there is no correction in the input image I.

In addition, referring to FIG. 11B, when it is determined whether the forgery is falsified, the service providing server may highlight (N) an area of high probability of forgery in the input image I and further indicate forgery type information.

The method of providing the uncorrected detection service has the advantage of being able to accurately determine whether to correct the color image even in the case of a compressed image, and to provide the user with intuitive recognition of the determined result. .

The embodiments according to the present invention described above may be implemented in the form of program instructions that can be executed through various computer components and can be recorded in a computer-readable recording medium. The computer-readable recording medium may include program instructions, data files, data structures, or the like alone or in combination. The program instructions recorded on the computer-readable recording medium may be specially designed and configured for the present invention or may be known and usable by those skilled in the computer software field. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical recording media such as CD-ROMs and DVDs, and magneto-optical media such as floptical disks. medium), and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine language codes produced by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like. The hardware device can be changed to one or more software modules to perform the processing according to the present invention, and vice versa.

The specific implementations described in the present invention are examples, and do not limit the scope of the present invention in any way. For brevity of the specification, descriptions of conventional electronic configurations, control systems, software, and other functional aspects of the systems may be omitted. In addition, the connection or connection members of the lines between the components shown in the drawings are illustrative examples of functional connections and / or physical or circuit connections. In the actual device, alternative or additional various functional connections, physical It can be represented as a connection, or circuit connections. In addition, unless specifically mentioned, such as “essential”, “important”, etc., it may not be a necessary component for application of the present invention.

In addition, the detailed description of the present invention has been described with reference to preferred embodiments of the present invention. And it will be understood that various modifications and changes can be made to the present invention without departing from the technical scope. Therefore, the technical scope of the present invention should not be limited to the contents described in the detailed description of the specification, but should be defined by the claims.

Claims (12)

A deep learning-based image correction detection system that determines forgery and alteration in the input image,
A specialized convolutional neural network for detecting correction feature information in the pixel domain of the input image;
A Markov statistics-based network that detects whether the input image is compressed in the frequency domain and whether it is compressed and correction characteristic information in the compressed image;
An integrated feature refiner that combines the correction feature information of the specialized convolutional neural network with the correction feature information of the Markov characteristic based network; And
It includes; a correction discrimination unit for discriminating forgery and alteration of the input image based on the correction feature information combined by the integrated feature refinement unit
Deep learning based image correction detection system. According to claim 1,
Further comprising an image block unit for image blocking the input image to a standard size
Deep learning based image correction detection system. According to claim 1,
The specialized convolutional neural network,
A correction feature pre-processing unit, a correction feature extraction unit and a first feature purification unit
Deep learning based image correction detection system. The method of claim 3,
The correction feature extraction unit,
It includes a plurality of layers, and each layer includes a convolutional layer, a batch normalization, and a rectified linear unit (ReLU) function.
Deep learning based image correction detection system. The method of claim 4,
The size of all convolutional layers included in the plurality of layers is 5 × 5 or less, and the stride is 1 person.
Deep learning based image correction detection system. According to claim 1,
The Markov characteristic based network,
A domain conversion unit, a pixel difference calculation unit, a threshold function, a matrix conversion unit and a second feature refinement unit
Deep learning based image correction detection system. The method of claim 6,
The domain conversion unit,
Discrete cosine transform (DCT) of an image block blocked by an image block unit to process it as data in the frequency domain
Deep learning based image correction detection system. According to claim 1,
The integrated feature purification unit,
It consists of a fully connected layer,
The vector has the structure of combining the vector representing the correction characteristic information of the specialized convolutional neural network and the vector representing the correction characteristic information of the Markov characteristic-based network as it is and transmitting it to the classifier.
Deep learning based image correction detection system. According to claim 1,
The correction discrimination unit,
Detecting pixels with possible correction, calculating the probability of correction of the pixels, and outputting a forgery confirmation map displaying pixels with high probability of forgery in different colors according to the probability value
Deep learning based image correction detection system. A method for providing a correction-free detection service using the deep learning-based image correction detection system of claim 1,
Receiving an input image for determining forgery and alteration from the user;
Generating an image block by blocking the input image to a certain standard;
Inputting the image block into the deep learning based image correction detection system, and outputting a forgery confirmation map from the deep learning based image correction detection system; And
Including the step of deep learning the forgery check map and the input image to output whether or not the forgery or false label;
How to provide uncompensated detection services. The method of claim 10,
If it is determined whether the forgery is falsified, the method further includes generating an input image highlighting an area having high possibility of forgery and providing the generated input image to the user.
How to provide uncompensated detection services. The method of claim 10,
If it is determined that the falsification is No, further comprising the step of synthesizing a confirmation stamp that there is no correction in the input image and providing it to the user.
How to provide uncompensated detection services.

Images