KR102533984B1 - Advanced system and method for detecting video forgery - Google Patents

Abstract

The present invention relates to an improved manipulation image detection system and method. The disclosed improved manipulation image detection system and method is an improved manipulation image detection system that is robust against anti-forensic technology that hides whether or not the image has been manipulated. An improved manipulation image detection system and method comprising a non-learnable block that transforms into an arbitrary space in which learning is impossible and a detector that performs learning in a conversion domain based on data converted through the non-learnable block. to provide. According to the present invention, there is an advantage in that it is possible to provide an improved manipulation image detection system and method that is robust to anti-forensic technology that hides whether or not the image has been manipulated.

Description

Improved manipulation video detection system and method {ADVANCED SYSTEM AND METHOD FOR DETECTING VIDEO FORGERY}

The present invention relates to an improved manipulation image detection system and method, and more particularly, to an improved manipulation image detection system and method designed to be robust against GAN-based anti-forensics in a system for detecting whether an image has been manipulated. it's about

The contents described in this part merely provide background information on the present embodiment and do not constitute prior art.

Thanks to rapidly developing science and technology, digital images are acquired in various ways and used in various places. As a side effect of the development of digital image culture, digital images are modified and manipulated in various ways, resulting in various difficulties such as determining the authenticity of images in places such as courts.

To solve this, the development of digital forensic technology is indispensable. Digital forensic technology includes a technique using temporal/spatial inconsistency and a technique using characteristics of an encoding/decoding pattern that inevitably occurs in a double compression environment when manipulating an image/video.

However, with the development of forensic technology, various anti-forensic methods such as erasing or hiding such features are also being developed in order to respond to forensic methods that identify and detect features that occur during manipulation. In particular, due to the development of artificial intelligence technology, a technology that prevents detection by general forensic detectors by transforming features indistinguishable to the human eye in a very detailed and sophisticated manner has recently been greatly developed.

Among these technologies, in particular, the Generative Adversarial Networks (GAN) method performs an excellent anti-forensic role of deleting traces of video/video based on the adversarial correspondence between generators and detectors. As a representative manipulation technology based on GAN, there is DeepFake technology, which has emerged as a major social issue. It is necessary to develop a detector as a counter anti-forensic technology that can neutralize these latest anti-forensic technologies.

Therefore, in the present invention, it is intended to propose an improved manipulation image detection system and method that is robust to anti-forensic technology compared to the prior art.

Korean Registered Patent Publication No. 10-2097422, published on April 6, 2020 (Name: Apparatus and Method for Detecting Manipulated Images) Korean Registered Patent Publication No. 10-1725041, published on April 10, 2017 (Name: Fake Image Detection Method and Device) Korean Registered Patent Publication No. 10-1311309, published on September 25, 2013 (Name: Video manipulation detection method and Video manipulation detection device)

(Non-Patent Document 1) B. Bayar, and M. C. Stamm, "A generic approach towards image manipulation parameter estimation using convolutional neural networks." In Proceedings of the 5th ACM Workshop on Information Hiding and Multimedia Security, pp. 147-157, 2017 (Non-Patent Document 2) B. Li, H. Zhang, H. Luo, and S. Tan. "Detecting double JPEG compression and its related anti-forensic operations with CNN." Multimedia Tools and Applications 78, no. 7 (2019): 8577-8601.

The present invention has been proposed to solve the problems of the prior art described above, and the main object of the present invention is to provide an improved manipulation image detection system and method that are robust against anti-forensic technology that hides whether or not the image has been manipulated.

In addition, another object of the present invention is to provide an improved manipulation image detection system and method that can maintain a high detection rate even in the learned GAN-based forensic technology when detecting whether or not an image has been manipulated using learning-based technology. .

The problem to be solved by the present invention is not limited to those mentioned above, and another problem to be solved that is not mentioned will be clearly understood by those skilled in the art from the description below.

One aspect of the present invention for achieving the above object is an improved manipulation image detection system that is robust against anti-forensic technology that hides whether or not the image has been manipulated, in order to discriminate between the original image and the manipulation image, the original image and the manipulation image. a non-learnable block that converts an image into an arbitrary space in which learning is impossible; and a detector that performs learning in a conversion domain based on the data converted through the non-learnable block.

The non-learnable block may have a conversion equation representing a conversion relationship between an input and an output that cannot be learned.

The manipulation image may be generated through a generator inside the GAN-based anti-forensic system.

If the detection system is a system that determines whether an image is double-compressed for manipulation detection, the non-learnable block may have a function that compresses the input image once more.

The detector may determine an image in which the compressed input image is further compressed.

The non-learnable block may be a non-differentiable quantization function as a discontinuous function.

The non-learnable block may be one in which a non-differentiable function is used for an activation function in the network.

Another aspect of the present invention is an improved manipulation image detection method that is robust against anti-forensic technology that hides whether or not an image has been manipulated. entering into; converting an image from the non-learnable block into an arbitrary space in which learning of the input original image and the manipulated image is impossible; and learning the detector in a conversion domain based on the converted data.

The manipulation image may be generated through a generator inside the GAN-based anti-forensic system.

The learned detector may detect whether an arbitrary image has been manipulated.

Another aspect of the present invention provides a computer-readable recording medium recording a program for executing the improved method for detecting an operation image according to any one of claims 8 to 10.

According to the improved manipulation image detection system and method of the present invention, it is possible to provide an improved manipulation image detection system and method that are robust against anti-forensic technology that hides whether or not the image has been manipulated.

In addition, there is an effect of providing an improved manipulation image detection system and method that can maintain a high detection rate even in the learned GAN-based forensic technology when detecting whether or not an image has been manipulated using learning-based technology.

The effects obtainable in the present invention are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the description below. .

BRIEF DESCRIPTION OF THE DRAWINGS The accompanying drawings, which are included as part of the detailed description to aid understanding of the present invention, provide examples of the present invention and, together with the detailed description, explain the technical features of the present invention.
1 is a diagram illustrating the configuration of a GAN-based manipulator and detector according to the prior art.
2 is a diagram illustrating the configuration of an improved manipulation image detection system according to an embodiment of the present invention.
3 is a diagram illustrating the configuration of a backpropagation algorithm.
4 is a diagram illustrating a quantization function.
5 is a diagram illustrating an improved method for detecting an operation image according to an embodiment of the present invention.

Hereinafter, preferred embodiments according to the present invention will be described in detail with reference to the accompanying drawings. The detailed description set forth below in conjunction with the accompanying drawings is intended to describe exemplary embodiments of the present invention and is not intended to represent the only embodiments in which the present invention may be practiced. The following detailed description includes specific details for the purpose of providing a thorough understanding of the present invention. However, those skilled in the art to which the present invention pertains understand that the present invention may be practiced without these specific details.

In some cases, in order to avoid obscuring the concept of the present invention, well-known structures and devices may be omitted or may be shown in block diagram form centering on core functions of each structure and device.

Throughout the specification, when a part is said to "comprising" or "including" a certain element, it means that it may further include other elements, not excluding other elements, unless otherwise stated. do. In addition, terms such as “… unit”, “… unit”, and “module” described in the specification mean a unit that processes at least one function or operation, which may be implemented as hardware or software or a combination of hardware and software. there is. Also, "a or an", "one", "the" and similar related words in the context of describing the invention (particularly in the context of the claims below) Unless indicated or otherwise clearly contradicted by context, both the singular and the plural can be used.

In describing the embodiments of the present invention, if it is determined that a detailed description of a known function or configuration may unnecessarily obscure the subject matter of the present invention, the detailed description will be omitted. In addition, terms to be described later are terms defined in consideration of functions in the embodiment of the present invention, which may vary according to the intention or custom of a user or operator. Therefore, the definition should be made based on the contents throughout this specification.

Each component in the drawings of the present invention is shown independently to represent different characteristic functions in the improved manipulation image detection system and method, and each component is composed of separate hardware or a single software unit. doesn't mean That is, each component is listed and included as each component for convenience of explanation, and at least two components of each component can be combined to form one component, or one component can be divided into a plurality of components to perform a function, and each of these components can be divided into a plurality of components. Integrated embodiments and separated embodiments of components are also included in the scope of the present invention as long as they do not depart from the essence of the present invention.

In addition, some of the components may be optional components for improving performance rather than essential components that perform essential functions in the present invention. The present invention can be implemented by including only components essential to implement the essence of the present invention, excluding components used for performance improvement, and a structure including only essential components excluding optional components used for performance improvement. Also included in the scope of the present invention.

Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings.

1 is a diagram illustrating the configuration of a GAN-based manipulator and detector according to the prior art.

In the case of a general detector 101, a distinguishing feature between an unmanipulated image/video and a manipulated image/video is identified, and the detector 101 is developed based on this. Recently, a more powerful detector 101 can be made using an artificial intelligence-based learning method such as deep learning.

However, with the development of deep learning-based technology, as well as the remarkable development of detection technology, the performance of manipulation technology has also greatly improved. In particular, in the GAN-based method using the adversarial response method, it is possible to make the features and traces of the manipulated image almost similar to the original non-manipulated image by using only changes that are barely noticeable to the human eye in the generator 103. . As such an anti-forensic technology comes out, a problem arises in which most of the existing detectors 101 are incapacitated. As such, the images in which manipulation traces from the generator 103 are erased or hidden by the GAN-based anti-forensic technology have excellent performance in avoiding most of the manipulation detectors 101 that have been developed, and thus have a high potential to cause social problems. .

In order to solve this problem, if a detector 101 that is superior to anti-forensic technology, which has the highest performance at present, is developed, it is possible to detect a manipulated image. However, the generator 103 based on the GAN-based anti-forensic technology has a big problem in that it has the ability to continuously evolve the anti-forensic technology based on the newly developed detector 101. For example, suppose that the anti-forensic technology of the existing GAN-based generator 103 is neutralized by developing a highly accurate detector 101 D. However, if the deep learning network structure of detector 101 D is known, based on this, generator 103 G in GAN can once again learn adversarially based on detector 101 D, eventually detector 101 D It is possible to evolve into another form of generator (103) G' that can neutralize. That is, due to the GAN characteristic of a structure in which the performance of the generator 103 is updated in the process of constant hostile interaction between the generator 103 and the detector 101, no matter what new detector 101 is developed, the GAN-based anti-virus A big problem arises in that forensic technology cannot be completely neutralized.

One embodiment of the present invention proposes a new method for disabling continuous adversarial learning by the prior art. In general, the detector 101 developed based on learning such as deep learning technology is reused as a detector 101 for learning the generator 103 inside the GAN-based anti-forensic system to help improve the performance of the generator 103 constantly. In order to compensate for the problem that it is possible, we propose a detection system in which learning of the generator 101 inside the GAN is impossible using the detector 101.

To this end, a first method can be considered to design the detector 101 in a non-learning-based way.

The second method proposes a method of forcibly inserting a block that cannot be learned in the middle, as shown in Figure 2, in order to prevent the update of the generator inside the newly developed GAN.

2 is a diagram illustrating the configuration of an improved manipulation image detection system according to an embodiment of the present invention.

In one embodiment of the present invention, in order to discriminate between the original image x and the manipulated image x^, the original image x and the manipulated image x^ are first converted into an arbitrary space in which learning is impossible through the non-learnable block 201 ( Φ(x), Φ(x^)), and a method of learning the detector 203 in the transformation domain based on the transformed data is used.

In this case, a conversion equation representing a conversion relationship between an input and an output of the non-learnable block 201 may be any function that cannot be learned.

As an embodiment of the present invention, it is assumed that a system for determining whether an image is double-compressed for manipulation detection is assumed, and the non-learnable block 201 is a device having a function of compressing an input image once more. In this case, the image compressed once is actually compressed twice, and the image compressed twice is compressed three times.

In this case, the detector 203 determines whether the final input image is double-compressed by discriminating not the 1st/2x compressed images but the 2nd/3rd compressed images. The corresponding detector 203 has a non-learnable module called compression, that is, a non-learnable block 201 in the middle, and thus the error value of the detector 203 is more difficult due to the characteristics of the non-learnable compression system. Since error backpropagation cannot be performed, even if the entire network structure of the actual detector 203 is disclosed, the generator 205 in the GAN of the actual anti-forensic system cannot proceed with learning based on this information.

Error Backpropagation is a supervised learning algorithm for neural network learning by minimizing errors through backward weight adjustment.

3 is a diagram illustrating the configuration of a backpropagation algorithm.

Bringing the result value through the activation function while updating the weight from the input signal (Input) to the output signal (Output) is called forward propagation. However, error backpropagation appeared because there can be errors in the output values of such forward propagation. In error backpropagation, the result of forward propagation from input to output has an error, and the error is sent back between the hidden layer and the input layer in the reverse direction, updating the weights. It is to adjust the error that occurred in the output (Output). At this time, the weights are adjusted by returning more errors to the neurons that caused a lot of error in the result.

There are two main reasons why the video compression system including the non-learnable block 201 according to an embodiment of the present invention cannot learn. First, a quantization function as shown in FIG. 4, which is widely used for encoding information in a compression system, is a discontinuous function and is obviously an indifferentiable system.

4 is a diagram illustrating a quantization function.

Another reason is that in the video compression process, after coding the data with a certain rule, a rate-distortion function is generated using the coded information and the predicted video value to find the most efficient value. . At this time, since the rate value of the coded information is determined through the distribution of the entire coded information, it is impossible to measure the differential value from the current value. In this way, efficient and non-differentiable module insertion can be performed by using a non-differentiable function in the non-learnable block 201 for an activation function in the network.

To confirm the above process, the results of testing the actual GAN-based generator are shown in Table 1 below.

Method [One] [2] Proposal method Detection rate for general manipulation images 98.5% 98.5% 99.9% Detection rate after application of anti-forensic technology according to the present invention 58.0% 54.5% 90.1%

First, as a result of selecting a learning-based technology with a high detection rate and detecting manipulation, it can be seen that it has a high accuracy of close to 99% as shown in Table 1 above.

However, when an anti-forensic system is built based on the detector, the actual detection rate drops significantly to about 50%. On the other hand, it can be seen that the detector 203 of the proposed method cannot directly proceed with GAN learning, and is very robust, such as having a detection rate of 90% or more even with GAN-based anti-forensic techniques learned in other methods.

5 is a diagram illustrating an improved method for detecting an operation image according to an embodiment of the present invention.

In order to discriminate between the original image x and the manipulated image x^, the original image x and the manipulated image x^ are input to the unlearnable block 201 (S501).

In the non-learning block 201, a function that cannot be learned is selected for the input original image x and the manipulated image x^, and the image is converted into an arbitrary space that cannot be learned (S503).

Then, based on the transformed data (Φ(x), Φ(x^)), a method of learning the detector 203 in the transformation domain is used (S505).

The learned detector 203 can utilize for detecting whether or not the image has been manipulated (S507), and has robust anti-forensic performance compared to the prior art.

Although it is described in FIG. 5 that steps S501 to S507 are sequentially executed, this is only an illustrative example of the technical idea of this embodiment, and those skilled in the art to which this embodiment belongs will Since it will be possible to change and execute the order described in FIG. 5 without departing from the essential characteristics or to execute one or more steps of steps S501 to S507 in parallel, it will be possible to apply various modifications and variations, so FIG. It is not limited. For example, the learning step of S505 does not end as a one-time event, but may be repeated until it has sufficient performance.

Combinations of each block of the block diagrams and each step of the flowcharts attached hereto may be performed by computer program instructions. Since these computer program instructions may be loaded into a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing equipment, the instructions executed by the processor of the computer or other programmable data processing equipment are each block of the block diagram or flowchart. In each step, means to perform the functions described are created. These computer program instructions may also be stored in a computer usable or computer readable memory that can be directed to a computer or other programmable data processing equipment to implement functionality in a particular way, such that the computer usable or computer readable memory It is also possible for the instructions stored in to produce an article of manufacture containing instruction means for performing the function described in each block of the block diagram or each step of the flow chart. The computer program instructions can also be loaded on a computer or other programmable data processing equipment, so that a series of operational steps are performed on the computer or other programmable data processing equipment to create a computer-executed process to generate computer or other programmable data processing equipment. It is also possible that the instructions performing the processing equipment provide steps for executing the functions described in each block of the block diagram and each step of the flowchart.

Additionally, each block or each step may represent a module, segment or portion of code that includes one or more executable instructions for executing specified logical function(s). It should also be noted that in some alternative embodiments it is possible for the functions recited in blocks or steps to occur out of order. For example, two blocks or steps shown in succession may in fact be performed substantially concurrently, or the blocks or steps may sometimes be performed in reverse order depending on their function.

The above description is merely an example of the technical idea of the present invention, and various modifications and variations can be made to those skilled in the art without departing from the essential characteristics of the present invention. Therefore, the embodiments disclosed in the present invention are not intended to limit the technical idea of the present invention, but to explain, and the scope of the technical idea of the present invention is not limited by these embodiments. The protection scope of the present invention should be construed according to the claims below, and all technical ideas within the scope equivalent thereto should be construed as being included in the scope of the present invention.

According to the improved manipulation image detection system and method of the present invention, in that it can be used as a solution capable of providing an improved manipulation image detection system and method that is robust to anti-forensic technology that hides whether or not the image has been manipulated, existing technology As it goes beyond the limits of related technology, it is an invention with industrial applicability because it is not only available for use in related technology, but also has a sufficient possibility of marketing or business of the applied device, as well as a level that can be clearly implemented in reality.

101: detector 103; generator
201: unlearnable block 203: detector 205: generator

Claims (14)

In an improved manipulation image detection system that is robust to anti-forensic technology that hides whether or not the image has been manipulated,
In order to discriminate between the original image and the manipulated image, the non-learnable block transforms the original image and the manipulated image into an arbitrary space in which learning is impossible; and
A detector that performs learning in a conversion domain based on data converted through the non-learnable block,
The improved manipulation image detection system, characterized in that the non-learnable block has a conversion equation representing a conversion relationship between an input and an output that cannot be learned. delete According to claim 1,
The manipulation image is an improved manipulation image detection system, characterized in that generated through a generator inside the GAN-based anti-forensic system. According to claim 1,
If the detection system is a system that determines whether an image is double-compressed for manipulation detection,
The improved manipulation image detection system, characterized in that the non-learnable block has a function for compressing the input image once more. According to claim 4,
The detector,
An improved manipulation image detection system, characterized in that the once again compressed input image is discriminated. According to claim 1,
The non-learnable block,
An improved manipulation image detection system characterized in that it is a non-differentiable quantization function as a discontinuous function. According to claim 1,
The non-learnable block,
An improved manipulation image detection system characterized in that a non-differentiable function is used for an activation function in a network. In the method for detecting an improved manipulation image that is robust to anti-forensic technology that hides whether the image is manipulated by a computing system,
inputting the original image and the manipulated image into an unlearnable block in order to discriminate between the original image and the manipulated image;
converting an image from the non-learnable block into an arbitrary space in which learning of the input original image and the manipulated image is impossible; and
Learning a detector in a conversion domain based on the converted data;
The method of detecting an improved manipulation image, characterized in that the non-learnable block has a conversion equation representing a conversion relationship between input and output that cannot be learned. According to claim 8,
The method of detecting an improved manipulation image, characterized in that the manipulation image is generated through a generator inside the GAN-based anti-forensic system. According to claim 8,
If the method for detecting the improved manipulation image is a method for determining whether or not the image is double compressed for manipulation detection,
The method of detecting an improved manipulation image, characterized in that the non-learnable block has a function that compresses the input image once more. According to claim 10,
The detector,
An improved manipulation image detection method characterized in that the once more compressed input image is discriminated. According to claim 8,
The non-learnable block,
An improved manipulation image detection method characterized in that it is a non-differentiable quantization function as a discontinuous function. According to claim 8,
The non-learnable block,
An improved manipulation image detection method characterized by using a non-differentiable function as an activation function in a network A computer-readable recording medium recording a program for executing an improved manipulation image detection method robust to anti-forensic technology that hides manipulation of images performed by a computing system,
inputting the original image and the manipulated image into an unlearnable block in order to discriminate between the original image and the manipulated image;
converting an image from the non-learnable block into an arbitrary space in which learning of the input original image and the manipulated image is impossible; and
Learning a detector in a conversion domain based on the converted data;
The non-learnable block has a conversion equation representing a conversion relationship between an input and an output that cannot be learned.

Images