KR20230006982A - Apparatus and method for distinguishing manipulated image based on discrete cosine transform information - Google Patents

Abstract

An apparatus for distinguishing a manipulated image according to an embodiment of the present invention may include: an image acquiring part for acquiring a target image; an image processing part for extracting compressed tracking information related to a discrete cosine transform (DCT) coefficient from the target image; and a distinguishing part for inputting the compressed tracking information of the target image to the trained image discriminator based on a plurality of training images labeled with classes of whether the compressed tracking information for each pixel included in the training image is manipulated, and distinguishing whether the manipulated pixel is included in the target image.

Description

Apparatus and method for discriminating manipulated images based on DCT space information

The present invention relates to an apparatus and method for discriminating a manipulated image based on DCT spatial information.

With the development of image editing technology and an Internet environment capable of sharing images, digital images are easily captured, posted online, and transmitted to numerous people through various social network services.

These digital images may include manipulated images created by image editing programs. Splicing, one of the image manipulation methods, is a general image manipulation method in which a specific area of an image is copied and pasted onto another image. Contrary to what can be easily done, it is not easy to visually discriminate a manipulated image.

These manipulated images can cause misunderstanding by disseminating distorted information or cause negative effects used for propaganda. Accordingly, research on a manipulation image discrimination technology that automatically distinguishes a manipulated image from an original image without relying on a person is being actively conducted.

Republic of Korea Patent Registration No. 10-1181086 (registered on September 3, 2012)

The problem to be solved by the present invention is to manipulate a target image based on DCT spatial information according to the difference between the compression traces generated in the process of generating a digital image and the compression traces appearing in the part where the manipulated image is edited. It is to provide an apparatus and method for discriminating manipulated images based on a neural network learned to discriminate whether

However, the problems to be solved by the present invention are not limited to those mentioned above, but include objects that are not mentioned but can be clearly understood by those skilled in the art from the description below. can do.

An apparatus for determining a manipulated image according to an embodiment of the present invention includes an image acquiring unit acquiring a target image; an image processor extracting DCT (Discrete Cosine Transform) spatial information of the target image; and inputting the DCT spatial information of the target image to an image discriminator learned based on a plurality of training images labeled with a class of manipulation of DCT spatial information included in the training image to determine whether a manipulated pixel is included in the target image. It may include a determination unit for determining whether or not.

Also, the DCT spatial information may include a DCT coefficient for each pixel of the target image and a quantization table used for compression of the target image.

The image processing unit may extract the DCT spatial information after cropping the target image in units of multiples of the number of horizontal pixels and vertical pixels of the quantization table used to compress the target image.

In addition, the image discriminator is based on a transformation module that generates a feature map in which DCT coefficients are separated based on frequency for DCT coefficients for each pixel included in the target image, and features generated by the transformation module from the DCT spatial information. It may include a DCT neural network that extracts the first feature value of the target image by performing a convolution operation on the map.

In addition, the transformation module generates a separate feature map in which binary values are assigned to pixels having DCT coefficients of the same value within a predetermined range of DCT coefficients for each pixel included in the target image, and the generation It may include a binary volume transformation layer that creates a binary volume by connecting the separate feature maps.

In addition, the transformation module divides the feature map generated from the binary volume into units of sizes of quantization tables used in the target image, and generates a first frequency-specific feature map obtained by extracting the same component for each frequency of each quantization table. A separation layer for each frequency may be included.

In addition, the conversion module performs element-by-element multiplication of a feature map generated from the binary volume and a block array feature map generated by connecting the quantization table to have a size corresponding to that of the feature map generated from the binary volume, and then has the same size. A feature map in which DCT coefficients are separated based on frequency may be generated from DCT coefficients for each pixel included in the target image by combining a second feature map for each frequency from which components for each frequency are extracted and the first feature map for each frequency. there is.

In addition, the binary volume conversion layer, according to Equation 1 below, in the order of 0 or more and T (predetermined natural number) or less, with respect to the DCT coefficient included in the 2-dimensional pixel position (i, j) of the target image, 0 or A separate feature map converted to a value of 1 may be created, and the generated (T+1) feature maps may be connected.

[Equation 1]

(M: DCT coefficient for each pixel included in the target image, clip(M)ij is converted to T if the DCT coefficient of coordinates (i, j) among the DCT coefficients included in M is greater than T, and is less than -T - A function that converts to T, abs() is an absolute value function)

The image discriminator may include: an RGB neural network generating a second feature value of the target image by performing a convolution operation on a feature map generated based on RGB information for each pixel of the target image; and a synthetic neural network that receives the first feature value and the second feature value as inputs and performs a convolution operation to generate a third feature value in which DCT information and RGB information of the target image are reflected together.

A manipulation image discrimination method performed by a manipulation image discrimination apparatus according to an embodiment of the present invention includes acquiring a target image; extracting discrete cosine transform (DCT) spatial information of the target image; and inputting the DCT spatial information of the target image to an image discriminator learned based on a plurality of training images labeled with a class of manipulation of DCT spatial information included in the training image to determine whether a manipulated pixel is included in the target image. It may include a step of determining whether or not.

According to an embodiment of the present invention, DCT coefficients for each frequency can be used in an image discrimination deep learning algorithm by cropping a target image based on the grid size of a quantization table and converting DCT coefficients into discrete volumes based on this. By making the DCT coefficient domain and the RGB information domain located in the same domain, it is possible to determine whether the target image has been manipulated by analyzing the compression trace of the target image with high performance.

The effects obtainable in the present invention are not limited to the effects mentioned above, and other effects not mentioned can be clearly understood by those skilled in the art from the description below. will be.

1 is a result of analyzing DCT coefficient distributions for an original part and a manipulated part in a target image.
2 is a functional block diagram of a manipulation image discrimination device according to an embodiment of the present invention.
3 is an exemplary view showing the structure of a neural network of an image discriminator according to an embodiment of the present invention.
4 is an exemplary diagram of the structure of a convolution unit used in HRNet.
5 is an exemplary diagram of the structure of a fusion convolution unit used in HRNet.
6 is an exemplary diagram of the structure of a transformation module of a DCT neural network according to an embodiment of the present invention.
7 and 8 are exemplary diagrams for explaining an operation of a separation layer for each frequency according to an embodiment of the present invention.
9 is a flowchart of a method for determining a manipulated image according to an embodiment of the present invention.
10 is a diagram illustrating a result of a manipulation image determination method according to an embodiment of the present invention.

Advantages and features of the present invention, and methods of achieving them, will become clear with reference to the detailed description of the following embodiments taken in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below and can be implemented in various forms, only the present embodiments are intended to complete the disclosure of the present invention, and those of ordinary skill in the art to which the present invention belongs It is provided to fully inform the person of the scope of the invention, and the scope of the invention is only defined by the claims.

In describing the embodiments of the present invention, detailed descriptions of well-known functions or configurations will be omitted unless actually necessary in describing the embodiments of the present invention. In addition, terms to be described later are terms defined in consideration of functions in the embodiment of the present invention, which may vary according to the intention or custom of a user or operator. Therefore, the definition should be made based on the contents throughout this specification.

Hereinafter, terms such as 'unit' and 'unit' refer to a unit that processes at least one function or operation, and may be implemented by hardware or software or a combination of hardware and software.

Manipulation of an image may include an act of including a processed image generated by changing at least some elements in the original image, and if some elements in the original image are changed, such an image may be referred to as a manipulated image. For example, splicing is a general image manipulation method in which a specific region of one image is copied and pasted onto another image. Contrary to the fact that image manipulation can be easily performed, it is not easy to distinguish the manipulated image with the naked eye.

Embodiments of the present invention begin with the purpose of analyzing compression traces within pixels of a specific image in order to determine whether a target image has been manipulated. Typically, compression occurs once, based on lossy compression (e.g. JPEG), when a picture is taken by a camera. When the subsequent operations are performed and saved, compression occurs once more. At this time, the original part is compressed once more in the same lattice unit because there is no change in pixel value, but the manipulated part is compressed in a new lattice because the pixel value is different. In this way, the pixels of the original area can be compressed twice in the same lattice unit, but the manipulated part has a different compression process and environment in the process of generating the original image, so the compression traces between the original image and the manipulated image appear differently. . For example, discrete cosine transform (DCT) spatial information of the original part and the manipulated part in the target image is checked as follows.

1 is a result of analyzing DCT coefficient distributions for an original part and a manipulated part in a target image.

Referring to FIG. 1(a), a green part in an image is an original part, and an orange part is a manipulated part edited and pasted from another image. Referring to FIG. 1(b), it can be seen that the distribution of DCT coefficients derived from the original part and the distribution of DCT coefficients derived from the manipulated part appear different.

On the other hand, in using existing image discrimination deep learning algorithms, when the DCT coefficient distribution itself is used purely as input data, performance is poor due to the large decorrelation of DCT coefficients.

Therefore, the manipulation image discrimination apparatus 100 according to an embodiment of the present invention can utilize the DCT coefficients in the image discrimination deep learning algorithm based on the method of converting the DCT coefficients into discrete volumes, and the domain and RGB information of the DCT coefficients. We propose a method that allows the domains of to be located in the same domain so that the feature values derived from each can be used together.

2 is a functional block diagram of a manipulation image discrimination device 100 according to an embodiment of the present invention. Overall operations of the manipulation image discrimination apparatus 100 according to an embodiment of the present invention may be performed by one or more processors, and the one or more processors may control functional blocks included in FIG. 2 to perform operations to be described later. there is.

Referring to FIG. 2 , the apparatus 100 for determining a manipulated image according to an embodiment of the present invention includes a storage unit 110, an image acquisition unit 120, an image processing unit 130, a determination unit 140, and a discriminator generation unit. may include section 150 .

The storage unit 110 may store various data utilized according to an embodiment of the present invention. For example, the storage unit 110 may store a target image, a training image, and the image determiner 200 . The storage unit 110 may be configured as a hardware-type memory inside the manipulated image determining device 100 or may be configured as a module that interworks with a cloud database located outside the manipulated image determining device 100 .

The image acquisition unit 120 may obtain a target image. For example, the image acquisition unit 120 may obtain a target image through an external input or through loading of data stored in the storage unit 110 . The target image may be an object for determining manipulation in the image. For example, the target image may be a JPEG image lossy compressed according to the Joint Photographic Experts Group (JPEG) standard.

The image processor 130 may extract DCT (Discrete Cosine Transform) spatial information of the target image. For example, the DCT spatial information may include DCT coefficients (eg, Y-channel DCT coefficients) for each pixel of the target image and a quantization table (eg, Y-channel quantization table) used for compression of the target image.

For example, when the image processing unit 130 extracts DCT coefficients for each pixel of the target image, the image processing unit 130 extracts the target image in units of multiples of the number of horizontal and vertical pixels of the quantization table used for compression of the target image. After cropping, DCT coefficients for each pixel may be extracted. As such, the reason why the image processing unit 130 crops based on the size of the quantization table is to derive DCT coefficients for each frequency of the target image, which will be described in detail with reference to FIGS. 7 and 8 .

For example, when extracting the quantization table used for compression of the target image, the image processing unit 130 may obtain the quantization table by checking a header of an image file for the target image.

The image processing unit 130 may derive RGB information of pixels constituting the target image.

For example, the image processing unit 130 provides RGB information consisting of RGB element values (x, y, z) representing the unique colors of R (Red), G (Green), and B (Blue) of each pixel constituting the target image. can be derived, and three feature maps composed of only elements of R, G, and B can be generated. If there is no DCT coefficient information in the target image, the image processing unit 130 may directly calculate DCT coefficients by performing discrete cosine transformation from RGB pixel information and set the value of the quantization table to 1.

The determination unit 140 inputs DCT space information or RGB information of the target image to the image discriminator 200 learned based on a predetermined image discrimination deep learning algorithm to determine whether the manipulated pixels are included in the target image. It can be determined whether

3 is an exemplary view showing the structure of a neural network of the image discriminator 200 according to an embodiment of the present invention.

Referring to FIG. 3 , the image discriminator 200 according to an embodiment of the present invention may include a DCT neural network, an RGB neural network, and a synthetic neural network. The embodiments of the DCT neural network, the RGB neural network, and the synthetic neural network illustrated in FIG. 3 are each examples using the structure of a High-Resolution Representation Learning Network (HRNet). The embodiment of the present invention describes a structure in which the transformation module 215 according to the embodiment of the present invention is applied to a network structure based on HRNet for convenience of understanding, but in addition to the structure of various image discrimination deep learning algorithms A neural network of the image discriminator 200 may be designed through an embodiment in which the conversion module 215 is applied.

The DCT neural network 210 may include a transform module 215 and a network based on HRNet, and is designed to extract a first feature value of a target image by performing a convolution operation on a feature map generated from DCT spatial information. can

The transform module 215 may generate a feature map in which DCT coefficients are separated based on frequencies for DCT coefficients for each pixel included in the target image based on a method of converting DCT coefficients into discrete volumes. Accordingly, the conversion module 215 can utilize the DCT coefficients in the image discrimination deep learning algorithm, and position the DCT coefficient domain and the RGB information domain in the same domain, so that the feature values derived from each can be used together do. A detailed structure of the conversion module 215 will be described later in conjunction with FIGS. 6 to 8 .

The RGB neural network 220 may include an HRNet-based network, and is designed to extract a second feature value of the target image by performing a convolution operation on a feature map generated based on RGB information for each pixel of the target image. can

The synthetic neural network 230 may include an HRNet-based network, receives the first feature value and the second feature value as inputs and performs a convolution operation to obtain a third feature in which DCT spatial information and RGB information of the target image are reflected together. It can be designed to extract values.

As shown in the structure of FIG. 3, the discriminator generation unit 150 is designed in a structure in which the DCT neural network 210, the RGB neural network 220, and the synthetic neural network 230 are connected, end-to-end (end-to-end) of the first input end and the final output end of the neural network. -end) Through learning, if the DCT coefficient, quantization table, and RGB information of the target image are input to the input terminal, it may be learned to determine whether pixels included in the target image are manipulated. For example, the discriminator generating unit 150 trains the image discriminator 200 based on a plurality of training images labeled with classes of manipulation of DCT spatial information, or based on RGB information included in the training images. The image discriminator 200 may be trained on the basis of a plurality of training images labeled with classes of whether or not manipulation has been performed.

On the other hand, the image discriminator 200 illustrated in FIG. 3 includes all of the DCT neural network, the RGB neural network, and the synthetic neural network and exemplifies an operation learned and used together, but the image discriminator according to another embodiment of the present invention ( 200) may be composed of either a DCT neural network or an RGB neural network alone. The discriminator generation unit 150 may generate the image discriminator 200 composed of a single DCT neural network or a single RGB neural network by performing end-to-end learning of an input end and an output end of a single neural network.

The discriminator generation unit 150 may store the image discriminator 200 for which learning has been completed according to the above-described embodiment in the storage unit 110, and when determining whether or not the target image has been manipulated, the discriminator 140 DCT space information or RGB information extracted by the image processing unit 130 may be input to the image determiner 200 stored in the storage unit 110 to determine whether the target image has been manipulated.

4 is an exemplary diagram of a detailed structure of a convolution unit used in the HRNet structure of FIG. 3, and FIG. 5 is an exemplary diagram of a structure of a fusion convolution unit used in the HRNet structure of FIG. The examples of FIGS. 4 and 5 are only examples of the structure of the convolution unit or the structure of the fusion convolution unit, but do not limit the embodiments of the present invention, and the structure of the convolution unit of HRNet and the fusion convolution unit The structure of is a well-known structure and a detailed description thereof will be omitted.

6 is an exemplary diagram of the structure of the conversion module 215 of the DCT neural network according to an embodiment of the present invention.

The upper structure will be described based on the dotted line of (a) in FIG. 6 .

At the top of the dotted line in (a), the conversion module 215 includes a binary volume conversion layer 216, a convolutional layer and a frequency-by-frequency separation layer 217, based on a scheme for converting DCT coefficients into discrete volumes. For the DCT coefficients 20 for each pixel included in the target image, a feature map 25 in which DCT coefficients are separated based on frequency may be generated.

The discrete volume transform layer is 0 within a predetermined range of DCT coefficients (hereinafter, in the example, the DCT coefficient in the predetermined range is a natural number T) for each pixel included in the 2-dimensional pixel position (i, j) of the target image. DCT coefficients for each pixel included in the 2D pixel position (i, j) of the target image are determined in the order below T, and a separate feature map converted to a value of 0 or 1 according to Equation 1 below is generated. When (T + 1) times proceed from 0 to T, a feature map connecting the generated (T + 1) feature maps can be generated.

[Equation 1]

(M: DCT coefficient for each pixel included in the target image, clip(M)ij is converted to T if the DCT coefficient of coordinates (i, j) among the DCT coefficients included in M is greater than T, and is less than -T - A function that converts to T, abs() is an absolute value function)

의 변환이 이루어지며, 소정 정수값의 집합 Z로 이루어진 HxW 크기의 타겟 이미지는, {0 또는 1}의 원소값으로 이루어진 H x W 크기의 (T+1)개 특징맵으로 변환될 수 있다. That is, according to Equation 1, the feature map is

The conversion of is performed, and the HxW-sized target image composed of a set Z of predetermined integer values can be converted into (T+1) feature maps of HxW size composed of element values of {0 or 1}.

The transformation module 215 targets the feature map 23 generated through a convolution operation (eg, 3x3 convolution, batch normalization, ReLu, 1x1 convolution, batch normalization, ReLU layer in the example of FIG. 6) on a binary volume. The first frequency-specific component is divided into size units of the quantization table used in the image (ex. 8x8 in case of JPEG), and the components for each frequency are extracted from the lattice divided into each quantization table through the frequency-specific separation layer 217-1. A feature map 25 may be created.

7 and 8 are exemplary diagrams for explaining the operation of the frequency-by-frequency separation layer 217 according to an embodiment of the present invention.

Referring to FIG. 7, if the size of the feature map is 64x64 pixels and the size of the quantization table is 8x8, the frequency-specific separation layer 217 has 64x64 pixels as shown in FIG. The feature map can be divided into units of quantization table size 8x8. That is, in terms of horizontal pixels, (64 horizontal pixels of feature map)/(8 horizontal size of quantization table) = 8, and in terms of vertical pixels (64 vertical pixels of feature map)/(quantization table) 8 vertical size) = 8 can be distinguished. That is, the frequency-specific separation layer 217 separates a feature map with a size of 64x64 pixels into 64 grids each having a size of 8x8 pixels per grid (eg, one grid a1 and one grid a2) as shown in FIG. can do.

Referring to FIG. 8, the frequency-specific separation layer 217 has 64 element values corresponding to the coordinates of (0, 0) in all 64 grids separated according to the operation of FIG. 7 described above according to Equation 2 below. It is possible to create one feature map (A) by collecting them.

[Equation 2]

(c: index indicating the number of feature maps, i: horizontal pixel index of feature map, j: vertical pixel index of feature map)

In the case of FIG. 8, since there is only one feature map before frequency separation, c={0}, i={0, 1, 2, 3, 4, 5, 6, 7}, j={0, 1, 2, 3, 4, 5, 6, 7}. For example, the frequency-specific separation layer 217 derives element values in the order of (0, 0) element value a1 of the upper left lattice of the 0th feature map and (0, 0) element value a2 of the next lattice. , it is possible to generate a feature map for each frequency by performing a corresponding operation until the coordinates are from (0, 0) to (7, 7), and converting the dimension into 64 feature maps from which DCT coefficients for each frequency are derived.

의 차원 변환이 이루어지며, 주파수별 분리 레이어(217)는 소정 정수값의 집합 R로 이루어진

차원의 특징맵을

차원의 특징맵으로 변환하여 주파수 분리를 수행할 수 있다. That is, according to Equation 2, the feature map is

The dimensional transformation of is performed, and the frequency-specific separation layer 217 consists of a set R of predetermined integer values.

dimensional feature map

Frequency separation can be performed by transforming into a dimensional feature map.

Referring to the dotted line in (a) in FIG. 6, the lower structure and finally generated feature map will be described.

At the bottom of the dotted line in (a), the transform module 215 iteratively arranges the quantization table 30 through the block arrangement layer 218 to have the same size as the feature map 23 generated from the binary volume 21. After element-by-element multiplication of the block array feature map 31 and the feature map 23 generated from the binary volume 21, the second frequency components are extracted through the frequency-specific separation layer 217-2. A feature map 27 in which DCT coefficients are separated based on frequency may be generated from the DCT coefficients for each pixel included in the target image by combining the feature map 35 for each feature and the first feature map 25 for each frequency. .

Accordingly, the transform module 215 transforms the DCT coefficients 20 for each pixel cropped based on the grid size of the quantization table of the target image into a discrete volume, and generates a feature map derived from DCT coefficients for each frequency. The domain of coefficients and the domain of RGB information can be located in the same domain.

9 is a flowchart of a method for determining a manipulated image according to an embodiment of the present invention.

Each step of the manipulation image discrimination method according to FIG. 9 may be performed by the manipulation image discrimination device 100 described with reference to FIG. 2 , and each step is described as follows.

In step S910, the image acquiring unit 120 may obtain a target image.

In step S920, the image processing unit 130 may extract DCT spatial information of the target image.

In step S930, the discriminating unit 140 transmits the DCT spatial information of the target image to the learned image discriminator 200 based on a plurality of training images labeled with classes of manipulation of DCT spatial information included in the training image. input to determine whether or not the manipulated pixel is included in the target image.

On the other hand, since the operation for the constituent elements, which are the subject of each step described above, to perform the corresponding step has been described in FIGS. 2 to 8, duplicate descriptions will be omitted.

10 is a diagram showing a correct answer mask for an input image, a manipulation discrimination result according to an embodiment of the present invention, a manipulation discrimination result according to ManTra-Net, and a manipulation discrimination result according to EXIF-SC, respectively.

Referring to FIG. 10, it can be seen that the accuracy of distinguishing the original image from the manipulated image can be significantly improved when using the embodiment of the present invention, compared to the previously used methods of ManTra-Net and EXIF-SC. . That is, the embodiment of the present invention can utilize frequency-specific DCT coefficients in the image discrimination deep learning algorithm by cropping the target image based on the grid size of the quantization table and converting the DCT coefficients into discrete volumes based on this. By making the DCT coefficient domain and the RGB information domain located in the same domain, it is possible to determine whether the target image has been manipulated by analyzing the compression trace of the target image with high performance.

The above description is merely an example of the technical idea of the present invention, and various modifications and variations can be made to those skilled in the art without departing from the essential qualities of the present invention. Therefore, the embodiments disclosed herein are not intended to limit the technical spirit of the present invention, but to explain, and the scope of the technical spirit of the present invention is not limited by these embodiments. The protection scope of the present invention should be construed according to the claims below, and all technical ideas within the scope equivalent thereto should be construed as being included in the scope of the present invention.

100: manipulated image determination device 110: storage unit 120: image acquisition unit
130: image processing unit 140: determination unit 150: discriminator generation unit
200: image discriminator 210: DCT neural network 220: RGB neural network
230: synthetic neural network 200: image discriminator
215: transformation module 216: binary volume transformation layer
217: separation layer by frequency 218: block array layer

Claims (18)

an image acquiring unit acquiring a target image;
an image processor extracting DCT (Discrete Cosine Transform) spatial information of the target image; and
Whether or not a manipulated pixel is included in the target image by inputting the DCT spatial information of the target image to an image discriminator based on a plurality of training images labeled with a class of manipulation of DCT spatial information included in the training image. Including a discriminating unit for discriminating
Manipulated image discrimination device. According to claim 1,
The DCT spatial information,
Including a DCT coefficient for each pixel of the target image and a quantization table used for compression of the target image
Manipulated image discrimination device. According to claim 1,
The image processing unit,
Extracting the DCT spatial information after cropping the target image in units of multiples of the number of horizontal pixels and vertical pixels of a quantization table used for compression of the target image
Manipulated image discrimination device. According to claim 1,
The image discriminator,
Based on a transformation module that generates a feature map in which DCT coefficients are separated on the basis of frequency for DCT coefficients for each pixel included in the target image, a convolution operation is performed on the feature map generated by the transformation module from the DCT spatial information. Including a DCT neural network that performs and extracts a first feature value of the target image
Manipulated image discrimination device. According to claim 4,
The conversion module,
A separate feature map in which binary values are assigned to pixels having DCT coefficients of the same value within a predetermined range of DCT coefficients for each pixel included in the target image is generated, and the generated separate feature maps are connected. Including a binary volume conversion layer that creates a binary volume,
Manipulated image discrimination device. According to claim 5,
The conversion module,
A frequency-specific separation layer for dividing the feature map generated from the binary volume into units of the size of the quantization table used in the target image and generating a first frequency-specific feature map obtained by extracting the same component for each frequency of each quantization table.
Manipulated image discrimination device. According to claim 6,
The conversion module,
A block array feature map generated by connecting the quantization table so as to have a size corresponding to the feature map generated from the binary volume and a feature map generated from the binary volume are multiplied element by element, and components for each frequency are extracted. Generating a feature map in which DCT coefficients are separated based on frequency from the DCT coefficients for each pixel included in the target image by joining the feature map for each frequency and the first feature map for each frequency.
Manipulated image discrimination device. According to claim 5,
The binary volume conversion layer,
For the DCT coefficient included in the 2-dimensional pixel position (i, j) of the target image, a separate feature converted to a value of 0 or 1 according to Equation 1 below in an order of 0 or more and T (predetermined natural number) or less Creating a map and connecting the generated (T + 1) feature maps
[Equation 1]

(M: DCT coefficient for each pixel included in the target image, clip(M)ij is converted to T if the DCT coefficient of coordinates (i, j) among the DCT coefficients included in M is greater than T, and is less than -T - A function that converts to T, abs() is an absolute value function)
Manipulated image discrimination device. According to claim 4,
The image discriminator,
an RGB neural network generating a second feature value of the target image by performing a convolution operation on a feature map generated based on RGB information for each pixel of the target image; and
Further comprising a synthetic neural network that receives the first feature value and the second feature value as inputs and performs a convolution operation to generate a third feature value in which DCT information and RGB information of the target image are reflected together.
Manipulated image discrimination device. In the manipulation image discrimination method performed by the manipulation image discrimination device,
acquiring a target image;
extracting discrete cosine transform (DCT) spatial information of the target image; and
Whether or not a manipulated pixel is included in the target image by inputting DCT spatial information of the target image to an image discriminator based on a plurality of training images labeled with a class of manipulation of DCT spatial information included in the training image. Including the step of determining
How to determine tampered images. According to claim 10,
The DCT spatial information,
Including a DCT coefficient for each pixel of the target image and a quantization table used for compression of the target image
How to determine tampered images. According to claim 10,
The step of extracting DCT spatial information of the target image,
Extracting the DCT spatial information after cropping the target image in units of multiples of the number of horizontal pixels and vertical pixels of a quantization table used for compression of the target image
How to determine tampered images. According to claim 10,
The image discriminator,
Based on a transformation module that generates a feature map in which DCT coefficients are separated on the basis of frequency for DCT coefficients for each pixel included in the target image, a convolution operation is performed on the feature map generated by the transformation module from the DCT spatial information. Including a DCT neural network that performs and extracts a first feature value of the target image
How to determine tampered images. According to claim 13,
The conversion module,
A separate feature map is created in which binary values are assigned to pixels having DCT coefficients of the same value within a predetermined range of DCT coefficients for each pixel included in the target image, and the generated separate feature maps are connected. A binary volume transformation layer that creates a binary volume
How to determine tampered images. According to claim 14,
The conversion module,
A frequency-specific separation layer for dividing the feature map generated from the binary volume into units of the size of the quantization table used in the target image and generating a first frequency-specific feature map obtained by extracting the same component for each frequency of each quantization table.
How to determine tampered images. According to claim 13,
The image discriminator,
an RGB neural network generating a second feature value of the target image by performing a convolution operation on a feature map generated based on RGB information for each pixel of the target image; and
Further comprising a synthetic neural network that receives the first feature value and the second feature value as inputs and performs a convolution operation to generate a third feature value in which DCT information and RGB information of the target image are reflected together.
How to determine tampered images. A computer-readable recording medium storing a computer program,
acquiring a target image;
extracting discrete cosine transform (DCT) spatial information of the target image; and
Whether or not a manipulated pixel is included in the target image by inputting DCT spatial information of the target image to an image discriminator based on a plurality of training images labeled with a class of manipulation of DCT spatial information included in the training image. Including the step of determining
Including instructions for causing the processor to perform the manipulation image discrimination method
A computer-readable recording medium. As a computer program stored on a computer-readable recording medium,
acquiring a target image;
extracting discrete cosine transform (DCT) spatial information of the target image; and
Whether or not a manipulated pixel is included in the target image by inputting DCT spatial information of the target image to an image discriminator based on a plurality of training images labeled with a class of manipulation of DCT spatial information included in the training image. Including the step of determining
Including instructions for causing the processor to perform the manipulation image discrimination method
computer program.

Images