KR102490136B1 - Paper identification device and method for forgery detection of documents - Google Patents

Abstract

The present invention relates to a paper identification device and method for forgery detection of documents.
According to the present invention, the paper identification device for detecting forgery of a document includes an image acquisition unit that acquires a transmitted light image capable of observing a unique formation pattern for each brand of paper sold in the market, and a Fourier transform (Fourier transform) on the obtained transmitted light image. transform) to obtain an FFT image, an image conversion unit that analyzes the input image and builds a CNN model that analyzes the paper using features included in the image, and builds the transmitted light image and the A learning unit that learns the FFT image as each input value, and the transmitted light image obtained by passing infrared rays through the paper sample to be analyzed and the FFT image obtained from the transmitted light image Input to the plurality of CNN models on which learning has been completed, respectively and a determination unit for comparing each value derived from the plurality of CNN models to determine whether forgery or alteration is present.
According to the present invention, it is possible to determine forgery even in paper where it is difficult to observe a repetitive grid pattern through a transmitted light image, and since a deep learning-based CNN model is used, it is possible to predict whether a paper stream is forged or altered through a non-destructive method.

Description

Paper identification device and method for forgery detection of documents

The present invention relates to a paper identification device and method for detecting forgery of a document, and more particularly, by inputting an infrared transmitted light image and an FFT converted image to a trained CNN model to identify copy paper, thereby determining whether a document is forged or altered. It relates to a paper identification device capable of performing the same and a method therefor.

With the development of office automation equipment, even ordinary people can easily produce elaborately forged or falsified documents, and this production method is directly or indirectly used in crime. For the investigation of such forgery/falsification crimes and the analysis of related evidence, various analysis techniques must be established from the viewpoint of forensic science.

However, it is not easy to establish a document appraisal technique from a forensic scientific point of view. One of the reasons for this is that the analyte must not be damaged or damage must be minimized.

Most of the analysis samples of document appraisal are used as legal basis data for a trial between interested parties after analysis, or are likely to be re-analyzed later, and are often used for other issues besides appraisal. However, in most cases, documents damaged in the analysis process cannot be used in actual appraisal, so it is common to perform non-destructive analysis.

On the other hand, the most basic analysis method for document forgery relates to paper analysis. In other words, if the paper can be classified or the manufacturer can be identified, it can be of great help to the investigation by identifying forgery/falsification.

Paper analysis methods include destructive methods using components and physical properties and non-destructive methods of optical classification by analyzing the color and surface characteristics of copying paper.

Non-destructive methods can be used for real appraisal. However, since there are various factors such as the degree of contamination of the sample in the actual appraisal and the change in color due to the passage of time after document production, this analysis method has a problem of limited use.

Therefore, there is a need for a method capable of identifying the manufacturer of commercially available copy paper by a non-destructive analysis method without restrictions such as contamination and samples written with characters in ink.

Republic of Korea Patent Registration No. 10-1573857 (2015.12.04. Notice)

A technical problem to be achieved by the present invention is to provide a paper identification device and method capable of determining whether a document is forged or altered by inputting an infrared transmitted light image and an FFT converted image to a trained CNN model to identify copy paper.

In order to achieve this technical problem, the paper identification device for forgery detection of a document according to an embodiment of the present invention includes an image acquisition unit for acquiring a transmitted light image capable of observing a unique formation pattern for each brand of paper distributed in the market, the above An image conversion unit that obtains an FFT image by applying a Fourier transform to the obtained transmitted light image, and a CNN model that analyzes the input image and analyzes the paper using features included in the image, and constructs the above construction. Learning unit for learning the transmitted light image and the FFT image as respective input values to the CNN model, and learning the transmitted light image obtained by transmitting infrared rays through the paper sample to be analyzed and the FFT image obtained from the transmitted light image and a determination unit for determining whether forgery is falsified by inputting each of the plurality of completed CNN models and comparing each value derived from the plurality of CNN models.

The image conversion unit performs a Fourier transform on the transmitted light image using the following equation,

는　x축 방향으로 주파수가 u/W, y축 방향으로 주파수가 v/H인 sinusoidal 주기함수를 나타낸다)(here,

represents a sinusoidal periodic function with a frequency u/W in the x-axis direction and a frequency v/H in the y-axis direction)

A Fourier spectrum may be obtained using the pixel value H(u,v) obtained through the Fourier transform.

The image conversion unit may acquire a low-frequency region corresponding to a peak portion using the acquired Fourier spectrum, delete the low-frequency region, and perform an inverse Fourier transform to obtain an FFT image.

The learning unit extracts an image having a certain size from the FFT image in a cropped form, repeats this process a plurality of times to augment data for the FFT image, and then uses the transmitted light image and the augmented FFT image to extract the data. You can create three.

The learning unit may train a first CNN model using a first dataset generated using the transmitted light image and a second CNN model using a second dataset generated using the FFT image. there is.

The determination unit inputs a transmitted light image obtained by photographing the paper to be evaluated and an FFT image obtained from the transmitted light image to the first CNN model and the second CNN model on which learning has been completed, respectively, and obtains information from the first CNN model. It is possible to determine forgery or alteration by comparing the obtained result value with the result value obtained from the second CNN model.

The determination unit may obtain each result value wi from the first CNN model and the second CNN model using the following equation.

If the result obtained from the first CNN model and the result value obtained from the second CNN model are the same, the determination unit derives the obtained result value as a final result, and the result value obtained from the first CNN model. If the result value obtained from the second CNN model is different from that of the second CNN model, another part of the paper may be input and the result value may be repeatedly derived until the same result value is derived.

In addition, in the paper identification method using the paper identification device according to an embodiment of the present invention, the step of acquiring a transmitted light image from which a unique formation pattern can be observed for each brand of paper distributed in the market, Acquiring an FFT image by applying a Fourier transform, analyzing the input image and constructing a CNN model to analyze the paper using features included in the image, and using the built CNN model to transmit the transmitted light image. And learning the FFT image as each input value, and transmitting the transmitted light image obtained by passing infrared rays through the paper sample to be analyzed and the FFT image obtained from the transmitted light image to the plurality of CNN models on which learning has been completed, respectively input, and comparing each value derived from the plurality of CNN models to determine whether forgery or alteration is present.

As such, according to the present invention, it is possible to determine whether or not paper is forged or falsified even if it is difficult to observe a repetitive grid pattern through a transmitted light image, and since a deep learning-based CNN model is used, it is possible to predict forgery or falsification of paper streams through a non-destructive method.

1 is a configuration diagram for explaining a paper identification device according to an embodiment of the present invention.
2 is a flowchart illustrating a method of identifying paper using a paper identification device according to an embodiment of the present invention.
3 is an exemplary diagram for explaining formation patterns that appear differently for each manufacturer.
FIG. 4 is an exemplary view illustrating a process of converting an FFT image obtained in step S220 shown in FIG. 2 .
FIG. 5 is an exemplary view showing a result of observing whether a periodic component is present or not by performing FFT conversion in step S220 shown in FIG. 2 .
FIG. 6 is an exemplary view for explaining the CNN model built in step S230 shown in FIG. 2 .

Hereinafter, preferred embodiments according to the present invention will be described in detail with reference to the accompanying drawings. In this process, the thickness of lines or the size of components shown in the drawings may be exaggerated for clarity and convenience of description.

In addition, terms to be described later are terms defined in consideration of functions in the present invention, which may vary according to the intention or practice of a measurement target or operator. Therefore, definitions of these terms will have to be made based on the content throughout this specification.

Hereinafter, the paper identification device 100 according to an embodiment of the present invention will be described in more detail with reference to FIG. 1 .

1 is a configuration diagram for explaining a paper identification device according to an embodiment of the present invention.

As shown in FIG. 1, according to an embodiment of the present invention, the paper identification device 100 includes an image acquisition unit 110, an image conversion unit 120, a learning unit 130, and a determination unit 140. .

First, the image acquisition unit 110 acquires a transmitted light image.

In other words, in order to observe the surface of the paper, the user acquires a transmitted light image by photographing the paper in a state in which infrared or ultraviolet light is transmitted. Then, the acquired transmitted light image is transferred to the paper identification device 100 .

Next, the image conversion unit 120 converts the received transmitted light image into an FFT image by applying a Fourier transform.

The image conversion unit 120 sets the x-axis or the y-axis of the transmitted light image as a time axis and interprets a change in brightness of an image pixel that changes according to a change in coordinate as a signal. That is, the image converter 120 obtains a Fourier spectrum by using a Fourier transform coefficient derived from the transmitted light image. On the other hand, when FFT conversion is performed, a strong peak appears in the low-frequency region around the origin H (0, 0), and the value rapidly decreases as the distance from the origin increases, that is, toward the high-frequency region. In the image without repetitive patterns, a strong peak appears only in the low-frequency region and rapidly converges to 0 as the distance increases. However, when a repetitive pattern exists in an image, a strong peak is generated even in a high frequency region. Accordingly, the image conversion unit 120 removes the central portion, which is a low-frequency region shown in the Fourier spectrum, and extracts only values greater than or equal to a threshold value among the remaining components to obtain an FFT image.

Next, the learning unit 130 generates a first dataset using the transmitted light image, and generates a second dataset using the acquired FFT image. Then, the learning unit 130 trains the first CNN model built using the first dataset and the second CNN model built using the second dataset.

When the transmitted light image to be evaluated is received in a state in which learning is completed, the determination unit 140 inputs the received transmitted light image and the FFT image obtained from the transmitted light image to the first CNN model and the second CNN model to obtain respective result values. Acquire Then, the determination unit 140 compares the result value obtained from the first CNN model and the result value obtained from the second CNN model, and derives a final result when the two results are the same.

Hereinafter, a paper identification method using a paper identification device according to an embodiment of the present invention will be described in more detail with reference to FIGS. 2 to 6 .

2 is a flowchart illustrating a method of identifying paper using a paper identification device according to an embodiment of the present invention.

As shown in FIG. 2 , the paper identification method using the paper identification device 100 according to an embodiment of the present invention is divided into a learning step and a paper identification step.

First, looking at the learning step, the paper identification device 100 receives the transmitted light image of the paper (S210).

When the surface of a sample is observed using transmitted light for each manufacturer's paper, a unique weave pattern is observed for each sample, and this weave is called formation. Formation is most affected by the manufacturing process, especially the composition of the manufacturing line in the factory, and the difference is shown by the combination ratio of other components.

The manufacturing process of a paper company, especially the production line of a factory, can maintain its uniqueness for a long period of time because it is difficult to change once it is composed of a huge organic device. This uniqueness is expressed as a formation characteristic.

3 is an exemplary diagram for explaining formation patterns that appear differently for each manufacturer.

As shown in FIG. 3, the transmitted light image includes a unique formation pattern for each brand. Accordingly, the image acquisition unit 110 acquires a transmitted light image through which the formation pattern can be observed.

Next, the image conversion unit 120 acquires an FFT image by applying a Fourier transform to the acquired transmitted light image (S220).

First, the image converter 120 obtains a Fourier transform coefficient (H(u,v)) for a transmitted light image using Equation 1 below.

Here, H(u, v) is the coefficient of the periodic function component with frequency u in the x-axis direction and v in the y-axis direction.

However, the image is a discrete signal rather than a continuous signal, and is a signal defined in a limited finite period.

Accordingly, the image transform unit 120 calculates the Fourier transform coefficient by Equation 2 below.

는　x축 방향으로 주파수가 u/W, y축 방향으로 주파수가 v/H인 sinusoidal 주기함수를 나타낸다. here,

represents a sinusoidal periodic function with a frequency u/W in the x-axis direction and a frequency v/H in the y-axis direction.

FIG. 4 is an exemplary view illustrating a process of converting an FFT image obtained in step S220 shown in FIG. 2 .

As shown in FIG. 4, when Fourier transform is applied to the transmitted light image, a strong peak appears in the low-frequency region around the origin H (0, 0), and the value rapidly increases as the distance from the origin increases, that is, toward the high-frequency region. has a smaller form. In the image without repetitive patterns, a strong peak appears only in the low-frequency region and rapidly converges to 0 as the distance increases. However, when a repetitive pattern exists in an image, a strong peak is seen even in a high frequency region.

Accordingly, the image conversion unit 120 extracts the central portion of the FFT disease image so that the matching pattern appears well, removes the extracted central portion, and extracts only values greater than or equal to a threshold value among the remaining components.

FIG. 5 is an exemplary view showing a result of observing whether a periodic component is present or not by performing FFT conversion in step S220 shown in FIG. 2 .

As shown in FIG. 5, the image conversion unit 120 in the embodiment of the present invention applies Fourier transform to the seven transmitted light images shown in FIG. In Nos. 6 and 7, a clear dividing point was confirmed, but it was weakly observed in the remaining papers.

Next, the learning unit 130 generates a first dataset using the transmitted light image obtained in step S210, generates a second dataset using the FFT image obtained in step S220, and generates the first data. A pre-built CNN model is trained using the set and the second dataset (S230).

In detail, the learning unit 130 crops a 250 x 250 image at a random position from a 300 x 300 transmitted light image. This is repeated about 20 times to augment the data for the transmitted light image.

Also, the learning unit 130 crops a 112 x 112 image at a random location from a 300 x 300 FFT image. This is repeated approximately 20 times to augment the data for the FFT image.

Next, the learning unit 130 randomly extracts data for the augmented transmitted light image to form a first data set, and randomly extracts data for the augmented FFT image to form a second data set.

Then, the learning unit 130 learns the previously built CNN model using the first dataset and the second dataset.

To elaborate, the learning unit 130 constructs a Convolution Neural Network (CNN). The built CNN model finds unique features in a video or image, and when a new image is input, it automatically classifies by matching the most similar features based on the pre-learned unique features.

FIG. 6 is an exemplary view for explaining the CNN model built in step S230 shown in FIG. 2 .

As shown in FIG. 6, the learning unit 130 builds a CNN model based on ResNet50.

The CNN model based on ResNet50 according to an embodiment of the present invention uses a loss function to calculate the error between the predicted value and the actual label value, and transforms the final output stage of the CNN model using Equation 3 below. .

Then, the CNN model extracts the highest expected value as shown in Equation 4 below using the modified final output stage.

Then, the learning unit 130 inputs the first data set to the built first CNN model for learning, and inputs the second data set to the second CNN model for learning.

As described above, the first CNN model and the second CNN model are CNN models based on ResNet50. However, since the sizes and types of the input images are different, the network to be trained and the accuracy of the output are different, so the first CNN model and the second CNN model are classified and described according to the input images.

When step S230 is completed, the determination unit 140 receives the transmitted light image of the paper to be identified (S240).

At this time, the received transmitted light image is formed as a 300x300 gray image.

Next, the determination unit 140 receives an FFT image obtained by applying a Fourier transform to the received transmitted light image (S250).

In other words, the determination unit 140 receives the FFT image generated in the same way as in step S220. That is, the image conversion unit 120 obtains an FFT-converted image by applying a Fourier transform to a 300x300 gray image, extracts periodic components from the obtained FFT-converted image, and finally acquires an FFT image.

Next, the determination unit 140 inputs the acquired transmitted light image and the FFT image to the first CNN model and the second CNN model for which learning has been completed, and obtains a result value (S260).

To explain this again, the determination unit 140 repeatedly performs cropping on the obtained transmitted light image 50 times. Then, the determination unit 140 inputs 50 transmitted light images to the first CNN model, and the first CNN model extracts a resultant value for the transmitted light image through Equation 4.

In addition, the determination unit 140 repeats cropping 50 times from the obtained FFT image, and inputs the 50 FFT images obtained by cropping to the second CNN model. Then, the second CNN model extracts the result value for the FFT image through Equation 4.

Then, the determination unit 140 compares the result value obtained from the first CNN model and the result value obtained from the second CNN model, and derives the result value as the final result value if the result of the comparison is the same.

On the other hand, if the values are different, the determination unit 140 derives a new result by inputting the transmitted light image or the FFT image of the other part of the paper to be identified to the first CNN model and the second CNN model. That is, it is repeated until the same result value is derived.

As such, according to the present invention, it is possible to determine whether or not paper is forged or falsified even if it is difficult to observe a repetitive grid pattern through a transmitted light image, and since a deep learning-based CNN model is used, it is possible to predict forgery or falsification of paper streams through a non-destructive method.

Although the present invention has been described with reference to the embodiments shown in the drawings, this is only exemplary, and those skilled in the art will understand that various modifications and equivalent other embodiments are possible therefrom. will be. Therefore, the true technical protection scope of the present invention should be determined by the technical spirit of the claims below.

100: paper identification device
110: image acquisition unit
120: image conversion unit
130: learning unit
140: judgment unit

Claims (16)

In the paper identification device for forgery detection of documents,
An image acquisition unit for obtaining a transmitted light image to observe a unique formation pattern for each brand of paper distributed in the market;
An image conversion unit for acquiring an FFT image by applying a Fourier transform to the obtained transmitted light image;
A learning unit that analyzes an input image and builds a CNN model that analyzes paper using features included in the image, and learns the built CNN model with the transmitted light image and the FFT image as input values, respectively; And
A transmitted light image obtained by transmitting infrared rays through a paper sample to be analyzed and an FFT image obtained from the transmitted light image are respectively input into a plurality of trained CNN models, and each value derived from the plurality of CNN models is compared. Paper identification device including a determination unit for determining whether or not forgery. According to claim 1,
The image conversion unit,
Performing a Fourier transform on the transmitted light image using the following equation,

(here,

represents a sinusoidal periodic function with a frequency u/W in the x-axis direction and a frequency v/H in the y-axis direction)
Paper identification device for acquiring a Fourier spectrum using the pixel value (H (u, v) obtained through the Fourier transform. According to claim 1,
The image conversion unit,
A paper identification device for acquiring a low-frequency region corresponding to a peak portion using the obtained Fourier spectrum, deleting the low-frequency region, and then performing inverse Fourier transform to obtain an FFT image. According to claim 1,
The learning unit,
Extracting an image having a certain size from the FFT image in a cropped form, repeating this a plurality of times to augment data for the FFT image, and then generating a dataset using the transmitted light image and the augmented FFT image paper identification device. According to claim 1,
The learning unit,
Learning a first CNN model using a first dataset generated using the transmitted light image;
A paper identification device for learning a second CNN model using a second dataset generated using the FFT image. According to claim 5,
The judge,
By inputting the transmitted light image obtained by photographing the paper to be evaluated and the FFT image obtained from the transmitted light image to the first CNN model and the second CNN model on which learning has been completed, respectively,
A paper identification device for determining whether forgery is falsified by comparing the result value obtained from the first CNN model and the result value obtained from the second CNN model. According to claim 6,
The judge,
Paper identification device for obtaining each result value wi from the first CNN model and the second CNN model using the following equation:

According to claim 7,
The judge,
If the result value obtained from the first CNN model and the result value obtained from the second CNN model are the same, the obtained result value is derived as a final result;
If the result values obtained from the first CNN model and the result values obtained from the second CNN model are different, a paper identification device that receives different parts of the paper and repeatedly derives result values until the same result value is obtained. . In the paper identification method using a paper identification device,
Obtaining a transmitted light image capable of observing a unique formation pattern for each brand of paper distributed in the market;
Acquiring an FFT image by applying a Fourier transform to the obtained transmitted light image;
Analyzing the input image and constructing a CNN model that analyzes the paper using features included in the image, and learning the built CNN model with the transmitted light image and the FFT image as respective input values, and
A transmitted light image obtained by transmitting infrared rays through a paper sample to be analyzed and an FFT image obtained from the transmitted light image are respectively input into a plurality of trained CNN models, and each value derived from the plurality of CNN models is compared. Paper identification method comprising the step of determining whether forgery or falsification. According to claim 9,
Acquiring the FFT image,
Performing a Fourier transform on the transmitted light image using the following equation,

(here,

represents a sinusoidal periodic function with a frequency u/W in the x-axis direction and a frequency v/H in the y-axis direction)
Paper identification method for obtaining a Fourier spectrum using the pixel value (H (u, v) obtained through the Fourier transform. According to claim 9,
Acquiring the FFT image,
A paper identification method of acquiring a low-frequency region corresponding to a peak portion using the obtained Fourier spectrum, deleting the low-frequency region, and then performing inverse Fourier transform to obtain an FFT image. According to claim 9,
The learning step is
An image having a certain size is extracted from the FFT image in a cropped form, and data for the FFT image is augmented by repeating this multiple times, and then a plurality of datasets are generated using the transmitted light image and the augmented FFT image. How to identify the paper you create. According to claim 12,
The learning step is
Learning a first CNN model using a first dataset generated using the transmitted light image;
Paper identification method for learning a second CNN model using a second dataset generated using the FFT image. According to claim 13,
The step of determining whether the forgery is,
By inputting the transmitted light image obtained by photographing the paper to be evaluated and the FFT image obtained from the transmitted light image to the first CNN model and the second CNN model on which learning has been completed, respectively,
Paper identification method for determining whether forgery or falsification is determined by comparing the result value obtained from the first CNN model and the result value obtained from the second CNN model. According to claim 14,
The step of determining whether the forgery is,
Paper identification method for obtaining each result value wi from the first CNN model and the second CNN model using the following equation:

According to claim 15,
The step of determining whether the forgery is,
If the result value obtained from the first CNN model and the result value obtained from the second CNN model are the same, the obtained result value is derived as a final result;
If the result value obtained from the first CNN model and the result value obtained from the second CNN model are different, a paper identification method for repeatedly deriving the result value by receiving another part of the paper until the same result value is derived. .

Images