KR20190102115A - Device and method for determining banknote fitness based on deep learning - Google Patents

Abstract

A device for determining banknote suitability comprises: a banknote image input unit for receiving an input image of the banknote; a line image sensor for extracting a banknote region from the input image, normalizing the banknote region to a predetermined size and generating a normalized image; a convolutional neural network for receiving the normalized image and outputting a feature map; and a banknote suitability determining unit for generating usage suitability information according to the feature map. According to an embodiment of the present invention, banknote usage suitability can be determined regardless of an input direction of the banknote without intervention of the user.

Description

DEVICE AND METHOD FOR DETERMINING BANKNOTE FITNESS BASED ON DEEP LEARNING

The present invention relates to a technique for determining the suitability of banknotes, and more particularly to an apparatus and method for determining the suitability of deep learning-based banknotes.

The global ATM market is developing at a slower pace in developed countries, but is growing at an annual growth rate of 7.7% annually from 2015 thanks to fast-growing countries such as India and China. Although many markets reach saturation and grow at an average annual rate of 1%, they are still expected to develop globally. India has grown rapidly over the past decade and has emerged as the world's fourth largest market in 2015. Even in markets that have reached saturation, the development potential is expected to be higher when various technologies such as smart ATM are combined.

Machines for counting and sorting bills, such as ATM (cash dispensers) or bill counters, have functions such as bill types, counterfeit detection, serial number recognition, and suitability for use. Determination of suitability for use is a process of evaluating whether or not it is usable by evaluating physical conditions of banknotes such as staining, tearing and discoloration. This not only helps recycle paper money, but also affects the processing speed and performance of the above mentioned machines. Existing studies for banknote suitability detection detect the entire banknote area, but mainly related to technology that is vulnerable to lighting changes, sensor wavelengths, etc., mainly because it detects only the brightness characteristics of the banknote image. In order to compensate for this, a technique of finely detecting a region of interest has been developed, which has advantages of reducing an input image size and processing time. However, existing research using the region of interest finds the region of interest manually and is vulnerable to contamination of regions outside the region of interest.

Background of the present invention is disclosed in Korean Patent Laid-Open No. 2005-0075153.

The problem to be solved of the present invention is to provide an apparatus and method for determining a bill suitability for determining the suitability of the use of bills.

According to one aspect of the present invention, a banknote conformity determination apparatus is provided.

According to an embodiment of the present invention, a bill acceptor judging device receives a banknote image input unit that receives an input image of a banknote, a line extracting a banknote region from the input image, and normalizing the banknote region to a predetermined size to generate a normalized image. An image sensor; There is provided a bill conformance determination device including a convolutional neural network for receiving the normalized image and outputting a feature map and a bill conformance determination unit for generating usage suitability information according to the feature map.

The convolutional neural network uses a CNN structure including five convolutional layers, three pooling layers, and three fully connected layers, wherein the kernel sizes used in the convolutional layers are 7 × 7 × 1, 5 × 5 × 96, and 3 Any one of × 3 × 128 and 3 × 3 × 256, the padding number is any one of 0 to 2, the number of strides is 1 or 2, the number of filters is any one of 96, 128, and 256, respectively The kernel size of the pooling layer may be any one of 3 × 3 × 96, 3 × 3 × 128, and 3 × 3 × 256, the number of strides may be 2, and the number of padding may be zero.

The three fully connected layers may output 4096, 2048, and two feature maps, respectively, or may output 4096, 2048, and three feature maps, respectively.

The convolutional neural network may finally output three or two feature maps.

The usage suitability information may be information indicating a suitability, normal or inadequate use of the bill, or suitability or incompatibility for the use of the bill according to the feature map output from the convolutional neural network.

According to another aspect of the present invention, there is provided a banknote suitability determination method.

According to an embodiment of the present invention, a method for determining a bill suitability includes the steps of: receiving a bill input image of a bill by the apparatus for determining a bill suitability; Extracting a bill region from the input image, normalizing the bill region to a predetermined size, generating a normalized image, receiving the normalized image through a convolutional neural network, and outputting a feature map and using the feature map There is provided a banknote conformance determination method comprising generating conformance information.

According to one embodiment of the present invention, it is possible to determine whether the bill is suitable for use regardless of the input direction of the bill.

In addition, according to an embodiment of the present invention, it is possible to determine the suitability of the use of the bill without the user's intervention.

In addition, according to an embodiment of the present invention, it is possible to determine the suitability for use of all areas of the bill.

1 is a block diagram illustrating an apparatus for determining a bill fit according to an embodiment of the present invention.
FIG. 2 is a diagram illustrating an image of banknotes of various currencies determined by the banknote conformance determining apparatus according to an embodiment of the present invention. FIG.
3 is a diagram illustrating a normalized image extracted from an input image by a line image sensor according to an embodiment of the present invention.
4 is a diagram illustrating a structure of a convolutional neural network used by the device for determining a bill fit according to an embodiment of the present invention.
5 is a diagram illustrating a filter used by the device for determining the bill fits according to an embodiment of the present invention.
FIG. 6 is a flowchart illustrating a process of determining whether a banknote is suitable for use by a banknote conformity determining apparatus according to an embodiment of the present invention. FIG.
7 is a graph showing the accuracy of the use suitability judgment of the banknote suitability determination apparatus according to an embodiment of the present invention.
8 is a view showing the number of images used for cross-validation by the device for determining a bill according to an embodiment of the present invention.
9 is a diagram illustrating a test result of the use suitability determination on the bill of the original currency of the bill suitability determination apparatus according to an embodiment of the present invention.
10 is a diagram illustrating a test result of the use suitability determination on the banknote of the INR of the bill suitability determination apparatus according to an embodiment of the present invention.
11 is a diagram illustrating a test result of the use suitability determination for the banknotes of the USD of the bill suitability determination apparatus according to an embodiment of the present invention.
12 is a view comparing the test results of the original currency of the banknote conformity determination apparatus according to an embodiment of the present invention and the test results of the conventional banknote conformity determination apparatus.
13 is a view comparing the test result of the INR of the banknote conformity determination apparatus according to an embodiment of the present invention and the test result of the conventional banknote conformity determination apparatus.
14 is a view comparing the test result of the banknote conformity determination apparatus according to the embodiment of the present invention and the test result of the conventional banknote conformity determination apparatus.

As the inventive concept allows for various changes and numerous embodiments, particular embodiments will be illustrated in the drawings and described in detail in the written description. However, this is not intended to limit the present invention to specific embodiments, it should be understood to include all modifications, equivalents, and substitutes included in the spirit and scope of the present invention. In describing the present invention, when it is determined that the detailed description of the related known technology may unnecessarily obscure the subject matter of the present invention, the detailed description thereof will be omitted. Also, the singular forms used in the specification and claims are to be interpreted as generally meaning "one or more", unless stated otherwise.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings, and in the following description with reference to the accompanying drawings, the same or corresponding components are given the same reference numerals and redundant description thereof will be omitted. Shall be.

1 is a block diagram illustrating an apparatus for determining a bill fit according to an embodiment of the present invention, Figure 2 illustrates an image of a bill of various currencies determined by the apparatus for determining a bill fit according to an embodiment of the present invention FIG. 3 is a diagram illustrating a normalized image extracted from an input image by a line image sensor according to an embodiment of the present invention, and FIG. 4 illustrates a banknote conformity determination according to an embodiment of the present invention. 5 is a diagram illustrating a structure of a convolutional neural network used by the apparatus, and FIG. 5 is a diagram illustrating a filter used by the apparatus for determining a bill fit according to an embodiment of the present invention.

Referring to FIG. 1, the banknote conformance determining apparatus 100 according to an embodiment of the present invention may include a banknote image input unit 110, a line image sensor 120, a convolutional neural network 130, and a banknote conformance determination unit 140. Include.

The bill image input unit 110 receives an image (hereinafter, referred to as an input image) photographed about the bill. In this case, the input image may be an image of banknotes of various currencies, and may be an image of banknotes of various directions. For example, the bill image input unit 110 may receive input images for various bills of various currencies such as won, INR, and USD, as shown in 210, 220, and 230 of FIG. 2. The bill image input unit 110 transmits the input image to the line image sensor 120.

The line image sensor 120 extracts a region corresponding to the banknote from the input image, and extracts a normalized image in which the extracted region is normalized to a specified size. For example, the line image sensor 120 extracts a bill area from an input image composed of a bill area and a non-bill area as shown in the images illustrated on the left of FIG. 3, and normalizes the bill area to a predetermined size. Normalized images may be generated as shown in 300, 310, 320, and 330 of FIG. 3. The line image sensor 120 transmits the normalized image to the convolutional neural network 130.

The convolutional neural network 130 receives a normalized image and performs a learned process to determine whether the bill is suitable, normal, or inadequate for the use of the bill, or determines whether it is suitable or inadequate for the use of the bill. For example, with reference to FIG. 4, the convolutional neural network 130 may use a CNN structure including five convolutional layers, three pulling layers, and three fully connected layers. As an example of the convolutional neural network 130, the first convolutional layer generates a feature map using 96 filters for an input of 55 × 23 × 96 (width, height, and channel size). For example, the convolutional neural network 130 may output 27 × 11 × 96 through the maximum pooling layer after the acquired feature map passes through the ReLU layer and the CCN layer of the first convolutional layer, and the maximum pooling layer is a kernel. Size 3 × 3 × 96, stride 2. The kernel size used in the first convolutional layer is 7 × 7 × 1, the padding number is 0, the stride number is 2, and the number of filters is 96. 500, 510, and 520 of FIG. 5 illustrate filters used. As an example of the convolutional neural network 130, the second convolutional layer generates a feature map using 128 filters for an input of 27 × 11 × 128 (width, height, and channel size). For example, the convolutional neural network 130 may output 13 × 5 × 128 through the maximum pooling layer after the acquired feature map passes through the ReLU layer and the CCN layer of the second convolutional layer, and the maximum pooling layer is a kernel. Size 3 × 3 × 128, stride 2. The kernel size used in the second convolutional layer is 5x5x96, the padding number is 2, the stride number is 1, and the number of filters is 128. As an example of the convolutional neural network 130, the third convolutional layer generates a feature map using 256 filters for an input of 13 × 5 × 256 (width, height, and size of the channel). The convolutional neural network 130 is transmitted to the fourth convolutional layer after, for example, the obtained feature map passes through the ReLU layer of the third convolutional layer. The kernel size used in the third convolutional layer is 3 × 3 × 128, the padding number is 1, the stride number is 1, and the number of filters is 256. As an example of the convolutional neural network 130, the fourth convolutional layer generates a feature map using 256 filters for an input of 13 × 5 × 256 (width, height, and channel size). The convolutional neural network 130 is transmitted to the fifth convolutional layer, for example, after the acquired feature map passes through the ReLU layer of the fourth convolutional layer. The kernel size used in the fourth convolution layer is 3 × 3 × 256, the padding number is 1, the stride number is 1, and the number of filters is 256. As an example of the convolutional neural network 130, the fifth convolutional layer generates a feature map using 128 filters for an input of 13 × 5 × 128 (width, height, and size of the channel). The convolutional neural network 130 may output, for example, 6 × 2 × 128 through the maximum pooling layer after the acquired feature map passes through the ReLU layer of the second convolutional layer, and the maximum pooling layer is the kernel size 6 ×. 2 × 128, stride 2. The kernel size used in the fifth convolutional layer is 3 × 3 × 256, the padding number is 1, the stride number is 1, and the number of filters is 128. The input image can finally output the feature map through five convolutional layers, seven ReLU layers, two CCN layers, three pooling layers, and three fully connected layers.

Thereafter, the deep learning unit 132 may output, for example, feature maps of 4096, 4096, and 2 or 3 through three additional fully connected layers. Here, Equation (1) below is used for the Softmax function in the third fully connected layer.

(1)

(One)

Where r is an array of output neurons.

The convolutional neural network 130 may use the training data increase to solve low recognition accuracy in the CNN-based recognition system due to data overfitting.

The feature map that the convolutional neural network 130 finally outputs may indicate either good, normal, or non-conformance of the paper money, or may indicate good or bad for the use of the paper money. The convolutional neural network 130 transmits the feature map to the bill conformity determination unit 140.

The banknote conformity determining unit 140 generates and outputs usage conformance information indicating the suitability of using the banknote according to the feature map received from the convolutional neural network 130.

FIG. 6 is a flowchart illustrating a process of determining whether a banknote is suitable for use by a banknote conformity determining apparatus according to an embodiment of the present invention. Each step to be described below is to be collectively referred to as the bill fit determination device for the concise and clear description of the process or invention carried out through each functional unit constituting the bill fit determination device.

Referring to FIG. 6, in operation 610, the banknote conformance determining apparatus receives an input image of a banknote. For example, the banknote conformity determining apparatus may receive an input image from a photographing apparatus for photographing a banknote through visible light.

In operation 620, the banknote conformity determining apparatus extracts a banknote region corresponding to the banknote from the input image and normalizes the banknote region to a predetermined size to generate a normalized image.

In operation 630, the banknote conformity determining apparatus generates a feature map indicating usage suitability for the normalized image through the convolutional neural network 130.

In operation 640, the banknote conformity determining apparatus generates and outputs usage suitability information indicating usage suitability for the photographed banknote according to the feature map.

FIG. 7 is a graph showing the accuracy of the use suitability judgment of the banknote conformity determining apparatus according to an embodiment of the present invention, and FIG. 8 is the number of images used for the cross-validation by the apparatus for determining the bill conformance according to an embodiment of the present invention. 9 is a view illustrating a test result of the use suitability judgment for the bill of the original currency of the bill suitability determination apparatus according to an embodiment of the present invention, Figure 10 is a bill suitability according to an embodiment of the present invention Fig. 11 is a diagram illustrating the test result of the use suitability judgment on the banknote of the INR of the determination device, Figure 11 illustrates the test result of the use suitability determination on the USD bill of the banknote suitability determination apparatus according to an embodiment of the present invention Drawing.

As illustrated in FIG. 8, the input image of the banknote corresponding to the fit (Fit), the input image of the banknote corresponding to the normal (Fit), unsuitable for use (Unfit) for the banknote suitability determination apparatus according to an embodiment of the present invention After learning the convolutional neural network 130 for the input image of the bill corresponding to the), a test was conducted to confirm the accuracy of each fit with the input image.

Accordingly, the banknote conformity determining apparatus exhibits the same accuracy as the 720 and 11 of FIG. 7 with respect to 710 of FIG. 7 and FIG.

12 is a view comparing the test results of the original currency of the banknote conformity determination apparatus according to an embodiment of the present invention and the test results of the conventional banknote conformity determination apparatus, Figure 13 is a banknote according to an embodiment of the present invention FIG. 14 illustrates a comparison between a test result of an INR of a suitability determination device and a test result of a conventional banknote suitability determination device, and FIG. 14 illustrates a test result of USD and a conventional banknote suitability determination device according to an embodiment of the present invention. It is a figure which compared the test result with a banknote conformity judgment apparatus.

As illustrated in FIG. 12, the test was conducted by setting the directions of banknotes in four directions for two kinds of original currency, and the average accuracy of the conventional banknote suitability judgment apparatus was only 50% to 80%, The banknote conformity determining apparatus according to an embodiment of the present invention has a high accuracy of about 90% in all directions.

As illustrated in FIG. 13, six different types of INRs were tested by setting the directions of banknotes in four directions, and the average accuracy of the conventional banknote suitability determination device was distributed from 80% to 100%. The banknote conformity determining apparatus according to an embodiment of the present invention has a high accuracy of 99% or more in all directions.

In addition, as illustrated in 14, four kinds of USD were set in four directions for banknotes in the test, and the average accuracy of the conventional banknote suitability determination device was distributed from 70% to 90%. The previous test result of the banknote conformity determining apparatus according to an embodiment of the present invention shows more than 90%.

Therefore, the banknote conformity determining apparatus according to an embodiment of the present invention can perform the use suitability determination having a high accuracy for the bills of each country, regardless of the direction of the bills in the input image.

The method for determining a bill suitable for a bill according to an embodiment of the present invention may be implemented in the form of program instructions that can be executed by various computer means and recorded in a computer readable medium. Computer-readable media may include, alone or in combination with the program instructions, data files, data structures, and the like. The program instructions recorded on the computer readable medium may be those specially designed and constructed for the present invention, or may be known and available to those skilled in the computer software arts. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tape, optical media such as CD-ROMs, DVDs, and magnetic disks, such as floppy disks. Hardware devices specially configured to store and execute program instructions such as magneto-optical media and ROM, RAM, flash memory and the like. In addition, the above-described medium may be a transmission medium such as an optical or metal wire, a waveguide, or the like including a carrier wave for transmitting a signal specifying a program command, a data structure, and the like. Examples of program instructions include not only machine code generated by a compiler, but also high-level language code that can be executed by a computer using an interpreter or the like. The hardware device described above may be configured to operate as one or more software modules to perform the operations of the present invention, and vice versa.

So far, the present invention has been described with reference to the embodiments. Those skilled in the art will appreciate that the present invention can be implemented in a modified form without departing from the essential features of the present invention. Therefore, the disclosed embodiments should be considered in descriptive sense only and not for purposes of limitation. The scope of the present invention is shown in the claims rather than the foregoing description, and all differences within the scope will be construed as being included in the present invention.

Claims (11)

A bill image input unit which receives an input image of a bill;
A line image sensor extracting a bill region from the input image and normalizing the bill region to a predetermined size to generate a normalized image;
A convolutional neural network receiving the normalized image and outputting a feature map; And
A banknote conformance determination apparatus comprising a banknote conformity determination portion that generates use conformity information according to the feature map.
The method of claim 1,
The convolutional neural network
Use a CNN structure with five convolutional layers, three pooling layers, and three fully connected layers,
The kernel size used in each of the convolution layers is any one of 7 × 7 × 1, 5 × 5 × 96, 3 × 3 × 128, and 3 × 3 × 256, and the padding number is any one of 0 to 2, and stride The number is 1 or 2, the number of filters is any one of 96, 128, and 256,
The kernel size of each pooling layer is any one of 3 × 3 × 96, 3 × 3 × 128, and 3 × 3 × 256, the number of strides is 2, and the number of padding is 0.
The method of claim 2,
And the three fully connected layers output 4096, 2048 and two feature maps respectively, or output 4096, 2048 and three feature maps, respectively.
The method of claim 3,
And the convolutional neural network finally outputs three or two feature maps.
The method of claim 4, wherein
The use suitability information is information indicating the suitability, normal or inadequate for the use of the bill, or the information indicating the suitability or incompatibility for the use of the bill according to the feature map output from the convolutional neural network. Device.
In the method in which a banknote conformity judgment device judges the use suitability of banknotes,
Receiving an input image of a bill;
Extracting a banknote region from the input image and normalizing the banknote region to a predetermined size to generate a normalized image;
Outputting a feature map by receiving the normalized image through a convolutional neural network; And
And generating usage suitability information according to the feature map.
The method of claim 6,
The convolutional neural network
Use a CNN structure with five convolutional layers, three pooling layers, and three fully connected layers,
The kernel size used in each of the convolution layers is any one of 7 × 7 × 1, 5 × 5 × 96, 3 × 3 × 128, and 3 × 3 × 256, and the padding number is any one of 0 to 2, and stride The number is 1 or 2, the number of filters is any one of 96, 128, and 256,
The kernel size of each pooling layer is any one of 3 × 3 × 96, 3 × 3 × 128, and 3 × 3 × 256, the number of strides is 2, and the number of padding is zero.
The method of claim 7, wherein
And the three fully connected layers output 4096, 2048 and two feature maps respectively, or output 4096, 2048 and three feature maps, respectively.
The method of claim 8,
And the convolutional neural network finally outputs three or two feature maps.
The method of claim 9,
The use suitability information is information indicating the suitability, normal or inadequate for the use of the bill, or the information indicating the suitability or incompatibility for the use of the bill according to the feature map output from the convolutional neural network. Way.
A computer program which executes the method of determining a bill suitable for any one of claims 6 to 10 and recorded on a computer-readable recording medium.

Images