EP3726486A1

EP3726486A1 - Paper sheet contamination assessment device and paper sheet contamination assessment method

Info

Publication number: EP3726486A1
Application number: EP17934505.3A
Authority: EP
Inventors: Toshihiko BANJOYA; Ryo Ikemoto; Chihiro NAGAI
Original assignee: Glory Ltd
Current assignee: Glory Ltd
Priority date: 2017-12-15
Filing date: 2017-12-15
Publication date: 2020-10-21
Also published as: JP6878623B2; WO2019116542A1; JPWO2019116542A1; EP3726486A4

Abstract

In order to determine existence of a color stain over a wide area on a sheet, a sheet stain determination apparatus is provided with: an image acquisition unit configured to acquire an image of a sheet; a memory configured to store therein reference data for specifying color change of the sheet due to aging; and a determination unit configured to, based on the image of the sheet and the reference data, detect a sheet whose color has changed over a region of a predetermined area or more from an original color after printing of the sheet, and determine whether the detected sheet is a sheet whose color has changed due to aging or a color-stained sheet whose color has changed due to a color stain.

Description

TECHNICAL FIELD

The present invention relates to a sheet stain determination apparatus and a sheet stain determination method for determining a stain on a sheet, and particularly to those for determining existence of a color stain caused by an ink, a dye, or the like.

BACKGROUND ART

Conventionally, sheet handling apparatuses that recognize denominations, authenticity, fitness, etc., of sheets have been used. In recognizing fitness, stained sheets can be detected. For example, Patent Literature 1 discloses a sheet determination apparatus for determining the degree of stain on a sheet. This apparatus applies light of a predetermined wavelength onto a banknote, and detects wrinkles and stains in a predetermined area based on the states of reflection and transmission of the light. For easy detection of wrinkles and stains, the angle and the wavelength of the light applied to the banknote can be changed.

CITATION LIST

[PATENT LITERATURE]

[PTL 1] Japanese Laid-Open Patent Publication No. 2010-277252
[PTL 2] Japanese Laid-Open Patent Publication No. H7-182518

SUMMARY OF THE INVENTION

PROBLEMS TO BE SOLVED BY THE INVENTION

However, there are stains that cannot be accurately detected by the conventional art. For example, even when blue light is applied to a sheet stained in blue by an ink, a dye, or the like, this color stain is not likely to be detected. Since the conventional art detects local stains such as wrinkles and scribbles, a color stain over a wide area of a sheet is not likely to be detected. Furthermore, only detecting that the color of a sheet is different over a wide area from the color of a normal sheet is likely to cause erroneous detection in which fading due to aging is detected as a color stain.
For example, in the Hindu festival, Holi, where people throw colored powder and colored water on each other, many banknotes are dyed in red, green, etc. For another example, in order to deal with theft of banknotes, some cash handling apparatuses spray a special ink onto banknotes to color the banknotes upon detecting an abnormality. Although it is desired to detect those banknotes as color-stained notes separately from other banknotes, it is difficult for the conventional art to accurately detect them.
The present invention has been made in view of the problems of the conventional art, and an object of the present invention is to provide a sheet stain determination apparatus and a sheet stain determination method for detecting whether a sheet has a color stain.

SOLUTION TO THE PROBLEMS

In order to solve the above problems and achieve the object, a sheet stain determination apparatus according to one aspect of the present invention includes: an image acquisition unit configured to acquire an image of a sheet; a memory configured to store therein reference data for specifying color change of the sheet due to aging; and a determination unit configured to, based on the image of the sheet and the reference data, detect a sheet whose color has changed over a region of a predetermined area or more from an original color after printing of the sheet, and determine whether the detected sheet is a sheet whose color has changed due to aging or a color-stained sheet whose color has changed due to a color stain.
In the above configuration, a feature value is calculated from pixel values of a plurality of colors obtained through color separation of a color image of a sheet, and the feature value is substituted in a predetermined evaluation formula to obtain an evaluation value. The reference data is a threshold value for determining whether or not the obtained evaluation value is a value indicating that the sheet is a color-stained sheet. The determination unit calculates the evaluation value from an image of a sheet acquired by the image acquisition unit and compares the evaluation value with the threshold value to determine whether or not the sheet is a color-stained sheet.
In the above configuration, the feature value is a value indicating a correlation feature of the pixel values of the plurality of colors.
In the above configuration, the evaluation formula is an formula for calculating a principal component score of a predetermined principal component, the formula being obtained through principal component analysis performed on feature values of images of a plurality of sheets having no color change due to aging and a plurality of sheets having color change due to aging.
In the above configuration, each image of the plurality of sheets having color change due to aging is generated artificially by changing color components of an image obtained by capturing a sheet.
In the above configuration, the pixel values of the plurality of colors include a pixel value of an R component, a pixel value of a G component, and a pixel value of a B component.
In the above configuration, the determination unit determines, based on the evaluation value, whether or not the color stains on the plurality of color-stained sheets are of the same color.
In the above configuration, the determination unit calculates, for a partial region that is set as a block on the sheet, the evaluation value from pixel values of pixels forming the block to determine whether or not the sheet is a color-stained sheet.
In the above configuration, the block is one of a plurality of partial regions into which an entire face of the sheet is divided, and the determination unit determines, based on the evaluation value calculated for each block, whether the color-stained sheet is an entirely color-stained sheet or a partially color-stained sheet.
In the above configuration, the sheet stain determination apparatus further includes: an inlet unit configured to receive a plurality of sheets; a transport unit configured to transport the sheets received in the inlet unit one by one; and a plurality of stacking units configured to stack sheets therein. The plurality of sheets received in the inlet unit are sorted and stacked in the plurality of stacking units, based on result of determination by the determination unit.
A sheet stain determination method, by which a sheet stain determination apparatus determines existence of a color stain, according to one aspect of the present invention includes: acquiring an image of a sheet; and determining, based on an acquired image of a sheet whose color has changed over a region of a predetermined area or more from an original color after printing of the sheet and on reference data for specifying color change of the sheet due to aging, whether the sheet is a sheet whose color has changed due to aging or a color-stained sheet whose color has changed due to a color stain.

ADVANTAGEOUS EFFECTS OF THE INVENTION

According to the present invention, it is possible to detect that a region of a predetermined area or more on a sheet has changed into a color different from an original color immediately after the printing of the sheet. Then, it is possible to determine whether the detected color change is caused by aging or a color stain due to an ink, a dye, or the like, by using reference data that is prepared in advance for specifying a color change caused by aging. Thus, a sheet having a color stain can be detected distinguishably from a sheet whose color has changed due to aging.

BRIEF DESCRIPTION OF THE DRAWINGS

[FIG. 1] FIG. 1 illustrates sheet stain determination executed by a banknote handling apparatus.
[FIG. 2] FIGS. 2A and 2B illustrate the structure of the banknote handling apparatus.
[FIG. 3] FIG. 3 is a block diagram schematically showing the functional structure of the banknote handling apparatus.
[FIG. 4] FIG. 4 illustrates banknote images used for calculation of feature values.
[FIG. 5] FIGS. 5A, 5B and 5C illustrate a feature value calculation method.
[FIG. 6] FIGS. 6A and 6B show examples of sets obtained by plotting principal component scores of a first principal component, a second principal component, and a third principal component.
[FIG. 7] FIG. 7 is a flowchart showing an example of process procedures for fitness recognition and stain determination.

DESCRIPTION OF EMBODIMENTS

Hereinafter, preferred embodiments of a sheet stain determination apparatus and a sheet stain determination method according to the present invention will be described in detail with reference to the accompanying drawings. The sheet stain determination method is used in, for example, a sheet handling apparatus that can handle a plurality of sheets one by one continuously. The kind of sheets to be subjected to stain determination is, but not limited to, "banknote" in the present embodiment. Specifically, the sheet stain determination apparatus and the sheet stain method will be described in relation to a banknote handling apparatus (sheet handling apparatus) that recognizes denominations, authenticity, fitness, etc., of banknotes, and sorts the banknotes for each kind, based on the recognition result.
First, the outline of the sheet stain determination method will be described. FIG. 1 illustrates sheet stain determination executed by a banknote handling apparatus 1. The banknote handling apparatus 1 shown in FIG. 1 functions as the sheet stain determination apparatus. The banknote handling apparatus 1 receives a plurality of banknotes 100, feeds the received banknotes one by one into inside the apparatus 1, and recognizes a denomination, authenticity, fitness, etc., of each banknote. Based on the recognition result, the banknote handling apparatus 1 sorts the banknotes by kind, and discharges to stack the banknotes into reject units, stacking units, and the like for each kind.
The banknote handling apparatus 1 performs fitness recognition to sort the banknotes 100 into a fit note 101 and unfit notes 111 to 114 as shown in FIG. 1. For example, a banknote that is reusable in the market is recognized as a fit note while a banknote that is not reusable in the market is recognized as an unfit note. The banknote handling apparatus 1 acquires values indicating optical features from an image obtained by capturing a banknote to determine whether the banknote is stained, wrinkled, or torn, for example. Further, the banknote handling apparatus 1 acquires values indicating the thickness of the banknote to determine whether tape or the like is adhered to the banknote. Each determination is performed by comparing the value acquired from the banknote with a threshold value that is prepared in advance for distinguishing fit notes from unfit notes. The kind of banknotes to be recognized as unfit notes can be changed by changing the threshold value used as reference data.
Unfit notes include various kinds of banknotes such as torn notes, mutilated notes having a missing part, notes having serious damage such as creases and wrinkles, notes with scribbles, notes discolored due to aging, and notes having color stains due to an ink, a dye, or the like. The banknote handling apparatus 1 can distinguish and sort these unfit notes by kind. Specifically, the banknote handling apparatus 1 can distinguish color-stained notes, i.e., notes discolored by color stains, from normally soiled notes such as notes with scribbles and notes discolored due to aging. In order to describe this function, hereinafter, the description continues on the assumption that each of the unfit notes 111 to 114 is either a normally soiled note that has discolored due to aging or a color-stained note that has discolored due to color stains.
After the fit/unfit sorting through the fitness recognition, the banknote handling apparatus 1 further sorts the unfit notes 111 to 114 into the normally soiled note 111 and the color-stained notes 112 to 114. The banknote handling apparatus 1 can further sort, by color, the notes 112 and 113 stained in different colors. The banknote handling apparatus 1 can further sort the color-stained notes 112 to 114 into the entirely color-stained notes 112 and 113 and the partially color-stained note 114. In FIG. 1, different colors are shown schematically for the color-stained notes 112 to 114 that are stained in these colors different from the original color immediately after printing of banknotes. The note 112 is entirely stained in blue, the note 113 is entirely stained in green, and the note 114 is partially stained in red.
Each of the color-stained notes (color-stained sheets) 112 to 114 is a banknote in which a region equal to or greater than a predetermined area is colored (stained with color) by an ink, a dye, or the like. The banknote handling apparatus 1 can detect a color-stained note, which has been stained by an ink, a dye, or the like and has changed to a color different from that of the normal banknote, distinguishably from other banknotes. Specifically, the color-stained note can be detected distinguishably from a banknote with scribbles such as characters and lines, and a banknote stained in the small area. Furthermore, the color-stained note can be detected distinguishably from a banknote that has a color change, which is not caused by a color stain but caused by fading due to aging, over a wide area as compared with its original color immediately after printing of the banknote.
For example, a banknote is divided into a plurality of partial regions, i.e., blocks, and the banknote handling apparatus 1 determines a banknote in which at least one block is entirely stained with color, as a color-stained note. For example, a rectangular region of about 30 mm in both length and width is set as one block, and a banknote in which one block is entirely stained with color is detected as a color-stained note. Based on the number of color-stained blocks, the color-stained notes 112, 113 that are entirely stained with color can be distinguished from the color-stained note 114 that is partially stained with color. The setting of the size and positions of the blocks used for determining existence of a color stain can be changed.
After a long time has passed since issuance of a banknote, the banknote will fade due to aging, and its color would have changed entirely. Specifically, for example, a blank portion (paper color) of the banknote is faded yellow. For another example, colors of patterns such as characters, portraits, and geometric patterns, which are clearly printed with inks of red, blue, etc., become dull. The banknote handling apparatus 1 can distinguish the normally soiled note 111 whose color has changed due to aging and the color-stained notes 112 to 114 whose color has changed due to a stain of a dye or the like.
The banknote handling apparatus 1 uses a value indicating the color of each block set on a banknote as a feature value for color stain determination. For example, using a color image obtained by capturing a banknote, pixel values (luminance values) of R (red), G (green) and B (blue) of pixels that form a block on the image are obtained, and a feature value of the block is obtained from total sums of pixel values of each color of these pixels. Alternatively, for example, total sums of values indicating correlation features of pixel values of color components of pixels forming a block are used to obtain a feature value of the block. Specifically, for example, values of higher-order local auto-correlation (HLAC) features of pixel values of R, G, and B are used for the feature value, which will be described later in detail.
A diagram shown in a frame 121 in FIG. 1 schematically shows a state in which a plurality of feature values indicating the colors of blocks are calculated from a color image obtained by capturing a banknote, and are plotted on a multi-dimensional feature space. When data of the blocks are plotted based on the feature values, data 131 of blocks of a banknote that is unused after issuance and therefore has neither color stain nor fading (hereinafter referred to as "brand-new note"), form a set 130. Meanwhile, data 141 of blocks of a banknote that has faded due to aging (ordinary soiled note) form another set 140.
The degree of fading due to aging varies depending on the number of years and the usage state after issuance. Data of a banknote that is significantly discolored due to the fading is plotted in the set 140 of faded notes. Data 151 of a banknote that is faded due to aging but is not yet to be included in the set 140 of faded notes, is plotted on or near a straight line connecting a centroid of the set 130 of brand-new notes and a centroid of the set 140 of faded notes. As for data of a faded banknote, the greater the degree of fading is, the closer to the set 140 of faded notes the data is plotted. Determination as to whether a banknote discolored due to fading is a fit note or an unfit note can be made by setting a threshold value for fitness. For example, when a boundary between the set 130 of brand-new notes and the set 140 of faded notes is set as a threshold, the banknote is determined as a fit note when the data thereof is plotted on the side of the set 130 of brand-new notes with respect to the boundary, while it is determined as an unfit note when the data thereof is plotted on the side of the set 140 of faded notes with respect to the boundary. Setting of such a threshold value allows the banknote indicated by the data 151 to be sorted as either the fit note 101 or any of the unfit notes 111 to 114. Hereinafter, the description continues on the assumption that the banknote of the data 151 is sorted as the fit note 101.
A feature value calculated from pixel values of a plurality of colors indicating color components of a block moves in a band-like region that linearly connects the set 130 of brand-new notes and the set 140 of faded notes, as the color of the banknote gradually changes due to fading with age. Specifically, as the color change due to fading advances, the feature value moves from a position near the set 130 of brand-new notes toward a position near the set 140 of faded notes. Meanwhile, data 161, 162 of color-stained notes, in which almost the entirety of the block is colored, are plotted at positions outside the set 130 of brand-new notes, the set 140 of faded notes, and the band-like region connecting these sets.
As shown in the frame 121, the banknote handling apparatus 1 sorts the banknotes by kind, based on the positional relationship between the set 130 of brand-new notes, the set 140 of faded notes, and the data 131, 141, 151, 161, and 162 plotted in the feature space based on the feature values of the blocks. Specifically, the banknote indicated by the data 131 and the banknote indicated by the data 151 are sorted as the fit note 101. The banknote indicated by the data 141 is sorted as the normally soiled note 111. The banknote indicated by the data 161 and the banknote indicated by the data 162 are sorted as any of the color-stained notes 112 to 114 according to the area size and color of the color stain.
For more accurate detection of color-stained notes, the banknote handling apparatus 1 can perform stain determination using principal component analysis. When principal component analysis is performed with a multi-dimensional feature value indicating a color being a variable, a change in the feature value caused by a color change due to aging is obtained as a first principal component as shown by an arrow in the frame 121 in FIG. 1. The banknote handling apparatus 1 uses a second principal component and a third principal component, having the next highest contribution rates, for detecting color-stained notes.
A diagram shown in a frame 122 in FIG. 1 schematically shows a state in which the values of the second principal component and the third principal component calculated from the feature values are plotted on a principal component space. As shown in the frame 122, based on the second principal component and the third principal component, a set 171 including the data 131 of a brand-new note and the data 141, 151 of faded notes can be separated from a set 172 including the data 161, 162 of color-stained notes. The banknote handling apparatus 1 distinguishes color-stained notes from other banknotes based on this result.
Specifically, formulas for calculating principal component scores of the second principal component and the third principal component from feature values, which is obtained through principal component analysis, are used as evaluation formulas. The banknote handling apparatus 1 calculates feature values from pixel values indicating color components of a block, and substitutes the calculated feature values in the evaluation formulas to calculate principal component scores of the second principal component and the third principal component. Then, based on the calculated scores as evaluation values, the banknote handling apparatus 1 evaluates whether or not the block has a color stain.
As shown in the frame 122 in FIG. 1, when data plotted on the principal component space based on the principal component scores of the second principal component and the third principal component are included in the set 172 of color-stained notes, this block is determined to be a block of a color-stained note. In other words, it is determined that there is a color stain in this block. Thus, the color-stained notes 112 to 114 can be distinguished from the fit note 101 and the normally soiled note 111.
Based on the determination result for each block as to whether the block has a color stain, the banknote handling apparatus 1 distinguishes the entirely color-stained notes 112, 113 from the partially color-stained note 114. Furthermore, based on the positional relationship such as the distance between the two pieces of data 161 and 162 of color-stained notes and the directions thereof on the principal component space, the banknote handling apparatus 1 determines whether or not the color stains of the color-stained notes indicated by the data 161, 162 are the same color. Thus, the color-stained notes 112 and 113 having the color stains of different colors can be distinguished from each other.
Specifically, if at least one of all blocks set on a banknote is a color-stained block, the banknote handling apparatus 1 determines that this banknote is any of the color-stained notes 112 to 114. Then, if some of the blocks have color stains while the remaining blocks do not have color stains, the banknote handling apparatus 1 determines that this banknote is the partially color-stained note 114. If all the blocks have color stains, the banknote handling apparatus 1 determines that this banknote is the entirely color-stained note 112 or 113.
Based on the positional relationship between the data 161 and the data 162 both indicating color stains, the banknote handling apparatus 1 determines whether or not the color of the stain indicated by the data 161 is different from the color of the stain indicated by the data 162. Based on the determination result, the banknote handling apparatus 1 sorts, by color, the entirely color-stained notes 112, 113. Likewise, partially color-stained notes having stains of different colors can also be sorted by color by distinguishing color difference.
In FIG. 1, in order to simplify the description, a banknote whose color has changed due to fading with age is regarded as the normally soiled note 111 and distinguished from the color-stained notes 112 to 114. However, setting of fitness recognition is not limited thereto. For example, a banknote faded due to aging may be sorted as a fit note. Likewise, the color-stained notes 112 to 114 may also be sorted as fit notes. Furthermore, whether a color-stained banknote is sorted as a fit note or an unfit note may be determined based on the color of the stain on the banknote. For example, a banknote stained in yellow may be sorted as a fit note. Alternatively, whether a color-stained banknote is sorted as a fit note or an unfit note may be determined based on the number of color-stained blocks. For example, a banknote having three or more color-stained blocks may be sorted as an unfit note. The manner of sorting banknotes based on the results of fitness recognition and stain determination performed by the banknote handling apparatus 1, can be changed by setting.
Hereinafter, the structure of the banknote handling apparatus 1 will be described first, and thereafter, feature values and principal components for distinguishing color-stained notes from normally soiled notes will be described in detail. FIGS. 2A and 2B illustrate the structure of the banknote handling apparatus 1. FIG. 2A shows a perspective view of an external appearance of the banknote handling apparatus 1. FIG. 2B shows a schematic cross-sectional view of an internal structure of the banknote handling apparatus 1. The banknote handling apparatus 1 includes an inlet unit 11 for receiving a plurality of banknotes, and a feeding unit 10 for feeding the banknotes received at the inlet unit 11 one by one into the apparatus 1. An operation unit 51 is disposed at a front surface of the apparatus 1 for inputting information on setting change, information on banknote handling, instruction command on banknote handling, and the like. A display unit 52 is disposed at the front surface of the apparatus 1 for displaying information on the content of setting, information on the result of banknote handling, and the like.
As shown in FIG. 2B, a transport unit 70 for transporting, along a transport path, the banknotes fed into the apparatus 1 by the feeding unit 10, and a recognition unit 55 for recognizing and counting the banknotes being transported by the transport unit 70, are disposed inside the banknote handling apparatus 1. The banknote handling apparatus 1 is provided with two reject units 65 (65a, 65b) for rejecting, as reject banknotes, predetermined kinds of banknotes such as a banknote not to be handled, a counterfeit note, and a suspect note that is suspected to be a counterfeit note but cannot be determined as to authenticity. The banknote handling apparatus 1 can sort reject banknotes by kind, by discharging each reject banknote to the first reject unit 65a or the second reject unit 65b based on the kind.
The recognition unit 55 acquires, from the banknotes being transported by the transport unit 70, various kinds of data for recognizing denominations, authenticity, fitness, etc., of the banknotes, and counting the banknotes. The data acquired by the recognition unit 55 include data for determining existence of a color stain.
Stacking units 60 (60a to 60h) receive and stack the banknotes having been transported by the transport unit 70. Each stacking unit 60 has an opening at the front surface. An operator who uses the banknote handling apparatus 1 can take out the banknotes stacked in the stacking unit 60 from the opening. At the front surface of the apparatus 1, individual display units 62a to 62h for displaying information on the stacked banknotes therein are disposed above the first to eighth stacking units 60a to 60h, respectively.
Based on the data acquired by the recognition unit 55, a denomination, authenticity, fitness, etc., of each banknote are recognized. Based on the recognition result, reject banknotes are stacked in the reject units 65, and other banknotes are sorted and stacked in the stacking units 60a to 60h for each kind. The kinds of banknotes to be stacked in the respective stacking units 60a to 60h can be changed by setting. The operator can change setting on the determination process and the like and setting of transport destinations of the banknotes based on the determination result, by operating the operation unit 51 while confirming the setting content displayed on the display unit 52.
For example, color-stained notes can be stacked in the reject units 65 separately from other banknotes, or can be stacked in the stacking units 60. The color-stained notes can be sorted into entirely color-stained notes and partially color-stained notes to be stacked separately. Alternatively, the color-stained notes can be sorted and stacked by color by distinguishing the colors of the stains on the banknotes.
As shown in FIG. 2B, the transport unit 70 is provided with a plurality of diverters 71 for diverging the banknotes being transported along the transport path. Furthermore, the transport unit 70 is provided with a plurality of sensors 72 for detecting the banknotes being transported along the transport path. When the recognition result of each banknote is obtained based on the data acquired by the recognition unit 55, a transport destination of the banknote is determined from among the stacking units 60 and the reject units 65. The banknote can be transported to and stacked in the determined destination by controlling the diverging operations of the diverters 71 while detecting the transport position of the banknote with the sensors 72. Each stacking unit 60 is provided with a sensor 73 for detecting existence of stacked banknotes therein.
FIG. 3 is a block diagram schematically showing the functional structure of the banknote handling apparatus 1. As shown in FIG. 3, the banknote handling apparatus 1 includes a control unit 50 and a memory 56 in addition to the above-described components. The control unit 50 controls the respective units with reference to data stored in the memory 56, thereby realizing the functions and operations described in the present embodiment. The control unit 50 functions as a determination unit for performing a determination process related to color stains.
For example, the memory 56 is implemented by a nonvolatile semiconductor memory, and is used for storage of various kinds of data such as programs and settings required for the operation of the control unit 50. The memory 56 stores: settings on the recognition process and the determination process; reference data such as threshold values and templates; settings on the kinds of banknotes to be stacked in the reject units 65 and the stacking units 60; and the like. The memory 56 is also used for temporary storage of images obtained by capturing banknotes, recognition results, and the like.
The recognition unit 55 includes a light source 80 that applies light to the banknotes transported by the transport unit 70, and a line sensor 81 for capturing the banknotes. The recognition unit 55 functions as an image acquisition unit that acquires images of banknotes. The recognition unit 55 turns on the light source 80 to apply light to each banknote, and acquires an image of the entire face of the banknote by using the line sensor 81. As for an image acquisition method, white light may be applied to each banknote to acquire a color image, or R, G, and B lights may be individually applied to each banknote to acquire images corresponding to the respective colors.
In addition to the light source 80 and the line sensor 81, the recognition unit 55 includes a magnetic sensor for acquiring data regarding magnetic characteristics of each banknote, a thickness detection sensor for acquiring data regarding the thickness of each banknote, and the like. Recognition of banknotes is performed by using the data acquired by the respective sensors. Since the recognition can be performed in a manner similar to that of the conventional art, detailed description thereof is omitted.
Next, a feature value used for color stain determination will be described. FIG. 4 illustrates banknote images used for calculation of a feature value. The entire face of a banknote 200 is captured by using the line sensor 81 to acquire an R image 301, a G image 401, and a B image 501 as shown in FIG. 4. For example, the banknote 200 is divided into a plurality rectangular regions of a few millimeters in length and width, and each rectangular region corresponds to one pixel of each of the R image 301, the G image 401, and the B image 501.
When R, G, and B lights are sequentially applied to the banknote 200, the R image 301, the G image 401, and the B image 501 shown in FIG. 4 are directly obtained. When white light is applied to the banknote 200, a color image of the banknote 200 is obtained. This color image can be separated into three color component images of R, G, and B to obtain the R image 301, the G image 401, and the B image 501.
The R image 301 indicates pixel values of an R component of the color image obtained by capturing the banknote 200. The R image 301 is divided into a plurality of blocks 321. For example, the R image 301 is divided into 8 blocks 321 (R1, R2, ...), i.e., 2 blocks in a short edge direction (vertical direction in FIG. 4) × 4 blocks in a long edge direction (horizontal direction in FIG. 4).
The G image 401 indicates pixel values of a G component of the color image obtained by capturing the banknote 200. Pixels 411 of the G image 401 correspond to pixels 311 of the R image 301, and blocks 421 of the G image 401 correspond to the respective blocks 321 of the R image 301. The B image 501 indicates pixel values of a B component of the color image obtained by capturing the banknote 200. Pixels 511 of the B image 501 correspond to the respective pixels 311 of the R image 301, and blocks 521 of the B image 501 correspond to the respective blocks 321 of the R image 301.
Upon obtaining the R image 301, the G image 401, and the B image 501 of the banknote 200, the control unit 50 calculates feature values of the respective blocks. Since the feature value calculation method is common to the respective blocks, the calculation method will be described for the first blocks (R1, G1, B1).
FIGS. 5A to 5C illustrate the feature value calculation method. As shown in FIG. 5A, the control unit 50 calculates feature values from the pixel values of the corresponding pixels 311, 411, and 511 among pixels of the first block R1 of the R image 301, the first block G1 of the G image 401, and the first block B1 of the B image 501, respectively.
For example, the control unit 50 calculates higher-order local auto-correlation (HLAC) features from R-component pixel values, G-component pixel values, and B-component pixel values to obtain feature values. Specifically, for example, using a mask pattern 240 shown in FIG. 5B, the control unit 50 calculates a product of the pixel value of each pixel 311 included in the first block R1 of the R image 301. Then, the products of the pixel values calculated for all the pixels are summed up to obtain a total sum as one feature value (∑(R×R) shown in FIG. 5C). Likewise, the control unit 50 calculates a product of the pixel value of each pixel 311 included in the first block R1 of the R image 301 and the pixel value of the corresponding pixel 411 included in the first block G1 of the G image 401. Then, the products calculated for all the pixels are summed up to obtain a total sum as one feature value (∑(R×G) shown in FIG. 5C). Thus, as for each block, 6-dimensional feature values, i.e., a feature value (∑(R×R)) calculated from the R-component pixel values, a feature value (∑(R×G)) calculated from the R-component pixel values and the G-component pixel values, a feature value (∑(R×B)) calculated from the R-component pixel values and the B-component pixel values, a feature value (∑(G×G)) calculated from the G-component pixel values, a feature value (∑(G×B)) calculated from the G-component pixel values and the B-component pixel values, and a feature value (∑(B×B)) calculated from the B-component pixel values, are obtained as shown in FIG. 5C.
Next, a method for setting a determination condition for existence of a color stain will be described. First, multiple actual banknotes to be subjected to color stain determination are prepared. Specifically, a plurality of actual brand-new notes and a plurality of actual notes faded due to aging are prepared. Then, the prepared brand-new notes and faded notes are captured, and from each image, 6-dimensional feature values are calculated for each block as described above.
As for the actual faded notes to be prepared, since color change due to aging should be detected from the faded notes, the faded notes are preferably banknotes that do not have color change caused by reasons other than aging, such as scribbles, ink stain, etc. For example, when banknotes have just been issued and actual faded notes are not available, images of faded notes may be artificially generated from captured images of the available banknotes. Aging causes a blank portion of a banknote to be yellowed, and a portion printed with an ink of red, blue, etc., to be faded. Therefore, for example, fading is artificially realized by reducing the pixel values of R, G, and B components of an image of a brand-new note, and yellowing is artificially realized by significantly reducing the B-component pixel values as compared with the R-component pixel values and the G-component pixel values, whereby an image of a faded note is artificially generated. For example, an amount of change in each of the R, G, and B pixel values to realize fading may be determined with reference to a difference in color between an actual brand-new note of another denomination and an actual faded note of the same denomination.
When the feature values have been obtained for the respective blocks of the multiple brand-new notes and the multiple faded notes, block-based principal component analysis with the obtained feature value being a variable is performed. Then, in order from one having the highest rate of distribution, a first principal component to a third principal component are adopted. Since brand-new notes and faded notes are subjected to the principal component analysis, the first principal component is a component indicating color change when the brand-new note is faded. Meanwhile, the second principal component and the third principal component are components indicating color change different from fading. Information such as coefficients for calculating the values of the first principal component, the second principal component, and the third principal component of each block is stored in the memory 56 as a determination condition for detecting a color stain of each block. That is, an evaluation formula for calculating, from the feature values of each block, principal component scores of the second principal component and the third principal component to be used as evaluation values for color stain determination, is stored in the memory 56 as the determination condition in advance. Information on partial regions to be blocks on a banknote is also stored in the memory 56 in advance. For example, blocks and a determination condition are set for each kind of a banknote. For example, after acquisition of a banknote image, the kind of the banknote is recognized first, and then feature values are obtained from the blocks set for the kind, thereby advancing the process by using the determination condition corresponding to the kind of the banknote.
FIGS. 6A and 6B show examples of sets obtained by plotting the principal component scores of the first principal component, the second principal component, and the third principal component. For example, with respect to the same blocks of multiple banknotes including brand-new notes, faded notes (normally soiled notes), and color-stained notes, the values of the first principal components are obtained and plotted. As a result, as shown in FIG. 6A, a set 601 of brand-new notes is separated from a set 611 of faded notes and a set 612 of color-stained notes. Meanwhile, when the values of the second principal components and the values of the third principal components are obtained and plotted, the set 612 of color-stained notes is separated from the set 601 of brand-new notes and the set 611 of faded notes as shown in FIG. 6B.
Actual brand-new notes and actual faded notes are prepared, and a set 601 of brand-new notes and a set 611 of faded notes are obtained for each block as shown in FIG. 6A. Then, for example, a threshold value for the first principal component is set so as to distinguish the brand-new notes and the fit notes from other banknotes. Then, the threshold value is stored in the memory 56 as a determination condition for the corresponding block. When the banknote handling apparatus 1 recognizes a banknote, the control unit 50 calculates a value of the first principal component from each block of the banknote to be determined with reference to the determination condition stored in the memory 56, and compares the calculated value with the threshold value set for the block. Thus, the brand-new notes and the fit notes can be distinguished from other banknotes.
Actual brand-new notes and actual faded notes are prepared, and a set 601 of brand-new notes and a set 611 of faded notes are obtained for each block as shown in FIG. 6B. Then, for example, threshold values are set based on values forming an outer edge of a region including these two sets. Then, the threshold values are stored in the memory 56 as a determination condition for the corresponding block. When the banknote handling apparatus 1 recognizes a banknote, the control unit 50 calculates values of the second principal component and the third principal component from each block of the banknote to be determined with reference to the determination condition stored in the memory 56, and compares the calculated values with the threshold values set for the block. Then, if the values obtained from the block are included in neither the set 601 of brand-new notes nor the set 611 of faded notes, this block is determined to be a block of a color-stained note.
That is, a threshold value is set as reference data for determining existence of a color stain, and stored in the memory 56 as a determination condition in advance. Then, a principal component score that is calculated by substituting a feature value in a prepared evaluation formula is compared, as an evaluation value, with the reference data, thereby determining existence of a color stain.
Although actual brand-new notes and actual faded notes are prepared to set a determination condition for color stain determination in the above example, the determination condition setting method is not limited thereto. For example, actual color-stained notes may be used in addition to the actual brand-new notes and the actual faded notes. Multiple brand-new notes, faded notes, and color-stained notes are prepared to examine, and a set 612 of color-stained notes based on the values of the second principal component and the third principal component is obtained as shown in FIG. 6B. Then, for example, a threshold value may be set based on the values of the second principal component and the third principal component that form an outer edge of the set 612 of color-stained notes. Thus, values of a second principal component and a third principal component, which are obtained from a block of the banknote to be determined, are compared with the threshold values, and then if the values are included in the set 612 of color-stained notes, this block is determined to be a block of a color-stained note. Meanwhile, in order to distinguish the set 612 of color-stained notes from the set 601 of brand-new notes and the set 611 of faded notes, a threshold (boundary) that divides the principal component space into two regions may be set. In this case, values of a second principal component and a third principal component are obtained from a block of the banknote to be determined, and whether or not this block is a block of a color-stained note can be determined based on which of the two regions includes these values. In the same method as that for faded notes, an image of a color-stained note can be generated artificially by changing the R, G, B pixel values of a brand-new note, and generated images can be used to find the set 612 of color-stained notes.
The color stain determination method is not limited to the method of directly using the values of the second principal component and the third principal component calculated for each block. For example, a Mahalanobis distance may be calculated from the values of the second principal component and the third principal component, and used for determination. For example, a Mahalanobis distance from the position of data of a target block which is defined by the values of the second and third principal components, to a set including the set 601 of brand-new notes and the set 611 of faded notes, is calculated. When the calculated Mahalanobis distance exceeds a preset threshold value, it is determined that the block has a color stain. For another example, a Mahalanobis distance from the position of the data of the target block which is defined by the values of the second and third principal components, to the set 612 of color-stained notes, is calculated. When the calculated Mahalanobis distance does not exceed a preset threshold value, it is determined that the block is a block of a color-stained note.
Next, the flow of process steps of fitness recognition and stain determination based on the values of the first to third principal components will be described. FIG. 7 is a flowchart showing an example of process procedures for fitness recognition and stain determination. Although the banknote handling apparatus 1 recognizes a denomination, authenticity, fitness, etc., of a banknote by comprehensively evaluating data such as optical characteristics, magnetic characteristics, thickness, etc., of the banknote, FIG. 7 shows only processes regarding color stain determination.
An operator places a plurality of banknotes to be subjected to color stain determination in the inlet unit 11 to start banknote handling. The banknotes are fed from the inlet unit 11 by the feeding unit 10 one by one into the apparatus 1, and are transported along the transport path by the transport unit 70. During the transportation, the recognition unit 55 acquires a color image of the entirety of each banknote (step S1).
For each of a plurality of blocks set on the banknote in advance, the control unit 50 calculates feature values based on higher-order local auto-correlation features from pixel values of an R component, a G component, and a B component of the banknote image (step S2). Subsequently, based on the calculated feature values and the determination condition stored in the memory 56 in advance, the control unit 50 calculates, for each of the plurality of blocks, principal component scores of a first principal component, a second principal component, and a third principal component (step S3). Based on the value of the first principal component score and information stored as the determination condition in the memory 56 in advance, the control unit 50 determines whether or not the banknote is a fit note (step S4).
For example, the control unit 50 calculates a Mahalanobis distance between the value of the first principal component of each block and the set 601 of brand-new notes shown in FIG. 6A. When the Mahalanobis distances of all the blocks set on the banknote are equal to or smaller than a threshold value that is preset as reference data, the control unit 50 determines that the banknote is a fit note (Yes in step S4). When at least one of the plurality of blocks has the Mahalanobis distance exceeding the threshold value, the control unit 50 determines that the banknote is not a fit note (No in step S4).
When the determination result is that the banknote is a fit note (step S5), the control unit 50 ends the determination. When the determination result is that the banknote is not a fit note, i.e., the banknote is an unfit note, the control unit 50 determines whether or not the banknote is a color-stained note (step S6) .
For example, the control unit 50 calculates a Mahalanobis distance between the values of the second principal component and the third principal component of each block, and the set 612 of color-stained notes shown in FIG. 6B. When the Mahalanobis distances of all the blocks set on the banknote exceed a threshold value that is set as reference data in advance, the control unit 50 determines that the banknote is not a color-stained note (No in step S6). When at least one of the plurality of blocks has the Mahalanobis distance equal to or smaller than the threshold value, the control unit 50 determines that the banknote is a color-stained note (Yes in step S6).
When the determination result is that the banknote is not a color-stained note, that is, the banknote is an ordinary soiled note such as a banknote faded due to aging (step S8), the control unit 50 ends the determination. Also, when the determination result is that the banknote is a color-stained note (step S7), the control unit 50 ends the determination.
However, when the determination result that the banknote is a color-stained note has been obtained, the determination process may be further continued. For example, the color-stained note may be sorted by the color of the stain, based on the positional relationship between the set 612 of color-stained notes shown in FIG. 6B and a point at which data of the target block is plotted. Alternatively, for example, the color-stained note may be sorted as an entirely color-stained note in which all blocks are color-stained, or a partially color-stained note in which only some blocks are color-stained, based on the number of the blocks determined as color-stained blocks.
In FIG. 7, the data of the target block is plotted on the principal component space consisting of the second principal component and the third principal component, and existence of a color stain is determined based on a Mahalanobis distance from the set of color-stained notes. However, the determination method is not limited thereto. For example, as described above, the Mahalanobis distance from the set of brand-new notes and faded notes is obtained, and the target banknote may be determined to be a color-stained note when the calculated distance exceeds a predetermined threshold value. Alternatively, a Mahalanobis distance from a set of only faded notes may be obtained to determine existence of a color stain on the block of the target banknote.
In the present embodiment, the entire face of a banknote is divided into a plurality of blocks, but setting of blocks is not limited thereto. When it is not necessary to determine a color stain on the entire face of a banknote, blocks may be set only in a region to be subjected to color stain determination. For example, there is a machine that sprays a special ink onto banknotes to color the banknotes upon detecting an abnormality, in order to deal with theft of banknotes. If the special ink is adhered to a limited region, blocks may be set on this region so as to detect a color stain in this region.
In the present embodiment, a color-stained note is detected by using a feature value indicating the color of a block set on a banknote. However, the feature value indicating the color of the block may be used for recognizing a denomination or authenticity of the banknote. For example, it is assumed that there are banknotes on which similar patterns are printed but the colors of the patterns vary depending on denomination, and banknotes of the same denomination on which the colors of the patterns vary depending on issue year. In this case, as described above, the banknotes having the different colors are distinguished from each other by using the feature values obtained from the pixel values of R, G, and B and on the principal component scores obtained from the feature values, whereby the denomination and the issue year of each banknote can be recognized. Furthermore, for example, when a counterfeit note having a color different from a genuine note appears, these colors can be distinguished from each other by using the feature values obtained from the pixel values of R, G, and B and on the principal component scores obtained from the feature values, whereby authenticity of each banknote can be recognized. That is, a feature value indicating the color of a block can be used as a color feature of a banknote for various purposes to identify the kind of the banknote.
In the present embodiment, a feature value is calculated by using pixel values of R, G, and B components of a color image obtained by capturing a banknote. However, the colors to be used for feature value calculation are not limited to R, G, and B. A feature value may be calculated from pixel values of any colors as long as a color change of a banknote due to fading can be extracted. For example, a pixel value of an image obtained by capturing a banknote with light of infrared wavelength is used as an IR component, a pixel value of an image obtained by capturing the banknote with green light is used as a G component, a pixel value of an image obtained by capturing the banknote with violet light is used as a V component, and a feature value may be calculated from the pixel values of these three color components. Alternatively, a feature value may be calculated from pixel values of four or more color components.
In the present embodiment, 6-dimensional feature values are calculated as higher-order local auto-correlation features from pixel values of R, G, and B components in each block, but the feature values are not limited thereto. For example, a value indicating a correlation feature between a target pixel and an adjacent pixel may be used as a feature value. Specifically, for example, total sums of products calculated between pixel values of R, G, and B components of a target pixel and pixel values of R, G, and B components of a pixel adjacent to the target pixel may be added as feature values to provide 7 or more dimensional higher-order local auto-correlation features.
In the present embodiment, existence of a color stain is determined by using an image obtained by capturing one side of a banknote. However, the determination method is not limited thereto. For example, images of a face side and a back side of a banknote may be captured, and the image of each face may be subjected to the aforementioned color stain determination. In this case, for example, this banknote may be determined to be a color-stained note when either the face side or the back side has a color stain, and may be determined not to be a color-stained note when the both sides do not have color stains. Alternatively, it is possible to set that the banknote is determined not to be a color-stained note even when either the face side or the back side has a color stain.
As described above, according to the banknote handling apparatus 1 of the present embodiment, pixel values of a plurality of colors are obtained from a color image obtained by capturing a banknote, and a feature value indicating the color of the banknote can be calculated. The calculated feature value is substituted in a predetermined evaluation formula to calculate an evaluation value for determining existence of a color stain. The calculated evaluation value is compared with a threshold value that is previously prepared as reference data to determine existence of a color stain. The evaluation formula and the threshold value are obtained by subjecting the feature values, which are obtained from a plurality of actual banknotes including brand-new notes and faded notes, to principal component analysis. By using the evaluation formula and the threshold value, a banknote whose color has changed due to fading can be distinguished from a color-stained banknote. Thus, the color-stained banknote can be detected with high accuracy.

INDUSTRIAL APPLICABILITY

As described above, a sheet stain determination apparatus and a sheet stain determination method according to the present invention are useful for determining existence of a color stain on a sheet over a wide area.

DESCRIPTION OF THE REFERENCE CHARACTERS

1: Banknote handling apparatus
10: Feeding unit
11: Inlet unit
50: Control unit
51: Operation unit
52: Display unit
55: Recognition unit
56: Memory
60 (60a to 60h): Stacking unit
62 (62a to 62h): Individual display unit
65 (65a, 65b): Reject unit
70: Transport unit
71: Diverter
72, 73: Banknote detecting sensor
80: Light source
81: Line sensor

Claims

A sheet stain determination apparatus comprising:
an image acquisition unit configured to acquire an image of a sheet;

a memory configured to store therein reference data for specifying color change of the sheet due to aging; and

a determination unit configured to, based on the image of the sheet and the reference data, detect a sheet whose color has changed over a region of a predetermined area or more from an original color after printing of the sheet, and determine whether the detected sheet is a sheet whose color has changed due to aging or a color-stained sheet whose color has changed due to a color stain.
The sheet stain determination apparatus according to claim 1, wherein
the reference data is a threshold value for determining whether or not an evaluation value indicates that a sheet is a color-stained sheet, the evaluation value being calculated by substituting a feature value, which has been calculated from pixel values of a plurality of colors obtained through color separation of a color image of the sheet, in a predetermined evaluation formula, and
the determination unit calculates the evaluation value from an image of the sheet acquired by the image acquisition unit, compares the calculated evaluation value with the threshold value to determine whether or not the sheet is a color-stained sheet.
The sheet stain determination apparatus according to claim 2, wherein the feature value is a value indicating a correlation feature of the pixel values of the plurality of colors.
The sheet stain determination apparatus according to claim 2 or 3, wherein the evaluation formula is an formula for calculating a principal component score of a predetermined principal component, the formula being obtained through principal component analysis performed on feature values of images of a plurality of sheets having no color change due to aging and a plurality of sheets having color change due to aging.
The sheet stain determination apparatus according to claim 4, wherein each image of the plurality of sheets having color change due to aging is generated artificially by changing color components of an image obtained by capturing a sheet.
The sheet stain determination apparatus according to any one of claims 2 to 5, wherein the pixel values of the plurality of colors include a pixel value of an R component, a pixel value of a G component, and a pixel value of a B component.
The sheet stain determination apparatus according to any one of claims 2 to 6, wherein the determination unit determines, based on the evaluation value, whether or not the color stains on the plurality of color-stained sheets are of the same color.
The sheet stain determination apparatus according to any one of claims 2 to 6, wherein the determination unit calculates, for a partial region that is set as a block on the sheet, the evaluation value from pixel values of pixels forming the block to determine whether or not the sheet is a color-stained sheet.
The sheet stain determination apparatus according to claim 8, wherein
the block is one of a plurality of partial regions into which an entire face of the sheet is divided, and
the determination unit determines, based on the evaluation value calculated for each block, whether the color-stained sheet is an entirely color-stained sheet or a partially color-stained sheet.
The sheet stain determination apparatus according to any one of claims 1 to 9, further comprising:
an inlet unit configured to receive a plurality of sheets;

a transport unit configured to transport the sheets received in the inlet unit, one by one; and

a plurality of stacking units configured to stack sheets therein, wherein

the plurality of sheets received in the inlet unit are sorted and stacked in the plurality of stacking units, based on result of determination by the determination unit.
A sheet stain determination method by which a sheet stain determination apparatus determines existence of a color stain, the method comprising:
acquiring an image of a sheet; and

determining, based on an acquired image of a sheet whose color has changed over a region of a predetermined area or more from an original color after printing of the sheet and on reference data for specifying color change of the sheet due to aging, whether the sheet is a sheet whose color has changed due to aging or a color-stained sheet whose color has changed due to a color stain.