JP2005316727A - Paper sheet identifying device and method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a paper sheet identifying device for determining a paper sheet insertion direction from a photographed image of a paper sheet inserted into a device, efficiently extracting a characteristic image of the paper sheet, and accurately determining the type and identification of the paper sheet based on the extracted characteristic image even if the characteristic image formed on the paper sheet is displaced or deviated. <P>SOLUTION: An image of an area including the paper sheet characteristic image is photographed from the paper sheet inserted in the normal/reverse direction on the back/front side, a template image matching the insertion direction of the paper sheet of the characteristic image is moved on the photographed image for carrying out template matching. The insertion direction of the paper sheet is determined from the template image having the highest correlation value found by the template matching. Using the position on the photographed image matching the template image, which has the highest correlation value based on template matching using the template image matching the determined insertion direction of the paper sheet, as a base point, the characteristic image of the paper sheet is cut out from the photographed image. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a paper sheet identification apparatus and method, and in particular, determines the insertion direction of a paper sheet from a captured image of a paper sheet inserted into the apparatus, and misaligns a characteristic image formed on the paper sheet. The present invention relates to a paper sheet identification apparatus and method for efficiently extracting a feature image of a paper sheet without being influenced by the above and accurately determining the type and authenticity of the paper sheet based on the extracted feature image.

  In general, a paper sheet identification device that identifies the type and authenticity of paper sheets such as banknotes, checks, gift certificates, etc., uses a magnetic sensor or optical sensor to magnetically characterize the characteristics of the paper sheets that have been input into the apparatus. Alternatively, the type and authenticity of the paper sheet are identified by optically detecting and collating the detected characteristic of the paper sheet with the characteristic of the genuine note.

  For example, when an optical sensor is used, the transmitted light type optical sensor or the reflective optical sensor is used to optically detect the pattern, size, direction, etc. of the paper sheet, and it is extracted as a feature image. The type and authenticity of the paper sheet are identified by comparing the image with the standard image of the genuine note for each paper sheet.

  The matching between the feature image and the standard image is performed based on, for example, the similarity by pattern matching or the like. Depending on how the feature image is extracted from the detected image of the detected paper sheet, The collation process with the standard image may take a long time or may be complicated.

  In addition, since the insertion direction of the paper sheets inserted into the apparatus is inserted in various directions such as front and back, normal and reverse, a feature image is detected from the detection images of the paper sheets input corresponding to these insertion directions. And the collation processing of the extracted feature image and the standard image must be performed.

  For example, Patent Document 1 proposes an image recognition device that enables recognition of an input image even if the input image causes a rotational shift with respect to a standard pattern.

  Further, in Patent Document 2, the denomination, front / back, direction, and authenticity of a paper sheet are identified at high speed without being affected by some positional deviation or skew deviation of the conveyance of the paper sheet. A highly reliable paper sheet recognition apparatus has been proposed.

Patent Document 3 proposes an image processing apparatus that detects an original at an arbitrary position and an arbitrary angle on an original table and determines whether or not the detected original is a specific original.
Japanese Patent Laid-Open No. 05-012446 Japanese Patent Laid-Open No. 05-062045 Japanese Patent Laid-Open No. 02-210591

  By the way, in the above-mentioned proposed Patent Document 1, even if the input image has a rotational shift with respect to the standard pattern, the standard pattern rotated by rotating the standard pattern at a predetermined angular interval and the input image are used. It is disclosed that an input image can be recognized by calculating a coincidence point.

  In Patent Document 2, the skew angle of the inserted paper sheet is calculated based on the time difference between detection signals detected by a plurality of detection elements, and the inserted paper sheet is inserted based on the calculated skew angle. After correcting the inclination of the scanned image, the front and back of the boundary point image serving as a reference registered in advance, the image in each of the forward and reverse directions, and the number of matching pixels between the input input boundary point image and the paper sheet A method of identifying the denomination, front / back, direction, and authenticity is disclosed.

  Further, in Patent Document 3, a bill at an arbitrary angle with respect to an arbitrary position on an original table of a copying machine is scanned a plurality of times, an accurate position and angle at which the bill is placed is calculated, and a banknote is calculated according to the calculation result. A method is disclosed in which image data of a specific part is taken out and the authenticity of the banknote is determined based on the similarity with a pattern registered in advance.

  However, in the proposed Patent Document 1 to Patent Document 3, the insertion direction of the paper sheet is determined from the captured image of the paper sheet, and an image area corresponding to the pre-registered feature image of the paper sheet is determined. A method for extracting from a captured image is not disclosed.

  In addition, there is no disclosure of a method for accurately determining the type or authenticity of a paper sheet that has reduced the influence when the watermark pattern or the like, which is a characteristic image of the paper sheet, is misaligned with respect to the watermark area. .

  Therefore, the present invention determines the insertion direction of the paper sheet from the captured image of the paper sheet inserted into the apparatus, even when the feature image formed on the paper sheet is misaligned and has variations. An object of the present invention is to provide a paper sheet identification apparatus and method that efficiently extract a feature image of a paper sheet and accurately and quickly determine the type and authenticity of the paper sheet based on the extracted feature image.

  In order to achieve the above object, the invention according to claim 1 is a paper sheet identification device that cuts out a feature image of the paper sheet from a captured image of the paper sheet and identifies the paper sheet based on the cut out feature image. In, the image pickup means for picking up an image of a region including the feature image of the paper sheet from the paper sheet inserted in either the front and back, forward and reverse direction, on the image picked up by the image pickup means An image matching unit that performs template matching with a template image corresponding to each insertion direction of the paper sheet of the feature image, and a template image having a maximum maximum correlation value by template matching performed by the image matching unit Insertion direction discriminating means for discriminating the insertion direction, and image matching using a template image corresponding to the insertion direction of the paper sheet determined by the insertion direction discrimination means Characterized by comprising the image cut-out means for cutting out the feature image of the paper sheet to a position on the captured image corresponding to the maximum correlation value by grayed unit from the captured image as a reference point.

  According to a second aspect of the present invention, in the first aspect of the invention, the feature image is an image of a watermark area of the paper sheet, and the image matching unit includes one captured image captured by the imaging unit. Template matching is performed with four template images obtained by converting one template image data into the front, back, and forward / reverse four directions.

  According to a third aspect of the present invention, in the first aspect of the invention, the feature image is an image of a watermark area of the paper sheet, and the image matching unit is configured to capture one captured image captured by the imaging unit. Template matching is performed between the four images converted into the front and back and the forward and reverse four directions and one template image corresponding to one template image data.

  According to a fourth aspect of the present invention, in the second or third aspect of the present invention, the image matching means obtains the center of gravity of the image in the watermark area, and the insertion direction of the paper sheet is correct according to the position of the center of gravity. Whether the direction is the reverse direction on the back side, the reverse direction on the front side, or the forward direction on the back side is determined, and the template matching is selectively performed with respect to the two determined directions.

  The invention of claim 5 is a paper sheet identification method for cutting out a characteristic image of the paper sheet from a captured image of the paper sheet, and identifying the paper sheet based on the cut out characteristic image. An image of an area including the feature image of the paper sheet is captured by the imaging unit from the paper sheet inserted in either the forward or reverse direction, and the feature image is captured on the image captured by the imaging unit. Template matching is performed by an image matching unit with a template image corresponding to each insertion direction of the paper sheet, and the insertion direction of the paper sheet from the template image having the largest maximum correlation value calculated by template matching by the image matching unit Is determined by the insertion direction determination means, and the image map using the template image corresponding to the insertion direction of the paper sheet determined by the insertion direction determination means. Characterized in that cutting the feature image of the paper sheet from the captured image position as a base point on the captured image maximum correlation value by the quenching means corresponding to the most significant the template image.

  According to a sixth aspect of the present invention, in the fifth aspect of the invention, the feature image is an image of a watermark area of the paper sheet, and the image matching unit includes one captured image captured by the imaging unit. Template matching is performed with four template images obtained by converting one template image data into the front, back, and forward / reverse four directions.

  The invention according to claim 7 is the invention according to claim 5, wherein the feature image is an image of a watermark area of the paper sheet, and the image matching means captures one captured image captured by the imaging means. Template matching is performed between the four images converted into the front and back and the forward and reverse four directions and one template image corresponding to one template image data.

  In the invention of claim 8, in the invention of claim 6 or 7, the image matching means obtains the center of gravity of the image of the watermark region, and the insertion direction of the paper sheet is correct in the table according to the position of the center of gravity. Whether the direction is the reverse direction on the back side, the reverse direction on the front side, or the forward direction on the back side is determined, and the template matching is selectively performed with respect to the two determined directions.

  According to the paper sheet identification apparatus and method of the present invention, the apparatus is based on a correlation value obtained by template matching between a template image corresponding to a pre-registered paper sheet insertion direction and a captured image of the inserted paper sheet. Since the insertion direction of the inserted paper sheet is determined and the characteristic image of the paper sheet is extracted from the captured image, the apparatus can be used even when the characteristic image formed on the paper sheet has misalignment or variation. Can accurately determine the insertion direction of the paper sheets inserted in the paper, efficiently extract the feature images of the paper sheets from the captured image, and accurately and quickly determine the type and authenticity of the paper sheets based on the extracted feature images There is an effect that it can be determined.

  A paper sheet identification apparatus and method according to the present invention is an apparatus for identifying the type and authenticity of a paper sheet based on a specific pattern formed on a paper sheet, for example, a watermark pattern is formed. When identifying the type and authenticity of paper sheets such as securities and banknotes, the insertion direction of the inserted paper sheet is determined from the captured image of the paper sheet and corresponds to the pre-registered characteristic image of the paper sheet Device that efficiently extracts the image area to be extracted from the captured image of the paper sheet, reduces the influence of misalignment of the feature image formed on the paper sheet, and accurately determines the type and authenticity of the paper sheet It is used as

  In the following embodiments, a method and apparatus for identifying the type and authenticity of a paper sheet using a watermark pattern as a characteristic image in a paper sheet on which a watermark pattern is formed will be described as an example.

  An embodiment of a paper sheet identification apparatus and method according to the present invention will be described in detail with reference to the accompanying drawings.

  FIG. 1 is a configuration diagram showing a schematic configuration of a paper sheet identification apparatus to which a paper sheet identification method according to the present invention is applied.

  As shown in FIG. 1, the paper sheet identification device 1 includes a control unit 2 that controls the entire paper sheet identification device 1, a paper sheet insertion unit 3 that is a paper sheet insertion port, and an insertion port. A paper sheet transport unit 4 that transports the inserted paper sheet, a paper sheet receiving unit 5 that receives the paper sheet, and a drive unit that drives and controls the paper sheet transport unit 4 according to an instruction from the control unit 2 6 and an identification unit 7 for identifying the type and authenticity of the paper sheet.

  The paper sheet inserted from the paper sheet insertion unit 3 is identified by the identifying unit 7 as to the type and authenticity of the paper sheet while being transported to the paper sheet receiving unit 5 by the paper sheet transport unit 4. .

  When the identifying unit 7 identifies the paper sheet as a genuine note, the driving unit 6 controls the driving of the paper sheet conveying unit 4 based on the instruction instruction from the control unit 2, and the paper sheet is received by the paper sheet receiving unit. The paper sheet receiving unit 5 receives the paper sheet that has been transported.

  When the identification unit 7 identifies the paper sheet as a fake ticket, the drive unit 6 performs drive control of the paper sheet transport unit 4 based on the instruction instruction of the control unit 2 and inserts the paper sheet. It is transported to part 3 and the paper sheets are returned.

  Now, in the paper sheet identification apparatus 1, the identification unit 7 performs the type and true / false identification of the paper sheet.

  FIG. 2 is a block diagram illustrating an example of a functional configuration of the identification unit 7 of the paper sheet identification apparatus 1.

  As shown in FIG. 2, the identification unit 7 includes an identification control unit 70 that performs overall control of the identification unit 7, a transmitted light type optical sensor 73, a memory 71, a template image database 72, a pattern matching unit 74, The identification target image extraction unit 75, the scratch / dirt detection unit 76, and the scratch / dirt effect correction unit 77 are configured.

  In the transmitted light type optical sensor 73, a light-emitting element 731 and a light-receiving element 732 form a pair, and a paper sheet inserted from the paper sheet insertion unit 3 at a position facing the paper sheet conveyance unit 4 facing each other. It is arranged on a predetermined scanning line passing through an imaging region including a watermark region of paper sheets to be identified.

  The light emitting element 731 irradiates light to the image pickup area of the paper sheet conveyed by the paper sheet conveyance unit 4, and the light receiving element 732 receives the transmitted light transmitted through the image pickup area of the paper sheet, An electric signal corresponding to the amount of received transmitted light is output.

  Note that any of infrared, ultraviolet, and visible light can be applied to the transmitted light type optical sensor 73.

  The memory 71 corresponds to the digital signal corresponding to the signal level of the electrical signal output from the transmitted light type optical sensor 73 at a predetermined time interval in a predetermined storage area of the memory 71 in the order of output from the transmitted light type optical sensor 73. The stored digital signal information is assigned with continuous addresses, and temporarily stored as image data (hereinafter referred to as “captured image data”) of the imaging region of the paper sheet.

  In the template image database 72, each genuine paper sheet is inserted into the paper sheet recognition apparatus 1 in each insertion direction on the front and back sides of each type, and is captured and acquired in accordance with each insertion direction. Image data is stored as template image data.

  The template image data stores, for example, image data obtained by capturing images of a watermark pattern or the like corresponding to the type of paper sheet of a genuine note in the forward and reverse directions, the front and back directions, and the surfaces.

  The pattern matching unit 74 performs pattern matching between the input image data obtained when the paper sheet to be examined passes through the transmitted light sensor 73 and the template image data stored in the template image database 72. The image area of a similar input image is searched and specified, and a correlation value by pattern matching between identification target image data and corrected identification target image data described later is calculated and output.

  For example, the captured image data of the paper sheet temporarily stored in the memory 71 and the template image data stored in the template image database 72 are read out, and a correlation value by pattern matching between these two image data is calculated and calculated. Based on the correlation value, the image area of the captured image data most similar to the template image data is searched and specified.

  First, the maximum value of the correlation value by pattern matching (hereinafter referred to as “maximum correlation”) while moving the template image data of the genuine note in which the paper sheet of the genuine note is inserted in the positive direction and the surface portion is imaged. Value)).

  Next, the template image is a genuine template image data in which the back surface is imaged in the normal insertion direction, a genuine note template image data in which the front surface is imaged in the reverse insertion direction, and the reverse insertion direction. The template image data of the genuine note whose back side is imaged in step 1 is replaced sequentially, and for each template image data replaced, the maximum is obtained by pattern matching while moving each template image data on the captured image data in the same manner as the processing operation described above. A correlation value is calculated.

  And it inserts based on the template image data (henceforth "specific template image data") which became the largest value among the maximum correlation values calculated by pattern matching with each template image data and captured image data. In addition to determining the normal / reverse and front / back insertion direction of the paper sheet, the specific template image indicates the image area on the captured image data when the correlation value by the pattern matching between the specific template image data and the captured image data is the maximum correlation value. The image area most similar to the data is specified.

  The identification target image extraction unit 75 extracts the image area on the captured image data most similar to the specific template image data specified by the pattern matching unit 74 as identification target image data, and stores it in the memory 71.

  The flaw / dirt detection unit 76 calculates the difference between the identification target image data extracted by the identification target image extraction unit 75 and the specific template image data, and the flaws attached to the paper sheets existing in the identification target image data. Image data indicating dirt (or scratches and dirt adhering to the transmitted light type optical sensor) is detected.

  The flaw / dirt influence correcting unit 77 corrects the influence of the image data indicating the flaw and dirt detected by the flaw / dirt detecting unit 76.

  Specifically, for example, a configuration may be applied in which the influence of image data such as scratches and dirt adhering to paper sheets is corrected by erasing image data indicating scratches and dirt from the identification target image data. In such a configuration, the image data indicating scratches and dirt is erased and the watermark pattern image data is also erased, which may cause erroneous identification.

  Therefore, a configuration is applied in which image data indicating scratches and dirt existing in the identification target image data is converted into corrected identification target image data pasted at the same position and area in the specific template image data.

  However, if the influence of image data showing all scratches and dirt is corrected, even if it is a fake paper sheet without a watermark pattern, the effect of image data showing scratches and dirt is corrected. Since it is erroneously identified as a paper sheet of a ticket, a configuration may be used in which the total number of pixels for correcting the influence of image data indicating scratches and dirt is limited.

  An operation in the case where the identification unit 7 configured as described above identifies the type and authenticity of the paper sheet inserted into the paper sheet insertion unit 3 will be briefly described.

  The paper sheet inserted from the paper sheet insertion unit 3 is transported by the paper sheet transport unit 4, and when the paper sheet reaches the arrangement position of the transmitted light type optical sensor 73 of the identification unit 7, the light emitting element 731 is moved to the paper. Light is applied to the imaging region of the leaf, and the light receiving element 732 receives the transmitted light that has passed through the imaging region of the paper sheet.

  The transmitted light type optical sensor 73 outputs an electrical signal having a signal level corresponding to the amount of transmitted light received by the light receiving element 732, and the electrical signal output by the transmitted light type optical sensor 73 is digital corresponding to the signal level. It is converted into a signal and input to the memory 71.

  The memory 71 temporarily stores digital signal information corresponding to the signal level output from the transmitted light type optical sensor 73 as captured image data of the imaging region of the paper sheet.

  When the captured image data of the paper sheet is stored in the memory 71, the pattern matching unit 74 stores the captured image data of the paper sheet stored in the memory 71 and the genuine paper sheet stored in the template image database 72. Read out each template image data captured and stored in forward / reverse for each type of class, each direction on the front and back and each surface, perform pattern matching between the captured image data and each template image data, and out of each template image data The template image data most similar to the captured image data is specified as the specific template image data, and the image region most similar to the specific template image data is searched and specified from the captured image data.

  The identification target image extraction unit 75 extracts the image area of the captured image data that is most similar to the specific template image data specified by the pattern matching unit 74 as the identification target image data of the paper sheet, and sends it to the scratch / dirt detection unit 76. Output.

  The scratch / dirt detection unit 76 calculates a difference between the identification target image data input from the identification target image extraction unit 75 and the specific template image data, and obtains image data indicating scratches and dirt existing in the identification target image data. It is detected and output to the scratch / dirt effect correction unit 77.

  The flaw / dirt influence correcting unit 77 outputs the corrected identification target image data corrected for the influence of the image data indicating the flaw and dirt detected and input by the flaw / dirt detecting unit 76 to the pattern matching unit 74.

  The pattern matching unit 74 performs pattern matching between the corrected identification target image data input from the scratch / dirt influence correction unit 77 and the identification target image data, calculates a correlation value between these two image data, and an identification control unit 70 identifies the type and authenticity of the paper sheet based on the correlation value obtained by pattern matching between the corrected identification target image data and the identification target image data calculated by the pattern matching unit 74.

  Refer to the flowcharts shown in FIGS. 3 and 4 for the processing procedure for identifying and processing the type and authenticity of the inserted paper sheet by the paper sheet identifying apparatus 1 having the identifying unit 7 operating as described above. I will explain.

  When a paper sheet is inserted from the paper sheet insertion unit 3 (YES in step S301), the paper sheet identification device 1 images the inserted paper sheet using the transmitted light optical sensor 73 (step S301). S302), template matching between the captured image data acquired by imaging and the template image data corresponding to each direction and each surface of the front and back, and the front and back of the genuine paper sheet is performed in advance. The maximum correlation values A, B, C, and D between the template image data and the captured image data are calculated (steps S303, S304, S305, and S306).

  The maximum correlation value A is the maximum correlation value between the template image data (hereinafter referred to as “template image data (positive direction surface)”) that is inserted in the positive direction and the surface portion is imaged, and the maximum. The correlation value B is a maximum correlation value and a maximum correlation value C between template image data (hereinafter referred to as “template image data (back in the positive direction)”) that is inserted in the positive direction and whose back surface is imaged, and the captured image data. Is the maximum correlation value between the template image data (hereinafter referred to as “template image data (reverse direction surface)”) that has been inserted in the reverse direction and whose surface portion has been imaged, and the captured image data. The maximum correlation value between the template image data (hereinafter referred to as “template image data (reverse direction reverse side)”) that is inserted in the direction and the back side is imaged and the captured image data It is shows.

  Template image data having the maximum correlation value that is the maximum value among the calculated maximum correlation values A, B, C, and D is set as specific template image data (step S307), and is most similar to the specific template image data. The image area of the captured image data to be extracted is extracted as image data to be identified of paper sheets (step S308).

  The extracted identification target image data is subjected to scratch / dirt correction and identification processing (step S309), and the type and authenticity of the inserted paper sheet are identified.

  FIG. 4 is a flowchart showing the processing procedure of the flaw / dirt correction and identification process S309. The processing procedure of the flaw / dirt correction and identification process S309 will be described with reference to FIG.

  When the identification target image data is extracted from the captured image data of the inserted paper sheet, a difference between the identification target image data and the specific template image data is calculated to indicate scratches and dirt existing in the identification target image data Image data is detected (step S3091), and corrected identification target image data in which the influence of the detected image data indicating scratches and dirt is corrected is calculated (step S3092).

  Then, pattern matching between the corrected identification target image data and the identification target image data is performed, and a correlation value between the two images is calculated (step S3093).

  The authenticity of the paper sheet is determined based on the calculated correlation value (step S3094). If the paper sheet is determined to be a genuine note (YES in step S3094), the paper sheet is accepted. (Step S3095), and the processing procedure ends.

  If it is determined in step S3094 that the paper sheet is a fake ticket (NO in step S3094), the paper sheet is returned from the paper sheet insertion unit 3 (step S3096), and the processing procedure is terminated.

  Here, the paper sheet identification method according to the present invention performed by the identification unit 7 of the paper sheet identification apparatus 1 will be described in detail.

  FIG. 5 to FIG. 9 are explanatory diagrams for explaining a method of identifying the type and authenticity of the paper sheet performed by the identification unit 7, and the paper sheet according to the present invention with reference to FIG. 5 to FIG. The identification method will be described in detail.

  FIG. 5 is a diagram illustrating an example of captured image data and template image data. FIG. 5A illustrates an image of a paper sheet inserted into the paper sheet identification device 1 using a transmitted light type optical sensor 73. FIGS. 5 (b) to 5 (e) show the captured image data 20 acquired in this manner, and the genuine paper sheets are forward / reverse for each type, and the paper sheets are identified in the front and back directions and on each surface. It is a figure which shows the template image data 26, 27, 28, and 29 which were inserted in the apparatus 1 and imaged and acquired.

  As shown in FIG. 5A, the captured image data 20 of the paper sheet imaged by the transmitted light type optical sensor 73 is composed of, for example, each pixel of f (i, j) whose number of pixels is L × L. In the captured image data 20, a watermark area 21 and a normal area 22 other than the watermark area 21 exist, and a watermark pattern 23 exists in the watermark area 21.

  Further, the picked-up image data 20 includes paper sheet scratches and dirt 25.

  In FIG. 5A, a watermark pattern 23 indicated by a broken line indicates a displacement of the watermark pattern 23 formed on the genuine note, and i = 0, 1, 2,... L-1, j = 0, 1, 2,..., L-1.

  In addition, the amount of light when the light receiving element 732 of the transmitted light type optical sensor 73 receives light is such that the light receiving element 732 receives the transmitted light that has passed through the paper sheet. The amount of light transmitted through the watermark pattern 23 is less than the amount of light transmitted through the watermark region 21, and the amount of light transmitted through the normal region 22 is less than the amount of light transmitted through the watermark pattern 23.

  Further, the amount of light transmitted through the scratches and dirt varies depending on the level of the scratches and dirt.

  The template image database 72 is obtained by capturing images of genuine paper sheets in the forward and reverse directions, front and back directions and surfaces in advance, for example, as shown in FIGS. 5B to 5E. Each template image data 26, 27, 28, 29 is stored.

  FIG. 5B shows template image data (forward direction surface) 26 obtained by inserting a genuine paper sheet into the paper sheet identification apparatus 1 in the forward direction and imaging the surface portion of the paper sheet. FIG. 5C is a diagram illustrating template image data (backward in the forward direction) obtained by inserting a genuine paper sheet into the paper sheet identification device 1 in the forward direction and capturing an image of the back side of the paper sheet. FIG. 5D shows template image data (inverse) obtained by inserting a genuine paper sheet into the paper sheet identification device 1 in the reverse direction and imaging the surface portion of the paper sheet. FIG. 5E shows template image data obtained by inserting a genuine paper sheet into the paper sheet identification apparatus 1 in the reverse direction and capturing an image of the back surface of the paper sheet. It is a figure which shows (reverse direction back surface) 29. FIG.

  Each of the template image data 26, 27, 28, 29 is composed of, for example, each pixel of t (i, j) having the number of pixels of M × N, and each of the template image data 26, 27, 28, 29 is , For example, image data of a feature image such as a watermark pattern corresponding to the type of paper sheet of a genuine note is stored, i = 0, 1, 2,..., M−1, j = 0, 1, 2. ..., N-1.

  6 and 7 specify template image data having the highest similarity among the template image data 26, 27, 28, and 29 in the captured image data 20, and the image most similar to the specified specific template image data. It is explanatory drawing explaining the method to extract an area | region as identification object image data from the captured image data.

  A method for extracting the image area most similar to the template image data from the captured image data 20 as the identification target image data will be described with reference to FIGS.

  First, for example, as illustrated in FIG. 6A, the maximum of the captured image data 20 and the template image data (forward surface) 26 is obtained by template matching of the captured image data 20 and the template image data (forward surface) 26. A correlation value is calculated.

  For example, template matching between the captured image data 20 and the template image data (positive surface) 26 calculates each correlation value by pattern matching while moving the template image data (positive surface) 26 on the captured image data 20. The maximum correlation value that is the maximum value among the calculated correlation values is calculated.

  Further, for example, each correlation value by pattern matching is calculated while moving the captured image data 20 on the template image data (positive surface) 26, and the maximum correlation value that is the maximum value among the calculated correlation values is calculated. May be.

The case where each correlation value by pattern matching is calculated while moving the template image data (positive-direction surface) 26 on the captured image data 20 will be specifically described as an example. Each pixel f (i, j) of the captured image data 20 ), The position corresponding to the upper left pixel of the partial image data is extracted from the start point (u) when extracting the partial image data having the same size (the same number of pixels, for example, M × N) as the template image data (positive surface) 26. V), and a pattern of each pixel f (u + i, v + j) of the partial image data corresponding to the position of the start point (u, v) and each pixel t (i, j) of the template image data (positive surface) 26 Template matching is performed such that the correlation value is calculated while sequentially moving the position of the start point (u, v) in the captured image data 20.
The calculation of the correlation value by template matching between the captured image data 20 and the template image data (positive surface) 26 can be calculated as a normalized correlation value as shown by the following equation, for example.

  In the above expression, R (u, v) is a correlation value between the partial image data and the template image data (positive surface) 26 at the position of the starting point (u, v) of the partial image data on the captured image, and f ( u + i, v + j) is each pixel value of the partial image data accompanying the movement of the start point (u, v), f ′ is the average value of the pixels of the partial image data, and t (i, j) is the template image data (forward surface) ) 26 pixel values, t ′ indicates the average value of the pixels of the template image data (positive surface) 26, respectively.

  Further, as the pixel value, for example, a value such as a density value of each pixel constituting each image is used.

  The calculation result of the normalized correlation value between the partial image data corresponding to the start point (u, v) calculated by the above equation and the template image data (positive direction surface) 26 is, for example, a curve as shown in FIG. It can be obtained as a graph 30.

  As shown in FIG. 6B, the horizontal axis of the curve graph 30 indicates the position information of the start point (u, v) of the partial image data extracted from the captured image data 20, and the vertical axis indicates the start point (u, The normalized correlation value between the partial image data corresponding to v) and the template image data (positive-direction surface) 26 is shown.

  The correlation value indicates the similarity between the partial image data and the template image data (positive-direction surface) 26. In the curve graph 30, the vertex A of the curve graph 30 is the template image data (on the captured image data 20). The maximum correlation value A by the template matching of (positive-direction surface) 26 is shown.

  When the maximum correlation value A is calculated by template matching between the captured image data 20 and the template image data (forward direction surface) 26, the template image data (forward direction surface) 26 is sequentially replaced with the template image data 27, 28, and 29. The maximum correlation value between the replaced template image data 27, 28, 29 and the captured image data 20 is calculated by the template matching described above.

  As shown in FIG. 6C, the maximum correlation value by template matching between the captured image data 20 and the template image data (back in the positive direction) 27 is the vertex B of the curve graph 31 as shown in FIG. As shown in FIG. 7A, the maximum correlation value obtained by template matching between the captured image data 20 and the template image data (reverse surface) 28 is a curve graph as shown in FIG. As shown in FIG. 7C, the maximum correlation value obtained by template matching between the captured image data 20 and the template image data (reverse reverse side) 29 is calculated as shown in FIG. It is calculated as the vertex D of the curve graph 33.

  The template image data having the largest correlation value among the maximum correlation values A, B, C, and D of the calculated template image data 26, 27, 28, and 29 is temporarily set as specific template image data. To do.

  Further, an image region in which the specific template image data is superimposed at a position on the captured image data 20 when the maximum correlation value is obtained by template matching between the specific template image data and the captured image data 20 is extracted from the captured image data 20. As a result, paper sheet identification target image data 34 as shown in FIG. 7E is extracted.

  Further, by specifying the specific template image data from the template image data 26, 27, 28, and 29, the normal / reverse and front / back insertion directions of the paper sheets are determined corresponding to the specific template image data. .

  In this example, the specific template image data is template image data (forward surface) 26, and the insertion direction of the inserted paper sheet corresponds to the template image data (forward surface) 26. Positive direction, determined as surface.

  Further, the template image data corresponding to each insertion direction of the front and back and the front and back is, for example, stored in the template image database 72 template image data of only one direction and one side of a genuine note, and a captured image of the inserted paper sheet When performing template matching with data, a template image obtained by horizontally inverting, vertically inverting, horizontally inverting and then vertically inverting the template image data may be used as a template image corresponding to the normal and reverse insertion directions.

  Further, for example, template image data for only one direction and one side of a genuine note is stored in the template image database 72, and the captured image data of the inserted paper sheet corresponds to each insertion direction and each side of the front and back. The image data may be rotationally corrected, and template matching may be performed between the rotationally corrected captured image data and the template image data.

  Next, a method for detecting scratches and dirt present in the identification target image data 34 extracted from the captured image data 20 will be described.

  FIG. 8 is an explanatory diagram showing a method for detecting image data indicating scratches and dirt existing in the identification target image data 34 extracted from the captured image data 20.

  8A is a diagram showing specific template image data (positive surface) 26, FIG. 8B is a diagram showing identification target image data 34 including scratches and dirt, and FIG. 8C is a diagram. FIG. 9 is a diagram illustrating difference image data 35 obtained by calculating a difference between the specific template image data (forward-direction surface) 26 illustrated in FIG. 8A and the identification target image data 34 illustrated in FIG.

  When scratches and dirt are attached to the paper sheets, or when scratches and dirt are attached to the transmitted light type optical sensor 73, when the paper sheets are imaged, the scratches and dirt are caused by the light emitting element 731. By blocking the light from the image, an unnecessary image due to scratches and dirt is captured in the image of the imaging region of the paper sheet acquired by the light receiving element 732.

  In particular, when scratches and dirt are attached in the watermark area 21 of the paper sheet, or scratches and dirt are attached to the position of the transmitted light type optical sensor 73 that blocks the transmitted light in the watermark area 21. In such a case, the identification target image data includes an image showing a scratch and a stain.

  Therefore, as shown in FIGS. 6 to 7, even if the image data 34 is the identification target image data 34 obtained by extracting the image region having the highest similarity with the specific template image data (positive-direction surface) 26 from the captured image data 20, There may be image data indicating scratches and dirt in the identification target image data 34.

  Therefore, as shown in FIG. 8, the difference between the specific template image data (positive surface) 26 and the identification target image data 34 including scratches and dirt is calculated.

  As shown in FIGS. 8A and 8B, the common image data in the specific template image data (positive-direction surface) 26 and the identification target image data 34 is the image data of the watermark pattern 23 and is not common. The image data is flaw and dirt image data 25 present in the identification target image data 34.

  Therefore, by calculating the difference between the specific template image data (positive-direction surface) 26 and the identification target image data 34, the difference image data 35 of the image data 25 showing the scratches and dirt as shown in FIG. 8C is detected. can do.

  The detection formula for detecting scratches and dirt can be calculated by the following formula, for example.

  In the above equation, d is a difference between the specific template image data (positive surface) 26 and the identification target image data 34, t (i, j) is each pixel value of the specific template image data (positive surface) 26, t ′ is an average value of the pixels of the specific template image data (positive surface) 26, f (u + i, v + j) is each pixel value of the identification target image data 34, and f ′ is an average value of the pixels of the identification target image data 34. Each is shown.

  Further, as the pixel value, for example, a value such as a density value of each pixel constituting each image is used.

  Then, the influence of the detected image data 25 indicating flaws and dirt is corrected, pattern correction between the corrected identification target image data (not shown) and the identification target image data 34 is performed, and a correlation value between the two images is obtained. By calculating, the type and authenticity of the paper sheet are identified.

  Specifically, for example, the detected image data 25 indicating scratches and dirt of the identification target image data 34 is converted into specific template image data (positive surface) 26 corresponding to the same position and the same area of the identification target image data 34. The sheet is pasted and converted into corrected identification target image data (not shown), pattern matching between the corrected identification target image data and the identification target image data 34 is performed, and a correlation value between the two images is calculated. Identify the type and true / false of the class.

  FIG. 9 illustrates an example of a method for identifying the type and authenticity of paper sheets based on the corrected identification target image data 36 and the identification target image data 34 corrected for the influence of image data indicating scratches and dirt. FIG.

  FIG. 9A is a diagram showing an example of the difference image data 35 indicating the flaws and dirt 25 existing in the identification target image data 34 detected by the flaw / dirt detection unit 76, and FIG. FIG. 9C shows an example of the specific template image data (positive-direction surface) 26, FIG. 9C shows the corrected identification target image data 36, and FIG. 9D shows the identification target image data 34. It is.

  A method for identifying the type and authenticity of paper sheets will be described with reference to FIG.

  For example, as shown in FIG. 9D, if the identification target image data 34 extracted from the captured image data of the imaging region of the paper sheet has a scratch and a stain 25, the scratch / dirt detection unit 76 performs the processing shown in FIG. Difference image data 35 indicating scratches and dirt 25 as shown in (a) is detected.

  The scratch / dirt influence correcting unit 77 applies the difference image data 35 indicating the scratch and the stain 25 detected by the scratch / dirt detecting unit 76 to the specific template image data (forward direction surface) 26 shown in FIG. The identification target image data 34 is pasted so as to have the same position and the same area as the scratches and dirt 25 present in the identification target image data 34, and converted into corrected identification target image data 36 as shown in FIG. 9C.

  Then, pattern matching between the corrected identification target image data 36 and the identification target image data 34 is performed, and a correlation value between the two images is calculated. If the calculated correlation value is equal to or greater than a predetermined value, the inserted paper sheet It is determined that the class is a genuine note, and is determined to be a false note if the calculated correlation value is not equal to or greater than a predetermined value.

  In addition to the method of identifying the type and authenticity of the paper sheet based on the corrected identification target image data and the identification target image data, the difference image data of the difference between the identification target image data and the specific template image data is used. The configuration may be such that the type and authenticity of the paper sheet are identified based on it.

  Specifically, based on the difference image data of the difference between the identification target image data extracted from the captured image data of the paper sheet and the specific template image data, for example, when there is no image data indicating scratches and dirt in the difference image data All pixels of the difference image data are “white”.

  That is, by confirming that all the pixels of the difference image data are “white”, it is possible to identify the paper sheet as a genuine note.

  However, in actuality, it is considered that there is an image showing some kind of scratches and dirt in the identification target image data. Therefore, an image without paper sheets captured by the transmitted light type optical sensor 73 in a state without paper sheets in advance. A configuration may be adopted in which the type and authenticity of the paper sheet are identified by pattern matching between the data and the difference image data of the image indicating the scratch and dirt attached to the paper sheet detected by the scratch / dirt detection unit 76. .

  Further, by utilizing the fact that the shape of the watermark region 21 is different for each type of paper sheet to identify the type and authenticity of the paper sheet, for example, the perimeter of the watermark region extracted from the captured image of the paper sheet, It is also possible to identify the type and authenticity of paper sheets from the circularity and area.

  In addition, when identifying paper sheets whose watermark area is deviated from the center in the short direction, the insertion direction of the paper sheet is determined based on the center of gravity of the watermark area calculated from the captured image, and the type of paper sheet It is also possible to reduce the true / false identification process.

  In the actual paper sheet identification processing, the authenticity determination is not performed by identifying the type and authenticity of the paper sheet only by the paper sheet identification method according to the present invention, but by combination with other identification factors. You may defeat.

  In the first embodiment, paper sheets are inserted in forward and reverse directions, front and back directions and surfaces, and feature images such as watermark patterns of paper sheets are registered in advance as template images corresponding to the insertion directions and inserted. The template matching between the captured image acquired from the paper sheet and the template image corresponding to each insertion direction determines the insertion direction of the inserted paper sheet and extracts the image area most similar to the template image from the captured image. An example of how to do it was shown.

  Next, in the second embodiment, a template image corresponding to a feature image such as a watermark pattern of a paper sheet inserted in only one direction is registered in advance, and the center of gravity of the watermark area of the captured image of the inserted paper sheet is registered. An example of a method for determining the insertion direction of a paper sheet based on the image and extracting the image region most similar to the template image from the captured image by rotationally correcting the template image based on the determined insertion direction will be described.

  FIG. 10 determines the insertion direction of the inserted paper sheet based on the center of gravity of the watermark area of the captured image of the inserted paper sheet, and rotates the template image corresponding to the determined insertion direction of the paper sheet It is the block diagram which showed schematic structure of the paper sheet identification device provided with the identification part to which the method which correct | amends and extracts the image area most similar to a template image from a captured image was applied.

  As shown in FIG. 10, the paper sheet identification device 100 has the same configuration as the paper sheet identification device 1 except that the identification unit 7 of the paper sheet identification device 1 shown in FIG. The operations of the units other than the identification unit 8 of the paper sheet identification apparatus 100 are configured to operate in the same manner as the units illustrated in the paper sheet identification apparatus 1.

  Therefore, the configuration and operation of each unit other than the identification unit 8 of the paper sheet identification apparatus 100 will be omitted for convenience of explanation, and the identification unit 8 will be described.

  FIG. 11 is a block diagram illustrating an example of a functional configuration of the identification unit 8.

  As shown in FIG. 11, the identification unit 8 has a function of extracting the watermark area included in the captured image, a function of calculating the center of gravity of the extracted watermark area, and a calculation in addition to the function of the identification unit 7 illustrated in FIG. 2. A function for determining the insertion direction of the inserted paper sheet based on the center of gravity of the watermark area is added.

  The paper sheet identification apparatus 100 includes an identification control unit 80 that performs overall control of the identification unit 8, a transmitted light type optical sensor 83, a memory 81, a template image database 82, a pattern matching unit 84, and an identification target image extraction unit. A unit 85, a scratch / dirt detection unit 86, a scratch / dirt effect correction unit 87, a watermark region extraction unit 88, a watermark region centroid calculation unit 89, and an insertion direction determination unit 90.

  In the transmitted light type optical sensor 83, a pair of a light emitting element 831 and a light receiving element 832 forms a pair of paper sheets inserted from the paper sheet insertion unit 3 so as to face each other and sandwich the paper sheet transport unit 4. It is arranged on a predetermined scanning line passing through an imaging area including a watermark area of a paper sheet to be identified.

  The light emitting element 831 irradiates light to the image pickup area of the paper sheet conveyed by the paper sheet conveyance unit 4, and the light receiving element 832 receives the transmitted light that has passed through the image pickup area of the paper sheet, An electric signal corresponding to the amount of received transmitted light is output.

  The memory 81 corresponds to the digital signal corresponding to the signal level of the electrical signal output from the transmitted light type optical sensor 83 at a predetermined time interval in the predetermined storage area of the memory 81 in the order of output from the transmitted light type optical sensor 83. Are stored, assigned consecutive addresses, and temporarily stored as captured image data of the imaging region of the paper sheet.

  The watermark area extraction unit 88 reads the captured image data of the paper stored in the memory 81 and extracts the image data of the watermark area included in the captured image data.

  The watermark area centroid calculation unit 89 calculates the circumference, circularity, area, and the like of the watermark area from the watermark area image data extracted by the watermark area extraction unit 88, and calculates the circumference, circularity, area, etc. of the calculated watermark area. Based on the above, the center of gravity of the watermark area is calculated.

  The insertion direction determination unit 90 determines the insertion direction of the inserted paper sheet based on the center of gravity of the watermark area calculated by the watermark area center of gravity calculation unit 89.

  The determination of the insertion direction of the paper sheet is performed by, for example, referring to the front and back surfaces of each type of paper sheet registered and managed in advance and the gravity center position information of the watermark area corresponding to the insertion direction, and the watermark area gravity center calculation unit 89 Based on the calculated center of gravity of the watermark area, the insertion direction of the inserted paper sheet is determined.

  The template image database 82 stores, as template image data, image data obtained by capturing in advance a watermark region or the like, which is a characteristic image of a genuine paper sheet, and also stores gravity center position information of the watermark region of each template image. Has been.

  In the template image data, for example, a feature image such as a watermark pattern corresponding to the type of paper sheet of a genuine note is stored.

  The pattern matching unit 84 reads template image data stored in the template image database 82, and the read template image data is reversed left and right, up and down corresponding to the insertion direction of the paper sheet determined by the insertion direction determination unit 90. An image area most similar to the template image is searched and specified on the captured image data while performing rotation correction such as inversion. Further, a correlation value obtained by pattern matching between the identification target image data and the corrected identification target image data is calculated and output.

  The identification target image extraction unit 85 extracts the image region most similar to the template image data specified by the pattern matching unit 84 from the captured image data as the identification target image data, and stores it in the memory 81.

  The scratch / dirt detection unit 86 calculates the difference between the identification target image data extracted by the identification target image extraction unit 85 and the template image data, and the scratches and dirt attached to the paper sheets existing in the identification target image data. An image indicating dirt (or scratches and dirt attached to the transmitted light type optical sensor) is detected.

  The scratch / dirt influence correcting unit 87 corrects the influence of the image indicating the scratch and dirt detected by the scratch / dirt detecting unit 86.

  The operation in the case where the identification unit 8 configured in this way identifies the type and authenticity of the paper sheet inserted into the paper sheet insertion unit 3 will be briefly described.

  The paper sheet inserted from the paper sheet insertion unit 3 is transported by the paper sheet transport unit 4, and when the paper sheet reaches the arrangement position of the transmitted light type optical sensor 83 of the identification unit 8, the light emitting element 831 Light is irradiated to the imaging region of the leaf, and the light receiving element 832 receives the transmitted light transmitted through the imaging region of the paper sheet.

  The transmitted light type optical sensor 83 outputs an electric signal having a signal level corresponding to the amount of transmitted light received by the light receiving element 832, and the electric signal output by the transmitted light type optical sensor 83 is digital corresponding to the signal level. It is converted into a signal and input to the memory 81.

  The memory 81 temporarily stores digital signal information corresponding to the signal level output from the transmitted light type optical sensor 83 as captured image data of the imaging region of the paper sheet.

  When the captured image data of the paper sheet is stored in the memory 81, the watermark area extraction unit 88 reads the captured image data of the paper sheet stored in the memory 81, and the image data of the watermark area included in the captured image data (Hereinafter referred to as “watermark region image data”) is extracted, and the extracted watermark region image data is output to the watermark region centroid calculating unit 13.

  The watermark area centroid calculation unit 89 calculates the centroid of the watermark area from the watermark area image data input from the watermark area extraction unit 88 and outputs the calculated centroid information of the watermark area to the insertion direction determination unit 90.

  The insertion direction determination unit 90 is based on the input center-of-gravity information of the watermark area, and the center-of-gravity position information of each watermark area corresponding to the front and back surfaces of each genuine sheet and the insertion direction stored in the template image database 82. The insertion direction of the inserted paper sheet is determined with reference to FIG.

  The pattern matching unit 84 reads the template image data stored in the template image database 82, and the template image data is inverted upside down, left / right inverted, etc. based on the insertion direction of the paper sheet determined by the insertion direction determination unit 90. Rotation correction is performed, template matching between the corrected template image data and captured image data is performed, and an image region of captured image data that is most similar to the template image data is searched and specified.

  The identification target image extraction unit 85 extracts the image area of the captured image data most similar to the template image data specified by the pattern matching unit 84 as the identification target image data of the paper sheet, and outputs it to the scratch / dirt detection unit 86. To do.

  The scratch / dirt detection unit 86 calculates the difference between the input identification target image data and the specific template image data, detects image data indicating scratches and dirt existing in the identification target image data, and affects the scratch / dirt influence. Output to the correction unit 87.

  The scratch / dirt influence correcting unit 87 outputs corrected identification target image data corrected for the influence of the image data indicating the scratch and dirt detected and input by the scratch / dirt detecting unit 86 to the pattern matching unit 84.

  The pattern matching unit 84 performs pattern matching between the corrected identification target image data input from the scratch / dirt influence correction unit 87 and the identification target image data, calculates a correlation value between these two image data, and an identification control unit 80 identifies the type and authenticity of the paper sheet based on the correlation value obtained by pattern matching between the corrected identification target image data and the identification target image data calculated by the pattern matching unit 84.

  A processing procedure for identifying and processing the type and authenticity of the inserted paper sheet by the paper sheet identifying apparatus 100 having the identifying unit 8 operating as described above will be described with reference to the flowchart shown in FIG. To do.

  When a paper sheet is inserted from the paper sheet insertion unit 3 (YES in step S1201), the paper sheet identification apparatus 100 images the paper sheet using the transmitted light type optical sensor 83 (step S1202). The watermark area is extracted from the captured image data of the paper sheet obtained in this way, and the center of gravity of the extracted watermark area is calculated (step S1203).

  Based on the calculated center of gravity of the watermark area, the insertion direction of the paper sheet is determined with reference to a matrix table (not shown) that stores the center of gravity information corresponding to the insertion direction and the insertion surface (step S1204). ).

  If the determination result of the paper sheet insertion direction is positive (YES in step S1205), the maximum correlation value E is calculated by template matching between the template image data read from the template image database 82 and the captured image data. (Step S1206).

  In the template image database 82, feature images of paper sheets inserted on the surface in the forward direction are registered in advance as template images.

  Next, in order to rotationally correct the template image data into the image data when inserted on the back surface in the forward direction, the template image data is reversed left and right, and the template image data (hereinafter referred to as “template image data (left and right) Inverse) ”)) and the maximum correlation value F by template matching between the captured image data is calculated (step S1207).

  The maximum correlation value E and the maximum correlation value F are compared (step S1208). If the maximum correlation value E is large (YES in step S1208), the template image data is set as specific template image data (step S1209). Then, an image region most similar to the set specific template image data is extracted from the captured image data as an identification target image (step S1216).

  The maximum correlation value E and the maximum correlation value F are compared (step S1208). If the maximum correlation value F is large (NO in step S1208), the template image data (inverted horizontally) is set as the specific template image data. (Step S1210), an image region most similar to the set specific template image data is extracted from the captured image data as an identification target image (Step S1216).

  In step S1205, if the determination result of the insertion direction of the paper sheet is determined to be the reverse direction (NO in step S1205), the template image data is rotationally corrected to the image data when inserted on the reverse surface. Then, the template image data is inverted upside down, and a maximum correlation value G is calculated by template matching between the template image data after the upside down (hereinafter referred to as “template image data (upside down)”) and the captured image data (step). S1211).

  Next, in order to rotationally correct the template image data to the image data when inserted in the reverse reverse direction, the template image data (upside down) is reversed left and right, that is, after the template image data is turned upside down, A maximum correlation value H is calculated by template matching between the template image data (hereinafter referred to as “template image data (inverted vertically and horizontally reversed)”) that has been horizontally reversed and the captured image data (step S1212).

  The maximum correlation value G and the maximum correlation value H are compared (step S1213). If the maximum correlation value G is large (YES in step S1213), the template image data (upside down) is set as the specific template image data. (Step S1214), an image region most similar to the set specific template image data is extracted from the captured image data as an identification target image (Step S1216).

  The maximum correlation value G and the maximum correlation value H are compared (step S1213). If the maximum correlation value H is large (NO in step S1213), the template image data (inverted horizontally and vertically) is set as the specific template image data. At the same time (step S1215), an image region most similar to the set specific template image data is extracted from the captured image data as an identification target image (step S1216).

  When the identification target image data is extracted from the captured image data, the extracted identification target image data is subjected to scratch / dirt correction and identification processing S1217, and the type and authenticity of the inserted paper sheet are identified to complete the processing. To do.

  Since the processing method of the flaw / dirt correction and identification processing S1217 is the same as the processing shown in FIG. 4 in the first embodiment, detailed description thereof is omitted for convenience of description.

  Here, the paper sheet identification method according to the present invention performed by the identification unit 8 of the paper sheet identification apparatus 100 will be described in detail.

  FIG. 13 is an explanatory diagram for explaining a method of extracting a watermark area from a captured image of a paper sheet and calculating the center of gravity of the extracted watermark area.

  FIG. 13A is a diagram illustrating an example of captured image data 320 of the imaging region in which the surface portion of the paper sheet inserted in the positive insertion direction is captured by the transmitted light type optical sensor 83. FIG. FIG. 13B is a view showing the watermark region image 321 extracted from the captured image data 320 shown in FIG. 13A, and FIG. 13C is a view showing the center of gravity 330 of the extracted watermark region 321 image. is there.

  As illustrated in FIG. 13A, the captured image data 320 of the paper sheet includes, for example, each pixel of f (i, j) having a number of pixels of L × L. The captured image data 320 includes The watermark area 321 and the normal area 322 other than the watermark area exist, and the watermark pattern 323 exists in the watermark area 321.

  Further, the picked-up image data 320 includes a paper sheet scratch and stain 325.

  In FIG. 13A, i = 0, 1, 2,..., L−1, j = 0, 1, 2,.

  The circumference, circularity, area, and the like of the watermark area 321 are calculated from the extracted image of the watermark area 321. Based on the calculated circumference, circularity, area, etc. of the watermark area 321, as shown in FIG. The center of gravity 330 of the watermark area 321 is calculated.

  In addition, when extracting the image of the watermark area 321 from the captured image data 320 of the paper sheet, the watermark area 321 and the normal area 322 are obtained by performing binarization processing on the captured image data 320 as preprocessing. And the image of the watermark region 321 may be extracted from the captured image data 320 as shown in FIG.

  FIG. 14 is an explanatory diagram for explaining a determination method for determining the insertion direction of the inserted paper sheet based on the center of gravity of the watermark area of the captured image.

  FIG. 14A is a diagram showing an example of the center-of-gravity information of the watermark area corresponding to the front and back surfaces of the genuine paper sheet inserted in the forward direction, and FIG. 14B is the reverse direction. FIG. 14C shows an example of the center-of-gravity information of the watermark area corresponding to the front and back surfaces of the inserted genuine paper sheet. FIG. 14C shows the front and back surfaces of the inserted paper sheet in the forward direction. FIG. 14D is a diagram showing an example of the center of gravity of the watermark area of the captured image captured correspondingly, and FIG. 14D shows the captured image captured corresponding to the front and back surfaces of the paper sheet inserted in the reverse direction. It is a figure which shows an example of the gravity center of a watermark area | region.

  As shown in FIGS. 14A and 14B, generally, the centers of gravity 436 and 536 of the watermark areas 421 and 521 of the paper sheet are either in the vertical direction with respect to the captured image areas 420 and 520. 14 (a), when the paper sheet is inserted in the forward direction, the paper sheet is inserted on either the front surface or the back surface. However, the center of gravity 436 of the watermark area 421 exists below the position on the center line 150 of the imaging area, and when a paper sheet is inserted in the reverse direction as shown in FIG. The center of gravity 536 of the watermark region 521 exists above the position on the center line 150 of the imaging region even when the class is inserted on either the front surface or the back surface.

  Therefore, based on the center of gravity of the watermark area of the captured image data, for example, as shown in FIG. 14C, the center of gravity 626 of the watermark area 621 of the captured image 620 is positioned on the center line 151 of the captured image data 620. If the paper sheet is present in the lower direction, the insertion direction of the paper sheet is determined as the forward direction, and the captured image data 720 when the paper sheet is inserted in the reverse insertion direction as shown in FIG. When the center of gravity 726 of the watermark area 721 is above the position on the center line 151 of the captured image data 720, the insertion direction of the paper sheet is determined as the reverse direction.

  15 to 16 are explanatory diagrams for explaining a method of extracting an identification target image from the captured image based on the determined insertion direction of the paper sheet.

  FIG. 15 is an explanatory diagram for explaining a method of extracting the identification target image data from the captured image data 620 in which the insertion direction of the inserted paper sheet is determined to be the positive direction as shown in FIG. FIG. 16 illustrates a method for extracting the identification target image data from the captured image data 720 in which the insertion direction of the inserted paper sheet is determined to be the reverse direction as illustrated in FIG. FIG.

  15, FIG. 15 (a) is a diagram showing template image data 40 registered in advance in the template image database 82 corresponding to the type of inserted paper sheet.

  The template image data 40 includes t (i, j) having, for example, a number of pixels of M × N, in which an image area of a predetermined size including a watermark area in which a genuine note is inserted in a positive direction and a surface portion is imaged. ) Is registered in the template image database 82 as image data composed of each pixel.

  In FIG. 15A, i = 0, 1, 2,..., M−1, j = 0, 1, 2,.

  First, as shown in FIG. 15B, the maximum correlation value obtained by template matching between the captured image data 620 of the paper sheet whose insertion direction is determined to be the positive direction and the template image data 40 shown in FIG. Is calculated.

The maximum correlation value obtained by template matching between the captured image data 620 and the template image data 40 is the same as that of the template image data 40 in each pixel f (i, j) of the captured image data 620 as described in the first embodiment. The position corresponding to the upper left pixel of the partial image data having the same size (the same number of pixels, eg, M × N) is set as the start point (u, v), and each pixel f of the partial image data corresponding to the position of the start point (u, v) Pattern matching between (u + i, v + j) and each pixel t (i, j) of the template image data 40 is performed, and a correlation value is calculated while sequentially moving the position of the start point (u, v) in the captured image data 20. Perform template matching.
The correlation value obtained by template matching between the captured image data 620 and the template image data 40 is calculated as a normalized correlation value represented by the following equation, for example.

  In the above equation, R (u, v) is the correlation value between the partial image data and the template image data 40 at the start point (u, v) position of the partial image data of the captured image data 620, and f (u + i, v + j) is , Each pixel value of the partial image data accompanying the movement of the start point (u, v), f ′ is an average value of the pixels of the partial image data, t (i, j) is each pixel value of the template image data 40, and t ′ is The average values of the pixels of the template image data 40 are shown.

  Further, as the pixel value, for example, a value such as a density value of each pixel constituting each image is used.

  Each correlation value between the partial image data corresponding to the start point (u, v) calculated by the above equation and the template image data 40 is calculated as a curve graph 300 as shown in FIG. Vertex E indicates the maximum correlation value E by template matching of the template image data 40 on the captured image data 620.

  When the maximum correlation value E by template matching between the captured image data 620 and the template image data 40 is calculated, the template image data (left-right reversed) 41 as shown in FIG. And the maximum correlation value between the template image data (horizontal inversion) 41 and the captured image data is calculated by template matching in the same manner as described above.

  The correlation value obtained by template matching between the captured image data 620 and the template image data (horizontal inversion) 41 is calculated as a normalized correlation value as shown by the following equation, for example.

  In the above equation, R (u, v) is the correlation value between the partial image data and the template image data 41 at the start point (u, v) position of the partial image data of the captured image data 620, and f (u + i, v + j) is , Each pixel value of the partial image data accompanying the movement of the start point (u, v), f ′ is an average value of the pixels of the partial image data, t (i, M−j) is each pixel value of the template image data 41, t 'Indicates the average value of the pixels of the template image data 41.

  Further, as the pixel value, for example, a value such as a density value of each pixel constituting each image is used.

  Each correlation value between the partial image data corresponding to the start point (u, v) calculated by the above equation and the template image data 41 is calculated as a curve graph 301 as shown in FIG. Vertex F indicates the maximum correlation value F obtained by template matching between the captured image data 620 and the template image data 41.

  Template image data showing a large value among the maximum correlation values E and F of the calculated template image data 40 and 41 is temporarily set as specific template image data, and an image region most similar to the specific template image data is determined. By extracting from the captured image data 620, paper sheet identification target image data 634 as shown in FIG. 15F is extracted.

  Next, when it is determined that the insertion direction of the inserted paper sheet is the reverse direction, as shown in FIG. 16B, the captured image data 720 of the paper sheet whose insertion direction is determined to be the reverse direction, and A maximum correlation value is calculated by template matching between template image data (upside down) 42 (see FIG. 16A) obtained by vertically inverting the template image data 40 shown in FIG. 15A and the captured image data 720.

  The calculation of the correlation value by template matching between the template image data (upside down) 42 and the captured image data 720 can be calculated as a normalized correlation value as shown by the following equation, for example.

  In the above equation, R (u, v) is the correlation value between the partial image data and the template image data 42 at the position (u, v) of the partial image data of the captured image data 720, and f (u + i, v + j) is , Each pixel value of the partial image data accompanying the movement of the start point (u, v), f ′ is an average value of the pixels of the partial image data, t (N−i, j) is each pixel value of the template image data 42, t 'Indicates the average value of the pixels of the template image data 42, respectively.

  Further, as the pixel value, for example, a value such as a density value of each pixel constituting each image is used.

  The calculation result of the normalized correlation value between the partial image data corresponding to the start point (u, v) calculated by the above equation and the template image data 42 is calculated as a curve graph 302 as shown in FIG. The vertex G of the curve graph 302 indicates the maximum correlation value G obtained by template matching between the captured image data 720 and the template image data 42.

  When the maximum correlation value G by the template matching between the captured image data 720 and the template image data 42 is calculated, the template image data 42 is further inverted horizontally, that is, the template image data 40 shown in FIG. After the vertical inversion, the template image data (inverted up / down / left / right inverted) 43 as shown in FIG. 16C that is further inverted horizontally is replaced with the maximum correlation between the replaced template image data (up / down / left / right inverted) 43 and the captured image data 720. The value is calculated by template matching as described above.

  The correlation value obtained by template matching between the captured image data 720 and the template image data (vertical up / down / left / right reversal) 43 is calculated as a normalized correlation value represented by the following equation, for example.

  In the above formula, R (u, v) is the correlation value between the partial image data and the template image data 43 at the position (u, v) of the partial image data of the captured image data 720, and f (u + i, v + j) is , Each pixel value of the partial image data accompanying the movement of the start point (u, v), f ′ is an average value of the pixels of the partial image data, and t (N−i, M−j) is each pixel value of the template image data 43. , T ′ indicate the average values of the pixels of the template image data 43, respectively.

  Further, as the pixel value, for example, a value such as a density value of each pixel constituting each image is used.

  Each correlation value between the partial image data corresponding to the start point (u, v) calculated by the above equation and the template image data 43 is calculated as a curve graph 303 as shown in FIG. Of the captured image data 720 and the template image data 43 indicates the maximum correlation value H by template matching.

  Template image data showing a large value among the maximum correlation values G and H of the calculated template image data 42 and 43 is temporarily stored as specific template image data, and an image region most similar to the specific template image data is determined. By extracting from the captured image data 720, paper sheet identification target image data 734 as shown in FIG. 16F is extracted.

  When the identification target image data 634 or 734 is extracted, image data indicating scratches and dirt present in the extracted identification target image data 634 or 734 is detected, and the influence of the detected image data indicating scratches and dirt is corrected. The corrected identification target image data is calculated, and the type and authenticity of the captured paper sheet are identified based on the calculated corrected identification target image data and the identification target image data.

  It is to be noted that the type and authenticity of the captured paper sheet are identified based on the detection method of image data indicating scratches and dirt existing in the identification target image data, and the calculated corrected identification target image data and the identification target image data. Since the method is the same as that shown in FIGS. 8 and 9 of the first embodiment, detailed description thereof is omitted.

  In the second embodiment, the insertion direction of the inserted paper sheet is determined based on the center of gravity of the watermark area of the captured image of the inserted paper sheet, and the template image corresponding to the determined insertion direction of the paper sheet The method of extracting the image area most similar to the template image from the captured image has been described, but the insertion direction in which the template image was captured corresponding to the determined insertion direction of the paper sheet Alternatively, rotation correction may be performed on the image corresponding to the insertion surface, and an image region most similar to the template image obtained by template matching between the rotation-corrected captured image and the template image may be extracted from the captured image.

  In the first and second embodiments, the watermark area including the watermark pattern formed on the paper sheet is shown as an example of the characteristic image of the paper sheet. However, the present invention is not limited to the type and authenticity of the paper sheet. Can be used as a feature image as long as the image area can be identified and captured.

  As described above, the insertion direction of the paper sheet is determined by template matching between the template image corresponding to each insertion direction of the front and back sides of the feature image of the paper sheet registered in advance and the front and back, and the captured image of the paper sheet. At the same time, the feature image area of the paper sheet is extracted as the identification target image, so that the influence of the positional deviation of the characteristic image formed on the paper sheet is reduced and the correction based on the extracted identification target image and the template image is performed. The type and authenticity of the paper sheet can be determined accurately and at high speed by efficiently collating with the already identified image.

  In addition, the insertion direction of the inserted paper sheet is determined based on the center of gravity of the watermark area of the captured image of the paper sheet, and the template matching between the template image and the captured image is performed based on the determination result. Since the image area corresponding to the image is extracted as the identification target image, the influence of the positional deviation or the like of the feature image formed on the paper sheet is reduced, and the corrected identification target based on the extracted identification target image and the template image It becomes possible to accurately and quickly determine the type and authenticity of paper sheets by efficiently collating with images.

1 is a schematic configuration diagram of a paper sheet identification apparatus to which a paper sheet identification method according to the present invention is applied. Block diagram of a functional configuration of the identification unit 7 of the paper sheet identification apparatus 1 A flowchart of a processing procedure in which the identification unit 7 extracts an identification target image from the captured image. Flowchart of a processing procedure for identifying the type and authenticity of a paper sheet from the extracted identification target image The figure which shows an example of captured image data and template image data Explanatory drawing of the method of extracting an identification target image from a captured image Explanatory drawing of the method of extracting an identification target image from a captured image Explanatory drawing of the detection method of a crack and dirt which exist in a classification subject image Explanatory diagram of the method for identifying the type and authenticity of paper sheets after correcting the effects of scratches and dirt images The paper sheet identification device 1 is a schematic configuration diagram of another paper sheet identification device Block diagram of a functional configuration of the identification unit 8 of the paper sheet identification apparatus 100 A flowchart of a processing procedure for the identification unit 8 to extract an identification target image from the captured image. Explanatory drawing explaining the method of calculating the gravity center of the watermark area | region of a captured image Explanatory drawing explaining the method to determine the insertion direction of paper sheets based on the gravity center of the watermark area | region of a captured image Explanatory drawing of the method of extracting an identification target image from the captured image of the paper sheets inserted in the positive direction Explanatory drawing of the method of extracting an identification object image from the captured image of the paper sheets inserted in the reverse direction

Explanation of symbols

DESCRIPTION OF SYMBOLS 1,100 Paper sheet identification apparatus 2 Control part 3 Paper sheet insertion part 4 Paper sheet conveyance part 5 Paper sheet acceptance part 6 Drive part 7, 8 Identification part 20 Imaged image data of imaging area of paper sheet 21 Watermark Area 22 Normal area 23 Watermark pattern 24 Template image data 25 Scratches, wrinkles, dirt 26, 31 Correlation value graphs 27, 271, 272 Corrected identification target image data 28, 281, 282 Difference image data 29 Centroid of watermark area 30 Predetermined range Image data 70, 80 Identification control unit 71, 81 Memory 72, 82 Template image database 73, 83 Transmitted light sensor 74, 84 Pattern matching unit 75, 85 Identification target image extraction unit 76, 86 Scratch / dirt detection unit 77, 87 Scratch / Dirty Effect Correction Unit 88 Watermark Region Extraction Unit 89 Watermark Region Center of Gravity Calculation Unit 90 Insertion Direction Determination Unit 731 and 831 Light emitting element 732 and 832 Light receiving element

Claims (8)

In a paper sheet identification device that cuts out a feature image of the paper sheet from a captured image of the paper sheet, and identifies the paper sheet based on the cut out feature image,
Imaging means for capturing an image of a region including the characteristic image of the paper sheet from the paper sheet inserted in either the front or back direction, forward or reverse,
Image matching means for performing template matching with a template image corresponding to each insertion direction of the paper sheets of the feature image on the image captured by the imaging means;
An insertion direction discriminating unit for discriminating an insertion direction of the paper sheet from a template image having the largest maximum correlation value by template matching performed by the image matching unit;
Based on the position on the captured image corresponding to the maximum correlation value by the image matching means using the template image corresponding to the insertion direction of the paper sheet determined by the insertion direction determining means, the paper sheet is taken from the captured image as a base point. A paper sheet identification apparatus comprising: an image cutout unit that cuts out a feature image of a kind. The feature image is
It is an image of the watermark area of the paper sheet,
The image matching means includes
The paper sheet according to claim 1, wherein template matching is performed between one captured image captured by the imaging unit and four template images obtained by converting one template image data into the front, back, and forward / reverse four directions. Kind identification device. The feature image is
It is an image of the watermark area of the paper sheet,
The image matching means includes
The template matching is performed between four images obtained by converting one captured image captured by the imaging unit into the front and back surfaces and four directions of forward and reverse and one template image corresponding to one template image data. Item 1. A paper sheet identification device according to Item 1. The image matching means includes
Find the center of gravity of the image in the watermark area,
According to the position of the center of gravity, it is determined whether the insertion direction of the paper sheet is a forward direction on the front side, a reverse direction on the back side, a reverse direction on the front side, or a forward direction on the back side
The paper sheet identification apparatus according to claim 2 or 3, wherein the template matching is selectively performed with respect to the two determined directions. A paper sheet identification method for cutting out a feature image of the paper sheet from a captured image of the paper sheet, and identifying the paper sheet based on the cut out feature image,
An image of an area including the feature image of the paper sheet is captured by the imaging unit from the paper sheet inserted in the front or back direction, either forward or reverse,
Perform template matching by the image matching unit with the template image corresponding to each insertion direction of the paper sheet of the feature image on the image captured by the imaging unit,
The insertion direction discriminating means discriminates the insertion direction of the paper sheets from the template image having the largest maximum correlation value calculated by template matching by the image matching means,
The position on the captured image corresponding to the template image having the maximum maximum correlation value by the image matching unit using the template image corresponding to the insertion direction of the paper sheet determined by the insertion direction determining unit is used as a base point. A paper sheet identification method characterized by cutting out a characteristic image of the paper sheet from a captured image. The feature image is
It is an image of the watermark area of the paper sheet,
The image matching means includes
The paper sheet according to claim 5, wherein template matching is performed between one captured image captured by the imaging unit and four template images obtained by converting one template image data into the front, back, and forward / reverse four directions. Class identification method. The feature image is
It is an image of the watermark area of the paper sheet,
The image matching means includes
The template matching is performed between four images obtained by converting one captured image captured by the imaging unit into the front and back surfaces and four directions of forward and reverse and one template image corresponding to one template image data. Item 6. The paper sheet identification method according to Item 5. The image matching means includes
Find the center of gravity of the image in the watermark area,
According to the position of the center of gravity, it is determined whether the insertion direction of the paper sheet is a forward direction on the front side, a reverse direction on the back side, a reverse direction on the front side, or a forward direction on the back side
The paper sheet identification method according to claim 6 or 7, wherein the template matching is selectively performed with respect to the two determined directions.

Images