EP3723054B1 - Unité de reconnaissance de billets de banque, dispositif de manipulation de billets de banque et procédé de reconnaissance de billets de banque - Google Patents

Unité de reconnaissance de billets de banque, dispositif de manipulation de billets de banque et procédé de reconnaissance de billets de banque Download PDF

Info

Publication number
EP3723054B1
EP3723054B1 EP20165227.8A EP20165227A EP3723054B1 EP 3723054 B1 EP3723054 B1 EP 3723054B1 EP 20165227 A EP20165227 A EP 20165227A EP 3723054 B1 EP3723054 B1 EP 3723054B1
Authority
EP
European Patent Office
Prior art keywords
infrared
data set
image
area
banknote
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
EP20165227.8A
Other languages
German (de)
English (en)
Other versions
EP3723054A1 (fr
Inventor
Toshihiko BANJOYA
Ryo Ikemoto
Chihiro NAGAI
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Glory Ltd
Original Assignee
Glory Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Glory Ltd filed Critical Glory Ltd
Publication of EP3723054A1 publication Critical patent/EP3723054A1/fr
Application granted granted Critical
Publication of EP3723054B1 publication Critical patent/EP3723054B1/fr
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G07—CHECKING-DEVICES
    • G07D—HANDLING OF COINS OR VALUABLE PAPERS, e.g. TESTING, SORTING BY DENOMINATIONS, COUNTING, DISPENSING, CHANGING OR DEPOSITING
    • G07D7/00—Testing specially adapted to determine the identity or genuineness of valuable papers or for segregating those which are unacceptable, e.g. banknotes that are alien to a currency
    • G07D7/06—Testing specially adapted to determine the identity or genuineness of valuable papers or for segregating those which are unacceptable, e.g. banknotes that are alien to a currency using wave or particle radiation
    • G07D7/12—Visible light, infrared or ultraviolet radiation
    • G—PHYSICS
    • G07—CHECKING-DEVICES
    • G07D—HANDLING OF COINS OR VALUABLE PAPERS, e.g. TESTING, SORTING BY DENOMINATIONS, COUNTING, DISPENSING, CHANGING OR DEPOSITING
    • G07D7/00—Testing specially adapted to determine the identity or genuineness of valuable papers or for segregating those which are unacceptable, e.g. banknotes that are alien to a currency
    • G07D7/181—Testing mechanical properties or condition, e.g. wear or tear
    • G07D7/187—Detecting defacement or contamination, e.g. dirt
    • G—PHYSICS
    • G07—CHECKING-DEVICES
    • G07D—HANDLING OF COINS OR VALUABLE PAPERS, e.g. TESTING, SORTING BY DENOMINATIONS, COUNTING, DISPENSING, CHANGING OR DEPOSITING
    • G07D7/00—Testing specially adapted to determine the identity or genuineness of valuable papers or for segregating those which are unacceptable, e.g. banknotes that are alien to a currency
    • G07D7/20—Testing patterns thereon
    • G07D7/2016—Testing patterns thereon using feature extraction, e.g. segmentation, edge detection or Hough-transformation

Definitions

  • the present invention relates to banknote recognition units, banknote handling devices, and banknote recognition methods.
  • the present invention specifically relates to a banknote recognition unit and a banknote recognition method suitable for determining the authenticity of a banknote under transport, and a banknote handling device including such a banknote recognition unit.
  • Current banknote handling devices including a banknote recognition unit execute processes such as recognition of the denominations of banknotes, authenticity determination, fitness determination, and serial number recognition during transport of banknotes.
  • a banknote is read by an optical line sensor and an image thereof is analyzed, so that processes are executed such as recognition of the denomination of the banknote, determination of the authenticity of the banknote, determination of the fitness of the banknote, and letter recognition of the serial number printed on the banknote.
  • JP H06-90750 B discloses a sheet soil recognition device configured to emit light to a sheet such as a banknote, to receive light reflected on or light transmitted through the sheet by a light receiving sensor, and to recognize the presence or absence of soil on the sheet based on an output signal from the light receiving sensor.
  • This device is configured to set any detection area for detecting soil on a sheet along a scanning line, to add a binarized signal of the output signal from the light receiving sensor in the detection area at each reference timing and add the output signal from the light receiving sensor in the detection area at each reference timing, and to recognize the sheet as a sheet without soil, a partially soiled sheet, or an entirely soiled sheet based on the results of these two additions.
  • JP 2006-202075 A discloses multiple banknote recognition methods for recognizing a banknote by comparing an output pattern of an electric signal output from a light receiving unit that receives light reflected on the banknote or light transmitted through the banknote with a reference output pattern that has been stored in advance.
  • the patent literature discloses a method of excluding an area where the output of an electric signal from the light receiving unit is greater than the predetermined value is excluded from the comparison target for recognizing a banknote.
  • the patent literature also discloses a method for correcting the reference output pattern in accordance with the output of an electric signal from the light receiving unit.
  • US 2018/0247308 A1 discloses systems and methods for validation of a media object may include receiving the media object and detecting, using a sensor, a damaged portion of the media object.
  • the media object may be validated against a standard.
  • the damaged portion of the media object may be given less weight.
  • infrared non-absorbing ink may be used for printing of banknotes in some cases.
  • a portion printed with infrared non-absorbing ink absorbs visible light and does not absorb infrared light.
  • the pattern printed with this ink is visually observable under a visible light source, while the pattern cannot be imaged even with an optical line sensor under an infrared light source and the banknote is imaged with the pattern disappeared.
  • the portion printed with infrared non-absorbing ink has a relatively high output (hereinafter, also referred to as a reflected infrared output) from the optical line sensor based on the light generated by reflection of infrared light on a banknote and has a relatively low output (hereinafter, also referred to as a transmitted infrared output) from the optical line sensor based on the light generated by transmission of infrared light through the banknote.
  • a counterfeit portion printed not with infrared non-absorbing ink but with infrared absorbing ink absorbs infrared light.
  • the reflected infrared output and the transmitted infrared output are both low.
  • a portion having a low reflected infrared output relative to a reference template prepared in advance by learning from a genuine note can be determined as a suspect counterfeit portion.
  • This determination may be specifically achieved as follows, for example. First, a reflected infrared output (infrared reflection image) is collected for the entire printed area of a banknote to be determined. The collected reflected infrared output is then compared with the reference template. Pixels each having an output outside a threshold tolerance are defined as counterfeit pixels, and the counterfeit pixels are counted. The number of the counterfeit pixels is then compared with a separately set threshold, whereby the authenticity determination is performed.
  • the above determination method may misdetermine a medium as a counterfeit note in a case that the medium has an oil-stained portion that is difficult to observe visually but has an influence on the reflected infrared output and the transmitted infrared output.
  • JP H06-90750 B is configured to recognize the presence or absence of soil on a sheet. This literature fails to disclose both the issue of misdetermination of a soiled medium as a counterfeit note and the solution thereto.
  • the comparison target for recognizing a banknote is limited based on the output value of an electric signal by reflective light or transmissive light.
  • a counterfeit portion may be highly possibly excluded from the comparison target.
  • a banknote having multiple locally soiled portions may be misdetermined as a counterfeit note.
  • an object of the present invention is to provide a banknote recognition unit capable of reducing misdetermination of a soiled note as a counterfeit note, a banknote handling device, and a banknote recognition method.
  • one aspect of the present invention is a banknote recognition unit according to claim 1.
  • Another aspect of the present invention is directed to a banknote handling device including the above banknote recognition unit.
  • Another aspect of the present invention is directed to a banknote recognition method for recognizing a banknote, according to claim 3.
  • the banknote recognition unit, the banknote handling device, and the banknote recognition method of the present invention can reduce misdetermination of a soiled note as a counterfeit note.
  • the reflection image means an image based on the intensity distribution of light generated by reflection of light on a banknote.
  • the reflection image is also referred to as an infrared reflection image.
  • the transmission image means an image based on the intensity distribution of light generated by transmission of light through a banknote.
  • the transmission image is also referred to as an infrared transmission image.
  • a banknote recognition unit and a banknote recognition method of the present embodiment are first described with reference to Fig. 1 .
  • Main features of the present embodiment are the following features I and II.
  • banknote handling device 300 illustrated in Fig. 2 is a small banknote handling device to be used on a table, and includes a banknote recognition unit (not illustrated in Fig.
  • a hopper 301 configured to hold a plurality of banknotes to be processed in the form of a stack
  • two rejection units 302 configured to dispense rejected banknotes, such as counterfeit notes and suspect notes, among the banknotes fed from the hopper 301 into a housing 310
  • an operation unit 303 configured to receive instructions from an operator
  • four stacking units 306a to 306d configured to stack sorted banknotes whose denomination, authenticity, and fitness are recognized in the housing 310
  • a display 305 configured to display information such as the results of recognizing and counting banknotes and the stacking conditions of the stacking units 306a to 306d.
  • the stacking units 306a to 306c among the four stacking units 306a to 306d store fit notes, while the stacking unit 306d stores soiled notes. Any method may be used to sort banknotes into the stacking units 306a to 306d.
  • a banknote recognition unit 1 of the present embodiment includes a control unit 10, a detection unit 20, and a storage unit 30.
  • the control unit 10 includes programs for achieving a variety of processing stored in the storage unit 30, a central processing unit (CPU) that executes the programs, a variety of hardware controlled by the CPU, and a logical device such as a field programmable gate array (FPGA).
  • the control unit 10 controls the components of the banknote recognition unit 1 based on signals output from the components of the banknote recognition unit 1 and a control signal from the control unit 10 in accordance with the programs stored in the storage unit 30.
  • the control unit 10 also has functions as a recognition unit 11, an authenticity determination unit 12, and a fitness determination unit 13 owing to the programs stored in the storage unit 30.
  • the detection unit 20 includes an image acquisition unit 21, a magnetism detection unit 22, and a UV detection unit 23.
  • the image acquisition unit 21 acquires an image of a banknote.
  • the magnetism detection unit 22 includes a magnetism sensor (not illustrated) to detect the magnetism and detects the magnetism of magnetic ink printed on a banknote and a security thread, for example, with the magnetism sensor.
  • the magnetism sensor is a magnetism line sensor of a plurality of magnetism detection elements arranged in line.
  • the UV detection unit 23 includes a ultraviolet light emitting unit (not illustrated) and a light receiving unit (not illustrated) and detects, with the light receiving unit, fluorescence generated when the ultraviolet light emitting unit emits ultraviolet light to a banknote and ultraviolet light passed through the banknote.
  • the storage unit 30 includes a nonvolatile storage unit such as a semiconductor memory or a hard disk, and stores a variety of programs and a variety of data for controlling the banknote recognition unit 1.
  • the storage unit 30 also stores a reference data set for an infrared reflection image, a reference data set for an infrared transmission image, and a reference value (hereinafter, also referred to as a size reference value) of the size of an abnormal area.
  • the reference data set for an infrared reflection image and the reference data set for an infrared transmission image are reference templates to be used as determination references in authenticity determination processing by the authenticity determination unit 12, and are each a parameter group prepared for each denomination of a banknote to be recognized.
  • Each parameter group is prepared by machine learning in advance based on the pixel value (output value) of each pixel of the infrared reflection images and the infrared transmission images of fit banknotes (genuine notes).
  • the reference data set for an infrared reflection image and the reference data set for an infrared transmission image each indicate a reference value of the pixel value of each pixel of the infrared reflection image or the infrared transmission image of a banknote to be subjected to authenticity determination.
  • the size reference value is a threshold to be used as a determination reference of the size of an abnormal area in the authenticity processing by the authenticity determination unit 12.
  • the recognition unit 11 compares a characteristic pattern of an image of a banknote acquired by the image acquisition unit 21 with a denomination recognition template that is a characteristic pattern for each denomination of a banknote and determines the denomination of the banknote.
  • the authenticity determination unit 12 determines the authenticity of a banknote based on an image of the banknote obtained by infrared irradiation. The specification thereof is described later.
  • the authenticity determination unit 12 also matches a detection signal from the magnetism detection unit 22 and a detection signal from the UV detection unit 23 to a respective reference data set for authenticity recognition, and determines the authenticity of the banknote.
  • the authenticity determination unit 12 executes a plurality of authenticity determination processes on a banknote and determines the banknote as a genuine note in a case that all determination results demonstrate that the banknote is genuine, while it determines the banknote as a counterfeit note in a case that any one of the determination results demonstrates that the banknote is counterfeit.
  • the image acquisition unit 21 includes optical line sensors 110 and 120 arranged to face each other. Between the optical line sensors 110 and 120 is formed a gap through which a banknote BN is to be transported. This gap is a portion of a transport path 311 of the banknote handling device. The optical line sensors 110 and 120 are respectively placed on the upper and lower sides of the transport path 311.
  • the optical line sensor 110 includes a reflection light source 111, a condenser 112, and a light receiving unit 113.
  • the reflection light source 111 emits light at predetermined wavelengths (invisible light such as infrared light and visible light such as single-color light of red, green, blue, or the like, or white light) to the upper surface of a banknote BN.
  • the condenser 112 gathers light emitted from the reflection light source 111 and reflected on the banknote BN.
  • the light receiving unit 113 includes a plurality of light receiving elements (not illustrated) arranged in line along the direction (main scanning direction) perpendicular to the direction (subscanning direction) of transporting a banknote BN, and converts the light gathered by the condenser 112 into an electric signal.
  • the optical line sensor 110 converts the electric signal generated by conversion by the light receiving unit 113 into a digital signal and outputs this digital signal.
  • the optical line sensor 120 includes a reflection light source 121, a condenser 122, a light receiving unit 123, and a transmission light source 124.
  • the reflection light source 121 and the transmission light source 124 each emit light at predetermined wavelengths (invisible light such as infrared light and visible light such as single-color light of red, green, blue, or the like, or white light) to the lower surface of a banknote BN.
  • the condenser 122 gathers light emitted from the reflection light source 121 and reflected on the banknote BN.
  • the light receiving unit 123 includes a plurality of light receiving elements (not illustrated) arranged in line along the direction perpendicular to the direction of transporting a banknote BN, and converts the light gathered by the condenser 122 into an electric signal.
  • the optical line sensor 120 converts the electric signal generated by conversion by the light receiving unit 123 into a digital signal and outputs this digital signal.
  • the transmission light source 124 is placed on the optical axis of the condenser 112 of the optical line sensor 110. Part of the light emitted from the transmission light source 124 passes through the banknote BN, gathered by the condenser 112 of the optical line sensor 110, and detected by the light receiving unit 113.
  • the optical line sensors 110 and 120 each repetitively image a banknote BN (expose the light receiving elements to light) under transport in the transport direction at predetermined time intervals and output a signal.
  • the image acquisition unit 21 can acquire an image of the whole banknote BN (image acquisition step). Specifically, the image acquisition unit 21 acquires a reflection image of the upper surface of the banknote BN and a transmission image of the banknote BN based on the output signal from the optical line sensor 110 and acquires a reflection image of the lower surface of the banknote BN based on the output signal from the optical line sensor 120.
  • the image acquisition unit 21 acquires as reflection images of the banknote BN an infrared reflection image of the upper surface of the banknote BN and an infrared reflection image of the lower surface of the banknote BN.
  • the image acquisition unit 21 further acquires as a transmission image of the banknote BN an infrared transmission image of the banknote BN.
  • the authenticity determination unit 12 includes a first determination unit 12a, a second determination unit 12b, and a main determination unit 12c.
  • the first determination unit 12a executes a first determination processing (first determination step) of comparing an infrared reflection image 210 as a first infrared image of a banknote (the whole printed area) with a reference data set for an infrared reflection image as a first reference data set (a reference data set for an infrared reflection image corresponding to the denomination recognized by the recognition unit 11) to determine whether the infrared reflection image 210 includes an abnormal area 211 that is outside a tolerance relative to the reference data set for an infrared reflection image. This enables detection of a soiled portion due to oil stain, a locally soiled portion, and/or a counterfeit portion as an abnormal area 211.
  • the first determination unit 12a compares the pixel value of each pixel of the infrared reflection image 210 with the respective reference value included in the reference data set for an infrared reflection image and, in a case that a pixel having a pixel value smaller than the reference value is present, determines the area including such a pixel as an abnormal area 211.
  • the second determination unit 12b executes, on an infrared transmission image 220 as a second infrared image of the banknote (the whole printed area), a second determination processing (second determination step) of comparing each of one or more determination target areas 221 corresponding to the one or more abnormal areas 211 with a reference data set for an infrared transmission image as a second reference data set (a reference data set for an infrared transmission image corresponding to the denomination recognized by the recognition unit 11) to determine whether each determination target area 221 is within a tolerance relative to the reference data set for an infrared transmission image.
  • This enables determination of whether each of the abnormal areas 211 and the determination target areas 221 is a soiled portion due to oil stain.
  • the second determination unit 12b determines one or more areas in the infrared transmission image 220 corresponding to the one or more abnormal areas 211 (i.e., one or more areas at the same positions as the one or more abnormal areas 211) as the determination target areas 221.
  • the second determination unit 12b then compares the pixel value of each pixel of each determination target area 221 with the respective reference value included in the reference data set for an infrared transmission image to determine that the determination target area 221 is within the tolerance relative to the reference data set for an infrared transmission image in a case that the pixel value of each pixel is greater than the respective reference value, while it determines that the determination target area 221 is not within the tolerance relative to the reference data set for an infrared transmission image in a case that the pixel value of each pixel is not higher than the respective reference value.
  • the main determination unit 12c executes authenticity determination processing (main determination step) on the infrared reflection image 210 at least excluding one or more abnormal areas 211 corresponding to the one or more determination target areas 221 determined as being within the tolerance. This enables authenticity determination on an area excluding a soiled portion due to oil stain in a case that the abnormal area 211 is such a portion, capable of reducing misdetermination of a soiled note due to oil stain as a counterfeit note.
  • the main determination unit 12c defines an area at least excluding one or more abnormal areas 211 in the infrared reflection image 210 corresponding to the one or more determination target areas 221 determined as being within the tolerance (i.e., one or more areas at the same positions as the one or more determination target areas 221) as an authenticity determination area to be processed in authenticity determination processing.
  • the main determination unit 12c then executes authenticity determination processing on the authenticity determination area by a common method. For example, in an example illustrated in Figs.
  • the abnormal area 211 on the right side of the infrared reflection image 210 is excluded from the authenticity determination area because the pixel value of each pixel of the corresponding determination target area 221 on the right side of the infrared transmission image 220 is greater than the respective reference value.
  • the main determination unit 12c executes the following processing. Specifically, the main determination unit 12c compares the size of each of one or more abnormal areas 211 corresponding to the one or more determination target areas 221 determined as being not within the tolerance with the size reference value stored in the storage unit 30. When the size of an abnormal area 211 is smaller than the size reference value, the main determination unit 12c executes authenticity determination processing (second main determination step) on the infrared reflection image 210 at least excluding this abnormal area 211. This enables authenticity determination on an area excluding a locally soiled portion having a small size in a case that the abnormal area 211 is such a portion, capable of reducing misdetermination of a soiled note due to local soil as a counterfeit note.
  • the main determination unit 12c specifies one or more abnormal areas 211 corresponding to the one or more determination target areas 221 determined as being not within the tolerance (i.e., one or more areas at the same portions as the one or more determination target areas 221) and calculates the size of each area.
  • the main determination unit 12c compares the calculated size of each abnormal area 211 with the size reference value stored in the storage unit 30.
  • the main determination unit 12c determines the area excluding this abnormal area 211 (the abnormal area having a size smaller than the size reference value) in the infrared reflection image 210 as the authenticity determination area.
  • the main determination unit 12c executes authenticity determination processing on the authenticity determination area by a common method. For example, in an example illustrated in Figs. 5 and 6 , the pixel value of each pixel of the determination target areas 221 at the center and on the left side of the infrared transmission image 220 is not higher than the respective reference value, but the size of each corresponding abnormal area 211 on the left side in the infrared reflection image 210 is smaller than the size reference value.
  • these abnormal areas 211 are excluded from the authenticity determination area.
  • the size of the corresponding abnormal area 211 at the center in the infrared reflection image 210 is not smaller than the size reference value.
  • this abnormal area 211 is not excluded from the authenticity determination area.
  • Calculation of the size of each abnormal area 211 by the second determination unit 12b may be performed by a method in which the infrared reflection image 210 is binarized, followed by filtering or labelling, and the number of pixels constituting each abnormal area 211 is calculated, for example.
  • the infrared reflection image 210 is first binarized and the binarized banknote area is subjected to black-white inversion. Thereby, as illustrated in the upper case of Fig. 7 , portions having a low infrared reflection output, i.e., suspected as being counterfeit, become white pixels. Then, a predetermined filtering operation is performed. The pixel values of the respective pixels are compared with a predetermined threshold. Pixels exceeding the threshold are determined as counterfeit pixels and the number thereof is counted. Provided that the threshold is set to 3, for example, the four pixels at the center in the left case of Fig. 7 are within the tolerance of the threshold and no counterfeit pixel is detected. In contrast, the four pixels at the center in the right case of Fig. 7 are outside the tolerance of the threshold and detected as counterfeit pixels.
  • the processing by the banknote recognition unit 1 is described with reference to Fig. 8 . This processing is performed on banknotes fed into the banknote recognition unit 1 one by one in a repetitive manner.
  • the recognition unit 11 recognizes, as described above, the denomination of the banknote based on images of the upper and lower surfaces of the banknote acquired by the image acquisition unit 21 and a template for denomination recognition (S12).
  • the first determination unit 12a executes a first determination processing of comparing the infrared reflection image (first infrared image) 210 of the banknote with a reference data set for an infrared reflection image (first reference data set) corresponding to the denomination recognized in Step S12 to determine whether the infrared reflection image 210 includes an abnormal area 211 that is outside a tolerance relative to the reference data set for an infrared reflection image (S13).
  • the second determination unit 12b executes a second determination processing of comparing one or more determination target areas 221 in the infrared transmission image (second infrared image) 220 of the banknote corresponding to the one or more abnormal areas 211 with a reference data set for an infrared transmission image (second reference data set) corresponding to the denomination recognized in Step S12 to determine whether each determination target area 221 is within a tolerance relative to the reference data set for an infrared transmission image (S14).
  • the main determination unit 12c excludes, as described above, one or more abnormal areas 211 corresponding to the one or more determination target areas 221 determined as being within the tolerance from the authenticity determination area of the infrared reflection image 210 (S15).
  • the size of each of one or more abnormal areas 211 corresponding to the one or more determination target areas 221 determined as being not within the tolerance is compared with the size reference value.
  • this abnormal area 211 is also excluded from the authenticity determination area of the infrared reflection image 210 (S15).
  • the main determination unit 12c executes authenticity determination processing on the authenticity determination area defined in Step S15 to determine the authenticity of the banknote (S16) .
  • the processing relating to the authenticity determination through Steps S13 to S16 is performed on both the upper surface and the lower surface of the banknote.
  • the aforementioned authenticity determination processing based on the infrared reflection image 210 of the upper surface of the banknote and the infrared transmission image 220 of the banknote is performed and the aforementioned authenticity determination processing based on the infrared reflection image 210 of the lower surface of the banknote and the infrared transmission image 220 of the banknote is performed.
  • the authenticity determination unit 12 matches each of the detection signal from the magnetism detection unit 22 and the detection signal from the UV detection unit 23 to the respective reference data set for authenticity recognition to determine the authenticity of the banknote (S17) .
  • Steps S16 and S17 each demonstrate that the banknote is genuine (S18: YES)
  • the authenticity determination unit 12 determines the banknote as a genuine note (S19).
  • the authenticity determination unit 12 determines the banknote as a counterfeit note (S20).
  • the fitness determination unit 13 determines the fitness of the banknote (S21).
  • control unit 10 feeds the banknote after the authenticity determination and the fitness determination out of the banknote recognition unit 1 (S22), completing the processing by the banknote recognition unit 1.
  • the first determination unit 12a detects, as an abnormal area 211, a portion suspected as being counterfeit based on the infrared reflection image 210 of a banknote; the second determination unit 12b determines whether the determination target area 221 corresponding to the abnormal area 211 is a soiled portion due to oil stain based on the infrared transmission image 220; and in a case that the determination target area 221 is determined as a soiled portion due to oil stain, the main determination unit 12c executes authenticity determination processing on an authenticity determination area at least excluding the abnormal area 211 in the infrared reflection image 210. This can reduce misdetermination of a soiled note due to oil stain as a counterfeit note.
  • the main determination unit 12c determines whether the abnormal area 211 is a locally soiled portion based on the size of the abnormal area 211 corresponding to the determination target area 221.
  • the main determination unit 12c executes authenticity determination processing with the abnormal area 211 also being excluded from the authenticity determination area. This can reduce misdetermination of a soiled note due to local soil as a counterfeit note.
  • the determination is performed first on the infrared reflection image 210 and then on the infrared transmission image 220, i.e., the case where the first infrared image is the infrared reflection image 210, the first determination unit 12a compares the infrared reflection image 210 with the reference data set for an infrared reflection image, the second infrared image is the infrared transmission image 220, and the second determination unit 12b compares the determination target area 221 of the infrared transmission image 220 with the reference data set for an infrared transmission image.
  • the determination may be performed first on the infrared transmission image 220 and then on the infrared reflection image 210.
  • the first infrared image may be the infrared transmission image 220
  • the first determination unit 12a may compare the infrared transmission image 220 with the reference data set for an infrared transmission image
  • the second infrared image may be the infrared reflection image 210
  • the second determination unit 12b may compare the determination target area of the infrared reflection image 210 with the reference data set for an infrared reflection image.
  • the present invention provides a technique useful for reducing misdetermination of determining a soiled note as a counterfeit note using a banknote recognition unit and in a banknote recognition method.

Landscapes

  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Toxicology (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Inspection Of Paper Currency And Valuable Securities (AREA)

Claims (3)

  1. Unité de reconnaissance de billets de banque (1) pour reconnaître un billet de banque, comprenant :
    une unité d'acquisition d'image (21) configurée pour acquérir une image de réflexion infrarouge (210) et une image de transmission infrarouge (220) d'un billet de banque (BN) ;
    une unité de stockage (30) configurée pour stocker un ensemble de données de référence pour une image de réflexion infrarouge et un ensemble de données de référence pour une image de transmission infrarouge chacun basé sur un billet de banque adapté, l'ensemble de données de référence pour une image de réflexion infrarouge comprenant une valeur de référence d'une valeur de pixel de chaque pixel de l'image de réflexion infrarouge, et l'ensemble de données de référence pour une image de transmission infrarouge comprenant une valeur de référence d'une valeur de pixel de chaque pixel de l'image de transmission infrarouge,
    une première unité de détermination (12a) configurée pour comparer l'image de réflexion infrarouge (210) acquise par l'unité d'acquisition d'image avec l'ensemble de données de référence pour une image de réflexion infrarouge afin de déterminer si l'image de réflexion infrarouge (210) comprend une zone anormale (211) qui est en dehors d'une tolérance par rapport à l'ensemble de données de référence pour une image de réflexion infrarouge ;
    une deuxième unité de détermination (12b) configurée pour comparer, dans un cas où l'image de réflexion infrarouge (210) comprend la zone anormale (211), une zone cible de détermination (221) de l'image de transmission infrarouge (220) avec l'ensemble de données de référence pour une image de transmission infrarouge pour déterminer si la zone cible de détermination (221) est dans une tolérance par rapport à l'ensemble de données de référence pour une image de transmission infrarouge, la zone cible de détermination (221) étant une zone correspondant à la zone anormale (211) ; et
    une unité de détermination principale (12c) configurée pour exécuter un traitement de détermination d'authenticité sur une zone excluant au moins la zone anormale (211) de l'image de réflexion infrarouge (210) dans un cas où la deuxième unité de détermination (12b) détermine que la zone cible de détermination (221) est dans la tolérance par rapport à l'ensemble de données de référence pour une image de transmission infrarouge,
    dans laquelle la première unité de détermination (12a) est configurée pour déterminer, en tant que zone anormale (211), une zone comprenant des pixels ayant chacun une valeur de pixel inférieure à la valeur de référence respective qui est incluse dans l'ensemble de données de référence pour une image de réflexion infrarouge dans l'image de réflexion infrarouge,
    dans un cas où l'image de réflexion infrarouge (210) comprend la zone anormale (211), la deuxième unité de détermination (12b) est configurée pour déterminer premièrement une zone dans l'image de transmission infrarouge (220) correspondant à la zone anormale (211) en tant que zone cible de détermination (221) et ensuite pour comparer la zone cible de détermination (221) avec l'ensemble de données de référence pour une image de transmission infrarouge afin de déterminer si la zone cible de détermination (221) est dans une tolérance par rapport à l'ensemble de données de référence pour une image de transmission infrarouge, où la deuxième unité de détermination (12b) est configurée pour déterminer que la zone cible de détermination (221) est dans la tolérance par rapport à l'ensemble de données de référence pour une image de transmission infrarouge dans un cas où la valeur de pixel de chaque pixel de la zone cible de détermination (221) est supérieure à la valeur de référence respective qui est incluse dans l'ensemble de données de référence pour une image de transmission infrarouge, et pour déterminer que la zone cible de détermination (221) n'est pas dans la tolérance par rapport à l'ensemble de données de référence pour une image de transmission infrarouge dans un cas où la valeur de pixel de chaque pixel de la zone cible de détermination (221) n'est pas supérieure à la valeur de référence respective qui est incluse dans l'ensemble de données de référence pour une image de transmission infrarouge,
    l'unité de stockage (30) stocke en outre une valeur de référence d'une taille de la zone anormale (211), et
    dans un cas où l'image de réflexion infrarouge (210) comprend la zone anormale (221) et la deuxième unité de détermination (12b) détermine que la valeur de pixel de chaque pixel de la zone cible de détermination (221) n'est pas supérieure à la valeur de référence respective qui est incluse dans l'ensemble de données de référence pour une image de transmission infrarouge, l'unité de détermination principale (12c) est configurée pour comparer la taille de la zone anormale (211) avec la valeur de référence stockée dans l'unité de stockage et pour exécuter, dans un cas où la taille de la zone anormale (211) est inférieure à la valeur de référence, un traitement de détermination d'authenticité sur une zone au moins excluant la zone anormale (211) de l'image de réflexion infrarouge (210).
  2. Dispositif de manipulation de billets de banque comprenant l'unité de reconnaissance de billets de banque selon la revendication 1.
  3. Procédé de reconnaissance de billets de banque pour reconnaître un billet de banque, comprenant :
    une étape d'acquisition d'image (S12) consistant à acquérir une image de réflexion infrarouge (210) et une image de transmission infrarouge (220) d'un billet de banque (BN) ;
    une première étape de détermination (S13) consistant à comparer l'image de réflexion infrarouge (210) avec un ensemble de données de référence pour une image de réflexion infrarouge afin de déterminer si l'image de réflexion infrarouge (210) comprend une zone anormale (211) qui est en dehors d'une tolérance par rapport à l'ensemble de données de référence pour une image de réflexion infrarouge, l'ensemble de données de référence pour une image de réflexion infrarouge comprenant une valeur de référence d'une valeur de pixel de chaque pixel de l'image de réflexion infrarouge et basée sur un billet de banque adapté ;
    une deuxième étape de détermination (S14) consistant à comparer, dans un cas où l'image de réflexion infrarouge (210) comprend la zone anormale (211), une zone cible de détermination (221) de l'image de transmission infrarouge (220) avec l'ensemble de données de référence pour une image de transmission infrarouge pour déterminer si la zone cible de détermination (221) est dans une tolérance par rapport à l'ensemble de données de référence pour une image de transmission infrarouge, la zone cible de détermination (221) étant une zone correspondant à la zone anormale (211), l'ensemble de données de référence pour une image de transmission infrarouge comprenant une valeur de référence d'une valeur de pixel de chaque pixel de l'image de transmission infrarouge et basée sur un billet de banque adapté ; et
    une étape de détermination principale (S16-S21) consistant à exécuter un traitement de détermination d'authenticité sur une zone excluant au moins la zone anormale (211) de l'image de réflexion infrarouge (210) dans un cas où la zone cible de détermination (221) est déterminée comme étant à l'intérieur de la tolérance par rapport à l'ensemble de données de référence pour une image de transmission infrarouge dans la deuxième étape de détermination (S14),
    dans laquelle la première étape de détermination (S13) comprend la détermination, en tant que zone anormale (211), d'une zone comprenant des pixels ayant chacun une valeur de pixel inférieure à la valeur de référence respective qui est incluse dans l'ensemble de données de référence pour une image de réflexion infrarouge dans l'image de réflexion infrarouge,
    dans un cas où l'image de réflexion infrarouge (210) comprend la zone anormale (211), la deuxième étape de détermination (S14) comprend premièrement la détermination d'une zone dans l'image de transmission infrarouge (220) correspondant à la zone anormale (211) en tant que zone cible de détermination (221), ensuite la comparaison de la zone cible de détermination (221) avec l'ensemble de données de référence pour une image de transmission infrarouge pour déterminer si la zone cible de détermination (221) est dans une tolérance par rapport à l'ensemble de données de référence pour une image de transmission infrarouge, où la deuxième étape de détermination (S14) détermine en ce que la zone cible de détermination (221) est dans la tolérance par rapport à l'ensemble de données de référence pour une image de transmission infrarouge dans un cas où la valeur de pixel de chaque pixel de la zone cible de détermination (221) est supérieure à la valeur de référence respective qui est incluse dans l'ensemble de données de référence pour une image de transmission infrarouge, et détermine que la zone cible de détermination (221) n'est pas dans la tolérance par rapport à l'ensemble de données de référence pour une image de transmission infrarouge dans un cas où la valeur de pixel de chaque pixel de la zone cible de détermination (221) n'est pas supérieure à la valeur de référence respective qui est incluse dans l'ensemble de données de référence pour une image de transmission infrarouge,
    dans un cas où l'image de réflexion infrarouge (210) comprend la zone anormale (221) et la deuxième étape de détermination (S14) détermine que la valeur de pixel de chaque pixel de la zone cible de détermination (221) n'est pas supérieure à la valeur de référence respective qui est incluse dans l'ensemble de données de référence pour une image de transmission infrarouge, l'étape de détermination principale (S16-S21) comprend la comparaison d'une taille de la zone anormale (211) avec une valeur de référence de la taille de la zone anormale et l'exécution, dans un cas où la taille de la zone anormale (211) est inférieure à la valeur de référence, d'un traitement de détermination d'authenticité sur une zone excluant au moins la zone anormale (211) de l'image de réflexion infrarouge (210).
EP20165227.8A 2019-03-27 2020-03-24 Unité de reconnaissance de billets de banque, dispositif de manipulation de billets de banque et procédé de reconnaissance de billets de banque Active EP3723054B1 (fr)

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
JP2019060934A JP7227818B2 (ja) 2019-03-27 2019-03-27 紙幣識別装置、紙幣処理装置及び紙幣識別方法

Publications (2)

Publication Number Publication Date
EP3723054A1 EP3723054A1 (fr) 2020-10-14
EP3723054B1 true EP3723054B1 (fr) 2022-07-13

Family

ID=69960411

Family Applications (1)

Application Number Title Priority Date Filing Date
EP20165227.8A Active EP3723054B1 (fr) 2019-03-27 2020-03-24 Unité de reconnaissance de billets de banque, dispositif de manipulation de billets de banque et procédé de reconnaissance de billets de banque

Country Status (3)

Country Link
EP (1) EP3723054B1 (fr)
JP (1) JP7227818B2 (fr)
CN (1) CN111754675A (fr)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2022110367A (ja) * 2021-01-18 2022-07-29 株式会社リコー 不正確認補助装置および不正確認方法
JP2023135883A (ja) * 2022-03-16 2023-09-29 グローリー株式会社 紙葉類識別装置、紙葉類処理装置及び紙葉類識別方法
CN117314897B (zh) * 2023-11-28 2024-03-19 恒银金融科技股份有限公司 纸币图像的失真判别方法和装置

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH0690750B2 (ja) 1987-01-26 1994-11-14 沖電気工業株式会社 紙葉類汚損判別装置
JP2006202075A (ja) 2005-01-21 2006-08-03 Mars Engineering Corp 紙幣識別装置および紙幣識別方法
JP2011159073A (ja) * 2010-01-29 2011-08-18 Glory Ltd 紙葉類識別装置および紙葉類識別方法
CN105096446B (zh) * 2015-07-20 2017-12-05 深圳怡化电脑股份有限公司 一种折角钞光变油墨鉴伪识别的方法及系统
US10810589B2 (en) * 2017-02-27 2020-10-20 Ncr Corporation Validation of damaged banknotes
AU2017403990A1 (en) * 2017-03-15 2019-10-31 Glory Ltd. Sheet detection device, sheet processing device, and sheet detection method

Also Published As

Publication number Publication date
EP3723054A1 (fr) 2020-10-14
JP7227818B2 (ja) 2023-02-22
CN111754675A (zh) 2020-10-09
JP2020160908A (ja) 2020-10-01

Similar Documents

Publication Publication Date Title
EP3723054B1 (fr) Unité de reconnaissance de billets de banque, dispositif de manipulation de billets de banque et procédé de reconnaissance de billets de banque
US9690841B2 (en) Method and apparatus for determining a class reference data record for the classification of value documents
JP2001126107A (ja) 紙葉類の識別方法および装置
US8989433B2 (en) Paper sheet recognition apparatus and paper sheet recognition method
EP3598401B1 (fr) Dispositif de détection de feuilles de papier, appareil de traitement de feuilles de papier et procédé de détection de feuilles de papier
US9460345B2 (en) Apparatus and method for recognizing media, financial device
US20190066428A1 (en) Invisible-feature detection device, sheet recognition device, sheet handling device, print inspection device, and invisible-feature detection method
JP5502111B2 (ja) 紙葉類識別装置および紙葉類識別方法
JPWO2019202715A1 (ja) 紙葉類処理装置及び紙葉類処理方法
EP3680867B1 (fr) Dispositif d'acquisition d'images, dispositif de manipulation de feuilles, dispositif de manipulation de billets de banque et procédé d'acquisition d'images
CN109074696B (zh) 用于检查有价文件特别是钞票的装置和方法及有价文件处理系统
CA2920541C (fr) Procede de verification d'un document de valeur ayant un substrat polymere et une fenetre transparente, et moyen servant a mettre en oeuvre le procede
EP2355056B1 (fr) Appareil de reconnaissance de feuilles de papier et procédé de reconnaissance de feuilles de papier
US9685020B2 (en) Apparatus and method for checking value documents
KR20120069841A (ko) 컨택 이미지 센서 모듈을 이용한 지폐 식별 장치
EP4317040A1 (fr) Dispositif de détection d'alimentation multiple et procédé de détection d'alimentation multiple
JP7508400B2 (ja) 重送検知装置及び重送検知方法
JP2021012587A (ja) 記番号読取装置、紙葉類処理装置、及び記番号読取方法
JP2020191014A (ja) 紙葉類処理装置、紙葉類処理方法及び紙葉類処理プログラム
KR20090008708A (ko) 컨택 이미지 센서를 이용한 위폐 식별방법

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE

17P Request for examination filed

Effective date: 20200324

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

AX Request for extension of the european patent

Extension state: BA ME

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: EXAMINATION IS IN PROGRESS

17Q First examination report despatched

Effective date: 20210127

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: GRANT OF PATENT IS INTENDED

INTG Intention to grant announced

Effective date: 20220221

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE PATENT HAS BEEN GRANTED

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 602020003946

Country of ref document: DE

REG Reference to a national code

Ref country code: AT

Ref legal event code: REF

Ref document number: 1504671

Country of ref document: AT

Kind code of ref document: T

Effective date: 20220815

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: LT

Ref legal event code: MG9D

REG Reference to a national code

Ref country code: NL

Ref legal event code: MP

Effective date: 20220713

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220713

Ref country code: RS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220713

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20221114

Ref country code: NO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20221013

Ref country code: NL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220713

Ref country code: LV

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220713

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220713

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220713

Ref country code: ES

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220713

REG Reference to a national code

Ref country code: AT

Ref legal event code: MK05

Ref document number: 1504671

Country of ref document: AT

Kind code of ref document: T

Effective date: 20220713

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220713

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20221113

Ref country code: HR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220713

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20221014

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 602020003946

Country of ref document: DE

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SM

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220713

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220713

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220713

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220713

Ref country code: AT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220713

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220713

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220713

26N No opposition filed

Effective date: 20230414

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: AL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220713

P01 Opt-out of the competence of the unified patent court (upc) registered

Effective date: 20230531

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220713

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MC

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220713

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

REG Reference to a national code

Ref country code: BE

Ref legal event code: MM

Effective date: 20230331

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20230324

REG Reference to a national code

Ref country code: IE

Ref legal event code: MM4A

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20230331

Ref country code: IT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220713

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20230324

Ref country code: FR

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20230331

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20230331

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20230331

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: DE

Payment date: 20240320

Year of fee payment: 5

Ref country code: GB

Payment date: 20240321

Year of fee payment: 5