EP3680867A1 - Dispositif d'acquisition d'images, dispositif de manipulation de feuilles, dispositif de manipulation de billets de banque et procédé d'acquisition d'images - Google Patents

Dispositif d'acquisition d'images, dispositif de manipulation de feuilles, dispositif de manipulation de billets de banque et procédé d'acquisition d'images Download PDF

Info

Publication number
EP3680867A1
EP3680867A1 EP20150651.6A EP20150651A EP3680867A1 EP 3680867 A1 EP3680867 A1 EP 3680867A1 EP 20150651 A EP20150651 A EP 20150651A EP 3680867 A1 EP3680867 A1 EP 3680867A1
Authority
EP
European Patent Office
Prior art keywords
light
image
banknote
emitted
transmission
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
EP20150651.6A
Other languages
German (de)
English (en)
Other versions
EP3680867B1 (fr
Inventor
Akira BOGAKI
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Glory Ltd
Original Assignee
Glory Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Glory Ltd filed Critical Glory Ltd
Publication of EP3680867A1 publication Critical patent/EP3680867A1/fr
Application granted granted Critical
Publication of EP3680867B1 publication Critical patent/EP3680867B1/fr
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G07—CHECKING-DEVICES
    • G07D—HANDLING OF COINS OR VALUABLE PAPERS, e.g. TESTING, SORTING BY DENOMINATIONS, COUNTING, DISPENSING, CHANGING OR DEPOSITING
    • G07D7/00—Testing specially adapted to determine the identity or genuineness of valuable papers or for segregating those which are unacceptable, e.g. banknotes that are alien to a currency
    • G07D7/06—Testing specially adapted to determine the identity or genuineness of valuable papers or for segregating those which are unacceptable, e.g. banknotes that are alien to a currency using wave or particle radiation
    • G07D7/12—Visible light, infrared or ultraviolet radiation
    • G—PHYSICS
    • G07—CHECKING-DEVICES
    • G07D—HANDLING OF COINS OR VALUABLE PAPERS, e.g. TESTING, SORTING BY DENOMINATIONS, COUNTING, DISPENSING, CHANGING OR DEPOSITING
    • G07D7/00—Testing specially adapted to determine the identity or genuineness of valuable papers or for segregating those which are unacceptable, e.g. banknotes that are alien to a currency
    • G07D7/06—Testing specially adapted to determine the identity or genuineness of valuable papers or for segregating those which are unacceptable, e.g. banknotes that are alien to a currency using wave or particle radiation
    • G07D7/12—Visible light, infrared or ultraviolet radiation
    • G07D7/1205—Testing spectral properties

Definitions

  • the present invention relates to image acquisition devices, sheet handling devices, banknote handling devices, and image acquisition methods.
  • the present invention specifically relates to an image acquisition device, a sheet handling device, a banknote handling device, and an image acquisition method suitable for detecting a feature of a sheet having a transparent portion.
  • Sheets such as banknotes (printed money), gift vouchers, and checks have a variety of security features for anti-counterfeiting.
  • paper usually used for sheets is made of vegetable fibers
  • paper made of synthetic fibers or a polymer sheet made of synthetic resin may be used in order to improve the properties such as durability, water resistance, and security.
  • Banknotes made of polymer sheets are called polymer banknotes and are difficult to counterfeit.
  • an optical sensor such as an optical line sensor is usually used.
  • a transparent portion transmits light emitted from the optical sensor, and a sheet having a transparent portion may therefore need to undergo different processes from a common sheet having no transparent portion.
  • JP 2013-77163 A discloses an optical reading device and a sheet handling device that irradiate one surface of a sheet with light beams at different wavelengths from two different light sources and receive light beams emitted in such irradiation and passed through the sheet to achieve detection of a watermarked image on the sheet and detection of the shape and the presence or absence of a defect at the same stage.
  • the image is usually taken by an image acquisition device including an optical line sensor.
  • a transparent portion of the sheet transmits light, such as infrared light, emitted from a light source of the image acquisition device.
  • a sheet having a transparent portion needs to undergo different processes from a common sheet having no transparent portion.
  • the external shape (outline) of the sheet first needs to be detected (extracted) from image data based on an output of the optical line sensor.
  • the image based on the output of the optical line sensor includes not only the sheet but also the background thereof (a region other than the sheet), and the region corresponding to the sheet in the overall image needs to be specified and the external shape of the region needs to be extracted.
  • the region corresponding to the sheet may not be correctly extracted from the overall image.
  • a transmission image of a banknote having a transparent portion is taken using infrared light in order to detect a feature of ink of the banknote in its opaque portion
  • the infrared light may pass through the transparent portion and a transmissive image 210 as illustrated in FIG. 8 may be obtained.
  • a transparent region 212 which corresponds to the transparent portion
  • a medium region 211 which corresponds to the banknote
  • a background region 213 which corresponds to the region other than the banknote.
  • the quantity of light applied to the banknote needs to be high so that the output of a light receiving unit of an optical line sensor is high. In this case, the output of the light receiving unit in the transparent portion is saturated.
  • the banknote may be misrecognized as being torn into two pieces. Even when the transparent portion of the banknote has a shading pattern, this shading pattern cannot be detected from the transmission image 210 including the transparent region 212 because the transparent region 212 is a saturated region. Similarly, even when the transparent portion of the banknote has a defect, this defect in the transparent portion cannot be detected from the transmission image 210 including the transparent region 212.
  • transmissive light alone of such light of different wavelengths fails to enable correct detection of the external shape and the presence or absence of a variety of sheets having a transparent portion and to enable correct detection of the shading pattern and the presence or absence of a defect in the transparent portion.
  • a banknote is irradiated with visible light and infrared light each at a quantity of light that enables detection of a feature of ink in an opaque portion
  • the output of the light receiving unit is saturated for each light in the transparent portion which has a high transmittance, causing a failure in detecting the external shape.
  • an object of the present invention is to provide an image acquisition device, a sheet handling device, a banknote handling device, and an image acquisition method enabling acquisition of an image from which a feature of a sheet is detectable in an opaque portion and an image from which a feature of the sheet is detectable in a transparent portion.
  • one aspect of the present invention is an image acquisition device comprising: a light source configured to emit light to a sheet; a light receiving unit configured: to receive a first transmitted light generated by transmission of a first emitted light having a first quantity of light through the sheet to output a first image signal, the first emitted light being emitted from the light source; and to receive a second transmitted light generated by transmission of a second emitted light having a second quantity of light through the sheet to output a second image signal, the second emitted light being emitted from the light source; and an image generating unit configured: to generate a first transmission image from the first image signal; and to generate a second transmission image from the second image signal, wherein the second quantity of light is set to be smaller than the first quantity of light.
  • the first quantity of light is set to a quantity of light at which the first transmission image includes a saturated region
  • the second quantity of light is set to a quantity of light at which a region in the second transmission image corresponding to the saturated region is unsaturated.
  • the saturated region is a region where an image signal has a maximum output
  • the quantity of light at which a region is unsaturated is a quantity of light at which an image signal has an output lower than the maximum output
  • the image acquisition device further comprises a control unit configured to control the light source and the image generating unit, wherein the control unit is configured: to control the light source to emit the first and second emitted lights in accordance with timings such that the first and second emitted lights are emitted one after the other in a cyclic manner, and to control the image generating unit to read out the first and second image signals from the light receiving unit synchronously with the timings of emitting of the first and second emitted lights.
  • the light receiving unit is configured to receive light emitted from the light source in the absence of the sheet to output a third image signal
  • the image generating unit is configured to generate a reference waveform from the third image signal
  • the second quantity of light is set to allow the reference waveform to satisfy a predetermined condition.
  • the light receiving unit is configured to receive light generated by transmission of light emitted from the light source through a reference medium to output a fourth image signal
  • the image generating unit is configured to generate a reference medium waveform from the fourth image signal
  • the first quantity of light is set based on a transmittance of the reference medium calculated from the reference medium waveform.
  • the first quantity of light is set such that the transmittance of the reference medium as a whole is uniform.
  • the sheet is a banknote, a gift voucher, or a check having a transparent portion
  • the image generating unit is configured to generate a transmission image including an image of the transparent portion
  • the transparent portion is a portion having a transmittance of not lower than 30% and not higher than 90% with respect to the first emitted light.
  • the light source is configured to emit infrared light to the sheet.
  • the ratio of the second quantity of light to the first quantity of light is not less than 1/16 and not more than 1/4.
  • the light source is configured to emit visible light to the sheet.
  • Another aspect of the present invention is a sheet handling device comprising the above image acquisition device.
  • Another aspect of the present invention is a banknote handling device comprising the above image acquisition device.
  • Another aspect of the present invention is an image acquisition method comprising: receiving a first transmitted light generated by transmission of a first emitted light having a first quantity of light through a sheet to output a first image signal, the first emitted light being emitted from a light source; receiving a second transmitted light generated by transmission of a second emitted light having a second quantity of light through the sheet to output a second image signal, the second emitted light being emitted from the light source; generating a first transmission image from the first image signal; and generating a second transmission image from the second image signal, wherein the second quantity of light is set to be smaller than the first quantity of light.
  • the image acquisition device, the sheet handling device, the banknote handling device, and the image acquisition method of the present invention enables acquisition of an image from which a feature of a sheet is detectable in an opaque portion and an image from which a feature of the sheet is detectable in a transparent portion.
  • the image acquisition device Preferred embodiments of the image acquisition device, the sheet handling device, the banknote handling device, and the image acquisition method of the present invention are described below with reference to the drawings.
  • Examples of the sheet to be detected in the present invention include banknotes, checks, gift vouchers, bills, ledgers, documents of value, and card-like media.
  • the banknote handling device is an embodiment of the sheet handling device.
  • the present invention is described with devices and methods for banknotes taken as examples. Described in the following as a preferred embodiment of the image acquisition device of the present invention is a banknote recognition device also having a function of an image acquisition device.
  • the banknote recognition device may be a device constituting part of the banknote handling device, or may be a device separate from and not associated with the banknote handling device. Described in the following are examples of a banknote detection device (image acquisition device), a banknote handling device, and an image acquisition method.
  • the term “reflection image” means an image based on the intensity distribution of light generated from light applied to and reflected on a sheet.
  • transmission image means an image based on the intensity distribution of light generated from light applied to and passed through a sheet.
  • the banknote to be handled is preferably a polymer banknote having a transparent portion such as a clear window that transmits light such as infrared light and visible light applied.
  • a banknote having no transparent portion such as a paper banknote, may also be handled.
  • Preferred among paper banknotes are a medium having a highly transmissive watermark and a medium having a high transmittance such as an oil-stained banknote.
  • the transparent portion is preferably made from a synthetic resin (polymer).
  • the banknote to be handled is preferably formed from a polymer sheet.
  • the banknote to be handled may also be a sheet (hybrid banknote) whose transparent portion is formed from a polymer sheet and whose opaque portion is formed from paper made of vegetable fibers or synthetic fibers.
  • the base material of the banknote to be handled is preferably a polymer or a composite of paper and a polymer.
  • the transparent portion may partially include an optically variable device (OVD) such as rainbow hologram.
  • ODD optically variable device
  • FIG. 1 illustrates a banknote BN1 that is an exemplary banknote to be handled.
  • the banknote BN1 includes a band-shaped transparent portion BNla at a central portion in the longitudinal direction and opaque portions BN1b at both sides of the transparent portion BN1a.
  • the transparent portion BNla may be a region having an infrared (e.g., wavelength range: 760 to 1100 nm) transmittance of not lower than 30% and not higher than 90%, for example, and the front and back surfaces thereof have respective patterns printed thereon.
  • the opaque portions BNlb each may be a region having an infrared transmittance of not higher than 10%, for example, and the front and back surfaces thereof have respective patterns, such as a portrait or the denomination thereof, printed thereon.
  • Each of the patterns on the transparent portion BNla and the opaque portions BNlb is at least partially printed with infrared absorption ink, so that the pattern is a shading pattern including regions having different infrared transmittances.
  • the summary of the present embodiment is described.
  • a transmission image of a banknote BN1 having a transparent portion BNla is acquired in a conventional case, for example, the output of a transparent region 212 corresponding to the transparent portion BNla may be saturated and the region may exhibit highlight clipping, as illustrated in FIG. 8 .
  • a feature of the transparent portion BNla may not be obtained. This is because light having a relatively large quantity of light needs to be applied to the banknote BN1 so as to enable extraction of a feature of ink at the opaque portion BN1b.
  • the transmission image 210 enables detection of a feature of ink in the opaque portions BNlb and the presence or absence of a defect in the opaque portions BN1b.
  • the latter light having a small quantity of light leads to a transmission image 220, as illustrated in FIG. 2(b) , in which the output of the transparent region 222 corresponding to the transparent portion BNla is unsaturated and a feature of the transparent portion BNla is imaged.
  • the transmission image 220 enables detection of the shading pattern of the transparent portion BNla and the presence or absence of a defect in the transparent portion BN1a. For the latter transmission image 220, the light applied to the opaque portions BNlb has a small quantity of light.
  • the transparent portion BNla is preferably a portion having a transmittance of not lower than 30% with respect to light emitted from the light source of the banknote recognition device (image acquisition device) of the present embodiment, and may be a portion having a transmittance of not lower than 30% and not higher than 90%.
  • the output of the transparent region may be saturated.
  • light having a smaller quantity of light can lead to a transmission image in which the output of the transparent region is unsaturated.
  • the transparent portion BNla has a transmittance of higher than 90%, no threshold for determining the presence or absence of a medium may be obtained.
  • the banknote handling device of the present embodiment may have a structure illustrated in FIG. 3 , for example.
  • a banknote handling device 300 illustrated in FIG. 3 is a small banknote handling device to be used on a table, and includes a banknote recognition device (not illustrated in FIG.
  • a hopper 301 that supports a stack of banknotes to be handled, two rejecters 302 to which banknotes dispensed from the hopper 301 into a housing 310 are discharged when they are rejected banknotes such as counterfeit notes or suspect notes, an operation unit 303 with which an operator input the instructions, four stackers 306a to 306d that accumulate sorted banknotes whose denomination, authenticity, and fitness are recognized in the housing 310, and a display 305 that displays the information such as the recognition count results of banknotes and the accumulation states of the stackers 306a to 306d.
  • the stackers 306a to 306c store fit notes and the stacker 306d stores unfit notes among the four stackers 306a to 306d.
  • a method of sorting banknotes into the stackers 306a to 306d may be selected as appropriate.
  • an imaging unit 21 includes optical line sensors 110 and 120 arranged to face each other. Between the optical line sensors 110 and 120 is formed a gap through which a banknote BN is transported. This gap is a portion of a transport path 311 of the banknote handling device of the present embodiment.
  • the optical line sensors 110 and 120 are respectively placed on the upper and lower sides of the transport path 311.
  • the optical line sensor 110 includes two reflection light sources 111, a condenser 112, and a light receiving unit 113.
  • the reflection light sources 111 each emit light at predetermined wavelengths (invisible light such as infrared light and visible light such as single-color light of red, green, blue, or the like, or white light) to the main surface (hereinafter, referred to as the surface A) on the light receiving unit 113 side of a banknote BN.
  • the condenser 112 gathers light emitted from the reflection light sources 111 and reflected on the banknote BN.
  • the light receiving unit 113 includes solid state image sensors (not illustrated) arranged in line along the direction (main scanning direction) perpendicular to the direction (sub-scanning direction) of transporting a banknote BN, and receives the light gathered by the condenser 112 and converts the light into an electric signal. The electric signal is then amplified and converted into digital data by analog-to-digital conversion. The resulting digital data is output as an image signal.
  • the optical line sensor 120 includes two reflection light sources 121, a condenser 122, a light receiving unit 123, and a transmission light source 124.
  • the reflection light sources 121 each emit light at predetermined wavelengths (invisible light such as infrared light and visible light such as single-color light of red, green, blue, or the like, or white light) to the main surface (hereinafter, referred to as the surface B) on the light receiving unit 123 side of the banknote BN.
  • the condenser 122 gathers light emitted from the reflection light sources 121 and reflected on the banknote BN.
  • the light receiving unit 123 includes solid state image sensors (not illustrated) arranged in line along the direction perpendicular to the direction of transporting a banknote BN, and receives the light gathered by the condenser 122 and converts the light into an electric signal. The electric signal is then amplified and converted into digital data by analog-to-digital conversion. The resulting digital data is output as an image signal.
  • the transmission light source 124 emits light at predetermined wavelengths (invisible light such as infrared light and visible light such as single-color light of red, green, blue, or the like, or white light) to the surface B of the banknote BN.
  • the transmission light source 124 is placed on the optical axis of the condenser 112 of the optical line sensor 110. Part of the light emitted from the transmission light source 124 passes through the banknote BN, gathered by the condenser 112 of the optical line sensor 110, and detected by the light receiving unit 113.
  • the light sources 111, 121, and 124 each include a linear light guide (not illustrated) extending in the direction (main scanning direction) perpendicular to the paper surface of FIG. 4 and a plurality of LED elements (not illustrated) arranged at each end (optionally one end) of the light guide.
  • the reflection light sources 111 and 121 each may include, as the LED elements, LED elements capable of emitting light within different wavelength ranges, and are each configured to emit light within the selected wavelength ranges.
  • the reflection light sources 111 and 121 each include LED elements that emit infrared (IR) light and LED elements that emit visible light (single-color light of red, green, blue, or the like, or white light), and these reflection light sources 111 and 121 emit infrared light and visible light to a sheet.
  • the transmission light source 124 may include, as the LED elements, LED elements capable of emitting light within different wavelength ranges, and is configured to emit light within the selected wavelength ranges.
  • the transmission light source 124 includes LED elements that emit infrared (IR) light and LED elements that emit visible light (single-color light of red, green, blue, or the like, or white light), and the transmission light source 124 emits infrared light and visible light to a sheet.
  • IR infrared
  • visible light single-color light of red, green, blue, or the like, or white light
  • the optical line sensors 110 and 120 repetitively image a banknote BN under transport in the transport direction and output image signals.
  • the banknote recognition device of the present embodiment can acquire an image of the whole banknote BN.
  • the banknote recognition device of the present embodiment acquires reflection images of the surface A of the banknote BN and transmission images of the banknote BN based on the output signals from the optical line sensor 110, and acquires reflection images of the surface B of the banknote BN based on the output signals from the optical line sensor 120.
  • a banknote recognition device (image acquisition device) 1 of the present embodiment includes a control unit 10, a detection unit 20, and a memory unit 30.
  • the control unit 10 includes components such as programs for executing a variety of processing operations stored in the memory unit 30, a central processing unit (CPU) configured to execute the programs, a variety of hardware components to be controlled by the CPU, and a logical device, e.g., a field programmable gate array (FPGA).
  • the control unit 10 controls the components of the banknote recognition device 1 in accordance with the programs stored in the memory unit 30 based on the signals output from the components of the banknote recognition device 1 and the control signals from the control unit 10.
  • the control unit 10 also has functions of a light source control unit 11, a sensor control unit 12, an image generating unit 13, a shape detection unit 14, and a recognition unit 15 in accordance with the program stored in the memory unit 30.
  • the detection unit 20 also includes a magnetism detection unit 22, a thickness detection unit 23, and a UV detection unit 24 along the transport path for banknotes.
  • the imaging unit 21 images a banknote as described above and outputs an image signal (image data).
  • the magnetism detection unit 22 includes a magnetism sensor (not illustrated) that determines the magnetism.
  • the magnetism sensor detects the magnetism of magnetic ink printed on a banknote or of a security thread, for example.
  • the magnetism sensor is a magnetism line sensor including magnetism detection devices arranged in line.
  • the thickness detection unit 23 includes a thickness detection sensor (not illustrated) that measures the thickness of a banknote.
  • the thickness detection sensor detects abnormal feeding such as double feeding and banknotes with material such as tape, for example.
  • the thickness detection sensor includes rollers arranged to face each other with the transport path in between, and detects the displacements of the rollers in passing of a banknote by sensors provided for the respective rollers.
  • the UV detection unit 24 includes a ultraviolet light emitting unit (not illustrated) and a light receiving unit (not illustrated), and the light receiving unit detects fluorescence generated when the ultraviolet light emitting unit emits ultraviolet light to a banknote and ultraviolet light passed through the banknote.
  • the memory unit 30 includes a non-volatile memory such as a semiconductor memory or a hard disk drive, and stores a variety of programs and a variety of data for controlling the banknote recognition device 1.
  • the memory unit 30 stores imaging parameters such as the wavelength ranges of light emitted from the light sources 111, 121, and 124, the timings of turning on and off the light sources 111, 121, and 124, the values of forward currents applied to the LED elements of the light sources 111, 121, and 124, and the timings of reading out the signals from the optical line sensors 110 and 120, during one cycle of imaging by the imaging unit 21.
  • the one cycle of imaging means an imaging pattern in which the parameters, such as the wavelength ranges of the light emitted from the light sources 111, 121, and 124, the timings of turning on and off the light sources 111, 121, and 124, the values of forward currents applied to the LED elements, and the timings of reading out the signals, are set.
  • One cycle of imaging is taken as one period and repetitive consecutive execution of this cycle enables acquisition of an image of the whole banknote.
  • the light source control unit 11 performs dynamic lighting control in which the light sources 111, 121, and 124 are sequentially turned on so as to acquire distinct images of a banknote by the light sources 111, 121, and 124. Specifically, the light source control unit 11 controls turning on and off of the light sources 111, 121, and 124 based on the timings set in the imaging parameters. This control is performed using a mechanical clock that changes in accordance with the rate of transporting a banknote and a system clock that is always output at a constant frequency regardless of the rate of transporting a banknote. The light source control unit 11 sets the levels of forward currents applied to the LED elements based on the imaging parameters.
  • the sensor control unit 12 controls the timings of reading out the image signals from the optical line sensors 110 and 120 and reads out the image signals from the line sensors synchronously with the timings of turning on and off the light sources 111, 121, and 124 based on the timings set in the imaging parameters. This control is performed using the mechanical clock and the system clock. The sensor control unit 12 then sequentially stores the read-out image signals in a ring buffer (line memory) of the memory unit 30.
  • the image generating unit 13 has a function of generating an image based on a variety of signals relating to a banknote acquired from the detection unit 20. Specifically, the image generating unit 13 first decomposes the data (image signals) stored in the ring buffer into data sets for the respective conditions of light application and light reception. More specifically, the data is decomposed into received light intensity data of light generated from visible light applied and reflected, received light intensity data of light generated from infrared light applied and reflected, received light intensity data of light generated from visible light applied and transmitted, and received light intensity data of light generated from infrared light applied and transmitted.
  • the image generating unit 13 then performs correction processing such as cutting of dark outputs, gain adjustment, and correction of bright output levels in accordance with the properties of each decomposed data set, generates a variety of images (image data) of the banknote, and stores the images in the memory unit 30.
  • the shape detection unit 14 detects (extracts) the external shape (outline) of a banknote based on an infrared transmission image A of the banknote to be described later. The specification thereof will be described later.
  • the shape detection unit 14 also outputs the outline information (extracted partial image region) of the banknote to the recognition unit 15.
  • the recognition unit 15 utilizes a variety of signals relating to a banknote acquired from the detection unit 20 to execute recognition processing.
  • the recognition unit 15 recognizes at least the denomination and authenticity of the banknote.
  • the recognition unit 15 may have a function of determining the fitness of the banknote.
  • the recognition unit 15 has a function of detecting the presence or absence of a defect such as soil, fold, or tear in the banknote and detecting the presence or absence of material such as tape attached to the banknote based on the thickness of the banknote, and thereby determining whether the banknote is handled as a fit note to be reused in the market or as an unfit note unsuitable to circulation in the market.
  • the recognition unit 15 uses an image of a banknote taken by the imaging unit 21 for recognition of the information such as the denomination, the authenticity, and the fitness, it utilizes the outline information of the banknote acquired by the shape detection unit 14. For example, based on the outline information of a banknote acquired by the shape detection unit 14, the recognition unit 15 defines a medium region corresponding to the banknote as a recognition target area within the whole image including the banknote and the other region, divides the image data within the area into blocks, and executes recognition processing by, for example, pattern matching.
  • the recognition unit 15 may further detect the shading pattern of the transparent portion of a banknote or may detect the presence or absence of a defect in the transparent portion of a banknote based on the infrared transmission image A to be described later.
  • FIG. 6 illustrates the contents and timings of turning on the light sources and of reading out the signals.
  • the light source control unit 11 is configured to control the light sources to emit infrared light having a quantity of light a and infrared light having a quantity of light b in accordance with timings such that the infrared light having a quantity of light a and the infrared light having a quantity of light b are emitted one after the other in a cyclic manner. Specifically, as illustrated in the upper row of FIG.
  • a banknote under transport is irradiated with infrared light having a quantity of light a from the transmission light source 124, then with infrared light having a quantity of light b from the transmission light source 124, then with visible light from the transmission light source 124, then with infrared light from the reflection light source 111, and then with visible light from the reflection light source 111, during one cycle.
  • the quantity of light a and the quantity of light b correspond respectively to the second quantity of light and the first quantity of light.
  • the infrared light having a quantity of light a and the infrared light having a quantity of light b correspond respectively to the first emitted light and the second emitted light.
  • the sensor control unit 12 is configured to read out the image signals from the light receiving unit 113 synchronously with the timings of emitting of the infrared light having a quantity of light a and the infrared light having a quantity of light b. In other words, each time the light applied to the banknote is switched, the image signal by the light before switching is read out from the optical line sensor 110.
  • the optical line sensor 110 acquires data of one line constituting a transmission image by the infrared light having the quantity of light a (hereinafter, referred to as an infrared transmission image A), data of one line constituting a transmission image by the infrared light having the quantity of light b (hereinafter, referred to as an infrared transmission image B), data of one line constituting a transmission image by the visible light (hereinafter, referred to as a visible transmission image), data of one line constituting a reflection image of the surface A by the infrared light (hereinafter, referred to as an infrared reflection image of the surface A), and data of one line constituting a reflection image of the surface A by the visible light (hereinafter, referred to as a visible reflection image of the surface A) during one cycle.
  • an infrared transmission image A data of one line constituting a transmission image by the infrared light having the quantity of light b
  • an infrared transmission image B data of one line constituting
  • Repetitive consecutive execution of this one cycle of imaging enables acquisition of the infrared transmission image A, the infrared transmission image B, the visible transmission image, the infrared reflection image of the surface A, and the visible reflection image of the surface A of the whole banknote.
  • the infrared transmission image A and the infrared transmission image B correspond respectively to the second transmission image and the first transmission image.
  • the reflection light source 121 is first turned off for a predetermined period of time (a period of time during which the sheet is irradiated with light from the transmission light source 124), thereafter the banknote under transport is irradiated with infrared light from the reflection light source 121, and then with visible light from the reflection light source 121, during one cycle.
  • the imaging devices of the light receiving unit 123 are exposed to light and thereby store electric charges. Also in this case, each time the light applied to the banknote is switched, the image signal by the light before switching is read out from the optical line sensor 120.
  • the optical line sensor 120 acquires data of one line constituting a reflection image of the surface B by the infrared light (hereinafter, referred to as an infrared reflection image of the surface B) and data of one line constituting a reflection image of the surface B by the visible light (hereinafter, referred to as a visible reflection image of the surface B) during one cycle.
  • Repetitive consecutive execution of this one cycle of imaging enables acquisition of the infrared reflection image of the surface B and the visible reflection image of the surface B.
  • the durations of applying the respective types of light are set to be equal to each other.
  • the durations of storing electric charges by the imaging devices are set to be equal to each other regardless of the type of light.
  • the quantity of light a is set to be lower than the quantity of light b.
  • the ratio of the quantity of light a to the quantity of light b is preferably not less than 1/16 and not more than 1/4, although it can be set as appropriate.
  • the quantity of light is expressed by (level of forward current applied to LED element) ⁇ (duration of irradiation).
  • the durations of irradiation are equal to each other as described above, and thus the value of forward current applied to each LED element of the transmission light source 124 is set to be lower in imaging of the infrared transmission image A than in imaging of the infrared transmission image B.
  • the level of forward current applied to an LED element is proportional to the radiant intensity of the LED element, so that the quantity of light can be regarded as being proportional to the radiant intensity of the LED element.
  • the quantity of light b is preferably set to a quantity of light at which the infrared transmission image B (specifically, a medium region in the infrared transmission image B corresponding to the banknote, usually a transparent region corresponding to the transparent portion) includes a saturated region (a region exhibiting highlight clipping).
  • a saturated region a region exhibiting highlight clipping.
  • this region is a region where an output of an image signal is to be a predetermined maximum value (e.g., 255 digit) after the outputs of the optical line sensors 110 and 120 are subjected to dark output correction in which the output with the light sources turned off is set to zero and bright output correction in which gain correction is performed on each pixel such that the output of a white reference medium with the light sources turned on is uniform, and then data constituting a transmission image in the absence of the white reference medium is acquired.
  • a predetermined maximum value e.g., 255 digit
  • the quantity of light a is preferably set to a quantity of light at which a region in the infrared transmission image A corresponding to the saturated region of the infrared transmission image B is unsaturated (does not exhibit highlight clipping). This enables detection of a feature of the banknote, such as the external shape, the shading pattern, or the presence or absence of a defect in the transparent portion, in a transparent region corresponding to the transparent portion of the banknote from the infrared transmission image A.
  • the quantity of light at which a region is unsaturated is a quantity of light at which an image signal has an output lower than the maximum value.
  • the shape detection unit 14 detects (extracts) the external shape (outline) of a banknote based on the infrared transmission image A.
  • the shape detection unit 14 specifies a medium region (partial image region) from the whole infrared transmission image A including the medium region corresponding to the banknote and a background region corresponding to the region other than the banknote, and then detects the external shape of the medium region.
  • the shape detection unit 14 first binarizes the infrared transmission image A based on a predetermined threshold. In other words, each pixel value of the infrared transmission image A is compared with the predetermined threshold; the pixel with a pixel value of lower than the threshold is determined as medium-present, and the pixel data thereof is replaced by 1 (white), while the pixel with a pixel value of not lower than the threshold is determined as medium-absent, and the pixel data thereof is replaced by 0 (black).
  • the shape detection unit 14 then executes edge detection on the binarized infrared transmission image A, and thereby detects the four sides of the banknote from the results of the edge detection.
  • the shape detection unit 14 subsequently executes Hough transform to compute the straight lines passing the four sides of the banknote, thereby determines the four apexes corresponding to the four corners of the banknote.
  • the quantity of light a in acquiring the infrared transmission image A is set to a quantity of light that is lower than the quantity of light b in acquiring the infrared transmission image B and at which a region in the infrared transmission image A corresponding to the saturated region in the infrared transmission image B is unsaturated.
  • the shape detection unit 14 can determine the transparent region corresponding to the transparent portion of the banknote as medium-present. This can prevent highlight clipping of the transparent region in the infrared transmission image A and the resulting assimilation thereof to the background region.
  • the external shape of the banknote can be detected not only in the opaque regions corresponding to the opaque portions of the banknote but also in the transparent region.
  • the processing by the banknote recognition device 1, in particular the method for acquiring the infrared transmission images A and B of a banknote is described.
  • First described is a method of setting the quantity of light a and the quantity of light b, in other words, a method of initial setting of the transmission light source 124.
  • the quantity of light a is preferably set as follows. Specifically, the transmission light source 124 is turned on with no medium such as a banknote placed at the imaging position of the optical line sensor 110.
  • the light receiving unit 113 is made to receive the light (e.g., infrared light) emitted from the transmission light source 124 and to output an image signal (ID data) corresponding to the third image signal.
  • the image generating unit 13 is made to generate a reference waveform from this image signal.
  • the quantity of light a is set to allow this reference waveform to satisfy a predetermined condition.
  • the predetermined condition may be set as appropriate in accordance with the characteristics of a medium to be imaged.
  • the quantity of light a may be adjusted by controlling the level of forward current applied to each LED element and the duration of irradiation such that the maximum value of the reference waveform is 200 digit (255 digit indicates saturation, 200 digit indicates a transmittance of 100%), and then the reference waveform may be subjected to correction processing such as cutting of dark outputs, gain adjustment, and correction of bright output levels.
  • the quantity of light a may be calculated from the waveform obtained with a reference medium, which has a lower transmittance than the transparent portion of a banknote to be imaged, placed at the imaging position of the optical line sensor 110.
  • the quantity of light b is preferably set as follows. Specifically, the transmission light source 124 is turned on with an entirely white reference medium placed at the imaging position of the optical line sensor 110.
  • the light receiving unit 113 is made to receive the light (e.g., infrared light) emitted from the transmission light source 124 and to output an image signal (ID data) corresponding to the fourth image signal.
  • the image generating unit 13 is made to generate a reference medium waveform from this image signal.
  • the transmittance of the reference medium is calculated from this reference medium waveform, and the quantity of light b is set based on the calculated transmittance of the reference medium. Specifically, the quantity of light b may be set such that the transmittance of the reference medium as a whole is uniform.
  • the level of forward current applied to each LED element and the duration of irradiation may be controlled such that the maximum value of the reference medium waveform is about 128 digit that is about a half of 255 digit.
  • the output of the image signal may be further controlled such that the maximum value of the reference medium waveform is 4/5 of the original value after each LED element is controlled.
  • the light receiving unit 113 first receives light generated by transmission of light having a quantity of light a emitted from the transmission light source 124 through a banknote to output an image signal S1, and receives light generated by transmission of light having a quantity of light b emitted from the transmission light source 124 through the banknote to output an image signal S2 (S11).
  • the quantity of light a is set to be lower than the quantity of light b.
  • the light generated by transmission of light having a quantity of light a through a banknote and the light generated by transmission of light having a quantity of light b through a banknote correspond respectively to the second transmitted light and the first transmitted light.
  • the image signal S1 and the image signal S2 correspond respectively to the second image signal and the first image signal.
  • the sensor control unit 12 reads out the image signals S1 and S2 from the optical line sensor 110 and sequentially stores the read-out image signals S1 and S2 in the ring buffer of the memory unit 30 (S12).
  • the image generating unit 13 generates an infrared transmission image A from the data based on the image signal S1 and generates an infrared transmission image B from the data based on the image signal S2 (S13).
  • the quantity of light b is set to a quantity of light at which the infrared transmission image B includes a saturated region
  • the quantity of light a is set to a quantity of light at which a region in the infrared transmission image A corresponding to this saturated region is unsaturated.
  • the light receiving unit 113 is configured to receive light generated by transmission of light having a quantity of light a through a banknote to output an image signal S1, the light having the quantity of light a being emitted from the transmission light source 124, and to receive light generated by transmission of light having a quantity of light b through the banknote to output an image signal S2, the light having the quantity of light b being emitted from the transmission light source 124; and the image generating unit 13 is configured to generate an infrared transmission image A from the image signal S1 and to generate an infrared transmission image B from the image signal S2.
  • the quantity of light a is set to be smaller than the quantity of light b.
  • the quantity of light b is set to a quantity of light at which the infrared transmission image B includes a saturated region
  • the quantity of light a is set to a quantity of light at which a region in the infrared transmission image A corresponding to this saturated region is unsaturated.
  • Described in the above embodiment is a case where the values of forward currents applied to the LED elements of the transmission light source 124 are different from each other while the durations of applying the respective types of light (i.e., the durations of receiving the respective types of light and storing electric charges by the imaging devices) are set to be equal to each other such that the quantity of light a is lower than the quantity of light b. Still, these durations may be different from each other. Specifically, the duration of irradiating a banknote with light from the transmission light source 124 may be shorter in imaging of the infrared transmission image A than in imaging of the infrared transmission image B, while the value of forward current applied to each LED element of the transmission light source 124 may be equal in both imaging processes.
  • the duration of irradiating a banknote with light from the transmission light source 124 may be shorter in imaging of the infrared transmission image A than in imaging of the infrared transmission image B and the value of forward current applied to each LED element of the transmission light source 124 may be smaller in the former imaging than in the latter imaging.
  • the light having a quantity of light a and the light having a quantity of light b are infrared light. Still, the light having a quantity of light a and the light having a quantity of light b each may be visible light of red, green, blue, or the like, for example.
  • the ratio of the quantity of light a to the quantity of light b is preferably not less than 1/16 and not more than 1/4, although it can be set as appropriate. In other words, the ratio of the quantity of light a to the quantity of light b may vary in accordance with the wavelength range used. The level of the quantity of light a and the level of the quantity of light b may vary in accordance with the wavelength range used.
  • FIG. 9 illustrates the control of the light sources by the light source control unit 11 and the control of readout of the signals from the optical line sensors 110 and 120 by the sensor control unit 12 in the case where the light having a quantity of light a and the light having a quantity of light b are visible light.
  • FIG. 9 illustrates the contents and timings of turning on the light sources and of reading out the signals. This case is the same as the case of FIG. 6 , except for partial difference in the contents and the timings of turning on the light sources.
  • a banknote under transport is irradiated with visible light having a quantity of light a from the transmission light source 124, then with visible light having a quantity of light b from the transmission light source 124, then with infrared light from the transmission light source 124, then with infrared light from the reflection light source 111, and then with visible light from the reflection light source 111, during one cycle.
  • the optical line sensor 110 acquires data of one line constituting a transmission image by the visible light having the quantity of light a (hereinafter, referred to as a visible transmission image A), data of one line constituting a transmission image by the visible light having the quantity of light b (hereinafter, referred to as a visible transmission image B), data of one line constituting a transmission image by the infrared light (hereinafter, referred to as an infrared transmission image), data of one line constituting an infrared reflection image of the surface A, and data of one line constituting a visible reflection image of the surface A, during one cycle.
  • a visible transmission image A data of one line constituting a transmission image by the visible light having the quantity of light b
  • an infrared transmission image data of one line constituting a transmission image by the infrared light
  • Repetitive consecutive execution of this one cycle of imaging enables acquisition of the visible transmission image A, the visible transmission image B, the infrared transmission image, the infrared reflection image of the surface A, and the visible reflection image of the surface A of the whole banknote.
  • the visible transmission image A and the visible transmission image B correspond respectively to the second transmission image and the first transmission image.
  • imaging by the optical line sensor 120 is the same as in the case illustrated in FIG. 6 .
  • the durations of applying the respective types of light are set to be equal to each other.
  • the durations of storing electric charges by the imaging devices are set to be equal to each other regardless of the type of light.
  • the quantity of light a is set to be lower than the quantity of light b.
  • the ratio of the quantity of light a to the quantity of light b can be set as appropriate. Still, as described above, the ratio of the quantity of light a to the quantity of light b is preferably not less than 1/16 and not more than 1/4 in the case of applying green light.
  • the quantity of light b is preferably set to a quantity of light at which the visible transmission image B (specifically, a medium region in the visible transmission image B corresponding to the banknote, usually a transparent region corresponding to the transparent portion) includes a saturated region (a region exhibiting highlight clipping).
  • the visible transmission image B specifically, a medium region in the visible transmission image B corresponding to the banknote, usually a transparent region corresponding to the transparent portion
  • a saturated region a region exhibiting highlight clipping.
  • the quantity of light a is preferably set to a quantity of light at which a region in the visible transmission image A corresponding to the saturated region of the visible transmission image B is unsaturated (does not exhibit highlight clipping). This enables detection of a feature of the banknote, such as the external shape, the shading pattern, or the presence or absence of a defect in the transparent portion, in a transparent region corresponding to the transparent portion of the banknote from the visible transmission image A.
  • Described in the above embodiment is a case where light having a quantity of light a and light having a quantity of light b are within the same wavelength range. Still, the wavelength range of the light having a quantity of light a and the wavelength range of the light having a quantity of light b may be different from each other.
  • the shape detection unit 14 detects the external shape of a banknote as a feature of the banknote and the recognition unit 15 detects the shading pattern of the transparent portion and the presence or absence of a defect in the transparent portion as features of the banknote based on the infrared transmission image A acquired with a small quantity of light a.
  • the banknote recognition device (image acquisition device) 1 may detect the presence or absence of a banknote based on the infrared transmission image A or the visible transmission image A. Thereby, even when a banknote has a transparent portion at an edge, the banknote recognition device 1 can detect this edge as described above, and thus can correctly detect passing of the banknote. Accordingly, the banknote recognition device (image acquisition device) 1 can also suitably be used as a tracking sensor for detecting the presence or absence of a banknote under transport.
  • the present invention provides a technique useful for acquiring an image from which a feature of a sheet is detectable in an opaque portion and an image from which a feature of the sheet is detectable in a transparent portion.

Landscapes

  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Toxicology (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Spectroscopy & Molecular Physics (AREA)
  • Inspection Of Paper Currency And Valuable Securities (AREA)
  • Image Input (AREA)
EP20150651.6A 2019-01-11 2020-01-08 Dispositif d'acquisition d'images, dispositif de manipulation de feuilles, dispositif de manipulation de billets de banque et procédé d'acquisition d'images Active EP3680867B1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2019003721 2019-01-11
JP2020000385A JP7391668B2 (ja) 2019-01-11 2020-01-06 画像取得装置、紙葉類処理装置、紙幣処理装置及び画像取得方法

Publications (2)

Publication Number Publication Date
EP3680867A1 true EP3680867A1 (fr) 2020-07-15
EP3680867B1 EP3680867B1 (fr) 2022-05-04

Family

ID=71667909

Family Applications (1)

Application Number Title Priority Date Filing Date
EP20150651.6A Active EP3680867B1 (fr) 2019-01-11 2020-01-08 Dispositif d'acquisition d'images, dispositif de manipulation de feuilles, dispositif de manipulation de billets de banque et procédé d'acquisition d'images

Country Status (3)

Country Link
EP (1) EP3680867B1 (fr)
JP (1) JP7391668B2 (fr)
AU (1) AU2020200185A1 (fr)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPWO2023007845A1 (fr) * 2021-07-30 2023-02-02
JP2023135883A (ja) * 2022-03-16 2023-09-29 グローリー株式会社 紙葉類識別装置、紙葉類処理装置及び紙葉類識別方法

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1011079A1 (fr) * 1998-12-14 2000-06-21 Kabushiki Kaisha Toshiba Appareil pour déterminer le degré de salissure de matières imprimées
US20010008275A1 (en) * 2000-01-14 2001-07-19 Takahiro Yanagiuchi Paper sheet discriminating device
US20060263101A1 (en) * 2003-04-23 2006-11-23 Seiji Takamatsu Sheet identifying device and method
US20100102234A1 (en) * 2007-03-29 2010-04-29 Glory Ltd. Paper-sheet recognition apparatus, paper-sheet processing apparatus, and paper-sheet recognition method
JP2013077163A (ja) 2011-09-30 2013-04-25 Toshiba Corp 光学読取装置および紙葉類処理装置
US20160216207A1 (en) * 2011-06-06 2016-07-28 Sicpa Holding Sa In-line decay-time scanner
WO2018167876A1 (fr) * 2017-03-15 2018-09-20 グローリー株式会社 Dispositif de détection de feuilles de papier, appareil de traitement de feuilles de papier et procédé de détection de feuilles de papier

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP3705267B2 (ja) 2002-12-09 2005-10-12 ノーリツ鋼機株式会社 フィルムスキャナ
JP2009075866A (ja) 2007-09-20 2009-04-09 Toshiba Corp 通過検知器及びそれを備えた紙葉類処理装置
JP6403391B2 (ja) 2014-01-27 2018-10-10 グローリー株式会社 紙幣処理装置及び紙幣処理方法
EP3467794B1 (fr) 2016-05-24 2022-10-26 Glory Ltd. Dispositif de collecte d'informations de feuilles de papier et procédé de collecte d'informations de feuilles de papier

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1011079A1 (fr) * 1998-12-14 2000-06-21 Kabushiki Kaisha Toshiba Appareil pour déterminer le degré de salissure de matières imprimées
US20010008275A1 (en) * 2000-01-14 2001-07-19 Takahiro Yanagiuchi Paper sheet discriminating device
US20060263101A1 (en) * 2003-04-23 2006-11-23 Seiji Takamatsu Sheet identifying device and method
US20100102234A1 (en) * 2007-03-29 2010-04-29 Glory Ltd. Paper-sheet recognition apparatus, paper-sheet processing apparatus, and paper-sheet recognition method
US20160216207A1 (en) * 2011-06-06 2016-07-28 Sicpa Holding Sa In-line decay-time scanner
JP2013077163A (ja) 2011-09-30 2013-04-25 Toshiba Corp 光学読取装置および紙葉類処理装置
WO2018167876A1 (fr) * 2017-03-15 2018-09-20 グローリー株式会社 Dispositif de détection de feuilles de papier, appareil de traitement de feuilles de papier et procédé de détection de feuilles de papier

Also Published As

Publication number Publication date
AU2020200185A1 (en) 2020-07-30
EP3680867B1 (fr) 2022-05-04
JP7391668B2 (ja) 2023-12-05
JP2020113270A (ja) 2020-07-27

Similar Documents

Publication Publication Date Title
US9210332B2 (en) Determining document fitness using illumination
US8509515B2 (en) Paper sheet identifying device and paper sheet identifying method
US20100128964A1 (en) Sequenced Illumination
EP3598401B1 (fr) Dispositif de détection de feuilles de papier, appareil de traitement de feuilles de papier et procédé de détection de feuilles de papier
US8989433B2 (en) Paper sheet recognition apparatus and paper sheet recognition method
EP3723054B1 (fr) Unité de reconnaissance de billets de banque, dispositif de manipulation de billets de banque et procédé de reconnaissance de billets de banque
EP3680867B1 (fr) Dispositif d'acquisition d'images, dispositif de manipulation de feuilles, dispositif de manipulation de billets de banque et procédé d'acquisition d'images
US20190066428A1 (en) Invisible-feature detection device, sheet recognition device, sheet handling device, print inspection device, and invisible-feature detection method
US7519213B2 (en) Optical double feed detection
JP3892081B2 (ja) 紙葉類の真偽判定方法
US20110019872A1 (en) Paper identifying apparatus and paper identifying method
US20230015962A1 (en) Optical sensor and sheet recognition unit
JP2011034172A (ja) 証券類識別装置および証券類識別方法
JP7473677B2 (ja) 光学センサ、紙葉類識別装置、紙葉類処理装置及び光検出方法
WO2020208806A1 (fr) Dispositif d'acquisition d'image, dispositif de manipulation de feuille de papier et procédé d'acquisition d'image
WO2022210372A1 (fr) Dispositif de détection d'alimentation multiple et procédé de détection d'alimentation multiple
EP3982340A1 (fr) Dispositif de détection de fluorescence/phosphorescence, dispositif de traitement de feuilles de papier, et procédé de détection de fluorescence/phosphorescence
WO2023176530A1 (fr) Dispositif d'identification de feuille de papier, dispositif de traitement de feuille de papier et procédé d'identification de feuille de papier
JP2022045035A (ja) 紙葉類識別装置、紙葉類処理装置及び紙葉類識別方法
JPH10208105A (ja) 蛍光センサを用いた紙葉類の真偽判定装置
JP2021012587A (ja) 記番号読取装置、紙葉類処理装置、及び記番号読取方法

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE

17P Request for examination filed

Effective date: 20200108

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

AX Request for extension of the european patent

Extension state: BA ME

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: EXAMINATION IS IN PROGRESS

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: EXAMINATION IS IN PROGRESS

17Q First examination report despatched

Effective date: 20201204

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: GRANT OF PATENT IS INTENDED

INTG Intention to grant announced

Effective date: 20220119

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE PATENT HAS BEEN GRANTED

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: AT

Ref legal event code: REF

Ref document number: 1489932

Country of ref document: AT

Kind code of ref document: T

Effective date: 20220515

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 602020002876

Country of ref document: DE

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: LT

Ref legal event code: MG9D

REG Reference to a national code

Ref country code: NL

Ref legal event code: MP

Effective date: 20220504

REG Reference to a national code

Ref country code: AT

Ref legal event code: MK05

Ref document number: 1489932

Country of ref document: AT

Kind code of ref document: T

Effective date: 20220504

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220504

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220905

Ref country code: NO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220804

Ref country code: NL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220504

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220504

Ref country code: HR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220504

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220805

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220504

Ref country code: ES

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220504

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220804

Ref country code: AT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220504

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: RS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220504

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220504

Ref country code: LV

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220504

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220904

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SM

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220504

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220504

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220504

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220504

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220504

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220504

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 602020002876

Country of ref document: DE

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: AL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220504

26N No opposition filed

Effective date: 20230207

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220504

P01 Opt-out of the competence of the unified patent court (upc) registered

Effective date: 20230531

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20230108

REG Reference to a national code

Ref country code: BE

Ref legal event code: MM

Effective date: 20230131

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20230131

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20230131

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: FR

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20230131

Ref country code: BE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20230131

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220504

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20230108

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: DE

Payment date: 20240119

Year of fee payment: 5

Ref country code: GB

Payment date: 20240123

Year of fee payment: 5