JP2000083930A - Personal identifying device using iris - Google Patents
Personal identifying device using irisInfo
- Publication number
- JP2000083930A JP2000083930A JP10256452A JP25645298A JP2000083930A JP 2000083930 A JP2000083930 A JP 2000083930A JP 10256452 A JP10256452 A JP 10256452A JP 25645298 A JP25645298 A JP 25645298A JP 2000083930 A JP2000083930 A JP 2000083930A
- Authority
- JP
- Japan
- Prior art keywords
- person
- eye
- image
- identified
- sight
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000001514 detection method Methods 0.000 claims description 2
- 210000000554 iris Anatomy 0.000 description 22
- 238000010586 diagram Methods 0.000 description 11
- 238000000034 method Methods 0.000 description 6
- 238000013459 approach Methods 0.000 description 2
- 239000000284 extract Substances 0.000 description 2
- 239000011521 glass Substances 0.000 description 2
- 208000016339 iris pattern Diseases 0.000 description 2
- 235000002673 Dioscorea communis Nutrition 0.000 description 1
- 241000544230 Dioscorea communis Species 0.000 description 1
- 241001469893 Oxyzygonectes dovii Species 0.000 description 1
- 208000035753 Periorbital contusion Diseases 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 210000000744 eyelid Anatomy 0.000 description 1
- 230000001678 irradiating effect Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 210000001747 pupil Anatomy 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/18—Eye characteristics, e.g. of the iris
Landscapes
- Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Ophthalmology & Optometry (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)
- Collating Specific Patterns (AREA)
- Image Input (AREA)
Abstract
Description
【０００１】[0001]
【発明の属する技術分野】本発明は、目の中の虹彩（ア
イリス）の模様によって個人識別を行う個人識別装置に
関する。[0001] 1. Field of the Invention [0002] The present invention relates to a personal identification device for performing personal identification based on the pattern of an iris in an eye.
【０００２】[0002]
【従来の技術】図３にアイリスを用いた個人識別装置の
構成を示す。同装置は、被識別者の目の画像を取得する
ための読取部３１と、取得した画像を基にして個人識別
を行う識別部３９とからなり、部屋に出入する人物を管
理する入退室管理装置や金融機関に設置される入出金装
置等に用いられる。2. Description of the Related Art FIG. 3 shows a configuration of a personal identification device using an iris. The device includes a reading unit 31 for acquiring an image of an eye of a person to be identified and an identification unit 39 for performing individual identification based on the acquired image. It is used for devices and deposit / withdrawal devices installed in financial institutions.
【０００３】図中、３１は読取部であり、３２は窓、３
３はカメラ、３４はディスプレイ、３５はハーフミラ
ー、３６はライト、３７はスタートスイッチ、３８は制
御部、３９は識別部である。図示するように個人識別装
置の読取部３１は、被識別者が覗くための窓３２が設け
られており、その奥にはハーフミラー３５が設けられて
いる。また、窓３２の外の光景を撮影するカメラ３３
と、カメラ３３が撮影した画像をリアルタイムで表示す
るディスプレイ３４が設けられている。なお、図中の矢
印は窓３２から装置内に入った光（すなわち、窓３２の
外の光景）の流れを示している。その上、窓３２を覗い
ている被識別者の目に向けて光を照射するライト３６が
設けられている。In the figure, reference numeral 31 denotes a reading unit, 32 denotes a window, 3
Reference numeral 3 denotes a camera, 34 denotes a display, 35 denotes a half mirror, 36 denotes a light, 37 denotes a start switch, 38 denotes a control unit, and 39 denotes an identification unit. As shown in the figure, the reading unit 31 of the personal identification device is provided with a window 32 through which a person to be identified can look through, and a half mirror 35 is provided behind the window 32. In addition, a camera 33 for photographing a scene outside the window 32
And a display 34 for displaying an image captured by the camera 33 in real time. The arrows in the figure indicate the flow of light that has entered the apparatus from the window 32 (that is, the scene outside the window 32). In addition, a light 36 for irradiating light toward the eyes of the identified person looking through the window 32 is provided.
【０００４】同装置は以下のようにして動作する。すな
わち、被識別者が読取部３１に近づくと、制御部３８が
図示せぬセンサにより被識別者が接近したことを検知
し、ライト３６を点灯する。またカメラ３３とディスプ
レイ３４を起動して、カメラ３３によって窓３２の外の
光景を撮影するとともに、撮影した画像をリアルタイム
でディスプレイ３４に表示する。そして、被識別者が窓
３２から中を覗くことによって、カメラ３３は被識別者
の目を撮影し、ディスプレイ３４は被識別者の目の画像
を表示する。被識別者は、ディスプレイ３４を見なが
ら、目が最も鮮明に映るように顔を前後に動かして目の
位置をカメラ３３の焦点位置に合わせ、焦点が合ったら
スタートスイッチ３７を押下する。すると、制御部３８
はカメラ３３が撮影した被識別者の目の画像（動画像）
から静止画像を取得し、認識部３９に出力する。[0004] The device operates as follows. That is, when the identified person approaches the reading unit 31, the control unit 38 detects that the identified object has approached by a sensor (not shown), and turns on the light 36. In addition, the camera 33 and the display 34 are activated, and a scene outside the window 32 is photographed by the camera 33, and the photographed image is displayed on the display 34 in real time. When the identified person looks into the window 32, the camera 33 takes an image of the identified person's eyes, and the display 34 displays an image of the identified person's eyes. The distinguished person moves the face back and forth so that the eyes appear most sharply while looking at the display 34, and adjusts the position of the eyes to the focal position of the camera 33. When the subject is focused, the start switch 37 is pressed. Then, the control unit 38
Is an image (moving image) of the eye of the identified person photographed by the camera 33
, And outputs the still image to the recognition unit 39.
【０００５】識別部３９は被識別者の目の静止画像から
アイリスの模様を抽出する。この模様は個人毎に異なっ
ており、識別部３９はその模様から個人を特定可能なデ
ータ（以後、アイリスデータという）を生成する。そし
て、生成した被識別者のアイリスデータを予め登録され
たアイリスデータと比較して、被識別者が誰であるの
か、あるいは被識別者が照合しようとする人物本人であ
るのかを識別する。[0005] The identification unit 39 extracts an iris pattern from a still image of the eye of the identified person. This pattern differs for each individual, and the identification unit 39 generates data (hereinafter, referred to as iris data) that can specify the individual from the pattern. Then, the generated iris data of the identified person is compared with pre-registered iris data to identify who the identified person is or whether the identified person is the person to be collated.
【０００６】[0006]
【発明が解決しようとする課題】ところで、悪意を持つ
者（以下、第三者という）は様々な方法で他人者になり
すますものと予測される。例えば、他人のアイリスを撮
影した写真を取得してその写真を眼鏡や義眼に貼り付
け、その眼鏡や義眼を装着した状態で照合を受けたり、
または他人の名義で登録を行ったりして他人になりすま
すことを試みることが考えられる。[0005] By the way, it is expected that a malicious person (hereinafter, referred to as a third party) will impersonate another person in various ways. For example, you can take a picture of another person's iris, paste the picture into glasses or artificial eyes, or receive verification while wearing the glasses or artificial eyes,
Alternatively, it is conceivable to try to impersonate another person by registering in the name of another person.
【０００７】本発明はこのような不正を検出し、第三者
が他人になりすますことができないアイリスを用いた個
人識別装置を提供することを目的とする。An object of the present invention is to provide a personal identification device using an iris that detects such fraud and prevents a third party from impersonating another person.
【０００８】[0008]
【課題を解決するための手段】上記課題を解決するため
に本発明は、被識別者の目の画像を取得する撮影手段
と、個別に点消灯を制御することができ、被識別者の視
線を誘導するための複数の光源と、被識別者の視線の方
向を検出する検出手段とを有し、前記複数の光源を点消
灯させながら前記撮影手段により被識別者の目の画像を
取得し、取得した画像から前記検出手段により被識別者
の視線の方向を検出する。SUMMARY OF THE INVENTION In order to solve the above-mentioned problems, the present invention provides a photographing means for acquiring an image of an eye of a discriminated person, and can individually control turning on and off of the eyes of the discriminated person. A plurality of light sources for guiding the object, and detecting means for detecting the direction of the line of sight of the identified person, and acquiring an image of the eye of the identified person by the photographing unit while turning on and off the plurality of light sources. The direction of the line of sight of the identified person is detected from the acquired image by the detection means.
【０００９】また、被識別者の視線が誘導する方向に動
いた場合に、取得した目の画像に基づいて個人識別を行
う。Further, when the line of sight of the person to be identified moves in a direction to be guided, personal identification is performed based on the acquired eye image.
【００１０】さらに、前記検出手段は、被識別者の両目
の視線の方向を検出することができ、被識別者の両目の
視線が誘導する方向に動いた場合に、取得した目の画像
に基づいて個人識別を行う。Further, the detecting means can detect the direction of the line of sight of the eyes of the identified person, and when the line of sight of the identified person moves in the direction in which the eyes are guided, based on the acquired eye image. Perform personal identification.
【００１１】本発明は、これにより、前記撮影手段によ
り取得した目の画像が生体によるものか否かを判断で
き、生体によるものと判断できる場合にのみ個人識別を
行いえる。According to the present invention, it is possible to determine whether or not the image of the eye acquired by the photographing means is from a living body, and to perform personal identification only when it can be determined that the image is from a living body.
【００１２】[0012]
【発明の実施の形態】本発明は、個別に点消灯を制御す
ることができ、被識別者の視線を誘導するための複数の
光源を設けるとともに、制御部３８に光源の点消灯を制
御する機能および被識別者の視線の方向を検出する機能
を付与している。そして、制御部３８によって、光源の
点消灯を制御しながらカメラ３３が撮影した被識別者の
目の動画像を検証し、被識別者の視線の方向を検出す
る。これにより、カメラ３３が撮影した被識別者の目の
画像が生体によるものか否かを判断する。According to the present invention, a plurality of light sources for guiding the line of sight of a discriminated person can be provided, and the control unit 38 controls the turning on and off of the light sources. The function and the function of detecting the direction of the line of sight of the identified person are provided. Then, the control unit 38 verifies the moving image of the eye of the identified person captured by the camera 33 while controlling the turning on and off of the light source, and detects the direction of the line of sight of the identified person. Accordingly, it is determined whether or not the image of the eye of the identified person captured by the camera 33 is due to a living body.
【００１３】以下、図面を参照して本発明の実施の形態
を説明する。なお、各図はこの発明を理解出来る程度に
概略的に示してあるにすぎない。また、各図において、
共通する要素については、同一の番号を付し、説明を省
略する。An embodiment of the present invention will be described below with reference to the drawings. It should be noted that the drawings are only schematically shown to the extent that the present invention can be understood. In each figure,
Elements that are common are assigned the same numbers, and descriptions thereof are omitted.
【００１４】＜構成について＞図１は本発明の実施の形
態を示す図であり、図２は本発明の光源の構成を示す図
である。<Structure> FIG. 1 is a diagram showing an embodiment of the present invention, and FIG. 2 is a diagram showing a structure of a light source according to the present invention.
【００１５】１は被識別者の視線を誘導するための複数
の光源となるＬＥＤ群である。本実施例では、ＬＥＤ群
１を、右側に視線を誘導する主ＬＥＤ２と、左側に視線
を誘導する副ＬＥＤ３により構成する。Reference numeral 1 denotes an LED group serving as a plurality of light sources for guiding the line of sight of the person to be identified. In this embodiment, the LED group 1 is composed of a main LED 2 for guiding the line of sight to the right and a sub-LED 3 for guiding the line of sight to the left.
【００１６】＜動作について＞同装置は以下のようにし
て動作する。すなわち、被識別者が読取部３１に近づく
と、制御部３８が図示せぬセンサにより被識別者が接近
したことを検知し、ライト３６を点灯する。またカメラ
３３とディスプレイ３４を起動して、カメラ３３によっ
て窓３２の外の光景を撮影するとともに、撮影した画像
をリアルタイムでディスプレイ３４に表示する。そし
て、被識別者が窓３２から中を覗くことによって、カメ
ラ３３は被識別者の目を撮影し、ディスプレイ３４は被
識別者の目の画像を表示する。このとき、被識別者は、
ディスプレイ３４を見ながら、目が最も鮮明に映るよう
に顔を前後に動かして目の位置をカメラ３３の焦点位置
に合わせ、焦点が合ったらスタートスイッチ３７を押下
する。<Regarding Operation> The device operates as follows. That is, when the identified person approaches the reading unit 31, the control unit 38 detects that the identified object has approached by a sensor (not shown), and turns on the light 36. In addition, the camera 33 and the display 34 are activated, and a scene outside the window 32 is photographed by the camera 33, and the photographed image is displayed on the display 34 in real time. When the identified person looks into the window 32, the camera 33 takes an image of the identified person's eyes, and the display 34 displays an image of the identified person's eyes. At this time, the identified person
While looking at the display 34, the face is moved back and forth so that the eyes appear most clearly, and the position of the eyes is adjusted to the focal position of the camera 33. When the eyes are in focus, the start switch 37 is pressed.
【００１７】すると、制御部３８は所定のタイミングで
ＬＥＤ群１の主ＬＥＤ２と副ＬＥＤ３をランダムに点消
灯させて、被識別者の視線を誘導する（点灯されたＬＥ
Ｄの方に向かせる）。同時に、カメラ３３が撮影した被
識別者の目の画像（動画像）から被識別者の視線の方向
を検出する。制御部３８は、主ＬＥＤ２と副ＬＥＤ３の
点消灯するタイミングとカメラ３３が撮影した画像のタ
イミングを対応付けて管理しており、主ＬＥＤ２と副Ｌ
ＥＤ３のいずれが点灯したときに被識別者の視線がその
方向を向いているのか否かを検証する。これにより、カ
メラ３３が撮影した被識別者の目の画像が生体によるも
のか否かを判断する。なお、被識別者の視線の方向を検
出する方法については後述する。Then, the control unit 38 randomly turns on and off the main LED 2 and the sub LED 3 of the LED group 1 at a predetermined timing to guide the line of sight of the identified person (the illuminated LE).
D). At the same time, the direction of the line of sight of the identified person is detected from the image (moving image) of the eye of the identified person captured by the camera 33. The control unit 38 manages the timing of turning on / off the main LED 2 and the sub LED 3 and the timing of the image captured by the camera 33 in association with each other.
It is verified whether or not the ED 3 is turned on and the line of sight of the identified person is in that direction. Accordingly, it is determined whether or not the image of the eye of the identified person captured by the camera 33 is due to a living body. The method of detecting the direction of the line of sight of the identified person will be described later.
【００１８】制御部３８は、被識別者の視線が誘導した
方向通りに向いていることを検出したら、カメラ３３が
撮影した被識別者の目の画像が生体によるものであると
判断して、カメラ３３が撮影した被識別者の目の画像
（動画像）から静止画像を取得し、認識部３９に出力す
る。識別部３９は被識別者の目の静止画像からアイリス
の模様を抽出し、アイリスデータを生成する。そして、
生成した被識別者のアイリスデータを予め登録されたア
イリスデータと比較して、被識別者が誰であるのか、あ
るいは被識別者が照合しようとする人物本人であるのか
を識別する。When the control unit 38 detects that the line of sight of the identified person is directed in the direction in which it has been guided, the control unit 38 determines that the image of the eye of the identified person captured by the camera 33 is due to the living body. A still image is acquired from the image (moving image) of the eye of the identified person captured by the camera 33 and output to the recognition unit 39. The identification unit 39 extracts an iris pattern from the still image of the eye of the identified person and generates iris data. And
The generated iris data of the identified person is compared with the iris data registered in advance to identify who the identified person is or whether the identified person is the person to be collated.
【００１９】制御部３８は、被識別者の視線が誘導した
方向通りに向いていないことを検出したら、カメラ３３
が撮影した被識別者の目の画像が生体によるものでない
と判断して、アラームを識別部３９または図示しない外
部の機器に出力する。アラームを受信した識別部３９ま
たは図示しない外部の機器は、個人識別を一時中止する
とともに、設備の管理者に警報を発して、以後の処理を
管理者に対応させる。When the control unit 38 detects that the line of sight of the identified person is not directed in the direction in which it has been guided, the camera 33
Determines that the image of the eye of the identified person is not from a living body, and outputs an alarm to the identification unit 39 or an external device (not shown). Upon receiving the alarm, the identification unit 39 or an external device (not shown) temporarily suspends the personal identification and issues an alarm to the facility manager to make subsequent processes correspond to the manager.
【００２０】＜被識別者の視線の方向を検出する方法に
ついて＞同装置は以下のようにして被識別者の視線の方
向を検出する。なお、図４は右目の目尻のテンプレート
を示す図であり、図５は右目の目頭のテンプレートを示
す図、図６は視線の方向を検出する方法を説明する図で
ある。<Method for detecting the direction of the line of sight of the person to be identified> The apparatus detects the direction of the line of sight of the person to be identified as follows. FIG. 4 is a diagram showing a template of the outer corner of the right eye, FIG. 5 is a diagram showing a template of the inner corner of the right eye, and FIG. 6 is a diagram for explaining a method of detecting the direction of the line of sight.
【００２１】４は右目の目尻を検出するためのテンプレ
ートであり、５は右目の目頭を検出するためのテンプレ
ートである。なお、図示しないが、左目用の目尻と目頭
のテンプレートも同様の構成となっている。また、６は
カメラ３３によって撮影された右目の画像、７は右の黒
目（瞳孔とアイリスからなる部分）、８はカメラ３３に
よって撮影された左目の画像、９は左の黒目、１０は右
の黒目の中心と目尻との距離（以下、ＤＲという）、１
１は左の黒目の中心と目尻との距離（以下、ＤＬとい
う）である。Reference numeral 4 denotes a template for detecting the outer corner of the right eye, and reference numeral 5 denotes a template for detecting the inner corner of the right eye. Although not shown, the templates for the left and right outer corners of the eye have the same configuration. Reference numeral 6 denotes an image of the right eye taken by the camera 33, reference numeral 7 denotes a right iris (a portion including a pupil and an iris), reference numeral 8 denotes an image of the left eye taken by the camera 33, reference numeral 9 denotes a left iris, and reference numeral 10 denotes a right iris. Distance between the center of the iris and the outer corner of the eye (hereinafter referred to as DR), 1
1 is the distance between the center of the left iris and the outer corner of the eye (hereinafter, referred to as DL).
【００２２】被識別者が窓３２を覗いてからスタートス
イッチ３７を押下するまでの間に、同装置の制御部３８
は、カメラ３３が撮影した被識別者の目の画像（図６参
照）から左右の黒目の位置を検出する。これは、画像の
中で特に濃度が濃く（黒く）、かつ円形になっている領
域を検出することによりできる。次に、左右の目の輪郭
を切り出す。これは、黒目の周囲の白目とその周囲の瞼
とで濃度が濃くなり（白から黒に変わり）、かつ右目で
あればテンプレート４とテンプレート５（左目であれば
前述の左目用のテンプレート）の形状に近似した形状と
なる領域を切り出すことによりできる。次に、左右の黒
目を切り出すとともに、その中心位置を検出する。これ
は、黒目と白目とで濃度が大きく変わる領域を切り出す
とともに、円形状になっている黒目の外周（すなわち、
アイリスの外周）からその中心を割り出すことによりで
きる。次に、左右の黒目の中心位置からその目尻までの
距離ＤＲとＤＬを算出する。なお、このとき、算出した
距離ＤＲとＤＬを以後ＤＲ０およびＤＬ０という。以上
の動作は、左右同時に行ってもよいし、左右のどちらか
を先に行い、他方を後から行ってもよい。During a period from when the identified person looks through the window 32 to when the start switch 37 is pressed, the control unit 38
Detects the positions of the left and right black eyes from the image of the eye of the identified person taken by the camera 33 (see FIG. 6). This can be achieved by detecting a region having a particularly high density (black) and a circular shape in the image. Next, the outlines of the left and right eyes are cut out. This is because the density of the white eye around the black eye and the eyelids around it becomes darker (changes from white to black), and for the right eye, the template 4 and the template 5 (for the left eye, the template for the left eye described above) This can be achieved by cutting out a region having a shape similar to the shape. Next, the left and right irises are cut out, and the center position is detected. This cuts out a region where the density changes greatly between the iris and the iris, and the outer periphery of the circular iris (ie,
This can be done by calculating its center from the outer periphery of the iris). Next, distances DR and DL from the center positions of the left and right irises to the corners of the eyes are calculated. At this time, the calculated distances DR and DL are hereinafter referred to as DR0 and DL0. The above operation may be performed simultaneously on the left and right, or one of the left and right may be performed first, and the other may be performed later.
【００２３】被識別者がスタートスイッチ３７を押下し
たら、同装置の制御部３８は、所定のタイミングでＬＥ
Ｄ群１の主ＬＥＤ２と副ＬＥＤ３をランダムに点消灯さ
せて、被識別者の視線を点灯されたＬＥＤの方に向かせ
る。すなわち、被識別者の視線を誘導する。このとき、
カメラ３３が被識別者の動きを撮影しており、制御部３
８はカメラ３３が撮影した画像と主ＬＥＤ２と副ＬＥＤ
３の点消灯するタイミングとを対応付けられて管理す
る。そして、制御部３８は、被識別者が窓３２を覗いて
からスタートスイッチ３７を押下するまでの間と同様の
動作を行って、左右の黒目の中心位置からその目尻まで
の距離ＤＲとＤＬを算出する。なお、このとき、算出し
た距離ＤＲとＤＬを以後ＤＲ１およびＤＬ１という。そ
して、制御部３８は、ＤＲ１とＤＲ０およびＤＬ１とＤ
Ｌ０の差分を算出して左右の目の視線がどちらの方向に
動いているのかを検出する。When the discriminator presses the start switch 37, the control unit 38 of the apparatus performs the LE at a predetermined timing.
The main LED 2 and the sub LED 3 of the D group 1 are turned on and off at random, and the line of sight of the identified person is directed toward the lit LED. That is, the gaze of the identified person is guided. At this time,
The camera 33 captures the movement of the identified person, and the control unit 3
8 is an image taken by the camera 33, the main LED2 and the sub LED
The timing of turning on and off the light of No. 3 is managed in association with each other. Then, the control unit 38 performs the same operation as before the distinguished person looks through the window 32 and presses the start switch 37, and determines the distances DR and DL from the center positions of the left and right black eyes to the outer corners of the eyes. calculate. At this time, the calculated distances DR and DL are hereinafter referred to as DR1 and DL1. Then, the control unit 38 determines that DR1 and DR0 and DL1 and D0
The difference of L0 is calculated to detect in which direction the eyes of the left and right eyes are moving.
【００２４】すなわち、“ＤＲ１−ＤＲ０”が０よりも
大きければ右目の視線が被識別者から見て左側に動いた
ことになり、“ＤＲ１−ＤＲ０”が０であれば右目の視
線が動かなかったことになり、“ＤＲ１−ＤＲ０”が０
よりも小さければ右目の視線が被識別者から見て右側に
動いたことになる。同様に、“ＤＬ１−ＤＬ０”が０よ
りも大きければ左目の視線が被識別者から見て右側に動
いたことになり、“ＤＬ１−ＤＬ０”が０であれば左目
の視線が動かなかったことになり、“ＤＬ１−ＤＬ０”
が０よりも小さければ左目の視線が被識別者から見て左
側に動いたことになる。That is, if "DR1-DR0" is larger than 0, the line of sight of the right eye has moved to the left side as viewed from the identified person. If "DR1-DR0" is 0, the line of sight of the right eye has not moved. That is, “DR1−DR0” is 0.
If it is smaller than this, it means that the line of sight of the right eye has moved to the right as viewed from the identified person. Similarly, if “DL1−DL0” is larger than 0, the line of sight of the left eye has moved to the right as viewed from the identified person, and if “DL1−DL0” is 0, the line of sight of the left eye has not moved. "DL1-DL0"
Is smaller than 0, it means that the line of sight of the left eye has moved to the left as viewed from the identified person.
【００２５】制御部３８は、このようにして左右の目の
視線がどちらの方に向いているのかを検出するととも
に、検出した視線の方向が視線を誘導した方向と合致す
るのかを検出する。例えば、被識別者の視線を右側に誘
導した場合、このとき、撮影した目の画像は、“ＤＲ１
−ＤＲ０”が０より小さくなり、しかも“ＤＬ１−ＤＬ
０”が０より大きくなる。また、このとき、ＬＥＤ群１
は主ＬＥＤ２が点灯したことになる。制御部３８はこれ
らの条件を満たしているのか否かを検証する。The control unit 38 detects in which way the line of sight of the left and right eyes is directed, and detects whether the direction of the detected line of sight matches the direction in which the line of sight was guided. For example, when the line of sight of the identified person is guided to the right side, the image of the captured eye at this time is “DR1
−DR0 ”becomes smaller than 0 and“ DL1-DL ”
0 ″ is larger than 0. At this time, the LED group 1
Means that the main LED 2 is turned on. The control unit 38 verifies whether these conditions are satisfied.
【００２６】制御部３８は、被識別者の視線が誘導した
方向通りに向いていることを検出したら、カメラ３３が
撮影した被識別者の目の画像が生体によるものであると
判断して、カメラ３３が撮影した被識別者の目の画像
（動画像）から静止画像を取得し、認識部３９に出力す
る。When the control unit 38 detects that the line of sight of the identified person is directed in the direction in which it has been guided, the control unit 38 determines that the image of the eye of the identified person captured by the camera 33 is due to the living body. A still image is acquired from the image (moving image) of the eye of the identified person captured by the camera 33 and output to the recognition unit 39.
【００２７】制御部３８は、被識別者の視線が誘導した
方向通りに向いていないことを検出したら、カメラ３３
が撮影した被識別者の目の画像が生体によるものでない
と判断して、アラームを識別部３９または図示しない外
部の機器に出力する。When the control unit 38 detects that the line of sight of the person to be identified is not directed in the guided direction, the camera 33
Determines that the image of the eye of the identified person is not from a living body, and outputs an alarm to the identification unit 39 or an external device (not shown).
【００２８】なお、本実施例では、両目とも正常な人で
あれば左右の目の視線がともに同じ方向を向く特性を利
用するために、左右の目の視線がともに同じ方向（誘導
する方向）に向いているのか否かを検証するようにし
た。しかしながら、片方の目しか正常でない人の場合は
左右の目のどちらかしか動かない。そこで、このような
場合を想定して以下のように構成することもできる。In this embodiment, if both eyes are normal, the left and right eyes have the same direction (guiding direction) in order to use the characteristic that both eyes have the same direction. It was made to verify whether it was suitable for. However, in a case where only one eye is normal, only one of the left and right eyes moves. Therefore, the following configuration can be adopted assuming such a case.
【００２９】すなわち、制御部３８が動いた方の目が正
しく誘導した方向に向いたか否かを検証し、誘導した方
向通りに向いていることを検出したら、動いた方の目に
基づいて認識部３９で個人識別を行う。その際に、例え
ば被識別者が片方の目しか正常でない旨のデータが認識
部３９または図示しない外部の機器に予め登録されてい
たら、個人識別の結果をもって識別処理を終了する。動
いた方の目が誘導した方向通りに向いていないことを検
出したり、被識別者が片方の目しか正常でない旨のデー
タが登録されていないときには、アラームを識別部３９
または図示しない外部の機器に出力する。そして、アラ
ームを受信した識別部３９または図示しない外部の機器
は、個人識別を一時中止するとともに、設備の管理者に
警報を発して、以後の処理を管理者に対応させる。That is, the control unit 38 verifies whether or not the moving eye is directed correctly in the direction in which it has been guided. If it is detected that the eye is directed in the guided direction, the recognition is performed based on the moving eye. The unit 39 performs personal identification. At this time, if, for example, data indicating that the identified person has only one eye is normal in the recognition unit 39 or an external device (not shown), the identification process ends with the result of personal identification. If it is detected that the moving eye is not directed in the direction in which it has been guided, or if data indicating that only one of the eyes of the identified person is normal is not registered, an alarm is issued to the identification unit 39.
Alternatively, the data is output to an external device (not shown). Then, the identification unit 39 or the external device (not shown) that has received the alarm temporarily suspends the personal identification and issues an alarm to the facility manager to make the subsequent processing correspond to the manager.
【００３０】なお、本発明は本実施例に限定されること
なく、本発明の要旨を逸脱しない範囲で種々の応用及び
変形が考えられる。例えば、本実施例では、ＬＥＤ群１
をディスプレイ３４の表示面上に設置したが、ディスプ
レイ３４の周囲に設置することも可能である。また、Ｌ
ＥＤ群１を、左右２つのＬＥＤにより構成したが、上下
方向や斜め方向に配置したＬＥＤにより構成することも
可能である。さらに、ＬＥＤ群１を用いなくても、被識
別者の目の画像と光を発する画像とを合成してディスプ
レイ３４に表示するようにすることも可能である。本発
明はこれらの構成を含む。The present invention is not limited to the present embodiment, and various applications and modifications can be considered without departing from the gist of the present invention. For example, in this embodiment, the LED group 1
Is installed on the display surface of the display 34, but may be installed around the display 34. Also, L
Although the ED group 1 is configured by two LEDs on the left and right, the ED group 1 may be configured by LEDs arranged in a vertical direction or an oblique direction. Further, without using the LED group 1, it is also possible to combine the image of the eye of the identified person with the image emitting light and display the image on the display 34. The present invention includes these configurations.
【００３１】[0031]
【発明の効果】以上説明した本発明には、次の効果があ
る。すなわち、本発明は被識別者の生体反応を検出して
いるので、カメラによって取得した目の画像が生体によ
るものか否かを判断でき、生体によるものと判断できる
場合にのみ個人識別を行いえる。そのため、第三者が他
人になりすますことを検出でき、様々な不正を防止する
ことができる。The present invention described above has the following effects. That is, since the present invention detects the biological reaction of the identified person, it is possible to determine whether or not the image of the eye acquired by the camera is due to a living body, and to perform personal identification only when it can be determined that the image is due to a living body. . Therefore, it is possible to detect that a third party impersonates another person, and it is possible to prevent various frauds.
【図１】本発明の実施の形態を示す図である。FIG. 1 is a diagram showing an embodiment of the present invention.
【図２】光源の構成を示す図である。FIG. 2 is a diagram showing a configuration of a light source.
【図３】従来装置の構成を示す図である。FIG. 3 is a diagram showing a configuration of a conventional device.
【図４】右目の目尻のテンプレートを示す図である。FIG. 4 is a diagram showing a template of the outer corner of the right eye.
【図５】右目の目頭のテンプレートを示す図である。FIG. 5 is a diagram showing a template of the inner corner of the right eye.
【図６】視線の方向を検出する方法を説明する図であ
る。FIG. 6 is a diagram illustrating a method of detecting the direction of a line of sight.
１ ＬＥＤ群 ３１ 読取部 ３２ 窓 ３３ カメラ ３４ ディスプレイ ３５ ハーフミラー ３６ ライト ３７ スタートスイッチ ３８ 制御部 ３９ 識別部 1 LED group 31 reading unit 32 window 33 camera 34 display 35 half mirror 36 light 37 start switch 38 control unit 39 identification unit
Claims (3)
と、 個別に点消灯を制御することができ、被識別者の視線を
誘導するための複数の光源と、 被識別者の視線の方向を検出する検出手段とを有し、 前記複数の光源を点消灯させながら前記撮影手段により
被識別者の目の画像を取得し、取得した画像から前記検
出手段により被識別者の視線の方向を検出することを特
徴とするアイリスを用いた個人識別装置。1. A photographing means for acquiring an image of an eye of a distinguished person, a plurality of light sources capable of individually controlling turning on / off and guiding a line of sight of the distinguished person; Detecting means for detecting the direction of the eye, acquiring an image of the eye of the identified person by the photographing means while turning on and off the plurality of light sources, from the acquired image of the line of sight of the identified person by the detecting means A personal identification device using an iris characterized by detecting a direction.
場合に、取得した目の画像に基づいて個人識別を行うこ
とを特徴とする請求項１に記載のアイリスを用いた個人
識別装置。2. The personal identification device using an iris according to claim 1, wherein the personal identification is performed based on the acquired image of the eye when the line of sight of the identified person moves in a guiding direction. .
の方向を検出することができ、 被識別者の両目の視線が誘導する方向に動いた場合に、
取得した目の画像に基づいて個人識別を行うことを特徴
とする請求項１に記載のアイリスを用いた個人識別装
置。3. The detection means can detect the direction of the line of sight of the identified person's eyes, and when the direction of the line of sight of the identified person's eyes moves,
The personal identification device using an iris according to claim 1, wherein personal identification is performed based on the acquired eye image.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| JP25645298A JP3562970B2 (en) | 1998-09-10 | 1998-09-10 | Biological identification device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| JP25645298A JP3562970B2 (en) | 1998-09-10 | 1998-09-10 | Biological identification device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| JP2000083930A true JP2000083930A (en) | 2000-03-28 |
| JP3562970B2 JP3562970B2 (en) | 2004-09-08 |
Family
ID=17292854
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| JP25645298A Expired - Fee Related JP3562970B2 (en) | 1998-09-10 | 1998-09-10 | Biological identification device |
Country Status (1)
| Country | Link |
|---|---|
| JP (1) | JP3562970B2 (en) |
Cited By (31)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2000298727A (en) * | 1999-03-23 | 2000-10-24 | Lg Electronics Inc | Method for discriminating forgery in iris recognition system |
| JP2001297322A (en) * | 2000-04-12 | 2001-10-26 | Oki Electric Ind Co Ltd | Guide unit |
| JP2002092605A (en) * | 2000-09-18 | 2002-03-29 | Oki Electric Ind Co Ltd | Iris recognition device |
| WO2006052004A1 (en) * | 2004-11-11 | 2006-05-18 | Matsushita Electric Industrial Co., Ltd. | Iris authentication device |
| US7068820B2 (en) | 2001-10-31 | 2006-06-27 | Matsushita Electric Industrial Co., Ltd. | Iris image pickup apparatus and iris authentication apparatus |
| JP2006172209A (en) * | 2004-12-16 | 2006-06-29 | Advanced Telecommunication Research Institute International | Apparatus for sight line direction estimation, method for sight line direction estimation, and program for executing the method for sight line direction estimation on computer |
| JP2007527270A (en) * | 2003-07-16 | 2007-09-27 | オムニパーセプション リミティド | Biological face recognition system |
| JP2008000464A (en) * | 2006-06-23 | 2008-01-10 | Glory Ltd | Authentication device and authentication method |
| US7761453B2 (en) | 2005-01-26 | 2010-07-20 | Honeywell International Inc. | Method and system for indexing and searching an iris image database |
| US7933507B2 (en) | 2006-03-03 | 2011-04-26 | Honeywell International Inc. | Single lens splitter camera |
| US8045764B2 (en) | 2005-01-26 | 2011-10-25 | Honeywell International Inc. | Expedient encoding system |
| US8050463B2 (en) | 2005-01-26 | 2011-11-01 | Honeywell International Inc. | Iris recognition system having image quality metrics |
| US8049812B2 (en) | 2006-03-03 | 2011-11-01 | Honeywell International Inc. | Camera with auto focus capability |
| US8064647B2 (en) | 2006-03-03 | 2011-11-22 | Honeywell International Inc. | System for iris detection tracking and recognition at a distance |
| US8063889B2 (en) | 2007-04-25 | 2011-11-22 | Honeywell International Inc. | Biometric data collection system |
| US8085993B2 (en) | 2006-03-03 | 2011-12-27 | Honeywell International Inc. | Modular biometrics collection system architecture |
| US8090246B2 (en) | 2008-08-08 | 2012-01-03 | Honeywell International Inc. | Image acquisition system |
| US8090157B2 (en) | 2005-01-26 | 2012-01-03 | Honeywell International Inc. | Approaches and apparatus for eye detection in a digital image |
| US8098901B2 (en) | 2005-01-26 | 2012-01-17 | Honeywell International Inc. | Standoff iris recognition system |
| US8213782B2 (en) | 2008-08-07 | 2012-07-03 | Honeywell International Inc. | Predictive autofocusing system |
| US8280119B2 (en) | 2008-12-05 | 2012-10-02 | Honeywell International Inc. | Iris recognition system using quality metrics |
| US8285005B2 (en) | 2005-01-26 | 2012-10-09 | Honeywell International Inc. | Distance iris recognition |
| US8436907B2 (en) | 2008-05-09 | 2013-05-07 | Honeywell International Inc. | Heterogeneous video capturing system |
| US8442276B2 (en) | 2006-03-03 | 2013-05-14 | Honeywell International Inc. | Invariant radial iris segmentation |
| US8472681B2 (en) | 2009-06-15 | 2013-06-25 | Honeywell International Inc. | Iris and ocular recognition system using trace transforms |
| US8630464B2 (en) | 2009-06-15 | 2014-01-14 | Honeywell International Inc. | Adaptive iris matching using database indexing |
| US8705808B2 (en) | 2003-09-05 | 2014-04-22 | Honeywell International Inc. | Combined face and iris recognition system |
| WO2014077447A1 (en) * | 2012-11-19 | 2014-05-22 | 아이리텍 인크 | Method and apparatus for identifying living eye |
| US8742887B2 (en) | 2010-09-03 | 2014-06-03 | Honeywell International Inc. | Biometric visitor check system |
| WO2016059786A1 (en) * | 2014-10-15 | 2016-04-21 | 日本電気株式会社 | Impersonation detection device, impersonation detection method, and recording medium |
| CN109300145A (en) * | 2018-08-20 | 2019-02-01 | 彭楷文 | NEW ADAPTIVE intelligence dazzle system |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2014206932A (en) | 2013-04-15 | 2014-10-30 | オムロン株式会社 | Authentication device, authentication method, control program, and recording medium |
-
1998
- 1998-09-10 JP JP25645298A patent/JP3562970B2/en not_active Expired - Fee Related
Cited By (44)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2000298727A (en) * | 1999-03-23 | 2000-10-24 | Lg Electronics Inc | Method for discriminating forgery in iris recognition system |
| JP2001297322A (en) * | 2000-04-12 | 2001-10-26 | Oki Electric Ind Co Ltd | Guide unit |
| JP2002092605A (en) * | 2000-09-18 | 2002-03-29 | Oki Electric Ind Co Ltd | Iris recognition device |
| JP4529263B2 (en) * | 2000-09-18 | 2010-08-25 | 沖電気工業株式会社 | Iris recognition device |
| US7068820B2 (en) | 2001-10-31 | 2006-06-27 | Matsushita Electric Industrial Co., Ltd. | Iris image pickup apparatus and iris authentication apparatus |
| JP2007527270A (en) * | 2003-07-16 | 2007-09-27 | オムニパーセプション リミティド | Biological face recognition system |
| US8705808B2 (en) | 2003-09-05 | 2014-04-22 | Honeywell International Inc. | Combined face and iris recognition system |
| WO2006052004A1 (en) * | 2004-11-11 | 2006-05-18 | Matsushita Electric Industrial Co., Ltd. | Iris authentication device |
| JP4687097B2 (en) * | 2004-12-16 | 2011-05-25 | 株式会社国際電気通信基礎技術研究所 | Gaze direction estimation apparatus, gaze direction estimation method, and program for causing computer to execute gaze direction estimation method |
| JP2006172209A (en) * | 2004-12-16 | 2006-06-29 | Advanced Telecommunication Research Institute International | Apparatus for sight line direction estimation, method for sight line direction estimation, and program for executing the method for sight line direction estimation on computer |
| US7761453B2 (en) | 2005-01-26 | 2010-07-20 | Honeywell International Inc. | Method and system for indexing and searching an iris image database |
| US8285005B2 (en) | 2005-01-26 | 2012-10-09 | Honeywell International Inc. | Distance iris recognition |
| US8045764B2 (en) | 2005-01-26 | 2011-10-25 | Honeywell International Inc. | Expedient encoding system |
| US8050463B2 (en) | 2005-01-26 | 2011-11-01 | Honeywell International Inc. | Iris recognition system having image quality metrics |
| US8090157B2 (en) | 2005-01-26 | 2012-01-03 | Honeywell International Inc. | Approaches and apparatus for eye detection in a digital image |
| US8098901B2 (en) | 2005-01-26 | 2012-01-17 | Honeywell International Inc. | Standoff iris recognition system |
| US8488846B2 (en) | 2005-01-26 | 2013-07-16 | Honeywell International Inc. | Expedient encoding system |
| US7933507B2 (en) | 2006-03-03 | 2011-04-26 | Honeywell International Inc. | Single lens splitter camera |
| US8761458B2 (en) | 2006-03-03 | 2014-06-24 | Honeywell International Inc. | System for iris detection, tracking and recognition at a distance |
| US8085993B2 (en) | 2006-03-03 | 2011-12-27 | Honeywell International Inc. | Modular biometrics collection system architecture |
| US8064647B2 (en) | 2006-03-03 | 2011-11-22 | Honeywell International Inc. | System for iris detection tracking and recognition at a distance |
| US8442276B2 (en) | 2006-03-03 | 2013-05-14 | Honeywell International Inc. | Invariant radial iris segmentation |
| US8049812B2 (en) | 2006-03-03 | 2011-11-01 | Honeywell International Inc. | Camera with auto focus capability |
| JP2008000464A (en) * | 2006-06-23 | 2008-01-10 | Glory Ltd | Authentication device and authentication method |
| US8063889B2 (en) | 2007-04-25 | 2011-11-22 | Honeywell International Inc. | Biometric data collection system |
| US8436907B2 (en) | 2008-05-09 | 2013-05-07 | Honeywell International Inc. | Heterogeneous video capturing system |
| US8213782B2 (en) | 2008-08-07 | 2012-07-03 | Honeywell International Inc. | Predictive autofocusing system |
| US8090246B2 (en) | 2008-08-08 | 2012-01-03 | Honeywell International Inc. | Image acquisition system |
| US8280119B2 (en) | 2008-12-05 | 2012-10-02 | Honeywell International Inc. | Iris recognition system using quality metrics |
| US8472681B2 (en) | 2009-06-15 | 2013-06-25 | Honeywell International Inc. | Iris and ocular recognition system using trace transforms |
| US8630464B2 (en) | 2009-06-15 | 2014-01-14 | Honeywell International Inc. | Adaptive iris matching using database indexing |
| US8742887B2 (en) | 2010-09-03 | 2014-06-03 | Honeywell International Inc. | Biometric visitor check system |
| US10152792B2 (en) | 2012-11-19 | 2018-12-11 | Iritech, Inc. | Method and apparatus for identifying living eye |
| WO2014077447A1 (en) * | 2012-11-19 | 2014-05-22 | 아이리텍 인크 | Method and apparatus for identifying living eye |
| KR101417415B1 (en) * | 2012-11-19 | 2014-07-08 | 아이리텍 잉크 | Method and apparatus to detect live eye |
| JP2016504660A (en) * | 2012-11-19 | 2016-02-12 | イリテック インコーポレイテッド | Method and apparatus for identifying living human eyes |
| WO2016059786A1 (en) * | 2014-10-15 | 2016-04-21 | 日本電気株式会社 | Impersonation detection device, impersonation detection method, and recording medium |
| JPWO2016059786A1 (en) * | 2014-10-15 | 2017-08-03 | 日本電気株式会社 | Spoofing detection device, spoofing detection method, and spoofing detection program |
| US10515262B2 (en) | 2014-10-15 | 2019-12-24 | Nec Corporation | Spoofing detection device, spoofing detection method, and recording medium |
| JP2020194608A (en) * | 2014-10-15 | 2020-12-03 | 日本電気株式会社 | Living body detection device, living body detection method, and living body detection program |
| US11210500B2 (en) | 2014-10-15 | 2021-12-28 | Nec Corporation | Spoofing detection device, spoofing detection method, and recording medium |
| US11699302B2 (en) | 2014-10-15 | 2023-07-11 | Nec Corporation | Spoofing detection device, spoofing detection method, and recording medium |
| US12051271B2 (en) | 2014-10-15 | 2024-07-30 | Nec Corporation | Spoofing detection device, spoofing detection method, and recording medium |
| CN109300145A (en) * | 2018-08-20 | 2019-02-01 | 彭楷文 | NEW ADAPTIVE intelligence dazzle system |
Also Published As
| Publication number | Publication date |
|---|---|
| JP3562970B2 (en) | 2004-09-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP3562970B2 (en) | Biological identification device | |
| WO2006088042A1 (en) | Biometric discrimination device, authentication device, and biometric discrimination method | |
| KR100601963B1 (en) | Authentication apparatus and method using eye gaze | |
| CN104781830B (en) | The recognition methods of living person's eyes and identification device | |
| US20050084137A1 (en) | System and method for iris identification using stereoscopic face recognition | |
| US20020131622A1 (en) | Apparatus and method for adjusting focus position in iris recognition system | |
| EP2005888A2 (en) | Imaging device and authentication device using the same | |
| JPWO2009016846A1 (en) | Iris authentication device and iris authentication system | |
| JP3879719B2 (en) | Image input device and authentication device using the same | |
| JP2005149515A (en) | Apparatus and method for human distinction using infrared light | |
| CN110135370B (en) | Method and device for detecting living human face, electronic equipment and computer readable medium | |
| KR102160137B1 (en) | Apparatus and Method for Recognizing Fake Face By Using Minutia Data Variation | |
| WO2006013678A1 (en) | Living body determination device, authentication device using the device, and living body determination method | |
| CN108140114A (en) | Iris recognition | |
| JP2007135149A (en) | Mobile portable terminal | |
| JP2007319175A (en) | Photographing device and authentication device using the same | |
| JP3808014B2 (en) | Eye image pickup device and individual authentication device | |
| JP2006136450A (en) | Iris certification device | |
| JP2006212185A (en) | Living body discrimination device, authentication apparatus and living body discrimination method | |
| JPH11191856A (en) | Device and method for picking up image | |
| JPH04174309A (en) | Driver's eye position detecting apparatus and condition detecting apparatus | |
| JP2006318374A (en) | Glasses determination device, authentication device, and glasses determination method | |
| JP2008134757A (en) | Personal authentication apparatus, and personal authenticating method | |
| KR20040006703A (en) | Iris recognition system | |
| US11948402B2 (en) | Spoof detection using intraocular reflection correspondences |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| TRDD | Decision of grant or rejection written | ||
| A01 | Written decision to grant a patent or to grant a registration (utility model) |
Free format text: JAPANESE INTERMEDIATE CODE: A01 Effective date: 20040525 |
|
| A61 | First payment of annual fees (during grant procedure) |
Free format text: JAPANESE INTERMEDIATE CODE: A61 Effective date: 20040601 |
|
| R150 | Certificate of patent or registration of utility model |
Free format text: JAPANESE INTERMEDIATE CODE: R150 |
|
| FPAY | Renewal fee payment (event date is renewal date of database) |
Free format text: PAYMENT UNTIL: 20090611 Year of fee payment: 5 |
|
| FPAY | Renewal fee payment (event date is renewal date of database) |
Free format text: PAYMENT UNTIL: 20090611 Year of fee payment: 5 |
|
| FPAY | Renewal fee payment (event date is renewal date of database) |
Free format text: PAYMENT UNTIL: 20100611 Year of fee payment: 6 |
|
| FPAY | Renewal fee payment (event date is renewal date of database) |
Free format text: PAYMENT UNTIL: 20100611 Year of fee payment: 6 |
|
| FPAY | Renewal fee payment (event date is renewal date of database) |
Free format text: PAYMENT UNTIL: 20110611 Year of fee payment: 7 |
|
| FPAY | Renewal fee payment (event date is renewal date of database) |
Free format text: PAYMENT UNTIL: 20110611 Year of fee payment: 7 |
|
| FPAY | Renewal fee payment (event date is renewal date of database) |
Free format text: PAYMENT UNTIL: 20120611 Year of fee payment: 8 |
|
| FPAY | Renewal fee payment (event date is renewal date of database) |
Free format text: PAYMENT UNTIL: 20130611 Year of fee payment: 9 |
|
| LAPS | Cancellation because of no payment of annual fees |