JP6759065B2 - Blood vessel imaging device and personal authentication system - Google Patents

Description

The present invention relates to a device that captures a blood vessel image of a user and a system that performs personal authentication using the captured blood vessel image.

Patent Document 1 acquires an opening formed on the surface of a housing, a plurality of light sources arranged on the side of the opening and arranged in a grid pattern, and position information of a hand presented on the opening. A light amount control unit that selects an irradiation light source to irradiate the hand from a plurality of light sources based on a sensor and position information and controls the amount of light of the irradiation light source, and a hand irradiated with light from the irradiation light source. A blood vessel imaging apparatus including an imaging unit that captures an image of a blood vessel included in a finger portion of the light source is disclosed.

WO2016 / 0842414

In Patent Document 1, since the blood vessels of a plurality of fingers are photographed at the same time, the main direction of the hand, which is the direction from the wrist side of the arm to the fingertip, and the light irradiation direction from the light source are close to parallel when the hand is closed. The light source is arranged so as to. However, in the open posture of the finger, the main direction of the finger is not parallel to the irradiation direction of the light source. At this time, since the imaging unit directly receives the light emitted and reflected on the finger side surface, a brightness saturation (halation) region is generated on the finger side surface of the finger blood vessel image, and a clear blood vessel image cannot be obtained.

In order to solve the above problems, the vascular imaging apparatus disclosed in the present invention includes an opening formed on the surface of the housing, a plurality of light sources arranged on the side of the opening, and a finger presented on the opening. A sensor that acquires the position information and the main direction information of the finger, and an irradiation light source that irradiates the finger based on the position information and the main direction information are selected from a plurality of light sources, and the amount of light of the selected irradiation light source is selected. It includes a light amount control unit for controlling and an imaging unit for capturing an image of a blood vessel existing inside a finger irradiated with light from an irradiation light source.

Even in the posture in which the finger is open, the main direction of the finger and the irradiation direction of the light source are close to parallel, so that the halation region is reduced and a clear blood vessel image can be obtained.

The figure which shows the structure of the personal authentication system. The figure which shows the appearance and the internal structure of the blood vessel imaging apparatus. The figure for demonstrating the halation area. The figure which shows the image when the finger is opened and closed at the time of taking a blood vessel image. The figure which shows the appearance when the blood vessel imaging apparatus is configured by using a plurality of light source units. The figure which shows the appearance when the blood vessel image taking apparatus is constructed using the light refraction part. The figure which shows the structure of the blood vessel imaging apparatus in consideration of the rotation of a hand. The figure which shows the flowchart of the process performed by a personal authentication system. The figure which shows the flowchart of the process which feedback-controls the amount of light. The figure which shows the mechanism for miniaturizing the blood vessel imaging apparatus. The figure which shows the appearance when the blood vessel imaging apparatus is miniaturized. The figure which looked at the blood vessel imaging apparatus from the side of the opening. The figure which shows the structure of the blood vessel imaging apparatus in consideration of the rotation of a hand. The figure which shows the modification of the light source arrangement in the blood vessel imaging apparatus. The figure which shows the image at the time of acquiring the blood vessel image by the imaging timing control. The figure which shows the flowchart of the process which acquires the blood vessel image using a plurality of frames. The figure which shows the device which it is easy for a user to present a hand with a finger open and non-contact. The figure which shows the device which it is easy for a user to present a hand with a finger closed and non-contact. The figure which shows the device which it is easy for a user to present a hand with a finger closed and non-contact. The figure which shows the modification of the opening 3. The figure which shows the device which is easy to make contact with the presentation surface when a user presents a hand. The figure which shows the device which makes it easy for a user to present a hand to the presentation surface in a non-contact manner when presenting a hand.

In this embodiment, a configuration will be described in which a clear finger blood vessel image is taken and authentication is performed by controlling the light source according to the opening / closing posture of the finger to be presented and the rotation posture of the hand. The blood vessel imaging device described in this embodiment can be used not only when capturing data at the time of authentication but also when capturing data at the time of registration.

FIG. 1 is a configuration diagram of a personal authentication system. An opening 3 is provided on the surface of the housing of the blood vessel imaging device 2 so that the hand 1 can be presented above the opening 3 of the blood vessel imaging device 2 when taking a blood vessel image of a finger. The distance sensor 4 arranged inside the housing below the opening 3 receives the light that has passed through the optical filter 12 for distance measurement, converts the received light into an electric signal, and hands through the data input unit 50. It is taken into the computer 5 as the distance data between 1 and the distance sensor. The CPU 7 calculates the position of the hand 1, the posture of the hand 1, the position of the finger, the posture of the finger, and the like by the program stored in the memory 6 based on the distance data taken into the computer 5. The light source control unit 51 controls the light source array 9 arranged inside the opening 8 based on the calculation result, selects an irradiation light source from a plurality of point light sources 10 constituting the light source array 9, and performs irradiation. Determine the light intensity value of the light source. Then, each point light source irradiates the finger with light corresponding to the light amount value thus determined. The imaging unit 11 arranged below the opening 3 receives the light that has passed through the optical filter 12. The received light is converted into an electric signal by the image pickup unit 11, and is taken into the computer 5 as an image via the image input unit 52.

The captured image is once stored in the memory 6. Then, the CPU 7 collates the image stored in the memory with the image stored in the storage device 14 in advance by the program stored in the memory 6 and authenticates the image.

In this embodiment, personal authentication will be described as an example of executing personal authentication in the same housing as the blood vessel imaging device, but the authentication processing may be executed in an authentication unit such as a server installed outside the housing. That is, the housing itself may not be subjected to the authentication process, and the information of the captured blood vessel image may be transmitted to the authentication unit such as a server.

Further, in the calculation of the position and posture of the hand 1, the position and posture of the hand may be detected by using the image of the hand taken by the imaging unit 11, or the image of the hand 1 taken by the imaging unit 11 and the hand 1 may be calculated. Both the distance data between and the distance sensor 4 may be used.

Further, as a means for notifying the user of the authentication result, the voice of the speaker 15 may be used, or the authentication result may be displayed on the display unit 16.

Further, a visible light source 17 is provided in the opening 3, and different colors of light are emitted to the user at the time of standby, when a hand is detected, at the time of authentication processing, at the time of successful authentication, at the time of authentication failure, etc. You may inform the state of.

Further, the registrant data to be collated may be narrowed down from a large number of registrant data by having the user ID input unit 18 input a password or ID or read an IC chip in the stage before authentication. .. This narrowing down has the effect of improving the image search speed and authentication accuracy. In particular, when the authentication target can be uniquely specified by narrowing down, it is called 1: 1 authentication, and the above-mentioned effect is further improved.

The opening 3 uses a material that transmits the irradiation light from each point light source 10. For example, a transparent member such as acrylic or glass is assumed. Further, a film that allows only near-infrared light to pass may be attached to the opening 3 made of these transparent members. In this way, the inside of the device can be made invisible to the user.

FIG. 2 is a diagram showing the appearance and internal structure of the blood vessel imaging apparatus. As shown in FIG. 2, the back side of the opening 3 has a radial arc, and the light source array 9 is arranged perpendicular to the isolated side. The matters described with reference to FIG. 1 will be omitted.

FIG. 3 is a diagram for explaining a halation region. As described above, when the light source array 9 is arranged in a straight line behind the opening 3, when the hand 1 is presented with the fingers open, the main direction 24 of the fingers is the direction from the base of the fingers to the tips of the fingers. And the irradiation direction 22 of the light source are not parallel. Therefore, the light radiated to the side surface of the finger is directly reflected, and the light is received by the imaging unit, so that the halation region 25 is generated on the side surface of the finger in the finger blood vessel image. As a result, as described above, a clear blood vessel image cannot be obtained.

FIG. 4 is a diagram showing an image when a finger is opened and closed during blood vessel imaging. When the hand is presented in front of the light source array 9, the main direction 23 of the hand, which is the direction from the wrist side of the arm to the fingertip, and the irradiation light direction 22 of the light source become parallel, and the back side of each finger is irradiated with light to form a blood vessel. Images can be acquired.

The main direction 24 of each finger of the hand to be presented is concentrated on a point near the wrist when going from the fingertip toward the finger base. This point near the wrist is referred to as the center point 26. The position of the center point 26 near the wrist where the main directions 24 of the fingers are concentrated does not change significantly regardless of whether the finger opening is large as shown in FIG. 4a or the finger opening is small as shown in FIG. 4b. .. Therefore, by arranging the point light sources 10 radially around the center point 26 near the wrist, even if the opening / closing posture of the finger fluctuates, the irradiation light direction 22 of the point light source 10 and the main direction of each finger are always present. 24 becomes close to parallel. As a result, the amount of light directly applied to the side surface of the finger is reduced, the expansion of the halation region 25 on the side surface of the finger can be suppressed, and the blood vessel image can be clearly taken.

FIG. 5 is a diagram showing a blood vessel imaging device when a plurality of light source units are used. In the example of FIG. 5, the light source array 9 is configured by arranging light source units 20 formed by arranging a plurality of light sources in a grid pattern in a radial pattern centered on an opening 3 or a hand presenting portion 19. ..

Although this example is an example using three light source units 20 having different irradiation angles, the number of light source units 20 may be two or four or more. As the number of light source units increases, the shape of the light source units 9 approaches radial, and the above-mentioned halation reduction effect increases.

FIG. 6 is a diagram showing a blood vessel imaging device when a light refracting portion is used. As shown in FIG. 6, a plurality of light sources may be arranged in a grid pattern, and the irradiation light direction 22 may be refracted by a light refracting portion 21 such as a prism to realize radial irradiation light. Even in this case, the halation reduction effect described above can be increased as compared with the case where the light refraction unit 21 is not installed.

FIG. 7 is a diagram showing a configuration of a blood vessel imaging device when the rotation of the hand is taken into consideration. By expanding the range of the light sources arranged on the circumference in this way, it is possible to acquire a clear blood vessel image of the finger not only when the posture changes when the finger is open but also when the rotation of the hand changes. At this time, by making the width of the wrist side narrower than the width of the fingertip side of the hand presented by the opening 3, the presentation position of the wrist or hand is guided to the center point 26 where the irradiation light directions of the plurality of point light sources 10 are gathered. It also has the effect of

Further, the presentation position of the hand 1 may be guided to the presentation unit 19 by providing the presentation unit 19 of the hand 1. As a result, even when the user's hand rotates, the rotation axis is close to the presentation unit 19, so that a clear finger blood vessel image can be obtained even when the rotation fluctuates.

FIG. 8 is a diagram showing a flowchart of processing performed by the authentication system. Prior to the description of FIG. 8, a method of controlling a plurality of point light sources 10 constituting the light source array 9 will be described. First, the reason why the light source array 9 needs to be controlled in the imaging of the finger blood vessels in the blood vessel imaging apparatus 2 will be described.

When all the point light sources 10 constituting the light source array 9 are turned on when taking a blood vessel image, not all of the irradiation light is applied to the finger. That is, the light that is not applied to the finger enters the device through the opening 3, is diffusely reflected in the device, and then is received by the imaging unit 11. This causes noise in the blood vessel image. Further, when all the point light sources 10 are turned on, even if the finger is irradiated with light, the irradiation light direction 22 of some point light sources and the main direction 24 of the finger are not parallel to each other. As a result, the light is strongly irradiated to the side surface of the finger, and a large halation region 25 is generated in the side surface region of the finger in the blood vessel image of the finger. This causes a decrease in the image quality of the captured blood vessel image.

Therefore, in order to suppress the generation of the halation region 25 on the side surface of the finger and to take a clear blood vessel image, the irradiation light direction 22 of the plurality of point light sources 10 constituting the light source array 9 and the main direction 24 of each finger are used. Need to be close to parallel.

Further, if the amount of light emitted by the point light source 10 is too large, the amount of light transmitted through the finger becomes too large, and the proportion of the halation region 25 in the entire area of the finger increases. On the other hand, if the amount of light is too small, sufficient light does not pass through the finger, and the contrast between the blood vessel portion and the region other than the blood vessel becomes small, resulting in an unclear blood vessel image of the finger being taken.

That is, it is necessary to select the point light source 10 to be turned on according to the main direction 24 of the finger in the light source array 9 and to control the amount of irradiation light thereof.

The flowchart of FIG. 8 will be described below. Step 101 is an operation in which the certifier presents a hand on the upper part of the opening 3. Step 102 is a process of detecting a hand by using a distance image or the like acquired by the distance sensor 4. Step 103 is a process of determining whether or not a hand has been detected. If no hand is detected here, the process returns to step 102, and if a hand is detected, the process proceeds to step 104. The step 104 is an initial light source control process, that is, a process of lighting the point light source 10 that irradiates the detected fingers of the hand with light at the initial light amount value. Step 105 is a process of acquiring a distance image by the distance sensor 4 and acquiring a near infrared image by the imaging unit 11. Step 106 is a process of detecting the position and posture of the finger by using the distance image and the near infrared image. Specifically, the position information and the posture information based on the three-dimensional shape of the finger are detected and acquired. In step 107, the light source array 9 is controlled according to the position of the finger and the posture of the finger detected in step 106, the point light source 10 to be turned on for photographing the finger blood vessel is selected, and the light intensity value of the point light source 10 to be turned on. Is the process of determining. Details of step 107 will be described later. Step 108 is a normalization process for correcting distortion due to a change in finger posture in the acquired finger blood vessel image. Step 109 is a process of extracting blood vessel features from the finger blood vessel image after the normalization process. Step 110 is a process of collating the extracted blood vessel features with the blood vessel features registered in the storage device 14 to calculate a collation score. Step 111 is a process of comparing the calculated collation score with the predetermined value TH1. If the collation score is larger than the predetermined threshold value TH1, the process proceeds to step 112 to perform post-authentication successful processing, and the process proceeds to step 114 to end the authentication flow. On the other hand, if the collation score is TH1 or less, the process proceeds to step 113 to determine the authentication timeout. If the timeout time has not elapsed, the process returns to step 105 and the authentication process is repeated. On the other hand, when the timeout time has elapsed, the process proceeds to step 114 to end the authentication flow.

Next, the details of step 107 will be described. In step 107, first, each finger of the hand presented in the opening 3 of the blood vessel imaging device 2 is detected, and the position in the three-dimensional space and the main direction of the finger are obtained for each finger. The main direction of the finger can be, for example, the direction of an approximate straight line of the center line passing near the skeleton of each finger obtained from distance data or a near-infrared image. Further, the direction of the approximate straight line of the finger contour line may be the main direction of the finger. Next, the point light source 10 to be turned on is determined according to the obtained finger position and the main direction of the finger. At this time, it is preferable to select a light source in which the angle generated by the irradiation light direction 22 and the main direction 24 of each finger is closest to 0 degrees. As a result, even if there is a change in the position of the finger, a change in the opening / closing posture of the finger, and a change in the rotational posture of the hand, the light is irradiated so as to be parallel to the main direction of each finger, and a clear finger blood vessel image is obtained. Can be obtained. Further, the light source 10 may be selected for each light source or for each column in the vertical direction.

Here, if the range of light irradiation is controlled from the vicinity of the first joint to the vicinity of the third joint of the finger, it is possible to reduce the amount of light entering the inside of the device through the opening 3 without irradiating the finger, which is preferable. Is.

Here, all the point light sources 10 determined to be turned on may be turned on at the same time, but in that case, the irradiation light of the plurality of point light sources 10 hits one finger and causes interference, so that the light is irradiated to the side surface of the finger. This may result in a halation region 25. In that case, the timing of turning on the light source 10 at each point is shifted, and the finger blood vessel image is taken a plurality of times at each timing, so that a clear blood vessel image of all fingers can be obtained.

More specifically, authentication is performed using the blood vessel images of the plurality of fingers acquired by first turning on all the point light sources 10, and when the authentication cannot be performed, the fingers in which the plurality of point light sources 10 are interfering are detected. It is preferable to detect or estimate and turn on only the point light source 10 that irradiates only the finger to perform re-imaging, because the number of re-imaging can be minimized.

In addition, the degree of interference of the irradiation light between the plurality of point light sources 10 is held in the lighting control program as a parameter from the preliminary experiment so that the light is turned on according to the presented finger position and the finger main direction 24. The point light source 10 to be turned on and the timing of lighting may be determined in advance based on the set point light source 10 and the parameters. As a result, the calculation load required for the above-mentioned re-shooting can be reduced.

Regarding the light amount control of the point light source 10 to be turned on, it is possible to irradiate with a constant amount of light regardless of the size of the hand, but in general, the larger the hand, the wider and thicker the finger, and the more difficult it is to transmit light. Therefore, it is preferable to control the amount of light to be emitted according to the width and thickness of the finger and the brightness information of the captured finger blood vessel image. This makes it possible to take a clear blood vessel image regardless of the size of the hand or finger.

FIG. 9 is a diagram showing a flowchart of a process of feedback-controlling the amount of light. Hereinafter, a mechanism for feedback-controlling the amount of light of the point light source 10 of the light source array 9 from information on the position and posture of the finger, the brightness of the finger blood vessel image, and the like will be described with reference to this figure.

Step 201 is an operation in which the certifier presents the hand 1 to the blood vessel imaging device 2. Step 202 is a process of detecting the presented hand using the distance image or the like acquired by the distance sensor 4. Step 203 is a process of determining whether or not a hand has been detected. If no hand is detected here, the process returns to step 202, and if a hand is detected, the process proceeds to step 204. Step 204 is a process of controlling the initial light source and lighting the point light source 10 that irradiates the detected fingers of the hand with light at the initial light amount value. Details of step 204 will be described later. Step 205 is a process of acquiring a distance image by the distance sensor 4 and acquiring a near infrared image by the imaging unit 11. Details when the ToF (Time of Flight) method is used as the distance sensor will be described later. Step 206 is a process of detecting the position and posture of the hand and the finger in the three-dimensional space by using the distance image acquired in step 205. In step 207, finger size information, finger region brightness information, and the like to be used for controlling the amount of light of the lighting point light source 10 are calculated from the finger blood vessel region of the distance image and the near infrared image acquired in step 205. It is a process. Step 208 is a process of determining a point light source 10 to be lit by the light source array 9 based on the calculation result of step 207. Step 209 is a process of controlling the amount of light of each point light source 10 to be turned on. Specifically, based on the size of each finger and the brightness information of the finger region calculated in step 207, the light amount value of the lit point light source 10 for clearly photographing the blood vessels of each finger is determined, and each point light source is used. Turn on 10. Details of the method for determining the light intensity value in step 209 will be described later. After the point light source 10 is turned on, the process returns to step 205, and the control of the light source array 9 is repeated. In this way, each point light source 10 is turned on with the light amount value determined in step 209, and by returning to 205 again and repeating the process, the light amount value is feedback-controlled by utilizing the width of the finger and the brightness of the finger area. Can be done. As the brightness information used for controlling the amount of light in step 209, information such as the average brightness of the finger region, the contrast value between the blood vessel and the non-blood vessel region, and the ratio of the halation region 25 in the finger region may be used. However, these may be used together.

The details of step 204 will be described below. In the initial light amount control of the point light source 10, since the finger blood vessel image cannot be captured yet, the point light source 10 is turned on with a preset initial light amount value. The initial light intensity may be fixed in advance to a light intensity value that allows a clear image of an average-sized finger blood vessel. Further, after the hand is detected in step 203, the width of the finger may be calculated from the distance image, the size of the finger may be estimated, and then the initial amount of light may be determined according to the estimated value of the size of the finger. In this way, by estimating the size of the finger based on the calculated width of the finger and determining the light intensity value, it is possible to turn on the light with an initial light intensity value close to the optimum light intensity value that enables clear blood vessels to be photographed for each finger at high speed. The amount of light can be controlled with high accuracy.

Hereinafter, the details of the method for determining the light intensity value in step 209 will be described. In the method of determining the light intensity value in step 209, the light intensity value of the point light source 10 to be irradiated may be independently determined based on the detected brightness information of the finger region of each finger, or the average brightness of all the detected finger regions may be determined. The light intensity of all light sources may be uniformly determined based on the information. Alternatively, a plurality of fingers whose main directions are close to each other may be grouped, and the light intensity value of the light source that irradiates the grouped fingers may be determined based on the average brightness information of the finger regions of all the grouped fingers. Here, when the irradiation light of the plurality of point light sources 10 hits one finger and the plurality of point light sources 10 interfere with each other, each point light source 10 can clearly photograph the blood vessels of all fingers according to the degree of interference. The light amount value of may be controlled. As a result, blood vessels of a plurality of fingers can be photographed simultaneously and clearly. Further, instead of using the above-mentioned luminance information, the light amount value of the point light source 10 to irradiate the finger may be determined according to the finger size information such as the finger width and the finger length.

Hereinafter, when the ToF method is used as the distance sensor 4, a method for solving the problem that the accuracy of finger position and posture detection is lowered due to fluctuations in the opening / closing posture of the finger and rotation of the hand will be described. The distance sensor 4 calculates the distance between the distance sensor 4 and the target when the distance sensor 4 receives the light reflected by the light emitted by itself when it hits the target whose distance is to be measured. The light emitted by the distance sensor 4 is emitted toward the opening 3 which is the presentation portion of the hand, and passes through the opening 3 such as acrylic or glass. However, in the portion of the opening 3 where the direction of the irradiation light of the distance sensor 4 and the direction of the surface of the opening 3 are perpendicular to each other, the light does not pass through the opening 3 and is totally reflected (specularly reflected) to the distance. Since the sensor 4 receives light, the distance cannot be measured. A portion where the direction of the irradiation light of the distance sensor 4 and the direction of the surface of the opening 3 are perpendicular to each other is defined as a distance-measurable portion. If the position of the finger comes near the non-distance measuring part by opening and closing the finger or rotating the hand, the distance of the finger cannot be measured, and even if it can be measured, the distance is measured near the non-distance measuring part. Since the accuracy of is lowered, the authentication accuracy is lowered.

Therefore, a polarizing filter is used as the optical filter 12, and the distance sensor 4 is passed through the polarizing filter before receiving light to block only the polarization of the specular reflection component at the opening 3, which is the cause of the unmeasurable portion. Then, reduce the area of the unmeasurable part. As a result, if the area of the non-distance measuring part becomes small, the adverse effect on the accuracy of distance measurement when the finger is located near the non-distance measuring part becomes small, and even if the finger opens / closes or the hand rotates, the authentication accuracy becomes low. The decrease can be suppressed.

In this embodiment, as in the first embodiment, a clear finger blood vessel image is taken by controlling the light source according to the opening / closing posture of the finger and the rotating posture of the hand, but the device can be further miniaturized and becomes clearer. A configuration that realizes various finger blood vessel imaging will be described. Since the same method as in the first embodiment can be used for the authentication processing flow and the light source control flow, the description thereof will be omitted. Further, the blood vessel imaging device 2 described in the present embodiment can be used not only at the time of authentication but also at the time of registration as in the case of the first embodiment.

FIG. 10 is a diagram showing a mechanism for miniaturizing the blood vessel imaging apparatus. A light source array 9 for irradiating a finger with light is arranged on the base side of the finger, and the main direction 24 of each finger and the irradiation direction of the light source are brought close to each other in parallel regardless of whether the finger is open or closed. A plurality of point light sources 10 are arranged radially around a position where the wrist is presented when the finger is held over the opening 3. By arranging the light source array 9 on the wrist side in this way, the light source array 9 can be further miniaturized.

Hereinafter, the configuration of the blood vessel imaging apparatus of Example 2 will be described with reference to FIGS. 11 and 12.

FIG. 11 is an example embodying the configuration of the blood vessel imaging apparatus of FIG. In the light source array 9, a plurality of point light sources 10 are arranged side by side on the surface of the opening 3, that is, the installation surface of the device. As shown in FIG. 12a, each point light source 10 is installed so as to irradiate diagonally downward in order to irradiate the presented fingers of the hand with light. As shown in FIG. 12b, a plurality of point light sources 10 may be arranged side by side in a direction parallel to the opening 3 and the installation surface of the device.

Hereinafter, a modified example of the blood vessel imaging apparatus of Example 2 will be described with reference to FIGS. 13 and 14.

FIG. 13 is a diagram showing a configuration of a blood vessel imaging device when the rotation of the hand is taken into consideration. When the blood vessel imaging device is configured in this way, as in FIG. 7, the width on the wrist side is narrower than the width on the fingertip side of the hand presented by the opening 3, so that the presentation position of the wrist or hand 1 can be set. It is possible to guide to the center point 26 where the irradiation light directions of the plurality of point light sources 10 are gathered. Further, by providing the hand presentation portion 19 and naturally guiding the presentation position of the hand 1 to the hand presentation portion 19, the user's hand can be easily rotated around the hand presentation portion 19 and the rotation fluctuates. A clear finger blood vessel image can be obtained even at any time.

FIG. 14 is a diagram showing a modified example of the arrangement of the light source in the blood vessel imaging apparatus. This example is effective when it is considered that the position of the hand fluctuates on the surface of the opening 3, that is, the installation surface of the device when the hand is rotated. That is, as in the example of FIG. 14, by arranging the plurality of point light sources 10 radially around the assumed plurality of center points 26 (positions of the hands), the presentation position of the hands is shifted or the hands are displaced. A clear finger blood vessel image can be obtained even when presented in a rotated posture.

In Example 1, it was explained that the process of step 107 in FIG. 8 should select a light source in which the angle generated by the irradiation light direction 22 and the main direction 24 of each finger is closest to 0 degrees. Needless to say, in the second embodiment, the arrangement of the light sources is opposite to that of the first embodiment, so that the reference angle is 180 degrees.

In Examples 1 and 2, the arrangement configuration of the point light source 10 capable of acquiring a clear finger blood vessel image according to the opening / closing posture of the finger to be presented and the rotation posture of the hand has been described. In this embodiment, even if the point light source 10 is arranged so that the blood vessel images of a plurality of fingers cannot be clearly captured by one shooting due to the opening and closing of the fingers and the rotation of the hand, the imaging process is performed at a plurality of timings. , A method of acquiring a clear blood vessel image and realizing highly accurate authentication will be described.

Further, the blood vessel imaging device 2 described in this embodiment can be used not only at the time of authentication but also at the time of registration as in the case of Example 1 or Example 2.

FIG. 15 is a diagram showing an image when an authentication blood vessel image is acquired by controlling the imaging timing. Specifically, it represents the position of the hand 1 at a plurality of timings when the hand 1 is presented to the opening 3 while operating the hand 1 at the time of authentication. At the timing T1, the main direction 24 of the finger 41 and the irradiation light direction 22 of the point light source 10 that lights up are close to parallel, and the finger blood vessel of the finger 41 can be clearly photographed. Similarly, at the timing T2, the finger main direction 24 of the finger 42 and the irradiation light direction 22 of the point light source 10 are close to parallel, and the finger blood vessel of the finger 42 can be clearly photographed. Similarly, at the timing T3, the finger main direction 24 of the finger 43 and the finger 44 and the irradiation light direction 22 of the point light source 10 are close to parallel, and the finger blood vessels of the finger 43 and the finger 44 can be clearly photographed.

As a method for acquiring clear blood vessel images of multiple fingers, in addition to the method of selecting a finger blood vessel image of a certain frame, a blood vessel image of multiple frames of the same finger is imaged by a method such as HDR (high dynamic range). It may be synthesized to generate a clear finger blood vessel image. In the example of FIG. 15, at each timing of T1, T2, and T3, a finger having a clear blood vessel and a finger having a non-clear blood vessel are different. Here, if each finger is tracked between a plurality of timings and the same finger is associated with each other, the displacement of the same finger between the plurality of timings and the distortion of the blood vessel shape due to the change in posture are precisely corrected (registration). After that, the finger blood vessel image can be combined to generate a clearer blood vessel image.

Needless to say, the method of selecting or generating a clear finger blood vessel image from a plurality of frames can be used not only at the time of authentication but also at the time of data acquisition for registration of each finger blood vessel image.

FIG. 16 is a diagram showing a flowchart of finger angiography and authentication processing of this embodiment. Step 301 is an operation in which the user presents a hand. Step 302 is a process of detecting the hand. Step 303 is a process of determining whether or not a hand has been detected. If no hand is detected, the process returns to the detection of the hand in step 302, and if a hand is detected, the process proceeds to step 304. Step 304 is a process of controlling the initial light source and lighting the point light source 10 of the light source array 9 with the initial amount of light according to the detected position of the hand so that the light hits the finger. Step A 305 is a process of acquiring a distance image by the distance sensor 4 and acquiring a near infrared image by the imaging unit 11. Step 306 is a process of detecting the position and posture of the hand or finger using the distance image and the near infrared image. In other words, it is a process of detecting and acquiring position information and posture information based on the three-dimensional shape of the finger. In step 307, the light source array 9 is controlled according to the position of the finger and the posture of the finger detected in step 306, the point light source 10 to be turned on for photographing the finger blood vessel is selected, and the light intensity value of the point light source 10 to be turned on. Is the process of determining. Step 308 is a process of performing normalization processing such as correction of rotation of the finger in each finger region and correction of distortion due to fluctuation of the posture of the finger in the near infrared image. Step 309 is a process of selecting each normalized blood vessel image so that a low-quality finger blood vessel image having a halation region 25 on the side surface of the finger is not used in the subsequent processing. Step 310 is a process of extracting features for collation from the finger blood vessel images normalized in step 308 and selected in step 309. Step 311 is a process of collating the finger blood vessel features registered in the database with the blood vessel features of each finger. In step 312, when the matching degree of the matched finger is larger than the matching degree by the matching in the past frame, the matching degree of the finger is updated and the matching score calculated from the matching degree of the plurality of fingers is recalculated. This is a process of updating the matching score based on the result. The details of step 312 will be described later. Step 313 is a process of performing an authentication determination based on the updated collation score. If the collation score is larger than the threshold value TH2, the process proceeds to step 314 to perform post-authentication successful processing, and the process proceeds to step 316 to end the authentication process. On the other hand, if the collation score is equal to or less than the threshold value TH2, the process proceeds to step 315 to determine the authentication timeout, and if the time-out occurs, the process proceeds to step 316 to end the authentication process. If it is not the authentication timeout, the process returns to step 304 and the authentication process is repeated.

In the selection of the finger blood vessel image in step 309, only the clear finger blood vessel image is used for collation to speed up the authentication time, but all the finger blood vessels are selected without selecting the finger blood vessel image. Images may be used. Further, in order to perform authentication at a higher speed, in the detection of the finger region in step 306, only the finger region in which the irradiation light direction 22 of the light source 10 and the finger main direction 24 are close to parallel is selected, and the process after step 308 is performed. It may be configured to be used.

The details of step 312 will be described below. When updating the collation score in step 312, it is premised that the plurality of fingers photographed in the plurality of frames from the past to the present used in the update process are all the same. Therefore, a method for ensuring the identity of a plurality of fingers shot in a plurality of frames used for the update process will be described below.

As an example of the method of ensuring identity, it is said that there is identity for a finger in which multiple frames used for the update process are continuously photographed and it can be confirmed that the detected areas of each finger are continuously connected. It is possible to judge.

Next, regarding a method of ensuring the identity of the fingers when the hand returns to the viewing angle after frame-out outside the viewing angle of the distance sensor 4 or the imaging unit 11 in the movement of the hand. Describe. In this case, since the hand is out of the viewing angle of the camera, the continuity of the position of the finger region cannot be guaranteed only by the method described above. In such a case, the detected blood vessel image features of the finger are collated, and when the high degree of matching exceeds the preset threshold value TH3, it is determined that the finger is identical, thereby ensuring the identity of the finger. be able to. Further, the degree of coincidence may be determined by using a modality other than the blood vessel image of the finger such as a fingerprint, a wrinkle of the finger, and the size and shape of the finger, and the identity of the finger may be determined. Further, the continuity of the finger region between the frames and the determination of the degree of coincidence of the finger blood vessel image features can be used in combination to ensure the identity of the finger.

Even if the identity of the finger is guaranteed, it is necessary to take a separate measure against counterfeiting to update the collation score by using an object other than the finger. As an example of countermeasures, it is conceivable to detect the living body of the finger and exclude artificial objects that are not detected as living bodies without using them for authentication. At that time, as a biological detection method, a method of detecting pulsation and heartbeat from time-series changes in the brightness of the finger region of the finger blood vessel image, and a method of calculating in advance by machine learning from the undulation information of the brightness of the finger blood vessel image and the brightness histogram. A method of distinguishing between a finger and an object other than the finger based on the specified parameters can be considered.

In this embodiment, the shape and light source arrangement of the blood vessel imaging device for authentication and the blood vessel imaging device for registration are different, or the presentation posture and presentation such as the opening / closing posture of the user's finger are presented at the time of authentication and at the time of registration. A method for improving compatibility between devices by increasing the degree of matching between the acquired data of the authentication device and the registration device even if the positions are different will be described.

First, as a general theory, in many scenes in which a user registers biometric data used for biometric authentication, a blood vessel image can be accurately taken at a window according to the guidance of a guide or the like. On the other hand, at the time of authentication, as can be seen from the entrance / exit gates, it is assumed that there are many cases where the user authenticates independently in the absence of a guide. In order to improve the certification system in such a situation, a mechanism for increasing the degree of matching between the blood vessel image at the time of registration and the blood vessel image at the time of authentication is required.

In this embodiment, a scene in which a blood vessel image taken with the hand open and a blood vessel image taken with the hand closed are collated will be described. As described above, the blood vessel image taken with the hand open is more likely to have a halation region 25 than the blood vessel image taken with the hand closed. Therefore, the degree of coincidence when collating the finger blood vessel images with the finger open and the finger blood vessel images with the finger closed is lower than the degree of agreement when collating the finger blood vessel images when the finger is closed. ..

Here, even when the position and size of the halation region 25 on the side surface of the finger fluctuate due to the fluctuation of the opening / closing posture of the finger, the halation region 25 does not occur near the center of the inside of the finger, and the finger blood vessel can be photographed. Therefore, by using only the features near the center of the inside of the finger and not using the features near the side of the finger when collating the features of the finger blood vessels at the time of authentication, it is highly resistant to fluctuations in the opening and closing posture of the finger. Accurate authentication can be realized. Further, the characteristics of the finger blood vessels may be weighted so that the weight of the characteristics near the center of the inside of the finger is increased and the weight of the characteristics near the side surface of the finger is decreased at the time of collation. Further, if the halation region 25 of the finger can be estimated or specified from the relationship between the main direction of the finger and the irradiation direction of the point light source 10 and the brightness of the acquired finger blood vessel image, the halation region 25 in the finger blood vessel image The weight of the feature of the above may be reduced, and the weight of the region other than the halation region 25 may be increased. In this case, it is preferable to adopt a configuration in which the regions can be distinguished by setting a region in which the brightness of the finger blood vessel image exceeds a predetermined value as a halation region or the like.

In this embodiment, even if the shape and light source arrangement of the authentication device and the registration device are different and the presentation posture such as the opening / closing posture of the user's finger and the tendency of the presentation position are different, the acquired data of the authentication device and the registration device are used. This section describes how to improve the degree of matching and compatibility.

In addition, FIG. 17 described below shows a device in which the user can easily present his / her hand with his / her finger open and non-contact, and FIG. 18 shows a device in which the user can easily present his / her hand with his / her finger closed. FIG. 20 shows a device in which the user can easily present his / her hand with his / her finger closed and non-contact, FIG. 21 shows a device in which the user can easily make contact with the presentation surface when presenting his / her hand, and FIG. 22 shows the user. Each device is shown so that the presentation surface can be easily presented without contact when presenting the hand.

FIG. 17 is an example of an authentication device in which the user can easily present his / her hand with his / her finger open and non-contact. The reason why it is easy to present the hand with the fingers open is that the area of the opening 3 is smaller than that of FIGS. 18 and 19 described later. The reason why the hand is easily presented in a non-contact manner is that since the light source array 9 is arranged on the fingertip side, a large distance between the hand and the opening 3 can be secured as compared with FIG. 21 described later.

In the authentication device shown in FIG. 17, an exterior cover long in the direction of travel of the user is attached to the outside of the authentication terminal in order to secure a lead wire for the user to be authenticated, and the user travels in a straight line along the lead wire. Induce to do. By walking in a straight line along the lead wire, the user has the effect of improving the reproducibility of the presentation position and presentation posture of the hands and fingers. Further, by arranging the light source array 9 on the side of the opening 3 and above the surface of the opening 3, even if the authenticated hand moves from the installation surface of the device to the traveling direction of the user, the hand is the light source. It can be prevented from hitting the array 9.

FIG. 18 is an example of a registration device in which the user can easily present his / her hand with his / her finger closed and non-contact. The reason why it is easy to present the hand with the fingers closed is that the area of the opening 3 is smaller than that of the device of FIG.

The registration device of FIG. 18 is suitable for installation in an environment where space is restricted. That is, in order to install the device in a narrow place, it is necessary to reduce the size of the device. Assuming that the user is stopped and his / her hand is brought close to the opening 3 at the time of registration, the light source array 9 is brought close to the fingertip when the hand is held over the opening 3. 9 can be arranged, and the size of the entire device can be reduced. Further, if the permissible range of the amount of fluctuation of the position above (in the height direction) from the surface of the opening 3 of the hand to be presented at the time of data acquisition for registration can be narrowed, the height direction constituting the light source array 9 is formed. By reducing the number of point light sources 10 in the above, the size of the registration device can be reduced. Further, by narrowing the width of the opening 3 from the fingertip to the wrist direction according to the shape of the hand presenting the shape of the opening 3, the presentation position of the hand, particularly the presentation position of the palm, and the device can be restricted. Both miniaturization can be achieved.

The light-shielding portion 28 may be attached to limit the height at which the hand can be presented. By using a material that does not transmit light, the light-shielding portion 28 has the effect of blocking ambient light entering the opening 3 and taking a clearer finger blood vessel image. Further, if a material that transmits visible light but blocks near-infrared light is used for the light-shielding portion 28, the user can confirm the hand to be presented through the roof at the time of registration. Since this roof material blocks near-infrared light, it is possible to obtain a clear finger blood vessel image by blocking the near-infrared component contained in the ambient light when taking a finger blood vessel image.

When the opening / closing posture of the finger is different between the authentication device and the registration device, as described in the fourth embodiment, the weight of the halation region 25 on the side surface of the finger is reduced to match the features of the finger blood vessel image, so that the different devices can be used. It is possible to improve compatibility between acquired data and realize highly accurate authentication.

FIG. 19 is an example of an authentication device in which a user can easily present his / her hand with his / her finger closed and non-contact. The authentication device shown is suitable when the installation width is limited. When the width of the device is narrow, the width of the opening 3 for presenting the hand is also small. Since the width of the opening 3 is small, the user can easily present his / her hand with his / her fingers closed.

On the other hand, if the width of the opening 3 is small, it is possible that a person with a large hand cannot fit all the fingers within the width of the opening 3. If the blood vessel images of all the fingers of the hand presented by a user are registered and all the fingers cannot be photographed at the time of authentication, the number of fingers that can be used for authentication decreases, and the compatibility between the registered data and the authentication data becomes compatible. descend. In addition, if the position and posture of the hand presented by the user are likely to fluctuate, there is a high possibility that many fingers will not be photographed at the same time, and the compatibility between the registration data and the authentication data will be reduced. Countermeasures for this problem will be described later with reference to FIG.

FIG. 20 is a diagram showing a modified example of the opening 3 in the authentication device of FIG. By marking the hand contour guide 30 for guiding the hand presentation position on the palm presentation portion 19 outside the opening 3, it is possible to suppress the rotation of the hand and the fluctuation of the hand presentation position. By recessing the inside of the opening 3 for presenting the finger and the contour line guide 30 of the hand in the height direction of the device installation surface rather than the surroundings, the rotation of the hand and the fluctuation of the presentation position are further suppressed, and the finger is closed. It is possible to make it easier to present the hand in the state. If all the fingers do not fit in the opening 3 even when the hand is closed, mark the contour guide 30 of the hand like a mittens glove as shown in FIG. 27, and open one finger. By guiding the finger to escape to a predetermined position outside of 3, the reproducibility of the finger presentation position is improved, and the remaining fingers can be easily presented to the opening 3 in a state where the main directions 24 of the fingers are aligned.

Even when the guide for limiting the position and posture of the finger is provided in this way, when many people use the authentication device 2, it is assumed that the finger protrudes from the opening 3. In that case, only a part of the finger can be photographed. If only a part of the finger can be photographed, the degree of matching with the characteristics of the finger for which the entire finger can be photographed at the time of registration will be low, so it is used when calculating the matching score from the degree of matching of multiple fingers. It may not be done. That is, if a part of the plurality of fingers protrudes from the opening 3 at the time of authentication, the collation score cannot be calculated for the protruding fingers, and there is a possibility that authentication cannot be performed. Therefore, instead of dividing the features of the blood vessel image for each finger, the features of a plurality of fingers are registered as a unit of a group, and the features of a group of a plurality of fingers are collated and authenticated. If a certain percentage of the amount of information of a group of features registered at the time of authentication can be extracted, verification is performed, so that even if multiple fingers protrude from the opening 3, authentication can be performed while maintaining authentication accuracy. It becomes possible to do.

FIG. 21 is an example of a registration device that is easily brought into contact with the presentation surface when the user presents the hand.
The reason why the hand easily comes into contact with the presentation surface is that since the light source array 9 is arranged on the wrist side, it is difficult to secure the distance between the hand and the opening 3 as compared with FIG. 22 described above.

As described above in FIG. 10, the registration device of FIG. 21 arranges the light source array 9 so as to irradiate light from the finger base side toward the fingertip side, so that the entire device can be miniaturized. Further, as compared with the case of irradiating the point light source 10 from the fingertip side to the finger base side, the irradiation light is not concentrated near the finger contour of the finger crotch when the point light source 10 is irradiated from the finger base side, and the halation region. The occurrence of 25 can be suppressed, and a clear finger blood vessel image can be obtained.

The light source array 9 may be provided with a light-shielding portion 28 that covers the opening 3. By attaching the light-shielding portion 28, it is possible to prevent ambient light from entering the inside of the device through the opening 3 and limit the position of the hand presented by the user at the time of registration in the height direction from the surface of the opening 3. it can.

FIG. 22 is an example of a device that makes it easy for the user to present the hand without contacting the presentation surface when presenting the hand. In the device of FIG. 22, a palm presentation portion 19 and an opening 3 for presenting a finger are provided, respectively, and the opening 3 is provided more than the palm presentation portion 19 in the upward direction, that is, in the height direction of the installation surface of the device. Place it low. Since the presentation portion 19 of the palm is in a high position, when the presenting hand is brought closer to the opening 3, even if the hand touches the presentation portion 19 of the palm, the finger does not touch the opening 3, so that the finger It is possible to prevent deterioration of authentication accuracy due to pressing.

Hereinafter, the authentication accuracy is improved when authentication is performed using the authentication data obtained by contacting the hand with the opening 3 without contacting the registration data obtained by contacting the hand with the opening 3 or vice versa. The mechanism will be described.

First, a problem concerning the difference between the finger blood vessel images when the finger opening 3 is not in contact and when the finger is in contact will be described. When the opening 3 of the finger is not in contact, the light emitted from the point light source 10 is reflected by the surface of the opening 3 such as acrylic, and the component hits the finger and the ventral side of the hand, and the entire finger is photographed in a bright state. On the other hand, when the finger opening 3 is in contact with the finger, there is no gap between the finger and the opening 3, so that there is no light component directly irradiated to the ventral side of the finger. Therefore, there is a difference in the amount of light emitted to the entire surface on the ventral side of the finger when the finger opening 3 is not in contact with the finger. Due to such differences in lighting conditions, the characteristics extracted from the finger blood vessel images acquired in each state during non-contact and contact also differ, and the characteristics of the finger blood vessel image during non-contact and the finger blood vessel image during contact also differ. The degree of matching of the features of the above is reduced, and the authentication accuracy is deteriorated. Therefore, by irradiating the skin surface on the ventral side of the finger with uniform light to equalize the lighting conditions when the finger is in contact with the opening 3 and when the finger is not in contact with the opening 3, the finger blood vessels during contact and non-contact are made uniform. The degree of matching of image features can be improved, and the authentication accuracy can be improved. At this time, the uniform light irradiating the surface on the ventral side of the finger can be created by passing the irradiating light above the light source 31 (not shown) arranged below the surface of the opening 3 through a diffuser plate or the like. At this time, the light source 31 is arranged so that the imaging unit 11 does not directly receive the uniform light. A shading means may be provided so that the imaging unit 11 does not directly receive the uniform light. Further, the uniform light irradiating the surface on the ventral side of the finger is generated on the surface of the finger when the finger opening 3 is in contact with and when the finger opening is not in contact, without reducing the contrast of the blood vessel pattern of the acquired finger blood vessel image as much as possible. It is desirable that the amount of light is such that the difference in lighting conditions is eliminated. The amount of uniform light may always be constant, or the brightness of the surface on the ventral side of the finger is calculated from the brightness information of the finger blood vessel image, and the brightness of the surface on the ventral side of the finger is uniform. The amount of light may be adjusted so that the light intensity is reduced.

In addition, when the finger is in contact with the opening 3, the lighting conditions change, and by pressing the finger strongly against the presentation portion, the blood vessel shape of the acquired blood vessel image is deformed or the blood flow is stopped. Defects in blood vessel patterns may occur. Therefore, it is necessary to prevent pressing with an excessive force when the finger comes into contact with the opening 3. In response to this, light with a wavelength that does not interfere with the wavelength used for angiography (pressing detection light) is incident inside acrylic, etc., and the incident light is internally totally reflected at the interface between acrylic and the atmosphere. To do. When a finger or hand touches acrylic, the pressing detection light that was totally reflected at the interface between acrylic and the atmosphere enters the living body at the interface between acrylic and the living body. Therefore, when a camera capable of capturing the pressing detection light is arranged below the opening 3 and the hand and the finger are photographed, the finger and the portion touching the acrylic of the hand become bright. The stronger the finger is pressed, the brighter the pressed portion becomes. Therefore, the degree of pressing can be detected from the brightness information of the captured image. When a strong pressing force is detected, it is possible to urge the speaker 15 to loosen the pressing force by using a buzzer sound, voice guidance, a display unit 16, or the like.

1: Hand 2: Vascular imaging device 3: Opening 4: Distance sensor 5: Computer 6: Memory 7: CPU 8: Opening 9: Light source array 10: Point light source 11: Imaging unit 12: Optical filter 13: Interface 14 : Storage device 15: Speaker 16: Display unit 17: Visible light source 18: User ID input unit 19: Hand presentation unit 20: Light source unit 21: Light refracting unit 22: Irradiation light direction 23: Hand main direction 24: Finger Main direction 25: Halation area 26: Center point 28: Light-shielding part 30: Contour guide 41: Finger 42: Finger 43: Finger 44: Finger 50: Data input unit 51: Light amount control unit 52: Image input unit

Claims (9)

An opening formed on the surface of the housing and
A plurality of light sources arranged in a light source array portion protruding from the opening,
A sensor that acquires the position information of the finger presented on the opening and the main direction information of the finger, and
A light amount control unit that selects an irradiation light source to irradiate the finger from the plurality of light sources based on the position information and the main direction information and controls the light amount of the selected irradiation light source.
An imaging unit that captures an image of blood vessels inside the finger irradiated with light from the irradiation light source, and an imaging unit.
A blood vessel imaging apparatus comprising the above. In the blood vessel imaging apparatus according to claim 1,
The light quantity control unit is a blood vessel imaging apparatus characterized in that the light source in which the main direction of the finger and the irradiation direction are parallel and closest to each other is selected. In the blood vessel imaging apparatus according to claim 1,
The blood vessel imaging apparatus, wherein the plurality of light sources are arranged in a direction of irradiating light from the fingertip direction to the root direction of the finger, and are arranged radially. In the blood vessel imaging apparatus according to claim 3,
The plurality of light sources constitute a plurality of light source units including a predetermined number of light sources, and the plurality of light source units are arranged in a radial pattern. In the blood vessel imaging apparatus according to claim 1,
A blood vessel imaging apparatus, further comprising a member that refracts light between the opening and the plurality of light sources. In the blood vessel imaging apparatus according to claim 1,
The blood vessel imaging apparatus, wherein the plurality of light sources are arranged in a direction of irradiating light from the root direction of the finger toward the fingertip direction, and are arranged radially. In the blood vessel imaging apparatus according to claim 1,
The fingers are multiple
The light amount control unit selects the irradiation light source for each finger based on the position information and the main direction information of the plurality of fingers, and determines the timing for controlling the light amount for each irradiation light source.
The imaging unit is a blood vessel imaging apparatus characterized in that an image of a blood vessel existing inside a finger is captured at each timing. In a personal authentication system including the blood vessel imaging device according to any one of claims 1 to 7, and an authentication device that performs personal authentication using features extracted from the captured blood vessel image.
It is further provided with a feature extraction unit that extracts features for the authentication device to perform personal authentication from the captured image of the blood vessel.
The feature extraction unit estimates a region in which the brightness exceeds a predetermined value in the captured image of the blood vessel as a brightness saturation region, and compares the region estimated as the brightness saturation region with other regions to weight the features. A personal authentication system characterized by setting a small value. In the personal authentication system according to claim 8,
The feature extraction unit is a personal authentication system characterized in that it exists in the authentication device.

Images