JP6523925B2 - Authentication apparatus using biometric information and authentication method - Google Patents

Description

  The present invention relates to an authentication system that authenticates an individual using a living body, and more particularly to a small, convenient, and highly accurate authentication technique.

  Among various biometric authentication techniques, finger vein authentication is known as one that can realize highly accurate authentication. The finger vein authentication achieves excellent authentication accuracy because it uses a blood vessel pattern inside the finger, and high security can be realized by forgery and tampering compared to fingerprint authentication.

  In recent years, increasing cases of securing the security of each device by installing a biometric authentication device on devices such as mobile phones, laptop PCs (Personal Computers), smartphones and tablet terminals, lockers, safes, printers, etc. ing. Further, as fields to which biometrics authentication is applied, biometrics authentication has been used in recent years in addition to entry and exit management, attendance management, login to a computer, and the like. In particular, it is important for biometric authentication devices used in the public to realize reliable personal authentication. Furthermore, in view of the spread of tablet type portable terminals in recent years and the trend of wearable computing, it is also one of the important requirements to realize the miniaturization of the device while securing the convenience as described above. It is also important to provide an authentication device with high operability so that the user can easily use the authentication device.

  There is a patent document 1 as a technology related to miniaturization in an authentication device that performs personal authentication based on the shape of a blood vessel.

  Further, Patent Document 2 discloses a technique for capturing an image of a palm held in a non-contact state on a device, a reflection image of a visible light source including red with a color camera, and extracting palm veins and palm print for authentication. It is done.

Patent publication 2013-33555 gazette WO 13/136553

  In order to realize a small-sized, easy-to-use, high-accuracy personal identification device, it is necessary to capture a wide range of biological information from a narrow-area sensor and to effectively correct positional deviation when presenting a living body. It is important to guide the user to the correct position if there is a position shift.

  In the biometric authentication device described in Patent Document 1, although the sensor opening is made narrower than the width of the finger to miniaturize the device, the contour of the finger can not be photographed because it is presented by touching the finger, and hence The position correction in the case where the positional deviation of the finger occurs is difficult, and further, there is no mention of the solution. In addition, since it is necessary to place a finger at a predetermined position of the device, it is not possible to capture a wide range of biological information, and further, it is difficult to miniaturize the device because it is necessary to provide a finger rest.

  Patent Document 2 describes a technique for simultaneously photographing and authenticating a palm vein and a palm print using a color camera provided as a standard on smartphones and tablets, and light of an ambient light or a liquid crystal screen. . This technology has the advantage of being able to capture a wide range of palm biometrics without touching the device. However, there is no disclosure of guidance such as the user stopping the living body at an appropriate position.

  Therefore, in the present invention, in order to realize a compact, easy-to-use and highly accurate personal authentication, it is possible to acquire information of a biometric feature having a large amount of information characterizing an individual while maintaining a high degree of freedom when presenting a biometric. It aims at realizing a possible personal identification device.

  An authentication apparatus for authenticating an individual using characteristics of a living body, comprising: an installation area where the living body is installed, a light source for irradiating light to the living body presented in the installation area, and the light source reflecting the living body An imaging unit configured to capture light; an image processing unit configured to process an image captured by the imaging unit; a feature extraction unit configured to extract a biological feature of the living body from the image; a storage unit configured to hold the biological feature; A collating unit that compares the degree of similarity of the biological features, the imaging device and the plurality of light sources are disposed at a position facing the living body, the plurality of light sources respectively emit different wavelengths, and the imaging devices are a plurality of Based on the posture of the living body, it has means for photographing the wavelength and measuring the distance between the living body and the imaging device, determining the posture of the living body, detecting the optimum presentation position and guiding it to that position. Of the living body An authentication device and performs butterfly correction.

  According to the present invention, in a biometric authentication device using a finger, it is possible to capture a wide range of living body even though the device housing is small, and perform authentication with high accuracy by acquiring various biological features therefrom. In addition, it is possible to correct or detect the position even when the positional deviation of the finger occurs, and it is possible to provide a small-sized, highly convenient authentication device with high authentication accuracy.

BRIEF DESCRIPTION OF THE DRAWINGS It is a figure which shows the structure of the whole biometrics system of 1st Embodiment. It is a figure explaining apparatus composition of a biometric system of a 1st embodiment. It is an example of processing flow which carries out reflective non-contact authentication using an input device. It is an example of an image of a vein, a fingerprint, or a fat print taken by simultaneously emitting infrared light and green light. It is an example of the spectral sensitivity characteristic of each RGB element of a color camera. It is explanatory drawing of the detection method of the presentation position of a finger. It is explanatory drawing of the detection method of the position of a finger | toe outline and a finger tip. It is explanatory drawing which shows the method of detecting the attitude | position of a finger. It is explanatory drawing which shows the method to detect the rolling rotation angle of a finger. It is an example of a guidance screen which guides the user the position and posture of a finger suitable for authentication. It is explanatory drawing of a cutting-out process and three-dimensional correction process of a finger. It is an explanatory view of composition and operation method of an authentication device which can do guidance of a finger position easily. It is explanatory drawing of the operation method of the authentication apparatus which guide | induces a position easily by tapping a finger | toe. It is one Example of the reflection non-contact biometrics apparatus comprised with the infrared light source and one infrared camera which is 2nd Embodiment. It is an example of the processing flow in a 2nd embodiment. It is explanatory drawing of the detection method of the pitching angle of the finger in 2nd Embodiment. It is an Example of an authentication apparatus structure which determines that the distance of a to-be-photographed object is near by the shade area of an infrared light source. It is a figure explaining the apparatus structure of the authentication apparatus of 3rd Embodiment. It is a figure explaining the apparatus structure of the authentication apparatus of 4th Embodiment. It is explanatory drawing of the guide method which image | photographs a finger in 4th Embodiment. It is explanatory drawing of the guide method which image | photographs a finger by one-hand operation in 4th Embodiment. It is explanatory drawing of another Example of the guide method which image | photographs a finger in 4th Embodiment. It is explanatory drawing of a structure of the authentication apparatus in 5th Embodiment. It is explanatory drawing of a structure of the authentication apparatus in 6th Embodiment. It is explanatory drawing of another apparatus structure of the authentication apparatus in 6th Embodiment. It is explanatory drawing of a structure of the authentication apparatus in 7th Embodiment.

  FIG. 1 is a diagram showing an entire configuration of a biometric system using blood vessels of a finger according to the first embodiment. Incidentally, it goes without saying that the present invention may be configured as an apparatus in which all or a part of the configuration is mounted in a housing, not as a system. The apparatus may be a personal authentication apparatus including an authentication process, or the authentication process may be performed outside the apparatus and may be a blood vessel image acquisition apparatus specialized for acquiring a blood vessel image or a blood vessel image extraction apparatus. Also, as described later, the embodiment may be a terminal.

  The authentication system according to the first embodiment includes an input device 2, an authentication processing unit 10, a storage device 14, a display unit 15, an input unit 16, a speaker 17, and an image input unit 18.

  The input device 2 includes the light source 3 installed in the housing and the imaging device 9 installed inside the housing. The image processing function of the authentication processing unit 10 or the image processing function including the image input unit 18 may be referred to as an image processing unit. In any case, the authentication processing unit 10 is a generic name of a processing unit that executes processing related to authentication, and a determination unit that determines the distance between a living body (finger) and a system or the posture of a living body (finger) from an image, A state control unit that instructs the display unit etc. to correct the distance to (a finger) or the posture of a living body (a finger), an unnecessary information removal unit that removes unnecessary information (wrinkles, background, etc.) from the captured image, And a matching unit that matches the extracted feature information with the registered data stored in advance in the storage device.

The light source 3 is, for example, a light emitting element such as a light emitting diode (LED), and emits light to the finger 1 presented on the upper part of the input device 2. The imaging device 9 captures an image of the finger 1 presented to the input device 2. The finger 1 may be plural.
The image input unit 18 acquires an image captured by the imaging device 9 of the input device 2, and inputs the acquired image to the authentication processing unit 10.

  The authentication processing unit 10 includes a central processing unit (CPU: Central Processing Unit) 11, a memory 12 and various interfaces (IF) 13.

  The CPU 11 performs various processes by executing the programs stored in the memory 12. The memory 12 stores a program executed by the CPU. The memory 12 also temporarily stores the image input from the image input unit 18.

  The interface 13 connects the authentication processing unit 10 to an external device. Specifically, the interface 13 is connected to the input device 2, the storage device 14, the display unit 15, the input unit 16, the speaker 17, the image input unit 18, and the like.

  The storage device 14 stores in advance registration data of the user. The registration data is information for matching the user, and is, for example, an image of a finger vein pattern. Usually, the image of the finger vein pattern is an image obtained by imaging blood vessels (finger veins) mainly distributed subcutaneously on the palm side of the finger as a dark shadow pattern.

  The display unit 15 is, for example, a liquid crystal display, and is an output device that displays the information received from the authentication processing unit 10.

  The input unit 16 is, for example, a keyboard, and transmits information input by the user to the authentication processing unit 10. The speaker 17 is an output device that transmits information received from the authentication processing unit 10 as an acoustic signal (for example, voice).

  Here, the display unit 15 and the speaker 17 are devices (instruction unit) for instructing the user who uses the authentication system to correct the distance between the living body (finger) and the system and the posture of the living body (finger). And the present invention is not limited to this device.

  In addition, each processing unit described above may perform all processing by one CPU, or may use a CPU for each processing unit.

  FIG. 2 is a diagram for explaining the structure of the input device of the biometric authentication system according to the first embodiment.

  The input device 2 captures a biological feature distributed on the surface of the finger or subcutaneously. The input device 2 is surrounded by a device housing 21, and two imaging devices 9 are disposed inside the device casing 21, and a plurality of infrared light sources 31 and visible light sources 32 are alternately annularly arranged around the imaging device 9. It is arranged and can illuminate the finger 1 uniformly through the opening. The infrared light source 31 emits infrared light, and the visible light source 32 emits visible light. The wavelength of the visible light source 32 can be arbitrarily selected from about 450 nm to about 570 nm, that is, from blue to green. The light source 31 and the light source 32 can each be irradiated with an arbitrary intensity. As an example of a specific wavelength, the light source 31 selects infrared light of 850 nm, and the light source 32 selects a green wavelength of 550 nm. In addition, the light emitting element of each wavelength may be integrated. An acrylic plate 22 is fitted in the opening to prevent dust and the like from invading the inside of the apparatus, and has an effect of physically protecting members inside the apparatus. When the light emitted from the light source 31 and the light source 32 is reflected by the acrylic plate 22, the subject disappears, so all the light sources are turned on in time series to continuously shoot the subject, and each image is HDR (High dynamic range) It may be synthesized by technology to obtain a clear object without reflection component. The light source 31 and the light source 32 may be arranged in an annular or lattice between the two imaging devices 9 instead of the periphery of the imaging device 9, and the annular or lattice so as to surround the two imaging devices 9. It may be arranged in a shape.

  The imaging device 9 is a color camera and has a plurality of light receiving elements sensitive to wavelength bands of visible light and infrared light. The imaging device 9 has, for example, three types of CMOS or CCD elements having photosensitivity to blue (B), green (G) and red (R), and these are arranged in a grid as known as the Bayer arrangement. ing. Each element of RGB is also sensitive to near infrared light. The sensitivity of each light receiving element is constituted of, for example, a sensor having a peak of light receiving sensitivity in the vicinity of 450 nm for blue, 550 nm for green, and 620 nm for red. Further, it is assumed that a color image to be captured is an RGB color image format in which each color plane of RGB can be acquired independently. It goes without saying that the imaging device 9 may be a multispectral camera having light receiving elements exceeding three wavelengths. Further, the imaging device 9 is provided with a band pass filter 33 that transmits the wavelengths of all the light output from the light source 3 and blocks the other bands, and blocks unnecessary stray light to enhance the image quality. In this embodiment, only light of wavelengths of 550 nm and 850 nm is transmitted.

  The input device 2 can capture various biological features present in the skin of the fingertip by irradiating a plurality of visible lights. For example, there are fingerprints, wrinkles of the epidermis, wrinkles of joints, patterns of melanin such as veins, spots and moles, blood, and fat lobules in the subcutaneous tissue (herein referred to as fat pattern).

  FIG. 3 shows an example of a processing flow of performing authentication while estimating the posture of a living body with a stereo camera.

  First, the system prompts the user to present a finger (S401). Subsequently, while irradiating the infrared light and the green light, the two color cameras start shooting while being synchronized (S402). At this time, if luminance saturation is observed due to external light or the like, exposure adjustment may be performed to set an exposure time at which the luminance saturation disappears. Subsequently, the distance / three-dimensional shape measurement of the subject is performed from the photographed image (S403). Then, it is judged whether or not the subject is present within the range of the distance set in advance from the authentication device (S404). If it is not present, the living body is regarded as not being presented, and the living body returns to the first processing (S401). Wait until presented. On the other hand, if the subject is within the range of the predetermined distance, it is considered that the living body is presented, and the posture detection process of the finger is performed (S405). Next, a finger guiding process is performed using the posture determination result (S406). At this time, if the position or angle of the finger deviates from the assumption, or if the finger is bent improperly, guide the user using the screen display, guide LED, buzzer, etc. so as to obtain an appropriate shooting state. Perform posture determination and guidance repeatedly. On the other hand, if it is determined that the posture is appropriate (S407), the finger image cutout processing is performed based on the posture information of the finger (S408). The posture information of the finger includes the position of the fingertip or the root of the finger, and the image of each finger is cut out using the positional information of one or more fingers to be authenticated. Then, the enlargement ratio of the image is corrected according to the position, the direction, and the distance of the finger with respect to the extracted finger image, and the posture of the finger is normalized (S409). Subsequently, a biological feature is extracted from the finger image whose posture is normalized (S410). Thereafter, the biometric feature is collated (S411), the similarity to the registered data is calculated, and it is determined whether the user is a registrant (S412).

  Here, each processing block will be described in detail.

  First, the distance / shape measurement of the subject (S403) will be described in detail. In this embodiment, reflected images of near-infrared light and green light are captured using two color cameras, and distance measurement is performed by applying a technique of stereo vision (stereo matching) using the parallax. Note that internal parameters and external parameters for converting a coordinate system between two cameras are known. It is also possible to synchronize shooting between cameras. At this time, by using a general stereo vision technique, it is possible to associate an arbitrary pixel of the left camera with an arbitrary pixel of the right camera, and obtain the distance of the subject in pixel units. As a method of associating pixels between both cameras, template matching between partial images, association by feature points using SIFT (Scale-invariant Feature Transform), or the like can be used. As a result, distance information or shape information of an object photographed in common by both cameras can be acquired, and the three-dimensional structure of the object can be grasped.

  In general stereo matching, when strong edge information does not exist in a subject, it becomes difficult to associate pixels between both cameras, and it becomes difficult to obtain accurate distance information. In particular, a reflection image of infrared light to a living body such as a finger generally has a weak edge, so that it is difficult to obtain a corresponding point. Therefore, in the present embodiment, infrared light and green light are simultaneously emitted, and the reflected light is photographed, and information such as veins that can be photographed with infrared light, and fingerprints and fatty marks with fine edges that can be photographed with green light Earn at the same time.

  FIG. 4 shows an example of an image of a vein, a fingerprint or a fat print taken by simultaneously emitting infrared light and green light. The finger veins 62 are captured unclearly in the reflection image 61 of infrared light, fingerprints of the epidermis and the like are hardly noticeable, and feature points such as edges are relatively few. On the other hand, although the finger vein 62 can not be observed substantially in the reflected image 63 of green light, the fingerprint 64, the joint wrinkles 65, and the fat print 66 are observed. Since these feature quantities have strong edges, feature points can be easily obtained by a general feature point extraction method such as SIFT, and the accuracy of matching the coordinates of both images in stereo matching is improved. can do.

  However, fingerprints, joint wrinkles, and fatty marks are often locally similar to edges of other positions. Therefore, when obtaining the feature point and its feature amount in general image processing, if the area of the local image from which the feature amount is extracted is small, it can not be distinguished from the feature amounts of other feature points, and erroneous corresponding points It may be detected. Therefore, in the case of line features such as fingerprints and joint wrinkles, only the end or branch point of the line is extracted, or in the case of granular features such as fat fingerprints, multiple edges of surrounding fingerprints or fat fingerprints are extracted. To extract the feature quantity of the point from a somewhat wide range to include. This makes it possible to suppress the aperture problem generally known as a factor of erroneous correspondence of feature points.

  Here, an embodiment will be described in which infrared and green reflection images are simultaneously obtained by simultaneously irradiating infrared light and green light.

  When infrared light and green light are simultaneously irradiated while the living body is held up, these lights are absorbed or reflected on the surface or under the surface of the living body, and the reflected light reaches the imaging device. Thereby, the reflected light intensity distribution in the vicinity of the living body surface is visualized. At this time, since the infrared light and the green light are simultaneously irradiated, the light intensity is reflected on each of the RGB light receiving elements of the color camera.

  FIG. 5 is an example of the spectral sensitivity characteristic of each element of RGB of a color camera. As shown in this figure, the RGB elements have almost the same sensitivity distribution in the infrared light region of a longer wavelength than the wavelength of about 850 nm. On the other hand, only the sensitivity of the G element is high for green light. At this time, the light reception amounts (Ir, Ig and Ib, respectively) of the three elements of R, G and B and the wavelength of the emitted LED are determined by the following simultaneous equations.

Where I 550 and I 850 are the reflected light components of green light and infrared light alone, and Wr (λ), Wg (λ) and Wg (λ) mean the sensitivities of the R, G and B elements at wavelength λ, respectively. Do. From these equations, reflected light components (I550 and I850) of green light and infrared light can be determined. However, although the number of equations is three for two unknowns, since two unknowns can be obtained by two equations, by averaging three results of taking two equations, Equalize the errors contained in the solution of unknowns.

  As another embodiment, it is possible to alternately turn on the infrared light and the green light to use each image, but the frame rate for shooting is reduced to half as compared with this method, There is a possibility that the position of the subject appearing in the images of both wavelengths may be slightly shifted. Therefore, according to the method of the present embodiment, since the infrared image and the fatty pattern can be photographed simultaneously, the subject can be photographed at high speed, and since the positional deviation between the frames of the subject does not occur, highly accurate stereo matching can be realized.

  In general, the result of distance measurement by stereo vision obtained by the above-described method is performed on a pixel basis, but an inaccurate result may be generated on a pixel basis due to the influence of image noise. Therefore, noise removal may be performed by spatially smoothing by applying a graph cut or a speckle filter.

  Subsequently, an example of posture detection of a living body performed in step S405 will be described. Various situations are assumed for the presentation method of the finger. For example, hold one finger over the device, hold five fingers open, bend a finger, etc., for finger postures, etc., or the finger position may be off the camera and not properly reflected. There is something about presentation position. Therefore, by using the result of distance measurement processing of the subject mentioned above, the posture of the finger such as the position of the finger tip and the root of each finger and the presentation angle of the finger is estimated, and based on this, it becomes appropriate presentation position and posture Obtain information to guide the user or to make corrections on the image.

  FIG. 6 shows an example of a method of detecting the presentation position of a finger. First, as shown in FIG. 6A, the distance image 100 is integrated toward the x-axis to obtain a projected luminance distribution 101 obtained by taking an average value. Here, it is assumed that the brightness value of the distance image is higher as the distance is shorter, the right direction of the image is the x-axis direction, the lower direction is the y-axis direction, and the direction away from the camera (the paper surface) is the z-axis direction. At this time, the projected luminance distribution of the portion where the finger is observed has a high value, and the projected luminance distribution of the portion where the subject is not present, such as a finger gap, has a low value. In addition, since a plurality of fingers may appear in an image, there are a plurality of unevenness of the projection luminance. At this time, as shown in FIG. 6B, if the peak of the projection luminance is determined, the coordinates of the plurality of fingers in the x-axis direction can be determined.

  Once the rough presentation position of a plurality of fingers can be grasped, next, the region of interest 102 centered on each finger is defined. The region of interest 102 of the finger sets a threshold for the projected luminance distribution 101 as shown in FIG. 6B, and the local maximum point beyond it is the center in the x-axis direction, and an arbitrary width sufficiently wider than the width of the finger. Is an area having Further, the height of the area is the same height as the distance image 100. In the present embodiment, since three fingers are photographed at the same time, three regions of interest are set.

  In addition, it is also possible to process only one specific finger instead of the plurality of photographed fingers being the targets of the authentication process. FIG. 6C shows an example of processing for detecting a specific finger. Considering that lens distortion is reduced at the center of the image and that the user places a finger directly over the authentication device, it is considered most rational to target the finger image that appears at the center of the image as the processing target . Therefore, only the finger close to the center of the image is taken out. First, the projection distribution 101 on the x-axis is multiplied by a weighting factor 103 that increases in value toward the center of the image, and conversion is performed so that the value of the projection distribution closer to the center of the image becomes higher. Thereafter, the maximum value 104 of the projection distribution is determined, and this x coordinate is defined as the position of the finger closest to the center of the image in the x axis direction. This makes it possible to detect a finger appearing at the center of the image rather than a finger appearing at the edge of the image.

  Next, finger contour detection is performed on each region of interest. FIG. 7 is an explanatory view of an embodiment for detecting a contour of a finger and a fingertip. The contour detection of the finger may divide the finger and the background based on a method such as binarization processing based on discriminant analysis or graph cutting in the pixel value in each region of interest, and extract the boundary, and connect strong edges. Alternatively, it may be taken out by a tracking method. At this time, since there is a possibility that the adjacent finger may be included in the region of interest, the edge weight is increased closer to the center of the image to minimize the distance and curvature of the edge while maximizing the total amount of the edge. A connection method may be adopted. As a result, it is less likely to accidentally extract the contour of the next finger, and an effect of reducing an authentication error can be obtained. By this process, finger contour 120 can be obtained as shown in FIG.

  Once the finger contour 120 of each region of interest can be detected, then the finger centerline 121 is determined. The center line 121 of the finger may be obtained by linear approximation of the finger contour 120, or an inner region of the extracted finger contour is defined as a finger region, and a principal component analysis is performed with each pixel of the finger region as an element It may be determined as the direction (main direction) of the obtained first main component. Next, the fingertip 122 is determined using the finger contour 120 and the center line 121 of the finger. In this case, the intersection point of the center line of the finger and the contour line of the finger may be defined as the fingertip 122, or the projection distribution is calculated toward the y axis of the region of interest 102, and the image is directed from the lower side to the upper side. The value of the luminance distribution may be examined, and the y-coordinate below the threshold that can be determined to be background first may be determined as the fingertip. In this method, since the rough position of the fingertip can be determined without detecting the finger contour, the processing can be simplified and speeded up.

  By the above processing, the presentation position of the x coordinate, y coordinate, and z coordinate of each finger is detected, and the position of the finger contour and the fingertip can be extracted.

  Next, the posture of the finger is detected by using the measurement result of the distance and solid shape of the finger. The posture of the finger includes not only the finger x, y, z coordinates, but also bending or warping of the finger and rotation about each of the x, y, z axes, and detecting these states Is required. In addition, a finger shall be divided into three clauses by two finger joints, and shall analyze the skeletal structure of a finger which approximated the central axis of each clause in a straight line. Further, in the coordinate system shown in FIG. 6A, rotation about each of the x-axis, y-axis and z-axis is defined as pitching, rolling and yawing, respectively.

  FIG. 8 shows an embodiment for detecting the posture of a finger. As shown in FIG. 8A, in this example, the finger 1 including two joints 150 is captured in the region of interest 102 for the distance image, the finger is lightly bent, and further includes rolling. Depending on the position of the presented finger, it may be assumed that only one joint is included or no joint is included unlike the illustrated figure. In addition, it is assumed that the brighter area closer to the camera is the region inside the finger contour 120. First, as shown in FIG. 8B, the distance image is partially differentiated to calculate a normal vector 140 of each coordinate. Since the finger 1 has a shape similar to an elliptical cylinder, it can be seen that the normals are distributed radially. Next, as shown in FIG. 8C, the normal vector of the pixel through which the central axis 121 of the finger passes is examined. This normal vector is shown in FIG. The normal vector points in various directions due to the swelling of the finger tip and the mid-segment of the finger. At this time, since the joint 150 is a recess, the normal vector of the position crossing the joint points in the direction in which the directions cross. Therefore, the angle between the normal vector and the central axis 121 of the finger is determined, and the position where the direction is reversed is detected as the estimated position 151 of the joint. Finally, for the region of each node divided by the estimated position 151 of the joint, the direction most orthogonal to all the normal vectors is determined by principal component analysis, and this is used as the central axis 152 of the node. By such processing, it is possible to obtain a skeletal structure for each finger node. Further, the pitching of the finger can be obtained by linear approximation of the distribution of the distance value of the finger internal region along the central axis 152 of the node, and can be obtained as the inclination angle. The yawing is obtained from the inclination of the central axis 121 of the finger.

  For finger rolling, consider the cross section of the finger as an ellipse, estimate the major axis and minor axis of the ellipse from the coordinates of the finger surface of the observable part, and obtain the rolling angle from the angle formed by the major axis direction and the x axis Can. Specifically, first, as shown in FIG. 9A, when a cross section 160 of a finger passing through a certain target pixel 163 and orthogonal to the center axis 152 of the node determined above is defined, this cross section and the finger surface The curve formed by is an ellipse 161. At this time, not all the ellipses 161 can be observed, but only a portion where a plurality of normal vectors 140 obtained as an image are distributed is observed. For this part, the x, y, z coordinates of the finger surface are known. That is, only a part of the ellipse 161 is observed. By applying, for example, the Hough transform to this partial curve, the major axis 162 of the ellipse 161 and the length and direction of the minor axis can be obtained. Then, by changing the pixel of interest 163 to various positions in the direction along the center axis 152 of the node of the finger, a number of major axes 162 can be obtained, but the average value of the angles formed by all the major axes 162 and the x axis is determined Thus, the average rolling angle of the node of the corresponding finger can be estimated. Finally, if the average rolling angle of all the nodes is estimated and averaged, the average rolling angle of the finger can be obtained.

  As another example of acquiring finger posture information described above, a large amount of finger images for which finger posture information is known is used as teacher data, and based on machine learning such as Random forest, the finger distance information from unknown distance images is obtained. Posture information may be estimated.

  From the above, it is possible to grasp the positions of the x, y and z axes of the fingers, the bending and warping of the fingers, and the state of rotation.

  In addition, the position of x, y, z axes of each finger, the bending angle of the finger, and the rotation angle are accumulated in time series for each frame, and the movement speed or movement acceleration related to the position or posture of the finger is calculated from the information. These can also be used for finger pose determination. For example, when it is detected that the finger is moving at high speed, the image is blurred, so the user is guided not to move the finger at high speed, or when the finger is hardly moved, it happens accidentally Since there is a possibility that the finger has been photographed, it is also possible to explicitly indicate that authentication has been made by urging the finger to move. Also, by allowing shooting only when the user moves away from the camera, it is possible to shoot an image of an image quality that the tendency of blurring of the subject substantially matches. As a result, the image quality at the time of registration and at the time of input can be unified, so that the authentication accuracy can be improved.

  Next, one example of the display (S406) of the guide for guiding the finger shown in FIG. 3 will be described in detail. As described above, when the presentation position or posture of the finger can be detected, it is possible to guide the finger being photographed to an appropriate position.

  FIG. 10 shows an example of a guidance screen for guiding the position and posture of a finger suitable for authentication to the user. FIG. 10A shows a state where a finger contour guide 180 is displayed at a position suitable for authentication, and a guide message 181 for prompting a finger is displayed. On this screen, the images taken by the imaging device are displayed overlapping each other, so that the user moves the finger to the right so that the user moves the finger to the right so that the user can easily operate it. Reverse processing of The image displayed here may be a distance image obtained by combining the images of the left and right cameras, or may be an infrared light reflection image or a green light reflection image of either the left or right camera. In addition, the finger area may be filled for concealment of biological information.

  The user holds his finger in line with this indication. However, since the outline of the finger contour does not conform to the shape of any finger, it can not necessarily match the shape of the contour of the finger that is held up. Furthermore, since it is necessary to adjust not only the two-dimensional planar alignment but also the camera distance to an appropriate position, an appropriate presentation position even if the finger is held to match the displayed finger contour guide It is not always held up. Furthermore, it is not easy to align the positions of a plurality of fingers, and not all fingers may be in positions suitable for authentication. Therefore, it is necessary to guide while changing the contents of the guide sequentially.

  FIG. 10B shows an example when holding three fingers. First, the finger movement instruction arrow 182 in the left direction is displayed such that the finger detected at the center among the three fingers detected by the above-described processing is positioned at the center of the image. This allows for an overall alignment of the finger. Next, as shown in FIG. 10C, even when the central finger has moved to the center of the image, the left and right fingers are misaligned, so that it is notified that the finger is lightly closed, and A finger movement pointing arrow 182 pointing right or left is displayed near each finger to guide the position. Similarly, when the finger is unnaturally closed, it may be displayed that the finger is lightly opened.

  FIG. 10 (d) shows a state in which the finger is shifted to the right and is rotated, and is close to the camera position and the joint is bent. First, an arrow for moving the finger to the left is displayed, and then a camera distance indication icon 183 for releasing the camera distance is displayed. Thereafter, a rotation instruction icon 184 for restoring the rotation of the finger is displayed. Finally, instruct to stretch the joints of the finger straight. At this time, if the finger tip is too low at the bottom of the screen, this may be displayed.

  As described above, when the first and second positions of the finger and the camera distance are guided and then the rotation such as pitching and the bending and stretching of the finger are guided, the guidance can be guided from the rough presentation position in the order of the fine adjustment. It becomes possible.

  As described above, based on the above-described posture detection result, the presentation position and the posture of the finger can be guided to the correct position.

  In addition, although the number of imaging | photography numbers of the finger | toe suitable in the above-mentioned Example assumed three, imaging | photography may be implemented if it is between 1 and 5 pieces. When the number of fingers is large, guiding the average position of the rightmost finger and the leftmost finger to be at the center of the image allows the entire finger held to be balanced at the angle of view. Can be Alternatively, the number of fingers currently being held can be detected in real time, and the number of displayed finger contour guides 180 described above can be switched for each number of fingers to indicate an appropriate position. By these processes, a plurality of fingers can be efficiently presented, and more accurate multi-finger authentication can be realized.

  Next, step S407 for determining whether the finger is at an appropriate position will be described. Although the position and posture of the finger suitable for authentication can be uniquely determined, if the user does not allow any deviation from it, the user takes a long time to present the finger and the convenience is degraded. Therefore, an allowance is provided for the amount of deviation from the ideal state in the parameters such as the number of shooting fingers, the x, y, and z positions of the fingers, the rotation, and the bending and stretching state, etc. It is determined that the posture is correct. In addition, when the number of photographing fingers is set to, for example, three as an appropriate value, at the time of registration, guidance is continued until three are included in the image to secure the data amount, but at the time of authentication It is good to accept it. This is because there is a case where even one match can be determined probabilistically and clearly as the correct person, by doing so it is possible to ease the restriction of the finger posture at the time of authentication, and a highly convenient operation Can be realized.

  Next, an embodiment of the finger cutout process step S408 and the three-dimensional correction process step S409 will be described.

  As described above, even when finger guidance is performed, photographing at an ideal position is difficult in consideration of the convenience of the user, and thus includes finger posture variations. In this case, due to the deformation of the biological feature accompanying the change in posture, a feature amount different from the pattern at the time of registration can be obtained. On the other hand, for example, it is necessary to carry out matching using feature quantities that are robust to deformation such as SIFT feature quantities or to correct deformation of the original image. In this embodiment, on the premise of matching by template matching having a low processing cost, detection results of finger posture information (here, a state in which a finger tip, a joint, and a root are labeled in a three-dimensional shape of a finger) are used. The biological image of the two-dimensional plane being photographed is geometrically transformed and corrected to a normalized state.

  First, for the region of interest 102 determined with respect to the distance image 100 as shown in FIG. 11A, the fingertip position determined above for the infrared light reflection image and the green light reflection image 200 as shown in FIG. A cutout image 201 of the finger is generated on the basis of. As a result, the fingertip comes in contact with the upper side of the cutout image 201, and the positional deviation of the x axis and the y axis is corrected. The posture information such as the fingertip position, the finger contour, and the central axis of each finger node obtained as described above is obtained for the distance image 100, and the coordinate system for the reflection image captured by the two left and right cameras. There is a gap. Here, it is assumed that the deviation of the coordinate system between the distance image 100 and the reflection image 200 can be calibrated, and is determined by coordinate conversion. When an error occurs in coordinate conversion, the total edge amount of all pixels on the finger contour is measured while translating the finger contour obtained in the distance image in parallel on the reflection image, and the total edge amount on the finger contour is the largest. The position may be corrected to the place. Further, the reflected image 200 may be an image of either the left or right camera, or may be an image obtained by combining both the left and right images. Coordinate conversion is possible in any case.

  Next, as shown in FIG. 11C, the image size is enlarged or reduced such that the distance value of the finger obtained in the distance image becomes constant with respect to the contour of the finger shown in the cutout image 201. The calculation of the magnification is determined by the ratio of the optimal distance value to the current finger distance value. Thereby, even if there is a shift in the z-axis direction, the size of the finger can be kept constant. At this time, by partially enlarging or reducing the image so that the distance value of the partial region of the finger is constant, it is possible to eliminate the non-uniformity of the distance due to the pitching angle of the finger.

  Then, as shown in FIG. 11 (d), rotation correction is performed so that the central axis 152 of each finger segment points upward in the image. Since the central axis 152 may have different inclinations for each node of the finger, an average value of the central axis for each node may be obtained and rotational correction may be performed so that it points upward in the image.

  Finally, as shown in FIG. 11E, the coordinates of the two-dimensional plane are projected and transformed in a direction in which the ellipse 161 of the cross section of the finger obtained above and its major axis 162 are parallel to the x-axis of the image. Here, the x axis after projection conversion is illustrated as an x 'axis, but hereinafter this is also referred to as an x axis. As a result, even if the appearance of the two-dimensional image of the finger is deformed by rolling, it is converted into a projected image in which the ventral side of the finger is directed to the front, so that an image without deformation can be obtained.

  By the above-described processing, correction of the x-, y-, and z-axis coordinates of the finger and the rotational direction can be performed. The same effect can be obtained even if the correction shown here is performed after feature extraction processing described later.

  Finally, the feature extraction process (S410), the matching process (S411), and the determination process (S412) will be described. The feature extraction processing extracts feature amounts for each of the infrared reflection image and the green reflection image. Since information such as veins is confirmed in the infrared reflection image, and fingerprints, joint wrinkles, fat marks and the like are confirmed in the green reflection image, general vein pattern extraction processing and texture pattern extraction processing are performed. , Acquire biometric features. In particular, in order to sharpen the line pattern, unsharp mask processing may be performed as preprocessing. Although there are two video images for the left and right cameras, it is also possible to use only the video from one camera, or separate image extraction may be performed separately using both videos, and the pattern obtained at the end may be synthesized Alternatively, they may be used as independent patterns without being synthesized. In any case, since both videos have parallax and can be used as different information, accuracy improvement can be expected by using videos of a plurality of cameras. With regard to the matching process, the extracted pattern is determined for similarity by general template matching or feature point matching. Then, when the obtained degree of similarity falls below a predetermined threshold, the authentication is successful. In addition, although a plurality of biometric features can be obtained by using infrared images, green images, and images of a plurality of cameras, it is possible to obtain an average similarity of all when compared, or an average of a plurality of high similarity results. The determination of authentication may be performed based on the degree of similarity. Calculation of a plurality of similarities increases data redundancy and contributes to accuracy improvement.

Next, the difference between the guiding method at registration and the guiding method at authentication will be described in detail.
At the time of registration, it is desirable to photograph the living body in a state as stable as possible. Therefore, registration data is created after continuing guidance to an ideal presentation position so that the imaging | photography state of a biological body may become good. For example, the living body to be photographed appears as central as possible, photographing a wide area as much as possible, that the rotation of the finger is small and facing the camera, included in the set depth of field of the camera, the finger bent Guide your finger to meet that, etc.

  However, it is difficult to exactly match the position of the finger in a hollow position, and it may be assumed that some users can not stop at the ideal presentation position, and the convenience deteriorates. Therefore, the induction needs to have a certain margin.

  Further, as described above, even when the captured image is geometrically corrected, it is not always possible to capture an image of the same image quality when the position of the finger actually changes. Therefore, a plurality of registration data with various finger postures are acquired, and at the time of collation, collation with all the registration data is performed to acquire a plurality of similarities, and based on these, the final authentication judgment Conduct. This method is effective for performing robust authentication against posture variations because the registration data has various finger posture variations. At this time, when guiding the finger at the time of registration, if the reference of guiding the finger is the same, only shooting of a constant finger posture is always performed, and it becomes difficult to include variations in the finger posture.

  Therefore, the first one is guided so as to be photographed at the position where the image quality is the highest as described above, and then photographed at a position where the magnification ratio is a little close to the camera. Finally, shoot with the finger away from the camera. If these three sheets are registered, even if a finger held at the time of authentication changes so as to approach or move away from the camera with respect to the standard position, any one sheet has higher similarity. As a result, robust matching can be realized against fluctuations in magnification.

  Similarly, it is possible to take various combinations of finger postures, such as photographing in three cases of guiding to the standard position, guiding to the left from the standard position, and guiding to the right from the standard position. Needless to say. Since it is possible to consider finger postures in which the fluctuation of the living body becomes larger according to the characteristics of the camera and the shape of the device, it is more effective to guide the finger postures with large fluctuation. Such guidance can realize robust authentication against finger posture change.

  On the other hand, at the time of authentication, by requiring strict alignment, the operation time becomes longer and inconvenience of alignment occurs, which reduces the convenience. Therefore, the allowable amount of induction deviation is taken larger than at the time of registration, and the position deviation is absorbed by performing correction based on the posture information of the finger. At this time, in the case of a system configuration capable of capturing an image in real time and performing verification, if the finger posture close to the registered data is achieved somewhere while holding up the hand, authentication succeeds, so the positional deviation is relatively Can be correctly authenticated even if Therefore, even if the finger is not at the appropriate position, the authentication process may be performed, and the processing flow may be changed so as to display that the position is shifted on the screen.

  As described above, it is possible to capture an image of high quality at the time of registration, and to perform an authentication process by a highly convenient operation at the time of authentication.

  FIG. 12 is a diagram showing an example of the configuration and operation method of the authentication device for easily guiding the position of the finger. When performing non-contact shooting, since the finger is held in the air, it is difficult to know which area the camera is shooting without checking the screen, and it is also difficult to grasp the distance from the appropriate camera. Therefore, as shown in FIG. 12A, the fingertip guide mark 220 and the finger root guide mark 221 are placed on the device housing 21. The guide mark is indicated by a triangle, and the direction of the apex of the triangle indicates the fingertip side. This makes it possible to show the relationship between the orientation of the device and the finger. In addition, these marks should just show the direction and direction of a finger | toe, and a shape is not limited to a triangle.

  As shown in FIG. 12B, the user places the fingertip of the finger 1 on the fingertip guide mark 220, and places the finger root so as to be between the two finger base guide marks 221. The finger is then moved away from the device as shown in FIG. 12 (c). As described above, when the finger is first brought into contact with the device housing 21 and the finger is moved away from it, the possibility of passing through the optical axis center of the camera and the optimum distance from the camera increases, and a higher quality living body can be obtained. Shooting can be realized. In the present embodiment, the guide mark is arranged at the center of the housing, but the finger may be arranged so as to be easily moved to an appropriate photographing position according to the position of the camera and the number of fingers to be photographed. Further, the guide mark may be a recess formed in the device housing 21 according to the shape of the finger, and the finger may be placed there.

  An example of the process flow of the non-contact authentication apparatus in which a guide mark is arranged in the apparatus is shown below. First, the system gives guidance to the user, "Please put your finger on the device". Then, the light source is turned on to wait for the presentation of a living body while continuously capturing images. If the living body is placed on the finger rest, the image is saturated due to the strong reflected light of the light source. Since whether or not the image is saturated is determined by the distance between the light source and the finger, threshold processing is performed to determine in advance the average luminance value when the finger is placed on the device housing 21 and to determine whether the finger is placed. The placement of the finger can be determined by setting and placing it. When the user places a finger in accordance with the guide mark, it is determined that the finger is placed by the finger placement determination process.

  Then the system gives guidance to the user saying "Please keep your finger away". Accordingly, as shown in FIG. 12C, when the user moves the finger away from the device, the intensity of the reflected light gradually decreases, and it can be seen that the finger and the device are separated. Then, the above-described distance measurement process is performed, and when it is detected that the finger is at an appropriate distance, the feature amount of the living body is extracted from the image at that time. In the case of the registration process, the biometric feature is registered, and in the case of the authentication process, the biometric feature is used as input to collate with the registration data.

  According to the present embodiment, by placing the finger on the guide mark of the finger, the user can photograph the living body without the finger being largely deviated from the angle of view or the appropriate photographing distance from the camera. There is an effect that it is possible to determine an optimum photographed image in a series of operations without stopping the camera in the air.

  Note that, as described in the example of the guidance method at the time of registration described above, the threshold value of the optimum photographing distance is changed according to the number of registration trials when acquiring a plurality of registration data. Registration data may be included. This makes it possible to maintain the similarity to the registered data even if the distance of the finger changes at the time of authentication.

  Another method of operating the authentication device for easily guiding the position of the finger is to tap the device. The above-mentioned finger-removing operation is difficult to perform unless the device is fixed or held by one hand. Therefore, instead of the operation of moving the finger away, tapping in the vertical direction is performed such that the device is held with one hand and the fingertip of the same hand is placed at a predetermined position, and then the fixed distance is moved back again. As a result, it is possible to capture a living body image with an optimal camera distance only by one-handed operation.

  As shown in FIG. 13 (a), the user first grasps the device casing 21 and places, for example, a forefinger 240 on a guide mark of a fingertip. When the system instructs to tap a finger, the user starts tapping. At the time of tapping, since the direction of the finger is an oblique presentation as shown in FIG. 13B, the photographed finger image is also photographed in a state in which pitching occurs. However, if the entire finger is at an appropriate angle that fits within the angle of view of the image, and if the registration data is stored in a similar posture, it may be used for authentication. Therefore, the operation of moving the finger up and down is repeated several times, and several shots are taken at the timing when the finger falls within the angle of view. Then, feature data of the living body is generated from these images and registered. If the same operation is performed also at the time of authentication, the similarity to the registered data is increased at the moment when the finger posture becomes close to the state at the time of registration, and the authentication succeeds.

  In this way, by performing the tapping operation, the presentation position of the finger is contained within a certain range, so that the reproducibility with the registered data is enhanced, and the finger is easily positioned in the air without performing the alignment of the finger in the air. Since it is possible to take a finger and to carry out authentication while holding the device with one hand, the convenience of the device is improved.

  FIG. 14 shows an embodiment of a reflection non-contact biometric authentication device constituted by a light source emitting infrared light and one infrared camera according to the second embodiment.

  A plurality of infrared light sources 31 are arranged around the imaging device 9, and a band pass filter 33 for transmitting light only in the vicinity of the wavelength emitted by the infrared light source 31 is mounted on the lens portion of the imaging device 9. The imaging device 9 is an infrared camera, and can capture light reflected by the finger 1.

  FIG. 15 shows an example of the process flow in this embodiment. This is substantially the same as the processing flow shown in FIG. 3 of the first embodiment described above. However, the difference from the above is that one infrared camera is used instead of two color cameras, no distance measurement with stereo vision is performed, and a single wavelength image is used instead of multiple wavelengths. . The outline of the processing flow will be described below.

  First, the system prompts the user to present a finger (S501). Subsequently, imaging is started with one infrared camera while emitting infrared light (S502). At this time, if the external light is strong, exposure adjustment is also performed. Subsequently, distance and solid shape estimation of the subject is performed from the photographed image (S 503). If it is determined that the subject is present within the range of the distance set in advance from the authentication device (S504), it is regarded that the living body is not presented and waits until the living body is presented. On the other hand, if the subject is within the range of the predetermined distance, it is considered that the living body is presented, and the posture detection process of the finger is performed (S505). Next, a finger guidance process is performed using the posture determination result (S506). At this time, if the position or angle of the finger deviates from the assumption, or if the finger is bent improperly, guide the user using the screen display, guide LED, buzzer, etc. so as to obtain an appropriate shooting state. Perform posture determination and guidance repeatedly. On the other hand, when it is determined that the posture is appropriate (S507), the finger image cutout processing is performed based on the finger posture information (S508). The posture information of the finger includes the position of the fingertip or the root of the finger, and the image of each finger is cut out using the positional information of one or more fingers to be authenticated. Then, the enlargement ratio of the image is corrected according to the position, the direction, and the distance of the finger with respect to the extracted finger image, and the posture of the finger is normalized (S509). Subsequently, a biological feature is extracted from the finger image whose posture is normalized (S510). Thereafter, the biometric feature is collated (S511), the similarity to the registered data is calculated, and it is determined whether the user is a registrant (S512).

  Here, processing blocks in which there is a difference between FIG. 15 which is the present embodiment and FIG. 3 which is the first embodiment described above will be described in detail.

  First, the irradiation of infrared light and the photographing process (S502) by the camera will be described. In a scene where a biometric authentication device is put to practical use, various objects exist around the imaging device. Therefore, when photographing infrared reflection images, unnecessary subjects other than fingers are often photographed. Unnecessary subjects may be noise for correctly detecting fingers, and therefore it is necessary to remove unnecessary subjects in order to perform more accurate authentication. As one of the methods, as shown in FIG. 14, there is a method of mounting a band pass filter 33 which transmits only the wavelength of the light source 31. On the other hand, if reflected light of the same wavelength range exists in the environment, that component can not be removed. So, in a present Example, background removal is performed by the difference of the lighting image and light-extinguishing image of a light source.

  First, an image with the light source 31 turned off is photographed. Immediately after that, an image in which the light source 31 is turned on is photographed. Then, when the image captured at the time of turning off is subtracted from the image captured at the time of lighting, it is possible to remove the component of the unnecessary subject that appears bright regardless of the light source 31 while the image of the finger held up is left. Thus, the fingers can be cut out accurately regardless of the influence of external light. Further, if the external light component is irradiated on the surface of the finger, it is canceled by the calculation of the difference, so that malfunction due to the external light of the posture detection process described later can be reduced.

  Moreover, when the image which turned off the light source is very bright, it turns out that external light is irradiated strongly. At this time, by irradiating the light source strongly according to the brightness and controlling the exposure time of the camera to be short, measurement of the living body can be realized even in an environment where strong external light is irradiated. When adjusting the exposure, since the optimum parameters necessary for processing change in the posture detection processing of the finger or the like described later, the optimum parameters for the respective exposure times are determined in advance in each processing.

  As described above, in the present embodiment, an unnecessary background can be removed by the background difference due to the blinking of the light source, and the finger detection accuracy can be improved. However, this method reduces the frame rate for shooting by half, so if the frame rate is not high enough, use the bandpass filter 33 to remove unnecessary background, or an infrared reflection image in which the finger and the background are already separated. It is also possible to acquire a large amount in advance and apply a method of separating a finger and a background by machine learning such as Random forest.

  Next, measurement of the distance and solid shape of the subject (S503) will be described in detail. In this embodiment, since distance measurement by stereo vision can not be performed as in the above-described embodiment, it is necessary to perform distance measurement with one infrared camera.

  As a method of performing distance estimation with one infrared camera, there is a method using the inverse square law with respect to the light intensity of the reflected light irradiated. Since the light source is placed around the camera and can be regarded as a point light source in general, the light emitted toward the finger spreads like a sphere. Since the surface area of a sphere increases in proportion to the square of the radius of the sphere, the energy of light per unit area is inversely proportional to the square of the radius. Therefore, the brightness of the subject is determined by the reflectance and absorption rate of the subject, and the distance between the subject and the camera.

  The light quantity to be irradiated is Ii, the reflection / absorption coefficient on the skin is μ, the distance to the subject is D, and the light quantity to receive light is Io. Further, assuming that the reflected light on the finger surface is Lambertian reflection and the reflected light diffuses isotropically, the following approximate expression holds.

  Here, since the irradiation light intensity Ii and the average reflection intensity μ on the skin can be investigated in advance, the distance of each pixel of the image can be grasped by solving for the distance D. However, although the amount of irradiated light and the amount of received light are detected as the luminance value of the image, the luminance is generally saturated if Ii is directly measured as the luminance value, so the irradiated light amount Ii is physically determined by a filter whose attenuation factor is known. The image is photographed in the attenuated state, and calibration such as increasing the light amount Ii by the attenuated amount is performed. Alternatively, a mirror or the like that directly reflects the irradiation light Ii may be mounted in the apparatus so that the irradiation light Ii can be directly observed through the filter whose attenuation factor is known. By this method, it is possible to obtain a distance image from the luminance value of the finger that is held up.

  In addition, a camera capable of accurately measuring the distance and the imaging device of the present embodiment are juxtaposed to capture a finger, and a large amount of learning data in which the correct distance label is assigned to each pixel is collected. Based on the learning, a distance image of the finger may be determined from the infrared reflection image.

  However, there is also a possibility that an image brighter than expected may actually be taken due to the influence of external light and the like. In that case, the absolute value of the distance D includes an error. Therefore, it is also possible to grasp the three-dimensional structure of the subject by obtaining a relative distance distribution, where the distance of the subject at an arbitrary position of the image is 1. In addition, a rough conversion table of the luminance value and the distance value of the image is prepared in advance, this distance value is read from the luminance value of the pixel of the subject at the arbitrary position, and this distance value is used as the relative distance value of the entire image. Rough value images can also be obtained by multiplying the values.

  Next, the difference from the first embodiment will be described with regard to finger posture detection (S505). Here, the same processing as in the first embodiment may be performed using the method of acquiring the distance image described above, but as a simpler method, the posture of the finger is not converted to the distance image in the infrared reflection image. The example which detects is shown.

  First, the infrared reflection image is regarded as a distance image, and the region of interest 102 is acquired by the method shown in FIG. 6 of the first embodiment. However, the distance is considered to be closer as the luminance value of the reflected image is higher.

  First, with regard to position detection in the x-axis direction of the finger, as shown in FIG. 6 of the first embodiment, the infrared reflection image can be obtained from the coordinates in the x-axis direction obtained when determining the region of interest by regarding it as a distance image. . Further, the position detection of the finger tip in the y-axis direction can be obtained by the same method as shown in FIG. With regard to the distance to the camera (position detection of z axis), the law of inverse square holds between the brightness of the reflected image and the light source distance, and there is a correlation between the two, so By examining in advance the relationship between the average luminance of the finger and the camera distance, it is possible to create a conversion table from the average luminance to the camera distance. Then, when detecting the contour of the input finger, the average luminance inside the finger is obtained, and the camera distance is calculated from the average luminance value using the conversion table. This makes it possible to detect that the finger is too close to the camera when the luminance value is too bright, and that the finger is too far from the camera when the luminance value is too dark. Although this value changes due to the influence of external light, if the brightness due to the external light is grasped by photographing the image when the light source is off, the influence of the external light can be mitigated by subtracting the value.

  As to the detection of the yawing, as in the above-described method shown in FIG. 7, the central axis 121 of the finger is determined, and rotation correction is performed so that this axis is parallel to the image y axis.

  Regarding the detection of pitching, FIG. 16 shows an embodiment of a detection method. In this example, it is assumed that the fingertip is pitching and rotating in a direction away from the camera. First, the average luminance projection distribution 301 is calculated toward the y-axis with respect to the cutout image 300 of the finger appearing in the infrared reflection image. Here, the average luminance projection distribution at the coordinate y is described as Iave (y). Since the fingertip is far from the camera, Iave (y) has a low value near the fingertip. In addition, since the reflection image is accompanied by a change in luminance due to the presence of fine irregularities and blood vessels in the finger, fine relief is also observed in Iave (y). Next, the relative distance Dr (y) shown in the following equation 7 is calculated.

Here, yc is an image center coordinate of y axis of cutout image 300 of finger, Wa is an arbitrary proportionality coefficient, Wb is a calibration coefficient described later, and Sqrt {x} is a square root of x. Further, let D '(y) be a value proportional to the distance of the light source to move in the oblique direction from the coordinate y to D' (y), and let D (y) be a value proportional to the vertical distance from the height of the light source to the coordinate y (Fig. 16). (B)). The relative distance Dr (y) is a relative distance at each coordinate y when the vertical distance at the coordinate yc is defined as zero.

  It is estimated that the distance D (y) or D 'of the coordinate y at which Iave (y) is a small value is far. However, as shown in FIG. 16B, if the flat plate is placed horizontally in front of the camera, D (y) calculated from the average luminance projection distribution 302 of the flat plate should be constant regardless of y. D '(y) changes because the luminance value becomes dark at the y coordinate far from the light source. Here, since it is inconvenient to change the distance depending on the y coordinate, the distance D ′ (y) is converted to the distance D (y) by Equation 6. Equation 6 includes parameter Wb, which is determined in advance so that D ′ (y) takes a constant value regardless of y when the horizontal flat plate is held in front of the camera.

  FIG. 16 (c) shows the flow of calculation. First, D '(y) is obtained from Iave (y) as shown in equation 5 (the vertical axis is not shown in the figure). Next, this is converted to Dr (y) using Equation 6 and Equation 7. Then, the obtained Dr (y) is linearly approximated by the least squares method to obtain an estimated pitch straight line 303 of the finger. Finally, the slope of this straight line is determined. The inclination at this time is the estimated pitch angle, and if this angle is larger than a fixed value, a warning of pitch deviation is warned, or the infrared reflection image is normalized by the pitching correction method described in detail in the first embodiment. To

  Finally, with regard to rolling, the presence or absence of rolling and the direction of rotation thereof are estimated by the direction of the joint of the finger and the shape of the contour. Specifically, a large amount of rotational images of a finger whose rotational angle is known is collected, and this is learned by, for example, a convolutional neural network to estimate the rolling angle of the unknown image. Using the result, a warning is issued when the rotation angle exceeds a certain value, or the infrared reflection image is normalized by the rolling correction method described in detail in the first embodiment.

  According to the above method, the posture of the finger can be estimated without obtaining the distance image of the subject, and the amount of deviation from the standard posture can be determined, and the result is used to give guidance to the user or the pattern. Can be corrected.

  FIG. 17 shows an example of the configuration of an authentication apparatus that determines that the distance of an object is short due to the light shielding area of the infrared light source. A light shielding plate 320 is disposed above the imaging device 9 mounted on the input device 2 so as to surround the outer periphery thereof. The light shielding plate 320 is installed so as to hide the inside of the infrared light sources 31 arranged on the circumference, and shields part of infrared light. At this time, when the finger 1 approaches the input device 2, an area 321 where light does not reach occurs as illustrated. At this time, when the image is observed, an image in which the vicinity of the center of the imaging device 9 is dark and the periphery thereof is bright is captured. Therefore, by defining double circles with different radii centered on the center of the image and comparing the average brightness of the inner circle with the area surrounded by the inner circumference and the outer circumference, etc., Detects situations where the center of the image is dark and the surroundings are bright. If this situation is detected, it can be determined that the finger is too close and a warning is given to the user. Note that it is also possible to estimate the distance between the finger and the camera by measuring the area or radius of the central dark area.

  Next, the feature extraction process (S510) in the second embodiment will be described in detail. In the present embodiment, since a reflection image of only infrared wavelength is obtained, it is impossible to carry out image processing for a plurality of wavelengths as shown in the above-mentioned embodiments. Therefore, a vein pattern, a joint wrinkle pattern, and a finger contour shape that are likely to be reflected in a near infrared reflection image are extracted as feature amounts. First, to extract the veins, an unsharp mask is applied, which emphasizes the longitudinal lines of the finger more strongly and removes noise. This emphasizes the vein pattern, which tends to travel in the longitudinal direction of the finger, and blurs other information. By applying a finger vein pattern extraction process generally used as an existing technique from this image, a finger vein pattern can be extracted efficiently and stably from the infrared reflection image. Next, an unsharp mask is applied to emphasize lines in the direction orthogonal to the longitudinal direction of the finger more strongly and to remove noise so that the joint wrinkle pattern is emphasized. And if a pattern is extracted similarly, especially the line pattern of joint wrinkles will be extracted. Since the shape of the line pattern and the distance between the first and second joints have individuality, they can be used as authentication information. The finger contour information can be similarly obtained by the finger contour detection process described in the above embodiment. By this process, biometric features of veins, joint creases, and finger contours can be acquired.

  These pieces of information are acquired and registered as a plurality of biological patterns, each of which is collated independently, or each of which is collated with constraints on its positional relationship, and finally the collation results of each modal are generally The fusion is achieved by multi-modal biometric technology to obtain the final authentication result. Thereby, highly accurate authentication can be realized while using an image of a single wavelength.

  Further, as another embodiment related to distance measurement of a subject, a method of measuring a three-dimensional structure of the subject while continuously shooting the subject may be implemented. First, at the start of shooting, the system gives the user guidance to move the finger toward the input device in various directions and continuously shoots the image. If detection of how the subject has moved in the photographed image is performed by template matching or feature point matching including SIFT, the three-dimensional shape of the subject can be measured. This is generally a technique called Structure from motion (SfM), which enables acquisition of the three-dimensional structure of the object surface. However, when the finger is bent and stretched, it is necessary to perform feature point matching corresponding to the non-rigid body.

  As described above, in the method of moving the living body, even if it is a back surface biological feature that can not be observed in a specific posture, when it can be photographed by the movement, the three-dimensional biological feature is more than the moving amount and the information of the three dimensional structure. Since it can be acquired, strong matching against movement of the living body can be realized.

  FIG. 18 shows an embodiment of a reflection non-contact biometric authentication device constituted by a light source emitting visible light and one color camera according to the third embodiment.

  A plurality of visible light sources 341 are arranged around the color camera 340, and a low pass filter 342 for blocking infrared light is mounted on the lens of the camera. The wavelength of the visible light source 341 has, for example, a peak of emission intensity at around 450 nm for blue, around 550 nm for green, and around 620 nm for red, and the emission intensity of each wavelength can be adjusted individually.

  The authentication processing procedure is the same as that of the second embodiment described above. The difference from the second embodiment is that a color camera is used instead of an infrared camera, and a photographed image is not a single wavelength infrared reflection image but a plurality of visible reflection images of a plurality of wavelengths. Here, processing specific to the present embodiment will be described in detail.

  In the present embodiment, unlike the above-described embodiments, a living body can be photographed by an RGB color camera. Unlike infrared light, visible light has the property that it is difficult to reach the deep part of the surface of a living body, but on the other hand it can handle a plurality of wavelengths, so many biological information can be obtained according to the optical properties of living tissue on the skin surface. Can be taken.

  Skin tissue includes various biological features such as fingerprints and wrinkles of the epidermis, wrinkles of the finger joints, melanin, blood in capillaries, arteries, veins and subcutaneous fats, and the light absorption rate and reflectance rate, respectively. Is different. Therefore, it is possible to extract information on each living tissue independently by examining the reflection characteristics of each living tissue in advance and irradiating a plurality of visible lights and observing them simultaneously. As the method, it is possible to use a method of separating melanin and blood by independent component analysis on three images of RGB, and multi-layered biometric technology. In addition, since veins are easy to observe in red images and finger joints in green or blue, a finger joint component is removed from a red image by a combination of dividing the difference between the two images or any ratio, etc. to obtain a more pure vein. Since only images can be acquired, biometric features can be measured more clearly.

  As described above, in reflection non-contact measurement using a light source of three wavelengths and a color camera, biological information that is difficult to observe with infrared light can be obtained, so that an effect of improving authentication accuracy can be obtained.

  Needless to say, in the case where a camera capable of capturing not only visible light but also infrared light can be used, it is possible to add an infrared light source in addition to the RGB light source for shooting. As a result, it becomes possible to capture information such as veins present deep in the surface of the living body, and the number of living body features that can be used for authentication can be increased to improve authentication accuracy.

  In addition, distance measurement of an object based on the inverse square law described in the second embodiment described above can also be applied in this embodiment. However, in the present embodiment, since there are three light sources, it is possible to calculate each distance independently. For example, for the light of the three wavelengths irradiated, the average reflection characteristic of the skin is determined in advance, and each reflected light is simultaneously photographed by a color camera. At this time, the relationship between the light intensity emitted at each wavelength and the observed light intensity is described by the relationship between the attenuation due to reflection and the attenuation due to the distance of the object as shown in Equation 4. Assuming that the average attenuation of light at the finger surface by reflection is known, only the distance of the object is unknown. Thereby, the distance of the subject can be obtained for three wavelengths. Although the result of distance measurement at three wavelengths is different due to the influence of external light, measurement error, etc., it is basically expected to be around the correct value. In order to level the error, the three obtained distance values are averaged to obtain a final result. As a result, the influence of the error is alleviated as compared with the distance value estimated using only the infrared light as described above, so that more accurate distance measurement can be realized.

  FIG. 19 shows an example of the configuration of a reflection non-contact biometric authentication device in a smartphone or a tablet according to a fourth embodiment. The smartphone 360 has an in-camera 362 on the color liquid crystal panel 361 and an out-camera 363 on the back, and a white light source 364 for flash projection used for flash photography is mounted near the out-camera 363 It is done. The in-camera 362 and the out-camera 363 are color cameras capable of capturing visible light, and an optical filter for blocking infrared light is mounted inside. It also has a focus adjustment function and an exposure adjustment function.

  The authentication processing procedure in this embodiment is basically the same as that of the second embodiment described above. The difference from the second embodiment is that the reflected light source may not be available, and the sensitivity characteristics of the camera and the wavelength spectrum of the light source may not be grasped in advance. Here, processing specific to the present embodiment will be described in detail.

  First, photographing processing of an object, distance measurement, and finger posture detection will be described. With regard to distance measurement of an object, posture detection, and extraction processing of biological features, a method of enhancing the accuracy of distance measurement using a color camera, as in the method described in the third embodiment described above, and a plurality of skin internals A method of extracting a biological feature can be used. In addition, the color of the finger on the palm side can be individually registered in advance, and the color can be used to detect the finger area.

  On the other hand, when photographing the living body with the out-camera 363 that can use a white light source for flash, if the emission spectrum of the white light source can be grasped in advance, calibration for measuring the feature amount of the living body or each wavelength component in distance measurement Since the light emission intensity can be grasped, it is possible to improve the accuracy of detection of a finger area, distance measurement, feature extraction of a living body, and the like.

  When the in-camera 362 is used, the liquid crystal panel 361 faces the living body to be held, so the liquid crystal panel may be used as a light source for living body imaging. Thereby, distance measurement and living body measurement of an object can be realized by the same method as the apparatus using the visible light source of RGB shown in the above-mentioned third embodiment. In addition, since the liquid crystal panel can control planar light emission, for example, the light source position is optional such that only the upper half of the panel is lit and photographed, and then only the lower half is lit and photographed. It can be set to Since the light emitting position is known, in the configuration of the present invention, it is also possible to acquire the three-dimensional shape of the subject by the technique of photometric stereo.

  However, the light intensity of the liquid crystal panel is generally weaker than that of a light emitting element such as an LED, and there is a possibility that the subject can not be irradiated with a sufficient intensity. Therefore, it is necessary to perform imaging using ambient light distributed around the authentication device. In this case, the ambient light can not specify the position of the light source, which makes it difficult to measure the distance based on the intensity of the image luminance value. In addition, since the spectral distribution of light also varies depending on the environment, it is difficult to obtain biological information such as veins, melanin, and fat print using color information. Therefore, the captured finger image is separated into images of RGB color planes, the average luminance is determined for each image, and when it is darker than a predetermined threshold, the image of the color plane is discarded without using, and sufficient As described above, it is possible to apply vein enhancement filters and finger joint enhancement filters to an image of a color plane that can be determined to be bright to obtain a plurality of biological features.

  Further, when the finger joint can be detected, the position of the finger is observed so that the color of hemoglobin is dark. Therefore, it is possible to extract the color of hemoglobin by taking the difference in color between the finger joint and its surroundings. At this time, the color of general hemoglobin is observed in advance with various ambient light, and is expressed as color space information such as RGB space or HSV space. Then, by comparing the color space information of hemoglobin that can be grasped from the current image with the color space information of various hemoglobins prepared in advance, the environment light associated with the most similar color space information is the current environment light It can be estimated to be the spectrum of Ambient light may be estimated by this method, and may be used for the above-mentioned finger area detection and living body measurement.

  In addition, with regard to the detection of the posture of the finger, the position of the finger can be estimated because it is assumed that the reflected light is irradiated from a position close to the camera in the second embodiment described above. In particular, when performing imaging using the in-camera 362, the premise can not be used because there is no light source. Therefore, as an example of finger posture detection, a method of detecting the area of a finger appearing in an image using color information or edge information of the image is used.

  First, a finger is photographed using the camera of the smartphone according to the embodiment together with a distance sensor or the like that can calculate distance information and finger posture information as learning data. Thereby, the correct value of distance information and posture information is labeled on the image captured by the camera of the smartphone. At this time, the number of types of fingers, the number of types of postures, and the number of images are acquired as much as possible. Next, general machine learning is performed, and parameter learning is performed to estimate distance information and posture information of the finger only by the image of the smartphone. For example, in order to obtain distance information and posture information of a pixel at a certain coordinate, learning is performed based on a random forest while using the strength of the skin color component of the pixel value and the image edge amount in the vicinity as an evaluation value. Then, the image of the smartphone is thrown into a random forest after learning and traversed, classified into either the background region or the finger inner region for each pixel, and if it is in the finger, it is the fingertip or at the root It is determined whether there is a joint or a joint. At the same time, the distance value is also estimated. Since the estimation result of the distance value varies from pixel to pixel due to the influence of noise or the like, processing may be performed so as to be spatially smooth.

  As described above, when the inner area of the hand and the background area can be separated, the background portion is blacked out and the finger based on the luminance value of the reflected image without distance measurement as shown in the second embodiment described above. The attitude detection method of can be applied. Therefore, the authentication process can be realized by combining the biometric feature extraction methods shown in the third embodiment described above.

  FIG. 20 shows an example of a guide method for photographing a finger with the out camera in the present embodiment. The user holds the smartphone with the hand 380 holding the device and captures the other finger 1. At this time, a finger contour guide 381 is drawn on the liquid crystal panel 361, and the user shoots a finger so as to fit the guide. The system calculates the posture information of the finger in real time, captures an image when it enters an allowable range, extracts the biometric feature based on this image, and performs authentication. At this time, if the position of the crotch of the finger is clearly indicated on the finger contour guide 381 as shown in the drawing, the rough distance between the camera and the finger can be easily adjusted. Such indicia may be applied to any of the embodiments of the invention.

  If misalignment has occurred, guide the misalignment direction with visual means such as arrows as shown in the above embodiment, or use a speaker or vibration element mounted on the smartphone , Or sound or vibration function may be used to notify of finger slippage. Especially with regard to the distance between the camera and the living body, guidance on the screen is difficult, and the larger the positional deviation, the stronger the vibration amount of the vibrator function, or the larger the volume of the guide sound emitted from the speaker. Can be implemented.

  FIG. 21 shows an example of a guide method for photographing a finger with an out-camera that can be performed by one-hand operation. The user holds the smartphone with one hand as shown in the figure, and first, the system instructs the camera 363 to be closed with a fingertip. The user blocks the out camera 363 with the finger 1. When the authentication system detects this, it subsequently displays a finger contour guide 381 on the screen, and instructs the user to turn the finger so that the finger appears in the distance. The user bends the finger away from the out camera 363 according to the guidance while holding the smartphone 360 as shown in FIG.

  At this time, as shown in the liquid crystal panel 361, the finger 1 appears in an inclined state. When this state is detected, the system carries out an imaging process and an authentication process. By applying this guiding method, the finger alignment can be easily performed even with one hand, and the biometric authentication described in the present invention with a smartphone can be realized.

  Alternatively, an operation may be induced to lightly shake the smartphone 360 while the finger 1 is in a bent state. At this time, when a continuous image is taken, the finger 1 held up is observed at the same position as it is, but the unnecessary background 400 included in the image moves in accordance with the falling operation. Therefore, the presence or absence of movement is determined for each pixel, and the area that has not moved is regarded as a finger area, and the other area is regarded as a background area, whereby area detection of the finger can be performed. According to this method, the finger area can be accurately detected by a simple image processing, and the authentication accuracy can be improved.

  FIG. 22 shows an example of a guide method in the case of using an in-camera. The in-camera is often set so as to easily capture a face, and it is difficult to capture a finger near the liquid crystal panel. In this case, the finger is held up in the air, but since the liquid crystal screen is hidden by the finger, it is difficult to check. Therefore, the finger is placed at a predetermined position on the liquid crystal panel, and guidance is then given to move the finger in the air.

  First, when the authentication is started, as shown in FIG. 22A, a guidance 420 for placing a finger on the screen and a finger position guide 421 for placing the finger are displayed. In this embodiment, the direction in which the finger is placed is in the upper left direction, but the direction may be changed according to the handed hand, or may be held in the direction just beside. The user then puts his finger here. Since the liquid crystal panel 361 is a touch panel, when it is detected that the finger is placed at the designated position, next, as shown in FIG. 22B, guidance 422 indicating that the finger is floated in the hollow is displayed. At this time, since the finger is placed obliquely upward to the left, the guidance 422 indicating that the finger floats in the air is displayed at the lower left of the screen so that the guidance can be easily seen. However, since there is a possibility that the screen is covered with fingers, audio guidance may be used together. The user moves the fingers hollow according to this. Then, as shown in FIG. 22C, the finger enters the angle of view of the camera. When the system detects this, imaging is performed.

  The position of the guide 421 at which the finger is first placed has an effect of guiding the finger to a position where it can easily enter the angle of view. If it is desired to shoot a plurality of fingers, the guide 421 may be a finger contour of the plurality of fingers. At this time, when the finger is guided to open at an appropriate angle, an effect of making it easier to shoot a plurality of fingers simultaneously can be obtained. By such a guide method, the authentication process of the above embodiment can be realized even when an in-camera is used.

  As another guidance method, the in-camera or the liquid crystal panel in the vicinity of the in-camera is guided to be lightly repeated with a fingertip, and tapping is detected by the change in the brightness of the touch panel or in-camera. At this time, shooting is performed in a continuous frame, and it is detected that the distance between the fingers is appropriate by distance measuring means based on measurement of the width of the finger contour, etc. Authentication can also be performed.

  Such a tapping-based guiding method is a relatively simple operation and can easily guide the position of the finger, and it is easy to align because it is hollow and there is no need to hold the finger, and Since the distance to the camera constantly changes, there is a timing when the living body reaches a distance suitable for imaging, and there is an effect such that the living body can be imaged with an appropriate focus.

  The smartphone shown in the present embodiment refers to the case where the in-camera and the out-camera, which are generally popular at present, are mounted. If a camera capable of projecting a finger in the vicinity of the liquid crystal panel, an infrared camera, a multispectral camera, a distance camera, etc. is mounted on a smartphone or a tablet, those cameras are used to utilize the above-described embodiment. It goes without saying that distance measurement, finger posture detection processing, and authentication processing may be performed as described above. In addition, the present embodiment is not limited to a smartphone, and can be implemented similarly even in the case of a tablet or a portable game machine.

  In addition, for users who can not easily align in the air, it has a finger rest on which a finger can be placed keeping a certain distance from the camera, and authentication is performed using a pedestal that can be fitted into a smartphone or tablet. It is good. For example, in an administrative service window or a bank window, a stationary tablet may be installed as a terminal that general customers can easily operate. In this case, portability is not a problem in particular, and by installing the above-mentioned pedestal and tablet in combination, it becomes possible to carry out authentication with a simpler operation while eliminating the need for a dedicated authentication device. .

  FIG. 23 shows an embodiment in which the authentication device shown in the first to third embodiments described above is applied to a video game.

  In FIG. 23A, a motion capture device 441 for detecting the posture of the player's body and operating the game is installed at the top of the television 440, and the input device 2 for authentication is provided therein. In this way, the user's finger 1 is projected and authentication is performed. The user operates the game using the motion capture device 441 or a separately prepared game controller or the like, in which the input device 2 detects the user's finger and carries out authentication. At the time of authentication, guidance may be displayed on the television 440, or voice guidance may be performed using a speaker or the like.

  FIG. 23B shows an embodiment in which the input device 2 is mounted on the controller 442 of the video game. The user holds the controller 442 to perform a game operation, and the authentication input device 2 is incorporated therein, and can be used for the above-described game control and settlement of charges for items.

  FIG. 23C shows an embodiment mounted on a portable game machine. The central portion of the portable game machine 443 is provided with an input device 2 for authentication. The authentication method is the same as the example of mounting on the smartphone described in the fourth embodiment described above.

  In each game machine described above, the following authentication application can be implemented. First, it is easy to identify the user when playing a game and automatically read saved data, to control access to content on a game console or network, and to manage and limit the play time of the game. Will be able to Further, various parameters of the game character can be changed according to the result of the authentication, and the charge of the pay item can be automatically settled. In particular, it is possible to control the characteristics of the game character in accordance with the degree of similarity between the registered data and the input data, for example, to increase the moving speed of the character or to increase the attack power. It can also be used as

  Further, if a plurality of fingers are registered and only one finger is presented at the time of authentication, it is possible to divide the function according to the type of the presented finger. For example, in the case of an action RPG game, if the index finger is presented, it is a striking attack, if it is presented the middle finger, it is a magic attack, and if it is a ring finger, the recovery of strength by magic etc. At present, such operation switching is made to be selected on the menu screen or complex commands are input using a plurality of input keys. However, these operations can be simplified by this embodiment. Also, if the biometric information of the outer periphery of the finger along the central axis of the finger to be presented is registered individually, the finger is presented to the input device and the finger is rolled about the axis to cope with the presented rolling angles. It can be matched with registered data. According to this, since the rolling angle of the finger can be detected according to the matching registered data, this operation can be utilized as an interface to the game. For example, the number of rotations of the engine in the racing game can be changed in an analog manner according to the rolling angle of the finger.

  FIG. 24 shows an embodiment in which the authentication device of the present invention is applied to a walk-through gate that can be authenticated simply by holding a finger while walking.

  When approaching the authentication gate 460, the user brings the finger 1 close to the input device 2 as shown. The input device 2 continuously captures an image and performs authentication at high speed while detecting the posture of the finger. If the authentication is successful, the flapper 461 opens and the user can pass. The finger can be guided by the liquid crystal screen 462 or the speaker 463. The liquid crystal screen 462 can display not only the guide of the finger but also the authentication result and the result of charging performed at the gate.

  The registration may be performed by the authentication gate 460, and the CPU and the memory may be mounted inside the gate to store registration data, and the registration terminal on which the input device 2 is mounted may be installed in another place. The registration data may be transferred to the authentication gate 460 via the network. The registration data may be stored in a separately installed server or the like. The authentication may be performed inside the authentication gate, or the image may be transferred to the server and performed on the server. In addition, a device for wireless communication is mounted on the authentication gate 460, and the user carries a wireless IC tag or the like when passing through the gate, and transfers registration data stored in the wireless IC tag to the authentication gate 460 for authentication. May be implemented.

  According to the configuration of the present embodiment, since it is a reflection non-contact authentication device, authentication can be performed even when held rough while walking, and gate throughput can be improved. In addition, by applying the method based on the difference image of the blinking of the light source and the exposure adjustment function in the above-described embodiment, photographing can be performed outdoors, and the application range of the apparatus can be expanded.

  In addition to the reflected light authentication device of the present invention, when relatively large devices such as entrance gates for large-scale facilities are permitted or where security is strictly controlled in important management facilities, etc. It may be a hybrid authentication device combining general transmission light devices.

  FIG. 25 shows an embodiment of a transmission / reflection type device configuration. FIG. 25A shows a configuration in which the transmission light source 481 is vertically disposed in the gate authentication device 480. When the user holds the finger 1 from the right side of the drawing, the transmission light source 481 and the reflection light source 482 simultaneously emit light. The transmission light source 481 emits infrared light, and the reflection light source 482 emits visible light of green and red. Here, the wavelengths of green, red and infrared are respectively 550 nm, 620 nm and 850 nm. The imaging device 9 is an RGB color camera with infrared sensitivity, and simultaneously receives transmitted and reflected light. At this time, according to the spectral sensitivity characteristic of the camera, the luminance components at the above three wavelengths can be described as follows.

  In addition, the definition of a symbol is the same as what was shown to Formula 1-Formula 3 of Example 1. FIG. By solving the simultaneous equations, I 550 and I 620 of the reflected light component and I 850 of the transmitted light component can be obtained, and authentication can be performed on each image. The camera may be mounted one by one so that the transmitted light and the reflected light can be photographed separately, and the light source to be irradiated may be switched in time division, but the cost increases in the former and the latter is a frame There is a problem that the rate decreases. Therefore, as shown in this embodiment, a method of simultaneously photographing light of different wavelengths by one camera and performing separation processing can be photographed at low cost and at high speed. Further, in this embodiment, since the place where the finger is held is open, even a first-time user can easily operate.

  Further, FIG. 25 (b) shows a method of presenting fingers in a space in the device. A transmitted image and a reflected image can be photographed by putting the finger 1 between the infrared light source 481 and the imaging device 9 and irradiating the transmitted light from above and the reflected light from below. The fingers may be inserted from the right to the left in the drawing, but as shown in the lower part of the drawing, the fingers may be moved from the side of the apparatus. At this time, the side of the device is open so as not to impede the movement of fingers. In addition, a plurality of photosensors 483 are installed at positions slightly offset from each other on the side of the input side. This photo sensor responds when the finger passes directly below, and senses the movement speed of the finger, the finger angle, etc. from the shift in the reaction time of the four sensors shown and the distance between the sensors. it can. In response to this, the time when the finger passes the center of the camera is instantaneously predicted, and the imaging device 9 shoots the finger at a timing suitable for shooting. At that time, the exposure time is made extremely short so that blurring of the image due to the movement of the finger does not occur, and the light amount is also extremely intensely irradiated in a short time. According to this method, it is possible to make the authentication instantaneously only by passing the finger at high speed.

  In any of the methods of this embodiment, biological characteristics of both transmitted light and reflected light can be obtained, so it can be expected to improve the accuracy and the forgery resistance. Further, authentication can be stably realized even when the living body is held at a position where transmitted light can not be irradiated with reflected light, and the user can use the authentication device more easily. You can also get the effect that

  FIG. 26 shows an embodiment in which the authentication device of the present invention is applied to a mouse. A left button 501 and a right button 502 are mounted on the mouse 500, and a wheel 503 is mounted at the center. The above-described input device 2 for authentication is installed near the wheel 503 at the center of the mouse 500. The shape of the mouse 500 is a curve according to the shape of the finger, and the inclination from the fingertip side to the central part of the mouse and the inclination from the central part of the mouse to the palm side are opposite. When holding the mouse 500, the distance between the finger 1 and the input device 2 is short, but if the finger 1 is stretched straight as shown by the relationship of the tilt of the mouse, the shooting distance with the camera necessary for shooting Can be easily obtained, and a sufficient range can be photographed. Although the forefinger is extended in the figure, it goes without saying that those fingers can be photographed by extending the middle finger and the other fingers. The input device 2 may be mounted on the left button 501 or the right button 502 of the mouse.

  This finger extension operation can be easily performed with the mouse held, so various situations when operating the PC, such as identity verification at PC login, identity verification at net payment, confirmation of user access right for each application software Etc. can be realized in natural PC operation without impairing the convenience of the user.

  Although all the authentication devices described above are described with fingers as an example, they may be anywhere on the skin of a living body, for example, in any place such as face, ears, palms, back of hands, wrists, arms, necks, feet, etc. It can apply. In this case, although an optimum posture detection process is required according to each part, it is needless to say that it can be realized by a method of machine learning after collecting a large amount of teacher data.

DESCRIPTION OF SYMBOLS 1 ... finger, 2 ... input device, 3 ... light source, 9 ... camera, 10 ... authentication processing unit, 11 ... central processing unit, 12 ... memory, 13 ...・ Interface, 14 ・ ・ ・ Storage device, 15 ・ ・ ・ Display part, 16 ・ ・ ・ Input part, 17 ・ ・ ・ Speaker, 18 ・ ・ ・ Image input part, 21 ... Device housing, 31 ... Red External light source 32 Visual light source 33 Band pass filter 61 Reflected image of infrared light 62 Finger vein 63 Reflected image of green light 64 Fingerprint , 65: joint crease, 66: fat print, 100: distance image, 101: projected luminance distribution, 102: area of interest of finger, 103: weighting factor, 104: 104 Maximum value of projection distribution, 120: finger contour, 121: center line of finger, 122: fingertip, 140: method Vector, 150: joint, 151: estimated position of joint, 152: central axis of finger node, 160: cross section of finger, 161: oval, 162: major axis of oval, 163: attention pixel, 180: finger contour guide, 181: guide message, 182: finger movement instruction arrow, 183: camera distance instruction icon, 184: rotation instruction icon, 200 · · · Reflected image, 201 · · · cutout image, 220 · · · fingertip guide mark, 221 · · · finger root guide mark, 240 · · · index finger, 300 · · · · finger cut out image, 301 · · · average brightness Projection distribution, 302: Average luminance projection distribution of flat plate, 303: Estimated pitch straight line of finger, 320: Light shielding plate, 321: Area where light does not reach, 340: Color camera, 341 · · · Visible light source, 342 · · · · · · · · · · · · · · · · · · · · · · · visible light source, 342 · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · , 380: hand holding the device, 381: finger contour guide, 400: unnecessary background, 420: finger placement guidance, 421: finger position guide, 422: finger Floating guidance, 440: television, 441: motion capture device, 442: controller, 443: portable game machine, 460: authentication gate, 461: flapper, 462: liquid crystal Screen, 463: Speaker, 480: Authentication device for gate, 481: Transmission light source, 482: Reflection light source, 483: Photo sensor, 500 ... mouse, 501 ... left button, 502 ... right button, 503 ... wheel

Claims (10)

A storage unit that holds feature information on a finger;
An instruction unit that instructs correction of the distance from the imaging unit to the finger and the posture of the finger;
A light source for irradiating the finger with light;
An imaging unit configured to capture light reflected by the finger;
Based on the luminance value of the image taken by the imaging unit, distance information from the imaging unit to the finger is calculated, and an image centered on the finger is generated, and the pixel value in the generated image is generated. A determination unit that detects the contour of the finger, detects a plurality of clauses of the finger from the detected contour, and analyzes the skeletal structure of each clause to determine the posture of the finger;
A state control unit that instructs the instruction unit to correct the distance between the imaging unit and the finger and the posture of the finger based on the determination;
A feature extraction unit that extracts the feature information from the image;
A matching unit that compares the similarity between the extracted feature information and the feature information stored in the storage unit;
An authentication apparatus comprising: In the authentication device according to claim 1,
The state control unit instructs the instruction unit to correct the distance to the finger and the posture of the finger from the light amount emitted by the light source and the light amount received by the imaging unit. apparatus. In the authentication device according to claim 2,
The authentication device, wherein the state control unit instructs the instruction unit to correct the posture of the finger by obtaining a relative distance distribution with the distance 1 at an arbitrary position of the image as 1. The authentication apparatus according to any one of claims 1 to 3.
An authentication apparatus comprising: an unnecessary information removal unit that removes unnecessary information from the image. In the authentication device according to claim 4,
The authentication apparatus, wherein the unnecessary information removal unit removes the unnecessary information by comparing an image captured by turning on the light source with an image captured after being turned off. In the authentication device according to claim 4,
There are a plurality of light sources, and the plurality of light sources respectively emit light of different frequencies,
The authentication apparatus, wherein the unnecessary information removal unit removes the unnecessary information by comparing the images captured with different frequencies. In the authentication device according to claim 5 or 6,
An authentication apparatus, wherein the determination unit, the state control unit, the feature extraction unit, the check unit, and the unnecessary information removal unit are executed by one processing device. In the authentication device according to claim 7,
The instruction unit is a display unit that instructs correction of the distance of the finger and the posture of the finger by an image, or a sound generation unit that instructs correction of a distance between the imaging unit and the finger by a sound and the posture of the finger An authentication device characterized by In the authentication device according to claim 8,
The feature information is information on a vein. Irradiating the finger with light;
An imaging step of photographing light reflected by the finger;
Based on the luminance value of the image captured in the imaging step, distance information between the imaging unit and the finger is calculated, an image centered on the finger is generated, and the finger is calculated based on the pixel value in the generated image. A step of detecting the contour of the finger, detecting a plurality of clauses of the finger from the detected contour, and analyzing the skeletal structure of each clause to determine the posture of the finger;
A step of instructing a correction instruction of the distance and orientation of the finger of the finger and the imaging unit to the user of the authentication device based on the determination,
A feature extraction step of extracting the image or we feature information,
A matching step of comparing the similarity between the extracted feature information and the feature information held in advance;
An authentication method comprising:

Images