CN111563454B

CN111563454B - Dual living body verification hand vein recognition method and device

Info

Publication number: CN111563454B
Application number: CN202010380694.1A
Authority: CN
Inventors: 郑音飞
Original assignee: Zhejiang University ZJU
Current assignee: Zhejiang University ZJU
Priority date: 2020-05-08
Filing date: 2020-05-08
Publication date: 2023-08-08
Anticipated expiration: 2040-05-08
Also published as: CN111563454A

Abstract

The invention discloses a hand vein recognition method and device for double living body verification, and relates to the field of identity verification. The method comprises the following steps: acquiring physiological parameters of a hand to be verified; performing action characteristic living experience evidence when the physiological parameter is larger than a first preset threshold value; the living experience is verified to successfully calculate the palm similarity and the back of hand similarity; and if the palm similarity and the hand back similarity are both larger than the second preset threshold, the hand vein recognition is successful. The invention adopts a mode of combining action characteristics with physiological parameter detection to perform double living verification, respectively extracts palm vein images and back hand vein images to perform vein recognition, improves reliability through double living verification and double vein recognition of the palm vein images and the back hand vein images, and is particularly suitable for occasions with particularly high reliability requirements.

Description

Dual living body verification hand vein recognition method and device

Technical Field

The invention relates to the field of identity verification, in particular to a hand vein recognition method and device for double living body verification.

Background

Vein recognition is to acquire images of hand veins through a near infrared camera, and store digital images of the veins in a computer system to realize characteristic value storage. And during vein comparison, vein images are adopted in real time, advanced filtering, image binarization and refinement means are used for extracting characteristics of the digital images, and complex matching algorithms are adopted to compare and match the digital images with vein characteristic values stored in a host computer of a computer system, so that identity authentication is carried out on individuals, and identity is confirmed. The vein is positioned in the human body, is not influenced by rough epidermis and external environment, has the advantages of high accuracy, difficult replication, safety, convenience and the like when used for vein recognition, and has been used in the fields of entrance guard, social security and the like.

Most of devices related to vein recognition in the market adopt a vein recognition mode of one of back of hand, palm or fingers alone, and have relatively few characteristic points and relatively low recognition reliability and accuracy. The vein is hidden in the skin of a human body, trace is not easy to leave, the forging difficulty is higher than that of other biological recognition modes, but the vein can still be forged and cracked, the most common cracking mode is to draw lines on paper, wear rubber gloves and draw lines on the rubber gloves, for example, on Chaos Communication Congress hacker meeting held by Laiytin in Germany in 2018, researchers Jan Krissler and Julianallbrecht successfully deceive a vein authentication system through a waxing hand model; the simplest cracking mode is to obtain a vein pattern by using vein acquisition equipment, copy the vein pattern of the finger, and crack by using the pattern. Therefore, the existing vein recognition device has the problems of low recognition reliability and easiness in cracking.

Disclosure of Invention

The invention aims to provide a hand vein recognition method and device for double living body verification, which solve the problems of low recognition reliability and easiness in cracking of the existing vein recognition equipment.

In order to achieve the above object, the present invention provides the following solutions:

A dual in vivo verification hand vein recognition method comprising:

acquiring physiological parameters of the hand to be verified, which are detected by a physiological parameter detection sensor;

if the physiological parameter is larger than a first preset threshold, performing action characteristic living experience verification on the hand to be verified to obtain a verification result;

if the verification result shows that the living experience is successful, acquiring a palm vein image and a back vein image of the hand to be verified;

acquiring a pre-stored palm vein storage image and a pre-stored back hand vein storage image;

respectively calculating the palm similarity of the palm vein image and the palm vein storage image and the back similarity of the back hand vein image and the back hand vein storage image;

if the palm similarity and the back similarity are both larger than a second preset threshold, the hand vein of the hand to be verified is successfully identified.

Optionally, the acquiring the physiological parameter of the hand to be verified detected by the physiological parameter detection sensor specifically includes:

and acquiring the pulse blood oxygen value of the hand to be verified, which is detected by the blood oxygen heart rate sensor.

Optionally, performing the motion feature living body verification on the hand to be verified to obtain a verification result, which specifically includes:

Acquiring a fist-making image of the hand to be verified;

if the hand to be verified in the fist-making image is in a fist-making state, acquiring a semi-fist-making image of the hand to be verified;

if the hand to be verified in the semi-fist-making image is in a semi-fist-making state, acquiring hand images of the hand to be verified in different fist-making states from the fist-making state to the semi-fist-making state;

detecting the optical flow characteristics of the hand image by using an optical flow method, and comparing the similarity of the optical flow characteristics of the hand image with prestored optical flow characteristics;

if the similarity is greater than a third preset threshold, the live experience is successful;

if the similarity is smaller than or equal to the third preset threshold, the liveness experience fails.

Optionally, the detecting the optical flow characteristic of the hand image by using an optical flow method, and comparing the similarity between the optical flow characteristic of the hand image and a pre-stored optical flow characteristic specifically includes:

taking the hand image of the ith frame as a first comparison image, and extracting an interested region of the first comparison image to obtain a first interested region;

taking the hand image of the (i+1) th frame as a second comparison image, and extracting an interested region of the second comparison image to obtain a second interested region;

Detecting the positions of the first region of interest and the second region of interest, the fluctuation range of which is larger than a fourth preset threshold value, by using an optical flow method to obtain optical flow characteristics;

let i=i+1, return to step "take the hand image of the i frame as the first comparison image, and extract the region of interest of the first comparison image, get the first region of interest", get the optical flow characteristic of all said hand images;

acquiring prestored optical flow characteristics;

and comparing all the optical flow characteristics with corresponding pre-stored optical flow characteristics respectively to obtain similarity.

A dual in-vivo verified hand vein recognition device, comprising: a top acquisition zone, a hand placement panel zone, and a bottom acquisition zone; the top collecting area and the hand placing panel area are connected with the bottom collecting area;

the top acquisition area is positioned at the top of the hand vein recognition device and is used for acquiring the back hand vein image of the hand to be verified;

the hand placing panel area is positioned below the top collecting area and is used for collecting physiological parameters of the hand to be verified;

the bottom acquisition area is positioned at the bottom of the hand vein recognition device and below the hand placement panel area, and is used for acquiring living experience information and processing the living experience information to obtain a hand vein recognition result; the liveness experience information comprises the back hand vein image, the physiological parameter and the palm vein image of the hand to be verified.

Optionally, the top acquisition area specifically includes: a top camera, a top light source, and a display screen;

the top camera is used for collecting a back hand vein image of the hand to be verified; the output end of the top camera is connected with the bottom acquisition area;

the top light source is arranged corresponding to the top camera and used for irradiating the back of the hand to be verified;

the display screen is connected with the output end of the bottom collection area, and the display screen is used for displaying the hand vein recognition result of the bottom collection area.

Optionally, the hand placement panel area specifically includes: the hand placing area, the hand texture map, the bulge and the physiological parameter detection sensor;

the hand placing area is positioned right below the top camera and is used for placing the hand to be verified;

the hand texture patterns are engraved in the center of the hand placement area, the protrusions are located on the first joints of the middle fingers of the hand texture patterns, and the hand texture patterns and the protrusions are used for displaying the placement positions of the hands to be verified;

the physiological parameter detecting sensor is positioned below the bulge and is used for detecting the physiological parameter.

Optionally, the bottom acquisition area specifically includes: the hand vein recognition system comprises a bottom camera, a bottom light source, a hand vein recognition system and a power supply;

the bottom camera is used for collecting palm vein images of the hand to be verified; the output end of the bottom camera is connected with the input end of the hand vein recognition system;

the bottom light source is arranged corresponding to the bottom camera and used for irradiating the palm of the hand to be verified;

the hand vein recognition system is respectively connected with the top camera and the physiological parameter detection sensor; the hand vein recognition system is used for acquiring the living experience information and processing the living experience information to obtain a hand vein recognition result;

the power supply is respectively connected with the top light source, the bottom light source, the physiological parameter detection sensor and the hand vein recognition system, and is used for supplying power to the top light source, the bottom light source, the physiological parameter detection sensor and the hand vein recognition system.

Optionally, the hand vein recognition system specifically includes:

the physiological parameter acquisition module is used for acquiring physiological parameters of the hand to be verified, which are detected by the physiological parameter detection sensor;

The action characteristic living experience verification module is used for carrying out action characteristic living experience verification on the hand to be verified when the physiological parameter is larger than a first preset threshold value to obtain a verification result;

the vein image acquisition module is used for acquiring a palm vein image and a back hand vein image of the hand to be verified when the verification result indicates that the living experience is successful;

the vein storage image acquisition module is used for acquiring a pre-stored palm vein storage image and a hand back vein storage image;

the similarity calculation module is used for calculating the palm similarity of the palm vein image and the palm vein storage image and the back hand similarity of the back hand vein image and the back hand vein storage image respectively;

and the hand vein recognition module is used for successfully recognizing the hand veins of the hand to be verified when the palm similarity and the back similarity are both larger than a second preset threshold value.

Optionally, the action feature liveness experience verification module specifically includes:

a fist-making image acquisition unit for acquiring a fist-making image of the hand to be verified;

the semi-fist-making image acquisition unit is used for acquiring a semi-fist-making image of the hand to be verified when the hand to be verified in the fist-making image is in a fist-making state;

A hand image acquisition unit, configured to acquire, when the hand to be verified in the semi-fist-making image is in a semi-fist-making state, hand images of the hand to be verified in different fist-making states from the fist-making state to the semi-fist-making state;

the similarity comparison unit is used for detecting the optical flow characteristics of the hand image by using an optical flow method and comparing the similarity between the optical flow characteristics of the hand image and prestored optical flow characteristics;

the liveness experience verification success unit is used for verifying the liveness experience success when the similarity is larger than a third preset threshold value;

and the liveness experience failure unit is used for failing the liveness experience when the similarity is smaller than or equal to the third preset threshold value.

According to the specific embodiment provided by the invention, the invention discloses the following technical effects:

the invention provides a hand vein recognition method and device for double living body verification. The method comprises the following steps: acquiring physiological parameters of the hand to be verified, which are detected by a physiological parameter detection sensor; if the physiological parameter is larger than a first preset threshold, performing action characteristic liveness experience verification on the hand to be verified to obtain a verification result; if the verification result shows that the living experience is successful, acquiring a palm vein image and a back vein image of the hand to be verified; acquiring a pre-stored palm vein storage image and a pre-stored back hand vein storage image; respectively calculating the palm similarity of the palm vein image and the palm vein storage image and the back similarity of the back hand vein image and the back hand vein storage image; if the palm similarity and the back similarity are both larger than the second preset threshold, the hand vein recognition of the hand to be verified is successful. The invention adopts a mode of combining action characteristics with physiological parameter detection to perform double living verification, respectively extracts palm vein images and back hand vein images to perform vein recognition, improves reliability through double living verification and double vein recognition of the palm vein images and the back hand vein images, and is particularly suitable for occasions with particularly high reliability requirements.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions of the prior art, the drawings that are needed in the embodiments will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

FIG. 1 is a flow chart of a hand vein recognition method according to an embodiment of the present invention;

FIG. 2 is a block diagram of a hand vein recognition device according to an embodiment of the present invention;

FIG. 3 is a circuit diagram of a hand vein recognition device according to an embodiment of the present invention;

FIG. 4 is a bottom view of the top acquisition zone of an embodiment of the present invention;

FIG. 5 is a block diagram of a hand placement panel area according to an embodiment of the present invention;

FIG. 6 is a block diagram of a protrusion according to an embodiment of the present invention;

FIG. 7 is a flow chart of data processing of a blood oxygen heart rate sensor according to an embodiment of the present invention;

FIG. 8 is a top view of a bottom acquisition zone according to an embodiment of the present invention;

FIG. 9 is a flow chart of a hand vein recognition device according to an embodiment of the present invention;

FIG. 10 is a flow chart of an identity registration process according to an embodiment of the present invention;

FIG. 11 is a flow chart of the motion feature in vivo registration in accordance with an embodiment of the present invention;

FIG. 12 is a flowchart of a process for storing pictures according to an embodiment of the present invention;

FIG. 13 is a flow chart of an authentication process according to an embodiment of the present invention;

FIG. 14 is a flow chart of the motion feature active experience verification according to an embodiment of the present invention;

fig. 15 is a flowchart of a hand image processing according to an embodiment of the present invention.

Symbol description: 1. a top acquisition zone; 2. a hand placement panel area; 3. a bottom acquisition zone; 4. a top camera; 5. a top light source; 6. a display screen; 7. a hand placement area; 8. a hand texture map; 9. a protrusion; 10. a physiological parameter detection sensor; 11. a label; 12. a bottom camera; 13. a bottom light source; 14. a main controller; 15. and a power supply.

Detailed Description

The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.

In order that the above-recited objects, features and advantages of the present invention will become more readily apparent, a more particular description of the invention will be rendered by reference to the appended drawings and appended detailed description.

The present embodiment provides a dual in-vivo hand vein recognition method, and fig. 1 is a flowchart of a hand vein recognition method according to an embodiment of the present invention. Referring to fig. 1, the hand vein recognition method includes:

step 101, acquiring physiological parameters of the hand to be verified, which are detected by a physiological parameter detection sensor.

The step 101 specifically includes:

and acquiring a pulse blood oxygen value of the hand to be verified, which is detected by the blood oxygen heart rate sensor.

Step 102, if the physiological parameter is greater than the first preset threshold, performing motion feature living verification on the hand to be verified to obtain a verification result. The first preset threshold is 90.

Performing motion characteristic living verification on the hand to be verified to obtain a verification result, and specifically comprising the following steps:

and acquiring a fist-making image of the hand to be verified.

If the hand to be verified in the fist-making image is in a fist-making state, a semi-fist-making image of the hand to be verified is obtained.

If the hand to be verified in the semi-fist-making image is in the semi-fist-making state, acquiring hand images of the hand to be verified in different fist-making states from the fist-making state to the semi-fist-making state.

Detecting the optical flow characteristics of the hand image by using an optical flow method, and comparing the similarity of the optical flow characteristics of the hand image with prestored optical flow characteristics, wherein the method specifically comprises the following steps:

and taking the ith frame of hand image as a first comparison image, and extracting the region of interest of the first comparison image to obtain a first region of interest.

And taking the (i+1) th frame of hand image as a second comparison image, and extracting an interested region of the second comparison image to obtain a second interested region.

And detecting the positions of the first region of interest and the second region of interest, the fluctuation range of which is larger than a fourth preset threshold value, by using an optical flow method, so as to obtain optical flow characteristics. The principle of the step is that the optical flow method can detect the position of a moving object.

According to the embodiment, through collecting images of the process from the fist making process to the semi-fist making process of each person, the change of the back joints in the process can be detected by using an optical flow method, and the joint changes of each person are different, so that the living body recognition effect is achieved.

Let i=i+1, return to step "take the ith frame of hand image as the first comparison image, and extract the region of interest of the first comparison image, get the first region of interest", get the optical flow characteristic of all hand images.

And obtaining pre-stored optical flow characteristics.

And comparing all the optical flow characteristics with the corresponding pre-stored optical flow characteristics respectively to obtain the similarity.

If the similarity is greater than a third preset threshold, the live experience is successfully verified.

If the similarity is smaller than or equal to a third preset threshold, the live experience evidence fails.

And step 103, if the verification result shows that the living experience is successful, acquiring a palm vein image and a back vein image of the hand to be verified.

Step 104, acquiring pre-stored palm vein storage images and back hand vein storage images.

Step 105, calculating the palm similarity of the palm vein image and the palm vein storage image, and the back similarity of the back vein image and the back vein storage image, respectively.

And 106, if the palm similarity and the back similarity are both larger than the second preset threshold, the hand vein recognition of the hand to be verified is successful.

The optical flow method can detect the instantaneous speed of the pixel motion of a spatially moving object on an observation imaging plane. When an object moves, the brightness mode of the corresponding point on the image also moves correspondingly, and the apparent movement of the brightness mode of the image is optical flow. Optical flow is a study that uses temporal variations and correlations of intensity data of pixels in an image sequence to determine the "motion" of the respective pixel locations. Optical flow expresses changes in the image and can therefore be used by the observer to determine the motion of the object. Typically, optical flow is generated by camera motion, object motion in the scene, or a combination of both.

The optical flow field is derived from the optical flow, and refers to a two-dimensional instantaneous velocity field formed by projecting a three-dimensional velocity vector of a visible pixel point in a scene on an imaging surface. The transition of the motion field in space to the image is denoted as an optical flow field reflecting the gray scale trend of each point on the image. The optical flow field contains motion information of the observed object and information about the rich three-dimensional structure of the scene.

The optical flow method detects moving objects, the basic idea of which is to assign a velocity vector to each pixel in an image, thus forming a motion field for the image. The points on the image and the points on the three-dimensional object are in one-to-one correspondence at a certain specific motion moment, and the image is dynamically analyzed according to the speed vector characteristics of each pixel point. If there is no moving object in the image, the optical flow vector is continuously changed in the whole image area, and when there is relative motion in the object and the image background, the velocity vector formed by the moving object is necessarily different from the velocity vector of the neighborhood background, so that the position of the moving object can be detected.

The embodiment also provides a hand vein recognition device for double living body verification, and fig. 2 is a structural diagram of the hand vein recognition device according to the embodiment of the invention; fig. 3 is a circuit diagram of a hand vein recognition device according to an embodiment of the present invention. Referring to fig. 2 and 3, the hand vein recognition apparatus includes: a top acquisition zone 1, a hand placement panel zone 2 and a bottom acquisition zone 3; the top collecting area 1 and the hand placing panel area 2 are both connected with the bottom collecting area 3.

The top collection area 1 is located at the top of the hand vein recognition device, and the top collection area 1 is used for collecting hand back vein images and hand movements of the hand to be verified, and displaying hand vein recognition results of the bottom collection area 3.

The hand placing panel area 2 is located below the top collecting area 1, and the hand placing panel area 2 is used for placing a hand to be verified and collecting physiological parameters of the hand to be verified.

The bottom acquisition area 3 is positioned at the bottom of the hand vein recognition device and below the hand placement panel area 2, and the bottom acquisition area 3 is used for acquiring living experience information and processing the living experience information to obtain a hand vein recognition result; the liveness experience information comprises a back hand vein image, physiological parameters and palm vein images of the hand to be verified. The bottom acquisition area 3 processes the living experience information through a hand vein recognition system, which can be realized by adopting the main controller 14.

Fig. 4 is a bottom view of the top acquisition region of an embodiment of the present invention. Referring to fig. 4, the top acquisition zone 1 specifically includes: a top camera 4, a top light source 5 and a display screen 6.

The top camera 4 is used for collecting a hand back vein image of the hand to be verified; the output end of the top camera 4 is connected with the bottom acquisition area 3. The top camera 4 is also used for collecting hand movements of the hand to be verified and transmitting the collected image data to the hand vein recognition system of the bottom collecting area.

The top light source 5 is arranged corresponding to the top camera 4, and the top light source 5 is used for illuminating the back of the hand to be verified. The top light source 5 is connected with a power supply of a bottom acquisition area, and the power supply of the bottom acquisition area supplies power for the top light source. The top light source 5 is a near infrared lamp set comprising 6 near infrared lamps with a diameter of 850 nanometers (nm).

The display screen 6 is connected with the output end of the bottom collection area, and the display screen 6 is used for displaying the hand vein recognition result of the bottom collection area. The display screen 6 adopts a touch screen and is also used for carrying out man-machine interaction, wherein the man-machine interaction comprises user operation prompts.

Fig. 5 is a structural view of a hand placement panel area according to an embodiment of the present invention. Referring to fig. 5, the hand placement panel section 2 specifically includes: the hand placing area 7, the hand texture figure 8, the bulge 9 and the physiological parameter detecting sensor 10.

The hand placement area 7 is located directly below the top camera 4, and the hand placement area 7 is used for placing the hand to be verified. The hand placement area 7 is a glass panel, preferably a transparent acrylic panel, and is also used to support the placement of the hand.

The hand pattern figure 8 is engraved in the centre of the hand placement area 7. Fig. 6 is a view showing the structure of the protrusions according to the embodiment of the present invention, and referring to fig. 6, the protrusions 9 are located at the middle finger first joint of the hand texture fig. 8. The hand texture figure 8 and the protrusions 9 are used for displaying the placement position of the hand to be verified. The height of the protrusions 9 is 1 millimeter (mm). When the hand is used, one surface of the palm faces downwards, the first joint of the middle finger of the hand is arranged right above the bulge 9, and the middle finger is arranged in the middle finger area of the hand texture chart; the use of the bulges and the hand texture patterns can ensure that the positions of the hands to be verified to be placed each time are relatively fixed.

A physiological parameter detecting sensor 10 is located below the boss 9, the physiological parameter detecting sensor 10 being for detecting a physiological parameter. The physiological parameter detecting sensor needs to be in close contact with the finger so that it is placed under the hand placement area. The physiological parameter detection sensor 10 is connected with the bottom acquisition area 3, and transmits the acquired physiological parameters to the hand vein recognition system of the bottom acquisition area. The physiological parameter detecting sensor 10 is specifically located below the finger area close to the protrusion 9 in the hand texture figure 8, and determines whether the hand to be verified is a living body by detecting the physiological parameter of the middle finger. The physiological parameter detection sensor 10 adopts a MAX30100 blood oxygen heart rate sensor, can detect physiological parameters such as blood oxygen heart rate of a human body, and the physiological parameter detection sensor transmits the detected physiological parameters to the hand vein recognition system, wherein the physiological parameters are pulse blood oxygen values of the hand to be verified.

MAX30100 is a sensor integrated with pulse oximeter and heart rate detection, and the blood oxygen heart rate sensor is integrated with a red LED (Light Emitting Diode ), an infrared LED, a light device, a photoelectric sensor and a low-noise electronic circuit with ambient light suppression, and is a pulse blood oxygen saturation monitoring sensor with minimum size and low power consumption in the industry at present.

The detection by the oximetry heart rate sensor is based on photoplethysmography. Photoplethysmography is based on the determination of blood volume changes by detecting changes in the intensity of reflected and/or transmitted light by means of a photosensor based on the absorption properties of human body tissue for light. The blood oxygen heart rate sensor irradiates light with known wavelength to human body tissues, and the lambert-beer law shows that the absorption of light by non-blood tissues such as human skin, muscle and the like is unchanged, but the human heart stretches to cause periodic change of blood volume and pressure, and the beating of the heart is compared with the change of light intensity, so that the condition of the beating of the heart, namely the pulse, can be known by the light intensity detected by the photoelectric sensor. Meanwhile, the size of the blood vessel capacity depends on the size of the ejection blood volume, and when the heart contracts, the ejection blood volume increases, so that the blood vessel capacity increases; conversely, when the heart is relaxed, the ejection volume is reduced, so that the blood vessel capacity is reduced, the increase of the blood vessel capacity indicates the increase of the absorption quantity of the blood to the light, and the light intensity received by the receiving end is reduced due to the constant light intensity of the injected light source; conversely, the decrease of the blood vessel capacity indicates the decrease of the absorption quantity of the blood to the light, the increase of the light intensity received by the receiving end, and the pulse wave is obtained through photoelectric conversion.

Fig. 7 is a flow chart of data processing of the blood oxygen heart rate sensor according to the embodiment of the invention. Referring to fig. 7, the operating principle of the blood oxygen heart rate sensor for measuring blood oxygen saturation pulse is that an LED driver inside a photoelectric sensor drives an LED of the photoelectric sensor to alternately irradiate red light and infrared light according to a preset time sequence, and then the photoelectric sensor collects reflected light signals, and meanwhile the photoelectric sensor converts the reflected light signals into analog electric signals. Analog electric signals output by the photoelectric sensor are amplified, filtered, subjected to analog-to-digital conversion and finally stored in a FIFO (First Input First Output, first-in first-out) buffer, and the MAX30100 sensor can communicate through a serial port, and can particularly communicate through I ² The C bus transmits the digital signal to the main controller, thereby obtaining the pulse blood oxygen value. After receiving the pulse blood oxygen value, the main controller judges whether the pulse blood oxygen value is more than 90; if the pulse blood oxygen value is greater than 90, passing the physiological parameter living verification; otherwise, i.e. pulse oximetry value is less than or equal to 90, the physiological parameter is not validated in vivo.

The hand placement panel area 2 further comprises a label 11, the label 11 being located at the upper left of the hand placement area 7. The label 11 is a prompt label, and the label 11 is used for marking the using steps and notes of the hand vein recognition device, so that the hand vein recognition device is convenient for the primary use and the operation of users with less use.

Fig. 8 is a top view of the bottom acquisition region according to an embodiment of the present invention, and referring to fig. 8, the bottom acquisition region 3 specifically includes: a bottom camera 12, a bottom light source 13, a hand vein recognition system, and a power supply 15.

The bottom camera 12 is used for acquiring palm vein images of the hand to be verified; the output end of the bottom camera is connected with the input end of the hand vein recognition system, and the collected images are transmitted to the hand vein recognition system through a universal serial bus (Universal Serial Bus, USB) to be processed by the hand vein recognition system. The top camera 4 and the bottom camera 12 should correspond to the center of the hand texture map 8. The frame rate of the top camera 4 and the bottom camera 12 is greater than 30.

The bottom light source 13 is disposed corresponding to the bottom camera 12, and the bottom light source 13 is used for illuminating the palm of the hand to be verified. The bottom light source 13 is a near infrared lamp set including 6 near infrared lamps having a diameter of 850 nanometers (nm).

The top collecting area and the bottom collecting area respectively collect vein images of the palm and the back of the hand under the irradiation of the near infrared light according to the absorption principle of hemoglobin to near infrared light. According to the embodiment, vein images of the back of hand and the palm are required to be identified and verified simultaneously, so that the safety of vein biological identification is improved.

The hand vein recognition system is also respectively connected with the top camera 4 and the physiological parameter detection sensor 10, and the input end of the hand vein recognition system is also respectively connected with the output end of the top camera 4 and the physiological parameter detection sensor 10; the hand vein recognition system is used for acquiring the living experience information and processing the living experience information to obtain a hand vein recognition result. The hand vein recognition system is also used for carrying out information interaction with the touch screen, wherein the information interaction comprises the following steps: and receiving an instruction of the touch screen, and displaying a hand vein recognition result on the touch screen. The hand vein recognition system obtains a hand vein recognition result through living body detection, wherein the living body detection comprises two parts of living body detection of physiological parameters and living body detection of action characteristics, and the false of the prosthesis is almost impossible through the combination of the two living body detection modes; the processing of the living experience information includes the processing and recognition of vein images of the palm and back of the hand, and the processing of living body detection.

The power supply 15 is respectively connected with the top light source 5, the bottom light source 13, the physiological parameter detection sensor 10 and the hand vein recognition system, and the power supply 15 is used for supplying power, namely voltage, to the top light source 5, the bottom light source 13, the physiological parameter detection sensor 10 and the hand vein recognition system.

The hand vein recognition system specifically comprises:

the physiological parameter acquisition module is used for acquiring the physiological parameters of the hand to be verified, which are detected by the physiological parameter detection sensor.

The physiological parameter acquisition module specifically comprises:

the physiological parameter acquisition unit is used for acquiring pulse blood oxygen values of the hand to be verified, which are detected by the blood oxygen and heart rate sensor.

And the action characteristic living experience verification module is used for performing action characteristic living verification on the hand to be verified when the physiological parameter is greater than a first preset threshold value, so as to obtain a verification result.

The action characteristic liveness experience verification module specifically comprises:

and the fist-making image acquisition unit is used for acquiring a fist-making image of the hand to be verified.

And the semi-fist-making image acquisition unit is used for acquiring a semi-fist-making image of the hand to be verified when the hand to be verified in the fist-making image is in a fist-making state.

And the hand image acquisition unit is used for acquiring hand images of the hand to be verified, which are in different fist making states from the fist making state to the semi-fist making state, when the hand to be verified in the semi-fist making image is in the semi-fist making state.

And the similarity comparison unit is used for detecting the optical flow characteristics of the hand image by using an optical flow method and comparing the similarity between the optical flow characteristics of the hand image and the pre-stored optical flow characteristics.

The similarity comparison unit specifically includes:

the first interested region extraction subunit is configured to take the ith frame of hand image as a first comparison image, and extract the interested region of the first comparison image to obtain a first interested region.

And the second interested region extraction subunit is used for taking the (i+1) th frame hand image as a second comparison image, and extracting the interested region of the second comparison image to obtain a second interested region.

The optical flow feature detection subunit is used for detecting the positions of the first region of interest and the second region of interest, the fluctuation range of which is larger than a fourth preset threshold value, by using an optical flow method, so as to obtain optical flow features.

And the return subunit is used for making i=i+1 and returning to the first region of interest extraction subunit to obtain optical flow characteristics of all hand images.

The pre-stored optical flow characteristic acquisition subunit is used for acquiring pre-stored optical flow characteristics.

And the similarity calculation subunit is used for comparing all the optical flow characteristics with the corresponding pre-stored optical flow characteristics respectively to obtain similarity.

And the liveness experience verification success unit is used for verifying the liveness experience success when the similarity is larger than a third preset threshold value.

And the liveness experience failure unit is used for failing the liveness experience when the similarity is smaller than or equal to a third preset threshold value.

The vein image acquisition module is used for acquiring palm vein images and back hand vein images of the hand to be verified when the verification result shows that the living experience is successful.

The vein storage image acquisition module is used for acquiring a pre-stored palm vein storage image and a hand back vein storage image.

The similarity calculation module is used for calculating the palm similarity of the palm vein image and the palm vein storage image and the back similarity of the back hand vein image and the back hand vein storage image respectively.

And the hand vein recognition module is used for successfully recognizing the hand veins of the hand to be verified when the palm similarity and the hand back similarity are both larger than a second preset threshold value.

The use flow of the hand vein recognition device mainly comprises the following steps: identity registration and identity verification; the identity registration is to store biological information of the user, wherein the biological information comprises biological characteristics of the hand of the user; the authentication is to extract the biological characteristics of the hand to be authenticated and match the biological characteristics which are stored in advance, wherein the biological characteristics are the biological information of the user stored in the process of identity registration; the use flow of identity registration and identity verification is basically the same. Fig. 9 is a flowchart of a use procedure of the hand vein recognition device according to an embodiment of the present invention, referring to fig. 9, the use procedure of the hand vein recognition device is as follows: the touch screen function selection comprises identity registration and identity verification, and the identity registration is carried out through the touch screen selection registration function; and selecting a verification function through the touch screen to perform identity verification.

Fig. 10 is a flowchart of an identity registration process according to an embodiment of the present invention, referring to fig. 10, the identity registration process includes: placing the hand to be registered in a designated area according to the requirement; the requirements are as follows: the back of the hand faces upwards, the palm faces downwards, the middle finger is placed on the middle finger area of the palm image of the hand texture chart, and the first joint of the middle finger is placed on the bulge.

The physiological parameters detected by the physiological parameter detection sensor are obtained, and the living body is verified through the physiological parameters.

Judging whether the verification of the physiological parameter verification living body passes or not; if the verification is passed, verifying the living body through the action characteristics; otherwise, the liveness experience fails.

And (5) verifying the activity characteristics: and acquiring the optical flow characteristics of the hand to be registered when performing the registration action.

Optical flow features are stored.

Acquiring hand images through a top camera (top camera) respectively to acquire a back hand vein image; hand images are acquired through a bottom camera (bottom camera), and palm vein images are acquired. The hand image collected by the top camera is a hand image containing fingers and the back of the hand, and the back vein image is an image containing only the back of the hand. The hand image collected by the bottom camera is a hand image containing fingers and a palm, and the palm vein image is an image containing only a palm portion.

The dorsal venous images are segmented from the dorsal venous images. The dorsum manus vein image is an image including a portion of the dorsum manus vein.

Palm vein images are segmented from the palm vein map. The palm vein image is an image containing veins of the palm portion.

Respectively calculating image characteristic values of the back hand vein image and the palm vein image, storing, and returning to the step of acquiring hand images through top cameras (top cameras) respectively to acquire a back hand vein image; the hand image is acquired through a bottom camera (bottom camera), a palm vein image is acquired, the method is circularly executed for three times, and the image characteristic values obtained by three times of calculation are stored.

Fig. 11 is a flowchart of the motion feature living body registration according to the embodiment of the present invention, referring to fig. 11, the motion feature verification living body includes:

the display screen prompts that a user is pleased to make a fist, the top camera collects hand pictures and judges whether the user is in a fist making state or not through the hand pictures; if the hand is in a fist-making state, the display screen prompts 'please make a fist in half', and the collected hand pictures are stored; otherwise, returning to the step of collecting the hand picture by the top camera, and judging whether the hand picture is in a fist-making state or not through the hand picture.

The top camera collects hand pictures and judges whether the hand pictures are in a semi-fist making state or not through the hand pictures; if the hand is in the semi-fist-making state, stopping storing the hand picture; otherwise, returning to the step of collecting the hand picture by the top camera, and judging whether the hand picture is in a semi-fist-making state or not through the hand picture.

Processing the stored hand pictures by using an optical flow method to obtain optical flow characteristics; referring specifically to fig. 12, a hand image stored in the second frame is extracted, and a region of interest in the hand image stored in the second frame is extracted, where in this embodiment, the region of interest is between the first joint of the finger and the wrist, that is, a back of hand portion; comparing the extracted hand picture with a previous frame picture, wherein the optical flow characteristics of the interested region of the previous frame picture need to be extracted before the extracted hand picture is compared with the previous frame picture, and the previous frame picture is a hand picture stored in the previous frame of the currently extracted hand picture; judging whether the extracted hand picture is a hand picture stored in the last frame; if the hand picture stored in the last frame is the hand picture, drawing an optical flow angle and a modulus value of the hand picture stored in the first frame, and taking the optical flow angle and the modulus value as optical flow characteristics; if the hand image is not the hand image stored in the last frame, extracting the hand image stored in the next frame, and returning to the step of comparing the extracted hand image with the previous frame image, wherein before the comparison, the optical flow characteristics of the interested region of the previous frame image need to be extracted. The method comprises the steps of describing the distribution condition of the speed u of each pixel point in the x direction (horizontal direction) and the speed v in the y direction (vertical direction) in hand pictures stored in the first frame, describing a speed curve diagram of each point in the first frame to the last frame, wherein the tangent angle between the starting point and the end point of the speed curve is an optical flow angle, the distance between the starting point and the end point is a module value, the optical flow angle and the module value are characteristic values of optical flow characteristics of the starting point, and if the tangent angle and the distance are larger than a preset threshold value, judging the starting point as the characteristic point and storing the characteristic value of the starting point.

Optical flow features are stored.

Fig. 13 is a flowchart of an authentication process according to an embodiment of the present invention, referring to fig. 13, the authentication process includes: placing the hand to be verified in a designated area according to the requirement; the requirements are as follows: the back of the hand faces upwards, the palm faces downwards, the middle finger is placed on the middle finger area of the palm image of the hand texture chart, and the first joint of the middle finger is placed on the bulge.

Judging whether the verification of the activity of the action feature verification living body is passed or not; if the hand image passes the verification, acquiring a hand back vein image through a top camera (top camera), and acquiring a hand image through a bottom camera (bottom camera) to acquire a palm vein image; otherwise, the liveness experience fails.

The dorsal venous images are segmented from the dorsal venous images.

Palm vein images are segmented from the palm vein map.

And respectively calculating the similarity of the back hand vein image and the pre-stored back hand vein storage image, and the similarity of the palm vein image and the pre-stored palm vein storage image, judging whether the hand vein recognition verification is passed or not according to the similarity, and displaying a hand vein recognition verification result on the touch screen. The pre-stored back hand vein storage image and the pre-stored palm vein storage image are the back hand vein image and the palm vein image stored during identity registration.

The action feature verification living body includes: the touch screen prompts to make a fist, the top camera detects a fist making image, the semi-fist making action is detected, and the semi-fist making action is detected. Touch screen prompting: please make a fist, after the top camera detects the action of making a fist, the touch screen prompts: please semi-make a fist, if no fist making action is detected within 2s, prompt for failure of liveness experience. All pictures between the two movements of the fist making movement and the semi-fist making movement are extracted, the pictures of two adjacent frames are taken for comparison from the beginning of the fist making movement to the end of the semi-fist making movement, the position with larger fluctuation range in the pictures is tracked by an optical flow method, the position with larger fluctuation range is taken as an optical flow characteristic to be stored, and the stored optical flow characteristic is as follows: the position of a point with a large fluctuation range on the back of the hand and the direction information of the movement of the point. Repeating the comparison of the pictures of two adjacent frames, comparing the similarity of the extracted optical flow characteristics with the pre-stored optical flow characteristic pairs, and judging the similarity according to a third preset threshold value. The basis for verifying the activity of the motion characteristics is as follows: the movement of the joints in the process from fist making to semi-fist making of each person is specific, and the optical flow characteristics of the whole process from fist making to semi-fist making are taken as the basis for judgment. The optical flow features are fewer, and the third preset threshold value is lower, so that the optical flow features are only used for preliminary identity screening.

The flow of action feature liveness experience is specifically seen in fig. 14:

Processing the stored hand pictures by using an optical flow method to obtain optical flow characteristics, comparing the optical flow characteristics with the optical flow characteristics stored during identity registration, and if the similarity is greater than a third preset threshold value, indicating that the living experience passes; otherwise, the liveness experience fails.

In this embodiment, the fist-making state is determined as the fist-making state when the first joints of the index finger, the middle finger and the ring finger of the hand are recognized to be protruded. In the state process from fist making to palm flattening, the semi-fist making state is judged, and the second joint is certainly protruded, so that the semi-fist making state can be judged by identifying the second joint protrusion of the index finger, the middle finger and the ring finger.

Fig. 15 is a flowchart of a process of hand image according to an embodiment of the present invention, referring to fig. 15, step "acquire hand image by top camera (top camera) to acquire back of hand vein image, and acquire hand image by bottom camera (bottom camera) to acquire palm vein image; segmenting a dorsal vein image from the dorsal vein image; segmenting a palm vein image from the palm vein image; respectively calculating the similarity of the back hand vein image and the pre-stored back hand vein storage image and the similarity of the palm vein image and the pre-stored palm vein storage image, judging whether the hand vein recognition verification is passed or not according to the similarity, and displaying a hand vein recognition verification result on a touch screen, wherein the method specifically comprises the following steps of: the top camera and the bottom camera collect hand images; image segmentation is carried out on the back hand vein image and the palm vein image, so that a back hand vein image and a palm vein image are obtained; sequentially carrying out image normalization, niblack binarization, median filtering and SIFT (Scale-invariant feature transform, scale invariant feature transform) feature extraction on the back vein image and the palm vein image to obtain back hand features and palm features; respectively carrying out feature matching on the back hand features and the palm features with pre-stored back hand features and palm features to obtain matching scores; judging whether the matching score is larger than a second preset threshold value or not; if the matching score is larger than a second preset threshold, the hand vein recognition verification is successful; if the matching score is smaller than or equal to a second preset threshold value, the hand vein recognition verification fails; the back of hand feature and palm feature are stored. The algorithm adopted in this part is quite common, and this part is only completed to ensure the flow integrity of the whole hand vein recognition device. The pre-stored hand back features are obtained through hand back vein storage images stored during identity registration, and the pre-stored palm features are obtained through palm vein storage images stored during identity registration.

Firstly, detecting whether real blood flow exists by adopting a blood oxygen heart rate sensor, and judging whether the hand to be verified is a living body or not by using the blood oxygen heart rate sensor to detect pulse blood oxygen value, so that false prosthesis can be broken; then adopting action recognition to enable a person to be verified to make a fist making action and a semi-fist making action, when making a fist to the semi-fist making, jumping of joints of each person on the back of the hand is different, analyzing the living body by utilizing the fist making action according to the path of the jumping joints and the fist making action by utilizing an optical flow method, judging whether the hand to be verified is a living body or not, and improving the reliability of living experience; the vein lines of the palm and the back of the hand are extracted, and the mode of combining the back of the hand with the palm is adopted, so that the safety and the reliability are improved.

In the present specification, each embodiment is described in a progressive manner, and each embodiment is mainly described in a different point from other embodiments, and identical and similar parts between the embodiments are all enough to refer to each other. For the system disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant points refer to the description of the method section.

The principles and embodiments of the present invention have been described herein with reference to specific examples, the description of which is intended only to assist in understanding the methods of the present invention and the core ideas thereof; also, it is within the scope of the present invention to be modified by those of ordinary skill in the art in light of the present teachings. In view of the foregoing, this description should not be construed as limiting the invention.

Claims

1. A dual in vivo verification hand vein recognition method, comprising:

if the physiological parameter is larger than a first preset threshold, performing action characteristic living experience verification on the hand to be verified to obtain a verification result; the method specifically comprises the following steps:

acquiring a fist-making image of the hand to be verified;

detecting the optical flow characteristics of the hand image by using an optical flow method, and comparing the similarity of the optical flow characteristics of the hand image with prestored optical flow characteristics; the method specifically comprises the following steps:

acquiring prestored optical flow characteristics;

comparing all the optical flow characteristics with corresponding pre-stored optical flow characteristics respectively to obtain similarity;

if the similarity is smaller than or equal to the third preset threshold, the liveness experience fails;

2. The dual in-vivo verification hand vein recognition method according to claim 1, wherein the acquiring the physiological parameter of the hand to be verified detected by the physiological parameter detection sensor specifically comprises:

3. A dual in-vivo verification hand vein recognition apparatus, comprising: a top acquisition zone, a hand placement panel zone, and a bottom acquisition zone; the top collecting area and the hand placing panel area are connected with the bottom collecting area;

the top acquisition area is positioned at the top of the hand vein recognition device and is used for acquiring a hand back vein image of the hand to be verified;

the bottom acquisition area is positioned at the bottom of the hand vein recognition device and below the hand placement panel area, and is used for acquiring living experience information and processing the living experience information to obtain a hand vein recognition result; the liveness experience information comprises the back hand vein image, the physiological parameter and the palm vein image of the hand to be verified;

The bottom acquisition area specifically comprises a bottom camera, a bottom light source, a hand vein recognition system and a power supply; the hand vein recognition system specifically comprises:

the physiological parameter acquisition module specifically comprises:

the physiological parameter acquisition unit is used for acquiring pulse blood oxygen values of the hand to be verified, which are detected by the blood oxygen and heart rate sensor;

the action characteristic living experience verification module specifically comprises:

the fist-making image acquisition unit is used for acquiring a fist-making image of the hand to be verified;

the hand image acquisition unit is used for acquiring hand images of the hand to be verified, which are different from the fist making state to the semi-fist making state, when the hand to be verified in the semi-fist making image is in the semi-fist making state;

The similarity comparison unit specifically includes:

the first interested region extraction subunit is used for taking the ith frame of hand image as a first comparison image, and extracting the interested region of the first comparison image to obtain a first interested region;

a second region of interest extraction subunit, configured to use the i+1st frame of hand image as a second comparison image, and extract a region of interest of the second comparison image, to obtain a second region of interest;

the optical flow feature detection subunit is used for detecting the positions of the first region of interest and the second region of interest, the fluctuation range of which is larger than a fourth preset threshold value, by using an optical flow method to obtain optical flow features;

a return subunit, configured to return i=i+1 to the first region of interest extraction subunit, to obtain optical flow features of all hand images;

a pre-stored optical flow characteristic obtaining subunit, configured to obtain a pre-stored optical flow characteristic;

the similarity calculating subunit is used for comparing all the optical flow characteristics with corresponding pre-stored optical flow characteristics respectively to obtain similarity;

the liveness experience failure unit is used for failing the liveness experience when the similarity is smaller than or equal to a third preset threshold value;

The vein image acquisition module is used for acquiring palm vein images and back hand vein images of the hand to be verified when the verification result shows that the living experience is successful;

the similarity calculation module is used for calculating the palm similarity of the palm vein image and the palm vein storage image and the back similarity of the back vein image and the back vein storage image respectively;

4. The dual in-vivo verification hand vein recognition apparatus of claim 3, wherein said top acquisition region specifically comprises: a top camera, a top light source, and a display screen;

5. The dual in-vivo verification hand vein recognition apparatus as defined in claim 4, wherein said hand placement panel section specifically comprises: the hand placing area, the hand texture map, the bulge and the physiological parameter detection sensor;

6. The dual in-vivo verification hand vein recognition apparatus of claim 5, wherein,