CN115641649A - Face recognition method and system - Google Patents

Face recognition method and system Download PDF

Info

Publication number
CN115641649A
CN115641649A CN202110813929.6A CN202110813929A CN115641649A CN 115641649 A CN115641649 A CN 115641649A CN 202110813929 A CN202110813929 A CN 202110813929A CN 115641649 A CN115641649 A CN 115641649A
Authority
CN
China
Prior art keywords
face
spectral
spectral image
image
facial
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202110813929.6A
Other languages
Chinese (zh)
Inventor
黄翊东
崔开宇
张巍
冯雪
刘仿
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Heguang Technology Co ltd
Tsinghua University
Original Assignee
Beijing Heguang Technology Co ltd
Tsinghua University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Heguang Technology Co ltd, Tsinghua University filed Critical Beijing Heguang Technology Co ltd
Priority to CN202110813929.6A priority Critical patent/CN115641649A/en
Priority to PCT/CN2022/098308 priority patent/WO2023000864A1/en
Publication of CN115641649A publication Critical patent/CN115641649A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/60—Extraction of image or video features relating to illumination properties, e.g. using a reflectance or lighting model
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/40—Spoof detection, e.g. liveness detection

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • Software Systems (AREA)
  • Multimedia (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Computing Systems (AREA)
  • Artificial Intelligence (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Medical Informatics (AREA)
  • Databases & Information Systems (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Human Computer Interaction (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Molecular Biology (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Image Analysis (AREA)

Abstract

The invention provides a face recognition method and a face recognition system, wherein the face recognition method comprises the following steps: acquiring a human face spectral image to be recognized; inputting the facial spectral image to be recognized into a trained facial spectral image feature extraction model to obtain a target feature vector of the facial spectral image, wherein the trained facial spectral image feature extraction model is obtained by training a machine learning model by using a sample facial spectral image marked with a facial information label and a true and false type spectral information label; and identifying the target characteristic vector according to a similarity evaluation standard to obtain a face identification result and a living body identification result. The invention makes up the security loophole of the traditional face detection and improves the accuracy of the face recognition result and the security of the face recognition system.

Description

Face recognition method and system
Technical Field
The invention relates to the technical field of spectral imaging, in particular to a face recognition method and system.
Background
The face recognition is a biological recognition technology for carrying out identity recognition based on face feature information of people, and attributes such as identity, expression, age, gender and the like of people are automatically deduced by collecting images or videos containing faces for analysis. As one of the biological feature recognition technologies, the face recognition has the characteristics of non-mandatory property, non-contact property, simple operation, good concealment and the like, so that the face recognition technology is widely applied to the fields of security, management and supervision, multimedia entertainment and the like.
Most of the existing face recognition technologies are based on gray level images or color RGB images, the available image information is limited, the change of illumination conditions, shooting visual angles and the like has direct influence on the recognition result, the change of factors such as face expression, hair style, makeup, glasses and the like can cause the reduction of the recognition accuracy, and lawbreakers can cause serious interference on the recognition result through disguising, shielding, masking and photo printing. The face recognition technology based on the traditional imaging system only utilizes the space geometric characteristics of an observation object, is very sensitive to uncertainty caused by various condition changes, has poor system robustness, and has sharply reduced recognition performance in a complex environment, so that the problem of camouflage of a silica gel mask can not be solved even if 3D face information is introduced, and the recognition of a living face can not be realized.
Therefore, a face recognition method and system are needed to solve the above problems.
Disclosure of Invention
Aiming at the problems in the prior art, the invention provides a face recognition method and a face recognition system.
The invention provides a face recognition method, which comprises the following steps:
acquiring a human face spectral image to be recognized;
inputting the facial spectral image to be recognized into a trained facial spectral image feature extraction model to obtain a target feature vector of the facial spectral image, wherein the trained facial spectral image feature extraction model is obtained by training a machine learning model by using a sample facial spectral image marked with an identity information label and a true and false type spectral information label to which a face belongs;
and identifying the target characteristic vector according to a similarity evaluation standard to obtain a face identification result and a living body identification result.
The invention provides a face recognition method, which is used for acquiring a spectral image of a face to be recognized and comprises the following steps:
acquiring a human face spectral image to be recognized through a human face recognition spectral imaging chip;
face identification spectrum imaging chip includes light modulation layer, image sensor layer and signal processing circuit layer, the light modulation layer the image sensor layer with the signal processing circuit layer is followed the vertical direction from last to connecting gradually down, wherein:
the light modulation layer is used for receiving the light signal reflected by the face to be recognized and carrying out light modulation;
the image sensor layer is used for converting an optical signal reflected by the human face to be recognized after the optical modulation into an electric signal, and the electric signal comprises human face image space information and skin spectrum information;
and the signal processing circuit layer is used for processing the face image space information and the skin spectrum information output by the image sensor layer to obtain a face recognition result.
According to the face recognition method provided by the invention, the face spectral imaging chip further comprises a lens group, wherein the lens group is positioned on the upper surface of the light modulation layer, is connected with the light modulation layer and is used for focusing and imaging the optical signal reflected by the face to obtain the optical signal reflected by the face to be recognized.
According to the face recognition method provided by the invention, the light modulation layer comprises at least one light modulation unit, the light modulation unit comprises a plurality of micro-nano structure arrays, through holes are uniformly distributed and arranged in each micro-nano structure array according to different preset arrangement rules, and the shapes of the through holes of the micro-nano structure arrays are different.
According to the face recognition method provided by the invention, a plurality of photosensitive pixel units are distributed on the upper surface of the image sensor layer, and each micro-nano structure array corresponds to at least one photosensitive pixel unit.
According to the face recognition method provided by the invention, the trained face spectral image feature extraction model is obtained by training the following steps:
constructing a sample training set according to a sample face spectral image marked with an identity information label to which a face belongs and a true and false type spectral information label, wherein the true and false type spectral information label is a spectral information label of a living body face, a simulated face and a non-face type;
and inputting the sample training set into the machine learning model for training to obtain a trained face image feature extraction model, wherein the machine learning model is a convolutional neural network.
According to the face recognition method provided by the invention, the step of inputting the sample training set into the machine learning model for training to obtain a trained face spectral image feature extraction model comprises the following steps:
and training a convolution kernel in the convolutional neural network through the sample training set based on a deep learning algorithm, and if a preset training condition is met, obtaining a trained face spectral image feature extraction model, wherein the convolution kernel is used for detecting the face corner point outline and the skin spectral characteristic.
The invention also provides a face recognition system, comprising:
the acquisition spectral image module acquires a spectral image of a human face to be recognized;
the extraction feature vector module is used for inputting the facial spectral image to be recognized into a trained facial spectral image feature extraction model to obtain a target feature vector of the facial spectral image, wherein the trained facial spectral image feature extraction model is obtained by training a machine learning model by using a sample facial spectral image marked with a facial information label and an authenticity type spectral information label;
and the recognition module is used for recognizing the target characteristic vector according to the similarity evaluation standard to obtain a face recognition result.
According to a face recognition system provided by the invention, the system further comprises:
the system comprises a sample training set building module, a sample training set processing module and a sample analysis module, wherein the sample training set building module is used for building a sample training set according to a sample face spectral image marked with a face information label and an authenticity type spectral information label, the face information label is an identity information label to which a face belongs, and the authenticity type spectral information label is a spectral information label of a living body face, a simulated face and a non-face type;
and the training feature extraction model module is used for inputting the sample training set into the machine learning model for training to obtain a trained face image feature extraction model, and the machine learning model is a convolutional neural network.
The invention also provides a non-transitory computer-readable storage medium having stored thereon a computer program which, when executed by a processor, performs the steps of the face recognition method as described in any of the above.
The invention provides a face recognition method and a system, which are characterized in that a face spectral image is acquired and input into a face spectral image feature extraction model to obtain target feature vectors of face spatial characteristics and spectral reflection characteristics; compared with the traditional face recognition method, the method has the advantages that the target characteristic vectors and the characteristic vectors in the database are compared and recognized by utilizing the similarity evaluation standard, and the face recognition and the living body recognition are simultaneously realized by utilizing the skin spectral characteristics, so that the security loophole of the traditional face detection is made up, the accuracy of the face recognition result is improved, and the security of a face recognition system is improved.
Drawings
In order to more clearly illustrate the technical solutions of the present invention or the prior art, the drawings needed for the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and those skilled in the art can also obtain other drawings according to the drawings without creative efforts.
Fig. 1 is a schematic flow chart of a face recognition method provided by the present invention;
FIG. 2 is a schematic structural diagram of a face recognition spectral imaging chip provided by the present invention;
fig. 3 is a schematic structural diagram of a micro-nano structure array of a light modulation layer in the face recognition spectral imaging chip provided by the invention;
fig. 4 is a schematic structural diagram of a face recognition system provided by the present invention;
fig. 5 is a schematic structural diagram of an electronic device provided in the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is obvious that the described embodiments are some, but not all embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The common RGB camera can only shoot images of RGB three channels, and the in-vivo detection algorithm based on the common RGB camera only uses image information of RGB three channels, and generally needs to be recognized through relatively complex algorithms such as video sequence analysis, so that the real-time performance and the reliability are poor, and the high-simulation face recognition rate is low.
Fig. 1 is a schematic flow diagram of a face recognition method provided by the present invention, and as shown in fig. 1, the present invention provides a face recognition method, which includes:
step 101, acquiring a human face spectral image to be recognized;
102, inputting the facial spectral image to be recognized into a trained facial spectral image feature extraction model to obtain a target feature vector of the facial spectral image, wherein the trained facial spectral image feature extraction model is obtained by training a machine learning model by using a sample facial spectral image marked with an identity information label to which a face belongs and a true and false type spectral information label;
and 103, identifying the target characteristic vector according to the similarity evaluation standard to obtain the results of face identification and living body identification.
In the invention, in step 101, a human face spectrum image to be recognized can be acquired by a calculation spectrum device or a hyperspectral device.
Further, in step 102, the identity information tag to which the face belongs is a tag containing spatial characteristics of the face with different identity IDs. And training the machine learning model according to the sample face spectral image marked with the identity information label to which the face belongs and the authenticity category spectral information label to obtain a face spectral image feature extraction model. The method comprises the steps of inputting a human face spectral image to be recognized into a trained human face spectral image feature extraction model, wherein the human face spectral image feature extraction model is a machine learning model, training the machine learning model to convert the human face spectral image into a target feature vector in a high-dimensional space, and the obtained target feature vector comprises human face spatial characteristics and spectral reflection characteristics. It should be noted that the target feature vector may be a feature vector of a living human face, may be a feature vector of a simulated human face, and may also be a feature vector of a non-human face.
It should be noted that the facial spectral image feature extraction model may be a convolutional neural network, or may be any Machine learning model, and the Machine learning model may also be a Support Vector Machine (SVM), or may also be a perceptron. The present invention is described in terms of a convolutional neural network training process.
Further, in step 103, the similarity evaluation criterion may be an euclidean distance between the feature vectors, an included angle of the feature vectors in the high-dimensional space, or an algorithm for measuring the similarity between the vectors in the high-dimensional space. And selecting a sample feature vector with the highest similarity with a target feature vector to be recognized in an existing human face image database as a candidate recognition result, wherein the sample feature vector in the database comprises human face spatial characteristics and spectral reflection characteristics. If the similarity reaches a set threshold, the identification is considered to be successful, otherwise, the identification is considered to be failed. And comparing and identifying the target characteristic vector through the similarity evaluation standard so as to obtain the results of face identification and living body identification.
The face recognition method provided by the invention comprises the steps of obtaining a face spectral image, inputting the face spectral image into a face spectral image feature extraction model, and obtaining target feature vectors of face spatial characteristics and spectral reflection characteristics; compared with the traditional face recognition method, the method has the advantages that the skin spectral characteristics are utilized, the face recognition and the living body recognition are simultaneously realized, the security loopholes of the traditional face detection are made up, the accuracy of the face recognition result is improved, and the security of a face recognition system is improved.
Preferably, the face recognition method provided by the invention can be realized by a face recognition spectral imaging chip, and particularly can be realized by a signal processing circuit layer in the face recognition spectral imaging chip. And acquiring a human face spectrum image to be recognized through the human face recognition spectrum chip. The face to be recognized is shot by using the face recognition spectral imaging chip, the shot face spectral image reaches hundreds of channels, and the face spectral image to be recognized is obtained. The human face spectral image comprises human face image spatial information and skin spectral information; the face to be recognized may be a living body face, a face image, a face in a video, or an object.
Fig. 2 is a schematic structural diagram of the face recognition spectral imaging chip provided by the present invention, and as shown in fig. 2, the present invention provides a face recognition spectral imaging chip, which includes a light modulation layer 2021, an image sensor layer 2022 and a signal processing circuit layer 2023, wherein the light modulation layer 2021, the image sensor layer 2022 and the signal processing circuit layer 2023 are sequentially connected from top to bottom along a vertical direction, wherein:
the light modulation layer 2021 is used for receiving the light signal reflected by the face 203 to be recognized and performing light modulation;
the image sensor layer 2022 is configured to convert an optical signal reflected by the optically modulated human face 203 to be recognized into an electrical signal, where the electrical signal includes human face image spatial information and human face spectral information after optical modulation;
the signal processing circuit layer 2023 is configured to process the face image spatial information and the optically modulated face spectral information output by the image sensor layer 2022 to obtain a face recognition result.
Optionally, the face spectral imaging chip further includes a lens group 201, where the lens group 201 is located on the upper surface of the light modulation layer 2021, and is connected to the light modulation layer 2021, and is configured to focus and image the optical signal reflected by the face 203 to obtain the optical signal reflected by the face to be recognized.
In the invention, when the face 203 to be recognized is shot by adopting the face recognition spectral imaging chip, the lens group lens in the face recognition spectral imaging chip faces the face to be recognized. In the face recognition spectral imaging chip, a series of lens groups 201 are arranged on one side of a chip internal structure 202, as shown in fig. 2, a light modulation layer 2021, an image sensor layer 2022 and a signal processing circuit layer 2023 constitute the internal structure 202 of the face recognition spectral imaging chip, light reflected by a face passes through the lens groups 201 to obtain an optical signal reflected by the face after focusing and imaging, and the optical signal reflected by the face after focusing and imaging is used as an optical signal reflected by the face to be recognized; the light modulation layer 2021 is provided with a plurality of light modulation units, each light modulation unit comprises a plurality of micro-nano structure arrays, and the light modulation units are used for performing light modulation on received light signals reflected by the imaged human face to be recognized; a plurality of photosensitive pixel units are arranged on the upper surface of the image sensor layer 2022, and micro-nano structure arrays with different modulation effects on light with different wavelengths are directly prepared on the surface of the photosensitive pixel unit area, so that optical signals reflected by a human face to be identified after the light modulation can be converted into electric signals through the image sensor layer 2022, wherein the electric signals comprise human face image spatial information and light modulated skin spectrum information; the signal processing circuit layer 2023 performs data analysis processing on the face image spatial information and the light modulated face spectral information output by the image sensor layer 2022, and determines whether the target to be recognized is a living body face, a simulated face, or a non-face.
Further, the light modulation layer 2021 is directly formed on the image sensor layer 2022, for example, the light modulation layer is attached, adhered, bonded, deposited on the image sensor layer 2022, and the image sensor layer 2022 and the signal processing circuit layer 2023 are electrically connected.
Optionally, the micro-nano structure arrays have different structure array types, and different modulation modes of the micro-nano structure arrays include, but are not limited to, scattering, absorption, transmission, reflection, interference, excimer, and resonance enhancement.
Optionally, the micro-nano structure array includes, but is not limited to, one-dimensional photonic crystals, two-dimensional photonic crystals, surface plasmons, metamaterials, super-surfaces, and the like. Specific materials may include silicon, germanium, silicon germanium materials, silicon compounds, germanium compounds, III-V materials, and the like, and may also be metals, wherein the silicon compounds include, but are not limited to, silicon nitride, silicon dioxide, silicon carbide, and the like.
Optionally, the light modulation layer 2021 is formed by directly growing one or more layers of materials on the image sensor layer 2022, and etching to prepare a micro-nano structure, for example, depositing and then etching; the micro-nano structure can also be directly prepared on the image sensor layer 2022 by etching.
Alternatively, the image sensor layer may be a CIS wafer, or may be a CCD image sensor.
Furthermore, a face to be recognized is shot by adopting a face recognition spectral imaging chip, the face spectral images of hundreds of channels are obtained by shooting, and the information contained in the face spectral images is far higher than the images obtained by shooting by a common RGB camera. Besides the identification of the identity of the face, the spectral imaging chip is used for acquiring the spectral information of the face, and then the data processing is carried out through the signal processing circuit layer, so that the living body detection can be conveniently carried out to identify the disguised face.
In the invention, the light modulation layer and the image sensor are integrated in a single chip without discrete elements and without additional collimation elements, and the preparation of the spectral imaging chip can be completed through a CMOS (complementary metal oxide semiconductor) process one-time flow sheet, thereby being beneficial to improving the stability of devices, greatly promoting the miniaturization and the light weight of an imaging spectrometer and reducing the cost of face recognition equipment.
On the basis of the embodiment, the light modulation layer comprises at least one light modulation unit, the light modulation unit comprises a plurality of micro-nano structure arrays, through holes which are uniformly distributed and arranged are formed in each micro-nano structure array according to different preset arrangement rules, and the through holes of each micro-nano structure array are different in shape.
Fig. 3 is a schematic structural diagram of a micro-nano structure array of a light modulation layer in the face recognition spectral imaging chip, as shown in fig. 3, a plurality of light modulation units are engraved on the light modulation layer, each unit includes a plurality of micro-nano structure arrays, and the micro-nano structure on the light modulation layer may be a hole penetrating through a flat plate or a micro-nano structure with a certain depth. The light modulation effect can be changed by changing the structural size parameters and/or the structural shape of the micro-nano structure units in the micro-nano structure array, and the unit geometric shapes can include, but are not limited to, circles, crosses, regular polygons, rectangles and any combination thereof. The modulation effect can also be changed by changing parameters of the micro-nano structure, and the change of the structural parameters can include but is not limited to parameters such as micro-nano structure period, radius, side length, duty ratio, thickness and the like, and any combination thereof.
Alternatively, the light modulation layer is silicon and a compound of silicon with a thickness of 300nm, and the total number of the light modulation units is 1000, and the size of each light modulation unit is 400 μm 2 Each light modulation unit comprises 25 micro-nano structure arrays. Each micro-nano structure can be arranged according to different preset arrangement rules, each micro-nano structure can be periodically arranged in the same shape, and the duty ratio is 10% -90%. Each micro-nano structure array is any one of a one-dimensional photonic crystal, a two-dimensional photonic crystal, a surface plasmon polariton, a metamaterial, a super surface and the like.
On the basis of the embodiment, a plurality of photosensitive pixel units are distributed on the upper surface of the image sensor layer, and each micro-nano structure array corresponds to at least one photosensitive pixel unit.
In the invention, a plurality of photosensitive pixel units are arranged on the upper surface of the image sensor layer 2022, micro-nano structure arrays having different modulation effects on light with different wavelengths are directly prepared on the surface of the photosensitive pixel unit area, each micro-nano structure array corresponds to one or more photosensitive pixel units in the vertical direction, and light signals reflected by the human face to be recognized after light modulation can be converted into electric signals through the image sensor layer 2022.
On the basis of the above embodiment, the trained facial image feature extraction model is obtained by training through the following steps:
constructing a sample training set according to a sample face spectral image marked with an identity information label to which a face belongs and a true and false type spectral information label, wherein the true and false information label is a spectral information label of a living body face, a simulated face and a non-face type;
and inputting the sample training set into the machine learning model for training to obtain a trained face image feature extraction model, wherein the machine learning model is a convolutional neural network.
In the invention, labels are marked on sample human face spectral images in a training sample set, and the marked labels comprise two parts of information: one part is an identity information label to which the face belongs and is used for face recognition; and one part is an authenticity type spectrum information label, namely spectrum information of a living body face, a simulated face or a non-face is used for living body identification. And the two parts of label information are simultaneously used for training the same human face spectral image feature extraction model.
And further, inputting a sample human face spectral image training set marked with a human face spectral image information label into a convolutional neural network for training, and acquiring a trained human face spectral image feature extraction model for extracting a target feature vector of the human face spectral image to be recognized.
Alternatively, the simulated face may be a 3D or image, and the non-face may be an animal or object.
Optionally, the category of the whole label may be { non-face, disguised face, face 1, face 2, face 3 \8230 }, where if the training sample set has n face feature information with different identities, the number of the labeled label categories should be n +2, and the information labeled by the label includes the identity information of the face and the spectral reflection characteristic information of the genuineness category.
On the basis of the above embodiment, the inputting the sample training set into the machine learning model for training to obtain a trained human face spectral image feature extraction model includes:
and training a convolution kernel in the convolutional neural network through the sample training set based on a deep learning algorithm, and if a preset training condition is met, obtaining a trained face spectral image feature extraction model, wherein the convolution kernel is used for detecting the face corner point outline and the skin spectral characteristic.
In the invention, based on a deep learning algorithm, a large number of spectral reflection characteristics of real human faces and objects are collected to be used as a sample training set, the sample training set is input into a convolutional neural network to automatically train a convolutional kernel, a loss function value of the convolutional neural network is obtained, and if the loss function value is judged to meet a training convergence condition, a trained human face spectral image feature extraction model is obtained.
In one embodiment, the face recognition and living body recognition processes can be performed independently. Specifically, a face space image to be recognized is input into a trained face image feature extraction model, and a target feature vector of the face space image is obtained, wherein the trained face image feature extraction model is obtained by training a convolutional neural network through a sample face space image marked with a face information label. The target feature vector comprises a face spatial characteristic and is used for identifying the identity of the face. And comparing and identifying the target characteristic vector and the characteristic vector (which can only comprise the spatial characteristics of the human face) in the database according to the similarity evaluation standard, and if the similarity of the target characteristic vector and the characteristic vector reaches a set threshold value, judging that the human face identification is successful, otherwise, judging that the human face identification is failed.
And further, inputting the target feature vector into a trained classifier to obtain a living body recognition result. The trained classifier is obtained by automatically training the deep neural network through the sample feature vector marked with the authenticity category spectrum information label.
Optionally, the classifier may also be obtained by training through a machine learning algorithm such as a Support Vector Machine (SVM) and a perceptron.
It can be understood that there is a significant difference in spectral reflectance characteristics between living and non-living subjects, and thus the classifier can be trained to find a difference in spectral reflectance characteristics that corresponds to living and non-living subjects.
In one embodiment, a face image database is constructed, a target feature vector is matched with a feature vector of the face image database according to a similarity evaluation standard, and a spectral reflection characteristic curve corresponding to the target feature vector is obtained according to a matching result; judging a spectral reflection characteristic curve, and judging and obtaining that the human face spectral image to be identified is a living human face image if the characteristic peaks of the spectral reflection characteristic curve are two minimum value points at 545nm and 575nm wave bands; or the characteristic peak of the spectral reflection characteristic curve is a maximum value point at a 850nm wave band, and the human face spectral image to be recognized is judged and known to be a living human face image.
Optionally, matching filtering is performed on the human face skin spectrum by using the human face skin spectrum reflection characteristic and a convolution kernel with the designed filtering characteristic also being in a W shape, and judgment is performed by setting a threshold value to obtain a living body identification result.
It should be noted that the skin spectrum of a human face mainly represents the spectral characteristics of human skin. Hemoglobin in the skin absorbs 545nm and 575nm light, so that the reflection curve of the skin is W-shaped in a visible light wave band. The human skin reflectance reaches a maximum around 850nm, then diminishes rapidly with increasing wavelength length and increases again to a small extent around 1450 nm. Therefore, in the visible light wave band, the living body identification can be carried out according to at least one characteristic peak of hemoglobin, such as characteristic absorption peaks near the wavelengths of 545nm and 575 nm; and/or in the near infrared band, and also by means of an extreme point at the 850nm band.
In an embodiment, because the spectral imaging chip adopts a calculation imaging mode instead of direct imaging, the original gray level image output by the image sensor can be used for face recognition, and the original image is processed in preprocessing modes such as equalization and noise reduction; and then, carrying out face recognition on the processed original gray level image, then, restoring a face spectrum in the image after the face recognition (restoring some key points in the face spectrum) according to a face recognition spectrum imaging chip, namely acquiring face key point spectrum information, and finally, distinguishing whether the face is a living body face or not through the operational analysis of a signal processing circuit layer in the face spectrum imaging chip.
In another embodiment, three channels of RGB can be extracted from the multi-channel image captured by the spectral imaging chip, so that the face recognition algorithm applicable to the common RGB camera is also applicable to the face spectral image. And designing an angular point detector and a contour detector by utilizing an image analysis technology, abstracting the face image into a characteristic vector in a high-dimensional space, and comparing and identifying the feature vector with the characteristic vector in an existing face image database according to a similarity evaluation standard. The similarity evaluation criterion may be an Euclidean distance between feature vectors, an included angle of the feature vectors in a high-dimensional space, or an algorithm for measuring similarity between vectors in the high-dimensional space. And selecting a sample feature vector with the highest similarity with the feature vector of the target to be recognized in the existing face image database as a candidate recognition result. And if the similarity reaches a set threshold value, the preliminary face recognition is regarded as successful, otherwise, the preliminary face recognition is regarded as failed.
Further, a face image with successful primary face recognition is obtained according to the similarity evaluation standard, and spectral image information of key points of the face in the face image is obtained through shooting by a spectral imaging chip. Since spectral images contain RGB information and there are up to hundreds of channels, a 2D convolution kernel or a 3D convolution kernel can be constructed to extract the feature vectors of the spectral image from the spectral image data cube. Alternatively, the convolution kernel may be provided in a signal processing circuit layer of the face recognition spectral imaging chip. According to the information such as the corner outline of a human face spectral image in the spatial dimension and the existing priori knowledge of the hemoglobin absorption characteristic in the spectral dimension, a convolution kernel for detecting the corner outline of the human face and the spectral characteristic of the skin is constructed through the principle of matched filtering, for example, according to the W-shaped characteristic of the skin spectrum between 500nm and 600nm, the face skin spectrum is matched and filtered by using the convolution kernel with the filtering characteristic also in the W shape, and a filtered skin spectrum reflection characteristic curve is obtained. By utilizing the spectral reflection characteristics of human skin, whether the shot human face target to be recognized is a living human face is judged by detecting the spectrum of the skin spectral reflection characteristic curve at the characteristic peak.
In the invention, a face recognition algorithm and a living body detection algorithm are combined together, the spectral reflection characteristic of the face is obtained by shooting through a spectral imaging chip, the face information is greatly enriched, the defects of the traditional imaging technology are overcome, the anti-interference capability on the influence of factors such as illumination conditions, shooting visual angles, face expressions, hairstyles and makeup is stronger, the face recognition method has good recognition effect on ways such as camouflage, shielding, masks and print pictures, and the accuracy of the face recognition result is greatly improved.
Fig. 4 is a schematic structural diagram of a face recognition system provided by the present invention, and as shown in fig. 4, the present invention provides a face recognition system, which includes a spectral image obtaining module 401, a feature vector extracting module 402, and a recognition module 403, where the spectral image obtaining module 401 is configured to obtain a spectral image of a face to be recognized; the feature vector extraction module 402 is configured to input the facial spectral image to be recognized into a trained facial spectral image feature extraction model, and obtain a target feature vector of the facial spectral image, where the trained facial spectral image feature extraction model is obtained by training a machine learning model using a sample facial spectral image labeled with an identity information tag to which a face belongs and a true-false type spectral information tag; the recognition module 403 is configured to recognize the target feature vector according to the similarity evaluation criterion, and obtain a face recognition result and a living body recognition result.
Optionally, the lens group, the light modulation layer, and the image sensor layer in the face recognition spectral imaging chip may be regarded as the module 401 for acquiring a spectral image, and are used to acquire a face spectral image of a face to be recognized through the lens group, the light modulation layer, and the image sensor layer, where the face spectral image includes face image spatial information and face spectral information; the feature vector extraction module 402 and the recognition module 403 may be disposed in the signal processing circuit layer, and configured to obtain a target feature vector of the facial spectrum image, match the target feature vector with a feature vector of a facial image database according to a similarity evaluation criterion, obtain a feature vector in the facial image database that meets the highest similarity as a candidate recognition result, and obtain a facial recognition result and a living body recognition result if the similarity meets a set threshold.
The invention provides a face recognition system, which is characterized in that a face spectral image is acquired and input into a face spectral image feature extraction model to obtain a target feature vector of face spatial characteristics and spectral reflection characteristics; compared with the traditional face recognition method, the method has the advantages that the target characteristic vectors and the characteristic vectors in the database are compared and recognized by utilizing the similarity evaluation standard, and the face recognition and the living body recognition are simultaneously realized by utilizing the skin spectral characteristics, so that the security loophole of the traditional face detection is made up, the accuracy of the face recognition result is improved, and the security of a face recognition system is improved.
On the basis of the embodiment, the system further comprises a sample training set constructing module and a training feature extraction model module, wherein the sample training set constructing module is used for constructing a sample training set according to a sample face spectral image marked with an identity information label to which the face belongs and an authenticity type spectral information label, and the authenticity type spectral information label is a spectral information label of a living body face, a simulated face and a non-face type; and the training feature extraction model module is used for inputting the sample training set into the machine learning model for training to obtain a trained human face image feature extraction model, and the machine learning model is a convolutional neural network.
On the basis of the above embodiment, the training feature extraction model module further includes a training feature extraction model unit, and the training feature extraction model unit is configured to train a convolution kernel in the convolutional neural network through the sample training set based on a deep learning algorithm, and obtain a trained face image feature extraction model if a preset training condition is met, where the convolution kernel is configured to detect a face corner profile and a skin spectrum characteristic.
The system provided by the present invention is used for executing the above method embodiments, and for the specific processes and details, reference is made to the above embodiments, which are not described herein again.
Fig. 5 is a schematic structural diagram of an electronic device provided in the present invention, and as shown in fig. 5, the electronic device may include: a processor (processor) 501, a communication Interface (Communications Interface) 502, a memory (memory) 503, and a communication bus 504, wherein the processor 501, the communication Interface 502, and the memory 503 are configured to communicate with each other via the communication bus 504. The processor 501 may call logic instructions in the memory 503 to perform a face recognition method comprising: acquiring a human face spectral image to be recognized; inputting the facial spectral image to be recognized into a trained facial spectral image feature extraction model to obtain a target feature vector of the facial spectral image, wherein the trained facial spectral image feature extraction model is obtained by training a machine learning model by using a sample facial spectral image marked with an identity information label and a true and false type spectral information label to which a face belongs; and identifying the target characteristic vector according to a similarity evaluation standard to obtain a face identification result and a living body identification result.
In addition, the logic instructions in the memory 503 may be implemented in the form of software functional units and stored in a computer readable storage medium when the logic instructions are sold or used as independent products. Based on such understanding, the technical solution of the present invention or a part thereof which substantially contributes to the prior art may be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
In another aspect, the present invention also provides a computer program product, the computer program product including a computer program stored on a non-transitory computer-readable storage medium, the computer program including program instructions, when the program instructions are executed by a computer, the computer being capable of executing the face recognition method provided by the above methods, the method including: acquiring a human face spectral image to be recognized; inputting the facial spectral image to be recognized into a trained facial spectral image feature extraction model to obtain a target feature vector of the facial spectral image, wherein the trained facial spectral image feature extraction model is obtained by training a machine learning model by using a sample facial spectral image marked with an identity information label and a true and false type spectral information label to which a face belongs; and identifying the target characteristic vector according to a similarity evaluation standard to obtain a face identification result and a living body identification result.
In still another aspect, the present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored, the computer program being implemented by a processor to perform the face recognition methods provided above, the method including: acquiring a human face spectral image to be recognized; inputting the facial spectral image to be recognized into a trained facial spectral image feature extraction model to obtain a target feature vector of the facial spectral image, wherein the trained facial spectral image feature extraction model is obtained by training a machine learning model by using a sample facial spectral image marked with an identity information label and a true and false type spectral information label to which a face belongs; and identifying the target characteristic vector according to a similarity evaluation standard to obtain a face identification result and a living body identification result.
The above-described embodiments of the apparatus are merely illustrative, and the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.
Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware. Based on the understanding, the above technical solutions substantially or otherwise contributing to the prior art may be embodied in the form of a software product, which may be stored in a computer-readable storage medium, such as ROM/RAM, magnetic disk, optical disk, etc., and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method according to the various embodiments or some parts of the embodiments.
Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.

Claims (10)

1. A face recognition method, comprising:
acquiring a human face spectral image to be recognized;
inputting the facial spectral image to be recognized into a trained facial spectral image feature extraction model to obtain a target feature vector of the facial spectral image, wherein the trained facial spectral image feature extraction model is obtained by training a machine learning model by using a sample facial spectral image marked with an identity information label and a true and false type spectral information label to which a face belongs;
and identifying the target characteristic vector according to a similarity evaluation standard to obtain a face identification result and a living body identification result.
2. The face recognition method according to claim 1, wherein the acquiring of the spectral image of the face to be recognized comprises:
acquiring a human face spectral image to be recognized through a human face recognition spectral imaging chip;
face identification spectral imaging chip includes light modulation layer, image sensor layer and signal processing circuit layer, the light modulation layer the image sensor layer with signal processing circuit layer is from last to connecting gradually down along the vertical direction, wherein:
the light modulation layer is used for receiving the light signal reflected by the face to be recognized and carrying out light modulation;
the image sensor layer is used for converting an optical signal reflected by the modulated human face to be recognized into an electric signal, and the electric signal comprises human face image space information and modulated skin spectrum information;
and the signal processing circuit layer is used for processing the human face image space information and the light modulated skin spectrum information output by the image sensor layer to obtain a human face recognition result.
3. The face recognition method according to claim 2, wherein the face spectral imaging chip further comprises a lens set, wherein the lens set is located on the upper surface of the light modulation layer, is connected to the light modulation layer, and is configured to focus and image the optical signal reflected by the face to obtain the optical signal reflected by the face to be recognized.
4. The face recognition method according to claim 2, wherein the light modulation layer comprises at least one light modulation unit, the light modulation unit comprises a plurality of micro-nano structure arrays, through holes are uniformly distributed in each micro-nano structure array according to different preset arrangement rules, and the through holes of each micro-nano structure array are different in shape.
5. The face recognition method according to claim 2, wherein a plurality of photosensitive pixel units are distributed on the upper surface of the image sensor layer, and each micro-nano structure array corresponds to at least one photosensitive pixel unit.
6. The face recognition method according to claim 1, wherein the trained face spectral image feature extraction model is obtained by training through the following steps:
constructing a sample training set according to a sample face spectral image marked with an identity information label to which a face belongs and a true and false type spectral information label, wherein the true and false type spectral information label is a spectral information label of a living body face, a simulated face and a non-face type;
and inputting the sample training set into the machine learning model for training to obtain a trained face image feature extraction model, wherein the machine learning model is a convolutional neural network.
7. The method of claim 6, wherein the inputting the sample training set into the machine learning model for training to obtain a trained facial spectral image feature extraction model comprises:
and training a convolution kernel in the convolutional neural network through the sample training set based on a deep learning algorithm, and if a preset training condition is met, obtaining a trained face spectral image feature extraction model, wherein the convolution kernel is used for detecting the face corner point outline and the skin spectral characteristic.
8. A face recognition system, comprising:
the acquisition spectral image module acquires a spectral image of a human face to be recognized;
the extraction feature vector module is used for inputting the facial spectral image to be recognized into a trained facial spectral image feature extraction model to obtain a target feature vector of the facial spectral image, wherein the trained facial spectral image feature extraction model is obtained by training a machine learning model by using a sample facial spectral image marked with an identity information label and an authenticity type spectral information label to which a face belongs;
and the recognition module is used for recognizing the target characteristic vector according to the similarity evaluation standard and acquiring the results of face recognition and living body recognition.
9. The face recognition system of claim 8, wherein the system further comprises:
the system comprises a sample training set establishing module, a sample training set establishing module and a sample processing module, wherein the sample training set establishing module is used for establishing a sample training set according to a sample face spectral image marked with an identity information label to which a face belongs and a truth type spectral information label, and the truth type spectral information label is a spectral information label of a living body face, a simulated face and a non-face type;
and the training feature extraction model module is used for inputting the sample training set into the machine learning model for training to obtain a trained face image feature extraction model, and the machine learning model is a convolutional neural network.
10. A non-transitory computer-readable storage medium, on which a computer program is stored, wherein the computer program, when being executed by a processor, implements the steps of the face recognition method according to any one of claims 1 to 7.
CN202110813929.6A 2021-07-19 2021-07-19 Face recognition method and system Pending CN115641649A (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN202110813929.6A CN115641649A (en) 2021-07-19 2021-07-19 Face recognition method and system
PCT/CN2022/098308 WO2023000864A1 (en) 2021-07-19 2022-06-13 Face recognition method and system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110813929.6A CN115641649A (en) 2021-07-19 2021-07-19 Face recognition method and system

Publications (1)

Publication Number Publication Date
CN115641649A true CN115641649A (en) 2023-01-24

Family

ID=84940099

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110813929.6A Pending CN115641649A (en) 2021-07-19 2021-07-19 Face recognition method and system

Country Status (2)

Country Link
CN (1) CN115641649A (en)
WO (1) WO2023000864A1 (en)

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116311553B (en) * 2023-05-17 2023-08-15 武汉利楚商务服务有限公司 Human face living body detection method and device applied to semi-occlusion image
CN116797889B (en) * 2023-08-24 2023-12-08 青岛美迪康数字工程有限公司 Updating method and device of medical image recognition model and computer equipment
CN117273747B (en) * 2023-09-28 2024-04-19 广州佳新智能科技有限公司 Payment method, device, storage medium and equipment based on face image recognition
CN117688538A (en) * 2023-12-13 2024-03-12 上海深感数字科技有限公司 Interactive education management method and system based on digital identity security precaution
CN117854138B (en) * 2024-03-07 2024-05-10 深圳航天信息有限公司 Information acquisition and analysis method, device, equipment and storage medium based on big data

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP3369036A4 (en) * 2015-10-30 2019-05-22 Microsoft Technology Licensing, LLC Spoofed face detection
CN105956515B (en) * 2016-04-20 2019-04-23 西安电子科技大学 Solid-EO-1 hyperion face identification method based on pole imaging
CN107808115A (en) * 2017-09-27 2018-03-16 联想(北京)有限公司 A kind of biopsy method, device and storage medium
CN110502989A (en) * 2019-07-16 2019-11-26 山东师范大学 A kind of small sample EO-1 hyperion face identification method and system
CN112330331A (en) * 2020-11-19 2021-02-05 平安普惠企业管理有限公司 Identity verification method, device and equipment based on face recognition and storage medium

Also Published As

Publication number Publication date
WO2023000864A1 (en) 2023-01-26

Similar Documents

Publication Publication Date Title
CN115641649A (en) Face recognition method and system
KR100852629B1 (en) Iris recognizing system and method using multifocus image sequence
CN101030244B (en) Automatic identity discriminating method based on human-body physiological image sequencing estimating characteristic
CN101251889B (en) Personal identification method and near-infrared image forming apparatus based on palm vena and palm print
US20110200237A1 (en) Pattern matching device and pattern matching method
KR20090087895A (en) Method and apparatus for extraction and matching of biometric detail
CN102844766A (en) Human eyes images based multi-feature fusion identification method
JP2009544108A (en) Multispectral image for multiple biometric authentication
CN112232155B (en) Non-contact fingerprint identification method and device, terminal and storage medium
de Souza et al. On the learning of deep local features for robust face spoofing detection
CN112016525A (en) Non-contact fingerprint acquisition method and device
Seal et al. Minutiae based thermal face recognition using blood perfusion data
Maček et al. Multimodal biometric authentication in IoT: Single camera case study
Wasnik et al. Presentation attack detection for smartphone based fingerphoto recognition using second order local structures
Song et al. Face liveness detection based on joint analysis of rgb and near-infrared image of faces
Proença Unconstrained iris recognition in visible wavelengths
Dronky et al. A review on iris liveness detection techniques
Gomez-Barrero et al. Multi-spectral convolutional neural networks for biometric presentation attack detection
Seal et al. Minutiae based thermal human face recognition using label connected component algorithm
Mohammed et al. Conceptual analysis of Iris Recognition Systems
Khandizod et al. Analysis and Feature Extraction using Wavelet based Image Fusion for Multispectral Palmprint Recognition
Li et al. Iris recognition on mobile devices using near-infrared images
CN112232152A (en) Non-contact fingerprint identification method and device, terminal and storage medium
SulaimanAlshebli et al. The Cyber Security Biometric Authentication based on Liveness Face-Iris Images and Deep Learning Classifier
Singh et al. Design and Implementation of Gabor Filter and SVM based authentication system using machine learning

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination