CN112052703A - Face recognition method and system based on deep neural network - Google Patents
Face recognition method and system based on deep neural network Download PDFInfo
- Publication number
- CN112052703A CN112052703A CN201910484391.1A CN201910484391A CN112052703A CN 112052703 A CN112052703 A CN 112052703A CN 201910484391 A CN201910484391 A CN 201910484391A CN 112052703 A CN112052703 A CN 112052703A
- Authority
- CN
- China
- Prior art keywords
- image
- face recognition
- images
- infrared
- rgb
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000013528 artificial neural network Methods 0.000 title claims abstract description 18
- 238000000034 method Methods 0.000 title claims abstract description 18
- 238000007781 pre-processing Methods 0.000 claims abstract description 15
- 238000012549 training Methods 0.000 claims description 29
- 238000012545 processing Methods 0.000 claims description 25
- 238000003062 neural network model Methods 0.000 claims description 12
- 230000009466 transformation Effects 0.000 claims description 12
- 230000004927 fusion Effects 0.000 claims description 7
- 239000013598 vector Substances 0.000 claims description 6
- 230000008859 change Effects 0.000 claims description 5
- 238000005286 illumination Methods 0.000 claims description 4
- PXFBZOLANLWPMH-UHFFFAOYSA-N 16-Epiaffinine Natural products C1C(C2=CC=CC=C2N2)=C2C(=O)CC2C(=CC)CN(C)C1C2CO PXFBZOLANLWPMH-UHFFFAOYSA-N 0.000 claims description 3
- 230000004913 activation Effects 0.000 claims description 3
- 238000012937 correction Methods 0.000 claims description 3
- 238000007499 fusion processing Methods 0.000 claims description 3
- 238000011176 pooling Methods 0.000 claims description 3
- 238000005070 sampling Methods 0.000 claims description 3
- 239000011521 glass Substances 0.000 abstract description 6
- 238000005516 engineering process Methods 0.000 description 2
- 230000007613 environmental effect Effects 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000012790 confirmation Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 230000008520 organization Effects 0.000 description 1
- 230000000149 penetrating effect Effects 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/048—Activation functions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/14—Transformations for image registration, e.g. adjusting or mapping for alignment of images
- G06T3/147—Transformations for image registration, e.g. adjusting or mapping for alignment of images using affine transformations
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/40—Image enhancement or restoration using histogram techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Mathematical Physics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Biophysics (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Artificial Intelligence (AREA)
- Computational Linguistics (AREA)
- Software Systems (AREA)
- Life Sciences & Earth Sciences (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Human Computer Interaction (AREA)
- Multimedia (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The invention provides a face recognition method and a face recognition system based on a deep neural network, which relate to the field of face recognition, and the method comprises the following steps of 1: collecting RGB images and infrared images of a human face, and preprocessing the RGB images and the infrared images; step 2: and inputting the preprocessed RGB images and infrared images into a pre-trained face recognition model for face recognition to obtain a face recognition result. According to the invention, the infrared image and the visible light image are fused, the advantages of the infrared image and the visible light image are combined, the influence on the visible light image acquisition under the conditions of dark light, shielding and the like is prevented, the problems that the infrared image acquisition is easily influenced by the environment temperature, the glasses and other glass products are difficult to penetrate to form black shadow and the image acquisition quality is influenced are solved, and the accuracy of the face recognition under the special conditions of dark light, shielding and the like is improved.
Description
Technical Field
The invention relates to the field of face recognition, in particular to a face recognition method and system based on a deep neural network.
Background
The face recognition technology has very wide application prospects in the fields of computer vision, customer identity identification, multimedia data retrieval and the like, can reasonably correspond according to the expression or the expression of a user along with the continuous maturity and perfection of the face recognition technology, can distinguish according to the posture, the expression, accessories and the like, but the face characteristics can be greatly changed along with the change of visible light rays, the face recognition can be seriously influenced under the conditions of outdoor, dim light and non-uniform illumination, in addition, criminals who need to be caught when breaking a case in a public security organization often adopt the ways of disguising and shielding to avoid the collection of camera images, and great difficulty is brought to face recognition and identity confirmation.
Disclosure of Invention
In view of the above disadvantages of the prior art, an object of the present invention is to provide a method and a system for face recognition based on a deep neural network, which fuse an infrared image and a visible light image and solve the problem of low accuracy in the case of dim light and occlusion in the existing face recognition.
The invention provides a face recognition method based on a deep neural network, which comprises the following steps:
step 1: collecting RGB images and infrared images of a human face, and preprocessing the RGB images and the infrared images;
step 2: and inputting the preprocessed RGB images and infrared images into a pre-trained face recognition model for face recognition to obtain a face recognition result.
Further, the face recognition model is obtained by pre-training through the following steps:
step 2.1: collecting RGB images and infrared images containing human faces, preprocessing the RGB images and the infrared images to obtain training samples, wherein the image of each training sample is provided with an identity label;
step 2.2: and inputting the image of the training sample into a neural network model, and training to obtain a face recognition model.
Further, the specific steps of inputting the images of the training samples into the neural network model for training are as follows:
step 2.2.1: inputting the images of the training samples into the convolution layer of the neural network model for linear transformation;
step 2.2.2: carrying out nonlinear transformation on the image after each convolution processing by using a ReLU activation function;
step 2.2.3: using a Max Pooling layer to carry out down-sampling on the image after each nonlinear change;
step 2.2.4: and inputting the downsampled image into a full connection layer, and calculating network Loss through a Softmax Loss layer to further obtain the face recognition model.
Further, the steps of preprocessing the RGB image and the infrared image are as follows:
step 1.1: carrying out gray level image processing on the RGB image, and carrying out correction processing on the infrared image;
step 1.2: performing WF (pass-function) illumination removal processing on the RGB image after the gray level image processing, and performing histogram equalization processing on the corrected infrared image;
step 1.3: extracting features to obtain feature vectors of feature points of the RGB image and the infrared image;
step 1.4: carrying out feature matching on the homonymous points of the feature vectors of the RGB image and the infrared image;
step 1.5: and connecting the points with the same name to construct a transformation model, obtaining a registration image through affine transformation, and performing image fusion based on the registration image.
A face recognition system based on deep neural network, the device comprises
An image acquisition module: acquiring images of a human face to obtain RGB images and infrared images;
an image preprocessing module: carrying out image processing on the collected RGB image and the infrared image;
an image registration module: registering the RGB image and the infrared image after image processing;
an image fusion module: fusing the registered RGB image and the infrared image;
a face recognition module: and carrying out face recognition on the image subjected to the fusion processing to obtain a face recognition result.
Further, the face recognition module is obtained by pre-training based on a neural network model.
As described above, the face recognition method and system based on the deep neural network of the present invention have the following beneficial effects: according to the invention, the infrared image and the visible light image are fused, and the advantages of the infrared image and the visible light image are combined, so that the influence on the acquisition of the visible light image under the conditions of dark light, shielding and the like is prevented, the problems that the acquisition of the infrared image is easily influenced by the environmental temperature, the black shadow is difficultly formed by penetrating glass products such as glasses and the like, and the image acquisition quality is influenced are solved, and the accuracy of face recognition under the special conditions of dark light, shielding and the like is improved.
Drawings
FIG. 1 is a flowchart of a face recognition method based on a deep neural network disclosed in an embodiment of the present invention;
FIG. 2 is a flow chart illustrating the pre-processing of RGB images and IR images disclosed in an embodiment of the present invention;
FIG. 3 is a flowchart illustrating a pre-training process of a face recognition model disclosed in an embodiment of the present invention;
fig. 4 is a diagram illustrating a training procedure of the face recognition model disclosed in the embodiment of the present invention.
Detailed Description
The embodiments of the present invention are described below with reference to specific embodiments, and other advantages and effects of the present invention will be easily understood by those skilled in the art from the disclosure of the present specification. The invention is capable of other and different embodiments and of being practiced or of being carried out in various ways, and its several details are capable of modification in various respects, all without departing from the spirit and scope of the present invention. It is to be noted that the features in the following embodiments and examples may be combined with each other without conflict.
It should be noted that the drawings provided in the following embodiments are only for illustrating the basic idea of the present invention, and the components related to the present invention are only shown in the drawings rather than drawn according to the number, shape and size of the components in actual implementation, and the type, quantity and proportion of the components in actual implementation may be changed freely, and the layout of the components may be more complicated.
As shown in fig. 1, the present invention provides a face recognition method based on a deep neural network, the method includes the following steps:
step 1: collecting RGB images and infrared images of a human face, and preprocessing the RGB images and the infrared images;
the infrared camera and the camera of the common camera are close to each other in distance, the RGB images and the infrared images of the face are collected simultaneously, the collected RGB images and the collected infrared images are paired, and fusion and dislocation of the RGB images and the infrared images are prevented during preprocessing.
Specifically, as shown in fig. 2, the steps of preprocessing the RGB image and the infrared image are as follows:
step 1.1: carrying out gray level image processing on the RGB image, and carrying out correction processing on the infrared image;
step 1.2: performing WF (pass-function) illumination removal processing on the RGB image after the gray level image processing, and performing histogram equalization processing on the corrected infrared image;
the WF refers to a Weber face light invariant feature extraction method, and can effectively remove complex light influence; the histogram equalization processing can improve the quality of the infrared image and highlight the characteristics of the image.
Step 1.3: extracting features to obtain feature vectors of feature points of the RGB image and the infrared image;
step 1.4: carrying out feature matching on the homonymous points of the feature vectors of the RGB image and the infrared image;
step 1.5: connecting the same-name points to construct a transformation model, obtaining a registration image through affine transformation, and performing image fusion based on the registration image;
even if the infrared camera and the common camera are close to each other, the shot images still have a certain parallax, and before the two images are fused, the two images need to be registered and converted into the same visual angle.
Step 2: and inputting the preprocessed RGB images and infrared images into a pre-trained face recognition model for face recognition to obtain a face recognition result.
As shown in fig. 3 and 4, specifically, the face recognition model is obtained by pre-training through the following steps:
step 2.1: collecting RGB images and infrared images containing human faces, preprocessing the RGB images and the infrared images to obtain training samples, wherein the image of each training sample is provided with an identity label;
step 2.2: and inputting the image of the training sample into a neural network model, and training to obtain a face recognition model.
Specifically, the specific steps of inputting the images of the training samples into the neural network model for training are as follows:
step 2.2.1: inputting the images of the training samples into the convolution layer of the neural network model for linear transformation;
step 2.2.2: carrying out nonlinear transformation on the image after each convolution processing by using a ReLU activation function;
step 2.2.3: using a Max Pooling layer to carry out down-sampling on the image after each nonlinear change;
step 2.2.4: and inputting the downsampled image into a full connection layer, and calculating network Loss through a Softmax Loss layer to further obtain the face recognition model.
Based on the face recognition method based on the deep neural network, the invention provides a face recognition system based on the deep neural network, and the system comprises
An image acquisition module: acquiring images of a human face to obtain RGB images and infrared images;
an image preprocessing module: carrying out image processing on the collected RGB image and the infrared image;
an image registration module: registering the RGB image and the infrared image after image processing;
an image fusion module: fusing the registered RGB image and the infrared image;
a face recognition model: and carrying out face recognition on the image subjected to the fusion processing to obtain a face recognition result.
Further, the face recognition model is obtained by pre-training based on a neural network model.
In summary, the infrared image and the visible light image are fused, and the advantages of the infrared image and the visible light (RGB) image are combined, so that the influence on the visible light image acquisition under the conditions of dark light, shielding and the like is prevented, and the problems that the infrared image acquisition is easily influenced by the environmental temperature, the black shadow is difficult to penetrate glass products such as glasses and the like to influence the image acquisition quality are solved, and the accuracy of the face recognition under the special conditions of dark light, shielding and the like is improved. Therefore, the invention effectively overcomes various defects in the prior art and has high industrial utilization value.
The foregoing embodiments are merely illustrative of the principles and utilities of the present invention and are not intended to limit the invention. Any person skilled in the art can modify or change the above-mentioned embodiments without departing from the spirit and scope of the present invention. Accordingly, it is intended that all equivalent modifications or changes which can be made by those skilled in the art without departing from the spirit and technical spirit of the present invention be covered by the claims of the present invention.
Claims (6)
1. A face recognition method based on a deep neural network is characterized by comprising the following steps:
step 1: collecting RGB images and infrared images of a human face, and preprocessing the RGB images and the infrared images;
step 2: and inputting the preprocessed RGB images and infrared images into a pre-trained face recognition model for face recognition to obtain a face recognition result.
2. The method for recognizing the face based on the deep neural network as claimed in claim 1, wherein the face recognition model is obtained by pre-training through the following steps:
step 2.1: collecting RGB images and infrared images containing human faces, preprocessing the RGB images and the infrared images to obtain training samples, wherein the image of each training sample is provided with an identity label;
step 2.2: and inputting the image of the training sample into a neural network model, and training to obtain a face recognition model.
3. The method for recognizing the face based on the deep neural network as claimed in claim 1, wherein the specific steps of inputting the images of the training samples into the neural network model for training are as follows:
step 2.2.1: inputting the images of the training samples into the convolution layer of the neural network model for linear transformation;
step 2.2.2: carrying out nonlinear transformation on the image after each convolution processing by using a ReLU activation function;
step 2.2.3: using a Max Pooling layer to carry out down-sampling on the image after each nonlinear change;
step 2.2.4: and inputting the downsampled image into a full connection layer, and calculating network Loss through a Softmax Loss layer to further obtain the face recognition model.
4. The face recognition method based on the deep neural network as claimed in claim 2, wherein: the steps of preprocessing the RGB image and the infrared image are as follows:
step 1.1: carrying out gray level image processing on the RGB image, and carrying out correction processing on the infrared image;
step 1.2: performing WF (pass-function) illumination removal processing on the RGB image after the gray level image processing, and performing histogram equalization processing on the corrected infrared image;
step 1.3: extracting features to obtain feature vectors of feature points of the RGB image and the infrared image;
step 1.4: carrying out feature matching on the homonymous points of the feature vectors of the RGB image and the infrared image;
step 1.5: and connecting the points with the same name to construct a transformation model, obtaining a registration image through affine transformation, and performing image fusion based on the registration image.
5. A face recognition system based on a deep neural network is characterized in that the device comprises
An image acquisition module: acquiring images of a human face to obtain RGB images and infrared images;
an image preprocessing module: carrying out image processing on the collected RGB image and the infrared image;
an image registration module: registering the RGB image and the infrared image after image processing;
an image fusion module: fusing the registered RGB image and the infrared image;
a face recognition module: and carrying out face recognition on the image subjected to the fusion processing to obtain a face recognition result.
6. The deep neural network-based face recognition system of claim 5, wherein: the face recognition module is obtained by pre-training based on a neural network model.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910484391.1A CN112052703A (en) | 2019-06-05 | 2019-06-05 | Face recognition method and system based on deep neural network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910484391.1A CN112052703A (en) | 2019-06-05 | 2019-06-05 | Face recognition method and system based on deep neural network |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112052703A true CN112052703A (en) | 2020-12-08 |
Family
ID=73608531
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910484391.1A Pending CN112052703A (en) | 2019-06-05 | 2019-06-05 | Face recognition method and system based on deep neural network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112052703A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11706546B2 (en) * | 2021-06-01 | 2023-07-18 | Sony Semiconductor Solutions Corporation | Image sensor with integrated single object class detection deep neural network (DNN) |
| CN116895094A (en) * | 2023-09-11 | 2023-10-17 | 杭州魔点科技有限公司 | Dark environment imaging method, system, device and medium based on binocular fusion |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101303724A (en) * | 2007-05-10 | 2008-11-12 | 中国银联股份有限公司 | Authentication authorization method and system |
| CN106960202A (en) * | 2017-04-11 | 2017-07-18 | 广西师范大学 | A kind of smiling face's recognition methods merged based on visible ray with infrared image |
| CN108090477A (en) * | 2018-01-23 | 2018-05-29 | 北京易智能科技有限公司 | A kind of face identification method and device based on Multi-spectral image fusion |
| CN108921100A (en) * | 2018-07-04 | 2018-11-30 | 武汉高德智感科技有限公司 | A kind of face identification method merged based on visible images with infrared image and system |
| CN109117725A (en) * | 2018-07-09 | 2019-01-01 | 深圳市科脉技术股份有限公司 | Face identification method and device |
-
2019
- 2019-06-05 CN CN201910484391.1A patent/CN112052703A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101303724A (en) * | 2007-05-10 | 2008-11-12 | 中国银联股份有限公司 | Authentication authorization method and system |
| CN106960202A (en) * | 2017-04-11 | 2017-07-18 | 广西师范大学 | A kind of smiling face's recognition methods merged based on visible ray with infrared image |
| CN108090477A (en) * | 2018-01-23 | 2018-05-29 | 北京易智能科技有限公司 | A kind of face identification method and device based on Multi-spectral image fusion |
| CN108921100A (en) * | 2018-07-04 | 2018-11-30 | 武汉高德智感科技有限公司 | A kind of face identification method merged based on visible images with infrared image and system |
| CN109117725A (en) * | 2018-07-09 | 2019-01-01 | 深圳市科脉技术股份有限公司 | Face identification method and device |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11706546B2 (en) * | 2021-06-01 | 2023-07-18 | Sony Semiconductor Solutions Corporation | Image sensor with integrated single object class detection deep neural network (DNN) |
| CN116895094A (en) * | 2023-09-11 | 2023-10-17 | 杭州魔点科技有限公司 | Dark environment imaging method, system, device and medium based on binocular fusion |
| CN116895094B (en) * | 2023-09-11 | 2024-01-30 | 杭州魔点科技有限公司 | Dark environment imaging method, system, device and medium based on binocular fusion |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108537136B (en) | Pedestrian re-identification method based on attitude normalization image generation | |
| CN109344701B (en) | Kinect-based dynamic gesture recognition method | |
| CN106778664B (en) | Iris image iris area segmentation method and device | |
| CN106250877B (en) | Near-infrared face identification method and device | |
| CN109409272B (en) | Cable acceptance system and method based on machine vision | |
| CN111539255B (en) | Cross-modal pedestrian re-identification method based on multi-modal image style conversion | |
| CN108090477A (en) | A kind of face identification method and device based on Multi-spectral image fusion | |
| Konwar et al. | An American sign language detection system using HSV color model and edge detection | |
| EP3499414A1 (en) | Lightweight 3d vision camera with intelligent segmentation engine for machine vision and auto identification | |
| CN111144366A (en) | Strange face clustering method based on joint face quality assessment | |
| CN110263768A (en) | A kind of face identification method based on depth residual error network | |
| CN111445459A (en) | Image defect detection method and system based on depth twin network | |
| CN112052703A (en) | Face recognition method and system based on deep neural network | |
| CN111597978B (en) | Method for automatically generating pedestrian re-identification picture based on StarGAN network model | |
| CN112052730B (en) | 3D dynamic portrait identification monitoring equipment and method | |
| CN111460884A (en) | Multi-face recognition method based on human body tracking | |
| CN112149690A (en) | Tracing method and tracing system based on biological image feature recognition | |
| CN111582118A (en) | Face recognition method and device | |
| CN116052222A (en) | Cattle face recognition method for naturally collecting cattle face image | |
| CN111582154A (en) | Pedestrian re-identification method based on multitask skeleton posture division component | |
| CN111402331B (en) | Robot repositioning method based on visual word bag and laser matching | |
| CN112633222B (en) | Gait recognition method, device, equipment and medium based on countermeasure network | |
| CN116403004B (en) | Cow face fusion feature extraction method based on cow face correction | |
| CN113326778A (en) | Human body posture detection method and device based on image recognition and storage medium | |
| Grigoryan et al. | Color facial image representation with new quaternion gradients |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20201208 |