CN114627562A - Multispectral face recognition module and method - Google Patents
Multispectral face recognition module and method Download PDFInfo
- Publication number
- CN114627562A CN114627562A CN202210175114.4A CN202210175114A CN114627562A CN 114627562 A CN114627562 A CN 114627562A CN 202210175114 A CN202210175114 A CN 202210175114A CN 114627562 A CN114627562 A CN 114627562A
- Authority
- CN
- China
- Prior art keywords
- image
- camera
- rgb
- main controller
- living body
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Abstract
The invention discloses a multispectral face recognition module and a multispectral face recognition method, wherein the multispectral face recognition module comprises a camera assembly and a control assembly, the camera assembly comprises an IR camera, a light supplement lamp and an RGB camera, the control assembly comprises a main controller, a power management unit and a storage unit, the IR camera and the RGB camera are connected with the main controller, and IR images and RGB images of a shot face are processed by an image processing unit and then are sent to the main controller; the main controller performs infrared image living body detection and color image living body detection on the IR image and the RGB image, compares the IR image and the RGB image with the database for identification if the IR image and the RGB image both accord with living body standards, and outputs an identification result. According to the invention, the IR camera and the RGB camera structure are utilized, the living body detection and the face recognition are carried out by the main controller after the face is shot to obtain the IR image and the RGB image, and the recognition accuracy is improved.
Description
Technical Field
The invention relates to a multispectral face recognition module and a multispectral face recognition method, and belongs to the technical field of face recognition.
Background
At present, the face recognition technology is generally applied in various industries. However, in practical application, the existing face recognition technology generally has the problems of poor environmental adaptability, non-ideal recognition effect and the like.
Disclosure of Invention
The invention provides a multispectral face recognition module and a multispectral face recognition method in order to overcome the defects in the prior art.
The invention can be realized by adopting the following technical scheme:
a multispectral face recognition module comprises a camera assembly and a control assembly, wherein the camera assembly comprises an IR camera, a light supplement lamp and an RGB camera, the control assembly comprises a main controller, a power management unit and a storage unit,
the IR camera and the RGB camera are connected with the main controller, and the IR image and the RGB image of the shot human face are processed by the image processing unit and then sent to the main controller;
the light supplement lamp comprises a white light LED lamp bead and an infrared LED lamp bead which are arranged up and down and packaged together, the light supplement lamp is integrated with the IR camera and the RGB camera, and light supplement is carried out on the IR camera and the RGB camera according to ambient light conditions;
the main controller performs infrared image living body detection and color image living body detection on the IR image and the RGB image, compares the IR image and the RGB image with the database for identification if the IR image and the RGB image both accord with living body standards, and outputs an identification result;
the power management unit is connected with the main controller and provides power for the main controller;
the storage unit is connected with the main controller and used for storing data and programs.
Preferably, the IR camera is connected to the main controller through a DVP interface, and the RGB camera is connected to the main controller through an MIPI interface.
Preferably, the control assembly further comprises an encryption IC, the encryption IC being connected to the main controller.
Preferably, the control assembly further includes a voice interface, a USB interface, a UART interface, an LCD interface, and a TP interface connected to the main controller.
A multispectral face recognition method comprises the following steps:
(1) the method comprises the steps that an IR camera and an RGB camera shoot a human face, the IR camera and the RGB camera respectively perform photometry and detect ambient light conditions, intelligent dynamic exposure adjustment is respectively performed according to respective measured data, and an IR image and an RGB image are obtained through shooting and are sent to a main controller after being processed by an image processing unit;
(2) respectively carrying out living body detection on the IR image living body detection unit and the RGB image living body detection unit of the main controller, and if the detection results of the IR image living body detection unit and the RGB image living body detection unit are both living bodies, carrying out the next step; if any one of the detection results of the IR image living body detection unit and the RGB image living body detection unit is a non-living body, outputting a detection non-conforming result;
(3) the IR image recognition unit and the RGB image recognition unit of the main controller perform face recognition on the IR image and the RGB image according to the face information in the database, and if the face information in the database is not matched, the unmatched face information is output; and if the face information is matched with certain face information in the database, outputting corresponding matched face information.
Compared with the prior art, the invention has the beneficial effects that: the invention utilizes the structure of the IR camera and the RGB camera, and the IR camera and the RGB camera shoot the human face to obtain the IR image and the RGB image, and then the main controller carries out the living body detection and the human face recognition, thereby effectively improving the accuracy of the whole recognition, simultaneously, the IR camera and the RGB camera detection can adapt to different light environments, effectively improving the use efficiency of the recognition and expanding the use range.
Drawings
FIG. 1 is a block diagram of the multispectral face recognition module of the present invention;
FIG. 2 is a schematic structural view of a camera assembly of the present invention;
fig. 3 is a flow chart of the multispectral face recognition method of the present invention.
Detailed Description
The following detailed description of embodiments of the invention refers to the accompanying drawings.
Example 1
As shown in fig. 1 to 3, the multispectral face recognition module of the present embodiment includes a camera assembly 1 and a control assembly 2, the camera assembly includes an IR camera 13, an RGB camera 11 and a fill-in light 12, the control assembly 2 includes a main controller 21, a power management unit 27 and a storage unit 26,
the IR camera 13 and the RGB camera 11 are connected with the main controller 21, and the IR image and the RGB image of the shot human face are processed by the image processing unit and then sent to the main controller 21;
the light supplement lamp 12 is integrated with the IR camera 13 and the RGB camera 11 together and respectively connected with the main controller 21, and light supplement is carried out on the IR camera 13 and the RGB camera 11 according to ambient light conditions.
The RGB camera 11 and the IR camera 13 are arranged at two ends of the module support 14, the supplement light 12 is arranged in the middle of the module support 14, the supplement light 12 packages the infrared LED lamp beads 121 and the white LED lamp beads 122 together, and preferably, the up-down structure is adopted for packaging, wherein the white LED lamp beads 122 are preferably arranged above, so that the white light and the infrared light can be uniformly covered in the central position; the angles of the two LED lamp beads cover the window range of the camera, and preferably 120-degree large-angle lamp beads are adopted; the preferable light supplement lamp 12 is on the same horizontal plane with the IR camera 13 and the RGB camera 11; the camera assembly 1 is preferably connected to the core board by FPC lines.
The main controller 21 performs infrared image living body detection and color image living body detection on the IR image and the RGB image, compares the IR image and the RGB image with the database for identification if the IR image and the RGB image both conform to living body standards, and outputs an identification result;
the power management unit 27 is connected with the main controller 21 and provides power for the main controller 21;
the storage unit 26 is connected to the main controller 21 for storing data and programs.
In the present embodiment, it is preferred that,
the IR camera 13 with through DVP interface connection between the main control unit 21, the RGB camera 11 with through MIPI interface connection between the main control unit 21, make things convenient for main control unit 21 to be used for connecting IR + RGB camera and acquire image information.
The control component further comprises an encryption IC 28, and the encryption IC 28 is connected with the main controller 21 and used for protecting the face recognition module from being maliciously cracked.
The control assembly 2 further comprises a voice interface 29, a USB interface 22, a UART interface 23, an LCD interface 24 and a TP interface 25 connected to the main controller 21, wherein the LCD interface 24 is used for connecting a display screen and displaying image information and menu information acquired by the camera. The USB interface 22 is used for communicating with the cat eye module and implementing image transmission through the UVC protocol. The UART interface 23 is used for communicating with the lock control module and transmitting the information related to the face recognition result. The TP interface 25 is used for an external touch screen and can realize a man-machine interaction input function. The voice interface 29 is used for externally connecting a loudspeaker and outputting voice information.
The components are all common components, and the connection between the components adopts a common circuit connection mode.
Example 2
On the basis of the embodiment 1, as shown in fig. 3, the multispectral face recognition method of the present embodiment includes the following steps:
(1) the IR camera 13 and the RGB camera 11 shoot a face, the IR camera 13 and the RGB camera 11 respectively perform photometry and detect ambient light conditions, intelligent dynamic exposure adjustment is respectively performed according to the measured data, and an IR image and an RGB image are obtained by shooting and are sent to the main controller 21 after being processed by the image processing unit;
(2) respectively carrying out living body detection on the IR image and the RGB image by an IR image living body detection unit and an RGB image living body detection unit of the main controller 21, and carrying out the next step if the detection results of the IR image living body detection unit and the RGB image living body detection unit are both living bodies; if any one of the detection results of the IR image living body detection unit and the RGB image living body detection unit is a non-living body, outputting a detection non-conforming result;
(3) the IR image recognition unit and the RGB image recognition unit of the main controller 21 perform face recognition on the IR image and the RGB image according to the face information in the database, and if the face information in the database is not matched, the unmatched face information is output; and if the face information is matched with certain face information in the database, outputting corresponding matched face information.
In the above method, the specific operations are as follows:
1. after the module is powered on and started, the light supplement lamp 12 is turned off, an image is collected by the RGB camera 11, the brightness mean value X corresponding to the image is calculated, when the X is larger than a set threshold value, the ambient light is judged to be a strong light environment, parameters such as an exposure value of an IR sensor of the IR camera 13 are initialized to corresponding preset values through IIC, and parameters such as an exposure value of the RGB sensor of the RGB camera 11 are adjusted;
2. the IR sensor is initialized by adopting an exposure value of 1, a frame of RGB image Trgb1 is collected before initialization takes effect, the position of a human face is positioned, if the human face is positioned, an RGB anti-counterfeiting model is called to carry out RGB living body anti-counterfeiting judgment, if human face information is not positioned (at the moment, a dark environment is possible, or the human face is in a back light state, and the collected human face image is very black), the IR sensor is waited to reach the human face positioning information;
3. the IR sensor takes effect, when a frame synchronization (vsync) signal is received and an image starts to be collected, the infrared LED lamp is instantly started, the brightness is adjusted to 100%, the IR sensor collects a first frame of effective image Tir1, a preferred CMOS of the IR sensor adopts a global exposure device, and if a non-global exposure CMOS is adopted, the brightness of the LED lamp is kept consistent during the CMOS exposure period according to the collected image time sequence strictly.
4. The brightness of the infrared LED lamp beads is quickly adjusted to 30%, and the IR sensor is initialized by adopting an exposure value of 2;
5. during the initialization validation of the IR sensor, the face position information is positioned according to the IR image Tir1, if the face is not positioned, the loop of the step 3 is continued after the next frame of image is acquired. If the face information is positioned, the face position information positioned by the Tir1 is mapped to the Trgb1 image, the brightness of the face area is judged, parameters such as the exposure value of the RGB sensor are adjusted again, and independent exposure is effectively carried out on the face area.
Because traditional automatic exposure is directed at the adjustment of whole image, when there is stronger external light interference, because the proportion of people's face in whole image is less, consequently the sensor often will be with the background dim down, and people's face can follow darker this moment, leads to the people's face location failure. By the method, local effective imaging can be performed even by using a common IR CMOS sensor, so that face positioning and living body anti-counterfeiting judgment under strong light are improved.
6. When the brightness of the infrared LED lamp bead is 30%, an infrared image Tir2 is collected, living body detection judgment is carried out based on Tir1 and Tir2, and the main process is as follows:
positioning a face area of the two images by taking a nose as a center, and dividing the face area into 9 grids;
respectively calculating the brightness mean value Lxy of the corresponding grids of the two images;
carrying out variance operation on the two images corresponding to the grids, and corresponding to a formula
Obtaining the mean square error s (sigma) y of the corresponding grid;
and comprehensively judging the mean square error of each grid, judging as a prosthesis when s (sigma) is larger than a set threshold, and otherwise, judging as a living body.
The images aiming at the Tir1 and the Tir2 are subjected to a large amount of sampling and then sent to a neural network algorithm for training, and the actual effect is better.
7. Calling a neural network RGB anti-counterfeiting model according to the Trgb2 image for judgment, if the image quality is judged to be poor and cannot be judged, turning on a white light LED lamp bead for instant supplementary lighting, acquiring a new image Trgbn, calling the RGB anti-counterfeiting model again for living body detection judgment, and if the image is a prosthesis, returning to step 3 for repeating the process;
8. and (4) if the IR image and the RGB image live body detection pass at the same time, entering an image recognition process. And respectively comparing and identifying the IR image and the RGB image, and ending the process if the identification is passed.
The steps of IR image recognition are as follows: a registered image processing step of generating registered data from the input registered face image; and an identification image processing process for identifying the acquired face image, wherein the registration image processing process comprises: a registered face image feature extraction step of extracting features of registered face images from a plurality of registered face images to generate registered data, the identification image processing step including: a conversion image generation step, namely obtaining a conversion image from the IR image of the face to be recognized according to preset learning information; an occlusion region determining step, namely obtaining a difference part generated by an occlusion object according to the IR image of the face to be recognized and the difference image of the converted image; a step of extracting the characteristics of the face image to be recognized, which is to extract the characteristics of the face image to be recognized from the collected face image to be recognized; and a similarity calculation and evaluation step of calculating the similarity between the registered face image and the face image to be recognized by omitting the difference part recognized in the occlusion region determination step according to the features of the registered face image and the face image to be recognized; the learning information is a conversion formula from a sample image space to a registered face image feature space generated according to a plurality of learning sample images shot on the face image under the condition of no shielding; in the step of generating the converted image, the image obtains the characteristics of the space of the non-shielding face image from the face image to be recognized according to the conversion formula, and then the image is converted back to the original space by using the inverse operation of the conversion formula to obtain the converted image of the similar non-shielding face; in the step of determining the occlusion area, subtracting the converted image from the face image to be recognized to generate the difference image, and generating the difference part according to the size and distribution of pixels in the difference image after binarizing the difference image.
The steps of RGB image recognition may be the same as the IR image recognition steps and will not be described here.
The above IR image live body detection, RGB image live body detection, IR image recognition and RGB image recognition may also adopt other common methods as long as living body and face images can be recognized.
The present invention has been described in connection with the preferred embodiments, but the present invention is not limited to the embodiments disclosed above, and is intended to cover various modifications, equivalent combinations, which are within the spirit of the invention.
Claims (5)
1. A multispectral face identification module is characterized in that: comprises a camera assembly and a control assembly, wherein the camera assembly comprises an IR camera, a light supplement lamp and an RGB camera, the control assembly comprises a main controller, a power management unit and a storage unit,
the IR camera and the RGB camera are connected with the main controller, and the IR image and the RGB image of the shot human face are processed by the image processing unit and then sent to the main controller;
the light supplement lamp comprises a white light LED lamp bead and an infrared LED lamp bead which are arranged up and down and packaged together, the light supplement lamp is integrated with the IR camera and the RGB camera, and light supplement is carried out on the IR camera and the RGB camera according to ambient light conditions;
the main controller performs infrared image living body detection and color image living body detection on the IR image and the RGB image, compares the IR image and the RGB image with the database for identification if the IR image and the RGB image both accord with living body standards, and outputs an identification result;
the power management unit is connected with the main controller and provides power for the main controller;
the storage unit is connected with the main controller and used for storing data and programs.
2. The multispectral face recognition module of claim 1, wherein: the IR camera is connected with the main controller through a DVP interface, and the RGB camera is connected with the main controller through an MIPI interface.
3. The multi-spectral face recognition module of claim 1 or 2, wherein: the control assembly further comprises an encryption IC, and the encryption IC is connected with the main controller.
4. The multispectral face recognition module according to claim 1 or 2, wherein: the control assembly further comprises a voice interface, a USB interface, a UART interface, an LCD interface and a TP interface which are connected with the main controller.
5. A multispectral face recognition method is characterized in that: the method comprises the following steps:
(1) the method comprises the steps that an IR camera and an RGB camera shoot a human face, the IR camera and the RGB camera respectively perform photometry and detect ambient light conditions, intelligent dynamic exposure adjustment is respectively performed according to respective measured data, and an IR image and an RGB image are obtained through shooting and are sent to a main controller after being processed by an image processing unit;
(2) respectively carrying out living body detection on the IR image living body detection unit and the RGB image living body detection unit of the main controller, and if the detection results of the IR image living body detection unit and the RGB image living body detection unit are both living bodies, carrying out the next step; if any one of the detection results of the IR image living body detection unit and the RGB image living body detection unit is a non-living body, outputting a detection non-conforming result;
(3) the IR image recognition unit and the RGB image recognition unit of the main controller perform face recognition on the IR image and the RGB image according to the face information in the database, and if the face information in the database is not matched, the unmatched face information is output; and if the face information is matched with certain face information in the database, outputting corresponding matched face information.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210175114.4A CN114627562A (en) | 2022-02-24 | 2022-02-24 | Multispectral face recognition module and method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210175114.4A CN114627562A (en) | 2022-02-24 | 2022-02-24 | Multispectral face recognition module and method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114627562A true CN114627562A (en) | 2022-06-14 |
Family
ID=81900025
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210175114.4A Pending CN114627562A (en) | 2022-02-24 | 2022-02-24 | Multispectral face recognition module and method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114627562A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115623291A (en) * | 2022-08-12 | 2023-01-17 | 深圳市新良田科技股份有限公司 | Binocular camera module, service system and face verification method |
| CN117292399A (en) * | 2023-09-11 | 2023-12-26 | 广州市坤盛信息科技有限公司 | Device for recognizing wild protection animals by RGB (red, green and blue) cameras based on infrared light supplementing |
-
2022
- 2022-02-24 CN CN202210175114.4A patent/CN114627562A/en active Pending
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115623291A (en) * | 2022-08-12 | 2023-01-17 | 深圳市新良田科技股份有限公司 | Binocular camera module, service system and face verification method |
| CN117292399A (en) * | 2023-09-11 | 2023-12-26 | 广州市坤盛信息科技有限公司 | Device for recognizing wild protection animals by RGB (red, green and blue) cameras based on infrared light supplementing |
| CN117292399B (en) * | 2023-09-11 | 2024-04-05 | 广州市坤盛信息科技有限公司 | Device for recognizing wild protection animals by RGB (red, green and blue) cameras based on infrared light supplementing |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106874871B (en) | Living body face double-camera identification method and identification device | |
| CN108197546B (en) | Illumination processing method and device in face recognition, computer equipment and storage medium | |
| CN114627562A (en) | Multispectral face recognition module and method | |
| CN101882034B (en) | Device and method for discriminating color of touch pen of touch device | |
| US9460521B2 (en) | Digital image analysis | |
| CN110458041B (en) | Face recognition method and system based on RGB-D camera | |
| JP2014053855A (en) | Image processing device and method, and program | |
| EP3466324A1 (en) | Skin diagnostic device and skin diagnostic method | |
| CN111738970A (en) | Image fusion method and device and computer readable storage medium | |
| CN116309607B (en) | Ship type intelligent water rescue platform based on machine vision | |
| CN107490967A (en) | Adapt to the picking robot Target self-determination identifying system and its method of illuminance mutation | |
| CN109886195A (en) | Skin identification method based on depth camera near-infrared single color gradation figure | |
| US10748019B2 (en) | Image processing method and electronic apparatus for foreground image extraction | |
| CN111666869B (en) | Face recognition method and device based on wide dynamic processing and electronic equipment | |
| CN114565993B (en) | Intelligent lock system based on face recognition and cat eye | |
| US20140327796A1 (en) | Method for estimating camera response function | |
| CN112861645A (en) | Infrared camera dim light environment compensation method and device and electronic equipment | |
| CN101980299B (en) | Chessboard calibration-based camera mapping method | |
| CN115457459A (en) | Machine vision system capable of effectively improving detection efficiency | |
| CN101996324B (en) | Dynamic sampling-based method for extracting laser spot | |
| CN211557362U (en) | Front-end image acquisition device capable of adapting to image scene | |
| KR101049409B1 (en) | Apparatus and method for color correction in image processing system | |
| CN105323647A (en) | Method and device for detecting the intensity of light in television watching environment and smart television | |
| JP2022179121A (en) | Camera device and image processing method | |
| JP2018072884A (en) | Information processing device, information processing method and program |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |