WO2022077139A1 - 人脸识别方法、装置及可读存储介质 - Google Patents

人脸识别方法、装置及可读存储介质 Download PDF

Info

Publication number
WO2022077139A1
WO2022077139A1 PCT/CN2020/120265 CN2020120265W WO2022077139A1 WO 2022077139 A1 WO2022077139 A1 WO 2022077139A1 CN 2020120265 W CN2020120265 W CN 2020120265W WO 2022077139 A1 WO2022077139 A1 WO 2022077139A1
Authority
WO
WIPO (PCT)
Prior art keywords
face
mask
image
feature point
images
Prior art date
Application number
PCT/CN2020/120265
Other languages
English (en)
French (fr)
Inventor
杨进维
Original Assignee
鸿富锦精密工业(武汉)有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 鸿富锦精密工业(武汉)有限公司 filed Critical 鸿富锦精密工业(武汉)有限公司
Priority to CN202080003091.8A priority Critical patent/CN114698399A/zh
Priority to PCT/CN2020/120265 priority patent/WO2022077139A1/zh
Priority to US17/598,453 priority patent/US11922724B2/en
Publication of WO2022077139A1 publication Critical patent/WO2022077139A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/771—Feature selection, e.g. selecting representative features from a multi-dimensional feature space
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/80—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
    • G06V10/803—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level of input or preprocessed data
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/161—Detection; Localisation; Normalisation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/168—Feature extraction; Face representation
    • G06V40/171—Local features and components; Facial parts ; Occluding parts, e.g. glasses; Geometrical relationships
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/172—Classification, e.g. identification

Definitions

  • the present application relates to a face recognition method, device and non-volatile readable storage medium.
  • the present application aims to solve at least one of the technical problems existing in the prior art.
  • one aspect of the present application is to provide a face recognition method, which can realize accurate face recognition when people wear masks.
  • An embodiment of the present application proposes a face recognition method.
  • the method includes: extracting a face sample image from a preset face sample library; performing face feature point detection on the face sample image to obtain a plurality of faces Feature points; obtain a plurality of mask images, wherein each of the mask images corresponds to a mask type; select a first face feature point, a second face feature point, a third person from a plurality of the face feature points Face feature point and the fourth face feature point, the distance between the first face feature point and the second face feature point is taken as the mask image height, and the third face feature point and the The distance between the described fourth face feature points is used as the mask image width; each described mask image is resized according to the described mask image height and the described mask image width; each described mask image through size adjustment is respectively fuse with the face sample images to obtain a plurality of face mask images; save the plurality of face mask images to the preset face sample library; obtain a human face through training based on the preset face sample library Face recognition model for face recognition
  • the face sample images in the preset face sample library include a plurality of face identities, and each of the face identities includes at least one face sample image.
  • the face sample database is trained to obtain the steps of face recognition model, including:
  • the face recognition model is obtained by training based on the preset face sample library.
  • the method before the step of obtaining the face recognition model by training based on the preset face sample library, the method further includes:
  • a mapping relationship is established between each of the face sample images in the preset face sample library and face identity information, so that each of the face sample images carries a face identity tag.
  • the method before the step of obtaining the face recognition model by training based on the preset face sample library, the method further includes:
  • a mapping relationship is established between each face sample image in the preset face sample library and mask wearing information, so that each face sample image carries a mask wearing label.
  • the step of performing face feature point detection on the face sample image to obtain a plurality of face feature points includes:
  • a plurality of described facial feature points obtained by detection are demarcated, and a sequence number is added for each described facial feature point, so as to select the facial feature point of the specified sequence number as the first facial feature point, the described facial feature point respectively.
  • the second face feature point, the third face feature point and the fourth face feature point are demarcated, and a sequence number is added for each described facial feature point, so as to select the facial feature point of the specified sequence number as the first facial feature point, the described facial feature point respectively.
  • the step of adjusting the size of each mask image according to the mask image height and the mask image width includes:
  • a scaling process is performed on each of the mask images, so that the height of each mask image is equal to the height of the mask image, and the width of each mask image is equal to the width of the mask image.
  • the step of fusing each of the resized mask images with the face sample images to obtain a plurality of face mask images includes:
  • each of the mask images adjusted in size is respectively fused with the face sample images to obtain a plurality of the face mask images.
  • the step of extracting the face sample image from the preset face sample library it also includes:
  • Preprocessing is performed on each face sample image in the preset face sample library, wherein the preprocessing includes one or more of the following combinations: face image righting processing, face image enhancement processing, face image Normalized processing.
  • Another aspect of the present application is to provide a face recognition device, the device includes a processor and a memory, the memory stores several computer-readable instructions, and the processor is configured to execute the computer-readable instructions stored in the memory. Steps of implementing the above-mentioned face recognition method when instructed.
  • Another aspect of the present application is to provide a non-volatile readable storage medium, the non-volatile readable storage medium stores a plurality of instructions, and a plurality of the instructions can be executed by one or more processors , so as to realize the steps of the above-mentioned face recognition method.
  • the face recognition device, method, and non-volatile readable storage medium According to the face recognition device, method, and non-volatile readable storage medium according to the embodiments of the present application, it is possible to accurately perform face recognition when people wear masks, and improve the user's face recognition experience.
  • FIG. 1 is a functional block diagram of a face recognition device according to an embodiment of the present application.
  • FIG. 2 is a functional block diagram of a face recognition program according to an embodiment of the present application.
  • FIG. 3 is a schematic diagram of the distribution of face feature points of a face sample image according to an embodiment of the present application.
  • FIG. 4 is a flowchart of a face recognition method according to an embodiment of the present application.
  • memory 10 processor 20 face recognition program 30 Camera 40 display 50
  • Extract module 101 Detection module 102 get module 103 select module 104 adjustment module 105 fusion module 106 save module 107 training module 108 face recognition device 100
  • the terms “comprising”, “comprising” or any other variation thereof are intended to encompass a non-exclusive inclusion such that a process, method, article or device comprising a list of elements includes not only those elements, but also includes no explicit Other elements listed, or elements inherent to such a process, method, article or apparatus are also included. Without further limitation, an element qualified by the phrase “comprising a" does not preclude the presence of additional identical elements in a process, method, article or apparatus that includes the element.
  • FIG. 1 is a schematic diagram of a preferred embodiment of the applicant's face recognition device.
  • the face recognition device 100 can realize the function of face recognition.
  • the face recognition device 100 can be a consumer electronic device such as a mobile phone and a computer, or some components in the consumer electronic device such as a mobile phone and a computer.
  • the equipment is safe to use, and it can also be access control equipment, monitoring equipment, or some components in the access control equipment and monitoring equipment, to realize functions such as access control and security monitoring through face recognition.
  • the face recognition apparatus 100 may include a memory 10 , a processor 20 , and a face recognition program 30 stored in the memory 10 and executable on the processor 20 .
  • the processor 20 executes the face recognition program 30
  • the steps in the embodiment of the face recognition method are implemented, for example, steps S400 to S414 shown in FIG. 4 .
  • the processor 20 executes the face recognition program 30
  • the functions of the modules in FIG. 2 such as modules 101 to 108 , are implemented.
  • the face recognition device 100 may also be a face recognition device obtained by training the face recognition device 100 independently of consumer electronic devices such as mobile phones and computers, or independent of security control devices such as access control equipment and monitoring equipment.
  • the model can be processed by model quantification and compression, so as to integrate the model-processed face recognition model into consumer electronic devices such as mobile phones and computers, so as to ensure the safety of electronic devices through face recognition, or integrate To such as access control equipment, monitoring equipment, to achieve access control, security monitoring and other functions through face recognition.
  • the face recognition program 30 may be divided into one or more modules, which are stored in the memory 10 and executed by the processor 20 to complete the present application.
  • the one or more modules may be a series of computer-readable instruction segments capable of accomplishing specific functions, and the instruction segments are used to describe the execution process of the face recognition program 30 in the face recognition device 100 .
  • the face recognition program 30 can be divided into extraction module 101 , detection module 102 , acquisition module 103 , selection module 104 , adjustment module 105 , fusion module 106 , storage module 107 , and training module 108 in FIG. 2 .
  • the specific functions of each module please refer to the function of each module in Figure 2 below.
  • the face recognition apparatus 100 may further include a camera 40 and a display screen 50.
  • the camera 40 may be used to photograph the face to be recognized, and the display screen may be used to display the currently photographed face and the face recognition result.
  • the schematic diagram is only an example of the face recognition apparatus 100, and does not constitute a limitation on the face recognition apparatus 100, and may include more or less components than those shown in the figure, or combine some components , or different components, for example, the face recognition device 100 may further include a communication module, a bus, and the like.
  • the processor 20 may be a central processing unit (Central Processing Unit, CPU), or other general-purpose processors, digital signal processors (Digital Signal Processor, DSP), application specific integrated circuits (Application Specific Integrated Circuit, ASIC), ready-made processors. Field-Programmable Gate Array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.
  • the general-purpose processor can be a microprocessor or the processor 20 can also be any conventional processor, etc.
  • the processor 20 can use various interfaces and buses to connect various parts of the face recognition device 100 .
  • the memory 10 can be used to store the face recognition program 30 and/or modules, and the processor 20 realizes the face recognition by running or executing the computer-readable instructions and/or modules stored in the memory 10 and calling the data stored in the memory 10.
  • the memory 10 may include high-speed random access memory, and may also include non-volatile memory, such as hard disk, internal memory, plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) card, Flash Card, at least one magnetic disk storage device, flash memory device, or other volatile solid state storage device.
  • FIG. 2 is a functional block diagram of a preferred embodiment of the applicant's face recognition program.
  • the face recognition program 30 may include an extraction module 101 , a detection module 102 , an acquisition module 103 , a selection module 104 , an adjustment module 105 , a fusion module 106 , a storage module 107 and a training module 108 .
  • the above-mentioned modules may be programmable software instructions stored in the memory 10 and invoked by the processor 20 for execution. It can be understood that, in other embodiments, the above-mentioned modules may also be program instructions or firmware (firmware) solidified in the processor 20 .
  • the extraction module 101 is used for extracting face sample images from a preset face sample library.
  • the preset face sample library may pre-build a sample library for training a face recognition model, and the face sample images in the preset face sample library may include multiple face identities, Each face identity may include at least one face sample image, for example, each face identity may include 1-100 face sample images.
  • the face sample image may be a headshot, a life photo, etc., and the face area in the face sample image may be a front face, a side face, and the like.
  • the preset face sample library includes face sample images of 100 users, and each user has 50 face sample images.
  • a face image of a person's face identity can be pre-photographed by a camera, and the resolution of the face image can be 1080P, 720P, etc.
  • the face image can be processed in real time by the image processing software pre-installed on the computer, Get multiple face sample images. It is also possible to take a photo of a face identity through a camera to obtain a plurality of face sample images of the face identity.
  • each face identity corresponds to a folder, and face sample images of the same face identity are stored in the same folder.
  • the face sample images in the preset face sample library are preferably face images without masks.
  • the extraction module 101 may extract one face sample image or multiple face sample images from the preset face sample library each time.
  • the number of face sample images extracted by the extraction module 101 each time can be set according to actual needs.
  • the extraction module 101 can randomly extract a face sample image from a preset face sample library, or can extract a face sample image from the preset face sample library in a preset order, such as The face sample image extraction is performed in sequence according to the arrangement order of the folders and the arrangement order of the face sample images in the folders.
  • each face sample image in the preset face sample library may be preprocessed, and the preprocessing includes: A combination of one or more of the following: face image righting processing, face image enhancement processing, face image normalization processing.
  • face image righting processing can achieve the face sample image with the correct face position
  • face image enhancement processing can improve the quality of the face sample image
  • face image normalization processing can realize the face sample image conversion to size Consistent normalized face images with the same gray value range.
  • the detection module 102 is configured to perform facial feature point detection on the face sample image to obtain a plurality of facial feature points.
  • the detection module 102 can perform face feature point detection on the face sample image to obtain a plurality of face feature points.
  • the detection module 102 can call the Dlib module (the Dlib module includes a pre-trained facial feature point detection model) to detect the facial feature points, obtain 68 facial feature points, and analyze the detected 68 facial feature points. Calibration is performed, and a serial number is added to each face feature point, and then 68 face feature points are drawn on the face sample image. The distribution of 68 facial feature points is shown in Figure 3.
  • the acquisition module 103 is used to acquire a plurality of mask images, each mask image corresponds to a mask type.
  • the types of masks may include N95 masks, PM2.5 masks, activated carbon masks, cotton masks, medical masks, and the like.
  • the obtaining module 103 can obtain mask images of multiple mask types from a designated website or a designated server through the network. It is also possible to take pictures of masks of multiple mask types through a camera, and then the acquisition module 103 can acquire multiple mask images.
  • the selection module 104 is used to select the first human face feature point, the second human face feature point, the third human face feature point and the fourth human face feature point from a plurality of the human face feature points, and the first human face feature point is selected.
  • the distance between the face feature point and the second face feature point is taken as the mask image height, and the distance between the third face feature point and the fourth face feature point is taken as the mask image width.
  • the first face feature point, the second face feature point, the third face feature point, and the fourth face feature point may be set according to actual face recognition requirements.
  • the facial feature point of serial number 9 can be set as the first facial feature point
  • the facial feature point of serial number 30 can be set as the second facial feature point
  • the person with serial number 9 can be set as the second facial feature point.
  • the distance between the face feature point and the face feature point of serial number 30 is defined as the height of the mask image (that is, the height of the mask marked in Figure 3); the face feature point of serial number 5 is set as the third face feature point, and the serial number The face feature point of 13 is set as the third face feature point, and then the distance between the face feature point of serial number 5 and the face feature point of serial number 13 is defined as the width of the mask image (that is, the width of the mask marked in Figure 3) .
  • the facial feature point of serial number 9 can also be set as the first facial feature point
  • the facial feature point of serial number 29 can also be set as the second facial feature point
  • the facial feature point of serial number 4 can be set as the second facial feature point.
  • the face feature point is set as the third face feature point
  • the face feature point with serial number 14 is set as the third face feature point.
  • the adjustment module 105 is configured to adjust the size of each mask image according to the height of the mask image and the width of the mask image.
  • the adjustment module 105 may adjust the size of each mask image according to the height of the mask image and the width of the mask image. For example, the adjustment module 105 can scale each mask image according to the mask image height and the mask image width, so that the height of each mask image is equal to the height of the mask image, and the width of each mask image is equal to the width of the mask image .
  • the height of each mask image is not limited to be adjusted to be equal to the height of the mask image, and the width of each mask image is adjusted to be equal to the width of the mask image. The height of the mask image is adjusted to be equal to the height of the mask image, and the width of each mask image is adjusted to be equal to 1.1 times the width of the mask image.
  • the fusion module 106 is used to fuse each mask image adjusted in size with the face sample image respectively to obtain a plurality of face mask images.
  • the fusion module 106 may respectively fuse each resizing mask image with the face sample image to obtain multiple face mask images.
  • the extraction module 101 extracts the face sample image I1 from the preset face sample library, and the adjustment module 105 compares the N95 mask image I2, PM2.5 mask image I3, activated carbon mask image I4, and cotton mask image I5 acquired by the acquisition module 103.
  • the medical mask image I6 has been resized.
  • the fusion module 106 can fuse the face sample image I1 with the resized N95 mask image I2 to obtain a new face sample image I11, and fuse the face sample image I1 with the resized PM2.5 mask image I3.
  • the fusion module 106 can use an open source computer vision library (Open Source Computer Vision Library, OpenCV) to seamlessly fuse each mask image adjusted with the face sample image, respectively, to obtain a plurality of face mask images. .
  • OpenCV Open Source Computer Vision Library
  • the fusion module 106 may also use other image fusion algorithms to respectively fuse each mask image adjusted in size with the face sample image.
  • the saving module 107 is used for saving a plurality of the face mask images to the preset face sample library.
  • the saving module 107 can save the plurality of face mask images to a preset face sample library, so that the subsequent face model training can be based on There are face images with masks and face images without masks for model training.
  • the training module 108 is configured to train a face recognition model based on the preset face sample library to perform face recognition.
  • the training module 108 can be based on the preset face sample image.
  • the face recognition model is obtained by training a face sample database.
  • the training module 108 may use a preset face sample library to train a preset neural network (such as a convolutional neural network/recurrent neural network) to obtain the face recognition model.
  • a preset neural network such as a convolutional neural network/recurrent neural network
  • the face image to be recognized is input into the face recognition model to obtain a plurality of face feature vectors, for example, 612 face feature vectors can be obtained for face matching.
  • the training method of the face recognition model may adopt the existing training technology of the face recognition model, which will not be described in detail here.
  • the name of each face is displayed at the same time.
  • a mapping relationship can be established between each face sample image and face identity information in the preset face sample library (including face sample images wearing masks and face sample images not wearing masks), so that each face The sample images carry face identity tags.
  • the training module 108 trains a face recognition model based on the face sample images carrying face identity tags, and then the trained face recognition model can simultaneously display the face to be recognized when performing face recognition on the face to be recognized. name. For unknown faces, "Stranger” or "Unknown” can be displayed uniformly.
  • the information is displayed at the same time.
  • a mapping relationship can be established between each face sample image and the mask wearing information in the preset face sample library (including face sample images with masks and face sample images without masks), so that each face sample The image carries a mask wearing label.
  • the training module 108 trains a face recognition model based on the face sample images carrying masks and wearing labels, and then the trained face recognition model can simultaneously display whether a mask is worn when performing face recognition on the face to be recognized. information.
  • the above-mentioned face recognition program 30 can amplify the face sample libraries of the face recognition systems of different platforms, synthesize the face sample library wearing a mask, and then import the face sample library wearing a mask into the database.
  • these face recognition systems can successfully identify the face identity information of people wearing masks.
  • it can be the Neural Compute Stick (NCS) face recognition system platform, the Dlib face recognition system platform, and the Azure face recognition system platform.
  • NCS Neural Compute Stick
  • the face recognition model when the training module 108 trains a face recognition model based on the preset face sample library, the face recognition model can be processed by means of quantization and compression, so that the face recognition model obtained by training Can be integrated into the target terminal.
  • FIG. 4 is a flowchart of a face recognition method in an embodiment of the present application. According to different requirements, the order of the steps in the flowchart can be changed, and some steps can be omitted.
  • Step S400 extracting face sample images from a preset face sample library.
  • the preset face sample library may pre-build a sample library for training a face recognition model, and the face sample images in the preset face sample library may include multiple face identities, Each face identity may include at least one face sample image, for example, each face identity may include 1-100 face sample images.
  • the face sample image may be a headshot, a life photo, etc., and the face area in the face sample image may be a front face, a side face, and the like.
  • the preset face sample library includes face sample images of 100 users, and each user has 50 face sample images.
  • a face image of a person's face identity can be pre-photographed by a camera, and the resolution of the face image can be 1080P, 720P, etc.
  • the face image can be processed in real time by the image processing software pre-installed on the computer, Get multiple face sample images. It is also possible to take a photo of a face identity through a camera to obtain a plurality of face sample images of the face identity.
  • each face identity corresponds to a folder, and face sample images of the same face identity are stored in the same folder.
  • the face sample images in the preset face sample library are preferably face images without masks.
  • one face sample image or multiple face sample images may be extracted from the preset face sample library each time.
  • the number of face sample images extracted each time can be set according to actual needs.
  • a face sample image can be randomly extracted from a preset face sample library, or a face sample image can be extracted from the preset face sample library in a preset order, for example, according to a folder. and the arrangement order of the face sample images in the folder to extract the face sample images in turn.
  • each face sample image in the preset face sample library may be preprocessed, and the preprocessing includes one of the following or Various combinations: face image righting processing, face image enhancement processing, face image normalization processing.
  • face image righting processing can achieve the face sample image with the correct face position
  • face image enhancement processing can improve the quality of the face sample image
  • face image normalization processing can realize the face sample image conversion to size Consistent normalized face images with the same gray value range.
  • Step S402 performing face feature point detection on the face sample image to obtain a plurality of face feature points.
  • face feature point detection may be performed on the face sample image to obtain a plurality of face feature points.
  • the Dlib module contains a pre-trained face feature point detection model
  • you can call the Dlib module to detect face feature points, obtain 68 face feature points, and calibrate the 68 face feature points obtained from the detection.
  • the distribution of 68 facial feature points is shown in Figure 3.
  • Step S404 acquiring multiple mask images, each mask image corresponding to one type of mask.
  • the types of masks may include N95 masks, PM2.5 masks, activated carbon masks, cotton masks, medical masks, and the like.
  • Mask images of multiple mask types can be obtained from a designated website or a designated server through the network. It is also possible to take pictures of masks of multiple mask types through a camera, and then multiple mask images can be obtained.
  • Step S406 select the first face feature point, the second face feature point, the third face feature point and the fourth face feature point from a plurality of the face feature points, and use the first face feature point
  • the distance between the point and the second face feature point is taken as the mask image height
  • the distance between the third face feature point and the fourth face feature point is taken as the mask image width.
  • the first face feature point, the second face feature point, the third face feature point, and the fourth face feature point may be set according to actual face recognition requirements.
  • the facial feature point of serial number 9 can be set as the first facial feature point
  • the facial feature point of serial number 30 can be set as the second facial feature point
  • the person with serial number 9 can be set as the second facial feature point.
  • the distance between the face feature point and the face feature point of serial number 30 is defined as the height of the mask image; the face feature point of serial number 5 is set as the third face feature point, and the face feature point of serial number 13 is set as The third face feature point, and then the distance between the face feature point of serial number 5 and the face feature point of serial number 13 is defined as the width of the mask image.
  • the facial feature point of serial number 9 can also be set as the first facial feature point
  • the facial feature point of serial number 29 can also be set as the second facial feature point
  • the facial feature point of serial number 4 can be set as the second facial feature point.
  • the face feature point is set as the third face feature point
  • the face feature point with serial number 14 is set as the third face feature point.
  • Step S408 adjusting the size of each mask image according to the mask image height and the mask image width.
  • each mask image when the height of the mask image and the width of the mask image are determined, each mask image may be resized according to the height of the mask image and the width of the mask image.
  • each mask image can be scaled according to the mask image height and the mask image width, so that the height of each mask image is equal to the height of the mask image, and the width of each mask image is equal to the width of the mask image.
  • the height of each mask image is not limited to be adjusted to be equal to the height of the mask image, and the width of each mask image is adjusted to be equal to the width of the mask image.
  • the height of the mask image is adjusted to be equal to the height of the mask image, and the width of each mask image is adjusted to be equal to 1.1 times the width of the mask image.
  • step S410 each mask image adjusted in size is respectively fused with the face sample image to obtain a plurality of face mask images.
  • each resizing mask image can be fused with the face sample image respectively to obtain multiple face mask images.
  • the face sample image I1 is extracted from the preset face sample library, and the obtained N95 mask image I2, PM2.5 mask image I3, activated carbon mask image I4, cotton mask image I5 and medical mask image I6 are resized .
  • the face sample image I1 and the resized N95 mask image I2 can be fused to obtain a new face sample image I11, and the face sample image I1 and the resized PM2.5 mask image I3 can be fused to obtain a new face sample image I1.
  • the face sample image I12 is obtained by fusing the face sample image I1 with the size-adjusted activated carbon mask image I4 to obtain a new face sample image I13, and the face sample image I1 and the size-adjusted cotton mask image I5 are processed.
  • a new face sample image I14 is obtained by fusing, and a new face sample image I15 is obtained by fusing the face sample image I1 and the size-adjusted medical mask image I6.
  • For each face sample image in the preset face sample library it can be fused with N95 mask image I2, PM2.5 mask image I3, activated carbon mask image I4, cotton mask image I5 and medical mask image I6 to obtain 5 images. Face mask image.
  • OpenCV can be used to seamlessly fuse each resized mask image with a face sample image to obtain multiple face mask images.
  • other image fusion algorithms can also be used to fuse each mask image adjusted in size with the face sample image respectively.
  • Step S412 saving a plurality of the face mask images to the preset face sample library.
  • the plurality of face mask images can be saved to a preset face sample library, so that the subsequent face model training can be based on the faces of people wearing masks. Images and face images without masks are used for model training.
  • Step S414 a face recognition model is obtained by training based on the preset face sample library, so as to perform face recognition.
  • each face sample image in the preset face sample library when fused with multiple mask images and saved in the preset face sample library, it can be based on the preset face sample image.
  • the sample database is trained to obtain the face recognition model.
  • a preset neural network (such as a convolutional neural network/recurrent neural network) may be trained by using a preset face sample library to obtain the face recognition model. After the face recognition model is obtained, the face image to be recognized is input into the face recognition model to obtain a plurality of face feature vectors, for example, 612 face feature vectors can be obtained for face matching.
  • the training method of the face recognition model may adopt the existing training technology of the face recognition model, which will not be described in detail here.
  • the name of each face is displayed at the same time.
  • a mapping relationship can be established between each face sample image and face identity information in the preset face sample library (including face sample images wearing masks and face sample images not wearing masks), so that each face The sample images carry face identity tags.
  • a face recognition model is obtained by training a face sample image carrying a face identity label, and then the trained face recognition model can display the name of the face to be recognized when performing face recognition on the face to be recognized. For unknown faces, "Stranger” or "Unknown” can be displayed uniformly.
  • the information is displayed at the same time.
  • a mapping relationship can be established between each face sample image and the mask wearing information in the preset face sample library (including face sample images with masks and face sample images without masks), so that each face sample The image carries a mask wearing label.
  • a face recognition model is obtained by training a face sample image with a mask wearing label, and then the trained face recognition model can display information on whether a mask is worn or not at the same time when performing face recognition on the face to be recognized.
  • the above-mentioned face recognition program 30 can amplify the face sample libraries of the face recognition systems of different platforms, synthesize the face sample library wearing a mask, and then import the face sample library wearing a mask into the database.
  • these face recognition systems can successfully identify the face identity information of people wearing masks.
  • it can be the Neural Compute Stick (NCS) face recognition system platform, the Dlib face recognition system platform, and the Azure face recognition system platform.
  • NCS Neural Compute Stick
  • the face recognition model when a face recognition model is obtained by training based on the preset face sample library, the face recognition model can be processed by means of quantization and compression, so that the trained face recognition model can be integrated into target terminal.
  • the above-mentioned face recognition device, method and non-volatile readable storage medium can realize accurate face recognition when people wear masks, and improve the user's face recognition experience.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Human Computer Interaction (AREA)
  • Artificial Intelligence (AREA)
  • Computing Systems (AREA)
  • Databases & Information Systems (AREA)
  • Evolutionary Computation (AREA)
  • Medical Informatics (AREA)
  • Software Systems (AREA)
  • Image Analysis (AREA)

Abstract

一种人脸识别装置、方法及非易失性可读存储介质，所述方法包括：从预设人脸样本库中提取人脸样本图像，并进行人脸特征点检测，得到多个人脸特征点；获取多个口罩图像；从多个人脸特征点中选取第一至第四人脸特征点，将第一人脸特征点与第二人脸特征点之间的距离作为口罩图像高度，将第三人脸特征点与第四人脸特征点之间的距离作为口罩图像宽度；依据口罩图像高度与口罩图像宽度对每一口罩图像进行尺寸调整；将经过尺寸调整的每一口罩图像分别与人脸样本图像进行融合，得到多个人脸口罩图像并保存至预设人脸样本库；基于预设人脸样本库训练得到人脸识别模型，以进行人脸识别。可实现在人们佩戴口罩的情形下准确地进行人脸识别。

Description

人脸识别方法、装置及可读存储介质 技术领域
本申请涉及一种人脸识别方法、装置及非易失性可读存储介质。
背景技术
随着新型冠状病毒引发的新冠肺炎疫情在全球持续扩散，使得人们对于通过飞沫传播的流行病的防护意识越来越强。比如，人们为了确保避免遭受新冠肺炎的感染，都会主动戴起口罩来。但是，在一些公共场所(如车站、商场)、企业中都会使用人脸识别系统作为出入门禁，当人们戴起口罩时，站在人脸识别系统前做人脸识别时的失效可能性很高，导致人们需要摘掉口罩进行人脸识别，一定程度上增加了人们被新冠肺炎感染的风险。
发明内容
本申请旨在至少解决现有技术中存在的技术问题之一。为此，本申请的一个方面在于提出一种人脸识别方法，可实现在人们佩戴口罩的情形下准确地进行人脸识别。
本申请一实施例提出一种人脸识别方法，所述方法包括：从预设人脸样本库中提取人脸样本图像；对所述人脸样本图像进行人脸特征点检测，得到多个人脸特征点；获取多个口罩图像，其中每一所述口罩图像对应一种口罩类型；从多个所述人脸特征点中选取第一人脸特征点、第二人脸特征点、第三人脸特征点及第四人脸特征点，将所述第一人脸特征点与所述第二人脸特征点之间的距离作为口罩图像高度，并将所述第三人脸特征点与所述第四人脸特征点之间的距离作为口罩图像宽度；依据所述口罩图像高度与所述口罩图像宽度对每一所述口罩图像进行尺寸调整；将经过尺寸调整的每一所述口罩图像分别与所述人脸样本图像进行融合，得到多个人脸口罩图像；将多个 所述人脸口罩图像保存至所述预设人脸样本库；基于所述预设人脸样本库训练得到人脸识别模型，以进行人脸识别。
在一些实施例中，所述预设人脸样本库中的人脸样本图像包括多个人脸身份，每一所述人脸身份包括至少一张所述人脸样本图像，所述基于所述预设人脸样本库训练得到人脸识别模型的步骤，包括：
当所述预设人脸样本库中的每一人脸样本图像均完成与多个所述口罩图像的图像融合时，基于所述预设人脸样本库训练得到所述人脸识别模型。
在一些实施例中，所述基于所述预设人脸样本库训练得到所述人脸识别模型的步骤之前，还包括：
将所述预设人脸样本库中的每一所述人脸样本图像与人脸身份信息建立映射关系，以使得每一所述人脸样本图像携带有人脸身份标签。
在一些实施例中，所述基于所述预设人脸样本库训练得到所述人脸识别模型的步骤之前，还包括：
将所述预设人脸样本库中的每一所述人脸样本图像与口罩佩戴信息建立映射关系，以使得每一所述人脸样本图像携带有口罩佩戴标签。
在一些实施例中，所述对所述人脸样本图像进行人脸特征点检测，得到多个人脸特征点的步骤，包括：
利用预设人脸特征检测模型对所述人脸样本图像进行人脸特征点检测；
对检测得到的多个所述人脸特征点进行标定，并为每一所述人脸特征点添加序号，以分别选取指定序号的人脸特征点作为所述第一人脸特征点、所述第二人脸特征点、所述第三人脸特征点及所述第四人脸特征点。
在一些实施例中，所述依据所述口罩图像高度与所述口罩图像宽度对每一所述口罩图像进行尺寸调整的步骤，包括：
对每一所述口罩图像进行缩放处理，以使得每一所述口罩图像的高度等于所述口罩图像高度，每一所述口罩图像的宽度等于所述口罩图像宽度。
在一些实施例中，所述将经过尺寸调整的每一所述口罩图像分别与所述人脸样本图像进行融合，得到多个人脸口罩图像的步骤，包括：
利用OpenCV将经过尺寸调整的每一所述口罩图像分别与所述人脸样本图像进行融合，得到多个所述人脸口罩图像。
在一些实施例中，所述从预设人脸样本库中提取人脸样本图像的步骤之 前，还包括：
对所述预设人脸样本库中的每一人脸样本图像进行预处理，其中所述预处理包括以下一种或多种的组合：人脸图像扶正处理、人脸图像增强处理、人脸图像归一化处理。
本申请的另外一方面在于提出了一种人脸识别装置，所述装置包括处理器及存储器，所述存储器上存储有若干计算机可读指令，所述处理器用于执行存储器中存储的计算机可读指令时实现上述的人脸识别方法的步骤。
本申请的另外一方面在于提出了一种非易失性可读存储介质，所述非易失性可读存储介质存储有多条指令，多条所述指令可被一个或者多个处理器执行，以实现如上述的人脸识别方法的步骤。
根据本申请实施例的人脸识别装置、方法及非易失性可读存储介质，可实现在人们佩戴口罩的情形下准确地进行人脸识别，提升用户人脸识别体验。
附图说明
本申请的上述和/或附加的方面和优点从结合下面附图对实施例的描述中将变得明显和容易理解，其中：
图1是本申请一实施方式的人脸识别装置的功能模块图。
图2是本申请一实施方式的人脸识别程序的功能模块图。
图3是本申请一实施方式的人脸样本图像的人脸特征点的分布示意图。
图4是本申请一实施方式的人脸识别方法的流程图。
主要元件符号说明
存储器 10
处理器 20
人脸识别程序 30
摄像头 40
显示屏 50
提取模块 101
检测模块 102
获取模块 103
选取模块 104
调整模块 105
融合模块 106
保存模块 107
训练模块 108
人脸识别装置 100
具体实施方式
下面详细描述本申请的实施例，所述实施例的示例在附图中示出，其中自始至终相同或类似的标号表示相同或类似的元件或具有相同或类似功能的元件。下面通过参考附图描述的实施例是示例性的，仅用于解释本申请，而不能理解为对本申请的限制。
在本文中，术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含，从而使得包括一系列要素的过程、方法、物品或者装置不仅包括那些要素，而且还包括没有明确列出的其他要素，或者是还包括为这种过程、方法、物品或者装置所固有的要素。在没有更多限制的情况下，由语句“包括一个……”限定的要素，并不排除在包括该要素的过程、方法、物品或者装置中还存在另外的相同要素。
请参阅图1，为本申请人脸识别装置较佳实施例的示意图。
人脸识别装置100可以实现人脸识别功能，比如，人脸识别装置100可以是手机、电脑等消费性电子设备，或者手机、电脑等消费性电子设备内的部分组件，通过人脸识别保证电子设备使用安全，也可以是诸如门禁设备、监控设备，或者门禁设备、监控设备内的部分组件，实现通过人脸识别进行门禁管控、 安全监控等功能。人脸识别装置100可以包括存储器10、处理器20以及存储在存储器10中并可在处理器20上运行的人脸识别程序30。处理器20执行人脸识别程序30时实现人脸识别方法实施例中的步骤，例如图4所示的步骤S400～S414。或者，所述处理器20执行人脸识别程序30时实现图2中各模块的功能，例如模块101～108。
在一实施方式中，人脸识别装置100也可以是独立于手机、电脑等消费性电子设备、或者独立于门禁设备、监控设备等安全管控设备，经过人脸识别装置100训练得到的人脸识别模型，可以进行模型量化与压缩等方式的模型处理，以将经过模型处理后的人脸识别模型集成到手机、电脑等消费性电子设备内，实现通过人脸识别保证电子设备使用安全，或者集成到诸如门禁设备、监控设备内，实现通过人脸识别进行门禁管控、安全监控等功能。
人脸识别程序30可以被分割成一个或多个模块，所述一个或者多个模块被存储在存储器10中，并由处理器20执行，以完成本申请。所述一个或多个模块可以是能够完成特定功能的一系列计算机可读指令指令段，所述指令段用于描述人脸识别程序30在人脸识别装置100中的执行过程。例如，人脸识别程序30可以被分割成图2中的提取模块101、检测模块102、获取模块103、选取模块104、调整模块105、融合模块106、保存模块107及训练模块108。各模块具体功能参见下图2中各模块的功能。
人脸识别装置100还可以包括摄像头40及显示屏50，摄像头40可用于拍摄待识别人脸，显示屏可用于显示当前拍摄到的人脸及人脸识别结果。本领域技术人员可以理解，所述示意图仅是人脸识别装置100的示例，并不构成对人脸识别装置100的限定，可以包括比图示更多或更少的部件，或者组合某些部件，或者不同的部件，例如人脸识别装置100还可以包括通信模块、总线等。
处理器20可以是中央处理单元(Central Processing Unit，CPU)，还可以是其他通用处理器、数字信号处理器(Digital Signal Processor，DSP)、专用集成电路(Application Specific Integrated Circuit，ASIC)、现成可编程门阵列(Field-Programmable Gate Array，FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件等。通用处理器可以是微处理器或者处理器20也可以是任何常规的处理器等，处理器20可以利用各种接口和总线连接人脸识别装置100的各个部分。
存储器10可用于存储人脸识别程序30和/或模块，处理器20通过运行或执行存储在存储器10内的计算机可读指令和/或模块，以及调用存储在存储器10内的数据，实现人脸识别装置100的各种功能。存储器10可以包括高速随机存取存储器，还可以包括非易失性存储器，例如硬盘、内存、插接式硬盘，智能存储卡(Smart Media Card,SMC)，安全数字(Secure Digital,SD)卡，闪存卡(Flash Card)、至少一个磁盘存储器件、闪存器件、或其他易失性固态存储器件。
图2为本申请人脸识别程序较佳实施例的功能模块图。
参阅图2所示，人脸识别程序30可以包括提取模块101、检测模块102、获取模块103、选取模块104、调整模块105、融合模块106、保存模块107及训练模块108。在一实施方式中，上述模块可以为存储于存储器10中且可被处理器20调用执行的可程序化软件指令。可以理解的是，在其他实施方式中，上述模块也可为固化于处理器20中的程序指令或固件(firmware)。
提取模块101用于从预设人脸样本库中提取人脸样本图像。
在一实施方式中，所述预设人脸样本库可以预先构建用来进行人脸识别模型训练的样本库，所述预设人脸样本库中的人脸样本图像可以包括多个人脸身份，每一人脸身份可以包括至少一张人脸样本图像，比如每一人脸身份可以包括1～100张不等的人脸样本图像。人脸样本图像可以是大头照、生活照等，人脸样本图像中的脸部区域可以是正脸、侧脸等。比如，所述预设人脸样本库包括100个用户的人脸样本图像，每一用户具有50张人脸样本图像。
在一实施方式中，可以通过摄影机预先拍摄一人脸身份的人脸影像，人脸影像的清晰度可以是1080P、720P等，人脸影像可以通过电脑预先安装的影像处理软件进行即时的影像处理，得到多张人脸样本图像。还可以通过相机来对一人脸身份进行拍照，得到该人脸身份的多张人脸样本图像。
在一实施方式中，在所述预设人脸样本库中，每一人脸身份对应一个文件夹，同一人脸身份的人脸样本图像存储在同一个文件夹中。所述预设人脸样本库中的人脸样本图像优选是未佩戴口罩的人脸图像。
在一实施方式中，提取模块101每次可以从预设人脸样本库中提取一张人脸样本图像，或者多张人脸样本图像。提取模块101每次提取的人脸样本图像 的数量可以根据实际需求进行设定。
在一实施方式中，提取模块101可以随机从预设人脸样本库中提取一张人脸样本图像，也可以按照预设顺序从预设人脸样本库中提取一张人脸样本图像，比如按照文件夹的排列顺序及文件夹中的人脸样本图像的排列顺序依次进行人脸样本图像提取。
在一实施方式中，在提取模块101从预设人脸样本库中提取人脸样本图像之前，可以对该预设人脸样本库中的每一人脸样本图像进行预处理，所述预处理包括以下一种或多种的组合：人脸图像扶正处理、人脸图像增强处理、人脸图像归一化处理。人脸图像扶正处理可以实现得到人脸位置端正的人脸样本图像，人脸图像增强处理可以实现改善人脸样本图像的质量，人脸图像归一化处理可以实现将人脸样本图像转换为尺寸一致、灰度取值范围相同的标准化人脸图像。
检测模块102用于对所述人脸样本图像进行人脸特征点检测，得到多个人脸特征点。
在一实施方式中，当提取模块101从预设人脸样本库中提取人脸样本图像时，检测模块102可以对该人脸样本图像进行人脸特征点检测，得到多个人脸特征点。比如，检测模块102可以调用Dlib模块(该Dlib模块包含有预先训练好的人脸特征点检测模型)实现人脸特征点检测，得到68个人脸特征点，并对检测得到的68个人脸特征点进行标定，并为每一人脸特征点添加序号，进而实现在人脸样本图像上画出68个人脸特征点。68个人脸特征点的分布如图3所示。
获取模块103用于获取多个口罩图像，每一口罩图像对应一种口罩类型。
在一实施方式中，口罩类型可以包括N95口罩、PM2.5口罩、活性炭口罩、棉布口罩、医用口罩等。获取模块103可以通过网络从指定网站或者指定服务器获取多个口罩类型的口罩图像。也可以通过相机对多个口罩类型的口罩进行拍照，进而获取模块103可以获取多个口罩图像。
选取模块104用于从多个所述人脸特征点中选取第一人脸特征点、第二人脸特征点、第三人脸特征点及第四人脸特征点，将所述第一人脸特征点与所述第二人脸特征点之间的距离作为口罩图像高度，并将所述第三人脸特征点与所述第四人脸特征点之间的距离作为口罩图像宽度。
在一实施方式中，第一人脸特征点、第二人脸特征点、第三人脸特征点及第四人脸特征点可以根据实际人脸识别需求进行设定。比如，如图3所示，可以将序号9的人脸特征点设定为第一人脸特征点，将序号30的人脸特征点设定为第二人脸特征点，进而序号9的人脸特征点与序号30的人脸特征点之间的距离定义为口罩图像高度(即图3标注的口罩高度)；将序号5的人脸特征点设定为第三人脸特征点，将序号13的人脸特征点设定为第三人脸特征点，进而序号5的人脸特征点与序号13的人脸特征点之间的距离定义为口罩图像宽度(即图3标注的口罩宽度)。在其他实施例中，也可以将将序号9的人脸特征点设定为第一人脸特征点，将序号29的人脸特征点设定为第二人脸特征点，将序号4的人脸特征点设定为第三人脸特征点，将序号14的人脸特征点设定为第三人脸特征点。
调整模块105用于依据所述口罩图像高度与所述口罩图像宽度对每一所述口罩图像进行尺寸调整。
在一实施方式中，当确定了口罩图像高度与口罩图像宽度时，调整模块105可以根据口罩图像高度与口罩图像宽度对每一口罩图像进行尺寸调整。比如，调整模块105可以根据口罩图像高度与口罩图像宽度对每一所述口罩图像进行缩放处理，以使得每一口罩图像的高度等于该口罩图像高度，每一口罩图像的宽度等于该口罩图像宽度。在其他实施方式中，不限定将每一口罩图像的高度调整为等于口罩图像高度，将每一口罩图像的宽度调整为等于该口罩图像宽度，也可以是其他形式的尺寸调整，比如将每一口罩图像的高度调整为等于口罩图像高度，将每一口罩图像的宽度调整为等于该口罩图像宽度的1.1倍。
融合模块106用于将经过尺寸调整的每一所述口罩图像分别与所述人脸样本图像进行融合，得到多个人脸口罩图像。
在一实施方式中，当完成对口罩图像的尺寸调整时，融合模块106可以将经过尺寸调整的每一口罩图像分别与人脸样本图像进行融合，得到多个人脸口罩图像。比如，提取模块101从预设人脸样本库中提取人脸样本图像I1，调整模块105对获取模块103获取的N95口罩图像I2、PM2.5口罩图像I3、活性炭口罩图像I4、棉布口罩图像I5及医用口罩图像I6进行了尺寸调整。融合模块106可以将人脸样本图像I1与尺寸调整后的N95口罩图像I2进行融合得到一新的人脸样本图像I11，将人脸样本图像I1与尺寸调整后的PM2.5口罩图像I3 进行融合得到一新的人脸样本图像I12，将人脸样本图像I1与尺寸调整后的活性炭口罩图像I4进行融合得到一新的人脸样本图像I13，将人脸样本图像I1与尺寸调整后的棉布口罩图像I5进行融合得到一新的人脸样本图像I14，及将人脸样本图像I1与尺寸调整后的医用口罩图像I6进行融合得到一新的人脸样本图像I15。对于预设人脸样本库中的每一人脸样本图像，均可以分别与N95口罩图像I2、PM2.5口罩图像I3、活性炭口罩图像I4、棉布口罩图像I5及医用口罩图像I6进行融合得到5张人脸口罩图像。
在一实施方式中，融合模块106可以利用开源计算机视觉库(Open Source Computer Vision Library，OpenCV)将经过尺寸调整的每一口罩图像分别与人脸样本图像进行无缝融合，得到多个人脸口罩图像。在其他实施例中，融合模块106也可以其他图像融合算法将经过尺寸调整的每一口罩图像分别与人脸样本图像进行融合。
保存模块107用于将多个所述人脸口罩图像保存至所述预设人脸样本库。
在一实施方式中，当融合模块106融合得到多个人脸口罩图像时，保存模块107可以将该多个人脸口罩图像保存至预设人脸样本库，使得后续进行人脸模型训练时，可以基于有佩戴口罩的人脸图像及未佩戴口罩的人脸图像进行模型训练。
训练模块108用于基于所述预设人脸样本库训练得到人脸识别模型，以进行人脸识别。
在一实施方式中，当预设人脸样本库中的每一人脸样本图像均完成与多个口罩图像的图像融合，且被保存至预设人脸样本库时，训练模块108可以基于该预设人脸样本库训练得到所述人脸识别模型。
在一实施方式中，训练模块108可以利用预设人脸样本库对预设神经网络(比如卷积神经网络/循环神经网络)进行训练，得到所述人脸识别模型。当得到所述人脸识别模型后，待识别的人脸图像输入至人脸识别模型可以得到多个人脸特征向量，比如可以得到612个人脸特征向量，以进行人脸匹配。人脸识别模型的训练方式可以采用现有的人脸识别模型的训练技术，在此不再详述。
在一实施方式中，为了实现在进行人脸识别过程中，同时显示每一人脸的名字。可以将所述预设人脸样本库(包含佩戴口罩的人脸样本图像及未佩戴口罩的人脸样本图像)中的每一人脸样本图像与人脸身份信息建立映射关系，以 使得每一人脸样本图像携带有人脸身份标签。训练模块108再基于携带有人脸身份标签的人脸样本图像训练得到人脸识别模型，进而训练得到的人脸识别模型可以在对待识别的人脸进行人脸识别时，可以同时显示待识别人脸的名字。若是未知的人脸，可以统一显示“陌生人”或者“Unknown”。
在一实施方式中，为了实现在进行人脸识别过程中，同时显示每一人脸是否佩戴口罩的信息。可以将所述预设人脸样本库(包含佩戴口罩的人脸样本图像及未佩戴口罩的人脸样本图像)中的每一人脸样本图像与口罩佩戴信息建立映射关系，以使得每一人脸样本图像携带有口罩佩戴标签。训练模块108再基于携带有口罩佩戴标签的人脸样本图像训练得到人脸识别模型，进而训练得到的人脸识别模型可以在对待识别的人脸进行人脸识别时，可以同时显示是否佩戴口罩的信息。
在一实施方式中，上述人脸识别程序30可以对不同平台的人脸识别系统的人脸样本库进行扩增，合成戴口罩的人脸样本库，再将戴口罩的人脸样本库导入到原有人脸样本库中，使得该些人脸识别系统可以成功识别戴口罩的人脸身份信息。比如，可以是神经运算棒(Neural Compute Stick，NCS)人脸识别系统平台、Dlib人脸识别系统平台、Azure人脸识别系统平台。
在一实施方式中，当训练模块108基于该预设人脸样本库训练得到人脸识别模型时，可以对该人脸识别模型进行量化与压缩等方式的处理，使得训练得到的人脸识别模型可以集成到目标终端。
图4为本申请一实施方式中人脸识别方法的流程图。根据不同的需求，所述流程图中步骤的顺序可以改变，某些步骤可以省略。
步骤S400，从预设人脸样本库中提取人脸样本图像。
在一实施方式中，所述预设人脸样本库可以预先构建用来进行人脸识别模型训练的样本库，所述预设人脸样本库中的人脸样本图像可以包括多个人脸身份，每一人脸身份可以包括至少一张人脸样本图像，比如每一人脸身份可以包括1～100张不等的人脸样本图像。人脸样本图像可以是大头照、生活照等，人脸样本图像中的脸部区域可以是正脸、侧脸等。比如，所述预设人脸样本库包括100个用户的人脸样本图像，每一用户具有50张人脸样本图像。
在一实施方式中，可以通过摄影机预先拍摄一人脸身份的人脸影像，人脸影像的清晰度可以是1080P、720P等，人脸影像可以通过电脑预先安装的影像 处理软件进行即时的影像处理，得到多张人脸样本图像。还可以通过相机来对一人脸身份进行拍照，得到该人脸身份的多张人脸样本图像。
在一实施方式中，在所述预设人脸样本库中，每一人脸身份对应一个文件夹，同一人脸身份的人脸样本图像存储在同一个文件夹中。所述预设人脸样本库中的人脸样本图像优选是未佩戴口罩的人脸图像。
在一实施方式中，每次可以从预设人脸样本库中提取一张人脸样本图像，或者多张人脸样本图像。每次提取的人脸样本图像的数量可以根据实际需求进行设定。
在一实施方式中，可以随机从预设人脸样本库中提取一张人脸样本图像，也可以按照预设顺序从预设人脸样本库中提取一张人脸样本图像，比如按照文件夹的排列顺序及文件夹中的人脸样本图像的排列顺序依次进行人脸样本图像提取。
在一实施方式中，从预设人脸样本库中提取人脸样本图像之前，可以对该预设人脸样本库中的每一人脸样本图像进行预处理，所述预处理包括以下一种或多种的组合：人脸图像扶正处理、人脸图像增强处理、人脸图像归一化处理。人脸图像扶正处理可以实现得到人脸位置端正的人脸样本图像，人脸图像增强处理可以实现改善人脸样本图像的质量，人脸图像归一化处理可以实现将人脸样本图像转换为尺寸一致、灰度取值范围相同的标准化人脸图像。
步骤S402，对所述人脸样本图像进行人脸特征点检测，得到多个人脸特征点。
在一实施方式中，当从预设人脸样本库中提取人脸样本图像时，可以对该人脸样本图像进行人脸特征点检测，得到多个人脸特征点。比如，可以调用Dlib模块(该Dlib模块包含有预先训练好的人脸特征点检测模型)实现人脸特征点检测，得到68个人脸特征点，并对检测得到的68个人脸特征点进行标定，并为每一人脸特征点添加序号，进而实现在人脸样本图像上画出68个人脸特征点。68个人脸特征点的分布如图3所示。
步骤S404，获取多个口罩图像，每一口罩图像对应一种口罩类型。
在一实施方式中，口罩类型可以包括N95口罩、PM2.5口罩、活性炭口罩、棉布口罩、医用口罩等。可以通过网络从指定网站或者指定服务器获取多个口罩类型的口罩图像。也可以通过相机对多个口罩类型的口罩进行拍照，进而可 以获取多个口罩图像。
步骤S406，从多个所述人脸特征点中选取第一人脸特征点、第二人脸特征点、第三人脸特征点及第四人脸特征点，将所述第一人脸特征点与所述第二人脸特征点之间的距离作为口罩图像高度，并将所述第三人脸特征点与所述第四人脸特征点之间的距离作为口罩图像宽度。
在一实施方式中，第一人脸特征点、第二人脸特征点、第三人脸特征点及第四人脸特征点可以根据实际人脸识别需求进行设定。比如，如图3所示，可以将序号9的人脸特征点设定为第一人脸特征点，将序号30的人脸特征点设定为第二人脸特征点，进而序号9的人脸特征点与序号30的人脸特征点之间的距离定义为口罩图像高度；将序号5的人脸特征点设定为第三人脸特征点，将序号13的人脸特征点设定为第三人脸特征点，进而序号5的人脸特征点与序号13的人脸特征点之间的距离定义为口罩图像宽度。在其他实施例中，也可以将将序号9的人脸特征点设定为第一人脸特征点，将序号29的人脸特征点设定为第二人脸特征点，将序号4的人脸特征点设定为第三人脸特征点，将序号14的人脸特征点设定为第三人脸特征点。
步骤S408，依据所述口罩图像高度与所述口罩图像宽度对每一所述口罩图像进行尺寸调整。
在一实施方式中，当确定了口罩图像高度与口罩图像宽度时，可以根据口罩图像高度与口罩图像宽度对每一口罩图像进行尺寸调整。比如，可以根据口罩图像高度与口罩图像宽度对每一所述口罩图像进行缩放处理，以使得每一口罩图像的高度等于该口罩图像高度，每一口罩图像的宽度等于该口罩图像宽度。在其他实施方式中，不限定将每一口罩图像的高度调整为等于口罩图像高度，将每一口罩图像的宽度调整为等于该口罩图像宽度，也可以是其他形式的尺寸调整，比如将每一口罩图像的高度调整为等于口罩图像高度，将每一口罩图像的宽度调整为等于该口罩图像宽度的1.1倍。
步骤S410，将经过尺寸调整的每一所述口罩图像分别与所述人脸样本图像进行融合，得到多个人脸口罩图像。
在一实施方式中，当完成对口罩图像的尺寸调整时，可以将经过尺寸调整的每一口罩图像分别与人脸样本图像进行融合，得到多个人脸口罩图像。比如，从预设人脸样本库中提取人脸样本图像I1，对获取的N95口罩图像I2、PM2.5 口罩图像I3、活性炭口罩图像I4、棉布口罩图像I5及医用口罩图像I6进行了尺寸调整。可以将人脸样本图像I1与尺寸调整后的N95口罩图像I2进行融合得到一新的人脸样本图像I11，将人脸样本图像I1与尺寸调整后的PM2.5口罩图像I3进行融合得到一新的人脸样本图像I12，将人脸样本图像I1与尺寸调整后的活性炭口罩图像I4进行融合得到一新的人脸样本图像I13，将人脸样本图像I1与尺寸调整后的棉布口罩图像I5进行融合得到一新的人脸样本图像I14，及将人脸样本图像I1与尺寸调整后的医用口罩图像I6进行融合得到一新的人脸样本图像I15。对于预设人脸样本库中的每一人脸样本图像，均可以分别与N95口罩图像I2、PM2.5口罩图像I3、活性炭口罩图像I4、棉布口罩图像I5及医用口罩图像I6进行融合得到5张人脸口罩图像。
在一实施方式中，可以利用OpenCV将经过尺寸调整的每一口罩图像分别与人脸样本图像进行无缝融合，得到多个人脸口罩图像。在其他实施例中，也可以其他图像融合算法将经过尺寸调整的每一口罩图像分别与人脸样本图像进行融合。
步骤S412，将多个所述人脸口罩图像保存至所述预设人脸样本库。
在一实施方式中，当融合得到多个人脸口罩图像时，可以将该多个人脸口罩图像保存至预设人脸样本库，使得后续进行人脸模型训练时，可以基于有佩戴口罩的人脸图像及未佩戴口罩的人脸图像进行模型训练。
步骤S414，基于所述预设人脸样本库训练得到人脸识别模型，以进行人脸识别。
在一实施方式中，当预设人脸样本库中的每一人脸样本图像均完成与多个口罩图像的图像融合，且被保存至预设人脸样本库时，可以基于该预设人脸样本库训练得到所述人脸识别模型。
在一实施方式中，可以利用预设人脸样本库对预设神经网络(比如卷积神经网络/循环神经网络)进行训练，得到所述人脸识别模型。当得到所述人脸识别模型后，待识别的人脸图像输入至人脸识别模型可以得到多个人脸特征向量，比如可以得到612个人脸特征向量，以进行人脸匹配。人脸识别模型的训练方式可以采用现有的人脸识别模型的训练技术，在此不再详述。
在一实施方式中，为了实现在进行人脸识别过程中，同时显示每一人脸的名字。可以将所述预设人脸样本库(包含佩戴口罩的人脸样本图像及未佩戴口 罩的人脸样本图像)中的每一人脸样本图像与人脸身份信息建立映射关系，以使得每一人脸样本图像携带有人脸身份标签。基于携带有人脸身份标签的人脸样本图像训练得到人脸识别模型，进而训练得到的人脸识别模型可以在对待识别的人脸进行人脸识别时，可以同时显示待识别人脸的名字。若是未知的人脸，可以统一显示“陌生人”或者“Unknown”。
在一实施方式中，为了实现在进行人脸识别过程中，同时显示每一人脸是否佩戴口罩的信息。可以将所述预设人脸样本库(包含佩戴口罩的人脸样本图像及未佩戴口罩的人脸样本图像)中的每一人脸样本图像与口罩佩戴信息建立映射关系，以使得每一人脸样本图像携带有口罩佩戴标签。基于携带有口罩佩戴标签的人脸样本图像训练得到人脸识别模型，进而训练得到的人脸识别模型可以在对待识别的人脸进行人脸识别时，可以同时显示是否佩戴口罩的信息。
在一实施方式中，上述人脸识别程序30可以对不同平台的人脸识别系统的人脸样本库进行扩增，合成戴口罩的人脸样本库，再将戴口罩的人脸样本库导入到原有人脸样本库中，使得该些人脸识别系统可以成功识别戴口罩的人脸身份信息。比如，可以是神经运算棒(Neural Compute Stick，NCS)人脸识别系统平台、Dlib人脸识别系统平台、Azure人脸识别系统平台。
在一实施方式中，当基于该预设人脸样本库训练得到人脸识别模型时，可以对该人脸识别模型进行量化与压缩等方式的处理，使得训练得到的人脸识别模型可以集成到目标终端。
上述人脸识别装置、方法及非易失性可读存储介质，可实现在人们佩戴口罩的情形下准确地进行人脸识别，提升用户人脸识别体验。
对本领域的技术人员来说，可以根据本申请的发明方案和发明构思结合生产的实际需要做出其他相应的改变或调整，而这些改变和调整都应属于本申请所公开的范围。

Claims (10)

  1. 一种人脸识别方法，其特征在于，所述方法包括：
    从预设人脸样本库中提取人脸样本图像；
    对所述人脸样本图像进行人脸特征点检测，得到多个人脸特征点；
    获取多个口罩图像，其中每一所述口罩图像对应一种口罩类型；
    从多个所述人脸特征点中选取第一人脸特征点、第二人脸特征点、第三人脸特征点及第四人脸特征点，将所述第一人脸特征点与所述第二人脸特征点之间的距离作为口罩图像高度，并将所述第三人脸特征点与所述第四人脸特征点之间的距离作为口罩图像宽度；
    依据所述口罩图像高度与所述口罩图像宽度对每一所述口罩图像进行尺寸调整；
    将经过尺寸调整的每一所述口罩图像分别与所述人脸样本图像进行融合，得到多个人脸口罩图像；
    将多个所述人脸口罩图像保存至所述预设人脸样本库；
    基于所述预设人脸样本库训练得到人脸识别模型，以进行人脸识别。
  2. 如权利要求1所述的人脸识别方法，其特征在于，所述预设人脸样本库中的人脸样本图像包括多个人脸身份，每一所述人脸身份包括至少一张所述人脸样本图像，所述基于所述预设人脸样本库训练得到人脸识别模型的步骤，包括：
    当所述预设人脸样本库中的每一人脸样本图像均完成与多个所述口罩图像的图像融合时，基于所述预设人脸样本库训练得到所述人脸识别模型。
  3. 如权利要求2所述的人脸识别方法，其特征在于，所述基于所述预设人脸样本库训练得到所述人脸识别模型的步骤之前，还包括：
    将所述预设人脸样本库中的每一所述人脸样本图像与人脸身份信息建立映射关系，以使得每一所述人脸样本图像携带有人脸身份标签。
  4. 如权利要求2或3所述的人脸识别方法，其特征在于，所述基于所述预设人脸样本库训练得到所述人脸识别模型的步骤之前，还包括：
    将所述预设人脸样本库中的每一所述人脸样本图像与口罩佩戴信息建立映 射关系，以使得每一所述人脸样本图像携带有口罩佩戴标签。
  5. 如权利要求1所述的人脸识别方法，其特征在于，所述对所述人脸样本图像进行人脸特征点检测，得到多个人脸特征点的步骤，包括：
    利用预设人脸特征检测模型对所述人脸样本图像进行人脸特征点检测；
    对检测得到的多个所述人脸特征点进行标定，并为每一所述人脸特征点添加序号，以分别选取指定序号的人脸特征点作为所述第一人脸特征点、所述第二人脸特征点、所述第三人脸特征点及所述第四人脸特征点。
  6. 如权利要求1所述的人脸识别方法，其特征在于，所述依据所述口罩图像高度与所述口罩图像宽度对每一所述口罩图像进行尺寸调整的步骤，包括：
    对每一所述口罩图像进行缩放处理，以使得每一所述口罩图像的高度等于所述口罩图像高度，每一所述口罩图像的宽度等于所述口罩图像宽度。
  7. 如权利要求1所述的人脸识别方法，其特征在于，所述将经过尺寸调整的每一所述口罩图像分别与所述人脸样本图像进行融合，得到多个人脸口罩图像的步骤，包括：
    利用OpenCV将经过尺寸调整的每一所述口罩图像分别与所述人脸样本图像进行融合，得到多个所述人脸口罩图像。
  8. 如权利要求1所述的人脸识别方法，其特征在于，所述从预设人脸样本库中提取人脸样本图像的步骤之前，还包括：
    对所述预设人脸样本库中的每一人脸样本图像进行预处理，其中所述预处理包括以下一种或多种的组合：人脸图像扶正处理、人脸图像增强处理、人脸图像归一化处理。
  9. 一种人脸识别装置，所述装置包括处理器及存储器，所述存储器上存储有若干计算机可读指令，其特征在于，所述处理器用于执行存储器中存储的计算机可读指令时实现如权利要求1-8任一项所述的人脸识别方法的步骤。
  10. 一种非易失性可读存储介质，其特征在于，所述非易失性可读存储介质存储有多条指令，多条所述指令可被一个或者多个处理器执行，以实现如权利要求1-8任一项所述的人脸识别方法的步骤。
PCT/CN2020/120265 2020-10-12 2020-10-12 人脸识别方法、装置及可读存储介质 WO2022077139A1 (zh)

Priority Applications (3)

Application Number Priority Date Filing Date Title
CN202080003091.8A CN114698399A (zh) 2020-10-12 2020-10-12 人脸识别方法、装置及可读存储介质
PCT/CN2020/120265 WO2022077139A1 (zh) 2020-10-12 2020-10-12 人脸识别方法、装置及可读存储介质
US17/598,453 US11922724B2 (en) 2020-10-12 2020-10-12 Face recognition method utilizing a face recognition model and a face sample library to detect mask images

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2020/120265 WO2022077139A1 (zh) 2020-10-12 2020-10-12 人脸识别方法、装置及可读存储介质

Publications (1)

Publication Number Publication Date
WO2022077139A1 true WO2022077139A1 (zh) 2022-04-21

Family

ID=81207512

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/120265 WO2022077139A1 (zh) 2020-10-12 2020-10-12 人脸识别方法、装置及可读存储介质

Country Status (3)

Country Link
US (1) US11922724B2 (zh)
CN (1) CN114698399A (zh)
WO (1) WO2022077139A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
TWI818824B (zh) * 2022-12-07 2023-10-11 財團法人工業技術研究院 用於計算遮蔽人臉影像的人臉擺動方向的裝置及方法

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115578777B (zh) * 2022-11-10 2023-03-14 成都智元汇信息技术股份有限公司 一种基于空间映射获得目标的识图方法及装置

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20200019758A1 (en) * 2018-07-16 2020-01-16 Adobe Inc. Meta-learning for facial recognition
CN111460962A (zh) * 2020-03-27 2020-07-28 武汉大学 一种口罩人脸识别方法及系统
CN111582199A (zh) * 2020-05-12 2020-08-25 佛山市玖章智能科技有限公司 一种人脸识别模型训练方法和人脸识别方法
CN111582068A (zh) * 2020-04-22 2020-08-25 北京交通大学 人员口罩佩戴状态检测方法

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP4708909B2 (ja) * 2005-08-09 2011-06-22 富士フイルム株式会社 デジタル画像の対象物検出方法および装置並びにプログラム
CN107844744A (zh) * 2017-10-09 2018-03-27 平安科技（深圳）有限公司 结合深度信息的人脸识别方法、装置及存储介质
CN108875534B (zh) * 2018-02-05 2023-02-28 北京旷视科技有限公司 人脸识别的方法、装置、系统及计算机存储介质
CN109815845B (zh) * 2018-12-29 2022-03-29 达闼机器人有限公司 人脸识别的方法和装置、存储介质
US10990807B2 (en) * 2019-09-06 2021-04-27 Adobe, Inc. Selecting representative recent digital portraits as cover images
CN111695431A (zh) * 2020-05-19 2020-09-22 深圳禾思众成科技有限公司 一种人脸识别方法、装置、终端设备及存储介质

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20200019758A1 (en) * 2018-07-16 2020-01-16 Adobe Inc. Meta-learning for facial recognition
CN111460962A (zh) * 2020-03-27 2020-07-28 武汉大学 一种口罩人脸识别方法及系统
CN111582068A (zh) * 2020-04-22 2020-08-25 北京交通大学 人员口罩佩戴状态检测方法
CN111582199A (zh) * 2020-05-12 2020-08-25 佛山市玖章智能科技有限公司 一种人脸识别模型训练方法和人脸识别方法

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
TWI818824B (zh) * 2022-12-07 2023-10-11 財團法人工業技術研究院 用於計算遮蔽人臉影像的人臉擺動方向的裝置及方法

Also Published As

Publication number Publication date
US20220327864A1 (en) 2022-10-13
CN114698399A (zh) 2022-07-01
US11922724B2 (en) 2024-03-05

Similar Documents

Publication Publication Date Title
US10817705B2 (en) Method, apparatus, and system for resource transfer
CN110232369B (zh) 一种人脸识别方法和电子设备
WO2018028546A1 (zh) 一种关键点的定位方法及终端、计算机存储介质
EP2336949B1 (en) Apparatus and method for registering plurality of facial images for face recognition
TW201911130A (zh) 一種翻拍影像識別方法及裝置
WO2021051611A1 (zh) 基于人脸可见性的人脸识别方法、系统、装置及存储介质
WO2019033572A1 (zh) 人脸遮挡检测方法、装置及存储介质
KR20190096703A (ko) 영상 융합 방법 및 장치
WO2022077139A1 (zh) 人脸识别方法、装置及可读存储介质
CN111753643B (zh) 人物姿态识别方法、装置、计算机设备和存储介质
US20170004369A1 (en) Object recognition apparatus and control method therefor
JP6071002B2 (ja) 信頼度取得装置、信頼度取得方法および信頼度取得プログラム
JP2022133378A (ja) 顔生体検出方法、装置、電子機器、及び記憶媒体
JP6969878B2 (ja) 識別器学習装置および識別器学習方法
CN111353364A (zh) 一种人脸动态识别方法及装置、电子设备
CN113642639A (zh) 活体检测方法、装置、设备和存储介质
JP2019533269A (ja) 人間の強膜および瞳孔に基づいてデジタル画像の色を補正するシステムおよび方法
CN110633677B (zh) 人脸识别的方法及装置
CN111881740A (zh) 人脸识别方法、装置、电子设备及介质
US11620728B2 (en) Information processing device, information processing system, information processing method, and program
WO2020244076A1 (zh) 人脸识别方法、装置、电子设备及存储介质
CN115115552B (zh) 图像矫正模型训练及图像矫正方法、装置和计算机设备
CN112101479B (zh) 一种发型识别方法及装置
US20220319232A1 (en) Apparatus and method for providing missing child search service based on face recognition using deep-learning
JP2019113959A (ja) 鼻紋照合システム、鼻紋照合方法及び鼻紋照合プログラム

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20956921

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20956921

Country of ref document: EP

Kind code of ref document: A1