CN114241542A - Face recognition method based on image stitching - Google Patents

Face recognition method based on image stitching Download PDF

Info

Publication number
CN114241542A
CN114241542A CN202111118259.2A CN202111118259A CN114241542A CN 114241542 A CN114241542 A CN 114241542A CN 202111118259 A CN202111118259 A CN 202111118259A CN 114241542 A CN114241542 A CN 114241542A
Authority
CN
China
Prior art keywords
face
image
mask
face recognition
recognition method
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202111118259.2A
Other languages
Chinese (zh)
Inventor
胡建华
余君
曾文权
杨忠明
陈广意
张嘉诚
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Guangdong Institute of Science and Technology
Original Assignee
Guangdong Institute of Science and Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Guangdong Institute of Science and Technology filed Critical Guangdong Institute of Science and Technology
Priority to CN202111118259.2A priority Critical patent/CN114241542A/en
Publication of CN114241542A publication Critical patent/CN114241542A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00Image enhancement or restoration
    • G06T5/50Image enhancement or restoration using two or more images, e.g. averaging or subtraction
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/10Segmentation; Edge detection
    • G06T7/136Segmentation; Edge detection involving thresholding
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/10Segmentation; Edge detection
    • G06T7/155Segmentation; Edge detection involving morphological operators
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/10Segmentation; Edge detection
    • G06T7/194Segmentation; Edge detection involving foreground-background segmentation
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20081Training; Learning
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20212Image combination
    • G06T2207/20221Image fusion; Image merging

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Collating Specific Patterns (AREA)

Abstract

The invention discloses a face recognition method, a face recognition device and a face recognition medium based on image splicing, wherein the method comprises the following steps: determining whether the face is worn with a mask or not by distance detection on the face image; performing morphological processing on the face image wearing the mask, and replacing the area shielded by the mask with the existing corresponding area image to obtain a complete face image; and performing face matching and recognition on the complete face image. The invention has the following beneficial effects: the face image complementing method is used for complementing the face image when the face area is blocked by the mask or other objects to obtain a complete face image, and the face data used for detailed matching is used, so that the problem that most of characteristic information of the blocked face is lost is solved.

Description

Face recognition method based on image stitching
Technical Field
The invention relates to the field of computer graphic processing and artificial intelligence, in particular to a face recognition method, a face recognition device and a face recognition medium based on image splicing.
Background
The traditional face recognition algorithm mainly comprises three modules: face Detection (Face Detection), Face Alignment (Face Alignment), and Face Feature characterization (features Representations).
The input received by the human face feature characterization is a standardized human face image, vectorized human face features are obtained through feature modeling, and finally the recognition result is obtained through the discrimination of a classifier. Because the existing face recognition algorithm adopts a complete face image for training and recognition, the face shielded by a mask cannot obtain complete characteristic information, such as nose shape and mouth information, and the result is difficult to recognize under the condition of lacking a large amount of characteristic information.
The traditional method for identifying the shielding face comprises three types of subspace regression, robust error coding, robust feature extraction and the like. The subspace regression method divides different classes of faces into different subspaces, the occlusion is an independent subspace, then the occluded face image is the superposition of the face without the occlusion and the occlusion, the occluded face image recognition problem can be regarded as the problem that the non-occluded face image and the occlusion are respectively regressed to the subspaces to which the non-occluded face image and the occlusion belong, the most representative methods in the subspace regression method are a sparse representation classification method and a collaborative representation method, and the difficulty of the subspace regression method lies in the construction of the occlusion subspace.
The robust error coding method mainly comprises an addition model and a multiplication model. The "additive model" considers that an occluded image is a composite of an original facial image without occlusion and an error e caused by occlusion, i.e., y is y0+ e, and the important consideration is how to separate the error e from y. The "multiplicative model" sees the occluded image as a concatenation of the unoccluded y0 and the occlusion, and only y0 can be accurately reconstructed. It is important to consider how to separate its occluded and non-occluded regions.
The robust feature extraction method is characterized in that features contained in a face image are usually very rich, and include low-order features such as color, brightness, texture and direction, and high-order features such as posture, expression, age and race, and the robust feature extraction method needs to decompose the features.
The occlusion face recognition algorithms have advantages, but when most feature information of the face is occluded by a mask, an accurate recognition result is difficult to obtain, and most algorithms cannot detect the face, so that the next face recognition cannot be performed. The invention uses face characteristics to replace part of information of the mask, thereby being capable of carrying out normal face detection and recognition under the shielding condition.
The prior art also adopts a face with shielding and a face without shielding as training sets to retrain and develop a new face recognition method, the method can effectively improve the face recognition accuracy under the shielding condition, and the defects are that a new algorithm needs to be retrained, and the development difficulty is high.
After the mask is worn, the information of the five sense organs such as the nose, the mouth and the like is shielded, the information which can be used for distinguishing the face of the human face can be greatly reduced, and the recognizable information such as the outline of the face and the like also changes greatly in physical distribution.
Disclosure of Invention
The present invention is directed to solving at least one of the problems of the prior art. Therefore, the invention provides a face recognition method, a face recognition device and a face recognition medium based on image splicing, which are used for obtaining a complete face image of a wearer wearing a mask and using the used face data for detailed matching, so that the problem that most of feature information of a shielded face is lost is solved.
The technical scheme of the invention comprises a face recognition method based on image splicing, which is characterized by comprising the following steps: s100, collecting face images through a camera device, and determining whether a face wears a mask or not through distance detection on the face images; s200, performing morphological processing on the face image wearing the mask, and replacing the area shielded by the mask with the existing corresponding area image to obtain a complete face image; and S300, performing face matching and recognition on the complete face image.
According to the face recognition method based on image stitching, S100 includes: the image capture device is used for capturing images, adjusting the size of the images to be used for detecting the mask, detecting whether the mask is worn by a target through a Yolov3 target object detection algorithm, and if the mask is detected, acquiring the image of the mask with the face being blocked.
According to the face recognition method based on image stitching, S100 further includes: and (3) positioning the mask, converting the color space of the picture into hsv from rgb, removing the color of a white part, and setting a threshold value to remove the background image.
According to the face recognition method based on image stitching, S200 includes: and performing opening operation, determining the maximum outline of the mask, saving the mask outline picture and determining the size of the replaced area, wherein the opening operation comprises the steps of performing corrosion treatment on the image and then performing expansion treatment on the image.
According to the face recognition method based on image stitching, S200 further includes: acquiring a mouth image from the existing face data, confirming the size and width of the face, adjusting the mouth image to cover on the mask according to the size of the face, adjusting to an optimal position coordinate, and intercepting and acquiring a spliced part on the portrait image by utilizing the coordinate; and adjusting the size of the intercepted mouth image according to the size of the mask, and then splicing to obtain a complete face image.
According to the face recognition method based on image stitching, the temperature S300 comprises the following steps: and matching and identifying the complete face image with the face image stored in the database, and outputting a corresponding matching identification result.
The technical scheme of the invention also comprises a face recognition device based on image splicing, which comprises a camera device, a memory, a processor and a computer program which is stored in the memory and can run on the processor, and is characterized in that any one of the method steps is realized when the processor executes the computer program.
The present invention also includes a computer-readable storage medium, in which a computer program is stored, wherein the computer program, when executed by a processor, implements any of the method steps.
The invention has the beneficial effects that:
(1) the face data is used for processing the shielded face image, and even under the condition of having a small amount of face information, the identity information of the shielded person can be identified;
(2) firstly, mask detection is carried out through a detection algorithm, face recognition under the condition of no mask is compatible, higher accuracy is achieved, and if the mask is detected, face information shielded by the mask is restored and recovered by using face information of an existing database;
(3) because the existing face recognition is a face recognition method based on the non-shielding condition, the face cannot be detected under the shielding condition of a mask, and the other face recognition algorithm fails, so that the algorithm can be compatible with the existing face recognition algorithm through spliced complete face information, and the face can be correctly recognized;
(4) when the face recognition is compared, the face information part of the mask is replaced, and the information of the mask part in the database is used for each matching, and the information of the mask part is the same and has no difference, so that the face recognition algorithm applies the focus to the face characteristic information without the mask, and the accuracy rate of the face recognition of the mask is higher than that of the face recognition of the mask based on deep learning training;
(5) the mask part of the face mask is processed, the nose part and the mouth part are spliced to obtain the characteristic information of the whole face, the characteristic information of the shielded part is completed for face identification, and the accuracy of the face is greatly improved;
(6) in public places, the face recognition can be carried out without taking off the mask, so that the crowd gathering is reduced, and the safety and the detection efficiency of public spaces are greatly improved;
(7) the face recognition method can be used for security scenes, and the identity of the target can still be recognized under the condition that the face is shielded by the mask.
Drawings
The above and/or additional aspects and advantages of the present invention will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings of which:
fig. 1a, 1b, 1c, 1d, 1e are schematic diagrams illustrating a face recognition process of a mask wearing type according to an embodiment of the present invention;
FIG. 2 illustrates an overall flow diagram according to an embodiment of the invention;
FIG. 3 is a detailed flow diagram according to an embodiment of the present invention;
FIG. 4 is a schematic diagram of an apparatus and medium according to an embodiment of the invention.
Detailed Description
Reference will now be made in detail to embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to the same or similar elements or elements having the same or similar function throughout. The embodiments described below with reference to the accompanying drawings are illustrative only for the purpose of explaining the present invention, and are not to be construed as limiting the present invention.
In the description of the present invention, the meaning of a plurality of means is one or more, the meaning of a plurality of means is two or more, and larger, smaller, larger, etc. are understood as excluding the number, and larger, smaller, inner, etc. are understood as including the number.
Fig. 1a, 1b, 1c, 1d, 1e are schematic diagrams illustrating a face recognition process of a mask wearing type according to an embodiment of the present invention. Which refers in particular to the flow chart of figure 3.
Fig. 2 shows a general flow diagram according to an embodiment of the invention. The method comprises the following steps: s100, collecting face images through a camera device, and determining whether a face wears a mask or not through distance detection on the face images; s200, performing morphological processing on the face image wearing the mask, and replacing the area shielded by the mask with the existing corresponding area image to obtain a complete face image; and S300, performing face matching and recognition on the complete face image.
Fig. 3 is a detailed flowchart according to an embodiment of the present invention. The specific process is as follows: capturing a face image from a camera according to frames, carrying out mask detection on a mask through a Yolo v3 algorithm, obtaining the image information of the mask with the face shielded, carrying out morphological operation processing on the shielded partial image, intercepting the existing face nose and mouth image to replace the mask part, restoring the whole face, obtaining relatively complete face characteristic information, and finally carrying out face recognition on the spliced face to be matched with the existing face data.
The method comprises the steps of detecting whether a target wears a mask or not by using a Yolov3 target object detection algorithm, acquiring image information of the mask with the human face being blocked if the mask is detected, performing morphological operation processing on a blocked partial image, intercepting an existing human face mouth image to replace the mask part, and splicing the image information with facial feature information of eyes, heads and the like of an original image, so that the whole human face can be restored, relatively complete human face data is obtained to be matched with the existing human face data, and identity information of the blocked human face is identified.
The method comprises the steps of starting a camera to capture an image according to a frame, primarily adjusting the size of the image for mask detection, locating the position of a mask if the mask is detected, converting the color space of the image into hsv from rgb, removing a white part from the color, setting a threshold value (removing a background part), processing the mask part, performing morphological conversion on the mask part, performing opening operation (firstly corroding and then expanding), determining the maximum outline of the mask, and storing the mask outline image (determining the size of a replaced area) (see fig. 1c and fig. 1 d).
Acquiring a mouth image (as shown in figure 1a) from the existing face data, confirming the size and width of the face, adjusting the mouth image to be covered on the mask (as shown in figure 1b) according to the size of the face, adjusting to an optimal position coordinate, and intercepting and acquiring a spliced part on the portrait image by utilizing the coordinate. Then, the size of the intercepted mouth image is adjusted according to the size of the mask, and then, the mouth image is spliced to obtain a relatively complete face image, and face recognition is performed on the processed face image (as shown in fig. 1 e).
FIG. 4 is a schematic diagram of an apparatus and medium according to an embodiment of the invention. The apparatus comprises a memory 100, a processor 200 and an acquisition device 300, wherein the processor 200 stores a computer program for executing: determining whether the face is worn with a mask or not by distance detection on the face image; performing morphological processing on the face image wearing the mask, and replacing the area shielded by the mask with the existing corresponding area image to obtain a complete face image; and performing face matching and recognition on the complete face image. The acquisition device 300 is used for acquiring a face image, and the memory 100 is used for storing data.
The embodiments of the present invention have been described in detail with reference to the accompanying drawings, but the present invention is not limited to the above embodiments, and various changes can be made within the knowledge of those skilled in the art without departing from the gist of the present invention.

Claims (8)

1. A face recognition method based on image stitching is characterized by comprising the following steps:
s100, collecting face images through a camera device, and determining whether a face wears a mask or not through distance detection on the face images;
s200, performing morphological processing on the face image wearing the mask, and replacing the area shielded by the mask with the existing corresponding area image to obtain a complete face image;
and S300, performing face matching and recognition on the complete face image.
2. The image stitching-based face recognition method according to claim 1, wherein the S100 comprises:
the image capture device is used for capturing images, adjusting the size of the images to be used for detecting the mask, detecting whether the mask is worn by a target through a Yolov3 target object detection algorithm, and if the mask is detected, acquiring the image of the mask with the face being blocked.
3. The image stitching-based face recognition method according to claim 2, wherein the S100 further comprises:
and (3) positioning the mask, converting the color space of the picture into hsv from rgb, removing the color of a white part, and setting a threshold value to remove the background image.
4. The image stitching-based face recognition method according to claim 2, wherein the S200 comprises:
and performing opening operation, determining the maximum outline of the mask, saving the mask outline picture and determining the size of the replaced area, wherein the opening operation comprises the steps of performing corrosion treatment on the image and then performing expansion treatment on the image.
5. The image stitching-based face recognition method according to claim 4, wherein the step S200 further comprises:
acquiring a mouth image from the existing face data, confirming the size and width of the face, adjusting the mouth image to cover on the mask according to the size of the face, adjusting to an optimal position coordinate, and intercepting and acquiring a spliced part on the portrait image by utilizing the coordinate;
and adjusting the size of the intercepted mouth image according to the size of the mask, and then splicing to obtain a complete face image.
6. The image stitching-based face recognition method according to claim 5, wherein the temperature S300 comprises:
and matching and identifying the complete face image with the face image stored in the database, and outputting a corresponding matching identification result.
7. An image stitching based face recognition apparatus comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor when executing the computer program implements the method steps of any of claims 1 to 6.
8. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the method steps of any one of claims 1 to 6.
CN202111118259.2A 2021-09-23 2021-09-23 Face recognition method based on image stitching Pending CN114241542A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111118259.2A CN114241542A (en) 2021-09-23 2021-09-23 Face recognition method based on image stitching

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111118259.2A CN114241542A (en) 2021-09-23 2021-09-23 Face recognition method based on image stitching

Publications (1)

Publication Number Publication Date
CN114241542A true CN114241542A (en) 2022-03-25

Family

ID=80743007

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111118259.2A Pending CN114241542A (en) 2021-09-23 2021-09-23 Face recognition method based on image stitching

Country Status (1)

Country Link
CN (1) CN114241542A (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115063863A (en) * 2022-06-27 2022-09-16 中国平安人寿保险股份有限公司 Face recognition method and device, computer equipment and storage medium
CN115619410A (en) * 2022-10-19 2023-01-17 闫雪 Self-adaptive financial payment platform
CN115810214A (en) * 2023-02-06 2023-03-17 广州市森锐科技股份有限公司 Verification management method, system, equipment and storage medium based on AI face recognition

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115063863A (en) * 2022-06-27 2022-09-16 中国平安人寿保险股份有限公司 Face recognition method and device, computer equipment and storage medium
CN115619410A (en) * 2022-10-19 2023-01-17 闫雪 Self-adaptive financial payment platform
CN115619410B (en) * 2022-10-19 2024-01-26 闫雪 Self-adaptive financial payment platform
CN115810214A (en) * 2023-02-06 2023-03-17 广州市森锐科技股份有限公司 Verification management method, system, equipment and storage medium based on AI face recognition
CN115810214B (en) * 2023-02-06 2023-05-12 广州市森锐科技股份有限公司 AI-based face recognition verification management method, system, equipment and storage medium

Similar Documents

Publication Publication Date Title
Liu et al. Learning deep models for face anti-spoofing: Binary or auxiliary supervision
CN114241542A (en) Face recognition method based on image stitching
CN109389074B (en) Facial feature point extraction-based expression recognition method
CN108629336B (en) Face characteristic point identification-based color value calculation method
EP1255225A2 (en) Method for detecting eye and mouth positions in a digital image
CN104951773A (en) Real-time face recognizing and monitoring system
CN103116749A (en) Near-infrared face identification method based on self-built image library
CN105046219A (en) Face identification system
CN110059634B (en) Large-scene face snapshot method
JP2009069996A (en) Image processing device and image processing method, recognition device and recognition method, and program
CN111539911B (en) Mouth breathing face recognition method, device and storage medium
CN112541422A (en) Expression recognition method and device with robust illumination and head posture and storage medium
CN112115775A (en) Smoking behavior detection method based on computer vision in monitoring scene
CN111611849A (en) Face recognition system for access control equipment
CN113673607A (en) Method and device for training image annotation model and image annotation
CN113158883A (en) Face recognition method, system, medium and terminal based on regional attention
Xu et al. Identity-constrained noise modeling with metric learning for face anti-spoofing
CN117475353A (en) Video-based abnormal smoke identification method and system
CN114663985A (en) Face silence living body detection method and device, readable storage medium and equipment
CN108564020B (en) Micro-gesture recognition method based on panoramic 3D image
CN114360033A (en) Mask face recognition method, system and equipment based on image convolution fusion network
CN114170069A (en) Automatic eye closing processing method based on continuous multiple pictures
Paul et al. Automatic adaptive facial feature extraction using CDF analysis
CN111523406A (en) Deflection face correcting method based on generation of confrontation network improved structure
US20230377315A1 (en) Learning method, learned model, detection system, detection method, and program

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination