CN111523383B - Non-perception face recognition system and method based on pedestrian ReID - Google Patents
Non-perception face recognition system and method based on pedestrian ReID Download PDFInfo
- Publication number
- CN111523383B CN111523383B CN202010195478.XA CN202010195478A CN111523383B CN 111523383 B CN111523383 B CN 111523383B CN 202010195478 A CN202010195478 A CN 202010195478A CN 111523383 B CN111523383 B CN 111523383B
- Authority
- CN
- China
- Prior art keywords
- face
- human body
- pedestrian
- features
- new person
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/161—Detection; Localisation; Normalisation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/168—Feature extraction; Face representation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/172—Classification, e.g. identification
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- General Health & Medical Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Processing (AREA)
- Image Analysis (AREA)
- Collating Specific Patterns (AREA)
Abstract
The invention discloses a pedestrian reiD-based non-perception face recognition system and a recognition method, wherein the system comprises: the video stream acquisition module is used for acquiring a video frame image; the human body detection module is used for carrying out human body detection and human body feature extraction on a video frame image, then carrying out feature matching on the human body features extracted from the current frame and the human body features extracted from the previous frame of the current frame, and comparing the successfully matched human body features with the human body features stored in a pedestrian ReID (identification) library to obtain a pedestrian ID comparison result; and the face detection module is used for further extracting the face features of the successfully matched human body, and then comparing the extracted face features with the faces stored in a face library one by one to obtain a face comparison result. The method and the device detect the human body characteristics and the face characteristics of the pedestrian by utilizing the relevance of the human body characteristics in the front frame image and the back frame image, and reduce the missing rate of the pedestrian.
Description
Technical Field
The invention relates to the technical field of computer vision recognition, in particular to a pedestrian ReID-based non-perception face recognition system and a pedestrian ReID-based non-perception face recognition method.
Background
The existing face recognition method mainly realizes the recognition of the face through the detection and comparison of the face. However, the existing face recognition method mainly has the following two problems:
when the pedestrian is exposed at intervals or the exposed face cannot be identified due to shooting angles and the like, the detection omission of the pedestrian is likely to happen. Therefore, the existing face recognition method can accurately recognize the face only by properly matching detected personnel;
secondly, the face recognition result of the preamble frame in a short time cannot be utilized in the recognition process of the subsequent frame, that is, the face detection process of each frame image of the video stream is independent by the existing face detection method, and the association possibly possessed by the pedestrians in the two frames of images of the preamble frame and the subsequent frame is not utilized, so that the recognition rate of the face detection is low, and the false detection is easy to occur.
Disclosure of Invention
The invention aims to provide an imperceptible face recognition system and a recognition method based on pedestrian ReID, so as to solve the technical problems.
In order to achieve the purpose, the invention adopts the following technical scheme:
provided is a pedestrian ReID-based non-perception face recognition system, comprising:
the video stream acquisition module is used for acquiring a video frame image;
the human body detection module is connected with the video stream acquisition module and used for detecting a human body of the video frame image, extracting human body characteristics of the detected human body, performing characteristic matching on the human body characteristics corresponding to the human body extracted from the current frame and the human body characteristics corresponding to each human body extracted from the previous frame of the current frame, and comparing the successfully matched human body characteristics with the human body characteristics of each pedestrian stored in a pedestrian ReID (ReID) library to obtain a pedestrian ID comparison result;
the face detection module is connected with the human body detection module and is used for further extracting face features of the human body which is successfully matched, and then comparing the extracted face features with faces stored in a face library one by one to obtain a face comparison result;
a result correction module, respectively connected to the human body detection module and the face detection module, for correcting the pedestrian ID corresponding to the suspected new person after being determined as the suspected new person in the previous frame according to the face comparison result made for the face feature in the current frame;
and the data updating module is connected with the result correcting module and used for updating the correction result made by the result correcting module into the pedestrian ReID library.
As a preferred scheme of the present invention, the human body detection module specifically includes:
the human body detection unit is used for carrying out human body detection on the video frame image through a preset target detection algorithm;
the human body image intercepting unit is connected with the human body detecting unit and is used for intercepting the detected human body image from the video frame image;
the human body feature extraction unit is connected with the human body image intercepting unit and used for inputting the intercepted human body image into a feature extraction network so as to extract the human body features on the human body image;
the human body feature matching unit is connected with the human body feature extraction unit and is used for performing feature matching on the human body features corresponding to the pedestrians extracted from the current frame and the human body features corresponding to the pedestrians extracted from the previous frame of the current frame to obtain a human body feature matching result;
the pedestrian ID giving unit is connected with the human body feature matching unit and is used for giving the pedestrian ID corresponding to the human body feature on the current frame participating in feature matching when the human body feature matching fails;
and the pedestrian characteristic comparison unit is connected with the human body characteristic matching unit and used for comparing the human body characteristics in the current frame participating in characteristic matching with the pedestrian characteristics stored in the pedestrian ReID library after the human body characteristics are successfully matched to obtain the comparison result of the pedestrian ID.
As a preferred scheme of the present invention, the face detection module specifically includes:
the human face detection unit is used for carrying out human face detection on the human body image to obtain a human face image;
the face feature extraction unit is connected with the face detection unit and used for extracting the face features on the face image;
the face comparison unit is connected with the face feature extraction unit and used for comparing the extracted face features with the faces stored in the face library one by one to obtain a face comparison result;
the face ID endowing unit is connected with the face comparison unit and is used for endowing the face ID corresponding to the face characteristics when the face comparison fails;
and the face ID acquisition unit is connected with the face comparison unit and used for acquiring the face ID corresponding to the face features from the face library after the face comparison is successful.
As a preferred aspect of the present invention, the result correction module specifically includes:
a first correction unit, configured to, when the face feature comparison of the suspected new person in the current frame is successful, obtain the pedestrian ID corresponding to the suspected new person from the pedestrian ReID library by using the face ID corresponding to the suspected new person, and then correct the pedestrian ID given to the suspected new person in the previous frame according to the obtained pedestrian ID;
a second correction unit, configured to, when the comparison of the face features of the suspected new person in the current frame fails, perform feature-by-feature comparison between the face features of the suspected new person and all the face features in the previous frame by extracting the face features corresponding to all the human bodies in the previous frame of the current frame,
if the comparison is successful, firstly, acquiring the face ID corresponding to the successfully-compared face features from the face library, then acquiring the pedestrian ID corresponding to the face ID from the pedestrian ReID library, and then correcting the pedestrian ID given to the suspected new person by using the acquired pedestrian ID;
and if the comparison is not successful, finally determining the suspected new person as the new person.
The invention also provides a non-perception face recognition method based on the pedestrian ReID, which is realized by the non-perception face recognition system, and the method comprises the following steps:
s1, acquiring the video frame image;
s2, detecting a human body of the video frame image, extracting human body features of the detected human body, performing feature matching on the human body features extracted from the current frame and all the human body features extracted from the previous frame of the current frame, and comparing the successfully matched human body features with all the pedestrian features stored in the pedestrian ReID library to obtain a pedestrian ID comparison result;
s3, further extracting the face features of the successfully matched human body, and comparing the extracted face features with the faces stored in the face library one by one to obtain a face comparison result;
step S4, correcting the pedestrian ID given to the suspected new person after the previous frame is determined as the suspected new person based on a preset correction method according to the face comparison result made for the face features in the current frame;
and step S5, updating the correction result made in the step S4 into the pedestrian ReID library.
As a preferable scheme of the present invention, in step S2, human body detection on the video frame image is implemented through a Yolo v3 target detection algorithm.
As a preferred embodiment of the present invention, in the step S2, the body features of the detected body image are extracted through a residual neural network ResNet 50.
As a preferable scheme of the present invention, in step S2, a specific method for performing feature matching on the human body features in the front and rear two frames of images is as follows:
performing inner product operation on a first human body feature vector corresponding to the human body features in the current frame and a second human body feature vector corresponding to each human body feature extracted from the previous frame one by one, then judging whether an inner product value larger than a preset threshold value exists in each inner product value obtained by calculation,
if so, selecting a maximum inner product value from the inner product values larger than the threshold value, and outputting the human body feature corresponding to the second human body feature vector or the first human body feature vector when the maximum inner product value is obtained through calculation as a matching result;
and if not, indicating that the human body feature matching fails.
As a preferred embodiment of the present invention, in step S3, the recognition and extraction of the face features are implemented by a RetinaFace face recognition algorithm.
As a preferable aspect of the present invention, in step S2, the specific method for acquiring the pedestrian ID corresponding to the human body feature includes:
step S21, carrying out human body detection on the video frame image;
s22, intercepting the detected human body image from the video frame image;
step S23, extracting the human body features on the human body image;
step S24, the human body features extracted from the current frame are matched with all the human body features extracted from the previous frame of the current frame,
if the matching is successful, performing feature comparison on the human body features in the current frame participating in the matching and the pedestrian features stored in the pedestrian ReID library to obtain a pedestrian ID comparison result;
and if the matching is unsuccessful, regarding the pedestrian corresponding to the human body characteristics involved in the matching as the suspected new person and giving the pedestrian ID corresponding to the suspected new person.
As a preferred solution of the present invention, in the step S3, the specific method for acquiring the face ID corresponding to the face feature includes:
step S31, carrying out face detection on the human body to obtain a face image;
step S32, extracting the face features on the face image;
step S33, comparing the extracted face features with the faces stored in the face library one by one,
if the comparison is successful, the face ID corresponding to the face features is obtained from the face library;
and if the comparison fails, giving the face ID corresponding to the face features.
As a preferable aspect of the present invention, in the step S4, the preset correction method includes a first correction method, and the process of correcting the pedestrian ID by the first correction method is as follows:
and when the face features of the suspected new person in the current frame are successfully compared, acquiring the pedestrian ID corresponding to the suspected new person from the pedestrian ReID library by using the face ID corresponding to the suspected new person, and then correcting the pedestrian ID given to the suspected new person in the previous frame according to the acquired pedestrian ID.
As a preferable aspect of the present invention, in the step S4, the preset correction method includes a second correction method, and the process of correcting the pedestrian ID by the second correction method is as follows:
when the comparison of the face features of the suspected new person in the current frame fails, all the face features in the previous frame of the current frame are extracted, and then the similarity comparison is carried out between the face features of the suspected new person and all the face features in the previous frame,
if the comparison is successful, firstly, acquiring the face ID corresponding to the successfully compared face features from the face library, then acquiring the pedestrian ID corresponding to the face ID from the pedestrian ReID library, and then correcting the pedestrian ID previously given to the suspected new person by using the acquired pedestrian ID;
and if the comparison is not successful, finally determining the suspected new person as the new person.
The invention utilizes the relevance of the human body characteristics of the pedestrians in the front and rear frame images to detect the human body characteristics and the human face characteristics of the pedestrians and correct the detection result, thereby not only reducing the missing detection rate of the pedestrians, but also improving the accuracy rate of the identification and detection of the pedestrians.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required to be used in the embodiments of the present invention will be briefly described below. It is obvious that the drawings described below are only some embodiments of the invention, and that for a person skilled in the art, other drawings can be derived from them without inventive effort.
Fig. 1 is a schematic structural diagram of a non-perceptual face recognition system according to an embodiment of the present invention;
fig. 2 is a schematic diagram of an internal structure of a human body detection module in a non-perceptual face recognition system according to an embodiment of the present invention;
fig. 3 is a schematic diagram of an internal structure of a face detection module in the non-perceptual face recognition system according to an embodiment of the present invention;
fig. 4 is a schematic diagram of an internal structure of a result correction module in the non-perceptual face recognition system according to an embodiment of the present invention;
FIG. 5 is a diagram illustrating the steps of a non-perceptual face recognition method according to an embodiment of the present invention;
FIG. 6 is a diagram of the steps of a method for obtaining the pedestrian ID corresponding to the human body characteristic according to the present invention;
fig. 7 is a step diagram of the method for acquiring the face ID corresponding to the face feature according to the present invention.
Detailed Description
The technical scheme of the invention is further explained by the specific implementation mode in combination with the attached drawings.
Wherein the showings are for the purpose of illustration only and are shown by way of illustration only and not in actual form, and are not to be construed as limiting the present patent; to better illustrate the embodiments of the present invention, some parts of the drawings may be omitted, enlarged or reduced, and do not represent the size of an actual product; it will be understood by those skilled in the art that certain well-known structures in the drawings and descriptions thereof may be omitted.
The same or similar reference numerals in the drawings of the embodiments of the present invention correspond to the same or similar components; in the description of the present invention, it should be understood that if the terms "upper", "lower", "left", "right", "inner", "outer", etc. are used for indicating the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, it is only for convenience of description and simplification of description, but it is not indicated or implied that the referred device or element must have a specific orientation, be constructed in a specific orientation and be operated, and therefore, the terms describing the positional relationship in the drawings are only used for illustrative purposes and are not to be construed as limitations of the present patent, and the specific meanings of the terms may be understood by those skilled in the art according to specific situations.
In the description of the present invention, unless otherwise explicitly specified or limited, the term "connected" or the like, if appearing to indicate a connection relationship between the components, is to be understood broadly, for example, as being fixed or detachable or integral; can be mechanically or electrically connected; they may be directly connected or indirectly connected through intervening media, or may be connected through one or more other components or may be in an interactive relationship with one another. The specific meanings of the above terms in the present invention can be understood in a specific case to those of ordinary skill in the art.
An embodiment of the present invention provides a system for sensorless face recognition based on pedestrian ReID (Person Re-identification), referring to fig. 1, including:
the video stream acquisition module 1 is used for acquiring video frame images from camera equipment including a camera;
the human body detection module 2 is connected with the video stream acquisition module 1 and is used for detecting a human body of the video frame image, extracting human body characteristics of the detected human body, performing characteristic matching on the human body characteristics corresponding to a certain human body extracted from the current frame and the human body characteristics corresponding to each human body extracted from the previous frame of the current frame, and comparing the successfully matched human body characteristics with the human body characteristics of each pedestrian stored in the pedestrian ReID library 100 to obtain a pedestrian ID comparison result;
specifically, referring to fig. 2, the human body detection module 2 includes:
a human body detection unit 21, configured to perform human body detection on the video frame image through a preset target detection algorithm; preferably, the target detection algorithm adopts Yolo v3, and the Yolo v3 target detection algorithm is an existing target detection algorithm that can be used for detecting a human body, so the specific process of the target detection algorithm for detecting the human body is not described herein;
a human body image intercepting unit 22 connected to the human body detecting unit 21 for intercepting the detected human body image from the video frame image; there are many methods for capturing human body images, for example, human body images can be captured through a specific neural network structure, and since the process of capturing human body images is not within the scope of the claimed invention, the specific process of capturing human body images is not described herein;
a human body feature extraction unit 23 connected to the human body image capturing unit 22 for inputting the captured human body image into a feature extraction network to extract human body features on the human body image; the feature extraction network is preferably a residual neural network ResNet50;
a human body feature matching unit 24, connected to the human body feature extracting unit 23, configured to perform feature matching on the human body feature corresponding to a certain pedestrian extracted from the current frame and the human body features corresponding to each pedestrian extracted from the previous frame of the current frame, so as to obtain a human body feature matching result;
a pedestrian ID giving unit 25 connected to the human body feature matching unit 24, for giving a pedestrian ID corresponding to the human body feature on the current frame participating in the feature matching when the human body feature matching fails;
and the pedestrian feature comparison unit 26 is connected to the human body feature matching unit 24, and is configured to perform feature comparison between the human body features in the current frame participating in the feature matching and the features of each pedestrian stored in the pedestrian ReID library 100 after the human body features are successfully matched, so as to obtain a pedestrian ID comparison result.
In the above technical solution, a specific matching process of human body features is not set forth herein, and will be set forth in detail in the following non-perceptual face recognition method.
A pedestrian corresponds to a pedestrian ID, but sometimes the pedestrian ID of the pedestrian cannot be recognized according to the human body features due to the problem of the image capturing angle, or the pedestrian is a new person, and the pedestrian ID corresponding to the pedestrian is not available in the pedestrian ReID library 100. In the two cases, in order to ensure the identification accuracy of the identity of the pedestrian, the invention further identifies the human face characteristics of the pedestrian on the basis of identifying the human body characteristics of the pedestrian.
Referring to fig. 1, the non-perceptual face recognition system further includes:
and the face detection module 3 is connected with the human body detection module 2 and used for further extracting face features of a human body, comparing the extracted face features with the faces stored in a face library 200 one by one, and obtaining a face comparison result. It should be noted here that there are many existing face detection methods, and the present invention preferably implements recognition and extraction of face features through a retinaFace face recognition algorithm.
Referring to fig. 3, the face detection module 3 specifically includes:
a face detection unit 31, configured to perform face detection on a human body image to obtain a human face image;
a face feature extraction unit 32 connected to the face detection unit 31 for extracting face features from the face image;
the face comparison unit 33 is connected to the face feature extraction unit 32, and is configured to compare the extracted face features with each face in the face library one by one to obtain a face comparison result;
a face ID giving unit 34 connected to the face comparison unit 33, for giving a face ID corresponding to the face feature when the face comparison fails;
the face ID obtaining unit 35 is connected to the face comparing unit 33, and is configured to obtain, from the face library 200, a face ID corresponding to the face feature after the face comparison is successful.
In the above technical solution, when the human identity is identified by detecting human features, if the human features of a certain human in the current frame are not successfully matched with the human features corresponding to all the human in the previous frame, the system will determine the pedestrian in the current frame as a suspected new person and automatically assign the suspected new person with a pedestrian ID corresponding to the suspected new person. In order to reduce the false detection rate, the system will perform face detection and comparison again on the suspected new person, and when the face features of the suspected new person are successfully compared with the faces in the face library, it indicates that the human body is determined as a false person before, so the result of the preliminary determination needs to be corrected in this case.
In addition, another false detection situation may occur when the human identity is determined by detecting the human face features. For example, after the system preliminarily determines a certain pedestrian in the previous frame as a suspected new person, when the current frame performs further face feature detection on the suspected new person, if the face of the suspected new person is not compared, in a normal case, the system finally determines that the suspected new person is the new person, then assigns a corresponding face ID to the face feature of the new person, and updates the assigned face ID and the pedestrian ID assigned to the new person in the previous frame into the face library 200 or the pedestrian ReID library 100. But the suspected new person may not be a new person, and the facial features of the suspected new person in the current frame are not compared successfully, perhaps because the facial image is not clear due to shooting angle problems. In order to reduce the false detection rate, the invention also needs to correct the human body recognition result aiming at the situation.
Therefore, with reference to fig. 1, the system for identifying an imperceptible face provided by the present invention further includes:
and the result correction module 4 is respectively connected with the human body detection module 2 and the face detection module 3, and is used for correcting the pedestrian ID corresponding to the suspected new person after the previous frame is judged as the suspected new person according to the face comparison result of the face features in the current frame.
Referring to fig. 4, the result correction module 4 specifically includes:
a first correction unit 41, configured to, when the face features of the suspected new person in the current frame are successfully compared, obtain, from the pedestrian ReID library 100, a pedestrian ID corresponding to the suspected new person by using the face ID corresponding to the suspected new person, and then correct, according to the obtained pedestrian ID, the pedestrian ID given to the suspected new person in the previous frame;
a second correcting unit 42, configured to, when the comparison of the facial features of the suspected new person in the current frame fails, extract the facial features corresponding to all the human bodies in the previous frame of the current frame, perform feature comparison between the facial features of the suspected new person and all the facial features in the previous frame one by one,
if the comparison is successful, acquiring a pedestrian ID corresponding to the pedestrian corresponding to the compared face feature from the pedestrian ReID library 100 through the face ID corresponding to the pedestrian in the last frame successfully compared, and then correcting the pedestrian ID previously given to the suspected new person by using the acquired pedestrian ID;
and if the comparison is not successful, finally determining the suspected new person as the new person.
In order to dynamically update the recognition rate of the non-perceptual face recognition system provided by the present invention, please continue to refer to fig. 1, the non-perceptual face recognition system further includes:
and the data updating module 5 is connected with the result correcting module 4 and is used for updating the correction result made by the result correcting module 4 in the pedestrian ReID library 100. It should be noted that, in the present invention, the data in the pedestrian ReID library 100 includes all the data in the face library 200.
The following description focuses on the non-perceptual face recognition method based on the pedestrian ReID provided by the invention.
Referring to fig. 5, the method specifically includes the following steps:
step S1, acquiring a video frame image, wherein the channel for acquiring the video frame image is as described above and is not repeated herein;
s2, carrying out human body detection on the video frame image, if a human body is detected, further extracting human body characteristics of the detected human body, and if no human body is detected, returning to the S1 to continuously obtain the video frame image; in step S2, preferably, a Yolo v3 target detection algorithm is used to perform human body detection on the video frame image, and a human body frame is generated when a human body is detected, then the human body image in the human body frame is intercepted, and the human body image is input into a preset feature extraction network to perform human body feature extraction, and preferably, the feature extraction network selects a ResNet50 residual neural network to perform 512-dimensional feature extraction on the human body image. And finally, carrying out feature matching on the human body features extracted from the current frame and all the human body features extracted from the previous frame of the current frame, and comparing the successfully matched human body features with the features of all pedestrians stored in the pedestrian ReID to obtain a comparison result of the pedestrian ID. The comparison result of the pedestrian ID does not comprise two types, namely, the comparison of the pedestrian ID is successful, and the comparison of the pedestrian ID is unsuccessful. The successful comparison of the pedestrian ID indicates that the compared pedestrian has stored data in the pedestrian ReID library, namely that the pedestrian is not a new person. If the comparison of the pedestrian IDs is unsuccessful, two conditions are included, one is that the pedestrian is a new person, and the pedestrian ID data of the new person is not recorded in the pedestrian ReID library; in another case, the comparison error may occur due to an unclear image on the picture of one of the two frames or at least one of the two frames of the same pedestrian. Therefore, when the pedestrian ID is not successfully compared, the comparison result needs to be corrected by comprehensively considering the two situations so as to avoid errors. The correction procedure for the two possible false detection situations will be described in detail below, and will not be described herein.
In addition, in step S2, a specific method for performing feature matching on the human body features in the two frames before and after is:
performing inner product operation on a first human body feature vector corresponding to the human body features in the current frame and a second human body feature vector corresponding to each human body feature extracted from the previous frame one by one, then judging whether an inner product value larger than a preset threshold value exists in each inner product value obtained by calculation,
if so, selecting the largest inner product value from the inner product values larger than the threshold value, and outputting a second human body feature vector or a human body feature corresponding to the first human body feature vector as a matching result when the largest inner product value is obtained through calculation;
and if not, indicating that the human body feature matching fails.
In addition, in order to improve the recognition efficiency of the non-perceptual face recognition system provided by the present invention, in the human body detection process at the first stage, if the pedestrian ID corresponding to the pedestrian is successfully compared on the previous frame image, the pedestrian ID comparison is not performed on the pedestrian at the current frame.
Referring to fig. 5, the method for identifying an imperceptible face according to the present invention further includes:
s3, further extracting the face features of the successfully matched human body, and comparing the extracted face features with the faces stored in a face library one by one to obtain a face comparison result;
s4, according to a face comparison result made of the face features in the current frame, a preset correction method is given to correct the pedestrian ID given to the suspected new person after the previous frame is judged as the suspected new person;
and step S5, updating the correction result made in the step S4 into a pedestrian ReID library.
Referring to fig. 6, in step S2, the specific method for acquiring the pedestrian ID corresponding to the human body feature includes:
step S21, carrying out human body detection on the video frame image;
step S22, intercepting the detected human body image from the video frame image;
step S23, extracting human body features on the human body image;
step S24, matching the human body characteristics extracted from the current frame with all the human body characteristics extracted from the previous frame of the current frame,
if the matching is successful, comparing the human body characteristics in the current frame participating in the matching with the characteristics of each pedestrian stored in the pedestrian ReID library to obtain a pedestrian ID comparison result;
and if the matching is unsuccessful, regarding the pedestrian corresponding to the human body characteristics involved in the matching as the suspected new person and giving the pedestrian ID corresponding to the suspected new person.
It should be noted that, the result of comparing the human body characteristics with the characteristics of each pedestrian stored in the pedestrian ReID library has two cases of successful comparison and unsuccessful comparison,
if the comparison is successful, obtaining the pedestrian ID corresponding to the human body feature participating in the matching of the previous frame image and the next frame image in the current frame;
if the comparison is unsuccessful, the system regards the pedestrian corresponding to the human body characteristic which is not successfully compared as a suspected new person, namely the person without data storage record in the pedestrian ReID library. But many times these pedestrians, considered as suspected new people, are not actually new people, possibly due to the fact that the human feature matching is not successful because the images of the previous and next frames are not clear. Therefore, in order to avoid the occurrence of such an error, please refer to fig. 7, the present invention further provides a face recognition method, specifically, in step S3, the specific method for obtaining the face ID corresponding to the face feature includes the following steps:
step S31, detecting the human face to obtain a human face image; it should be emphasized here that, in order to increase the speed of pedestrian recognition, the present invention generally only detects faces that are deemed to be suspected new people;
step S32, extracting the face features on the face image;
step S33, comparing the extracted face features with each face stored in the face library one by one,
if the comparison is successful, acquiring a face ID corresponding to the face features from a face library;
and if the comparison fails, giving a face ID corresponding to the face feature.
In step S4, the preset correction method includes a first correction method, and the process of correcting the pedestrian ID by the first correction method is detailed as follows:
and when the face features of the suspected new person in the current frame are successfully compared, acquiring the pedestrian ID corresponding to the suspected new person from the pedestrian ReID library by using the face ID corresponding to the suspected new person, and then correcting the pedestrian ID given to the suspected new person in the previous frame according to the acquired pedestrian ID.
It should be noted that, for the same pedestrian, the present invention has a certain rule when encoding the pedestrian ID and the face ID of the same pedestrian, and the pedestrian ID and the face ID of the same pedestrian have a certain correlation, for example, for the pedestrian a, the present invention encodes the pedestrian ID of the pedestrian a as abbody 000, and encodes the face ID of the pedestrian a as Aface000, and the pedestrian ID of the pedestrian a and the face ID form the identification feature of the pedestrian a, so the present invention can acquire the pedestrian ID of the pedestrian through the face ID of a certain pedestrian.
In addition, when the comparison of the face features of the suspected new person in the current frame is successful, it indicates that the previous frame preliminarily determines that the pedestrian is wrong as the suspected new person, that is, the previous frame wrongly assigns the pedestrian ID of the suspected new person to the pedestrian. Therefore, the invention obtains the actual pedestrian ID of the suspected new person by comparing the face ID of the suspected new person so as to correct the previous wrong pedestrian ID assignment.
In step S4, the preset correction method further includes a second correction method, and the process of correcting the pedestrian ID by the second correction method is as follows:
when the comparison of the face features of the suspected new person in the current frame fails, the suspected new person is actually the new person, but there is also a case that the face comparison fails due to unclear face images of the suspected new person in the current frame. Therefore, in order to avoid the situation, when the comparison of the face features of the suspected new person in the current frame fails, the similarity comparison is carried out on the face features of the suspected new person and all the face features in the previous frame by further extracting all the face features in the previous frame of the current frame,
if the comparison is successful, firstly, acquiring a face ID corresponding to the successfully compared face features from a face library, then acquiring a pedestrian ID corresponding to the face ID from a pedestrian ReID library, and then correcting the pedestrian ID previously given to the suspected new person by using the acquired pedestrian ID;
and if the comparison is unsuccessful, finally determining the suspected new person as the new person.
It should be understood that the above-described embodiments are merely preferred embodiments of the invention and the technical principles applied thereto. Various modifications, equivalent substitutions, changes, etc., will also be apparent to those skilled in the art. However, such variations are within the scope of the invention as long as they do not depart from the spirit of the invention. In addition, certain terms used in the specification and claims of the present application are not limiting, but are used merely for convenience of description.
Claims (13)
1. A pedestrian ReID-based non-perceptual face recognition system, comprising:
the video stream acquisition module is used for acquiring a video frame image;
the human body detection module is connected with the video stream acquisition module and used for detecting a human body of the video frame image, extracting human body characteristics of the detected human body, performing characteristic matching on the human body characteristics corresponding to the human body extracted from the current frame and the human body characteristics corresponding to each human body extracted from the previous frame of the current frame, and comparing the successfully matched human body characteristics with the human body characteristics of each pedestrian stored in a pedestrian ReID (ReID) library to obtain a pedestrian ID comparison result;
the human face detection module is connected with the human body detection module and used for further extracting human face features of the human body which is successfully matched, and then comparing the extracted human face features with human faces stored in a human face library one by one to obtain a human face comparison result;
a result correction module, respectively connected to the human body detection module and the face detection module, for correcting the pedestrian ID corresponding to the suspected new person after being determined as the suspected new person in the previous frame according to the face comparison result made for the face feature in the current frame;
and the data updating module is connected with the result correcting module and used for updating the correction result made by the result correcting module into the pedestrian ReID library.
2. The system of claim 1, wherein the human detection module specifically comprises:
the human body detection unit is used for carrying out human body detection on the video frame image through a preset target detection algorithm;
the human body image intercepting unit is connected with the human body detecting unit and is used for intercepting the detected human body image from the video frame image;
the human body feature extraction unit is connected with the human body image intercepting unit and used for inputting the intercepted human body image into a feature extraction network so as to extract the human body features on the human body image;
the human body feature matching unit is connected with the human body feature extraction unit and is used for performing feature matching on the human body features corresponding to the pedestrians extracted from the current frame and the human body features corresponding to the pedestrians extracted from the previous frame of the current frame to obtain a human body feature matching result;
the pedestrian ID giving unit is connected with the human body feature matching unit and is used for giving the pedestrian ID corresponding to the human body feature on the current frame participating in feature matching when the human body feature matching fails;
and the pedestrian characteristic comparison unit is connected with the human body characteristic matching unit and used for comparing the human body characteristics in the current frame participating in the characteristic matching with the pedestrian characteristics stored in the pedestrian ReID library after the human body characteristics are successfully matched to obtain the pedestrian ID comparison result.
3. The system of claim 2, wherein the face detection module specifically comprises:
the human face detection unit is used for carrying out human face detection on the human body image to obtain a human face image;
the face feature extraction unit is connected with the face detection unit and used for extracting the face features on the face image;
the face comparison unit is connected with the face feature extraction unit and is used for comparing the extracted face features with the faces stored in the face library one by one to obtain a face comparison result;
the face ID endowing unit is connected with the face comparison unit and is used for endowing a face ID corresponding to the face characteristic when the face comparison fails;
and the face ID acquisition unit is connected with the face comparison unit and used for acquiring the face ID corresponding to the face features from the face library after the face comparison is successful.
4. The system of claim 3, wherein the result correction module specifically comprises:
a first correction unit, configured to, when the face feature comparison of the suspected new person in the current frame is successful, obtain, from the pedestrian ReID library, the pedestrian ID corresponding to the suspected new person by using the face ID corresponding to the suspected new person, and then correct, according to the obtained pedestrian ID, the pedestrian ID given to the suspected new person in the previous frame;
a second correction unit, configured to, when the comparison of the face features of the suspected new person in the current frame fails, perform feature-by-feature comparison between the face features of the suspected new person and all the face features in the previous frame by extracting the face features corresponding to all the human bodies in the previous frame of the current frame,
if the comparison is successful, firstly, acquiring the face ID corresponding to the successfully compared face features from the face library, then acquiring the pedestrian ID corresponding to the face ID from the pedestrian ReID library, and then correcting the pedestrian ID previously given to the suspected new person by using the acquired pedestrian ID;
and if the comparison is not successful, finally determining the suspected new person as the new person.
5. An imperceptible face recognition method based on pedestrian ReID, which is implemented by the imperceptible face recognition system according to any one of claims 1 to 4, comprising the steps of:
s1, acquiring the video frame image;
s2, detecting a human body of the video frame image, extracting human body features of the detected human body, performing feature matching on the human body features extracted from the current frame and all the human body features extracted from the previous frame of the current frame, and comparing the successfully matched human body features with all the pedestrian features stored in the pedestrian ReID library to obtain a pedestrian ID comparison result;
s3, further extracting the face features of the successfully matched human body, and comparing the extracted face features with the faces stored in the face library one by one to obtain a face comparison result;
step S4, correcting the pedestrian ID given to the suspected new person after the previous frame is determined as the suspected new person based on a preset correction method according to the face comparison result made for the face features in the current frame;
and step S5, updating the correction result made in the step S4 into the pedestrian ReID library.
6. The method of claim 5, wherein in step S2, the human body detection of the video frame image is realized by a Yolo v3 target detection algorithm.
7. The non-perceptual face recognition method of claim 5, wherein in the step S2, the human features of the detected human image are extracted through a residual neural network ResNet 50.
8. The method for identifying an unaesthetic face as claimed in claim 5, wherein in the step S2, the specific method for performing the feature matching on the human body features in the two frames of images comprises:
performing inner product operation on a first human body feature vector corresponding to the human body features in the current frame and a second human body feature vector corresponding to each human body feature extracted from the previous frame one by one, judging whether inner product values larger than a preset threshold value exist in each inner product value obtained by calculation,
if so, selecting a maximum inner product value from the inner product values larger than the threshold value, and outputting the human body feature corresponding to the second human body feature vector or the first human body feature vector when the maximum inner product value is obtained through calculation as a matching result;
and if not, indicating that the human body feature matching fails.
9. A method for unaware face recognition according to claim 5, wherein in step S3, the recognition and extraction of the face features are implemented by a RetinaFace face recognition algorithm.
10. The method for identifying an unaware face as in claim 5, wherein in the step S2, the specific method for obtaining the pedestrian ID corresponding to the human body feature comprises:
step S21, carrying out human body detection on the video frame image;
s22, intercepting the detected human body image from the video frame image;
step S23, extracting the human body features on the human body image;
step S24, the human body features extracted from the current frame are matched with all the human body features extracted from the previous frame of the current frame,
if the matching is successful, comparing the human body features in the current frame participating in the matching with the features of the pedestrians stored in the pedestrian ReID library to obtain a comparison result of the pedestrian IDs;
and if the matching is unsuccessful, regarding the pedestrian corresponding to the human body characteristics involved in the matching as the suspected new person and giving the pedestrian ID corresponding to the suspected new person.
11. The method for identifying an unaware face as in claim 5, wherein in the step S3, the specific method for obtaining the face ID corresponding to the face feature comprises:
step S31, carrying out face detection on the human body to obtain a face image;
step S32, extracting the face features on the face image;
step S33, comparing the extracted face features with the faces stored in the face library one by one,
if the comparison is successful, the face ID corresponding to the face features is obtained from the face library;
and if the comparison fails, giving the face ID corresponding to the face features.
12. The method for unaware face recognition according to claim 5, wherein in the step S4, the preset correction method comprises a first correction method, and the first correction method corrects the pedestrian ID as follows:
and when the face features of the suspected new person in the current frame are successfully compared, acquiring the pedestrian ID corresponding to the suspected new person from the pedestrian ReID library by using the face ID corresponding to the suspected new person, and then correcting the pedestrian ID given to the suspected new person in the previous frame according to the acquired pedestrian ID.
13. The non-perceptual face recognition method as set forth in claim 5, wherein the preset correction method in the step S4 comprises a second correction method, and the second correction method corrects the pedestrian ID as follows:
when the comparison of the face features of the suspected new person in the current frame fails, all the face features in the previous frame of the current frame are extracted, and then the face features of the suspected new person are compared with all the face features in the previous frame in a similarity manner,
if the comparison is successful, firstly, acquiring the face ID corresponding to the successfully compared face features from the face library, then acquiring the pedestrian ID corresponding to the face ID from the pedestrian ReID library, and then correcting the pedestrian ID previously given to the suspected new person by using the acquired pedestrian ID;
and if the comparison is not successful, finally determining the suspected new person as the new person.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010195478.XA CN111523383B (en) | 2020-03-19 | 2020-03-19 | Non-perception face recognition system and method based on pedestrian ReID |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010195478.XA CN111523383B (en) | 2020-03-19 | 2020-03-19 | Non-perception face recognition system and method based on pedestrian ReID |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111523383A CN111523383A (en) | 2020-08-11 |
| CN111523383B true CN111523383B (en) | 2023-04-07 |
Family
ID=71901921
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010195478.XA Active CN111523383B (en) | 2020-03-19 | 2020-03-19 | Non-perception face recognition system and method based on pedestrian ReID |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111523383B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112418169A (en) * | 2020-12-10 | 2021-02-26 | 上海芯翌智能科技有限公司 | Method and equipment for processing human body attribute data |
| CN112800940A (en) * | 2021-01-26 | 2021-05-14 | 湖南翰坤实业有限公司 | Elevator control and abnormity alarm method and device based on biological feature recognition |
| CN112949526B (en) * | 2021-03-12 | 2024-03-29 | 深圳海翼智新科技有限公司 | Face detection method and device |
| CN113269127B (en) * | 2021-06-10 | 2024-04-02 | 北京中科通量科技有限公司 | Face recognition and pedestrian re-recognition monitoring method and system for real-time automatic database establishment |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108491822A (en) * | 2018-04-02 | 2018-09-04 | 杭州高创电子科技有限公司 | A kind of Face datection De-weight method based on the limited caching of embedded device |
| CN108921008A (en) * | 2018-05-14 | 2018-11-30 | 深圳市商汤科技有限公司 | Portrait identification method, device and electronic equipment |
| CN109522782A (en) * | 2018-09-04 | 2019-03-26 | 上海交通大学 | Household member's identifying system |
| CN110147712A (en) * | 2019-03-27 | 2019-08-20 | 苏州书客贝塔软件科技有限公司 | A kind of intelligent cloud platform of pedestrian's analysis |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP3418944B1 (en) * | 2017-05-23 | 2024-03-13 | Canon Kabushiki Kaisha | Information processing apparatus, information processing method, and program |
-
2020
- 2020-03-19 CN CN202010195478.XA patent/CN111523383B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108491822A (en) * | 2018-04-02 | 2018-09-04 | 杭州高创电子科技有限公司 | A kind of Face datection De-weight method based on the limited caching of embedded device |
| CN108921008A (en) * | 2018-05-14 | 2018-11-30 | 深圳市商汤科技有限公司 | Portrait identification method, device and electronic equipment |
| CN109522782A (en) * | 2018-09-04 | 2019-03-26 | 上海交通大学 | Household member's identifying system |
| CN110147712A (en) * | 2019-03-27 | 2019-08-20 | 苏州书客贝塔软件科技有限公司 | A kind of intelligent cloud platform of pedestrian's analysis |
Non-Patent Citations (4)
| Title |
|---|
| Chen Change Loy 等.Wider Face and Pedestrian Challenge 2018 Methods and Results.《arXiv》.2019,第1-7页. * |
| KangGeon Kim 等.Face and Body Association for Video-Based Face Recognition.《2018 IEEE Winter Conference on Applications of Computer Vision》.2018,第39-48页. * |
| 毕君郁.结合行人检测和重识别的人员搜索框架在搜寻走失儿童中的应用分析.《无线互联科技》.2020,第17卷(第05期),第157-159页. * |
| 焦珊珊 ; 李云波 ; 陈佳林 ; 潘志松 ; .多目标跨摄像头跟踪技术.《国防科技》.2019,第40卷(第06期),第33-41页. * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111523383A (en) | 2020-08-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111523383B (en) | Non-perception face recognition system and method based on pedestrian ReID | |
| US11107225B2 (en) | Object recognition device and computer readable storage medium | |
| WO2021036436A1 (en) | Facial recognition method and apparatus | |
| WO2019071664A1 (en) | Human face recognition method and apparatus combined with depth information, and storage medium | |
| CN106991395B (en) | Information processing method and device and electronic equipment | |
| WO2019134346A1 (en) | Face recognition method, application server, and computer-readable storage medium | |
| CN110298312B (en) | Living body detection method, living body detection device, electronic apparatus, and computer-readable storage medium | |
| US11042725B2 (en) | Method for selecting frames used in face processing | |
| CN110532992B (en) | Human face recognition method based on visible light and near infrared | |
| WO2020244071A1 (en) | Neural network-based gesture recognition method and apparatus, storage medium, and device | |
| US20220147735A1 (en) | Face-aware person re-identification system | |
| US11281922B2 (en) | Face recognition system, method for establishing data of face recognition, and face recognizing method thereof | |
| WO2021012647A1 (en) | Face verification method and apparatus, server and readable storage medium | |
| CN110929555B (en) | Face recognition method and electronic device using same | |
| CN111582118A (en) | Face recognition method and device | |
| CN110874588B (en) | Method and device for dynamically optimizing light influence in face recognition | |
| US20090136137A1 (en) | Image processing apparatus and method thereof | |
| CN111611437A (en) | Method and device for preventing face voiceprint verification and replacement attack | |
| CN111325073B (en) | Monitoring video abnormal behavior detection method based on motion information clustering | |
| US9646386B2 (en) | Method and apparatus for generating temporally consistent superpixels | |
| CN111046804A (en) | Living body detection method, living body detection device, electronic equipment and readable storage medium | |
| CN112052706B (en) | Electronic device and face recognition method | |
| US20140369625A1 (en) | Image processing method | |
| CN109064485B (en) | Feature library maintenance method based on CMT algorithm | |
| WO2021025954A1 (en) | Techniques for detecting a three-dimensional face during facial recognition |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |