Identity security verification method and system based on identity document and face recognition
Technical Field
The invention belongs to the field of security authentication of biological feature recognition, and particularly relates to an identity security verification method and system based on identity document and face recognition.
Background
The identity security verification is widely applied to the scenes of banks, customs, airports, railway stations and the like, and along with the popularization of non-contact intelligent chips in identity documents and the high-speed development of face recognition technology, more and more occasions provide higher requirements for the simplicity, the high efficiency and the accuracy of an identity verification system.
Identity security verification requires verification of authenticity of a certificate on one hand and verification of consistency between a certificate holder and a certificate owner on the other hand. At present, the former method mainly utilizes the anti-counterfeiting point of the certificate to identify or manually judges whether the chip reading picture is consistent with the certificate surface printing picture. In the latter case, a worker is usually required to perform manual verification, which not only consumes a lot of time, but also the reliability of the verification depends on the experience and ability of the worker, so that the safety of the whole system is significantly reduced.
At present, identity security verification systems based on second-generation identity cards and face recognition technologies exist, in order to solve the problem that the image processing technology cannot be applied due to extremely low pixel of a chip storage photo, a method of obtaining an original certificate photo through networking and then comparing the original certificate photo with a field photo is adopted in part of systems, and the application range is severely limited due to networking requirements.
The proposal proposed by the patent document with the application number of CN102902959A and the name of 'method and system for recognizing human face based on second generation ID card storage certificate' does not need to be accessed to an ID card data center, but the proposal needs to take a great deal of complicated preprocessing steps for chip photos, and the decision-making still depends on a human face training library.
The patent document with the application number of CN102129555A entitled "method and system for identity authentication based on second generation identification card" completes the authentication of the identity of the card holder by comparing the surface scanning picture of the identification card with the real-time picture, but because the scanned and obtained picture has the influence of noise which cannot be normalized due to hardware, the extracted features cannot accurately express image information, so that the system has low recognition accuracy and cannot meet strict requirements in security inspection scenes.
In addition, at present, no identity security verification system capable of processing identity documents such as Hongkong and Macau passes, passports and the like exists in China, and the requirements of customs or international large-scale activity security inspection scenes cannot be met.
Disclosure of Invention
Aiming at the defects and the improvement requirements of the prior art, the invention provides an identity security verification method and an identity security verification system based on identity documents and face recognition, and aims to directly verify the consistency of a licensee and a document owner without depending on an external database, thereby improving the verification efficiency and accuracy, and simultaneously gradually expanding a background database along with the application of a verification system so as to realize the comprehensive authentication of various identity documents based on second-generation resident identity cards, Hongkong and Macau passes, passports and the like and further ensure the public security.
In order to achieve the above object, according to one aspect of the present invention, there is provided an identity security verification method based on identity document and face recognition, comprising the steps of:
(1) a system registration step: acquiring certificate information of a user identity certificate as registration information, acquiring a human face image with clear and non-crown front face of the user as a registration image, and storing the registration information and the registration image in a database; identity documents may include, but are not limited to, second generation resident identity cards, hong Kong and Macau passes, passports; the certificate information of the identity certificate comprises a certificate number and the name of a certificate holder;
(2) an information acquisition step: acquiring certificate information of a user identity certificate, an image in a chip and a scanned image of a printed photo on the surface of the certificate, and acquiring a real-time face image; inquiring a database according to the certificate information, and if related items are inquired, extracting a registration image from an inquiry result; if the related item cannot be inquired, creating a new item in the database according to the certificate information, wherein the registered image in the created new item is empty;
(3) certificate authentication: respectively extracting shallow features of the images in the chip and the scanned images obtained in the step (2), and performing consistency judgment; if the image in the chip is consistent with the scanned image, the step (4) is carried out; if the image in the chip is not consistent with the scanned image, the verification is not passed, and the verification is terminated;
(4) an image preprocessing step: if the registered image is obtained in the step (2), carrying out face detection and pretreatment on the real-time face image to obtain a first comparison image, and carrying out face detection and pretreatment on the registered image to obtain a second comparison image; if the registered image is not obtained in the step (2), carrying out face detection and pretreatment on the real-time face image to obtain a first comparison image, and carrying out face detection and pretreatment on the image with higher image quality in the image in the chip and the scanned image to obtain a second comparison image; the efficiency and the accuracy of face verification can be improved by carrying out face detection and pretreatment on the image;
(5) a human face verification step: respectively extracting deep features of the first contrast image and the second contrast image acquired in the step (4), and calculating the matching degree of the deep features and the second contrast image; if the matching degree is higher than the first threshold value, the step (6) is carried out; otherwise, the verification is not passed, and the verification is terminated;
(6) a database updating step: if the registered image is not obtained in the step (2), the acquired real-time face image is taken as the registered image, stored in an item corresponding to the database and transferred to the step (7); if the registered image is obtained in the step (2) and the time interval between the current verification time and the last database updating time exceeds a second threshold value, taking the real-time face image acquired this time as the registered image, updating a corresponding item in the database, and turning to the step (7); if the registered image is acquired in the step (2) and the time interval between the current verification time and the last database updating time does not exceed a second threshold value, switching to the step (7);
(7) and the verification is passed and the verification is finished.
Further, the first threshold value used for judging whether the face verification is passed in the step (5) is set according to experience; the larger the first threshold is, the lower the computational complexity of the face verification is, but the lower the accuracy of the face verification is; conversely, the smaller the first threshold value, the higher the accuracy of face verification, but the higher the computational complexity of face verification.
Further, in the step (5), the matching degree between the deep features of the first contrast image and the deep features of the second contrast image is a cosine distance of the deep features of the first contrast image and the deep features of the second contrast image.
Further, the step (6) determines whether to update the second threshold of the registered image according to the actual application scene; the smaller the second threshold value is, the more frequent the registered image is updated, the higher the accuracy of the face verification is, but the updating cost is higher; conversely, the smaller the second threshold value, the less frequent the registered image is updated, the less overhead is updated, but the accuracy of the face verification is also smaller.
Further, the shallow feature extracted in the step (3) is a feature formed by combining an SURF feature, a histogram horizontal projection feature, a histogram vertical projection feature and a gradient feature; to reduce computational complexity while increasing extraction speed.
Further, in the step (3), the method for determining consistency includes: calculating Euclidean distances between the extracted shallow features; if the Euclidean distance is larger than a third threshold value, determining that the image in the chip is consistent with the scanned image; otherwise, the image in the chip is judged to be inconsistent with the scanned image.
Further, the third threshold is set empirically; as the system is put into use, the database will dynamically expand, the system will be updated periodically, and the third threshold will be more accurate.
Further, in the step (4), a Haar feature and an Adaboost classifier can be adopted for face detection;
further, in step (4), the preprocessing of the image includes: histogram equalization, Gaussian filtering denoising and image size normalization;
further, in the step (4), the method for comparing the image quality of the on-chip image and the scanned image includes: comparing the sizes of the image files of the image and the scanned image in the chip, wherein the larger the image file is, the higher the image quality is; for the second generation of resident identification cards, the image quality of the scanned image is higher; for passports, the image quality of the image within the chip is higher.
Further, in step (5), the method for extracting deep features of the image comprises: obtaining high-dimensional convolution characteristics of the image, and then performing dimensionality reduction processing on the high-dimensional convolution characteristics to obtain low-dimensional characteristics suitable for calculation; taking the low-dimensional features as deep features for face verification; methods for obtaining high-dimensional convolution features of an image include, but are not limited to, deep convolution neural networks; methods for performing dimension reduction processing on high-dimensional convolution features include, but are not limited to, principal component analysis algorithms.
According to another aspect of the present invention, the present invention further provides a system for implementing the identity security verification method based on identity document and face recognition provided by the present invention, including: the system comprises a video stream image unit, a certificate information acquisition unit, an image preprocessing unit, a face verification unit and a database management unit;
the video stream image unit is used for acquiring a face image which is free from a crown and clear on the front side of a user as a registration image during system registration; the video stream image unit is also used for acquiring a real-time face image during information acquisition;
the certificate information acquisition unit is used for inputting certificate information as registration information during system registration; the certificate information acquisition unit is also used for reading the certificate information and the image in the chip of the identity certificate during information acquisition and scanning the identity certificate to acquire a scanned image of a printed photo on the surface of the certificate;
the image preprocessing unit is used for carrying out face detection and preprocessing on the image during image preprocessing so as to improve the accuracy of face verification;
the face verification unit is used for extracting an image in the chip and a scanned image which are acquired by the certificate information acquisition unit when the certificate is identified, and then respectively extracting shallow layer characteristics of the image in the chip and the scanned image and carrying out consistency judgment; the face verification unit is also used for extracting a first comparison image and a second comparison image which are obtained by the image preprocessing unit during face verification, then respectively extracting deep features of the first comparison image and the second comparison image and calculating the matching degree of the first comparison image and the second comparison image;
the database management unit is used for acquiring registration information from the certificate information acquisition unit during system registration, acquiring a registration image from the video stream image unit, and storing the acquired registration information and the acquired registration image into the database; the database management unit is also used for acquiring certificate information from the information acquisition unit during information acquisition and querying the database according to the acquired certificate information; the database management unit is also used for acquiring a real-time face image from the video stream image unit as a registration image when the database is updated, and storing or updating the acquired registration image into the database.
Further, the video stream image unit is a camera.
Further, the certificate information acquisition unit includes: a CIS (Contact Image Sensor) Image acquisition module and a chip card reader; the CIS image acquisition module is used for scanning the certificate to acquire a scanning image of a printed photo on the surface of the identity certificate, and the chip card reader is used for reading the certificate information of the identity certificate and the image in the chip.
Further, the shallow feature extracted from the image by the face verification unit is a feature formed by combining a SURF feature, a histogram horizontal projection feature, a histogram vertical projection feature and a gradient feature, so that the calculation complexity is reduced, and the extraction speed is increased.
Further, the method for extracting deep features from an image by the face verification unit comprises the following steps: obtaining high-dimensional convolution characteristics of the image, and then performing dimensionality reduction processing on the high-dimensional convolution characteristics to obtain low-dimensional characteristics suitable for calculation; and taking the low-dimensional features as deep features for face verification.
Generally, by the above technical solution conceived by the present invention, the following beneficial effects can be obtained:
(1) a database for storing identity document information and a registration image is established, and the registration image is updated regularly, so that on one hand, the method does not depend on an external database, and the efficiency of identity security verification can be improved; on the other hand, the contrast images for face verification are recent face images of the user and collected real-time face images, so that the accuracy of face verification can be obviously improved, and on the basis, the face verification does not depend on manual operation strongly, and the verification speed of the face verification can be obviously improved;
(2) when the certificate is identified, the authenticity of the certificate is identified by extracting the shallow layer characteristics of the image in the chip of the identity certificate and the scanned image of the printed photo on the surface of the certificate, the manual identification is not relied on, and the efficiency and the accuracy of the certificate identification can be improved; the extracted shallow feature is a feature formed by combining SURF feature, histogram horizontal projection feature, histogram vertical projection feature and gradient feature, so that the calculation complexity is reduced, and the extraction speed is increased;
(3) identity security verification can be completed aiming at various certificates such as second-generation resident identity cards, Hongkong and Macau pass cards, passports and the like, and the method can be suitable for various identity security verification scenes; meanwhile, along with the application of the system, the database stores various certificate information of the user, so that the comprehensive verification of various identity certificates can be realized;
(4) when the registration image can not be obtained, the image with higher image quality in the image in the chip of the identity document and the scanned image of the printed picture on the surface of the identity document can be selected as one of the contrast images for face verification, so that the accuracy of the face verification can be ensured.
Drawings
FIG. 1 is a flow chart of the identity security verification method based on identity document and face recognition provided by the present invention;
fig. 2 is a block diagram of an identity security verification system based on identity document and face recognition according to an embodiment of the present invention;
FIG. 3 is a first embodiment of a method for verifying identity security provided by the present invention;
fig. 4 is a second embodiment of the identity security verification method provided by the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. In addition, the technical features involved in the embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.
The identity security verification method based on identity document and face recognition, as shown in figure 1, comprises the following steps:
(1) a system registration step: acquiring the certificate information of the user identity certificate as registration information, acquiring a face image of the user as a registration image, and storing the registration information and the registration image into a database; the certificate information of the identity certificate comprises a certificate number and the name of a certificate holder;
(2) an information acquisition step: acquiring certificate information of a user identity certificate, an image in a chip and a scanned image of a printed photo on the surface of the certificate, and acquiring a real-time face image; inquiring a database according to the certificate information, and if related items are inquired, extracting a registration image from an inquiry result; if the related item cannot be inquired, creating a new item in the database according to the certificate information, wherein the registered image in the created new item is empty;
(3) certificate authentication: respectively extracting shallow layer characteristics of the images in the chip and the scanned images, and performing consistency judgment; if the image in the chip is consistent with the scanned image, the step (4) is carried out; if the image in the chip is not consistent with the scanned image, the verification is not passed, and the verification is terminated;
(4) an image preprocessing step: if the registered image is obtained in the step (2), carrying out face detection and pretreatment on the real-time face image to obtain a first comparison image, and carrying out face detection and pretreatment on the registered image to obtain a second comparison image; if the registered image is not obtained in the step (2), carrying out face detection and pretreatment on the real-time face image to obtain a first comparison image, and carrying out face detection and pretreatment on the image with higher image quality in the image in the chip and the scanned image to obtain a second comparison image; the efficiency and the accuracy of face verification can be improved by carrying out face detection and pretreatment on the image;
(5) a human face verification step: respectively extracting deep features of the first contrast image and the second contrast image, and calculating the matching degree of the deep features and the second contrast image; if the matching degree is higher than the first threshold value, the step (6) is carried out; otherwise, the verification is not passed, and the verification is terminated;
(6) a database updating step: if the registered image is not obtained in the step (2), the real-time face image is used as the registered image and is stored into the corresponding item of the database, and the step (7) is carried out; if the registered image is obtained in the step (2) and the time interval between the current verification time and the last database updating time exceeds a second threshold value, taking the real-time face image as the registered image, updating a corresponding entry in the database, and turning to the step (7); if the registered image is acquired in the step (2) and the time interval between the current verification time and the last database updating time does not exceed a second threshold value, switching to the step (7);
(7) and the verification is passed and the verification is finished.
In conjunction with the identity security verification method shown in fig. 1, an embodiment of the present invention provides a system for implementing the method, as shown in fig. 2, including: the system comprises a video stream image unit, a certificate information acquisition unit, an image preprocessing unit, a face verification unit and a database management unit;
the video stream image unit is used for acquiring a face image which is free from a crown and clear on the front side of a user as a registration image during system registration; the video stream image unit is also used for acquiring a real-time face image during information acquisition; in this embodiment, the video stream image unit is a camera capable of acquiring a face image in real time;
the certificate information acquisition unit is used for inputting certificate information as registration information during system registration; the certificate information acquisition unit is also used for reading the certificate information and the image in the chip of the identity certificate during information acquisition and scanning the identity certificate to acquire a scanned image of a printed photo on the surface of the certificate; in the embodiment, the certificate information acquisition unit comprises a CIS-based certificate scanner and a chip card reader; the CIS-based image scanner is used for scanning the certificate to acquire a scanned image of a printed photo on the surface of the certificate; the chip card reader is used for reading the certificate information of the identity certificate and the image in the chip; the certificate information acquisition unit also comprises equipment supporting manual input, such as a mouse, a keyboard and the like;
the image preprocessing unit is used for carrying out face detection and preprocessing on the image during image preprocessing so as to improve the accuracy of face verification;
the face verification unit is used for acquiring an image in the chip and a scanned image from the certificate information acquisition unit when the certificate is authenticated, and then respectively extracting the characteristics of the image and the scanned image and carrying out consistency judgment; the face verification unit is also used for acquiring a first comparison image and a second comparison image from the image preprocessing unit during face verification, then respectively extracting deep features of the first comparison image and the second comparison image and calculating the matching degree of the first comparison image and the second comparison image;
the database management unit is used for acquiring registration information from the certificate information acquisition unit during system registration, acquiring a registration image from the video stream image unit, and storing the acquired registration information and the acquired registration image into the database; the database management unit is also used for acquiring certificate information from the information acquisition unit during information acquisition and querying the database according to the acquired certificate information; the database management unit is also used for acquiring a real-time face image from the video stream image unit as a registration image when the database is updated, and storing or updating the acquired registration image into the database.
The superficial layer features extracted from the image by the face verification unit are features formed by combining SURF features, histogram horizontal projection features, histogram vertical projection features and gradient features so as to reduce the calculation complexity and improve the extraction speed; the method for extracting deep features from the image by the face verification unit comprises the following steps: obtaining high-dimensional convolution characteristics of the image, and then performing dimensionality reduction processing on the high-dimensional convolution characteristics to obtain low-dimensional characteristics suitable for calculation; and taking the low-dimensional features as deep features for face verification.
In the first application scenario, the user has completed the system registration step based on the second generation resident identification card, so that the database has stored the certificate information and the registration image of the user identification card.
In a first application scenario, as shown in fig. 3, the identity security verification method provided by the present invention specifically includes the following steps:
(11) an information acquisition step: reading the certificate information and the image in the chip of the second generation resident identification card of the user, scanning the second generation resident identification card of the user to obtain a scanned image of a printed photo on the surface of the certificate, and simultaneously acquiring a real-time face image; inquiring a database according to the read certificate information, and extracting a registration image stored in the database;
(12) certificate authentication: respectively extracting shallow layer characteristics of the images in the chip and the scanned images, and performing consistency judgment; if the image in the chip is consistent with the scanned image, the step (13) is carried out; if the image in the chip is not consistent with the scanned image, the verification is not passed, and the verification is terminated; in this embodiment, the method for consistency determination includes: respectively extracting shallow features of the images in the chip and the scanned images; the extracted shallow layer features are combined features of SURF features, histogram horizontal projection features, histogram vertical projection features and gradient features; so as to reduce the complexity of calculation and improve the extraction speed; calculating Euclidean distances between the extracted shallow features; if the Euclidean distance is larger than a third threshold value, determining that the image in the chip is consistent with the scanned image; otherwise, judging that the image in the chip is inconsistent with the scanned image; the third threshold value is set according to experience; as the system is put into use, the database is dynamically expanded, the system is periodically updated, and the third threshold is more accurate;
(13) an image preprocessing step: carrying out face detection and pretreatment on the real-time face image to obtain a first comparison image, and carrying out face detection and pretreatment on a registered image extracted from a query database to obtain a second comparison image; in this embodiment, a Haar feature and an Adaboost classifier are used for face detection; the pretreatment comprises the following steps: histogram equalization, Gaussian filtering denoising and image size normalization;
(14) a human face verification step: respectively extracting deep features of the first contrast image and the second contrast image, and calculating the cosine distance of the deep features as matching degree; if the matching degree is higher than the first threshold value, the step (15) is carried out; otherwise, the verification is not passed, and the verification is terminated; in this embodiment, the method for extracting deep features of an image includes: obtaining high-dimensional convolution characteristics of the image through a deep convolution neural network, and performing dimensionality reduction processing on the high-dimensional convolution characteristics through a principal component analysis algorithm to obtain low-dimensional characteristics suitable for calculation; taking the low-dimensional features as deep features for face verification;
(15) a database updating step: if the time interval between the current verification time and the last database updating time exceeds a second threshold value, the real-time face image is used as a registration image, corresponding items in the database are updated, and the step (16) is carried out; otherwise, go to step (16); the second threshold value is set according to experience;
(16) and the verification is passed and the verification is finished.
In the second application scenario, the document held by the user is a passport, and the system registration step has not been completed, so that the document information and the registration image of the user identity document do not exist in the database.
In a first application scenario, as shown in fig. 4, the identity security verification method provided by the present invention specifically includes the following steps:
(21) an information acquisition step: reading the certificate information and the image in the chip of the user passport, scanning the user passport to obtain a scanned image of a printed photo on the surface of the certificate, and simultaneously acquiring a real-time face image; inquiring a database according to the read certificate information, wherein the inquiry result is empty, creating a new entry in the database according to the certificate information, and the registered image corresponding to the new entry is empty;
(22) certificate authentication: respectively extracting shallow layer characteristics of the images in the chip and the scanned images, and performing consistency judgment; if the image in the chip is consistent with the scanned image, the step (23) is carried out; if the image in the chip is not consistent with the scanned image, the verification is not passed, and the verification is terminated; in this embodiment, the method for consistency determination includes: respectively extracting shallow features of the images in the chip and the scanned images; the extracted shallow layer features are combined features of SURF features, histogram horizontal projection features, histogram vertical projection features and gradient features; so as to reduce the complexity of calculation and improve the extraction speed; calculating Euclidean distances between the extracted shallow features; if the Euclidean distance is larger than a third threshold value, determining that the image in the chip is consistent with the scanned image; otherwise, judging that the image in the chip is inconsistent with the scanned image; the third threshold value is set according to experience; as the system is put into use, the database is dynamically expanded, the system is periodically updated, and the third threshold is more accurate;
(23) an image preprocessing step: carrying out face detection and pretreatment on the collected face image to obtain a first comparison image, and carrying out face detection and pretreatment on the image in the passport chip to obtain a second comparison image; in this embodiment, a Haar feature and an Adaboost classifier are used for face detection; the pretreatment comprises the following steps: histogram equalization, Gaussian filtering denoising and image size normalization;
(24) a human face verification step: respectively extracting deep features of the first contrast image and the second contrast image, and calculating the cosine distance of the deep features as matching degree; if the matching degree is higher than the first threshold value, the step (25) is carried out; otherwise, the verification is not passed, and the verification is terminated; in this embodiment, the method for extracting deep features of an image includes: obtaining high-dimensional convolution characteristics of the image through a deep convolution neural network, and performing dimensionality reduction processing on the high-dimensional convolution characteristics through a principal component analysis algorithm to obtain low-dimensional characteristics suitable for calculation; taking the low-dimensional features as deep features for face verification;
(25) a database updating step: storing the collected real-time face image as a registration image into an item corresponding to the database, and turning to the step (26);
(26) and the verification is passed and the verification is finished.
It will be understood by those skilled in the art that the foregoing is only a preferred embodiment of the present invention, and is not intended to limit the invention, and that any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the scope of the present invention.