KR102529513B1 - Identity verification system using smart phone - Google Patents

Abstract

The present invention acquires the subject's fingerprint image and face image from the camera of the smartphone, detects the characteristic points of the subject's fingerprint, compares them with the characteristic points of the fingerprint previously registered in the main server, and also obtains the subject's facial image from the subject's face image. The present invention relates to an identification system using a smartphone, which detects a feature point and compares the detected feature point with facial feature points pre-registered in a main server to confirm the identity of an examinee.
The present invention is provided with a camera to photograph the subject's fingerprint or face, or by photographing the subject's fingerprint from a fingerprint acquisition assistant equipped with a camera to acquire the subject's fingerprint and face image, and obtain the fingerprint from the acquired fingerprint and face image. a police smart phone that detects feature points and facial features; A gateway server that receives the subject's fingerprint and face images, and fingerprint and face feature points from the police smartphone and transmits them to the matching server; Receiving the subject's fingerprint and face image and fingerprint and facial feature points from the gateway server, and matching the subject's fingerprint and face feature points with the fingerprints and face feature points of a plurality of comparison targets pre-stored in the database unit, Characterized in that it includes; a matching server that performs identity verification.

Description

Identity verification system using smart phone {Identity verification system using smart phone}

The present invention acquires the subject's fingerprint image and face image from the camera built into the police smartphone, detects the subject's fingerprint and facial feature points in the police smartphone, and the matching server uses the fingerprint and facial feature points received from the police smartphone. The present invention relates to an identification system using a smartphone, which confirms the identity of an examinee by comparing a fingerprint and facial feature points of a comparison target pre-registered in a database unit.

If a police officer on duty at the frontline finds a drunken person, an elderly person with dementia, or a person suspected of being a missing child, a person who has died by accident, an illegal alien, a wanted person, or a criminal, etc., currently, after accompanying them to the police station, through the identification process.

In some cases, it is inconvenient to have an unrelated person go to the police station for identification.

Issues that are constantly being addressed in the field of fingerprint recognition are the degradation of fingerprint recognition performance and too low image quality, and the problem of image improvement occurs when there is no direction information due to too low image quality. Although there are various types of fingerprints, since all fingerprints have a certain pattern, the image quality can be improved by analyzing the pattern of the image and improving the direction information.

Therefore, a police officer's own smart phone or mobile device (laptop, PDF, etc.) is used to acquire the target's fingerprint and immediately verify the identity on the spot, but an identification system with improved fingerprint image quality is desired.

In addition, fingerprint recognition technology and face recognition technology can be employed to confirm the identity of the subject using a police officer's smart phone (hereinafter referred to as a police smart phone).

Fingerprint recognition technology is a technology for recognizing a user by acquiring a digital image of a fingerprint using a dedicated sensor, and includes optical fingerprint recognition, capacitive fingerprint recognition, and ultrasonic fingerprint recognition.

In the case of the national DB, since fingerprints are stored for all 10 fingers, it is a situation in which approximately 500 million fingerprints must be matched. Even if 100,000 fingerprints are matched per second, it takes a large amount of time of 80 to 100 minutes, so speed is important. is an element

Face recognition technology is a technology that analyzes and recognizes about 50 parts of the face that change, such as the nose, mouth, eyebrows, and chin, rather than the entire face. Face images are sensitive to lighting, the environment, and the angle of the images, and have disadvantages in distinguishing disguises, facial changes over time, plastic surgery, and similar facial features of twins.

The present invention requires an identification system that can more quickly and accurately confirm the identity in the field by acquiring the fingerprint and face image of the target using a smartphone or mobile device (laptop, PDF, etc.) carried by the police officer himself. do.

As a prior art, Korean Registered Utility Model Publication No. 20-0283571 relates to a portable identification device for verifying identity by scanning an identification card or a biometric fingerprint of an identification card holder and comparing it with information registered in an administrative computer network. In this case, it is impossible to check if the fingerprint is erased because it is recognized only with the fingerprint.

As another prior art, Korean Patent Laid-open Publication No. 2003-0086527 relates to a police radio that can confirm whether the identity inquiry person is a party by comparing a fingerprint corresponding to a resident registration number of an identity inquiry person with a fingerprint input by the identity inquiry person. , Even in this case, it is not possible to check if the fingerprint is erased because it is recognized only with the fingerprint.

As another prior art, Korean Patent Laid-Open Publication No. 2002-0011577 stores the identity of a user on the Internet, data such as fingerprints and face photos in advance in a verification database, and then inputs user's fingerprints and face photos, etc. It is about a user identification system on the Internet made to compare and confirm data of users, and it generally takes a lot of time to compare and analyze users' fingerprints and face photos.

The problem to be solved by the present invention is to acquire the subject's fingerprint image and face image from the camera built in the police smartphone, detect the subject's fingerprint and facial feature points in the police smartphone, and the matching server from the police smartphone An identification system using a smart phone is provided to confirm the identity of an examinee by comparing received fingerprints and facial feature points with fingerprints and facial feature points of a comparison target previously registered in a database unit.

Another problem to be solved by the present invention is to install a predetermined application on a smart phone or mobile device (laptop, PDF, etc.) carried by the police officer through a specific registration process such as authentication, and obtain an image of the subject's fingerprint and face. It is to provide an identification system using a smart phone that can be acquired and identified in the field more quickly and accurately.
Another problem to be solved by the present invention is to obtain a fingerprint image by photographing the examinee's fingerprint with a camera or by photographing the examinee's fingerprint from a fingerprint acquisition assistant equipped with a camera, and obtaining a fingerprint image from the obtained fingerprint image. A police smart phone that detects fingerprint feature points including end points and branch points, a gateway server that receives the subject's fingerprint image and fingerprint feature points from the police smartphone and transfers them to a matching server, and the examinee's fingerprint from the gateway server Provides an identification system using a smartphone including a matching server that receives image and fingerprint feature points and performs identification by matching the subject's fingerprint feature points with the fingerprint feature points of a plurality of comparison objects previously stored in the database unit will be.
Another problem to be solved by the present invention is a police smartphone, which acquires a facial image of the subject by photographing the subject's face with a camera of the police smartphone, and a face including a nose, mouth, eyebrows, and chin from the obtained face image. Facial feature points including the position of each part where the skeleton changes are detected, and the detected facial image and facial feature points of the subject are transferred to a matching server through a gateway server. It is to provide an identification system using a smartphone that performs identification by matching with stored facial feature points of a plurality of comparison objects.
Another problem to be solved by the present invention is that the matching server detects ridge direction information from the subject's fingerprint image, and when there are no two or more consecutive ridge directions among the front, rear, left, and right directions, the ridge direction information is determined to be incorrect, and corrects the feature point by correcting the direction of the ridge before, after, left, or right of any one of the ridges with incorrect ridge direction information, and corrects the subject's fingerprint feature point corrected, and the database unit It is to provide an identification system using a smart phone that performs identification by matching with fingerprint feature points of a plurality of comparison objects stored in advance.
Another problem to be solved by the present invention is that the police smart phone detects fingerprint feature points from the subject's fingerprint image captured by the camera, and transmits the subject's fingerprint image and fingerprint feature point to the matching server through the gateway server. Fingerprint image analysis step of; an identification analysis step of the matching server, in which the matching server performs identification by matching the subject's fingerprint feature points with the fingerprint feature points of a plurality of comparison objects previously stored in the database unit; To provide a driving method of an identification system using a smartphone including a.
Another problem to be solved by the present invention is that, in the identity analysis step of the matching server, the matching server overlaps the fingerprint feature points of the examinee and the feature points of the two fingerprint images of the comparison target previously stored in the database unit as much as possible. an alignment step of selecting an alignment reference point of the image by calculating an amount of rotation and transition of the image, converting the coordinates of the feature point to match the alignment reference point, and then determining a pair of corresponding feature points; a matching step in which the matching server calculates a degree of similarity using the coordinates, types, and angle information of the pair of corresponding feature points determined in the alignment step; In the matching server, a score acquisition step of outputting the degree of matching between the subject's fingerprint feature points and the fingerprint feature point of the comparison object previously stored in the database from the similarity of the corresponding feature point pair as a score according to preset rule data; , To provide a driving method of an identification system using a smartphone.

In order to solve the above problems, the identification system using a smartphone of the present invention is provided with a camera to photograph the subject's fingerprint, or to acquire a fingerprint image by photographing the subject's fingerprint from a fingerprint acquisition assistant equipped with a camera, A police smart phone that detects a fingerprint feature point including a break point and a divergence point of a ridge of a fingerprint from the obtained fingerprint image; A gateway server that receives the subject's fingerprint image and fingerprint feature points from the police smartphone and transmits them to the matching server; a matching server that receives a fingerprint image and fingerprint characteristics of the examinee from the gateway server, matches the fingerprint characteristics of the examinee with the fingerprint characteristics of a plurality of comparison objects previously stored in the database unit, and performs identification; It is characterized in that it includes.

The matching server detects ridge direction information from the subject's fingerprint image, and determines that the ridge direction information is incorrect when there are no two or more identical consecutive ridge directions among front, back, left, and right, and incorrect ridge direction information. As correction is made in the direction of any one of the ridges before, after, left, and right of the ridge with By matching with them, identification can be performed.

Alternatively, the police smartphone captures the subject's face with a camera of the police smartphone to acquire the subject's face image, and from the obtained face image, the position of each part where the facial skeleton including the nose, mouth, eyebrows, and chin is changed is determined. Detects facial feature points including facial features, and transfers the detected facial image and facial feature points of the subject to a matching server through a gateway server. By matching with them, identification can be performed.

The matching server standardizes the ridge intervals by obtaining the ridge intervals from the subject's fingerprint image and normalizing the ridge intervals using the maximum and minimum values of the ridge intervals, and from the subject's fingerprint image having the standardized ridge intervals, Direction information is detected, and when there are no two or more identical consecutive ridge directions among front, rear, left, and right, the ridge direction information is determined to be incorrect, and the front, rear, and left ridges with incorrect ridge direction information are determined. , As correction is made in the direction of any one of the consecutive previous ridges, the feature point is corrected, and the corrected subject's fingerprint feature point is matched with a plurality of comparison target fingerprint feature points previously stored in the database unit to perform identification. can

In addition, in the driving method of the identification system using a smartphone of the present invention, the police smart phone detects a fingerprint feature point from the subject's fingerprint image taken with a camera, and the subject's fingerprint image and fingerprint feature point are sent to a matching server through a gateway server. Passing to, the fingerprint image analysis step of the police smart phone; and an identification analysis step of the matching server, in which the matching server performs identification by matching the subject's fingerprint feature points with a plurality of comparison target fingerprint feature points stored in advance in the database unit.

The fingerprint image analysis step of the police smartphone is a fingerprint image receiving step in which the operation processing unit of the police smartphone receives a fingerprint image of a finger captured from a camera of the police smartphone or a fingerprint acquisition auxiliary device linked to the police smartphone. ; The police smart phone compares whether the fingerprint image taken in the finger fingerprint image reception step is a fingerprint of a finger or not, with the color of a previously stored finger, determines whether it is a finger print, and if it is determined that it is not a finger print, Alternatively, a finger color comparison step of notifying that it is not a finger print by voice and returning to the finger fingerprint image receiving step; After the finger color comparison step, the calculation processing unit of the police smartphone extracts the outline of the finger from the finger fingerprint image, determines whether the corresponding finger outline is smaller than the finger shooting guideline, and is within the finger shooting guideline and limits, If it is out of the limit or smaller than the finger photographing guideline, notifying by text or voice that the finger is out of the finger photographing guideline and returning to the finger fingerprint image receiving step, a fingerprint image input determination step; The operation processing unit of the police smartphone, in the fingerprint image input determination step, if the finger outline is greater than or equal to the finger shooting guideline, but within the finger shooting guideline and the limit, the fingerprint image is centered on the fingerprint guideline 100px × 100px is extracted as an image of the fingerprint image, detects horizontal ridges and vertical ridges in the fingerprint image, determines whether the horizontal ridges and vertical ridges form a pattern by repeating a certain portion of each, and if it is determined that the pattern is not formed, text or voice , a ridge check step of notifying that it is not a fingerprint image and returning to the finger fingerprint image reception step.

In the fingerprint image analysis step of the police smart phone, after the ridge check step, the arithmetic processing unit of the police smart phone detects fingerprint feature points including breakpoints and diverging points of the ridges of the fingerprint from the fingerprint image of the subject. A detection step; is further included.

The fingerprint feature detection step may include a pre-processing step of thinning the subject's fingerprint image by the calculation processing unit of the police smart phone; A candidate fingerprint feature point detection step of detecting candidate fingerprint feature points from the fingerprint image thinned in the preprocessing step by the calculation processing unit of the police smart phone; The calculation processing unit of the police smart phone includes a post-processing step of removing pseudo feature points from feature points detected in the candidate fingerprint feature point detection step.

In the identity analysis step of the matching server, the matching server calculates the amount of rotation and transition at which the feature points of the two fingerprint images overlap as much as possible between the fingerprint feature points of the subject and the fingerprint feature points of the comparison object previously stored in the database unit. an alignment step of selecting an alignment reference point, converting the coordinates of feature points to match the alignment reference point, and then determining corresponding feature point pairs; a matching step in which the matching server calculates a degree of similarity using the coordinates, types, and angle information of the pair of corresponding feature points determined in the alignment step; The matching server includes a score acquisition step of outputting, as a score, the degree of matching between the fingerprint feature points of the subject and the fingerprint feature points of the comparison object previously stored in the database based on the similarity between the corresponding feature point pairs, according to preset rule data. .

In the identity analysis step of the matching server, before the sorting step, the matching server obtains the ridge intervals for the subject's fingerprint image received from the calculation processing unit of the police smartphone and normalizes the ridge intervals using the maximum and minimum values of the ridge intervals. By doing, standardization, ridge spacing standardization step; After the ridge spacing standardization step, the matching server detects the ridge direction information from the subject's fingerprint image, and determines that the ridge direction information is incorrect when there is no identical direction of two or more consecutive ridges among front, back, left, and right. Further, a correction step of correcting the subject's fingerprint feature point by correcting the erroneous ridge direction information to the direction of any one of the consecutive ridges before, after, left, and right of the ridge having the erroneous ridge direction information. include

The identification system using a smartphone of the present invention acquires a fingerprint image and face image of a subject from a camera built in a police smartphone, detects the subject's fingerprint and facial feature points in the police smartphone, and the matching server is a police officer. The identity of the subject is confirmed by comparing the fingerprint and facial feature points received from the smartphone with the fingerprint and facial feature points of a comparison target previously registered in the database.

That is, the present invention installs a predetermined application on a smart phone or mobile device (laptop, PDF, etc.) carried by the police officer through a specific registration process such as authentication, and acquires the subject's fingerprint and face image at the scene. , can be identified more quickly and accurately.

In particular, in the identification system using a smartphone of the present invention, a predetermined application is installed in the smartphone through a specific registration process, and security functions are improved through user authentication during use.

In addition, it is possible to establish a database for drunk people or elderly people with dementia, etc., and confirm the subject's identity in a one-stop manner by linking with the database.
In the present invention, the fingerprint of the examinee is photographed with a camera, or the fingerprint of the examinee is photographed from a fingerprint acquisition assisting device equipped with a camera to obtain a fingerprint image, and from the obtained fingerprint image, the end point of the ridge of the fingerprint and A police smartphone that detects a fingerprint feature point including a divergence point, a gateway server that receives a fingerprint image and a fingerprint feature point of an examinee from the police smartphone and transfers the received fingerprint image and fingerprint feature point to a matching server, and receives the fingerprint image and fingerprint feature point of the examinee from the gateway server and provides an identification system using a smart phone including a matching server that performs identification by matching the subject's fingerprint feature points with the fingerprint feature points of a plurality of comparison objects previously stored in the database unit.
In addition, in the present invention, the police smartphone captures the subject's face with the camera of the police smartphone to acquire the subject's face image, and from the obtained face image, each part where the facial skeleton including the nose, mouth, eyebrows, and chin changes Detects a facial feature point including the position of the subject, and transfers the detected facial image and facial feature point of the subject to a matching server through a gateway server, and the matching server includes the subject's facial feature point and a plurality of comparison targets previously stored in the database unit. Provided is an identification system using a smartphone that performs identification by matching with facial feature points.
In addition, in the present invention, the matching server detects ridge direction information from the subject's fingerprint image, and determines that the ridge direction information is incorrect when there is no one identical to two or more consecutive ridge directions among front, back, left, and right, , As corrected in the direction of any one of the ridges before, after, left, or right of the ridge with incorrect ridge direction information, the feature point is corrected, and the corrected subject's fingerprint feature point and a number of previously stored in the database unit Provided is an identification system using a smart phone that performs identification by matching with fingerprint feature points of a comparison target.
In addition, the present invention, the police smartphone detects the fingerprint feature point from the subject's fingerprint image captured by the camera, and transfers the subject's fingerprint image and fingerprint feature point to the matching server through the gateway server, the fingerprint image analysis step of the police smartphone ; an identification analysis step of the matching server, in which the matching server performs identification by matching the subject's fingerprint feature points with the fingerprint feature points of a plurality of comparison objects previously stored in the database unit; It provides a driving method of an identification system using a smartphone including a.
In addition, in the present invention, in the identity analysis step of the matching server, the matching server rotates and transitions at which the characteristic points of the two fingerprint images of the subject's fingerprint characteristic points and the fingerprint characteristic points of the comparison object previously stored in the database overlap as much as possible. an alignment step of determining a pair of corresponding feature points after selecting an alignment reference point of the image by calculating , converting the coordinates of the feature point to match the alignment reference point; a matching step in which the matching server calculates a degree of similarity using the coordinates, types, and angle information of the pair of corresponding feature points determined in the alignment step; In the matching server, a score acquisition step of outputting the degree of matching between the subject's fingerprint feature points and the fingerprint feature point of the comparison object previously stored in the database from the similarity of the corresponding feature point pair as a score according to preset rule data; , and provides a driving method of an identification system using a smartphone.

1 is a schematic diagram for schematically explaining an identification system using a smartphone of the present invention.
FIG. 2 is an example of a fingerprint acquisition assisting device applied to the smart phone of FIG. 1 .
3 is an explanatory diagram for explaining image improvement according to direction information.
4 is an explanatory diagram for explaining distortion of a general fingerprint image.
5 is an explanatory diagram for explaining feature information coordinate change according to rotation angle determination.
6 is an example of outputting matching feature points between fingerprints to be compared and connection information between feature points.
7 is an explanatory diagram for explaining the use of feature information similarity around corresponding feature information when checking feature information similarity.
8 is an explanatory diagram for schematically explaining a fingerprint recognition process in a matching server.
9 is an example of capturing the index finger of the right hand on the fingerprint capturing screen of the police smartphone according to the present invention.
FIG. 10 is a diagram explaining whether or not a finger selected on the fingerprint capturing screen of FIG. 9 matches a finger capturing guideline.

Hereinafter, the identification system using a smartphone of the present invention will be described in detail with reference to the accompanying drawings.

In the present invention, a smart phone or mobile device (laptop, PDF, etc.) carried by a police officer is referred to as a 'police smart phone'.

1 is a schematic diagram for schematically explaining an identification system using a smartphone of the present invention.

With the police smartphone 100, a smartphone in which a predetermined application is installed through a specific registration process such as authentication, the fingerprint or face image of the subject is photographed on the police smartphone 100, and the fingerprint or face image is captured. When a feature point of a fingerprint or face is detected, an identification group is selected, and an identification request is selected, a feature point is detected in the subject's fingerprint image or face image, and the identification request signal, identification group signal, and fingerprint feature points are detected. , facial feature points are transmitted to the gateway (G/W) server 200.

Here, the identification group can be selected and entered by age group, gender, etc., and also can be set such as a drunkard, an elderly person with dementia, a missing child, a person who has died by accident, a foreigner, a wanted person, and a criminal.

In the present invention, fingerprint acquisition is possible without a fingerprint acquisition assistant, but a fingerprint acquisition assistant may be used in some cases.

In the case of fingerprint recognition, in the case of contact technology, the resolution is fixed at 500 dpi, but in the case of a smartphone in general, the misrecognition rate may be high during matching because the resolution is different depending on the distance at each point of time when the subject is photographed.

In general, when a fingerprint is acquired by a fingerprint sensor, the shape of the fingerprint is deformed according to the pressure of the position where the finger is applied. resolution compatibility.

In some cases, instead of the police smartphone 100 and the fingerprint acquisition assistant, the police smartphone 100 may receive and use a fingerprint image acquired using a live scanner, which is an existing fingerprint input device. .

The gateway (G/W) server 200 receives the identification request signal, identification group signal, fingerprint feature points, and facial feature points received from the police smart phone 100 and transmits them to the matching server 300.

The matching server 300 performs identification by comparing the fingerprint feature points of the identification group previously stored in the database unit 370 with the subject's fingerprint feature points received through the gateway server 200, and also performs data verification. Identity verification is performed by comparing the facial feature points of the identification group previously stored in the bus base unit 370 with the subject's facial feature points received through the gateway server 200, and the result is transmitted to the police through the gateway server 200. It is transmitted to the smartphone 100.

FIG. 2 is an example of a fingerprint acquisition assisting device applied to the smart phone of FIG. 1 .

The fingerprint acquisition assisting device 170 of FIG. 2 is formed in the form of a finger clip, that is, a thimble, and has a fingerprint detection sensor 177 at a portion where the bottom of the finger is closed.

The housing of the fingerprint acquisition assisting device 170 is formed in the form of a finger clip, that is, a thimble, which surrounds the front and rear surfaces of the subject's fingers and applies horizontal pressure to the fingers. A finger print is formed by (177).

The fingerprint detection sensor 177 is a means for obtaining image information of a fingerprint, detects a fingerprint image, and transmits the fingerprint image to a calculation processing unit (not shown) of the police smart phone 100. The fingerprint detection sensor 177 It can be a camera or an image sensor.

3 is an explanatory diagram for explaining image improvement according to direction information, FIG. 4 is an explanatory diagram for explaining distortion of a general fingerprint image, and FIG. 5 is an explanatory diagram for explaining feature information coordinate change according to rotation angle determination. FIG. 6 is an example of outputting matching feature points between fingerprints to be compared and connection information between feature points, and FIG. 7 is an explanatory diagram for explaining the use of feature information similarity around the corresponding feature information when checking the feature information similarity. .

Figure 3 (a) shows an example of low image quality. In Figure 3 (a), the green line is direction information that should actually be displayed (ie, direction information indicated by an expert), and Figure 3 (b) shows As an example of analyzing the direction information in the fingerprint image of FIG. 3(a) as a conventional analysis method, it can be seen that the direction information is properly wrong in the area within the square of FIG. 3(a). That is, there is a problem with direction information. FIG. 3(c) is an example of correcting a fingerprint image to improve direction information in the fingerprint image of FIG. 3(a).

That is, the present invention improves image quality by improving direction information by analyzing patterns of images.

Fig. 4(a) is an example of downward twisting, Fig. 4(b) is an example of upward twisting, Fig. 4(c) is an example of right twisting, and Fig. 4(d) is an example of left twisting. am.

That is, when a subject takes a fingerprint, image distortion occurs according to the pressure of the finger, that is, the direction in which the force of the finger is applied. Accordingly, the matching server 300 detects the distortion direction of the image when comparing fingerprints and corrects the distortion so that the image fits into a square.

In the present invention, the matching server 300 multiplexes rotation and displacement selection to improve matching performance of deformed images.

FIG. 5(a) shows an example of setting an angle change of an input fingerprint image, and FIG. 5(b) shows a fingerprint image whose coordinates are changed according to the angle change of FIG. 5(a).

The matching server 300 attempts matching using a plurality of rotation values because the rotation angle may vary depending on the image position when the rotation angle determination has a great influence on the feature information coordinates and the image deformation is large.

Figure 6 (a) is an example of a fingerprint stored in the database unit, and Figure 6 (b) is an example of a subject's fingerprint. Matching feature points between the two fingerprints are indicated by a red line, and connection information between feature points is indicated by a blue line. indicated by a line.

Visual fingerprint recognition result expression information is obtained from the police smart phone 100, and the matching server 300 receives it and uses it as an aid when determining whether the fingerprint is the same. That is, if it is difficult to determine whether the matching server 300 has the same fingerprint, it can determine using visual fingerprint recognition result expression information.

As shown in FIG. 6, it is possible to determine whether the fingerprints are the same or not by outputting matching feature points between fingerprints to be compared and connection information between feature points. Even when the distance between feature information or the value of the feature point direction changes due to fingerprint deformation, the deviation is small because the test feature point and surrounding information move together. Therefore, as shown in FIG. 7, similarity of feature information around the corresponding feature information is taken into consideration when testing feature information similarity.

Figure 7 (a) shows the surrounding feature points (M1 to M5) of the feature point (M-r) in the fingerprint stored in the database unit, and (b) of FIG. In M5), the similarity of the feature points around the feature point to be compared is examined, and the feature information similarity is detected using the similarity review result.

In the present invention, since index information is used for 1:N comparison in order to improve fingerprint recognition speed in the matching server 300, the speed is faster than when other algorithms are applied (eg, analyzing about 100,000 people per second). Since the types of fingerprint types, which are static classifications, are limited, the present invention dynamically selects objects and performs a high-speed search.

8 is an explanatory diagram for schematically explaining a fingerprint recognition process in a matching server.

In the fingerprint feature information processing step (S100), fingerprint feature information is processed for high-speed search.

In the preliminary target selection step (S200), the similarity is checked in the feature information processed in the fingerprint feature information processing step (S100), the fingerprint feature information stored in the database unit is sorted according to the similarity score, and the fingerprints stored in the aligned database unit are sorted. Among the feature information, a preliminary target determined to be similar to the subject's fingerprint feature information is selected. In the preliminary target selection step (S200), the number (size) of preliminary targets (candidates) is determined in consideration of the DB size of the test target group stored in the database unit.

9 is an example of capturing a right index finger on the fingerprint capturing screen of a police smartphone according to the present invention, and FIG. 10 is a diagram explaining whether a finger selected on the fingerprint capturing screen of FIG. 9 matches a finger capturing guideline.

9(a) is an example of selecting a right index finger on the fingerprint capture screen, indicating clicking the right index finger selection button in the right and left pictures at the bottom of the fingerprint capture screen. A finger capture guideline is displayed in the middle of the fingerprint capture screen, and a fingerprint guideline (a blue circle in the case of FIG. 9 (a)) is displayed within the finger capture guideline. Mark the limit (triangle) of the range that can deviate from this guideline. In addition, a message requesting to proceed with photographing by placing a predetermined finger selected in the finger photographing guideline is displayed on the top of the fingerprint photographing screen.

9(b) is an example of photographing by contacting the index finger of the right hand on the fingerprint photographing screen.

10(a) is an example of a case where the index finger of the right hand, which is a finger selected on the fingerprint capture screen, deviated from the finger capture guideline, and FIG. 10(c) is an example of a case where the index finger of the right hand coincides with the finger capturing guideline on the fingerprint capturing screen.

As shown in (c) of FIG. 10, if the boundary line of the finger is located between the finger capturing guideline and the limit point, then the fingerprint to be photographed is appropriately located within the fingerprint guideline.

In the present invention, a mobile-only app was developed to acquire a fingerprint image from the camera of the police smart phone 100 or the fingerprint acquisition assist device 177. When acquiring a fingerprint image through a mobile app, the app presents guidelines for finger capture for the fingerprint image input part, and checks three points for the finger input in the guideline, that is, the fingerprint image of the finger through color comparison It is possible to acquire a high-quality fingerprint image by checking whether or not the fingerprint image is properly input within the guideline, and checking whether the fingerprint image is present through the ridge check.

Next, a process of photographing a fingerprint and detecting a feature point in an operation processing unit (not shown) of the police smart phone 100 will be described.

In the finger fingerprint image receiving step, the operation processing unit of the police smartphone 100 receives the fingerprint image captured from the camera of the police smart phone or the fingerprint acquisition assisting device 170 interlocked with the police smart phone.

In the finger color comparison step, the calculation processing unit of the police smart phone 100 compares whether the finger fingerprint image taken in the finger fingerprint image receiving step is a fingerprint of a finger or not, with a pre-stored finger color, and compares whether it is a finger print or not. is primarily determined, and if it is determined that it is not a finger print, it is notified by text or voice that it is not a finger, and returns to the step of receiving a fingerprint image.

In the fingerprint image input determination step, after the finger color comparison step, the calculation processing unit of the police smart phone 100 extracts the outline of the finger from the finger fingerprint image, determines whether the corresponding finger outline is smaller than the finger shooting guideline, and determines whether the finger It is determined whether it is within the photographing guideline and the limit, and if it is out of the limit or smaller than the finger photographing guideline, it is notified by text or voice, and returns to the finger fingerprint image receiving step. At this time, the calculation processing unit of the police smart phone 100 determines that an appropriate fingerprint image is input within the fingerprint guideline, if the corresponding finger outline is greater than or equal to the finger capture guideline, but within the finger capture guideline and the limit. If it is out of the range or smaller than the finger capture guideline, it is determined that an appropriate fingerprint image is not input within the fingerprint guideline.

In the ridge check step, in the step of determining whether a fingerprint image is input, if it is determined that the corresponding finger outline is greater than or equal to the finger capture guideline but within the finger capture guideline and the limit, and an appropriate fingerprint image is input within the fingerprint guideline, the police The calculation processing unit of the smartphone 100 extracts the fingerprint image as a 100px×100px image based on the fingerprint guideline, detects the horizontal ridge and the vertical ridge in the fingerprint image, and the horizontal ridge and the vertical ridge If a pattern is formed by repeating a certain portion of each, it is determined that it is a fingerprint image of a finger, and if a pattern is not formed, it is determined that it is not a fingerprint image, and it is notified by text or voice that it is not a finger image, and returns to the fingerprint image receiving step. .

In the feature information detection step, the operation processing unit of the police smart phone 100 extracts feature information from the finger fingerprint image, which includes a pre-processing step, a feature point extraction step, and a post-processing step. It proceeds in 3 stages.

The pre-processing step is a step of thinning the fingerprint image, image enhancement is performed, binarization is performed on the image-enhanced fingerprint image, and thinning is performed. That is, in the preprocessing step, a conversion process into a block directional image, a binary image, and a thinned image is performed. The fingerprint image is divided into blocks of a certain size, and the direction representing the flow of ridges is determined for each block to be converted into a directional image for each block. The obtained ridge direction information for each block is used in the binarization and smoothing processes. During the binarization process, the fingerprint image is converted into a binary image expressed only in black and white. The binary image is then subjected to a smoothing process that removes noise and emphasizes the fusion, thereby improving the connectivity of the fusion and converting the fusion into a line with a thickness of 1 pixel. It is expressed and converted into a thinned image,

The feature point extraction step is a step of extracting candidate (preliminary target) feature points from the fingerprint image thinned in the preprocessing step. Here, a minutiae refers to a point at which a change occurs in the flow of a ridge, called a point where the flow of a ridge is cut off and a divergence point (a point where one ridge splits into two). The feature amount (T={m1, m2, , mm}) is composed of information on the feature point (mi) existing in the fingerprint image. In most cases, the feature point information used is a kind of feature point that indicates which of the divergence point and the demerit point. , the location in the fingerprint image, and the direction of the ridge where the feature point is located.

That is, the step of extracting feature points is a step of extracting candidate feature points, and stores information on points and branch points where changes occur in the flow of ridges from the thinned image. Candidate feature points including pseudo feature points are extracted from the ridge information of the thinned image. At this time, a false feature point may be generated due to an incorrect ridge of the thinned image. The pseudo feature point refers to a fake feature point generated during the thinning process due to noise at the time of fingerprint acquisition. Pseudo feature points increase the amount of meaningless computation and increase errors, thereby degrading the performance of the system.

The post-processing step is a step of removing fake feature points from the feature points extracted in the feature point extraction step. That is, in the post-processing step, a final feature point is extracted by removing the pseudo feature point by correcting the ridge portion that causes the pseudo feature point, and adding the missing feature point.

In the feature information transmission step, the operation processing unit of the police smart phone 100 transmits the feature information and the fingerprint image composed of the candidate feature points detected in the feature information detection step to the matching server 300 through the gateway server 200,

Next, an identification process in the matching server 300 will be schematically described.

In the ridge interval standardization step, the ridge interval is obtained for the fingerprint image received from the calculation processing unit of the police smartphone 100, and the ridge interval is normalized using the maximum and minimum values of the ridge interval to have a standardized interval. .

As a correction step, by analyzing the ridge direction information from the fingerprint image received from the calculation processing unit of the police smartphone 100, the direction information is incorrect (for example, when there are no two or more identical directions of consecutive ridges in the front, rear, left, and right directions) The direction information of is corrected (for example, the direction of the ridge before the wrong part is corrected).

In the fingerprint matching step, the matching server 300 uses the feature amount defined from the feature point information extracted from the calculation processing unit of the police smart phone 100 by using the matching algorithm of the fingerprint recognition method using the feature point to two fingerprint images. This step of determining the degree of similarity between images includes three steps: an image alignment step, a matching step, and a scoring step.

In the alignment step, the alignment reference point of the images is selected by calculating the amount of rotation and transition where the feature points of the two fingerprint images overlap the most. Then, after transforming the coordinates of feature points to match the alignment reference points, corresponding feature point pairs are determined. At this time, in order to find an ideal amount of rotational shift, it is necessary to perform as many candidate alignments as possible.

In the matching step, a similarity (Measure Vector) is calculated using the coordinates, type, and angle information of the pair of corresponding feature points determined in the alignment step. In order to realistically reflect the similarity of corresponding feature point pairs, various statistical techniques can be considered from various angles in the similarity determination method.

In the scoring step, the degree of matching between the two fingerprint images from the similarity (Measure Vector) of the pair of corresponding feature points is expressed as a score according to preset rule data. Rule data is pre-determined data using statistical grounds and mathematical modeling in decision making to accurately reflect the matching degree of images, similar to the matching step.

As a preliminary target selection step, the fingerprint feature information of the database unit in which the score obtained according to the similarity of the corresponding feature point pair in the scoring step exceeds a predetermined threshold value is selected as a preliminary target, or the fingerprint feature of the database unit with the highest score is selected. Information is selected as a preliminary target and transmitted to the police smartphone through the gateway server.

Existing traditional fingerprint image acquisition devices have been obtained using a live scanner, but in the present invention, a fingerprint image acquired from a camera of a conventional live scanner or police smartphone 100 or a camera of a fingerprint acquisition assistant 177 is compared Verify your identity.

However, in general, when a fingerprint image is acquired with a camera, if the distance from the camera to the finger is not constant, compatibility with a fingerprint image acquired with an existing device cannot be guaranteed. Therefore, the present invention, in order to standardize the fingerprint images acquired differently according to the live scanner, the camera of the police smartphone 100, etc., obtains the interval of the ridge for the acquired fingerprint image, improves the image based on the standardized interval, and improves A fingerprint template is created through the image.

The matching server 300 includes a matching controller unit (MCU) (not shown), a matching process unit (MPU) (not shown), and a matching monitoring unit (MMU) (not shown). includes

The MCU is a means for overall control of the matching server 300, and its main functions are fingerprint matching task management and MPU management.

The fingerprint matching task management function of the MCU fetches the fingerprint identification request information received from the G/W server 200, pre-stored data from the DB unit 370, requests a matching task to the MPU connected to the MCU, and MPU Matching results are collected to determine candidates and the results are delivered to the G/W server.

The MPU management function of the MCU monitors the status of the MPU through the MMU and manages the starting and stopping of the MPU. If a process of a specific MPU is stopped or deadlocked, the MCU can restart the process of the MPU after checking the status through MPU monitoring.

The MPU analyzes the fingerprint matching task requested from the MCU to determine the matching target, compares the feature points of the requested fingerprint and the target fingerprint, and transmits target fingerprint information with a high degree of similarity to the MCU.

The MMU monitors the status of the MPU interworking with the MCU and inquires the operation log of the MPU. The MMU transmits a status check message to the MPU, and the received MPU checks its own status and sends whether or not it operates normally to the MMU. If the MMU does not receive a message from the MPU normally or receives a message in an abnormal state, the MMU transmits the operating status of the MPU to the MCU, and the MCU sends a message to the MPU to restart the process to restart the MPU function. .

In the present invention, the matching server and database unit may be referred to as a fingerprint matching system.

The fingerprint data of the fingerprint matching system is managed to optimize the performance of the MPU by building in a distributed environment. Distributed processing of fingerprint data is performed in two ways: real-time distributed processing and batch distributed processing.

Real-time distributed processing is to distribute and process the changed fingerprint data to each MPU while the fingerprint matching system is operating. When it is detected that the change is detected, the changed fingerprint data is distributed and audited to ensure consistency of the data.

Batch distribution processing can be performed when an initial fingerprint matching system is built or when fingerprint data in the main database is batch distributed to the MPU when there are many changes in fingerprint data. The basic data is distributed and audited to ensure consistency of fingerprint data.

As described above, the present invention has been described by the limited embodiments and drawings, but the present invention is not limited to the above embodiments, and those skilled in the art in the field to which the present invention belongs can make various modifications and transformation is possible Therefore, the spirit of the present invention should be grasped only by the claims described below, and all equivalent or equivalent modifications thereof will be said to belong to the scope of the spirit of the present invention.

100: police smartphone 170: fingerprint acquisition assist device 177: fingerprint detection sensor
200: gateway server 300: matching server 370: database unit

Claims (11)

Acquiring a fingerprint image by photographing the examinee's fingerprint with a camera or photographing the examinee's fingerprint from a fingerprint acquisition assisting device equipped with a camera, and from the obtained fingerprint image, including the end point and branch point of the ridge of the fingerprint A police smart phone that detects fingerprint feature points;
A gateway server that receives the subject's fingerprint image and fingerprint feature points from the police smartphone and transmits them to the matching server;
a matching server that receives a fingerprint image and fingerprint characteristics of the examinee from the gateway server, matches the fingerprint characteristics of the examinee with the fingerprint characteristics of a plurality of comparison objects previously stored in the database unit, and performs identification;
In the identification system using a smartphone comprising a,
The matching server detects ridge direction information from the subject's fingerprint image, and determines that the ridge direction information is incorrect when there is no one of the same consecutive ridge directions among front, back, left, and right, and determines that the ridge direction information is incorrect. By correcting in the direction of any one of the ridges before, after, left, and right of the ridge with information, the subject's feature point is corrected,
The police smartphone captures the subject's face with the camera of the police smartphone to obtain the subject's face image, and from the obtained face image, the facial skeleton including the nose, mouth, eyebrows, and chin is changed. Detect facial feature points, and deliver the detected subject's face image and facial feature points to the matching server through the gateway server;
The matching server is an identification system using a smartphone, characterized in that the identification is performed by matching the subject's facial feature points with the facial feature points of a plurality of comparison targets previously stored in the database unit. According to claim 1,
The matching server is an identification system using a smartphone, characterized in that for performing identification by matching the corrected subject's fingerprint feature points with the fingerprint feature points of a plurality of comparison objects previously stored in the database unit. delete According to claim 2,
matching server,
Standardize the ridge intervals by obtaining the ridge intervals from the subject's fingerprint image and normalizing the ridge intervals using the maximum and minimum values of the ridge intervals,
From the fingerprint image of the subject with a standardized ridge interval, ridge direction information is detected, and when there is no one of the consecutive ridge directions of the front, rear, left, and right, and two or more are identical, the ridge direction information is determined to be incorrect, As correction is made in the direction of any one of the ridges before, after, left, or right of the ridge with incorrect ridge direction information, the feature point is corrected, and the corrected subject's fingerprint feature point is compared with a number of previously stored in the database unit. An identification system using a smartphone, characterized in that identification is performed by matching with fingerprint feature points of a subject. A fingerprint image analysis step of the police smart phone, in which the police smart phone detects a fingerprint feature point from the subject's fingerprint image captured by the camera, and transmits the subject's fingerprint image and fingerprint feature point to the matching server through the gateway server;
an identification analysis step of the matching server, in which the matching server performs identification by matching the subject's fingerprint feature points with the fingerprint feature points of a plurality of comparison objects previously stored in the database unit;
Including,
The identity analysis step of the matching server is,
The matching server detects ridge direction information from the subject's fingerprint image and determines that the ridge direction information is incorrect when there are no two or more identical consecutive ridge directions among front, back, left, and right, and incorrect ridge direction information A correction step of correcting the subject's fingerprint feature point by correcting in the direction of any one of the ridges before, after, left, or right of the ridge having incorrect ridge direction information;
After the correction step, the matching server selects the alignment reference point of the images by calculating the rotation and transition amount at which the feature points of the two fingerprint images overlap as much as possible between the fingerprint feature points of the subject and the fingerprint feature points of the comparison object previously stored in the database unit. and an alignment step of determining a pair of corresponding feature points after converting the coordinates of the feature points to match the alignment reference points;
a matching step in which the matching server calculates a degree of similarity using the coordinates, types, and angle information of the pair of corresponding feature points determined in the alignment step;
a score acquisition step of outputting, by the matching server, the degree of matching between the fingerprint feature points of the subject and the fingerprint feature points of the target of comparison previously stored in the database based on the similarity between the pair of corresponding feature points as a score according to preset rule data;
Characterized in that comprising a, driving method of the identification system using a smartphone. The method of claim 5, wherein the fingerprint image analysis step of the police smart phone,
A fingerprint image receiving step of receiving a fingerprint image captured by a camera of a police smart phone or a fingerprint acquisition auxiliary device linked with the police smart phone by an operation processing unit of the police smart phone;
The police smart phone compares whether the fingerprint image taken in the finger fingerprint image reception step is a fingerprint of a finger or not, with the color of a previously stored finger, determines whether it is a finger print, and if it is determined that it is not a finger print, Alternatively, a finger color comparison step of notifying that it is not a finger print by voice and returning to the finger fingerprint image receiving step;
After the finger color comparison step, the calculation processing unit of the police smartphone extracts the outline of the finger from the finger fingerprint image, determines whether the corresponding finger outline is smaller than the finger shooting guideline, and is within the finger shooting guideline and limits, If it is out of the limit or smaller than the finger photographing guideline, notifying by text or voice that the finger is out of the finger photographing guideline and returning to the finger fingerprint image receiving step, a fingerprint image input determination step;
The operation processing unit of the police smartphone, in the fingerprint image input determination step, if the finger outline is greater than or equal to the finger shooting guideline, but within the finger shooting guideline and the limit, the fingerprint image is centered on the fingerprint guideline 100px × 100px is extracted as an image of the fingerprint image, detects horizontal ridges and vertical ridges in the fingerprint image, determines whether the horizontal ridges and vertical ridges form a pattern by repeating a certain portion of each, and if it is determined that the pattern is not formed, text or voice , a ridge check step of notifying that it is not a fingerprint image and returning to the finger fingerprint image receiving step;
Characterized in that it comprises a, driving method of the identification system using a smartphone. The method of claim 6, wherein the fingerprint image analysis step of the police smart phone,
After the ridge check step, the operation processing unit of the police smart phone detects fingerprint feature points including end points and branch points of the ridges of the fingerprint from the subject's fingerprint image, a fingerprint feature detection step;
Characterized in that it further comprises, driving method of the identification system using a smartphone. The method of claim 7, wherein the fingerprint feature detection step,
A pre-processing step of thinning the subject's fingerprint image by the calculation processing unit of the police smart phone;
A candidate fingerprint feature point detection step of detecting candidate fingerprint feature points from the fingerprint image thinned in the preprocessing step by the calculation processing unit of the police smart phone;
A post-processing step of removing pseudo feature points from the feature points detected in the candidate fingerprint feature point detection step by the calculation processing unit of the police smart phone;
Characterized in that comprising a, driving method of the identification system using a smartphone. delete The method of claim 5, wherein the identity analysis step of the matching server,
Before the correction step, the matching server obtains the ridge intervals for the subject's fingerprint image received from the calculation processing unit of the police smartphone and normalizes the ridge intervals using the maximum and minimum values of the ridge intervals, thereby standardizing the ridge intervals. standardization stage;
Method of driving an identification system using a smartphone, characterized in that it further comprises. According to claim 1,
The fingerprint acquisition assist device forms a thimble and is characterized in that it has a camera on the contact part of the bottom of the finger, an identification system using a smartphone.

Images