KR101620556B1 - Method for Biometric Detection - Google Patents

Abstract

One embodiment of the present invention provides a biometric method capable of estimating joint information of a living body with a small number of feature points using depth information.
To this end, a biometric identification method according to an embodiment of the present invention includes generating a depth image of a subject using a 3D sensor, generating a histogram representing a distribution of depth values, separating the background through the depth image, A step (B) of estimating a head from the depth image from which the background is separated, designating a body as a region of interest (C), extracting a joint of the body in the region of interest (D) (F) generating a plurality of random points on the geodesic distance image, and inputting the plurality of random points to a SVM (Support Vector Machine) And a step (G) of sorting the parts.

Description

{Method for Biometric Detection}

One embodiment of the present invention relates to a biometric method.

In the transition to the information society, the information of individuals as well as specific organizations (hereinafter referred to as "personal information") has more value than any assets. In order to protect such important personal information, various passwords are used and a technique for identifying other persons is urgently required. In addition to protecting information, social needs such as credit cards, cash cards, and electronic identification cards are still expanding. However, since there is no secondary identification method other than passwords, there are many social problems such as computer crime have. In recent years, a biometric technique for identifying an individual using biometric information of a person such as a fingerprint, iris, or vein has been attracting attention as a method for solving such problems.

Japanese Patent Application Laid-Open No. 2001-0056007 (20010704)

One embodiment of the present invention provides a biometric method capable of estimating joint information of a living body with a small number of feature points using depth information.

In addition, one embodiment of the present invention provides a biometrics method that can be used at night and can be operated in an embedded-based system since only depth information is used.

A biometric identification method according to an embodiment of the present invention includes generating a depth image of a subject using a 3D sensor, generating a histogram representing a distribution of depth values, separating a background through the depth image, (C) extracting the joints of the living body in the region of interest, (D) extracting the center of the region of interest from the center of the region of interest (F) generating a plurality of random points on the geodesic distance image, and inputting the plurality of random points to a SVM (Support Vector Machine) to generate a joint part of a living body (G). ≪ / RTI >

In the step (A), a preprocessing process of converting the depth image into a form required by an application program may be performed.

In the step (C), the method of estimating the head includes calculating the head diameter of the living body, adjusting the size of the two-dimensional head template according to the head diameter of the living body, chamfer distance matching, Extracting a head candidate region of the living body, matching the extracted region with the 3D head template, and returning the head position and diameter of the living body.

The step of performing the chamfer distance matching may include extracting a boundary line, generating a distance transform image, and matching the two-dimensional head template, with the background separated depth images .

In the step (F), the coordinates of the plurality of random points may be the origin of the center point.

In the step (F), the coordinates of the plurality of random points may be normalized by applying the predetermined distance to the 3D sensor and the object.

In step (F), the SVM may classify the joint part into a head, a neck, a torso, a left shoulder, a right shoulder, a left arm, a right, a left hip, a right hip, a left leg and a right leg.

The method may further include, after the step (G), classifying the arm of the joint part into an elbow and a wrist joint, and classifying the leg into a knee and ankle joint.

Wherein the step (H) includes the steps of: generating a rectangle including two short sides and two long sides by wrapping the random points included in the joint part classified by the arm in a minimum width; Classifying the points having the greatest geodesic distance value into wrist joints and classifying the point that is 3/4 from the center of the quadrangle to the center of the other short side of the quadrangle into the elbow joints.

Wherein the step (H) includes the steps of creating a rectangle having two short sides and two long sides by wrapping the random points included in the joint part classified by the legs with a minimum width, Classifying the point having the greatest geodesic distance value into the ankle joints and classifying the points that are 3/4 from the center of the quadrangle to the center of the other side of the quadrangle into knee joints.

The biometric method according to an embodiment of the present invention can estimate the joint information of a living body with a small number of feature points using depth information.

In addition, the biometrics method according to an embodiment of the present invention uses only depth information, so that it can be used at night and can also be operated in an embedded-based system.

FIG. 1 is a flowchart illustrating a biometric detection method according to an exemplary embodiment of the present invention.
FIGS. 2A and 2B are front and back images of a biometric method according to an exemplary embodiment of the present invention.
3 is an image showing a result of biometric detection in the biometric method according to an embodiment of the present invention.
FIG. 4 is a flowchart illustrating a head estimation / detection step of a biometric method according to an exemplary embodiment of the present invention.
FIG. 5A is an image showing a two-dimensional hair template in the biometric method according to an embodiment of the present invention, and FIG. 5B is an image showing a three-dimensional hair template.
6 is an image showing the result of head detection in the biometric method according to an embodiment of the present invention.
FIG. 7 is a flowchart illustrating a joint extracting step in the biometric method according to an exemplary embodiment of the present invention.
8 is an image showing a geodesic distance in the biometric method according to an embodiment of the present invention.
9 is an image showing a coordinate change of a random point in the biometric method according to an embodiment of the present invention.
10 is a graph showing a relationship between a depth of a subject and a shoulder width of a living body in the biometric method according to an embodiment of the present invention.
11 is a graph showing normalization of coordinates of a random point in the biometric method according to an embodiment of the present invention.
12 is an image showing a result of classification of a living body joint through SVM in the biometric method according to an embodiment of the present invention.
FIG. 13 is a flowchart showing a step of classifying an arm (or a leg) into an elbow (or a knee) and a wrist (or ankle) joint in the biometric method according to an embodiment of the present invention.
FIG. 14 is an image showing a procedure and a result of classification into an arthroscopic elbow and a wrist joint in the biometric method according to an embodiment of the present invention.
FIG. 15 is an image showing a result of extracting a joint of a living body in the biometric method according to an embodiment of the present invention. FIG.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings, so that those skilled in the art can easily carry out the present invention.

As used herein, the term "and / or" includes any and all combinations of one or more of the listed items. In addition, the terminology used herein is for the purpose of describing particular embodiments only, and is not intended to be limiting of the invention. In addition, as used herein, the singular forms "a," "an," and "the" include plural referents unless the context clearly dictates otherwise. Furthermore, " comprise "and / or" comprising "as used herein specify the presence of stated steps, operations, elements, elements, numerical values and / But does not preclude the presence or addition of other steps, operations, elements, elements, numerical values and / or groups.

FIG. 1 is a flowchart illustrating a biometric detection method according to an exemplary embodiment of the present invention. FIGS. 2A and 2B illustrate an example of a biometric method according to an exemplary embodiment of the present invention. FIG. 3 is an image showing the result of biometric detection in the biometric method according to an embodiment of the present invention.

First, referring to FIG. 1, noise is reduced for a depth image input through a 3D sensor (for example, a depth camera, etc.) and a pre-operation for image size adjustment is performed. Here, the preprocessing process performs an average filter for noise attenuation and reduces the size of the input depth image in half for fast image processing.

Thereafter, referring to Figs. 2A and 2B, a background removal step is performed to detect a living body (hereinafter referred to as a person). The background removal step is performed through background model learning for a few seconds, assuming that the camera is fixed at a certain position.

The background removal algorithm is based on the adaptive gaussian mixture model. The background values of each pixel value of the image are obtained through several frames using several Gaussian mixture distributions rather than a single Gaussian distribution. It is a method to remove through the difference with the original image. Therefore, when learning the background model, there must be no human or other moving object to perform the learning work correctly.

That is, when the background model is removed from the depth image input every frame after learning the background model, only the depth image corresponding to the foreground remains, and all the pixels corresponding to the background are output as 0s. Therefore, when people enter the field of view of the camera, the foreground image with the background removed contains only the information of the person concerned.

Referring to FIG. 3, the rectangles surrounding the blobs of the foreground are obtained. When the rectangles are designated as regions of interest (hereinafter, referred to as ROIs), this is the ROI of the people.

At this time, those of the foreground blurbs that are too small are regarded as noise and removed. However, here, when two or more people are adjacent to each other, the ROI of the adjacent persons may be combined into one. In this case, the head detection step is performed on the extracted ROI.

Here, if two or more people are included in one ROI using the face information of the person obtained through head detection, the ROI for each person is separated. When two or more faces are detected in the ROI, the ROI is divided by dividing the center of the X-axis between two adjacent faces.

FIG. 4 is a flowchart illustrating a method of estimating / detecting a head in a biometric method according to an exemplary embodiment of the present invention. FIG. 5A is a diagram illustrating an image of a two-dimensional head template in a biometric method according to an exemplary embodiment of the present invention, FIG. 5B is an image showing a three-dimensional head template, and FIG. 6 is an image showing a result of head detection in the biometric method according to an embodiment of the present invention.

Next, with reference to Figs. 4 to 6, the above-described head detecting step will be described.

First, referring to FIG. 4, the head detection algorithm finds a position estimated to be a human face using a modified template matching technique called chamfer distance matching.

In the case of chamfer distance matching, the object to be searched can be found only when the image size of the template and the object to be searched are the same as the template matching. However, the size of a person's face changes depending on the relative position of the person to the camera. Due to the problem that the size of the face changes, a fixed size template can not detect a face that is smaller than or larger than the template size. To solve this problem, the head detection algorithm calculates the depth value of the point where the person exists and adjusts the size of the template by estimating the size of the human face through the depth value. In this case, the depth of the point where the person exists is obtained through the depth histogram of the input image subjected to the background removal. When this modified method is used, the number of chamfer distance matching is controlled more flexibly than the conventional method, which is advantageous in fast calculation.

Referring to FIG. 4, the head detection algorithm will be described in detail.

First, a histogram representing the distribution of the depth value of the input image is generated to determine the depth value at which a person is expected to exist for the ROI image from which the background has been removed. Here, since the background has been removed, the ROI image contains only the depth information of a person at a certain distance from the camera. That is, the histogram shows a high value only in the depth corresponding to the distance in which the person exists. Using this fact, we can find the depth at which the human is expected to exist if the local maximum is found in the generated histogram.

Of course, using the depth information of the distance that a person is expected to be present, the size of the human head at the distance can be predicted through the equation. In an embodiment of the present invention, the size of the human head according to the depth value is cubic but is transformed into a quadratic expression for the computation speed. The following equation (1) is an expression for obtaining the length (h) of the face through the depth (z).

That is, the face length of a person according to the depth value is estimated through Equation 1, and the size of the 2D template (see FIG. 5A) to be used for chamfer distance matching is adjusted using this information.

Here, the chamfer distance matching extracts only the edges of the input image, performs distance transform, performs template matching on the distance-transformed image, and detects an object of the type to be searched . That is, the chamfer distance matching has an advantage in that it can be detected robustly against deformation in the form of the object to be searched compared with a general template matching.

Perform chamfer distance matching on the input ROI image to find points that are supposed to be the human head. First, the boundary image is obtained through the Canny edge detection algorithm and the distance transformation is performed on the boundary image. Then, template matching is performed using a 2D template whose size is adjusted prior to the distance conversion image, so that objects of a type considered as a human head are detected.

Finally, candidates estimated to be human heads are confirmed to be human heads through template matching with a 3D template including depth information (see FIG. 5B). In the 3D template matching as described above, ROIs of candidates considered as the human head are extracted, and the size of the ROI is adjusted to be equal to the template size, and the matching operation is performed.

In the case where the head area is overlapped too close to the last detected head, the overlapping heads are integrated into one head. As a result, an image as shown in Fig. 6 can be output.

FIG. 7 is a flowchart illustrating a joint extracting step in the biometric method according to an embodiment of the present invention. FIG. 8 is an image showing a geodesic distance in the biometric method according to an embodiment of the present invention, 10 is a view showing a relationship between a depth of a subject and a shoulder width of a living body in a biometric method according to an embodiment of the present invention. FIG. 11 is a graph illustrating normalization of coordinates of a random point in the biometric method according to an embodiment of the present invention. FIG. 12 is a graph showing the normalization of coordinates of a random point in the biometric method according to an embodiment of the present invention. This is an image showing the classification result of the bioartile joint.

Next, with reference to FIGS. 7 to 12, the steps of extracting joints in the biometric method according to an embodiment of the present invention will be described.

The skeleton extraction algorithm extracts joints from the input ROI from the biometric detection and determines the position of the joint feature points (for example, head, shoulders, elbows, etc.) of a person existing in each ROI as a final result Return.

Thereafter, information on the human ROI is obtained from the biometric detection, and the joint information of the person existing in the ROI is estimated. Here, the information on the human ROI is composed of the ROI image, the center point of the person existing in the ROI, and the origin coordinates (the leftmost uppermost point of the ROI) of the ROI in the original image.

Then, a geodesic distance image based on the center point of the person is generated for the input ROI image. Here, the geodesic distance means a shortest distance to a point in a space formed by a node. Therefore, if the geodetic distance is calculated based on the center point of the body using the human ROI image having the depth information, the highest value is obtained at the fingertip or the toe. Here, the geodesic distance is calculated using Dijkstra's algorithm.

Thereafter, a geodesic iso level image is generated by dividing the generated geodetic distance image into a minimum value and a maximum value in 60 steps. Here, the reason why the geodetic ISO level image is generated is to effectively extract the feature points in the head, arm, leg, and the like from the vicinity of the center of the body of the person with less change of motion.

Of course, the step 60 may be arbitrarily set to the number of steps, and the present invention does not limit the number of ISO levels.

Here, referring to FIG. 8, in the geo-level ISO level image, the farther the geodesic distance is, the lighter the color.

Then randomly generate 200 random points in the next geodetic ISO image. At this time, 200 random points were generated only in non-zero values in the geodetic ISO image. The generated 200 random points will be used as input data for prediction in SVM (Support Vector Machine).

Here, the random points may be set to any number, and the number of random points is not limited in the present invention.

Here, the coordinates of the 200 random points generated using the geodetic ISO image are all set to the coordinate system with the origin at the leftmost uppermost point of the ROI. However, when SVM is trained and predicted with these coordinates, the consistency of the body part coordinates is degraded whenever the size of ROI is changed according to the human posture. Therefore, when the center of the human body is transformed into a coordinate system with the origin, the coordinates of the points other than the arms and legs having a lot of motion show a greater consistency, so that a better result can be obtained when the coordinate system is not changed. For this reason, coordinate transformation of 200 random points is performed by subtracting the coordinates of the center point from the coordinates of 200 random points so that the center point of the human body obtained from the biometric detection is the origin as shown in FIG.

In addition, since the size of ROI of a person changes according to the distance between a person and a camera in a video image, the coordinates of the parts of the body may be different depending on the distance between the person and the camera, even though the same operation is performed. In order to solve this problem, it is necessary to perform learning and prediction of SVM after normalization of coordinates of 200 random points by setting a reference size.

In order to perform the normalization of the coordinates, it is necessary to know the relationship between the change of the person's body length according to the depth. So, we use the relationship between distance and shoulder width. As a result, as shown in FIG. 10, experimental results on the relationship between the shoulder width and the distance can be obtained. In other words, by changing the relative distance to the camera, we obtain the following data by calculating the distance between two shoulder points obtained through OpenNI. Also, calculate the equation for the trend line of Excel and obtain the relation between distance and shoulder width. Thereby, Equation (2) defined for the distance (y) and the shoulder width ( x ) is obtained.

In Equation (2), the simplification of the equation and the position of the trend line are shifted upward in the y- axis from the data distribution, and in order to correct it, the line format is changed from 6,116 to 6,000 to decrease by 100 in the x- axis.

Based on Equation (2), an average distance value of 200 random points generated in the ROI is obtained, and an approximate shoulder width of a person is estimated. The coordinates of the random points were normalized so that the standard shoulder width was 12 pixels. In practice, the shoulder width is changed from 10 to 40 and the best result is obtained from the test result. Thus, the following equation (3) defines a scale value to be scaled for the ( x , y ) coordinates of 200 random points on the average of distance (z_avrg).

Here, since the scale can not be obtained if the average of the distance (z_avrg) is 6,000 or more, or negative, and the shape of the human being is 6000 or more, the maximum value of the distance average (z_avrg) is limited to 5,800 .

Thereafter, scaling is performed on the depth (z) value to obtain a normalized distance z_norm as shown in Equation (4).

Thus, through the above equations, as shown in FIG. 11, scaling is performed so that the distribution of 200 random points is normalized to be equal to the distribution of the ROI image with 12 pixels of shoulder width.

Then, when classification is performed by inputting 200 random points generated in the human ROI to the SVM classifier, 200 points are output as a result of the bio-articulating part to which each 200 points belong. Based on this, the central points of the points corresponding to each bio-articulating part can be obtained, and the characteristic points of the bio-articulating part can be obtained. However, in the case of the wrist and ankle, it is preferable to estimate the position by other methods because the number of points existing in the training data is smaller than that of the other bio-articulated parts and the classification result is not good.

In other words, first, the joint part corresponding to the head, neck, torso, shoulder, and pelvis finds feature points using the center moments of the respective parts of the classified points as shown in FIG.

Specifically, in the classification image as shown in FIG. 12, only the points of color of the bio-articulating part to be estimated are left. If the center moments of the points of the part are found, it becomes the joint point of the part.

Here, since the head, neck, torso, shoulder, and pelvis have less motion in the ROI image, it is possible to obtain a good result by estimating the joint point through the center moment, but the wrist or ankle has relatively many movements, The number of points is relatively small, so that the classification result is not good, and the joint point jumps to another place. To solve this problem, the wrist is learned with a label like an elbow, the ankle is learned with a label like a knee, and the information of the elbow and knee points and the information of the geodesic distance are combined in the prediction result. I found the joint point of the ankle. Specifically, in SVM learning, instead of the total of 15 joint parts of the body, head, neck, torso, left and right shoulders, left and right elbows, left and right wrists, left and right buttocks, left and right knees, left and right ankles, , Left and right buttocks, left and right legs, and a total of 11 pieces of bio-articulation to reduce the number of labels to perform learning and forecasting.

Then, using the information of the geodesic distance in the predicted results of the arms and legs, the joint points were separated from the arms to the elbows and the wrists, and the joint points were separated from the legs at the knees and ankles. The advantage of this method is that the SVM prediction operation time is reduced because the SVM reduces the number of classes to be learned and predicted.

FIG. 13 is a flowchart showing a step of classifying an arm (or a leg) into an elbow (or a knee) and a wrist (or ankle) joint in the biometric method according to an embodiment of the present invention, FIG. 15 is a view showing a result of extracting joints of a living body in a biometric method according to an embodiment of the present invention. FIG. 15 is a view illustrating a process of sorting an elbow and a wrist according to an embodiment of the present invention. Image.

Finally, with reference to FIGS. 13 to 15, a process of extracting a living body joint in the biometric method according to an embodiment of the present invention will be described. FIG.

Referring to FIG. 13, an algorithm for extracting the joint points of the elbow and the wrist from the point of the arm and extracting the joint points of the knee and ankle from the point of the leg can be confirmed.

Specifically, as described above, in the case of an arm, there are cases where points in the head most frequently appear in the arms because the movement is the most among the joint parts of the body. Therefore, when calculating the joint points of the wrist and the elbow including such points, a large error occurs. That is, to avoid such a situation, it is desirable to remove points belonging to a certain area around the head among the points classified as the arms.

Thereafter, when a rectangle that wraps the points classified with the arms in the minimum width is obtained, the shape is the same as the image shown in FIG.

Then, it can be confirmed that the region adjacent to one side of the shorter side of the shorter side of the rectangle in the generated rectangle corresponds to the elbow, and the region adjacent to the other side of the opposite side corresponds to the elbow. Here, if the geodesic distance information is used, it can be known which of the short short sides of the rectangle corresponds to the wrist. That is, the point having the greatest value of the geodesic distance among the nearby points of the short two short sides of the rectangle becomes the joint point of the wrist. And the joint point of the elbow is selected as one of the points existing on the line connecting the center of the rectangle and the center of the other short side opposite to the rectangle. Here, as a result of an experiment, it is preferable that the elbow be a point 3/4 from the center of the rectangle to the center of the side. Of course, this can be applied to find joint points of the knees and ankles in the legs.

As described above, if all of the above steps are performed, an image showing the result of finally extracting joints of a living body as shown in Fig. 15 can be obtained.

The present invention is not limited to the above-described embodiment, but may be embodied in many other forms without departing from the spirit or scope of the invention as defined in the appended claims. It will be understood by those of ordinary skill in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention.

x; Shoulder width y; Street

Claims (10)

(A) generating a depth image of a subject using a 3D sensor and generating a histogram representing a distribution of depth values;
(B) separating the background through the depth image;
A step (C) of estimating a head in the depth image from which background is separated and designating a living body as a region of interest;
(D) extracting a joint of a living body in the region of interest;
(E) generating a geodesic distance image based on a center point of the ROI;
(F) generating a plurality of random points on the geodesic distance image; And
(G) inputting the plurality of random points to a SVM (Support Vector Machine) and classifying joint parts of a living body; Lt; / RTI >
In the step (C)
The method of estimating the head
Calculating a head diameter of the living body;
Adjusting a size of the two-dimensional hair template according to the head diameter of the living body;
Performing chamfer distance matching;
Extracting a head candidate region of the living body and matching it with a three-dimensional head template; And
Returning the head position and diameter of the living body; And a biometric identification method. The method according to claim 1,
In the step (A)
And a preprocessing step of converting the depth image into a form required by an application program is performed. delete The method according to claim 1,
Wherein performing the chamfer distance matching comprises:
Extracting a boundary line from the background image;
Generating a distance transform image; And
Matching the two-dimensional head template; And a biometric identification method. The method according to claim 1,
In the step (F)
Wherein the coordinates of the plurality of random points are the origin of the center point. 6. The method of claim 5,
In the step (F)
Wherein the coordinates of the plurality of random points are normalized by applying the predetermined distance to the 3D sensor and the object. The method according to claim 1,
In the step (F)
Wherein the SVM classifies the joint part into a head, a neck, a torso, a left shoulder, a right shoulder, a left arm, a right, a left hip, a right hip, a left leg and a right leg. 8. The method of claim 7,
After the step (G)
Further comprising the step of classifying the arm of the joint part into an elbow and a wrist joint, and classifying the leg into a knee and an ankle joint (H). 9. The method of claim 8,
The step (H)
Creating a rectangle with two short sides and two long sides, wrapping the random points included in the joint part classified with the arms in a minimum width;
Classifying a point having the largest geodesic distance value of the geodesic distance image as a wrist joint among the random points adjacent to one side of the quadrangle; And
And dividing the point, which is 3/4 from the center of the rectangle to the center of the other side of the rectangle, into an elbow joint. 9. The method of claim 8,
The step (H)
Creating a rectangle comprising two short sides and two long sides, wrapping the random points included in the joint part classified by the legs in a minimum width;
Classifying a point having the largest geodesic distance value of the geodesic distance image as an ankle joint among the random points adjacent to one side of the quadrangle; And
And classifying a point, which is 3/4 from the center of the rectangle to the center of the other side of the rectangle, into a knee joint.

Images