TW202044112A - Face recognition system for three-dimensional living body recognition, three-dimensional living body recognition method and storage medium receiving a two-dimensional image obtained by three-dimensional shooting of an object and a range image corresponding to the two-dimensional image - Google Patents

Abstract

無。

Description

Face recognition system for three-dimensional living body recognition, three-dimensional living body recognition method and storage medium

The present invention relates to the technology of a face recognition system, and more particularly relates to a face recognition system for three-dimensional living body recognition, a three-dimensional living body recognition method and a storage medium capable of realizing the method.

The conventional face recognition system (such as the access control system and the face card system) is to take a two-dimensional image of the recognizer, and compare the two-dimensional image with the existing face image in the database to determine the pending recognition Whether the person can be identified. However, interested parties can use fake faces in flat photos to deceive this system. In order to increase the security of the face recognition system, there are the following two conventional methods to judge the authenticity of the face.

One way is to instruct the recognized person to perform different actions so as to use multiple two-dimensional images to recognize the authenticity of the human face. In the face recognition system using this method, the recognized person will be required to perform designated actions such as turning the head left and right, turning the head up and down, or other actions to determine whether the recognized person is alive. However, because the process of instructing the recognized person and waiting for the recognized person to complete the specified action takes time, this system also takes a long time to complete the recognition, and it is more inconvenient for the recognized person to use.

The other method uses two-dimensional images in combination with thermal images or infrared images. In the face recognition system using this method, there is no need to require the person to be recognized to perform additional actions, and the authenticity of the human face can be judged by the two-dimensional image combined with the thermal image or the infrared image according to the temperature characteristics of the human face. However, due to the influence of sunlight or other environmental factors outdoors, the fake human face in a flat photo can also have a temperature equivalent to body temperature, which may deceive this system.

Therefore, the face recognition technology still needs to be improved. In terms of face recognition, the recognition process is simplified and the risk of the system being deceived by fake faces in flat photos is also reduced.

One object of the present invention is to provide a technology for a face recognition system, which is used to determine whether the recognized object is a living body. This technology uses the depth information of the three-dimensionally photographed objects of the recognized person to perform three-dimensional live body recognition, so it can simplify the recognition process and reduce the risk of the system being deceived by fake faces in flat photos.

In order to achieve at least the above objectives, the present invention provides a three-dimensional living body recognition face recognition system, which includes a processing unit and a storage unit. The storage unit is electrically coupled to the processing unit, the storage unit stores the program code, and the processing unit executes multiple operations after accessing the program code. These operations include the following. The processing unit receives a two-dimensional image obtained by three-dimensional shooting of the object and a depth image corresponding to the two-dimensional image, and thereby determines whether there is a human face image in the two-dimensional image. The processing unit performs face detection and face feature point detection on the two-dimensional image to find the face image and the plurality of feature points in the face image respectively. The processing unit uses at least part of the coordinate positions of these feature points to find the discriminative area in the face image. The processing unit determines the discrimination feature value corresponding to the discrimination area based on the part of the depth image corresponding to the discrimination area and the three-dimensional reference surface corresponding to the discrimination area. The processing unit obtains a judgment result of whether the face image is a living human face at least according to the distinguishing feature value corresponding to the distinguishing area.

In an embodiment of the present invention, the processing unit determines the discriminant feature value corresponding to the discriminant area based on the distance from the three-dimensional discriminant surface corresponding to the depth information of the part of the depth image corresponding to the discriminant area to the three-dimensional reference surface.

In an embodiment of the present invention, the processing unit sets the three-dimensional reference surface as a three-dimensional plane, and the processing unit uses the depth information of the portion of the depth image corresponding to the discrimination area to determine the three-dimensional reference surface.

In an embodiment of the present invention, the processing unit obtains the judgment result of whether the face image is a living human face based on the discriminant feature value and the threshold value corresponding to the discriminative area, wherein if the discriminant feature value corresponding to the discriminant area is greater than or equal to the threshold Value, the processing unit obtains the judgment result that the face image is a living face; if the discriminant feature value corresponding to the discriminant area is less than the threshold value, the processing unit obtains the judgment result that the face image is not a living face.

In an embodiment of the present invention, the discrimination area is the first discrimination area, the discrimination feature value is the first discrimination feature value corresponding to the first discrimination area, and the processing unit uses the coordinate position of at least another part of these feature points to find the person The second discrimination area in the face image; the processing unit determines the second discrimination feature value corresponding to the discrimination area based on the face image and depth image corresponding to the second discrimination area and the three-dimensional reference surface corresponding to the second discrimination area; the processing unit at least according to The first discriminant feature value corresponding to the first discriminant region and the second discriminant feature value corresponding to the second discriminant region obtain a judgment result of whether the face image is a human face of a living body.

In an embodiment of the present invention, the face recognition system may further include a three-dimensional photographing unit, and the three-dimensional photographing unit is communicatively connected to the processing unit for outputting a two-dimensional image and a depth image.

In an embodiment of the present invention, the face recognition system may further include a communication unit, which is electrically coupled to the processing unit for communication with the remote 3D camera unit to obtain 2D images and depth images.

In order to achieve at least the above objectives, the present invention provides a three-dimensional living body recognition method, which can be used in a face recognition system. The method includes the following steps. Receive a two-dimensional image obtained by three-dimensional shooting of the object and a depth image corresponding to the two-dimensional image. Perform face detection and face feature point detection on the two-dimensional image to find out the face image and the multiple feature points in the face image respectively. Use at least part of the coordinate positions of these feature points to find the discriminative area in the face image. The discrimination feature value corresponding to the discrimination area is determined based on the part of the depth image corresponding to the discrimination area and the three-dimensional reference surface corresponding to the discrimination area. At least according to the discriminant feature value corresponding to the discriminant area, the judgment result of whether the face image is a living face is obtained.

In an embodiment of the present invention, the step of determining the discriminant feature value corresponding to the discriminant area includes: determining the discriminant area based on the distance from the 3D discriminant surface corresponding to the depth information of the depth image portion of the discriminant area to the 3D reference surface The corresponding discriminative feature value.

In an embodiment of the present invention, the three-dimensional reference surface is the three-dimensional plane closest to the three-dimensional discriminating surface.

In one embodiment of the present invention, the three-dimensional reference surface is a three-dimensional plane, and the three-dimensional reference surface is determined by using the depth information of the depth image corresponding to the discrimination area.

In an embodiment of the present invention, the step of obtaining the judgment result of whether the face image is a human face of a living body includes the following steps. If the discriminant feature value corresponding to the discriminant region is greater than or equal to the threshold value, then the result of judging that the face image is a living face is obtained. If the discriminant feature value corresponding to the discriminant area is less than the threshold value, the result of judging that the human face image is not a living human face is obtained.

In one embodiment of the present invention, the discrimination area is the first discrimination area, and the discrimination feature value is the first discrimination feature value corresponding to the first discrimination area. The three-dimensional living body recognition method further includes: using at least another part of these feature points The coordinate position finds the second discriminant area in the face image; the second discriminant feature value corresponding to the discriminant area is determined based on the face image and depth image corresponding to the second discriminant area and the three-dimensional reference surface corresponding to the second discriminant area; The step of obtaining the judgment result of whether the face image is a living face includes at least obtaining the judgment of whether the face image is a living body based on at least a first discriminant feature value corresponding to the first discriminant area and a second discriminant feature value corresponding to the second discriminant area The judgment result of the face.

In order to achieve at least the above-mentioned object, the present invention further provides a storage medium, which records a program code used for a computing device to execute at least one or a combination of the above-mentioned three-dimensional living body recognition method embodiments.

In this way, the above embodiments of the technology provided for the face recognition system can be used to determine whether the recognized object is a living body. This technology uses the depth information of the three-dimensionally photographed objects of the recognized person to perform three-dimensional live body recognition, so it can simplify the recognition process and reduce the risk of the system being deceived by fake faces in flat photos. This technology can be applied to face recognition systems with security requirements (such as access control systems, face punch card systems) to prevent the system from being deceived by intentional persons using fake faces and increase the security of the system.

In order to fully understand the purpose, features and effects of the present invention, the following specific embodiments are used in conjunction with the accompanying drawings to give a detailed description of the present invention. The description is as follows:

Please refer to FIG. 1, which illustrates an embodiment of a face recognition system for 3D living body recognition. The face recognition system of FIG. 1 can implement the 3D living body recognition method according to FIG. 4 or the embodiment based on FIG. 4 (which will be described later) Detailed description, temporarily skipped here), and can be used to further implement various methods based on face recognition. The face recognition system 1 for three-dimensional living body recognition as shown in FIG. 1 (hereinafter referred to as the face recognition system 1) includes a storage unit 11 and a processing unit 10. The processing unit 10 is electrically coupled to the storage unit 11, and is used to receive a two-dimensional image IM obtained by three-dimensional shooting of an object and a depth image DM corresponding to the two-dimensional image IM, and thereby determine whether there is a living body in the two-dimensional image IM Face image.

The processing unit 10 may be implemented by one or more circuits, such as a processor, a digital signal processor, or a programmable integrated circuit such as a microcontroller, a field programmable gate array (FPGA) ) Or one or more circuits in circuits such as application specific integrated circuit (ASIC), or a dedicated circuit or module. The storage unit 11 includes, for example, a memory device such as a memory or a storage device, and can be used to store data to be input or output by the processing unit 10, or can be used in a database for facial recognition and facial feature point recognition.

In an embodiment based on FIG. 1, the face recognition system 2 shown in FIG. 2 may further include a three-dimensional photographing unit 20, which is communicatively connected to the processing unit 10 for outputting two-dimensional images IM and depth images DM. For example, the three-dimensional photographing unit 20 can be communicatively connected to the processing unit 10 through an electrical connection, a communication interface (such as a USB, etc.), a data connection cable, or other connection methods. The face recognition system 2 can be further configured as a computing device or terminal device according to requirements, such as a camera or surveillance camera with in-depth detection, or as a smart device, such as a smart phone or a tablet, or can be configured as Computer systems such as notebook computers. The three-dimensional photographing unit 20 includes, for example, a two-dimensional image sensor 21 and a depth sensor 22. The two-dimensional image sensor 21 is, for example, a CMOS or other image sensors. The depth sensor 22 is, for example, a depth sensor using infrared rays or lasers, such as a time of flight (TOF) sensor. However, the implementation of the present invention is not limited by the above examples.

In an embodiment based on FIG. 1, the face recognition system 3 shown in FIG. 3 may further include a communication unit 12, which is electrically coupled to the processing unit 10, and is used for wired or wireless communication connection To the remote 3D camera unit 20 to obtain the 2D image IM and the depth image DM, the face recognition system 3 determines whether there is a human face image in the 2D image IM. In this embodiment, the face recognition system 3 can be configured as one or more computing devices, such as mobile devices, computers, servers, or cloud servers. On the other hand, the 3D photographing unit 20 may be connected to the data transmission device 30, and transmit the captured 2D image IM and depth image DM to the communication unit 12 through the data transmission device 30 in a wired or wireless manner. The data transmission device 30 may utilize a communication circuit, such as a communication circuit supporting Wi-Fi, USB or other communication protocols, or may be a computing device with a communication circuit. However, the implementation of the present invention is not limited by the above examples.

Please refer to FIG. 4, which is a schematic flowchart of an embodiment of a three-dimensional living body recognition method. The embodiment shown in FIG. 4 can be used as a face recognition system based on FIGS. 1 to 3, and various methods based on face recognition can be further implemented. As shown in FIG. 4, an embodiment of the three-dimensional living body recognition method includes the following steps. As shown in step S10, a two-dimensional image IM obtained by three-dimensional shooting of an object and a depth image DM corresponding to the two-dimensional image IM are received. As shown in step S20, face detection and face feature point detection are performed for the two-dimensional image IM to find the face image and the plurality of feature points in the face image respectively. As shown in step S30, use at least part of the coordinate positions of these feature points to find the discriminative area in the face image. As shown in step S40, the discrimination feature value corresponding to the discrimination area is determined based on the part of the depth image DM corresponding to the discrimination area and the three-dimensional reference surface corresponding to the discrimination area. As shown in step S50, the judgment result of whether the face image is a living human face is obtained at least according to the discriminant feature value corresponding to the discriminant area.

The following examples illustrate various implementation methods for each step.

Regarding step S10, for example, as shown in FIG. 1, FIG. 2 or FIG. 3, the processing unit 10 may be used to receive the two-dimensional image IM and the depth image DM to implement step S10. For example, a two-dimensional image IM includes an image with A´B pixels, such as an RGB image, a grayscale image, or other forms of images. A and B are positive integers and can be set according to the resolution requirements of the face recognition system The depth image DM includes depth information corresponding to at least some or all of the pixels in the two-dimensional image IM; the two-dimensional image IM and the depth image DM are synchronized and aligned. For example, the depth image DM corresponding to the two-dimensional image IM includes the depth information of the entity in the environment corresponding to at least some or all of the pixels in the two-dimensional image IM in terms of a reference origin (or reference plane), and this reference origin (Or reference plane) can usually be taken as the position of the aforementioned three-dimensional photographing unit 20 (or the reference plane for photographing). In addition, the reference origin (or reference plane) can also be set at other positions. The embodiments are not limited to the above examples.

According to step S10, the face recognition system 1, 2 or 3 can be configured to receive individual two-dimensional image IM and depth image DM corresponding to the two-dimensional image IM, or can be configured to receive a format based on three-dimensional coordinates (such as (x, y, z)) to represent the two-dimensional image IM and the depth image DM, where z is the depth value corresponding to the coordinate (x, y). Therefore, the implementation of the present invention is not limited by examples such as the format of the two-dimensional image or depth image received by the face recognition system.

In step S20, face detection is performed on the two-dimensional image IM to find the face image. Face detection finds the position and size of the face in the two-dimensional image IM to detect facial features and ignores other things such as buildings, trees, and bodies. Please refer to FIG. 5, which is a schematic diagram of an embodiment of step S20 in FIG. 4. As shown in FIG. 5, through an embodiment of step S20, the face image FI is found in the two-dimensional image IM. Step S20 performs face feature point detection on the two-dimensional image IM to find a plurality of feature points in the face image. For example, the detection of facial feature points can be performed on the basis of face detection. As shown in Figure 5, the detection of facial feature points can detect the feature points on the face image FI, such as the corners of the eyes, nose, and mouth. After positioning, the characteristic points C1~C4, C5~C7, C8~C9 are obtained. In addition, facial feature point detection can also be used to detect any feature points to be detected on the face image FI, such as the corners of the eyes, nose, mouth, chin, ears, forehead, and facial cubits. Feature points, or feature points in other parts of the face image FI. The implementation of the present invention is not limited by the above-mentioned feature points or the number of feature points.

In the embodiment of step S20, any one or more face detection technologies can be used to implement, for example, but not limited to, feature-based recognition algorithms based on facial feature points, and based on the entire face. Image recognition algorithms (appearance-based recognition algorithms), template-based recognition algorithms (template-based recognition algorithms), recognition algorithms using neural network (recognition algorithms using neural network) or support vector machines for recognition Recognition algorithms using support vector machine (SVM), or other suitable algorithms.

For example, in step S30, at least part of these feature points, such as the coordinate positions of at least one or more feature points, are used to find the discriminative region in the face image. Please refer to FIG. 6, which is a schematic diagram of an embodiment of step S30 in FIG. 4. In FIG. 6, in order to facilitate the drawing and description of the relationship between the discriminant area obtained in step S30 and the corresponding depth information, the face image is indicated by contours or lines as indicated by the symbol FD (such as FI in FIG. 5 above) The part in the corresponding depth image DM. If the depth information in the depth image DM uses a certain numerical range (such as 0~255 or other numerical range) to indicate the distance of the entity corresponding to the pixel from the far to short distance of the secondary reference origin (or reference plane), FD or DM can The gray scale diagram is used for illustration, but the implementation of the present invention is not limited by this. The following examples illustrate various possible ways to find the discriminative area in the face image.

As shown in Fig. 6, for example, the block near the nose is taken as the discrimination area to implement step S30. At least the center of the nose (such as the feature point C5) or the position of the feature point detected from the two-dimensional image IM (X, Y), you can directly define the coordinates (X+W, Y+H), (X+W, YH), (XW, Y+H), (XW, Y+H) near (X, Y) Discrimination area R3, where W and H are the width and height of the discrimination area respectively.

In addition, the implementation of the present invention is not limited by the implementation form of step S30; for example, in the method step, if the depth information corresponding to the region based on at least one feature point is used to determine whether the face image is related to a living face During processing, since the discrimination area has been indirectly defined, it can also be considered that step S30 has been implemented.

In addition, in some embodiments, step S30 may also use two or more feature points to find the discrimination area. For example, take the characteristic points C5, C6, and C7 of the nose in Figure 5, take the midpoints of C5, C6 and the midpoints of C6, C7 to estimate the width of the nose, and use the feature points C2, C3 midpoints and feature points of the corner of the eye C5 is used to estimate the height of the nose, so that the discrimination area R4 shown in Figure 6 can be obtained. Others, such as the discrimination area R1 located on the forehead, the discrimination area R2, R3 across the nose, or the discrimination area R5 below the mouth, can also be found by analogy. For another example, two or more feature points can be used to set a discrimination area. For example, use C8 and C9 of the characteristic points in Figure 5 to set a discrimination area. For example, use C5, C6, and C7 of the characteristic points in Figure 5 to set a discrimination area. For example, use C6, C7, C8, and C9 of the characteristic points in Figure 5 to set a discrimination area. You can also use any detected feature value on the face image FI to set one or more discrimination areas, such as at least one or more features in the corners of the eyes, nose, corners of the mouth, chin, ears, forehead, facial cubits or other parts Click to set one or more discrimination areas. However, the implementation of the present invention is not limited by the above examples of discrimination regions or the number of discrimination regions.

After the discrimination area is found in step S30, the subsequent recognition of whether a human face is alive can be further performed on the depth information corresponding to the discrimination area.

In an embodiment of the present invention, the step S40 of determining the discriminant feature value corresponding to the discriminant region includes: determining based on the distance from the 3D discriminant surface to the 3D reference surface corresponding to the depth information of the portion of the depth image DM corresponding to the discriminant region The discriminant feature value corresponding to the discriminant area. Please refer to FIG. 7A, which is a schematic diagram of an embodiment of step S40 in FIG. 4. In FIG. 7A, take the discrimination region R3 of the nose part of FIG. 6 as an example, and for the convenience of explanation, the depth information of the part of the depth image DM corresponding to the discrimination region R3 is based on the corresponding three-dimensional discrimination surface S as a cross-section. Said. In the embodiment shown in FIG. 7A, the three-dimensional reference plane PA is the three-dimensional plane closest to the three-dimensional discriminating surface. The three-dimensional discriminating surface S is a curved surface in this example and the highest point intersects the three-dimensional reference plane PA. In detail, the three-dimensional discriminant surface S is not an ideal curved surface. In fact, the three-dimensional discriminant surface S refers to the depth information (as represented by z) corresponding to the coordinate position (as represented by (x, y)) of the discrimination region R3 Set, for ease of description, multiple discrete coordinates (x, y, z) can be regarded as points on the three-dimensional discriminant surface S, so the implementation of the present invention is not subject to the above description of the three-dimensional discriminant surface S or other examples limit.

In the embodiment of step S40, the distance from the three-dimensional discrimination surface S corresponding to the discrimination area R3 to the three-dimensional reference surface PA can be used as the discrimination feature value corresponding to the discrimination area R3, for example, the points of the three-dimensional discrimination surface S to this three-dimensional reference The distance of surface PA is averaged or processed by other statistics. In order to be able to detect the situation that an interested person wants to deceive by using the photo of the face, when the photo of the face is contained in the two-dimensional image, because the photo of the face is also flat, the photo of the face is compared with the three-dimensional reference plane PA. The distance will be small, so the corresponding discriminative eigenvalue is small. Conversely, if the two-dimensional image is a real face, the curvature of the face will cause a higher distance difference, so the corresponding discriminant feature value is larger. After statistical analysis or calculation of empirical values, the relationship between the size of the distinguishing feature value and whether the face is real can be obtained. Therefore, the subsequent step S50 can be used to obtain whether the face image is at least based on the distinguishing feature value corresponding to the distinguished area. It is the judgment result of a living human face. In addition, based on the method in step S30 of FIG. 4, the number of regions can be determined by two or more. Correspondingly, through steps S40 and S50, the corresponding regions can also be determined by multiple methods to determine whether the facial image is The judgment result of the face of the living body.

Please refer to FIG. 7B, which is a schematic diagram of another embodiment of step S40 in FIG. 4. As shown in FIG. 7B, the three-dimensional reference plane PB can be set as a three-dimensional plane that is not the closest to the three-dimensional discrimination surface S. As shown in the example of FIG. 6, the three-dimensional reference plane PB is in front of the discrimination area R3 such as the nose.

Please refer to FIG. 7C, which is a schematic diagram of another embodiment of step S40 in FIG. 4. As shown in FIG. 7C, the three-dimensional reference plane PC can be set as a three-dimensional plane that is not the closest to the three-dimensional discriminating surface S. As shown in the example of Fig. 6, the three-dimensional reference plane PC is in the discriminating region R3, such as behind the nose.

In an embodiment of the present invention, the three-dimensional reference surface is a three-dimensional plane, and the three-dimensional reference surface is determined by using the depth information of the portion of the depth image DM corresponding to the discrimination area. Please refer to FIG. 8, which is a schematic flowchart of another embodiment of step S40 in FIG. As shown in step S400, the depth information of the part of the depth image DM corresponding to the discrimination area is used to obtain a plurality of modeling parameters of the model (or equation) of the three-dimensional reference surface. As shown in step S410, based on the modeling parameters and the depth information of the corresponding part of the depth image DM in the discrimination area, the three-dimensional discrimination surface corresponding to the depth information of the part of the depth image DM corresponding to the discrimination area is obtained. Multiple distance values of the 3D reference surface. As shown in step S420, the discriminant feature value corresponding to the discriminant area is obtained based on the multiple distance values. For example, a number of distance values are averaged or counted by other statistics to obtain a discriminative feature value.

或簡化為

(公式1)。For example, taking the three-dimensional reference surface of FIG. 7A as a three-dimensional plane as an example, the following formula can be used to express:

Or simplified to

(Formula 1).

舉例而言，為了簡化上述矩陣及找出合適的三維平面作為三維參考面，可以將參考原點移至此判別區域的中心位置(或可視為幾何重心)。藉此，最後得出的三維平面會通過判別區域的所有點之座標所求到的中心點。為此，以此判別區域的中心位置為新的參考原點，上述式子中的座標要經過座標轉換。據此，在經過針對此判別區域的中心位置為新參考原點之座標轉換以後，此判別區域中的所有三維的座標將符合以下等式：

， 故上述矩陣經過座標轉換以後可以簡化為：

最後，可以得出：

(公式2) 。Assuming that there are N points in a discriminative area, the following matrix can be listed by formula 1:

For example, in order to simplify the above matrix and find a suitable three-dimensional plane as the three-dimensional reference plane, the reference origin can be moved to the center position of the discrimination area (or can be regarded as the geometric center of gravity). In this way, the final three-dimensional plane will pass through the center point obtained by the coordinates of all points in the discrimination area. For this reason, the center position of the discrimination area is used as the new reference origin, and the coordinates in the above formula must undergo coordinate conversion. According to this, after the coordinate conversion for the center position of the discrimination area as the new reference origin, all three-dimensional coordinates in the discrimination area will conform to the following equation:

, So the above matrix can be simplified to:

Finally, it can be concluded:

(Formula 2).

(公式3)。Substituting the highest point in the discrimination area into the plane equation, d can be obtained. After obtaining a, b, and d through formula 2, substituting a, b, and d into formula 1, it is the plane equation closest to the discrimination area. The above a, b, and d are examples of the modeling parameters described in step S400. Thereby, the three-dimensional reference plane PA as shown in FIG. 7A can be obtained. If the three-dimensional reference plane PA is translated in a certain direction, the three-dimensional reference plane PB can be obtained, or the three-dimensional reference plane PA can be translated in another direction, and the three-dimensional reference plane PC can be obtained. Regarding step S410, the following formula 3 can be used to calculate the distance from a point (x ₀ , y ₀ , z ₀ ) to the plane:

(Formula 3).

Please refer to FIG. 9, which is a schematic flowchart of an embodiment of step S50 in FIG. 4. As shown in step S510, whether the distinguishing feature value corresponding to the distinguishing area is greater than or equal to the threshold value. As shown in step S520, if the discriminant feature value corresponding to the discriminant area is greater than or equal to the threshold value, it is determined that the human face image is a living human face. As shown in step S530, if the discriminant feature value corresponding to the discriminant area is less than the threshold value, it is determined that the human face image is not a living human face. After statistical analysis or calculation of empirical values, the relationship between the size of the distinguishing feature value and whether the face is real can be obtained. Therefore, the subsequent step S50 can be used to obtain whether the face image is at least based on the distinguishing feature value corresponding to the distinguished area. It is the judgment result of a living human face.

In addition, in the method based on FIG. 4, in addition, based on the method in step S30 of FIG. 4, the number of regions can be determined by two or more. Correspondingly, through steps S40 and S50, the corresponding regions can also be determined based on multiple methods. The feature value is used to determine whether the face image is a living face. In one embodiment, the discriminant area is the first discriminant area, and the discriminant feature value is the first discriminant feature value corresponding to the first discriminant area. The three-dimensional living body recognition method further includes: using the coordinate position of at least another part of these feature points to find The second discriminant area in the face image is determined; the second discriminant feature value corresponding to the discriminant area is determined based on the face image and depth image DM corresponding to the second discriminant area and the three-dimensional reference surface corresponding to the second discriminant area; The step of determining whether the human face image is a living human face includes determining whether the human face image is a living human face at least according to the first discriminant feature value corresponding to the first discriminant region and the second discriminant feature value corresponding to the second discriminant region The result of the judgment.

In some embodiments, steps S40 and S50 may use a method based on machine learning or neural network to obtain the judgment result of whether the face image is a human face of a living body. For example, before steps S40 and S50 are implemented, one or more discriminative regions and corresponding depth information in multiple real faces can be used as feature data and correspond to the ground truth category of the real face. Examples are given to a classifier for training, and one or more discriminative regions and corresponding depth information in multiple fake faces are used as feature data and correspond to the real category of fake faces. training. After a lot of training, this classifier has the ability to judge whether the face image is a living face in the case of one or more discriminative regions in the face image; when the classifier meets the required criteria When the rate is correct, it can be used to implement steps S40 and S50. However, the implementation of the present invention is not limited by the above examples.

In addition, in some embodiments, a computing device readable recording medium is provided, and the recording is used for a computing device (such as the face recognition system 1, 2 or 3 shown in Figure 1, Figure 2 or Figure above), A computing device (such as a terminal device or a server) executes the code of the three-dimensional living body recognition method or the code including the three-dimensional living body recognition method, wherein the method may include at least one or a combination of the embodiments based on the method of FIG. 4. For example, the program code is one or more programs or program modules, such as those used to implement steps S10 to S50 in FIG. 4, steps S400 and S420 in FIG. 8, or steps S500 to S530 in FIG. The code of the group works cooperatively and can be executed in any suitable order or in parallel. When the arithmetic device executes this program code, it can cause the arithmetic device to execute an embodiment of the three-dimensional living body recognition method based on FIG. 4. The above-mentioned readable recording medium is, for example, firmware, ROM, RAM, memory card, optical information storage medium, magnetic information storage medium, or any other kind of storage medium or memory, and the implementation of the present invention is not affected by this Example limitation.

In addition, any embodiment or combination of the three-dimensional living body recognition method or system based on FIG. 4 can be a part of applications such as an identity recognition system or a withdrawal system based on an identity recognition system, an access control system, a withdrawal system, or a payment system. , And help to strengthen the security of the above system to avoid being deceived by fake faces. The following is an example.

In one embodiment, in the process of face recognition by the identity recognition system, any embodiment or combination of the three-dimensional living body recognition method based on FIG. 4 can be used to determine whether the human face image is a living human face The result of the judgment. If the judgment result is yes, the identity recognition system may further recognize the identity of the face, so that the identity recognition system can determine whether the face is a registered or known face in the user database of the identity recognition system. If the face is a registered or known face in the user database of the system, the identity recognition system can proceed to the next step. For example, the system allows the user represented by the face to use a certain device , Or computer computing, network resource permissions, or specifically for this user to turn on or off a certain device or change the settings or other options of a certain device. In addition, if the result of the judgment that the face image is not a living face is obtained, the face identification system can reject the user represented by the face, or request that the face image be taken again and whether the face is a living person The judgment of the face, however, this embodiment is not limited by this. The above-mentioned embodiments can allow the identity recognition system to quickly screen out fake faces, strengthen the security of the face identity recognition system, and avoid wasting identity judgments such as reading the database in the face recognition aspect of the face recognition system The computing resources consumed.

In another embodiment, the face identification system can also perform face identification first, so that the face identification system can determine whether the face is registered or known in the user database of the face identification system human face. If the face is a registered or known face in the user database of this system, the face identification system can perform any embodiment or combination of the three-dimensional living body recognition method based on FIG. 4 to obtain the person The judgment result of whether the face image is a living human face. If the judgment result is yes, the face identification system has confirmed the authenticity of the user represented by the face, so the next step of the operation can be performed as in the foregoing example. This embodiment can also strengthen the security of the face recognition system.

In addition, the above embodiments of the identity recognition system can be applied to a withdrawal system, an access control system, a withdrawal system, a payment system or other systems based on an identity recognition system. After identification and confirmation of whether the human face is a living human face, certain operations such as withdrawal operation, door opening or closing operation, payment operation or other system operations are performed. In this way, the security of the withdrawal system, access control system, withdrawal system, payment system or other systems based on the identity recognition system can be strengthened.

In other embodiments, any one or a combination of the above-mentioned three-dimensional living body recognition method or system based on FIG. 4 may also include the step of identity recognition based on facial images, thereby becoming a method suitable for the identity recognition system. Or it can be used in applications such as withdrawal systems, access control systems, withdrawal systems, or payment systems based on identity recognition, which can help prevent the above systems from being deceived by fake faces and strengthen security. For example, the step of identification based on the face image can be implemented before or after steps S10 to S50, or between steps S10 to S50, and the implementation of the present invention is not limited by this.

The steps of identity recognition based on facial images are, for example, comparing unidentified facial images with facial images with known identities or corresponding codes in the facial database in the system or calculating similarity, and using or The output is a series of similarity scores, indicating the identity of the face to be recognized. For example, if the user with the highest similarity among the faces of the unidentified facial image and the faces of multiple users with known identities is user A, the identity of the facial image can be determined as user A. However, the implementation of the present invention is not limited by this example.

In other embodiments, any one or a combination of the above-mentioned three-dimensional living body recognition method or system based on FIG. 4 may not only include the step of identity recognition based on the face image, but may also include, after the identity recognition and face recognition After confirming whether it is a face of a living body, a certain operation step is further executed. For example, if the user represented by the face has the authority to use a certain device, or computer computing, network resources, or specifically for this user to turn on or turn off a certain device or change the settings or settings of a certain device Other options, such as withdrawal operation, door opening or closing operation, payment operation or the operation of other systems are not repeated here. However, the implementation of the present invention is not limited by this example.

In this way, the above embodiments of the technology provided for the face recognition system can be used to determine whether the recognized object is a living body. This technology uses the depth information of the three-dimensionally photographed objects of the recognized person to perform three-dimensional living body recognition. Compared with the conventional image processing method, in some embodiments, the face recognition system only needs one face to judge, so the recognition process can be simplified, and the system can be deceived by fake faces in flat photos. risk. This technology can be applied to face recognition systems with security requirements (such as access control systems, face check-in systems) to prevent the system from being deceived by intentional persons using fake faces in photos, and can increase the security of the system.

The present invention has been disclosed in a preferred embodiment above, but those skilled in the art should understand that the embodiment is only used to describe the present invention and should not be construed as limiting the scope of the present invention. It should be noted that all changes and substitutions equivalent to this embodiment should be included in the scope of the present invention. Therefore, the protection scope of the present invention should be defined by the scope of the patent application.

1, 2, 3: Face recognition system 10: Processing unit 11: storage unit 12: Communication unit 20: 3D photography unit 21: Two-dimensional image sensor 22: Depth sensor 30: Data transmission device S10~S50: steps S400~S420: steps S510~S530: steps IM: Two-dimensional image DM: Depth image FI: Face image FD: The part of the depth image corresponding to the face image C1~C9: Feature points R1~R5: Discrimination area S: Three-dimensional discriminant surface PA, PB, PC: three-dimensional reference surface

[Figure 1] is a schematic block diagram of an embodiment of a face recognition system for three-dimensional living body recognition. [Figure 2] is a schematic block diagram of another embodiment of the face recognition system for 3D living body recognition. [Figure 3] is a schematic block diagram of some embodiments of a face recognition system for three-dimensional living body recognition. [Figure 4] is a schematic flowchart of an embodiment of the three-dimensional living body recognition method. [Figure 5] is a schematic diagram of an embodiment of step S20 in Figure 4. [Figure 6] is a schematic diagram of an embodiment of step S30 in Figure 4. [Fig. 7A] is a schematic diagram of an embodiment of step S40 in Fig. 4. [Figure 7B] is a schematic diagram of another embodiment of step S40 in Figure 4. [Figure 7C] is a schematic diagram of another embodiment of step S40 in Figure 4. [Figure 8] is a schematic flowchart of an embodiment of step S40 in Figure 4. [Fig. 9] is a schematic flowchart of an embodiment of step S50 in Fig. 4.

S10~S50: steps

Claims (14)

A face recognition system for three-dimensional living body recognition, including: A processing unit; and A storage unit is electrically coupled to the processing unit, the storage unit stores the program code, and the processing unit executes multiple operations after accessing the program code. The operations include: Receiving a two-dimensional image obtained by three-dimensional shooting of an object and a depth image corresponding to the two-dimensional image and thereby determining whether there is a human face image in the two-dimensional image; Perform face detection and face feature point detection on the two-dimensional image to find a face image and a plurality of feature points in the face image respectively; Use at least part of the coordinate positions of the feature points to find a discriminative area in the face image; Determine a distinguishing feature value corresponding to the distinguishing area based on the part of the depth image corresponding to the distinguishing area and a three-dimensional reference surface corresponding to the distinguishing area; and At least according to the discriminant feature value corresponding to the discriminant area, a judgment result of whether the face image is a human face is obtained. The face recognition system according to claim 1, wherein the processing unit determines the discrimination area based on the distance from a three-dimensional discrimination surface corresponding to the depth information of the part of the depth image corresponding to the discrimination area to the three-dimensional reference surface The corresponding discriminative feature value. The face recognition system according to claim 2, wherein the processing unit sets the three-dimensional reference surface as a three-dimensional plane, and the processing unit uses the depth information of the portion of the depth image corresponding to the discrimination area to determine the three-dimensional reference surface . The face recognition system according to claim 1, wherein the processing unit obtains the judgment result of whether the face image is a living face based on the distinguishing feature value corresponding to the distinguishing area and a threshold value, wherein if the distinguishing area corresponds to If the discriminant feature value of is greater than or equal to the threshold value, the processing unit obtains the judgment result that the face image is a living human face; if the discriminant feature value corresponding to the discriminant area is less than the threshold value, the processing unit obtains The face image is not the judgment result of a living face. The face recognition system according to claim 1, wherein the discrimination area is a first discrimination area, the discrimination feature value is a first discrimination feature value corresponding to the first discrimination area, and the processing unit uses the feature points Find a second discriminant region in the face image based on the coordinate position of at least another part of the face image; the processing unit is based on the face image corresponding to the second discriminant region and the depth image and the second discriminant region corresponding to a three-dimensional The reference surface determines a second discriminant feature value corresponding to the discriminant region; the processing unit obtains the face at least according to the first discriminant feature value corresponding to the first discriminant region and the second discriminant feature value corresponding to the second discriminant region The judgment result of whether the image is a living human face. The face recognition system according to any one of claims 1 to 5, wherein the face recognition system further includes: A three-dimensional photographing unit, which is communicatively connected to the processing unit, for outputting the two-dimensional image and the depth image. The face recognition system according to any one of claims 1 to 5, wherein the face recognition system further includes: A communication unit, which is electrically coupled to the processing unit, and is used for communication with a remote 3D camera unit to obtain the 2D image and the depth image. A three-dimensional living body recognition method used in a face recognition system, the method includes: Receiving a two-dimensional image obtained by three-dimensional shooting of the object and a depth image corresponding to the two-dimensional image; Perform face detection and face feature point detection on the two-dimensional image to find a face image and a plurality of feature points in the face image respectively; Use at least part of the coordinate positions of the feature points to find a discriminative area in the face image; Determine a distinguishing feature value corresponding to the distinguishing area based on the part of the depth image corresponding to the distinguishing area and a three-dimensional reference surface corresponding to the distinguishing area; and At least according to the discriminant feature value corresponding to the discriminant area, a judgment result of whether the face image is a human face is obtained. The three-dimensional living body recognition method according to claim 8, wherein the step of determining a discriminant feature value corresponding to the discriminant region includes: The discrimination feature value corresponding to the discrimination area is determined based on the distance from a three-dimensional discrimination surface corresponding to the depth information of the part of the depth image corresponding to the discrimination area to the three-dimensional reference surface. The three-dimensional living body recognition method according to claim 9, wherein the three-dimensional reference surface is a three-dimensional plane closest to the three-dimensional discriminating surface. The three-dimensional living body recognition method according to claim 9, wherein the three-dimensional reference surface is a three-dimensional plane, and the three-dimensional reference surface is determined by using depth information of the portion of the depth image corresponding to the discrimination area. The three-dimensional living body recognition method according to claim 8, wherein the step of obtaining the judgment result of whether the face image is a living body face comprises: If the discriminant feature value corresponding to the discriminant area is greater than or equal to a threshold value, the result of the judgment that the face image is a living face is obtained; If the discriminant feature value corresponding to the discriminant area is less than the threshold value, then the result of judging that the face image is not a living face is obtained. The three-dimensional living body recognition method according to claim 8, wherein the discrimination area is a first discrimination area, the discrimination feature value is a first discrimination feature value corresponding to the first discrimination area, and the three-dimensional living body recognition method further includes: Finding a second discriminant area in the face image by using the coordinate position of at least another part of the feature points; Determine a second discriminant feature value corresponding to the discriminant area based on the face image and the depth image corresponding to the second discriminant area and a three-dimensional reference surface corresponding to the second discriminant area; The step of obtaining the judgment result of whether the face image is a living face includes obtaining the face at least according to a first discriminant feature value corresponding to the first discriminant region and a second discriminant feature value corresponding to the second discriminant region The judgment result of whether the image is a living human face. A storage medium, which records a program code used to allow an arithmetic device to execute the three-dimensional living body identification method according to any one of claims 8 to 13.

Images