TWI727337B - Electronic device and face recognition method - Google Patents

Abstract

A face recognition method is provided. The method includes capturing an image of a user, uploading a first user image to a server, receiving the first user image including a rectangular frame of a user face and user information transmitted by the server, detecting a rectangular frame of a user face in a second user image, determining whether a number of faces is the same in the first and second user image, uploading the second user image to the server to recognize a rectangular of the user face when the number of faces is the same in the first and second user image, determining difference between the rectangular frames of the user face detected by the electronic device and the server is less than or equal to a predefined value, and determining the user information in the second user image is the same as the user information in the first user image when the difference is less than or equal to the predefined value. An electronic device is also provided.

Description

Electronic device and face recognition method

The invention relates to the field of image recognition, in particular to an electronic device and a face recognition method

With the development of science and technology, the application of image recognition has become more and more extensive. For example, facial image recognition is used in access control systems, attendance systems, roll call systems and conference room systems. In the prior art, in order to improve the accuracy of face recognition, some users choose to pay for a cloud-based recognition system, such as Microsoft Cloud Cognitive Services. Under normal circumstances, the cloud recognition system needs to charge a fee every time it sends data back to the user terminal. Therefore, in real-time face recognition scenarios such as video conferences, the process of using the cloud recognition system for face recognition is more complicated, and it is easy to generate high amounts. The cost, thus causing inconvenience to users.

In view of this, it is necessary to provide an electronic device and a face recognition method to solve the above technical problems.

An electronic device includes at least a processor and a camera unit, the electronic device is in communication with a server, and the processor includes: a camera module for controlling the camera unit to photograph the front of the electronic device every preset time User's image; The upload module is used to upload the first user image captured by the camera unit to the server; the receiving module is used to receive the rectangular frame containing the face and user information recognized and returned by the server The first user image; the detection module is used to detect the face rectangle in the second user image of the user in front of the electronic device captured by the camera unit; the judgment module is used to judge the Whether the number of human faces in the second user image is the same as the number of human faces in the first user image; the upload module is also when the judgment module determines the human faces in the second user image When the number is the same as the number of faces in the first user image, upload the second user image to the server to identify the face rectangle in the second user image; determining module , For determining whether the difference between the rectangular frame of the face detected by the detection module and the rectangular frame of the face recognized by the server in the second user image is less than or equal to a preset value; and The determining module may also determine that the difference between the face rectangular frame detected by the detection module and the face rectangular frame recognized by the server in the second user image is less than or equal to the predetermined When setting the value, it is determined that the user information in the second user image is the same as the user information in the first user image.

A face recognition method applied to an electronic device, the electronic device includes at least a camera unit, the electronic device is also communicatively connected with a server, the method includes the following steps: controlling the camera unit to take pictures every preset time An image of the user in front of the electronic device; uploading the first user image captured by the camera unit to the server; Receive the first user image including the face rectangle and user information recognized and returned by the server; detect the face rectangle in the second user image of the user in front of the electronic device captured by the camera unit Box; determine whether the number of faces in the second user image is the same as the number of faces in the first user image; when it is determined that the number of faces in the second user image is the same as the first When the number of faces in the user image is the same, upload the second user image to the server to identify the face rectangle in the second user image; determine that the second user image is Whether the difference between the human face rectangular frame detected by the electronic device and the human face rectangular frame recognized by the server is less than or equal to a preset value; and when it is determined that the electronic device is in the second user image When the difference between the detected human face rectangular frame and the human face rectangular frame recognized by the server is less than or equal to the preset value, the user information in the second user image is determined to be the same as the first user The user information of the images is the same.

The above electronic device and face recognition method do not need to recognize the image again, which simplifies the face recognition when the number of faces in the images taken before and after is the same and the difference between the rectangular frames of different faces is small. The process can also reduce the payment of cloud resources, thus providing users with great convenience and effectively improving user experience.

1: Electronic device

10: processor

101: camera module

102: Upload module

103: receiving module

104: Detection Module

105: Judgment Module

106: Confirm module

107: display module

20: memory

30: camera unit

40: display unit

2: server

201: Face Image Database

202: Image recognition programming interface

S101~S110: steps

1 is a schematic diagram of the application structure of an electronic device in a preferred embodiment of the present invention.

FIG. 2 is a schematic diagram of using a cascaded convolutional neural network algorithm to detect a user's face in a second user image in a preferred embodiment of the present invention.

3 is a schematic diagram of a rectangular frame of a human face of a second user image in a preferred embodiment of the present invention.

Fig. 4 is a schematic flowchart of a face recognition method in a preferred embodiment of the present invention.

Please refer to FIG. 1, which is a schematic diagram of an application structure of an electronic device 1 provided by a preferred embodiment of the present invention. In this embodiment, the electronic device 1 is in communication connection with the server 2. The electronic device 1 is used to capture an image of the current user, and recognizes services based on the captured image and the image provided by the server 2. Identify the user information in the image. In this embodiment, the electronic device 1 is a smart phone or a personal computer, and the server 2 is a single server, a cloud server or a server cluster for providing image recognition services.

The electronic device 1 includes, but is not limited to, a processor 10, a memory 20, a camera unit 30, and a display unit 40. In this embodiment, the processor 10 is preferably a microprocessor. The memory 20 is preferably a read-only memory or a random access memory. The camera unit 30 is a camera built into the electronic device 1. In other embodiments, the camera unit 30 may also be a camera or a video camera externally connected via a USB data cable or connected via wireless communication. The display unit 40 is a Liquid Crystal Display (LCD) or Organic Light-Emitting Diode (OLED) display.

The server 2 at least includes a face image database 201 and an image recognition programming interface (API) 202. In this embodiment, the electronic device 1 is wirelessly connected to the server 2 via the Internet or WI-FI.

The electronic device 1 at least includes a camera module 101, an upload module 102, a receiving module 103, a detection module 104, a judgment module 105, a determination module 106, and a display module 107. In this embodiment, the aforementioned modules 101-107 are stored in the memory 20 and can be invoked and executed by the processor 10 Programmable software instructions. It can be understood that, in other implementation manners, the above-mentioned modules may also be program instructions or firmware that are solidified in the processor 10.

The camera module 101 is used to control the camera unit 30 to take a user image in front of the electronic device 1 at a preset time.

In this embodiment, the electronic device 1 is applied to an application scenario that needs to continuously detect and identify user information to verify user identity, such as a video conference, which can prevent unauthorized persons from obtaining conference content.

In this embodiment, the camera unit 30 is installed above the display unit 40 so as to capture the face image of the current user. Wherein, the preset time is five minutes. In other embodiments, the preset time can also be set to other times according to requirements.

The upload module 102 is used to upload the first user image captured by the camera unit 30 to the server 2. It should be noted that the first user image is the first user image captured by the camera unit 30 after the user enters the application scene.

In this embodiment, the electronic device 1 is installed with facial recognition software and has an image upload function. The user can manually upload the first user image taken by the camera unit 30 to the server 2 on the electronic device 1 through the face recognition software. In other embodiments, the electronic device 1 may also automatically upload the first user image to the server 2 when the image capturing unit 30 captures an image containing the user.

The receiving module 103 is configured to receive the first user image including the face rectangle and user information recognized and returned by the server 2.

In this embodiment, the face image database 201 of the server 2 stores and records multiple user face images required in the application scene, which are uploaded in advance by the application scene manager. A user’s face image contains the user’s identity information as a basis for identity recognition and verification.

Specifically, when the server 2 receives the first user image uploaded by the upload module 102, the image recognition programming interface 202 uses machine learning to identify the user in the first user image. And select the face in the form of a rectangular frame. Further, the server 2 analyzes the facial features of the user in the first user image through the image recognition programming interface 202, and compares the facial features of the user in the image with the facial image data The facial features of each face image in the library 201 are compared. When the similarity between the facial features of a face image and the facial features of the user in the first user image is greater than or equal to a preset percentage, the server 2 determines the user corresponding to the face image The identity information is the user information in the first user image. Wherein, the user information includes at least name, gender, and age. In this embodiment, the preset percentage is 90%.

The server 2 also returns a first user image including a rectangular frame with a human face and user information to the electronic device 1. It should be noted that when the similarity between the facial features of all facial images in the face image database 201 and the facial features of the user in the first user image is less than the preset percentage , The server 2 sends back to the electronic device 1 the prompt information that the user has not passed the identity verification in the first user image. That is, the receiving module 103 may also receive the prompt information that the user in the first user image returned by the server 2 has not passed the identity verification.

The detection module 104 is used to detect the face rectangle of the second user image of the former user of the electronic device 1 captured by the camera unit 30. It should be noted that the second user image may be any image captured by the camera unit 30 after the first user image is captured.

In this embodiment, the detection module 104 uses a Multi-task Cascaded Convolutional Networks (MTCNN) algorithm to detect the user's face in the second user image.

Specifically, please refer to FIG. 2. First, the detection module 104 scales the second user image to different sizes to generate an image pyramid of the second user image. Then, the detection module 104 performs three stages of processing on the second user image. Among them, the first stage adopts the proposed network Road (Proposal Network, P-Net). In the first stage, the detection module 104 uses a full convolutional network to generate a candidate rectangular frame for the face in the second user image according to the image pyramid and Bounding Box Regression Vectors, and use the Bounding Box Regression algorithm to correct the candidate rectangular boxes and use the Non-Maximum Suppression (NMS) algorithm to merge the overlapping candidate rectangular boxes.

The second stage is an optimization network (Refine Network, R-Net). In the second stage, the detection module 104 improves the candidate rectangular frame of the face in the second user image, specifically: Input the candidate rectangles output in the first stage into the optimization network to delete the wrong rectangles, and continue to use the edge regression algorithm to correct the remaining candidate rectangles and use the non-maximum suppression algorithm to merge the overlapping candidate rectangles .

The third stage is the output network (Output Network, O-Net). In the third stage, the detection module 104 outputs the processed first rectangular frame and corresponding facial feature points. 2. User image. In this embodiment, the number of the facial feature points is five, which are respectively located at the positions corresponding to the two ends of the eyes, nose, and mouth of the face image.

The judgment module 105 is used to judge whether the number of human faces in the second user image is the same as the number of human faces in the first user image.

In this embodiment, the judgment module 105 compares the number of rectangular frames in the second user image with that in the first user image. If the number of rectangular frames in the two images is the same, the determination module 105 determines that the number of faces in the second user image is the same as that in the first user image. If the numbers of rectangular frames in the two images are different, the determination module 105 determines that the number of faces in the second user image and the first user image are different.

The uploading module 102 also converts the second user image to the second user image when the judgment module 105 determines that the number of faces in the second user image is the same as the number of faces in the first user image. The image is uploaded to the server 2 to recognize the rectangular frame of the human face in the second user image. The receiving module 103 also receives a second user image returned by the server 2 including the rectangular frame of the human face detected by the detection module 104 and the rectangular frame of the human face detected by the server 2.

The determining module 106 is used to determine whether the difference between the rectangular frame of the face detected by the detection module 104 and the rectangular frame of the face recognized by the server 2 in the second user image is less than or Equal to a preset value.

In this embodiment, the difference between the rectangular frame of the human face detected by the detection module 104 and the rectangular frame of the human face recognized by the server 2 in the second user image is the Euclidean distance. That is, the determining module 106 determines the Euclidean frame between the rectangular frame of the human face detected by the detection module 104 and the rectangular frame of the human face recognized by the server 2 in the second user image. Whether the distance is less than or equal to the preset value.

Specifically, please refer to FIG. 3. In the figure, the light-colored line frame is the rectangular frame of the human face detected by the detection module 104, and the dark-colored line frame is the rectangular frame of the human face recognized by the server 2. The determining module 106 firstly determines the four end points of each face rectangular frame, and then determines a set of end points corresponding to the positions of the two face rectangular frames, for example, a set of upper left end points and a set of lower left end points. Point, a set of upper right end points and a set of lower right end points, the distance between the closest set of end points is calculated using the Euclidean distance algorithm. For example, the lower right end points a and b in FIG. 3 are illustrated by taking the planar rectangular coordinate system X0Y established in the second user image as an example. It is assumed that the face rectangle detected by the detection module 104 is The coordinates of the endpoint a of is (X1, Y1), and the coordinates of the endpoint b of the face rectangle identified by the server 2 are (X2, Y2), then the Euclidean distance between the endpoint a and the endpoint b The calculation formula of d is:

The determining module 106 is also used for determining the Euclidean rectangular frame of the face detected by the detection module 104 and the rectangular frame of the face recognized by the server 2 in the second user image. When the distance is less than or equal to the preset value, it is determined that the user information of the second user image is the same as the user information of the first user image.

In the technical solution of the present invention, when the number of faces in different user images taken before and after the electronic device 1 is the same, and the difference in the rectangular frames of the faces is small, it is determined that the users in the different user images are the same person. There is no need for the server 2 to perform user identification and verification again, which simplifies the process of user identification, and saves costs when the image recognition provided by the server 2 requires payment.

The upload module 102 is also configured to: when the judgment module 105 determines that the number of faces in the second user image is different from the number of faces in the first user image, the camera The second user image captured by the unit 30 is uploaded to the server 2 to identify user information in the second user image. The receiving module 103 also receives a second user image containing user information returned by the server 2.

The upload module 102 is also used when the determination module 106 determines that the face rectangle detected by the detection module 104 in the second user image and the person recognized by the server 2 When the difference between the face rectangles is greater than the preset value, the second user image taken by the camera unit 30 is uploaded to the server 2 to identify the user information in the second user image. The receiving module 103 also receives a second user image containing user information returned by the server 2.

The display module 107 is used to display the user image captured by the camera unit 30 on the display unit 40, and display a rectangular frame of the human face and the recognized user information on the user image.

In this embodiment, the display module 107 uses the Open Source Computer Vision (OpenCV) to draw the detection module 104 and the server on the user image displayed on the display unit 40 2 The determined rectangular frame of the user's face, and the user information in the user image recognized by the server 2.

In this embodiment, to facilitate identification, the display module 107 controls the user information corresponding to the female user in the user image to be displayed in red on the display unit 40, and controls the user information corresponding to the male user in the user image to be displayed on the display unit 40. The cell 40 is displayed in blue.

Please refer to FIG. 4, which is a schematic flowchart of a face recognition method in a preferred embodiment of the present invention. According to different requirements and according to different requirements, the order of the steps in the flow diagram can be changed, and some steps can be omitted or combined.

In step S101, the camera unit 30 is controlled to take an image of the user in front of the electronic device 1 every preset time.

Step S102, upload the first user image captured by the camera unit 30 to the server 2.

Step S103, receiving a first user image including a face rectangle and user information recognized and returned by the server 2.

Step S104: Detect the face rectangle of the second user image of the former user of the electronic device 1 captured by the camera unit 30.

Step S105: Determine whether the number of human faces in the second user image is the same as the number of human faces in the first user image. When the judgment result is yes, the process proceeds to step S106; when the judgment result is no, the process proceeds to step S109.

Step S106: Upload the second user image to the server 2 to identify the face rectangle in the second user image.

Step S107: Determine whether the difference between the human face rectangular frame detected by the electronic device 1 and the human face rectangular frame recognized by the server 2 in the second user image is less than or equal to a preset value. When the determination result is yes, the process proceeds to step S108; when the determination result is no, the process proceeds to step S109.

In step S108, it is determined that the user information of the second user image is the same as the user information of the first user image.

Step S109, upload the second user image captured by the camera unit 30 to the server 2 to identify user information in the second user image.

Step S110, receiving a second user image containing user information returned by the server 2.

Further, the method further includes the steps of displaying the user image captured by the camera unit 30 on the display unit 40, and displaying a rectangular frame of the face and the recognized user information on the user image.

In summary, the present invention meets the requirements of an invention patent, and Yan filed a patent application according to law. However, the above are only the preferred embodiments of the present invention. For those who are familiar with the technique of the present invention, equivalent modifications or changes made in accordance with the spirit of the present invention should be covered by the scope of the following patent applications.

1: Electronic device

10: processor

101: camera module

102: Upload module

103: receiving module

104: Detection Module

105: Judgment Module

106: Confirm module

107: display module

20: memory

30: camera unit

40: display unit

2: server

201: Face Image Database

202: Image recognition programming interface

Claims (10)

An electronic device comprising at least a processor and a camera unit. The improvement is that the electronic device is in communication with a server, and the processor is used to: control the camera unit to take pictures of the former user of the electronic device every preset time Image; uploading the first user image captured by the camera unit to the server; receiving the first user image including the face rectangle and user information recognized and returned by the server; detecting The face rectangle in the second user image of the user in front of the electronic device captured by the camera unit; determining whether the number of faces in the second user image is the same as that in the first user image The number of faces is the same; when it is determined that the number of faces in the second user image is the same as the number of faces in the first user image, upload the second user image to the server for identification The face rectangular frame in the second user image; determining whether the difference between the face rectangular frame detected in the second user image and the face rectangular frame recognized by the server is less than or equal to one A preset value; and when it is determined that the difference between the face rectangular frame detected in the second user image and the face rectangular frame recognized by the server is less than or equal to the preset value, determining the The user information of the second user image is the same as the user information of the first user image. The electronic device according to claim 1, wherein the processor is further configured to: when determining that the number of human faces in the second user image is different from the number of human faces in the first user image Uploading the second user image captured by the camera unit to the server to identify the user information in the second user image; and receiving the second user information returned from the server User image. The electronic device according to claim 1, wherein the difference between the face rectangular frame detected in the second user image and the face rectangular frame recognized by the server is the Euclidean distance, and the processing The device is also used to determine whether the Euclidean distance between the human face rectangular frame detected in the second user image and the human face rectangular frame recognized by the server is less than or equal to the preset value. The electronic device according to claim 1, wherein the processor is further configured to: when determining that the face rectangle detected in the second user image and the face rectangle frame recognized by the server When the difference is greater than the preset value, upload the second user image captured by the camera unit to the server to identify the user information in the second user image; and receive a response from the server The transmitted second user image containing user information. The electronic device according to claim 1, wherein the processor is further configured to: use a cascaded convolutional neural network algorithm to detect the user's face and determine the person in the second user image taken by the camera unit Face rectangle. A face recognition method is applied to an electronic device. The electronic device includes at least a camera unit, and the electronic device is also in communication with a server. The improvement is that the method includes the following steps: Take the image of the user in front of the electronic device at a preset time; upload the first user image taken by the camera unit to the server; receive the rectangular frame and the face of the user recognized and returned by the server The first user image of the user information; detecting the face rectangle in the second user image of the user in front of the electronic device captured by the camera unit; determining whether the number of faces in the second user image is The same as the number of human faces in the first user image; When it is determined that the number of human faces in the second user image is the same as the number of human faces in the first user image, upload the second user image to the server to identify the second user image. The face rectangle in the user image; determining whether the difference between the face rectangle frame detected by the electronic device and the face rectangle frame recognized by the server in the second user image is less than or equal to one A preset value; and when it is determined that the difference between the face rectangular frame detected by the electronic device and the face rectangular frame recognized by the server in the second user image is less than or equal to the preset value , It is determined that the user information of the second user image is the same as the user information of the first user image. The face recognition method according to claim 6, wherein the method further includes the following step: when it is determined that the number of faces in the second user image is different from the number of faces in the first user image At the same time, upload the second user image captured by the camera unit to the server to identify the user information in the second user image; and receive the user information returned by the server The second user image. The face recognition method according to claim 6, wherein the difference between the rectangular frame of the face detected by the electronic device and the rectangular frame of the face detected by the server in the second user image is Europe The step “determine whether the difference between the rectangular frame of the face detected by the electronic device and the rectangular frame of the face recognized by the server in the second user image is less than or equal to a preset value" is specifically The method includes: determining whether the Euclidean distance between the face rectangular frame detected by the electronic device and the face rectangular frame recognized by the server in the second user image is less than or equal to the preset value . The face recognition method according to claim 6, wherein the method further includes the following step: when it is determined that the rectangular frame of the face detected by the electronic device in the second user image is recognized by the server When the difference between the received rectangular frames of the human face is greater than the preset value, upload the second user image captured by the camera unit to the server to identify the user information in the second user image; and Receiving a second user image containing user information returned by the server. The face recognition method according to claim 6, wherein the step of "detecting the face rectangle of the second user image of the user in front of the electronic device captured by the camera unit" specifically includes: using a cascaded convolutional neural network The road performance algorithm detects the user's face in the second user image and determines the face rectangle.

Images