WO2022095884A1

WO2022095884A1 - Facial recognition application method

Info

Publication number: WO2022095884A1
Application number: PCT/CN2021/128348
Authority: WO
Inventors: 陈普
Original assignee: 华为云计算技术有限公司
Priority date: 2020-11-03
Filing date: 2021-11-03
Publication date: 2022-05-12
Also published as: CN114529959A

Abstract

A facial recognition application method, system and apparatus, and a device and a non-volatile memory. The method comprises: acquiring a picture comprising a face (110); performing facial recognition on the picture to acquire value information bound to a facial recognition result (120); and a computing device inputting the value information into an application by means of an operating system of the computing device (130).

Description

An application method of face recognition

This application claims the priority of the Chinese patent application with the application number of 202011210990.3 and the application title of "An Application Method for Face Recognition", which was submitted to the China Patent Office on November 3, 2020, the entire contents of which are incorporated herein by reference. middle.

technical field

The present application relates to the field of face recognition service (Face Recognition Service), in particular to a face recognition application method, system, device, device and non-volatile memory.

Background technique

Face Recognition Service (FRS for short) is an intelligent service that uses computers to process, analyze and understand face images based on human facial feature information for identity recognition. Face recognition is provided to users in the form of an open API (Application Programming Interface). Users can obtain face processing results by accessing and calling the API in real time, helping users to automatically identify, compare and query faces. etc., to build an intelligent business system and improve business efficiency. At present, face recognition provides the following sub-services: face detection, face comparison, face search, liveness detection, etc. It is suitable for many application scenarios such as enterprise, security, electronic identity, residential security management, public security criminal investigation and so on. In the prior art, the face recognition function is completed by integrating the face recognition cloud service in the application, which requires modification of the client of the application software to integrate the face recognition cloud service, which may also bring security risks to the application.

SUMMARY OF THE INVENTION

The following describes in detail the application method, system, device, device, and non-volatile memory for face recognition provided by the embodiments of the present application, in order to solve the security problem of implementing face recognition through the client in the application.

In a first aspect of the present application, an application method for face recognition is provided. The method includes: acquiring a picture including a human face; performing face recognition on the picture to acquire value information bound to the face recognition result; and inputting the value information into an application by a computing device through its own operating system.

The application method of face recognition provided by this application, through the operating system to open up the communication between the face recognition system and the application, on the premise of avoiding the modification of the application, integrate the face recognition client or face recognition on the computing device system.

In a possible design of the first aspect, the above process of acquiring a face picture may be program-automatic, and the computing device may be continuously reading a video stream. In a possible implementation manner, when the computing device detects a human face in the video stream, it will automatically trigger an action of acquiring a human face picture.

In a possible design of the first aspect, the above process of acquiring a face picture may be triggered by a user, and the triggering method may be set in advance.

In a possible design of the first aspect, the face recognition application method provided by the present application is to input the value information into the application through a simulated input event of the operating system of the computing device.

The operating system of the computing device may provide a program interface for implementing analog input events. After the operating system receives the simulated input event, it will notify the application for execution. During the process, the application of the computing device does not need to communicate directly with the client or system that provides face recognition.

In a possible design of the first aspect, when there are one or more applications in a running state in the computing device, the computing device detects the application located in the uppermost window of the desktop of the operating system of the computing device, and simulates the application. Events are notified to the application for execution.

In a possible design of the first aspect, the computing device may trigger different applications to execute the analog input event according to different triggering methods.

In a possible design of the first aspect, the computing device notifies an input event to a specific application for execution, and the specific application is set in advance.

In a possible design of the first aspect, the picture of the human face may be acquired by the server, or acquired by the computing device.

In a possible design of the first aspect, the computing device performs face recognition on the picture and obtains value information bound with the face recognition result.

A possible design of the first aspect, the performing face recognition on the picture and obtaining the value information bound with the face recognition result includes: the server performing face recognition on the picture and obtaining the face recognition result. Binding value information.

In a possible design of the first aspect, the performing face recognition on the picture and obtaining the value information bound with the face recognition result includes: the server performing face recognition on the picture; The value information bound to the face recognition result.

In a possible design of the first aspect, the value information includes: ID number, mobile phone number, membership number, bank card number, order number, appointment number, pickup code, payment account number, logistics number, license plate number, address, Any one or a combination of information such as billing number and service number.

In a second aspect, the present application provides a system, including a computing device and a server: the computing device is used to obtain a picture including a human face; the server performs face recognition on the picture, and obtains a result bound to the face recognition result. Value information; the computing device inputs the value information to the application through the operating system of the computing device.

In a third aspect, the present application provides a system, including a computing device and a server: the computing device obtains a picture including a human face; the server performs face recognition on the picture; the computing device obtains and binds the face recognition result value information; the computing device inputs the value information into the application through the operating system of the computing device.

In a fourth aspect, the present application provides a system, including a computing device and a server: the server obtains a picture including a human face; the server performs face recognition on the picture; the computing device obtains a result bound to the face recognition result Value information; the computing device inputs the value information into the application through the computing device's operating system.

In a fifth aspect, the application provides a system, including a computing device and a server: the server obtains a picture including a human face; the server performs face recognition on the picture, and obtains value information bound with the face recognition result; The computing device inputs the value information to the application through an operating system of the computing device.

In a sixth aspect, the application provides a system, including a computing device and a server: the computing device obtains a picture including a human face; the server performs face recognition on the picture, and obtains value information bound with the face recognition result ; the computing device inputs the value information into the application through the operating system of the computing device.

In a seventh aspect, the present application provides a face recognition application device, which includes a face processing unit and an operating system. The face processing unit is used to obtain a picture including a human face, perform face recognition on the picture, and obtain value information bound with the face recognition result; the operating system is used to input the value information into the application.

In a possible design of the seventh aspect, the operating system is configured to input the value information to the application by simulating an input event.

In an eighth aspect, the present application provides a computing device including a processor and a memory. The processor executes the instructions in the memory, so that the computing device implements the method implemented by the computing device in the first aspect or any possible design of the first aspect of the present application.

In a ninth aspect, the present application provides a server including a processor and a memory. The processor executes the instructions in the memory, so that the computing device implements the method implemented by the server in the first aspect or any possible design of the first aspect of the present application.

In a tenth aspect, the present application provides a non-volatile memory, comprising instructions, the computer instructions instructing the computer device to perform the first aspect or the method of any possible design of the first aspect.

Description of drawings

FIG. 1 is a schematic diagram of an example of an application system architecture of face recognition provided by an embodiment of the present application.

FIG. 2 is a schematic diagram of an example of a face recognition system architecture provided by an embodiment of the present application

FIG. 3 is a flowchart of an example of a face recognition method provided by an embodiment of the present application.

FIG. 4 is a schematic diagram of an example of an application scenario provided by an embodiment of the present application

FIG. 5 is a schematic diagram of an example structure of a face recognition system provided by an embodiment of the present application.

FIG. 6 is a schematic diagram of an example structure of a face recognition apparatus 300 provided by an embodiment of the present application.

FIG. 7 is an example flowchart of an example of a face recognition method provided by an embodiment of the present application.

FIG. 8 is an exemplary structural diagram of a computing device 500 provided by an embodiment of the present application.

Detailed ways

In order to make the purpose, technical solutions and principles of the present application clearer, the technical solutions of the present invention will be described in further detail below with reference to the accompanying drawings 1 to 7 of the embodiments of the present application.

FIG. 1 is a schematic diagram illustrating an example of the architecture of a face recognition system provided by an embodiment of the present application. As shown in FIG. 1 , the system may include a computing device 101 and a camera 102 . In one embodiment, computing device 101 may be a computing device with an operating system installed, and computing device 101 has a memory, a processor, and a display. In one embodiment, the photographing apparatus 102 is a device equipped with a camera, which may be independent of the computing device 101 or integrated in the computing device 101 . For example, between the photographing device 102 and the computing device 101, a serial interface (Serial port), a universal serial bus (Universal Serial Bus, USB), a Transmission Control Protocol/Internet Protocol (Transmission Control Protocol/Internet Protocol, TCP/ IP), Bluetooth (Bluetooth) and other protocols to transmit data. Optionally, in another possible embodiment, the photographing apparatus 102 may also be capable of automatically acquiring a human face. For example, the photographing apparatus 102 automatically acquires a picture containing a human face after detecting a human face in a video stream.

2 is a schematic diagram of an example of the architecture of a face recognition system provided by an embodiment of the present application, including a computing device 101, a photographing device 102, and a server 103. The server 103 is a device that can provide a face recognition service.

In some possible embodiments, the server 103 may also be a server in a cluster or a data center. A server (eg, server 103 ) in the cluster or data center deploys an instance for providing a face recognition cloud service.

The server 103 deploys an instance for implementing a face instance, and the instance may be a virtual machine or a container. Alternatively, the server 103 does not deploy an instance, but directly provides the face recognition function. Regardless of whether the server 103 provides the face recognition capability through an instance or directly provides the face recognition capability, this application is collectively referred to as the server 103 providing the face recognition capability, such as providing a face recognition service for the computing device 101 .

FIG. 3 is an example flowchart of a method for face recognition provided by an embodiment of the present application, and the method may be applied to the system as shown in FIG. 1 or FIG. 2 . FIG. 3 is a flow chart of integrating face recognition on a computing device provided by this embodiment, and the steps of an embodiment of the method may include:

Step 110: Obtain a picture including a human face.

In a possible implementation manner, the computing device 101 acquires a video stream from the photographing device 102, and processes the video stream to obtain a picture including a human face. Illustratively, the processing may include detecting human faces and capturing human faces.

Optionally, the above manner may also be: the server 103 obtains a video stream from the photographing device 102, processes the video stream, and obtains a picture including a human face.

In a possible implementation, the local photographing device 102 supports the functions of face detection and face capture, and directly sends the captured face photo to the computing device 101, then the computing device 101 does not need to process the video Stream to get face pictures.

In a possible implementation manner, the above manner may also be: the local photographing device 102 directly captures the captured person

The face photo is sent to the server 103 .

In a possible implementation scenario, the above-mentioned process of obtaining a face picture may be program-automatic, and the photographing device 102 may continuously read a video stream to obtain a face picture therefrom. In a possible implementation method, the photographing device 102 will then perform face recognition after acquiring the face.

Optionally, the above process of acquiring a face picture may also be triggered by a user. For example, the acquisition and recognition of the face picture can be triggered by setting a shortcut key or other methods.

Step 120: Perform face recognition on the picture to obtain value information bound to the face recognition result.

In a possible implementation manner, the computing device 101 sends the face picture to the server 103 that can provide a face recognition service, and the server 103 obtains the identification information bound to the face. Alternatively, the computing device 101 acquires the identification information of the face binding based on the face recognition result fed back by the server 103 . The server 103 can be deployed in public cloud clusters and data centers, and can also be any other device that can provide remote face recognition services.

In a possible implementation manner, the computing device 101 implements face recognition locally, and obtains value information bound to the face recognition result.

For example, the value information can be user name, ID number, mobile phone number, membership number, bank card number, order number, appointment number, pickup number, payment account number, logistics number, license plate number, address, bill number, business number A combination of one or more of such valuable informational content.

Step 130 , the computing device 101 inputs the value information into the application through the operating system of the computing device 101 .

After the computing device 101 obtains the value information, its operating system inputs the content of the value information into the input user interface (User Interface, UI) of the application by simulating an input event. For example, in the Windows operating system, an API function keybd_event() for simulating keyboard operations is provided, and this function can simulate corresponding keyboard actions. The keybd_event() function can trigger a key event, which means that a WM_KEYDOWN or WM_KEYUP message will be generated.

In a possible implementation manner, the frontmost application may receive the event message sent by the operating system, where the frontmost application is the application where the window located at the top layer of the system desktop is located. An example of an implementation scenario, when one or more applications are running in a computing device, the computing device detects an application located in the topmost window of the desktop of the operating system of the computing device, and notifies the application of the simulation event implement.

Optionally, the computing device may trigger different applications to execute the analog input event according to different triggering modes. As an example of an application scenario, different trigger modes may correspond to different applications, for example, the shortcut key "K" corresponds to application A, and the shortcut key "M" corresponds to application B.

Optionally, the computing device notifies the input event to a specific application for execution, and the specific application is set in advance. For example, the user sets in advance to notify the "XX gym membership management system" application for execution of the simulated input event.

An example of an application scenario is shown in Figure 4. The photographing device 102 may be an external camera device with a face picture capture function, and the computing device 101 is used for processing the face photos and presenting the results in the application. For example, as shown in FIG. 4 , the photographing device 102 sends the captured face photo to the computing device 101 , and the computing device 101 obtains the bound value information according to the face recognition result after recognizing the face image.

Computing device 101 simulates input events by generating an operating system, and the events carry content data and value information. For example, the Windows system can implement analog input through keybd_event(), the first parameter of the keybd_event() function is the virtual key code, and the virtual key code corresponding to the character in the value information can be set as the first parameter of the function. After receiving the input event from the operating system, the application on the computing device transfers the event to the UI component that can input the user interface that handles the focus for processing. For example, the text input box in focus in Figure 4, the text input box can display the content of the input value information into the component.

In some other possible implementations, in the process of simulating input, if some related configurations are added, new input may be added, for example, a "carriage return" can be simulated after entering the mobile phone number, which can better Adapt the input of the application. It is also possible to execute the configured pre-input shortcut key (usually a combination shortcut key) to trigger a certain input box to be in focus before entering the mobile phone number.

As shown in FIG. 5 , it is a schematic diagram showing an example of the structure of the face recognition system of the computing device 101 provided by the embodiment of the present application. Inside the computing device 101, the face processing unit 202 is configured to obtain a picture including a human face, perform face recognition on the picture, and obtain value information bound to the face recognition result. The operating system 207 is used to input the value information to the application 201 . The application 201 is an application installed and running in the computing device 101, and in this embodiment of the present application, is used to receive the value information input by the operating system, and present the input result.

For example, the face processing unit 202 may be a local application program on the computing device 101, a plug-in based on a web browser, a Javascript program running on a web basis, or other similar methods. The application 201 is an application that requires face recognition, such as a library management system, a gym membership management system, etc., and may also be an application based on a web browser. The photographing device 102 is an external device with an image or video stream acquisition function or a device integrated on the computing device 103 . Optionally, the photographing device 102 may have the function of face detection and acquisition.

In some possible implementations, as shown in FIG. 5 , the face processing unit 202 may further include some subunits: a face acquisition unit 203 , a face recognition unit 204 , an analog input unit 205 , and a face import unit 206 . The four units are described below.

The face obtaining unit 203 is used to obtain the video stream from the photographing device 102, and the video stream is processed to obtain a picture containing the face, and then the obtained face photo is sent to the face recognition unit 204 for face recognition. Optionally, the face obtaining unit 203 may also directly obtain the captured face picture from the photographing device 102 .

The face recognition unit 204 is used for recognizing the face picture and obtaining the value information corresponding to the face. As an example of an application scenario, the face recognition unit 204 accesses the server 103 that provides the face recognition cloud service, and obtains the value information corresponding to the face returned by the server 103 . In this example scenario, the face recognition unit 204 can be used to package and send data, call the cloud face recognition function, obtain identification information returned by the cloud face recognition, and the like. Optionally, the face recognition cloud service may be a face recognition cloud service provided by a public cloud or a private cloud, or a remote face recognition service provided by a set of physical servers. Optionally, the service can also be a module on local computing, integrated in the face recognition unit. In some possible implementations, after acquiring the identification information, the face recognition unit 204 will trigger the analog input of the analog input unit 205 unit.

The analog input unit 205 is used to generate an analog input event of the operating system 207, and then the operating system 207 inputs the value information corresponding to the face into the application. An example of a possible implementation scenario is that the simulated input unit 205 uses the acquired value information to trigger a simulated input event through the interface capability of the operating system 207. For example, the interface provided by the operating system can be used to implement simulated keyboard input.

The face importing unit 206, which is an optional unit, is used to bind the face picture and the identification information corresponding to the face, and upload it to the local or cloud face database. Optionally, the face picture may be obtained through the face obtaining unit 203, or may be locally uploaded from a computing device.

An example of a possible implementation scenario is that the face recognition cloud service cluster establishes a face database based on the cloud service, and imports one or more face information into the face database. Optionally, the face database may also be deployed locally or in other remote devices. The face information includes a face picture and other information bound to it, such as face identification information. The number of faces that can be imported in the face database depends on the actual application scenario. For example, a small store may have several hundred members; a large chain store may have hundreds of thousands of members in a city. In practice, corresponding to the face import function, this module should also have additional functions such as face query, face update, face deletion, and batch deletion of faces.

FIG. 6 is a face recognition device 300 provided by the present application, and the device includes a face processing unit 202 and an operating system 207 . Wherein, regarding the face processing unit 202 and the operating system 207, refer to the description of this part in FIG. 5 .

FIG. 7 is an example flowchart of a method for a face recognition application provided by an embodiment of the present application. As an example of an application scenario, before starting face recognition, first configure the face recognition device 202, such as cloud IoT information (cloud address, port), authentication information (authentication token Token, AK/SK, user name and password, etc.), face recognition configuration information (identification trigger method, trigger shortcut key, whether live detection, etc.), etc. Next, optionally, upload the face picture and identification information or other related operations through the face import module 206 in FIG. 5 , please refer to the foregoing description. Finally, a face recognition process as shown in steps 410 to 470 in FIG. 7 can be implemented on a computing device.

Step 410, trigger face recognition

For example, the user may actively trigger the face obtaining unit 203 of the face processing unit 202 to start face obtaining through a shortcut key or other means.

Optionally, the manner of triggering identification may also be automatic. For example, the face acquisition unit 203 continuously reads the video stream, and when the face picture is acquired from the camera, the face recognition unit 204 is triggered to recognize and perform subsequent operations.

The face recognition unit 204 can realize the collaborative service between the face recognition unit 204 and multiple applications by adding a pre-detection module to recognize the currently installed application.

Step 411: Read the camera video stream.

For example, when the trigger recognition method is active triggering, in this step, the face acquisition unit 203 turns on the shooting function of the shooting device 102 and starts to read the video stream.

Step 420 , face detection, and capture the face.

For example, when the photographing device 102 does not have the function of face detection and capture, the face acquisition unit 203 will complete the function in this step. The face acquisition unit 203 performs face detection on the video stream, and captures a face picture, where the face picture includes a clearly identifiable face.

Step 430, face recognition.

For example, the face recognition unit 204 accesses the face recognition cloud service, for example, the service interface of the called face recognition cloud service. Optionally, if the face recognition calculation is performed locally, the local computing module is called to recognize the current face picture.

Step 440, return the bound value information.

For example, after the face recognition is successful, the server that provides the face recognition cloud service returns the value information bound to the face. For example, face recognition may fail. For example, there is no binding information for the face in the current picture in the face database, and an error message is returned. Optionally, after the recognition fails, you can continue to acquire and recognize face pictures, or stop acquiring and recognizing.

Step 450, trigger the analog input.

For example, after monitoring the return message from the cloud server, the face recognition unit 204 will send the returned message to the analog input unit, and immediately trigger the analog input unit 205 to generate an analog input.

Step 460: Generate analog input through the operating system.

For example, the simulated input unit will input value information or other relevant information into the user interface of the application through the simulated key input of the operating system. For the specific implementation, please refer to the foregoing example description.

This embodiment of the present application further provides a computing device 500 , as shown in FIG. 7 , the computing device 500 may be the computing device 101 or the server 103 . Computing device 500 includes bus 501 , processor 502 , memory 503 , and when computing device 500 represents computing device 101 , may also include an optional component: display 504 . The processor 502 , the memory 503 , and the display 504 communicate through the bus 501 .

The bus 501 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus or the like. The bus can be divided into address bus, data bus, control bus and so on. For ease of presentation, only one thick line is used in FIG. 8, but it does not mean that there is only one bus or one type of bus.

For the computing device 101, the processor 502 may be a central processing unit (Central Processing Unit, CPU), and in addition, there may be other processor chips such as a graphics processing unit (graphics processing unit, GPU). The memory 503 may be a random access memory (random access memory, RAM) or a solid-state drive (solid-state drive, SSD) and other devices or memory instances with storage capabilities. In some possible implementations, the processor 502 may also control other interfaces to receive data. Wherein, the other interface may be a camera or the like.

For the server 103, the processor 502 can be a central processing unit (Central Processing Unit, CPU), and can also be a graphics processing unit (graphics processing unit, GPU), a neural network processor (Neural network Processing Unit, NPU) and an FPGA (Field Programmable Gate Array), etc. one or more.

The memory 503 may be a random access memory (random access memory, RAM) or a solid-state drive (solid-state drive, SSD) and other devices or instances with storage capabilities. Executable program codes are stored in the memory 503, and the processor 502 executes the executable program codes to realize the functions of the aforementioned computing device 101 or the aforementioned functions of the server 103, or to execute the methods described in the aforementioned embodiments by the computing device 101 or the server 103. Steps performed by the server 103 . Optionally, for the computing device 101, the processor 502 controls the display 504 to present the relevant results to the user.

Display 504 is an input/output (I/O) device. The device can display electronic files such as images and text on the screen for users to view. According to different manufacturing materials, the display 504 can be classified into a liquid crystal display (LCD), an organic light emitting diode (OLED) display, and the like.

The descriptions of the processes or structures corresponding to each of the above-mentioned drawings have their own emphasis, and for the parts that are not described in detail in a certain process or structure, reference may be made to the related descriptions of other processes or structures.

Claims

An application method for face recognition, characterized in that the method comprises:

Get a picture including a human face;

Perform face recognition on the picture, and obtain the value information bound with the face recognition result;

The computing device inputs the value information to the application through an operating system of the computing device.
The method according to claim 1, wherein the inputting the value information into the application through the operating system of the computing device comprises:

The value information is input to the application through simulated input events of the operating system of the computing device.
The method according to claim 1 or 2, wherein acquiring a picture including a human face comprises:

The server obtains a picture including a human face; or,

The computing device acquires a picture including a human face.
The method according to any one of claims 1 to 3, wherein the performing face recognition on the picture and obtaining the value information bound with the face recognition result, comprising:

The computing device performs face recognition on the picture and obtains value information bound with the face recognition result.
The method according to any one of claims 1 to 3, wherein the performing face recognition on the picture and obtaining the value information bound with the face recognition result, comprising:

The server performs face recognition on the picture and obtains the value information bound with the face recognition result.
The method according to any one of claims 1 to 3, wherein the performing face recognition on the picture and obtaining the value information bound with the face recognition result, comprising:

The server performs face recognition on the picture;

The computing device acquires the value information bound with the face recognition result.
The method according to any one of claims 1 to 6, wherein the value information comprises any one or a combination of the following:

User name, ID number, mobile phone number, membership number, bank card number, order number, appointment number, pickup code, payment account number, logistics number, license plate number, address, bill number, business number.
A system, characterized in that it includes a computing device and a server;

The computing device acquires a picture including a human face;

The server performs face recognition on the picture, and obtains value information bound with the face recognition result;

The computing device inputs the value information to an application through an operating system of the computing device.
A system, characterized in that it includes a computing device and a server;

The computing device acquires a picture including a human face;

The server performs face recognition on the picture;

The computing device obtains the value information bound with the face recognition result;

The computing device inputs the value information to an application through an operating system of the computing device.
A system, characterized in that it includes a computing device and a server;

The server obtains a picture including a human face;

The server performs face recognition on the picture;

The computing device obtains the value information bound with the face recognition result;

The computing device inputs the value information to an application through an operating system of the computing device.
A system, characterized in that it includes a computing device and a server;

The server obtains a picture including a human face;

The server performs face recognition on the picture, and obtains value information bound with the face recognition result;

The computing device inputs the value information to an application through an operating system of the computing device.
A system, characterized in that it includes a computing device and a server;

The computing device acquires a picture including a human face;

The server performs face recognition on the picture, and obtains value information bound with the face recognition result;

The computing device inputs the value information to an application through an operating system of the computing device.
An application device for face recognition, wherein the device comprises:

a face processing unit, configured to obtain a picture including a face, perform face recognition on the picture, and obtain value information bound to the face recognition result;

An operating system for inputting the value information to the application.
The apparatus of claim 13, wherein the operating system is configured to input the value information to the application by simulating an input event.
A computing device, characterized in that the computing device includes a processor and a memory; the processor executes the instructions in the memory, so that the computing device implements any one of claims 1 to 7 implemented by the computing device method.
A server, characterized in that the server includes a processor and a memory; the processor executes the instructions in the memory, so that the server implements the method implemented by the server in any one of claims 1 to 7.
A non-volatile memory, characterized in that the non-volatile memory includes instructions for implementing the method of any one of claims 1 to 7 .