CN108875512B - Face recognition method, device, system, storage medium and electronic equipment - Google Patents
Face recognition method, device, system, storage medium and electronic equipment Download PDFInfo
- Publication number
- CN108875512B CN108875512B CN201711269206.4A CN201711269206A CN108875512B CN 108875512 B CN108875512 B CN 108875512B CN 201711269206 A CN201711269206 A CN 201711269206A CN 108875512 B CN108875512 B CN 108875512B
- Authority
- CN
- China
- Prior art keywords
- face
- image
- server
- recognition
- frame
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/161—Detection; Localisation; Normalisation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/56—Provisioning of proxy services
- H04L67/568—Storing data temporarily at an intermediate stage, e.g. caching
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- Image Processing (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a face recognition method, a face recognition device, a face recognition system, a storage medium and electronic equipment, wherein the face recognition method comprises the following steps: carrying out face detection snapshot on the collected frame image to obtain a face image; transmitting the face image of the frame of image to a server for face recognition; when the transmission is started, continuously carrying out face detection snapshot on at least the next frame of image, and caching the face image of the at least the next frame of image; and when receiving a return result from the server, transmitting at least one of the cached face images to the server, and returning to the step of continuously carrying out face detection snapshot on at least the next frame image when starting transmission. The face recognition scheme of the invention can immediately transmit the cached face image after the server returns the result, thereby realizing seamless connection and improving the efficiency of the whole face recognition.
Description
Technical Field
The present invention relates to the field of face recognition technologies, and in particular, to a face recognition method, an apparatus, a system, a storage medium, and an electronic device.
Background
The face recognition technology is being widely used in production and life, and particularly after an algorithm based on deep learning rises, the precision of the whole face recognition system is greatly improved. On the other hand, the face recognition technology based on deep learning is relatively resource-consuming, so that it is difficult for traditional low-end devices (such as low-end gate inhibition machines, low-end mobile phones, etc.) to directly carry the whole face service.
Based on this, a snapshot-mode-based deep learning face system is beginning to be popular. In short, the low-end device (access control machine, low-end mobile phone, snapshot machine) at the client end performs face detection and tracking, and transmits the detected face through the network to the back-end server for identification and comparison. Because the face recognition may not pass the comparison threshold value once, if the result fed back by the server is not ideal, the client needs to take a snapshot again.
Therefore, a typical flow is a repeated periodic operation of "detection-snapshot" and "recognition", and the "detection-snapshot" and the "recognition" are usually performed in series, and the overall time is the sum of the two times of "detection-snapshot" and "recognition", which is repeated for many times, and an effective policy scheduling is lacking.
Disclosure of Invention
In order to solve the above problems, the present invention provides a scheme related to face recognition, which can be used for face recognition and also for recognition of any target object, and only needs to replace a face with another target object. The following briefly describes the scheme of the present invention for face recognition, and more details will be described in the following detailed description with reference to the drawings.
According to an aspect of the present invention, there is provided a face recognition method, including: carrying out face detection snapshot on the collected frame image to obtain a face image; transmitting the face image of the frame of image to a server for face recognition; when the transmission is started, continuously carrying out face detection snapshot on at least the next frame of image, and caching the face image of the at least the next frame of image; and when receiving a return result from the server, transmitting at least one of the cached face images to the server, and returning to the step of continuously carrying out face detection snapshot on at least the next frame image when starting transmission.
In an embodiment of the present invention, the returned result includes a returned result indicating that the recognition was unsuccessful, and, when the returned result indicating that the recognition was unsuccessful is received from the server, at least one of the face images having the same face as the face image of the frame of image that has been transmitted to the server among the cached face images is transmitted to the server.
In an embodiment of the present invention, the returned result includes a returned result of successful recognition, and when the returned result of successful recognition is received from the server, at least one of the face images having a different face from the face image of the one frame of image that has been transmitted to the server among the cached face images is transmitted to the server.
In an embodiment of the present invention, the capturing of the face detection for the acquired frame image includes: detecting a face from the acquired frame image to obtain a face image; and performing quality inspection on the detected face image, and determining the face image passing the quality inspection as an image to be transmitted to a server side for face recognition.
In an embodiment of the present invention, the buffering the face image of the at least next frame image includes: and caching the face image with the optimal quality aiming at the face images with the same face.
In an embodiment of the present invention, the capturing a face detection for an acquired frame image further includes: after the face is detected, the same face identification number is identified for different face images of the same person, so as to determine whether the different face images include the same face based on the face identification number.
In one embodiment of the invention, the quality check comprises checking at least one of: image blur, face pose, and image brightness.
According to another aspect of the present invention, a face recognition apparatus for implementing the face recognition method is provided, the apparatus includes a detection snapshot module, a communication module, and a buffer module, wherein: the detection snapshot module is configured to perform face detection snapshot frame by frame aiming at the collected images to obtain face images; the communication module is configured to transmit the face image from the detection snapshot module or from the cache module to a server for face recognition; the detection snapshot module is further configured to continue face detection snapshot for at least the next frame of image when the communication module starts to implement transmission of the face image, and the cache module caches the face image of the at least the next frame of image; and the communication module is also configured to transmit at least one of the facial images cached by the caching module to the server when a return result is received from the server.
In an embodiment of the present invention, the returned result includes a returned result indicating that the recognition was unsuccessful, and when the returned result indicating that the recognition was unsuccessful is received from the server, the communication module transmits, to the server, at least one of the face images that have the same face as the face image of the frame of image that has been transmitted to the server, among the face images cached by the cache module.
In an embodiment of the present invention, the returned result includes a returned result of successful recognition, and when the returned result of successful recognition is received from the server, the communication module transmits, to the server, at least one of the facial images cached by the caching module, which has a different face from the facial image of the frame of image that has been transmitted to the server.
In an embodiment of the present invention, the detection snapshot module further includes a face detection module and a face snapshot module, wherein the face detection module detects a face from the acquired frame image to obtain a face image; the face snapshot module performs quality inspection on the face image detected by the face detection module, and determines the face image passing the quality inspection as an image to be transmitted to a server side by the communication module for face recognition.
In an embodiment of the present invention, the caching module caches the face image of the at least next frame of image, including: and caching the face image with the optimal quality aiming at the face images with the same face.
In an embodiment of the present invention, the face detection module is further configured to: after the face is detected, the same face identification number is identified for different face images of the same person, so that the communication module or the cache module determines whether the different face images comprise the same face or not based on the face identification number.
In one embodiment of the invention, the quality check performed by the face capture module comprises checking at least one of: image blur, face pose, and image brightness.
According to a further aspect of the present invention, there is provided a face recognition system, the system comprising a storage device and a processor, the storage device having stored thereon a computer program for execution by the processor, the computer program, when executed by the processor, performing the face recognition method of any of the above.
According to a further aspect of the present invention, there is provided a storage medium having stored thereon a computer program which, when executed, performs the face recognition method of any one of the above.
According to another aspect of the present invention, an electronic device is provided, which includes an image acquisition apparatus and the above-mentioned face recognition apparatus.
According to the face recognition method, the device, the system, the storage medium and the electronic equipment, the face image obtained by detection and snapshot is transmitted to the server side for recognition, meanwhile, the face image obtained by detection and snapshot is continuously detected and cached, and after the server side returns a result, the cached face image can be immediately transmitted, so that seamless connection is realized, and the efficiency of overall face recognition is improved.
Drawings
The above and other objects, features and advantages of the present invention will become more apparent by describing in more detail embodiments of the present invention with reference to the attached drawings. The accompanying drawings are included to provide a further understanding of the embodiments of the invention and are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the description serve to explain the principles of the invention and not to limit the invention. In the drawings, like reference numbers generally represent like parts or steps.
FIG. 1 illustrates a schematic block diagram of an example electronic device for implementing a face recognition method, apparatus, system, and storage medium in accordance with embodiments of the present invention;
FIG. 2 shows a schematic flow diagram of a face recognition method according to an embodiment of the invention;
FIG. 3 shows a schematic block diagram of a face recognition apparatus according to an embodiment of the present invention; and
FIG. 4 shows a schematic block diagram of a face recognition system according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, exemplary embodiments according to the present invention will be described in detail below with reference to the accompanying drawings. It is to be understood that the described embodiments are merely a subset of embodiments of the invention and not all embodiments of the invention, with the understanding that the invention is not limited to the example embodiments described herein. All other embodiments, which can be derived by a person skilled in the art from the embodiments of the invention described herein without inventive step, shall fall within the scope of protection of the invention.
First, an example electronic device 100 for implementing a face recognition method, apparatus, system, and storage medium according to an embodiment of the present invention is described with reference to fig. 1.
As shown in FIG. 1, electronic device 100 includes one or more processors 102, one or more memory devices 104, an input device 106, an output device 108, and an image capture device 110, which are interconnected via a bus system 112 and/or other form of connection mechanism (not shown). It should be noted that the components and structure of the electronic device 100 shown in fig. 1 are exemplary only, and not limiting, and the electronic device may have other components and structures as desired.
The processor 102 may be a Central Processing Unit (CPU) or other form of processing unit having data processing capabilities and/or instruction execution capabilities, and may control other components in the electronic device 100 to perform desired functions.
The storage 104 may include one or more computer program products that may include various forms of computer-readable storage media, such as volatile memory and/or non-volatile memory. The volatile memory may include, for example, Random Access Memory (RAM), cache memory (cache), and/or the like. The non-volatile memory may include, for example, Read Only Memory (ROM), hard disk, flash memory, etc. On which one or more computer program instructions may be stored that may be executed by processor 102 to implement client-side functionality (implemented by the processor) and/or other desired functionality in embodiments of the invention described below. Various applications and various data, such as various data used and/or generated by the applications, may also be stored in the computer-readable storage medium.
The input device 106 may be a device used by a user to input instructions and may include one or more of a keyboard, a mouse, a microphone, a touch screen, and the like.
The output device 108 may output various information (e.g., images or sounds) to an external (e.g., user), and may include one or more of a display, a speaker, and the like.
The image capture device 110 may capture images (e.g., photographs, videos, etc.) desired by a user and store the captured images in the storage device 104 for use by other components. The image capture device 110 may be a camera.
For example, an example electronic device for implementing the face recognition method and apparatus according to the embodiment of the present invention may be implemented as an access control device, a mobile phone, a snapshot device, a tablet computer, or the like.
Next, a face recognition method 200 according to an embodiment of the present invention will be described with reference to fig. 2. As shown in fig. 2, the face recognition method 200 may include the following steps:
in step S210, a face detection snapshot is performed on the acquired frame image to obtain a face image.
In one embodiment, the detection snapshot of the face for the acquired one-frame image may include face detection and face snapshot. Wherein, the face detection may include detecting a face from the acquired frame image to obtain a face image. The face capturing may include performing quality inspection on the detected face image, and determining the face image passing the quality inspection as an image to be transmitted to a server for face recognition.
Illustratively, the quality check of the face snapshot on the detected face image may include checking at least one of: image blur, face pose, and image brightness. For example, it is checked whether the image blur degree is below a predetermined threshold value; checking whether the three-dimensional angle of the face is within a preset threshold range; and checking whether the brightness of the face image is within a preset threshold range or not, and the like, and determining that the face image passes the quality check when at least one or more conditions are met. In other examples, the quality check of the face snapshot on the detected face image may also include other items. Based on the quality inspection of the detected face image, the image which is not good in quality and is not beneficial to face recognition can be filtered, so that the success rate of face recognition is improved, and the efficiency is improved.
In addition, in some application scenarios, the step S210 may also perform face detection only on the acquired frame image to obtain a face image, without performing face snapshot.
In step S220, the face image of the frame of image is transmitted to a server for face recognition.
In one embodiment, the "server" described in step S220 may be understood as a local server or a remote server including a base library required for face recognition. Illustratively, the facial image may be transmitted to the server via any suitable wired or wireless network for face recognition by the server.
In step S230, while beginning to implement the transmission, the face detection snapshot is continuously performed on at least one next frame image, and the face image of the at least one next frame image is buffered.
Here, it should be noted that, although step S230 is shown as being subsequent to step S220 in the flowchart shown in fig. 2, step S230 and step S220 are performed in parallel. In other words, during the period of implementing the transmission of the face image and the face recognition, at least the detection snapshot of the next frame image can be performed in parallel with the transmission of the face image, rather than the detection snapshot of the face of the next frame image after the server side performing the face recognition returns the result. Obviously, the scheme of the invention can greatly improve the speed of face recognition.
Furthermore, the detection of the face is continued for the "at least" next frame image in step S230 because the time consumed for the transmission of the face image and the face recognition may be sufficient for the detection of more than one frame image, and of course, may be sufficient or even insufficient for the detection of only one frame image. However, in any case, during the period of carrying out the transmission of the face image and the face recognition, the face recognition of at least the next frame image is continued in parallel therewith, and the efficiency of the overall face recognition can be improved.
In step S240, when receiving a return result from the server, transmitting at least one of the cached face images to the server, and returning to the step of continuously performing face detection snapshot on at least the next frame image when starting transmission.
In one embodiment, the return results received from the server at step S240 may include identifying unsuccessful return results. Wherein the unsuccessful recognition may indicate that no corresponding match is found from the base library based on the previous frame of face image transmitted to the server. Therefore, no matching result is found, it may be that no corresponding matching result exists in the base library, or it may be that only one frame of face image is insufficient to obtain a matching result. In any case, when a return result indicating that the recognition is unsuccessful is received from the server, at least one of the face images having the same face (note that the same face is understood to be the face of the same person) as the face image of the frame of image that has been transmitted to the server in the face image cached in step S230 may be transmitted to the server, so that the server performs face recognition on the person again.
Further, when a return result indicating that the identification is unsuccessful is received from the server, a face image with the best quality may be selected from the face images having the same face as the face image of the frame image that has been transmitted to the server, which are cached in step S230, and transmitted to the server.
In addition, when the face image is cached, if a plurality of face images subjected to the detection snapshot processing are included for the face of the same person, only the face image of the person with the best quality may be cached. Based on the above, when the cached face image is transmitted to the server, only the face image with the best quality is transmitted, so that the transmission time is saved, and the recognition success rate and efficiency are improved.
Furthermore, while the transmission is performed, the operation of returning to step S230 may be continued, that is, while the transmission is performed, the detection snapshot of the face is continued for at least the next frame image, and the face image of the at least the next frame image is buffered.
In another embodiment, the return result received from the server at step S240 may include a return result that identifies success. Wherein, successful recognition can indicate that a corresponding match has been found from the base library based on the previous frame of face image transmitted to the server. When a return result indicating that the recognition is successful is received from the server, at least one of the face images having different faces (note that different faces are understood as faces of different persons) from the face image of the frame of image that has been transmitted to the server in the face image buffered in step S230 may be transmitted to the server, so that the server performs face recognition on the other persons. Further, similarly to the previous embodiment, only the face image with the best quality may be transmitted at the time of transmission, or only the face image with the best quality may be buffered for the face of the same person at the time of buffering.
Likewise, while the transmission is performed, the operation of returning to step S230 may be continued, that is, while the transmission is performed, the detection snapshot of the face is continued for at least the next frame image, and the face image of the at least the next frame image is buffered.
It can be seen from the foregoing embodiments that, in the scheme of the present invention, during the transmission and recognition period, the image of the previous frame after being detected and captured is buffered after being detected and captured, so as to prepare that when the previous frame is unsuccessfully recognized, the image which is detected and captured and is already buffered can be directly transmitted, and it is not necessary to start detecting and capturing the next frame of image when the previous frame is unsuccessfully recognized from the server, thereby effectively improving the efficiency of overall face recognition. Or, even if the previous frame is successfully identified, the next face identification task (face identification of other people) can be performed based on the cached face image, and the efficiency of the whole face identification can be improved.
In a word, no matter whether the face image transmitted to the server side is successfully identified or not, the face image transmission is started, meanwhile, the face detection snapshot is continuously carried out on at least the next frame of image, and the face image of the at least the next frame of image is cached for subsequent face identification, so that the detection snapshot and the transmission identification can be effectively executed in parallel, and the efficiency of the whole face identification can be effectively improved.
Further, in the above-described embodiment, it may be determined whether or not two face images have faces of the same person based on the face identification numbers. The face identification number may be given during the face detection. That is, after a face is detected, a face identification number may be identified for the face, and the face identification numbers of the same person are also the same in different frames. Therefore, as corresponding to the foregoing example, a face image having the same face (or a different face) as the face image previously transmitted to the server may be found from the cache based on the face identification number; in addition, a face image with the best quality in the face images corresponding to the face identification number can be cached based on the face identification number.
In the following we show the beneficial effects of the solution of the present invention by a specific example. For example, let us assume that the time required for detecting a captured image is Td, the time required for capturing is Ta, the time required for transmitting a face image to a server is Tn, and the time required for face recognition of the face image is Tr. In addition, it is assumed that the recognition of N frames results in a successful recognition result.
Then, it is conventional to perform the four processes of detection, capturing, transmission and identification in series, and each process is repeated N times, i.e. the total time T1 is T1 ═ N (Td + Ta + Tn + Tr).
In contrast, according to the above-described aspect of the present invention, while the face image is transmitted to the server for recognition, the detection snapshot is locally continued, and the total time T2 required is T2 ═ Td + Ta + Tn + Tr + (N-1) × Max (Td + Ta, Tn + Tr), where Max (Td + Ta, Tn + Tr) means taking the larger value of Td + Ta and Tn + Tr. It is clear that T2 is smaller than T1 because the local detection snapshot and the server side transmission identification are effectively processed in parallel.
The overall improvement efficiency varies depending on the values of Td, Ta, Tn, and Tr. For example, assuming that Td is 200ms, Ta is 50ms, Tn is 250ms, Tr is 150ms (where ms represents ms), and N is 3, for a certain device as an example, then: t1 ═ 3 × 3 (200+50+250+50) ═ 1.95s (where s denotes seconds); t2 ═ 2 × (200+50+250+150) + Max (200+50,250+150) ═ 1.45s, so the time required was optimized to approximately 26%.
Based on the above description, according to the face recognition method provided by the embodiment of the invention, the face image obtained by detection and snapshot is transmitted to the server for recognition, meanwhile, the detection and snapshot is continuously carried out to obtain the face image and the face image is cached, and after the server returns a result, the cached face image can be transmitted immediately, so that seamless connection is realized, and the efficiency of overall face recognition is improved.
The face recognition method according to the embodiment of the present invention is exemplarily described above. Illustratively, the face recognition method according to the embodiments of the present invention may be implemented in a device, apparatus or system having a memory and a processor.
In addition, the face recognition method provided by the embodiment of the invention has high processing speed, can be conveniently deployed on an access control machine and a snapshot machine, and can also be conveniently deployed on mobile equipment such as a smart phone, a tablet personal computer and a personal computer.
A face recognition apparatus according to another aspect of the present invention is described below with reference to fig. 3. Fig. 3 shows a schematic block diagram of a face recognition apparatus 300 according to an embodiment of the present invention.
As shown in fig. 3, the face recognition apparatus 300 according to the embodiment of the present invention includes a detection snapshot module 310, a communication module 320, and a cache module 330. The various modules may perform the various steps/functions of the face recognition method 200 described above in connection with fig. 2, respectively. Only the main functions of the respective modules of the face recognition apparatus 300 will be described below, and the details that have been described above will be omitted.
The detection snapshot module is configured 310 to perform a detection snapshot of a face frame by frame for the acquired image to obtain a face image. The communication module 320 is configured to transmit the face image from the detection snapshot module 310 or from the cache module 330 to a server for face recognition. The detection snapshot module 310 is further configured to continue the detection snapshot of the human face for at least the next frame of image when the communication module 320 starts to implement the transmission of the human face image, and the buffer module 330 buffers the human face image of the at least the next frame of image. The communication module 320 is further configured to transmit at least one of the face images cached by the caching module 330 to the server upon receiving a return result from the server.
In one embodiment, the detection snapshot module 310 may include a face detection module and a face snapshot module (not shown in fig. 3). The face detection module can detect a face from the acquired frame image to obtain a face image. The face capture module may perform quality inspection on the face image detected by the face detection module, and determine the face image passing the quality inspection as an image to be transmitted to the server by the communication module 320 for face recognition.
Illustratively, the quality check of the face snapshot by the face snapshot module on the detected face image may include checking at least one of: image blur, face pose, and image brightness. For example, it is checked whether the image blur degree is below a predetermined threshold value; checking whether the three-dimensional angle of the face is within a preset threshold range; and checking whether the brightness of the face image is within a preset threshold range or not, and the like, and determining that the face image passes the quality check when at least one or more conditions are met. In other examples, the quality check of the face snapshot performed by the face snapshot module on the detected face image may further include other items. Based on the quality inspection of the face image detected by the face detection module by the face snapshot module, the image which is not good in quality and is not beneficial to face recognition can be filtered, so that the success rate of the face recognition is improved, and the efficiency is improved.
In addition, in some application scenarios, the detection snapshot module 310 may also only include a face detection module, and perform face detection on the acquired frame of image to obtain a face image, without including a face snapshot module to perform face snapshot.
In one embodiment, the "server" to which the communication module 320 transmits the facial images may be understood as a local server or a remote server that includes a base library required for face recognition. For example, the communication module 320 may transmit the facial image to the server via any suitable wired or wireless network for the server to perform face recognition on the facial image.
While the communication module 320 starts to implement the transmission, the detection snapshot module 310 continues to perform face detection snapshot on at least the next frame of image, and the buffer module 330 buffers the face image of the at least the next frame of image.
When the communication module 320 receives a return result from the server, at least one of the face images cached by the caching module 330 may be transmitted to the server. And, when the communication module 320 starts transmission, the detection snapshot module 310 continues to perform face detection snapshot for at least the next frame image.
In one embodiment, the return results received by the communication module 320 from the server may include return results identifying the unsuccessful. Wherein the unsuccessful recognition may indicate that no corresponding match is found from the base library based on the previous frame of face image transmitted to the server. Therefore, no matching result is found, it may be that no corresponding matching result exists in the base library, or it may be that only one frame of face image is insufficient to obtain a matching result. In any case, when the communication module 320 receives a return result indicating that the recognition is unsuccessful from the server, at least one of the face images of the frame of image that has been transmitted to the server, which has the same face (note that the same face is understood to be the face of the same person), among the face images buffered by the buffer module 330, may be transmitted to the server, so that the server performs face recognition on the person again.
Further, when the communication module 320 receives a return result indicating that the recognition is unsuccessful from the server, the face image with the best quality may be selected from the face images which have the same face as the face image of the frame of image which has been transmitted to the server and are buffered by the buffering module 330, and transmitted to the server.
In addition, when the caching module 330 caches the face images, if a plurality of face images subjected to the detection snapshot processing are included for the face of the same person, only the face image of the person with the best quality may be cached. Based on this, when the communication module 320 transmits the cached face image to the server, only the face image with the best quality is transmitted, which not only saves transmission time, but also improves the recognition success rate and efficiency.
Further, while the communication module 320 performs the transmission, the detection snapshot module 310 may continue to perform face detection snapshot on at least the next frame of image, and the buffer module 330 buffers the face image of the at least the next frame of image.
In another embodiment, the return results received by the communication module 320 from the server may include return results that identify a success. Wherein, successful recognition can indicate that a corresponding match has been found from the base library based on the previous frame of face image transmitted to the server. When receiving a return result of successful recognition from the server, the communication module 320 may transmit at least one of the face images having different faces (note that different faces are understood as faces of different persons) from the face image of the frame of image that has been transmitted to the server among the face images buffered by the buffer module 330 to the server, so as to perform face recognition on the other persons by the server. Further, similar to the previous embodiment, only the face image with the best quality may be transmitted when the communication module 320 transmits, or only the face image with the best quality may be buffered for the face of the same person when the buffering module 330 buffers.
Likewise, while the communication module 320 performs the transmission, the detection snapshot module 310 may continue to perform face detection snapshot on at least the next frame of image, and the buffer module 330 buffers the face image of the at least the next frame of image.
Further, in the above-described embodiment, the communication module 320 or the cache module 330 may determine whether the two face images have the same person's face based on the face identification number. The face identification number may be given when the face detection module performs the face detection. That is, after the face detection module detects a face, a face identification number may be identified for the face, and the face identification numbers of the same person are also the same in different frames. Therefore, corresponding to the foregoing example, the communication module 320 may find, from the cache, a face image having the same face (or a different face) as the face image previously transmitted to the server based on the face identification number; in addition, the caching module 330 may cache, based on the face identification number, one of the face images with the best quality corresponding to the face identification number.
Based on the above description, the face recognition device according to the embodiment of the present invention continues to perform detection and snapshot to obtain a face image and perform caching while transmitting the face image obtained by detection and snapshot to the server for recognition, and can immediately transmit the cached face image after the server returns a result, thereby achieving seamless connection, and improving the efficiency of overall face recognition.
FIG. 4 shows a schematic block diagram of a face recognition system 400 according to an embodiment of the present invention. The face recognition system 400 includes a storage device 410 and a processor 420.
Wherein the storage means 410 stores program codes for implementing respective steps in the face recognition method according to an embodiment of the present invention. The processor 420 is configured to run the program codes stored in the storage 410 to perform the corresponding steps of the face recognition method according to the embodiment of the present invention, and is configured to implement the corresponding modules in the face recognition apparatus according to the embodiment of the present invention.
In one embodiment, the program code, when executed by the processor 420, causes the face recognition system 400 to perform the steps of: carrying out face detection snapshot on the collected frame image to obtain a face image; transmitting the face image of the frame of image to a server for face recognition; when the transmission is started, continuously carrying out face detection snapshot on at least the next frame of image, and caching the face image of the at least the next frame of image; and when receiving a return result from the server, transmitting at least one of the cached face images to the server, and returning to the step of continuously carrying out face detection snapshot on at least the next frame image when starting transmission.
In one embodiment, the returned result includes a returned result indicating that the recognition was unsuccessful, and, when the returned result indicating that the recognition was unsuccessful is received from the server, at least one of the face images having the same face as the face image of the one frame of image that has been transmitted to the server is transmitted to the server.
In one embodiment, the returned result includes a returned result of successful recognition, and when the returned result of successful recognition is received from the server, at least one of the face images of the cached face images having a different face from the face image of the frame of image that has been transmitted to the server is transmitted to the server.
In one embodiment, the detection of a face for an acquired frame of image capturing that the program code when executed by the processor 420 causes the face recognition system 400 to perform comprises: detecting a face from the acquired frame image to obtain a face image; and performing quality inspection on the detected face image, and determining the face image passing the quality inspection as an image to be transmitted to a server side for face recognition.
In one embodiment, the caching of the face image of the at least next frame image that the face recognition system 400 is caused to perform when the program code is executed by the processor 420 comprises: and caching the face image with the optimal quality aiming at the face images with the same face.
In one embodiment, the detection of a face for an acquired frame of image capturing performed by the face recognition system 400 when the program code is executed by the processor 420 further comprises: after the face is detected, the same face identification number is identified for different face images of the same person, so as to determine whether the different face images include the same face based on the face identification number.
In one embodiment, the quality check that the face recognition system 400 performs when the program code is executed by the processor 420 includes checking at least one of: image blur, face pose, and image brightness.
Furthermore, according to an embodiment of the present invention, there is also provided a storage medium on which program instructions are stored, which when executed by a computer or a processor are used for executing the corresponding steps of the face recognition method according to an embodiment of the present invention, and for implementing the corresponding modules in the face recognition apparatus according to an embodiment of the present invention. The storage medium may include, for example, a memory card of a smart phone, a storage component of a tablet computer, a hard disk of a personal computer, a Read Only Memory (ROM), an Erasable Programmable Read Only Memory (EPROM), a portable compact disc read only memory (CD-ROM), a USB memory, or any combination of the above storage media. The computer-readable storage medium may be any combination of one or more computer-readable storage media.
In one embodiment, the computer program instructions may implement the functional modules of the face recognition apparatus according to the embodiment of the present invention when executed by a computer, and/or may perform the face recognition method according to the embodiment of the present invention.
In one embodiment, the computer program instructions, when executed by a computer or processor, cause the computer or processor to perform the steps of: carrying out face detection snapshot on the collected frame image to obtain a face image; transmitting the face image of the frame of image to a server for face recognition; when the transmission is started, continuously carrying out face detection snapshot on at least the next frame of image, and caching the face image of the at least the next frame of image; and when receiving a return result from the server, transmitting at least one of the cached face images to the server, and returning to the step of continuously carrying out face detection snapshot on at least the next frame image when starting transmission.
In one embodiment, the returned result includes a returned result indicating that the recognition was unsuccessful, and, when the returned result indicating that the recognition was unsuccessful is received from the server, at least one of the face images having the same face as the face image of the one frame of image that has been transmitted to the server is transmitted to the server.
In one embodiment, the returned result includes a returned result of successful recognition, and when the returned result of successful recognition is received from the server, at least one of the face images of the cached face images having a different face from the face image of the frame of image that has been transmitted to the server is transmitted to the server.
In one embodiment, the computer program instructions, when executed by a computer or processor, cause the computer or processor to perform the detection snap-shot of a human face for an acquired frame of image comprises: detecting a face from the acquired frame image to obtain a face image; and performing quality inspection on the detected face image, and determining the face image passing the quality inspection as an image to be transmitted to a server side for face recognition.
In one embodiment, the computer program instructions, when executed by a computer or processor, cause the computer or processor to perform the caching of the face image of the at least next frame of image comprises: and caching the face image with the optimal quality aiming at the face images with the same face.
In one embodiment, the computer program instructions, when executed by a computer or processor, cause the computer or processor to perform the snap-shot of face detection for an acquired frame of image further comprises: after the face is detected, the same face identification number is identified for different face images of the same person, so as to determine whether the different face images include the same face based on the face identification number.
In one embodiment, the quality check, which the computer program instructions when executed by a computer or processor cause the computer or processor to perform, comprises checking at least one of: image blur, face pose, and image brightness.
In addition, according to the embodiment of the invention, the electronic equipment is also provided, and the electronic equipment can comprise an image acquisition device and a face recognition device. The image acquisition device may be configured to acquire an image, and the face recognition device may perform the face recognition method described above with reference to fig. 2 on the image acquired by the image acquisition device. The face recognition apparatus may be implemented by the face recognition apparatus 300 described above with reference to fig. 3, or may also be implemented by the face recognition system 400 described above with reference to fig. 4. The operation of the face recognition device included in the electronic device according to the embodiment of the present invention and the operation of the electronic device can be understood by those skilled in the art with reference to the foregoing description about fig. 3 or fig. 4, and for brevity, no further description is provided here.
According to the face recognition method, the device, the system, the storage medium and the electronic equipment, the face image obtained by detection and snapshot is transmitted to the server side for recognition, meanwhile, the face image obtained by detection and snapshot is continuously detected and cached, and after the server side returns a result, the cached face image can be immediately transmitted, so that seamless connection is realized, and the efficiency of overall face recognition is improved.
Although the illustrative embodiments have been described herein with reference to the accompanying drawings, it is to be understood that the foregoing illustrative embodiments are merely exemplary and are not intended to limit the scope of the invention thereto. Various changes and modifications may be effected therein by one of ordinary skill in the pertinent art without departing from the scope or spirit of the present invention. All such changes and modifications are intended to be included within the scope of the present invention as set forth in the appended claims.
Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
In the several embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other ways. For example, the above-described device embodiments are merely illustrative, and for example, the division of the units is only one logical functional division, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another device, or some features may be omitted, or not executed.
In the description provided herein, numerous specific details are set forth. It is understood, however, that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure an understanding of this description.
Similarly, it should be appreciated that in the description of exemplary embodiments of the invention, various features of the invention are sometimes grouped together in a single embodiment, figure, or description thereof for the purpose of streamlining the invention and aiding in the understanding of one or more of the various inventive aspects. However, the method of the present invention should not be construed to reflect the intent: that the invention as claimed requires more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive aspects lie in less than all features of a single disclosed embodiment. Thus, the claims following the detailed description are hereby expressly incorporated into this detailed description, with each claim standing on its own as a separate embodiment of this invention.
It will be understood by those skilled in the art that all of the features disclosed in this specification (including any accompanying claims, abstract and drawings), and all of the processes or elements of any method or apparatus so disclosed, may be combined in any combination, except combinations where such features are mutually exclusive. Each feature disclosed in this specification (including any accompanying claims, abstract and drawings) may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise.
Furthermore, those skilled in the art will appreciate that while some embodiments described herein include some features included in other embodiments, rather than other features, combinations of features of different embodiments are meant to be within the scope of the invention and form different embodiments. For example, in the claims, any of the claimed embodiments may be used in any combination.
The various component embodiments of the invention may be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. Those skilled in the art will appreciate that a microprocessor or Digital Signal Processor (DSP) may be used in practice to implement some or all of the functionality of some of the modules according to embodiments of the present invention. The present invention may also be embodied as apparatus programs (e.g., computer programs and computer program products) for performing a portion or all of the methods described herein. Such programs implementing the present invention may be stored on computer-readable media or may be in the form of one or more signals. Such a signal may be downloaded from an internet website or provided on a carrier signal or in any other form.
It should be noted that the above-mentioned embodiments illustrate rather than limit the invention, and that those skilled in the art will be able to design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention may be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by one and the same item of hardware. The usage of the words first, second and third, etcetera do not indicate any ordering. These words may be interpreted as names.
The above description is only for the specific embodiment of the present invention or the description thereof, and the protection scope of the present invention is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present invention, and the changes or substitutions should be covered within the protection scope of the present invention. The protection scope of the present invention shall be subject to the protection scope of the claims.
Claims (9)
1. A face recognition method, comprising:
carrying out face detection snapshot on the collected frame image to obtain a face image;
transmitting the face image of the frame of image to a server for face recognition;
when the transmission is started, continuously carrying out face detection snapshot on at least the next frame of image, and caching the face image of the at least the next frame of image; and
when receiving a return result from the server, transmitting at least one of the cached face images to the server, and returning to the step of continuously carrying out face detection snapshot on at least the next frame of image when starting transmission;
wherein the returned results include returned results that have not been successfully identified and returned results that have been successfully identified, and,
when a return result that the identification is unsuccessful is received from the server, at least one of the face images which have the same face as the face image of the frame of image transmitted to the server in the cached face images is transmitted to the server;
and when a return result of successful recognition is received from the server, transmitting at least one of the face images which have different faces from the face image of the frame of image transmitted to the server in the cached face images to the server.
2. The method according to claim 1, wherein the capturing of the face detection for the acquired frame image comprises:
detecting a face from the acquired frame image to obtain a face image; and
and performing quality inspection on the detected face image, and determining the face image passing the quality inspection as an image to be transmitted to a server side for face recognition.
3. The method of claim 2, wherein the buffering the face image of the at least next frame image comprises:
and caching the face image with the optimal quality aiming at the face images with the same face.
4. The method according to claim 3, wherein the capturing of the detection of the human face for the acquired frame image further comprises:
after the face is detected, the same face identification number is identified for different face images of the same person, so as to determine whether the different face images include the same face based on the face identification number.
5. The method of claim 2, wherein the quality check comprises checking at least one of: image blur, face pose, and image brightness.
6. A face recognition apparatus for implementing the face recognition method according to any one of claims 1 to 5, wherein the apparatus comprises a detection snapshot module, a communication module and a buffer module, wherein:
the detection snapshot module is configured to perform face detection snapshot frame by frame aiming at the collected images to obtain face images;
the communication module is configured to transmit the face image from the detection snapshot module or from the cache module to a server for face recognition;
the detection snapshot module is further configured to continue face detection snapshot for at least the next frame of image when the communication module starts to implement transmission of the face image, and the cache module caches the face image of the at least the next frame of image; and
the communication module is also configured to transmit at least one of the facial images cached by the caching module to the server when a return result is received from the server.
7. A face recognition system, characterized in that the system comprises a storage means and a processor, the storage means having stored thereon a computer program to be run by the processor, the computer program, when being run by the processor, performing the face recognition method according to any one of claims 1-5.
8. A storage medium, characterized in that the storage medium has stored thereon a computer program to be run by a processor, which computer program, when run by the processor, performs the face recognition method according to any one of claims 1-5.
9. An electronic device, characterized in that the electronic device comprises an image acquisition apparatus and the face recognition apparatus of claim 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711269206.4A CN108875512B (en) | 2017-12-05 | 2017-12-05 | Face recognition method, device, system, storage medium and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711269206.4A CN108875512B (en) | 2017-12-05 | 2017-12-05 | Face recognition method, device, system, storage medium and electronic equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108875512A CN108875512A (en) | 2018-11-23 |
| CN108875512B true CN108875512B (en) | 2021-04-23 |
Family
ID=64325591
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201711269206.4A Active CN108875512B (en) | 2017-12-05 | 2017-12-05 | Face recognition method, device, system, storage medium and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108875512B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109618207B (en) * | 2018-12-21 | 2021-01-26 | 网易（杭州）网络有限公司 | Video frame processing method and device, storage medium and electronic device |
| CN109978914B (en) * | 2019-03-07 | 2021-06-08 | 北京旷视科技有限公司 | Face tracking method and device |
| CN110263680B (en) * | 2019-06-03 | 2022-01-28 | 北京旷视科技有限公司 | Image processing method, device and system and storage medium |
| CN112241672B (en) * | 2019-07-19 | 2024-05-03 | 杭州海康威视数字技术股份有限公司 | Identity data association method and device, electronic equipment and storage medium |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106951876A (en) * | 2017-03-27 | 2017-07-14 | 北京品恩科技股份有限公司 | A kind of three-dimensional portrait synthesis recognition system |
| CN107346426A (en) * | 2017-07-10 | 2017-11-14 | 深圳市海清视讯科技有限公司 | A kind of face information collection method based on video camera recognition of face |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101546377A (en) * | 2009-04-28 | 2009-09-30 | 上海银晨智能识别科技有限公司 | Human face image capture system and human face image capture method |
| CN103824064A (en) * | 2014-03-11 | 2014-05-28 | 深圳市中安视科技有限公司 | Huge-amount human face discovering and recognizing method |
| US10146797B2 (en) * | 2015-05-29 | 2018-12-04 | Accenture Global Services Limited | Face recognition image data cache |
| CN106709424B (en) * | 2016-11-19 | 2022-11-11 | 广东中科人人智能科技有限公司 | Optimized monitoring video storage system |
| CN106778645B (en) * | 2016-12-24 | 2018-05-18 | 深圳云天励飞技术有限公司 | A kind of image processing method and device |
-
2017
- 2017-12-05 CN CN201711269206.4A patent/CN108875512B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106951876A (en) * | 2017-03-27 | 2017-07-14 | 北京品恩科技股份有限公司 | A kind of three-dimensional portrait synthesis recognition system |
| CN107346426A (en) * | 2017-07-10 | 2017-11-14 | 深圳市海清视讯科技有限公司 | A kind of face information collection method based on video camera recognition of face |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108875512A (en) | 2018-11-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108875512B (en) | Face recognition method, device, system, storage medium and electronic equipment | |
| CN107278369B (en) | Personnel searching method, device and communication system | |
| US20210224812A1 (en) | Fraudulent transaction identification method and apparatus, server, and storage medium | |
| CN105938552B (en) | Face recognition method and device for automatically updating base map | |
| CN106650662B (en) | Target object shielding detection method and device | |
| US10803544B2 (en) | Systems and methods for enhancing machine vision object recognition through accumulated classifications | |
| CN108875516B (en) | Face recognition method, device, system, storage medium and electronic equipment | |
| US8463025B2 (en) | Distributed artificial intelligence services on a cell phone | |
| CN108256404B (en) | Pedestrian detection method and device | |
| EP2336949B1 (en) | Apparatus and method for registering plurality of facial images for face recognition | |
| CN108875478B (en) | People-authentication-integrated verification method, device and system and storage medium | |
| CN108875476B (en) | Automatic near-infrared face registration and recognition method, device and system and storage medium | |
| CN108875517B (en) | Video processing method, device and system and storage medium | |
| CN110263680B (en) | Image processing method, device and system and storage medium | |
| US9594958B2 (en) | Detection of spoofing attacks for video-based authentication | |
| EP2966591B1 (en) | Method and apparatus for identifying salient events by analyzing salient video segments identified by sensor information | |
| CN109241888B (en) | Neural network training and object recognition method, device and system and storage medium | |
| CN109783680B (en) | Image pushing method, image acquisition device and image processing system | |
| CN110751021A (en) | Image processing method, image processing device, electronic equipment and computer readable medium | |
| CN108875549B (en) | Image recognition method, device, system and computer storage medium | |
| CN110765851A (en) | Registration method, device and equipment | |
| CN110263830B (en) | Image processing method, device and system and storage medium | |
| CN108875538B (en) | Attribute detection method, device and system and storage medium | |
| CN105807888A (en) | Electronic equipment and information processing method | |
| JP2017162179A (en) | Information processing apparatus, information processing method, and program |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |