CN108875476B

CN108875476B - Automatic near-infrared face registration and recognition method, device and system and storage medium

Info

Publication number: CN108875476B
Application number: CN201710655686.1A
Authority: CN
Inventors: 曹志敏
Original assignee: Beijing Kuangshi Technology Co Ltd; Beijing Megvii Technology Co Ltd
Current assignee: Beijing Kuangshi Technology Co Ltd; Beijing Megvii Technology Co Ltd
Priority date: 2017-08-03
Filing date: 2017-08-03
Publication date: 2022-03-15
Anticipated expiration: 2037-08-03
Also published as: CN108875476A

Abstract

The embodiment of the invention provides an automatic near-infrared face registration method and device, an automatic near-infrared face recognition method and device and a storage medium. The automatic near-infrared face registration method comprises the following steps: step S210: automatically acquiring a face near-infrared image containing a target face, which is automatically acquired by a near-infrared camera while an RGB camera acquires a face RGB image containing the target face; step S220: obtaining a target near-infrared image containing a target face at least based on the face near-infrared image; step S230: acquiring target object information related to a target face in a face RGB image, wherein the target object information is identification information of an object to which the target face belongs; step S240: acquiring near-infrared registration information corresponding to a target face at least based on the target near-infrared image and the target object information; and step S250: and storing the near-infrared registration information into a near-infrared database. The method has low matching requirement on the user, good user experience and high face recognition rate.

Description

Automatic near-infrared face registration and recognition method, device and system and storage medium

Technical Field

The invention relates to the field of face recognition, in particular to an automatic near-infrared face registration method, device and system, an automatic near-infrared face recognition method, device and system and a storage medium.

Background

The current face recognition system can be divided into two main types, one is an RGB face recognition system imaging in RGB visible light wave band, and is characterized in that the source of training data is wide, and after a deep learning system is widely adopted in recent years, the recognition accuracy of the system exceeds that of human eyes, and the system is widely applied to on-line and off-line systems. Another broad category is near-infrared face recognition systems, which suffer from a lack of extensive training data. In addition, before the near-infrared face recognition system is used, a user (i.e., a person who needs to perform face recognition) needs to register near-infrared face data (for example, a near-infrared image including a face of the user, personal information of the user, and the like) on a specified device in advance, and the matching requirement on the user is high. Therefore, it is desirable to provide a method for collecting near-infrared face data of a user more conveniently.

Disclosure of Invention

The present invention has been made in view of the above problems. The invention provides an automatic near-infrared face registration method, device and system, an automatic near-infrared face recognition method, device and system and a storage medium.

According to one aspect of the invention, an automatic near-infrared face registration method is provided. The method comprises the following steps: step S210: automatically acquiring a face near-infrared image containing a target face, which is automatically acquired by a near-infrared camera while an RGB camera acquires a face RGB image containing the target face; step S220: obtaining a target near-infrared image containing a target face at least based on the face near-infrared image; step S230: acquiring target object information related to a target face in a face RGB image, wherein the target object information is identification information of an object to which the target face belongs; step S240: acquiring near-infrared registration information corresponding to a target face at least based on the target near-infrared image and the target object information; and step S250: and storing the near-infrared registration information into a near-infrared database.

Exemplarily, step S240 includes: storing the target near-infrared image and the target object information into a temporary image library; and when the number of the target near-infrared images containing the target face stored in the temporary image library reaches a preset number, selecting one or more target near-infrared images with the best image quality from the target near-infrared images containing the target face stored in the temporary image library to obtain near-infrared registration information, wherein the near-infrared registration information comprises one or more target near-infrared images or identification characteristics of one or more target near-infrared images and target object information.

Exemplarily, step S240 includes: and determining that the near-infrared registration information comprises the target near-infrared image or the identification characteristics of the target near-infrared image and target object information.

Exemplarily, before step S230, the method further comprises: acquiring a human face RGB image; obtaining a target RGB image containing a target face based on the face RGB image; and searching in the RGB base based on the target RGB image to determine whether a base RGB image matched with the target RGB image exists in the RGB base or not, or searching in the RGB base based on the identification feature of the target RGB image to determine whether the identification feature of the base RGB image matched with the identification feature of the target RGB image exists in the RGB base or not.

Exemplarily, step S230 includes: and determining the base object information which is stored in the RGB base and is related to the retrieved base RGB image or the identification characteristic of the retrieved base RGB image as the target object information.

Exemplarily, step S230 includes: under the condition that the base RGB image matched with the target RGB image is not retrieved or the identification feature of the base RGB image matched with the identification feature of the target RGB image is not retrieved, outputting prompt information for prompting that the target face is an unknown face; and receiving input information of an operator and determining target object information based on the input information.

Exemplarily, after step S230, the method further comprises: the target RGB image or the identifying characteristics of the target RGB image, along with the target object information, are stored in an RGB base.

Exemplarily, step S230 includes: input information of an operator is received and target object information is determined based on the input information.

Illustratively, the method further comprises: acquiring a human face RGB image; obtaining a target RGB image containing a target face based on the face RGB image; and storing the target RGB image or the identification features of the target RGB image in an RGB base together with the target object information.

Illustratively, obtaining a target RGB image containing a target face based on the face RGB image includes: and carrying out face detection on the face RGB image to obtain an image only containing a target face as a target RGB image.

Exemplarily, step S220 includes: carrying out face detection on the face near-infrared image to determine all faces contained in the face near-infrared image; determining the position of a target face in a face near-infrared image according to the pixel corresponding relation between the face RGB image and the face near-infrared image and the position of the target face in the face RGB image, wherein the pixel corresponding relation is determined based on the physical position relation between an RGB camera and a near-infrared camera; and extracting an image area where the target face is located according to the position of the target face in the near-infrared image of the face and a face detection result of the near-infrared image of the face to obtain an image only containing the target face as the target near-infrared image.

Exemplarily, before step S210, the method further comprises: acquiring an initial RGB image which is acquired by an RGB camera and contains a target face; and judging whether the image quality of the initial RGB image meets the preset requirement, and if so, determining that the initial RGB image is a face RGB image.

According to another aspect of the present invention, there is provided an automatic near-infrared face recognition method, including: step S310: acquiring a face RGB image which is acquired by an RGB camera and contains a face to be recognized; step S320: under the condition that the image quality of the face RGB image does not meet the preset requirement, automatically acquiring a face near-infrared image which is automatically acquired by a near-infrared camera while the RGB camera acquires the face RGB image and contains a face to be recognized; step S330: obtaining a near-infrared image to be recognized containing a face to be recognized at least based on the face near-infrared image; and step S340: and retrieving in the near-infrared base library in the automatic near-infrared face registration method based on the near-infrared image to be recognized or the identification characteristics of the near-infrared image to be recognized so as to determine whether the face to be recognized is a known face.

Exemplarily, after step S310, the method further comprises: under the condition that the image quality of the face RGB image meets the preset requirement, acquiring a to-be-recognized RGB image containing a to-be-recognized face based on the face RGB image; and retrieving in the RGB base based on the RGB image to be recognized or the identification characteristics of the RGB image to be recognized so as to determine whether the face to be recognized is a known face.

For example, step S320 is only executed when the number of bottom library near-infrared images stored in the near-infrared bottom library or the number of bottom library near-infrared images to which the identification features stored in the near-infrared bottom library belong reaches a first number threshold and/or the number of objects corresponding to bottom library near-infrared images stored in the near-infrared bottom library or the number of objects corresponding to the identification features of bottom library near-infrared images stored in the near-infrared bottom library reaches a second number threshold.

According to another aspect of the present invention, there is provided an automatic near-infrared face registration apparatus, comprising: the face near-infrared image acquisition module is used for automatically acquiring a face near-infrared image which is automatically acquired by the near-infrared camera when the RGB camera acquires a face RGB image containing a target face; the target near-infrared image acquisition module is used for acquiring a target near-infrared image containing a target face at least based on the face near-infrared image; the system comprises an object information acquisition module, a face RGB image acquisition module and a face information acquisition module, wherein the object information acquisition module is used for acquiring target object information related to a target face in the face RGB image, and the target object information is identification information of an object to which the target face belongs; the near-infrared registration information acquisition module is used for acquiring near-infrared registration information corresponding to a target face at least based on the target near-infrared image and the target object information; and the near-infrared registration information storage module is used for storing the near-infrared registration information into the near-infrared bottom library.

Illustratively, the near-infrared registration information obtaining module includes: the temporary storage sub-module is used for storing the target near-infrared image and the target object information into a temporary image library; and the near-infrared registration information obtaining sub-module is used for selecting one or more target near-infrared images with the best image quality from the target near-infrared images containing the target faces stored in the temporary image library to obtain the near-infrared registration information when the number of the target near-infrared images containing the target faces stored in the temporary image library reaches a preset number, wherein the near-infrared registration information comprises one or more target near-infrared images or identification characteristics of the one or more target near-infrared images and target object information.

Illustratively, the near-infrared registration information obtaining module includes: and the near-infrared registration information determining submodule is used for determining that the near-infrared registration information comprises a target near-infrared image or the identification characteristics of the target near-infrared image and target object information.

Illustratively, the automatic near-infrared face registration apparatus further includes: the first face RGB image acquisition module is used for acquiring a face RGB image before the object information acquisition module acquires target object information related to a target face in the target RGB image; the second target RGB image acquisition module is used for acquiring a target RGB image containing a target face based on the face RGB image; and the RGB retrieval module is used for retrieving in the RGB base based on the target RGB image so as to determine whether the RGB base has the base RGB image matched with the target RGB image or not, or retrieving in the RGB base based on the identification characteristic of the target RGB image so as to determine whether the RGB base has the identification characteristic of the base RGB image matched with the identification characteristic of the target RGB image or not.

Illustratively, the object information acquiring module includes: and the object information determining submodule is used for determining the base object information which is stored in the RGB base and is related to the searched base RGB image or the identification characteristic of the searched base RGB image as the target object information.

Illustratively, the object information acquiring module includes: the prompt output sub-module is used for outputting prompt information for prompting that the target face is an unknown face under the condition that the base RGB image matched with the target RGB image is not retrieved or the identification feature of the base RGB image matched with the identification feature of the target RGB image is not retrieved; and a first information receiving and determining sub-module for receiving input information of an operator and determining target object information based on the input information.

Illustratively, the automatic near-infrared face registration apparatus further includes: and the first RGB storage module is used for storing the target RGB image or the identification characteristics of the target RGB image and the target object information in an RGB base after the target object information related to the target face in the target RGB image is acquired by the object information acquisition module.

Illustratively, the object information acquiring module includes: and a second information receiving and determining sub-module for receiving input information of an operator and determining target object information based on the input information.

Illustratively, the automatic near-infrared face registration apparatus further includes: the second face RGB image acquisition module is used for acquiring a face RGB image; the second target RGB image acquisition module is used for acquiring a target RGB image containing a target face based on the face RGB image; and the second RGB storage module is used for storing the target RGB image or the identification characteristics of the target RGB image and the target object information in an RGB base.

Illustratively, the first target RGB image capturing module or the second target RGB image capturing module includes: and the first face detection submodule is used for carrying out face detection on the face RGB image so as to obtain an image only containing a target face as a target RGB image.

Illustratively, the target near-infrared image acquisition module includes: the second face detection submodule is used for carrying out face detection on the face near-infrared image so as to determine all faces contained in the face near-infrared image; the position determining submodule is used for determining the position of the target face in the face near-infrared image according to the pixel corresponding relation between the face RGB image and the face near-infrared image and the position of the target face in the face RGB image, wherein the pixel corresponding relation is determined based on the physical position relation between the RGB camera and the near-infrared camera; and the target near-infrared image obtaining sub-module is used for extracting the image area where the target face is located according to the position of the target face in the face near-infrared image and the face detection result of the face near-infrared image so as to obtain the image only containing the target face as the target near-infrared image.

Illustratively, the automatic near-infrared face registration apparatus further includes: the system comprises an initial RGB image acquisition module, a near infrared image acquisition module and a near infrared image acquisition module, wherein the initial RGB image acquisition module is used for acquiring an initial RGB image which is acquired by an RGB camera and contains a target face before the near infrared image acquisition module automatically acquires a face RGB image which is acquired by the near infrared camera and contains the target face at the same time when the RGB camera acquires the face RGB image containing the target face; and the face RGB image determining module is used for judging whether the image quality of the initial RGB image meets the preset requirement, and if so, determining the initial RGB image as the face RGB image.

According to another aspect of the present invention, there is provided an automatic near-infrared face recognition apparatus, including: the face RGB image acquisition module is used for acquiring a face RGB image which is acquired by the RGB camera and contains a face to be recognized; the face near-infrared image acquisition module is used for automatically acquiring a face near-infrared image which is automatically acquired by the near-infrared camera while the RGB camera acquires the face RGB image and contains a face to be recognized under the condition that the image quality of the face RGB image does not meet the preset requirement; the system comprises a to-be-recognized near-infrared image acquisition module, a face recognition module and a face recognition module, wherein the to-be-recognized near-infrared image acquisition module is used for acquiring a to-be-recognized near-infrared image containing a to-be-recognized face at least based on the face near-infrared image; and the near-infrared retrieval module is used for retrieving in the near-infrared bottom library in the automatic near-infrared face registration method based on the near-infrared image to be recognized or the identification characteristics of the near-infrared image to be recognized so as to determine whether the face to be recognized is a known face.

Illustratively, the automatic near-infrared face recognition device further comprises: the face RGB image acquisition module is used for acquiring a face RGB image which is acquired by the RGB camera and contains a face to be recognized, and acquiring the RGB image to be recognized containing the face to be recognized based on the face RGB image under the condition that the image quality of the face RGB image meets the preset requirement; and the RGB retrieval module is used for retrieving in the RGB base based on the RGB image to be recognized or the identification characteristics of the RGB image to be recognized so as to determine whether the face to be recognized is a known face.

Illustratively, the face near-infrared image acquisition module is only started when the number of bottom library near-infrared images stored in the near-infrared bottom library or the number of bottom library near-infrared images to which identification features stored in the near-infrared bottom library belong reaches a first number threshold and/or the number of objects corresponding to bottom library near-infrared images stored in the near-infrared bottom library or the number of objects corresponding to identification features of bottom library near-infrared images stored in the near-infrared bottom library reaches a second number threshold.

According to another aspect of the present invention, there is provided an automatic near-infrared face registration system, comprising an image acquisition device, a near-infrared light source, a processor and a memory, wherein the image acquisition device comprises an RGB camera and a near-infrared camera, the near-infrared light source is configured to emit near-infrared light for illuminating a target face, the memory stores computer program instructions, and the computer program instructions when executed by the processor are configured to perform the following steps: step S210: automatically acquiring a face near-infrared image containing a target face, which is automatically acquired by a near-infrared camera while an RGB camera acquires a face RGB image containing the target face; step S220: obtaining a target near-infrared image containing a target face at least based on the face near-infrared image; step S230: acquiring target object information related to a target face in a face RGB image, wherein the target object information is identification information of an object to which the target face belongs; step S240: acquiring near-infrared registration information corresponding to a target face at least based on the target near-infrared image and the target object information; and step S250: and storing the near-infrared registration information into a near-infrared database.

Illustratively, the step S240 for execution by the computer program instructions when executed by the processor comprises: storing the target near-infrared image and the target object information into a temporary image library; and when the number of the target near-infrared images containing the target face stored in the temporary image library reaches a preset number, selecting one or more target near-infrared images with the best image quality from the target near-infrared images containing the target face stored in the temporary image library to obtain near-infrared registration information, wherein the near-infrared registration information comprises one or more target near-infrared images or identification characteristics of one or more target near-infrared images and target object information.

Illustratively, the step S240 for execution by the computer program instructions when executed by the processor comprises: and determining that the near-infrared registration information comprises the target near-infrared image or the identification characteristics of the target near-infrared image and target object information.

Illustratively, before the step S230 for execution by the processor, the computer program instructions are further for performing the following steps when executed by the processor: acquiring a human face RGB image; obtaining a target RGB image containing a target face based on the face RGB image; and searching in the RGB base based on the target RGB image to determine whether a base RGB image matched with the target RGB image exists in the RGB base or not, or searching in the RGB base based on the identification feature of the target RGB image to determine whether the identification feature of the base RGB image matched with the identification feature of the target RGB image exists in the RGB base or not.

Illustratively, the step S230 for execution by the computer program instructions when executed by the processor comprises: and determining the base object information which is stored in the RGB base and is related to the retrieved base RGB image or the identification characteristic of the retrieved base RGB image as the target object information.

Illustratively, the step S230 for execution by the computer program instructions when executed by the processor comprises: under the condition that the base RGB image matched with the target RGB image is not retrieved or the identification feature of the base RGB image matched with the identification feature of the target RGB image is not retrieved, outputting prompt information for prompting that the target face is an unknown face; and receiving input information of an operator and determining target object information based on the input information.

Illustratively, after step S230 for execution by the processor, the computer program instructions are further for performing the following steps when executed by the processor: the target RGB image or the identifying characteristics of the target RGB image, along with the target object information, are stored in an RGB base.

Illustratively, the step S230 for execution by the computer program instructions when executed by the processor comprises: input information of an operator is received and target object information is determined based on the input information.

Illustratively, the computer program instructions when executed by the processor are further for performing the steps of: acquiring a human face RGB image; obtaining a target RGB image containing a target face based on the face RGB image; and storing the target RGB image or the identification features of the target RGB image in an RGB base together with the target object information.

Illustratively, the step of obtaining a target RGB image containing a target face based on a face RGB image, the computer program instructions being executable by the processor to perform the steps of: and carrying out face detection on the face RGB image to obtain an image only containing a target face as a target RGB image.

Illustratively, the step S220 for execution by the computer program instructions when executed by the processor comprises: carrying out face detection on the face near-infrared image to determine all faces contained in the face near-infrared image; determining the position of a target face in a face near-infrared image according to the pixel corresponding relation between the face RGB image and the face near-infrared image and the position of the target face in the face RGB image, wherein the pixel corresponding relation is determined based on the physical position relation between an RGB camera and a near-infrared camera; and extracting an image area where the target face is located according to the position of the target face in the near-infrared image of the face and a face detection result of the near-infrared image of the face to obtain an image only containing the target face as the target near-infrared image.

Illustratively, before the step S210 for execution by the processor, the computer program instructions are further for performing the following steps when executed by the processor: acquiring an initial RGB image which is acquired by an RGB camera and contains a target face; and judging whether the image quality of the initial RGB image meets the preset requirement, and if so, determining that the initial RGB image is a face RGB image.

According to another aspect of the present invention, there is provided an automatic near-infrared face recognition system, comprising an image acquisition device, a near-infrared light source, a processor and a memory, wherein the image acquisition device comprises an RGB camera and a near-infrared camera, the near-infrared light source is configured to emit near-infrared light for illuminating a face to be recognized, the memory stores computer program instructions, and the computer program instructions when executed by the processor are configured to perform the steps of: step S310: acquiring a face RGB image which is acquired by an RGB camera and contains a face to be recognized; step S320: under the condition that the image quality of the face RGB image does not meet the preset requirement, automatically acquiring a face near-infrared image which is automatically acquired by a near-infrared camera while the RGB camera acquires the face RGB image and contains a face to be recognized; step S330: obtaining a near-infrared image to be recognized containing a face to be recognized at least based on the face near-infrared image; and step S340: and retrieving in the near-infrared base library in the automatic near-infrared face registration method based on the near-infrared image to be recognized or the identification characteristics of the near-infrared image to be recognized so as to determine whether the face to be recognized is a known face.

Illustratively, after step S310 for execution by the processor, the computer program instructions are further for performing the following steps when executed by the processor: under the condition that the image quality of the face RGB image meets the preset requirement, acquiring a to-be-recognized RGB image containing a to-be-recognized face based on the face RGB image; and retrieving in the RGB base based on the RGB image to be recognized or the identification characteristics of the RGB image to be recognized so as to determine whether the face to be recognized is a known face.

For example, the step S320 executed by the computer program instructions when executed by the processor is only executed when the number of bin near-infrared images stored in the near-infrared bin or the number of bin near-infrared images to which the identification feature stored in the near-infrared bin belongs reaches a first number threshold and/or the number of objects corresponding to the bin near-infrared images stored in the near-infrared bin or the number of objects corresponding to the identification feature of the bin near-infrared images stored in the near-infrared bin reaches a second number threshold.

According to another aspect of the present invention, there is provided a storage medium having stored thereon program instructions operable when executed to perform the steps of: step S210: automatically acquiring a face near-infrared image containing a target face, which is automatically acquired by a near-infrared camera while an RGB camera acquires a face RGB image containing the target face; step S220: obtaining a target near-infrared image containing a target face at least based on the face near-infrared image; step S230: acquiring target object information related to a target face in a face RGB image, wherein the target object information is identification information of an object to which the target face belongs; step S240: acquiring near-infrared registration information corresponding to a target face at least based on the target near-infrared image and the target object information; and step S250: and storing the near-infrared registration information into a near-infrared database.

Illustratively, step S240 for execution of the program instructions when executed includes: storing the target near-infrared image and the target object information into a temporary image library; and when the number of the target near-infrared images containing the target face stored in the temporary image library reaches a preset number, selecting one or more target near-infrared images with the best image quality from the target near-infrared images containing the target face stored in the temporary image library to obtain near-infrared registration information, wherein the near-infrared registration information comprises one or more target near-infrared images or identification characteristics of one or more target near-infrared images and target object information.

Illustratively, step S240 for execution of the program instructions when executed includes: and determining that the near-infrared registration information comprises the target near-infrared image or the identification characteristics of the target near-infrared image and target object information.

Illustratively, prior to step S230, in which the program instructions are for execution at runtime, the program instructions are further for performing the following steps at runtime: acquiring a human face RGB image; obtaining a target RGB image containing a target face based on the face RGB image; and searching in the RGB base based on the target RGB image to determine whether a base RGB image matched with the target RGB image exists in the RGB base or not, or searching in the RGB base based on the identification feature of the target RGB image to determine whether the identification feature of the base RGB image matched with the identification feature of the target RGB image exists in the RGB base or not.

Illustratively, the step S230 for execution of the program instructions when running includes: and determining the base object information which is stored in the RGB base and is related to the retrieved base RGB image or the identification characteristic of the retrieved base RGB image as the target object information.

Illustratively, the step S230 for execution of the program instructions when running includes: under the condition that the base RGB image matched with the target RGB image is not retrieved or the identification feature of the base RGB image matched with the identification feature of the target RGB image is not retrieved, outputting prompt information for prompting that the target face is an unknown face; and receiving input information of an operator and determining target object information based on the input information.

Illustratively, after step S230, in which the program instructions are for execution at runtime, the program instructions are further for executing the following steps at runtime: the target RGB image or the identifying characteristics of the target RGB image, along with the target object information, are stored in an RGB base.

Illustratively, the step S230 for execution of the program instructions when running includes: input information of an operator is received and target object information is determined based on the input information.

Illustratively, the program instructions are further operable when executed to perform the steps of: acquiring a human face RGB image; obtaining a target RGB image containing a target face based on the face RGB image; and storing the target RGB image or the identification features of the target RGB image in an RGB base together with the target object information.

Illustratively, the step of obtaining a target RGB image containing a target face based on a face RGB image, which program instructions are operable to execute at runtime, comprises: and carrying out face detection on the face RGB image to obtain an image only containing a target face as a target RGB image.

Illustratively, the step S220 for execution of the program instructions when executed includes: carrying out face detection on the face near-infrared image to determine all faces contained in the face near-infrared image; determining the position of a target face in a face near-infrared image according to the pixel corresponding relation between the face RGB image and the face near-infrared image and the position of the target face in the face RGB image, wherein the pixel corresponding relation is determined based on the physical position relation between an RGB camera and a near-infrared camera; and extracting an image area where the target face is located according to the position of the target face in the near-infrared image of the face and a face detection result of the near-infrared image of the face to obtain an image only containing the target face as the target near-infrared image.

Illustratively, prior to step S210, in which the program instructions are for execution at runtime, the program instructions are further for performing the following steps at runtime: acquiring an initial RGB image which is acquired by an RGB camera and contains a target face; and judging whether the image quality of the initial RGB image meets the preset requirement, and if so, determining that the initial RGB image is a face RGB image.

According to another aspect of the present invention, there is provided a storage medium having stored thereon program instructions operable when executed to perform the steps of: step S310: acquiring a face RGB image which is acquired by an RGB camera and contains a face to be recognized; step S320: under the condition that the image quality of the face RGB image does not meet the preset requirement, automatically acquiring a face near-infrared image which is automatically acquired by a near-infrared camera while the RGB camera acquires the face RGB image and contains a face to be recognized; step S330: obtaining a near-infrared image to be recognized containing a face to be recognized at least based on the face near-infrared image; and step S340: and retrieving in the near-infrared base library in the automatic near-infrared face registration method based on the near-infrared image to be recognized or the identification characteristics of the near-infrared image to be recognized so as to determine whether the face to be recognized is a known face.

Illustratively, after step S310, in which the program instructions are for execution at runtime, the program instructions are further for executing the following steps at runtime: under the condition that the image quality of the face RGB image meets the preset requirement, acquiring a to-be-recognized RGB image containing a to-be-recognized face based on the face RGB image; and retrieving in the RGB base based on the RGB image to be recognized or the identification characteristics of the RGB image to be recognized so as to determine whether the face to be recognized is a known face.

For example, the program instructions, when executed, perform step S320 only when the number of bottom library near-infrared images stored in the near-infrared bottom library or the number of bottom library near-infrared images to which the identification features stored in the near-infrared bottom library belong reaches a first number threshold and/or the number of objects corresponding to bottom library near-infrared images stored in the near-infrared bottom library or the number of objects corresponding to the identification features of bottom library near-infrared images stored in the near-infrared bottom library reaches a second number threshold.

According to the automatic near-infrared face registration method, the automatic near-infrared face registration device and the automatic near-infrared face recognition method, the automatic near-infrared face recognition device and the automatic near-infrared face recognition system, and the storage medium, the near-infrared face registration and recognition of the user can be achieved without active registration of the user, the matching requirement on the user is low, the user experience is good, and the face recognition rate is high.

Drawings

The above and other objects, features and advantages of the present invention will become more apparent by describing in more detail embodiments of the present invention with reference to the attached drawings. The accompanying drawings are included to provide a further understanding of the embodiments of the invention and are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the description serve to explain the principles of the invention and not to limit the invention. In the drawings, like reference numbers generally represent like parts or steps.

FIG. 1 illustrates a schematic block diagram of an example electronic device for implementing an automatic near-infrared face registration method and apparatus in accordance with embodiments of the present invention;

FIG. 2 shows a schematic flow diagram of a method for automatic near-infrared face registration according to one embodiment of the invention;

FIG. 3 shows a schematic flow diagram of a method of automatic near-infrared face recognition according to one embodiment of the present invention;

FIG. 4 shows a schematic block diagram of an automatic near-infrared face registration apparatus according to one embodiment of the present invention;

FIG. 5 shows a schematic block diagram of an automatic near-infrared face recognition apparatus according to one embodiment of the present invention; and

FIG. 6 shows a schematic block diagram of an automatic near-infrared face registration system according to one embodiment of the present invention.

Detailed Description

In order to make the objects, technical solutions and advantages of the present invention more apparent, exemplary embodiments according to the present invention will be described in detail below with reference to the accompanying drawings. It is to be understood that the described embodiments are merely a subset of embodiments of the invention and not all embodiments of the invention, with the understanding that the invention is not limited to the example embodiments described herein. All other embodiments, which can be derived by a person skilled in the art from the embodiments of the invention described herein without inventive step, shall fall within the scope of protection of the invention.

In order to solve the above-mentioned problems, embodiments of the present invention provide an automatic near-infrared face registration method and apparatus, and a corresponding automatic near-infrared face recognition method and apparatus. According to the automatic near-infrared face registration method provided by the embodiment of the invention, the near-infrared image of the target face is automatically acquired while the RGB image of the target face is acquired, and the near-infrared registration information is acquired and stored to finish the near-infrared face registration of the target face. The automatic near-infrared face registration can be carried out in the process of RGB face registration or in the process of RGB face recognition. The accumulated near-infrared registration information can be used for automatically performing near-infrared face recognition when the image quality of the RGB image is poor, and can also be used as training data of a near-infrared face recognition system. Through the method, the near-infrared face data (namely the near-infrared registration information) of the user can be accumulated under the condition that the user does not sense, and the subsequent near-infrared face recognition is favorably realized. The automatic near-infrared face registration method and the automatic near-infrared face recognition method provided by the embodiment of the invention can be well applied to various fields adopting face recognition technology.

First, an example electronic device 100 for implementing an automatic near-infrared face registration method and apparatus according to an embodiment of the present invention is described with reference to fig. 1.

As shown in FIG. 1, electronic device 100 includes one or more processors 102, one or more memory devices 104, an input device 106, an output device 108, an image capture device 110, and a near-infrared light source 112, which are interconnected via a bus system 114 and/or other form of connection (not shown). It should be noted that the components and structure of the electronic device 100 shown in fig. 1 are exemplary only, and not limiting, and the electronic device may have other components and structures as desired.

The processor 102 may be a Central Processing Unit (CPU) or other form of processing unit having data processing capabilities and/or instruction execution capabilities, and may control other components in the electronic device 100 to perform desired functions.

The storage 104 may include one or more computer program products that may include various forms of computer-readable storage media, such as volatile memory and/or non-volatile memory. The volatile memory may include, for example, Random Access Memory (RAM), cache memory (cache), and/or the like. The non-volatile memory may include, for example, Read Only Memory (ROM), hard disk, flash memory, etc. On which one or more computer program instructions may be stored that may be executed by processor 102 to implement client-side functionality (implemented by the processor) and/or other desired functionality in embodiments of the invention described below. Various applications and various data, such as various data used and/or generated by the applications, may also be stored in the computer-readable storage medium.

The input device 106 may be a device used by a user to input instructions and may include one or more of a keyboard, a mouse, a microphone, a touch screen, and the like.

The output device 108 may output various information (e.g., images and/or sounds) to an external (e.g., user), and may include one or more of a display, a speaker, etc.

The image capture device 110 may capture images (including video frames) and store the captured images in the storage device 104 for use by other components. The image capture device 110 may be a surveillance camera. It should be understood that the image capture device 110 is merely an example, and the electronic device 100 may not include the image capture device 110. In this case, the RGB image and the near-infrared image may be captured by using another image capturing device, and the captured images may be transmitted to the electronic apparatus 100.

The image capturing device 110 may be an RGB and near-infrared binocular camera including an RGB camera and a near-infrared camera for capturing an RGB image and a near-infrared image, respectively.

The near-infrared light source 112 is any light source capable of generating near-infrared light, such as a near-infrared Light Emitting Diode (LED) or the like. The near-infrared light source 112 is merely an example, and the electronic device 100 may not include the near-infrared light source 112. In this case, near infrared light can be generated using a separate light emitting device.

Illustratively, the exemplary electronic device for implementing the automatic near-infrared face registration method and apparatus according to the embodiments of the present invention may be implemented on a device such as a personal computer or a remote server.

Next, an automatic near-infrared face registration method according to an embodiment of the present invention will be described with reference to fig. 2. FIG. 2 shows a schematic flow diagram of a method 200 for automatic near-infrared face registration in accordance with one embodiment of the present invention. As shown in fig. 2, the automatic near-infrared face registration method 200 includes the following steps.

In step S210, a near-infrared image of a face including a target face is automatically acquired while the RGB camera acquires RGB images of the face including the target face.

The target face may be the face of any object (i.e., user) that needs RGB face registration or RGB face recognition. In the process of RGB face registration or RGB face recognition, an RGB image of a target face (i.e., a face RGB image) needs to be acquired for registration or recognition. Meanwhile, in the process, a near-infrared image of the target face (namely, a face near-infrared image) can be automatically acquired for near-infrared face registration.

Illustratively, the face RGB image and the face near-infrared image may be complete images acquired by the RGB camera and the near-infrared camera for respective image acquisition regions, respectively. The face RGB image and the face near-infrared image may be original images acquired by an RGB camera and a near-infrared camera, or may be images obtained by preprocessing the original images. In addition, the face RGB image and the face near-infrared image may be a single still image, or may be a certain video frame in a video stream.

The RGB image of the face and/or the near-infrared image of the face may be transmitted to the electronic device 100 by a client device (such as a security device including RGB and near-infrared binocular cameras) to be subjected to RGB face registration or recognition and near-infrared face registration by the processor 102 of the electronic device 100, or may be collected by the image collecting means 110 (including RGB camera and near-infrared camera) included in the electronic device 100 and transmitted to the processor 102 to be subjected to RGB face registration or recognition and near-infrared face registration.

The near-infrared light source 112 and the near-infrared camera may be automatically turned on when the RGB image of the face containing the target face is acquired using the RGB camera. The near-infrared light source 112 is used for emitting near-infrared light to illuminate the target human face. The near-infrared camera is used for acquiring an image of a target face under the irradiation of near-infrared light so as to obtain a face near-infrared image containing the target face.

In step S220, a target near-infrared image including a target face is obtained based on at least the face near-infrared image.

Illustratively, before step S230, the method 200 may further include: and obtaining a target RGB image containing a target face based on the face RGB image.

In one embodiment, some processing may be performed on the face RGB image and the face near-infrared image to obtain new images as the target RGB image and the target near-infrared image. For example, obtaining a target RGB image containing a target face based on the face RGB image may include: and carrying out face detection on the face RGB image to obtain an image only containing a target face as a target RGB image. Similarly, an image area only including the target face in the near-infrared image of the face can be extracted through a face detection mode and the like, and a new image is obtained and is taken as the target near-infrared image. In another embodiment, the face RGB image may be directly used as the target RGB image, and the face near-infrared image may be directly used as the target near-infrared image for subsequent operations. Some embodiments of step S220 are described in cases below.

In some cases, the face RGB image and the face near-infrared image may contain only the target face. For example, when RGB face registration is performed, it is determined that only one person who is an object to which a target face belongs and no other unrelated object exists in image capturing regions of an RGB camera and a near-infrared camera. For this case, according to an example, the face RGB image may be directly taken as the target RGB image, and the face near-infrared image may be directly taken as the target near-infrared image for the subsequent operation. According to another example, the face detection may be performed on the face RGB image and the face near-infrared image, and image areas each including only the target face are extracted, thereby obtaining the target RGB image and the target near-infrared image.

In some cases, it may not be possible to determine how many faces will appear in the face RGB image and the face near-infrared image. In this case, the face RGB image and the face near-infrared image may be subjected to face detection, and the face in the face RGB image and the face in the face near-infrared image are aligned according to the pixel correspondence between the face RGB image and the face near-infrared image, so as to ensure that the target face used for near-infrared face registration is consistent with the target face used for RGB face registration or recognition, thereby ensuring that the association between the target face used for near-infrared face registration and the target object information is correct.

Exemplarily, step S220 may include: carrying out face detection on the face near-infrared image to determine all faces contained in the face near-infrared image; determining the position of a target face in a face near-infrared image according to the pixel corresponding relation between the face RGB image and the face near-infrared image and the position of the target face in the face RGB image, wherein the pixel corresponding relation is determined based on the physical position relation between an RGB camera and a near-infrared camera; and extracting an image area where the target face is located according to the position of the target face in the near-infrared image of the face and a face detection result of the near-infrared image of the face to obtain an image only containing the target face as the target near-infrared image.

Before determining the position of the target face in the near-infrared image of the face, the position of the target face in the RGB image of the face may be determined in advance. For example, when performing RGB face registration, face recognition systems typically require that the target face be located at a particular location in the image capture area of the RGB camera and/or that no other extraneous objects (as described above) be available in the image capture area of the RGB camera in order to obtain correct RGB face data. In this case, the position of the target face in the RGB image of the face can be recognized very easily without worrying about the interference of other faces with the target face.

The pixel correspondence of the face RGB image and the face near-infrared image is known. After the RGB camera and the near-infrared camera are installed, the physical position relationship between the two cameras is basically fixed, and therefore, the pixel correspondence relationship between the images acquired by the two cameras is also fixed. The pixel correspondence may be pre-calculated.

For example, if it is determined that the face a appearing in the RGB face image should appear in a certain image region in the near-infrared face image according to the pixel correspondence, and a face is detected at the image region in the near-infrared face image, it may be determined that the face is the face a. Optionally, an image region where the face a is located in the face near-infrared image may be extracted, and a target near-infrared image is obtained after normalization processing.

Under the condition that the image acquisition environment is complex (for example, in a place with large pedestrian volume), the position of the target face in the face near-infrared image can be accurately determined according to the pixel corresponding relation, and the near-infrared registration information is prevented from being wrong due to the complex environment.

In step S230, target object information related to the target face in the RGB image of the face is obtained, where the target object information is identification information of an object to which the target face belongs.

For example, the target object information may include, but is not limited to, one or more of: the name, sex, age, identity card number, native place, address, etc. of the target human face. The target object information may be input by an operator or retrieved from a base library storing a large amount of base library object information, which will be described later. The operator described herein may be the user himself or an administrator of the face recognition system.

Taking the entrance guard management system of the office building with the face recognition function as an example, in order for an employee to enter and exit the office building, the employee may first perform RGB face registration at the entrance guard management system. The registration process may be that an RGB image containing the face of the employee is captured by an RGB camera and personal information (i.e., target object information) of the employee is input by a system administrator (or the employee himself), illustratively, the name and job number of the employee may be input. Subsequently, the entrance guard management system can associate the acquired RGB image with the name and the job number of the employee, and store the RGB image in the RGB base, thereby completing the RGB face registration of the employee. The personal information of the employee, which is input by the system administrator (or the employee himself/herself), is used to mark the RGB image of the employee, but it may be automatically used by the entrance guard management system for near-infrared face registration of the employee.

In step S240, near-infrared registration information corresponding to the target face is obtained based on at least the target near-infrared image and the target object information.

The near-infrared registration information may include a target near-infrared image corresponding to the target face or an identification feature of the target near-infrared image. The near-infrared registration information may further include target object information corresponding to a target face. Following the example of the entrance guard management system, during the process of RGB face registration of the employee, the near-infrared camera may be used to automatically acquire a near-infrared image including the face of the employee, and the acquired near-infrared image may be associated with target object information input by a system administrator (or the employee himself) and stored in the near-infrared base, so as to complete the near-infrared face registration of the employee.

In one embodiment, each time a target near-infrared image is obtained, the target near-infrared image itself or the identification features thereof are directly associated with target object information as near-infrared registration information corresponding to a target face. Exemplarily, step S240 may include: and determining that the near-infrared registration information comprises the target near-infrared image or the identification characteristics of the target near-infrared image and target object information. The target near-infrared images are not screened, and a large amount of near-infrared registration information can be quickly collected. Under the condition that the image acquisition condition of the near-infrared camera is good, the working mode has high efficiency.

In another embodiment, for the same face, the corresponding target near-infrared images are temporarily stored and gradually accumulated, when enough target near-infrared images are accumulated, the target near-infrared images with good quality are selected from the accumulated target near-infrared images, and the selected target near-infrared images or the identification features of the selected target near-infrared images are associated with the target object information to serve as the near-infrared registration information corresponding to the target face. The method can ensure the definition of the near-infrared images of the bottom database stored in the near-infrared bottom database or the near-infrared images of the bottom database to which the identification features stored in the near-infrared bottom database belong, ensure the quality of the near-infrared registration information, and further ensure the identification accuracy when the near-infrared bottom database is subsequently used for near-infrared face identification.

Exemplarily, step S240 may include: storing the target near-infrared image and the target object information into a temporary image library; and when the number of the target near-infrared images containing the target face stored in the temporary image library reaches a preset number, selecting one or more target near-infrared images with the best image quality from the target near-infrared images containing the target face stored in the temporary image library to obtain near-infrared registration information, wherein the near-infrared registration information comprises one or more target near-infrared images or identification characteristics of one or more target near-infrared images and target object information.

In one example, the RGB camera and the near-infrared camera may capture video streams, and in a certain continuous period, the near-infrared camera may capture a plurality of face near-infrared images for a target face, and obtain a plurality of target near-infrared images correspondingly. In another example, the RGB camera and the near-infrared camera may acquire still images, and the near-infrared camera respectively acquires a plurality of face near-infrared images for a target face at a plurality of different time periods, and correspondingly acquires a plurality of target near-infrared images.

The following continues with the example of the concierge management system described above. For example, it is assumed that the RGB base stores the base object information and the base RGB image corresponding to the face B, and it is assumed that the employee X to which the face B belongs passes in and out of the office building ten times in total, and RGB face recognition is performed on the employee X each time. Therefore, the identity of the employee X is recognized every time, a human face near-infrared image of the employee X is collected at the same time, and the human face near-infrared image is used as a target near-infrared image. The obtained target near-infrared image is associated with personal information of employee X each time. And finally, accumulating to obtain ten target near-infrared images of employee X. Assuming that the preset number is ten, one or more target near-infrared images with the best quality can be selected from the accumulated ten target near-infrared images. For example, the identification features of the selected one or more target near-infrared images can be extracted and the extracted identification features and the target object information can be associated together to form the required near-infrared registration information.

In step S250, the near-infrared registration information is stored in the near-infrared base.

Illustratively, the near-infrared base library may store gradually accumulated near-infrared images of the base library or identification features of the near-infrared images of the base library and related base library object information. And when the near-infrared face recognition is subsequently carried out, the near-infrared registration information can be retrieved from the near-infrared bottom library so as to judge whether the near-infrared registration information matched with the face to be recognized exists.

The identification features described herein may be image features extracted from corresponding images (e.g., target near-infrared image, target RGB image, base near-infrared image, base RGB image, near-infrared image to be recognized, RGB image to be recognized, etc.) based on any existing or future feature extraction method that may occur.

It is to be understood that the target near-infrared image is converted into a bin near-infrared image after it is stored in the near-infrared bin. Similarly, after the identifying features of the target near-infrared image are stored in the near-infrared ground library, they are converted into identifying features of the near-infrared image of the ground library. The target RGB image, the identification feature of the target RGB image, the target object information, and other information have similar transition conditions, and are not described again.

According to the automatic near-infrared face registration method provided by the embodiment of the invention, the near-infrared image of the face is automatically acquired while the RGB image of the target face is acquired, the near-infrared registration information is obtained and stored, and the near-infrared face registration of the target face is completed. The automatic near-infrared face registration can be carried out in the process of RGB face registration or in the process of RGB face recognition.

The automatic near-infrared face registration method provided by the embodiment of the invention has low matching requirement on the user, and can realize automatic accumulation of near-infrared registration information under the condition that the user does not sense the information. The accumulated near-infrared registration information can be used for automatically performing near-infrared face recognition when the image quality of the RGB image is poor, and can also be used as training data of a near-infrared face recognition system. For example, the problem of low RGB face recognition rate in the case of poor lighting conditions can be solved by using the accumulated near-infrared registration information for automatic near-infrared face recognition.

Illustratively, the automatic near-infrared face registration method according to the embodiment of the present invention may be implemented in a device, an apparatus or a system having a memory and a processor.

The automatic near-infrared face registration method according to the embodiment of the invention can be deployed at an image acquisition end, for example, the method can be deployed at the image acquisition end of an access control management system or the image acquisition end of a security monitoring system in public places such as stations, shopping malls, banks and the like. Alternatively, the automatic near-infrared face registration method according to the embodiment of the present invention may also be distributively deployed at the server side (or cloud side) and the client side. For example, a scene RGB image and a scene near-infrared image may be collected at a client, the client transmits the collected images to a server (or a cloud), and the server (or the cloud) performs automatic near-infrared face registration.

According to an embodiment of the present invention, before step S230, the method 200 may further include: acquiring a human face RGB image; obtaining a target RGB image containing a target face based on the face RGB image; and searching in the RGB base based on the target RGB image to determine whether a base RGB image matched with the target RGB image exists in the RGB base or not, or searching in the RGB base based on the identification feature of the target RGB image to determine whether the identification feature of the base RGB image matched with the identification feature of the target RGB image exists in the RGB base or not.

The method for obtaining the target RGB image based on the face RGB image has been described above, and will not be described herein again.

As described above, the near-infrared face registration may be implemented in the process of RGB face recognition. The RGB master may store a master RGB image of a large number of known objects or identifying characteristics of the master RGB image and associated master object information. For example, the target RGB image may be compared with the RGB images in the RGB master library one by one, and the similarity may be calculated separately. And if the bottom library RGB images with the similarity larger than the preset similarity threshold exist, determining that the target face is the known face. Assuming that the retrieved RGB image of the base library belongs to the object C, it indicates that the object to which the target face belongs is the object C.

According to the embodiment of the present invention, step S230 may include: and determining the base object information which is stored in the RGB base and is related to the retrieved base RGB image or the identification characteristic of the retrieved base RGB image as the target object information.

The near-infrared face registration can be performed by using the recognition result of the RGB face recognition. Following the above example, assume that a base RGB image matching the target RGB image is retrieved from the RGB base, the base RGB image belongs to the object C, and the base object information related to the retrieved base RGB image is the base object information of the object C. In this way, the base object information of the object C can be used as the required target object information for storing in association with the target near-infrared image or the identification feature of the target near-infrared image.

According to the embodiment of the present invention, step S230 may include: under the condition that the base RGB image matched with the target RGB image is not retrieved or the identification feature of the base RGB image matched with the identification feature of the target RGB image is not retrieved, outputting prompt information for prompting that the target face is an unknown face; and receiving input information of an operator and determining target object information based on the input information.

The RGB face registration may be a user passive registration. For example, in the RGB face recognition process, if the user is found to be a new user, the user may be prompted to perform RGB face registration. And simultaneously, the near-infrared face registration is carried out by using the object information during the RGB face registration.

For example, if the recognition result of the RGB face recognition shows that the target face is an unknown face, prompt information may be output to prompt the operator to input target object information of an object to which the target face belongs. Assuming that the target face belongs to an object K, the object has never been subjected to RGB face registration before, and therefore RGB registration information matching the target face cannot be retrieved from the RGB base library. In this case, the operator can input the object information of the object K through the input device 106. Target object information of the object K may be determined based on the input information of the operator, and the target object information may be used for being stored in association with a target near-infrared image of the object K or an identification feature of the target near-infrared image of the object K.

Exemplarily, after the step S230, the method 200 may further include: the target RGB image or the identifying characteristics of the target RGB image, along with the target object information, are stored in an RGB base.

Similar to the near-infrared face registration, RGB registration information may be obtained based on the target RGB image and the target object information. Illustratively, the RGB registration information may include the target RGB image or an identifying characteristic of the target RGB image. The RGB registration information may also include target object information. In the RGB face registration stage or the RGB face recognition stage, the identification features of the target RGB image or the target RGB image may be stored in the RGB base together with target object information determined based on input information of an operator for later RGB face recognition. The base database data which can be used for RGB face recognition can be increased by storing the RGB registration information, and the face recognition rate can be improved.

According to the embodiment of the present invention, step S230 may include: input information of an operator is received and target object information is determined based on the input information.

The RGB face registration may also be user active registration, that is, the user is directly regarded as a new user to register without performing RGB face recognition. In the RGB face registration process, an operator directly inputs object information of an object to which a target face belongs. Based on the operator input information, desired target object information may be determined, which may be used for storage in association with the target near-infrared image or the identifying characteristics of the target near-infrared image.

Illustratively, the method 200 may further include: acquiring a human face RGB image; obtaining a target RGB image containing a target face based on the face RGB image; and storing the target RGB image or the identification features of the target RGB image in an RGB base together with the target object information.

The manner and meaning of obtaining the target RGB image based on the face RGB image and storing the target RGB image or the identification features of the target RGB image and the target object information have been described above, and the implementation of this embodiment can be understood with reference to the above description, and will not be described again.

According to an embodiment of the present invention, before step S210, the method 200 may further include: acquiring an initial RGB image which is acquired by an RGB camera and contains a target face; and judging whether the image quality of the initial RGB image meets the preset requirement, and if so, determining that the initial RGB image is a face RGB image.

The image quality of the RGB image is related to the illumination condition of the RGB image, the illumination condition is good, and the image quality is good. Only the RGB images with good image quality are used for subsequent RGB face registration, so that the effectiveness of RGB registration information (the RGB images of the bottom library or the identification characteristics of the RGB images of the bottom library) stored in the RGB bottom library can be ensured, and the accuracy of subsequent RGB face recognition is further ensured. Similarly, only the RGB image with good image quality is used for RGB face recognition, so that the accuracy of RGB face recognition can be ensured.

For example, the quality of the acquired initial RGB image may be evaluated to obtain a quality score of the initial RGB image. And only regarding the initial RGB image with the quality score larger than a preset score threshold value, using the initial RGB image as the face RGB image for RGB face registration or RGB face recognition.

According to another aspect of the invention, an automatic near-infrared face recognition method is provided. FIG. 3 shows a schematic flow diagram of a method 300 for automatic near-infrared face recognition according to one embodiment of the present invention. As shown in fig. 3, the automatic near-infrared face recognition method 300 includes the following steps.

In step S310, a face RGB image including a face to be recognized acquired by the RGB camera is acquired.

In step S320, when the image quality of the RGB image of the face does not meet the preset requirement, the near-infrared image of the face including the face to be recognized, which is automatically acquired by the near-infrared camera while the RGB camera acquires the RGB image of the face, is automatically acquired.

In step S330, a near-infrared image to be recognized including a face to be recognized is obtained based on at least the face near-infrared image.

In step S340, a search is performed in the near-infrared base library described in the automatic near-infrared registration method 200 based on the near-infrared image to be recognized or the identification feature of the near-infrared image to be recognized, so as to determine whether the face to be recognized is a known face.

The near-infrared database of near-infrared registration information that automatically accumulates faces as described above may be used for automatic near-infrared face recognition.

Similar to the RGB face image and the near-infrared face image involved in the automatic near-infrared face registration method 200, the RGB face image and the near-infrared face image involved in the automatic near-infrared face recognition method 300 may be complete images acquired by the RGB camera and the near-infrared camera with respect to their respective image acquisition regions. The face RGB image and the face near-infrared image may be original images acquired by an RGB camera and a near-infrared camera, or may be images obtained by preprocessing the original images. In addition, the face RGB image and the face near-infrared image may be a single still image, or may be a certain video frame in a video stream.

The RGB image to be recognized including the face to be recognized may be obtained based on the face RGB image, and the implementation manner of the RGB image to be recognized is similar to the implementation manner of obtaining the target RGB image based on the face RGB image in the automatic near-infrared face registration method 200. Those skilled in the art can understand the RGB image to be recognized and the obtaining manner thereof based on the above description, and the details are not repeated herein. Similarly, the near-infrared image to be recognized including the face to be recognized may be obtained based on at least the near-infrared image of the face, and the implementation manner of the method is similar to the implementation manner of obtaining the target near-infrared image including the target face based on at least the near-infrared image of the face in the automatic near-infrared face registration method 200. Those skilled in the art can understand the near-infrared image to be recognized and the obtaining manner thereof based on the above description, and the details are not repeated herein.

Illustratively, when the face to be recognized is subjected to RGB face recognition, if it is found that the image quality of the acquired RGB image of the face is not good, RGB face recognition cannot be performed, or the recognition result may be inaccurate, the near-infrared image of the face to be recognized may be automatically acquired, and the near-infrared database may be used for retrieval. For example, if the near-infrared images of the bottom library are stored in the near-infrared bottom library, the near-infrared images to be recognized may be compared with all the near-infrared images of the bottom library one by one, and the similarity is calculated respectively, and the near-infrared images of the bottom library matching the near-infrared images to be recognized are the similarity greater than the preset similarity threshold. And if the near-infrared image of the bottom library matched with the near-infrared image is retrieved from the near-infrared bottom library, the face to be recognized is a known face, otherwise, the face to be recognized is an unknown face.

The preset requirements for determining the image quality of the RGB images of the scene may be the same as or different from the preset requirements for determining the image quality of the initial RGB images described above.

If the near-infrared image identification features of the bottom library are stored in the near-infrared bottom library, the identification features of the near-infrared image to be identified can be compared with the identification features of all bottom library near-infrared images one by one, and the similarity is calculated respectively. A person skilled in the art can understand the retrieval process of the identification features of the near-infrared image of the base library according to the retrieval process of the near-infrared image of the base library, and details are not repeated.

In an environment with poor illumination conditions, such as a dark environment or a point light source environment, the RGB image acquired by the RGB face recognition system has uneven illumination or low signal to noise, so that the face recognition rate is greatly reduced. Because the near-infrared camera can filter most of the energy of the visible light wave band, the imaging quality is stable, and the near-infrared camera is basically not interfered by external light. According to the automatic near-infrared face recognition method provided by the embodiment of the invention, under the condition that the image quality of the RGB image is poor, the face recognition is automatically carried out by adopting a near-infrared face recognition mode, so that the face recognition rate can be greatly improved. In addition, the automatic near-infrared face recognition method provided by the embodiment of the invention does not need user cooperation, and realizes near-infrared face recognition under the condition that the user does not sense, so that the user experience is better. In addition, because the near-infrared face recognition is performed by adopting the near-infrared database which automatically accumulates the near-infrared registration information of the user in the prior art, the automatic near-infrared face recognition method according to the embodiment of the invention is a method which can be automatically upgraded. The automatic near-infrared face recognition and the registration can be performed alternately or simultaneously, and the recognition rate of the automatic near-infrared face recognition is higher and higher along with the larger and larger information amount in the near-infrared bottom library.

According to an embodiment of the present invention, after step S310, the method 300 may further include: under the condition that the image quality of the face RGB image meets the preset requirement, acquiring a to-be-recognized RGB image containing a to-be-recognized face based on the face RGB image; and retrieving in the RGB base based on the RGB image to be recognized or the identification characteristics of the RGB image to be recognized so as to determine whether the face to be recognized is a known face.

If the image quality of the face RGB image is good enough, the face recognition of RGB can be directly carried out, and if the image quality of the face RGB image is not good, the face recognition of near infrared can be automatically carried out. The method is a working mode of automatic switching of double recognition modes, and the mode can automatically select a proper recognition mode to perform face recognition under the condition that a user does not sense the face recognition mode. Therefore, the working mode of automatic switching of the double recognition modes does not affect the user experience, and simultaneously, the high-quality and high-efficiency face recognition can be realized.

According to the embodiment of the present invention, step S320 is executed only when the number of the near-infrared images of the bottom library stored in the near-infrared bottom library or the number of the near-infrared images of the bottom library to which the identification features stored in the near-infrared bottom library belong reaches a first number threshold and/or the number of the objects corresponding to the near-infrared images of the bottom library stored in the near-infrared bottom library or the number of the objects corresponding to the identification features of the near-infrared images of the bottom library stored in the near-infrared bottom library reaches a second number threshold.

In one example, when performing near-infrared face registration, stored in the near-infrared base is identification features of a near-infrared image. Assuming that the first number threshold is 10000, before the identification features of 10000 bottom base near-infrared images are stored in the near-infrared bottom base, the near-infrared face recognition function is not started, and when the identification features of the 10000 bottom base near-infrared images are stored in the near-infrared bottom base, the near-infrared face recognition function is started. After the near-infrared face recognition function is started, when the image quality of the face RGB image is poor, the near-infrared face recognition can be automatically carried out.

In one example, when near-infrared face registration is performed, stored in the near-infrared base library are near-infrared images. Assuming that the second number threshold is 100, the near-infrared face recognition function is not started before the near-infrared images of the bottom banks corresponding to 100 objects are stored in the near-infrared bottom bank, and the near-infrared face recognition function is started when the near-infrared images of the bottom banks corresponding to 100 objects are stored in the near-infrared bottom bank. After the near-infrared face recognition function is started, when the image quality of the face RGB image is poor, the near-infrared face recognition can be automatically carried out.

In the starting condition of the near-infrared face recognition function, the number of the near-infrared images of the base library or the number of the objects corresponding to the identification features of the near-infrared images of the base library or the identification features of the base library can be considered at the same time, and the description is omitted.

According to another aspect of the invention, an automatic near-infrared face registration apparatus is provided. Fig. 4 shows a schematic block diagram of an automatic near-infrared face registration apparatus 400 according to an embodiment of the present invention.

As shown in fig. 4, the automatic near-infrared face registration apparatus 400 according to the embodiment of the present invention includes a face near-infrared image acquisition module 410, a target near-infrared image acquisition module 420, an object information acquisition module 430, a near-infrared registration information acquisition module 440, and a near-infrared registration information storage module 450. The modules may respectively perform the steps/functions of the automatic near-infrared face registration method described above in connection with fig. 2. Only the main functions of the respective components of the automatic near-infrared face registration apparatus 400 will be described below, and the details that have been described above will be omitted.

The face near-infrared image acquisition module 410 is configured to automatically acquire a face near-infrared image including a target face, which is automatically acquired by the near-infrared camera while the RGB camera acquires a face RGB image including the target face. The facial near-infrared image acquisition module 410 may be implemented by the processor 102 in the electronic device shown in fig. 1 executing program instructions stored in the storage 104.

The target near-infrared image obtaining module 420 is configured to obtain a target near-infrared image including a target face based on at least the face near-infrared image. The target near-infrared image acquisition module 420 may be implemented by the processor 102 in the electronic device shown in fig. 1 executing program instructions stored in the storage 104.

The object information obtaining module 430 is configured to obtain target object information related to a target face in the RGB image of the face, where the target object information is identification information of an object to which the target face belongs. The object information acquisition module 430 may be implemented by the processor 102 in the electronic device shown in fig. 1 executing program instructions stored in the storage 104.

The near-infrared registration information obtaining module 440 is configured to obtain near-infrared registration information corresponding to a target face based on at least the target near-infrared image and the target object information. Near-infrared registration information obtaining module 440 may be implemented by processor 102 in the electronic device shown in fig. 1 executing program instructions stored in storage 104.

The near-infrared registration information storage module 450 is configured to store the near-infrared registration information in a near-infrared database. Near-infrared registration information storage module 450 may be implemented by processor 102 in the electronic device shown in fig. 1 executing program instructions stored in storage 104.

Illustratively, the near-infrared registration information obtaining module 440 includes: the temporary storage sub-module is used for storing the target near-infrared image and the target object information into a temporary image library; and the near-infrared registration information obtaining sub-module is used for selecting one or more target near-infrared images with the best image quality from the target near-infrared images containing the target faces stored in the temporary image library to obtain the near-infrared registration information when the number of the target near-infrared images containing the target faces stored in the temporary image library reaches a preset number, wherein the near-infrared registration information comprises one or more target near-infrared images or identification characteristics of the one or more target near-infrared images and target object information.

Illustratively, the near-infrared registration information obtaining module 440 includes: and the near-infrared registration information determining submodule is used for determining that the near-infrared registration information comprises a target near-infrared image or the identification characteristics of the target near-infrared image and target object information.

Illustratively, the automatic near-infrared face registration apparatus 400 further includes: a first face RGB image acquisition module (not shown) for acquiring a face RGB image before the object information acquisition module 430 acquires target object information related to a target face in the target RGB image; a second target RGB image obtaining module (not shown) for obtaining a target RGB image containing a target face based on the face RGB image; and an RGB retrieval module (not shown) for performing a retrieval in the RGB master based on the target RGB image to determine whether a master RGB image matching the target RGB image exists in the RGB master, or performing a retrieval in the RGB master based on the identification feature of the target RGB image to determine whether an identification feature of the master RGB image matching the identification feature of the target RGB image exists in the RGB master.

Illustratively, the object information acquiring module 430 includes: and the object information determining submodule is used for determining the base object information which is stored in the RGB base and is related to the searched base RGB image or the identification characteristic of the searched base RGB image as the target object information.

Illustratively, the object information acquiring module 430 includes: the prompt output sub-module is used for outputting prompt information for prompting that the target face is an unknown face under the condition that the base RGB image matched with the target RGB image is not retrieved or the identification feature of the base RGB image matched with the identification feature of the target RGB image is not retrieved; and a first information receiving and determining sub-module for receiving input information of an operator and determining target object information based on the input information.

Illustratively, the automatic near-infrared face registration apparatus 400 further includes: a first RGB storing module (not shown) for storing the target RGB image or the identification features of the target RGB image in the RGB base together with the target object information after the object information acquiring module 430 acquires the target object information related to the target face in the target RGB image.

Illustratively, the object information acquiring module 430 includes: and a second information receiving and determining sub-module for receiving input information of an operator and determining target object information based on the input information.

Illustratively, the automatic near-infrared face registration apparatus 400 further includes: a second face RGB image obtaining module (not shown) for obtaining a face RGB image; a second target RGB image obtaining module (not shown) for obtaining a target RGB image containing a target face based on the face RGB image; and a second RGB storage module (not shown) for storing the target RGB image or the identification features of the target RGB image together with the target object information in the RGB master.

Illustratively, the target near-infrared image acquisition module 420 includes: the second face detection submodule is used for carrying out face detection on the face near-infrared image so as to determine all faces contained in the face near-infrared image; the position determining submodule is used for determining the position of the target face in the face near-infrared image according to the pixel corresponding relation between the face RGB image and the face near-infrared image and the position of the target face in the face RGB image, wherein the pixel corresponding relation is determined based on the physical position relation between the RGB camera and the near-infrared camera; and the target near-infrared image obtaining sub-module is used for extracting the image area where the target face is located according to the position of the target face in the face near-infrared image and the face detection result of the face near-infrared image so as to obtain the image only containing the target face as the target near-infrared image.

Illustratively, the automatic near-infrared face registration apparatus 400 further includes: an initial RGB image obtaining module (not shown) configured to obtain an initial RGB image including a target face, which is collected by the RGB camera, before the face near-infrared image obtaining module 410 automatically obtains a face near-infrared image including the target face, which is automatically collected by the near-infrared camera while the RGB camera collects a face RGB image including the target face; and the face RGB image determining module is used for judging whether the image quality of the initial RGB image meets the preset requirement, and if so, determining the initial RGB image as the face RGB image.

According to another aspect of the invention, an automatic near-infrared face recognition device is provided. Fig. 5 shows a schematic block diagram of an automatic near-infrared face recognition apparatus 500 according to an embodiment of the present invention.

As shown in fig. 5, the automatic near-infrared face recognition apparatus 500 according to the embodiment of the present invention includes a face RGB image obtaining module 510, a face near-infrared image obtaining module 520, a near-infrared image to be recognized obtaining module 530, and a near-infrared retrieval module 540. The modules may respectively perform the steps/functions of the automatic near-infrared face recognition method described above in connection with fig. 3. Only the main functions of the respective components of the automatic near-infrared face recognition apparatus 500 will be described below, and the details that have been described above will be omitted.

The face RGB image obtaining module 510 is configured to obtain a face RGB image which includes a face to be recognized and is collected by the RGB camera. The face RGB image acquisition module 510 may be implemented by the processor 102 in the electronic device shown in fig. 1 executing program instructions stored in the storage 104.

The face near-infrared image acquisition module 520 is configured to, when the image quality of the RGB image of the face does not meet the preset requirement, automatically acquire a face near-infrared image including the face to be recognized, which is automatically acquired by the near-infrared camera while the RGB camera acquires the RGB image of the face. The facial near-infrared image acquisition module 520 may be implemented by the processor 102 in the electronic device shown in fig. 1 executing program instructions stored in the storage 104.

The to-be-recognized near-infrared image obtaining module 530 is configured to obtain a to-be-recognized near-infrared image including a to-be-recognized face based on at least the face near-infrared image. The near-infrared image acquisition module to be recognized 530 may be implemented by the processor 102 in the electronic device shown in fig. 1 executing program instructions stored in the storage device 104.

The near-infrared retrieval module 540 is configured to perform retrieval in the near-infrared database in the automatic near-infrared face registration method 200 based on the near-infrared image to be recognized or the identification features of the near-infrared image to be recognized, so as to determine whether the face to be recognized is a known face. The near infrared retrieving module 540 may be implemented by the processor 102 in the electronic device shown in fig. 1 executing program instructions stored in the storage 104.

Illustratively, the automatic near-infrared face recognition apparatus 500 further includes: a to-be-recognized RGB image obtaining module (not shown) configured to, after the face RGB image obtaining module 510 obtains a face RGB image which is acquired by the RGB camera and includes a face to be recognized, obtain, based on the face RGB image, a to-be-recognized RGB image including the face to be recognized, when the image quality of the face RGB image meets a preset requirement; and the RGB retrieval module is used for retrieving in the RGB base based on the RGB image to be recognized or the identification characteristics of the RGB image to be recognized so as to determine whether the face to be recognized is a known face.

Illustratively, the face near-infrared image obtaining module 520 is only activated when the number of bottom library near-infrared images stored in the near-infrared bottom library or the number of bottom library near-infrared images to which the identification features stored in the near-infrared bottom library belong reaches a first number threshold and/or the number of objects corresponding to bottom library near-infrared images stored in the near-infrared bottom library or the number of objects corresponding to the identification features of bottom library near-infrared images stored in the near-infrared bottom library reaches a second number threshold.

Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.

It is understood that the automatic near-infrared face registration apparatus 400 and the automatic near-infrared face recognition apparatus 500 can be implemented on the same device (e.g., the electronic device 100).

FIG. 6 shows a schematic block diagram of an automatic near-infrared face registration system 600 in accordance with one embodiment of the present invention. The automatic near-infrared face registration system 600 includes an image acquisition device 610, a storage device 620, a processor 630, and a near-infrared light source 640.

The image capturing device 610 is used for capturing images, such as a human face RGB image and a human face near-infrared image. The image capture device 610 is optional and the automatic near-infrared face registration system 600 may not include the image capture device 610. In this case, other image capturing devices may be used to capture RGB images of a human face and near-infrared images of a human face, and send the captured images to the automatic near-infrared human face registration system 600.

The storage means 620 stores computer program instructions for implementing the corresponding steps in the automatic near-infrared face registration method according to an embodiment of the present invention.

The processor 630 is configured to run the computer program instructions stored in the storage device 620 to execute the corresponding steps of the automatic near-infrared face registration method according to the embodiment of the present invention, and is configured to implement the face near-infrared image acquisition module 410, the target near-infrared image acquisition module 420, the object information acquisition module 430, the near-infrared registration information acquisition module 440, and the near-infrared registration information storage module 450 in the automatic near-infrared face registration apparatus 400 according to the embodiment of the present invention.

In one embodiment, the computer program instructions, when executed by the processor 830, are for performing the steps of: step S210: automatically acquiring a face near-infrared image containing a target face, which is automatically acquired by a near-infrared camera while an RGB camera acquires a face RGB image containing the target face; step S220: obtaining a target near-infrared image containing a target face at least based on the face near-infrared image; step S230: acquiring target object information related to a target face in a face RGB image, wherein the target object information is identification information of an object to which the target face belongs; step S240: acquiring near-infrared registration information corresponding to a target face at least based on the target near-infrared image and the target object information; and step S250: and storing the near-infrared registration information into a near-infrared database.

According to another aspect of the present invention, an automatic near-infrared face recognition system is provided, which includes an image acquisition device, a storage device, a processor, and a near-infrared light source. The image acquisition device, the storage device, the processor and the near-infrared light source in the automatic near-infrared face recognition system are similar in structure to the image acquisition device 610, the storage device 620, the processor 630 and the near-infrared light source 640 in the automatic near-infrared face registration system 600, respectively, and those skilled in the art can understand the structure of the automatic near-infrared face recognition system with reference to fig. 6, and for the sake of brevity, they are not separately shown.

The image acquisition device is used for acquiring images, such as human face RGB images and human face near infrared images. The image acquisition device is optional and the automatic near-infrared face recognition system may not include the image acquisition device. In this case, other image acquisition devices may be used to acquire the RGB image of the human face and the near-infrared image of the human face, and the acquired images may be sent to the automatic near-infrared face recognition system.

The storage device stores computer program instructions for implementing the corresponding steps in the automatic near-infrared face recognition method according to an embodiment of the present invention.

The processor is configured to run the computer program instructions stored in the storage device to execute the corresponding steps of the automatic near-infrared face recognition method according to the embodiment of the present invention, and is configured to implement the face RGB image acquisition module 510, the face near-infrared image acquisition module 520, the to-be-recognized near-infrared image acquisition module 530, and the near-infrared retrieval module 540 in the automatic near-infrared face recognition apparatus 500 according to the embodiment of the present invention.

In one embodiment, the computer program instructions, when executed by the processor, are for performing the steps of: step S310: acquiring a face RGB image which is acquired by an RGB camera and contains a face to be recognized; step S320: under the condition that the image quality of the face RGB image does not meet the preset requirement, automatically acquiring a face near-infrared image which is automatically acquired by a near-infrared camera while the RGB camera acquires the face RGB image and contains a face to be recognized; step S330: obtaining a near-infrared image to be recognized containing a face to be recognized at least based on the face near-infrared image; and step S340: and retrieving in the near-infrared base library in the automatic near-infrared face registration method based on the near-infrared image to be recognized or the identification characteristics of the near-infrared image to be recognized so as to determine whether the face to be recognized is a known face.

Furthermore, according to an embodiment of the present invention, there is also provided a storage medium, on which program instructions are stored, which when executed by a computer or a processor are used to execute corresponding steps of the automatic near-infrared face registration method according to an embodiment of the present invention, and are used to implement corresponding modules in the automatic near-infrared face registration apparatus according to an embodiment of the present invention. The storage medium may include, for example, a memory card of a smart phone, a storage component of a tablet computer, a hard disk of a personal computer, a Read Only Memory (ROM), an Erasable Programmable Read Only Memory (EPROM), a portable compact disc read only memory (CD-ROM), a USB memory, or any combination of the above storage media.

In one embodiment, the program instructions, when executed by a computer or a processor, may cause the computer or the processor to implement the functional modules of the automatic near-infrared face registration apparatus according to the embodiment of the present invention, and/or may execute the automatic near-infrared face registration method according to the embodiment of the present invention.

In one embodiment, the program instructions are operable when executed to perform the steps of: step S210: automatically acquiring a face near-infrared image containing a target face, which is automatically acquired by a near-infrared camera while an RGB camera acquires a face RGB image containing the target face; step S220: obtaining a target near-infrared image containing a target face at least based on the face near-infrared image; step S230: acquiring target object information related to a target face in a face RGB image, wherein the target object information is identification information of an object to which the target face belongs; step S240: acquiring near-infrared registration information corresponding to a target face at least based on the target near-infrared image and the target object information; and step S250: and storing the near-infrared registration information into a near-infrared database.

The modules in the automatic near-infrared face registration system according to the embodiment of the present invention may be implemented by a processor of an electronic device implementing automatic near-infrared face registration according to the embodiment of the present invention running computer program instructions stored in a memory, or may be implemented when computer instructions stored in a computer-readable storage medium of a computer program product according to the embodiment of the present invention are run by a computer.

According to another aspect of the present invention, there is also provided a storage medium on which program instructions are stored, which when executed by a computer or a processor are used for executing the corresponding steps of the automatic near-infrared face recognition method according to the embodiment of the present invention, and for implementing the corresponding modules in the automatic near-infrared face recognition apparatus according to the embodiment of the present invention. The storage medium may include, for example, a memory card of a smart phone, a storage component of a tablet computer, a hard disk of a personal computer, a Read Only Memory (ROM), an Erasable Programmable Read Only Memory (EPROM), a portable compact disc read only memory (CD-ROM), a USB memory, or any combination of the above storage media.

In one embodiment, the program instructions, when executed by a computer or a processor, may cause the computer or the processor to implement the functional modules of the automatic near-infrared face recognition apparatus according to the embodiment of the present invention, and/or may execute the automatic near-infrared face recognition method according to the embodiment of the present invention.

In one embodiment, the program instructions are operable when executed to perform the steps of: step S310: acquiring a face RGB image which is acquired by an RGB camera and contains a face to be recognized; step S320: under the condition that the image quality of the face RGB image does not meet the preset requirement, automatically acquiring a face near-infrared image which is automatically acquired by a near-infrared camera while the RGB camera acquires the face RGB image and contains a face to be recognized; step S330: obtaining a near-infrared image to be recognized containing a face to be recognized at least based on the face near-infrared image; and step S340: and retrieving in the near-infrared base library in the automatic near-infrared face registration method based on the near-infrared image to be recognized or the identification characteristics of the near-infrared image to be recognized so as to determine whether the face to be recognized is a known face.

The modules in the automatic near-infrared face recognition system according to the embodiment of the present invention may be implemented by a processor of an electronic device implementing automatic near-infrared face recognition according to the embodiment of the present invention running computer program instructions stored in a memory, or may be implemented when computer instructions stored in a computer-readable storage medium of a computer program product according to the embodiment of the present invention are run by a computer.

Although the illustrative embodiments have been described herein with reference to the accompanying drawings, it is to be understood that the foregoing illustrative embodiments are merely exemplary and are not intended to limit the scope of the invention thereto. Various changes and modifications may be effected therein by one of ordinary skill in the pertinent art without departing from the scope or spirit of the present invention. All such changes and modifications are intended to be included within the scope of the present invention as set forth in the appended claims.

In the several embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other ways. For example, the above-described device embodiments are merely illustrative, and for example, the division of the units is only one logical functional division, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another device, or some features may be omitted, or not executed.

In the description provided herein, numerous specific details are set forth. It is understood, however, that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure an understanding of this description.

Similarly, it should be appreciated that in the description of exemplary embodiments of the invention, various features of the invention are sometimes grouped together in a single embodiment, figure, or description thereof for the purpose of streamlining the invention and aiding in the understanding of one or more of the various inventive aspects. However, the method of the present invention should not be construed to reflect the intent: that the invention as claimed requires more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive aspects lie in less than all features of a single disclosed embodiment. Thus, the claims following the detailed description are hereby expressly incorporated into this detailed description, with each claim standing on its own as a separate embodiment of this invention.

It will be understood by those skilled in the art that all of the features disclosed in this specification (including any accompanying claims, abstract and drawings), and all of the processes or elements of any method or apparatus so disclosed, may be combined in any combination, except combinations where such features are mutually exclusive. Each feature disclosed in this specification (including any accompanying claims, abstract and drawings) may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise.

Furthermore, those skilled in the art will appreciate that while some embodiments described herein include some features included in other embodiments, rather than other features, combinations of features of different embodiments are meant to be within the scope of the invention and form different embodiments. For example, in the claims, any of the claimed embodiments may be used in any combination.

The various component embodiments of the invention may be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. It will be appreciated by those skilled in the art that a microprocessor or Digital Signal Processor (DSP) may be used in practice to implement some or all of the functions of some of the modules in an automatic near-infrared face registration apparatus or an automatic near-infrared face recognition apparatus according to embodiments of the present invention. The present invention may also be embodied as apparatus programs (e.g., computer programs and computer program products) for performing a portion or all of the methods described herein. Such programs implementing the present invention may be stored on computer-readable media or may be in the form of one or more signals. Such a signal may be downloaded from an internet website or provided on a carrier signal or in any other form.

It should be noted that the above-mentioned embodiments illustrate rather than limit the invention, and that those skilled in the art will be able to design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention may be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by one and the same item of hardware. The usage of the words first, second and third, etcetera do not indicate any ordering. These words may be interpreted as names.

The above description is only for the specific embodiment of the present invention or the description thereof, and the protection scope of the present invention is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present invention, and the changes or substitutions should be covered within the protection scope of the present invention. The protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. An automatic near-infrared face registration method comprises the following steps:

step S210: automatically acquiring a face near-infrared image which is automatically acquired by a near-infrared camera while an RGB camera acquires a face RGB image containing a target face;

step S220: obtaining a target near-infrared image containing the target face at least based on the face near-infrared image;

step S230: acquiring target object information related to the target face in the face RGB image, wherein the target object information is identification information of an object to which the target face belongs;

step S240: obtaining near-infrared registration information corresponding to the target face at least based on the target near-infrared image and the target object information; and

step S250: storing the near-infrared registration information into a near-infrared bottom library;

wherein, before the step S210, the method further comprises:

and when the RGB camera is used for collecting the RGB image of the face, automatically starting a near-infrared light source and the near-infrared camera, wherein the near-infrared light source is used for emitting near-infrared light to irradiate the target face.

2. The method of claim 1, wherein the step S240 comprises:

storing the target near-infrared image and the target object information into a temporary image library; and

when the number of target near-infrared images containing the target face, which are stored in the temporary image library, reaches a preset number, one or more target near-infrared images with the best image quality are selected from the target near-infrared images containing the target face, which are stored in the temporary image library, so as to obtain the near-infrared registration information, wherein the near-infrared registration information comprises the one or more target near-infrared images or identification features of the one or more target near-infrared images and the target object information.

3. The method of claim 1, wherein the step S240 comprises:

determining that the near-infrared registration information includes the target near-infrared image or the identification feature of the target near-infrared image, and the target object information.

4. The method of claim 1, wherein, prior to the step S230, the method further comprises:

acquiring the face RGB image;

obtaining a target RGB image containing the target face based on the face RGB image; and

and searching in an RGB base library based on the target RGB image to determine whether a base library RGB image matched with the target RGB image exists in the RGB base library or searching in the RGB base library based on the identification feature of the target RGB image to determine whether the identification feature of the base library RGB image matched with the identification feature of the target RGB image exists in the RGB base library.

5. The method of claim 4, wherein the step S230 comprises:

and determining the base library object information which is stored in the RGB base library and is related to the retrieved base library RGB image or the identification characteristic of the retrieved base library RGB image as the target object information.

6. The method of claim 4, wherein the step S230 comprises:

under the condition that a bottom library RGB image matched with the target RGB image is not retrieved or the identification feature of the bottom library RGB image matched with the identification feature of the target RGB image is not retrieved, outputting prompt information for prompting that the target face is an unknown face; and

input information of an operator is received and the target object information is determined based on the input information.

7. The method of claim 6, wherein after the step S230, the method further comprises:

storing the target RGB image or the identification features of the target RGB image and the target object information in an RGB base.

8. The method of claim 1, wherein the step S230 comprises:

9. The method of claim 8, wherein the method further comprises:

acquiring the face RGB image;

10. The method of claim 4 or 9, wherein the obtaining a target RGB image containing the target face based on the face RGB image comprises:

and carrying out face detection on the face RGB image to obtain an image only containing the target face as the target RGB image.

11. The method of claim 1, wherein the step S220 comprises:

carrying out face detection on the face near-infrared image to determine all faces contained in the face near-infrared image;

determining the position of the target face in the face near-infrared image according to the pixel corresponding relation between the face RGB image and the face near-infrared image and the position of the target face in the face RGB image, wherein the pixel corresponding relation is determined based on the physical position relation between the RGB camera and the near-infrared camera; and

and extracting an image area where the target face is located according to the position of the target face in the face near-infrared image and a face detection result of the face near-infrared image so as to obtain an image only containing the target face as the target near-infrared image.

12. The method of claim 1, wherein, prior to the step S210, the method further comprises:

acquiring an initial RGB image which is acquired by the RGB camera and contains the target face; and

and judging whether the image quality of the initial RGB image meets a preset requirement, and if so, determining that the initial RGB image is the face RGB image.

13. An automatic near-infrared face recognition method comprises the following steps:

step S310: acquiring a face RGB image which is acquired by an RGB camera and contains a face to be recognized;

step S320: under the condition that the image quality of the face RGB image does not meet the preset requirement, automatically acquiring a face near-infrared image which is automatically acquired by a near-infrared camera while the RGB camera acquires the face RGB image and contains the face to be recognized;

step S330: obtaining a near-infrared image to be recognized containing the face to be recognized at least based on the face near-infrared image; and

step S340: retrieving in the near-infrared base of any of claims 1 to 12 based on the near-infrared image to be recognized or the identification features of the near-infrared image to be recognized, to determine whether the face to be recognized is a known face.

14. The method of claim 13, wherein after step S310, the method further comprises:

under the condition that the image quality of the human face RGB image meets the preset requirement,

obtaining an RGB image to be recognized containing the face to be recognized based on the face RGB image; and

and retrieving in an RGB base library based on the RGB image to be recognized or the identification characteristics of the RGB image to be recognized so as to determine whether the face to be recognized is a known face.

15. The method according to claim 13, wherein the step S320 is performed only when the number of bottom library near-infrared images stored in the near-infrared bottom library or the number of bottom library near-infrared images to which the identification features stored in the near-infrared bottom library belong reaches a first number threshold and/or the number of objects corresponding to bottom library near-infrared images stored in the near-infrared bottom library or the number of objects corresponding to the identification features of bottom library near-infrared images stored in the near-infrared bottom library reaches a second number threshold.

16. An automatic near-infrared face registration apparatus, comprising:

the face near-infrared image acquisition module is used for automatically acquiring a face near-infrared image which is automatically acquired by a near-infrared camera when an RGB camera acquires a face RGB image containing a target face and contains the target face;

the target near-infrared image acquisition module is used for acquiring a target near-infrared image containing the target face at least based on the face near-infrared image;

an object information obtaining module, configured to obtain target object information related to the target face in the target RGB image, where the target object information is identification information of an object to which the target face belongs;

a near-infrared registration information obtaining module, configured to obtain near-infrared registration information corresponding to the target face based on at least the target near-infrared image and the target object information; and

the near-infrared registration information storage module is used for storing the near-infrared registration information into a near-infrared bottom library;

wherein the apparatus further comprises:

the starting module is used for automatically starting the near-infrared light source and the near-infrared camera when the RGB camera is used for collecting the face RGB image before the face near-infrared image containing the target face is automatically collected by the face near-infrared image collecting module when the RGB camera collects the face RGB image containing the target face, and the near-infrared light source is used for emitting near-infrared light to irradiate the target face.

17. An automatic near-infrared face recognition device, comprising:

the face RGB image acquisition module is used for acquiring a face RGB image which is acquired by the RGB camera and contains a face to be recognized;

the face near-infrared image acquisition module is used for automatically acquiring a face near-infrared image which is automatically acquired by a near-infrared camera while the RGB camera acquires the face RGB image and contains the face to be recognized under the condition that the image quality of the face RGB image does not meet the preset requirement;

the to-be-recognized near-infrared image acquisition module is used for acquiring a to-be-recognized near-infrared image containing the to-be-recognized face at least based on the face near-infrared image; and

a near-infrared retrieval module, configured to perform a retrieval in the near-infrared base of any one of claims 1 to 12 based on the near-infrared image to be recognized or an identification feature of the near-infrared image to be recognized, so as to determine whether the face to be recognized is a known face.

18. An automatic near-infrared face registration system comprising an image acquisition device, a near-infrared light source, a processor and a memory, wherein the image acquisition device comprises an RGB camera and a near-infrared camera, the near-infrared light source is configured to emit near-infrared light for illuminating a target face, the memory has stored therein computer program instructions, which when executed by the processor are configured to perform the steps of:

step S210: automatically acquiring a face near-infrared image which is automatically acquired by the near-infrared camera and contains the target face when the RGB camera acquires a face RGB image containing the target face;

step S230: acquiring target object information related to the target face in the target RGB image, wherein the target object information is identification information of an object to which the target face belongs;

wherein, prior to step S210 for which the computer program instructions are executed by the processor, the computer program instructions are further for executing, by the processor, the steps of:

19. An automatic near-infrared face recognition system comprising an image acquisition device, a near-infrared light source, a processor and a memory, wherein the image acquisition device comprises an RGB camera and a near-infrared camera, the near-infrared light source is configured to emit near-infrared light for illuminating a face to be recognized, the memory has stored therein computer program instructions that, when executed by the processor, are configured to perform the steps of:

20. A storage medium having stored thereon program instructions which when executed are for performing the steps of:

wherein, before step S210 in which the program instructions are executed, the program instructions are further operable to perform, when executed, the steps of:

21. A storage medium having stored thereon program instructions which when executed are for performing the steps of: