WO2023140099A1

WO2023140099A1 - Biometric authentication system and biometric authentication method

Info

Publication number: WO2023140099A1
Application number: PCT/JP2022/048688
Authority: WO
Inventors: 毅藤山; 剛中村
Original assignee: パナソニックＩｐマネジメント株式会社
Priority date: 2022-01-19
Filing date: 2022-12-28
Publication date: 2023-07-27
Also published as: JP2023105624A

Abstract

This biometric authentication system is configured to include a terminal device, and a server capable of communicating with the terminal device, wherein: the terminal device outputs an imaging instruction to a camera that images a person, acquires a captured image in which the person is imaged, from the camera, extracts a facial region of the person shown in the captured image, generates an imaging instruction on the basis of the size of the facial region, outputs the generated imaging instruction to the camera, extracts a lip print of the person from the captured image, and sends the lip print to the server; and the server identifies the person on the basis of the extracted lip print.

Description

Biometric authentication system and biometric authentication method

The present disclosure relates to a biometric authentication system and a biometric authentication method.

Patent Document 1 discloses a face authentication system that performs image processing on a face image of a visitor's face, extracts features, searches registered face images based on the features, and performs personal authentication. The face authentication system normalizes the size of a face image to detect contours, detects general features consisting of closed regions with contours closed, and unique features consisting of line segments and isolated points where the contours are open ends, and recognizes a visitor by pattern-matching a feature-processed face image file registered by overwriting the general features and specific features on a registrant's face image and a feature-processed image file with the general features and specific features. Here, general features refer to contours of the face, hair, eyes, eyebrows, nostrils, and mouth. Unique features refer to wrinkles, blemishes, moles, etc. caused by unevenness of the face.

Japanese Patent Application Laid-Open No. 2005-242432

However, in Patent Document 1, since the size of the face image is normalized, there is a possibility that the image quality will deteriorate. Moreover, when detecting general features and specific features from a face image whose image quality has been degraded by normalization, the face authentication system may not be able to extract the visitor's lip print from the face image.

The present invention has been made in view of the conventional situation described above, and aims to provide a biometric authentication system and a biometric authentication method that more efficiently acquire a captured image from which a lip print used for biometric authentication can be acquired.

The present disclosure is a biometric authentication system including a terminal device and a server capable of communicating with the terminal device, wherein the terminal device outputs an image capturing command to a camera that captures an image of a person, acquires a captured image of the person captured by the camera, extracts the face area of the person reflected in the captured image, generates the image capturing command based on the size of the face area, outputs the generated image capturing command to the camera, extracts the lip print of the person from the captured image, and extracts the lip print of the person from the captured image. and the server provides a biometric authentication system that identifies the person based on the extracted lip print.

Also, the present disclosure is a biometric authentication method performed by one or more computers, which outputs an imaging command to a camera that captures an image of a person, acquires a captured image of the person captured by the camera, extracts a face region of the person in the captured image, generates the imaging command based on the size of the face region, outputs the generated imaging command to the camera, extracts the lip print of the person from the captured image, and identifies the living person based on the extracted lip print. Provide an authentication method.

According to the present invention, it is possible to more efficiently acquire captured images from which lip prints used for biometric authentication can be acquired.

1 is a diagram showing an internal configuration example of a biometric authentication system according to Embodiment 1; FIG. Flowchart showing an example of the operation procedure of the biometric authentication system according to the first embodiment Diagram for explaining an example of image processing FIG. 2 shows an example of the overall configuration of a biometric authentication system according to a modification of Embodiment 1; A table showing the correspondence relationship between the operation procedure example of each embodiment shown in FIGS. 2 and 7 and the action subject of the operation procedure FIG. 2 shows an example of the overall configuration of a biometric authentication system according to Embodiment 2; Flowchart showing an example of the operation procedure of the biometric authentication system according to the second embodiment

Hereinafter, each embodiment specifically disclosing the biometric authentication system and biometric authentication method according to the present disclosure will be described in detail with reference to the drawings as appropriate. However, more detailed description than necessary may be omitted. For example, detailed descriptions of well-known matters and redundant descriptions of substantially the same configurations may be omitted. This is to avoid unnecessary verbosity in the following description and to facilitate understanding by those skilled in the art. It should be noted that the accompanying drawings and the following description are provided for a thorough understanding of the present disclosure by those skilled in the art and are not intended to limit the claimed subject matter.

(Embodiment 1)
First, referring to FIG. 1, the internal configuration of biometric authentication system 100 according to Embodiment 1 will be described. FIG. 1 is a diagram showing an internal configuration example of a biometric authentication system 100 according to Embodiment 1. As shown in FIG.

The biometric authentication system 100 includes a camera C1, a terminal device P1, and a server S1. The biometric authentication system 100 captures an image of a person to be authenticated using a camera C1. The biometric authentication system 100 uses the terminal device P1 to estimate the distance between the camera C1 and the person appearing in the captured image, controls the zoom magnification of the camera C1 based on the estimated distance, and captures a captured image used for biometric authentication (that is, a captured image from which the lip print of the person to be authenticated can be obtained).

The biometric authentication system 100 uses the terminal device P1 to generate an image (hereinafter referred to as a "lip image") that extracts (cuts out) a lip region in which a person's lips appear from the captured image, and acquires a lip print based on the generated lip image. The biometric authentication system 100 identifies a person to be authenticated and determines whether or not the person to be authenticated is a person registered in the database DB by collating the lip print acquired by the terminal device P1 with the lip print of at least one person registered in the database DB by the server S1.

The camera C1 is connected to the terminal device P1 so that data can be transmitted and received. The camera C1 performs zoom magnification adjustment processing and imaging processing based on the control command transmitted from the terminal device P1. The camera C1 captures an image of a person to be authenticated, and transmits the captured image to the terminal device P1.

The terminal device P1 is implemented using, for example, a PC (Personal Computer), a notebook PC, a tablet terminal, a smartphone, or the like. The terminal device P1 acquires the captured image transmitted from the camera C1, and extracts a face region in which a person's face is captured from the acquired captured image. The terminal device P1 determines whether or not the captured image captured by the camera C1 is a captured image from which a lip print used for biometric authentication can be obtained, based on the size (area) of the face region for the entire captured image.

When the terminal device P1 determines that the captured image captured by the camera C1 is a captured image from which a lip print used for biometric authentication can be obtained, the terminal device P1 generates a lip image by extracting (cutting out) the lip region from the captured image. The terminal device P1 extracts a lip print from the generated lip image, generates lip print data, and transmits the generated lip print data to the server S1. On the other hand, when the terminal device P1 determines that the captured image captured by the camera C1 is not a captured image from which a lip print used for biometric authentication can be obtained, the terminal device P1 outputs to the display unit 16 an image capture support screen (not shown) that requests the person to approach or leave the camera C1, or executes control to adjust the zoom magnification of the camera C1 to recapture the person.

The terminal device P1 includes a communication unit 10, a processor 11, a memory 15, and a display unit 16.

The communication unit 10 is connected to the camera C1 so that data can be transmitted and received, and is connected to the communication unit 20 of the server S1 through the network NW so as to be able to communicate wirelessly or by wire. The wireless communication here is communication via a wireless LAN (Local Area Network) such as Wi-Fi (registered trademark).

The communication unit 10 outputs the captured image transmitted from the camera C1 to the processor 11, and outputs the matching result transmitted from the server S1 to the processor 11. The communication unit 10 also transmits various control commands output from the processor 11 (for example, a control command for adjusting the zoom magnification, a control command for executing imaging processing, etc.) to the camera C1, and transmits lip print data output from the processor 11 to the server S1.

The processor 11 is configured using, for example, a CPU (Central Processing Unit), a DSP (Digital Signal Processor) or an FPGA (Field Programmable Gate Array), and controls the operation of each part. The processor 11 cooperates with the memory 15 to collectively perform various processes and controls. Specifically, the processor 11 refers to the programs and data held in the memory 15 and executes the programs, thereby implementing the functions of the imaging control unit 12, the lip print extraction unit 14, and the like.

The imaging control unit 12 executes imaging control of the camera C1. The imaging control unit 12 generates a control command for changing the zoom magnification of the camera C1, and transmits the generated control command to the camera C1 to execute processing for adjusting the zoom magnification of the camera C1. Also, the imaging control unit 12 is configured to be able to implement the function of the distance measurement unit 13 .

The distance measuring unit 13 performs image analysis on the captured image transmitted from the camera C1 to detect a person's face area. The distance measurement unit 13 estimates the distance between the camera C1 and the captured person based on the size (area, number of pixels) of the face region of the person with respect to the entire area (total number of pixels) of the captured image, and outputs information about the estimated distance to the imaging control unit 12.

The imaging control unit 12 determines whether or not the distance output from the distance measuring unit 13 is equal to or less than a predetermined distance. The predetermined distance referred to here is a distance determined based on the resolution of an image sensor (not shown) provided in the camera C1, and may be set to a distance at which the lip print of the captured person can be obtained.

When the imaging control unit 12 determines that the distance estimated by the distance measuring unit 13 is equal to or less than the predetermined distance, the imaging control unit 12 outputs the captured image transmitted from the camera C1 to the lip print extraction unit 14 . On the other hand, when the imaging control unit 12 determines that the estimated distance is not equal to or less than the predetermined distance, it determines the zoom magnification of the camera C1 based on the difference between the estimated distance and the predetermined distance. The imaging control part 12 changes the currently set zoom magnification to the determined new zoom magnification, generates a control command for imaging the person again, outputs it to the communication part 10, and transmits it to the camera C1.

The lip print extraction unit 14 detects a person's face from the captured image captured by the camera C1, and extracts a face area containing the detected face. Based on the information of the extracted face area, the lip print extraction unit 14 detects each part (part) constituting the face such as the eyes, nose, and mouth (lips) of the person to be authenticated from the face area, cuts (extracts) the lip area (that is, the area where the mouth is shown) from the captured image, and generates a lip image.

The lip print extraction unit 14 corrects the contrast of the lip image in order to make it easier to detect lip wrinkles (edges) from the generated lip image. Here, the lip pattern extracting unit 14 corrects the contrast of the lip image so that the contrast ratio or the contrast difference between the contrast of the lip wrinkle portion and the contrast of the lip portion other than the lip wrinkle is equal to or greater than a predetermined value.

The lip print extraction unit 14 extracts lip creases (edges) from the contrast-corrected lip image. Further, the lip pattern extraction unit 14 performs binarization processing on the contrast-corrected lip image based on the extracted lip wrinkle (edge) information. As a result, the lip print extraction unit 14 can generate a lip image from which a lip print based on lip wrinkles can be more easily obtained from the lip image. A lip print extraction unit 14 extracts a lip print from the lip image after the binarization process. The lip print extraction unit 14 outputs the extracted lip print data to the communication unit 10 and causes it to be transmitted to the server S1.

The memory 15 has, for example, a RAM (Random Access Memory) as a work memory used when executing each process of the processor 11, and a ROM (Read Only Memory) for storing programs and data that define the operation of the processor 11. The RAM temporarily stores data or information generated or acquired by the processor 11 . A program that defines the operation of the processor 11 is written in the ROM.

The display unit 16 is configured using a display such as an LCD (Liquid Crystal Display) or an organic EL (Electroluminescence). The display unit 16 displays an imaging support screen (not shown) output from the processor 11, a screen (not shown) for notifying the authentication result, and the like.

The server S1 is realized by a general server or a cloud server. A server S1 acquires lip print data transmitted from a terminal device P1, compares the acquired lip print feature quantity with the lip print feature quantity of at least one person registered in a database DB, and calculates a collation score. The server S1 identifies a person to be authenticated based on the calculated matching score, generates an authentication result including information about the identified person, information indicating authentication success or authentication failure, etc., and transmits the authentication result to the terminal device P1.

The server S1 includes a communication unit 20, a processor 21, a memory 25, and a database DB. Note that the server S1 shown in FIG. 1 shows an example including a database DB, but is not limited to this. For example, the database DB may be configured separately from the server S1 and implemented by an external device communicably connected to the server S1. Also, there may be a plurality of databases DB.

The communication unit 20 is connected to the communication unit 10 of the terminal device P1 via the network NW so as to be capable of wireless communication or wired communication, and transmits and receives data. The wireless communication here is communication via a wireless LAN such as Wi-Fi (registered trademark).

The communication unit 20 outputs to the processor 21 the lip print data transmitted from the communication unit 10 of the terminal device P1. In addition, the communication unit 20 transmits the authentication result information output from the processor 21 to the communication unit 10 of the terminal device P1.

The processor 21 is configured using, for example, a CPU or FPGA, and cooperates with the memory 25 to perform various types of processing and control. Specifically, the processor 21 refers to the program held in the memory 25 and implements the functions of each unit such as the matching unit 22 and the score determination unit 24 by executing the program.

The matching unit 22 extracts a feature amount indicating the individuality of the person to be authenticated from the lip print data transmitted from the terminal device P1, compares the extracted feature amount of the lip print with the feature amount of the lip print of at least one person registered in the database DB, and calculates a matching score indicating the degree of matching between the extracted feature amount and the feature amount of the lip print registered in the database DB. The collation unit 22 is configured to be able to implement the function of the feature amount extraction unit 23 .

The matching score here is a score that is calculated higher as the distance between the feature amount of the lip print of the person to be authenticated and the feature amount of the lip print of the person registered in the database DB is smaller (that is, the closer they are to each other). Also, the distance here may be the Euclidean distance. The collation score is calculated such that the smaller the Euclidean distance between the acquired lip print feature amount and the lip print feature amount registered in the database DB, the higher the score calculated, and the larger the Euclidean distance, the lower the score calculated.

Also, the matching score may be a score indicating the degree of similarity between the feature amount of the lip print of the person to be authenticated and the feature amount of the lip print of the person registered in the database DB. In such a case, if the matching score is a score indicating similarity, the higher the similarity, the higher the matching score, and the lower the similarity, the lower the matching score.

The feature quantity extraction unit 23 extracts the feature quantity of the lip print of the person to be authenticated from the lip print data transmitted from the terminal device P1. The feature amount extraction unit 23 outputs the extracted feature amount of the lip print to the matching unit 22 .

The collation unit 22 collates the feature amount of the lip print output from the feature amount extraction unit 23 with the feature amount of the lip print registered in the database DB, and calculates a collation score. The matching unit 22 selects the highest matching score among the calculated matching scores.

The matching unit 22 extracts the personal information of the person corresponding to the selected matching score from the database DB, associates the selected matching score with the extracted personal information, and outputs them to the score determination unit 24 .

Based on the matching score output from the matching unit 22, the score determining unit 24 determines whether the person to be authenticated and the person corresponding to this matching score are the same person. When the score determination unit 24 determines that the matching score output from the matching unit 22 is equal to or less than a predetermined matching score, the score determining unit 24 may determine that the person to be authenticated is not registered in the database DB (that is, authentication has failed). The score determination unit 24 generates an authentication result based on the determination result of determining whether or not the person to be authenticated and the person corresponding to the matching score are the same person, and transmits the authentication result to the terminal device P1 via the communication unit 20. It should be noted that the authentication result is generated including the specified person information, information indicating authentication success or authentication failure, and the like.

For example, the score determination unit 24 determines whether the matching score is 80 or less when the matching score is calculated as a value of "0 (zero) to 100". When the score determination unit 24 determines that the matching score is not equal to or less than 80, it determines that the person to be authenticated and the person corresponding to this matching score are the same person (that is, authentication is successful).

On the other hand, when the score determination unit 24 determines that the matching score is 80 or less, it determines that the person to be authenticated and the person corresponding to this matching score are not the same person (that is, authentication failure). As a result, the score determination unit 24 can more effectively suppress authentication errors that can occur when the matching score is low (that is, the similarity between the feature amount of the lip print of the person to be authenticated and the feature amount of the lip print registered in the database DB is low).

The memory 25 has, for example, a RAM as a work memory that is used when executing the processing of the processor 21, and a ROM that stores a program that defines the processing of the processor 21. Data generated or acquired by the processor 21 is temporarily stored in the RAM. A program that defines the processing of the processor 21 is written in the ROM.

The database DB is a so-called storage, and is configured using a storage medium such as flash memory, HDD (Hard Disk Drive), or SSD (Solid State Drive). The database DB stores (registers) personal information (for example, name, age, gender, etc.) for each person in association with a lip print or a feature amount of the lip print.

It should be noted that the database DB may store (register) each of a plurality of lip prints with different mouth openings for each person. For example, the lip prints registered in the database DB may include, for each person, a lip print with the mouth closed, a lip print while uttering the vowel "a", and a lip print while uttering the vowel "i". In such a case, the database DB stores (registers) the person information, the lip print, the information on how to open the mouth when the lip print is acquired, or the information on the uttered vowels when the lip print is acquired, etc., in association with each other.

The network NW connects the terminal device P1 and the server S1 for wired communication or wireless communication, and enables data transmission/reception.

Although the example shown in FIG. 1 shows an example in which the camera C1 and the terminal device P1 are configured as separate devices, the camera C1 may be configured integrally with the terminal device P1 and configured to be able to implement various functions of the terminal device P1.

Next, the operation procedure of the biometric authentication system 100 according to Embodiment 1 will be described with reference to FIGS. 2 and 3. FIG. FIG. 2 is a flow chart showing an example of the operation procedure of the biometric authentication system 100 according to the first embodiment. FIG. 3 is a diagram illustrating an example of image processing. FIG. 5 is a table showing the correspondence relationship between the operation procedure example of each embodiment shown in FIGS. 2 and 7 and the action subject of the operation procedure. In the biometric authentication system 100 according to the first embodiment, as shown in FIG. 5, the terminal device P1 executes steps St11 to St17, and the server S1 executes steps St18 to St19.

The terminal device P1 acquires the captured image Pt11 including the face of the person to be authenticated transmitted from the camera C1 (St11). The terminal device P1 detects a face area Ar11 including a person's face from the acquired captured image Pt11, and estimates the distance between the subject (that is, the person to be authenticated) and the camera C1 based on the size (area) of the face area Ar11 with respect to the entire captured image (St12).

The terminal device P1 determines the zoom magnification of the camera C1 based on the estimated distance and the predetermined distance. The terminal device P1 changes the currently set zoom magnification to the determined new zoom magnification, and generates a control command for re-imaging the person. The terminal device P1 transmits the generated control command to the camera C1 and controls the zoom magnification of the camera C1 (St13).

After adjusting the zoom magnification based on the control command transmitted from the terminal device P1, the camera C1 executes the imaging process again and transmits the captured image to the terminal device P1.

The terminal device P1 acquires again the captured image Pt12 including the person's face transmitted from the camera C1 (St14). The terminal device P1 detects a face area Ar12 including a person's face from the acquired captured image Pt12, and re-estimates the distance between the subject (person to be authenticated) and the camera C1 based on the size (area) of the face area Ar12 with respect to the entire captured image.

Here, if the terminal device P1 determines that the re-estimated distance is a distance at which the person's lip print can be obtained from the captured image, the process proceeds to step St15. On the other hand, when the terminal device P1 determines that the re-estimated distance is not the distance at which the person's lip print can be obtained from the captured image, the terminal device P1 proceeds to the processing of step St13, and generates a control command to adjust the zoom magnification of the camera C1 again.

The terminal device P1 extracts the face area Ar13 of the person shown in the captured image Pt12 (St15), and further extracts the lip area Pt14 showing the lips (mouth) of the person from the extracted face area Ar13 (St16). The terminal device P1 generates a lip image obtained by cutting out the extracted lip region Pt14, and corrects the contrast so that the lip print can be detected from the wrinkles of the lips reflected in the generated lip image (St17). The terminal device P1 extracts lip creases (edges) from the contrast-corrected lip image, and performs binarization processing on the lip image (St17). The terminal device P1 extracts the lip print of the person to be authenticated from the binarized lip image Pt15, generates lip print data, and transmits the generated lip print data to the server S1.

Based on the lip print data transmitted from the terminal device P1, the server S1 extracts a feature quantity indicating the individuality of the person. The server S1 collates the extracted feature amount based on the lip print of the person to be authenticated with the feature amount based on the lip print of at least one person registered in the database DB (St18).

The server S1 calculates a matching score between the extracted lip print feature amount and the lip print feature amount registered in the database DB. The server S1 selects the highest matching score among the calculated matching scores, and determines (identifies) that the person corresponding to the selected matching score is the same person as the person to be authenticated (St19). The server S1 generates an authentication result including personal information corresponding to the specified person, and transmits it to the terminal device P1. The terminal device P1 generates a screen (not shown) including the authentication result transmitted from the server S1, and outputs and displays it on the display unit 16. FIG.

As described above, the biometric authentication system 100 according to Embodiment 1 can more efficiently acquire a captured image from which a lip print used for biometric authentication can be acquired. In addition, the biometric authentication system 100 can further improve authentication accuracy by executing biometric authentication using this captured image.

(Modification of Embodiment 1)
The biometric authentication system 100 according to the first embodiment described above estimates the distance between the camera C1 and the person to be authenticated based on the size of the person's face in the captured image captured by the camera C1. An example in which the biometric authentication system 100A according to the modification of the first embodiment measures the distance between the camera C1 and the person to be authenticated using the ranging sensor SS1 will be described.

In addition, in the description of the biometric authentication system 100A according to the modified example of the first embodiment, the same reference numerals are given to the same configurations as the biometric authentication system 100 according to the first embodiment, and the description thereof will be omitted.

First, referring to FIG. 4, the overall configuration of a biometric authentication system 100A according to the modification of Embodiment 1 will be described. FIG. 4 is a diagram showing an overall configuration example of a biometric authentication system 100A according to a modification of the first embodiment.

The biometric authentication system 100A includes a camera C1, a ranging sensor SS1, a terminal device P1A, and a server S1. The biometric authentication system 100A measures the distance between the person to be authenticated by the ranging sensor SS1 and the camera C1, and when the terminal device P1A determines that the measured distance is a distance at which a captured image from which a lip print can be obtained can be captured, the camera C1 captures an image of the person.

The biometric authentication system 100A uses the terminal device P1A to generate a lip image by extracting (cutting out) a lip region in which a person's lips appear from the captured image, and acquires a lip print based on the generated lip image. The biometric authentication system 100A identifies a person to be authenticated and determines whether or not the person to be authenticated is a person registered in the database DB by collating the lip print acquired by the terminal device P1A with the lip print of at least one person registered in the database DB by the server S1.

The camera C1 is connected to the terminal device P1A so that data can be transmitted and received. The camera C1 is controlled by the terminal device P1A to capture an image of a person to be authenticated, and transmits the captured image to the terminal device P1A.

The ranging sensor SS1 is realized by, for example, a TOF (Time Of Flight) sensor or the like. The ranging sensor SS1 is connected to the terminal device P1A so as to be able to transmit and receive data, measures the distance between the camera C1 and the person to be authenticated, and transmits the measured distance information to the terminal device P1A.

The terminal device P1A is implemented using, for example, a PC, notebook PC, tablet terminal, smartphone, or the like. The terminal device P1A is connected to each of the distance measuring sensor SS1, the camera C1, and the server S1 so that data can be transmitted and received.

A terminal device P1A in the modification of Embodiment 1 includes a communication unit 10A, a processor 11A, a memory 15, and a display unit 16.

The communication unit 10A is connected to enable data transmission/reception between the distance measuring sensor SS1 and the camera C1, respectively, and is connected to the communication unit 20 of the server S1 through the network NW to enable wireless or wired communication.

The communication unit 10A outputs distance information transmitted from the ranging sensor SS1 to the processor 11A, outputs captured images transmitted from the camera C1 to the processor 11A, and outputs matching results transmitted from the server S1 to the processor 11A. Further, the communication unit 10A transmits various control commands output from the processor 11A (for example, a control command for adjusting the zoom magnification, a control command for executing imaging processing, etc.) to the camera C1, and transmits lip print data output from the processor 11A to the server S1.

The processor 11A is configured using, for example, a CPU, DSP or FPGA, and controls the operation of each unit. The processor 11A cooperates with the memory 15 to collectively perform various processes and controls. Specifically, the processor 11A refers to the programs and data held in the memory 15 and executes the programs, thereby implementing the functions of the imaging control section 12A, the lip print extraction section 14, and the like.

The imaging control unit 12A determines whether or not the distance between the camera C1 and the person to be authenticated is equal to or less than a predetermined distance based on the distance information transmitted from the ranging sensor SS1. When the image capturing control unit 12A determines that the distance is equal to or less than the predetermined distance, the image capturing control unit 12A generates a control command for capturing an image of the person to be authenticated, and transmits the control command to the camera C1.

The imaging control unit 12A detects a person's face from the captured image transmitted from the camera C1, and extracts a face area containing the detected face. Based on the information of the extracted face area, the lip print extraction unit 14 detects each part (part) constituting the face such as the eyes, nose, and mouth (lips) of the person to be authenticated from the face area, cuts (extracts) the lip area (that is, the area where the mouth is shown) from the captured image, and generates a lip image.

Although the example shown in FIG. 4 shows an example in which the camera C1 and the terminal device P1A are configured as separate devices, the camera C1 may be configured integrally with the terminal device P1A and configured to be able to implement various functions of the terminal device P1A.

Here, in the biometric authentication system 100A according to the modification of the first embodiment, in the operation procedure shown in FIG. 2, the distance measuring sensor SS1 performs the process of step St12, the terminal device P1A performs the process of steps St13 to St17, and the server S1 performs the processes of steps St18 to St19 (see FIG. 5). Note that the process of step St11 is omitted in the modification of the first embodiment.

As described above, the biometric authentication system 100A according to the modification of the first embodiment can more efficiently acquire a captured image from which a lip print used for biometric authentication can be acquired. In addition, the biometric authentication system 100A can further improve authentication accuracy by executing biometric authentication using this captured image.

(Embodiment 2)
The biometric authentication system 100 according to the first embodiment described above estimates the distance between the camera C1 and the person to be authenticated by the terminal device P1, and executes zoom magnification and imaging control of the camera C1. An example will be described in which the biometric authentication system 200 according to the second embodiment estimates the distance between the camera C1B and the person to be authenticated using the ranging sensor SS2, and executes zoom magnification and imaging control of the camera C1B.

In addition, in the description of the biometric authentication system 200 according to the second embodiment, the same reference numerals are given to the same configurations as the biometric authentication system 100 according to the first embodiment or the biometric authentication system 100A according to the modification of the first embodiment, and the description thereof will be omitted.

The internal configuration of the biometric authentication system 200 according to Embodiment 2 will be described with reference to FIG. FIG. 6 is a block diagram showing an internal configuration example of the biometric authentication system 200 according to the second embodiment.

The biometric authentication system 200 according to Embodiment 2 includes a camera C1B, a ranging sensor SS2, a terminal device P1B, and a server S1. The biometric authentication system 200 measures the distance between the camera C1B and the person to be authenticated by the distance measuring sensor SS2, and if it is determined that the measured distance is the distance at which the captured image from which the lip print can be obtained can be captured, the camera C1B captures the person.

The biometric authentication system 200 uses the terminal device P1B to generate a lip image by extracting (cutting out) a lip region in which a person's lips appear from the captured image, and acquires a lip print based on the generated lip image. The biometric authentication system 200 identifies a person to be authenticated and determines whether or not the person to be authenticated is a person registered in the database DB by collating the lip print acquired by the terminal device P1B with the lip print of at least one person registered in the database DB by the server S1.

The camera C1B is connected so as to be able to transmit and receive data between the ranging sensor SS2 and the terminal device P1B. The camera C1B is controlled by the ranging sensor SS2 to capture an image of the person to be authenticated, and transmits the captured image to the terminal device P1B.

The distance measuring sensor SS2 is implemented by, for example, a computer, device, or the like including a distance measuring unit 30 such as a TOF sensor. The ranging sensor SS2 is connected to the camera C1B so as to be able to transmit and receive data. The ranging sensor SS2 includes a ranging section 30, a communication section 31, a processor 32, and a memory 33. FIG. Note that the distance measuring unit 30 is not essential and may be omitted. In such a case, the distance measuring sensor SS2 estimates the distance between the camera C1B and the person to be authenticated based on the size (area) of the person's face region in the captured image captured by the camera C1B.

The distance measurement unit 30 measures the distance between the camera C1B and the person to be authenticated, and outputs the measured distance information to the processor 32.

The communication unit 31 is connected to the camera C1B for wireless communication or wired communication, and executes data transmission/reception. The wireless communication here is communication via a wireless LAN such as Wi-Fi (registered trademark). The communication unit 31 transmits various control commands output from the processor 32 (for example, a control command for adjusting the zoom magnification, a control command for executing an imaging process, etc.) to the camera C1B, outputs the captured image transmitted from the camera C1B to the processor 32, and transmits the captured image to the terminal device P1B.

The processor 32 is configured using, for example, a CPU or FPGA, and cooperates with the memory 33 to perform various types of processing and control. Specifically, the processor 32 refers to the program held in the memory 33 and implements the function of each part by executing the program.

Based on the distance information output from the distance measuring unit 30, the processor 32 determines whether the distance between the camera C1B and the person to be authenticated is equal to or less than a predetermined distance. When the processor 32 determines that the distance between the camera C1B and the person to be authenticated is equal to or less than a predetermined distance, the processor 32 generates a control command for imaging the person to be authenticated, outputs it to the communication unit 31, and transmits it to the camera C1B.

On the other hand, when the processor 32 determines that the distance between the camera C1B and the person to be authenticated is not equal to or less than the predetermined distance, it determines the zoom magnification of the camera C1B based on the difference between the measured distance and the predetermined distance. The processor 32 changes the currently set zoom magnification to the determined new zoom magnification, generates a control command to image the person again, outputs it to the communication unit 31, and transmits it to the camera C1B.

In addition, when the distance measurement unit 30 is omitted, the processor 32 acquires the captured image transmitted from the camera C1B and detects the face area of the person who is the authentication target in the acquired captured image. The processor 32 estimates the distance between the camera C1B and the person to be authenticated based on the size (area, number of pixels) of the face area of the person with respect to the entire area (total number of pixels) of the captured image, and determines whether the estimated distance is equal to or less than a predetermined distance.

When the processor 32 determines that the estimated distance is equal to or less than the predetermined distance, the processor 32 transmits the captured image transmitted from the camera C1B to the terminal device P1B. On the other hand, when the processor 32 determines that the estimated distance is not equal to or less than the predetermined distance, the processor 32 determines the zoom magnification of the camera C1B based on the difference between the estimated distance and the predetermined distance. The processor 32 changes the zoom magnification currently set for the camera C1B to the determined new zoom magnification, and generates a control command to image the person again. The processor 32 outputs the generated control command to the communication unit 31 and causes it to be transmitted to the camera C1B.

The memory 33 has, for example, a RAM as a work memory that is used when executing each process of the processor 32, and a ROM that stores programs and data that define the operation of the processor 32. Data or information generated or obtained by the processor 32 is temporarily stored in the RAM. A program that defines the operation of the processor 32 is written in the ROM.

A terminal device P1B according to Embodiment 2 includes a communication unit 10B, a processor 11B, a memory 15, and a display unit 16.

The communication unit 10B is connected to enable data transmission/reception between the distance measuring sensor SS2 and the camera C1B, and is connected to the communication unit 20 of the server S1 via the network NW to enable wireless or wired communication.

The communication unit 10B outputs the captured image transmitted from the camera C1B or the ranging sensor SS2 to the processor 11B, and outputs the matching result transmitted from the server S1 to the processor 11B. Further, the communication unit 10B transmits various control commands output from the processor 11B (for example, a control command for adjusting the zoom magnification, a control command for executing imaging processing, etc.) to the camera C1B, and transmits lip print data output from the processor 11B to the server S1.

The processor 11B is configured using, for example, a CPU, DSP or FPGA, and controls the operation of each unit. The processor 11B cooperates with the memory 15 to collectively perform various processes and controls. Specifically, the processor 11B refers to the programs and data held in the memory 15 and executes the programs, thereby implementing the functions of the lip print extractor 14 and other parts.

Next, referring to FIG. 7, the operation procedure of biometric authentication system 200 according to Embodiment 2 will be described. FIG. 7 is a flow chart showing an example of the operation procedure of the biometric authentication system 200 according to the second embodiment. In the biometric authentication system 200 according to the second embodiment, as shown in FIG. 5, the distance measuring sensor SS2 performs steps St21 to St23 and steps St25, the terminal device P1B performs steps St24 to St27, and the server S1 performs steps St28 to St29.

It should be noted that the operation procedure example shown in FIG. 7 shows an operation procedure example when the distance measurement unit 30 is omitted from the configuration of the distance measurement sensor SS2. If the distance measuring unit 30 is not omitted from the configuration of the distance measuring sensor SS2, the biometric authentication system 200 according to Embodiment 2 transmits the captured image of the distance measuring sensor SS2 to the terminal device P1B after the process of step St23, and the terminal device P1B executes processing for extracting a face area from the transmitted captured image.

The ranging sensor SS2 acquires the captured image Pt11 (see FIG. 3) including the face of the person who is the authentication target, transmitted from the camera C1B (St21). The terminal device P1 detects a face area Ar11 including a person's face from the captured image Pt11, and estimates the distance between the subject (person to be authenticated) and the camera C1 based on the size (area) of the face area Ar11 (see FIG. 3) in the entire captured image (St22). The ranging sensor SS2 determines whether or not the estimated distance is equal to or less than a predetermined distance (St23).

When the ranging sensor SS2 determines in the process of step St23 that the estimated distance is equal to or less than the predetermined distance (St23, YES), it transmits the captured image to the terminal device P1B.

On the other hand, when the ranging sensor SS2 determines in the processing of step St23 that the estimated distance is not equal to or less than the predetermined distance (St23, NO), it determines the zoom magnification of the camera C1B based on the difference between the estimated distance and the predetermined distance. The distance measuring sensor SS2 generates a control command to change the currently set zoom magnification to the determined new zoom magnification and to image the person again. Ranging sensor SS2 transmits the generated control command to camera C1B to control the zoom magnification of camera C1B.

In addition, the ranging sensor SS2 generates a control command for displaying an imaging support screen (not shown) requesting the person to be authenticated to approach or leave the camera C1B, and transmits it to the terminal device P1B. The terminal device P1B generates an imaging support screen based on the control command transmitted from the ranging sensor SS2, and outputs and displays the generated imaging support screen on the display unit 16 (St25).

After adjusting the zoom magnification based on the control command transmitted from the ranging sensor SS2, the camera C1B executes the imaging process again and transmits the captured image to the ranging sensor SS2.

The terminal device P1B acquires the captured image Pt12 (see FIG. 3) including the person's face transmitted from the ranging sensor SS2. The terminal device P1B extracts a face area Ar13 (see FIG. 3) of the person reflected in the acquired captured image Pt12 (see FIG. 3) (St24), and further extracts a lip area Pt14 (see FIG. 3) showing the lips (mouth) of the person from the extracted face area Ar13 (St26).

The terminal device P1B generates a lip image obtained by cutting out the extracted lip region Pt14, and corrects the contrast so that the lip print can be detected from the wrinkles of the lips reflected in the generated lip image (St27). The terminal device P1B extracts lip wrinkles (edges) from the contrast-corrected lip image, and performs binarization processing on the lip image (St27). The terminal device P1B extracts the lip print of the person to be authenticated from the binarized lip image Pt15 (see FIG. 3), generates lip print data, and transmits the generated lip print data to the server S1.

Based on the lip print data transmitted from the terminal device P1B, the server S1 extracts a feature amount indicating the individuality of the person. The server S1 collates the extracted feature amount of the lip print with the feature amount of the lip print registered in the database DB (St28).

The server S1 calculates a matching score between the extracted lip print feature amount and the lip print feature amount registered in the database DB. The server S1 selects the highest matching score among the calculated matching scores, and determines (identifies) that the person corresponding to the selected matching score is the same person as the person to be authenticated (St29). The server S1 generates an authentication result including personal information corresponding to the specified person, and transmits it to the terminal device P1B. The terminal device P1B generates a screen (not shown) including the authentication result transmitted from the server S1, and outputs and displays it on the display unit 16. FIG.

As described above, the biometric authentication system 200 according to Embodiment 3 can more efficiently acquire a captured image from which a lip print used for biometric authentication can be acquired. In addition, the biometric authentication system 200 can further improve authentication accuracy by executing biometric authentication using this captured image.

As described above, the

biometric authentication systems

100, 100A, and 200 according to the first embodiment, the modified example of the first embodiment, and the second embodiment include the terminal devices P1, P1A, and P1B and the server S1 that can communicate with the terminal devices P1, P1A, and P1B. Terminal devices P1, P1A, and P1B output control commands (examples of imaging commands) to cameras C1 and C1B that capture images of people, acquire captured images of people captured by cameras C1 and C1B, extract the face regions of the people in the captured images, generate control commands based on the size of the face regions, output the generated control commands to camera C1, extract the lip prints of the people from the captured images, and transmit the commands to server S1. The server S1 identifies a person based on the extracted lip print.

Thus, the

biometric authentication systems

100, 100A, and 200 according to the first embodiment, the modified example of the first embodiment, and the second embodiment estimate the distance between the camera C1 and the person based on the size of the face region, and generate a control command when determining that the estimated distance is equal to or less than a predetermined distance, thereby more efficiently acquiring a captured image from which a lip print used for biometric authentication can be acquired.

As described above, when the terminal device P1 of the biometric authentication system 100 according to the first embodiment compares the extracted lip print with the registered lip print of at least one registered person and determines that there is a registered lip print identical or similar to the extracted lip print, it identifies the person corresponding to the same or similar registered lip print as the person of the lip print. Thereby, the biometric authentication system 100 according to Embodiment 1 can perform biometric authentication based on the lip print appearing in the captured image captured by the camera C1.

As described above, the terminal device P1 of the biometric authentication system 100 according to Embodiment 1 estimates the distance between the camera C1 and the person based on the size of the face area, and generates a control command when determining that the estimated distance is equal to or less than the predetermined distance. Thus, the biometric authentication system 100 according to Embodiment 1 can determine whether or not the image captured by the camera C1 is an image from which a lip print used for biometric authentication can be obtained.

Further, as described above, when the terminal device P1 of the biometric authentication system 100 according to the first embodiment determines that the estimated distance is not equal to or less than the predetermined distance, it generates a control command (an example of a zoom command) for changing the zoom magnification of the camera C1, and outputs the generated control command to the camera C1. As a result, when the biometric authentication system 100 according to Embodiment 1 determines that the captured image captured by the camera C1 is not a captured image from which a lip print used for biometric authentication can be obtained, the biometric authentication system 100 can more efficiently acquire a captured image from which a lip print used for biometric authentication can be obtained by changing the zoom magnification of the camera C1.

As described above, the terminal device P1 of the biometric authentication system 100 according to the first embodiment determines the zoom magnification of the camera C1 based on the difference between the estimated distance and the predetermined distance, and generates a control command to change the current zoom magnification of the camera C1 to the determined zoom magnification. Thus, the biometric authentication system 100 according to Embodiment 1 can more efficiently acquire a captured image from which a lip print used for biometric authentication can be acquired by changing the zoom magnification of the camera C1.

Also, as described above, the

biometric authentication systems

100A and 200 according to the modification of the first embodiment and the second embodiment acquire the distance between the cameras C1 and C1B and the person, generate a control command based on the acquired distance, and output the generated control command to the camera. As a result, the

biometric authentication systems

100A and 200 according to the modification of the first embodiment and the second embodiment can determine whether or not the cameras C1 and C1B are capable of capturing a captured image from which a lip print used for biometric authentication can be obtained, based on the distance between the cameras C1 and C1B and the person to be authenticated.

Also, as described above, the

biometric authentication systems

100A and 200 according to the modification of the first embodiment and the second embodiment generate a control command when determining that the acquired distance is equal to or less than the predetermined distance. As a result, the

biometric authentication systems

100A and 200 according to the modification of the first embodiment and the second embodiment can cause the cameras C1 and C1B to image the person when it is determined that the distance between the cameras C1 and C1B and the person to be authenticated is a distance at which the lip print used for biometric authentication can be obtained. Therefore, the

biometric authentication systems

100A and 200 can more efficiently acquire captured images from which lip prints used for biometric authentication can be acquired.

Also, as described above, the

biometric authentication systems

100, 100A, and 200 according to the first embodiment, the modified example of the first embodiment, and the second embodiment generate a lip image obtained by extracting a lip region including a person's lips from a captured image, and extract a person's lip print from the generated lip image. As a result, the

biometric authentication systems

100, 100A, and 200 according to the first embodiment, the modified example of the first embodiment, and the second embodiment can acquire the lip print of the person to be authenticated from the images captured by the cameras C1 and C1B.

Also, as described above, the

biometric authentication systems

100, 100A, and 200 according to the first embodiment, the modified example of the first embodiment, and the second embodiment binarize the lip image and extract the lip print from the binarized lip image. As a result, the

biometric authentication systems

100, 100A, and 200 according to the first embodiment, the modified example of the first embodiment, and the second embodiment can acquire a lip print more suitable for biometric authentication.

Further, as described above, the

biometric authentication systems

100, 100A, and 200 according to the first embodiment, the modified example of the first embodiment, and the second embodiment correct the contrast of the lip image, extract the lip edge (that is, the lip wrinkle) from the contrast-corrected lip image, and binarize the lip image based on the extracted lip edge. As a result, the

biometric authentication systems

Although various embodiments have been described above with reference to the accompanying drawings, the present disclosure is not limited to such examples. It is clear that a person skilled in the art can conceive of various modifications, modifications, substitutions, additions, deletions, and equivalents within the scope of the claims, and it is understood that they also belong to the technical scope of the present disclosure. In addition, the constituent elements of the various embodiments described above may be combined arbitrarily without departing from the gist of the invention.

This application is based on a Japanese patent application (Japanese Patent Application No. 2022-006567) filed on January 19, 2022, the contents of which are incorporated herein by reference.

The present disclosure is useful as a biometric authentication device, a biometric authentication system, and a biometric authentication method that more efficiently acquire a captured image from which a lip print used for biometric authentication can be acquired.

10, 10A, 10B, 20, 31

communication units

11, 11A, 11B, 21, 32

processors

12, 12A imaging control units 13, 30 distance measurement unit 14 lip

print extraction units

15, 25, 33 memory 22 matching unit 23 feature amount extraction unit 24

score determination units

100, 100A, 200 biometric authentication systems C1, C1B camera DB databases P1, P 1A, P1B terminal device S1 server SS1, SS2 ranging sensor

Claims

a terminal device;
A biometric authentication system comprising a server communicable with the terminal device,
The terminal device
Output an imaging command to a camera that captures a person,
Acquiring a captured image in which the person is captured from the camera,
extracting a face region of the person reflected in the captured image;
generating the imaging command based on the size of the face area;
outputting the generated imaging command to the camera;
extracting the lip print of the person from the captured image and transmitting it to the server;
The server is
identifying the person based on the extracted lip print;
biometric authentication system.
The server is
When it is determined that there is a registered lip print identical or similar to the extracted lip print based on collation of the extracted lip print and registered lip prints of at least one registered person, identifying a person corresponding to the same or similar registered lip print as the person of the lip print;
The biometric authentication system according to claim 1.
The terminal device
estimating a distance between the camera and the person based on the size of the face area;
generating the imaging command when it is determined that the estimated distance is equal to or less than a predetermined distance;
The biometric authentication system according to claim 2.
The terminal device
generating a zoom command for changing the zoom magnification of the camera when it is determined that the estimated distance is not equal to or less than a predetermined distance;
outputting the generated zoom command to the camera;
The biometric authentication system according to claim 3.
The terminal device
determining a zoom magnification of the camera based on the difference between the estimated distance and the predetermined distance;
generating the zoom command to change the current zoom factor of the camera to the determined zoom factor;
The biometric authentication system according to claim 4.
The terminal device
obtaining the distance between the camera and the person;
generating the imaging command based on the obtained distance;
outputting the generated imaging command to the camera;
The biometric authentication system according to claim 1.
The terminal device
generating the imaging command when it is determined that the obtained distance is equal to or less than a predetermined distance;
The biometric authentication system according to claim 6.
The terminal device
generating a lip image by extracting a lip region including the lips of the person from the captured image;
extracting the lip print from the generated lip image;
The biometric authentication system according to claim 1.
The terminal device
binarizing the lip image and extracting the lip print from the binarized lip image;
The biometric authentication system according to claim 8.
The terminal device
correcting the contrast of the lip image and extracting the edge of the lip from the contrast-corrected lip image;
binarizing the lip image based on the extracted edge of the lip;
The biometric authentication system according to claim 9.
A biometric authentication method performed by one or more computers,
Output an imaging command to a camera that captures a person,
Acquiring a captured image in which the person is captured from the camera,
extracting a face region of the person reflected in the captured image;
generating the imaging command based on the size of the face area;
outputting the generated imaging command to the camera;
extracting a lip print of the person from the captured image;
identifying the person based on the extracted lip print;
biometric method.