JP6675893B2 - Personal authentication system, personal authentication device, personal authentication method, and program - Google Patents

Description

  The present invention relates to a personal authentication system, a personal authentication device, a personal authentication method, and a program.

  There is a personal authentication method for identifying an individual using a plurality of authentication methods (for example, Patent Document 1). Thus, for example, based on using an authentication method (first authentication) that is highly convenient for the user to be identified, if the individual cannot be identified by the first authentication, it is more convenient than the first authentication. However, it is possible to perform personal authentication using an authentication method (second authentication) that is inferior but has higher authentication accuracy in a complementary manner. For the first authentication, for example, a personal authentication method with a low work load on the user and high convenience, such as face authentication for automatically capturing a user's face to identify an individual, is used. For the second authentication, it is desirable to use an authentication method with higher authentication accuracy, such as fingerprint authentication or ID / password authentication.

WO2002 / 025457

  In the personal authentication method as described above, since the convenience of the user increases as the degree of success of the personal authentication by the first authentication increases, it is required to increase the authentication accuracy in the first authentication. However, according to the conventional personal authentication method (for example, Patent Document 1), even if personal authentication is repeatedly performed, the authentication accuracy of a specific personal authentication method such as the first authentication does not increase. .

  The present invention has been made in view of the above points, and provides a personal authentication system, a personal authentication method, a personal authentication device, and a program that can increase the authentication accuracy in personal authentication.

(1) The present invention has been made to solve the above-described problems. One aspect of the present invention is a personal authentication system having a first authentication server and a second authentication server, wherein the first authentication A server configured to store, for each user, face authentication data based on the face image of the user; an acquisition unit configured to acquire face authentication data based on the face image of the target user for personal authentication; and a face authentication acquired by the acquisition unit. A first authentication unit that performs a first authentication including a face authentication for identifying an individual based on a result of matching data with the face authentication data stored in the storage unit, wherein the second authentication server includes the first authentication unit. A second authentication unit that performs a second authentication for specifying the user based on an input from a terminal when the authentication is not successful; and the storage unit when the first authentication or the second authentication is successful. Is the previously remembered face A personal authentication system and to store the face authentication data which the acquiring unit in addition to the testimony data acquired.

(2) In one aspect of the present invention, the face authentication is performed by comparing the face authentication data acquired by the acquisition unit with all of at least one face authentication data stored in the storage unit. And determining whether or not the person indicated by the face authentication data acquired by the acquisition unit and the person indicated by the face authentication data stored in the storage unit are the same person, and the ratio of the number of cases determined to be the same person. The personal authentication system according to (1), wherein it is determined that the authentication has been successful when is greater than or equal to a predetermined ratio.

(3) Also, as one aspect of the present invention, an imaging unit that captures a face image of the user, and face authentication data based on the face image captured by the imaging unit are transmitted to the first authentication server. The personal authentication system according to (1), further including a robot including: a communication unit configured to receive information indicating a response to the user from the first authentication server.

(4) According to another aspect of the invention, the first authentication unit repeatedly performs the first authentication based on the face authentication data periodically acquired by the acquisition unit. (1) The personal authentication system described in the above.

(5) Also, as one aspect of the present invention, the information input by the second authentication unit from the terminal includes an identifier for identifying the robot, and the second authentication server does not pass through the robot. Performing the second authentication, transmitting the information identifying the user to the robot identified by the identifier if the second authentication is successful, and if the first authentication or the second authentication is successful, The personal authentication system according to (3), wherein the storage unit further stores the face authentication data acquired by the acquisition unit while the response to the user is being made.

(6) According to another aspect of the present invention, a storage unit that stores face authentication data based on a face image of a user for each user, and an obtaining unit that obtains face authentication data based on a face image of a target user for personal authentication. A first authentication unit that performs a first authentication including a face authentication for identifying an individual based on a result of matching the face authentication data obtained by the obtaining unit with the face authentication data stored in the storage unit; A second authentication unit that performs a second authentication that specifies the user based on an input from a terminal when the authentication is not successful, and stores the memory when the first authentication or the second authentication is successful. The unit stores the face authentication data acquired by the acquiring unit in addition to the already stored face authentication data.

(7) Also, as one aspect of the present invention, the storage unit stores, for each user, face authentication data based on the user's face image, and the acquisition unit stores the face image of the target user for personal authentication. Acquiring the face authentication data based on the information, and the first authentication unit identifies the individual based on a result of the comparison between the face authentication data acquired by the acquisition unit and the face authentication data stored by the storage unit. A first authentication step of performing a first authentication including a face authentication, and a second authentication step of performing a second authentication for specifying the user based on an input from a terminal when the first authentication is not successful. A second authentication step, and in the case where the first authentication or the second authentication is successful, the storage unit stores the face authentication data obtained by the obtaining unit in addition to the already stored face authentication data. 2 memory station A personal authentication method characterized by comprising the flop, the.

(8) According to another aspect of the present invention, a first storage step of storing face authentication data based on a user's face image in a computer for each user, and face authentication data based on a face image of a target user for personal authentication. And performing first authentication including face authentication for identifying an individual based on a result of matching the face authentication data obtained in the obtaining step with the face authentication data stored in the first storage step. A first authentication step, a second authentication step of performing a second authentication for specifying the user based on an input from a terminal when the first authentication is not successful, and wherein the first authentication or the second authentication is performed. If successful, executing a second storing step of storing the face authentication data acquired in the acquiring step in addition to the already stored face authentication data. A program for causing.

  According to the present invention, authentication accuracy in personal authentication can be increased.

It is a schematic diagram showing an example of composition of a personal identification system concerning a 1st embodiment of the present invention. FIG. 2 is a block diagram illustrating a functional configuration of a robot included in the personal authentication system according to the first embodiment of the present invention. FIG. 2 is a block diagram illustrating a functional configuration of a first authentication server included in the personal authentication system according to the first embodiment of the present invention. FIG. 2 is a block diagram illustrating a functional configuration of a terminal included in the personal authentication system according to the first embodiment of the present invention. FIG. 2 is a block diagram illustrating a functional configuration of a second authentication server included in the personal authentication system according to the first embodiment of the present invention. 4 is a flowchart illustrating an operation of the personal authentication system according to the first embodiment of the present invention. 4 is a flowchart illustrating an operation of the personal authentication system according to the first embodiment of the present invention. 4 is a flowchart illustrating an operation of the personal authentication system according to the first embodiment of the present invention. 4 is a flowchart illustrating an operation of the personal authentication system according to the first embodiment of the present invention. 4 is a flowchart illustrating an operation of the personal authentication system according to the first embodiment of the present invention.

(First embodiment)
(Configuration of personal authentication system)
Hereinafter, a first embodiment of the present invention will be described with reference to the drawings.
FIG. 1 is a schematic diagram illustrating an example of a configuration of a personal authentication system 1 according to the first embodiment of the present invention.

  The personal authentication system 1 is configured to include a robot 10, a first authentication server 20 (robot management server), a terminal 30, and a second authentication server 40. The personal authentication system 1 is, for example, a system used in reception of a store or the like, and identifies a visitor (hereinafter, also referred to as a user) by personal authentication including face authentication, and sends the identified user to the identified user via the robot 10. It is a system that responds by a corresponding dialogue or the like. Further, data acquired at the time of personal authentication of the user and at the time of response to the user are stored (stored), and can be used at the next time of personal authentication of the user.

  The robot 10 is installed at, for example, a reception in a store, recognizes the face of a user who visits the reception, and transmits face authentication data based on the face image to the first authentication server 20. The robot 10 and the first authentication server are communicatively connected via a communication line 50. The communication line 50 is, for example, a wireless communication line such as LTE (Long Term Evolution) or 3G (3rd Generation; third generation mobile communication system) line, or a public wireless LAN (Wi-Fi; Wireless Fidelity). Wi-Fi), LAN (Local Area Network; private communication network), and WAN (Wide Area Network; wide area network).

  The first authentication server 20 (robot management server) can store (accumulate) face authentication data based on the face image of the user for each user. The first authentication server 20 performs a first authentication including a face authentication for identifying an individual based on a result of matching the face authentication data acquired from the robot 10 with the face authentication data stored in the first authentication server 20 of the first authentication server 20. Do. The first authentication server 20 includes, for example, a server device using a general-purpose computer, a personal computer, or the like.

  When the first authentication is successful (when the user can be identified), the first authentication server 20 (robot management server) responds to the user via the robot 10. When the first authentication is not successful (when the user cannot be identified), the first authentication server 20 transmits, for example, voice data indicating a message to the user, and outputs a voice based on the voice data to the robot. 10 is output. The message includes, for example, a message to the effect that "Please start an application (for performing the second authentication) at the terminal".

  When the second authentication application is started in the terminal 30 by the above message, the second authentication server 40 converts the input data from the terminal 30 (for example, signature data based on fingerprint information obtained by acquiring a fingerprint) into the input data. Based on this, a second authentication (for example, fingerprint authentication) for specifying the individual is performed.

  The first authentication server 20 and the terminal 30 are connected for communication via a communication line 60. The communication line 60 includes, for example, a wireless communication line such as an LTE or 3G line, and a dedicated line such as a public wireless LAN, a LAN, and a WAN.

  The terminal 30 includes a configuration for receiving an input in the second authentication, for example, a sensor for reading a fingerprint, a touch panel for receiving an input of a user ID (Identifier) or a password, and the like. The terminal 30 is, for example, a smartphone, a tablet terminal, or a small personal computer.

The second authentication server 40 is configured to include a server device using, for example, a general-purpose computer or a personal computer.
The terminal 30 and the second authentication server 40 are connected for communication via a communication line 70. For example, it includes a wireless communication line such as an LTE or 3G line, and a dedicated line such as a public wireless LAN (Wi-Fi), a LAN, and a WAN.
Further, the first authentication server 20 and the second authentication server 40 are connected for communication via a communication line 80. The communication line 80 includes, for example, a dedicated line such as a LAN and a WAN.

If the second authentication is successful, the first authentication server 20 (robot management server) responds to the user via the robot 10.
The face authentication data based on the face images acquired at the time of the first authentication, at the time of the second authentication, and at the time of responding to the user are stored in the first authentication server 20, and are stored in the next first authentication of the user. Can be used.

(Robot configuration)
Next, the functional configuration of the robot 10 in the personal authentication system 1 will be described with reference to the drawings.
FIG. 2 is a block diagram illustrating a functional configuration of the robot 10 included in the personal authentication system 1 according to the first embodiment of the present invention.
The robot 10 includes a control unit 100, an imaging unit 101, a face detection unit 102, a voice input unit 103, a data conversion unit 104, a voice output unit 105, a storage unit 106, a display unit 107, a communication unit 108.

  The control unit 100 controls various processes in the robot 10. The control unit 100 includes, for example, a CPU (Central Processing Unit).

  The imaging unit 101 captures an image of the front of the robot 10. The imaging unit 101 includes, for example, a camera. The shooting may be a moving image, or may be a still image taken at regular intervals (for example, every second). The imaging unit 101 outputs image data indicating a captured image to the face detection unit 102. In the case of a moving image, the imaging unit 101 outputs image data indicating a still image of a screen shot at regular intervals (for example, every one second) to the face detection unit 102, for example.

  The face detection unit 102 analyzes an image captured by the imaging unit 101. The face detection unit 102 can detect an image that is presumed to be a human face in the image. The face detection unit 102 outputs face authentication data indicating an image (hereinafter, referred to as a face image) including an image estimated to be a person's face to a communication unit 108 described below.

The voice input unit 103 generates a voice signal based on a user's voice uttered to the robot 10 from the front of the robot 10. The voice input unit 103 includes, for example, a microphone. The audio signal generated by the audio input unit 103 is appropriately converted to audio data of a predetermined data format via the data conversion unit 104, and output to the communication unit 108 described later.
In the present embodiment, the robot 10 includes the data conversion unit 104, but is not limited thereto. For example, the first authentication server 20 (robot management server) may include a data conversion unit. When the data conversion unit is provided on the first authentication server 20 (robot management server) side, the voice signal generated by the voice input unit 103 is transmitted via the communication unit 108 to the first authentication server 20 (robot management server). ) And converted into audio data in a predetermined data format.

  The audio output unit 105 outputs audio based on audio data obtained from the first authentication server 20 via the communication unit 108. The voice based on the voice data acquired from the first authentication server 20 is, for example, a voice indicating a response message to the user. The audio output unit 105 includes, for example, a speaker.

  The storage unit 106 stores various application programs and various data in advance. The storage unit 106 may also be used as a temporary storage area for various processes in the robot 10. The storage unit 106 includes a storage medium such as a hard disk drive (HDD), a flash memory, an electrically erasable programmable read only memory (EEPROM), and a random access memory (RAM). , ROM (Read Only Memory), or an arbitrary combination thereof.

  The display unit 107 displays various information for the user. The display unit 107 includes a display, for example, a liquid crystal display (LCD) or an organic EL (Electro Luminescence) display.

The communication unit 108 is a communication interface for communicatively connecting to the first authentication server 20 (robot management server) via the communication line 50.
The communication unit 108 transmits the face authentication data based on the image including the face image of the user and the voice data based on the voice of the user to the communication unit 201 of the first authentication server 20 via the communication line 50.
The communication unit 108 transmits the voice data indicating a response message to the user and various information used in personal authentication (for example, information indicating the result of personal authentication) to the first authentication server via the communication line 50. 20 (robot management server).

(Configuration of first authentication server)
Next, the functional configuration of the first authentication server 20 (robot management server) in the personal authentication system 1 will be described with reference to the drawings.
FIG. 3 is a block diagram illustrating a functional configuration of the first authentication server 20 included in the personal authentication system 1 according to the first embodiment of the present invention.
The first authentication server 20 (robot management server) according to the present embodiment includes a control unit 200, a communication unit 201 (acquisition unit), a face authentication unit 202 (first authentication unit), an authentication information storage unit 203, The keyword authentication unit 204, the keyword generation unit 205, the voiceprint authentication unit 206, the response control unit 207, the device authentication unit 208, the response information storage unit 209, the user ID authentication unit 210, and the authentication data learning unit 211 , An authentication status update unit 212 and an authentication cooperation unit 213.

  The control unit 200 controls various processes in the first authentication server 20. The control unit 200 includes, for example, a CPU.

The communication unit 201 (acquisition unit) is a communication interface for communicatively connecting to the robot 10 via the communication line 50.
The communication unit 201 transmits voice data indicating a response message to the user and various information used in personal authentication (for example, information indicating a result of personal authentication) to the robot 10 via the communication line 50.
The communication unit 201 receives face authentication data based on an image including a face image of the target user for personal authentication and voice data based on the voice of the user from the communication unit 108 of the robot 10 via the communication line 50.

The communication unit 201 (acquisition unit) is a communication interface for communicatively connecting to the terminal 30 via the communication line 60.
The communication unit 201 transmits various information used in personal authentication (for example, information indicating the result of personal authentication) to a communication unit 302 of the terminal 30 via the communication line 60, which will be described later.
In addition, the communication unit 201 receives various information used in personal authentication (for example, information indicating a request for personal authentication) from a communication unit 302 of the terminal 30 described later via the communication line 60.

  The face authentication unit 202 (first authentication unit) includes face authentication data acquired from the robot 10 via the communication unit 201 and face authentication data stored in an authentication information storage unit 203 (to be described later) of the first authentication server 20 of its own. Face authentication is performed based on the matching result based on the degree of conformity with the data.

The authentication information storage unit 203 stores a face authentication database that stores (accumulates) face authentication data based on a user's face image for each user.
The authentication information storage unit 203 stores a voiceprint authentication database that stores (accumulates) voiceprint data based on the voiceprint of the user's voice for each user.
Further, the authentication information storage unit 203 stores an identifier for identifying the robot 10 (for example, a device address of a Bluetooth (registered trademark) device included in the robot 10) in advance. When there are a plurality of robots 10, the authentication information storage unit 203 stores in advance a list of identifiers corresponding to each robot 10.
The authentication information storage unit 203 stores a customer information database including a list of user IDs for identifying users in advance.
The authentication information storage unit 203 is configured to include a storage medium, for example, an HDD, a flash memory, an EEPROM, a RAM, or any combination thereof.

The keyword authentication unit 204 performs keyword authentication using data indicating a keyword generated by a keyword generation unit 205 described later.
The keyword generation unit 205 generates data (for example, data in a text format) indicating an arbitrary keyword used for keyword authentication in response to a request from the keyword authentication unit 204.

  Specifically, the keyword authentication unit 204 outputs information indicating a request for keyword generation to the keyword generation unit 205. When information indicating a request for keyword generation is input from the keyword authentication unit 204, the keyword generation unit 205 generates data (for example, text format data) indicating an arbitrary keyword used for keyword authentication. The keyword generation unit 205 transmits data indicating the generated keyword to the communication unit 302 of the terminal 30 via the communication unit 201, which will be described later.

Then, the keyword displayed on the terminal 30 is uttered to the robot 10 by the user.
The keyword authentication unit 204 acquires data (for example, voice data) indicating a keyword based on the utterance of the user from the communication unit 108 of the robot 10 via the communication unit 201 of the keyword authentication unit 204 of the robot 10. The keyword authentication unit 204 performs data format conversion (for example, conversion from voice data to text format data) as appropriate, and collates the transmitted keyword with the received keyword. When the transmitted keyword matches the received keyword, the keyword authentication unit 204 determines that the keyword authentication has been successful. When the transmitted keyword does not match the received keyword, the keyword authentication unit 204 determines that the keyword authentication has failed.

  The voiceprint authentication unit 206 detects voiceprint authentication data indicating the voiceprint of the user from voice data based on the voice indicating the user's voice acquired from the robot 10 via the communication unit 201. The voiceprint authentication unit 206 performs voiceprint authentication based on the matching degree between the detected voiceprint authentication data and the voiceprint authentication data stored in the authentication information storage unit 203.

The response control unit 207 is configured to include a dialogue engine that analyzes the content of an utterance made by the user to the robot 10 and selects (or generates) a response message according to the analyzed utterance content.
The response control unit 207 acquires voice data based on the voice uttered by the user from the communication unit 108 of the robot 10 via the communication unit 201, and analyzes the utterance content based on the voice data. The response control unit 207 appropriately obtains the response message text from the response message text database stored in the response information storage unit 209 described below, according to the analysis result of the utterance content. Then, the response control unit 207 generates a response message using the appropriately acquired response message text. The response control unit 207 converts the information indicating the generated response message into audio data, and transmits the audio data to the communication unit 108 of the robot 10 via the communication unit 201.

The device authentication unit 208 performs device authentication in response to a request from the terminal 30. Specifically, the device authentication unit 208 acquires, from the communication unit 302 of the terminal 30, a device identifier (for example, a Bluetooth (registered trademark) device address) that identifies the device via the communication unit 201. The device authentication unit 208 checks whether or not the same identifier as the acquired device identifier exists in the identifier (identifier for identifying the robot 10) stored in the authentication information storage unit 203. Thereby, it is determined that the device corresponding to the obtained device identifier is the robot 10 managed by its own first authentication server (robot management server).
The device authentication unit 208 transmits information indicating the determination result of the device authentication to the communication unit 302 of the terminal 30 via the communication unit 201.

  The response information storage unit 209 stores the wording database for the response message as described above. The data format of the text data stored in the text database may be any data format. For example, the text data may be any of text format data, binary format data, voice data, and the like, as long as the data format is appropriately converted when communicating with the robot 10.

The user ID authentication unit 210 acquires a user ID for identifying a user from the communication unit 302 of the terminal 30 via the communication unit 201. The device authentication unit 208 checks whether the same user ID as the acquired user ID exists in the list of user IDs stored in the authentication information storage unit 203. Thereby, it is determined whether or not the user ID acquired from the terminal 30 is a user ID managed by its own first authentication server (robot management server).
The user ID authentication unit 210 transmits information indicating the determination result of the user ID authentication to the communication unit 302 of the terminal 30 via the communication unit 201.

  The authentication data learning unit 211 authenticates information (for example, face authentication data based on a user's face image, voiceprint authentication data based on a user's voiceprint, and the like) acquired during various types of authentication and when responding to the user. The information is stored (stored) in various databases stored in the information storage unit 203.

  The authentication status update unit 212 updates the authentication status data indicating the authentication status of each user based on the determination result of each personal authentication. Note that the authentication status data is stored in, for example, the authentication information storage unit 203. For example, the authentication status data is updated to data indicating an authentication status of “authorized” when the first authentication or the second authentication is performed, and when the interaction between the user and the robot ends or when the user speaks for a certain period of time. When a timeout occurs due to no occurrence, the data is updated to data indicating the authentication state of “unauthenticated”.

  The authentication cooperation unit 213 is a communication interface for communicatively connecting to the second authentication server 40 via the communication line 80. By the communication connection, data synchronization between the first authentication server 20 and the second authentication server 40 is achieved. The data referred to here is, for example, customer information or data indicating a user authentication state.

(Judgment by the face authentication unit)
Hereinafter, an example of the determination by the face authentication unit 202 (first authentication unit) based on the matching degree of the face authentication data will be described. The face authentication described above is specifically performed in the following procedure, for example.
The face authentication unit 202 determines, for each user, the degree of conformity between the face authentication data acquired from the robot 10 via the communication unit 201 and the face authentication data in the face authentication database stored in the authentication information storage unit 203. .

  For example, it is assumed that 10 pieces of face authentication data based on a face image of a certain user are stored in the face authentication database. The face authentication data is, for example, data based on a face image captured when a user has visited a store in the past. The face authentication unit 202 collates the face authentication data acquired from the robot 10 with all of the ten face authentication data described above to correspond to the user photographed by the robot 10 and the face authentication data. It is determined whether or not the user is the same person.

In the determination results of the suitability of the ten face authentication data, the ratio of the number of the determination results determined to be the same person (because of the high suitability) is equal to or more than a predetermined ratio (for example, 80% or more). (That is, if it is determined that eight out of ten face images are the same person), the face authentication unit 202 determines that the face authentication has been successful. Note that the face authentication unit 202 identifies the user by performing the above-described comparison on the entire face authentication data stored in the face authentication database.
The face authentication data may be image data based on a face image of the user, or may be feature amount data indicating a feature amount extracted from the image data based on the face image of the user.

(Terminal configuration)
Next, the functional configuration of the terminal 30 in the personal authentication system 1 will be described with reference to the drawings.
FIG. 4 is a block diagram illustrating a functional configuration of the terminal 30 included in the personal authentication system 1 according to the first embodiment of the present invention.
The terminal 30 according to the present embodiment includes a control unit 300, a display unit 301, a communication unit 302, a fingerprint input unit 303, a device detection unit 304, an operation input unit 305, and a storage unit 306. Be composed.

The control unit 300 controls various processes in the terminal 30. The control unit 300 includes, for example, a CPU.
The display unit 301 displays various information for the user. The display unit 107 includes a display, for example, a liquid crystal display (LCD) or an organic EL display.

The communication unit 302 is a communication interface for communication connection with the first authentication server 20 (robot management server) via the communication line 60.
The communication unit 302 transmits various information used in personal authentication (for example, information indicating a request for personal authentication) to the communication unit 201 of the first authentication server 20 via the communication line 60.
In addition, the communication unit 302 receives various information used in personal authentication (for example, information indicating a result of the personal authentication) from the communication unit 201 of the first authentication server 20 via the communication line 60.

Further, the communication unit 302 is a communication interface for communicatively connecting to the second authentication server 40 via the communication line 70.
The communication unit 302 transmits various information used in personal authentication (for example, information indicating a request for personal authentication) to the communication unit 401 of the second authentication server 40 via the communication line 60, which will be described later.
In addition, the communication unit 302 receives various information used in personal authentication (for example, information indicating a result of the personal authentication) from a communication unit 401 (described later) of the second authentication server 40 via the communication line 60.

  The fingerprint input unit 303 includes a sensor that reads a user's fingerprint. The fingerprint input unit 303 identifies a user by comparing fingerprint data based on the read fingerprint with fingerprint data included in a fingerprint data list stored in the storage unit 306 described below.

  In the present embodiment, biometric authentication (fingerprint authentication) using FIDO, which is a standardization technology of passwordless authentication specified by the FIDO (Fast Identity Online; Fido) Alliance, will be described as an example of the second authentication. Not limited to For example, instead of fingerprint authentication, authentication using a user ID and a password may be used as the second authentication.

  The device detection unit 304 can detect a device identifier that identifies a nearby robot 10. The device detection unit 304 acquires information indicating a device identifier (for example, a Bluetooth (registered trademark) device address) that identifies the robot 10 via, for example, an antenna (not illustrated) provided in the terminal 30. The device detection unit 304 causes the display unit 301 to display the acquired device identifier list, and allows the user to select a specific device identifier included in the device identifier list using the operation input unit 305 described later. Then, the operation input unit 305 transmits data indicating the selected device identifier to the first authentication server 20 (robot management server) via the communication unit 302.

The operation input unit 305 receives various operation inputs from the user. The operation input unit 305 includes, for example, a touch panel, input buttons, a keyboard, a touch pad, and the like.
The storage unit 306 stores in advance various data including a user's fingerprint data and various application programs including an application program of an application used in fingerprint authentication (second authentication). The storage unit 306 can also be used as a temporary storage area for various processes in the terminal 30. The storage unit 306 is configured to include a storage medium, for example, an HDD, a flash memory, an EEPROM, a RAM, a ROM, or any combination thereof.

(Configuration of second authentication server)
Next, a functional configuration of the second authentication server 40 in the personal authentication system 1 will be described with reference to the drawings.
FIG. 5 is a block diagram illustrating a functional configuration of the second authentication server 40 included in the personal authentication system 1 according to the first embodiment of the present invention.
The second authentication server 40 according to the present embodiment includes a control unit 400, a communication unit 401, a fingerprint authentication processing unit 402 (second authentication unit), a storage unit 403, and an authentication cooperation unit 404. Is done.

The control unit 400 controls various processes in the second authentication server 40. The control unit 400 includes, for example, a CPU.
The communication unit 401 is a communication interface for communicatively connecting to the terminal 30 via the communication line 70.
The communication unit 401 transmits various information used in personal authentication (for example, information indicating the result of personal authentication) to the communication unit 401 of the terminal 30 via the communication line 60.
In addition, the communication unit 302 receives various information used in personal authentication (for example, information indicating a request for personal authentication) from a communication unit 401 (described later) of the second authentication server 40 via the communication line 60.

  The fingerprint authentication processing unit 402 (second authentication unit) acquires the signature data generated according to the result of the fingerprint authentication performed in the terminal 30 from the communication unit 302 of the terminal 30 via the communication unit 401. The fingerprint authentication processing unit 402 verifies the obtained signature data, and transmits the verification result to the communication unit 302 of the terminal 30 via the communication unit 401.

  In the FIDO (Fido), the fingerprint authentication itself is completed in the terminal 30, only the signature data not including the fingerprint data is transmitted from the terminal 30 to the second authentication server 40, and the personal authentication is performed. You. In addition, the public key and the private key used in the signature and the signature verification are securely shared in advance between the terminal 30 and the second authentication server 40.

  The storage unit 403 stores various data used in the second authentication server 40 and various application programs in advance. The storage unit 403 may be used as a temporary storage area for various processes in the second authentication server 40. The storage unit 403 includes a storage medium, for example, an HDD, a flash memory, an EEPROM, a RAM, a ROM, or any combination thereof.

  The authentication cooperation unit 404 is a communication interface for communication connection with the first authentication server 20 (robot management server) via the communication line 80. By the communication connection, data synchronization between the first authentication server 20 and the second authentication server 40 is achieved. As described above, the data here is, for example, customer information, data indicating the authentication state of the user, and the like.

(Operation of personal authentication system)
Next, the operation of the personal authentication system 1 will be described with reference to the drawings.
6 to 10 are flowcharts showing the operation of the personal authentication system 1 according to the first embodiment of the present invention. This flowchart is started when the robot 10 starts shooting with the imaging unit 101.

(Step st001) The imaging unit 101 of the robot 10 photographs the front of the robot 10. The face detection unit 102 analyzes an image captured by the imaging unit 101. The face detection unit 102 detects an image included in the image and estimated to be a human face. The face detection unit 102 outputs, to the communication unit 108, face authentication data indicating an image (face image) including an image estimated to be a human face. Thereafter, the process proceeds to step st002.

(Step st002) The communication unit 108 of the robot 10 transmits the face authentication data input from the face detection unit 102 to the communication unit 201 of the first authentication server 20 via the communication line 50. Thereafter, the process proceeds to step st003.

(Step st003) The face authentication unit 202 (first authentication unit) of the first authentication server 20 includes the face authentication data acquired from the robot 10 via the communication unit 201 and authentication information of the own first authentication server 20 described later. The face authentication is performed based on the matching degree between the face authentication data stored in the storage unit 203 and the face authentication data.

When the face authentication is successful (that is, when the user corresponding to the face image can be identified), the face authentication unit 202 outputs information indicating a request to execute keyword authentication to the keyword authentication unit 204. Thereafter, the process proceeds to step st004.
When the face authentication is not successful (that is, when the user corresponding to the face image cannot be identified), the face authentication unit 202 transmits information indicating a request for starting an application (for performing the second authentication) to: The data is transmitted to the communication unit 108 of the robot 10 via the communication unit 201. Thereafter, the process proceeds to step st015.

(Step st004) When information indicating a request for executing keyword authentication is input from the face authentication unit 202, the keyword authentication unit 204 outputs information indicating a request for keyword generation to the keyword generation unit 205. When information indicating a request for keyword generation is input from the keyword authentication unit 204, the keyword generation unit 205 generates data (for example, text data) indicating an arbitrary keyword used for keyword authentication. Thereafter, the process proceeds to step st005.

(Step st005) The keyword authentication unit 204 transmits, to the communication unit 108 of the robot 10 via the communication unit 201, information indicating a standby instruction for waiting for acquisition of a voice indicating a keyword read out by the user. Thereby, the robot 10 is in a state of waiting for the acquisition of the voice indicating the keyword read out by the user. Thereafter, the process proceeds to step st006.

(Step st006) The keyword generation unit 205 transmits data indicating the keyword generated in step st004 to the communication unit 302 of the terminal 30 via the communication unit 201, which will be described later. The communication unit 302 of the terminal 30 receives the data indicating the keyword. Thereafter, the process proceeds to step st007.

(Step st007) The display unit 301 of the terminal 30 displays a keyword based on the data received by the communication unit 302. Note that, instead of the keyword being displayed on the display unit 301, for example, a configuration may be used in which the keyword is output by voice using a speaker (not shown) or the like provided in the terminal 30. Thereafter, the process proceeds to step st008.

(Step st008) The user reads out the keyword displayed on the display unit 301 in step st007 to the voice input unit 103 (for example, a microphone) of the robot 10. The voice input unit 103 performs voice recognition of a voice uttered by the user, and generates a voice signal based on a voice in which the keyword is read out. Thereafter, the process proceeds to step st009.

(Step st009) The audio signal generated by the audio input unit 103 is appropriately converted to audio data of a predetermined data format via a data conversion unit 104 described later, and output to the communication unit 108. The communication unit 108 transmits voice data indicating the keyword spoken by the user to the communication unit 201 of the first authentication server 20 via the communication line 50. Thereafter, the process proceeds to step st010.

(Step st010) The communication unit 201 of the first authentication server 20 outputs the voice data received from the robot 10 in step st009 to the keyword authentication unit 204. The keyword authentication unit 204 appropriately converts the input voice data into a data format (for example, converts voice data into text format data). The keyword authentication unit 204 collates the keyword based on the voice data with the keyword generated by the keyword generation unit 205 in step st004.

  When the keyword based on the voice data matches the keyword generated by the keyword generation unit 205 in step st004, the keyword authentication unit 204 determines that the keyword authentication has been successful. When the keyword authentication is successful (that is, when it is not confirmed that the user holding the terminal 30 and the user speaking to the robot 10 are the same), the keyword authentication unit 204 sets the voiceprint authentication unit Information indicating a request to execute voiceprint authentication is output to 206. Thereafter, the process proceeds to step st011.

  If the keyword based on the audio data does not match the keyword generated by the keyword generation unit 205 in step st004, it is determined that the keyword authentication has failed. When the keyword authentication is not successful (that is, when it is not confirmed that the user holding the terminal 30 and the user speaking to the robot 10 are the same), the keyword authentication unit 204 determines ( Information indicating a request for starting the application (for performing the second authentication) is transmitted to the communication unit 108 of the robot 10 via the communication unit 201. Thereafter, the process proceeds to step st015.

(Step st011) The communication unit 201 of the first authentication server 20 outputs the voice data received from the robot 10 in step st009 to the voice print authentication unit 206. The voiceprint authentication unit 206 appropriately converts the input voice data into a data format (for example, conversion from voice data to information indicating a voiceprint (feature amount)). The keyword authentication unit 204 performs voiceprint authentication based on the matching degree between the information indicating the voiceprint based on the voice data and the voiceprint data stored in the authentication information storage unit 203.

  If the voiceprint authentication is successful (that is, if the user corresponding to the voiceprint based on the voice data received from the robot 10 can be identified in step st009), the voiceprint authentication unit 206 returns the authentication status data indicating the authentication status of the user. Information indicating an update request (that is, a request for updating to the authentication status data indicating that the user is authenticated) is output to the authentication status update unit 212. Thereafter, the process proceeds to step st012.

  If the voiceprint authentication is not successful (that is, if the user corresponding to the voiceprint based on the voice data received from the robot 10 in step st009 cannot be identified), the voiceprint authentication unit 206 (to perform the second authentication) (A) Information indicating a request to start an application is transmitted to the communication unit 108 of the robot 10 via the communication unit 201. Thereafter, the process proceeds to step st015.

(Step st012) The authentication status update unit 212 updates the authentication status data indicating that the user is authenticated, based on the authentication status data update request input from the voice print authentication unit 206. The authentication status update unit 212 outputs to the authentication data learning unit 211, information indicating that the user has been updated to authentication status data indicating that the user has been authenticated. Thereafter, the process proceeds to step st013.

(Step st013) Upon acquiring the information output from the authentication state updating unit 212 in step st012, the authentication data learning unit 211 stores the face authentication data based on the user's face image received by the communication unit 201 in step st003 in the authentication information. The various face authentication databases stored in the unit 203 are stored (stored).
When acquiring the information output from the authentication state updating unit 212 in step st012, the authentication data learning unit 211 stores the voiceprint authentication data based on the user's voiceprint received by the communication unit 201 in step st010 in the authentication information storage unit 203. It is stored (stored) in the stored various voiceprint authentication databases. Thereby, the user's characteristics (for example, a face image and a voiceprint) are learned.
The authentication data learning unit 211 outputs, to the response control unit 207, information indicating that the learning of the user characteristics described above has been completed. Thereafter, the process proceeds to step st014.

(Step st014) When the response control unit 207 acquires the information output from the authentication data learning unit 211 in step st013, the response control unit 207 gives the user identified by the face authentication in step st003 or the user identified by the voiceprint authentication in step st011. A corresponding response message is selected (or generated), and data indicating the response message is transmitted to the communication unit 108 of the robot 10 via the communication unit 201. Thereafter, the process proceeds to step st401 shown in FIG.

(Step st015) The communication unit 201 of the robot 10 receives information indicating a request to start an application from the first authentication server 20. Based on the information indicating the request to start the application, the robot 10 outputs, for example, a response message including a message to the effect that "Please start the application (for performing the second authentication) at the terminal". The output unit 105 outputs audio. The robot 10 may display the response message on the display unit 107 as text information. Thereafter, the process proceeds to step st101 shown in FIG.

(Step st101) The response authentication message output (or displayed) in step st015 activates the second authentication application in the terminal 30 by a user operation. Thereafter, the process proceeds to step st102.

(Step st <b> 102) When the application for the second authentication is started by the user operation on the terminal 30, the control unit 300 of the terminal 30 transmits the information indicating the start of the second authentication processing to the second communication via the communication unit 302. The information is transmitted to the communication unit 401 of the authentication server 40. Thereafter, the process proceeds to step st103.

(Step st103) Upon receiving the information indicating the start of the second authentication process transmitted from the terminal 30, the communication unit 401 of the second authentication server 40 outputs the information to the fingerprint authentication processing unit 402 (second authentication unit). I do. When the information is input, the fingerprint authentication processing unit 402 generates a challenge (data to be subjected to a digital signature generated by a random number). The fingerprint authentication processing unit 402 transmits the generated challenge to the communication unit 302 of the terminal 30 via the communication unit 401. Thereafter, the process proceeds to step st104.

(Step st104) The communication unit 302 of the terminal 30 receives the challenge transmitted from the second authentication server in step st103. Further, the fingerprint input unit 303 reads the fingerprint of the user based on an operation by the user (for example, contact of the finger with the fingerprint input unit 303). The fingerprint input unit 303 identifies a user by comparing fingerprint data based on the read fingerprint with fingerprint data included in the fingerprint data list stored in the storage unit 306.

Note that, as described above, in the present embodiment, fingerprint authentication using FIDO (fido) will be described as an example of the second authentication, but the present invention is not limited to this. For example, instead of fingerprint authentication, authentication using a user ID and a password may be used as the second authentication.
When the user is identified, the control unit 300 digitally signs the challenge received by the communication unit 302 described above. Thereafter, the process proceeds to step st105.

(Step st105) The control unit 300 transmits the digitally signed challenge to the communication unit 401 of the second authentication server 40 via the communication unit 302. Thereafter, the process proceeds to step st106.

(Step st106) The communication unit 401 of the second authentication server 40 receives the digitally signed challenge transmitted from the terminal 30 in step st105. The communication unit 401 outputs the challenge with the digital signature to the fingerprint authentication processing unit 402 (second authentication unit). The fingerprint authentication processing unit 402 verifies the digital signature with respect to the input challenge with the digital signature. As described above, the public key and the secret key used in the signature and the signature verification are securely shared in advance between the terminal 30 and the second authentication server 40. Thereafter, the process proceeds to step st107.

(Step st107) The fingerprint authentication processing unit 402 (second authentication unit) transmits the data indicating the result of the verification of the digital signature performed in step st106 to the communication unit 302 of the terminal 30 via the communication unit 401. Thereafter, the process proceeds to step st108.

(Step st108) The communication unit 302 of the terminal 30 receives the data indicating the verification result of the digital signature transmitted from the second authentication server 40 in step st107.
If the verification result indicates that the digital signature is valid, the control unit 300 determines that the fingerprint authentication has been successful. Thereafter, the process proceeds to step st201 shown in FIG.
If the digital signature is a verification result indicating that it is not valid, the control unit 300 determines that the fingerprint authentication has not been successful. Thereafter, the process proceeds to step st405 shown in FIG.

(Step st201) The control unit 300 of the terminal 30 outputs, to the device detection unit 304, information indicating that fingerprint authentication has been determined to be successful. When the information is input, the device detection unit 304 detects a device identifier that identifies the nearby robot 10. As described above, the device detection unit 304 acquires information indicating a device identifier (for example, a Bluetooth (registered trademark) device address) that identifies the robot 10 via, for example, an antenna (not shown) provided in the terminal 30. I do. Thereafter, the process proceeds to step st202.

(Step st202) The device detection unit 304 causes the display unit 301 to display the acquired list of device identifiers. Thereafter, the process proceeds to step st203.

(Step st203) The user refers to the list of device identifiers displayed on the display unit 301, and selects a device identifier associated with the robot 10 facing the user. The device identifier associated with the robot 10 is known to the user in advance, or the device identifier is an identifier that allows the user to recognize that the device identifier is the device identifier associated with the robot 10. . Thereafter, the process proceeds to step st204.

(Step st204) The control unit 300 transmits the data indicating the device identifier and the data indicating the user ID selected by the operation input unit 305 by the user operation in step st203, and the first authentication server 20 (robot) via the communication unit 302. By transmitting the request to the communication unit 201 of the management server, a request for device authentication and user ID authentication is made. It is assumed that the user ID has been input by the user in advance. Thereafter, the process proceeds to step st205.

(Step st205) The communication unit 201 of the first authentication server 20 (robot management server) receives the data indicating the device identifier and the data indicating the user ID transmitted from the terminal 30 in step st204. The communication unit 201 outputs data indicating the device identifier to the device authentication unit 208. The communication unit 201 outputs data indicating the user ID to the user ID authentication unit 210.

  When the data is input, the device authentication unit 208 determines whether the same identifier as the device identifier based on the data exists in the identifier (the identifier for identifying the robot 10) stored in the authentication information storage unit 203. Check whether or not. Thereby, it is determined whether or not the device corresponding to the obtained device identifier is the robot 10 managed by its own first authentication server (robot management server). Thereafter, the process proceeds to step st206.

  In the above description, the user selects the device identifier associated with the robot 10; however, the present invention is not limited to this. For example, the control unit 300 may transmit the device identifier list acquired by the device detection unit 304 to the communication unit 201 of the first authentication server 20 (robot management server) via the communication unit 302. Then, in the device authentication unit 208 of the first authentication server 20 (robot management server), the device identifier associated with the robot 10 managed by its own first authentication server 20 is included in the list of device identifiers. Device authentication may be performed by verifying whether the device exists.

(Step st206) When the device identifier received in step st205 is an identifier associated with the robot 10 managed by its own first authentication server 20 (robot management server) (that is, device authentication has succeeded). If it is determined that the device authentication has been performed, the device authentication unit 208 outputs information indicating that the device authentication has been successful to the user ID authentication unit 210. Thereafter, the process proceeds to step st301 shown in FIG.
If the device identifier received in step st205 is not the identifier associated with the robot 10 managed by its own first authentication server 20 (robot management server) (that is, if the device authentication is successful) If not determined), the processing of this flowchart ends.

  The reason for ending this flowchart when the device authentication is not successful is that the user speaks to the device other than the robot 10 managed by the first authentication server 20 (robot management server) or transmits information to the device. This is because, for example, there is a possibility that the information is illegally stolen or the information is falsified by inputting.

(Step st301) When the information indicating that the device authentication is successful is input from the device authentication unit 208, the user ID authentication unit 210 of the first authentication server 20 (robot management server) receives the data input in step st205. It is determined whether the same user ID as the user ID based on the user ID exists in the user IDs stored in the authentication information storage unit 203. Thereby, it is determined whether the user corresponding to the acquired user ID is a user managed by the first authentication server of the user. Thereafter, the process proceeds to step st302.

(Step st302) If the same user ID as the user ID based on the data input in step st205 exists in the user ID stored in the authentication information storage unit 203, the user ID authentication unit 210 It is determined that the ID authentication has been successful. The user ID authentication unit 210 outputs information indicating that the user ID authentication has succeeded to the authentication data learning unit 211. Thereafter, the process proceeds to step st303.

  If the same user ID as the user ID based on the data input in step st205 does not exist in the user IDs stored in the authentication information storage unit 203, the user ID authentication unit 210 Is determined not to be successful. The user ID authentication unit 210 outputs a request for updating the authentication status data to the response control unit 207. Thereafter, the process proceeds to step st405 shown in FIG.

(Step st303) Based on the authentication status data update request input from the user ID authentication unit 210, the authentication status update unit 212 updates the authentication status data indicating that the user is authenticated. . The authentication status update unit 212 outputs to the authentication data learning unit 211, information indicating that the user has been updated to authentication status data indicating that the user has been authenticated. Thereafter, the process proceeds to step st304.

(Step st304) Upon acquiring the information output from the authentication state updating unit 212 in step st303, the authentication data learning unit 211 stores the face authentication data based on the user's face image received by the communication unit 201 in step st003 in the authentication information. The various face authentication databases stored in the unit 203 are stored (stored).
When acquiring the information output from the authentication state updating unit 212 in step st012, the authentication data learning unit 211 stores the voiceprint authentication data based on the user's voiceprint received by the communication unit 201 in step st010 in the authentication information storage unit 203. It is stored (stored) in the stored various voiceprint authentication databases. Thereby, the user's characteristics (for example, a face image and a voiceprint) are learned.
The authentication data learning unit 211 outputs, to the control unit 200, information indicating that the learning of the user characteristics described above has been completed. Thereafter, the process proceeds to step st305.

(Step st305) When information indicating that the learning of the characteristics of the user has been completed is input from the authentication data learning unit 211, the control unit 200 transmits information indicating that the device authentication and the user ID authentication have succeeded to the communication unit. The data is transmitted to the communication unit 302 of the terminal 30 via 201. When the communication unit 302 of the terminal 30 receives information indicating that the device authentication and the user ID authentication have been successful, the communication unit 302 may cause the display unit 301 to display the information, for example.
Thereafter, the process proceeds to step st306.

(Step st306) The control unit 200 outputs information indicating that the device authentication and the user ID authentication have been successful to the response control unit 207. When the information is input, the response control unit 207 selects (or generates) a response message corresponding to the user identified by the user ID authentication in step st301, and transmits data indicating the response message to the communication unit 201. Is transmitted to the communication unit 108 of the robot 10 via. Thereafter, the process proceeds to step st401 shown in FIG.

(Step st401) The communication unit 108 of the robot 10 receives data indicating a response message from the first authentication server 20. The data is appropriately converted by the data conversion unit 104 and then output to the audio output unit 105. The sound output unit 105 outputs a sound based on the data. As described above, a response (individual response) corresponding to the user is made. Thereafter, the process proceeds to step st402.

(Step st402) The imaging unit 101 of the robot 10 continues shooting while a response to the user is being made. The photographing may be continuous photographing, or may be photographing at fixed intervals (for example, at one-second intervals). The captured image is analyzed by the face detection unit 102, and a face image of the user is detected. Thus, not only the user's face image at the time of face authentication, but also the user's face image at the time of response is stored (stored) in the storage unit 106. As a result, more face images that can be used for the face authentication of the user can be collected, and the authentication accuracy in the face authentication of the user can be increased.

  The voice data based on the input from the voice input unit 103 of the robot 10 is transmitted to the first authentication server 20 (robot management server), and the voiceprint authentication unit 206 detects the voiceprint while the response to the user is being performed. Is continued. Note that the voiceprint detection process may be performed continuously, or may be performed at regular intervals (for example, at one-second intervals). As a result, not only the user's voice data at the time of voiceprint authentication but also the user's voice data at the time of response are stored (stored) in the storage unit 106. As a result, more voice data that can be used for the user's voiceprint authentication can be collected, and the authentication accuracy of the user's voiceprint authentication can be increased.

(Step st403) The face authentication data and the voice data acquired in step st402 are transmitted to the communication unit 201 of the first authentication server 20 (robot management server) via the communication unit 108. Thereafter, the process proceeds to step st404.

(Step st404) The communication unit 201 of the first authentication server 20 (robot management server) receives the face authentication data and the voice data acquired by the robot 10 at the time of the individual response. The communication unit 201 stores (accumulates) the received face authentication data in the face authentication database stored in the authentication information storage unit 203. Further, the authentication data learning unit 211 stores (accumulates) voiceprint authentication data indicating the voiceprint of the user based on the received voice data in the voiceprint authentication database stored in the authentication information storage unit 203. Thereby, the user's characteristics (for example, a face image and a voiceprint) are learned. This is the end of the processing of this flowchart.

(Step st405) The control unit 200 transmits information indicating that the user ID authentication has not been successful to the communication unit 302 of the terminal 30 via the communication unit 201. When the communication unit 302 of the terminal 30 receives information indicating that the user ID authentication has not been successful, for example, the communication unit 302 may cause the display unit 301 to display the information.
Thereafter, the process proceeds to step st406.

(Step st406) The control unit 200 outputs to the response control unit 207, information indicating that the user ID authentication has not been successful. When the information is input, the response control unit 207 selects (or generates) a response message corresponding to the unidentified user (anonymous user), and transmits data indicating the response message via the communication unit 201. To the communication unit 108 of the robot 10. Thereafter, the process proceeds to step st407.

(Step st407) The communication unit 108 of the robot 10 receives data indicating a response message from the first authentication server 20. The data is appropriately converted by the data conversion unit 104 and then output to the audio output unit 105. The sound output unit 105 outputs a sound based on the data. As described above, a response to the unidentified user (anonymous response) is made. This is the end of the processing of this flowchart.

As described above, the personal authentication system 1 according to the first embodiment of the present invention uses the first authentication for identifying a user by face authentication and the second authentication for identifying a user using a terminal. Personal authentication can be performed.
The personal authentication system 1 according to the present embodiment attempts to identify a user by the second authentication when the user cannot be identified by the first authentication. Even if the user cannot be identified by the first authentication, if the user can be identified by the second authentication, the personal authentication system 1 according to the present embodiment uses the data ( For example, it is possible to store the face authentication data and voice print authentication data acquired at the time of the first authentication, and the face authentication data and voice print authentication data acquired at the time of responding to the user. Thereby, in the personal authentication system 1 according to the present embodiment, as the number of times of authentication (opportunity) increases, valid data for the first authentication is accumulated, so that the accuracy of the authentication by the first authentication is increased. It is possible to go.

  This also allows the personal authentication system 1 according to the present embodiment to support the transition from the conventionally used authentication method (for example, the authentication method used in the second authentication) to the authentication method using face authentication. You can also.

  In the conventional face authentication technology, for example, the same person may be recognized as another person due to factors such as posture, lighting, facial expression, attached matter, and aging that change the individual or the environment. According to the personal authentication system 1 according to the present embodiment, by accumulating valid data for the first authentication, it is possible to reduce the influence of the occurrence of erroneous recognition due to these variable factors. . Thereby, the personal authentication system 1 according to the present embodiment can increase the authentication accuracy of the personal authentication (by the first authentication).

  Further, according to the personal authentication system 1 according to the present embodiment, not only the face authentication data and the voice print authentication data acquired at the time of responding to the user are stored, but also the face authentication data acquired periodically during the response. The personal authentication can be repeatedly performed during the response to the user by using the voiceprint authentication data. Thus, the personal authentication system 1 according to the present embodiment can respond to each user, for example, even when the users are replaced during the response. Further, the personal authentication system 1 according to the present embodiment, when a user is replaced in the middle of a response, gives a response to the user before the replacement to the user before the replacement, thereby providing the user before the replacement. It is also possible to prevent personal information from being leaked.

  In the personal authentication system 1 according to the present embodiment, the second authentication is performed by communication between the terminal 30 and an authentication device (second authentication server). Therefore, the personal authentication system 1 according to the present embodiment can perform authentication without passing through the robot 10, which is often installed in a highly public place. As a result, the personal authentication system 1 according to the present embodiment is also effective for preventing unauthorized access or the like aimed at the robot 10 that often has relatively low security.

(Second embodiment)
Hereinafter, a personal authentication system 1 according to the second embodiment will be described.
In the personal authentication system 1 according to the first embodiment described above, it is determined that the first authentication has succeeded only when all three of the face authentication, the keyword authentication, and the voiceprint authentication have succeeded. If at least one of the three authentications does not succeed, the second authentication is performed.
Further, in the personal authentication system 1 according to the first embodiment, it is determined that the second authentication is successful only when all of the three authentications of the fingerprint authentication, the device authentication, and the user ID authentication are successful. If at least one of the three authentications does not succeed, the personal authentication is not performed, and depending on the case, an anonymous response is performed or a response to the user is not performed.

The personal authentication system 1 according to the second embodiment described below has a simpler configuration than the personal authentication system 1 according to the first embodiment.
The personal authentication system 1 according to the second embodiment uses face authentication as the authentication method of the first authentication. The personal authentication system 1 according to the second embodiment uses authentication using a user ID and a password as an authentication method of the second authentication.

  First, the robot 10 starts photographing in front, and analyzes the photographed image. When detecting a portion presumed to be a face image of a person in the analyzed image, the robot 10 calculates a feature amount of the detected face image data, and transmits the calculated feature amount data to the first authentication server 20 (robot management server). Send to When the first authentication server 20 acquires the feature amount data, the first authentication server 20 compares the acquired feature amount data with the feature amount data for each user (a group of data acquired in the past) stored in its own first authentication server 20. Perform matching based on the degree of conformity.

  The first authentication server 20 determines that the first authentication is successful when there is a user whose ratio of the number of feature amount data having a high degree of conformity is equal to or more than a predetermined ratio, and the user responds to the robot 10. Identify that you are a user. It should be noted that the method of identifying a user when there are a plurality of users whose ratio of the number of pieces of feature amount data having a high degree of matching is equal to or higher than a predetermined ratio is arbitrary. For example, when there are a plurality of users whose ratio of the number of feature amount data having a high degree of matching is equal to or more than a predetermined ratio, the user having the highest ratio may be selected, or the user is not identified. May be used.

  If the first authentication is successful, the first authentication server 20 stores the feature amount data acquired in the first authentication, and uses the feature amount data in the next authentication. In addition, the first authentication server 20 performs a response (individual response) according to the identified user. Further, the first authentication server 20 also stores feature amount data based on a face image photographed at the time of a response, and is used for the next authentication.

  The first authentication server 20 determines that the first authentication has not been successful when there is no user whose ratio of the number of feature amount data having a high degree of matching is equal to or higher than a predetermined ratio. In that case, the first authentication server transmits, to the robot 10, data indicating a response message for causing the user to perform the second authentication. When receiving the data, the robot 10 displays a response message based on the data. The response message includes, for example, a content such as “Please input a user ID and password at a terminal”.

The user refers to the response message displayed on the robot 10 and inputs a user ID and a password at the terminal 30 owned by the user. Data indicating the input user ID and password is appropriately encrypted and transmitted to the second authentication server.
Upon receiving the data indicating the user ID and the password, the second authentication server 40 collates the received user ID and password with the user ID and password included in the customer database stored in the second authentication server 40 of the second authentication server 40. Performs the second authentication.

When the second authentication is successful, the first authentication server 20 stores the feature amount data acquired in the first authentication, and uses the feature data in the next authentication. In addition, the first authentication server 20 performs a response (individual response) according to the identified user. Further, the first authentication server 20 also stores feature amount data based on a face image photographed at the time of a response, and is used for the next authentication.
If the second authentication is not successful, the first authentication server 20 performs a user-independent response (anonymous response) because the user has not been identified.
This is the end of the description of the personal authentication system 1 according to the second embodiment of the present invention.

  Although the embodiments of the present invention have been described in detail above, the specific configuration is not limited to the above-described one, and various design changes and the like can be made without departing from the gist of the present invention. is there.

  Note that a part or all of the first authentication server 20 and the second authentication server 40 in the above-described embodiment may be realized by a computer. In this case, a program for realizing the control function may be recorded on a computer-readable recording medium, and the program recorded on the recording medium may be read and executed by a computer system.

  Here, the “computer system” is a computer system built in the first authentication server 20 and the second authentication server 40, and includes an OS and hardware such as peripheral devices. The “computer-readable recording medium” refers to a portable medium such as a flexible disk, a magneto-optical disk, a ROM, and a CD-ROM, and a storage device such as a hard disk built in a computer system.

  Further, the "computer-readable recording medium" is a medium that dynamically holds the program for a short time, such as a communication line when transmitting the program via a communication line such as a network such as the Internet or a telephone line, In this case, a program holding a program for a certain period of time, such as a volatile memory in a computer system serving as a server or a client, may be included. Further, the above-mentioned program may be for realizing a part of the above-mentioned functions, or may be for realizing the above-mentioned functions in combination with a program already recorded in a computer system.

  Further, the first authentication server 20 and the second authentication server 40 in the above-described embodiment may be realized as an integrated circuit such as an LSI (Large Scale Integration). Each functional block of the first authentication server 20 and the second authentication server 40 may be individually formed into a processor, or a part or all may be integrated into a processor. The method of circuit integration is not limited to an LSI, and may be realized by a dedicated circuit or a general-purpose processor. Further, in the case where a technology for forming an integrated circuit that replaces the LSI appears due to the advance of the semiconductor technology, an integrated circuit based on the technology may be used.

DESCRIPTION OF SYMBOLS 1 ... Personal authentication system, 10 ... Robot, 20 ... 1st authentication server (robot management server), 30 ... Terminal, 40 ... 2nd authentication server, 50 ... Communication line Reference numeral 60: communication line, 70: communication line, 80: communication line, 100: control unit, 101: imaging unit, 102: face detection unit, 103: voice input unit , 104: data conversion unit, 105: audio output unit, 106: storage unit, 107: display unit, 108: communication unit, 200: control unit, 201: communication Unit (acquisition unit), 202: face authentication unit (first authentication unit), 203: authentication information storage unit, 204: keyword authentication unit, 205: keyword generation unit, 206: voiceprint Authentication unit, 207: Response control unit, 208: Device authentication unit, 209 Response information storage unit, 210: user ID authentication unit, 211: authentication data learning unit, 212: authentication status update unit, 213: authentication cooperation unit, 300: control unit, 301 .. Display unit, 302 communication unit, 303 fingerprint input unit, 304 device detection unit, 305 operation input unit, 306 storage unit, 400 control unit, 401: communication unit, 402: fingerprint authentication processing unit (second authentication unit), 403: storage unit, 404: authentication cooperation unit

Claims (6)

A personal authentication system having a first authentication server and a second authentication server,
The first authentication server,
A storage unit that stores face authentication data based on the face image of the user for each user;
An acquisition unit that acquires face authentication data based on a face image of a target user for personal authentication,
A first authentication unit that performs first authentication including face authentication for identifying an individual based on a result of matching between the face authentication data obtained by the obtaining unit and the face authentication data stored in the storage unit;
With
The second authentication server,
If the first authentication is not successful, a second authentication unit that performs a second authentication that specifies the user based on an input from a terminal;
With
If the first authentication or the second authentication is successful, the storage unit stores the face authentication data obtained by the obtaining unit in addition to the already stored face authentication data.
Personal authentication system,
The face authentication is performed by comparing the face authentication data acquired by the acquisition unit with all of the at least one face authentication data stored in the storage unit, and identifying the person indicated by the face authentication data acquired by the acquisition unit. It is determined whether the person indicated by the face authentication data stored in the storage unit is the same person or not. If the ratio of the number of cases determined to be the same person is equal to or more than a predetermined ratio, the authentication is successful. Judge
A personal authentication system, characterized in that: A personal authentication system having a first authentication server and a second authentication server,
The first authentication server,
A storage unit that stores face authentication data based on the face image of the user for each user;
An acquisition unit that acquires face authentication data based on a face image of a target user for personal authentication,
A first authentication unit that performs first authentication including face authentication for identifying an individual based on a result of matching between the face authentication data obtained by the obtaining unit and the face authentication data stored in the storage unit;
With
The second authentication server,
If the first authentication is not successful, a second authentication unit that performs a second authentication that specifies the user based on an input from a terminal;
With
When the first authentication or the second authentication is successful, the storage unit stores the face authentication data obtained by the obtaining unit in addition to the already stored face authentication data.
Personal authentication system,
The first authentication unit repeatedly performs the first authentication based on the face authentication data acquired by the acquisition unit periodically;
A personal authentication system, characterized in that: A personal authentication system having a first authentication server and a second authentication server,
The first authentication server,
A storage unit that stores face authentication data based on the face image of the user for each user;
An acquisition unit that acquires face authentication data based on a face image of a target user for personal authentication,
A first authentication unit that performs first authentication including face authentication for identifying an individual based on a result of matching between the face authentication data obtained by the obtaining unit and the face authentication data stored in the storage unit;
With
The second authentication server,
If the first authentication is not successful, a second authentication unit that performs a second authentication that specifies the user based on an input from a terminal;
With
When the first authentication or the second authentication is successful, the storage unit stores the face authentication data obtained by the obtaining unit in addition to the already stored face authentication data.
Personal authentication system,
The personal authentication system further includes:
An imaging unit that captures a face image of the user;
A communication unit that transmits face authentication data based on the face image captured by the imaging unit to the first authentication server, and receives information indicating a response to the user from the first authentication server;
Having a robot with
The information input by the second authentication unit from the terminal includes an identifier for identifying the robot,
The second authentication server performs the second authentication without the intervention of the robot, and when the second authentication is successful, transmits information specifying the user to the robot specified by the identifier,
When the first authentication or the second authentication is successful, the storage unit further stores the face authentication data obtained by the obtaining unit while a response to the user is being performed,
A personal authentication system, characterized in that: A storage unit that stores face authentication data based on the face image of the user for each user;
An acquisition unit that acquires face authentication data based on a face image of a target user for personal authentication,
A first authentication unit that performs first authentication including face authentication for identifying an individual based on a result of matching between the face authentication data obtained by the obtaining unit and the face authentication data stored in the storage unit;
If the first authentication is not successful, a second authentication unit that performs a second authentication that specifies the user based on an input from a terminal;
With
If the first authentication or the second authentication is successful, the storage unit stores the face authentication data obtained by the obtaining unit in addition to the already stored face authentication data.
Personal authentication device,
The face authentication is performed by comparing the face authentication data acquired by the acquisition unit with all of the at least one face authentication data stored in the storage unit, and identifying the person indicated by the face authentication data acquired by the acquisition unit. It is determined whether the person indicated by the face authentication data stored in the storage unit is the same person or not. If the ratio of the number of cases determined to be the same person is equal to or more than a predetermined ratio, the authentication is successful. Judge
A personal authentication device, characterized in that: A first storage step in which the storage unit stores face authentication data based on a face image of the user for each user;
An acquiring step of acquiring face authentication data based on a face image of a target user for personal authentication,
A first authentication unit configured to perform a first authentication including a face authentication for identifying an individual based on a result of a comparison between the face authentication data obtained by the obtaining unit and the face authentication data stored by the storage unit; Steps and
If the first authentication is not successful, a second authentication unit performs a second authentication for specifying the user based on an input from a terminal;
If the first authentication or the second authentication is successful, the storage unit stores the face authentication data acquired by the acquisition unit in addition to the already stored face authentication data, a second storage step,
Have a,
The face authentication is performed by comparing the face authentication data acquired by the acquisition unit with all of the at least one face authentication data stored in the storage unit, and identifying the person indicated by the face authentication data acquired by the acquisition unit. It is determined whether the person indicated by the face authentication data stored in the storage unit is the same person or not. If the ratio of the number of cases determined to be the same person is equal to or more than a predetermined ratio, the authentication is successful. Judge
Personal authentication method which is characterized a call. On the computer,
A first storage step of storing face authentication data based on a user's face image for each user;
An acquisition step of acquiring face authentication data based on a face image of a target user for personal authentication,
A first authentication step of performing a first authentication including a face authentication for identifying an individual based on a comparison result between the face authentication data obtained in the obtaining step and the face authentication data stored in the first storage step;
A second authentication step of performing a second authentication for specifying the user based on an input from a terminal when the first authentication is not successful;
A second storage step of storing the face authentication data obtained in the obtaining step in addition to the already stored face authentication data when the first authentication or the second authentication is successful;
A program for executing,
The face authentication is performed by comparing the face authentication data acquired by the acquisition unit with all of the at least one face authentication data stored in the storage unit, and identifying the person indicated by the face authentication data acquired by the acquisition unit. It is determined whether the person indicated by the face authentication data stored in the storage unit is the same person or not. If the ratio of the number of cases determined to be the same person is equal to or more than a predetermined ratio, the authentication is successful. Judge
program.

Images