JP2020087093A - Vending machine settlement system, vending machine, face authentication server, vending machine settlement method, and program - Google Patents

Abstract

To provide a vending machine settlement system capable of performing quick settlement based on face authentication.SOLUTION: A face authentication server 101 holds first and second feature vectors generated from a customer ID and a face image, and performs face authentication. A vending machine 103 generates a third feature vector corresponding to the first feature vector from the face image of a customer. The face authentication server extracts the first feature vector stored in a database, that is, the first feature vector having higher similarity to the third feature vector than a predetermined threshold. The face authentication server generates a list composed of the customer ID corresponding to the extracted first feature vector as a face authentication candidate. The vending machine generates a fourth feature vector from the face image of the customer who tries to purchase merchandise. The face authentication server performs face authentication of the customer who tries to purchase the merchandise on the basis of the fourth feature vector and the second feature vector corresponding to the customer ID included in the face authentication candidate.SELECTED DRAWING: Figure 1

Description

The present invention relates to a vending machine settlement system, a vending machine, a face authentication server, a vending machine settlement method and a program.

In recent years, vending machines have become highly functional. For example, vending machines that are compatible with not only ordinary payment using cash (purchase of goods) but also payment using an IC (Integrated Circuit) card have become popular. There is also a system that connects a vending machine to a server via a terminal such as a smartphone and provides various services to customers through the server. For example, Patent Document 1 discloses an automatic vending machine that gives points according to a customer's product purchase.

JP, 2017-220031, A

Vending machines are often installed not only in stations and commercial facilities but also in workplaces and schools. Many people at work do not always carry cash such as wallets or IC cards, and if such a person wants to purchase a product at a vending machine, he or she needs to bring out the wallet and go to the vending machine. However, some people think that it is a hassle to bring out a wallet or the like, and there is a desire to purchase a product by hand. In order to meet such a demand, it is possible to incorporate a face recognition function into a system including a vending machine.

However, if the face authentication function is simply incorporated, the time from the product selection by the customer to the completion of payment becomes long, and the convenience of the vending machine is reduced. That is, it takes a lot of processing time to specify a customer to be settled from a database having a huge amount of data (customer identifier and authentication information).

In other words, the customer has to wait in front of the vending machine for a long time from the selection of the product until the settlement is completed, and such a coercion to the customer is "I want to purchase the product quickly". It is against the customer's intention and is not acceptable.

Therefore, it is desired to provide an automatic vending machine payment system, an automatic vending machine, a face authentication server, an automatic vending machine payment method, and a program that contribute to enabling quick payment by face authentication.

According to a first aspect of the present invention or disclosure, a database that stores a customer ID (Identifier) and first and second feature vectors generated from a face image of a person corresponding to the customer ID in association with each other. Access to and perform face authentication of the requested person, perform face authentication server and payment related to product purchase, request payment of face authentication from the payment server and the face authentication server, and authentication by the face authentication server is successful. A vending machine that requests the settlement server to settle a product purchased by the customer, and the vending machine detects the customer present in the vicinity of the device, and detects the face of the customer. A third feature vector corresponding to the first feature vector is generated from an image, and the face authentication server is the first feature vector stored in the database, and is between the third feature vector and the third feature vector. Of the first feature vector whose similarity is higher than a predetermined threshold value is generated, and a list of the customer IDs corresponding to the extracted first feature vector is generated as a face authentication candidate. , A fourth feature vector is generated from a face image of a customer who intends to purchase a product at a timing different from the timing of detecting the customer, and the face authentication server is configured to generate the fourth feature vector and the face authentication candidate. An automatic vending machine payment system is provided, which performs face authentication of a customer who intends to purchase the product based on the second feature vector corresponding to the customer ID included in.

According to a second aspect of the present invention and the disclosure, a database that stores a customer ID (Identifier) and first and second feature vectors generated from a face image of a person corresponding to the customer ID in association with each other. Is connected to a face authentication server that performs face authentication of a requested person and performs payment related to product purchase, and requests face authentication to the face authentication server. A device that requests the settlement server to settle a product purchased by a customer who has succeeded in authentication, and in response to detecting a customer present in the vicinity of the device, the first image is detected from the face image of the customer. Generating a third feature vector corresponding to the feature vector, transmitting the third feature vector to the face authentication server, and causing the face authentication server to store the first feature vector stored in the database. A list of the customer IDs corresponding to the extracted first feature vector, the first feature vector having a similarity with the third feature vector higher than a predetermined threshold is extracted. Is generated as a face authentication candidate, and a fourth feature vector is generated from a face image of a customer who wants to purchase a product at a timing different from the timing of detecting the customer, and , Requesting face authentication of a customer who intends to purchase the product, based on the fourth feature vector and the second feature vector corresponding to the customer ID included in the face authentication candidate, A vending machine is provided.

According to a third aspect of the present invention or the disclosure, a database that stores a customer ID (Identifier) and first and second feature vectors generated from a face image of a person corresponding to the customer ID in association with each other. Between the first feature vector stored in the database and the third feature vector generated by the vending machine, the server performing the face authentication of the requested person. A first feature vector whose degree of similarity is higher than a predetermined threshold value is extracted, a list including the customer ID corresponding to the extracted first feature vector is generated as a face authentication candidate, and the vending machine sells the product. A customer who intends to purchase the product based on a fourth feature vector generated from the face image of the customer who wants to purchase the product and the second feature vector corresponding to the customer ID included in the face authentication candidate. A face authentication server for performing face authentication is provided.

According to a fourth aspect of the present invention or the disclosure, a database that stores a customer ID (Identifier) and first and second feature vectors generated from a face image of a person corresponding to the customer ID in association with each other. Access to and perform face authentication of the requested person, perform face authentication server and payment related to product purchase, request payment of face authentication from the payment server and the face authentication server, and authentication by the face authentication server is successful. In a vending machine settlement system including an vending machine that requests the settlement server to settle a product purchased by the customer, the face of the detected customer is detected in response to detecting a customer existing in the vicinity of the own device. Generating a third feature vector corresponding to the first feature vector from an image, and calculating the similarity between the first feature vector stored in the database and the third feature vector. Extracting a first feature vector higher than a predetermined threshold value, generating a list of the customer IDs corresponding to the extracted first feature vector as a face authentication candidate, and detecting the customer. Generating a fourth feature vector from a face image of a customer who intends to purchase a product at different timings, the fourth feature vector, and the second feature vector corresponding to the customer ID included in the face authentication candidate. And a step of performing face authentication of a customer who intends to purchase the product based on the feature vector.

According to a fifth aspect of the present invention or disclosure, a database that stores a customer ID (Identifier) and first and second feature vectors generated from a face image of a person corresponding to the customer ID in association with each other. Is connected to a face authentication server that performs face authentication of a requested person and performs payment related to product purchase, and requests face authentication to the face authentication server. The face of the detected customer is detected by the computer mounted in the vending machine, which requests the settlement server to settle the product purchased by the customer who has succeeded in authentication, in response to the detection of the customer around the device. A process of generating a third feature vector corresponding to the first feature vector from an image, transmitting the third feature vector to the face authentication server, and storing the third feature vector in the database in the face authentication server. Of the extracted first feature vector, the first feature vector having a similarity between the third feature vector and the third feature vector higher than a predetermined threshold value, and corresponding to the extracted first feature vector. And a process of requesting to generate a list of the customer IDs as face authentication candidates, and a fourth feature vector is generated from a face image of a customer who wants to purchase a product at a timing different from the timing of detecting the customer. And the second feature vector corresponding to the customer ID included in the face authentication candidate, with respect to the face authentication server, based on the processing A process for requesting to perform face recognition of a customer and a program for executing the process are provided.
Note that this program can be recorded in a computer-readable storage medium. The storage medium may be a non-transient one such as a semiconductor memory, a hard disk, a magnetic recording medium, an optical recording medium, or the like. The present invention can also be embodied as a computer program product.

According to the respective aspects of the present invention and the disclosure, there are provided an automatic vending machine payment system, an automatic vending machine, a face authentication server, an automatic vending machine payment method and a program, which contribute to enabling quick payment by face authentication. To be done.

It is a figure for explaining the outline of one embodiment. It is a figure which shows an example of the vending machine payment system which concerns on 1st Embodiment. It is a sequence diagram which shows an example of operation|movement of the vending machine payment system which concerns on 1st Embodiment. It is a figure which shows an example of the outline of the vending machine which concerns on 1st Embodiment. It is a figure which shows an example of the hardware constitutions of the vending machine which concerns on 1st Embodiment. It is a figure which shows an example of a process structure of the vending machine which concerns on 1st Embodiment. It is a figure for demonstrating operation|movement of a face image extraction part. It is a figure for demonstrating operation|movement of a face image extraction part. It is a figure for explaining operation of a feature vector generation part. It is a figure for explaining operation of a feature vector generation part. It is a figure which shows an example of the hardware constitutions of the face authentication server which concerns on 1st Embodiment. It is a figure which shows an example of a process structure of the face authentication server which concerns on 1st Embodiment. It is a figure which shows an example of the information stored in the internal database of the face authentication server which concerns on 1st Embodiment. It is a sequence diagram which shows an example of operation|movement of the vending machine payment system which concerns on 1st Embodiment. It is a sequence diagram which shows an example of operation|movement of the vending machine payment system which concerns on 1st Embodiment. It is a sequence diagram which shows an example of operation|movement of the vending machine payment system which concerns on 1st Embodiment. It is a figure for demonstrating operation|movement of the vending machine which concerns on 1st Embodiment. It is a figure for demonstrating operation|movement of the vending machine which concerns on 1st Embodiment. It is a sequence diagram which shows an example of another operation|movement of the vending machine payment system which concerns on 1st Embodiment. It is a figure for demonstrating operation|movement of the vending machine which concerns on 1st Embodiment. It is a figure for demonstrating operation|movement of the face image extraction part which concerns on 2nd Embodiment. It is a figure for demonstrating operation|movement of the face authentication server which concerns on 2nd Embodiment. It is a figure for demonstrating operation|movement of the payment control part which concerns on 3rd Embodiment. It is a figure showing an example of another outline of a vending machine.

First, an outline of one embodiment will be described. Note that the reference numerals attached to the outline are added to the respective elements for convenience as an example for facilitating understanding, and the description of the outline is not intended to limit the invention. The connecting lines between blocks in each figure include both bidirectional and unidirectional. The unidirectional arrows schematically show the flow of main signals (data), and do not exclude bidirectionality. Further, although not explicitly shown in the circuit diagram, block diagram, internal configuration diagram, connection diagram, etc. disclosed in the present disclosure, an input port and an output port exist at each of the input end and the output end of each connection line. The same applies to the input/output interface.

The vending machine payment system according to one embodiment includes a face authentication server 101, a payment server 102, and a vending machine 103 (see FIG. 1 ). The face authentication server 101 accesses the database that stores the customer ID (Identifier) and the first and second feature vectors generated from the face image of the person corresponding to the customer ID in association with each other, and requests the requested person. Face recognition. The payment server 102 makes payment related to product purchase. The vending machine 103 requests the face authentication server 101 for face authentication, and requests the payment server 102 to settle a product purchased by a customer whose authentication by the face authentication server 101 has succeeded. The vending machine 103 generates a third feature vector corresponding to the first feature vector from the detected face image of the customer in response to detecting the customer existing around the device. The face authentication server 101 extracts a first feature vector stored in the database and having a degree of similarity with the third feature vector higher than a predetermined threshold value. Further, the face authentication server 101 generates a list of customer IDs corresponding to the extracted first feature vector as a face authentication candidate. The vending machine 103 generates the fourth feature vector from the face image of the customer who wants to purchase the product at a timing different from the timing of detecting the customer. The face authentication server 101 performs face authentication of a customer who intends to purchase a product based on the fourth feature vector and the second feature vector corresponding to the customer ID included in the face authentication candidates.

When the vending machine 103 recognizes the customer in front of its own device, the vending machine 103 generates a third feature vector (a general feature vector described later) from the face image of the recognized customer. The face authentication server 101 uses the third feature vector transmitted from the vending machine 103, and features similar to the customers existing in front of the vending machine 103 among the huge number of customers stored in the database. Customers in advance. For the narrowing down, a first feature vector (narrowing down feature vector described later) corresponding to the third feature vector generated from the face image of the customer is used. After that, the vending machine 103 generates a fourth feature vector (detailed feature vector described later) from the face image of the customer who intends to purchase the product. The face authentication server 101 treats a customer included in the face authentication candidates generated previously as an authentication target. The face authentication server 101 uses a second feature vector (authentication feature vector described later) that characterizes the face of the customer included in the face authentication candidates in detail, and a fourth feature vector acquired from the vending machine 103. Perform face recognition.

As described above, in the vending machine payment system shown in FIG. 1, when the vending machine 103 recognizes a customer (future customer), the feature vector characterizing the face of the customer is transmitted to the face authentication server 101, and the face authentication server 101 Target customers who will be authenticated in advance. After that, when the customer actually indicates the intention to purchase the product, the vending machine 103 transmits a feature vector characterizing the face of the customer in more detail to the face authentication server 101 to request execution of face authentication. Since the face authentication server 101 only needs to identify (authenticate) a customer who intends to purchase a product from among customers who have been narrowed down in advance, quick settlement by face authentication can be realized.

Hereinafter, specific embodiments will be described in more detail with reference to the drawings. In addition, in each embodiment, the same components are denoted by the same reference numerals, and the description thereof will be omitted.

[First Embodiment]
The first embodiment will be described in more detail with reference to the drawings.

FIG. 2 is a diagram illustrating an example of the vending machine payment system according to the first embodiment. Referring to FIG. 2, the vending machine payment system includes vending machines 10-1 to 10-N (N is a positive integer, the same applies hereinafter), a face authentication server 20, and a payment server 30. To be done.

In the following description, if there is no special reason for distinguishing the vending machines 10-1 to 10-N, they are simply referred to as "vending machine 10".

The vending machine 10 is a device for selling products such as beverages unattended. The vending machine 10 supports payment by face authentication or the like in addition to existing payment means using cash and an IC card.

The face authentication server 20 is a device that authenticates a customer who wants to purchase a product at the vending machine 10. The face authentication server 20 processes requests for face authentication from a plurality of vending machines 10 included in the system. The face authentication server 20 stores a previously registered identifier (ID; IDentifier) for identifying a customer and a feature amount (feature vector) extracted from the face image of the customer in association with each other. There are two types of feature vectors registered in the face authentication server 20, and details of each feature vector will be described later.

The payment server 30 is a device that performs payment related to product purchases by customers. More specifically, the payment server 30 performs a payment process related to the purchase of the product of the customer whose validity is confirmed by the face authentication. The payment server 30 stores a customer identifier (ID) registered in advance and information used for payment (for example, information about a credit card) in association with each other.

Note that a customer who uses the vending machine payment system shown in FIG. 2 transmits his or her face image to the operating company of the vending machine 10 or the like, and sends the feature vector extracted from the face image to the face authentication server 20 in advance. You need to register. Similarly, the customer needs to contact the operating company or the like with the card information used for the payment and register the card information in the payment server 30 in advance. Further, although not particularly limited, it is desirable that the customer IDs registered in the face authentication server 20 and the settlement server 30 are common. This is because by using the same customer ID in the two servers, mutual conversion of customer IDs is unnecessary.

Such information registration can take various forms such as registration using the website of the operating company, registration using a terminal such as a smartphone, and registration by sending a document in which necessary information is entered. Further, these registration processes are obvious to those skilled in the art, and detailed description thereof will be omitted.

The feature vector registered in the face authentication server 20 needs to match the information (feature vector) transmitted from the vending machine 10. A description of this point will be given later.

The configuration shown in FIG. 2 is an example, and is not intended to limit the configuration of the system. For example, the vending machine settlement system may include a vending machine that does not support face authentication. Alternatively, one server device having the functions of the face authentication server 20 and the payment server 30 may be prepared. Alternatively, the system may include a database server or the like that manages a database used by the face authentication server 20 or the like.

[General operation of the system]
Subsequently, a schematic operation of the vending machine payment system according to the first embodiment will be described with reference to FIG.

FIG. 3 is a sequence diagram showing an example of the operation of the vending machine payment system according to the first embodiment. The sequence diagram shown in FIG. 3 shows the operation when the customer selects the face authentication as the settlement means when purchasing the product. Details of operations including payment by cash will be described later.

The vending machine 10 detects a person when a "person" appears around the device (step S01).

When the "person" is detected, the vending machine 10 sets the operation mode of its own device to the "selection mode". In addition, the selection mode is a mode in which it is assumed that a person (customer) who exists in the vicinity of the vending machine 10 selects a product to be purchased (is trying to purchase the product).

The operation mode before transitioning to the selection mode is the “standby mode”. The standby mode is a mode of waiting for a customer to appear in front of the vending machine 10.

The automatic vending machine 10 in the selection mode acquires face images of persons existing in the vicinity and generates a feature vector from the face images (step S02).

When the generation of the feature vector is completed, the vending machine 10 transmits a “narrowing down request” to the face authentication server 20. At that time, the vending machine 10 includes the ID of its own device and the generated feature vector in the narrowing-down request and transmits it to the face authentication server 20.

Hereinafter, the feature vector transmitted together with the narrowing-down request will be referred to as a “schematic feature vector”. The general feature vector is a vector (a set having a plurality of feature amounts as an element) that characterizes a rough feature of a face image, and has a characteristic of low accuracy when used for face authentication.

The face authentication server 20 receives the narrowing-down request and acquires a rough feature vector. The face authentication server 20 uses the acquired outline feature vector and the feature vector stored in the internal database (database that associates the customer ID with the feature vector) to generate a "face authentication candidate" (step S11).

The face authentication candidate is a feature vector registered in the internal database, and a customer (customer ID corresponding to a feature vector whose similarity to the general feature vector acquired from the vending machine 10 is higher than a predetermined threshold value). ) Is a list of. More specifically, the face authentication server 20 extracts a small number of customers who are similar to the acquired general feature vector (similar in face) from the large number of customers registered in the internal database, and extracted the extracted customers. Set the customer as a "candidate for face authentication processing". Note that a plurality of customers (feature vectors) are usually extracted from the above-described characteristics (authentication accuracy is low) of the general feature vector.

The vending machine 10 determines whether or not the customer has selected a product, in parallel with the generation processing regarding the “face authentication candidate” of the face authentication server 20. Specifically, the vending machine 10 detects a product selection operation by a customer (for example, pressing a button associated with a product sample) (step S03).

When the customer purchase operation is detected, the vending machine 10 performs a display for the customer to select a payment method (displays a payment selection screen; step S04). For example, the vending machine 10 displays a message prompting the customer to select either payment using an IC card or face authentication. Here, it is assumed that the payment by “face authentication” is selected.

When face settlement is selected, the vending machine 10 sets the operation mode of its own device to the “authentication mode”. The authentication mode is a mode in which a customer performs face authentication for completing product purchase.

The vending machine 10 in the authentication mode acquires the face image of the customer who has selected the product and generates a feature vector from the face image (step S05).

When the generation of the feature vector is completed, the vending machine 10 transmits an “authentication request” to the face authentication server 20. At that time, the vending machine 10 includes the ID of its own device and the generated feature vector in the authentication request, and transmits the authentication request to the face authentication server 20.

Hereinafter, the feature vector transmitted together with the authentication request will be referred to as “detailed feature vector”. The detailed feature vector is a vector that characterizes detailed features of the face image, and has a characteristic of high accuracy when used for face authentication.

The face authentication server 20 receives the authentication request and acquires the detailed feature vector. The vending machine 10 uses the feature vector corresponding to each customer included in the face authentication candidates generated previously and the acquired detailed feature vector to correspond to the detailed feature vector transmitted from the face authentication server 20. The person is authenticated (step S12).

The face authentication server 20 transmits the authentication result to the vending machine 10. Specifically, the face authentication server 20 determines that the person corresponding to the detailed feature vector transmitted from the vending machine 10 (person to be authenticated) has the proper authority (when accepting the proof of the person to be authenticated). , "Authentication successful" is transmitted as the authentication result.

On the other hand, the face authentication server 20 transmits “authentication failure” as the authentication result when the person to be authenticated does not have the proper authority (when rejecting the authentication of the person to be authenticated). It should be noted that the face authentication server 20 also transmits the ID of the customer who has succeeded in the authentication at least when transmitting the authentication result related to the successful authentication to the vending machine 10.

When the authentication result of “successful authentication” is acquired from the face authentication server 20, the vending machine 10 specifies the customer ID authenticated by the face authentication server 20 and relates to the purchase of the product by the customer with respect to the payment server 30. A payment process is requested (step S06).

When the payment server 30 acquires the payment information (customer ID, payment amount), the payment server 30 executes payment for the customer with the specified ID and notifies the vending machine 10 of the result (step S21).

When the vending machine 10 receives the notification of “successful payment is completed” from the payment server 30, the vending machine 10 discharges the selected product (step S07). At this time, the vending machine 10 sets the operation mode of its own device to the “standby mode” because the settlement by face authentication is completed, and prepares for the purchase of a product by a new customer.

Next, details of each device that constitutes the vending machine payment system will be described.

[Appearance of vending machines]
FIG. 4 is a diagram showing an example of an outline of the vending machine 10. As shown in FIG. 4, the vending machine 10 includes a product sample 11 and a selection button 12 for selecting a product. The customer selects a desired product by pressing the selection button 12.

The vending machine 10 supports a plurality of types of payment means. Specifically, the vending machine 10 supports payment by cash, an IC card, and face authentication. Note that the payment means is an example, and for example, the vending machine 10 may not support payment using an IC card.

A customer who wishes to make a payment by cash inserts an amount equal to or more than the price of the desired product through the cash slot 13, and presses the selection button 12 after the desired product can be selected.

On the other hand, a customer who wishes to make a payment using an IC card or face authentication makes a payment by any means after pressing the selection button 12 corresponding to the desired product. At this time, the vending machine 10 displays to select whether the IC card payment or the face authentication payment is desired, and performs the payment process according to the customer's selection.

Specifically, when the customer wishes to pay by IC card, the customer brings the IC card closer to the IC card reader 14 and completes the payment. If the customer desires to pay by face authentication, the customer does not perform any special operation and waits until the settlement by face authentication is completed. However, since the display regarding the acquisition of the face image is displayed on the liquid crystal panel 15 (display means) as necessary, the customer completes the settlement by the face authentication according to the displayed instruction.

When the payment by cash, IC card or face authentication is normally completed, the product is discharged from the discharge port 16.

Although FIG. 4 shows the appearance of the vending machine of the type displaying the product sample 11, it goes without saying that the vending machine of the type displaying the selection button corresponding to the product sample on the liquid crystal panel may be used. Is.

[Hardware configuration of vending machines]
FIG. 5: is a figure which shows an example of the hardware constitutions of the vending machine 10 which concerns on 1st Embodiment. The vending machine 10 has the configuration illustrated in FIG. For example, the vending machine 10 includes a CPU (Central Processing Unit) 21, a memory 22, a display module 23, a camera module 24, a wireless communication module 26 connected to an antenna 25, and the like, which are mutually connected by an internal bus.

The configuration shown in FIG. 5 is not intended to limit the hardware configuration of the vending machine 10. The vending machine 10 also includes hardware (not shown). Specifically, FIG. 5 shows hardware for realizing the function of the vending machine 10 as an automatic vending machine (storage means for storing products, means for discharging a product selected by the customer, products The means for processing the purchase price, etc.) are not shown. These hardware can use the means used in the existing vending machine.

The vending machine 10 accesses the network via the wireless communication module 26 and communicates with the face authentication server 20 and the payment server 30. The vending machine 10 may be connected to a network by wire such as Ethernet (registered trademark) instead of wireless communication.

The memory 22 is a RAM (Random Access Memory), a ROM (Read Only Memory), an HDD (Hard Disk Drive), or the like.

The display module 23 is configured to include the liquid crystal panel 15. The display module 23 receives display data from the CPU 21, and drives the liquid crystal panel 15 using the display data to provide information to the customer. The display module 23 also includes an operation device such as a touch panel. The operation module detects the customer's operation on the touch panel or the like, and outputs the result (for example, the customer's pressed coordinate) to the CPU 21 as operation data.

Note that a GUI (Graphical User Interface) can be constructed by integrally configuring the liquid crystal panel 15 and the touch panel. The vending machine 10 according to the first embodiment will be described assuming that the liquid crystal panel 15 and the touch panel are integrally configured.

The camera module 24 is configured to include an optical system component such as a lens and an electronic system component such as a light receiving element and a memory. The camera module 24 is configured to take a picture around the vending machine 10 in a predetermined cycle and output the taken image data to the CPU 21. Alternatively, the CPU 21 may instruct the camera module 24 to periodically acquire the imaging data. The camera of the vending machine 10 is installed so that at least the front surface including the face of the customer who wants to purchase the product can be imaged.

Each processing module of the vending machine 10 described later is realized by the CPU 21 executing a program stored in the memory 22, for example. In addition, the program can be downloaded via a network or updated using a storage medium storing the program. Further, the processing module may be realized by a semiconductor chip. That is, it is sufficient that there is a means for executing the function performed by the processing module with some hardware and/or software.

[Processing configuration of vending machines]
The vending machine 10 has a function of requesting face authentication from the face authentication server 20 and requesting the payment server 30 to settle a product purchased by a customer who has been successfully authenticated by the face authentication server 20. The vending machine 10 includes the following processing module to realize the function.

FIG. 6 is a diagram illustrating an example of a processing configuration of the vending machine 10 according to the first embodiment. Referring to FIG. 6, the vending machine 10 includes a communication control unit 301, a vending machine control unit 302, a settlement control unit 303, a face image extraction unit 304, a feature vector generation unit 305, and a storage unit 306. , And are included.

The communication control unit 301 is means for controlling communication with other devices (face authentication server 20, payment server 30).

The vending machine control unit 302 is means for controlling (implementing) the function of the vending machine. For example, the vending machine control unit 302 realizes control of the selection button 12 (lighting or extinguishing of the button), payment by an IC card, discharge of products for which payment is completed, and the like.

The payment control unit 303 is a unit that mainly controls the vending machine 10 relating to face payment of a customer. Specifically, the settlement control unit 303 manages the state transition regarding the above-mentioned three operation modes (standby mode, selection mode, authentication mode).

Further, the payment control unit 303 controls the display module 23 to display the above-described payment selection screen and the like on the liquid crystal panel 15 and provide the GUI to the customer.

Furthermore, the settlement control unit 303 acquires image data (image pickup data) obtained by picking up an image of the situation reflected on the lens from the camera module 24. When the payment control unit 303 acquires the image data from the camera module 24, the payment control unit 303 delivers the image data to the face image extraction unit 304.

The face image extraction unit 304 is a unit that extracts a face image (a region corresponding to a human face) from the acquired image data.

For example, as disclosed in Japanese Unexamined Patent Application Publication No. 2014-170979 (reference 1), the face image extraction unit 304 stores the acquired image data (data including the face image) and the template image of the face image. Compare. After that, the face image extraction unit 304 determines whether or not a face image is present in the acquired image data depending on whether or not the difference between the two is less than or equal to a threshold value.

When a face image exists in the acquired image data, the face image extraction unit 304 cuts out the area of the face image and extracts the face image.

Various methods are conceivable for extracting the face image, and it is needless to say that the method is not limited to the above. For example, a model in which color information, edge directions, and densities are combined is stored in advance, and when a region similar to the model is detected from the input frame, it is determined that a face image exists, and the face image may be extracted. it can.

Furthermore, it is also possible to extract a face image using a template created by utilizing the fact that the contour of the face (head) is elliptical and the eyes and mouth are rectangular. Furthermore, a face detection method that uses the characteristics of the brightness distribution that the cheeks and forehead parts have high brightness and the eyes and mouth parts have low brightness, and face detection using the symmetry of the face and the skin color area and position. You may use the method etc. which perform.

Alternatively, a method of statistically learning the feature amount distribution obtained from a large number of face and non-face learning samples and determining whether the feature amount obtained from the input image belongs to the face or non-face distribution is used. May be. That is, a technique related to machine learning such as deep learning or support vector machine may be used to extract the face image.

The face image extraction unit 304 switches the face image extraction operation according to the operation mode of the vending machine 10.

When the operation mode is the “standby mode”, the face image extraction unit 304 operates so as to extract as many face images as possible from the acquired image data. For example, when applying the technique of Reference 1, the face image extraction unit 304 sets a low threshold value used for presence/absence of a face image so that many images are extracted as face images.

More specifically, even when the customer is not directly facing the vending machine 10, or when the customer is located at a distance from the vending machine 10 and the area assumed to be a face image is small, The face image extraction unit 304 tries to extract a face image from the image data.

For example, it is assumed that the face image extraction unit 304 has acquired the image data shown in FIG. 7. In this case, the face image extraction unit 304 extracts a region surrounded by a dotted line from each of the image data of FIGS. 7A and 7B as a face image. That is, not only the face image of an appropriate size as shown in FIG. 7A, but also the face image of a small size as shown in FIG. 7B is extracted.

When the face image extraction unit 304 succeeds in extracting the face image, the face image extraction unit 304 stores the extracted face image in the storage unit 306 and notifies the settlement control unit 303 of the fact (success of face image extraction).

When the face image extraction unit 304 extracts at least one face image in the “standby mode”, the payment control unit 303 determines that there is a customer in front of the own device and sets the operation mode to the “selection mode”. In this way, the vending machine 10 determines that there is a customer in the vicinity of the own device when the face image is included in the image data captured in the standby mode.

When the operation mode is the “authentication mode”, the face image extraction unit 304 operates so as to extract a face image that satisfies a predetermined condition. For example, the face image extraction unit 304 determines that the predetermined condition is satisfied when the size of the face image of the customer who wants to purchase the product is larger than the predetermined size. More specifically, the face image extraction unit 304 performs threshold processing on the area (image size) of the extracted face image to extract a face image having a predetermined area or more.

For example, it is assumed that the face image extraction unit 304 has acquired the two image data shown in FIG. In this case, the face image extraction unit 304 determines that the face image shown in FIG. 8A has a predetermined area or more and extracts it. On the other hand, the face image extraction unit 304 determines that the face image shown in FIG. 8B is smaller than the predetermined area and does not extract it. When the face image extraction unit 304 succeeds in extracting the face image satisfying the predetermined condition, the face image extraction unit 304 stores the extracted face image in the storage unit 306 and notifies the settlement control unit 303 of the fact.

When the face image is extracted but the face image does not satisfy the predetermined condition, the face image extraction unit 304 notifies the settlement control unit 303 of that fact. For example, the face image extraction unit 304 notifies that the size of the face image has been extracted but is smaller than the predetermined size.

The feature vector generation unit 305 is means for generating a feature vector from the face image extracted by the face image extraction unit 304. The feature vector generation unit 305 switches the feature vector generation processing according to the operation mode of the vending machine 10.

More specifically, when the operation mode is the “selection mode”, the feature vector generation unit 305 generates the “outline feature vector”. When the operation mode is the “authentication mode”, the feature vector generation unit 305 generates the “detailed feature vector”.

The feature vector generation unit 305 normalizes the face image when generating the feature vector. The face images extracted by the face image extraction unit 304 have inconsistent sizes and sizes depending on the environment and conditions at the time of image data acquisition. The feature vector generation unit 305 normalizes the image by performing enlarging/reducing processing so that the face image has a predetermined size, and averaging the brightness.

In the normalized face image, for example, as shown in FIG. 9, feature points A1 and A2 having endpoints of both eyes as feature points, feature points A3 and A4 having endpoints of mouth as feature points, and nose head are featured. A characteristic point A5 that is a point is included. The feature vector generation unit 305 can generate a feature vector by extracting these feature points from the face image using a technique such as template matching and arranging the coordinates (X coordinate, Y coordinate) of the feature points.

At that time, the feature vector generation unit 305 switches the feature points to be selected according to the operation mode. For example, it is assumed that the coordinates of each of the characteristic points A1 to A5 are as shown in FIG.

When the operation mode is the “selection mode”, the feature vector generation unit 305 generates the “outline feature vector” having the positions (X coordinates, Y coordinates) of the endpoints (feature points A1 and A2) of both eyes as elements. To do. The feature vector generated in this case is a four-dimensional vector represented by the following equation (1).
FV={X1, Y1, X2, Y2} (1)

When the operation mode is the “authentication mode”, the feature vector generation unit 305 generates a “detailed feature vector” having the positions of the above five feature points as elements. The feature vector generated in this case is a 10-dimensional vector represented by the following equation (2).
FV={X1, Y1, X2, Y2,..., X5, Y5} (2)

The operation of the feature vector generation unit 305 is merely an example, and it goes without saying that the generation of the two feature vectors is not limited to the above. That is, the two feature vectors generated by the feature vector generation unit 305 may be any as long as they have the above-described features (the rough feature vector has low accuracy; the detailed feature vector has high accuracy).

Here, a feature vector with a low degree is usually generated quickly but has low accuracy when used for authentication, and a feature vector with a high degree requires a long time to generate but has high accuracy when used for authentication. According to the fact, the feature vector generation unit 305 switches between generation of a high-order feature vector (detailed feature vector) and a low-order feature vector (a general feature vector of lower order than the detailed feature vector) according to the operation mode. It can be said that there is.

When the feature vector generation process is completed, the feature vector generation unit 305 stores the feature vector in the storage unit 306 and notifies the settlement control unit 303 that the feature vector generation process is completed. The payment control unit 303 sends the generated feature vector to the face authentication server 20.

Next, the face authentication server 20 will be described.

[Hardware configuration of face authentication server]
FIG. 11 is a diagram illustrating an example of a hardware configuration of the face authentication server 20 according to the first embodiment. The face authentication server 20 is a so-called computer and has the configuration illustrated in FIG. 11. For example, the face authentication server 20 includes a CPU 31, a memory 32, an input/output interface 33, a NIC (Network Interface Card) 34 as a communication unit, and the like.

The input/output interface 33 is a unit that serves as an interface of an input/output device (not shown). The input/output device includes, for example, a display device and an operation device. The display device is, for example, a liquid crystal display or the like. The operation device is, for example, a keyboard or a mouse.

Each processing module of the face authentication server 20, which will be described later, is realized by the CPU 31 executing a program stored in the memory 32, for example, as in the vending machine 10.

[Processing configuration of face authentication server]
The face authentication server 20 has a function of performing a face authentication of the requested person by accessing a database that stores the customer ID and two feature vectors generated from the face image of the person corresponding to the customer ID in association with each other. Have. The face authentication server 20 includes the following processing module to realize the function.

FIG. 12 is a diagram illustrating an example of a processing configuration of the face authentication server 20 according to the first embodiment. Referring to FIG. 12, the face authentication server 20 includes a communication control unit 401, a storage unit 402, a candidate generation unit 403, and a face authentication unit 404.

The communication control unit 401 is means for controlling communication with another device (the vending machine 10).

The storage unit 402 stores the customer ID and the feature vector in association with each other. For example, a database as shown in FIG. 13 is constructed in the storage unit 402. As shown in FIG. 13, a customer ID and two types of feature vectors (narrowing feature vector and authentication feature vector) are stored (registered) in the internal database of the face authentication server 20 in association with each other.

As a method of calculating the two types of feature vectors registered in the face authentication server 20, a method similar to the two feature vectors calculated by the vending machine 10 is used. Specifically, the “narrowing feature vector” is generated by the same method as the “outline feature vector” generated by the vending machine 10 (for example, the feature point at the same location) and stored in the internal database of the face authentication server 20. It Similarly, the “authentication feature vector” is generated by the same method as the “detailed feature vector” generated by the vending machine 10 and stored in the internal database of the face authentication server 20. In other words, a feature vector that can be compared with each of the two feature vectors generated by the vending machine 10 is registered in advance in the internal database of the face authentication server 20.

The candidate generation unit 403 is a unit that generates the above-mentioned “face authentication candidate”. Specifically, when the candidate generation unit 403 acquires the “narrowing request” from the vending machine 10, the candidate generating unit 403 relates to each of the narrowing-down feature vectors stored in the internal database with the general characteristic vector acquired from the vending machine 10. Calculate the similarity.

For example, the candidate generation unit 403 calculates the chi-square distance, Euclidean distance, etc. between the two feature vectors. The calculated chi-square distance or Euclidean distance is an index indicating the degree of similarity between two feature vectors (two face images characterized by the feature vector). The index indicating the degree of similarity between the two feature vectors is not limited to the Euclidean distance or the chi-square distance. The similarity may be an index such as a correlation value (Correlation) of two feature vectors.

The candidate generation unit 403 executes threshold value processing on each of the calculated similarities, and extracts narrowing-down feature vectors higher than a predetermined threshold value. The customer (customer ID) corresponding to the extracted narrowed-down feature vector becomes a member constituting the face authentication candidate. It should be noted that the rough feature vector and the narrowed-down feature vector calculated by the same method as the rough feature vector usually have a low degree, and thus the time required for the similarity calculation processing is short.

The face authentication unit 404 is means for performing face authentication using the detailed feature vector transmitted from the vending machine 10. Specifically, when the face authentication unit 404 acquires the “authentication request” from the vending machine 10, the detailed characteristics acquired from the vending machine 10 regarding each of the authentication feature vectors obtained from the face authentication candidates calculated previously. Calculate the similarity with the vector. For example, the face authentication unit 404 calculates the chi-square distance, Euclidean distance, etc. between the two feature vectors.

The face authentication unit 404 executes threshold value processing on the calculated similarity, and determines whether or not there is an authentication feature vector higher than a predetermined threshold value.

As a result of the determination, if there is an authentication feature vector having a degree of similarity higher than a predetermined threshold value, the face authentication unit 404 authenticates the customer corresponding to the detailed feature vector acquired from the vending machine 10 (sets authentication success). ).

If the result of the determination is that there is no authentication feature vector having a degree of similarity higher than a predetermined threshold value, the face authentication unit 404 does not authenticate the customer corresponding to the detailed feature vector acquired from the vending machine 10 (set as authentication failure). To). The authentication result is transmitted to the vending machine 10 via the communication control unit 401.

[Payment server]
The payment server 30 is configured by an information processing device (computer) like the face authentication server 20, and since the realization of the function is obvious to those skilled in the art, detailed description of the payment server 30 will be omitted.

[System operation]
Next, the operation of the vending machine payment system will be described with reference to the drawings.

14 to 16 are sequence diagrams showing an example of the operation of the vending machine payment system according to the first embodiment.

In step S101, the vending machine 10 determines whether or not there is a customer in front of itself. Specifically, the vending machine 10 determines that there is a customer in front of the own device when at least one face image can be extracted from the image data by the camera attached to the own device.

If there is no customer (step S101, No branch), the vending machine 10 repeats the above determination process.

If there is a customer (step S101, Yes branch), the vending machine 10 generates an "outline feature vector" from the image data, and sends it to the face authentication server 20 together with the narrowing request (step S102). That is, the vending machine 10 generates a rough feature vector corresponding to the narrowing down feature vector held by the face authentication server 20 from the detected customer's face image in response to detecting the customer existing around the device. ,Send.

The face authentication server 20 calculates the degree of similarity between the acquired "outline feature vector" and each of the plurality of "narrowed-down feature vectors" stored in the internal database (step S201).

The face authentication server 20 performs threshold processing on the calculated similarity and extracts the customer ID corresponding to the “narrowing feature vector” higher than the predetermined similarity to generate face authentication candidates ( Step S202).

As described above, the face authentication server 20 extracts the narrowing-down feature vector stored in the internal database and having the similarity with the acquired rough feature vector higher than a predetermined threshold value. After that, the face authentication server 20 generates a list of customer IDs corresponding to the extracted narrowed-down feature vector as a face authentication candidate.

The vending machine 10 detects whether the customer has inserted cash into the cash slot 13 (step S103).

When cash is thrown in (step S103, Yes branch), the vending machine 10 (vending machine control unit 302) determines that the customer wants to pay in cash and performs a purchase process in cash (step). S104).

When cash is not input (step S103, No branch), the vending machine 10 determines whether or not the customer has selected the product (step S105).

If the product has not been selected (step S105, No branch), the vending machine 10 returns to step S103 and repeats the determination process related to cash insertion and product selection.

If the product is selected (step S105, Yes branch), the vending machine 10 displays the payment selection screen on the liquid crystal panel 15 (step S106). The vending machine 10 displays, for example, as shown in FIG.

Note that it is also assumed that the customer throws cash into the cash slot 13 even though the settlement selection screen is displayed. In preparation for such a case, the vending machine 10 detects whether cash has been inserted even while the payment selection screen is displayed (step S107).

When cash is thrown in (step S107, Yes branch), the vending machine 10 performs the cash purchase processing in step S104.

If the cash is not input (step S107, No branch), the vending machine 10 performs the payment process by the payment means selected by the customer.

Specifically, if the IC card is selected (step S108, Yes branch), the vending machine 10 performs the purchase process using the IC card (step S109).

If the IC card is not selected (step S108, No branch), the vending machine 10 determines whether the settlement by face authentication is selected (step S110).

If the face authentication is selected (Yes at Step S110), the vending machine 10 executes the processing at Step S111 and subsequent steps.

If the face authentication is not selected (step S110, No branch), the vending machine 10 returns to step S107 and repeats the determination process regarding the payment means selection by the customer.

The processing after step S111 will be described with reference to FIGS.

When face settlement is selected, the vending machine 10 sets the operation mode of its own device to the “authentication mode”.

In step S111, the vending machine 10 extracts the face image of the customer from the image data. Then, the vending machine 10 determines whether or not the extracted face image satisfies a predetermined condition (step S112). Specifically, the vending machine 10 determines that the predetermined condition is not satisfied when the area of the extracted face image is smaller than a predetermined value.

When the extracted face image does not satisfy the predetermined condition (step S112, No branch), the vending machine 10 displays the “image correction screen” (step S113). The image correction screen is a screen that requests (reminds) the customer to perform a predetermined operation so that the face image extracted from the image data of the image of the customer satisfies the predetermined condition. The vending machine 10 displays, for example, as shown in FIG. In FIG. 18, a guide serving as a guide for the size of the face is displayed.

The customer who comes into contact with the display as shown in FIG. 18 changes the standing position and presses the OK button when the face and the frame of the customer almost match. In this way, the customer who comes into contact with the screen as shown in FIG. 18 recognizes that the settlement by the face authentication has once failed and the face image is acquired again. In other words, when the vending machine 10 determines that the face image extracted from the image data of the image of the customer does not satisfy the predetermined condition, the vending machine 10 displays a message indicating that settlement by face authentication has failed, and a face image. A screen including a message notifying re-imaging of is displayed.

After the customer follows the contents of the image correction screen (for example, after pressing the OK button), the vending machine 10 extracts the face image again (return to step S111).

When the extracted face image satisfies the predetermined condition (step S112, Yes branch), the vending machine 10 generates a "detailed feature vector" and sends it to the face authentication server 20 together with the authentication request (step S114). As described above, the vending machine 10 in the authentication mode generates the detailed feature vector from the face image of the customer who wants to purchase the product at a timing different from the timing of first detecting the customer, and causes the face authentication server 20 to generate the detailed feature vector. Send.

The face authentication server 20 calculates the degree of similarity between the acquired "detailed feature vector" and each of the plurality of "authentication feature vectors" specified from the face authentication candidates (step S203).

The face authentication server 20 performs threshold processing on the calculated similarity and performs face authentication of the customer corresponding to the acquired “detailed feature vector” (step S204).

The face authentication server 20 transmits the authentication result (authentication success, authentication failure) to the vending machine 10. At that time, when the face authentication server 20 succeeds in the authentication, the face authentication server 20 also transmits the successful customer ID to the vending machine 10. In this way, the face authentication server 20 performs face authentication of a customer who intends to purchase a product based on the detailed feature vector and the authentication feature vector corresponding to the customer ID included in the face authentication candidates.

The vending machine 10 confirms the authentication result by the face authentication server 20 and determines whether or not the face authentication of the customer is successful (step S115).

When the face authentication is successful (step S115, Yes branch), the vending machine 10 transmits the payment information including at least the ID of the successfully authenticated customer and the payment amount to the payment server 30 (step S116).

The operation after step S117 will be described with reference to FIG.

When the payment server 30 acquires the payment information, the payment server 30 identifies the customer who is the object of the payment based on the payment information, and performs the payment using the card information of the identified customer (step S301). After that, the payment server 30 transmits the payment result (payment is normally completed, payment is not possible) to the vending machine 10.

The vending machine 10 confirms the settlement result by the settlement server 30 and determines whether the settlement is normally completed (step S117).

When the settlement is normally completed (step S117, Yes branch), the vending machine 10 ejects the selected product from the outlet 16 (step S118).

When the authentication fails (step S115, No branch) or the payment fails (step S117, No branch), the vending machine 10 executes the error process of step S119.

For example, the vending machine 10 displays on the liquid crystal panel 15 that payment by face authentication has failed, and prompts the customer to select another payment method. Alternatively, the vending machine 10 displays on the liquid crystal panel 15 a screen prompting a retry by face recognition. At that time, the vending machine 10 displays a message notifying that the face image is captured together with the frame (guide) shown in FIG. Alternatively, the vending machine 10 may change to the image correction screen as shown in FIG. 18 (without displaying the image correction screen) and display a screen including a message prompting selection of another payment method.

Alternatively, the vending machine 10 may turn on an LED (Light Emitting Diode) arranged around the IC card reader 14 and the cash slot 13 in order to prompt payment by another payment means. Alternatively, the vending machine 10 may capture a face image of a customer and perform face authentication again while reproducing a moving image that introduces a new product. Alternatively, the vending machine 10 may re-perform face recognition while providing a customer with a game in which products and points can be obtained.

[Modification of First Embodiment]
The configuration and operation of the vending machine payment system described above are merely examples, and the following modifications are possible.

[Modification 1]
For example, in the sequence diagram of FIG. 14 and the like, operations relating to cash payment and payment using an IC card and face authentication are separated. In other words, when cash is thrown in, it is determined that the customer desires to make payment in cash, and the payment selection screen shown in FIG. 17 does not display the options for payment in “cash”. However, the payment selection methods based on cash, IC card, and face recognition can be unified.

For example, as shown in FIG. 19, when the vending machine 10 detects a customer in front of itself, the vending machine 10 detects whether or not a product is selected, and if selected, a payment selection screen including cash as an option. May be displayed (step S106a). In this case, the vending machine 10 displays the payment selection screen as shown in FIG. When cash is selected (step S107a), the vending machine 10 executes cash purchase processing (step S104). In FIG. 19, the same processes as those shown in FIG. 14 are designated by the same reference numerals.

[Modification 2]
In the above embodiment, the case where the feature vector generation unit 305 uses the same kind of feature amount (for example, the coordinate position of the feature point) for the generation of the rough feature vector and the generation of the detailed feature vector has been described. However, the elements forming the two feature vectors may be different types of feature amounts. More specifically, each of the detailed feature vector and the authentication feature vector may include, as an element, a feature amount of a type different from the feature amount included in each of the general feature vector and the narrowed-down feature vector.

For example, as disclosed in Japanese Patent Laid-Open No. 2015-097000 (Reference 2), feature points (for example, center points and end points of eyes, nose, mouth, etc.) are extracted from a face image, and a gray value near the feature points is extracted. Alternatively, the characteristics (periodicity, directionality, color distribution, etc.) may be treated as a feature amount and used as an element of the feature vector. Alternatively, in addition to the coordinates of the feature points, a value that can be calculated from the coordinates of the feature points (for example, the length between the eyebrows) may be used as an element such as the detailed feature vector.

As described above, the elements included in the general feature vector (narrowing down feature vector) and the detailed feature vector (authentication feature vector) may be calculated from different types of feature amounts. That is, the feature vector generation unit 305 may generate a detailed feature vector having a feature amount that can be expected to be highly accurate when used for authentication, although the calculation takes some time. Higher authentication accuracy can be realized by using such a detailed feature vector.

Alternatively, the feature vector generation unit 305 may generate the detailed feature vector by adding the feature amount and the like disclosed in Reference 2 to the general feature vector. That is, the feature vector generation unit 305 may generate a detailed feature vector having a higher order than the order of the rough feature vector while including the rough feature vector.

Normally, a feature vector with a high degree takes a long time to generate, but it has high accuracy when used for authentication, and the features of a face image can be expressed without omission. By using such a detailed feature vector, high authentication accuracy can be obtained.

In addition, there is a high possibility that sufficient accuracy cannot be obtained by the authentication using the rough feature vector. As described above, the rough feature vector is also generated from a small face image. The small face image contains no features that characterize the details of the face. In addition, even if a smaller face image is enlarged in the normalization process of the feature vector generation unit 305, a feature that is not included from the beginning does not appear. For this reason, face recognition using a rough feature vector cannot be expected to have sufficiently high accuracy.

As described above, in the vending machine payment system according to the first embodiment, as soon as the vending machine 10 recognizes the customer in front of the self-apparatus (before selecting the product), the outline of the face image of the customer is recognized. Generate a feature vector. Using the general feature vector, the face authentication server 20 narrows down in advance a large number of customers stored in the internal database and having similar characteristics to the customers existing in front of the vending machine 10. .. That is, the face authentication server 20 previously generates a list of customers who are candidates for face authentication that will occur in the future as “face authentication candidates”. After that, when the customer selects a product and desires payment by face authentication, the vending machine 10 generates a detailed feature vector from the face image of the customer. The face authentication server 20 treats a customer included in the face authentication candidates generated in advance as an authentication target, and realizes accurate (high accuracy) face authentication using the detailed feature vector and the authentication feature vector. The operation of the vending machine settlement system described above enables quick settlement by face authentication.

Generally, when a customer purchases a product, there is some time (a time lag) between being in front of the vending machine 10 and selecting the product. In the vending machine payment system according to the first embodiment, this time lag is used to preliminarily narrow down the customers who have faces similar to the face-to-face customers. Since the feature vectors used for the narrowing down (the rough feature vector and the narrowing down feature vector) have a low degree, the time required for these processes is short even if there are many targets of the similarity generation process or the threshold process. It can be said that there is. On the other hand, when the payment process is actually required, face authentication is performed using a feature vector (detailed feature vector, authentication feature vector) that provides high authentication accuracy. These two feature vectors have a high degree and require a long time for the similarity generation processing and the threshold processing. However, since the customers to be processed are not the many customers stored in the internal database of the face authentication server 20 but the already narrowed-down customers, the time required to obtain the authentication result is short. In this way, the vending machine payment system according to the first embodiment can provide quick payment using face authentication.

Further, in the vending machine payment system according to the first embodiment, when extracting a face image used for face authentication, it is determined whether the extracted face image satisfies a predetermined condition. That is, the vending machine 10 extracts a high-quality face image that satisfies a predetermined condition, and generates a detailed feature vector used for face authentication from the face image. The detailed feature vector generated in this manner includes features that appear on the face of the customer who wants to purchase the product without omission, and high authentication accuracy can be expected. In this way, the vending machine payment system according to the first embodiment can also provide highly accurate payment by face authentication.

[Second Embodiment]
Next, the second embodiment will be described in detail with reference to the drawings.

The second embodiment will describe an automatic vending machine payment system that shortens the time from when a customer selects a product to when performing authentication.

In the first embodiment, the description has been made on the assumption that there is only one customer who is imaged in the standby mode, but face images of two or more persons may be obtained in the standby mode. For example, when the vending machine 10 is installed in a cafeteria, a factory or the like, it is assumed that a line can be formed in front of the vending machine 10 at a fixed time (for example, lunch break). The operation of the vending machine 10 when two or more face images are extracted from the image data acquired in the standby mode under such a situation will be described.

When the face image extraction unit 304 according to the second embodiment recognizes the presence of a plurality of face images from one image data in the standby mode, each of the plurality of face images is extracted and stored in the storage unit 306. .. For example, when one image data includes two face images 501 and 502 as shown in FIG. 21, the face image extraction unit 304 extracts the two face images 501 and 502, and the storage unit 306. To store.

Then, the feature vector generation unit 305 generates a “schematic feature vector” for each of the extracted face images. The payment control unit 303 collectively transmits the generated plurality of schematic feature vectors to the face authentication server 20.

The candidate generation unit 403 of the face authentication server 20 according to the second embodiment executes a process related to generation of face authentication candidates for each of the acquired plurality of general feature vectors.

After that, the candidate generation unit 403 integrates the face authentication candidates generated from the respective outline feature vectors to generate a face authentication candidate (final face authentication candidate) used for the subsequent face authentication. For example, in the example of FIG. 21, the candidate generation unit 403 generates the first face authentication candidate using the general feature vector of the face image 501, and uses the general feature vector of the face image 502 to generate the second face. Generate authentication candidates.

After that, the candidate generation unit 403 generates a face authentication candidate by integrating the first and second face authentication candidates. For example, the candidate generation unit 403 integrates a plurality of face authentication candidates by generating a union of face authentication candidates having the customer ID as an element.

Here, even if two or more face images are included in one piece of image data, only one person will finally purchase the product. That is, in principle, the number of face images acquired in the authentication mode is one, and the number of detailed feature vectors generated correspondingly is also one. Therefore, the face authentication server 20 does not simultaneously acquire a plurality of detailed feature vectors, and the face authentication unit 404 is requested by the vending machine 10 to perform face authentication for one person.

The face authentication unit 404 according to the second embodiment, when acquiring the detailed feature vector, performs face authentication as in the first embodiment. That is, the face authentication server 20 generates a plurality of face authentication candidates corresponding to the outline feature vector generated from each of a plurality of face images, integrates the generated face authentication candidates, and uses the face for face authentication. Generate authentication candidates. After that, the face authentication server 20 executes face authentication using the generated face authentication candidates (face authentication candidates in which a plurality of face authentication candidates are integrated).

For example, in the above example, the first and second face authentication candidates are integrated regardless of whether the person in the face image 501 shown in FIG. 21 is authenticated or the person in the face image 502 is authenticated. Face authentication is performed using the authentication feature vector obtained from the new face authentication candidate. As a result, even if any one of the plurality of persons shown in one image data purchases the product, the authentication process can be completed quickly.

For example, in the example of FIG. 21, consider a case where the face image 501 is extracted and the face image 502 is not extracted (not extracted). In this case, the face authentication server 20 generates face authentication candidates corresponding to the face image 501, but does not generate face authentication candidates corresponding to the face image 502. In such a situation, if the person corresponding to the face image 502 selects the product, the detailed feature vector of the person is transmitted to the face authentication server 20. However, since the face authentication candidate corresponding to the face image 502 has not been created yet, the face authentication server 20 transmits “authentication failure” to the vending machine 10. In this case, although processing related to the person settlement of the face image 502 is performed by the recovery processing and the error processing of the vending machine 10, it takes a lot of time until the settlement is completed.

However, according to the second embodiment, the face authentication candidate is generated from the plurality of schematic feature vectors, so that the above problem does not occur. For example, in the above example, even if the person in the face image 502 selects a product, a customer having characteristics similar to the person is already extracted as a candidate in the face authentication candidates already generated. The authentication server 20 can authenticate the person at an early stage.

In the above description, a case has been described in which a plurality of face authentication candidates are integrated to generate a new face authentication candidate, but the system may be operated without integrating a plurality of face authentication candidates. That is, the candidate generation unit 403 extracts a narrowing-down feature vector having a similarity with the rough feature vector generated from each of the plurality of face images that is higher than a predetermined threshold, and corresponds to the extracted narrowing-down feature vector. A plurality of lists of customer IDs to be used may be generated as face authentication candidates. The face authentication unit 404 can use the generated plurality of face authentication candidates for face authentication.

In addition, the face authentication server 20 may generate a generated candidate list that associates the face authentication candidate with the rough feature vector used to generate the face authentication candidate for each vending machine 10. When the face authentication server 20 acquires the same general feature vector from the same vending machine 10, it does not perform the process of extracting the narrowed feature vector from the internal database, and the face authentication candidate in the generated candidate list is used as the face authentication. Can be used.

The face authentication server 20 confirms the ID of the vending machine 10 and the acquired general feature vector, and when the same general feature vector is acquired from the same vending machine 10, generation of face authentication candidates can be omitted. .. That is, the face authentication server 20 according to the second embodiment can end the process related to the creation of face authentication candidates at an early stage. As a result, for example, even when the customer immediately selects a product and the transmission interval between the general feature vector and the detailed feature vector is short, it is possible to prevent the face authentication candidate generation process from becoming a bottleneck.

Alternatively, the face authentication server 20 may generate, for each vending machine 10, an authentication success list including narrowed-down feature vectors that have succeeded in face authentication. The face authentication server 20 can perform face authentication by referring to the authentication success list in preference to the face authentication candidates.

When a customer purchases a product at the vending machine 10 and succeeds in face authentication, the authentication feature vector of the customer is stored in the authentication success list together with the ID of the vending machine 10. If the same customer purchases a product at the same vending machine 10, the detailed feature vector of the customer is compared with the authentication feature vector that has been successfully authenticated in the past, so that the result of “authentication success” is obtained at an early stage. can get. That is, the face authentication server 20 caches the detailed feature vector that has been successfully authenticated for each vending machine 10. Then, since the detailed feature vector of the same customer hits the cache, useless similarity calculation is unnecessary.

Note that holding a large number of detailed feature vectors that have been successfully authenticated negates the meaning of calculating face authentication candidates. Therefore, it is effective to limit the number of detailed feature vectors to be retained (cached), or to manage the number of times of successful authentication in association with each other, and to retain the detailed feature vector with a high number of successful times. is there. For example, the face authentication server 20 manages the number of times of successful face authentication in the authentication success list in association with the authentication feature vector, and selects from the authentication success list the authentication feature vector in which the number of times of successful face authentication is smaller than a predetermined value. delete.

Alternatively, the face authentication server 20 may divide the plurality of vending machines 10 into groups and share the generated candidate list and the authentication success list for the vending machines 10 belonging to the same group. For example, the vending machines can be grouped according to the position (installation location) of the vending machine 10. More specifically, the vending machines 10 existing in the area of the predetermined range are managed as one group, and the generated candidate list and the authentication success list are shared within the group.

For example, as shown in FIG. 22, if the customer is successfully authenticated by a certain vending machine 10, the corresponding customer ID is registered in the authentication success list (a list managed for each group). When the face authentication server 20 acquires the detailed feature vector from another vending machine 10 that belongs to the same group as the vending machine 10, the face authentication server 20 prioritizes the authentication process regarding the authentication feature vector of the customer registered in the authentication success list. To do it.

By sharing information between groups in this way, processing such as face recognition can be speeded up. The customer's range of activity is often near the workplace or home. Therefore, by grouping the vending machines 10 at the installation position of the vending machines 10, the information generated by the vending machines 10 located in a close place is shared to speed up the authentication process and the like.

As described above, according to the second embodiment, the vending machine payment system can provide more rapid payment using face authentication.

[Third Embodiment]
Subsequently, a third embodiment will be described in detail with reference to the drawings.

In the third embodiment, an automatic vending machine payment system that performs face recognition of a customer more accurately will be described.

In the first embodiment, the face image extraction unit 304 has described that the face image extraction unit 304 extracts a face image that satisfies a predetermined condition in the authentication mode. More specifically, in the first embodiment, the size of the face image is set as the predetermined condition. The accuracy of face recognition can be further improved by changing this predetermined condition or adding a condition.

For example, it can be assumed that two customers are standing in front of the vending machine 10. In such a case, the vending machine 10 cannot determine which customer's face image should be acquired. Therefore, when the image data acquired in the authentication mode includes two or more face images, the payment control unit 303 according to the third embodiment allows the customer who wants to purchase the product to be displayed on the liquid crystal panel 15. Display the image correction screen that prompts you to.

The vending machine 10 displays, for example, as shown in FIG. As described above, when the vending machine 10 detects a plurality of face images from the captured image data, it determines that the predetermined condition is not satisfied, and one customer is displayed on the liquid crystal panel 15 by the image correction screen. Can be urged to. By having only one customer standing in front of the self-apparatus, the vending machine 10 can accurately identify the customer who intends to make a payment.

The face image extraction unit 304 according to the third embodiment may determine that the predetermined condition is satisfied when there is a change in the face image included in each of the plurality of image data obtained during the predetermined period. More specifically, the face image extraction unit 304 extracts a face image from a plurality of image data, and stores a face image in the storage unit 306 when a change is recognized in these extracted face images. Good. The change in the face image confirmed by the face image extraction unit 304 may be, for example, a positional shift or blinking. By detecting these changes, the face image extraction unit 304 can confirm the presence of a “person” who truly stands in front of the device itself (can eliminate the face authentication by information other than the person).

As described above, the vending machine 10 according to the third embodiment determines whether or not the face image extracted from the data obtained by capturing the image of the customer who intends to purchase the product satisfies the predetermined condition. Then, when the face image does not satisfy the predetermined condition, the vending machine 10 displays a screen requesting the customer to perform a predetermined operation so as to satisfy the predetermined condition. As a result, a face image and a detailed feature vector suitable for face authentication can be obtained, and thus the vending machine payment system according to the third embodiment can provide payment by face authentication with higher authentication accuracy.

[Modification]
The configuration and operation of the vending machine payment system described in the first to third embodiments are examples, and various modifications are possible. For example, the vending machine payment system and its constituent devices may have the following configurations or operations.

In the above-described embodiment, it is determined whether or not a person is present around the vending machine 10 by using image data (imaging data) captured by the camera. However, the person is detected by a human sensor using infrared rays or the like. May be detected. Alternatively, the vending machine 10 may use the image data to detect whether or not a person is truly present in the vicinity of the own device after detecting an object with the motion sensor. In other words, the power of the camera module 24 may be turned off to reduce the power consumption of the vending machine 10 until the detection of a person by the motion sensor.

In the above embodiment, it is premised that the vending machine 10 acquires a still image and extracts a face image from the still image. However, the camera module 24 may acquire a moving image, extract frames at predetermined intervals from the moving image, and acquire image data. Further, there is no limitation on the format or size of the image data acquired by the vending machine 10, and for example, the vending machine 10 may acquire a monochrome image or a color image.

In the above-described embodiment, the configuration in which the liquid crystal panel 15 is incorporated in the vending machine 10 has been described. However, the tablet equipped with the liquid crystal panel and the existing vending machine 10 are connected to realize the functions of the vending machine 10. Good. For example, as shown in FIG. 24, the tablet 17 may be mounted on the vending machine 10a, and the camera function and the communication function of the tablet 17 may be used to implement the vending machine 10 described in the above embodiment. In this case, the vending machine control unit 302 shown in FIG. 6 corresponds to the controller of the existing vending machine 10a, and the settlement control unit 303 corresponds to the controller of the tablet 17. The functions of the vending machine 10 are realized by the cooperation of the two controllers (transmission/reception of control information).

In the above embodiment, a case has been described in which two feature vectors (a rough feature vector and a detailed feature vector) are generated using an image captured by one camera module 24. However, the camera module to be used may be selectively used according to the generated feature vector (operation mode of the vending machine 10). For example, the camera installed on the upper part of the vending machine 10 is used to generate the general feature vector in the standby mode, and the center of the vending machine 10 is used to generate the detailed feature vector in the authentication mode. You may use the camera installed in.

In addition, the face image may be distorted depending on the relationship between the position of the camera attached to the vending machine 10 (for example, the upper part of the vending machine 10 as described above) and the position of the customer. In such a case, the feature vector generation unit 305 may acquire an image suitable for calculation of the feature vector by appropriately performing geometric transformation such as affine transformation.

In the above embodiment, the authentication result by the face authentication server 20 is transmitted to the payment server 30 via the vending machine 10, but the authentication result may be transmitted from the face authentication server 20 to the payment server 30. In this case, the payment server 30 issues a product discharge instruction to the vending machine 10.

Furthermore, the information transmitted and received between each device (the vending machine 10, the face authentication server 20, the payment server 30) may be encrypted by an encryption method such as common key encryption or public key encryption, or in plain text. It may be transmitted. However, since the settlement information and the feature vector can be regarded as personal information, it is desirable that the information exchanged between the devices is encrypted.

In the above-described embodiment, the person detected by the vending machine 10 has been described on the assumption that the person purchases the product, but in reality, there are people who leave without purchasing the commodity. In such a case, the “face authentication candidate” prepared in advance by the face authentication server 20 before face authentication is unnecessary. Therefore, the vending machine 10 transmits the rough feature vector to the face authentication server 20, but if the customer leaves without indicating the purchase intention of the product, the vending machine 10 specifies the transmitted rough feature vector to be a “face recognition candidate”. May be transmitted to the face authentication server 20. By deleting unnecessary face authentication candidates, the resources of the face authentication server 20 can be effectively used.

In the above embodiment, it has been described that two types of feature vectors are registered in the face authentication server 20, but it is also possible to express two types of feature vectors with one feature vector. For example, if some of the elements of the authentication feature vector are treated as narrowed down feature vectors, the number of feature vectors registered in the face authentication server 20 can be one. For example, if the authentication feature vector is a 10-dimensional vector, and the narrowed-down feature vector is determined to be the four elements (four-dimensional vector) from the beginning of the authentication feature vector, one feature vector has substantially two feature vectors. Will be included.

In the above embodiment, the vending machine 10 generates the feature vector, but the vending machine 10 may send the face image (or image data including the face image) to the face authentication server 20. That is, the face image extraction processing and the feature vector generation processing may be executed on the server side. However, in this case, it is necessary to secure a sufficient communication band in the network between the vending machine 10 and the face authentication server 20.

In the above embodiment, when the face image used for generating the detailed feature vector is extracted, if the predetermined condition is not satisfied, the customer is asked to change the standing position using the image correction screen. However, when the predetermined condition is not satisfied, the vending machine 10 may control the camera (lens) so as to satisfy the predetermined condition. For example, if the face area is set to a predetermined condition, the vending machine 10 may make the face image satisfy the predetermined condition by moving the lens or zooming.

In the plurality of flowcharts used in the above description, the plurality of steps (processes) are described in order, but the execution order of the steps executed in each embodiment is not limited to the order described. In each of the embodiments, the order of the illustrated steps can be changed within a range that does not hinder the contents, for example, executing each process in parallel. Further, the above-described respective embodiments can be combined within a range in which the contents do not conflict with each other.

It should be noted that the disclosures of each of the above cited patent documents and the like are incorporated herein by reference. Modifications and adjustments of the exemplary embodiments and examples are possible within the scope of the overall disclosure (including the claims) of the present invention and based on the basic technical concept of the invention. Further, various combinations of various disclosed elements (including each element of each claim, each element of each embodiment or example, each element of each drawing, and the like) or selection within the framework of the entire disclosure of the present invention Is possible. That is, it goes without saying that the present invention includes various variations and modifications that can be made by those skilled in the art according to the entire disclosure including the claims and the technical idea. In particular, with regard to the numerical range described in this specification, any numerical value or small range included in the range should be construed as specifically described even if not otherwise specified.

10, 10-1 to 10-N, 10a, 103 Vending machine 11 Product sample 12 Selection button 13 Cash slot 14 IC card reader 15 Liquid crystal panel 16 Discharge port 17 Tablet 20, 101 Face authentication server 21, 31 CPU (Central Processing Unit)
22, 32 memory 23 display module 24 camera module 25 antenna 26 wireless communication module 30, 102 settlement server 33 input/output interface 34 NIC (Network Interface Card)
301, 401 Communication control unit 302 Vending machine control unit 303 Payment control unit 304 Face image extraction unit 305 Feature vector generation unit 306, 402 Storage unit 403 Candidate generation unit 404 Face authentication unit 501, 502 Face image

Claims (15)

A database that stores the customer ID (Identifier) and the first and second feature vectors generated from the face image of the person corresponding to the customer ID in association with each other is accessed to perform face authentication of the requested person. , Face recognition server,
A payment server that performs payment related to product purchases,
A vending machine that requests face authentication from the face authentication server and requests the payment server to settle a product purchased by a customer who has been successfully authenticated by the face authentication server.
Including,
The vending machine, in response to detecting a customer around the device itself,
Generating a third feature vector corresponding to the first feature vector from the detected face image of the customer,
The face authentication server extracts a first feature vector stored in the database, the first feature vector having a similarity with the third feature vector higher than a predetermined threshold value; A list of the customer IDs corresponding to the extracted first feature vector is generated as a face authentication candidate,
The vending machine, at a timing different from the timing of detecting the customer,
Generate a fourth feature vector from the face image of the customer who wants to purchase the product,
The face authentication server is
Face authentication of a customer who intends to purchase the product is performed based on the fourth feature vector and the second feature vector corresponding to the customer ID included in the face authentication candidate.
Vending machine payment system. The vending machine payment system according to claim 1, wherein the orders of the first and third feature vectors are lower than the orders of the second and fourth feature vectors. 3. The vending machine settlement according to claim 1, wherein each of the second and fourth feature vectors includes, as an element, a feature amount of a type different from the feature amount included in each of the first and third feature vectors. system. The vending machine is
When the captured image data includes a face image, it is determined that a customer exists around the own device, and the third feature vector is generated from the face image of the customer existing around the own device. The vending machine payment system according to any one of 1 to 3. The vending machine is
The vending machine payment system according to claim 4, wherein when the captured image data includes a plurality of face images, the third feature vector is generated from each of the plurality of face images. The face authentication server is
The first feature vector whose similarity to the third feature vector generated from each of the plurality of face images is higher than a predetermined threshold is extracted, and the extracted first feature vector corresponds to the extracted first feature vector. The vending machine settlement system according to claim 5, wherein a plurality of lists including the customer IDs are generated as face authentication candidates, and the plurality of generated face authentication candidates are used for the face authentication. A plurality of said vending machines are included,
7. The vending machine payment system according to claim 1, wherein the face authentication server processes face authentication requests from the plurality of vending machines. The face authentication server is
For each vending machine, generate a generated candidate list that associates the face authentication candidate with the third feature vector used to generate the face authentication candidate,
When the same third feature vector is acquired from the same vending machine, the process of extracting the first feature vector from the database is not performed, and the face recognition candidates in the generated candidate list are used for face recognition. The vending machine payment system according to claim 7. The face authentication server is
For each of the vending machines, generate an authentication success list including the fourth feature vector that has succeeded in face authentication,
9. The vending machine settlement system according to claim 8, wherein face authentication is performed by referring to the authentication success list in preference to the face authentication candidate. The face authentication server is
In the authentication success list, the number of successful face authentications is managed in association with the fourth feature vector, and the fourth feature vector in which the number of successful face authentications is smaller than a predetermined value is set to the authentication success list. The vending machine payment system according to claim 9, which is deleted from the vending machine payment system. The face authentication server is
The vending machine settlement system according to claim 9 or 10, wherein the plurality of vending machines are divided into groups, and the generated candidate list and/or the authentication success list regarding the vending machines belonging to the same group are shared. .. A database that stores the customer ID (Identifier) and the first and second feature vectors generated from the face image of the person corresponding to the customer ID in association with each other is accessed to perform face authentication of the requested person. , Face recognition server,
Connected to the payment server, which performs payment related to product purchase,
A device for requesting face authentication from the face authentication server and requesting the payment server to settle a product purchased by a customer who has been successfully authenticated by the face authentication server,
In response to detecting a customer existing around the own device, a third feature vector corresponding to the first feature vector is generated from the detected face image of the customer,
The third feature vector is transmitted to the face authentication server, and the face recognition server stores the first feature vector stored in the database, which is similar to the third feature vector. Requesting to extract a first feature vector whose degree is higher than a predetermined threshold value and generate a list of the customer IDs corresponding to the extracted first feature vector as a face authentication candidate,
At a timing different from the timing of detecting the customer, a fourth feature vector is generated from the face image of the customer who wants to purchase the product,
The face authentication of the customer who intends to purchase the product is performed on the face authentication server based on the fourth feature vector and the second feature vector corresponding to the customer ID included in the face authentication candidate. Request to do,
vending machine. A database that stores the customer ID (Identifier) and the first and second feature vectors generated from the face image of the person corresponding to the customer ID in association with each other is accessed to perform face authentication of the requested person. , A server,
Extracting a first feature vector stored in the database, the first feature vector having a similarity with a third feature vector generated by a vending machine higher than a predetermined threshold value; A list of the customer IDs corresponding to the extracted first feature vector is generated as a face authentication candidate,
The product based on the fourth feature vector generated from the face image of the customer who is going to purchase the product by the vending machine, and the second feature vector corresponding to the customer ID included in the face authentication candidate. Perform face recognition for customers who are trying to purchase
Face recognition server. A database that stores the customer ID (Identifier) and the first and second feature vectors generated from the face image of the person corresponding to the customer ID in association with each other is accessed to perform face authentication of the requested person. , Face recognition server,
A payment server that performs payment related to product purchases,
A vending machine that requests face authentication from the face authentication server and requests the payment server to settle a product purchased by a customer who has been successfully authenticated by the face authentication server.
In vending machine payment system including
Generating a third feature vector corresponding to the first feature vector from the detected face image of the customer in response to detecting the customer existing in the vicinity of the own device;
A first feature vector stored in the database, the first feature vector having a similarity to the third feature vector higher than a predetermined threshold is extracted, and the extracted first feature vector is extracted. Generating a list of the customer IDs corresponding to the feature vectors as face authentication candidates,
Generating a fourth feature vector from a face image of a customer who intends to purchase a product at a timing different from the timing of detecting the customer,
Performing face authentication of a customer who intends to purchase the product, based on the fourth feature vector and the second feature vector corresponding to the customer ID included in the face authentication candidate;
Vending machine payment method including. A database that stores the customer ID (Identifier) and the first and second feature vectors generated from the face image of the person corresponding to the customer ID in association with each other is accessed to perform face authentication of the requested person. Connected to a face authentication server and a payment server for performing payment related to product purchase, requesting face authentication from the face authentication server, and payment of a product purchased by a customer who has been successfully authenticated by the face authentication server. The computer installed in the vending machine, which requests the payment server,
A process of generating a third feature vector corresponding to the first feature vector from the detected face image of the customer in response to detecting the customer existing around the device itself;
The third feature vector is transmitted to the face authentication server, and the face recognition server stores the first feature vector stored in the database, which is similar to the third feature vector. A process of extracting a first feature vector whose degree is higher than a predetermined threshold value, and generating a list of the customer IDs corresponding to the extracted first feature vector as a face authentication candidate;
A process of generating a fourth feature vector from a face image of a customer who intends to purchase a product at a timing different from the timing of detecting the customer;
The face authentication of the customer who intends to purchase the product is performed on the face authentication server based on the fourth feature vector and the second feature vector corresponding to the customer ID included in the face authentication candidate. Process that requires you to do
A program to execute.

Images