JP2021092952A - Automated transaction device - Google Patents

Abstract

To provide an automated transaction device that allows for giving advice and guidance at various operation stages by using emotions based on various facial expressions of users.MEANS FOR SOLVING THE PROBLEM: An automated transaction device has an interface unit having a screen that accepts operation of a user and displays information for the user. The automatic transaction device includes an imaging unit that captures a part of a body including a face of the user and generates imaging data, and an emotion recognition artificial intelligence that recognizes the user's emotions based on the imaging data and generates emotion data. A message corresponding to a predetermined emotion data determined from the emotion data at the operation stage corresponding to the emotion data is output to the screen of the interface unit.SELECTED DRAWING: Figure 2

Description

The present invention relates to an automated teller machine.

Known automatic teller machines include automated teller machines (ATMs) and cash dispensers (CDs) of financial institutions. ATMs are installed in branches of financial institutions such as banks, stores such as convenience stores, and transportation facilities such as train stations. The user can perform transactions such as deposit, withdrawal, transfer, and balance inquiry by performing various operations with the input keys and the touch panel according to the display screen of the interface unit of the automatic teller machine.

Generally, an automated teller machine is equipped with an automated teller telephone. The automatic call telephone is known as a response system in which a user can call a specific person such as an operator by raising the handset of the automatic call telephone and the operator can respond to a user's inquiry (problem) (Patent Document 1). ).

Further, there is known a system in which a robot provides an appropriate operation guidance of an automatic processing machine to a customer according to a customer's situation or a processing state of the automatic processing machine (Patent Document 2).

Japanese Unexamined Patent Publication No. 2001-34812 JP-A-2017-87411

The automated teller machine system of Patent Document 1 cannot cope with a user's problem without an action such as picking up a receiver from the user. Therefore, there is a problem that it is not possible to deal with the user or solve the problem without the transmission from the user.

In the operation guidance system of Patent Document 2, based on the customer's photographed face imaged data captured by the image pickup unit provided in the robot and the emotional data in which the face imager data stored in the storage unit and human emotions are associated with each other. The customer's emotions are estimated and recognized, or the customer's emotions are estimated and recognized from the photographed customer's face imaging data (paragraphs 0062, 0150), but the customer's emotions when the advertisement is displayed, Data of emotions and facial expressions recognized from the voice emitted and data of the age and gender estimation result of the customer are stored in a predetermined database, and the stored information is utilized, for example, in marketing related to the advertisement. (Paragraph 0151), and there is a problem that the operation guidance does not reflect any recognized customer's emotions. There is a problem that a mobile robot requires a large space.

The present invention has been made in view of the above problems, and it is possible to give advice and guidance at various operation stages by utilizing emotions based on various facial expressions of the user in front of the interface portion of the automated teller machine. One of the issues is to provide an automated teller machine. Another object of the emotion determination of the present invention is, for example, to provide support for resolving mistakes and confusion about the operation of the user's automated teller machine.

The automated teller machine of the present invention is an automated teller machine having an interface unit having a screen that accepts a user's operation and displays information to the user.
An imaging unit that captures a part of the body including the user's face and generates imaging data,
It has an emotion recognition artificial intelligence unit that recognizes the user's emotions based on the imaged data and generates emotion data.
It is characterized in that a message corresponding to a predetermined emotion data determined from the emotion data at an operation stage corresponding to the emotion data is output to the screen of the interface unit.

According to the present invention, in a compact automated teller machine equipped with a camera, a message corresponding to a predetermined emotion data determined from the emotion data at an operation stage corresponding to the user's emotion data based on the imaging data is transmitted. Since it is output to the screen of the interface unit, it is possible to give advice and guidance to the user about various operations, and it is possible to provide the user with support for eliminating mistakes and confusion about the operation of the automated teller machine.

It is a schematic perspective view which shows the automatic teller machine of the Example by this invention. It is a schematic explanatory drawing which shows an example of the configuration example of the monitoring system including the automatic teller machine of this Example. It is a flow chart which shows the flow of operation when the user in the automatic teller machine which concerns on this Example makes a transaction to transfer money with a "cash card".

Hereinafter, an example of an automated teller machine (hereinafter, also referred to as an ATM) that handles cash according to an embodiment of the present invention will be described with reference to the drawings. The present invention is not limited to the following examples. In the examples, components having substantially the same function and configuration are designated by the same reference numerals, so that duplicate description will be omitted.

(ATM configuration)
As shown in FIG. 1, the ATM is provided with an interface unit IF on the front side thereof, which makes it easy for users to perform transaction operations while facing each other. The interface unit IF has a function of transmitting information about transactions and receiving operation instructions with the user. The interface unit IF includes a camera 1 which is an imaging unit, an operation display unit 2, a paper medium processing unit 3 having a paper medium inlet / output 3a for inputting / outputting paper media such as a passbook and a payment slip, and bills and coins. A cash processing unit 4 having a cash deposit / withdrawal port 4a for input / output, a card reader / receipt printer unit 5 having a card inlet / outlet 5a and a receipt issuing port 5b, and a handset 6a of an automatic calling telephone unit (not shown) are provided. ing.

Further, in the ATM, in addition to the interface unit IF, as shown in FIG. 2, a storage unit 7 in which a program for controlling the operation of the ATM is stored and a program stored in the storage unit 7 are stored in the CPU. (Central Processing Unit) includes a control unit 8 that controls the operation of each function unit of the ATM, an emotion recognition artificial intelligence unit 9, and an automatic call telephone unit 6 that is the main unit of the handset 6a. ..

The camera 1 has a function of acquiring imaging data of a moving image or a still image. The camera 1 itself is arranged at a position where the entire face of the user can be photographed and recognized. For example, the camera 1 can acquire an image of a part of the body including the user's face and an image around the ATM including an image around the ATM. The camera 1 may be mounted inside the housing on the rear side of the rear confirmation half mirror (not shown) on the operation display unit 2 of the ATM so as not to be seen from the outside.

The operation display unit 2 is a touch panel that displays an operation screen for a transaction commanded by the control unit 8 and enables selection of a transaction type and input of a password, transaction amount, and the like. The operation display unit 2 receives selections and inputs from the user, transmits the instruction contents to the control unit 8, confirms various information from the control unit 8 to the user, and displays an operation screen such as selection input. .. The touch panel is an electronic component that combines an image display device such as a liquid crystal display and a position input device such as a touch pad, and senses screen position information by touching a display on the screen and transmits it to the control unit 8. Based on the control of the control unit 8, the operation screen of the operation display unit 2 (hereinafter, also simply referred to as a screen) displays, for example, transaction types such as "deposit" and "withdrawal" as initial screens. Then, the user advances the transaction by touching a display portion such as a button of a desired operation on the screens of various operation stages displayed under the control of the control unit 8.

The paper medium processing unit 3 takes in paper media such as payment slips and passbooks from the paper medium inlet / outlet 3a, processes the paper medium based on the control of the control unit 8, and discharges the paper medium from the paper medium inlet / outlet 3a. .. As a passbook processing function of the paper medium processing unit 3, the magnetic stripe data in the passbook inserted by the user is read and written, and the transaction content (booking data) is booked and printed. As a payment slip processing function of the paper medium processing unit 3, the payment slip inserted from the user into the passbook entrance / exit is optically read, or if necessary, printed.

Based on the control of the control unit 8, the cash processing unit 4 performs processing related to cash deposit / withdrawal (transportation and storage of cash from the cash deposit / withdrawal port 4a, determination of the authenticity of cash, determination of denomination, etc.), and cashes the cash. It can be discharged from the deposit / withdrawal port 4a.

The card reader / receipt printer unit 5 has user identification information (for example, name, account number, etc.) recorded on a card medium such as a cash card inserted from the user into the card inlet / outlet 5a under the control of the control unit 8. ) And recognizes the user. The card reader / receipt printer unit 5 can also read and write the data recorded on the card medium. The card reader / receipt printer unit 5 prints the transaction result represented by the signal in response to the transaction detail print signal supplied from the control unit 8 on the statement slip sheet, and discharges the transaction result from the receipt issuing port 5b. ..

As shown in FIG. 2, the automated teller machine (ATM), the server 11, and the operator terminal 12a in the operator room 12 in the monitoring system are all connected to the communication network NW, and each of them controls / information data and voice via the communication network NW. Data can be transmitted. The automatic calling telephone unit 6 has a function of outputting voice via the handset 6a and a function of acquiring voice information. The automatic call telephone unit 6 is a communication device that automatically calls a predetermined party only by the user lifting the handset 6a. When the handset 6a of the automatic call telephone unit 6 is raised, for example, the automatic call telephone unit 6 calls the operator room 12 that manages the ATM, and the user can talk to the operator via the handset 6a and the operator terminal 12a. The automatic calling telephone unit 6 can be provided with a function of transmitting an announcement of voice guidance for operation to the user via the handset 6a.

The emotion recognition artificial intelligence unit 9 is connected to the camera 1 and the control unit 8 and analyzes emotions by the movement of muscles such as the user's face and the movement of the line of sight from the image from the camera 1, and joy, surprise, fear, anger, and anger. Extract emotions from analysis results such as disgust, sadness, contempt, satisfaction, confusion, excitement, guilt, conviction, and impatience. The analysis result is transmitted to the control unit 8. That is, the emotion recognition artificial intelligence unit 9 executes emotion analysis based on the imaged data of the facial expression of the user imaged by the camera 1, generates emotion data, and outputs the emotion data to the control unit 8. The emotion recognition artificial intelligence unit 9 performs emotion determination processing based on emotion data in the CPU and memory (ROM, RAM, HDD, etc.) of the control unit 8 by the cooperation of the CPU and the memory based on the program installed in the memory. It may be configured to do so. The emotion recognition software program is stored in advance in, for example, a storage unit 7 provided in the device, and each program is read into, for example, a RAM at the time of execution and executed by the CPU. Emotion recognition software programs include Microsoft's Face API and Affectiva's Affdex. The control unit 8 requires support from a user such as "confused" or "hurried" (hereinafter, these are also simply referred to as "confused"), which are predetermined emotion data determined in the received analysis result (emotion data). Detects the presence or absence of negative emotional results. This is called an emotion determination step by the emotion recognition artificial intelligence unit 9 and the control unit 8.

The communication unit 10 is a communication module for transmitting and receiving data to and from another information processing device (communication with the operator terminal 12a of the server 11 or the operator room 12 via the communication network NW) by wire or wirelessly. .. The communication unit 10 transmits the user's emotion analysis result, a call establishment request, a call disconnection request, screen operation data, and the like to the server 11. In addition, the server 11 receives the message of the main question and transmits it to the control unit 8. For example, the communication unit 10 is connected to a LAN (Local Area Network), a telephone line, or the like, and communicates with the operator terminal 12a or the server 11 via the communication network NW. The communication unit 10 transmits / receives information about the processing performed at the ATM to / from the server 11 via the communication network NW. The communication network NW is a wired or wireless transmission line for information transmitted from the connected communication unit 10. For example, the communication network NW may include a public line network (Internet, telephone line network, satellite communication network), various LANs, or a WAN (Wide Area Network). Further, the communication network NW may include a dedicated line network such as IP-VPN (Internet Protocol-Virtual Private Network).

The components used in this embodiment are mainly a camera 1, an operation display unit 2, an emotion recognition artificial intelligence unit 9, a control unit 8, an automatic call telephone unit 6, and a communication unit 10. Therefore, for convenience, these are described below. For the configurations other than the elements, detailed explanations will be omitted, and the emotion determination step and the puzzle handling processing step will be described.

As shown in FIG. 2, the automated teller machine (ATM), the server 11, and the operator terminal 12a of the operator room 12 are all connected to the communication network NW, and each transmits control / information data and voice data via the communication network NW. ..

A: Emotion determination step of an automated teller machine (ATM) An image of a user's facial expression in a state where a screen of one operation stage of a certain transaction is displayed (a state including screen information of the operation stage) is captured. The emotion recognition artificial intelligence unit 9 analyzes the user's emotions from the image from the camera 1 to generate emotion data, and transmits the analysis result to the control unit 8. When the control unit 8 receives the analysis result (emotion data), it detects the presence or absence of an emotional result such as "confused" or "impulsive" that the user needs support (emotion determination of embarrassment). The above-mentioned processing by the emotion recognition artificial intelligence unit 9 and the control unit 8 is the emotion determination step.

B: Start of puzzle handling processing step in an automated teller machine (ATM) After the first emotion determination step, when the control unit 8 determines that the required emotion result is absent, the screen of the next stage is displayed on the operation display unit 2. Display it. After the emotion determination step, when the control unit 8 determines that the required emotion result is present (that is, confused), the control unit 8 instructs the operation display unit 2 to output a message as to whether or not support is required. When the user inputs the input "necessary" for the message on the screen of the operation display unit 2 from the screen, the input information is transmitted from the operation display unit 2 to the control unit 8. When support request is required, the control unit 8 transmits a request for question data to the server 11 via the communication unit 10 together with the current screen information (screen information at the current operation stage). The process of request transmission of question data by the control unit 8 is the first stage of starting the puzzle response processing step. In the case of the user's "support not required" input, the control unit 8 continues the process as it is (displays the screen of the next stage).

C: Start of the puzzle handling processing step on the server 11 The middle stage server 11 receives the signal from the communication unit 10 and then transmits the question data for the current screen information to the communication unit 10. The process of transmitting the question data by the server 11 is the middle stage of starting the puzzle response processing step.

D: Start of puzzle handling processing step in automated teller machine (ATM) The communication unit 10 receives question data from the server 11 and transmits it to the control unit 8. The control unit 8 instructs the operation display unit 2 to output the received question data. The operation display unit 2 outputs question data (options) from the control unit 8 to the screen, and prompts the user to input which question is applicable or not. As a result, the support of the user is started by exchanging data between the operation display unit 2, the control unit 8, the communication unit 10, and the server 11. If the question (option) output on the screen does not correspond to the user, the user inputs "Not applicable" on the screen. In this case, the operation display unit 2 notifies the control unit 8 that "not applicable" has been input. The control unit 8 issues an instruction to the operation display unit 2 to output a handset message urging the handset 6a to be picked up, and the operation display unit 2 displays the handset message. The process of transmitting question data by the ATM is the start and end stage of the puzzle response processing step.

E: One stage of user support in the puzzle handling process step in an automated teller machine (ATM) When the user lifts the handset 6a by looking at the handset message, the automatic call phone unit 6 tells that the handset 6a has been lifted. Detects and transmits to the control unit 8. As a result, the control unit 8 requests the server 11 to make a call connection with the operator via the communication unit 10.

F: Two stages of user support in the puzzle handling process step on the server 11 The server 11 that receives the call connection request establishes a call between the vacant operator terminal 12a and the handset 6a of the automated teller machine (ATM). , Voice data is transmitted and received between them.

G: Three stages of user support in the puzzle handling process step on the server 11 At the same time as the call is established, the server 11 receives the support information (at the current operation stage) that the user received from the automated teller machine (ATM) at the operator terminal 12a. Screen information) is sent. The operator responds to the user's problem based on the received support information via the operator terminal 12a.

As described above, the camera 1 and the emotion recognition artificial intelligence unit 9 are used to analyze the user's emotions, and the message is output from the screen to support the user who sends the message from the automated teller machine (ATM). it can.

Further, by preparing a countermeasure method for a main question in advance on the server 11 and outputting it to the screen, it is possible to obtain an effect that the problem of the user can be solved by the automatic teller machine without connecting to the operator terminal 12a. Further, the storage unit 7 of the automatic trading device (ATM) stores the screen information of the current operation stage corresponding to the emotion data and the question data corresponding to the predetermined emotion data "confused", and stores the question data corresponding to the predetermined emotion data in the predetermined emotion data. If the corresponding question data is output to the screen of the interface unit IF as a message corresponding to the predetermined emotion data, the user's problem can be solved by the ATM itself without connecting to the server 11 and the operator terminal 12a. The effect is obtained.

In addition, if the control unit 8 analyzes "confused" or "impulsive" at the time of the transfer screen, the measures against the transfer fraud should be strengthened by outputting a message to alert the transfer fraud to the screen of the interface unit IF. Is also possible.

Next, an example of the operation flow of the ATM of this embodiment when the user transfers with a "cash card" will be described. FIG. 3 is a flow chart showing an operation flow when a user in an automated teller machine according to the present embodiment makes a transaction to transfer money with a “cash card”.

The ATM starts the operation by making each component wait as step S1.

In order for the user to start a transaction at an ATM, it is necessary to select a transaction on the initial screen of the interface unit IF. The control unit 8 detects whether or not there is movement of the user from the image data from the camera 1 (step S2), and when the movement is detected, starts emotion detection. If not, the ATM returns to step S1 and waits until the motion is detected again. In the case of the transfer transaction of this example, the user selects the "transfer" button display on the initial screen.

When the button selection is detected and the process proceeds to step S3, the "cash" and "cash card" selection buttons are displayed on the screen (transfer method selection screen). In this screen state, the emotion recognition artificial intelligence unit 9 and the control unit 8 perform the emotion determination of the selection (step S3).

Next, it is determined whether or not the result of the determination step S3 is confusing (step S4). If the control unit 8 determines that there is confusion and "Y", the process proceeds to step S5 and the confusion handling process is performed. In this example, when the user transfers cash, he or she selects to pay from the user's account by pressing the "cash card" button instead of "cash". Therefore, if the control unit 8 determines that there is no confusion, "N", the process proceeds to step S6.

If the process proceeds to step S6, a screen is displayed to prompt the user to enter his / her personal identification number for authentication (password input screen). The user enters the PIN. In this screen state, the emotion recognition artificial intelligence unit 9 and the control unit 8 perform the emotion determination of the input (step S6).

Next, it is determined whether or not the result of the emotion determination step S6 is confusing (step S7). If the control unit 8 determines that there is a confusion and "Y", the process proceeds to step S8 and the confusion handling process is performed. If the control unit 8 determines that there is no confusion, "N", the process proceeds to step S9.

If the process proceeds to step S9, a message prompting the user to enter his / her own telephone number is displayed on the screen (telephone number input screen). The phone number will be used by the bank to contact you if a mistake is discovered. The user enters the phone number. In this screen state, the emotion recognition artificial intelligence unit 9 and the control unit 8 perform the emotion determination of the input (step S9).

Next, it is determined whether or not the result of the emotion determination step S9 is confusing (step S10). If the control unit 8 determines that there is a confusion and "Y", the process proceeds to step S11 and the confusion handling process is performed. If the control unit 8 determines that there is no confusion, "N", the process proceeds to step S12.

When proceeding to step S12, the account name of the user (own) is displayed on the screen when transferring with a "cash card". (Name input confirmation screen). The user confirms the name. This name does not have to be the user name but may be a name that the payer can understand. In this screen state, the emotion recognition artificial intelligence unit 9 and the control unit 8 perform the emotion determination of the confirmation (step S12).

Next, it is determined whether or not the result of the emotion determination step S12 is confusing (step S13). If the control unit 8 determines that there is a confusion and "Y", the process proceeds to step S14 to perform the confusion handling process. If the control unit 8 determines that there is no confusion, "N", the process proceeds to step S15.

When the process proceeds to step S15, a selection button for selecting the financial institution to transfer to or selecting the branch name is displayed on the screen (financial institution input screen). The user presses the button of the corresponding financial institution name and branch name from each selection button list, selects the financial institution of the transfer destination, and selects the branch name. In this screen state, the emotion recognition artificial intelligence unit 9 and the control unit 8 perform the emotion determination of the selection (step S15).

Next, it is determined whether or not the result of the emotion determination step S15 is confusing (step S16). If the control unit 8 determines that there is a confusion and "Y", the process proceeds to step S17 and the confusion handling process is performed. If the control unit 8 determines that there is no confusion "N", the process proceeds to step S18.

If the process proceeds to step S18, a selection button for selecting the transfer destination account item (account type) is displayed on the screen (account item selection screen). The user selects a subject by pressing the button of the corresponding subject from the selection button list. In this screen state, the emotion recognition artificial intelligence unit 9 and the control unit 8 perform the emotion determination of the selection (step S18).

Next, it is determined whether or not the result of the emotion determination step S18 is confusing (step S19). If the control unit 8 determines that there is a confusion and "Y", the process proceeds to step S20 and the confusion handling process is performed. If the control unit 8 determines that there is no confusion, "N", the process proceeds to step S21.

If the process proceeds to step S21, a message prompting the user to enter the transfer destination account number is displayed on the screen (transfer destination account number input screen). The user enters the transfer account number. In this screen state, the emotion recognition artificial intelligence unit 9 and the control unit 8 perform emotion determination of the input (step S21).

Next, it is determined whether or not the result of the emotion determination step S21 is confusing (step S22). If the control unit 8 determines that there is a confusion and "Y", the process proceeds to step S23 to perform the confusion handling process. If the control unit 8 determines that there is no confusion, "N", the process proceeds to step S24.

If the process proceeds to step S24, a message prompting the user to enter the transfer amount is displayed on the screen (transfer amount input screen). The user enters the transfer account number. In this screen state, the emotion recognition artificial intelligence unit 9 and the control unit 8 perform the emotion determination of the input (step S24).

Next, it is determined whether or not the result of the emotion determination step S24 is confusing (step S25). If the control unit 8 determines that there is a confusion and "Y", the process proceeds to step S26 to perform a confusion response process. If the control unit 8 determines that there is no confusion, "N", the process proceeds to step S27.

If the process proceeds to step S27, a message prompting the user to confirm the transfer details is displayed on the screen (transfer confirmation screen). In the case of cash card transfer, the screen will switch to the confirmation screen of the transfer details, and if the user has no difference in the details, press the "Confirm" button, and if there is an error, correct it with the "Correction" button, but return to the step of correction ( Step S27).

Next, if the "Confirm" button is selected, the completion is displayed on the screen. When transferring with a "cash card", the transaction is completed and the transfer statement is output.

According to this embodiment, the predetermined emotion data "confused" determined from the emotion data at the operation stage (the operation screen state of input, selection, and confirmation) corresponding to the emotion data of the user's facial expression based on the imaging data. Since the corresponding message is output to the screen of the interface section, it is possible to give advice and guidance to the user about various operation stages, and an automated teller machine that can provide support to eliminate mistakes and confusion about the operation of the automated teller machine. Obtainable.

1 Camera 2 Operation display unit 3 Paper media processing unit 4 Cash processing unit 5 Card reader / receipt Printer unit 6 Automatic calling telephone unit 7 Storage unit 8 Control unit 9 Emotion recognition artificial intelligence unit 10 Communication unit 11 Server 12 Operator room 12a Operator terminal IF Interface part

Claims (10)

An automated teller machine having an interface section that has a screen that accepts user operations and displays information to the user.
An imaging unit that captures a part of the body including the user's face and generates imaging data,
It has an emotion recognition artificial intelligence unit that recognizes the user's emotions based on the imaged data and generates emotion data.
An automated teller machine that outputs a message corresponding to a predetermined emotion data determined from the emotion data at an operation stage corresponding to the emotion data to the screen of the interface unit. A communication unit capable of communicating with the server is provided, and a request for screen information of the current operation stage corresponding to the emotion data and question data corresponding to the predetermined emotion data is transmitted to the server via the communication unit. The automatic teller machine according to claim 1. It is characterized in that the question data corresponding to the predetermined emotion data transmitted by the server and received via the communication unit is output to the screen of the interface unit as a message corresponding to the predetermined emotion data. The automated teller machine according to claim 2. The operator terminal is provided with an automatic call telephone unit capable of making a call, and the message corresponding to the predetermined emotion data includes a non-applicable option in which there is no corresponding question data, and the handset of the automatic call telephone unit is picked up. The automatic teller machine according to claim 3, wherein a message prompting the user is output to the screen of the interface unit. The automatic trading device according to claim 4, wherein when the handset of the automatic calling telephone unit is lifted after displaying the message including the option not applicable, a call is established with the operator terminal. A storage unit for storing screen information of the current operation stage corresponding to the emotion data and question data corresponding to the predetermined emotion data is provided, and the question data corresponding to the predetermined emotion data is converted into the predetermined emotion data. The automatic trading device according to claim 1, wherein the corresponding message is output to the screen of the interface unit. The operator terminal is provided with an automatic call telephone unit capable of making a call, and the message corresponding to the predetermined emotion data includes a non-applicable option in which there is no corresponding question data, and the handset of the automatic call telephone unit is picked up. The automatic teller machine according to claim 6, wherein a message prompting the user is output to the screen of the interface unit. The automated teller machine according to claim 7, wherein when the handset of the automatic calling telephone unit is lifted after displaying the message including the option not applicable, a call is established with the operator terminal. The automated teller machine according to any one of claims 1 to 8, wherein the determined predetermined emotion data is a negative emotion result that the user needs support. The automatic transaction according to any one of claims 1 to 9, wherein when the determined predetermined emotion data is analyzed, a transfer fraud alert message is output to the screen of the interface unit. apparatus.

Images