JP7358956B2 - automatic transaction device - Google Patents

Description

The present invention relates to an automatic transaction device.

Automated teller machines (ATMs) and cash dispensers (CDs) of financial institutions are known as automatic transaction devices. Automatic transaction devices are installed in branches of financial institutions such as banks, stores such as convenience stores, and facilities of transportation facilities such as stations. Users can perform transactions such as deposits, withdrawals, transfers, and balance inquiries by performing various operations using the input keys and touch panel according to the display screen of the interface section of the automatic transaction device.

Generally, automated transaction machines are equipped with an automated calling telephone. An automatic calling telephone is known as a response system in which the user picks up the handset of the automatic calling telephone to call a specific party, such as an operator, and the operator can respond to the user's inquiry (issue) (Patent Document 1) ).

Furthermore, a system is known in which a robot provides appropriate operational guidance for an automatic processing machine to a customer depending on the customer's situation or the processing state of the automatic processing machine (Patent Document 2).

Japanese Patent Application Publication No. 2001-34812 JP2017-87411A

The automatic transaction device system of Patent Document 1 cannot respond to the user's problem unless the user takes action such as picking up the phone. Therefore, there is a problem in that it is not possible to respond to users or solve problems without information from users.

The operation guidance system disclosed in Patent Document 2 is based on captured face image data of a customer captured by an image capture unit included in a robot, and emotional data that associates the face image data stored in a storage unit with human emotions. Alternatively, the customer's facial expression is recognized and the customer's emotion is estimated and recognized from the photographed customer's facial image data (Paragraphs 0062, 0150). Data on emotions and facial expressions recognized from the voices uttered and data on the age and gender estimation results of the customer are stored in a predetermined database, and the stored information is used, for example, for marketing related to the advertisement. (Paragraph 0151), and there is a problem in that the operation guidance does not reflect the customer's recognized emotions at all. There is a problem that mobile robots require a large amount of space.

The present invention was created in view of the above-mentioned problems, and it is possible to provide advice and guidance at various stages of operation by utilizing emotions based on various facial expressions of the user in front of the interface of the automated transaction device. One of the challenges is to provide automatic transaction equipment. Another purpose of the emotion determination of the present invention is to provide support for eliminating mistakes and confusion made by users regarding the operation of automatic transaction devices, for example.

The automatic transaction device of the present invention is an automatic transaction device having an interface section having a screen that receives operations from a user and displays information to the user,
an imaging unit that generates imaging data by imaging a part of the user's body including the user's face;
an emotion recognition artificial intelligence unit that recognizes the user's emotion based on the imaging data and generates emotion data;
A message corresponding to predetermined emotional data determined from the emotional data is output to the screen of the interface unit in the operation step corresponding to the emotional data.

According to the present invention, in a compact automatic transaction device equipped with a camera, a message corresponding to predetermined emotional data determined from the emotional data of the user is sent to the Since it is output to the screen of the interface section, it is possible to advise and guide the user regarding various operations, and it is possible to provide the user with support to eliminate mistakes and confusion regarding the operation of the automatic transaction device.

1 is a schematic perspective view showing an automatic transaction device according to an embodiment of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS It is a schematic explanatory diagram which shows an example of the structure of the monitoring system containing the automatic transaction device of a present Example. It is a flowchart showing the flow of operations when a user performs a transaction to transfer money using a "cash card" in the automatic transaction device according to the present embodiment.

DESCRIPTION OF THE PREFERRED EMBODIMENTS An example of an automated teller machine (hereinafter also referred to as an ATM) that handles cash according to an embodiment of the present invention will be described below with reference to the drawings. Note that the present invention is not limited to the following examples. Note that, in the embodiments, components having substantially the same functions and configurations are designated by the same reference numerals and redundant explanation will be omitted.

(ATM configuration)
As shown in FIG. 1, an ATM is provided with an interface section IF on the front side of the ATM that allows a user to easily perform transaction operations while facing the ATM. The interface unit IF has a function of transmitting transaction-related information and receiving operational instructions from users. The interface unit IF includes a camera 1 as an imaging unit, an operation display unit 2, a paper media processing unit 3 having a paper media input/output port 3a for inputting/outputting paper media such as passbooks and payment slips, and a paper media processing unit 3 for inputting and outputting paper media such as banknotes and payment slips. A cash processing unit 4 having a cash input/output port 4a, a card reader/receipt printer unit 5 having a card input/output port 5a and a receipt issuing port 5b, and a receiver 6a of an automatic calling telephone unit (not shown) are provided. ing.

In addition to the interface section IF, the ATM also includes a storage section 7 in which a program for controlling the operation of the ATM is stored, and a CPU that stores the program stored in the storage section 7, as shown in FIG. (Central Processing Unit) to control the operation of each functional part of the ATM, an emotion recognition artificial intelligence part 9, and an automatic call telephone part 6 of the main device of the handset 6a. .

The camera 1 has a function of acquiring imaging data of moving images and still images. The camera 1 itself is placed at a position where the user's entire face can be photographed and recognized. For example, the camera 1 can acquire an image of a part of the user's body, including the user's face, and an image of the ATM surroundings, including an image of the area around the ATM. The camera 1 may be mounted on the operation display section 2 of the ATM, for example, inside a casing behind a rear confirmation half mirror (not shown) so as not to be visible from the outside.

The operation display unit 2 is a touch panel that displays an operation screen for transactions instructed by the control unit 8 and allows selection of the type of transaction and input of a password, transaction amount, etc. The operation display unit 2 receives selections and inputs from the user, transmits the instruction contents to the control unit 8, and displays an operation screen for checking various information and inputting selections from the control unit 8 to the user. . The touch panel is an electronic component that combines an image display device such as a liquid crystal display and a position input device such as a touch pad, and senses screen position information by touching the display on the screen and transmits the information to the control unit 8. Based on the control of the control unit 8, the operation screen (hereinafter also simply referred to as screen) of the operation display unit 2 displays transaction types such as “deposit” and “withdrawal” as an initial screen. Then, the user can proceed with the transaction by touching a display part such as a button for a desired operation on the screens of various operation stages displayed under the control of the control unit 8.

The paper media processing unit 3 takes in paper media such as payment slips and passbooks from the paper media input/output port 3a, processes the paper media based on the control of the control unit 8, and discharges the paper media from the paper media input/output port 3a. . The passbook processing function of the paper medium processing unit 3 is to read and write magnetic stripe data in a passbook inserted by the user, and to print out transaction details (bookkeeping data). The payment slip processing function of the paper medium processing unit 3 is to optically read the payment slip inserted into the passbook entrance by the user, or print it if necessary.

The cash processing unit 4 performs processing related to cash deposits and withdrawals (transporting and storing cash from the cash deposit/withdrawal port 4a, determining the authenticity and denomination of cash, etc.) under the control of the control unit 8, and converts cash into cash. It can be discharged from the deposit/withdrawal port 4a.

The card reader receipt printer section 5, under the control of the control section 8, stores user identification information (for example, name, account number, etc.) recorded on a card medium such as a cash card inserted into the card slot 5a by the user. ) and has the function of recognizing the user. The card reader receipt printer section 5 can also read and write data recorded on the card medium. The card reader receipt printer section 5 prints the transaction result represented by the signal on a statement sheet in response to the transaction statement print signal supplied from the control section 8, and discharges it from the receipt issuing port 5b. .

As shown in FIG. 2, the automatic transaction machine (ATM) in the monitoring system, the server 11, and the operator terminal 12a in the operator room 12 are all connected to the communication network NW, and each can transmit control/information data and voice information via the communication network NW. Data can be transmitted. The automatic calling telephone section 6 has a function of outputting voice via the handset 6a and a function of acquiring voice information. The automatic calling telephone unit 6 is a communication device that automatically calls a predetermined party simply by a user lifting the handset 6a. When the user picks up the handset 6a of the automatic call telephone section 6, for example, the automatic call telephone section 6 calls the operator room 12 that manages the ATM, and the user can talk to the operator via the receiver 6a and the operator terminal 12a. Note that the automatic calling telephone section 6 can be provided with a function of transmitting an announcement of voice guidance for operation to the user via the receiver 6a.

The emotion recognition artificial intelligence unit 9 is connected to the camera 1 and the control unit 8, and analyzes the user's emotions from the image from the camera 1 based on the movement of the muscles of the user's face and the movement of the line of sight, and detects joy, surprise, fear, anger, etc. Extract emotions from analysis results such as disgust, sadness, contempt, satisfaction, confusion, excitement, guilt, satisfaction, impatience, etc. The analysis results are transmitted to the control section 8. That is, the emotion recognition artificial intelligence unit 9 performs emotion analysis based on the captured data of the user's facial expression captured by the camera 1 to generate emotion data, and outputs this to the control unit 8 . The emotion recognition artificial intelligence unit 9 performs emotion judgment processing based on emotion data in the CPU and memory (ROM, RAM, HDD, etc.) in the control unit 8 through cooperation between the CPU and memory based on a program installed in the memory. It may be configured to do so. The emotion recognition software program is, for example, stored in advance in the storage unit 7 provided in the device, and each program is read into the RAM and executed by the CPU at the time of execution, for example. Emotion recognition software programs include Microsoft's Face API and Affectiva's Affdex. In the received analysis results (emotional data), the control unit 8 determines whether the user needs support, such as ``confused'' or ``impatient'' (hereinafter also simply referred to as ``confused''), which are determined predetermined emotional data. Detect the presence or absence of negative emotional results. This is called an emotion determination step by the emotion recognition artificial intelligence section 9 and the control section 8.

The communication unit 10 is a communication module for transmitting and receiving data with other information processing devices (communicates with the server 11 and the operator terminal 12a of the operator room 12 via the communication network NW) by wire or wirelessly. . The communication unit 10 transmits the user's emotional analysis results, call establishment requests, call disconnection requests, screen operation data, etc. to the server 11. It also receives messages about major questions from the server 11 and transmits them to the control unit 8. For example, the communication unit 10 is connected to a LAN (Local Area Network), a telephone line, etc., and communicates with the operator terminal 12a and the server 11 via the communication network NW. The communication unit 10 transmits and receives information regarding processing performed at the ATM to and from the server 11 via the communication network NW. The communication network NW is a wired or wireless transmission path for information transmitted from the connected communication units 10. For example, the communication network NW may include a public line network (the Internet, a telephone line network, a satellite communication network), various LANs, or a WAN (Wide Area Network). Furthermore, the communication network NW may include a dedicated line network such as an IP-VPN (Internet Protocol-Virtual Private Network).

The components used in this embodiment are mainly the camera 1, the operation display section 2, the emotion recognition artificial intelligence section 9, the control section 8, the automatic call telephone section 6, and the communication section 10. Detailed description of the configuration other than the elements will be omitted, and the emotion determination step and the confusion handling step will be described.

As shown in FIG. 2, the automatic transaction machine (ATM), the server 11, and the operator terminal 12a in the operator room 12 are all connected to the communication network NW, and each transmits control/information data and voice data via the communication network NW. .

A: Emotion determination step for automated teller machine (ATM) Capturing the facial expression of a user when a screen for one operation step of a certain transaction is displayed (a state that includes screen information for the operation step) The emotion recognition artificial intelligence unit 9 analyzes the user's emotions based on the video captured by the camera 1, generates emotion data, and transmits the analysis results to the control unit 8. When the control unit 8 receives the analysis result (emotional data), it detects whether there is an emotional result such as "confused" or "impatient" that indicates that the user needs support (confused emotion determination). The above processing performed by the emotion recognition artificial intelligence section 9 and the control section 8 is an emotion determination step.

B: Initial stage of starting the confusion response processing step in the automated teller machine (ATM) After the emotion determination step, if the control unit 8 determines that the desired emotion result is null, it displays the screen for the next stage on the operation display unit 2. Display. After the emotion determination step, if the control unit 8 determines that the desired emotion result is present (that is, confused), the control unit 8 instructs the operation display unit 2 to output a message indicating whether support is required. When the user inputs an input indicating that support is “required” in response to a message on the screen of the operation display unit 2, the input information is transmitted from the operation display unit 2 to the control unit 8. When the support request is "required", the control unit 8 transmits a request for question data to the server 11 via the communication unit 10 along with current screen information (screen information at the current operation stage). The process of transmitting a request for question data by the control unit 8 is the first stage of starting the confusion handling process step. If the user inputs "no support required", the control unit 8 continues the process (displays the screen for the next stage).

C: Middle stage of start of confusion response processing step in server 11 After receiving the signal from the communication unit 10, the server 11 transmits question data regarding the current screen information to the communication unit 10. The process of transmitting the question data by the server 11 is the middle stage of starting the puzzle handling process step.

D: Start and final stage of confusion handling processing steps in automatic transaction machine (ATM) The communication unit 10 receives question data from the server 11 and transmits it to the control unit 8 . The control section 8 instructs the operation display section 2 to output the received question data. The operation display unit 2 outputs the question data (options) from the control unit 8 on the screen, and prompts the user to input which question is applicable or not. As a result, user support is started through data exchange between the operation display section 2, the control section 8, the communication section 10, and the server 11. If there is no question (option) that matches the question (option) displayed on the screen, the user enters "Not applicable" on the screen. In this case, the operation display section 2 notifies the control section 8 that "not applicable" has been input. The control unit 8 instructs the operation display unit 2 to output a receiver message urging the user to pick up the receiver 6a, and the operation display unit 2 displays the receiver message. The process of transmitting question data by the ATM is the start and final stage of the confusion handling process step.

E: 1st stage of user support in the confusion handling process step in an automated teller machine (ATM) When the user lifts up the handset 6a by looking at the handset message, the automatic calling telephone section 6 notifies that the handset 6a has been lifted. detects it and transmits it to the control section 8. As a result, the control unit 8 requests the server 11 via the communication unit 10 to establish a telephone connection with the operator.

F: Two stages of user support in the confusion response processing step at the server 11 The server 11 that receives the call connection request establishes a call between the available operator terminal 12a and the handset 6a of the automatic transaction machine (ATM). , audio data is transmitted and received between them.

G: Three stages of user support in the confusion response processing step at the server 11 At the same time as the call is established, the server 11 sends the support information that the user received from the automatic transaction machine (ATM) (current operating stage) to the operator terminal 12a. screen information). The operator responds to the user's problem based on the received support information via the operator terminal 12a.

As described above, by using the camera 1 and the emotion recognition artificial intelligence unit 9, the user's emotions are analyzed, and a message is output from the screen to support the user sent from an automatic transaction machine (ATM). can.

Further, by preparing in advance countermeasures for major questions on the server 11 and outputting them on the screen, it is possible to obtain the effect that the user's problems can be solved by the automatic transaction device without being connected to the operator terminal 12a. Furthermore, screen information of the current operation stage corresponding to the emotion data and question data corresponding to the predetermined emotion data "confused" are stored in the storage unit 7 of the automatic transaction machine (ATM), and the screen information corresponding to the predetermined emotion data "confused" is stored. By outputting the corresponding question data to the screen of the interface section IF as a message corresponding to the predetermined emotion data, the user's problem can be solved at the ATM itself without connecting to the server 11 and operator terminal 12a. Effects can be obtained.

Furthermore, if the control unit 8 analyzes the remittance screen as "confused" or "impatient," a message alerting the user to remittance fraud is output on the screen of the interface unit IF, thereby strengthening countermeasures against remittance fraud. is also possible.

Next, an example of the operation flow of the ATM of this embodiment when a user makes a transfer using a "cash card" will be explained. FIG. 3 is a flowchart showing the flow of operations when a user performs a money transfer transaction using a "cash card" in the automatic transaction device according to the present embodiment.

The ATM starts operation by putting each component on standby in step S1.

In order for a user to start a transaction at an ATM, it is necessary to select the transaction on the initial screen of the interface section IF. The control unit 8 detects whether there is a movement of the user from the image data from the camera 1 (step S2), and when the movement is detected, starts emotion detection. If not, the process returns to step S1 and the ATM waits until movement is detected again. In the case of the transfer transaction in this example, the user selects the "Transfer" button display on the initial screen.

If the selection of the button is detected and the process proceeds to step S3, selection buttons for "cash" and "cash card" are displayed on the screen (transfer method selection screen). In this screen state, the emotion recognition artificial intelligence unit 9 and the control unit 8 determine the emotion of the selection (step S3).

Next, it is determined whether the result of determination step S3 is confusion (step S4). When the control unit 8 determines that there is confusion (Y), the process proceeds to step S5, and a puzzle handling process is performed. In this example, when the user wants to transfer cash, he or she presses the "cash card" button instead of "cash" to select payment from the user's account. Therefore, when the control unit 8 determines that there is no confusion "N", the process proceeds to step S6.

If the process proceeds to step S6, a message is displayed on the screen to prompt the user to enter his/her own PIN number for authentication (PIN number input screen). The user enters the PIN number. In this screen state, the emotion recognition artificial intelligence unit 9 and the control unit 8 determine the emotion of the input (step S6).

Next, it is determined whether the result of the emotion determination step S6 is confusion (step S7). If the control unit 8 determines that there is confusion (Y), the process proceeds to step S8, and a puzzle handling process is performed. When the control unit 8 determines that there is no confusion ("N"), the process advances to step S9.

If the process advances to step S9, a message prompting the user to input his or her own telephone number is displayed on the screen (telephone number input screen). The phone number will be used by the bank to contact you if a mistake is discovered. User enters phone number. In this screen state, the emotion recognition artificial intelligence unit 9 and the control unit 8 determine the emotion of the input (step S9).

Next, it is determined whether the result of the emotion determination step S9 is confusion (step S10). When the control unit 8 determines that there is confusion (Y), the process proceeds to step S11, and a puzzle handling process is performed. When the control unit 8 determines that there is no confusion "N", the process advances to step S12.

If the process proceeds to step S12, the user's (self) account name will be displayed on the screen if the transfer is made using a "cash card". (Name input confirmation screen). User confirms name. This name does not have to be the user's name as long as it can be understood by the payment source. In this screen state, the emotion recognition artificial intelligence section 9 and the control section 8 perform emotion determination for the confirmation (step S12).

Next, it is determined whether the result of the emotion determination step S12 is confusion (step S13). When the control unit 8 determines that there is confusion (Y), the process proceeds to step S14, and a puzzle handling process is performed. When the control unit 8 determines that there is no confusion "N", the process advances to step S15.

If the process proceeds to step S15, selection buttons for selecting the financial institution to which the transfer is to be made or the name of the branch are displayed on the screen (financial institution input screen). The user presses the appropriate financial institution name or branch name button from the respective selection button lists, selects the financial institution to which the transfer is to be made, and selects the branch name. In this screen state, the emotion recognition artificial intelligence unit 9 and the control unit 8 determine the emotion of the selection (step S15).

Next, it is determined whether the result of the emotion determination step S15 is confusion (step S16). When the control unit 8 determines that there is confusion (Y), the process proceeds to step S17, and a puzzle handling process is performed. When the control unit 8 determines that there is no confusion ("N"), the process advances to step S18.

If the process proceeds to step S18, a selection button for selecting the account item (account type) to which the transfer is to be made is displayed on the screen (account item selection screen). The user selects a subject by pressing the appropriate subject button from the selection button list. In this screen state, the emotion recognition artificial intelligence unit 9 and the control unit 8 determine the emotion of the selection (step S18).

Next, it is determined whether the result of the emotion determination step S18 is confusion (step S19). When the control unit 8 determines that there is confusion (Y), the process proceeds to step S20, and a puzzle handling process is performed. When the control unit 8 determines that there is no confusion "N", the process advances to step S21.

If the process proceeds to step S21, a prompt to input the account number of the transfer destination is displayed on the screen (transfer destination account number input screen). The user enters the bank account number. In this screen state, the emotion recognition artificial intelligence unit 9 and the control unit 8 determine the emotion of the input (step S21).

Next, it is determined whether the result of the emotion determination step S21 is confusion (step S22). When the control unit 8 determines that there is confusion (Y), the process proceeds to step S23, and a puzzle handling process is performed. When the control unit 8 determines that there is no confusion "N", the process advances to step S24.

If the process proceeds to step S24, a prompt to input the transfer amount is displayed on the screen (transfer amount input screen). The user enters the bank account number. In this screen state, the emotion recognition artificial intelligence unit 9 and the control unit 8 determine the emotion of the input (step S24).

Next, it is determined whether the result of the emotion determination step S24 is confusion (step S25). When the control unit 8 determines that there is confusion (Y), the process proceeds to step S26, and a puzzle handling process is performed. When the control unit 8 determines that there is no confusion ("N"), the process proceeds to step S27.

If the process proceeds to step S27, a prompt to confirm the details of the transfer is displayed on the screen (transfer confirmation screen). In the case of a cash card transfer, the screen switches to a confirmation screen for the transfer details, and if there are no discrepancies in the details, the user presses the "Confirm" button, and if there is a mistake, the user presses the "Correct" button to correct it, but returns to the correction step ( Step S27).

Next, if the "Confirm" button is selected, a message indicating completion will be displayed on the screen. When making a transfer using a cash card, the transaction is completed and a transfer statement is printed.

According to this embodiment, the predetermined emotional data "confused" determined from the emotional data at the operation stage (operation screen state of input, selection, and confirmation) corresponding to the emotional data of the user's facial expression based on the imaging data. Since the corresponding messages are output on the screen of the interface section, it is possible to advise and guide the user regarding various operation steps, and the automatic transaction device can provide support to eliminate mistakes and confusion regarding the operation of the automatic transaction device. Obtainable.

1 Camera 2 Operation display section 3 Paper media processing section 4 Cash processing section 5 Card reader receipt printer section 6 Automatic call telephone section 7 Storage section 8 Control section 9 Emotion recognition artificial intelligence section 10 Communication section 11 Server 12 Operator room 12a Operator terminal IF Interface section

Claims (8)

An automatic transaction device having an interface section having a screen that receives operations from a user and displays information to the user,
an imaging unit that generates imaging data by imaging a part of the user's body including the user's face;
an emotion recognition artificial intelligence unit that recognizes the user's emotion based on the imaged data and generates emotion data;
a control unit that determines whether or not the user has negative emotions based on the emotion data;
It has a communication unit that can communicate with the server ,
When the control unit determines that the negative emotional result is present, the control unit outputs a message to the screen of the interface unit indicating whether support corresponding to the negative emotional result is necessary ;
When the input information requiring the support is input, the control unit requests screen information of the operation stage when the negative emotional result was determined and question data corresponding to the negative emotional result, Send to the server via the communication unit
An automatic transaction device characterized by: The control unit outputs question data corresponding to the negative emotion result transmitted by the server and received via the communication unit to the screen of the interface unit as a message corresponding to the negative emotion result. The automatic transaction device according to claim 1 , characterized in that: The control unit includes an automatic calling telephone section that can make a call to an operator terminal, and the control section includes an option of "Not applicable" in which there is no corresponding item in the question data in the message corresponding to the negative emotional result , and 3. The automatic transaction device according to claim 2 , wherein a message prompting the user to pick up the telephone receiver is output to the screen of the interface unit. The automatic transaction according to claim 3 , wherein the control unit establishes a call with the operator terminal when the handset of the automatic calling telephone unit is lifted after displaying the message including the not applicable option. Device. An automated transaction device having an interface section having a screen that receives operations from a user and displays information to the user,
an imaging unit that generates imaging data by imaging a part of the user's body including the user's face;
an emotion recognition artificial intelligence unit that recognizes the user's emotion based on the imaged data and generates emotion data;
a control unit that determines whether or not the user has negative emotions based on the emotion data;
a storage unit that stores screen information of an operation stage corresponding to the negative emotion result and question data corresponding to the negative emotion result ,
When the control unit determines that the negative emotional result is present, the control unit outputs a message to the screen of the interface unit indicating whether support corresponding to the negative emotional result is necessary;
When the input information requiring the support is input, the control unit converts the stored screen information of the operation stage and the question data corresponding to the negative emotional result into a response to the negative emotional result . An automatic transaction device characterized in that the automatic transaction device outputs a message to the screen of the interface section. The control unit includes an automatic calling telephone section that can make a call to an operator terminal, and the control section includes an option of "Not applicable" in which there is no corresponding item in the question data in the message corresponding to the negative emotional result , and 6. The automatic transaction device according to claim 5 , further comprising: outputting a message to the screen of the interface section to prompt the section to pick up the telephone receiver. The automatic transaction according to claim 6 , wherein the control unit establishes a call with the operator terminal if the handset of the automatic calling telephone unit is lifted after displaying the message including the not applicable option. Device. Claim 1, wherein when the operation by the user is a money transfer operation, the control unit outputs a message alerting to money transfer fraud to the screen of the interface unit if the control unit analyzes the negative emotion result. The automatic transaction device according to any one of items 7 to 7 .

Images