WO2015141051A1

WO2015141051A1 - Trading system and trading device

Info

Publication number: WO2015141051A1
Application number: PCT/JP2014/078664
Authority: WO
Inventors: 悠介柴崎
Original assignee: 沖電気工業株式会社
Priority date: 2014-03-20
Filing date: 2014-10-28
Publication date: 2015-09-24
Also published as: JP2015184762A; JP6191519B2

Abstract

The present invention relates to a trading system comprising a trading device and a terminal device. The trading device accepts operations by an operator, and presents an operating screen displaying information relating to trading. The trading device is provided with an operating screen presentation unit capable of attaching to an object capable of being operated on, at least on the operating screen, an encoded image in which tag data for identifying content to be displayed on the terminal device have been encoded. The terminal device analyses an image captured by an image-capturing unit, and if it detects an encoded image, it displays a composite image in which an image of the content corresponding to the encoded image is superimposed on the captured image.

Description

Transaction system and transaction apparatus

The present invention relates to a transaction system and a transaction apparatus, for example, a transaction system including an ATM (Automated Teller Machine).

Conventionally, the technology using Augmented Reality (hereinafter referred to as AR) has been utilized in various technologies in each industry. AR is a technique for superimposing information such as digital composition on a real landscape.

In the system using the AR technology, there are two methods for specifying a real landscape to be overlaid. The first method is a specifying method based on position information using GPS or the like, and the second method is a specifying method using image analysis by a camera. Furthermore, among image analysis by cameras, there are a method for adding information to 2D data called “markers” that can be recognized by a computer, and a method for recognizing images of actual objects and figures and adding information. .

And, conventionally, there is a security technology for a personal identification number using AR even in a transaction device such as ATM. Location information is used as a method for identifying a real landscape. Based on the position information, the ATM to be overlaid is specified. Even on the password input screen, the password input button is not displayed on the ATM customer operation screen. Instead of display, virtual buttons are displayed in a superimposed manner through a head mounted display attached by the ATM user. The user touches the ATM customer operation screen using the superimposed video and performs input.

However, in the technique described in Japanese Patent Application Laid-Open No. 2012-174208, since there is no display on the actual ATM screen, it is impossible to determine whether the correct key is pressed when viewed from the operator. There is a problem that key errors are likely to occur. In ATM, if a mistaken operation is performed a certain number of times with a personal identification number, the transaction will be stopped, or the amount of money will be erroneously manipulated.

Therefore, there is a demand for a transaction system and a transaction apparatus that can support operations related to customer transactions using a terminal device that displays augmented reality.

According to the first aspect of the present invention, the transaction system includes a transaction device that performs a transaction with an operator, and a terminal device that includes an imaging unit and a display unit that displays an image acquired by the imaging unit. (1) The transaction apparatus presents an operation screen for receiving the operation of the operator and displaying information related to the transaction, and at least an object that can accept the operation on the operation screen, the terminal An operation screen presentation unit capable of attaching an encoded image in which tag data for identifying content to be displayed on the device is encoded, and (2) the terminal device is (2-1) content data (2-2) When the encoded image is detected by analyzing the image captured by the imaging unit, the content data corresponding to the encoded image is stored in the container. Acquired from the data tree holding portion is held, a composite image in which the imaging unit is superposed the image of the contents acquired in the image captured and an image processing unit for displaying on the display unit.

According to the second aspect of the present invention, the transaction device presents an operation screen for receiving information on the transaction and accepting the operation of the operator, and is an object that can accept the operation on the operation screen. And an operation screen presenting unit capable of attaching an encoded image in which tag data for identifying content to be displayed on the terminal device possessed by the operator is encoded.

According to the present invention, it is possible to provide a transaction system and a transaction apparatus that can support an operation related to a customer transaction using a terminal device that displays augmented reality.

It is the block diagram shown about the whole structure of the WebATM system which concerns on 1st Embodiment. It is the block diagram shown about the internal structure of the WebAP server which concerns on 1st Embodiment. It is the block diagram shown about the internal structure of the customer information database server which concerns on 1st Embodiment. It is explanatory drawing shown about the example of the data structure memorize | stored in the customer information database server which concerns on 1st Embodiment. It is the block diagram shown about the structural example of the smart phone terminal which concerns on 1st Embodiment. It is a top view shown about the appearance of the smart phone terminal concerning a 1st embodiment. It is a bottom view shown about the appearance of the smart phone terminal concerning a 1st embodiment. It is the block diagram shown about the structural example of AR content download server which concerns on 1st Embodiment. It is the sequence diagram shown about operation | movement of the WebATM system which concerns on 1st Embodiment. It is explanatory drawing shown about the structural example of the screen displayed by ATM of 1st Embodiment. It is explanatory drawing shown about the example of AR content displayed with the smart phone terminal of 1st Embodiment. It is explanatory drawing shown about the example of the synthesized image displayed with the smart phone terminal of 1st Embodiment. It is the block diagram shown about the structural example of the WebAP server which concerns on 2nd Embodiment. It is explanatory drawing shown about the structural example of the screen displayed by ATM of 2nd Embodiment. It is explanatory drawing (the 1) shown about the example of the synthesized image displayed with the smart phone terminal of 2nd Embodiment. It is explanatory drawing (the 2) shown about the example of the synthesized image displayed with the smart phone terminal of 2nd Embodiment. It is the block diagram shown about the whole structure of the WebATM system which concerns on 3rd Embodiment. It is explanatory drawing (the 1) shown about the structural example of the screen displayed by ATM which concerns on 3rd Embodiment. It is explanatory drawing (the 1) shown about the structural example of the synthesized image displayed with the smart phone terminal which concerns on 3rd Embodiment. It is explanatory drawing (the 2) shown about the structural example of the screen displayed by ATM which concerns on 3rd Embodiment. It is explanatory drawing (the 2) shown about the structural example of the synthesized image displayed with the smart phone terminal which concerns on 3rd Embodiment.

(A) First Embodiment Hereinafter, a first embodiment of a transaction system and a transaction apparatus according to the present invention will be described in detail with reference to the drawings. This embodiment demonstrates the example which applied the transaction system and transaction apparatus of this invention to the WebATM system and ATM.

(A-1) Configuration of the First Embodiment FIG. 1 is an explanatory diagram showing the overall configuration of the WebATM system 1 of this embodiment.

First, an overall outline of the WebATM system 1 will be described.

The WebATM system 1 includes an ATM 10, a smartphone terminal 20 as a terminal device, a customer information database server 30, an AR content download server 40 as a content providing device, a billing system host 50, and a WebAP server 60 as an operation screen providing device. ing.

The WebATM system 1 is an ATM system in which the ATM 10 is made into a thin client with the WebAP server 60 as the center. The basis of the WebATM system 1 includes an ATM 10, a WebAP server 60, and a billing system host 50.

In the WebATM system 1, the transaction application (content of the transaction operation screen) provided by the ATM 10 is stored in the WebAP server 60. The content stored in the WebATM system 1 includes a page (operation screen) with an AR marker. Further, in the WebATM system 1, transaction information is provided from the account host 50 to the ATM 10 via the WebAP server 60.

Furthermore, in the WebATM system 1, the AR content is stored in the AR content download server 40 and provided to the smartphone terminal 20 by communication.

The content data to be overlaid on the AR marker is stored in the AR content download server 40. The AR content download server 40 is connected to a customer information database server 30 that manages customer information.

In the first embodiment, content to be superimposed on (attached to) an AR marker (hereinafter referred to as “AR content”) is stored in the WebAP server 60 and downloaded to the smartphone terminal 20 via the AR content download server 40. The When the AR content is stored in the AR content download server 40, it may be stored offline via a medium or the like, not via the network. And the smart phone terminal 20 can superimpose and display the image of AR content using the AR marker on the customer operation screen of the ATM 10 as a mark.

Next, an example of the internal configuration of each device constituting the WebATM system 1 will be described.

First, the internal configuration of the ATM 10 will be described.

The ATM 10 includes a control unit 11 that controls the entire apparatus, a customer operation screen 12 as an operation screen presentation unit configured using a touch panel display, a passbook unit 13 that performs processing of the inserted passbook (for example, passbook entry, etc.), It has a card unit 14 for processing the inserted card (for example, reading data) and a cash processing unit 15 for depositing and withdrawing cash. Further, it is assumed that a telephone 80 capable of making a call with a call center operator is also arranged in the vicinity of the ATM 10.

The control unit 11 can be constructed by installing the transaction program of the embodiment in a computer including a CPU, a memory, and the like. The control unit 11 includes at least a main control program 111 for controlling each component in the apparatus and a web browser 112. The Web browser 112 accesses the WebAP server 60 by communication, acquires Web content related to the transaction operation screen, displays the operation screen on the customer operation screen 12, and performs processing for providing the customer. That is, the ATM 10 is configured to operate as a thin client including only the Web browser 112 as an application.

Next, the internal configuration of the WebAP server 60 will be described with reference to FIG.

The WebAP server 60 is connected to the Web server process 61 serving as an operation screen data providing unit that provides a web page (operation screen related to transaction processing) to the ATM 10 and the accounting host 50, and information processing (transaction) associated with the transaction processing. And a content storage unit 63 for storing Web content (operation screen related to transaction processing) and AR content data.

The content storage unit 63 stores ATM display content 64, which is a Web page to be supplied to the ATM 10, and AR content data 65 to be supplied to the smartphone terminal 20 via the AR content download server 40. . In the AR content data 65, data of AR content (content such as image data and text data) corresponding to each tag data obtained by decoding an AR marker (encoded image including contents obtained by encoding tag data) is stored. It shall be included.

Next, the internal configuration of the customer information database server 30 will be described with reference to FIG.

The customer information database server 30 includes a customer information database storage unit 32 for managing and storing customer data, and a database for providing data of the customer information database storage unit 32 to the AR content download server 40 as a database client. It has a server process 31.

The customer information database storage unit 32 stores data for managing information on customers who use the ATM 10 and the smartphone terminal 20. In this embodiment, the information stored in the customer information database storage unit 32 is assumed to have contents as shown in FIG.

As shown in FIG. 4, the customer information database storage unit 32 stores information including a customer name for each customer and a customer ID for identifying each customer's smartphone terminal 20 (customer ID used for application login). It shall be.

Next, the internal configuration of the smartphone terminal 20 will be described with reference to FIG.

The smartphone terminal 20 includes a control unit 21, a touch panel display 22, a camera 23, and a communication unit 24 as a network interface that performs wireless communication.

The control unit 21 performs various data processing and control of each device. The control unit 21 is configured by installing various control programs in a computer including a CPU, a memory, and the like. In this embodiment, it is assumed that an AR application 211 for processing at least AR content is installed in a computer constituting the control unit 21.

The AR application 211 serving as a content holding unit and an image processing unit displays an image captured by the camera 23 (hereinafter referred to as “camera captured image”) on the touch panel display 22 and analyzes the captured image to detect an AR marker. Check if there is any. When the AR application 211 detects an AR marker, the AR application 211 acquires AR content corresponding to the AR marker via the AR content download server 40, and an image based on the acquired AR content (hereinafter referred to as “AR content image”). ) And a camera-captured image superimposed image (hereinafter referred to as “composite image”) is displayed on the touch panel display 22. It should be noted that various AR technology applications can be applied to the AR marker detection processing, AR content (AR content image) acquisition processing, and composition processing performed by the AR application 211.

Note that the AR application 211 may download and cache the necessary AR content data from the AR content download server 40 in advance, or acquire it whenever necessary (each time the AR marker is detected). It may be.

Further, a touch panel display 22 is disposed on one surface of the plate-shaped smartphone terminal 20 (see FIG. 6A), and a camera 23 is disposed on the other surface (rear surface) (see FIG. 6B). Thereby, the customer can view the composite image on the touch panel display 22 while imaging the portion including the AR marker with the camera 23 (while holding the smartphone terminal 20).

Next, the internal configuration of the AR content download server 40 will be described with reference to FIG.

The AR content download server 40 has an application server process 41 that provides AR content to the smartphone terminal 20 that operates as a client. The AR content download server 40 (application server process 41) accepts the login from the smartphone terminal 20 (login using the customer ID managed by the customer information database server 30), accepts the authentication, and accepts the smartphone terminal 20 that accepts the connection. to manage. Then, the application server process 41 performs processing for acquiring and providing the AR content corresponding to the request (tag data) from the logged-in smartphone terminal 20 from the WebAP server 60. Here, the application server process 41 performs processing to acquire tag data obtained by decoding the AR marker, request the Web AP server 60 to acquire AR content corresponding to the acquired tag data, and return the AR content to the smartphone terminal 20.

Note that the AR content download server 40 may download and cache all the AR content (AR content data 65 stored in the content storage unit 63) from the WebAP server 60 in advance. You may make it download every time (every request | requirement from AR content data 65). In this embodiment, the master data of the AR content is stored in the WebAP server 60. However, the storage position of the master data of the AR content is not limited, and is stored in the AR content download server 40. Alternatively, it may be stored only in each smartphone terminal 20 (stored together with the AR application 211).

(A-2) Operation of the First Embodiment Next, the operation of the WebATM system 1 of the first embodiment having the above configuration will be described.

First, the basic operation of transaction support using AR content in the WebATM system 1 will be described using the sequence diagram of FIG.

First, the AR application 211 is launched on the smartphone terminal 20 by the customer (S101), and a login request is made to the AR content download server 40 (S102).

Then, the AR content download server 40 performs authentication by a process of collating with the customer data of the customer information database server 30 based on the login request from the smartphone terminal 20, and here, it is assumed that the authentication is successful (S103). .

The authentication method in the AR content download server 40 is not limited. In the AR content download server 40, if there is no security problem when accepting login, the authentication process may be omitted. Further, the AR content download server 40 may accept a request for AR content from the smartphone terminal 20 without performing login processing.

Then, transaction processing starts at the ATM 10 in accordance with the operation of the customer (the customer who owns the smartphone terminal 20) (S104). Note that the login process (steps S101 to S103) by the smartphone terminal 20 may be after the transaction process in the ATM 10 is started.

Thereafter, in the ATM 10, in the course of transaction processing with the customer, the operation screen displayed on the customer operation screen 12 is changed, and the operation screen including the AR marker is displayed (S105).

Then, the customer operation screen 12 is imaged by the smartphone terminal 20 (camera 23) by the customer's operation, and the AR marker is detected by the smartphone terminal 20 (AR application 211) (S106).

When the AR marker is detected, the smartphone terminal 20 (AR application 211) analyzes the image of the AR marker, extracts tag data, attaches the tag data, and sends a data request for AR content to the AR content download server 40. Transmit (S107).

Upon receiving the AR content data request, the AR content download server 40 acquires the AR content corresponding to the tag data from the WebAP server 60 (S108) and transmits it to the smartphone terminal 20 (S109).

When the AR content is acquired, the smartphone terminal 20 (AR application 211) starts a process of displaying the image of the AR content and the image obtained by combining the camera captured image on the customer operation screen 12 (S110).

Next, a specific operation for supporting a transaction (operation) using the ATM 10 for a customer will be described.

FIG. 9 shows an operation screen displayed on the customer operation screen 12 when accepting the input of the password at the ATM 10.

FIG. 10 is an explanatory diagram showing an image of the AR content displayed on the touch panel display 22 of the smartphone terminal 20 when accepting the input of the password at the ATM 10. Originally, the smartphone terminal 2 displays a composite image obtained by combining the AR content image and the camera-captured image, but FIG. 10 shows only the AR content image extracted for the sake of simplicity.

FIG. 11 illustrates a composite image displayed on the touch panel display 22 of the smartphone terminal 20 when accepting the input of the password at the ATM 10. That is, FIG. 11 illustrates a state in which the camera captured image obtained by capturing the operation screen illustrated in FIG. 9 and the composite image obtained by combining the AR content image illustrated in FIG.

As shown in FIG. 9, the PIN code input operation screen (number input screen) displayed on the ATM 10 (customer operation screen 12) includes 10 objects corresponding to 10 numbers from 0 to 9 (operable). Buttons B100 to B109, and fields F101 to F104 for displaying the number of input digits are arranged.

As shown in FIG. 9, images that function as AR markers are displayed on the buttons B100 to B109, respectively. As shown in FIG. 9, the AR marker images attached to the buttons B100 to B109 appear to have symbols A to J in the form of a 7-segment display as viewed from the customer's eyes. It has a visible configuration.

Therefore, on the operation screen of ATM 10 (customer operation screen 12), which button corresponds to which number is not displayed, so the customer displays AR content on the smartphone terminal 20 (touch panel display 22), It is necessary to grasp the numbers corresponding to the buttons.

In the fields F101 to F104, every time the customer inputs a digit number (each time one button is pressed), a symbol “*” indicating that the PIN is entered in the order of the fields F101 to F104 is displayed. indicate. In this case, an image functioning as an AR marker is displayed in the field where “*” is displayed.

In the case of this embodiment, first, the customer refers to the AR content displayed on the touch panel display 22 of the smartphone terminal 20 in order to input “1” as the first character of the personal identification number. The customer recognizes that the button corresponding to “1” displayed on the touch panel display 22 of the smartphone terminal 20 is the “A” button on the operation screen of the ATM 10 (customer operation screen 12), and “A”. Press the button. As a result, the ATM 10 determines that the number “1” corresponding to the “A” button has been input, and displays an AR marker image in which tag data similar to the “A” button is encoded in F101. In this way, the customer inputs the reference number by pressing the button including the AR marker displayed on the customer operation screen 12 of the ATM 10 while referring to the AR content displayed on the touch panel display 22 of the smartphone terminal 20. To go.

That is, the customer cannot recognize numbers entered in each field on the operation screen of the ATM 10 (customer operation screen 12), but displays the AR content on the smartphone terminal 20 (touch panel display 22), and displays in each field. The corresponding numbers can be grasped.

As described above, when the operation screen is imaged on the ATM 10 (customer operation screen 12) illustrated in FIG. 9 with the camera 23 of the smartphone terminal 20, the AR content displayed on the touch panel display 22 is, for example, as illustrated in FIG. Become. At this time, the composite image displayed on the touch panel display 22 is as shown in FIG.

10 and 11, on the touch panel display 22, images G100 to G109 of AR content are displayed at positions overlapping the buttons B100 to B109 on the ATM 10 side. As shown in FIGS. 10 and 11, the touch panel display 22 displays fields F201 to F204 for displaying the input numbers (numbers) at positions overlapping the fields F101 to F104 on the ATM 10 side. . 9 shows a state in which only a two-digit password is entered, so only numbers of two digits are displayed in FIGS. 10 and 11 as well.

Regarding the buttons B100 to B109 displayed on the ATM 10 (customer operation screen 12), which button corresponds to which number may be random, but the image displayed on each button corresponds to the number corresponding to the button. Must be an AR marker. For example, in FIG. 9, the button B101 to which the image A is attached in the 7-segment display format is a button that can input the number “1”, so that the image attached to the button B101 corresponds to “1”. It is necessary to use tag data of AR content to be used as an AR marker. Thereby, the smartphone terminal 20 (AR application 211) can acquire AR content (image data of each button) corresponding to the captured AR marker and display it on the touch panel display 22.

As described above, in the WebATM system 1, it is possible to display the password as the AR content on the smartphone terminal 20 side without displaying the code number key or the input content on the ATM 10 side. In other words, the smartphone terminal 20 displays the virtual image of the personal identification number superimposed on the AR marker, thereby allowing the customer to press the AR marker (AR tag) as a button.

In addition, when the smartphone terminal 20 (AR application 211) first displays the AR content as illustrated in FIG. 11, the smartphone terminal 20 (AR application 211) freezes the displayed composite image for a predetermined time or until the screen is touched. You may make it be the state which carried out. This eliminates the need for the customer to continue to be ready to input the password while holding the smartphone terminal 20 over the ATM 10, thus increasing customer convenience.

(A-3) Effects of the First Embodiment According to the first embodiment, the following effects can be achieved.

(A-3-1) On the customer operation screen 12 of the ATM 10, when accepting the password input (numerical input), the operation buttons on the ATM 10 side are displayed with only the AR marker, and correspond to each operation button. The numbers are displayed as AR content on the smartphone terminal 20 side. In addition, on the customer operation screen 12 of the ATM 10, only the AR marker (image with “*” symbol added as a dummy) is displayed for the input number (number), and the input number is displayed on the smartphone terminal 20 side. Display as content. Thereby, the possibility that the password entered in the ATM 10 is peeped by others other than the customer is suppressed.

For convenience of explanation, alphabetic symbols are used as an example of the AR marker to be displayed. However, the shape of the AR marker is not limited to this, and for example, an irregular figure or pattern is associated with the AR marker. The relationship with the numbers can be made more difficult, and the possibility of being peeped by a third party other than the customer can be suppressed.

In particular, since the ATM display screen is provided at a low position so that various people can operate it, there is a possibility of peeping from behind, so the ATM 10 and the smartphone terminal 20 are linked to provide input support. That is convenient for customers.

(A-3-2) By the way, the real landscape specifying method using position information cannot accurately place virtual buttons on the customer operation screen. Moreover, it is very dangerous in terms of security to attach a GPS as an external sensor to an ATM terminal and connect it directly to the ATM terminal. Therefore, in the WebATM system 1 of the first embodiment, the above-described problem is avoided by using AR markers (image analysis) instead of position information for specifying the display position of the AR content.

(B) Second Embodiment Hereinafter, a second embodiment of the transaction system and the transaction apparatus according to the present invention will be described in detail with reference to the drawings. This embodiment demonstrates the example which applied the transaction system and transaction apparatus of this invention to the WebATM system and ATM.

(B-1) Configuration of Second Embodiment The overall configuration of the WebATM system 1A of the second embodiment can also be shown using FIG. Below, the difference with 1st Embodiment is demonstrated about WebATM system 1A of 2nd Embodiment.

In the WebATM system 1 of the first embodiment, the same AR content is distributed to all customers. In the second embodiment, the AR marker of the same tag data is displayed in the ATM 10. Also, the AR content selected from a plurality of patterns according to the customer (customer type) is displayed on the smartphone terminal 20.

Accordingly, in the WebAP server 60A of the second embodiment, a plurality of AR content data 65 is stored, and the AR content download server 40 selects and selects the AR content data 65 corresponding to the customer (customer type). The AR content (AR content corresponding to the acquired tag data) included in the AR content data 65 is read and distributed to the smartphone terminal 20.

In the second embodiment, as shown in FIG. 12, the WebAP server 60A stores two AR content data 65-1 and 65-2. Here, it is assumed that the AR content data 65-1 includes AR content for sighted persons (healthy persons), and the AR content data 65-2 includes AR content for hearing-impaired persons.

The configuration in which the AR content download server 40 determines the customer type is not limited. For example, the AR content download server 40 can set the smart phone terminal 20 (AR When accepting a login from the application 211), data that can identify the type of the customer (whether they are sighted or hearing impaired) may be acquired from the AR application 211. Thereby, the AR content download server 40 can select an acquisition destination (AR content data 65-1 or AR content data 65-2) when acquiring the AR content according to the type of customer.

Further, for example, information for managing the type of each customer is also stored in the customer information database server 30, and the AR content download server 40 is based on the ID (customer ID) used when accepting login from the smartphone terminal 20. The information (customer type information of the customer ID) of the customer information database server 30 may be acquired.

(B-2) Operation of the Second Embodiment Next, the operation of the WebATM system 1A of the second embodiment having the above configuration will be described.

Here, an example of adding (enlarged display) information to the operation screen being displayed using the AR content in the ATM 10 will be described.

FIGS. 13 to 15 are explanatory diagrams showing an example in which transaction support (operation support) is performed using AR contents when the customer confirms information related to the transaction at the ATM 10. In FIG. 13, a message indicating that a fee is charged for executing a transaction for the customer and a confirmation button B201 for accepting confirmation from the customer are arranged. Further, in the operation screen shown in FIG. 13, the user can transition to the next operation screen by pressing the confirmation button. Further, on the operation screen shown in FIG. 13, a button B202 (a button displaying “consultation counter not using voice”) for displaying guidance for the hearing impaired is also arranged. Furthermore, on the operation screen of ATM 10 shown in FIG. 13, an AR marker image G201 in which tag data of AR content for adding (enlarged display) information related to the operation screen is arranged.

FIGS. 14 and 15 show a composite image displayed on the touch panel display 22 when the operation screen of FIG. 13 (screen on which the AR marker image G201 is displayed) is captured by the camera 23 of the smartphone terminal 20. .

FIG. 14 shows a composite image displayed on the smartphone terminal 20 when the customer type is a sighted person (healthy person). FIG. 15 shows a composite image displayed on the smartphone terminal 20 when the customer type is a hearing impaired person.

As shown in FIG. 14, when the type of customer is a sighted person (healthy person), the smartphone terminal 20 displays an AR content image “The current time fee is as follows. An image G301 in which the message “...” is displayed in an enlarged manner and a message “Prompt to contact the operator with the handset on the left if you need help.” An enlarged image G302 is arranged.

As shown in FIG. 15, when the type of customer is a hearing impaired person, the smartphone terminal 20 prompts the user to press the button B202 as an AR content image. An image G401 with an enlarged message “Please contact us at the consultation desk” and an image G402 with an enlarged message “The current time fee is as follows. Has been.

As described above, in the WebATM system 1A of the second embodiment, the AR content for transaction support (operation support) is displayed (enlarged display) on the smartphone terminal 20 without using voice for the hearing impaired person. The sighted person (healthy person) is configured to display (enlarged display) AR content on the smartphone terminal 20 for transaction support (operation support) using sound.

(B-3) Effects of Second Embodiment According to the second embodiment, the following effects can be achieved.

(B-3-1) In the WebATM system 1A of the second embodiment, information for adding the operation screen displayed on the customer operation screen 12 of the ATM 10 to the smartphone terminal 20 is displayed on the smartphone terminal 20 as AR content. (Enlarged display). Thereby, in the second embodiment, additional information on the operation screen is displayed (enlarged display) without increasing the information displayed on the customer operation screen 12 of the ATM 10 (without complicating the display of the customer operation screen 12). ). Since ATM has a limited installation space, the size of the display screen is limited, and a plurality of information relating to transactions cannot be displayed together. For example, information for assisting transactions is displayed by pressing the help button to change the screen. Thus, as in the second embodiment, by coordinating the ATM 10 and the smartphone terminal 20, the above-described problems can be solved and customer transactions (operations) can be supported.

Especially, the display contents of the customer operation screen of the screen in ATM must be concise and clear. This is because the display content needs to shorten the operation time if the erroneous recognition of the operation content is reduced. However, on the other hand, due to a lack of information, there are many patterns that cause a shortage of information in response to a sudden specification change or a complicated display content confirmation operation. Inquiries to call centers due to lack of information can cause pressure on banking operations. Therefore, in the WebATM system 1A of the second embodiment, it is possible to expand the information provided to the customer while minimizing the influence on the ATM 10 (customer operation screen 12) by adding information by AR content.

(B-3-2) In the second embodiment, different AR contents are displayed on the smartphone terminal 20 in accordance with the customer (customer type). Specifically, in the second embodiment, AR content adapted to characteristics of a sighted person (a healthy person) and a hearing impaired person is displayed from the smartphone terminal 20. Thereby, in the WebATM system 1A, addition of information tailored to the customer (display of AR content) is realized without changing the content of the operation screen displayed on the customer operation screen 12 of the ATM 10. Further, as described above, in the WebATM system 1A, only the AR content is replaced without changing the content of the operation screen displayed on the ATM 10 (customer operation screen 12), so the configuration of the operation screen of the ATM 10 ( (ATM application) can be reduced.

(C) Third Embodiment Hereinafter, a third embodiment of the transaction system and the transaction apparatus according to the present invention will be described in detail with reference to the drawings. This embodiment demonstrates the example which applied the transaction system and transaction apparatus of this invention to the WebATM system and ATM.

(C-1) Configuration of the Third Embodiment FIG. 16 is a block diagram showing the overall configuration of the WebATM system 1B of the third embodiment.

In the WebATM system 1B of the third embodiment, a call center terminal 70 installed in a call center (not shown) is added as a system component. Further, the customer can receive a transaction (operation) support by speaking with the operator of the call center terminal 70 using the telephone 80 installed in the vicinity of the ATM 10.

In this embodiment, it is assumed that the contents of the AR marker (tag data) on the operation screen currently displayed on the ATM 10 can be changed according to the operator's operation on the call center terminal 70. That is, when the operator operates the call center terminal 70 and induces a transaction (operation) to a customer using voice / video / text, the object described (the button on the operation screen) is used by using the AR content. Or message).

For example, it is assumed that the call center terminal 70 is configured to be able to display the same screen as that currently distributed from the WebAP server 60B to the ATM 10. The WebAP server 60B can change the AR marker on the operation screen distributed to the ATM 10 in accordance with a control command (for example, a control command using a communication message) from the call center terminal 70. To do. In addition, about the structure which acquires the screen distributed to ATM10 in the call center terminal 70, or the structure which changes the AR marker on the operation screen distributed to ATM10 according to control from the call center terminal 70 by the WebAP server 60B, Various web application configurations can be applied.

(C-2) Operation of Third Embodiment Next, the operation of the WebATM system 1B of the third embodiment having the above configuration will be described.

Here, an example of adding information (enlarged display) to the operation screen being displayed using the AR content in the ATM 10 will be described with reference to FIGS. 17A and 17B and FIGS. 18A and 18B.

FIG. 17A shows an example of an operation screen displayed on the customer operation screen 12 of the ATM 10. In the operation screen of FIG. 17A, a message indicating that a fee is charged for dealing with a customer and a confirmation button B501 for accepting confirmation from the customer are arranged. In the operation screen shown in FIG. 17A, the customer can transition to the next operation screen by pressing the confirmation button B501. In addition, on the operation screen shown in FIG. 17A, a button B502 (a button displaying “consultation counter not using voice”) for displaying guidance for the hearing impaired is also arranged. In addition, a field F501 for displaying an AR marker image is arranged on the operation screen of the ATM 10 shown in FIG. 17A. In FIG. 17A, an image “1” in the form of a 7-segment display is arranged in the field F501 as an AR marker.

FIG. 17B shows a composite image displayed on the touch panel display 22 when the operation screen of FIG. 17A (screen on which the AR marker in the field F501 is displayed) is captured by the camera 23 of the smartphone terminal 20.

In FIG. 17B, as an AR content image corresponding to the AR marker image displayed in the field F501 on the smartphone terminal 20 (touch panel display 22), an arrow indicating a message in the operation screen on the ATM 10 side, An image G601 with a message “Please see” is displayed. 18A shows an example in which the AR marker displayed in the field F501 is replaced on the operation screen in FIG. 17A (an example in which the image is replaced with an image “2” in the 7-segment display format).

Then, when the operation screen in FIG. 18A (screen on which the AR marker in the field F501 is displayed) is captured by the camera 23 of the smartphone terminal 20, the composite image in FIG. 18B is displayed on the touch panel display 22.

In FIG. 18B, as an AR content image corresponding to the AR marker image displayed in the field F501 on the smartphone terminal 20 (touch panel display 22), an arrow indicating the confirmation button B501 in the operation screen on the ATM 10 side, An image G602 with a message “please see here” is displayed.

That is, the operator of the call center terminal 70 performs an operation (operation control for the WebAP server 60) for replacing the AR marker image displayed in the field F501, thereby replacing the AR marker image displayed in the field F501. It is possible. Accordingly, when the operator of the call center terminal 70 wants to explain the message displayed on the operation screens of FIGS. 17A and 18A, the operator of the call center terminal 70 transmits an instruction to display the AR marker “1” in the field F 501 to the WebAP server 60. Thus, it is necessary to operate the call center terminal 70. In addition, when the operator of the call center terminal 70 guides the customer to press the confirmation button B501 displayed on the operation screens of FIGS. 17A and 18A, the operator instructs the field F501 to display the AR marker “2”. It is necessary to operate the call center terminal 70 to transmit to the WebAP server 60. Upon receiving the instruction from the call center terminal 70, the WebAP server 60B temporarily creates a target screen after changing the AR marker, and transmits a screen data download request (reload request) to the ATM 10 to be changed. To do. And ATM10 which received the instruction | indication downloads and displays the target screen after AR marker change.

In the call center terminal 70, an interface for accepting an AR marker replacement instruction displayed on the operation screen on the ATM 10 side is not limited. For example, the call center terminal 70 may accept a predetermined operation on a keyboard (not shown) or by pressing a button on an operation screen (not shown).

(C-3) Effects of Third Embodiment According to the third embodiment, the following effects can be achieved.

In the WebATM system 1B, the AR marker displayed on the customer operation screen 12 of the ATM 10 can be replaced in accordance with the control of the call center terminal 70. Thereby, in the WebATM system 1B, it is possible to perform display indicating an object explained by the call center operator using the AR content, regardless of the operation environment of the customer.

At this time, in the WebATM system 1B, it is useful that the display contents of the ATM 10 (customer operation screen 12) can be changed in a timely manner, and the “AR marker” itself can be used variably. This is because it is possible to relax the following requirements due to the operating environment of the customer. As a first requirement, when the display content is changed on the smartphone terminal 20, downloading is necessary, so the customer needs to be in a place where the smartphone terminal 20 can communicate (a place where wireless communication is established). That is. As a second requirement, when AR content is downloaded and switched on the smartphone terminal 20 side, in order to receive the switching completion on the call center side (call center terminal 70 side), the smartphone terminal 20 side to the call center (call center terminal 70) Communication processing occurs. In the WebATM system 1B, information communication from a general terminal (such as the call center terminal 70) to a terminal having a high security risk (such as ATM 10) should be avoided.

(D) Other Embodiments The present invention is not limited to the above-described embodiments, and may include modified embodiments as exemplified below.

(D-1) In each of the above embodiments, the customer information database server 30 and the AR content download server 40 are arranged, but when the AR content is held (cached) in advance on the smartphone terminal 20 side. The customer information database server 30 may be arranged and the AR content download server 40 may be omitted.

(D-2) In the first and second embodiments described above, the ATM 10 has been described as a configuration that functions as a thin client. However, the content of the operation screen is provided inside, and the accounting system does not pass through the WebAP server 60. It may be configured to connect directly to the host 50.

The entire disclosure of Japanese Patent Application No. 2014-058453 is incorporated herein by reference. All documents, patent applications, and technical standards mentioned in this specification are to the same extent as if each individual document, patent application, and technical standard were specifically and individually described to be incorporated by reference, Incorporated herein by reference.

Claims

A trading system,
A transaction device for performing transactions with the operator;
A terminal device including an imaging unit and a display unit that displays an image acquired by the imaging unit;
Have
The transaction device is
In order to identify the content to be displayed on the terminal device as an object that accepts the operation of the operator and presents an operation screen that displays information related to the transaction and that can accept the operation on at least the operation screen An operation screen presenting unit capable of attaching an encoded image obtained by encoding the tag data of
The terminal device is
A content holding unit for holding content data;
When the encoded image is detected by analyzing the image captured by the imaging unit, content data corresponding to the encoded image is acquired from the data held by the content holding unit, and the imaging unit captures the image. An image processing unit that causes the display unit to display a composite image obtained by superimposing the acquired content image on the image;
Comprising
Trading system.
The operation screen presenting unit arranges a plurality of operation keys each assigned a different number, presents a number input screen that accepts one or more digits input,
The transaction system according to claim 1, wherein an encoded image obtained by encoding tag data of content indicating a number of the operation key is attached to each operation key constituting the number input screen.
The transaction system according to claim 2, wherein an encoded image obtained by encoding tag data of content indicating a number already input by the operation key is displayed on the number input screen.
The transaction system according to claim 1, wherein an encoded image obtained by encoding tag data of a content for enlarging and displaying information related to the operation screen being presented is displayed on the operation screen presented by the operation screen presentation unit.
A content providing device that provides content data to the terminal device, and provides the terminal device with content selected according to the type of the terminal device from a plurality of contents corresponding to the same tag data. The transaction system according to claim 1.
An operation screen data providing unit that provides operation screen data to the transaction device, and an encoded image change receiving unit that replaces the encoded image displayed on the operation screen presented by the operation screen presenting unit according to the control of the external device The transaction system according to claim 1, further comprising: an operation screen providing device comprising:
In addition to accepting the operation of the operator and presenting an operation screen for displaying information related to the transaction, the content displayed on the terminal device possessed by the operator is displayed on the object that can accept the operation on the operation screen. A transaction apparatus having an operation screen presenting unit capable of attaching an encoded image in which tag data for identification is encoded.