JP7021321B1 - Terminal devices, payment methods and payment programs - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a terminal device, an analysis server, a payment method, a payment system and a payment program capable of improving the reading accuracy of various codes. SOLUTION: In a payment system 1, a terminal device 100 has a transmission unit for transmitting an image including information about payment contents (for example, an image taken by a terminal device 100 used by a user) to an analysis server 10, and an image. It is provided with a control unit having a receiving unit for receiving information on payment contents obtained by analyzing the above, and a payment processing unit for executing payment processing based on the received information on payment contents. [Selection diagram] Fig. 2

Description

The present invention relates to a terminal device , a payment method and a payment program.

Conventionally, there has been provided a technique for reading information indicated by a barcode printed on a medium such as paper and realizing processing according to the read information. As an example of such a technology, there is provided a technology for acquiring the price of a transaction target by reading a barcode printed on a package or price tag of the transaction target. Further, in order to improve the reading accuracy of such a barcode, when reading the barcode, it is determined whether the barcode conforms to a predetermined standard, and the information indicated by the barcode is read according to the determination result. The technology to realize the above is proposed.

Japanese Unexamined Patent Publication No. 10-187866

However, with the above-mentioned technique, there is room for improving the reading accuracy of various codes such as barcodes and QR codes (registered trademarks).

For example, a general barcode indicates information used for processing and a check digit for determining whether or not the information is correct. When the barcode is read, the reading device that reads the barcode determines whether or not the information used for processing is correctly read by using the check digit. However, even if the information indicated by the barcode or the check digit cannot be read correctly, if the information that is the reading result and the check digit happen to correspond, it is erroneously determined that the information used for processing has been read correctly. There is a risk of doing it.

The present application has been made in view of the above, and an object thereof is to improve the reading accuracy of various codes.

The terminal device according to the present application has a transmission unit that transmits an image including information on payment details to an analysis server, a reception unit that receives information on payment details obtained by analyzing the images, and the payment details received. It is characterized by having a payment processing unit that executes payment processing based on information about the above.

According to one aspect of the embodiment, there is an effect that the reading accuracy of various codes can be improved.

FIG. 1 is a diagram showing an example of transmission / reception of information between an analysis server and a terminal device according to an embodiment. FIG. 2 is a diagram showing a configuration example of the terminal device according to the embodiment. FIG. 3 is a diagram showing a configuration example of the second application according to the embodiment. FIG. 4 is a diagram showing a configuration example of the analysis server according to the embodiment. FIG. 5 is a flowchart showing an example of the payment processing procedure according to the embodiment. FIG. 6 is a hardware configuration diagram showing an example of a computer that realizes the functions of the terminal device.

Hereinafter, a terminal device, an analysis server, a payment method, a payment system, and a mode for implementing a payment program (hereinafter referred to as “the embodiment”) according to the present application will be described in detail with reference to the drawings. Note that this embodiment does not limit the terminal device, analysis server, payment method, payment system, and payment program according to the present application. Further, in each of the following embodiments, the same parts are designated by the same reference numerals, and duplicate explanations are omitted.

[1. Embodiment]
A payment process realized by the terminal device or the like of the present embodiment will be described with reference to FIG. FIG. 1 is a diagram showing an example of payment processing according to an embodiment. In FIG. 1, it is assumed that the payment processing and the like according to the embodiment are realized by the analysis server 10, the payment server 12, and the terminal device 100, which are examples of the payment device according to the present embodiment.

As shown in FIG. 1, the payment system 1 according to the embodiment includes an analysis server 10, a payment server 12, and a terminal device 100. The analysis server 10, the payment server 12, and the terminal device 100 are connected to each other via a network N (see, for example, FIG. 2) so as to be able to communicate with each other by wire or wirelessly. The network N is, for example, a WAN (Wide Area Network) such as the Internet. The payment system 1 shown in FIG. 1 may include a plurality of analysis servers 10, a payment server 12, and a plurality of terminal devices 100. Further, the analysis server 10 and the payment server 12 may be realized by physically or logically the same server device or cloud system.

The analysis server 10 shown in FIG. 1 is an information processing device that receives an image including information on payment details captured by the terminal device 100 and analyzes the received image, and is realized by, for example, a server device or a cloud system. .. For example, the analysis server 10 analyzes and reads information on payment details such as a barcode included in the received image. The information regarding the read payment details is transmitted to the terminal device 100 that has captured the image. That is, the analysis server 10 performs a calculation in which an image including information on payment details is input and information on payment details is output. Here, the information regarding the payment content is, for example, a QR code (registered trademark) image or a data string indicated by the QR code (registered trademark) in the case of a barcode image.

The payment server 12 shown in FIG. 1 is an information processing device that provides an electronic payment service related to electronic payment using the terminal device 100 and performs various payments, and is realized by, for example, a server device or a cloud system. For example, the payment server 12 manages the accounts of the provider of the transaction target and the user to which the transaction target is provided, and remits electronic money between the accounts in accordance with the payment request from the user. Realize various payments. The electronic money may be, for example, points or currencies independently used by various companies, or may be money provided by the state such as Japanese yen or dollars that can be electronically traded. good.

The terminal device 100 shown in FIG. 1 is an information processing device used by a user. The terminal device 100 is realized by, for example, a smartphone, a tablet terminal, a notebook PC (Personal Computer), a desktop PC, a mobile phone, a PDA (Personal Digital Assistant), or the like. Further, the terminal device 100 displays the information distributed by the analysis server 10 and / or the payment server 12 by a web browser or an application. In the example shown in FIG. 1, a case where the terminal device 100 is a smartphone provided with a photographing device (camera) is shown.

Here, an example of payment (electronic payment) using the terminal device 100 will be described prior to the payment processing executed by the analysis server 10, the payment server 12, and the terminal device 100. In the following description, an example in which the user U makes a payment using the terminal device 100 by using the store identification information that identifies the store, which is a two-dimensional code arranged in the store, will be described. The form is not limited to this. The example of payment described below can be applied even when an arbitrary user U makes a payment at an arbitrary store using an arbitrary terminal device 100. Further, the store identification information may be not only a QR code (registered trademark) but also a barcode, a predetermined mark, a number, or the like.

For example, when the user U makes a payment for using or purchasing a payment target such as various products or services at a store, the user U activates a payment application pre-installed on the terminal device 100. Then, the user U captures the store identification information installed in the store via the application. In such a case, the terminal device 100 displays a screen for inputting the price to be settled, and accepts the input of the payment amount from the user U or the store clerk. Then, the terminal device 100 converts the user information that identifies the user U, the store identification information (or the information indicated by the store identification information, that is, the information indicating the store), and the payment information indicating the payment amount into the payment server. Send to 12.

In such a case, the payment server 12 transfers the electronic money of the amount indicated by the payment amount from the account of the user U indicated by the user information to the account of the store indicated by the store identification information. Then, the payment server 12 transmits a notification to the terminal device 100 that the payment has been completed. In such a case, the terminal device 100 notifies that the payment has been made by electronic money by outputting a screen indicating that the payment has been completed and a predetermined voice.

The payment using the terminal device 100 is not limited to the above-mentioned processing. For example, the payment using the terminal device 100 may be performed using a store terminal installed in the store. For example, the terminal device 100 displays the user identification information for identifying the user U on the screen. In such a case, the store terminal installed in the store reads the user identification information displayed on the terminal device 100, and setstles the user identification information, the payment amount, and the payment information indicating the store identification information. It sends to the server 12. In such a case, the payment server 12 transfers the electronic money of the amount indicated by the payment amount from the account of the user U indicated by the user identification information to the account of the store, and transfers the electronic money of the amount indicated by the payment amount to the store terminal or the terminal device 100 of the store. On the other hand, it may be notified that the payment has been made by outputting a screen indicating that the payment has been completed or a predetermined voice.

Further, the payment using the terminal device 100 is not only a process of transferring the electronic money from the account charged with the electronic money in advance by the user U to the account of the store, but also, for example, a credit card registered in advance by the user U. It may be the payment used. In such a case, for example, the payment server 12 may remit the electronic money of the payment amount to the account of the store and charge the payment amount to the credit card management company of the user U.

Further, the payment using the terminal device 100 is not only a process of transferring electronic money from the user U's account to the store's account, but also, for example, from the user U's account to the predetermined transaction target provider's account. It may be a payment that transfers electronic money. In such a case, the user U uses a camera provided with the terminal device 100 to provide a barcode included in the payment form (for example, the transfer form used in the convenience store storage agency service) sent by the provider to the user U. Read with. Then, when the terminal device 100 displays the payee and the payment amount indicated by the read barcode on the screen and receives an operation from the user U to approve the remittance to the payee, the user identification information and the user identification information are displayed. The payment information indicating the payment amount and the information for identifying the payee is transmitted to the payment server 12. In such a case, the payment server 12 transfers the electronic money of the amount indicated by the payment amount from the account of the user U to the account of the payment destination.

Here, conventionally, when reading a barcode, there is known a technique for determining whether or not the barcode is correctly read by using the check digit indicated by the barcode. However, in the method using the check digit, if the information that is the reading result and the check digit coincide with each other, it may be erroneously determined that the barcode has been read correctly.

Further, when reading the bar code, the lighting condition at the time of shooting, the blurring of the camera of the terminal device 100 at the time of shooting, the blurring of the subject at the time of shooting, the focus state at the time of imaging, and the image sensor of the camera of the terminal device 100. In some cases, the captured image may not be clear. In such a case, it may be difficult or impossible to correctly read the barcode from the captured image with the computational resources of the terminal device 100.

Therefore, the analysis server 10 executes the analysis process according to the embodiment. Hereinafter, the payment process executed by the analysis server 10 will be described with reference to FIG. In the following description, an example in which the terminal device 100 is used by the user U will be shown. Further, in the following description, the terminal device 100 may be equated with the user U. That is, in the following, the user U can be read as the terminal device 100.

First, the terminal device 100 photographs the payment form created for the user U by the provider who provides the predetermined transaction target (step S1). In the example of FIG. 1, it is assumed that the predetermined water supply company prepares the payment form R1 for the user U and sends it to the user U. As shown in FIG. 1, the payment form R1 is a character string T1 indicating a payment content including a payment amount (billing amount), a payee (a water supply company that issued the payment form R1), a payment due date, and the payment content. A bar code BC indicating the payment details, and a number indicating the payment details (for example, the payment amount, the identifier of the water supply company that issued the payment form R1, the payment due date, etc.), and the number BCN (visually) indicated by the bar code BC. Possible characters) are included. In the example of FIG. 1, the user U activates an application for electronic payment (application provided by the analysis server 10 and / or the payment server 12) pre-installed in the terminal device 100, and pays via the application. Take an image of form R1.

Subsequently, the terminal device 100 determines whether or not the captured image contains the barcode BC (step S2). Here, if it is determined that the captured image does not include the barcode BC, the user U may be urged to repeat the procedure from step S1 on the assumption that the barcode BC could not be captured normally. Then, the transition to step S1 may be performed automatically. If it is determined that the captured image contains the barcode BC, the terminal device 100 proceeds to the next step S3.

Subsequently, the terminal device 100 transmits an image including the barcode BC to the analysis server 10 (step S3). Here, what is transmitted to the analysis server 10 is not limited to the image including the barcode BC. That is, the context information may be transmitted by attaching to the image including the barcode BC or before and after transmitting the image including the barcode BC. The context information is, for example, information about the hardware of the terminal device 100 such as a camera, information about software such as middleware, and sensor information acquired by various sensors of the terminal device at the time of shooting (for example, GPS data of the shooting location, shooting). Information on the posture of the terminal device 100 at the time and changes in the posture with respect to the time axis (posture, speed, acceleration, angular speed, angular acceleration, etc. of the terminal device 100), ambient light state, network connection status, etc.), terminal device 100 Model, shooting time zone, flash status at the time of shooting, focal length of the lens at the time of shooting, focus status at the time of shooting, etc. On the contrary, information other than the information related to the payment content may be deleted from the image including the barcode and transmitted. That is, when the payment form R1 contains the personal information of the user U, if the personal information is not the information related to the payment contents, the portion including the personal information may not be transmitted to the analysis server 10. At this time, the portion including the personal information may be deleted from the image by image trimming or the like, or the portion including the personal information in the image may be masked with black color or a mosaic or the like.

Subsequently, the analysis server 10 receives the image transmitted from the terminal device 100 in step S3, and obtains information on the payment content by analyzing this image (step S4). For example, an image including a barcode is analyzed and the number information indicated by the barcode is read. At this time, the image may be analyzed based on the model of the terminal device 100 that captured the image. For example, in image analysis using deep learning, a trained model may be prepared for each model of the terminal device 100, and the image may be analyzed with a trained model that matches the model of the terminal device 100 that captured the received image. good.

Subsequently, the analysis server 10 transmits the information regarding the payment content obtained by analyzing the image to the terminal device 100 (step S5). For example, the analysis server 10 may transmit the number information indicated by the bar code as the information regarding the payment content to the terminal device 100. Here, if the analysis fails in step S4, the information indicating that the analysis has failed may be transmitted to the terminal device 100. Also, at that time, information about the cause of the failure (for example, the image is too dark, the image is too bright, the image is out of focus, the image is out of focus, the image does not contain a part of the barcode, etc. ) May also be transmitted.

Subsequently, the terminal device 100 checks the information regarding the received payment content (step S6). If there is no error in the received information, the information regarding the received payment details may be displayed on the display unit of the terminal device 100. Here, when the analysis fails in step S4 and the analysis server 10 transmits the information that the analysis has failed to the terminal device 100 in step S5, the information that the image analysis has failed is received in step S6. It becomes. In this case, the display unit of the terminal device 100 may display that the image analysis has failed. Further, when the information regarding the cause of the failure is received, the cause may be displayed on the display unit of the terminal device 100.

Subsequently, the terminal device 100 transmits the information regarding the payment content received in step S6 to the payment server 12, and the payment server 12 from the user U's account to the payment destination's account according to the received payment content information. Remittance of electronic money to (step S7). For example, if the payment details indicated by the specified character strings and the payment details indicated by the specified barcode number match, the payment server 12 can use the payment details from the user U's account based on these payment details. Remittance of electronic money to the account of the water company (“A city water supply”) that is the payment destination.

As described above, the terminal device 100 according to the embodiment determines whether or not the image of the payment form R1 contains information on the payment content such as a barcode, and if it is determined that the information is included. Sends the captured image to the analysis server 10. The analysis server 10 analyzes the image received from the terminal device 100, and when the information regarding the payment content is read, the analysis server 10 transmits the read information regarding the payment content to the terminal device 100. When the terminal device 100 receives the information regarding the payment content from the analysis server 10, the terminal device 100 transmits this information to the payment server 12 and executes the payment (remittance) process. As a result, when an image is analyzed by the terminal device 100, it is difficult to perform highly accurate analysis with the computational resources of the terminal device 100, and as a result, analysis failure (misreading) may occur. In general, the analysis server 10 is easier to secure computational resources than the terminal device 100 and can perform image analysis with high accuracy. Therefore, by analyzing the image with the analysis server 10, the image analysis fails (misreading). The chances can be reduced.

[2. Configuration of terminal equipment]
Next, the terminal device 100 for realizing the above-mentioned payment processing will be described with reference to FIG. FIG. 2 is a diagram showing a configuration example of the terminal device according to the embodiment. As shown in FIG. 2, the terminal device 100 includes a communication unit 110, a storage unit 120, a camera 130, a touch panel 140, and a control unit 150.

(About communication unit 110)
The communication unit 110 is realized by, for example, a NIC (Network Interface Card) or the like. Then, the communication unit 110 is connected to the network N by wire or wirelessly, and transmits / receives information to / from a payment server 12 or the like that distributes an application for electronic payment.

(About the storage unit 120)
The storage unit 120 is realized by, for example, a semiconductor memory element such as a RAM (Random Access Memory) or a flash memory (Flash Memory), or a storage device such as a hard disk or an optical disk. Further, the storage unit 120 stores an image taken by the camera 130.

(About camera 130)
The camera 130 is an image pickup device for capturing an image (moving image or still image). The camera 130 is composed of an image pickup device such as a CCD (Charged-coupled devices) sensor or a CMOS (Complementary metal-oxide-semiconductor) sensor.

(About touch panel 140)
The touch panel 140 is an input device realized by a liquid crystal display or the like, displaying various information, and accepting various operations from the user with a user's finger or a dedicated pen.

(About control unit 150)
The control unit 150 is a controller, and for example, various programs stored in a storage device inside the terminal device 100 by a CPU (Central Processing Unit), an MPU (Micro Processing Unit), or the like have a RAM as a work area. It is realized by being executed. Further, the control unit 150 is a controller, and is realized by, for example, an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

Here, the control unit 150 realizes various functions related to the terminal device 100 by executing a plurality of applications. For example, in the example shown in FIG. 2, the control unit 150 is executing the first application 151 and the second application 152. In addition to the application shown in FIG. 2, the control unit 150 may execute an arbitrary number of applications for exerting an arbitrary function.

The first application 151 is an application that serves as an OS (Operating System) of the terminal device 100. The second application 152 causes the terminal device 100 to execute the above-mentioned payment process. Hereinafter, an example of the functional configuration of the second application 152 will be described with reference to FIG. FIG. 3 is a diagram showing a configuration example of the second application according to the embodiment. As shown in FIG. 3, the second application 152 according to the embodiment has a photographing unit 1521, a transmitting unit 1522, a receiving unit 1523, a payment processing unit 1524, and a notification unit 1525, and will be described below. Realize or execute the functions and actions of information processing.

(About the shooting unit 1521)
The photographing unit 1521 photographs an image. For example, in the example of FIG. 1, the photographing unit 1521 controls the camera 130 to take an image of the payment form R1 including the barcode, and stores the taken image in the storage unit 120.

The photographing unit 1521 may control the camera 130 in the electronic payment using the terminal device 100 and acquire an image of the store identification information that identifies the store.

(About transmitter 1522)
The transmission unit 1522 transmits the image of the payment form R1 photographed by the photographing unit 1521 to the analysis server 10. For example, the transmission unit 1522 transmits the image of the payment form R1 photographed by the photographing unit 1521 and stored in the storage unit 120 from the communication unit 110 of the terminal device 100 to the analysis server 10 via the network N.

Further, the transmission unit 1522 transmits the image of the payment form R1 photographed by the photographing unit 1521 and the context information indicating the context when the image is photographed to the analysis server 10. The context information includes information on the hardware of the terminal device 100, information on software, information on sensors provided in or connected to the terminal device 100, and the like, but specific examples are as described above and are not repeated here.

Further, the transmission unit 1522 determines whether or not the captured image contains information regarding the payment content, and if it is determined that the captured image contains the information, the transmission unit 1522 transmits the image to the analysis server 10. For example, the terminal device 100 checks the image of the payment form R1 taken by the photographing unit 1521 and stored in the storage unit 120, determines whether or not the information regarding the payment content such as a barcode is included, and includes the terminal device 100. If it is determined that the payment form R1 is stored in the storage unit 120, the image of the payment form R1 is transmitted from the communication unit 110 of the terminal device 100 to the analysis server 10 via the network N.

Further, the transmission unit 1522 determines whether or not the captured image contains personal information, and if it is determined that the captured image contains personal information, the transmission unit 1522 transmits an image obtained by removing the personal information from the captured image to the analysis server 10. do. For example, the terminal device 100 checks the image of the payment form R1 taken by the photographing unit 1521 and stored in the storage unit 120, determines whether or not the image contains personal information, and determines that the image contains personal information. In that case, the personal information is deleted by trimming or masking (blackening, mosaic, etc.) the part related to the personal information in the image of the payment form R1 stored in the storage unit 120, and the deleted image is used as the terminal. It is transmitted from the communication unit 110 of the device 100 to the analysis server 10 via the network N. Here, the personal information included in the image may include information on personal privacy such as GPS information included in the Exif data of the image file, and after deleting these information, the image file is transmitted to the analysis server 10. You may.

Further, the transmission unit 1522 transmits the context information of the image to the analysis server 10 together with the captured image or before and after the captured image. For example, the terminal device 100 captures context information (for example, the degree of flash) when the image is captured together with the image of the payment form R1 captured by the photographing unit 1521 and stored in the storage unit 120, or before and after the transmission of the image. , Image resolution, shooting date and time, environment brightness by illuminance sensor, camera shake information by acceleration sensor and gyro sensor, camera lens focal length, information on camera lens focus, etc.) from the communication unit 110 of the terminal device 100. It is transmitted to the analysis server 10 via the network N.

(About receiver 1523)
The receiving unit 1523 receives the information regarding the payment content read from the image by the analysis server 10 by the image analysis. For example, the terminal device 100 receives the information regarding the payment contents read by the analysis server 10 by analyzing the image of the payment form R1 by making the communication unit 110 function as the reception unit 1523 via the network N.

Further, the receiving unit 1523 receives the information related to the analysis generated when the image is analyzed by the analysis server 10 together with the information related to the payment contents or instead of the information related to the payment contents. For example, when analyzing an image with the analysis server 10, it is assumed that the analysis fails because the image is unclear. In such a case, the analysis server 10 transmits to the terminal device 100 via the network N that the analysis has failed because it was unclear as information regarding the analysis.

(About notification unit 1525)
When the notification unit 1525 receives the information related to the analysis generated when the analysis server 10 analyzes the image, the notification unit 1525 notifies the user U of the information related to this analysis. For example, when the analysis server 10 analyzes an image, if the analysis of the image fails due to the image being unclear, the terminal device 100 indicates that the analysis failed because the image was unclear. Receives from the analysis server 10 via the network N. The terminal device 100 visually notifies the user U of the information related to the analysis received in this way by displaying it on the display of the terminal device 100 or the like. The terminal device 100 may notify the user U of information regarding analysis by emitting a sound or vibration.

(About the payment processing unit 1524)
When it is determined that the payment contents match, the payment processing unit 1524 executes the payment processing according to the payment contents. For example, in the example of FIG. 1, if the payment content indicated by each character string included in the captured image matches the payment content indicated by the specified barcode number, the payment processing unit 1524 sets the payment content. Execute the payment processing according to it.

Further, the payment processing unit 1524 may notify the payment content to the external server device that realizes the remittance between the accounts as the payment processing. For example, in the example of FIG. 1, the payment processing unit 1524 makes a remittance request including a photographed image and a specified barcode number, and a remittance request corresponding to the payment content indicated by the number is a payment server. Send to 12.

The payment processing unit 1524 may execute payment processing related to electronic payment using the terminal device 100. For example, the payment processing unit 1524 has user information for identifying the user U, information indicating the store identification information photographed by the photographing unit 1521, and a payment amount input to the terminal device 100 by the user U or a store clerk. The payment information indicating that is transmitted to the payment server 12.

[3. Analysis server configuration]
Next, the configuration of the analysis server 10 will be described with reference to FIG. FIG. 4 is a diagram showing a configuration example of the analysis server 10 according to the embodiment. As shown in FIG. 4, the payment server 12 has a communication unit 20, a storage unit 30, and a control unit 40.

(About communication unit 20)
The communication unit 20 is realized by, for example, a NIC or the like. Then, the communication unit 20 is connected to the network N by wire or wirelessly, and transmits / receives information to / from the terminal device 100 and the like.

(About the storage unit 30)
The storage unit 30 is realized by, for example, a semiconductor memory element such as a RAM or a flash memory, or a storage device such as a hard disk or an optical disk.

(About control unit 40)
The control unit 40 is a controller, and is realized by, for example, a CPU, an MPU, or the like executing various programs stored in a storage device inside the analysis server 10 using the RAM as a work area. Further, the control unit 40 is a controller, and is realized by, for example, an integrated circuit such as an ASIC or FPGA. As shown in FIG. 4, the control unit 40 according to the embodiment includes a reception unit 41, an analysis unit 42, a transmission unit 43, and a learning unit 44, and has functions and operations of information processing described below. Realize or execute.

(About receiver 41)
The receiving unit 41 receives an image taken by the terminal device 100 used by the user U. For example, in the example of FIG. 1, the receiving unit 41 receives the image of the payment form R1 from the terminal device 100.

Further, the receiving unit 41 may receive context information indicating the context when the image is taken, in addition to the image taken by the terminal device 100 used by the user U. The context information is as described above and will not be repeated here.

(About analysis unit 42)
The analysis unit 42 analyzes the image received from the terminal device 100 of the user U. For example, the barcode BC included in the image of the payment form R1 is analyzed, and the data string indicated by the barcode is read. The image may be stored in the storage unit 30 for a predetermined period. Further, the image stored in the storage unit 30 may be used later when analyzing another image.

Further, when the analysis unit 42 receives the context information in addition to the image, the analysis unit 42 may obtain information on the payment content from the image based on the context information when analyzing the received image. For example, on the premise that it is known that an image that is out of focus tends to be acquired for a predetermined model of the terminal device 100, when a certain image and context information are received, the context information is examined. However, if it is found that the model is a predetermined model, the image may be analyzed after applying a filter for correcting the out-of-focus.

Further, the analysis unit 42 makes the model learn the characteristics of the context indicated by the context information when the image capable of analyzing the information regarding the payment content is taken, and the image is based on the model and the context information learned by the learning unit 44. May be analyzed.

(About transmitter 43)
The transmission unit 43 transmits the data string, which is the information regarding the payment content analyzed by the analysis unit 42, to the terminal device 100. For example, in the example of FIG. 1, the transmission unit 43 transmits a data string obtained by analyzing a barcode included in an image transmitted from the terminal device 100 to the terminal device 100.

(About the learning unit 44)
The learning unit 44 causes the model to learn the characteristics of the context indicated by the context information when the image capable of analyzing the information regarding the payment content is taken.

Further, the learning unit 44 may be trained by a learning model that inputs an image and outputs information about payment contents. In that case, context information may be added to the input in addition to the image. Further, the learning model may be created for each model of the terminal device 100.

[4. Payment processing flow]
A procedure for payment processing of the terminal device 100 according to the embodiment will be described with reference to FIG. FIG. 5 is a flowchart showing an example of the payment processing procedure according to the embodiment.

As shown in FIG. 6, the terminal device 100 determines whether or not an image including a barcode has been captured (step S101). If no image has been taken (step S101; No), the terminal device 100 waits until the image is taken. On the other hand, when the image is taken (step S101; Yes), the terminal device 100 transmits the image including the barcode to the analysis server 10 (step S102).

Subsequently, the terminal device 100 determines whether or not information has been received from the analysis server 10 (step S103). When the information is not received (step S103; No), the terminal device 100 waits until the information is received. On the other hand, when the information is received (step S103; Yes), the terminal device 100 determines whether or not the information received from the analysis server 10 includes information regarding the payment content (step S104).

If the information received from the analysis server 10 does not include information about the payment details (step S104; No), the terminal device 100 displays an error (step S106) and again takes an image including a barcode. To prompt the user to execute step S101. If the information received from the analysis server 10 includes payment details (step S104; Yes), the terminal device 100 executes the payment process (step S105) and ends the process.

[5. Modification example]
The above embodiment shows an example, and various modifications and applications are possible.

[5-1. About the code]
In the above-described embodiment, an example in which the code is a barcode has been shown, but the code is not limited to such an example, and any information can be used as long as the information can be read by the terminal device used by the user. There may be. For example, the code may be a two-dimensional code (for example, a QR code (registered trademark)).

[5-2. About processing mode]
Of the processes described in the above embodiments, all or part of the processes described as being automatically performed can be performed manually, and conversely, all the processes described as being performed manually. Alternatively, a part thereof can be automatically performed by a known method. In addition, information including processing procedures, specific names, various data and parameters shown in the above documents and drawings can be arbitrarily changed unless otherwise specified. For example, the various information shown in each figure is not limited to the information shown in the figure.

Further, each component of each of the illustrated devices is a functional concept, and does not necessarily have to be physically configured as shown in the figure. That is, the specific form of distribution / integration of each device is not limited to the one shown in the figure, and all or part of them may be functionally or physically distributed / physically in any unit according to various loads and usage conditions. Can be integrated and configured.

In addition, the above-described embodiments can be appropriately combined as long as the processing contents do not contradict each other.

[6. effect〕
[First effect]
As described above, the terminal device 100 according to the embodiment includes a transmission unit that transmits an image including information on payment details to an analysis server, and a reception unit that receives information on payment details obtained by analyzing the images. It has a payment processing unit that executes payment processing based on the information regarding the received payment contents.

As a result, the analysis server can be made to perform the image analysis processing performed by the terminal device 100.

[Second effect]
In the terminal device 100 described in the item of the first effect, the transmitting unit further transmits the image and the context information indicating the context when the image is taken to the analysis server, and the receiving unit receives the image from the image based on the context information. You may receive information about the payment details obtained.

As a result, the image can be analyzed based on the context information, so that more accurate image analysis becomes possible.

[Third effect]
In the terminal device 100 described in the item of the second effect, the transmission unit may further transmit information indicating the type of the terminal device to the analysis server as context information.

As a result, the image can be analyzed based on the information of the model of the terminal device, and the accuracy of the analysis can be improved.

[Fourth effect]
Further, in the terminal device 100 described in the item of the second or third effect, even if the transmission unit transmits the sensor information acquired by the sensor of the terminal device when the image is taken as the context information to the analysis server. good.

As a result, the image can be analyzed based on the information of the sensor, and the accuracy of the analysis can be improved.

[Fifth effect]
In the terminal device 100 described in the second to fourth effect items, the transmission unit may further transmit information indicating software executed by the terminal device when the image is taken to the analysis server as context information.

As a result, the image can be analyzed based on the software information of the terminal device, and the accuracy of the analysis can be improved.

[Sixth effect]
In the terminal device 100 described in the first to fifth effect items, the transmission unit further determines whether or not the image contains information regarding the payment content, and if it is determined that the image is included, the image is displayed. It may be sent to the analysis server.

As a result, since the image is transmitted to the analysis server only when the information regarding the payment content is included, the communication cost and the execution cost of the server can be reduced.

[Seventh effect]
In the terminal device 100 described in the items of the first to sixth effects, the transmission unit further determines whether or not the image contains personal information, and if it is determined that the image contains the personal information, the personal information is obtained from the image. The image excluding the above may be sent to the analysis server.

As a result, it is not necessary to send personal information to an external server, so that the risk of leakage of personal information can be reduced.

[Eighth effect]
In the terminal device 100 described in the first to seventh effect items, the transmitting unit further transmits a plurality of images including information on payment details to the analysis server, and the receiving unit analyzes the plurality of images. You may receive information about the payment details made.

As a result, information on the payment details can be obtained from the analysis of a plurality of images, so that more accurate analysis becomes possible.

[Ninth effect]
In the terminal device 100 described in the items of the first to eighth effects, the receiving unit further receives the information related to the analysis, and when the terminal device receives the information related to the analysis, the terminal device relates to the user of the terminal device regarding the analysis. It may further have a notification unit for notifying based on information.

This makes it possible to notify the user that the analysis has failed. Furthermore, by adding the reason for the failure, the cause of the failure can be known in more detail, and it is possible to make people aware of the shooting of an image that is difficult to fail.

[Tenth effect]
The analysis server 10 according to the embodiment has a receiving unit that receives an image containing information on payment details from a terminal device, an analysis unit that analyzes an image received by the receiving unit to obtain information on payment details, and an analysis unit. It has a transmission unit that transmits information about the payment details made to the terminal device.

As a result, the terminal device 100 can offload the image analysis work to the analysis server 10.

[Eleventh effect]
In the analysis server-10 described in the item of the tenth effect, the receiving unit further receives the image and the context information indicating the context when the image is taken, and the analysis unit pays from the image based on the context information. You may get information about the content.

As a result, the analysis server 10 can analyze the image based on the context information, so that the analysis can be performed with high accuracy.

[Twelfth effect]
The analysis server 10 described in the item of eleventh effect further has a learning unit for learning the feature of the context indicated by the context information when the image capable of analyzing the information regarding the payment contents is taken, and analyzes the information. The department may obtain information about the payment content from the image based on the context information and the model.

As a result, by continuing the learning, highly accurate image analysis can be expected as the learning progresses.

[7. Hardware configuration]
Further, the terminal device 100, the analysis server 10, and the payment server 12 according to each of the above-described embodiments are realized by, for example, a computer 1000 having a configuration as shown in FIG. Hereinafter, the terminal device 100 will be described as an example. FIG. 6 is a hardware configuration diagram showing an example of a computer that realizes the functions of the terminal device. The computer 1000 has a CPU 1100, a ROM 1200, a RAM 1300, an HDD 1400, a communication interface (I / F) 1500, an input / output interface (I / F) 1600, and a media interface (I / F) 1700.

The CPU 1100 operates based on a program stored in the ROM 1200 or the HDD 1400, and controls each part. The ROM 1200 stores a boot program executed by the CPU 1100 when the computer 1000 is started, a program depending on the hardware of the computer 1000, and the like.

The HDD 1400 stores a program executed by the CPU 1100, data used by such a program, and the like. The communication interface 1500 receives data from another device via the communication network 500 (corresponding to the network N of the embodiment) and sends the data to the CPU 1100, and the communication interface 1500 receives data generated by the CPU 1100 via the communication network 500 to another device. Send to the device.

The CPU 1100 controls an output device such as a display or a printer, and an input device such as a keyboard or a mouse via the input / output interface 1600. The CPU 1100 acquires data from the input device via the input / output interface 1600. Further, the CPU 1100 outputs the data generated via the input / output interface 1600 to the output device.

The media interface 1700 reads a program or data stored in the recording medium 1800 and provides the program or data to the CPU 1100 via the RAM 1300. The CPU 1100 loads the program from the recording medium 1800 onto the RAM 1300 via the media interface 1700, and executes the loaded program. The recording medium 1800 is, for example, an optical recording medium such as a DVD (Digital Versatile Disc) or PD (Phase change rewritable Disk), a magneto-optical recording medium such as MO (Magneto-Optical disk), a tape medium, a magnetic recording medium, or a semiconductor memory. And so on.

For example, when the computer 1000 functions as the terminal device 100, the CPU 1100 of the computer 1000 realizes the function of the control unit 150 by executing the program loaded on the RAM 1300. Further, each data in the storage device of the terminal device 100 is stored in the HDD 1400. The CPU 1100 of the computer 1000 reads and executes these programs from the recording medium 1800, but as another example, these programs may be acquired from another device via a predetermined communication network.

[8. others〕
Although some of the embodiments of the present application have been described in detail with reference to the drawings, these are examples, and various modifications are made based on the knowledge of those skilled in the art, including the embodiments described in the disclosure column of the invention. It is possible to carry out the present invention in other modified forms.

Further, the configuration of the analysis server 10 and the payment server 12 described above can be flexibly changed, such as by calling an external platform or the like by API (Application Programming Interface), network computing, or the like depending on the function.

Further, the "part" described in the claims can be read as "means" or "circuit". For example, the acquisition unit can be read as an acquisition means or an acquisition circuit.

10 Analysis server 12 Payment server 20 Communication unit 30 Storage unit 40 Control unit 41 Reception unit 42 Analysis unit 43 Transmission unit 44 Learning unit 100 Terminal device 110 Communication unit 120 Storage unit 130 Camera 140 Touch panel 150 Control unit 151 1st application 152 2nd Application 1521 Shooting unit 1522 Transmitting unit 1523 Receiving unit 1524 Payment processing unit 1525 Notification unit

Claims (12)

It ’s a terminal device,
An image containing information about payment details , as well as a transmitter that sends context information indicating the context when the image was taken to the analysis server, and a transmitter.
A receiving unit that receives information on the payment content obtained by analyzing the image based on the context information , or information on the fact that the analysis has failed and on the cause of the failure.
A payment processing unit that executes payment processing based on the payment content information when the payment content information is received.
A terminal device comprising a notification unit that notifies a user of the terminal device based on the information that the analysis has failed when the information indicating that the analysis has failed is received. It ’s a terminal device,
An analysis server that analyzes the image to obtain information on the payment details, along with an image containing information on the payment details, and context information indicating the environment in which the terminal device is placed when the image is taken. A transmitter that sends information about the content to an analysis server that analyzes based on a model that has learned the characteristics of the context indicated by the context information when an image that can be analyzed is taken.
A receiving unit that receives information on payment details obtained by analyzing the image transmitted to the analysis server based on the context information.
A terminal device having a payment processing unit that executes payment processing based on the received information on payment details. The terminal device according to claim 1 or 2 , wherein the transmission unit transmits information indicating the type of the terminal device to the analysis server as the context information. One of claims 1 to 3 , wherein the transmitting unit transmits the sensor information acquired by the sensor of the terminal device when the image is taken as the context information to the analysis server. The terminal device described in 1. One of claims 1 to 4 , wherein the transmitting unit transmits, as the context information, information indicating software executed by the terminal device when the image is taken, to the analysis server. The terminal device described in. Claim 1 is characterized in that the transmitting unit determines whether or not the image contains information regarding the payment content, and if it is determined that the image contains the information, transmits the image to the analysis server. The terminal device according to any one of 5 to 5 . The transmitting unit determines whether or not the image contains personal information, and if it is determined that the image contains personal information, the transmitting unit transmits an image obtained by removing the personal information from the image to the analysis server. The terminal device according to any one of claims 1 to 6 . The transmission unit transmits a plurality of images including information on the payment contents to the analysis server.
The terminal device according to any one of claims 1 to 7 , wherein the receiving unit receives information regarding the payment content obtained by analyzing the plurality of images. It is a payment method executed by the terminal device.
A transmission process that sends context information indicating the context when the image was taken, along with an image containing information about payment details, to the analysis server.
A receiving process for receiving information on the payment details obtained by analyzing the image based on the context information , or information indicating that the analysis has failed and on the cause of the failure.
A payment processing process that executes payment processing based on the payment content information when the payment content information is received, and
A payment method comprising a notification step of notifying a user of the terminal device based on the information that the analysis has failed when the information that the analysis has failed is received. A transmission procedure for transmitting to the analysis server context information indicating the context when the image was taken, along with an image containing information on payment details.
A receiving procedure for receiving information on the payment details obtained by analyzing the image based on the context information , or information indicating that the analysis has failed and on the cause of the failure.
A payment processing procedure for executing payment processing based on the payment content information when the payment content information is received, and a payment processing procedure.
A payment program for causing a computer to execute a notification procedure for notifying a user of a terminal device based on the information that the analysis has failed when the information that the analysis has failed is received. It is a payment method executed by the terminal device.
An analysis server that analyzes the image to obtain information on the payment details, along with an image containing information on the payment details, and context information indicating the environment in which the terminal device is placed when the image is taken. A transmission process that sends information about the content to an analysis server that analyzes based on a model that has learned the characteristics of the context indicated by the context information when an image that can be analyzed is taken.
A receiving process for receiving information on payment details obtained by analyzing the image transmitted to the analysis server based on the context information, and
A payment method comprising: a payment processing step of executing payment processing based on the received information regarding payment contents. An analysis server that analyzes the image to obtain information on the payment details, along with an image containing information on the payment details, and context information indicating the environment in which the terminal device was placed when the image was taken. The transmission procedure to send to the analysis server that analyzes based on the model that learned the characteristics of the context indicated by the context information when the image that can be analyzed is taken, and the transmission procedure.
A receiving procedure for receiving information on payment details obtained by analyzing the image transmitted to the analysis server based on the context information, and a receiving procedure.
A payment program for causing a computer to execute a payment processing procedure that executes payment processing based on the received information on payment details.

Images