JP2024052468A - Application program, service providing system, and information providing method - Google Patents

Abstract

[Problem] To provide useful information to users. [Solution] An application program that causes a computer of a terminal device to acquire an image captured by an imaging unit, identify service information corresponding to the feature information contained in the image by referring to correspondence information stored in a storage unit that associates feature information contained in the image with service information related to electronic payment services, and display the identified service information on a display unit of the terminal device by superimposing it on the image captured by the imaging unit. [Selected Figure] Figure 1

Description

The present invention relates to an application program, a service providing system, and an information providing method.

Conventionally, a support device has been disclosed that combines other information with an actual image and displays the image on a terminal device (see, for example, Patent Document 1).

Patent No. 6704578

Conventional technology sometimes fails to obtain information about features contained in an image, and is therefore unable to provide useful information to users.

The present invention was made in consideration of these circumstances, and one of its objectives is to provide an application program, a service providing system, and an information providing method that can provide useful information to users.

One aspect of the present invention is an application program that causes a computer of a terminal device to acquire an image captured by an imaging unit, identify service information corresponding to feature information contained in the image by referring to correspondence information stored in a storage unit that associates feature information contained in the image with service information related to electronic payment services, and superimpose the identified service information on the image captured by the imaging unit and display it on a display unit of the terminal device.

Another aspect of the present invention is an information provision method in which a computer of a terminal device acquires an image captured by an imaging unit, identifies service information corresponding to the feature information contained in the image by referring to correspondence information stored in a storage unit that associates feature information contained in the image with service information related to electronic payment services, and displays the identified service information on a display unit of the terminal device by superimposing it on the image captured by the imaging unit.

According to one aspect of the present invention, it is possible to provide an application program, a service providing system, and an information providing method that can provide useful information to users.

FIG. 1 is a diagram showing an example of a configuration for realizing an electronic payment service. FIG. 1 is a sequence diagram (part 1) illustrating an example of a general flow of electronic payment. FIG. 2 is a sequence diagram (part 2) illustrating an example of a general flow of electronic payment. FIG. 2 is a configuration diagram of a payment server 100 according to the first embodiment. FIG. 11 is a diagram showing an example of the contents of user information 172. FIG. 13 is a diagram showing an example of the contents of affiliated store/store information 176. FIG. 13 is a diagram showing an example of the contents of first correspondence information 178A. FIG. 13 is a diagram showing an example of the contents of second correspondence information 178B. FIG. 13 is a diagram showing an example of how the user terminal device 10 captures an image including the logo of an electronic payment service (the logo of a specific brand). 2 is a diagram showing an example of content displayed on a display unit of a user terminal device 10. FIG. 13 is a diagram showing another example of content displayed on the display unit of the user terminal device 10. FIG. FIG. 13 is a diagram showing an example of how the user terminal device 10 captures an image including the logo of a store where electronic payment services are available. 13 is a diagram showing another example of content displayed on the display unit of the user terminal device 10. FIG. 13 is a diagram showing another example of content displayed on the display unit of the user terminal device 10. FIG. 11 is a flowchart showing an example of a process flow related to an information provision service executed by a payment application 20 (information provision application 30) and a payment server 100. 13 is a flowchart showing another example of the flow of a process related to an information provision service executed by the payment application 20 (information provision application 30) and the payment server 100. 1 is a diagram showing an example of service information displayed on a display unit of a user terminal device 10. FIG. 1 is a diagram showing an example of service information displayed on a display unit of a user terminal device 10. FIG. FIG. 11 is a diagram showing an example of content for a user to make a payment using an electronic payment service. 13 is a flowchart showing an example of a process related to ordering a product, which is executed by a payment application 20 and a payment server 100. FIG. 13 is a diagram showing another example of the correspondence information A. FIG. 4 is a diagram showing an example of content displayed on a display unit.

Below, with reference to the drawings, an embodiment of the application program, the service providing system, the server device, and the information providing method of the present invention will be described. The application program and the payment server work together to provide an electronic payment service. In the following description, the application program is referred to as a payment app. The electronic payment service is a service that supports payments related to the purchase of goods and services at a store. The store is, for example, a physical store (real store) that exists in the real world, but may also include a virtual store for electronic commerce. The virtual store may also include a store provided by an entity other than the operator of the electronic payment service. In that case, when making a payment for a purchase at the virtual store, the screen is controlled to transition to an interface screen of the electronic payment service. In the electronic payment service, the store is treated as belonging to, for example, an affiliated store (brand), and processing such as payment when a purchase is made at the store is mainly performed between the user and the affiliated store. Alternatively, processing such as payment may be performed between the user and the store.

[Electronic payment service]
1 is a diagram showing an example of a configuration for realizing an electronic payment service. The electronic payment service is realized mainly by a payment server 100. The payment server 100 communicates with, for example, one or more user terminal devices 10, one or more first store terminal devices 50, and one or more second store terminal devices 70 via a network NW. The network NW includes, for example, the Internet, a LAN (Local Area Network), a wireless base station, a provider device, and the like.

The user terminal device 10 is, for example, a portable terminal device such as a smartphone or a tablet terminal. The user terminal device 10 is a computer device having at least an optical reading function (e.g., a camera), a communication function, a display function, an input reception function, a position identification function (e.g., a position identification unit), and a program execution function. In the following description, the components for realizing these functions are respectively referred to as a camera, a communication device, a touch panel, a self-position identification unit, a CPU (Central Processing Unit), etc. In the user terminal device 10, a processor such as a CPU executes a payment app 20, thereby operating to provide an electronic payment service to a user in cooperation with a payment server 100. The payment app 20 is installed in the user terminal device 10 from, for example, an application store, and controls the camera, communication device, touch panel, etc. The position identification unit identifies the position (position coordinates) of the user terminal device 10 using a Global Navigation Satellite System (GNSS).

The user terminal device 10 operates to provide an information provision service (described later) to the user in cooperation with the payment server 100 by executing the information provision app 30 by a processor such as a CPU. The information provision app 30 may be, for example, an application program included in the payment app 20, or may be a separate application program that is installed on the user terminal device 10 from an application store. The information provision app 30 controls a camera, a communication device, a touch panel, etc.

The first store terminal device 50 is installed, for example, in a store. The first store terminal device 50 is a computer device having at least a product price acquisition function, an optical reading function, a program execution function, and a communication function. The first store terminal device 50 includes a so-called POS (Point of Sale) device, and the product price acquisition function and the optical reading function may be realized by the POS device. The store code image 60 is placed in the store, and is a code image such as a QR code (registered trademark) printed on a paper or plastic medium. The store code image 60 may be displayed on a display placed in the store (which may be the display of a terminal device such as a smartphone).

The second store terminal device 70 is used by the operator of the affiliated store. The second store terminal device 70 is a smartphone, a tablet terminal, a personal computer, etc. An interface 72 for affiliated stores runs on the second store terminal device 70. The interface 72 for affiliated stores may be an app for affiliated stores or a browser. The interface 72 for affiliated stores accepts coupon settings and the like made by the operator of the affiliated store and transmits them to the payment server 100. The second store terminal device 70, which is a smartphone, has the function of displaying a code image equivalent to a store code image and reading a code image displayed by the user terminal device 10 by executing the app for affiliated stores.

The payment server 100 realizes electronic payment based on payment information received from the user terminal device 10 or the first store terminal device 50. The first store terminal device 50 may include a POS device and an affiliated store server, in which case the payment information is sent from the POS device to the payment server 100 via the affiliated store server. In the following explanation, no distinction is made between these two, and it is assumed that the payment information is sent from the first store terminal device 50.

Figures 2 and 3 are sequence diagrams illustrating the general flow of electronic payment. There may be two patterns for electronic payment: pattern 1 and pattern 2.

In the case of pattern 1 (hereinafter referred to as user scan) shown in FIG. 2, the user terminal device 10 with the payment application 20 activated decodes the store code image 60 by the optical reading function (S1). The store code image 60 includes store URL (Uniform Resource Locator) information. This store URL is the domain of the electronic payment service to which information capable of identifying the store is added, and is associated with the affiliated store ID, store ID, etc. in the payment server 100 (described later). The payment application 20 transmits the first payment information including the store URL and the account ID to the payment server 100 (S2). The payment server 100 searches for store information (described later) from the affiliated store ID and store ID corresponding to the store URL, acquires the affiliated store name and store name information (S3), and transmits it to the payment application 20 (S4). The user inputs the payment amount into the user terminal device 10 on the screen on which the affiliated store name and store name are displayed (S5). Then, the user terminal device 10 generates second payment information including at least the payment amount and transmits it to the payment server 100 (S6). The payment server 100 performs electronic payment based on the received second payment information (S7). The payment server 100 then transmits a payment completion notice (information for displaying a payment completion screen) to the payment application 20 (S8), and the payment application 20 displays the payment completion screen (S9). Note that when the store code image 60 is displayed on a display installed in the store, the store code image 60 may include not only the store URL but also information on the payment amount. In this case, the procedure in which the user inputs the payment amount is omitted, and the first payment information is transmitted to the payment server 100 with the information on the payment amount included. Information on the affiliated store name and store name may be included and displayed on the payment completion screen.

In the case of pattern 2 (hereinafter referred to as store scan) shown in FIG. 3, when the payment app 20 is started, when a payment operation is performed in the payment app 20, when the automatic update timing (for example, every minute) occurs, and at other timings, the payment app 20 sends a request to issue a one-time code to the payment server 100 (S11). The payment server 100 generates a one-time code (S12) and sends it to the payment app 20 (S13). The payment app 20 displays a code image such as a QR code or a barcode generated based on the one-time code (S14). The user holds (presents) the display surface of the user terminal device 10 over the first store terminal device 50, and the first store terminal device 50 decodes the code image using an optical reading function and obtains the one-time code, etc. (S15). Then, the first store terminal device 50 generates payment information including the one-time code, payment amount, affiliated store ID, store ID, etc., and sends it to the payment server 100 (S16). The payment amount information is acquired in advance by reading a barcode or manually entering it. Based on the received information, the payment server 100 identifies the user corresponding to the one-time code and performs electronic payment (S17). The payment server 100 then sends a payment completion notice to the payment application 20 (S18), and the payment application 20 displays a payment completion screen (S19).

Note that electronic payment may be made using only one of the above patterns. Furthermore, the "account ID" described in FIG. 2 may be other information (e.g., a phone number) that can be used as user identification information. Furthermore, issuance of a one-time code may be omitted in the store scan, and the payment app 20 may display a code image generated based on the user's account ID. In that case, the payment server 100 identifies the user corresponding to the account ID instead of identifying the user corresponding to the one-time code.

[Payment server]
4 is a configuration diagram of the payment server 100 according to the first embodiment. The payment server 100 includes, for example, a communication unit 110, a payment content providing unit 120, a payment processing unit 130, an information management unit 140, a specification unit 142, an information providing unit 144, and a storage unit 170. The components other than the communication unit 110 and the storage unit 170 are realized by, for example, a hardware processor such as a CPU executing a program (software). Some or all of these components may be realized by hardware (including circuitry) such as an LSI (Large Scale Integration), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a GPU (Graphics Processing Unit), or may be realized by cooperation between software and hardware. The program may be stored in advance in a storage device (a storage device having a non-transient storage medium) such as a hard disk drive (HDD) or a flash memory, or may be stored in a removable storage medium (a non-transient storage medium) such as a DVD or a CD-ROM, and may be installed in the storage device by mounting the storage medium in a drive device. Note that a part of the functional configuration included in the payment server 100 may be distributed to other devices.

The storage unit 170 is a HDD, flash memory, RAM (Random Access Memory), etc. The storage unit 170 may be a NAS (Network Attached Storage) device that the payment server 100 can access via a network. The storage unit 170 stores information such as user information 172, payment content information 174, affiliated store/shop information 176, and correspondence information 178. The correspondence information 178 includes first correspondence information 178A and second correspondence information 178B.

The communication unit 110 is a communication interface for connecting to the network NW. The communication unit 110 is, for example, a network interface card.

The payment content providing unit 120 has, for example, a web server function, and provides information (content) for displaying various screens of the electronic payment service to the user terminal device 10. The payment content providing unit 120 reads out necessary content from the payment content information 174 as appropriate and provides it to the user terminal device 10. The user terminal device 10 accepts various inputs by the user while the content is being played by the payment application 20, and transmits the above-mentioned payment information and the like to the payment server 100.

The payment processing unit 130 performs payment processing based on the payment information transmitted by the user terminal device 10 or the first store terminal device 50. The payment processing unit 130 performs payment processing while referring to the user information 172.

5 is a diagram showing an example of the contents of the user information 172. The user information 172 is an example of the registration information of a user. The user information 172 is, for example, a user URL, an account ID, a telephone number, a password, as well as information associated with an email address, a user ID, a name, an address, a date of birth, a registration date, a charge balance, a deferred payment setting, a deferred payment limit, a deferred payment usage amount, a deferred payment available amount, a payment method setting, a bank account, a credit card number, a charge history information, and a payment history information. The user URL is used for a remittance process between users. When registering for the electronic payment service, it is necessary to register a telephone number and a password. The account ID is issued to the user by the payment server 100, and the user ID is an ID that can be set by the user at will (does not have to be set). The email address, and the name, address, and date of birth are also information that can be set by the user at will (does not have to be set). The registration date is the date on which the user registered for the electronic payment service (the date on which the account was created). Hereinafter, the user instance (electronic payment account) to which this information is associated will be referred to as an account.

The charge balance is information indicating the balance of electronic money that is set by the user by transferring money to the account in advance. The means of transfer include transfer from an ATM (Automatic Teller Machine) of a designated company (bank) and transfer from a registered bank account. The deferred payment setting is information indicating whether or not the setting for enabling electronic payment by deferred payment has been completed, and is set to either "completed" or "not completed." The deferred payment limit is the limit of deferred payment that can be used each month, the deferred payment usage amount is the amount of deferred payment that has already been used in the current month, and the available deferred payment amount is the amount of deferred payment that can be used in the current month, which is calculated by subtracting the deferred payment usage amount from the deferred payment limit. In the figure, only one deferred payment limit is shown, but in reality, there is also a daily upper limit, and the lower of these may be set as the deferred payment limit. Further details of deferred payment will be described later. The payment method setting is setting information indicating whether the user will make electronic payment using the charge balance or payment by deferred payment at that time. The bank account and credit card number are information on a bank account or credit card number (account number, card number) that can be used to deposit funds into an electronic payment service. The charge history information is a history of the user transferring funds to the electronic payment service in advance to increase the charge balance. The payment history information is information that indicates the details of payments made by the user for each payment (date and time, store ID of the store where the purchase was made, payment amount, payment method, etc.).

Figure 6 is a diagram showing an example of the contents of affiliated store/store information 176. Affiliated store/store information 176 includes, for example, a first table 176A in which an affiliated store ID and a store ID are associated with a store URL, a second table 176B in which an affiliated store name and sales amount (described above) are associated with an affiliated store ID, and a third table 176C in which a store ID is associated with a store ID. In addition to this information, affiliated store/store information 176 may also include information such as the affiliated store or store category, the store's location, and payment patterns.

The information management unit 140 manages the user information 172 and the affiliated store/store information 176 based on information acquired from the user terminal device 10 and the second store terminal device 70. The information management unit 140 adds new records, edits, deletes, etc. for the user information 172 and the affiliated store/store information 176.

The identification unit 142 identifies service information corresponding to the image obtained from the payment application 20 or the feature information contained in the image. For example, the identification unit 142 identifies the service information by referring to the correspondence information 178 (details will be described later).

The information providing unit 144 provides the service information identified by the identification unit 142 to the payment application 20.

[Electronic Payment]
When payment information is acquired from the user terminal device 10 or the first store terminal device 50, the payment processing unit 130 refers to the user information 172 to acquire the "payment method setting" of the user. For a user whose "payment method setting" is set to "charge balance", the payment processing unit 130 performs electronic payment as follows. For example, the payment processing unit 130 performs electronic payment by decreasing the charge balance managed in association with the user ID and increasing the item value of the affiliated store's sales. The item value of the affiliated store's sales is not used as electronic money itself, for example, but an amount corresponding to the item value of the sales is transferred to a bank account in a cycle according to an agreement between the affiliated store and the electronic payment service.

The payment processing unit 130 performs electronic payment for users whose "setting information" is set to "deferred payment" as follows. Deferred payment is set separately from "credit payment" in cooperation with a credit card company, which is a separate entity from the operator of the electronic payment service, and the operator of the electronic payment service acts as the creditor and allows electronic payment that is not dependent on the charge balance within the deferred payment limit. In order to receive the deferred payment service, it may be required to obtain a credit card provided by the operator of the electronic payment service. The amount used for deferred payment is settled on the payment date of the following month, for example, by debiting from a bank account, for one month's worth. In this case, the payment processing unit 130 performs provisional payment by adding the payment amount to the deferred payment amount and subtracting the same amount from the available deferred payment amount, and when the closing date comes, it performs processing to debit the payment for the current month on the payment date of the following month as described above. In addition, if the payment amount exceeds the available deferred payment amount at the time of provisional payment, an error notification is returned to the payment application 20.

[Information provision service]
The information providing application 30 (information providing application 30) and the payment server 100 cooperate to provide an information providing service. At this time, the payment server 100 refers to the first correspondence information 178A and/or the second correspondence information 178B and provides the information providing application 30 with service information corresponding to the image acquired from the information providing service or the feature information included in the image. The information providing application 30 causes the display unit of the user terminal device 10 to display the provided service information. For example, the service information is displayed superimposed on a landscape image.

A landscape image is an image of a landscape captured by the camera of the user terminal device 10. The characteristic information contained in the image is information indicating the characteristics of the logo that represents the brand of the electronic payment service, and information indicating the characteristics of the logos of stores such as affiliated shops and shops that can be used with the electronic payment service. Examples of logos include letters, figures, patterns, and combinations of these. The logo is a so-called AR (Augmented Reality) marker.

Service information is information related to electronic payment services. Information related to electronic payment services includes information related to stores (affiliated stores or shops) where electronic payment services can be used. Service information is, for example, one or more of the following information: coupon information available at electronic payment services (or specified affiliated stores or shops), campaigns in electronic payment services (lottery information, which will be described later, may also be included in the campaign information), or electronic stamp card information available at electronic payment services. A stamp card is a tool for providing a service in which stamps are awarded in accordance with electronic payments at a target store, and incentives are awarded to users when stamps are accumulated.

(First correspondence information)
The first correspondence information 178A is information for identifying service information corresponding to the logo of an electronic payment service, which is characteristic information included in an image. Fig. 7 is a diagram showing an example of the contents of the first correspondence information 178A. The first correspondence information 178A is information in which, for example, identification information, characteristic information of the logo of an electronic payment service, location information, and service information are associated with each other. In the first correspondence information 178A, information on stores located in the vicinity of the location where the logo of the characteristic information is located is set as service information. These stores are stores where the electronic payment service can be used.

(Second response information)
The second correspondence information 178B is information for identifying service information corresponding to the logo of an electronic payment service store, which is characteristic information included in the image. Fig. 8 is a diagram showing an example of the contents of the second correspondence information 178B. The second correspondence information 178B is information in which, for example, identification information, characteristic information of the logo of an electronic payment service store, location information, and service information are associated with each other. In the second correspondence information 178B, information related to the store at the location where the logo of the characteristic information is present is regarded as service information. In the example of Fig. 8, information such as coupons, stamp cards, and campaigns is service information.

[Information provision service using first response information]
Fig. 9 is a diagram showing an example of how the user terminal device 10 captures an image including a logo of an electronic payment service (a logo of a specific brand). As shown in Fig. 10, when the logo is captured, information (service information) of nearby stores where multiple (one or more) electronic payment services are available is displayed on the display unit of the user terminal device 10. For example, any information such as the name of the store, the store's logo, and the products handled by the store may be displayed. The service information is displayed superimposed on the captured landscape image.

When a user performs a predetermined operation, such as a tap operation, on an area corresponding to store information, content including other information about the store is displayed, as shown in FIG. 11. For example, as shown in FIG. 11, content including the store's business hours and the store's menu may be displayed, or advantageous information for the user, such as store coupons and campaigns, may be displayed. In addition, information on the route from the location of the user terminal device 10 to the store in the map information, information indicating the location of the user terminal device 10 and the location of the store in the map information, etc. may be displayed on the display unit of the user terminal device 10.

[Information provision service using second response information]
Fig. 12 is a diagram showing an example of how the user terminal device 10 captures an image including a logo of a store where an electronic payment service is available. As shown in Fig. 12, when the logo is captured, information about a plurality of stores (one or more) is displayed on the display unit of the user terminal device 10. For example, any information such as information about campaigns being held at the store, information about coupons that can be used at the store, and products handled at the store may be displayed.

When a user performs a predetermined operation, such as a tap operation, on an area corresponding to predetermined information, content corresponding to the information is displayed as shown in FIG. 13. This service information is displayed superimposed on the captured landscape image.

For example, when an area corresponding to a coupon is operated, content including detailed information about the coupon is displayed, as shown in FIG. 14. Furthermore, depending on the operation, the user can acquire a coupon. Acquiring a coupon means that coupon information is associated with the identification information of the user of the payment app 20 so that the coupon can be used in the payment app 20. This allows the user to use the coupon by performing a specified operation in electronic payment using the payment app 20.

The location information may be omitted from the first correspondence information 178A or the second correspondence information 178B. In this case, the service information corresponding to the feature information is provided without taking into account the location information.

[Flowchart (part 1)]
15 is a flowchart showing an example of a flow of a process related to an information provision service executed by the payment application 20 (information provision application 30) and the payment server 100. In this process, the payment application 20 (information provision application 30) acquires an image captured by a camera, transmits the acquired image or feature information contained in the image to the payment server 100 (server device), and requests the payment server 100 to transmit service information corresponding to feature information contained in an image specified by the payment server 100 with reference to correspondence information 178 in which the feature information is associated with service information related to an electronic payment service. In response to the request, the payment application 20 acquires the service information transmitted by the payment server 100, and displays the acquired service information on the display unit of the user terminal device 10 by superimposing it on the image captured by the camera.

First, the payment application 20 activates the camera based on the user's operation to capture an image (S100). Next, the payment application 20 transmits the captured image, the user's identification information, and the location information of the user terminal device 10 to the payment server 100 (S102).

Next, the payment server 100 acquires service information corresponding to the logo and location information based on the logo included in the image, the location information of the user terminal device 10, and the correspondence information 178 (S104). Next, the payment server 100 transmits the service information to the payment application 20 (S106). Next, the payment application 20 displays content in which the service information is superimposed on an image of a landscape on the display unit (S108).

When a user performs an operation on the service information, the payment app 20 accepts the operation on the service information (S110) and requests the payment server 100 to provide information corresponding to the operation (S112). Next, the payment server 100 acquires information corresponding to the request (S114) and provides the acquired information corresponding to the request to the payment app 20 (S116). Next, the payment app 20 displays the information provided by the payment server 100 on the display unit (S118). At this time, the provided information may be superimposed on a landscape image as shown in Figures 10 and 13 described above. For example, the process executed by the payment app 20 described above is a function executed by the information providing app 30 included in the payment app 20.

As a result, when a user captures an image that includes a logo, service information corresponding to the logo is displayed on the display unit of the user terminal device 10. For example, a user can obtain useful information by capturing an image of a logo while walking down the street. In this way, it is possible to provide users with information that is useful to them while improving entertainment value. Furthermore, the information provision service is also ideal for operators of stores and electronic payment services, as it serves as a promotion for the store or electronic payment service.

For example, if the image does not contain a logo in the process of S104, the payment server 100 may send information indicating that the logo is not included to the payment app 20, or may provide predetermined information such as information on stores near the user's location where electronic payment services are available or campaigns for electronic payment services. Also, if the image does not contain a logo, the payment server 100 may provide the payment app 20 with service information corresponding to the location information of the user terminal device 10. In this case, for example, the correspondence information 178 includes information that associates location information with service information.

In addition, in S102, the payment app 20 transmits an image to the payment server 100, but instead (or in addition), an image in which the logo features or an area containing the logo is extracted from the image may be transmitted to the payment server 100. For example, the payment app 20 uses a predetermined image processing algorithm or information indicating the logo features stored in the memory unit of the user terminal device 10 to extract the logo features, extract an area containing the logo from the image, and transmits the logo features or the extracted area containing the logo to the payment server 100. In this way, if only a part of the captured image, rather than the entire image, or the results of processing the image (logo features) are transmitted to the payment server 100, the communication load is reduced.

In addition, in S112, the payment app 20 sent a request for information to the payment server 100, but instead (or in addition) it may send a request to another server. The other server is, for example, a predetermined server device, such as a server device corresponding to the store of the service information corresponding to the operation. For example, when the service information of "XX coffee" is operated, the payment server 100 requests the server device corresponding to "XX coffee" to provide the information. For example, the service information provided to the user is associated with information of the server device that is requested to provide the information in advance (for example, information of the access destination). The payment app 20 refers to this information and requests the other server device to provide the information.

[Flowchart (part 2)]
16 is a flowchart showing another example of the flow of processing related to the information provision service executed by the payment application 20 (information provision application 30) and the payment server 100. In this processing, the payment application 20 acquires the correspondence information 178, and identifies service information from a logo using the correspondence information 178. The payment application 20 (information provision application 30) acquires an image captured by a camera, identifies service information corresponding to the feature information included in the image by referring to correspondence information stored in the storage unit of the user terminal device 10, which associates feature information included in the image with service information related to electronic payment services, and displays the identified service information on the display unit of the user terminal device 10 by superimposing it on the image captured by the camera.

First, the payment application 20 acquires the correspondence information 178 from the payment server 100 or another server device (S200). Next, the payment application 20 activates the camera based on the user's operation to acquire an image (S202).

Next, the payment server 100 acquires service information corresponding to the logo and location information based on the logo included in the image, the location information of the user terminal device 10, and the correspondence information 178 (S204). Next, the payment server 100 displays content in which the service information is superimposed on an image of a landscape on the display unit (S206).

When the user performs an operation on the service information, the payment app 20 accepts the operation on the service information (S208) and requests the payment server 100 to provide information corresponding to the operation (S210). Next, the payment server 100 acquires information corresponding to the request (S212) and provides the acquired information corresponding to the request to the payment app 20 (S214). Next, the payment app 20 displays the information provided by the payment server 100 on the display unit (S216). At this time, the provided information may be superimposed on a landscape image as shown in Figures 10 and 13 described above. For example, the process executed by the payment app 20 described above is a function executed by the information providing app 30 included in the payment app 20.

As a result, when a user captures an image that includes a logo, service information corresponding to the logo is displayed on the display unit of the user terminal device 10. In addition, in this process, the payment app 20 identifies the service information, so that it is possible to provide the user with information that is useful to the user while reducing the communication load.

[Another example of service information (part 1)]
The service information may be entertaining information such as a lottery. FIG. 17 is a diagram showing an example of service information displayed on the display unit of the user terminal device 10. For example, if the service information is a lottery, tapping an area corresponding to the lottery information causes the lottery to be animated and the winning result to be displayed. For example, if the service information is a logo of an electronic payment service, the lottery is a lottery for winning points for the electronic payment service or a special benefit such as a discount at a store where the electronic payment service can be used. For example, if the service information is a logo of a store that provides an electronic payment service, the lottery is a lottery for winning points for the store or a special benefit such as a discount at the store.

[Another example of service information (part 2)]
The service information may be information for making a payment using an electronic payment service. The information for making a payment is, for example, information for guiding a user to a payment using an electronic payment service. For example, the information for guiding a user to a payment is information for ordering or reserving a product, service, or the like. For example, it is information for making a mobile order.

Figure 18 is a diagram showing an example of service information displayed on the display unit of the user terminal device 10. For example, the display unit displays, as service information, information for making a mobile order (or reservation) for an item at the logo store (an example of a "reception unit"). When the user operates the area for information for making a mobile order, the content shown in Figure 19 is displayed.

Figure 19 is a diagram showing an example of content for a user to make a payment using an electronic payment service. For example, this content includes information about the product to be purchased, the price of the product, information about the store where the product is purchased, the time until the product is provided, the payment method in the electronic payment service, and a button for confirming the payment. The time until the product is provided is obtained from the store's second store terminal device 70 or a server device that manages store information. When the button for confirming the payment is operated, the payment is made and the product is ordered. The content shown in Figure 19 is an example of "content related to the electronic payment, which is provided by the server device in response to a request for the electronic payment and order of the product or service from a server device."

[Flowchart for ordering products]
Fig. 20 is a flowchart showing an example of a process for ordering a product, which is executed by the payment application 20 and the payment server 100. In this process, it is assumed that information for mobile ordering is displayed on the display unit as shown in Fig. 18 described above.

When a user operates the area for information for mobile ordering, the payment application 20 transmits information indicating that the operation has been performed to the payment server 100 (S300). The payment server 100 identifies payment content (e.g., see FIG. 19) corresponding to the operation (S302), and transmits the identified payment content to the payment application 20 (S304).

Next, the payment application 20 displays the transmitted payment content on the display unit (S306). When the user operates a button to confirm the payment in the payment content, the payment application 20 transmits payment instruction information to the payment server 100 instructing the user to confirm the payment (S308). Next, upon acquiring the payment instruction information, the payment server 100 performs the payment (S310) and orders the product from the store according to the payment (S312). For example, the payment server 100 transmits product order information to the second store terminal device 70 of the store. This causes the ordered product to be prepared at the store.

Next, the payment server 100 provides the payment application 20 with content indicating that the payment and order have been completed (S314). The payment application 20 displays the provided content on the display unit (S316).

As described above, users can easily order products by manipulating the service information.

[Modification 1]
The payment application 20 or the payment server 100 may determine the service information to be provided for each user. For example, the payment application 20 or the payment server 100 determines the service information based on the payment history of the user for the electronic payment service. For example, when providing service information corresponding to the logo of the electronic payment service, the member stores or stores where the user frequently makes electronic payments may be preferentially displayed on the display unit, or the member stores or stores where the user makes a large number of payments or the user makes a large amount of payments may be preferentially displayed on the display unit.

For example, correspondence information 178 may be prepared for each user, and service information may be provided by referencing correspondence information 178 appropriate for the user. FIG. 21 is a diagram showing another example of first correspondence information 178A. For example, as described above, service information is associated with characteristic information based on the user's payment history.

[Modification 2]
The payment application 20 or the payment server 100 may determine the service information to be provided for each user. For example, when providing service information corresponding to a store's logo, the information of the product or service for which the user has paid using the electronic payment service may be preferentially displayed on the display unit, or the product or service for which the user has paid many times or the product or service for which the user has paid a large amount may be preferentially displayed on the display unit. In the second modification, the payment server 100 is assumed to manage information of the product for which payment has been made as a payment history. For example, the payment server 100 is assumed to manage information of electronic payments (including information of purchased products or services) using the electronic payment service in mobile orders and online shopping.

For example, when an image including a store's logo is captured, the payment app 20 causes the display unit to display information about products that have been purchased frequently at that store in the user's payment history as service information. FIG. 22 is a diagram showing an example of content displayed on the display unit. For example, when a user frequently places mobile orders for lunch set A and lunch set B at the store using electronic payment, the payment app 20 causes the display unit to display service information for mobile orders for lunch set A and lunch set B.

As a result, when a user captures an image of the logo of an affiliated store or shop, information for mobile ordering is displayed on the display, and the user can easily order products by operating the area for information for mobile ordering. Note that in variant example 2, the process is described as being performed when an image of an affiliated store or shop logo is captured, but instead (or in addition), the process may also be performed when an image of an electronic payment service logo is captured. For example, information for ordering products that are frequently purchased by the user and that are located near the location of the user terminal device 10 is displayed on the display.

In addition to (in addition to) determining the service information to be displayed for each user as described above, the service information to be provided may be determined according to the environment in which the user is located. For example, if it is a hot day, a cold drink may be provided as the service information, and if it is lunchtime, information on stores that serve lunch may be provided as the service information.

In each of the above embodiments, the information providing app 30 is included in the payment app 20, and the information providing app 30 executes processing together with the payment app 20. However, instead of (or in addition to) this, the information providing app 30 may be a separate application program that is not included in the payment app 20. In this case, the information providing app 30, the payment app 20, and the payment server 100 similarly cooperate to provide various services. Also, only the information providing app 30 may be provided to the user terminal device 10, and the information providing app 30 may run on the user terminal device 10.

In addition, the information provision app 30 has been described as being installed on a user terminal device 10 such as a smartphone, but instead (or in addition), the information provision service may be executed on a terminal device dedicated to executing the processing of the information provision app 30.

In the above example, the service information is identified from a logo included in the image. However, instead of (or in addition to) this, the service information may be identified from an object or display that has characteristics other than a logo. The object or display that has characteristics may be arbitrarily determined, such as a distinctive building, an advertisement in a city, or a person in an advertisement.

According to the embodiment described above, the payment application 20 can provide the user with useful information by providing the user with service information corresponding to the logo contained in the image.

The above describes the form for carrying out the present invention using an embodiment, but the present invention is not limited to such an embodiment, and various modifications and substitutions can be made without departing from the spirit of the present invention.

10 User terminal device 20 Payment application 30 Information provision application 100 Payment server 120 Payment content provision unit 142 Identification unit 144 Information provision unit 178 Corresponding information

Claims (13)

The terminal device computer
Acquire an image captured by the imaging unit;
Identifying service information corresponding to the feature information included in the image by referring to correspondence information stored in a storage unit in which the feature information included in the image is associated with service information related to an electronic payment service;
displaying the specified service information on a display unit of the terminal device in a state in which the specified service information is superimposed on the image captured by the imaging unit;
Application program. The correspondence information corresponds characteristic information of the logo of the electronic payment service to service information which is information of a store where the electronic payment service can be used,
the image captured by the imaging unit includes a characteristic of the logo of the electronic payment service,
The computer of the terminal device
and identifying service information, which is information on a store where the electronic payment service is available and which corresponds to the characteristic information included in the image, by referring to the correspondence information.
2. The application program according to claim 1. The correspondence information includes correspondence between characteristic information of the logo of the electronic payment service, service information which is information of a store where the electronic payment service can be used, and location information,
the image captured by the imaging unit includes a characteristic of the logo of the electronic payment service,
The computer of the terminal device
and identifying service information, which is information on a store where the electronic payment service is available and which corresponds to the feature information and the location information included in the image, by referring to the correspondence information.
3. The application program according to claim 2. The correspondence information corresponds characteristic information of the logo of a store where the electronic payment service can be used with service information which is information about the store;
The image captured by the imaging unit includes a feature of the store's logo,
The computer of the terminal device
and identifying service information, which is information on a store where the electronic payment service is available and which corresponds to the characteristic information included in the image, by referring to the correspondence information.
2. The application program according to claim 1. The correspondence information includes correspondence between characteristic information of the logo of a store where the electronic payment service is available, service information which is information about the store, and location information;
The image captured by the imaging unit includes a feature of the store's logo,
The computer of the terminal device
and identifying service information, which is information on a store where the electronic payment service is available and which corresponds to the feature information included in the image and the location information, by referring to the feature information.
5. The application program according to claim 4. The service information displayed on the display unit of the terminal device is one or more of information on coupons available for use in electronic payment services, campaigns in electronic payment services, and information on electronic stamp cards available for use in electronic payment services.
6. The application program according to claim 1. The service information displayed on the display unit of the terminal device includes a reception unit for purchasing or reserving a product or service provided at a store where the electronic payment service is available, using electronic payment of the electronic payment service.
6. The application program according to claim 1. When the reception unit is operated,
The computer of the terminal device
requesting a server device to process the electronic payment and order for the product or service;
displaying on a display unit the content relating to the electronic payment provided by the server device in response to the request;
8. The application program according to claim 7. the service information displayed on the display unit of the terminal device is determined based on the user's payment history of the electronic payment service;
6. The application program according to claim 1. If the image captured by the imaging unit includes a characteristic of the logo of the electronic payment service,
The computer of the terminal device
displaying, on a display unit of the terminal device, information on a store where the user has a high frequency of payments in the payment history of the electronic payment service, as the service information with priority;
10. The application program according to claim 9. If the image captured by the imaging unit includes a characteristic logo of a store where the electronic payment service is available,
The computer of the terminal device
displaying information on products that the user has paid for frequently in the payment history of the electronic payment service as the service information on a display unit of the terminal device;
10. The application program according to claim 9. An application program according to any one of claims 1 to 11;
a server device that provides the correspondence information to the application program;
A service providing system comprising: The terminal device computer
The image captured by the imaging unit is acquired,
Identifying service information corresponding to the feature information included in the image by referring to correspondence information stored in a storage unit in which the feature information included in the image is associated with service information related to an electronic payment service;
displaying the specified service information on a display unit of the terminal device in a state in which the specified service information is superimposed on the image captured by the imaging unit;
How to provide information.

Images