JP7496638B2 - Program, mobile terminal and system - Google Patents

Description

The present invention relates to a program, a mobile terminal, and a system .

There is a known system in which customers can register products themselves using a mobile device equipped with a camera (see, for example, Patent Document 1).

JP 2018-147252 A

However, because customers register products themselves using a mobile terminal, there is a risk of problems occurring, such as customers thinking they have scanned the barcode of a product but actually failing to do so, or customers maliciously pretending to scan the barcode and putting the product in their shopping cart.

The present invention was made in light of these circumstances, and its purpose is to provide technology that can prevent problems from occurring when customers register products themselves.

In order to solve the above-mentioned problems, a product registration system according to one aspect of the present invention includes an acquisition means capable of acquiring information for product registration, an identification means for identifying shopper behavior, and an output means capable of outputting information related to the shopper behavior identified by the identification means, and is characterized in that the output means outputs information related to the identified behavior when shopper behavior is identified by the identification means and information for product registration is not acquired by the acquisition means.

With the above configuration, information about shopper behavior is output as needed, preventing problems from occurring.

The product registration system may include an imaging means for imaging the actions of the shopper, and the identification means may identify the behavior of the shopper based on the imaging information of the imaging means.

According to the above configuration, the behavior of shoppers can be appropriately identified using the imaging means.

In the above-mentioned product registration system, the output means may transmit information regarding the behavior to a specified information processing device.

According to the above configuration, information regarding shopper behavior can be suitably processed in a specified information processing device.

The product registration system may include a display control means and may display notification information corresponding to the information related to the behavior output by the output means.

With the above configuration, it is possible to check notification information regarding the behavior of shoppers.

In the above-mentioned product registration system, the acquisition means may acquire information for product registration as a product that requires the intervention of a store clerk in registration, depending on the behavior of the shopper identified by the identification means.

With the above configuration, registration can be conveniently performed with the assistance of a store clerk.

To solve the above-mentioned problems, a product registration system according to another aspect of the present invention includes a recognition means capable of recognizing product identification information that identifies a product, and an output means capable of outputting information corresponding to the recognition result by the recognition means, the recognition means being capable of recognizing predetermined identification information different from the product identification information, and the output means outputting information corresponding to the recognition result when the recognition means is in a state where it does not recognize either the predetermined identification information or the product identification information.

With the above configuration, information according to the recognition results is output as needed, preventing problems from occurring.

To solve the above-mentioned problems, a product registration system according to another aspect of the present invention includes a recognition means capable of recognizing product identification information that identifies a product, and an output means capable of outputting information corresponding to the recognition result by the recognition means, the recognition means being capable of recognizing predetermined identification information different from the product identification information, and the output means outputting information corresponding to the recognition result based on the fact that the recognition means does not recognize the product identification information during a period in which the recognition means does not recognize the predetermined identification information.

With the above configuration, information according to the recognition results is output as needed, preventing problems from occurring.

To solve the above-mentioned problems, another aspect of the present invention is a program that causes a computer to function as a mobile terminal used by a shopper to register products, and causes the computer to function as a reading means capable of reading information for product registration, a specifying means for specifying the shopper's behavior, and an output means capable of outputting information related to the shopper's behavior specified by the specifying means, the output means being characterized in that, when the shopper's behavior is specified by the specifying means and the reading means is unable to read the information for product registration, the output means outputs information related to the specified behavior.

With the above configuration, information about shopper behavior is output as needed, preventing problems from occurring.

In this way, problems that arise when customers register products themselves can be prevented.

FIG. 1 is a conceptual diagram of a network for explaining a sales system. FIG. 2 is an explanatory diagram illustrating the state near the entrance of a store in which the sales system 1 is installed. FIG. 11 is an explanatory diagram illustrating setting of a mobile terminal in a shopping cart. FIG. 2 is an explanatory diagram for explaining a settlement device. FIG. 2 is a sequence diagram outlining the overall flow of product purchase through sales system 1. 10 is a flowchart illustrating an example of an operation of a mobile terminal in the sales system 1. 10 is a flowchart illustrating an example of an operation of a mobile terminal in the sales system 1. 10 is a flowchart illustrating an example of an operation of a mobile terminal in the sales system 1. 10 is a flowchart illustrating an example of an operation of a mobile terminal in the sales system 1. 10 is a flowchart illustrating an example of an operation of a mobile terminal in the sales system 1. 13 shows examples of displays on a display unit of a mobile terminal. 13 is a display example on a display unit of a mobile terminal. 13 is a display example on a display unit of a mobile terminal. 13 is a display example on a display unit of a mobile terminal. 13 is an example of a display on the display unit of a settlement device. 13 is an example of a display on the display unit of a settlement device.

1 is a conceptual diagram of a network for explaining a sales system 1. The sales system 1 shown in FIG. 1 includes a management device (e.g., a store controller) 10, a cloud server 20, a registration settlement device 30, a settlement device 40, a monitoring device 50 (e.g., a personal computer, etc.), and a mobile terminal 60 (e.g., a smartphone, etc.). Although one registration settlement device 30 is illustrated in FIG. 1, the number of registration settlement devices 30 in one store may be two or more. Although two settlement devices 40 are illustrated in FIG. 1, the number of settlement devices 40 in one store may be one or three or more. Although one monitoring device 50 is illustrated in FIG. 1, the number of monitoring devices 50 in one store may be two or more. Although one management device 10 is basically provided in one store, two or more management devices 10 may be provided in one store. The sales system 1 can be introduced in various stores (supermarkets, convenience stores, drug stores, home centers, home appliance retailers, etc.).

The sales system 1 shown in FIG. 1 provides various services, such as a service corresponding to the mode in which a store clerk registers an item using the registration and settlement device 30, and a service corresponding to the mode in which a customer himself registers an item using a mobile terminal 60 (hereinafter referred to as service A). Note that the part of the sales system 1 that provides service A may be referred to as a specific sales system. In other words, the specific sales system is a subsystem of the sales system 1, and provides service A.

The management device 10, the registration settlement device 30, the settlement device 40, and the monitoring device 50 are installed in the store and are connected to each other so that they can communicate with each other via a LAN 19 (which may be wired or wireless). The management device 10 can communicate with the cloud server 20.

The management device 10 is a device that mainly manages various devices within the LAN 19. For example, the management device 10 has a communication function with external devices, and receives product files (product masters) from external devices (e.g., a headquarters server not shown, which may be the cloud server 20) and supplies them to the registered settlement device 30, etc., and receives transaction information from the settlement device 40, etc. and transmits it to the external device (e.g., the cloud server 20, etc.).

The cloud server 20 is a server that provides various services. For example, the cloud server 20 provides service A (a service corresponding to the mode in which the customer himself/herself registers products using the mobile terminal 60). For example, the cloud server 20 manages various information related to service A (for example, transaction information (basket information), etc.).

The registration and settlement device 30 is a device that executes the registration process and the settlement process. The registration and settlement device 30 is used by a store clerk (the registration process and the settlement process are executed by the operation of the store clerk).

The settlement device 40 is a device that executes the settlement process. The settlement device 40 is basically used by the customer (the settlement process is executed by the customer's operation). The settlement device 40 is used at least when using service A.

The monitoring device 40 is a device that mainly monitors various devices within the LAN 19. The monitoring device 40 monitors the operating status of other devices (e.g., the registration settlement device 30, the settlement device 40, etc.), displays various information on the other devices, and outputs various commands to the other devices.

The mobile terminal 60 is operated by a customer. The mobile terminal 60 is used when using service A. Specifically, when using service A, the mobile terminal 60 is set in the holder 71 of the shopping cart 70 (see FIG. 3).

The portable terminal 60 has a communication function, an imaging function (camera), and a recognition function for recognizing various codes (barcodes, two-dimensional codes, etc.). For example, the portable terminal 60 recognizes barcodes attached to products. In other words, the portable terminal 60 reads product identification information (product codes). The portable terminal 60 also recognizes barcodes (markers 73, described below) in which specific information is encoded. In other words, the portable terminal 60 reads the specific information. The portable terminal 60 may also extract feature points in an image captured by the imaging function (an image acquired as a through image) and recognize objects (various codes).

Although it is assumed that the mobile terminals 60 are basically owned by each customer, they may be loaned by the store. The number of mobile terminals 60 (the number in operation) varies depending on the number of customers, etc., and in FIG. 1, two mobile terminals 60 are shown as an example to show that multiple units are possible.

The mobile terminal 60 displays various screens related to service A. For example, an application program for using service A (hereinafter also referred to as a specific app) may be installed on the mobile terminal 60, and the mobile terminal 60 may display various screens related to service A.

In addition, part or all of the sales system 1 (or a specific sales system) may be referred to by other names (e.g., a product registration system, a settlement system, a product sales data processing system, etc.).

In the above, the cloud server 20, settlement device 40, and mobile terminal 60 are described as being used for service A, but the management device 10 is also used for service A (for example, used to send and receive information between the settlement device 40 and the cloud server 20). Note that other devices (such as the registered settlement device 30 and monitoring device 50) may also be used for service A.

Figure 2 is an explanatory diagram that explains the state near the entrance of a store that has introduced sales system 1 (specifically, a specific sales system). As shown in Figure 2, near the entrance (or the entrance if the entrance and exit are separate) there are a rental shopping cart 70, a rental shopping basket 80, and a sales shopping bag 90. When a customer visits the store, they place the shopping basket 80 in the shopping cart 70, place the shopping bag 90 in the shopping basket 80, etc., and proceed inside the store, but when using service A provided by the specific sales system, one of the preparations the customer makes is to place a mobile terminal 60 in the shopping cart 70 (details will be described later).

The customer may bring the shopping bag 90 that he or she purchased this time as his or her own bag the next time he or she visits the store, or may bring a bag other than the shopping bag 90 as his or her own bag. In addition, the customer may purchase a plastic bag.

Although not shown in FIG. 2, the shopping cart 70 is also provided with a reserved basket (see FIG. 14) for storing reserved items (described below) in addition to the shopping basket 80.

Figure 3 is an explanatory diagram for explaining how to set the mobile terminal 60 in the shopping cart 70. Figure 3(A) is a schematic diagram of the upper part of the shopping cart 70 on the front side in the direction of travel, as viewed from the side. Figure 3(B) is a schematic diagram of Figure 3(A) as viewed from above.

As shown in FIG. 3A, the shopping cart 70 includes a holder 71 and an arm 72. A marker 73 is attached to the tip of the arm 72. When the shopping cart 70 is properly set in the holder 71, the mobile terminal 60 can recognize the marker 73 as an object in the captured image (through image) as shown in FIG. 3B. More specifically, when the mobile terminal 60 is properly set in the holder 71, if there is nothing blocking the marker 73 in front of the imaging unit of the mobile terminal 60, the mobile terminal 60 will recognize the marker 73, but if there is something blocking the marker 73 in front of the imaging unit of the mobile terminal 60, the mobile terminal 60 will not recognize the marker 73. Note that in the case of FIG. 3B, the arm 72 should also be captured in the image as shown in FIG. 12A, which will be described later, but this is omitted.

Figure 4 is an explanatory diagram explaining the settlement device 40. Figure 4 (A) is a diagram showing an example of the appearance of the settlement device 40 (oblique view seen from the customer side). Figure 4 (B) is a diagram showing an example of the configuration of the settlement device 40. In Figures 4 (A) and 4 (B), the same parts are given the same reference numerals. Note that the settlement device 40 is basically operated by the customer.

Below, an example of the configuration of the settlement device 40 shown in FIG. 4(B) will be described with reference to FIG. 4(A). The settlement device 40 comprises a CPU 401, a ROM 402, a RAM 403, a hard disk 404, a display unit 405, a scanner unit 406, a card payment unit 408, a change dispenser 409, a printing unit 413, an audio output unit 414, and a communication unit 415. These can communicate with each other via a bus. The settlement device 40 may further comprise a key operation unit, an imaging unit (camera), and an alarm unit (a lamp on a sign pole, etc.).

The CPU 401 is a central processing unit, and controls the operation of the settlement device 40 by reading and executing programs stored in the ROM 402 .
The ROM 402 is a read-only memory, and stores various types of information used by the CPU 401, including programs.

RAM 403 is a read/write memory that stores various information. For example, RAM 403 stores information read from ROM 402 or hard disk 404, information acquired from the outside, information generated during processing, etc.

The hard disk 404 stores various information. For example, instead of the ROM 402, the hard disk 404 may store programs executed by the CPU 401. Also, instead of the RAM 403, the hard disk 404 may store information read from the ROM 402, information obtained from the outside, information generated during processing, etc.

The display unit 405 is a touch display that displays various information and accepts various inputs.

The scanner unit 406 optically reads various information. For example, the scanner unit 406 scans a code (such as a two-dimensional code) displayed on the display unit of the mobile terminal 60 and reads the information required for the settlement process. The scanner unit 406 may also scan a code (such as a barcode, two-dimensional code, etc.) printed on a receipt (registered trademark) emitted by another device (such as the registered settlement device 30) and read the information required for the settlement process. The scanner unit 406 may also scan a barcode attached to a product and read the product code. The scanner unit 406 may also scan a barcode attached to a store clerk's name tag and read the store clerk code.

The card payment unit 408 is a payment mechanism that uses various cards (credit cards, prepaid cards such as transportation cards, cash cards for debit payments, point cards for point payments, etc.). The card payment unit 408 is equipped with a card recognition unit (reading unit), a display unit, and an operation unit, but it is sufficient that it is equipped with at least the card recognition unit. The card recognition unit may also recognize various cards (e.g., point cards, membership cards, shareholder cards, etc.) for the purpose of granting benefits, discounts, etc.

The change dispenser 409 (cash settlement unit) is a mechanism for settlement using cash, and has an input port for banknotes and coins, and an output port for banknotes and coins. It calculates the amount inserted into the input port, calculates the change amount, which is the difference between the input amount and the purchase amount, and outputs the change from the output port. When banknotes or coins are inserted into the input port, they are detected by a sensor (detecting that they have been inserted, detecting the number of coins by denomination, etc.).

The printing unit 413 prints and issues various media (receipts, bills, etc.).
The audio output unit 414 outputs audio. For example, the audio output unit 414 outputs audio guidance and the like.
The communication unit 415 transmits and receives information to and from other devices (the management device 10, the registered settlement device 30, the other settlement devices 40, the monitoring device 50, etc.).

Note that while the appearance and hardware configuration of the settlement device 40 have been described above, a description of the appearance and hardware configuration of the registration settlement device 30, etc. will be omitted. Note that, as described in FIG. 1, the registration settlement device 30 is basically used by a store clerk (the registration process and settlement process are carried out by operation of the store clerk).

The registration and settlement device 30 may be configured to have a display unit and a scanner unit on each side (clerk side, customer side), and a settlement unit (card settlement unit, cash settlement unit) on one side (customer side). The registration and settlement device 30 in this configuration may be a device that can be switched to any of a plurality of operation modes (a mode in which the registration process is performed by the operation of the clerk and the settlement process is performed by the operation of the customer, a mode in which the registration process is performed by the operation of the clerk and the settlement process is performed by the operation of the customer, etc.) depending on, for example, the operating status. Also, if the settlement process is performed by the operation of the customer, the registration and settlement device 30 can be used for service A. That is, the customer can perform the registration process on the mobile terminal 60 and perform the settlement process on the registration and settlement device 30.

Figure 5 is a sequence diagram outlining the overall flow of purchasing a product through the sales system 1 (specifically, service A provided by a specific sales system). Specifically, Figure 5 shows an example of the flow of processing in each of the mobile terminal 60, the settlement device 40 installed in the store, and the cloud server 20 installed outside, such as a data center, from when a customer (a customer possessing a mobile terminal 60 with a specific application installed) visits a store (a store where service A is running), registers a product displayed in the store using the mobile terminal 60, and completes payment for the registered product. The left side of Figure 5 shows the processing of the mobile terminal 60, the center shows the processing of the settlement device 40, and the right side shows the processing of the cloud server 20. Note that in Figure 5, it is assumed that product registration using the mobile terminal 60 is always performed normally (the product code is successfully read in step S5).

Step S1: The mobile terminal 60 acquires information that identifies the store (store identification information). For example, a two-dimensional code for identifying the store may be displayed near the entrance of the store (the two-dimensional code may be output on a display screen, a medium with the two-dimensional code printed on it may be attached, etc.), and a customer who visits the store may scan (read) the two-dimensional code with the mobile terminal 60, thereby causing the mobile terminal 60 to acquire the store identification information. Note that when a customer who visits the store starts a specific app, a screen instructing the customer to scan the two-dimensional code may be displayed as the initial screen, or when a customer who visits the store scans the two-dimensional code with the mobile terminal 60, the specific app may be started and a screen informing the customer that they are connected to the cloud server 20 may be displayed as the initial screen.

For example, since a store is identified by its location, a customer who visits the store may obtain location information (GPS information) on the mobile terminal 60 at the store (i.e., the store's location information may be obtained as store identification information). When a customer who visits the store starts a specific app, location information may be obtained and a screen may be displayed as the initial screen informing the customer that a connection to the cloud server 20 is in progress. If multiple stores are detected from the location information and it is not possible to identify one, a selection screen may be displayed to allow the customer to select. Alternatively, a mode may be switched to forcefully obtain a two-dimensional code.

The mobile terminal 60 that has acquired the store-specific information transmits the acquired store-specific information together with the customer identification information to the cloud server 20 as a transaction start request. Note that the customer identification information is stored in the customer information of the cloud server 20 based on the information entered in the name field of the registration form using the mobile terminal 60 when registering the customer (e.g., when installing a specific app on the mobile terminal 60), but may also be stored in the storage unit of the mobile terminal 60 in addition to the cloud server 20.

The destination information (the address of the cloud server 20) may also be obtained when the customer is registered and stored in the memory of the mobile terminal 60. When scanning a two-dimensional code, the destination information may be converted into a two-dimensional code in addition to the store-specific information, and the mobile terminal 60 may obtain the destination information along with the store-specific information by scanning the two-dimensional code.

Step S2: The cloud server 20, which has received the customer identification information and the store identification information as a transaction start request from the mobile terminal 60, generates a basket (basket information) for the transaction. The basket information may include, for example, basket identification information, transaction start date and time, transaction end date and time, customer identification information, registered product information, pending product information, cancellation information, etc. The basket identification information is identification information that uniquely identifies the basket. For example, the basket identification information may be composed of store identification information, a date, and a serial number (for example, a serial number for each store and date). The transaction start date and time is the start date and time of the transaction (for example, the creation date and time of the basket information). The transaction start date and time may be the registration date and time of one product item. The creation date and time of the basket information and the registration date and time of one product item may both be stored separately. When the basket information is generated, the basket identification information, transaction start date and time, and customer identification information are generated, but since it is before the start of the actual transaction (product registration), other information is not generated.

Step S3: The cloud server 20 that generated the basket for the transaction generates screen information and transmits the generated screen information together with the basket identification information to the mobile terminal 60.

Step S4: The mobile terminal 60 receives the basket identification information and screen information from the cloud server 20, stores the basket identification information, and displays the registration screen on the display unit.

Step S5: The customer's action causes the mobile terminal 60 to scan the barcode attached to the product and read the product code. After reading the product code, the mobile terminal 60 transmits the read product code together with the basket identification information to the cloud server 20.

Step S6: The cloud server 20 receives the basket identification information and the product code from the mobile terminal 60 and identifies the basket of the transaction from the basket identification information.

Step S7: The cloud server 20 updates the product data in the identified basket. Specifically, when the cloud server 20 receives a product code as the Nth item, it stores the product code as the product code of the registered product information (registered product N) in the basket information of the identified basket, obtains the product name and price corresponding to the product code from the product information, and stores them as the product and price of the registered product information (registered product N). The cloud server 20 also updates the registered product information (total) in the basket information of the identified basket.

Step S8: After updating the product data in the basket, the cloud server 20 generates screen information after the product is registered (screen information corresponding to the updated product data) and transmits the generated screen information to the mobile terminal 60 together with the basket identification information.

Step S9: The mobile terminal 60 receives the basket identification information and screen information from the cloud server 20, and after confirming whether the basket identification information matches the stored basket identification information, displays a registration screen based on the screen information on the display unit. In other words, the product is added to the registration screen of the mobile terminal 60.

Note that while FIG. 5 has simplified the process for ease of explanation, steps S5 to S9 are repeated each time a product code is read in step S until a checkout instruction is received in step S10 below.

Step S10: The mobile terminal 60 accepts accounting instructions as operations by the customer.

Step S11: The mobile terminal 60 that has received the accounting instruction generates a two-dimensional code. In other words, the mobile terminal 60 converts information required to execute the settlement process for the items purchased by the mobile terminal 60 (e.g., basket identification information) into a two-dimensional code. The mobile terminal 60 that has generated the two-dimensional code displays the generated two-dimensional code on the display unit.

Step S12: The settlement device 40 scans (reads) the two-dimensional code displayed on the display unit of the mobile terminal 60. For example, the settlement device 40 scans the two-dimensional code displayed on the display unit of the mobile terminal 60 that is pointed by the customer within the recognition range of the scanner unit 406.

Step S13: The settlement device 40 that reads the two-dimensional code requests the cloud server 20 to calculate the subtotal amount. For example, the settlement device 40 sends a calculation request (subtotal calculation request information) requesting calculation of the subtotal amount, together with the basket identification information obtained from the two-dimensional code, to the cloud server 20 via the management device 10.

Step S14: The cloud server 20 receives the basket identification information and the sub-calculation output request information from the mobile terminal 60 and identifies the basket of the transaction from the basket identification information.

Step S15: The cloud server 20, having identified the basket, calculates the subtotal amount of the identified basket.
Step S16: The cloud server 20, which has calculated the subtotal amount, updates the basket information (stores the subtotal amount (calculated subtotal amount)) and transmits the subtotal information indicating the calculated subtotal amount together with the basket identification information to the settlement device 40 via the management device 10.

Step S17: Having received the basket identification information and subtotal information from the cloud server 20, the settlement device 40 displays the subtotal amount on the display unit 405.

Step S18: The settlement device 40, which has displayed the subtotal amount on the display unit 405, executes the payment (settlement). Specifically, the settlement device 40 accepts the selection of the payment type. In the case of cash, it accepts the deposit, calculates the change amount, and, if there is change, dispenses the change and issues a receipt. In addition, when the settlement is completed, the settlement device 40 sends settlement completion information together with the basket information to the cloud server 20 via the management device 10, and the cloud server 20 stores the transaction end date and time for that basket (settlement date and time).

Figure 6 is a flowchart explaining an example of the operation of the mobile terminal 60 by the sales system 1 (specifically, service A provided by a specific sales system). Specifically, Figure 6 shows an example of the operation of the mobile terminal 60 when a customer (a customer who has a mobile terminal 60 with a specific application installed) visits a store (a store where service A is in operation), correctly places the mobile terminal 60 in the shopping cart 70, which is the preparation stage before purchasing a product, and receives an instruction to start registering the product.

Step S101: The mobile terminal 60 determines whether or not the marker 73 (a barcode in which specific information is encoded) is recognized. Note that, during the preparation stage before purchasing a product, if the mobile terminal 60 is correctly set in the holder 71, it will recognize the marker 73 and enter a recognized state. If the marker 73 is not recognized (step S101: NO), proceed to step S102. If the marker 73 is recognized (step S101: YES), proceed to step S103.

Step S102: The mobile terminal 60 displays information (message) on the display unit that prompts the customer to correct the set position of the mobile terminal 60. Then, the process returns to step S101. In other words, the above information is displayed on the display unit until the set position of the mobile terminal 60 is corrected and the marker 73 is recognized. Note that the mobile terminal 60 may display information (message) on the display unit that notifies the customer that the set position of the mobile terminal 60 is incorrect, instead of or in addition to the above information.

Step S103: The mobile terminal 60 displays information (message) on the display unit that prompts the user to start shopping (product registration). Then, the process proceeds to step S104.

Step S104: The mobile terminal 60 determines whether or not an instruction to start shopping has been received. If an instruction to start shopping has not been received (step S104: NO), the process returns to step S103. In other words, the above information is displayed on the display unit until an instruction to start shopping is received. If an instruction to start shopping has been received (step S104: YES), the process proceeds to step S105.

Step S105: The mobile terminal 60 sets the shopping flag (for example, stores a value of 1). Then, the flowchart in FIG. 6 ends. The shopping flag is set during the period between the start of shopping (S104 in FIG. 6: YES) and the end of product registration (Step S290 in FIG. 7).

The process in FIG. 6 (steps S101 to S105) is performed before the process in step S4 in FIG. 5 (displaying the registration screen on the display unit). In addition, the process in steps S1 to S2 in FIG. 5 and the process in FIG. 6 can be performed in any order.

Figures 7 to 10 are flow charts for explaining an example of the operation of the mobile terminal 60 by the sales system 1 (specifically, service A provided by a specific sales system). Specifically, Figures 7 to 10 show an example of the operation of the mobile terminal 60 until the shopping flag is cleared. In other words, they show an example of the operation of the mobile terminal 60 when the preparation stage before the product purchase (see Figure 6) is completed and a product is registered using the mobile terminal 60 set in the shopping cart 70. In the explanation of Figures 7 to 10, it is assumed that the mobile terminal 60 is correctly set in the shopping cart 70. Furthermore, when the shopping flag is set, the mobile terminal 60 displays information on the display unit that guides the user on how to register a product before registering the first product, and after registering the first product, displays a button on the display unit that accepts an instruction to proceed to checkout (i.e., to end product registration).

Step S201 in FIG. 7: The mobile terminal 60 determines whether or not the marker 73 is recognized. As described above, in the explanations of FIG. 7 to FIG. 10, it is assumed that the mobile terminal 60 is correctly set in the holder 71, so the recognition state of the marker 73 is maintained if there is nothing blocking the marker 73 in front of the imaging unit of the mobile terminal 60. If the marker 73 is recognized (step S201: YES), proceed to step S290. If the marker 73 is not recognized (step S201: NO), proceed to step S210.

Step S210: The mobile terminal 60 determines whether or not the recognition of the product code was successful. When the customer brings the product into the space between the imaging unit of the mobile terminal 60 and the marker 73, the mobile terminal 60 becomes able to read the barcode attached to the product. If the recognition of the product code is successful (step S210: YES), proceed to step S225. If the recognition of the product code is not successful (step S210: NO), proceed to step S340 in FIG. 8.

Cases where product code recognition is unsuccessful include cases where product code recognition fails and cases where even something resembling a barcode attached to the product is not recognized. When the mobile terminal 60 recognizes something resembling a barcode (an object having a striped pattern resembling a barcode) but is unable to correctly recognize it as a barcode (when product code recognition is unsuccessful), it is determined that product code recognition has failed. Cases where product code recognition is unsuccessful include cases where the customer brings the product (barcode) into the space between the imaging unit of the mobile terminal 60 and the marker 73, but the barcode cannot be recognized due to its position, angle, dirt, etc. Cases where even something resembling a barcode attached to the product is not recognized include cases where the customer does not bring the product (barcode) into the space between the imaging unit of the mobile terminal 60 and the marker 73.

Step S225: The mobile terminal 60 determines whether or not the registration failure flag set (described later) is set. If the registration failure flag set is set (step S225: YES), the process proceeds to step S226. If the registration failure flag set is not set (step S225: NO), the process proceeds to step S230.

Step S226: The mobile terminal 60 clears the registration failure flag. Then, the process proceeds to step S230.

Step S230: The mobile terminal 60 transmits the product code, etc. to the cloud server 20. The process then proceeds to step S231. Note that the process of step S230 corresponds to the process following the process of step S5 in FIG. 5 (the process of reading the product code) (the process of transmitting the read product code together with the basket identification information to the cloud server 20).

Step S231: The mobile terminal 60 displays the product. Then, the process proceeds to step S290. Note that the process of step S231 corresponds to the process of step S9 in FIG. 5 (the process of adding the product to the registration screen).

Step S290: The mobile device 60 determines whether an instruction to proceed to checkout has been accepted. That is, the mobile device 60 determines whether a button has been operated to accept an instruction to proceed to checkout (i.e., to end product registration). If an instruction to proceed to checkout has been accepted (Step S290: YES), the process proceeds to Step S292. If an instruction to proceed to checkout has not been accepted (Step S290: NO), the process returns to Step S201. Note that the process of Step S10 in FIG. 5 (the process of accepting a checkout instruction as an operation by the customer) corresponds to this Step S290 (YES).

Step S292: The mobile terminal 60 clears the shopping flag. After that, the mobile terminal 60 generates and displays a two-dimensional code (see step S11 in FIG. 5), but the explanation is omitted.

Step S340 in FIG. 8: The mobile terminal 60 determines whether or not the recognition of the product code has failed. In other words, the mobile terminal 60 determines whether or not it has recognized something resembling a barcode (an object having a striped pattern similar to a barcode). If the recognition of the product code has failed (step S340: YES), proceed to step S342. If the recognition of the product code has not failed (step S340: NO), proceed to step S350.

Step S340 (YES) occurs when the product code cannot be recognized (step S210 (NO) in FIG. 7), but something that looks like a barcode can be recognized. In other words, something that looks like a barcode can be recognized, but the product code cannot be recognized. For example, if a customer is trying to register a product but is unable to register it, step S340 (YES) occurs. On the other hand, step S340 (NO) occurs when the product code cannot be recognized (step S210 (NO) in FIG. 7), and something that looks like a barcode cannot be recognized. In other words, the product code cannot be recognized, let alone something that looks like a barcode. For example, if the customer is pretending to register, step S340 (NO) occurs.

Step S342: The portable terminal 60 sets a registration failure flag indicating that recognition of the product code has failed, and then proceeds to step S347.
Step S347: The portable terminal 60 displays on the display unit information (message) informing the customer that registration has failed and information (message) urging the customer to try reading again, and then proceeds to step S350.

Step S350: The mobile device 60 determines whether the unrecognized state of the marker 73 is equal to or longer than a second threshold (e.g., 10 seconds). Note that the second threshold is greater than the first threshold in step S352 described below. If the unrecognized state of the marker 73 is equal to the second threshold (step S350: YES), the process proceeds to step S375. If the unrecognized state of the marker 73 is not equal to or longer than the second threshold (step S350: NO), the process proceeds to step S351.

Step S351: The mobile terminal 60 determines whether or not the marker 73 is in a recognized state. That is, the mobile terminal 60 determines whether or not the marker 73 returned to a recognized state before the unrecognized state of the marker 73 reached the second threshold. If the marker 73 is in a recognized state (step S351: YES), proceed to step S352. If the marker 73 is not in a recognized state (step S351: NO), return to step S210 in FIG. 7.

Step S352: The mobile device 60 determines whether the unrecognized state of the marker 73 is equal to or greater than a first threshold (e.g., 5 seconds). Note that this first threshold is smaller than the second threshold of step S350. If the unrecognized state of the marker 73 is equal to or greater than the first threshold (step S352: YES), the process proceeds to step S355. If the unrecognized state of the marker 73 is not equal to or greater than the first threshold (step S352: NO), the process proceeds to step S365.

Step S355: The mobile terminal 60 determines whether or not the registration failure flag is set. If the registration failure flag is set (step S355: YES), the process proceeds to step S367. If the registration failure flag is not set (step S355: NO), the process proceeds to step S356.

Step S356: The mobile terminal 60 counts up the number of times the marker was not recognized correctly. Then, the process returns to step S290 in FIG. 7. Note that step S355 (NO) occurs not because of the act of registering a product, but because it assumes a situation in which recognition of the marker 37 is temporarily lost (for a relatively short period of time, less than the first threshold), and therefore the number of times the marker 37 was not recognized correctly is simply counted. The number of times the marker was not recognized correctly is initialized for each transaction, but may be saved as historical information for the customer or added as a cumulative number before being initialized for each transaction.

Step S365: The mobile terminal 60 determines whether the registration failure flag is set. If the registration failure flag is set (step S365: YES), the process proceeds to step S367. If the registration failure flag is not set (step S365: NO), the process proceeds to step S368.

Step S367: The mobile terminal 60 displays on the display unit information (message) informing the customer that registration has failed and information (message) urging the customer to scan again. Then, proceed to step S400 in FIG. 9.

Step S368: The mobile terminal 60 displays on the display unit information (message) informing the customer that the correct operation (the operation of reading the barcode attached to the product in order to register the product) was not performed, and information (message) urging the customer to perform the correct operation. Then, proceed to step S500 in FIG. 10.

Step S375: The mobile terminal 60 determines whether the registration failure flag is set. If the registration failure flag is set (step S375: YES), the process proceeds to step S377. If the registration failure flag is not set (step S375: NO), the process proceeds to step S378.

Step S377: The mobile terminal 60 displays on the display unit information (message) informing the customer that registration has not yet been completed and information (message) urging the customer to read the card. Then, the process proceeds to step S400 in FIG. 9.

Step S378: The mobile terminal 60 displays on the display unit information (message) informing the customer that the correct operation has not yet been performed and information (message) urging the customer to read. Then, proceed to step S500 in FIG. 10.

Step S400 in FIG. 9: The mobile terminal 60 determines whether a predetermined time (e.g., 10 seconds) has elapsed. The predetermined time in step S400 may be different from the first threshold value and the second threshold value in FIG. 8. If the predetermined time has elapsed (step S400: YES), proceed to step S468. If the predetermined time has not elapsed (step S400: NO), proceed to step S410.

The elapsed time in step S400 may basically be based on the display in step S367 (or step S377) in Fig. 8. For example, it may be any of the following.
(a) In cases where the display such as in step S367 of FIG. 8 does not interfere with reading the product (i.e., when a display such as in step S367 of FIG. 8 is provided that does not interfere with reading the product), the elapsed time from the start of the display such as in step S367 of FIG. 8 is compared with the above-mentioned specified time.
(a) If the display in step S367 in Fig. 8 etc. interferes with the reading of the commodity (i.e., if a display that may interfere with the reading of the commodity is displayed in step S367 in Fig. 8 etc.), the time that has elapsed since the display in step S367 in Fig. 8 etc. was deleted is compared with the above-mentioned predetermined time. Note that the display in step S367 in Fig. 8 etc. may be deleted automatically or by the customer's operation.

Step S410: The mobile terminal 60 determines whether or not the recognition of the product code was successful. If the recognition of the product code was successful (step S410: YES), the process proceeds to step S426. If the recognition of the product code was not successful (step S410: NO), the process returns to step S400.

Step S426: The mobile terminal 60 clears the registration failure flag. Then, the process proceeds to step S430.

Step S430: The mobile terminal 60 transmits the product code, etc. to the cloud server 20. The process then proceeds to step S431. Note that the process of step S430 corresponds to the process subsequent to the process of step S5 in FIG. 5 (the process of reading the product code) (the process of transmitting the read product code together with the basket identification information to the cloud server 20).

Step S431: The mobile terminal 60 displays the product. Then, the process returns to step S290 in FIG. 7. Note that the process of step S431 corresponds to the process of step S9 in FIG. 5 (the process of adding the product to the registration screen).

Step S468: The mobile terminal 60 displays on the display unit information (message) informing the customer that registration has failed and information (message) prompting the customer to confirm the product purchase (confirmation operation). Then, proceed to step S480.

Step S480: The mobile terminal 60 determines whether or not an instruction to purchase a product has been received. If an instruction to purchase a product has not been received (step S480: NO), the process proceeds to step S481. If an instruction to purchase a product has been received (step S480: YES), the process proceeds to step S484.

Step S481: The mobile terminal 60 determines whether or not an instruction to not purchase the product has been received. If an instruction to not purchase the product has not been received (step S481: NO), the mobile terminal 60 returns to step S480. If an instruction to not purchase the product has been received (step S481: YES), the mobile terminal 60 proceeds to step S482.

Step S482: The mobile terminal 60 transmits pending information (1) to the cloud server 20. The pending information (1) includes information that can identify that the registration of the product has failed and information that can identify that the purchase of the product has been canceled. Then, proceed to step S483.

Step S483: The mobile terminal 60 displays on the display unit information (message) informing the customer that an instruction not to purchase the product has been accepted, and information (message) informing the customer that the product will be separated as a reserved product. A reserved product is a product that has not been successfully registered, and should be placed in a reserved basket (see FIG. 14) so that it can be easily distinguished from successfully registered products. In addition to the above information, step S483 may also display information (message) informing the customer that registration of the product has failed. The process then proceeds to step S486.

Step S484: The mobile terminal 60 transmits pending information (2) to the cloud server 20. The pending information (2) includes information that can identify that the registration of the product has failed and information that can identify that the product is planned to be purchased. Then, proceed to step S485.

Step S485: The mobile terminal 60 displays on the display unit information (message) informing the customer that the instruction to purchase the product has been accepted, and information (message) informing the customer that the product will be placed on hold. In addition to the above information, in step S485, information (message) may be displayed informing the customer that registration of the product has failed. The process then proceeds to step S486.

Step S486: The mobile terminal 60 sets the registration failure flag. Then, the process returns to step S290 in FIG. 7.

Step S500 in FIG. 10: The mobile terminal 60 determines whether a predetermined time (e.g., 10 seconds) has elapsed. The predetermined time in step S500 may be different from the first threshold value and the second threshold value in FIG. 8. The predetermined time in step S500 may also be different from the predetermined time in step S400 in FIG. 9. If the predetermined time has elapsed (step S500: YES), proceed to step S568. If the predetermined time has not elapsed (step S500: NO), proceed to step S510.

The elapsed time in step S500 may basically be based on the display in step S368 (or step S378) in Fig. 8. For example, it may be any of the following.
(a) In cases where the display such as in step S368 of FIG. 8 does not interfere with reading the product (i.e., when a display such as in step S368 of FIG. 8 is provided that does not interfere with reading the product), the elapsed time from the start of the display such as in step S368 of FIG. 8 is compared with the above-mentioned specified time.
(a) In the case where the display in step S368 in Fig. 8 etc. interferes with the reading of the commodity (i.e., when a display that may interfere with the reading of the commodity is displayed in step S368 in Fig. 8 etc.), the time that has elapsed since the display in step S368 in Fig. 8 etc. was deleted is compared with the above-mentioned predetermined time. Note that the display in step S368 in Fig. 8 etc. may be deleted automatically or by the customer's operation.

Step S510: The mobile terminal 60 determines whether or not the product code has been successfully recognized. If the product code has been successfully recognized (Step S510: YES), the process proceeds to Step S526. If the product code has not been successfully recognized (Step S510: NO), the process proceeds to Step S540.

Step S526: The mobile terminal 60 clears the registration failure flag. Then, the process proceeds to step S530.

Step S530: The mobile terminal 60 transmits the product code, etc. to the cloud server 20. The process then proceeds to step S531. Note that the process of step S530 corresponds to the process following the process of step S5 in FIG. 5 (the process of reading the product code) (the process of transmitting the read product code together with the basket identification information to the cloud server 20).

Step S531: The mobile terminal 60 displays the product. Then, the process returns to step S290 in FIG. 7. Note that the process of step S531 corresponds to the process of step S9 in FIG. 5 (the process of adding the product to the registration screen).

Step S540: The mobile terminal 60 determines whether or not the recognition of the product code has failed. If the recognition of the product code has failed (step S540: YES), the process proceeds to step S542. If the recognition of the product code has not failed (step S540: NO), the process returns to step S500.

Step S542: The portable terminal 60 sets a registration failure flag indicating that recognition of the product code has failed, and then proceeds to step S547.
Step S547: The portable terminal 60 displays on the display unit information (message) informing the customer that registration has failed and information (message) urging the customer to try reading again. Then, the process proceeds to step S400 in FIG.

Step S568: The mobile terminal 60 displays on the display unit information (message) informing the customer that the correct operation was not performed and information (message) prompting the customer to confirm the product purchase (confirmation operation). Then, proceed to step S580.

Step S580: The mobile terminal 60 determines whether or not an instruction to purchase a product has been received. If an instruction to purchase a product has not been received (step S580: NO), the mobile terminal 60 proceeds to step S581. If an instruction to purchase a product has been received (step S580: YES), the mobile terminal 60 proceeds to step S584.

Step S581: The mobile terminal 60 determines whether or not an instruction to not purchase the product has been received. If an instruction to not purchase the product has not been received (step S581: NO), the mobile terminal 60 returns to step S580. If an instruction to not purchase the product has been received (step S581: YES), the mobile terminal 60 proceeds to step S582.

Step S582: The mobile terminal 60 transmits hold information (3) to the cloud server 20. The hold information (3) includes information that can identify that a correct operation has not been performed and information that can identify that the purchase of the product has been canceled. Then, proceed to step S583.

Step S583: The mobile terminal 60 displays on the display unit information (message) informing the customer that an instruction not to purchase the product has been accepted, and information (message) informing the customer that the product will be placed on hold. In addition to the above information, in step S583, information (message) may be displayed informing the customer that registration of the product has failed. Then, the process returns to step S290 in FIG. 7.

Step S584: The mobile terminal 60 transmits hold information (4) to the cloud server 20. The hold information (4) includes information that can identify that a correct operation has not been performed and information that can identify that a product is planned to be purchased. Then, proceed to step S485.

Step S585: The mobile terminal 60 displays on the display unit information (message) informing the customer that the instruction to purchase the product has been accepted, and information (message) informing the customer that the product will be placed on hold. In addition to the above information, in step S585, information (message) may be displayed informing the customer that registration of the product has failed. Next, the process returns to step S290 in FIG. 7.

Step S585: The mobile terminal 60 displays on the display unit information (message) informing the customer that the instruction to purchase the product has been accepted, and information (message) informing the customer that the product will be placed on hold. In addition to the above information, in step S585, information (message) may be displayed informing the customer that registration of the product has failed. Next, the process returns to step S290 in FIG. 7.

According to the above flowcharts of FIG. 7 to FIG. 10, in the case where the reading of the product is successful in one attempt, the process proceeds from step S210 (YES) in FIG. 7 to step S230 in FIG. 7 to step S231 in FIG. 7. On the other hand, since there are many cases where the reading of the product is not successful in one attempt, some patterns will be explained. For example, if the reading fails and the product is immediately (below the first threshold) placed in the shopping cart 80, the process proceeds from step S352 (YES) in FIG. 8 to step S355 (YES) in FIG. 8 to step S367 in FIG. 8 (continued to step S400 in FIG. 9). Also, in the case where the reading fails and the product is placed in the shopping cart 80 relatively early (between the first threshold and the second threshold), the process proceeds from step S352 (NO) in FIG. 8 to step S365 (YES) in FIG. 8 to step S367 in FIG. 8 (continued to step S400 in FIG. 9). Also, if the reading does not even fail for a long time (greater than the second threshold), the process proceeds from step S350 (YES) in FIG. 8 to step S375 (NO) in FIG. 8 to step S378 in FIG. 8 (continues to step S500 in FIG. 10).

Although an example of the operation of the mobile terminal 60 by the sales system 1 has been described above using Figures 6 to 10, the operation of the mobile terminal 60 is not limited to this.

As an example, in the examples of Figures 7 to 10, the failure is notified several times (for example, step S347 in Figure 8, step S367 in Figure 8, step S468 in Figure 9, step S547 in Figure 10), but some notifications may be omitted. For example, in the examples of Figures 7 to 10, if the mobile terminal 60 is unsuccessful in recognizing the product code, the mobile terminal notifies the customer on the display (for example, steps S367, S368, S377, S378 in Figure 8) and gives a predetermined time to recognize the product code (for example, step S400 in Figure 9, step S500 in Figure 10), but the predetermined time does not have to be given. For example, the process may proceed from step S355 (YES) in Figure 8 or step S365 (YES) in Figure 8 to step S468 in Figure 9 (step S367 in Figure 8 is not executed). Similarly, step S375 (YES) in FIG. 8 may proceed to step S468 in FIG. 9 (step S377 in FIG. 8 is not executed). Also, step S365 (NO) in FIG. 8 may proceed to step S568 in FIG. 10 (step S368 in FIG. 8 is not executed). Similarly, step S375 (NO) in FIG. 8 may proceed to step S568 in FIG. 10 (step S378 in FIG. 8 is not executed).

Figures 11 to 14 show examples of displays on the display unit of the mobile terminal 60. Figure 11 shows a screen that is displayed when a customer comes to the store (for example, at the entrance as shown in Figure 2). Figure 11 (A) shows a screen that prompts the customer to confirm whether or not they have placed a shopping bag (such as shopping bag 90).

Figure 11 (B) is a screen that is displayed when the marker 73 cannot be read. For example, the mobile terminal 60 may display a screen as shown in Figure 11 (B) in step S102 of Figure 6. Note that the screen of Figure 11 (B) is erased by touching the OK button and does not interfere with the reading of the marker 73, but the message "Please move the marker to the center..." may be displayed continuously in a position that does not interfere with the reading of the marker 73 (e.g., the top of the screen, etc.) until the marker 73 can be read. Also, instead of the screen of Figure 11 (B), a screen as shown in Figure 11 (D) may be displayed.

FIG. 11(C) is a screen for accepting an instruction to start shopping. For example, the mobile terminal 60 may display a screen as shown in FIG. 11(C) in step S103 of FIG. 6. The mobile terminal 60 may also set a shopping in progress flag when the OK button on the screen is touched (step S104 of FIG. 6).

Figures 12 (A) and 12 (B) are screens that are displayed during shopping (e.g., steps S4 to S9 in Figure 5). Figure 12 (A) is the screen when a product has not been read. Figure 12 (A) is the screen when a product is being read. Note that "XX Yogurt" in Figure 12 (B) is a product that was added in step S9 in Figure 5.

Figure 12 (C) is a screen displaying a two-dimensional code for settlement, which is displayed, for example, in step S11 of Figure 5. Figure 12 (D) shows how the settlement device 40 reads the two-dimensional code (step S12 of Figure 5). Note that while Figure 12 (D) is an illustration for the purpose of explaining how the settlement device 40 reads the two-dimensional code, the mobile terminal 60 may also display guidance information such as that shown in Figure 12 (D) for the customer.

Figures 13 and 14 are screens that are displayed when product registration is unsuccessful. Figure 13 (A) is a screen that displays information indicating that registration has failed and prompting the user to scan again. For example, the mobile terminal 60 may display a screen such as that shown in Figure 13 (A) in step S347 of Figure 8, step S367 of Figure 8, and step S547 of Figure 10. Figure 13 (B) is a screen that displays information indicating that the correct operation has not been performed and prompting the user to perform the correct operation. For example, the mobile terminal 60 may display a screen such as that shown in Figure 13 (B) in step S368 of Figure 8.

Figure 13 (C) is a screen that displays information to the effect that registration has not yet been performed and to prompt the user to read the information. For example, the mobile terminal 60 may display a screen such as that shown in Figure 13 (C) in step S377 of Figure 8. Figure 13 (E) is a screen that displays information to the effect that correct operation has not yet been performed and to prompt the user to read the information. For example, the mobile terminal 60 may display a screen such as that shown in Figure 13 (D) in step S378 of Figure 8.

Figure 14 (A) is a screen that displays information indicating that registration has failed and prompting the user to confirm the purchase of a product. For example, the mobile terminal 60 may display a screen such as that shown in Figure 14 (A) in step S468 of Figure 9. Figure 14 (B) is a screen that displays information indicating that the correct operation has not been performed and prompting the user to confirm the purchase of a product. For example, the mobile terminal 60 may display a screen such as that shown in Figure 14 (B) in step S568 of Figure 10.

Figure 14 (C) is a screen that displays that an instruction not to purchase has been accepted and that the item will be placed on hold. For example, the mobile terminal 60 may display a screen such as that shown in Figure 14 (C) in step S483 of Figure 9 and step S583 of Figure 10. Figure 14 (D) is a screen that displays that an instruction to purchase has been accepted and that the item will be placed on hold. For example, the mobile terminal 60 may display a screen such as that shown in Figure 14 (D) in step S485 of Figure 9 and step S585 of Figure 10.

Figure 15 is an example of a display on the display unit 405 of the settlement device 40. Figure 15 (A) is an example of a settlement screen displayed in step S17 of Figure 5. The settlement screen shown in Figure 15 (A) displays information (symbol a) about the successfully registered product "XX yogurt", information (symbol b) about the successfully registered product "XX shampoo", the total points for the successfully registered products ("XX yogurt", "XX shampoo") "2 points" (symbol f), and the total amount for the successfully registered products ("XX yogurt", "XX shampoo") "610 yen" (symbol g).

In addition, the checkout screen shown in FIG. 15(A) also displays information about products that were not successfully registered. Products displayed as "Pending (planned purchase) (Failed)" (symbol c), products displayed as "Pending (planned purchase) (Fraud)" (symbol d), and products displayed as "Pending (cancelled) (Failed)" (symbol e) are all information about products that were not successfully registered.

The product indicated by symbol c as "Pending (Planned Purchase) (Failed)" is a product for which registration by the customer has failed and for which an instruction to purchase has been received from the customer. Specifically, this is a product that corresponds to the pending information (2) in step S484 of FIG. 9. The product is in the pending basket. In other words, the customer has placed the product in the pending basket in accordance with the instructions in step S485 of FIG. 9 (e.g., FIG. 14 (D)).

The product marked with symbol d "Pending (Planned to be purchased) (Fraudulent)" is a product for which the customer has not performed the correct action and for which an instruction to purchase has been received from the customer. Specifically, this is a product that corresponds to the pending information (4) in step S584 of FIG. 10. The product in question is in the pending basket. In other words, the customer has placed the product in the pending basket in accordance with the instruction in step S585 of FIG. 10 (e.g., FIG. 14 (D)).

The product indicated by symbol e as "Pending (Cancelled) (Failed)" is a product for which the customer's registration has failed and for which the customer has instructed not to purchase the product. Specifically, this product corresponds to the pending information (1) in step S482 of FIG. 9. The product is in the pending basket. In other words, the customer has placed the product in the pending basket in accordance with the instructions in step S483 of FIG. 9 (e.g., FIG. 14(C)).

Although not shown in FIG. 15(A), some products are displayed as "Pending (Cancelled) {Fraudulent}." Products displayed as "Pending (Cancelled) {Fraudulent}" are products for which the customer has not performed the correct action and for which the customer has given instructions not to purchase the product. Specifically, these products correspond to the pending information (2) in step S582 in FIG. 10. These products will also be placed in the pending basket in accordance with the instructions in step S583 in FIG. 10 (e.g., FIG. 14(C)).

Products for which a customer has instructed them to purchase (items planned for purchase) are registered by a store clerk. Products for which a customer has instructed them not to purchase (cancelled items) are confirmed as cancelled by a store clerk. The details of the product registration by the store clerk are reflected in the transaction information. Note that the details of the confirmation by the store clerk of the cancellation of a product registration may also be reflected in the transaction information.

Note that if there are products on the checkout screen that were not successfully registered (failed or fraudulent) (i.e., if products that were not successfully registered are displayed on the checkout screen), in other words, if there are products to be purchased or canceled products, a notification may be issued to call a store clerk, as the store clerk will confirm the product registration or cancellation. For example, the checkout device 40 may send store clerk call information to the monitoring device 50 (or a terminal carried by the store clerk (smartphone, tablet, etc.)) when the checkout screen displays products that were not successfully registered.

In the columns for items to be purchased (symbols c and d), a "barcode scan" button and an "enter barcode" button are provided as buttons to be operated (touched) when registering items. The "barcode scan" button is operated when registering an item to be purchased using the scanner unit 406. The "enter barcode" button is operated when registering an item to be purchased by inputting the barcode number of the settlement device 40.

The Cancelled Item column (symbol e) contains a "Confirm" button that is operated to confirm the cancellation.

The number (symbol h) at the bottom right of the screen indicates the number of times the marker was not recognized correctly (the number of times counted up in step S356 in FIG. 8).

In addition, the display of "on hold (scheduled to be purchased)" or "on hold (cancelled)" allows the clerk to distinguish between successfully registered products, products to be purchased, and canceled products. Furthermore, the checkout screen shown in FIG. 15(A) has different display formats, allowing the clerk to more easily distinguish between them. Specifically, as shown in FIG. 15(A), the display areas for successfully registered products, products to be purchased, and canceled products are different, and the display colors (shown as dots in the drawing) are also different. In the example shown in FIG. 15(A), the gap between successfully registered products and products not successfully registered (products to be purchased, canceled products) is wider than the gap between products to be purchased and canceled products. This makes it easy to distinguish between products that do not require action (successfully registered products) and products that require action (confirmation of product registration or cancellation).

The store clerk can distinguish between the two by the display of "Failed" and "Fraud", but the display format may also be different. For example, the display area or color may be different for the symbol c "Pending (Planned Purchase) Failed", the symbol d "Pending (Planned Purchase) Fraud", the symbol e "Pending (Cancelled) Failed", and the symbol not shown "Pending (Cancelled) Fraud".

The display of "Fraud" is also effective in deterring customers. However, as this is a somewhat harsh expression, it may be a little softer (for example, "reading operation not detected"), or information that only the store (clerk) can understand (for example, "F"). It may be possible to set how these expressions are used. For example, it may be possible to select the expression (display information) in the monitoring device 50. It may be possible to make the selection on a customer-by-customer basis.

Figure 15 (B) is an example of a checkout screen in the middle of processing by a store clerk. Specifically, this is an example of a checkout screen at a stage where product registration for the product to be purchased (symbol c) has been completed, and confirmation of the cancellation of the canceled product (symbol e) has been completed, but product registration for the product to be purchased (symbol d) has not yet been completed.

Figure 15 (C) is the screen (small screen, window) that is displayed when the "Barcode Input" button is operated. In addition to the barcode input field, the screen in Figure 15 (C) also has a "Preset" button (symbol i) for identifying a product from preset information, a "Classification" button (symbol j) for identifying a product from its classification, and a "Amount" button (symbol k) for entering the amount.

The canceled product (symbol e) in Figure 15 (A) is erased in Figure 15 (B), but if a store clerk confirms the cancellation (the store clerk confirms the existence of the canceled product → presses the "Confirm" button), the information about the canceled product is erased.

The product to be purchased (symbol c) in FIG. 15(A) has product information (product "XX bread", etc.) displayed in FIG. 15(B), but if the product is registered by a store clerk, the product information will be displayed. In addition, when the product to be purchased is registered, the total points and total price are also updated.

The "[Input]" to the right of the product "XX Bread" marked with the symbol c in Figure 15(B) indicates that the product was registered by entering a barcode. Although not shown in Figure 15(B), if the product was registered by scanning a barcode, the word "[Scan]" will be displayed.

In Fig. 15(A) and (B), buttons ("barcode scan" button, "barcode input") are displayed for each item to be purchased, but a single set of buttons ("barcode scan" button, "barcode input") may be displayed on the checkout screen regardless of the number of items to be purchased. In this case, when the buttons ("barcode scan" button, "barcode input") are operated, the screen in Fig. 15(C) may be displayed, and product registration may be performed for the items to be purchased (if there are multiple items, any one of the items to be purchased; for example, the item to be purchased displayed at the top). Also, even if there are items to be purchased, the buttons ("barcode scan" button, "barcode input") do not need to be displayed. In this case, when the column for an item to be purchased is touched, the screen in Fig. 15(C) may be displayed, and product registration may be performed for that item to be purchased.

The above describes an embodiment of the present invention in detail with reference to the drawings, but the specific configuration is not limited to this embodiment, and includes designs that do not deviate from the gist of the present invention.

For example, in the above description, the mobile terminal 60 is configured to recognize the marker 73, which is a specific barcode, except when the customer is attempting to register a product. However, as long as the mobile terminal 60 can recognize it as something different from the barcode attached to the product, the thing that is recognized except when the customer is attempting to register a product does not have to be a barcode. For example, the mobile terminal 60 may be configured to recognize a store logo or the like except when the customer is attempting to register a product.

Also, the markers used for identification do not have to be predetermined. After scanning the product and registering it in the basket information, when the registration information is displayed on the screen of the mobile terminal, an image inside the shopping bag 90 (within the imaging range) or something distinctive or distinctive may be recognized as a marker and updated. Furthermore, if there is something on the shopping bag that can be used as a marker without the need to fix it with an arm or the like, a method in which the marker is registered when shopping begins may be used.

In the above explanation, products that are not successfully registered are placed in the holding basket, both for purchase and for non-purchase (see steps S483 and S485 in FIG. 9, steps S583 and S585 in FIG. 10, FIG. 14, etc.), but it is also possible to place the products to be purchased in the holding basket and not to place the products not to be purchased in the holding basket. In other words, in the above explanation, the store clerk returns the products not to be purchased to the shelf at the checkout stage, but the customer may return the products not to be purchased to the shelf at the purchase stage.

When the customer returns the product not purchased to the shelf at the purchase stage, a display or the like according to the mode may be performed. For example, the mobile terminal 60 may display the screen of FIG. 16(A) instead of the screen of FIG. 14(A) in step S468 of FIG. 9. The mobile terminal 60 may display the screen of FIG. 16(B) instead of the screen of FIG. 14(B) in step S568 of FIG. 10. The mobile terminal 60 may display the screen of FIG. 16(C) instead of the screen of FIG. 14(C) in step S483 of FIG. 9 and step S583 of FIG. 10. The mobile terminal 60 may display the screen of FIG. 16(D) instead of the screen of FIG. 14(D) in step S485 of FIG. 9 and step S585 of FIG. 10. Note that in FIG. 16(C), the customer is instructed to return the product not purchased (cancelled product) to the shelf.

At the checkout stage, the canceled items (items not purchased) are basically treated as reserved items in order to check whether the customer has forgotten to return them to the shelves. For example, the checkout screen may display the presence or absence of canceled items and the number of canceled items, and the store clerk may operate a confirmation button. It is possible to check whether all customers have forgotten to return the canceled items to the shelves, but the presence or absence (necessity) of confirmation may be different depending on the customer. For example, the response may be different depending on the degree of use of service A (recent number of times used, cumulative number of times used up to now, etc.). Specifically, the canceled items by customers who have little experience using service A may be treated as reserved items (confirmation work may be performed), and the canceled items by customers who have much experience using service A may not be treated as reserved items (confirmation work may not be performed). In addition, instead of or in addition to the degree of use of service A, the response may be different depending on the purchasing history of service A (for example, customer rank based on RFM analysis, etc.) or the purchasing history of the entire store including service A. Furthermore, there is no need to check whether the customer has forgotten to return the canceled item to the shelf at the checkout stage.

The mobile terminal 60 may also generate a captured image (image data) of the product and store it or transmit it to an external device (such as the cloud server 20). As an example, the mobile terminal 60 may release the shutter (generate image data) when acquiring a through image of the area around the product's barcode, and ultimately store the image data if it fails to read the product.

Appendix A1 to Appendix A8 are disclosed below.
(Appendix A1)
An acquisition means capable of acquiring information for product registration (e.g., product code, etc.);
An identification means for identifying a shopper's behavior (e.g., an action that affects the reading of the marker 73, an action of registering a product, etc.);
an output means for outputting information on the behavior of the shopper identified by the identification means,
The output means includes:
A product registration system characterized in that when a shopper's behavior is identified by the identification means and information for product registration is not acquired by the acquisition means (for example, when the marker 73 becomes unreadable and the product code cannot be acquired), information regarding the identified behavior (for example, reserved information (1) to reserved information (4), etc.) is output.

(Appendix A2)
The shopping mall has an imaging means for imaging the actions of a shopper,
The identification means is
The merchandise registration system according to claim A1, characterized in that the behavior of a shopper is identified based on the imaging information of the imaging means.

(Appendix A3)
The output means includes:
The product registration system according to claim A1 or A2, characterized in that the information regarding the behavior is transmitted to a predetermined information processing device.

(Appendix A4)
A display control means is provided,
The product registration system according to any one of claims A1 to A3, wherein notification information corresponding to the information on the behavior output by the output means is displayed.

(Appendix A5)
The acquisition means includes:
A product registration system as described in any of Appendices A1 to A4, characterized in that information for product registration is obtained as a product that requires the intervention of a store clerk in the registration, depending on the behavior of the shopper identified by the identification means.

(Appendix A6)
A recognition means capable of recognizing product identification information (e.g., a product code, etc.) that identifies a product;
an output means capable of outputting information corresponding to a recognition result by the recognition means (e.g., reserved information (1) to reserved information (4) etc.);
The recognition means includes:
It is possible to recognize predetermined identification information (e.g., information on a marker 73, etc.) different from the commodity identification information,
The output means includes:
A product registration system, characterized in that, when the recognition means is in a state where it does not recognize either the predetermined identification information or the product identification information, it outputs information corresponding to the recognition result.

(Appendix A7)
A recognition means capable of recognizing product identification information for identifying a product;
output means for outputting information corresponding to a recognition result by said recognition means;
The recognition means includes:
A predetermined identification information different from the product identification information can be recognized,
The output means includes:
A product registration system comprising: a recognition means for detecting whether or not the product identification information is recognized by the recognition means during a period in which the recognition means does not recognize the predetermined identification information; and a product registration system for detecting whether or not the product identification information is recognized by the recognition means.

(Appendix A8)
A program that causes a computer to function as a mobile terminal (e.g., a mobile terminal 60, etc.) used by a shopper to register a product,
The computer,
A reading means capable of reading information for product registration;
Identification measures to identify shopper behavior;
functioning as an output means capable of outputting information regarding the behavior of the shopper identified by the identification means;
The output means includes:
A program characterized by outputting information relating to the identified behavior when the behavior of a shopper is identified by the identification means and information for product registration cannot be read by the reading means.

The programs for implementing the above-described sales system 1, management device 10, cloud server 20, registered settlement device 30, settlement device 40, monitoring device 50, mobile terminal 60, etc. may be recorded in a computer-readable recording medium, and the programs may be read and executed by a computer system. The term "computer system" as used herein includes hardware such as an OS and peripheral devices. The term "computer-readable recording medium" refers to portable media such as flexible disks, optical magnetic disks, ROMs, and CD-ROMs, and storage devices such as hard disks built into a computer system. The term "computer-readable recording medium" also includes those that hold a program for a certain period of time, such as volatile memory (RAM) inside a computer system that becomes a server or client when a program is transmitted via a network such as the Internet or a communication line such as a telephone line. The above program may be transmitted from a computer system that stores the program in a storage device, etc., to another computer system via a transmission medium, or by a transmission wave in the transmission medium. Here, the "transmission medium" that transmits the program refers to a medium that has a function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line. The program may also be one that realizes some of the functions described above. Furthermore, it may be one that realizes the functions described above in combination with a program already recorded in the computer system, a so-called differential file (differential program).

1. Sales system (including specific sales systems)
10...Management device (store controller, etc.)
20 cloud server 30 registration settlement device 40 settlement device 50 monitoring device 60 mobile terminal 70 shopping cart 80 shopping basket 90 shopping bag 401 CPU
402...ROM
403...RAM
404: Hard disk 405: Display unit 406: Scanner unit 408: Card payment unit 409: Change machine 403: Printing unit 414: Audio output unit 415: Communication unit

Claims (5)

A program for causing a computer to function as a mobile terminal equipped with an imaging means, set at a predetermined position, and used by a shopper when registering a product, comprising:
The computer,
an operation receiving means for receiving an operation to start product registration from a shopper;
a marker recognition means for recognizing, as a marker, an object that has a distinctiveness that can be recognized when the mobile terminal is set at a predetermined position from the captured image captured by the imaging means;
when the marker recognition means recognizes the marker from the captured image, the marker recognition means functions as an instruction means for instructing a shopper to start product registration by the operation acceptance means ;
The instruction means is
a program for instructing a shopper to correct a position at which the mobile terminal is set when the marker recognition means does not recognize the marker from the captured image; a registration means for registering a product based on product identification information that identifies the product from the captured image;
a registration success/failure determination means for determining whether the registration of the product by the registration means has been successful;
Functioning as a
The registration success/failure determination means
The program according to claim 1, characterized in that if the marker recognition means no longer recognizes the marker from the captured image after the operation acceptance means accepts the start operation, it is determined whether the product has been successfully registered by the registration means. The instruction means is
3. The program according to claim 2 , wherein when said registration success/failure judging means judges that the registration of the product by said registration means has not been successful, said means instructs the shopper to scan the product again. A mobile terminal that is set in a predetermined position and used by a shopper to register a product,
An imaging means;
an operation receiving means for receiving an operation to start product registration from a shopper;
a marker recognition means for recognizing, as a marker, a distinctive object that can be recognized when the mobile terminal is set at a predetermined position from the captured image captured by the imaging means;
an instruction means for instructing a shopper to start product registration by said operation receiving means when said marker recognition means recognizes said marker from said captured image ,
The instruction means is
A mobile terminal characterized in that, if the marker recognition means does not recognize the marker from the captured image, the mobile terminal instructs the shopper to correct the position at which the mobile terminal is set . A system including a mobile terminal that is set at a predetermined position and that is used by a shopper when registering an item, and a payment device that settles a payment for the item registered using the mobile terminal,
The mobile terminal,
An imaging means;
an operation receiving means for receiving an operation to start product registration from a shopper;
a marker recognition means for recognizing, as a marker, a distinctive object that can be recognized when the mobile terminal is set at a predetermined position from the captured image captured by the imaging means;
an instruction means for instructing a shopper to start product registration by said operation receiving means when said marker recognition means recognizes said marker from said captured image ,
The instruction means is
A system characterized in that, if the marker recognition means does not recognize the marker from the captured image, the system instructs the shopper to correct the position at which the mobile terminal is set .

Images