JP2023138696A - Commodity sales data processing system, settlement device, and program - Google Patents

Abstract

To efficiently resolve a commodity-unidentified state in a commodity sales data processing system in which customers themselves register a commodity by using a portable terminal.SOLUTION: A commodity sales data processing system in which a customer registers a commodity by using a terminal device includes: imaging means that images a commodity to be registered; commodity identification means that identifies the commodity to be registered on the basis of a captured image that is obtained by imaging by the imaging means; and storage means that, when an identification failure result in which a commodity has not been identified by the commodity identification means is generated, stores, as an auxiliary image corresponding to the identification failure result, a captured image corresponding to a commodity identified by the commodity identification means for the first time after the identification failure result is generated.SELECTED DRAWING: Figure 16

Description

The present invention relates to a product sales data processing system, a payment device, and a program.

A system is known in which customers themselves register products using mobile terminals (see, for example, Patent Document 1).

JP2020-067947A

In a product sales data processing system in which customers themselves register products using mobile terminals, an unspecified state may occur in which the product to be registered cannot be specified. For products that are in an unspecified state, the unspecified state is resolved by, for example, an operation by a store clerk. It is preferable that such an unspecified state be resolved efficiently.

The present invention has been made in view of the above circumstances, and is intended to efficiently resolve the unspecified state of products in a product sales data processing system in which customers themselves register products using mobile terminals. The purpose is to

One aspect of the present invention for solving the above-mentioned problems is a product sales data processing system in which a customer registers a product using a terminal device, the system including an imaging means for capturing an image of the product to be registered. and a product specifying means for specifying a product to be registered based on a captured image obtained by capturing an image by the image capturing means, and when an unspecified result in which the product cannot be specified by the product specifying means is generated, the unspecified product is identified. and storage means for storing a captured image corresponding to a product first specified by the product specifying means after the timing at which the result is generated, as an auxiliary image corresponding to the unspecified result. .

Further, one aspect of the present invention is a payment device in a product sales data processing system in which a customer registers a product using a terminal device, the payment device being configured to register a product using the terminal device. Information acquisition means for acquiring registration result information regarding registration results; and information about the specified product in the product registration performed by the customer using the terminal device, based on the registration result information acquired by the information acquisition means. Display a registration result information list showing specific registration information and unidentified information indicating an unidentified result in which no product was identified, and select the unidentified information from among the registration result information shown in the registration result information list. A display control means is provided for displaying specified unspecified information and information regarding a product that is first specified after an unspecified result corresponding to the specified unspecified information is generated in response to the specified unspecified information. It is a payment device.

A computer serving as a payment device in a product sales data processing system in which a customer registers a product using a terminal device obtains registration result information regarding the result of product registration performed using the terminal device. an information acquisition means for determining the specified registration information for the specified product and information on the specified product in the product registration performed by the customer using the terminal device, based on the registration result information obtained by the information obtaining means; A list of registration result information is displayed in which unspecified information indicating unspecified results that were not specified is displayed, and in response to the unspecified information being specified from among the registration result information shown in the registration result information list, the specified information is displayed. This is a program for functioning as a display control means for displaying the specified unspecified information and information regarding the first specified product after the unspecified result corresponding to the specified unspecified information is generated.

It is a diagram showing an example of the configuration of a shopping system in this embodiment. It is a figure showing an example of installation of the payment device in this embodiment. It is a figure showing an example of the appearance of the payment device in this embodiment. It is a figure showing an example of composition of a settlement device in this embodiment. It is a figure showing an example of composition of a customer terminal device in this embodiment. It is a figure showing an example of composition of a transaction management device in this embodiment. It is a diagram showing an example of the configuration of a transaction status management device in this embodiment. It is a figure showing an example of customer information, store information, and cart information in this embodiment. It is a diagram showing an example of the definition of a reserved product in this embodiment. FIG. 2 is a sequence diagram showing an example of a processing procedure executed by the customer terminal device, settlement device, and transaction management device according to the present embodiment in response to accounting for one transaction in self-operation. It is a figure showing an example of a screen of a shopping application in this embodiment. It is a figure showing an example of a registered product presentation screen, a product registration screen, and a guidance image in this embodiment. It is a figure showing an example of the payment screen in this embodiment. It is a figure showing an example of a pending product processing screen in this embodiment. It is a figure showing an example of a situation confirmation screen in this embodiment. It is a figure showing an example of a product information input screen in this embodiment. It is a flowchart which shows the example of the process procedure which the customer terminal device and transaction management apparatus in this embodiment perform in connection with product registration. It is a flowchart which shows the example of a process procedure which the settlement apparatus in this embodiment performs in connection with the cancellation|release of hold of a hold product. It is a figure which shows an example of a situation confirmation screen in a 2nd modification. It is a figure which shows an example of the payment screen in a 3rd modification.

<Embodiment>
[Shopping system configuration example]
FIG. 1 shows an example of the configuration of a shopping system (an example of a product sales data processing system) of this embodiment. In the shopping system shown in the figure, for example, while a customer takes out a product to be purchased from a shelf, the customer registers the product using the customer terminal device 50, and the same customer also registers the product to be purchased by using the customer terminal device 50. This corresponds to accounting performed using the payment device 40 (accounting for customer terminals). Accounting here refers to procedures performed in response to one transaction, including product registration and settlement of registered products.
The shopping system of this embodiment includes a management device 10, a transaction management device 60, a payment device 40, a customer terminal device 50, a transaction status management device 80, and a credit card payment server 70.

The management device 10, the settlement device 40, and the transaction status management device 80 are installed in the store, and are communicably connected via a LAN 19 (which may be wired or wireless). The management device 10 is a business terminal used by a store clerk for store management, etc., and is capable of communicating with the transaction management device 60.

The transaction status management device 80 is a business terminal that monitors the status of transactions performed by customers using the customer terminal device 50 in the store, and is capable of communicating with the transaction management device 60.
Although four payment devices 40 are illustrated in FIG. 1, the number of payment devices 40 in one store is not particularly limited.

The customer terminal device 50 is operated by a customer (such as a shopper who is a member of the store, hereinafter also simply referred to as a customer). In addition to general communication functions and imaging functions (camera), the customer terminal device 50 has a recognition function that scans barcodes attached to products and reads product codes, that is, recognizes barcodes attached to products. Equipped with functions. Note that the recognition function provided in the customer terminal device 50 only needs to be capable of reading the product code, and does not need to be capable of recognizing which product the read product code is for. In other words, if a barcode exists as an object in an image captured by the imaging function (an image acquired as a through image), the customer terminal device 50 can read the product code from the barcode. It should be like this.

Such a customer terminal device 50 may be a smartphone, a tablet terminal, etc. owned by the customer. A shopping application having a function of registering products in accordance with the shopping system of this embodiment is installed on a smartphone, a tablet terminal, or the like as the customer terminal device 50.

Further, the customer terminal device 50 may be, for example, a terminal lent to a customer from a store. An application having a function of registering products in accordance with the shopping system of this embodiment is also installed on the customer terminal device 50 lent to the customer by the store. The customer terminal device 50 lent to the customer from the store may be used by the customer while being held in the hand, or may be used by the customer while being attached to a shopping cart, for example. It may be of the form.

Further, the customer terminal device 50 captures an image of the product (for example, the surrounding area to which the barcode is attached) (for example, releases the shutter), and generates a captured image (image data). The customer terminal device 50 may perform imaging by releasing the shutter according to the operation of the customer who is the operator, but in this embodiment, it is preferable that the customer terminal device 50 performs imaging by releasing the shutter based on its own judgment.

Further, the customer terminal device 50 may be equipped with an image recognition technology that extracts feature points from images (through images, captured images) and recognizes imaged targets (objects, etc.). For example, the customer terminal device 50 may identify (estimate) the photographed product using image recognition technology.

The payment device 40 is a payment device that allows payment by at least cash as a payment method. The settlement device 40 is configured to be able to communicate with the transaction management device 60. Note that the settlement device 40 may be able to communicate with the transaction management device 60, for example, via a relay of the management device 10.

The transaction management device 60 manages transactions performed in accordance with the shopping system of this embodiment.

The credit card payment server 70 executes payment processing corresponding to a credit card in response to a credit card payment request transmitted from the payment device 40, customer terminal device 50, or the like.

[About the payment device]
FIG. 2 is a diagram showing an example of installation of the payment device 40. FIG. 2(A) is a perspective view of the payment device 40 and the like as viewed from the customer side. FIG. 2(B) is a perspective view of the payment device 40 and the like as viewed from the clerk's side. As shown in FIG. 2(A), a counter is placed on the right side of the payment device 40 when viewed from the customer side.

FIG. 3 is a diagram showing an example of the appearance of the payment device 40. FIG. 3(A) is a perspective view of the payment device 40 viewed from the customer side. FIG. 3(B) is a perspective view of the payment device 40 viewed from the store clerk side.
FIG. 4 is a diagram showing a configuration example of the payment device 40. In FIGS. 3 and 4, the same parts are denoted by the same reference numerals.

Hereinafter, a configuration example of the payment device 40 shown in FIG. 4 will be described with reference to FIG. 3. The payment device 40 includes a CPU 401, a ROM 402, a RAM 403, a hard disk 404, a customer side display section 405, a customer side scanner section 406, a card payment section 408, a change machine 409, a clerk side display section 410, It includes a key operation section 411, a clerk side scanner section 412, a printing section 413, an audio output section 414, and a communication section 415. These can communicate with each other via a bus.

The CPU 401 is a central processing unit, and controls the operation of the payment device 40 by reading and executing a program stored in the ROM 402 .
The ROM 402 is a read-only memory and stores various information used by the CPU 401 including programs.

RAM 403 is a read/write memory and stores various information. For example, the RAM 403 stores information read from the ROM 402 or the hard disk 404, information acquired from the outside, information generated during processing, and the like.

Hard disk 404 stores various information. For example, instead of the ROM 402, the hard disk 404 may store programs executed by the CPU 401. Further, instead of the RAM 403, information read from the ROM 402, information acquired from the outside, information generated during processing, etc. may be stored.

The customer-side display unit 405 is a touch display for customers, and displays various information to the customer and receives various inputs from the customer.
The customer-side scanner unit 406 is a scanner unit for customers, and for example, scans a code attached to a product (for example, a code other than a barcode such as a barcode or two-dimensional code may be used), and obtains the product code. Read. Further, the customer side scanner unit 406 may scan a code (bar code, two-dimensional code, etc.) printed on the checkout ticket (registered trademark) and read information necessary for payment. Further, the customer side scanner unit 406 may scan a code (two-dimensional code, barcode, etc.) displayed on the display unit of the customer terminal device 50 and read information necessary for payment.

Note that the customer-side scanner unit 406 is used when a customer registers a product, but the customer may register the product using other methods. For example, if a preset key (button for ordering a product) corresponding to a product is displayed on the customer side display section 405, the customer may operate (press) the preset key to register the product.

The card payment unit 408 is a payment mechanism using various cards (credit cards, prepaid cards such as transportation cards, etc.). The card payment unit 408 of this embodiment includes a card recognition unit (reading unit), a display unit, and an operation unit, but it may be any device that includes at least a card recognition unit. Note that the card recognition unit may recognize various cards (for example, membership cards, point cards, etc.) that are not directly used for payment (settlement).

The change machine 409 (cash payment unit) is a cash payment mechanism, has an input slot for banknotes and coins, and a discharge port for banknotes and coins, calculates the amount inserted into the slot, and compares the input amount and the purchase amount. The difference in change amount is calculated and the change is discharged from the discharge port. Note that the change machine 409 faces the customer and is operated by the customer. Note that when banknotes or coins are inserted into the input slot, they are detected by a sensor (detection of insertion, detection of number of coins for each denomination, etc.).

In other words, the change machine 409 is used in the settlement device 40 when paying for registered products in cash (money). The change machine 409 has a bill slot for inserting bills, a coin slot for inserting coins, a bill discharge slot for discharging bills, a coin discharge slot for discharging coins, and a coin discharge slot for depositing or discharging coins. It includes a counter for counting coins, a coin conveyance mechanism between the input port or discharge port and the storage section, the above-mentioned sensor, and the like. Note that the bill slot and the coin slot are also referred to as deposit slots. The banknote outlet and the coin outlet are also referred to as change outlet. Note that the banknote input port and the banknote discharge port may be common, and the coin input port and the coin discharge port may be common.

Further, the change machine 409 counts the money replenished during the closing process and stores it in the storage section. Further, the change machine 409 counts the money to be withdrawn at the time of closing processing and discharges it from the change outlet. The closing process is a process of adjusting the amount (cash amount/cash amount) stored in the change machine 409 to a reference amount after closing or before opening.

The clerk-side display unit 410 is a touch display for the clerk, and displays various information to the clerk and receives various inputs from the clerk.
The key operation section 411 is composed of various keys (buttons), and receives various inputs from the store clerk.
The store clerk side scanner section 412 is a scanner section for the store clerk, and for example, scans a bar code attached to a product and reads the product code. Further, the clerk-side scanner unit 412 scans a bar code or the like attached to the clerk's name tag and reads the clerk code.

Note that although the clerk-side scanner unit 412 is used when a store clerk registers a product, the store clerk may register the product using other methods. For example, if a key corresponding to a product (for example, a key corresponding to a sports newspaper, etc.) is arranged in the key operation unit 411, the store clerk can operate (press) the key and register the product. good. Furthermore, if a preset key corresponding to a product is displayed on the store clerk side display section 410, the store clerk may register the product by operating the preset key.

The printing unit 413 prints and issues various media (receipts, billing tickets, etc.). The printing unit 413 is rotatably changeable in direction (direction of the media issuing port) from the clerk's side to the customer's side and from the customer's side to the clerk's side. The orientation of the printing unit 413 may be changed manually, or may be changed automatically (mechanically controlled, etc.), for example, in accordance with the transition (switching) of the operation mode, which will be described later. Note that a sensor or the like may be used to detect whether the printing unit 413 is oriented correctly or incorrectly.

Audio output unit 414 outputs audio. For example, the audio output unit 414 outputs audio guidance and the like.
The communication unit 415 executes communication with other devices (other payment devices 40 and management devices 10).

Note that the payment device 40 of this embodiment has a customer terminal compatible payment mode in which payment processing is performed based on customer terminal compatible accounting, and three different patterns of accounting (product registration, product registration, etc.) as operating modes. It is possible to switch between a total of four operating modes, including three accounting modes (first to third accounting modes) corresponding to (accounting).
The first checkout mode is an operating mode in which the clerk performs operations corresponding to product registration and checkout on the checkout device 40. The second checkout mode is an operating mode in which a store clerk performs an operation corresponding to product registration on the payment device 40, and a customer performs an operation corresponding to payment on the same payment device 40. The third checkout mode is an operation mode in which the customer performs operations corresponding to product registration and checkout on the checkout device 40.
As a result, in a store that has introduced the shopping system of this embodiment, by setting a customer terminal-compatible payment mode for some of the plurality of payment devices 40, it is possible to support customer terminal-compatible payment and use other payment devices. 40, for example, by setting the first checkout mode or the second checkout mode, the clerk can interact with the customer and make the checkout. Note that other payment devices 40 may be set to the third payment mode.
Such mode switching may be performed, for example, in accordance with a predetermined time period within the store's business hours.
Note that the payment device 40 of this embodiment may be configured as a dedicated device specialized for the above-mentioned customer terminal compatible payment mode.

[Example of configuration of customer terminal device]
FIG. 5 shows an example of the configuration of the customer terminal device 50. As shown in FIG. The customer terminal device 50 in the figure includes a CPU 501, a storage section 502, a RAM 503, a display section 504, an operation section 505, an imaging section 506, and a network compatible communication section 507.
The CPU 501 is a central processing unit, and controls the operation of the customer terminal device 50 by reading and executing a program stored in the storage unit 502.
The storage unit 502 is an auxiliary storage device for the CPU 501, and stores various information used by the CPU 501, including programs.
RAM 503 is the main storage device of CPU 501.
The display unit 504 displays images according to the display control of the CPU 501. The display unit 504 may be configured as a touch panel that allows operations on the screen.
The operation unit 505 collectively represents various operators provided in the customer terminal device 50, various input devices connected to the customer terminal device 50, and the like.
The imaging unit 506 performs imaging under the control of the CPU 401. Furthermore, the imaging unit 506 of this embodiment can also function as a part that reads two-dimensional codes in a code reader that processes two-dimensional codes.
The network compatible communication unit 507 is a unit that performs wireless communication via a network. The network compatible communication unit 507 enables the customer terminal device 50 to communicate with the transaction management device 60.

[Example of configuration of transaction management device]
FIG. 6 shows a configuration example of the transaction management device 60. The transaction management device 60 in the figure includes a CPU 601, a storage section 602, a RAM 603, and a network compatible communication section 604.
The CPU 601 is a central processing unit, and controls the operation of the transaction management device 60 by reading and executing programs stored in the storage unit 602.
The storage unit 602 is an auxiliary storage device for the CPU 601, and stores various information used by the CPU 501, including programs.
RAM 603 is the main storage device of CPU 601.
The network compatible communication unit 604 is a unit that performs communication via a network.

The storage unit 602 of the transaction management device 60 stores customer information, store information, product master (product information), and cart information.

[Example of configuration of transaction status management device]
FIG. 7 shows a configuration example of the transaction status management device 80. The transaction status management device 80 in the figure includes a CPU 801, a storage section 802, a RAM 803, an in-store communication section 804, a network communication section 805, a display section 806, and an operation section 807.
The CPU 801 is a central processing unit, and controls the operation of the transaction status management device 80 by reading and executing programs stored in the storage unit 802.
The storage unit 602 is an auxiliary storage device for the CPU 801, and stores various information used by the CPU 801, including programs.
RAM803 is the main storage device of CPU801.
The in-store communication unit 804 communicates with terminals such as the payment device 40 and the management device 10 in the store via the LAN 19.
The network compatible communication unit 805 is a unit that communicates with the outside via a network. In the case of the transaction status management device 80, the network compatible communication unit 805 can communicate with the transaction management device 60 on the cloud.
The display unit 806 displays images under the control of the CPU 801. Display unit 806 may be configured as a touch panel.
The operation unit 807 collectively represents the operators included in the transaction status management device 80 and the input devices connected to the transaction status management device 80.

[About information stored by the transaction management device]
The customer information stored in the storage unit 602 is information for managing individual customers. The transaction management device 60 generates customer information at the time of customer registration (memorizing the customer information of a certain customer may mean that the customer has been registered as a customer). Further, the transaction management device 60 updates customer information as appropriate based on cart information and the like. Transaction management device 60 may refer to cart information and update customer information, for example, at a predetermined time every day.

FIG. 8(A) shows an example of customer information stored in the storage unit 602 of the transaction management device 60. The customer information in the figure includes customer identification information, customer name, customer registration date, cancellation information, customer rank, number of points, etc.
Customer identification information is identification information that uniquely identifies a customer.
Note that the customer identification information may be application identification information that is uniquely assigned to each installed shopping application, for example, when the shopping application of this embodiment is installed. The application identification information may be, for example, a number uniquely (for example, sequentially) assigned to each installed shopping application. Alternatively, the application identification information may be terminal identification information unique to the customer terminal device 50 in which the shopping application is installed. The terminal identification information is unique to the hardware of the customer terminal device 50, such as a telephone number assigned to the customer who is the user of the customer terminal device 50, a MAC (Media Access Control) address, etc. It may be hardware identification information.
The customer name is the customer's full name or nickname. The customer registration date is the date and time of customer registration. The cancellation information is information regarding cancellation of a registered product after registration. The customer rank is a rank according to the customer's purchase history. When generating customer information for a new customer, customer identification information, customer name, and customer registration date are generated, but since this is before the actual transaction (product registration) begins, other information (cancellation information, etc.) is not generated.

For example, the transaction management device 60 uses the shopping service provided by the transaction management device 60 at the time of customer registration (for example, when the customer terminal device 50 uses the shopping service provided by the transaction management device 60 from an external device (for example, a predetermined server that provides all applications, the transaction management device 60). We generate and store customer identification information (when you download or install the Shopping Application). Further, for example, during customer registration, the transaction management device 60 uses the customer terminal device 50 to acquire information entered in the name column of the registration form (input form) and stores it as the customer name. Further, the transaction management device 60 obtains, for example, the current date and time at the time of customer registration, and stores it as the customer registration date.

Note that, instead of or in addition to storing the customer information in the storage unit within the transaction management device 60, the transaction management device 60 stores a part or part of the customer information in another device (such as a file server that the transaction management device 60 can access). You can memorize everything.

FIG. 8(B) shows an example of store information stored in the storage unit 602 of the transaction management device 60. The store information in the figure includes store identification information, store name (branch name), store specific information 1, and store specific information 2.
Store identification information is identification information that uniquely identifies a store. The store identification information shown in FIG. 8(B) is composed of a store (trade name) or company code and a branch code.
The store name is the name of the store. The store name shown in FIG. 8(B) is composed of a store (trade name) or company and a branch name.
Store specification information 1 is information on a two-dimensional code (such as a QR code (registered trademark)) for specifying a store where transactions are made (a store where products are bought and sold).
Store specification information 2 is store location information (GPS information) for specifying the store with which transactions are made.
Note that in the example shown in FIG. 8(B), the store identification information and the store specific information 1 are different, but the store identification information and the store specific information 1 may be the same.

Note that the transaction management device 60 may acquire store information and the like from the outside (such as a server (not shown) at the headquarters that controls each store) and store it. Further, instead of or in addition to storing the store information in the storage unit within the transaction management device 60, the transaction management device 60 stores part of the store information or You can memorize everything.

Cart information is information for managing individual transactions. Transaction management device 60 generates cart information at the start of a transaction. Additionally, the transaction management device 60 updates the cart information as the transaction progresses (each time a product is registered). ).

FIG. 8C is an example of cart information stored in the storage unit 602 of the transaction management device 60.
The cart information in the figure includes cart identification information, transaction start date and time, transaction end date and time, customer identification information, product information in the cart (registered product information, pending product information), cancellation information, and the like.

Cart identification information is identification information that uniquely identifies cart information. For example, the cart identification information includes store identification information, a date, and a serial number (for example, a serial number for each store and date).
The transaction start date and time is the start date and time of the transaction. The transaction start date and time may be, for example, the generation date and time of the cart information. Note that the transaction start date and time may be the registration date and time of the first product item (the date and time when the registered product information (registered product 1) in FIG. 8(C) was stored). Both the generation date and time of cart information and the registration date and time of one item of product may be stored separately.

The transaction end date and time is the end date and time of the transaction. For example, the transaction end date and time may be the settlement date and time. The customer identification information is customer identification information that identifies the customer of the transaction. Note that when cart information is generated, cart identification information, transaction start date and time, and customer identification information are generated, but since this is before the actual transaction (product registration) begins, other information (transaction end date and time, etc.) is not generated. Not done. The payment date and time may be the payment start date and time or the payment end date and time. As the transaction end date and time, both the settlement start date and time and the settlement end date and time may be stored separately.

Registered product information (total) is information that is updated every time a product is registered. The registered product information (total) includes the number of items (number of products), approximate subtotal amount (for example, subtotal amount before discount etc. is reflected), subtotal amount, etc. The registered product information (registered product 1) is the registered information of the first item of product. The registered product information (registered product 2) is the registered information of the second product item. In the example shown in FIG. 8(C), illustration of registered product information (registered product 3) to registered product information (registered product 5) is omitted. The registered product information (N; N is an integer) includes a product code, product name (product name), price, and the like.

The registered product information (N) may include the registration date and time of the N items of products. That is, the transaction management device 60 may store the registration date and time of the registered product as the registered product information. The registration date and time of each product may also be used as a material for determining whether or not to apply a service such as time service and the effect after application.

The pending product information (total) is information that is updated every time a pending product is registered.
A suspended product is a product that is temporarily registered as being on hold because it was not successfully registered due to a predetermined factor.
The on-hold product information (total) includes the number of on-hold products (number of products), the number of NON-FILE items among the on-hold products, the number of items that could not be read (unauthorized operation confirmation required) among the on-hold products, and the like.

NON-FILE means that the barcode or product code was successfully scanned at the store (the product code could be read), but the product code is not stored as product information in the product master. (unregistered product).

Reading NG is an example of a failure in scanning a product code at a store (failing to read the product code) or a case in which scanning of a product code at a store fails. In other words, reading failure refers to an event in which a timeout process is performed when it can be determined that a product has been imaged for a certain period of time using image recognition technology but barcode recognition has not been achieved, for example, or an event in which a timeout process has been performed.
For example, due to wrinkles on the package, scratches or stains on barcode printing, barcodes printed on transparent film, etc., the barcode may not be correctly read even though the customer has performed the operation to read the barcode. If the image cannot be acquired (recognized), it is determined that the reading is not successful.

The suspended product information (suspended product 1) is information about the first item of suspended merchandise. The suspended product information (suspended product 2) is information about the second item of suspended merchandise. The on hold product information (on hold product 3) is information on the three items of on hold products.

The held product information (held item N; N is an integer) includes the type of held item (information indicating whether the held item is a NON-FILE or NG reading), image data (image data captured when reading NG). including. For example, if the product N is a product on hold due to NON-FILE, the product information on hold (product N on hold) includes the product type ``1 (NON-FILE),'' a product code, and image data. Further, when the product N is a product on hold due to NG reading, the product information on hold (Hold product N) includes the type of product on hold “2 (NG reading)” and image data.

FIG. 9 shows an example of the definition of reserved products corresponding to the cart information of this embodiment. The figure shows an example in which pending products are classified into eight categories. A classification number from "1" to "8" is assigned to each of the eight categories of pending products.
The classification of the suspended product with classification number 1 is "NON-FILE" as explained in FIG.
The classification of the pending product with classification number 2 is "Reading NG" as explained in FIG.

The classification of the suspended product with classification number 3 is "pseudo scan input". A pending product with "pseudo scan input" is a product for which the customer attempted to have the customer terminal device 50 read the code of the product, but the customer terminal device 50 was unable to read the code normally.

As a specific example, "pseudo scan input" corresponds to a situation in which a product is placed within the imaging range of the imaging unit 506, but the code of the product is not included in the imaging range of the imaging unit 506.
Furthermore, "pseudo scan input" corresponds to a situation in which a product is placed within the imaging range of the imaging unit 506, but a portion without a code is imaged.
Furthermore, "pseudo scan input" corresponds to a situation in which a product is entered into the imaging range of the imaging unit 506 with the code portion hidden by a finger or the like.
In addition, "pseudo scan input" includes cases where the code of the product is imaged by the imaging unit 506, but the product is passed through the imaging range of the imaging unit 506 at a speed that the customer terminal device 50 cannot read the code. This situation applies.

The classification of the pending product with classification number 4 is "non-scanned input". A “non-scan-input” pending product is a product that is placed in a shopping cart without performing an operation to position the product within the imaging range of the imaging unit 506.
This non-scanning input may occur, for example, in a case where the customer terminal device 50 is fixedly attached to a shopping cart so as to be able to take an image of the inside of the shopping cart placed in the shopping cart. For example, if the customer terminal device 50 determines based on the captured image that a product has been placed in the shopping cart even though the code of the product has not been scanned, the customer terminal device 50 determines that the product has been placed in the shopping cart without being scanned. It can be determined that this has been done.

The classification of the pending product with classification number 5 is "age confirmation product." "Age confirmation products" are products, such as alcoholic beverages and cigarettes, that require a store clerk to confirm the age of a customer before they can be sold.

The classification of the suspended product with classification number 6 is "pharmaceuticals." "Medicine" is a product that requires confirmation from the purchaser by a qualified person such as a pharmacist.

The classification of the suspended product with classification number 7 is "security tag product." A "security tag product" is a product that has a security tag attached to it and therefore requires a store clerk to remove the security tag from the product at the time of sale.

The classification of the suspended product with classification number 8 is "cancelled product." A “cancelled product” is a product that was once registered but canceled by a customer's cancellation operation on the customer terminal device 50. The "cancelled item" is collected by a store clerk from the customer at the time of checkout, for example.

The product master is stored in the management device 10, for example. The product master stores product information for each product sold at the store. Product information corresponding to one product includes a plurality of predetermined information items including a product code and a product name.
The transaction management device 60 stores the product master acquired from the management device 10. The transaction management device 60 uses the stored product master for processing related to product registration.

For pending products that fall under classification number 1 "NON-FILE", the product code was obtained by scanning the code according to the product registration operation, but the product information associated with the obtained product code was not registered in the product master. It has not been. Therefore, it is not specified which of the products registered in the product master the pending product corresponding to the classification number 1 "NON-FILE" is.
Pending products that fall into one of the three classifications of classification numbers 2 to 4: "Unscanned", "Pseudo Scan Input", and "Non-Scan Input" will receive a product code as a result of code scanning according to the product registration operation. is failing to obtain. Therefore, held products that fall under any of the three classifications of classification numbers 2 to 4, "Unscanned", "Pseudo Scan Input", and "Non-Scan Input", cannot be stored in any of the products registered in the product master. It has not been determined whether

Pending products that fall into one of the four categories of classification numbers 1 to 7: "NON-FILE", "Unreadable", "Pseudo Scan Input", and "Non-Scan Input" are It is a specific product. An unspecified product is, for example, a product whose registration result is incomplete because it has not been specified which of the products registered in the product master.
Unspecified products are further divided into code-acquired products whose product codes have been obtained through product registration operations, such as "NON-FILE" with classification number 1, and "read NG" and "pseudo scan input" with classification numbers 2 to 4. ” and “Non-scan input” products, which have no product code obtained through the product registration operation, and products with no code obtained.

Note that instead of or in addition to storing the cart information in the storage section within the transaction management device 60, the transaction management device 60 stores part or all of the cart information in another device (such as a file server that the transaction management device 60 can access). You can memorize everything.

[Example of basic processing procedure for self-operation]
The sequence diagram in FIG. 10 shows an example of a processing procedure executed by the customer terminal device 50, the settlement device 40, and the transaction management device 60 in response to accounting for one transaction in self-operation. Specifically, the process shown in the figure is a process in which a customer registers a product to be purchased from among the products displayed in the store, and the customer terminal device 50 and the payment process until the payment for the registered product is completed. An example of the processing of the device 40 and the transaction management device 60 is shown.

Step S1: The customer terminal device 50 acquires information specifying the store (store specifying information). For example, by displaying a two-dimensional code to identify the store near the entrance of the store (outputting the two-dimensional code on the display screen, pasting a medium on which the two-dimensional code is printed, etc.), customers visiting the store can By scanning (reading) the two-dimensional code with the terminal device 50, the customer terminal device 50 may acquire store specific information. Note that when a customer who visits the store starts the shopping application, a screen may be displayed as the startup screen that instructs the customer to scan the two-dimensional code, or the customer who visits the store can scan the two-dimensional code using the customer terminal device 50. When the shopping application is scanned, the shopping application may be started, and a screen that notifies the customer that the customer is being connected to the transaction management device 60 may be displayed as the startup screen.

For example, since a store is specified by its location, a customer visiting the store may acquire location information (GPS information) using the customer terminal device 50 at the store (i.e., acquire location information of the store as store identification information). ). Note that when a customer visiting the store starts up the shopping application, the location information may be acquired and a screen that notifies the customer that the shopping application is being connected to the transaction management device 60 may be displayed as the startup screen. If multiple stores are detected from the location information and it is not possible to specify one store, a selection screen may be displayed to allow the customer to make a selection. Alternatively, the mode may be switched to a mode in which the two-dimensional code is forcibly acquired.

FIG. 11A shows an example of a shopping application startup screen displayed on the display unit 504 of the customer terminal device 50. In the startup screen shown in the figure, the store name is displayed. This store name is displayed based on the store specific information acquired in step S1.
When starting product registration for shopping, the customer operates a transaction start instruction button BT10 arranged on the startup screen.
In response to the operation of the transaction start instruction button BT10, the customer terminal device 50 transmits a transaction start request including the acquired store specific information and customer identification information to the transaction management device 60. The transaction start request may be sent in response to the customer's operation on the customer terminal device 50 as described above, or may be sent in response to the store specific information being acquired without receiving any particular operation. You can do it like this.
Regarding customer identification information, the information entered in the name field of the registration form using the customer terminal device 50 during customer registration (when downloading or installing the shopping application on the customer terminal device 50) is stored in the transaction management device 60. Although it may be stored in the customer information, it may also be stored in the storage unit of the customer terminal device 50 in addition to the transaction management device 60. In addition, if a store is identified (when the initial product registration screen described below is obtained), the store name, services being offered (information on flyers distributed on that day), and available coupon information will be displayed. It may be displayed on a screen (product registration initial screen or a screen different from the product registration initial screen). Note that the information on services and coupons may be acquired from the transaction management device 60 as, for example, screen information.

Further, the destination information (address of the transaction management device 60) is also obtained during customer registration (when downloading or installing the shopping application on the customer terminal device 50) and stored in the storage section of the customer terminal device 50. You can leave it there. In addition, in the case where the two-dimensional code is scanned, in addition to the store specific information, the destination information is also converted into a two-dimensional code, and by scanning the two-dimensional code with the customer terminal device 50, the customer terminal The device 50 may also acquire destination information as well as store specific information.

Step S2: The transaction management device 60, which receives the customer identification information and store specific information as a transaction start request from the customer terminal device 50, generates cart information for the transaction. In addition, in the initial state when cart information is generated, cart identification information, transaction start date and time, and customer identification information are stored, but since the actual transaction (product registration) has not yet started, other information (transaction end date and time) is stored. Date and time, product information (registered product information, pending product information, etc.) are not stored.

As described above, the transaction management device 60 stores the store information as shown in FIG. However, if the received store identification information is a two-dimensional code, the store identification information is obtained by referring to store identification information 1, and the received store identification information is location information (GPS information). If there is, the store identification information is acquired by referring to the store identification information 2. Note that if the store identification information received from the customer terminal device 50 is store identification information converted into a two-dimensional code, the transaction management device 60 may acquire it as is.

That is, the transaction management device 60, which has received the customer identification information and store specific information as a transaction start request from the customer terminal device 50, acquires the store identification information from the store specific information received from the customer terminal device 50, and also sets the current date. A serial number is obtained (assigned), and the store identification information, current date, and serial number are combined and stored as cart identification information in the cart information. Further, the transaction management device 60, which has received the store specific information and customer identification information as a transaction start request from the customer terminal device 50, acquires the current date and time and stores it as the transaction start date and time (generation date and time) in the cart information. Further, the transaction management device 60, which has received the store specific information and customer identification information as a transaction start request from the customer terminal device 50, stores the customer identification information received from the customer terminal device 50 as customer identification information in the cart information.

Step S3: The transaction management device 60 that has generated the cart information for the transaction generates product registration initial screen information (screen information of the product registration screen that is the initial screen) and transmits it to the customer terminal device 50. Specifically, the transaction management device 60 generates, for example, product registration initial screen information, and transmits the generated product registration initial screen information together with cart identification information to the customer terminal device 50.

Step S4: The customer terminal device 50, which has received the cart identification information and product registration initial screen information from the transaction management device 60, stores the cart identification information and displays the registration screen on the display unit. Specifically, the customer terminal device 50 displays a product registration initial screen as shown in FIG. 11(B), for example.

Step S5: In response to the customer's operation, the customer terminal device 50 scans the barcode attached to the product and reads the product code. In addition, in the same figure, it is assumed that the barcode scanning was successful. Steps S5 to S9 are repeatedly executed each time a barcode attached to a product is scanned.

The customer terminal device 50 that has acquired the barcode transmits cart identification information and product registration information including the product code obtained by scanning to the transaction management device 60.

Step S6: The transaction management device 60, which has received the product registration information (cart identification information and product code) from the customer terminal device 50, identifies the cart information of the transaction from the cart identification information included in the received product registration information. do.

Step S7: The transaction management device 60 updates the product data in the specified cart information. Specifically, if the received product registration information includes a product code for item N, the transaction management device 60 stores the product code as registered product information (registered product N) in the specified cart information. ), and the product name and price corresponding to the product code are acquired from the product information and stored as the product and price of the registered product information (registered product N). Further, the transaction management device 60 updates the registered product information (total) in the specified cart information.

Step S8: The transaction management device 60 that has updated the product data in the cart information generates product registration update screen information (screen information of the product registration screen that is the update screen to which the registered product has been added), and Send to. Specifically, the transaction management device 60 displays, for example, a product registration update screen on the customer terminal device 50 as shown in FIG. The product registration update screen information is generated and the generated product registration update screen information is transmitted to the customer terminal device 50 together with the cart identification information.

The product registration screen (product registration update screen) shown in FIG. 11C is displayed on the customer terminal device 50 after "〇〇 loaf of bread" is registered as the third product. That is, when the transaction management device 60 stores “〇〇 yogurt” as one item in the cart information, the transaction management device 60 generates product registration update screen information such that “〇〇 yogurt” is displayed on the customer terminal device 50, When the product registration update screen information is sent to the customer terminal device 50 along with the cart identification information, and “〇〇 Chocolate” is stored in the cart as the second item, “〇〇 Yogurt” and “〇〇 Chocolate” are stored in the customer terminal device 50. ” is displayed, the generated product registration update screen information is sent to the customer terminal device 50 along with the cart identification information, and “〇〇 loaf of bread” is stored as the third item in the cart information. In this case, as shown in FIG. 11(C), product registration update screen information is generated such that "XX yogurt", "XX chocolate", and "XX bread" are displayed on the customer terminal device 50. , the generated product registration update screen information is transmitted to the customer terminal device 50 together with the cart identification information.

Note that in response to the registration of one product, a registered product presentation screen WD1 that presents the currently registered product may be displayed, as shown in FIG. 12(A), for example. In addition, on the registered product presentation screen WD1 shown in the same figure, there is a "Yes" button BT1 and a "No" button for confirming with the customer whether or not to eat or drink by eat-in, depending on whether the target product is compatible with eat-in. An example in which BT2 is arranged is shown. In this case, the price of the product is calculated so that the consumption tax rate for the target product is different depending on whether the "Yes" button BT1 is operated or the "No" button BT2 is operated. become.

Step S9: The customer terminal device 50, which has received the cart identification information and product registration update screen information from the transaction management device 60, adds the product to the registration screen. Specifically, the customer terminal device 50 displays a product registration update screen as shown in FIG. 11(C), for example. As described above, the product registration screen (product registration update screen) shown in FIG. It is.

Step S10: The customer terminal device 50 receives a payment transfer instruction operation as a customer operation. For example, a touch on the "proceed to checkout" button shown in FIG. 11(C) is accepted.

Step S11: The customer terminal device 50 that has received the payment transfer instruction operation generates a two-dimensional code. That is, the customer terminal device 50 converts information (for example, cart identification information) necessary for the customer terminal device 50 to perform payment processing for the purchased product into a two-dimensional code. The customer terminal device 50 that has generated the two-dimensional code displays the generated two-dimensional code on the display section. For example, a code display screen on which two-dimensional codes as shown in FIG. 11(D) are arranged is displayed on the display unit.

Step S12: The payment device 40 scans (reads) the two-dimensional code displayed on the display section of the customer terminal device 50. For example, the payment device 40 scans a two-dimensional code displayed on the display section of the customer terminal device 50 that is pointed within the recognition range by the customer side scanner section 406 by the customer.

Step S13: After reading the two-dimensional code displayed on the display section of the customer terminal device 50, the payment device 40 requests the transaction management device 60 to calculate the subtotal amount. For example, the payment device 40 transmits a calculation request (subcalculation request information) requesting calculation of a subtotal amount to the transaction management device 60 together with cart identification information obtained from the two-dimensional code.

Step S14: The transaction management device 60, which has received the cart identification information and the subcalculation request information from the customer terminal device 50, specifies the cart information of the transaction from the cart identification information.

Step S15: The transaction management device 60 that has identified the cart calculates the subtotal amount using information such as the price in the registered product information included in the cart information corresponding to the identified cart information.

Step S16: The transaction management device 60 that has calculated the subtotal amount updates the cart information (memorizes the subtotal amount (subtotal amount after calculation)), and sends the subtotal information indicating the calculated subtotal amount to the payment device 40 along with the cart identification information. Send.

Step S17: After receiving the cart identification information and subtotal information (settlement information) from the transaction management device 60, the settlement device 40 displays the subtotal amount on the customer side display section 405.

Step S18: The payment device 40 displays the subtotal amount on the customer side display section 405 and executes payment (settlement). Specifically, the payment device 40 accepts the selection of the payment type. In the case of cash, the deposit is accepted, the amount of change is calculated, and if there is change, the change is released and a receipt is issued. Further, when the payment is completed, the payment device 40 transmits payment completion information together with the cart information to the transaction management device 60, and the transaction management device 60 stores the transaction end date and time of the cart (settlement date and time).

The customer terminal device 50 sends the product code to the transaction management device 60 after scanning the product (S5); The product code may not be transmitted for products other than those listed above (for example, products scanned after being moved to another store). For example, the customer terminal device 50 acquires and stores location information (GPS information) when visiting the store (or when displaying the product registration initial screen). Additionally, the customer terminal device 50 acquires location information when scanning each product, and combines the location information acquired when scanning the product and the location information acquired when visiting the store (or when displaying the initial product registration screen). compare. Then, if the two match (or almost match), the customer terminal device 50 allows the transmission of the product code of the product to the transaction management device 60, and if they do not match (or almost match), the customer terminal device 50 allows the product code of the product to be sent to the transaction management device 60. Transmission of the product code of the product to the transaction management device 60 may be prohibited.
This makes it possible to prevent inappropriate product registration (for example, product registration for a cart created at another store, etc.).

If the payment device 40 prohibits the transmission of the product code as described above, the payment device 40 displays an error message (e.g., "Registration is not possible because you are not in 〇〇 store") on the customer side display section 405 after scanning the product. May be displayed. Further, the payment device 40 may display the above message on the clerk side display section 410 instead of or in addition to the customer side display section 405.

In addition, the payment device 40 requests the transaction management device 60 to calculate the subtotal amount after reading the two-dimensional code displayed on the display section of the customer terminal device 50 (S13). The calculation of the subtotal amount may not be requested for products other than the above (for example, products scanned at other stores). For example, the payment device 40 may refer to the store identification information of the store (it may store and refer to the store identification information of the store within the self-payment device 40, or may store and refer to the store identification information of the store within the self-payment device 40, or may store and refer to the store identification information of the store within the self-payment device 40, or store and refer to the store identification information of the store within the self-payment device 40). When the two-dimensional code displayed on the display section of the customer terminal device 50 is read, the cart identification information obtained from the two-dimensional code and the store identification of the store Compare information. Then, if the store identification information included in the cart identification information includes the store identification information of the store, the payment device 40 permits the request for calculation of the subtotal amount, and includes the store identification information of the store. If it is not configured, the request to calculate the subtotal amount may be prohibited.
This makes it possible to prevent inappropriate payment (for example, payment for a product registered at another store, etc.).

If the payment device 40 prohibits the request for the subtotal amount as described above, the payment device 40 sends an error message (for example, "Payment cannot be made because it includes products from stores other than 〇〇 store") to the customer after reading the two-dimensional code. It may be displayed on the display section 405. Further, the payment device 40 may display the above message on the clerk side display section 410 instead of or in addition to the customer side display section 405.

In the example of the processing procedure shown in FIG. 10, the transaction management device 60 calculates the subtotal amount. However, for example, the payment device 40 may acquire cart information from the transaction management device 60 and calculate the subtotal amount based on the acquired cart information. Alternatively, the transaction management device 60 sends cart information to the management device 10 to request calculation of the subtotal amount, and the management device 10 calculates the subtotal amount using the received cart information, and the calculated subtotal amount is , may be transmitted to the settlement device 40 via the transaction management device 60.

[About the operation of the customer terminal device when an unspecified product is registered]
In FIG. 10, step S5 is a process corresponding to the case where the product code has been successfully read and the product corresponding to the read product code has been identified.
However, when the customer terminal device 50 is performing the product code reading process in response to step S5, the product registration result indicates that the product is registered as an unspecified product according to one of the classification numbers 1 to 4 in FIG. There may be cases where the result (unspecified result) is that it has been carried out.

As a specific example, FIG. 12(B) shows a case in which the customer terminal device 50 determines that an unspecified result as a pseudo scan of classification number 3 has occurred, using a captured image of the product registration screen. ing. The figure shows a product registration screen displayed on the customer terminal device 50. In the product registration screen shown in the figure, a captured image of a customer trying to have the customer terminal device 50 read the barcode of the product with the barcode portion hidden by his or her finger is displayed. It is shown.

In such a case, the customer terminal device 50 obtains a processing result indicating that the presence of the barcode was recognized from the captured image, but the code could not be extracted from the barcode. Since such a processing result is included in one type of processing result in a pseudo scan, the customer terminal device 50 detects that a pseudo scan has occurred. In this case, the customer terminal device 50 transmits, to the transaction management device 60, product registration information indicating that the product is an unspecified product that has been pseudo-scanned with classification number 3.

Furthermore, when transmitting product registration information in response to the occurrence of an unspecified result as described above, the customer terminal device 50 transmits a captured image corresponding to the generated unspecified result. The captured image to be transmitted is configured such that the captured image captured by the imaging unit 506 for reading the product code is buffered for a certain period of time, and then extracted from the buffered captured image. It's fine. Further, the captured images to be transmitted may be, for example, a predetermined number of still images, or may be moving images for a predetermined period of time depending on the occurrence of an unspecified result.

Further, when the product to be registered is determined to be an unspecified product due to the occurrence of the unspecified result as described above, the customer terminal device 50 treats the unspecified product as a pending product. In this case, at the timing when the customer terminal device 50 determines that an unspecified result has occurred, as shown in FIG. A guide image WD2 may be displayed to prompt the user. Specifically, in the guide image WD2 shown in the figure, a message is displayed saying, "If the product cannot be scanned, please put the product in a special bag." Note that the form of the guide image WD2 is not particularly limited.
The guide image WD2 in the figure is displayed so as to be superimposed on the product registration screen. In the guide image WD2 of the figure, a confirmation button BT3 is arranged. In this case, guide image WD2 is deleted in response to the operation on confirmation button BT3. Alternatively, guide image WD2 may be erased when a certain period of time has elapsed since the start of display. After the guide image WD2 is deleted, the screen may be returned to, for example, the product registration screen shown in FIG. 11(B) or FIG. 11(C).

Unspecified products are treated as pending products as described above. On the product registration screen, a registered product list is displayed as shown in Figure 11(C), but for products classified as unspecified products among pending products, list items are not displayed in the registered product list. Tomoyoshi. Pending products that are not classified as unspecified products are displayed as list items in the registered product list. Note that for unspecified products as well, a list item may be displayed in the registered product list to indicate that the products are unspecified products.

After completing the registration of all the products to be purchased, the customer performs an operation to instruct payment transfer, and causes the payment device 40 to read the two-dimensional code displayed on the customer terminal device 50. The payment device 40 that has read the two-dimensional code requests the transaction management device 60 to calculate the subtotal amount, as shown in step S13 in FIG. Here, when requesting calculation of the subtotal amount, the payment device 40 of this embodiment requests payment information to be used for payment processing.

In response to a request for settlement information, transaction management device 60 specifies relevant cart information and generates settlement information based on the specified cart information. The payment information includes registered product information in the cart information and information such as the subtotal amount depending on the registered product. Further, among the registered product information included in the payment information, the registered product information corresponding to the non-registered product includes a captured image corresponding to the unspecified result. Transaction management device 60 transmits the generated settlement information to settlement device 40.
Upon receiving the payment information, the payment device 40 causes the customer side display unit 405 to display a payment screen that reflects the content of the received payment information.

FIG. 13 shows an example of a payment screen displayed on the customer side display section 405 of the payment device 40.
In the payment screen shown in the figure, a registered product list area AR3 is arranged. The registered product list area AR3 is an area that presents a list of products registered by the customer terminal device 50.

In the following description, product information (registered product information, pending product information) corresponding to each product list item in the registered product list area AR3 may also be referred to as a record.
In addition, regarding registered product information and pending product information corresponding to classification by age verification products (classification number 5), medicines (classification number 6), and security tag products (classification number 7), the product must be specified. Therefore, it is also called a product specific record. Among the product identification records, the registered product information is also called a normal registration record because it corresponds to a normally registered product rather than a pending product. Among the pending product information, those corresponding to unspecified products (classification numbers 1 to 4) are also referred to as unspecified records.

If a pending product is included among the registered products, in the registered product list area AR3, the list items for the pending product are prioritized and arranged from the top of the left column. That is, in the registered product list area AR3, the list items of pending products are arranged in order from the top.
In the example of the figure, in response to the fact that five suspended products were included among the registered products, in the registered product list area AR3, a suspended product list area AR31 that includes list items of five suspended products from the upper left An example is shown in which a is formed. The list items in the pending product list area AR31 are different from the list items of other products in aspects such as background color, for example.

Then, in the pending product list area AR31, list items corresponding to product identification records in which products are identified among the pending products are arranged in order, and then list items corresponding to unidentified products in which the products have not been identified are arranged in order. The list items are arranged in order. In other words, the pending product list area AR31 is divided into list items corresponding to product identification records in which products are specified among the pending products, and list items corresponding to unspecified products in which the products are not specified. It is an array.

In the registered product list area AR3, list items corresponding to normal registration records are arranged following the pending product list area AR31. In the figure, an example is shown in which list items corresponding to normal registration records are arranged in descending order of product price in the registered product list area AR3.
Among the list items in the registered product list area AR3, the list item corresponding to the product specification record shows the product name, registered quantity, price corresponding to the registered quantity, etc. of the specified product.

In the list item for each pending product in the pending product list area AR31, a pending release declaration button BT31 is arranged. The hold release declaration button BT31 is a button for performing an operation to declare release of hold for the corresponding product. However, the hold release declaration button BT31 cannot be operated on the payment screen shown in the figure, but can be operated on the hold product processing screen described later. On the payment screen, the hold release declaration button BT31 has a function as an icon indicating that the corresponding list item corresponds to a held product.

Each hold release declaration button BT31 shows the name of the classification of the hold product to which the corresponding product corresponds.
Specifically, for example, for age verification products (classification number 5), medicines (classification number 6), crime prevention tag products (classification number 7), and canceled products (classification number 8), the corresponding button BT31 is pressed to release the hold release declaration button BT31. A name indicating the classification is displayed. Furthermore, for unspecified products (classification numbers 1 to 4), the name "pending product" is displayed all together.
In the same figure, as a display on the hold release declaration button BT31, the list item on the first line is displayed as "age confirmation" corresponding to the age confirmation product (classification number 5), and the list item on the second line is displayed as "age confirmation". The item is displayed as "Security Tag" in response to the fact that it is a product with a security tag (classification number 7), and the list item on the third line is displayed as "Security Tag" in correspondence to the fact that it is a pharmaceutical product (classification number 6). An example of a product labeled as "Medicine" is shown. Furthermore, an example is shown in which the list items on the fourth and fifth lines are each displayed as "pending product."
Thumbnail images TN are arranged in the list items on the fourth and fifth lines. The thumbnail image TN is a captured image obtained by capturing an image using the customer terminal device 50 while a product registration operation is being performed for a corresponding unspecified product. The thumbnail image TN may be a still image or a moving image.
Depending on whether or not the thumbnail image TN is placed, the store clerk visually distinguishes between the list items for pending products, those corresponding to unidentified products, and the list items corresponding to pending products whose products have been identified. can. Furthermore, the store clerk can grasp what the product registered as an unspecified product actually is based on the content displayed in the thumbnail image TN.
Note that the display indicating the corresponding classification in the hold release declaration button BT31 may be a picture such as an icon indicating the classification.

Among the list items of suspended products, list items corresponding to identified products such as age verification products (classification number 5), medicines (classification number 6), security tag products (classification number 7), etc. The product name, registered quantity, price corresponding to the registered quantity, etc. of the product are displayed.
On the other hand, in a list item corresponding to an unspecified product, contents are displayed according to the classification of the corresponding pending product. Specifically, in the case of an unspecified product as a NON-FILE (classification number 1), the list item in the fifth line of the same figure indicates that it is a NON-FILE, and the product that has been read is Code may be shown. In addition, in the case of unspecified products (products with no code obtained) that are not scanned (classification number 2), pseudo-scanned (classified number 3), or non-scanned (classified number 4), the 4th line in the figure A message indicating that the product code could not be read may be displayed as a list item.

As shown in the figure, if a pending product list area AR31 is formed in the registered product list area AR3 of the checkout screen, the store clerk needs to perform an operation to process the pending products. When the payment screen in which the pending product list area AR31 is displayed as described above, the payment device 40 calls a store clerk to have the store clerk perform an operation for processing the pending product. To call the store clerk, the payment device 40 may, for example, display a display device such as a sign pole in a predetermined manner, or send a notification of the clerk call to a terminal device that the store clerk owns or monitors. good.
In addition, as shown in the same figure, when the registration result includes a pending product and the payment screen in which the pending product list area AR31 is displayed is displayed, the payment device 40 displays information related to the customer's payment. It is configured not to accept any operations. Further, for example, in the area above the registered product list area AR3 on the payment screen in the figure, a message is displayed to inform the customer that the clerk who received the call will come to the payment device 40. There is.

The clerk who goes to the settlement device 40 in response to the clerk's call performs a login operation to log into the settlement device 40 as a clerk. In this case, the login operation is, for example, an operation of having a scanner section (customer side scanner section 406 or clerk side scanner section 412) read a code (two-dimensional code or barcode) printed on the clerk's ID card. good. Alternatively, it may be an operation of inputting a clerk number or the like by, for example, inputting a number.
In response to the store clerk logging in, the display on the customer side display section 405 changes from the payment screen to the pending product processing screen.

FIG. 14 shows an example of a pending product processing screen. In this figure, the same parts as in FIG. 13 are given the same reference numerals.
On the pending product processing screen, the contents of the registered product list area AR3 are displayed in a state inherited from the payment screen (FIG. 13). Therefore, in the initial display of the pending product processing screen, the list items corresponding to the registered product information in the registered product list area AR3 are arranged in order of price. Further, the contents of characters and the like displayed on the hold release declaration button BT31 arranged in the list item for each held product in the held product list area AR31 are also carried over from the payment screen (FIG. 13).

On the left side of the area above the registered product list area AR3 on the hold product processing screen in the same figure, it is indicated that the screen is for product management, and an instruction is given to the store staff to release the hold of the hold product. A message is displayed.

Further, on the right side of the area above the registered product list area AR3, sort buttons BT32 (price order button BT32-1, registration order button BT32-2, department order button BT32-3) are arranged.
The sort button BT32 is a button for sorting the list items corresponding to the registered product information in the registered product list area AR3 according to the corresponding sort order. Among the sort buttons BT32, the price order button BT32-1 is a button for sorting list items corresponding to registered product information in order of price. The registration order button BT32-2 is a button for sorting the list items corresponding to the registered product information in the order in which they were registered by the customer using the customer terminal device 50. The department order button BT32-3 is a button for sorting the list items corresponding to the registered product information by dividing them into departments to which the products correspond.

When canceling a hold on a product in a category for which the product has been identified (age verification product (classification number 5), pharmaceutical product (classification number 6), security tag product (classification number 7) among the products on hold, the store clerk will performs predetermined response work according to each classification, and then performs an operation (for example, tap operation) on the hold release declaration button BT31 arranged in the corresponding list item in the held product list area AR31. , the entire list of items on hold may be tap-operable.
When the product to be released from hold is an age-confirmed product, the store clerk confirms that the customer is an adult as a receptionist, and operates the hold release declaration button BT31 placed in the corresponding list item.
In addition, if the product to be released from hold is a pharmaceutical product, the store clerk must either explain the situation himself or herself if he or she is a pharmacist, or explain it to another store employee who is qualified as a pharmacist. After doing so, the user operates the hold release declaration button BT31 placed in the corresponding list item.
Additionally, if the product to be released from hold is a product with a security tag, the store clerk will remove the security tag attached to the product and press the hold release declaration button BT31 placed in the corresponding list item. Manipulate.
When a hold is released for a held product in a category for which the product can be identified as described above, the display on the operated hold release declaration button BT31 changes from the state indicating the corresponding classification as shown in FIG. 13. , for example, changes to a state indicating that the hold has been released, such as "confirmed" (or may be "hold released") shown in FIG.

In addition, the store clerk will check the pending products that are classified as unspecified products (NON-FILE (classification number 1), NG reading (classification number 2), false scan input (classification number 3), non-scanning input (classification number 4)). When canceling the hold, first, in the hold product list area AR31, the user operates the hold release declaration button BT31 arranged in the list item corresponding to the hold product to be released.
In response to the operation of the suspension release declaration button BT31 corresponding to an unspecified product in this way, a status confirmation screen is displayed on the customer side display section 405.

FIG. 15 shows an example of the status confirmation screen P10. The status confirmation screen P10 is a screen showing the status of the unspecified product corresponding to the operated hold release declaration button BT31, which is registered by the customer's operation on the customer terminal device 50. The status confirmation screen P10 may be displayed, for example, so as to be superimposed on the pending product processing screen of FIG. 14.
The status confirmation screen P10 in the same figure exemplifies a case where an unspecified product to be processed for release from hold is classified into pseudo scan input (classification number 3).
In addition, the status confirmation screen P10 in the figure shows that the customer terminal device 50 is held in a holder provided in the cart, and in the state that the customer terminal device 50 is held in the holder, the customer terminal device 50 is in the shopping cart. This corresponds to cases in which images can be taken. The customer causes the customer terminal device 50 held in the holder section to scan the product code.

In the situation confirmation screen P10 shown in the figure, captured images G11, G12, and G13 are arranged in this order from left to right. The captured images G11, G12, and G13 are captured images obtained when the customer terminal device 50 was trying to scan the product code of a corresponding unspecified product.

The arrangement order of the captured images G11, G12, and G13 from left to right corresponds to the lapse of time in which the images were captured.
First, the second (center) captured image G12 is a captured image captured when the product to be processed is within the imaging range of the customer terminal device 50 during the product registration operation.
The first captured image G11 is an image captured in the past than the second captured image G12, and is a captured image captured at a timing before the product to be processed falls within the imaging range.
The third captured image G13 is an image captured after the second captured image G12, and is a captured image captured at a timing when the product to be processed is out of the imaging range.

In this case, by looking at the captured images G11, G12, and G13, the store clerk can grasp the situation at the time of the product registration operation that resulted in the unspecified result.
Specifically, in the example shown in the figure, the store clerk first determines by transitioning from the captured image G11 to the captured image G12 that the product to be processed has not been placed in the shopping cart before the product registration operation is performed. I can understand what wasn't there.
In addition, the store clerk can understand from the captured image G12 that the product code part of the product was hidden by his hand, so that an unspecified result was generated as a pseudo-scan input (classification number 3).
Further, the store clerk can understand what the product to be processed is based on the captured image G12. Specifically, in the example shown in the figure, the store clerk understands that the product to be processed is "canned yakitori."
In addition, the store clerk can understand that the product to be processed has been placed in the shopping cart after the product registration operation has been performed by the transition from the captured image G12 to the captured image G13.

After understanding the situation during the product registration operation as described above, the store clerk operates the screen transition button BT35 to display the product information input screen. In response to the screen transition button BT35 being operated, a product information input screen is displayed instead of the status confirmation screen P10.
In addition, while the status confirmation screen P10 is displayed, the store clerk inputs the product code of the product to be processed that was placed in the shopping cart into the scanner section of the payment device 40 (customer-side scanner section 406, store clerk An operation may be performed to read the information using the side scanner section 412). When the payment device 40 successfully reads the product code, a product information input screen is displayed.

FIG. 16 shows an example of the product information input screen P40. The product information input screen P40 is a screen on which a store clerk performs an operation to input product information of a product to be processed as an unspecified product. The product information input screen P40 may also be displayed to be superimposed on the pending product processing screen, for example, similarly to the status confirmation screen P10.

In the product information input screen P40, a product information area AR41, an input operation area AR42, a registration button BT41, a cancel button BT42, and a captured image area AR43 (target captured image area AR43-1, next captured image area AR43-2) are arranged. Ru.

The product information area AR41 is an area where product information regarding corresponding unspecified products is shown. In the product information area AR41 in the same figure, the product name, product code, price, registered quantity, etc. are shown. The registered quantity can be input (registered) by operating the plus button and minus button arranged in the product information area AR41.

The input operation area AR42 is an area where a software keyboard and buttons for inputting product information in the product information area AR41 are arranged.
In the input operation area AR42, it is possible to switch between a keyboard with a key arrangement corresponding to numerical input and a keyboard with a key arrangement corresponding to character input according to an operation.

A preset call button BT421 is arranged in the input operation area AR42. By operating the preset call button BT421, a preset key screen is displayed in which preset keys corresponding to each product are arranged.
For example, if the unspecified product to be processed is one of the following: NG reading (classification number 2), pseudo-scanning (classification number 3), or non-scanning (classification number 4), the store clerk may If the product name is known, a preset key screen may be displayed. The store clerk can enter the product information of the product to be processed into the product information area AR41 by operating the preset key of the corresponding product from among the preset keys arranged on the displayed preset key screen. .

Furthermore, a product code input declaration button BT422 is arranged in the input operation area AR42. The product code input declaration button BT422 is a button that is operated when declaring that a product code input operation will be performed.
The store clerk operates the product code input declaration button BT422, and then inputs, for example, the numerical value of the product code indicated by printing etc. on the product to be processed into the keyboard arranged in the input operation area AR42. I do. As a result, the product code is input into the product information area AR41.

If the product information input screen P40 is displayed in response to the clerk having the payment device 40 read the product code, the read product code has already been entered in the product information area AR41, and the NON- If it is an unspecified product other than FILE (classification number 1), for example, the payment device 40 identifies the product from the product master based on the product code, so the product name, price, etc. are also input. It is in.

Furthermore, if the product to be processed is classified as NON-FILE (classification number 1), the product code is entered in the product information area AR41 when the product information input screen P40 is displayed. in a state. In this case, the store clerk inputs the product name, price, etc. into the product information area AR41 by operating a keyboard arranged in the input operation area AR42.

The registration button BT41 is a button operated to confirm the information input in the product information area AR41 as described above, release the hold on the corresponding unspecified product, and make it a normally registered product. .
The cancel button BT42 is a button for performing an operation to instruct cancellation of registration for the corresponding unspecified product.

After confirming the content entered in the product information area AR41, the store clerk operates the registration button BT41. As a result, the payment device 40 releases the unspecified product from the pending state and processes the product to be included in the registered products.
After the registration button BT41 is operated, the product information input screen P40 is cleared and the screen returns to the pending product processing screen (FIG. 14). At the stage of returning to the pending product processing screen, the pending release declaration button BT31 for the list item of the product that was to be processed in the pending product list area AR31 indicates that the pending state has been released by displaying, for example, "confirmed". shows.

Alternatively, if the store clerk determines that the product to be processed should not be registered, the store clerk operates the cancel button BT42 to cancel the registration and collect the product from the shopping cart.
Even after the cancel button BT42 is operated, the product information input screen P40 is cleared and the screen returns to the pending product processing screen (FIG. 14). When the screen returns to the pending product processing screen, the list item of the product to be processed in the pending product list area AR31 has changed to a predetermined state indicating that the registration has been cancelled.

Note that the classification number, classification name, etc. of the corresponding unspecified product may be displayed on the status confirmation screen P10 and the product information input screen P40.

Further, in the captured image area AR43, the target captured image area AR43-1 is an area where the target captured image is displayed. The target captured image is a captured image obtained by capturing an unspecified product to be processed this time when the product is within the imaging range during the product registration operation when the unspecified product is a product for which a code has not been obtained. The captured image displayed in the target captured image area AR43-1 may be the same as the captured image G12 displayed on the situation confirmation screen P10, for example.
In the captured image area AR43, the next captured image area AR43-2 is an area where the next captured image is displayed. The next captured image is a captured image obtained during a product registration operation performed subsequent to the product registration operation corresponding to the product for which the code has not been acquired, which is the current processing target. In other words, the next captured image is the image captured at the time of the first product registration operation in which the product code was successfully read and the product was identified, after the product registration operation corresponding to the product for which the code has not been acquired. It is an image.
Among the captured images obtained in response to the next product registration operation, if there is a captured image in which an article as a product is detected, the captured image containing the detected product is selected as the next captured image. Ru.

If a customer performs a product registration operation for a certain product and the customer terminal device 50 is unable to properly read the product code and the product is registered as a product with no code obtained, the customer may register the same product again. Often attempts to register products. In such a case, the customer will perform the product registration operation twice in a row for the same product. In such a case, if the product code is successfully read by the second product registration operation, the same product will be registered as a pending product for which the code has not been obtained, and the product code will be registered based on the read product code. This will result in double registration with the normal registration in which the product was identified.
If double registration is performed in this way, the procedure for releasing the hold by the store staff is to delete the registration of the held product as a product for which the code has not been obtained, and to leave the normal registration for the same product. Bye. In order to perform such processing smoothly, it is required that the store clerk be able to easily understand whether or not a product for which the code for which the hold is to be released has been registered twice.

Therefore, in this embodiment, a captured image area AR43 (target captured image area AR43-1, next captured image area AR43-2) is arranged on the product information input screen P40 to display the target captured image and the next captured image. That's what I do.
If the target captured image and the next captured image show the same product being captured, the store clerk can confirm that double registration has been performed. Therefore, the store clerk in this case operates, for example, the cancel button BT42 arranged on the product information input screen P40. As a result, the products to be processed whose codes have not been acquired are deleted from the list of registered products.

Further, in the next captured image area AR43-2 in the figure, a quantity display section NM is arranged. The quantity display section NM is an image section that shows the quantity (registered quantity) registered, for example, by a customer's operation, for a product that was successfully registered by the product registration operation when the next captured image was captured.
By arranging the quantity display section NM, at the stage of performing a hold release operation, a store clerk can grasp the registered quantity of a product that has been successfully registered in duplicate with an unspecified product to be released from hold. Able to accurately perform work and operations related to release from hold.

Note that if the product to be processed is an unspecified product as a NON-FILE (classification number 1), the customer terminal device 50 may successfully read the product code but not identify the product. will be announced. In this case, since the customer can know that the product code has been read successfully, there is a low possibility that double registration will occur by performing the product registration operation for the same product again. Therefore, if the registration result of the product to be processed corresponds to NON-FILE, the captured image area AR43 (image to be processed, next captured image) may not be displayed.
However, some customers may misunderstand that the product code has failed to be read even if a notification is made that corresponds to the non-registration result, for example as a NON-FILE, and may end up registering the product again. . For this reason, even if the registration result of the product to be processed corresponds to NON-FILE, the captured image area AR43 (image to be processed, next captured image) may be displayed. If the captured image area AR43 is displayed, the store clerk can easily understand that the unregistered NON-FILE results for the same product have been registered twice.

[Processing procedure example]
With reference to the flowchart of FIG. 17, an example of a processing procedure executed by the customer terminal device 50 and the transaction management device 60 of this embodiment in connection with product registration will be described. The process shown in FIG. 9 corresponds to FIG. 9, and more specifically shows the process of steps S5 to S11 as including the process corresponding to the registration result of an unspecified product.

First, an example of a processing procedure executed by the customer terminal device 50 will be described.
Step S101: With the product registration screen displayed, the customer terminal device 50 determines whether or not the product code has been successfully read.

Step S102: If the product code has been successfully read, the customer terminal device 50 further determines that the product registration result corresponding to the previous product registration operation is code not obtained result (reading NG (classification number 2), false scan input (classification No. 3) or non-scan input (classification No. 4).
The code not obtained result corresponds to a case where a pending product is registered as a code not obtained product as a product registration result.

Step S103: If it is determined in step S102 that the product registration result corresponding to the previous product registration operation is a code-unacquired result, the customer terminal device 50 selects one of the captured images captured by the imaging unit 506. , during the period corresponding to the current product registration operation, acquire (extract) a captured image in which the product is stored. The captured image acquired by the customer terminal device 50 in step S103 is the captured image in the next captured image area AR43-2 of the product information input screen P40, which targets the product whose code has not been acquired and was registered in the previous product registration operation to be released from hold. It will be used as the next captured image.

Note that the format (video, still image) of the captured image acquired by the customer terminal device 50 in step S103, the number of captured images, the time of the captured image to be acquired, the period, etc. of the detected product for which the code has not been acquired may be made different depending on the classification.

Step S104: If it is determined in step S102 that the result is not a code not obtained, or after the process in step S103, the customer terminal device 50 transmits product registration information.
The product registration information includes the product code read in response to step S101 and cart identification information. After that, when the process reaches step S104 after passing through step S103, the customer terminal device 50 transmits product registration information further including the captured image acquired in step S103. Further, in the case where it is determined in step S102 that the result is not a code not obtained, and the product registration result corresponds to an unspecified product of NON-FILE (classification number 1), the customer terminal device 50 Product registration information indicating that the product is registered as FILE (classification number 1) is transmitted.

Step S105: In response to receiving the registration completion notification transmitted from the transaction management device 60 in response to the transmission of the product registration information in step S102, the customer terminal device 50 registers the product as illustrated in FIG. Update the registration screen.

Step S106: If it is determined in step S101 that the code was not read normally, or after the process in step S105, the customer terminal device 50 determines that the product registration result corresponding to the current product registration operation is the code not obtained result. It is determined whether or not it was. The code not obtained result is a product registration result that corresponds to one of the pending products: NG reading (classification number 2), pseudo-scanning (classification number 3), and non-scanning (classification number 4).

First, the customer terminal device 50 is in a state where it is recognized that the entire barcode of the product is captured in the captured image, but a certain period of time has elapsed without successfully completing (successfully) reading the barcode. In this case, it is determined that a code not obtained result as reading NG (classification number 2) has occurred.

In addition, when the product is recognized in the captured image but the barcode reading is not completed normally because the barcode cannot be recognized at all, the customer terminal device 50 performs a pseudo scan (classification number 3). It is determined that a code not obtained result has occurred.
Alternatively, if the customer terminal device 50 is unable to successfully read the barcode because it can only recognize a portion of the product's barcode in the captured image, a code not obtained result as a pseudo scan may occur. It is determined that
Alternatively, the customer terminal device 50 may recognize the barcode of the product in the captured image, but the barcode reading may not be completed normally because the product was moving too fast. It is determined that a code not obtained result has occurred.
A code-unrecovered result that is classified as a pseudo-scan is when a customer places a product in their shopping cart simply by pretending to scan the product's barcode. There is a possibility that there are.

Furthermore, the customer terminal device 50 set in the shopping cart can take an image of the contents of the shopping cart. When the customer terminal device 50 detects that a new product has been added to the shopping cart even though no attempt is made to read the barcode of the product in the captured image. In this case, it is determined that a code not obtained result as a non-scan input (classification number 4) has occurred.
Note that, for example, a customer may move already registered products within the shopping cart in order to organize the contents of the shopping cart. The customer terminal device 50 may be configured to be able to identify, based on the captured image, such movement of the product and the additional insertion of a new product corresponding to the result of non-scanning and no-code acquisition.
The occurrence of code not obtained results that are classified as non-scanned items is due to fraud in which the customer places the item in the shopping cart without having the customer terminal device 50 read the barcode of the item. There is a possibility that there are.

In addition, in the case where it is determined in step S106 that the product registration result is not a code-unacquired result, the customer terminal device 50 analyzes the captured image obtained by the imaging unit 506, and the product registration operation is performed. This applies if it is determined that this has not been done.

Step S107: If it is determined in step S106 that the code has not been obtained, the customer terminal device 50 selects an image taken during the period corresponding to the current product registration operation from among the images taken by the imaging unit 506. Acquire (extract) an image.
The captured image acquired by the customer terminal device 50 in step S107 is the captured image G11 displayed on the status confirmation screen P10 when the code-unacquired product registered in the current product registration operation is targeted for release of hold. , G12, and G13. In addition, the captured image acquired by the customer terminal device 50 in step S107 is the target captured image on the product information input screen P40 when the code-unacquired product registered in the current product registration operation is targeted for release of hold. It may also be used as a target captured image of area AR43-1.

Step S108: The customer terminal device 50 transmits to the transaction management device 60 product registration information indicating the product registration result as the code not obtained this time. Furthermore, the product registration information transmitted in step S108 includes the captured image acquired in step S107.

Step S109: The customer terminal device 50 updates the product registration screen in response to receiving the registration completion notification transmitted from the transaction management device 60 in response to the transmission of the product registration information in step S108.

Step S110: If it is determined in step S106 that the result is not a code not obtained, or after the process in step S109, the customer terminal device 50 determines whether or not a payment transfer instruction operation has been received.
If the payment transfer instruction operation has not been accepted, the process returns to step S101.

Step S111: Upon receiving the payment transfer instruction operation, the customer terminal device 50 converts the cart identification information into a two-dimensional code, and displays the generated two-dimensional code on the code display screen illustrated in FIG. 11(D), for example. 504.

Next, an example of a processing procedure executed by the transaction management device 60 will be described.
Step S201: The transaction management device 60 determines whether the product registration information transmitted from the customer terminal device 50 in step S104 or step S108 has been received.

Step S202: When the product registration information is received, the transaction management device 60 specifies the cart information indicated by the cart identification information included in the received product registration information.

Step S203: The transaction management device 60 updates the cart information specified in step S202. That is, the transaction management device 60 generates registered product information or pending product information based on the product code included in the product registration information received in response to step S201. Transaction management device 60 adds the generated registered product information or pending product information to the cart information specified in step S202. As a result, one product is registered in the cart information.
Furthermore, if the product registration information received in step S201 includes a captured image, the transaction management device 60 associates the captured image with the registered product information or pending product information added to the cart information. memorize it.

Step S204: After the processing in step S203, the transaction management device 60 transmits a registration completion notification to the customer terminal device 50.

In the process shown in the figure, the customer terminal device 50 selects captured images that are not sent to the transaction management device 60 (i.e., images that will not be used as target captured images or next captured images) among captured images obtained through regular imaging. images) may be deleted at a predetermined timing. For example, if a product is identified and registered twice in a row, it is determined that the captured image corresponding to the first registration will not be used as the next captured image. Therefore, the customer terminal device 50 may delete the captured image corresponding to the first registration at the timing when the product is identified and registered twice in a row. Alternatively, the customer terminal device 50 may delete the captured images that have not been transmitted to the transaction management device 60, for example, at the timing when the payment transfer instruction operation is performed and product registration for all products is completed.

In addition, in the shopping system of the present embodiment, the customer terminal device 50 does not transmit product registration information each time a product is registered, but transmits product registration information when product registration for all products is completed. The product registration information for all the products that have been registered may be sent at once. At this time, the customer terminal device 50 transmits the product registration information including the captured image to be used as the target captured image and the next captured image.
When transmitting product registration information all at once in this manner, the customer terminal device 50 may send captured images (target captured images, target captured images, (a captured image that will not be used as the next captured image) may be deleted.

With reference to the flowchart of FIG. 18, an example of a processing procedure executed by the payment device 40 in connection with release of the hold of a held product will be described.
Step S301: The payment device 40 prevents the store clerk from logging in to release the reservation while displaying the payment screen including the list items of products on hold on the customer side display section 405, as illustrated in FIG. stand by.

Step S302: In response to the store clerk logging in, the payment device 40 changes the display on the customer side display section 405 from the payment screen to the pending product processing screen (FIG. 14).

Step S303: While the pending product processing screen is displayed, the payment device 40 determines whether an unspecified record designation operation has been performed. The unspecified record designation operation is an operation on the hold release declaration button BT31 placed in the list item corresponding to the unspecified product among the list items of the held product (unspecified record) in the held product list area AR31.

Step S304: When the unspecified record designation operation is performed, the payment device 40 displays a status confirmation screen P10 on the customer side that makes the unspecified product corresponding to the list item specified by the unspecified record designation operation the target of the hold release process. It is displayed on the display unit 405.

Step S305: With the status confirmation screen P10 displayed, the payment device 40 waits for a screen transition trigger to the product information input screen P40 to occur. The screen transition trigger is acceptance of an operation on the screen transition button BT35 on the status confirmation screen P10, or completion (success) of reading the product code by the payment device 40.

Step S306: When the screen transition trigger occurs, the payment device 40 displays a product information input screen P40 for the unspecified product to be processed.
Step S307: The payment device 40 determines that the product registration information (product code, product name, price, quantity, etc.) input operation has been performed by the store clerk while the product information input screen P40 is being displayed. Enter product registration information accordingly.
Step S308: The payment device 40 transmits registered product information or a cancellation request to the transaction management device 60 in response to the operation of the registration button BT41 or the cancellation button BT42. The registered product information is transmitted in response to the operation of the registration button BT41, and has content that reflects the information input into the product information area AR41. The cancellation request is a command that is sent in response to the operation of the cancellation button BT42 and requests that the pending product information to be processed stored in the cart information be placed in a canceled state.

Step S309: If it is determined in step S303 that the unspecified record designation operation has not been performed, the payment device 40 determines whether or not a pending product confirmation operation has been performed. The pending product confirmation operation is an operation on the pending release declaration button BT31 in the list item corresponding to the pending product that is not an unspecified product. In other words, the held products that are not unidentified products are age-confirmed products (classification number 5), medicines (classification number 6), and security tag products (classification number 7), which have been identified. For these held products that are not unspecified products, the store clerk performs a predetermined service and then operates the hold release declaration button BT31 of the corresponding list item to release the held products.

Step S310: If the held product confirmation operation has been performed, the settlement device 40 transmits a hold release request corresponding to the held product for which the current held product confirmation operation has been performed to the transaction management device 60.

Step S311: The transaction management device 60 updates the corresponding cart information in response to receiving the registered product information or cancellation request transmitted from the payment device 40 in step S308. That is, when the transaction management device 60 receives registered product information, it changes the pending product information of the target product to product registration information based on the received registered product information. At this time, the transaction management device 60 may change, for example, the hold release flag included in the product information to indicate that the hold has been released.
Further, when the transaction management device 60 receives a cancellation request, it changes the pending product information of the target product to indicate that it has been cancelled. In addition, in response to receiving the hold release notification transmitted from the settlement device 40 in step S310, the transaction management device 60 changes the product information type so that the held product information of the target product in the cart information is managed as registered product information. change. In this case, the transaction management device 60 may change, for example, the hold release flag included in the product information to indicate that the hold has been released.
After updating the cart information as described above, the transaction management device 60 transmits settlement information corresponding to the updated cart information to the settlement device 40. The payment device 40 receives the transmitted payment information.

Step S312: The settlement device 40 determines whether or not the hold has been released for all the held products. If there are still held products that have not been released from hold, the process returns to step S302.
When the process returns from step S312 to step S302, the payment device 40 displays a pending product processing screen updated with the content of the payment information received in step S311.

Step S313: If it is determined in step S312 that the hold has been released for all held products, the payment device 40 displays a payment screen that reflects the content of the payment information received in step S311. do. Since no list items for pending products are displayed in the registered product list area AR3 of the payment screen displayed in step S312, the payment device 40 can execute the payment process in accordance with the customer's subsequent operations. state.

<Modified example>
Modifications of this embodiment will be described below. The modified examples described below may be combined with the above embodiment and other modified examples as appropriate.

[First modification]
In the embodiment described above, the transaction management device 60 identifies the product based on the product code read by the customer terminal device 50. Product identification is not limited to such processing. For example, the customer terminal device 50 may send a captured image of a product to the transaction management device 60, and the transaction management device 60 may identify the product by comparing the received captured image with a database of product images. It's okay to be.

[Second modification]
The next captured image may also be displayed on the situation confirmation screen P10, for example. FIG. 19 shows an example of the situation confirmation screen P10 corresponding to this modification. In the status confirmation screen P10 of the same figure, in addition to the captured images G11, G12, and G13 taken when the product registration operation was performed for an unspecified product, a next captured image G21 is arranged.
In this case, the store clerk can determine whether or not double registration has occurred at the stage when the status confirmation screen P10 is displayed before the product information input screen P40 is displayed.
Note that the arrangement of the captured images G11, G12, G13 and the next captured image G21 on the situation confirmation screen P10 is not limited to the example shown in the figure. For example, the next captured image G21 may be placed above or below the area where the captured images G11, G12, and G13 are placed. Alternatively, it may be arranged so as to fall within the range of any one of the captured images G11, G12, and G13. Alternatively, for example, on the status confirmation screen P10 where the captured images G11, G12, and G13 are displayed but the next captured image G21 is not displayed, for example, when operating a button to display the next captured image, etc. Accordingly, the next captured image may be displayed.
Note that a quantity display section NM similar to that illustrated in FIG. 16 may be arranged in correspondence with the next captured image G21 shown in FIG. 19.

[Third modification]
On the checkout screen (Figure 13) or the pending product processing screen (Figure 14), for example, the list item of pending products as unspecified products (unspecified record) will be registered successfully for the first time in the subsequent product registration operation, regardless of the sort order. The items may be arranged in the same order as the list items (normally registered records) of the products that have been registered.
FIG. 20 shows an example of a payment screen corresponding to this modification. In the figure, out of unspecified products, products for which codes have not been obtained (pending products that fall under any of the following categories: NG reading (classification number 2), pseudo-scanning (classification number 3), or non-scanning (classification number 4)) The list items are arranged in the same order as the list item of the product that was successfully registered first after the registration of the product for which the code has not been obtained.
In the figure, an example is shown in which the list items of products whose codes have not been obtained are placed first, and the list items of products that have been successfully registered first after the registration of the products whose codes have not been obtained are arranged later. In addition, in the figure, the list items of products whose codes have not been obtained and the list items of normally registered products are surrounded by one frame FL. This makes it easier to visually understand the combination of list items for products whose codes have not been obtained and list items for products that have been successfully registered, which may be double-registered.

[Fourth modification]
In the embodiment described above, in response to the suspension of a suspended product being released, the corresponding list item displayed in the suspended product list area AR31 of the suspended product processing screen is changed, for example, in the display on the suspension release declaration button BT31. This was to indicate that the hold had been released. In this case, the list item of the held product targeted for release from hold is arranged at the same position in the held product list area AR31 even after the hold is released.
However, in response to the hold being released, the list items of the held products are moved to the registered product list area AR3 outside the held product list area AR31, and are treated as normal registration records in the currently set arrangement order (sort order). ) may be arranged according to

[Fifth modification]
In the embodiment described above, the captured image corresponding to the first product registration operation in which the product was identified after the unspecified product was registered is displayed as the next captured image.
In contrast, in this modification, for example, even if the next product registration operation after an unspecified product is registered results in a registration result indicating that the product is an unspecified product, the next product registration operation will be Sometimes, the corresponding captured image can be displayed as the next captured image. Alternatively, even if a product registration operation performed within a certain period of time after an unspecified product has been registered results in a registration result indicating that the product is an unspecified product, this will be handled during the next product registration operation. The captured image may be displayed as the next captured image.
For example, if a customer performs a product registration operation for a certain product and the product is registered as an unspecified product, when the customer performs the product registration operation for the same product again, the same product is registered as an unspecified product. There are cases like this. That is, when registering a product, there are cases where the same product is registered twice so that the results of registration as an unspecified product overlap. With the configuration of this modified example, it becomes easier for store staff to understand such double registration and take prompt countermeasures.
Additionally, if the next product registration operation after an unspecified product is registered as described above results in a registration result indicating that the product is an unspecified product, or within a certain period of time after the unspecified product is registered, If the performed product registration operation results in a registration result indicating that the product is an unspecified product, the transaction management device 60 may, for example, register the same product as one unspecified record. You can process it as you like.

[Sixth variation]
In addition, after the customer terminal device 50 regularly transmits the captured image obtained by the imaging unit 506 to the transaction management device 60, the transaction management device 60 side records the time and classification of the code not obtained result. Depending on the result, image analysis result, etc., a captured image corresponding to the code-unacquired result may be acquired.

[Seventh modification]
Further, the transaction management device 60 is configured to store captured images regularly transmitted by the customer terminal device 50, and the customer terminal device 50 or the transaction management device 60 determines whether the code has not been obtained based on the captured image. In response to the determination that a code has not been obtained, for example, a flag or identification information may be added to the corresponding captured image, so that the corresponding captured image is managed as one corresponding to the code not obtained result.

[Eighth modification]
Further, the next captured image to be displayed on the payment device 40 may be displayed by a store clerk's search. For example, when a store clerk selects one unspecified product as a target for release from hold, the clerk may perform an operation on the checkout device to instruct a search for a captured image corresponding to the first normally registered product after the unspecified product is registered. Do this for 40. The settlement device 40 requests the transaction management device 60 to search for the instructed captured image. The transaction management device 60 executes a search in response to a request, and sends the searched captured image to the settlement device 40. The payment device 40 displays the received captured image as the next captured image.

[Ninth modification]
Furthermore, based on the results of analyzing the changes in the captured image obtained by imaging the inside of the shopping cart by the customer terminal device 50, it is possible to determine whether the registered product is not placed in the shopping cart or because of a person accompanying the customer, etc. The customer terminal device 50, transaction management device 60, etc. may be able to detect the occurrence of a situation in which a product is placed in a shopping cart without being registered.
Further, if the occurrence of the above situation is detected, the customer terminal device 50 may ask the customer to place the registered item in the shopping cart, or the customer may place the registered item in the shopping cart without registering the item. For example, a display may be used to guide the user to register the products included in the product.

[10th modification]
On the product information input screen P40 in FIG. 16, it may be possible to search for products registered in the corresponding transaction. Furthermore, the search that is possible in this modification may be a vague search that searches for products that "contain" the input keyword.
For example, in the product information input screen P40 of this modification, a button for instructing a fuzzy search is arranged. When the button is operated, a keyboard for character input on which, for example, 50 syllables (or alphabets) are arranged is displayed in the input operation area AR42. The store clerk performs an operation to input a keyword, which can be a part of the name of the product to be searched, on the keyboard for character input. At this time, it may be possible to convert hiragana characters input as keywords into words. Then, the store clerk operates a search execution button arranged on the product information input screen P40.
In response to the operation of the search execution button, the transaction status management device 80 executes a vague search and displays the search results. The search results may be displayed as a pop-up screen, for example. Further, in the search results, the product name, price, number of registered items, etc. for each searched product may be displayed.
The store clerk identifies unspecified products to be processed this time from among the products presented as the search results, and performs an operation to select the identified products. Depending on the operation, the display may return to the product information input screen P40, for example. In the displayed input operation area AR42, the product name, price, registered number, product code, etc. of the selected product are input. The clerk operates the registration button BT41 after confirming the content displayed in the input operation area AR42. This makes it possible for store staff to efficiently release hold on unspecified products without having to enter a large number of digits as a product code or narrow down the search based on product classification, for example. Become.

[Eleventh modification]
For example, when a target unspecified product is registered through an operation on a screen such as the status confirmation screen P10 in FIG. 15 (and FIG. 19) or the product information input screen P40 in FIG. A moving image (or a time-lapse image) may be displayed. The operation for displaying a video image may be, for example, an operation on a video display button placed on the status confirmation screen P10, the product information input screen P40, etc., or, for example, the captured image G12 on the status confirmation screen P10 or the product information input screen The operation may be performed on the target captured image area AR43-1 or the next captured image area AR43-2 in P40.
For example, when the next captured image is displayed, a video is displayed from when the product registration operation is started when the target unspecified product is registered until the product corresponding to the next captured image is captured. You may choose to do so. Note that the moving image may be deleted, for example, upon completion of payment. However, if for some reason the transaction is terminated without payment being completed, the video image will be saved for a certain period of time after the transaction so that the store clerk can check it even after the transaction is completed. You may be allowed to do so. Furthermore, the moving image may be stored in the transaction management device 60 or the transaction status management device 80 in association with the date, member identification information, and the like. The saved moving images can be played back on a terminal such as the transaction status management device 80.

[12th modification]
In the above-described embodiments and modified examples, a part of the processing executed by the customer terminal device 50, the transaction status management device 80, etc., is performed by a higher-level device such as the transaction management device 60 that is connected to the customer terminal device 50 via communication. may be executed. Further, in the above embodiments and modifications, a part of the processing executed by the settlement device 40 may be executed by a higher-level device such as the transaction management device 60 that is connected to the settlement device 40 via communication.
As one of the aspects described above, the shopping system of this embodiment may include a thin client system configuration.
As an example, the transaction management device 60 may be configured as a thin client compatible server (for example, a cloud server), and the customer terminal device 50, transaction status management device 80, etc. may function as a thin client. In this case, the thin client terminals such as the customer terminal device 50 and the transaction status management device 80 transmit information such as captured images and commands corresponding to operations performed on the thin client terminals to the transaction status management device 80. The transaction status management device 80 may execute processing in response to information received from the thin client terminal, and may transmit image data or web page data as a screen or the like on which the processing results are reflected to the thin client terminal. The customer terminal device 50, transaction status management device 80, etc. as thin client terminals display the transmitted image data and web page data using an image viewer or a web browser.

<Summary of embodiments>
[Technical field]
The present invention relates to a product sales data processing system, a payment device, and a program.
[Background technology]
2. Description of the Related Art A system is known in which a customer registers a product by himself/herself using a mobile terminal (see, for example, Patent Document 1).
[Prior art documents]
[Patent document]
[Patent Document 1] Japanese Patent Application Publication No. 2020-067947 [Summary of the invention]
[Problem to be solved by the invention]
In a product sales data processing system in which customers themselves register products using mobile terminals, an unspecified state may occur in which the product to be registered cannot be specified. For products that are in an unspecified state, the unspecified state is resolved by, for example, an operation by a store clerk. It is preferable that such an unspecified state be resolved efficiently.
The present invention has been made in view of the above circumstances, and is intended to efficiently resolve the unspecified state of products in a product sales data processing system in which customers themselves register products using mobile terminals. The purpose is to

[Means to solve the problem]
(1) As explained above, one aspect of the present embodiment is a product sales data processing system in which a customer registers a product using a terminal device (for example, customer terminal device 50). , an imaging means (for example, the imaging unit 506) that takes an image of a product to be registered; a product specifying means that identifies a product to be registered based on a captured image obtained by the imaging means; and the product specifying means. When an unidentified result in which the product cannot be identified occurs, a captured image corresponding to the product first identified by the product identifying means after the timing at which the unidentified result occurs is used to correspond to the unidentified result. The product sales data processing system includes a storage means (step S203) for storing as an auxiliary image (for example, a next captured image).

According to the above configuration, in response to the registration of a pending product as an unspecified product, the image corresponding to the registration result in which the product is identified for the first time after the unspecified product is registered is set as the next captured image. can be obtained. By displaying the next captured image at a predetermined opportunity, it is possible to efficiently resolve the unspecified state of the product (for example, release of the hold on a pending product in the unspecified state).

(2) One aspect of the present embodiment is the product sales data processing system described in (1), in which the product sales data processing system stores the product as registration result information indicating the result of product registration by a customer using a terminal device. A list of registration result information (e.g., a pending product The registered product list area AR3) on the processing screen is displayed, and in response to unspecified information being specified from among the registration result information shown in the registration result information list, unspecified information corresponding to the specified unspecified information is displayed. A captured image when a result occurs (for example, the target captured image in the target captured image area AR43-1) and an auxiliary image corresponding to the unspecified result indicated by the specified unspecified information (for example, the next captured image area AR43) -2 next captured image).

According to the above configuration, the target captured image obtained by capturing the status of the product registration operation corresponding to the unspecified product targeted for release of hold and the next captured image corresponding to the unspecified product can be displayed. Can be done. Thereby, for example, by looking at the target captured image and the next captured image, a store clerk can easily determine whether or not a customer is registering the same product twice.

(3) One aspect of the present embodiment is the product sales data processing system described in (2), wherein the display control means is configured to arrange the unspecified information in a consecutive order in the registration result information list. to be displayed.

According to the above configuration, in the registered product list area AR3 of the pending product processing screen, it is possible to form a pending product list area AR31 in which list items of unspecified products are arranged in order from the top, for example. This makes it easier for store staff to release unspecified products from hold.

(4) One aspect of the present embodiment is the product sales data processing system described in (2), wherein in the registration result information list, the unspecified information and the unspecified results corresponding to the unspecified information are After the timing of occurrence, the product is displayed so that it is arranged in the order of the specific registration information about the product first specified by the product specifying means.

According to the above configuration, in the registered product list area AR3 of the pending product processing screen, the list item of unspecified products is changed to the list of products that are first identified after the product registration corresponding to the unspecified product is performed. Items can be arranged so that the items and arrangement order are consecutive. As a result, for example, in the registered product list area AR3, a list item for an unspecified product and a list item for a product that may have been registered twice corresponding to the unspecified product are arranged side by side. This will improve the efficiency of the work of store staff in releasing unspecified products from hold.

(5) One aspect of the present embodiment is the product sales data processing system according to any one of (2) to (4), in which the display control means is configured to display the auxiliary image along with the auxiliary image. The registered quantity of the corresponding product is displayed (for example, the quantity display section NM is displayed).

According to the above configuration, the quantity display section NM can be displayed in correspondence with the next captured image. This allows store staff to grasp the quantity of products that have been successfully registered in duplicate with unspecified products that are subject to release from hold, and perform work and operations related to release from hold. Can be done accurately.

(6) One aspect of the present embodiment is the product sales data processing system according to any one of (2) to (5), in which editing means for editing the unspecified information (for example, product information input screen P40), the display control means is configured to change the information from unspecified information in the registration result information list in response to editing by the editing means to change the unspecified information to specific registered information. The specified registration information is arranged in an arrangement order according to a predetermined arrangement rule for registration result information.

According to the above configuration, in the registered product list area AR3 in which the list items of pending products are arranged in the order in which they are arranged, the list item corresponding to the product whose pending product is released is set as the list item of the normally registered product. You can rearrange them according to the order of arrangement.

(7) One aspect of the present embodiment is a payment device (40) in a product sales data processing system in which a customer registers a product using a terminal device (for example, customer terminal device 50). , information acquisition means for acquiring registration result information regarding product registration results performed using the terminal device; and information acquisition means for acquiring registration result information regarding product registration results performed using the terminal device; When registering a product, a list of registration result information shows specific registration information about the specified product and unspecified information indicating unspecified results in which the product was not specified (for example, registered product information on the pending product processing screen). List area AR3) is displayed, and in response to the unspecified information being specified from among the registration result information shown in the registration result information list, the specified unspecified information (for example, target captured image area AR43- 1 target captured image) and information regarding the first product identified after the unspecified result corresponding to the specified unspecified information occurs (for example, the next captured image in the next captured image area AR43-2). This is a payment device equipped with display control means for displaying information.

(8) One aspect of the present embodiment is that a computer is used as a payment device in a product sales data processing system in which a customer registers a product using a terminal device. an information acquisition means for acquiring registration result information regarding the registration result of a product that has been registered, and a product that is identified in the product registration performed by a customer using a terminal device based on the registration result information acquired by the information acquisition means; displaying a registration result information list showing specific registration information for , and unidentified information indicating an unidentified result in which no product was identified; Display control means for displaying specified unspecified information and information regarding a product that is first specified after an unspecified result corresponding to the specified unspecified information is generated in response to the specified information. This is a program to make it function as a.

Note that programs for realizing the functions of the management device 10, payment device 40, customer terminal device 50, transaction management device 60, transaction status management device 80, etc. in each of the above embodiments are stored in a computer-readable recording medium. The above-mentioned management device 10, settlement device 40, customer terminal device 50, transaction management device 60, transaction status management device 80, etc. You may also perform processing as follows. Here, "reading a program recorded on a recording medium into a computer system and executing it" includes installing the program on the computer system. The "computer system" here includes hardware such as an OS and peripheral devices. Further, a "computer system" may include a plurality of computer devices connected via a network including the Internet, a WAN, a LAN, a communication line such as a dedicated line, etc. Furthermore, the term "computer-readable recording medium" refers to portable media such as flexible disks, magneto-optical disks, ROMs, and CD-ROMs, and storage devices such as hard disks built into computer systems. In this way, the recording medium storing the program may be a non-transitory recording medium such as a CD-ROM. The recording medium also includes a recording medium provided internally or externally that can be accessed from the distribution server to distribute the program. The code of the program stored in the recording medium of the distribution server may be different from the code of the program in a format executable by the terminal device. That is, as long as it can be downloaded from the distribution server and installed in an executable form on the terminal device, the format in which it is stored on the distribution server does not matter. Note that a configuration in which a program is divided into a plurality of parts and downloaded at different timings and then combined on a terminal device, or a distribution server that distributes each of the divided programs may be different. Furthermore, a ``computer-readable recording medium'' refers to a storage medium that retains a program for a certain period of time, such as volatile memory (RAM) inside a computer system that serves as a server or client when a program is transmitted via a network. This shall also include things. Moreover, the above-mentioned program may be for realizing a part of the above-mentioned functions. Furthermore, it may be a so-called difference file (difference program) that can realize the above-mentioned functions in combination with a program already recorded in the computer system.

10 management device, 40 payment device, 50 customer terminal device, 60 transaction management device, 70 credit card payment server, 80 transaction status management device

Claims (10)

A system in which a customer registers a product using a terminal device,
an imaging means for imaging the product to be registered;
a product specifying means for specifying a product to be registered based on a captured image obtained by capturing the image with the image capturing means;
When the product identification means produces an unidentified result in which the product cannot be identified, a captured image when the product that caused the unidentified result is within the imaging range, and an image captured before the captured image. A system comprising: an image; and a storage means for storing the image. A system in which a customer registers a product using a terminal device,
an imaging means for imaging the product to be registered;
a product specifying means for specifying a product to be registered based on a captured image obtained by capturing the image with the image capturing means;
When the product identification means produces an unidentified result in which the product cannot be identified, a captured image when the product with the unidentified result is within the imaging range, and an image captured after the captured image. A system comprising storage means for storing and. The registration result information indicating the result of product registration by the customer using the terminal device includes specific registration information about the product specified by the product specifying means and unspecified information indicating the unspecified result. Display a list of registration result information,
When, in response to unspecified information being specified from among the registration result information shown in the registration result information list, a product for which an unspecified result corresponding to the specified unspecified information has occurred is within the imaging range; The system according to claim 1, further comprising display control means for displaying a captured image and a captured image captured before the captured image. The registration result information indicating the result of product registration by the customer using the terminal device includes specific registration information about the product specified by the product specifying means and unspecified information indicating the unspecified result. Display a list of registration result information,
When, in response to unspecified information being specified from among the registration result information shown in the registration result information list, a product for which an unspecified result corresponding to the specified unspecified information has occurred is within the imaging range. The system according to claim 2, further comprising a display control means for displaying a captured image and a captured image captured after the captured image. A payment device in a system in which a customer registers a product using a terminal device,
If the terminal device identifies the product to be registered based on the captured image obtained by imaging the product to be registered using the imaging means, and an unidentified result in which the product cannot be identified is generated, the unidentified result is A payment device comprising: a storage means for storing a captured image when a produced product is within the captured image and a captured image captured before the captured image. A payment device in a system in which a customer registers a product using a terminal device,
If the terminal device identifies the product to be registered based on the captured image obtained by imaging the product to be registered using the imaging means, and an unidentified result in which the product cannot be identified is generated, the unidentified result is A payment device comprising: a storage means for storing a captured image when a produced product is within the captured image and a captured image captured after the captured image. A computer is used as a payment device in a system where customers register products using terminal devices.
If the terminal device identifies the product to be registered based on the captured image obtained by imaging the product to be registered using the imaging means, and an unidentified result in which the product cannot be identified is generated, the unidentified result is A program for functioning as a storage means for storing a captured image when a generated product is within the captured image and a captured image captured before the captured image. A computer is used as a payment device in a system where customers register products using terminal devices.
If the terminal device identifies the product to be registered based on the captured image obtained by imaging the product to be registered using the imaging means, and an unidentified result in which the product cannot be identified is generated, the unidentified result is A program for functioning as a storage means for storing a captured image when a produced product is within an image capturing range and a captured image captured after the captured image. A system in which a customer registers a product using a terminal device includes a first program corresponding to a first computer and a second program corresponding to a second computer, the first program being , for functioning as a first program for functioning as a product specifying means for specifying a product to be registered based on a captured image obtained by capturing an image of the product to be registered by an imaging device;
The second program is configured to display a captured image when the product that caused the unspecified result is within the imaging range, when the product identification means generates an unspecified result in which the product cannot be identified, and a captured image that is larger than the captured image. A program that functions as a storage means for storing previously captured images. A system in which a customer registers a product using a terminal device includes a first program corresponding to a first computer and a second program corresponding to a second computer, the first program being , for functioning as a first program for functioning as a product specifying means for specifying a product to be registered based on a captured image obtained by capturing an image of the product to be registered by an imaging device;
The second program is configured to display a captured image when the product that caused the unspecified result is within the imaging range, when the product identification means generates an unspecified result in which the product cannot be identified, and a captured image that is larger than the captured image. A program that functions as a storage means for storing captured images captured later.

Images