JP6443184B2 - Checkout system, product registration device, checkout device, program, and checkout method - Google Patents

Description

  The present invention relates to a settlement system, a commodity registration apparatus, a settlement apparatus, a program, and a settlement method.

  At cash register terminals in supermarkets and other stores, (1) processing for registering products for payment (reading barcodes, etc.) and (2) processing for performing payments for registered products (such as receiving payment and returning a change) Two processes are performed.

  Then, there is an operation method in which the function of the cash register terminal is divided into two devices, and the processing of (1) and the processing of (2) are performed by different devices. For example, Patent Literature 1 discloses a self-POS system in which a registration device used for the process (1) and an accounting device used for the process (2) are separately provided. The store clerk registers the product using the registration device. The registration device images the customer by the imaging unit. The store clerk operating the registration device selects a customer image to be associated with the accounting data (data regarding the registered product) from the customer image captured by the imaging unit. As a result, the registration device associates the customer image with the transaction data. Thereafter, the customer performs accounting using the accounting device. The accounting apparatus captures the customer and acquires accounting data associated with the customer image that is highly consistent with the generated customer image. Then, the customer performs accounting based on this accounting data.

JP 2012-248162 A

  A customer who uses a registration device may be different from a customer who uses a payment device to check out a product registered by the registration device. For example, when a couple is shopping, a husband may use a registration device and a wife may use an accounting device. For this reason, it is difficult for the store clerk to determine the customer who uses the accounting apparatus or to automatically determine the customer using the apparatus.

  In the invention of Patent Document 1, a store clerk determines a customer image to be associated with information related to accounting. Therefore, customers other than the customer who uses the registration device cannot use the accounting device.

  The present invention has been made in view of the above problems. An object of the present invention is to provide a technique that allows a customer to flexibly perform settlement.

The settlement system of the present invention has a product registration device and a settlement device.
The product registration device includes: a first imaging unit that generates a first customer image including a customer; a first display control unit that displays the first customer image on a display device that can be browsed by the customer; A first input receiving unit that receives an input for selecting a first customer image; and a generating unit that generates related information in which the selected first customer image is associated with payment information used for payment of a product.
The settlement apparatus includes a second imaging unit that generates a second customer image including a customer, and an acquisition unit that acquires the settlement information associated with the first customer image corresponding to the second customer image and the related information. And having.

  The merchandise registration apparatus of the present invention is a merchandise registration apparatus included in the settlement system of the present invention.

  The settlement apparatus of the present invention is a settlement apparatus included in the settlement system of the present invention.

  The first program of the present invention is a program that causes a computer to operate as the commodity registration apparatus of the present invention.

  The second program of the present invention is a program that causes a computer to operate as the settlement apparatus of the present invention.

  The settlement method of the present invention is executed by a settlement system having a product registration apparatus and a settlement apparatus. In the settlement method, the product registration device generates a first customer image including a customer, and the product registration device displays the first customer image on a display device that the customer can view. A display control step, a first input receiving step in which the product registration device receives an input for selecting a first customer image of a customer who performs a checkout process, and the product registration device has the selected first customer image and product A generating step for generating related information associated with the checkout information used for checkout, a second imaging step in which the checkout device generates a second customer image including a customer, and the checkout device in the second customer image. And acquiring the settlement information associated with the first customer image corresponding to the information in the related information.

  ADVANTAGE OF THE INVENTION According to this invention, the technique which enables a customer to pay flexibly is provided.

It is a figure which illustrates arrangement | positioning of the goods registration apparatus and payment apparatus which a payment system has. FIG. 2 is a block diagram illustrating the information processing system according to the first embodiment. 3 is a flowchart illustrating processing executed by the product registration device according to the first embodiment. 3 is a flowchart illustrating a flow of processing executed by the settlement apparatus according to the first embodiment. It is a figure which illustrates the hardware constitutions of the computer which implement | achieves the goods registration apparatus of Embodiment 1. It is a figure which illustrates the hardware constitutions of the computer which implement | achieves the adjustment apparatus of Embodiment 1. It is a figure which illustrates payment information in a table format. It is a figure which illustrates the display in a display apparatus. It is a figure which illustrates the composition of association information in a table format. It is a block diagram which illustrates the information processing system of Embodiment 1 which has a 2nd display control part and the 2nd input reception part. It is a block diagram which illustrates the settlement apparatus which has a judgment part. It is a block diagram which illustrates the payment system of Embodiment 2. It is a figure which illustrates the correlation information in Embodiment 2. 10 is a flowchart illustrating a flow of processing executed by the warning device according to the second embodiment. It is a figure which illustrates the operating environment of the payment system of Embodiment 2. 10 is a flowchart illustrating a flow of processing executed by the warning device according to the third embodiment.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. In all the drawings, the same reference numerals are given to the same components, and the description will be omitted as appropriate.

[Embodiment 1]
FIG. 1 is a diagram exemplifying an arrangement of a product registration apparatus 3000 and a settlement apparatus 4000 that the settlement system 2000 has. The settlement system 2000 includes a product registration device 3000 and a settlement device 4000. As shown in FIG. 1, the merchandise registration device 3000 and the checkout device 4000 are provided as independent devices. In a store that uses the checkout system 2000, the checkout of the product is performed using the product registration device 3000 and the checkout device 4000. Here, the installation number of the merchandise registration apparatus 3000 and the settlement apparatus 4000 is 1 or more, respectively.

  The product registration device 3000 is used for processing for registering a product as a payment target. A customer who wants to purchase a product goes to the product registration apparatus 3000 with the product. A store clerk operating the product registration apparatus 3000 receives a product from a customer and registers the product as a payment target. Specifically, the store clerk causes a reader included in the product registration device 3000 to read a barcode or the like attached to each product. Along with the registration of the product, the product registration device 3000 generates settlement information. The checkout information is information related to the checkout process for a product registered as a checkout target.

  The settlement apparatus 4000 is used for settlement of merchandise registered as a settlement target (such as receiving payment). After the product registration is completed, the customer moves to an arbitrary checkout device 4000. The settlement apparatus 4000 acquires settlement information relating to the product purchased by the customer, and performs settlement processing based on the settlement information. Specifically, the checkout device 4000 presents the price to the customer, counts the price input by the customer, refunds the change to the customer, issues a receipt, and the like.

<Configuration of Product Registration Device 3000 and Checkout Device 4000>
FIG. 2 is a block diagram illustrating a checkout system 2000 according to the first embodiment. In FIG. 2, each block represents a functional unit configuration, not a hardware unit configuration.

  The product registration device 3000 includes a first imaging unit 3020, a first display control unit 3040, a first input receiving unit 3060, and a generation unit 3080. The first imaging unit 3020 generates an image including a customer. Hereinafter, this image is referred to as a first customer image. The 1st display control part 3040 displays a 1st customer image on the display apparatus 10 which a customer can browse. The 1st input reception part 3060 receives the input which selects the 1st customer image of the customer who performs checkout processing. The generation unit 3080 generates related information that associates the selected first customer image with the checkout information.

  The payment apparatus 4000 includes a second imaging unit 4020 and an acquisition unit 4080. The second imaging unit 4020 generates an image including the customer. Hereinafter, this image is referred to as a second customer image. The acquisition unit 4080 acquires the adjustment information associated with the first customer image corresponding to the second customer image using the related information. The “first customer image corresponding to the second customer image” is a first customer image that is considered to include the same customer as that shown in the second customer image. Further specific description regarding the “first customer image corresponding to the second customer image” will be described later.

<Action and effect>
In general, since there are a plurality of customers in a store, there is a possibility that a plurality of customers are reflected in an image captured by the first imaging unit 3020. Therefore, it may be difficult to determine which first customer image is the first customer image (the first customer image in which the customer is shown) that should be associated with the settlement information regarding the product purchased by a certain customer.

  Further, the customer who uses the settlement apparatus 4000 and the customer who uses the product registration apparatus 3000 are not necessarily the same. For example, when a couple is shopping, a husband may use a product registration device 3000 and a wife may use a checkout device 4000. For this reason, it is difficult for a store clerk to determine a customer who uses the checkout device 4000 or to automatically determine the customer using the device.

  When associating the settlement information with the first customer image showing the customer, the product registration device 3000 according to the present embodiment causes the customer to select the first customer image to be associated with the settlement information. Therefore, the customer who uses the product registration device 3000 not only displays the first customer image in which he / she is reflected but also the first customer image in which a customer other than himself / herself (for example, a family standing next to the customer) is reflected. You can also choose. Therefore, according to the settlement system 2000 of the present embodiment, the customer can flexibly perform settlement.

  Hereinafter, the present embodiment will be described in further detail.

<Process flow>
FIG. 3 is a flowchart illustrating the process executed by the product registration device 3000 according to the first embodiment. The first imaging unit 3020 generates a first customer image (S102). The first display control unit 3040 displays the first customer image on the display device 10 (S104). The first input receiving unit 3060 receives the selection of the first customer image (S106). The generation unit 3080 associates the selected first customer image with the checkout information and generates related information (S108).

  FIG. 4 is a flowchart illustrating the flow of processing executed by the settlement apparatus 4000 according to the first embodiment. The second imaging unit 4020 generates a second customer image (S202). The acquisition unit 4080 acquires the settlement information associated with the first customer image corresponding to the second customer image (S204).

<Example of Hardware Configuration of Product Registration Device 3000>
Each functional component of the product registration apparatus 3000 may be realized by hardware (eg, a hard-wired electronic circuit) that implements each functional component, or a combination of hardware and software (eg: It may be realized by a combination of an electronic circuit and a program for controlling it).

  FIG. 5 is a diagram illustrating a hardware configuration of a computer 1000 that implements the commodity registration apparatus 3000 according to the first embodiment. The computer 1000 may be mounted using a dedicated device such as a cash register terminal, or may be mounted using a general-purpose device such as a PC (Personal Computer) or a portable terminal.

  The computer 1000 includes a bus 1020, a processor 1040, a memory 1060, a storage 1080, an input / output interface 1100, and a network interface 1120. The bus 1020 is a data transmission path through which the processor 1040, the memory 1060, the storage 1080, the input / output interface 1100, and the network interface 1120 transmit / receive data to / from each other. However, the method of connecting the processors 1040 and the like is not limited to bus connection. The processor 1040 is an arithmetic processing unit such as a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit). The memory 1060 is a memory such as a RAM (Random Access Memory) or a ROM (Read Only Memory). The storage 1080 is a storage device such as a hard disk, an SSD (Solid State Drive), or a memory card. The storage 1080 may be a memory such as a RAM or a ROM.

  The input / output interface 1100 is an interface for connecting the computer 1000 and an input / output device. The input / output device is, for example, a keyboard or a mouse.

  The network interface 1120 is an interface for connecting the computer 1000 to an external device so as to be communicable. The network interface 1120 may be a network interface for connecting to a wired line or a network interface for connecting to a wireless line. For example, a computer 1000 configuring the product registration apparatus 3000 is connected to a computer 6000 configuring the settlement apparatus 4000 via a network.

  The storage 1080 stores program modules for realizing the first imaging unit 3020, the first display control unit 3040, the first input receiving unit 3060, and the generation unit 3080, respectively. The processor 1040 implements the functions of the corresponding functional components by executing these program modules. Here, when executing the above modules, the processor 1040 may execute the modules after reading them onto the memory 1060 or without reading them onto the memory 1060.

  The hardware configuration of the computer 1000 is not limited to the configuration shown in FIG. For example, each program module may be stored in the memory 1060. In this case, the computer 1000 may not include the storage 1080.

  The input / output device connected to the input / output interface 1100 includes a reader 40 that reads product information symbols. The specific mounting method of the reader 40 depends on the product information symbol attached to the product. The merchandise information symbol is a symbol attached to the merchandise, and is a symbol representing information for identifying the merchandise (product information ID (Identifier) or the like). The symbol is a bar code, a two-dimensional code (such as a QR code (registered trademark)), or a character string symbol. The character string here includes a numeric string. The product information symbol is a barcode or the like in which information specifying product information is encoded, or a character string symbol representing information specifying product information.

  For example, when the product information symbol is a barcode, the reader 40 is a barcode reader. For example, when the product information symbol is a two-dimensional code, the reader 40 is a two-dimensional code reader. For example, when the product information symbol is a character string symbol, the reader 40 includes an image sensor that captures the character string, and a processing unit that analyzes the captured character string symbol and determines a product ID and the like. Note that various known techniques relating to character string analysis can be used for analyzing character string symbols. A description of these known techniques is omitted.

  When a plurality of types of symbols are used as the product information symbol, a plurality of types of readers are connected to the input / output interface 1100. For example, when a barcode and a two-dimensional code are used as the product information symbol, the input / output interface 1100 is connected to a barcode reader and a two-dimensional code reader.

  The input / output device connected to the input / output interface 1100 includes the display device 10. The display device 10 is a display device that is controlled by the first display control unit 3040. The display device 10 includes a touch panel 12, for example. The display device 10 is installed facing the customer so that the customer who uses the product registration device 3000 can browse.

<< Hardware Configuration of First Imaging Unit 3020 >>
The first image capturing unit 3020 includes an image sensor that records a captured result. For example, the first imaging unit 3020 is configured using a camera having this imaging element. The first imaging unit 3020 may capture a still image or a moving image. In the latter case, the first customer image is each frame constituting the moving image.

  The first imaging unit 3020 is installed at a position where a customer who uses the product registration device 3000 can be imaged. The first imaging unit 3020 may be provided integrally with the product registration device 3000 or may be provided separately. In the latter case, for example, the camera 30 that implements the first imaging unit 3020 is connected to the input / output interface 1100.

<Hardware configuration example of checkout device 4000>
Each functional component of the settlement apparatus 4000 may be realized by hardware (eg, a hard-wired electronic circuit) that implements each functional component, or a combination of hardware and software (eg, electronic A combination of a circuit and a program for controlling the circuit may be realized.

  FIG. 6 is a diagram illustrating a hardware configuration of a computer 6000 that realizes the settlement apparatus 4000 according to the first embodiment. The computer 6000 may be mounted using a dedicated device such as a cash register terminal, or may be mounted using a general-purpose device such as a PC (Personal Computer) or a portable terminal. The configuration of the computer 6000 is the same as that of the computer 1000.

  The storage 6080 stores program modules for realizing the functions of the second imaging unit 4020 and the acquisition unit 4080, respectively. The processor 6040 implements the functions of the corresponding functional components by executing these program modules. Here, when executing the above modules, the processor 6040 may execute the modules after reading them onto the memory 6060 or without reading them onto the memory 6060.

  The hardware configuration of the computer 6000 is not limited to the configuration shown in FIG. For example, each program module may be stored in the memory 6060. In this case, the computer 6000 may not include the storage 6080.

  The input / output devices connected to the input / output interface 6100 include the display device 20 controlled by the second display control unit 4040. The display device 20 includes, for example, a touch panel 22. The display device 20 is installed toward the customer so that the customer using the checkout device 4000 can browse. This display device may be provided integrally with the checkout device 4000 or may be provided separately.

  The input / output devices connected to the input / output interface 6100 include a device for a customer to input money, a device for refunding a change, a device for issuing a receipt, and the like (not shown). ).

<< Hardware Configuration of Second Imaging Unit 4020 >>
The second imaging unit 4020 is realized with the same hardware configuration as the first imaging unit 3020, for example. The camera 35 is a camera that implements the second imaging unit 4020. The camera 35 or the like that implements the second imaging unit 4020 is installed at a position where a customer who uses the settlement apparatus 4000 can be imaged.

<Product Registration Processing by Product Registration Device 3000>
As described above, the product registration device 3000 generates checkout information related to a product registered as a checkout target. The merchandise registration apparatus 3000 stores the generated checkout information in a storage unit inside or outside the merchandise registration apparatus 3000.

  A checkout object in one checkout process may include a plurality of products. For example, the merchandise registration device 3000 receives one or a plurality of merchandise registered after receiving an operation for instructing the start of the registration process for the settlement target and before receiving an operation for instructing the end of the registration process for the settlement target. Register as a checkout target in one checkout process. The settlement information related to a certain settlement process indicates the ID of each product registered as the target of the settlement process. Further, the settlement information may further indicate a transaction number, the amount of each product, and the total amount.

  FIG. 7 is a diagram illustrating the settlement information in a table format. The settlement information shown in FIG. The settlement information 200 has two columns, a settlement information ID 202 and a product list 204. The product list 204 is a list of product IDs registered as settlement targets. The product ID 206 is an ID of a product registered as a payment target.

  It should be noted that various known techniques can be used as a specific method for generating the settlement information using the product ID read by the reader or the like, and thus description of this method is omitted.

<Details of processing performed by first imaging unit 3020>
The first imaging unit 3020 generates a first customer image (FIG. 3: S102). There are various timings at which the first imaging unit 3020 performs imaging. For example, the first imaging unit 3020 always performs repeated imaging while the product registration device 3000 is operating. The frequency with which the first imaging unit 3020 repeatedly performs imaging is, for example, 1/30 second, which is the same as a general moving image frame rate.

  Further, for example, the first imaging unit 3020 may detect that the customer is in the vicinity of the product registration device 3000 and perform imaging in response to the detection. For example, an infrared sensor that detects a person is provided in the vicinity of the first imaging unit 3020. The first imaging unit 3020 can recognize that there is a customer in the vicinity of the first imaging unit 3020 by receiving the notification from the infrared sensor. Therefore, for example, the first image capturing unit 3020 captures an image of a customer when the infrared sensor detects that there is a customer near the first image capturing unit 3020, and generates a first customer image.

  Note that the first imaging unit 3020 may repeatedly image the customer for a predetermined period from when the above detection is performed. For example, the first imaging unit 3020 repeatedly images a customer for a predetermined time from when the above detection is performed. In addition, for example, the first imaging unit 3020 may repeatedly image the customer from when the detection is performed until registration of the product purchased by the customer is completed (until settlement information is generated). .

  The above-mentioned “predetermined time” may be set in the first imaging unit 3020 in advance, or may be stored in a storage unit provided inside or outside the product registration device 3000. In the latter case, the first imaging unit 3020 acquires and uses the predetermined time from the storage unit. Unless otherwise specified, various predetermined values (predetermined time, predetermined value, etc.) used by each functional component described below are similarly set in the functional component or obtained from the storage unit. Used.

  Further, the first imaging unit 3020 may be operated by a store clerk who operates the product registration device 3000. For example, the store clerk guides the customer so that the customer's face enters the imaging range of the first imaging unit 3020 by a method such as calling out to the customer. Then, the store clerk confirms that the customer's face is in the imaging range of the first imaging unit 3020, and causes the first imaging unit 3020 to capture the customer image.

  Further, the guidance may be performed by the first imaging unit 3020. For example, the first imaging unit 3020 guides the customer by displaying guidance on the display device 10 with characters or performing voice guidance with sound so that the customer's face is imaged by the first imaging unit 3020. To do.

  As described above, an image captured by the first imaging unit 3020 may include a plurality of customers. Therefore, the first image capturing unit 3020 extracts customers in the image recorded on the image sensor of the first image capturing unit 3020, and generates a first customer image for each customer. Here, for the process of extracting each customer shown in the image, a known object recognition process for extracting a person shown in the image can be used. Therefore, details of this process are omitted.

  When the first imaging unit 3020 images a customer multiple times, a plurality of original images showing the same customer are generated. Therefore, the first imaging unit 3020 generates a first customer image using any one of the plurality of original images. There are various ways to determine which original image is used to generate the first customer image. For example, the 1st imaging part 3020 makes an image with a good customer image a 1st customer image. The “image with a good customer image” is an image in which the customer is greatly reflected or an image in which the blur is small. Here, a method for calculating the size of an object (customer) shown in an image or calculating the size of a shake is a known technique. Therefore, the description regarding these methods is omitted.

<Details of Processing Performed by First Display Control Unit 3040>
The first display control unit 3040 displays the first customer image generated by the first imaging unit 3020 on the display device 10 (FIG. 3: S104). FIG. 8 is a diagram illustrating a display on the display device 10. In FIG. 8A, the first display control unit 3040 displays one first customer image 50 on the touch panel 12 of the display device 10. On the other hand, in FIG. 8B, the first display control unit 3040 displays three first customer images on the touch panel 12 of the display device 10. When there are many customers imaged by the first imaging unit 3020, the first display control unit 3040 displays the button 70 on the touch panel of the display device 10. When the customer presses the button 70, the first display control unit 3040 displays the first customer image 50 that is not currently displayed on the touch panel 12.

<Details of Processing Performed by First Input Accepting Unit 3060>
The first input receiving unit 3060 receives selection of the first customer image displayed on the display device 10 (FIG. 3: S106). As a method of accepting selection of the first customer image, various known techniques for accepting selection of an image, a button, or the like displayed on the display can be used. For example, when the display apparatus 10 has the touch panel 12 and the first input receiving unit 3060 displays the first customer image on the touch panel 12, the first input receiving unit 3060 displays the first customer image displayed on the touch panel 12. By detecting that it has been pressed, the selection of the first customer image is accepted.

  In the case of FIG. 8A described above, the customer confirms whether or not the customer shown in the first customer image 50 displayed on the touch panel 12 is himself. When the customer shown in the first customer image 50 is himself, the customer selects the first customer image 50 by pressing the first customer image 50.

  In the case of FIG. 8B described above, the customer is reflected in the first customer image 50 displayed on the touch panel 12 or the first customer image displayed on the touch panel 12 when the button 70 is pressed. It is confirmed whether or not there is a first customer image 50. When there is a first customer image 50 in which the user is reflected, the customer selects the first customer image 50 by pressing the first customer image 50.

  In addition, when the 1st display control part 3040 displays a some 1st customer image like FIG.8 (b), the 1st input reception part 3060 may receive the input which selects a some 1st customer image. Good. For example, when three family members use the product registration device 3000, the three first customer images are displayed on the display device 10, respectively. This family may select the first customer images for each of the three people. By doing this, all three people can use the checkout device 4000 to check out the product. For example, in FIG. 8B, the customer selects the plurality of first customer images 50 by pressing the plurality of first customer images 50.

  Here, the 1st input reception part 3060 may receive the input which shows that there is no 1st customer image in which it is reflected from the customer. For example, in FIG. 8, the customer presses the button 60 to input that there is no first customer image in which the customer is shown. For example, when this input is made, the first imaging unit 3020 captures the customer again and generates a first customer image. Then, the first display control unit 3040 displays the generated first customer image on the display device 10.

  Note that the method by which the first input receiving unit 3060 receives input is not limited to a method using a touch panel. For example, the first input receiving unit 3060 may receive an input via a keyboard or the like.

<Details of processing performed by generation unit 3080>
The generation unit 3080 associates the payment information with the first customer image (FIG. 3: S108). Information indicating this association is referred to as association information. FIG. 9 is a diagram illustrating the configuration of the association information in a table format. The association information illustrated in FIG. 9 is referred to as association information 300. The association information 300 has two columns of a first customer image 302 and a settlement information ID 304. The first customer image 302 indicates an image file in which the first customer image is recorded. The settlement information ID 304 indicates the ID of the settlement information.

  The generation unit 3080 associates the first customer image with the checkout information related to the product purchased by the customer shown in the first customer image. Specifically, the generation unit 3080 associates the settlement information generated by the registration process with the first customer image generated during the period from the start of the registration process of the product to the end of the registration process. . In addition, when the input which selects a some 1st customer image is performed with respect to the 1st input reception part 3060, the production | generation part 3080 shows the relevant information which linked | related these several 1st customer images with the adjustment information. Generate.

  The generation unit 3080 stores the association information in a storage unit accessible from each settlement apparatus 4000. This storage unit may be inside or outside the product registration apparatus 3000. For example, the storage unit outside the product registration device 3000 is a database server that can be accessed from the product registration device 3000 and the checkout device 4000.

<Regarding the settlement process by the settlement apparatus 4000>
The settlement apparatus 4000 acquires settlement information and performs settlement processing based on the settlement information. A known method can be used as a method for performing the settlement process based on information (settlement information) used by the settlement apparatus 4000 for the settlement process of the product. Details of this known technique are omitted.

<Details of Processing Performed by Second Imaging Unit 4020>
The second imaging unit 4020 generates a second customer image (FIG. 4: S202). The process in which the second imaging unit 4020 generates the second customer image is the same as the process in which the first imaging unit 3020 generates the first customer image.

<Details of Processing Performed by Acquisition Unit 4080>
The acquisition unit 4080 acquires the adjustment information associated with the first customer image corresponding to the second customer image using the related information (FIG. 4: S204). Specifically, the acquisition unit 4080 accesses the storage unit in which the generation unit 3080 stores the related information, and acquires the adjustment information indicated by the related information indicating the first customer image corresponding to the second customer image. .

  The “first customer image corresponding to the second customer image” is a first customer image that is considered to include the same customer as that shown in the second customer image. For example, the “first customer image corresponding to the second customer image” is a first customer image in which a customer having the highest similarity with the customer in the second customer image is shown. In this case, the acquisition unit 4080 calculates the similarity between the customer shown in the second customer image and the customer shown in each first customer image.

  Here, there are various methods by which the acquisition unit 4080 calculates the similarity. For example, the acquisition unit 4080 calculates the feature amount of the customer in each of the first customer image and the second customer image. Then, the acquisition unit 4080 calculates the similarity of the feature amounts of these customers. Here, various known methods used for object matching can be used as the method for extracting the feature amount of the object from the image and the method for calculating the similarity of the feature amount. Further, the method for calculating the similarity between the first customer image and the second customer image is not limited to the method using the feature amount. For the calculation of the first similarity, various known methods for calculating the similarity of the object shown in the image can be used.

  Further, for example, the checkout device 4000 displays a candidate for the first customer image (hereinafter referred to as a candidate image) in which the same customer as the customer shown in the second customer image is displayed on a display device that can be browsed by the customer. You may accept selection of an image. In this case, the payment apparatus 4000 further includes a second display control unit 4040 and a second input reception unit 4060 (FIG. 10). The second display control unit 4040 displays the candidate image on the display device 20 that can be browsed by the customer. The second input receiving unit 4060 receives an input for selecting a candidate image displayed on the display screen. Then, the second input receiving unit 4060 acquires the adjustment information associated with the selected candidate image.

<Details of Processing Performed by Second Display Control Unit 4040>
The 2nd display control part 4040 makes the 1st customer image in which the customer with a high similarity with the customer reflected in the 2nd customer image is reflected as a candidate image. For example, the 2nd display control part 4040 makes the 1st customer image whose similarity with the customer reflected in the 2nd customer image is more than a predetermined value as a candidate image. For example, the 2nd display control part 4040 makes the 1st customer image in which the order at the time of arranging in the ascending order of the similarity the above-mentioned predetermined order be a candidate image. The method by which the second display control unit 4040 calculates the similarity is the same as the method by which the acquisition unit 4080 calculates the similarity.

<Details of Processing Performed by Second Input Accepting Unit 4060>
Second input reception unit 4060 receives selection of candidate images. The method by which the second input receiving unit 4060 receives selection of candidate images is the same as the method by which the first input receiving unit 3060 receives selection of the first customer image.

<About other functions>
The payment apparatus 4000 may further include a determination unit 4100. FIG. 11 is a block diagram illustrating a settlement apparatus 4000 having a determination unit 4100. In a predetermined case, the determination unit 4100 determines that “there is no first customer image in which the same customer as the customer in the second customer image is reflected”. In this predetermined case, for example, 1) An input to the second input receiving unit 4060 that there is no candidate image showing a customer using the checkout device 4000 (such as an operation of pressing the button 60 in FIG. 8). 2) When the second input reception unit 4060 does not receive the candidate image selection operation (when the customer does not select the candidate image), or 3) is displayed by the second display control unit 4040. This is a case where there is no candidate image to be displayed.

  If the determination unit 4100 determines that “there is no first customer image showing the same customer as the customer shown in the second customer image”, the checkout device 4000 performs various exception processes. For example, the second display control unit 4040 displays a message on the display device 20 that there is no checkout information related to the product purchased by the customer. Further, for example, the settlement apparatus 4000 may execute the series of processes shown in FIG. 4 again. As a result, the degree of similarity is calculated using the second customer image generated by newly imaging the customer. In this way, the customer can check out more reliably.

[Embodiment 2]
FIG. 12 is a block diagram illustrating a settlement system 2000 according to the second embodiment. In FIG. 12, each block represents a functional unit configuration, not a hardware unit configuration.

  The settlement system 2000 of the second embodiment has a function of performing a warning process when a customer who is going out of the area where the settlement apparatus 4000 is installed has not finished the settlement process. For this purpose, the settlement system 2000 of the second embodiment is configured as follows.

  The related information generated by the generation unit 3080 of the second embodiment further indicates a completion flag. The completion flag indicates whether or not the settlement using the settlement information indicated by the related information is completed. That is, the generation unit 3080 of the second embodiment generates related information that associates the first customer image, the checkout information, and the completion flag. Hereinafter, it is assumed that the value of the completion flag indicating that the settlement is completed is “settlement complete” and the value of the completion flag indicating that the settlement is not completed is “unsettled”. FIG. 13 is a diagram illustrating association information 300 according to the second embodiment. The association information 300 further includes a completion flag 306 in addition to the first customer image 302 and the settlement information ID 304.

  The payment system 2000 of the second embodiment further includes a warning device 5000. The warning device 5000 includes a third imaging unit 5020 and a warning unit 5040. The third imaging unit 5020 performs imaging and generates an image including a customer who is at an exit of an area where the checkout device 4000 is installed (hereinafter also simply referred to as “exit”). Hereinafter, an image generated by the third imaging unit 5020 is referred to as a third customer image.

  The warning unit 5040 performs warning processing when the first warning condition is satisfied. The first warning condition is “the completion flag associated with the first customer image showing the same customer as the customer shown in the third customer image indicates that the account is not settled”.

<Process flow>
FIG. 14 is a flowchart illustrating the flow of processing executed by the warning device 5000 according to the second embodiment. The third imaging unit 5020 images the customer at the exit and generates a third customer image (S302). The warning unit 5040 determines whether or not the first warning condition is satisfied (S304). When the first warning condition is satisfied (S304: YES), the warning unit 5040 performs a warning process (S306). If the first warning condition is not satisfied (S304: NO), the process of FIG. 14 ends.

<Hardware configuration example>
For example, the warning device 5000 is realized by a computer having a hardware configuration similar to that of the computer that implements the commodity registration device 3000 and the checkout device 4000. Hardware (for example, a camera) constituting the third imaging unit 5020 is connected to the input / output interface of the computer. The third imaging unit 5020 is provided near the exit. For example, the third imaging unit 5020 is provided on the ceiling near the exit. For example, when there is a gate through which a customer passes at the exit, the third imaging unit 5020 may be provided at the gate. However, the 3rd imaging part 5020 should just image the customer who is in an exit, and the installation place is not limited to the above-mentioned example. Regarding other points, the third imaging unit 5020 is mounted in the same manner as the first imaging unit 3020 and the second imaging unit 4020.

  The computer storage that implements the warning device 5000 has a program module that implements the warning unit 5040. The processor of the computer that implements the warning device 5000 implements the function of the warning unit 5040 by executing this program module.

<Details of Processing Performed by Third Imaging Unit 5020>
The processing performed by the third imaging unit 5020 (FIG. 14: S302) is the same as the processing performed by the first imaging unit 3020 and the second imaging unit 4020.

<Details of processing performed by warning unit 5040>
The warning unit 5040 determines whether or not the first warning condition is satisfied (FIG. 14: S304), and performs a warning process when the first customer image is satisfied (FIG. 14: S306). Various warning processes are performed by the warning unit 5040. For example, the warning unit 5040 outputs an alarm sound at a place where a customer imaged by the first imaging unit 3020 or a store clerk or a security guard can hear. For example, this place is near the exit, near the product registration device 3000, or a security room. In this case, a speaker or the like that outputs an alarm sound is installed at a place where the alarm sound is output.

  In addition, for example, the warning unit 5040 displays a warning message at a place where a customer imaged by the third imaging unit 5020, a store clerk, a guard, or the like can see. For example, a display device that a customer can browse is provided near the exit. The warning unit 5040 displays a warning message indicating that the payment is not completed on the display device. Further, for example, the warning unit 5040 displays a warning message on the display device 10 of each product registration device 3000, the display device in the security guard room, or the like.

  Moreover, when the gate is installed in the exit, the warning part 5040 may perform the process which closes the gate, when the 1st warning condition is satisfy | filled. By doing so, the customer can no longer exit from the exit, so that it is possible to prevent the customer who has not been checked out from returning by mistake.

  In addition, when a membership system is introduced in a store that operates the payment system 2000, the warning unit 5040 may issue a warning by using member information. Specifically, it is assumed that a member image (such as a face photo) and a member contact information (such as an email address) are registered in advance as member information. When the first warning condition is satisfied, the warning unit 5040 accesses a database or the like in which member information is stored, and searches for a member image in which the same customer as the customer shown in the third customer image is shown. To do. Then, when there is such an image, the warning unit 5040 transmits a warning message to the contact information of the customer indicated by the member information indicated by the image. For example, the warning unit 5040 transmits an email including a warning message to the customer's email address. By using the member information, a warning message can be sent directly to the customer. Therefore, even if the customer accidentally forgets the payment and leaves the exit, a warning can be given to the customer so that other people do not know it. Therefore, customer privacy can be protected.

<Specific example>
The operation of the payment system 2000 of the second embodiment will be specifically exemplified. FIG. 15 is a diagram illustrating an operational environment of the payment system 2000 according to the second embodiment. In FIG. 15, the settlement apparatus 4000 is installed in the area 80. The exit 90 is an exit of the area 80. The gate 100 is a gate through which a customer passes when exiting the exit 90. In FIG. 15, the third imaging unit 5020 is provided in the gate 100.

  Usually, after using the product registration device 3000, the customer performs payment using the payment device 4000. When the settlement of the customer is completed, the settlement apparatus 4000 uses the second input reception unit 4060 to change the value of the completion flag associated with the first customer image selected by the customer to “settlement complete”.

  When the customer passes through the gate 100, the third imaging unit 5020 images the customer and generates a third customer image. Then, the warning unit 5040 determines whether or not the completion flag associated with the first customer image showing the same customer as the customer shown in the third customer image indicates “unsettled”. As described above, if the customer has completed the settlement, the completion flag indicates “settlement complete”. Therefore, the warning unit 5040 does not perform warning processing.

  On the other hand, if the customer has not settled, the completion flag associated with the first customer image showing the same customer as the customer shown in the third customer image indicates “unsettled”. In this case, the warning unit 5040 performs warning processing. For example, the warning unit 5040 closes the gate 100 and displays a warning message on a display device provided in the gate 100.

<Action and effect>
The payment system 2000 of the present embodiment performs warning processing when the first warning condition is satisfied. Here, the fact that the first warning condition is satisfied indicates that the customer has tried to exit (or has exited) before paying for the merchandise to be purchased. Therefore, according to the payment system 2000 of the present embodiment, it is possible to prevent the customer from forgetting the payment and return home, and thus it is possible to prevent a trouble from occurring between the customer and the store. .

[Embodiment 3]
The settlement system 2000 according to the third embodiment is represented in FIG. 13 in the same manner as the settlement system 2000 according to the second embodiment. Except as described below, the settlement system 2000 of the third embodiment is the same as the settlement system 2000 of the second embodiment.

  In the settlement system 2000 of the third embodiment, when there are a plurality of customer candidates who perform settlement based on the settlement information, all the candidate customers try to exit before the settlement based on the settlement information is performed. Perform warning processing. For this purpose, the settlement system 2000 of the third embodiment is configured as follows.

  The first input receiving unit 3060 of the third embodiment receives selection of a plurality of first customer images. The specific method by which the first input receiving unit 3060 receives selection of a plurality of first customer images is as described in the first embodiment.

  The generation unit 3080 according to the third embodiment generates related information by associating a plurality of selected first customer images with settlement information. Thereby, the customer who performs the payment based on the payment information is a customer shown in any of the associated first customer images. In other words, it means that there are a plurality of customer candidates who perform settlement based on the settlement information. The specific method for generating the related information by associating the plurality of first customer images with the settlement information is as described in the first embodiment.

  The warning unit 5040 of the third embodiment performs warning processing when the second warning condition is satisfied in addition to the first warning condition. On the other hand, the warning unit 5040 does not perform warning processing when the second warning condition is not satisfied.

  The second warning condition is “For all the first customer images associated with the checkout information together with the first customer image in which the same customer as the third customer image is reflected, the third customer image in which the same customer is reflected. It has been generated ”. Here, the third customer image in which the same customer as the first customer image is generated is that the customer in the first customer image is at the exit of the area where the checkout device 4000 is installed. It means that it came out. Therefore, the second warning condition is that the customer who is about to exit the current out of the customer candidates (each customer associated with the settlement information) who will settle the product purchased by the customer who is about to exit. This means that all customers except for have already exited. Therefore, if the customer who is going to exit now also exits, all the customers who are candidates for settlement will exit the exit without performing settlement based on the settlement information. Therefore, the warning unit 5040 of the third embodiment performs warning processing in such a case.

  On the other hand, if the second warning condition is not satisfied, one or more of the candidate customers who will settle the products purchased by the customer who is currently leaving the exit has not yet exited. . In this case, there is no problem if payment is performed by a customer who has not yet exited, and the warning unit 5040 does not perform warning processing.

<Process flow>
FIG. 16 is a flowchart illustrating the flow of processing executed by the warning device 5000 according to the third embodiment. The flowchart of FIG. 16 is the same as the flowchart of FIG. 14 except that the branch of S402 is included.

  When the first warning condition is satisfied (S304: YES), the warning unit 5040 determines whether or not the second warning condition is satisfied (S402). When the second warning condition is satisfied (S402: YES), the warning unit 5040 performs a warning process (S306). If the second warning condition is not satisfied (S402: NO), the process of FIG. 16 ends.

<Hardware configuration example>
The hardware configuration of the settlement system 2000 of the third embodiment is the same as the hardware configuration of the settlement system 2000 of the second embodiment. However, a program module for realizing the function of the warning unit 5040 of the third embodiment is stored in the storage 1080 of the computer 1000 used for mounting the warning device 5000.

<Details of Processing Performed by Third Imaging Unit 5020>
The third imaging unit 5020 of the third embodiment stores the generated third customer image in a storage unit inside or outside the warning device 5000. That is, the warning device 5000 according to the third embodiment stores the history of the third customer image.

<Details of processing performed by warning unit 5040>
The warning unit 5040 determines whether or not the second warning condition is satisfied (FIG. 16: S402). Specifically, the warning unit 5040 performs processing as follows.

  The warning unit 5040 acquires each first customer image other than the first customer image from the related information indicating the first customer image in which the same customer as the customer shown in the third customer image is shown. Then, the warning unit 5040 determines, for each acquired first customer image, whether there is a third customer image in which the same customer as the customer in the first customer image is present.

  The warning unit 5040 uses the third customer image stored in the storage unit by the third imaging unit 5020. Specifically, the warning unit 5040 compares the customer shown in the acquired first customer image with the customer shown in each third customer image stored in the storage unit. A third customer image in which the same customer as that shown in one customer image is shown is searched.

  For example, the warning unit 5040 calculates the degree of similarity between the customer shown in the first customer image and the customer shown in the third customer image by the same processing as the processing performed by the second display control unit 4040. Then, when there is a third customer image whose similarity is equal to or greater than a predetermined threshold, the warning unit 5040 determines that there is a third customer image in which the same customer as that in the first customer image is shown. On the other hand, the warning unit 5040 determines that there is no third customer image showing the same customer as the customer shown in the first customer image when there is no third customer image whose similarity is equal to or greater than a predetermined threshold.

  For all the acquired first customer images, when there is a third customer image in which the same customer as the customer in the first customer image is present, the second warning condition is satisfied. In this case, the warning unit 5040 performs warning processing. On the other hand, for any one or more of the acquired first customer images, the second warning condition is not satisfied when there is no third customer image showing the same customer as the customer shown in the first customer image. In this case, the warning unit 5040 does not perform warning processing.

  Note that the warning process performed by the warning unit 5040 of the third embodiment is the same as the warning process performed by the warning unit 5040 of the second embodiment.

<Action and effect>
The warning unit 5040 according to the present embodiment determines whether or not each of the other customers associated with the settlement information has exited the settlement information associated with the customer at the exit of the area where the settlement apparatus 4000 is installed. (It is reflected in any of the third customer images generated by the third imaging unit 5020). The warning unit 5040 performs warning processing when each of these customers has exited. By doing so, when there are a plurality of candidate customers who perform settlement based on certain settlement information, a warning is issued only when all the candidate customers exit the exit. On the other hand, if there is a customer who is still in the area among the candidate customers who perform the settlement process based on the settlement information, no warning is given. Therefore, according to the payment system 2000 of the present embodiment, the customer is prevented from forgetting the payment and going home, and the customer can flexibly perform the payment.

  As mentioned above, although embodiment of this invention was described with reference to drawings, these are the illustrations of this invention, Various structures other than the above are also employable.

Hereinafter, examples of the reference form will be added.
1. A payment system having a product registration device and a payment device,
The product registration device
First imaging means for generating a first customer image including the customer;
First display control means for displaying the first customer image on a display device that can be browsed by the customer;
A first input receiving means for receiving an input for selecting a first customer image of a customer who performs a checkout process;
Generating means for generating related information associating the selected first customer image and checkout information used for checkout of a product;
The settlement apparatus is
Second imaging means for generating a second customer image including the customer;
A payment system comprising: a first customer image corresponding to the second customer image; and acquisition means for acquiring the payment information associated in the related information.
2. Second display control means for displaying a candidate image that is a candidate for the first customer image in which the same customer as the customer shown in the second customer image is displayed on a display device that the customer can view;
Second input receiving means for receiving an input for selecting a candidate image displayed on the display device;
The acquisition unit acquires the settlement information associated with the selected candidate image in the related information. Checkout system described in.
3. The first input receiving means receives selection of a plurality of first customer images,
The related information generates the related information by associating the selected first customer images with the checkout information. Or 2. Checkout system described in.
4). The related information further associates a completion flag indicating whether or not the settlement using the settlement information is completed with respect to the settlement information,
The settlement system is
Third imaging means for generating a third customer image including a customer at an exit of an area where the settlement apparatus is installed;
When the completion flag associated with the first customer image showing the same customer as the customer shown in the third customer image indicates that the payment using the payment information is not completed, A warning device having warning means for performing warning processing; To 3. The settlement system according to any one of the above.
5). The first input receiving means receives selection of a plurality of first customer images,
The related information generates the related information by associating the selected first customer images with the checkout information,
The warning means that the completion flag associated with the first customer image in which the same customer as the customer in the third customer image is reflected is that the settlement using the settlement information has not been completed. In the case shown
If there is a third customer image showing the same customer as the customer shown in the first customer image for all of the plurality of first customer images associated with the checkout information, the warning process is performed,
For any one of the plurality of first customer images associated with the checkout information, if there is no third customer image showing the same customer as the customer shown in the first customer image, the warning process is performed. No, 4 Checkout system described in.
6). 1. To 5. The product registration device according to any one of the above.
7). 1. To 5. The settlement apparatus according to any one of the above.
8). 5. Computer. A program for operating as the product registration apparatus according to any one of the above.
9. 6. Computer. A program that operates as the settlement apparatus according to any one of the above.
10. A checkout method executed by a checkout system having a product registration device and a checkout device,
A first imaging step in which the commodity registration device generates a first customer image including a customer;
A first display control step in which the product registration device displays the first customer image on a display device that a customer can browse;
A first input receiving step in which the product registration device receives an input for selecting a first customer image of a customer who performs a checkout process;
The product registration device generates a related information in which the selected first customer image is associated with payment information used for product payment;
A second imaging step in which the settlement apparatus generates a second customer image including the customer;
A settlement method comprising: an acquisition step in which the settlement apparatus obtains the settlement information associated with the first customer image corresponding to the second customer image and the related information.
11. A second display control step in which the settlement apparatus displays a candidate image that is a candidate for the first customer image in which the same customer as the customer in the second customer image is displayed on a display device that the customer can view; ,
A second input step for receiving an input for selecting a candidate image displayed on the display device;
The acquisition step acquires the settlement information associated with the selected candidate image in the related information. Checkout method described in 1.
12 The first input receiving step receives selection of a plurality of first customer images,
The generating step generates the related information by associating the selected first customer images with the checkout information. Or 11. Checkout system described in.
13. The related information further associates a completion flag indicating whether or not the settlement using the settlement information is completed with respect to the settlement information,
The checkout system further includes a warning device,
A third imaging step in which the warning device generates a third customer image including a customer at an exit of an area where the checkout device is installed;
The warning flag indicates that the completion flag associated with the first customer image in which the same customer as the customer in the third customer image is reflected is not settled using the settlement information. If so, a warning step for performing warning processing is included. To 12. The settlement system according to any one of the above.
14 The first input receiving step receives selection of a plurality of first customer images,
The related information generates the related information by associating the selected first customer images with the checkout information,
In the warning step, the completion flag associated with the first customer image showing the same customer as the customer shown in the third customer image indicates that the payment using the payment information is not completed. In the case shown
If there is a third customer image showing the same customer as the customer shown in the first customer image for all of the plurality of first customer images associated with the checkout information, the warning process is performed,
For any one of the plurality of first customer images associated with the checkout information, if there is no third customer image showing the same customer as the customer shown in the first customer image, the warning process is performed. No, 13 Checkout system described in.

10, 20 Display device 12, 22 Touch panel 30, 35 Camera 40 Reader 50 First customer image 60, 70 Button 80 Area 90 Exit 100 Gate 200 Checkout information 202 Checkout information ID
204 Product List 206 Product ID
300 Association Information 302 First Customer Image 304 Checkout Information ID
306 Completion flag 1000, 6000 Computer 1020, 6020 Bus 1040, 6040 Processor 1060, 6060 Memory 1080, 6080 Storage 1100, 6100 Input / output interface 1120, 6120 Network interface 2000 Settlement system 3000 Product registration device 3020 First imaging unit 3040 First display Control unit 3060 First input receiving unit 3080 Generating unit 4000 Payment device 4020 Second imaging unit 4040 Second display control unit 4060 Second input receiving unit 4080 Acquisition unit 4100 Determination unit 5000 Warning device 5020 Third imaging unit 5040 Warning unit 6000 Computer

Claims (10)

A payment system having a product registration device and a payment device,
The product registration device
First imaging means for generating a first customer image including the customer;
First display control means for displaying the first customer image on a display device that can be browsed by the customer;
A first input receiving means for receiving an input for selecting a first customer image of a customer who performs a checkout process;
Generating means for generating related information associating the selected first customer image and checkout information used for checkout of a product;
The settlement apparatus is
Second imaging means for generating a second customer image including the customer;
A payment system comprising: a first customer image corresponding to the second customer image; and acquisition means for acquiring the payment information associated in the related information. Second display control means for displaying a candidate image that is a candidate for the first customer image in which the same customer as the customer shown in the second customer image is displayed on a display device that the customer can view;
Second input receiving means for receiving an input for selecting a candidate image displayed on the display device;
The payment system according to claim 1, wherein the acquisition unit acquires the payment information associated with the selected candidate image in the related information. The first input receiving means receives selection of a plurality of first customer images,
The payment system according to claim 1, wherein the generation unit generates the related information by associating the selected first customer images with the payment information. The related information further associates a completion flag indicating whether or not the settlement using the settlement information is completed with respect to the settlement information,
The settlement system is
Third imaging means for generating a third customer image including a customer at an exit of an area where the settlement apparatus is installed;
When the completion flag associated with the first customer image showing the same customer as the customer shown in the third customer image indicates that the payment using the payment information is not completed, The settlement system according to any one of claims 1 to 3, further comprising a warning device having warning means for performing warning processing. The first input receiving means receives selection of a plurality of first customer images,
The related information generates the related information by associating the selected first customer images with the checkout information,
The warning means that the completion flag associated with the first customer image in which the same customer as the customer in the third customer image is reflected is that the settlement using the settlement information has not been completed. In the case shown
If there is a third customer image showing the same customer as the customer shown in the first customer image for all of the plurality of first customer images associated with the checkout information, the warning process is performed,
For any one of the plurality of first customer images associated with the checkout information, if there is no third customer image showing the same customer as the customer shown in the first customer image, the warning process is performed. The settlement system according to claim 4, wherein there is no settlement.   The merchandise registration apparatus according to any one of claims 1 to 5.   The settlement apparatus according to any one of claims 1 to 5.   A program that causes a computer to operate as the commodity registration device according to claim 6.   A program that causes a computer to operate as the settlement apparatus according to claim 7. A checkout method executed by a checkout system having a product registration device and a checkout device,
A first imaging step in which the commodity registration device generates a first customer image including a customer;
A first display control step in which the product registration device displays the first customer image on a display device that a customer can browse;
A first input receiving step in which the product registration device receives an input for selecting a first customer image of a customer who performs a checkout process;
The product registration device generates a related information in which the selected first customer image is associated with payment information used for product payment;
A second imaging step in which the settlement apparatus generates a second customer image including the customer;
A settlement method comprising: an acquisition step in which the settlement apparatus obtains the settlement information associated with the first customer image corresponding to the second customer image and the related information.

Images