JP2022164939A - Sales system - Google Patents

Abstract

To provide a sales system capable of efficiently performing accounting processing even on an article to which an RF tag cannot be attached.SOLUTION: A sales system 100 according to the present invention comprises: a recognition part which recognizes a customer based upon an image generated by a camera 10; a specifying part which specifies an article G picked up by the customer; and a register device 40 which presents accounting information on the specified article to the recognized customer.SELECTED DRAWING: Figure 2

Description

The present invention relates to a sales system used for merchandise sales in stores.

For example, a technique described in Patent Literature 1 has been proposed in order to improve the efficiency of accounting processing in merchandise sales. The technology described in Patent Literature 1 is a sales system that uses RFID technology to streamline product sales at an event site or the like, and includes a gate arranged at the entrance/exit of the event site. The gate includes a reader that reads the product ID and the user ID from the RF tag attached to each product and the user's ID card, and a transmitter that transmits the read product ID and user ID to the server. . The server stores the product ID and the user ID received from the gate in the memory as purchase information, and transmits payment information generated based on the purchase information to the cash register device.

JP 2020-191042 A

However, in the sales system of the above-mentioned patent document, it is necessary to attach RF tags to all the products being sold. It is difficult to use the above-mentioned sales system at a store that picks up and pays the bill.

SUMMARY OF THE INVENTION It is an object of the present invention to provide a sales system capable of efficiently performing checkout processing even for products to which an identification medium such as an RF tag cannot be attached.

In order to achieve the above object, the sales system of the present invention includes a recognition unit that recognizes a customer based on an image generated by a camera, a specification unit that specifies products picked up by the customer, and a checkout processing device that presents checkout information of the identified product.

Further, the recognition unit recognizes a plurality of customers, and the accounting processing device presents accounting information of one customer and accounting information of other customers following the one customer.

The sales system of the present invention can efficiently perform checkout processing even for products to which RF tags cannot be attached.

Schematic diagram of a store where a sales system according to the first embodiment of the present invention is used (a) hardware configuration diagram of the sales system, (b) hardware configuration diagram of the camera of the sales system, (c) hardware configuration diagram of the GPU of the sales system, (d) (e) Hardware configuration diagram of the cash register device of the sales system (a) Front view of a display device of a sales system according to a second embodiment of the present invention, (b) Side cross-sectional view of the display device A diagram showing the hardware configuration of the display device

A sales system according to an embodiment of the present invention will be described below with reference to the drawings.

[First embodiment]

As shown in FIG. 1, a store equipped with a sales system 100 (FIG. 2) according to the first embodiment of the present invention has a tray stand 1 on which many unused trays T are placed, a display area where products G are displayed, It has a sales area including E1 and an accounting area E2 for accounting. The tray table 1, the display area E1, and the checkout area E2 are arranged linearly, and lanes 3 for placing trays T and customer movement routes R are provided along the display area E1 and the checkout area E2. The display area E1 includes a display stand 2, which is a display device for displaying a plurality of product groups. The display stand 2 has a placement surface 2a that extends horizontally along the movement path R, and the product G is placed on the placement surface 2a. A lane 3 on which the tray T is placed is provided horizontally from the placement surface 2a toward the movement route R. As shown in FIG. Two register devices 40 (checkout processing devices) are installed along the moving route R in the checkout area E2. Note that the store may be, for example, a store that sells donuts as product G, or an udon restaurant that sells toppings such as tempura as product G.

As shown in FIG. 2, the sales system 100 used in the store described above includes a camera 10, an image processing device (hereinafter referred to as a GPU 20), and a server 30 in addition to the two register devices 40 described above. ing. The camera 10 is communicably connected to the GPU 20 by wire or wirelessly. A network N is formed in the store, and the GPU 20 and each cash register device 40 are connected to the server 30 via the network N so as to be able to communicate with each other.

The camera 10, as shown in FIG. 1, is installed on the ceiling of the sales area, captures an image of the sales area at a predetermined frame rate, and generates an image of the sales area. As shown in FIG. 2, such a camera 10 includes an image sensor 11 functioning as an imaging unit for imaging the sales area, a CPU 12 controlling the image sensor 11 and functioning as a generation unit for generating an image, and a and a communication module 14 functioning as a transmitting unit for transmitting the image to the GPU 20 .

The GPU 20 includes a communication module 21 functioning as a receiving unit for receiving an image from the camera 10, a CPU 22 functioning as a processing unit for processing the received image, and a memory functioning as a storage unit for storing the image processing program and the image. 23 and a network module 24 that functions as a transmission unit that transmits information acquired by image processing to the server 30 . Each time an image is received from the camera 10, the CPU 22, which is the processing unit, recognizes the customer and the product G included in the image. The method of recognizing the customer and the product G is not particularly limited, but the customer and the product G can be recognized based on their appearance characteristics, for example. Specifically, appearance features (shape, color, size) when the customer or product G is observed from above are determined in advance, and a pixel pattern that matches or approximates the appearance feature is recognized as the customer or product G. be. Machine learning may be used as another recognition method. Specifically, a learned model is generated in advance using an image of a person or product G as teacher data. By inputting the image received from the camera 10 to the trained model, the customer and the product G included in the image are recognized.

Here, when the CPU 22, which is a processing unit, recognizes the product G in the image, it specifies the product ID based on the appearance features of the recognized product G and the learned model. When the product ID is specified, the network module 24 functioning as a transmission unit associates the product ID with the position information (coordinates of the product G in the image) of the recognized product G and transmits the product ID to the server 30 . Further, when the CPU 22 recognizes a customer, the CPU 22 assigns a customer ID to the recognized customer, associates the customer ID with the customer's location information (coordinates of the customer in the image) via the network module 24, and sends the information to the server. 30.

The server 30 includes a network module 34 that functions as a receiving unit that receives information from the GPU 20 and a transmitting unit that transmits information to the cash register device 40, and a CPU 32 that executes processing such as identification processing, determination processing, and generation processing. , and a memory 33 that functions as a storage unit that stores a program that causes the CPU 32 to execute these processes and information received from the GPU 20 .

The specifying process executed by the CPU 32 is a process of matching and specifying the picked-up product G and the customer who picked up the product G based on the information received from the GPU 20 . The identification of the picked-up product G, for example, specifies that the product G having position information (coordinate values) on lane 3 (FIG. 1) is the picked-up product G. Further, the identification of the customer who picked up the product G is identified as the customer who picked up the product G, for example, the customer whose location information is similar to the location information of the identified product G. Then, the product ID of the specified product G is associated with the customer ID of the specified customer. In this manner, the CPU 32 of the server 30 functions as an identification unit that associates and identifies the picked-up product G and the customer who picked up the product G. FIG. In addition, the memory 33 functioning as a storage unit stores the picked-up product G (product ID) and the customer (customer ID) who picked up the product G in association with each other.

Judgment processing executed by the CPU 32 includes first judgment processing for judging whether or not the customer location information received from the GPU 20 is included in the accounting area E2. and a second determination process of determining a cash register device 40 close to the customer based on the location information of the customer when it is determined to be included. In this manner, the CPU 32 of the server 30 functions as a determination unit that determines the cash register device 40 closer to the customer. Also, the memory 33 functioning as a storage unit stores the determination results of the first determination process and the second determination process.

The generation process executed by the CPU 32 is a process of generating accounting information. The accounting information includes the name of the product G picked up and the price of the product G, and is generated based on the product ID specified in the above specifying process. Specifically, the memory 33 (storage unit) of the server 30 stores in advance a product table that defines the correspondence between the name of each product G, the price of each product G, and the product ID. Accounting information is generated by extracting the name and price of the product G using the ID as a search key. The accounting information generated by the generation process is stored in the memory 33 functioning as a storage unit, and transmitted via the network module 34 to the cash register device 40 determined in the determination process.

The cash register device 40 has a network module 44 functioning as a receiving unit for receiving accounting information from the server 30, a memory 43 functioning as a storage unit for storing the received accounting information, and displaying the received accounting information to the customer. A display 45 functioning as a display unit, a CPU 42 functioning as an accounting unit that performs accounting processing based on accounting information, and a cashier 46 are provided. The network module 44 of the cash register device 40 also functions as a transmission unit that transmits to the server 30 that the transaction has been completed when the transaction is processed.

In the sales system 100 configured as described above, an image of the sales area is generated by the camera 10 that captures an image of the sales area at a predetermined frame rate, and the GPU 20 that processes the image detects the presence of the product G and the customer in the sales area. is recognized. Then, the server 30 identifies the product G picked up by the customer, transmits the accounting information of the identified product G to the cash register device 40, and the cash register device 40 executes the accounting process. As described above, according to the sales system 100 of the present embodiment, the product G picked up by the customer can be specified and the accounting process can be performed without attaching an identification medium such as an RF tag to the product G as in the conventional art. can be done. In addition, since there is no need to input the information of the picked-up product G in the checkout process, the checkout process can be performed efficiently.

Further, in the above embodiment, when the GPU 20 recognizes a plurality of customers, the GPU 20 transmits the customer IDs of the plurality of customers and their position information, and transmits the product ID of the recognized product G and its position information to the server 30. Send. The server 30 that has received these pieces of information identifies the product G picked up by the customer based on the customer's position information and the position information of the product G in the identification process. Specifically, the difference value of the position information of the customer and the position information of each product G is obtained, and the product G whose difference value is within a certain value is specified as the picked product G, and the product ID of the product G is identified. is associated with the customer ID. In addition, in the second determination process of the determination process, the CPU of the server 30 recognizes that the plurality of customers belong to a group when the cash register device 40 close to the location information of the plurality of customers is common. 40 to send accounting information of multiple customers. When the cash register device 40 receives a plurality of pieces of accounting information from the server 30, the cash register device 40 presents on the display a selection screen to select whether to perform these transactions individually or collectively. According to such a mode, it is also possible to collectively process the accounts of customers visiting the store as a group.

[Second embodiment]

In the first embodiment described above, the customer and the product G are recognized based on the image generated by the camera 10, but the second embodiment recognizes the customer based on the image generated by the camera 10, This differs from the first embodiment in that the product G is recognized based on the output of the load sensor. In the following, the points different from the first embodiment will be mainly described.

In the display area E1 of the second embodiment, as shown in FIG. 3, a display device 50 for displaying the products G is provided. The display device 50 includes a display shelf 51 for displaying the merchandise G and a sensor 55 for detecting the displayed merchandise G. The display shelf 51 includes a plurality of shelf boards 52 provided vertically and side walls 53 that support the shelf boards 52 . A plurality of trays T are placed on each of the shelf boards 52, and a plurality of products G of the same kind are placed on each of the trays T. As shown in FIG. A load cell 55 (load sensor) as the sensor 55 is provided between each tray T and the shelf plate 52 . The load cell 55 is a sensor that detects the weight of a plurality of commodities G placed on the tray T, and as shown in FIG. It is input to a microcomputer (hereinafter referred to as MCU 57). The MCU 57 has a CPU and a memory, and the CPU executes a program including recognition processing and transmission processing pre-stored in the memory.

The recognition process is a process of recognizing that the product G placed on the tray T has been picked up based on the output of the load cell 55, and includes an acquisition process, a difference process, a comparison process, and a determination process. Acquisition processing is processing for acquiring the output value of the load cell 55 via the ADC 56 at a predetermined cycle, and the acquired output value is stored in the memory. The difference process is a process of obtaining the difference between the output value acquired in the acquisition process and the output value acquired in the previous acquisition process. The comparison process is a process of comparing the difference value obtained in the difference process with a threshold value. This threshold is a value corresponding to the weight of each product G placed on the tray T, and the output value of the load cell 55 is experimentally set in advance with one product G placed on the tray T. Determined by obtaining The determination process is a process of determining whether or not the product G has been picked up. As a result of the comparison process, if the difference value is less than the threshold value, it is determined that the product G has not been picked up from the tray T, If the difference value exceeds the threshold, it is determined that the product G has been picked up from the tray T. The CPU executes the above-described acquisition processing to determination processing for the output of the load cell 55 of each tray T. FIG. As a result of the CPU executing the recognition process in this way, when it is determined that the product G has been picked up, the CPU identifies the tray ID of the tray T corresponding to the load cell 55 and obtains the tray ID via the network module 58. It transmits to the server 30 (transmission processing).

The server 30 of this embodiment receives customer position information and customer ID from the GPU 20 as in the first embodiment, and receives the tray ID from the display device 50 . In the identification process, the CPU 32 of the server 30 associates and identifies the picked-up product G and the customer who picked up the product G based on the received tray ID and customer position information. Specifically, in the memory 33 of the server 30 of this embodiment, in addition to the product table of the first embodiment, a tray table and a product-tray correspondence table are stored in advance. In the tray table, the tray ID and the position information of the tray T are associated with each other. The product-tray correspondence table associates tray IDs with product IDs. Then, the CPU 32 of the server 30 extracts the position information of the tray T corresponding to the received tray ID from the tray table, and extracts the product ID corresponding to the tray ID from the product-tray correspondence table. Then, the CPU 32 of the server 30 searches for position information of a customer close to the extracted position information of the tray T, and associates the extracted product ID with the customer ID of the customer.

As in the first embodiment, the CPU 32 of the server 30 executes determination processing and generation processing, transmits accounting information to the corresponding cash register device 40 , and the cash register device 40 executes accounting processing.

Although the sales system of the present invention has been described based on the first and second embodiments, each of the above embodiments may be modified as follows.

<Modification 1>
It may be recognized that the product G has been picked up based on the image generated by the camera that captures the display area E1 and the output of the load sensor provided in the display area E1. For example, in the second embodiment, a display shelf may be provided with a plurality of cameras. Specifically, a camera may be provided on the lower surface of the shelf located above each tray T, and each camera may take an image of the product G placed on the tray T. FIG. The image of the product G on the tray T individually generated by each camera is transmitted to the GPU 20, and the product G picked up from the tray T is recognized by the GPU 20.

<Modification 2>
Although the single camera 10 that captures the sales area is used in the first embodiment, a plurality of cameras 10 may be used. When the sales area is captured by a plurality of cameras 10, each camera 10 transmits the generated image to the GPU. The GPU combines images received from a plurality of cameras 10 to generate an area image of the sales area, and performs the image processing of the first embodiment on the area image.

Also, a GPU may be provided for each of a plurality of cameras. In this case, each GPU performs the image processing of the first embodiment, associates the position information of the certified product G with the product ID, and transmits the position information to the server 30 . Also, each GPU associates the customer ID with the location information of the authorized customer and transmits the customer ID to the server 30 . The server 30 identifies the position information of the product G and the customer in the entire sales area based on the position information of the product G and the customer received from each GPU, and then executes the above identification processing, determination processing, and generation processing. .

<Modification 3>
The camera 10 used in the first and second embodiments may be a TOF camera. The TOF camera outputs color information of each pixel and distance information of each pixel, and transmits the color information and distance information to the GPU. The GPU can generate an image based on the color information received from the TOF camera, and can accurately detect the position information of the product or customer recognized based on the generated image based on the distance information. In addition, the GPU can detect the recognized arm of the customer based on the distance information, detect the motion of the arm based on the distance information received for each frame, and identify the picked up product based on the motion. be able to.

<Modification 4>
In the above-described embodiment, the product G is picked up on the tray T, but a bag (bag) or box brought by the customer or a bag (bag) or box prepared in advance at the store for bringing the product G home may be used. It doesn't matter if there is. In this aspect, the product G can be put into the bag or box brought by the customer at the same time when the customer picks up the product.

<Modification 5>
A delivery area may be provided between the display area E1 and the checkout area E2. The delivery area is provided with a stand on which the product to be handed is placed, and the product is recognized by the camera 10. - 特許庁According to this aspect, it is possible to deal with products specially ordered by customers, for example, products such as "less ice".

<Modification 6>
The accounting processing device may perform accounting processing using not only cash payments but also electronic money and credit cards. Alternatively, the transaction may be processed only with electronic money or a credit card. In addition, when performing transaction processing using a credit card, the transaction processing apparatus may further include a credit card reader disposed upstream of the tray table 1 . A credit card reader reads a credit card owned by a customer and transmits the read credit card information to a server. When the server receives the credit card information, it inquires the credit card information to the credit company's system and obtains whether the credit card can be used. The server executes the specifying process, the determination process, and the generation process, associates the accounting information generated in the generation process with the credit card information, and transmits the information to the accounting processing device. The accounting processing device executes accounting processing using the accounting information and credit card information received from the server.

10 camera 20 GPU
30 server 40 cash register device 50 display device 100 sales system

Claims (2)

a recognition unit that recognizes the customer based on the image generated by the camera;
an identification unit that identifies the product picked up by the customer;
an accounting processor that presents accounting information for the identified product to the identified customer;
A sales system characterized by comprising: The recognition unit recognizes a plurality of customers,
The accounting processing device presents the accounting information of one customer and the accounting information of another customer that is continuous with the customer,
The sales system according to claim 1.

Images