JP2020077275A - Commodity settlement system, commodity conveying cart and commodity settlement method - Google Patents

Abstract

To accurately identify a commodity selected by a user to achieve efficient settlement of the commodity.SOLUTION: A commodity conveying cart 10 images a commodity mounted on a basket part by a user with a commodity recognition camera to subject the image to image processing to recognize the commodity on the basis of an appearance feature. Moreover, the commodity conveying cart 10 images the face of the user, an adjacent commodity shelf, motion of the user, surrounding persons and the like with a state recognition camera to recognize a peripheral state. The commodity conveying cart 10 uses the recognition result of the commodity and the recognition result of the state to identify the commodity to be purchased, and performs settlement of the identified commodity. Accordingly, the commodity selected by the user can be accurately identified to achieve efficient settlement of the commodity.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a product payment system for payment of a product, a product transportation cart used by a user to carry a product to be purchased, and a product payment method.

  Conventionally, when a product is sold at a store such as a supermarket, a shopper who is a user puts a product to be purchased in a shopping cart or a product transportation cart, and a clerk operates a POS (Point Of Sale) register. Registered and settled the product to be purchased.

  If the POS register is used for product registration and settlement in this way, the customer's waiting time becomes long. Therefore, there has been considered a technology for improving the efficiency of settlement of products by identifying the products selected by the user and registering them in advance.

  For example, Patent Document 1 discovers a moving object such as a shopper or a basket moving in a store, defines the area of the moving object, captures an image while tracking the movement of the moving object, and displays the state in the shelf. Always capture images, compare the captured images before and after the object is taken out from the shelf, define the area of the product to be recognized from the captured image, specify the product from the defined image area, and A system for doing so is disclosed.

JP, 2017-157216, A

  However, the conventional techniques typified by the above have a problem in that the product cannot be specified with high accuracy and the settlement efficiency cannot be realized in many cases. For example, in the system according to Patent Document 1 described above, when there are a plurality of shopping carts near the product shelf, it is not possible to identify which shopping cart the product is in.

  For these reasons, it has been an important issue to identify the product selected by the user with high accuracy and make the payment of the product efficient.

  The present invention has been made to solve the above-described problems, and a product settlement system, a product transportation cart, and a product that can specify a product selected by a user with high accuracy and streamline the payment of the product. The purpose is to provide a payment method.

  In order to solve the above-mentioned problems and achieve the object, the present invention provides a product recognizing means for recognizing a product placed on a product carrying cart used by a user for carrying a product to be purchased, and a peripheral part of the product carrying cart. A state recognition means for recognizing a state, a purchase target commodity specifying means for specifying a purchase target commodity by using the recognition result by the commodity recognition means and the recognition result by the state recognition means, and the purchase target commodity specifying means And a payment processing means for paying the settled product.

  Further, the present invention is characterized in that, in the above-mentioned invention, the state recognition means recognizes the arrangement of products around the product transportation cart by image processing.

  Further, the present invention is characterized in that, in the above-mentioned invention, the state recognition means recognizes, by image processing, an operation in which the user places the product on the product transport cart.

  Further, the present invention is characterized in that, in the above-mentioned invention, the state recognition means recognizes, by image processing, an operation in which the user takes out and returns the product placed on the product transport cart.

  Further, the invention is characterized in that, in the above invention, the product recognition means recognizes the product by image processing.

  Further, the present invention is characterized in that, in the above-mentioned invention, the product recognizing unit recognizes the product by inputting an image of the product into a trained model generated in advance.

  Further, in the present invention according to the above-mentioned invention, when the product recognizing means acquires the identification information attached to the product, the product recognizing unit further recognizes the product by using the identification information.

  Further, the present invention is characterized in that, in the above invention, the purchase target product specifying means is provided in the product transport cart.

  Further, the invention is characterized in that, in the above-mentioned invention, the settlement processing means is provided in the commodity transport cart.

  Further, the present invention is a product transportation cart used by a user to carry a product to be purchased, the product recognition means for recognizing a product placed by the user, and a state recognition means for recognizing a peripheral condition. , A purchase target product specifying unit that specifies a purchase target product using the recognition result by the product recognition unit and the recognition result by the state recognition unit, and a payment that performs payment for the product specified by the purchase target product specifying unit And a processing means.

  Further, the present invention provides a product recognition step of recognizing a product placed on a product carrying cart used by a user to carry a product to be purchased, a state recognition step of recognizing a state around the product carrying cart, and the product. A purchase target product specifying step of specifying a purchase target product using the recognition result of the recognition step and the recognition result of the state recognition step; and a payment processing step of performing payment for the product specified by the purchase target product specifying step. It is characterized by including.

  ADVANTAGE OF THE INVENTION According to this invention, the product which the user selected can be pinpointed accurately and the payment of a product can be made efficient.

FIG. 1 is an explanatory diagram of the concept of the product payment system according to the present embodiment. FIG. 2 is an explanatory diagram of the configuration of the store. FIG. 3 is a system configuration diagram of the product settlement system. FIG. 4 is a functional block diagram showing the functional configuration of the product transport cart. FIG. 5 is a flowchart showing the processing procedure of the product transport cart. FIG. 6 is a flowchart showing details of the purchase target product specifying process shown in FIG. FIG. 7 is a flowchart showing details of the settlement process shown in FIG. FIG. 8 is an explanatory diagram of variations of various processes. FIG. 9 is an explanatory diagram of the structure of the learned model. FIG. 10 is an explanatory diagram of purchase of actual products and products of online stores. FIG. 11 is an explanatory diagram of a usage example of data managed by the store management server.

  Hereinafter, preferred embodiments of a product payment system, a product transportation cart, and a product payment method according to the present invention will be described in detail with reference to the accompanying drawings.

  First, the concept of the product payment system according to the present embodiment will be described. FIG. 1 is an explanatory diagram of the concept of the product payment system according to the present embodiment. In the present embodiment, the products are displayed on the product shelves, and the shopper, who is the user, puts the products to be purchased in the product transport cart 10 and carries them.

  The product transportation cart 10 is a so-called shopping cart, and includes a basket part for storing products, a handle part gripped by a user, and wheels (casters). Further, the product transportation cart 10 includes a product recognition camera for capturing an image of the product placed on the basket by the user and recognizing the product, and a state recognition camera for capturing the surroundings and recognizing the state. ..

  The product transport cart 10 performs image processing on the image of the product captured by the product recognition camera, and recognizes the product based on the external characteristics. If the image includes identification information such as a bird code that contributes to the identification of the product, the identification information may be read and used for recognition of the product.

  However, since a store such as a supermarket handles a large number of products, there are many products with similar external characteristics, and it is difficult to uniquely identify the product from the image. Therefore, the product transportation cart 10 further identifies the product placed in the basket portion by the user, that is, the product to be purchased, by further using the recognition result of the surrounding state.

  Specifically, the product transportation cart 10 captures an image of the user's face, nearby product shelves, user's movements, people in the vicinity, and the like with a state recognition camera. The imaging result of the user's face can be used for user authentication or the like. The imaging result of the nearby product shelves can be used to identify which product is displayed on the shelf. Further, by comparing a plurality of images obtained by continuously capturing images of the product shelves, it is possible to detect taking out of the product from the product shelf or returning of the product to the product shelf. The user's action can be used to detect the removal of the product by the user or the return of the product to the product shelf. The image pickup result of the person in the vicinity can be used for adding a product to the basket part or detecting removal of the product from the basket part performed by a third party other than the user.

  When the product carrying cart 10 identifies the product to be purchased using the result of the product recognition and the result of the state recognition, the product carrying cart 10 performs the payment process of the product to be purchased. The product transportation cart 10 has a function of reading a payment card (credit card, prepaid card, cash card) with a card reader or the like and performing a payment process. It is also possible to identify the user by face authentication and perform settlement processing. In addition, the purchase target product specified by the product transport cart 10 can be transmitted to a nearby device for settlement processing.

  In this way, the product transportation cart 10 recognizes the product placed by the user and the surrounding state, and identifies the product to be purchased by using the product recognition result and the condition recognition result. Settle the goods. Therefore, the product selected by the user can be specified with high accuracy, and the settlement of the product can be made efficient.

  Next, the structure of the store will be described. FIG. 2 is an explanatory diagram of the configuration of the store. In the configuration shown in FIG. 2, a plurality of product shelves, a manned cashier area, a self cashier area, and a cart storage area near the doorway are arranged.

  The plurality of product shelves are arranged so as to form a passage through which the user can push while pressing the product transport cart 10. The user who has entered the store receives the product transportation cart 10 from the cart storage area, and puts a desired product in the basket while moving along the passage. After that, the selected product is settled and the product transport cart 10 is returned, and the store is closed.

  Here, if the user performs the payment process using the payment function of the product transport cart 10, the user can exit the store without stopping at the manned cashier area or the self-checkout area. However, in order to perform the payment process using the product transport cart 10, it is necessary that all the products selected by the user be specified and that payment by a payment card or face authentication be possible.

  When there is a possibility that a product that cannot be specified is included, the product transport cart 10 guides the user to the manned cashier area, and causes the clerk to specify the product and make a payment.

  If all of the products selected by the user have been specified as purchase target products, but payment by payment card or face authentication is not possible, payment processing is performed in the self-checkout area. The product transport cart 10 notifies the cashier terminal device for payment installed in the self-checkout area of the product to be purchased. Therefore, when the payment process is performed in the self-checkout area, the product registration operation is not required, and only the payment process needs to be performed.

  Next, the system configuration of the product settlement system will be described. FIG. 3 is a system configuration diagram of the product settlement system. In the product settlement system shown in FIG. 3, the product transport cart 10 can communicate with the store management server 20 by wireless communication. Further, the store management server 20 is communicably connected to the cashier terminal device 30, the in-store camera 40, and the backyard terminal device 50.

  As described above, the product transportation cart 10 includes the product recognition camera and the state recognition camera, and can perform product recognition processing, status recognition processing, purchase target product identification processing, payment processing, and the like. The product transport cart 10 can also send the results of various processes to the store management server 20. Furthermore, the product transportation cart 10 can transmit the specified purchase target product to the cashier terminal device 30.

  The cashier terminal device 30 is a terminal device installed in a manned cashier area or a self-checkout area. The cashier terminal device 30 installed in the manned cashier area can perform product registration processing and payment processing, and is operated by a clerk. The cashier terminal device 30 installed in the self-checkout area is a device that receives the purchase target product specified by the product transport cart 10 and performs a payment process, and is operated by the user. In addition, the cashier terminal device 30 installed in the manned cashier area or the self-checkout area can send the results of various processes to the store management server 20.

  The in-store cameras 40 are appropriately installed at various places in the store, capture images of users and product shelves, and send them to the store management server 20. The backyard terminal device 50 is a terminal device operated by a store clerk in the backyard, and can display an image captured by the in-store camera 40.

  The store management server 20 is a device that manages the entire store. Specifically, the store management server 20 manages store inventories, sales records, sales, clerk work status, and the like. Furthermore, the store management server 20 manages the results of various processes performed by the product transport cart 10, the results of various processes performed by the cashier terminal device 30, the results of imaging by the in-store camera 40, and the like.

  Next, the functional configuration of the product transport cart 10 will be described. FIG. 4 is a functional block diagram showing a functional configuration of the product transportation cart 10. As shown in FIG. 4, the product transportation cart 10 includes a product recognition camera 11, a state recognition camera 12, a communication unit 13, a card reader / writer 14, a storage unit 15, and a control unit 16.

  The product recognition camera 11 images the product placed on the basket portion by the user, and outputs the imaging result to the control unit 16. The state recognition camera 12 images the surroundings and outputs the imaging result to the control unit 16. The communication unit 13 is a communication interface that communicates with the store management server 20 and the cashier terminal device 30. The card reader / writer 14 is a device that accepts a user card for identifying a user and a payment card (credit card, prepaid card, cash card, etc.) used for payment and reads / writes data.

  The storage unit 15 is a storage device including a hard disk device, a non-volatile memory, and the like, and stores a learned model 15a, product data 15b, payment history data 15c, and the like. The learned model 15a is data indicating the structure and parameters of a multilayer neural network adjusted to identify a product by performing supervised machine learning using a product image for learning. The product data 15b indicates a purchase target product that is a product that the user put in the basket portion. The payment history data 15c is data indicating a history of payment processing performed by the product transportation cart 10.

  The control unit 16 is a control unit that controls the entire product transportation cart 10 and includes a product recognition unit 16a, a state recognition unit 16b, a purchase target product specification unit 16c, and a payment processing unit 16d.

  The product recognition unit 16a performs image processing on the image of the product captured by the product recognition camera 11 and recognizes the product based on the appearance characteristics. Specifically, the product recognition unit 16a inputs the image of the product to the learned model 15a. The learned model 15a extracts the external characteristic of the product from the input image and outputs the corresponding product as the identification result. Alternatively, if the product cannot be uniquely identified from the input image, the identification result is “unknown”.

  If the image includes identification information that contributes to the identification of the product, such as a barcode, the product recognition unit 16a reads the identification information and uses it to recognize the product.

  The state recognition unit 16b performs a process of recognizing the surrounding state by using the image pickup result by the state recognition camera 12. The state recognition camera 12 captures an image of the user's face, nearby product shelves, user's actions, people in the vicinity, and the like. The imaging result of the user's face can be used for user authentication or the like. The imaging result of the nearby product shelves can be used to identify which product is displayed on the shelf. Further, by comparing a plurality of images obtained by continuously capturing images of the product shelves, it is possible to detect taking out of the product from the product shelf or returning of the product to the product shelf. The user's action can be used to detect the removal of the product by the user or the return of the product to the product shelf. The image pickup result of the person in the vicinity can be used for adding a product to the basket part or detecting removal of the product from the basket part performed by a third party other than the user.

  The purchase target product specifying unit 16c performs a process of specifying the purchase target product using the recognition result by the product recognition unit 16a and the recognition result by the state recognition unit 16b. Specifically, when the product recognition unit 16a identifies the product, the identified product is added to the product data 15b as a purchase target product.

  Even if the recognition result by the product recognition unit 16a is "unknown", if the user recognizes the operation of putting the product on the product transport cart and the number of products on the product shelf decreases before and after that, the product is displayed on the product shelf. The existing product is added to the product data 15b as a purchase target product. Further, if the user recognizes the operation of placing the product on the product transport cart and the number of products on the product shelf decreases before and after that, the product displayed on the product shelf is added to the product data 15b as a purchase target product.

  Further, if the user recognizes the operation of taking out and returning the product from the product transport cart, and if the number of products on the product shelf increases before and after that, the corresponding product is selected from the product data 15b and deleted.

  The purchase target product specifying unit 16c sets a “manned cash register guidance flag” when there is a product that cannot be specified even by using the recognition result by the product recognition unit 16a and the recognition result by the state recognition unit 16b. This flag affects selectable payment methods.

  In addition, when it is detected that a third party other than the user adds a product to the basket part or removes a product from the basket part, the purchase target product specifying unit 16c causes the “manned cash register guidance flag”. Set. It is desirable to re-identify the product in the manned cashier area because there may be excess or deficiency in the product to be purchased, such as the product that the user does not intend or the product is insufficient. Is. The user may be notified when the addition or removal of the product by the third party is detected. Further, it is also possible to transmit data related to the addition and removal of products by a third party to the cashier terminal device 30 and the store management server 20.

  The payment processing unit 16d performs a payment process for the purchase target product specified by the purchase target product specifying unit 16c. The payment processing unit 16d permits payment by a payment card or face authentication, provided that all the products selected by the user can be identified and payment by a payment card or face authentication is possible. For example, if the user has already been authenticated by reading the user card or authenticating the face image, and the payment card or face authentication is set to be permitted, payment by the payment card or face authentication can be performed.

  When the payment processing unit 16d performs payment processing by a payment card or face authentication, the payment processing unit 16d notifies the user of the completion of the payment, adds the details of the payment to the payment history data 15c, and transmits it to the store management server 20, The product data 15b is cleared.

  On the other hand, if there is a possibility that an unspecified product is included, the payment processing unit 16d notifies the user to the manned cashier area, and causes the clerk to specify the product and make a payment. .. In this case, when the payment at the manned cash register is completed, the product data 15b is cleared and the manned cash register guidance flag is reset.

  Further, if all the products selected by the user have been specified as purchase target products, but if payment by payment card or face authentication cannot be performed, the payment processing unit 16d performs payment processing in the self-checkout area. The payment processing unit 16d transmits the product data 15b to the cashier terminal device 30 for payment installed in the self-checkout area. Then, when the payment is completed at the cashier terminal device 30, the product data 15b is cleared.

  Next, a processing procedure of the product transport cart 10 will be described. FIG. 5 is a flowchart showing a processing procedure of the product transport cart 10. When the process is started, first, the state recognition unit 16b of the product transportation cart 10 authenticates the user (step S101). This user authentication is performed by face authentication using the face image of the user captured by the state recognition camera 12 and reading of the user card by the card reader / writer 14.

  After that, the purchase target product specifying unit 16c performs a purchase product specifying process (step S102), and the payment processing unit 16d determines whether or not a payment start operation has been received (step S103). If (step S103; No), the process proceeds to step S102, and the purchased product specifying process is repeated.

  When the payment start operation is accepted (step S103; Yes), the payment processing unit 16d executes the payment process (step S104), and the process ends.

  FIG. 6 is a flowchart showing details of the purchase target product specifying process shown in FIG. When the purchase target product specifying process is started, the purchase target product specifying unit 16c determines whether or not a product is added or taken out (step S201). Specifically, when the product recognizing unit 16a detects an increase in the product or when the state recognizing unit 16b recognizes the operation of putting the product in the basket, it is determined as "addition of product". Similarly, when the product recognizing unit 16a detects the decrease of the product, or when the state recognizing unit 16b recognizes the operation of returning the product to the product shelf, it is determined that the product is taken out.

  If the addition or removal of the product is not detected (step S201; No), the process directly returns to the original process. On the other hand, when the addition or removal of the product is detected (step S201; Yes), the product recognition unit 16a recognizes the product (step S202), and the state recognition unit 16b recognizes the state (step S203).

  If the product cannot be identified by either the product recognition or the state recognition (step S204; No), the purchase target product specifying unit 16c performs a detailed check (step S206). This detailed check can be performed, for example, by transmitting a detailed check request to the store management server 20, the store management server 20 performing image processing on the image captured by the in-store camera 40 to identify the product, and notifying the product transport cart 10. Good. Alternatively, when the store management server 20 receives the detailed check request, the salesclerk in charge operates the backyard terminal device 50 to confirm the image pickup result of the in-store camera 40 to identify the product and notify the product transport cart 10. You may.

  If the product cannot be specified even by the detailed check (step S207; No), the purchase target product specifying unit 16c sets a manned cash register guidance flag (step S208) and returns to the original process.

  When the product can be specified from the recognition of the product or the recognition of the state (step S204; Yes), or the product can be specified by the detailed check (step S207; Yes), the purchase target product specifying unit 16c determines the specified product. Is added or deleted to update the product data 15b (step S205), and the process returns to the original process.

  FIG. 7 is a flowchart showing details of the settlement process shown in FIG. When the payment processing unit 16d starts the payment processing, it first determines whether or not the manned cash register guidance flag is set (step S301). If the manned cash register guidance flag is set, the settlement processing unit 16d gives a notice for guiding the user to the manned cash register (step S308). Then, the product data 15b is transmitted to the cashier terminal device 30 in the manned cashier area (step S307).

  If the manned cash register guidance flag is not set (step S301; No), the payment processing unit 16d determines whether cart payment (payment card or face authentication) is possible (step S302). If the cart payment cannot be made (step S306; No), the payment processing unit 16d gives a notice for guiding the user to the self-checkout (step S306). Then, the product data 15b is transmitted to the cashier terminal device 30 in the self cashier area or the manned cashier area (step S307).

  Further, although cart payment is possible (step S302; Yes), if cart payment is not selected (step S303; No), the product data 15b is transmitted to the cashier terminal device 30 in the self cashier area or the manned cashier area. (Step S307).

  If the cart payment is possible (step S302; Yes) and the cart payment is selected (step S303; Yes), the payment processing unit 16d executes payment by a payment card or face authentication (step S304).

  After step S304 or step S307, the payment processing unit 16d ends the payment (step S305) and returns to the original processing.

  Next, variations of various processes will be described. FIG. 8 is an explanatory diagram of variations of various processes. First, as a method of recognizing a product, identification by deep learning or reading of identification information such as a barcode can be used. Further, in the state recognition, it is possible to recognize the state and change of the product shelf, the user's action (picking up and returning the item), the user's face, the user's position, the surrounding person and action and the like. The position of the user can be inferred from the surrounding situation, for example. By estimating the position of the user, it is possible to know which product category the user is in, and narrow down the candidates for image recognition of the product.

  Further, the product to be purchased may be specified by the product transport cart 10, or may be requested by an external device, for example, the store management server 20. Further, there are variations such as payment by face authentication in the product transportation cart 10, payment by a payment card in the product transportation cart 10, and requesting the payment to the cashier terminal device 30.

  Here, a learned model using deep learning will be described. FIG. 9 is an explanatory diagram of the structure of the learned model. As shown in FIG. 9, the learned model is a multilayer neural network having an input layer, a plurality of intermediate layers, and an output layer. The plurality of input nodes existing in the input layer correspond to a plurality of pixels included in the image of the product, for example, and the pixel value becomes an input value to the input node. The output layer corresponds to the product identification result, and one output node is provided for each identification result.

  The intermediate nodes included in the intermediate layer are connected to the nodes in the layers before and after, respectively, and the parameters of this internode connection are optimized by the learning process. Then, when the image of the product is input to the input layer, the value of each output node is obtained based on the input value and the parameter of the inter-node connection. The value of the output node indicates the strength of the feature of the identification result assigned to the output node. Then, the strengths of the features of the identification results indicated by the plurality of output nodes are compared, and if the features of the particular identification result appear sufficiently stronger than the features of other identification results, the identification result Is output.

  In the description so far, the case where the actual product displayed on the product shelf is purchased has been described as an example, but it is also possible to purchase the product at an online store and settle the payment together with the actual product. FIG. 10 is an explanatory diagram of purchase of actual products and products of online stores. In FIG. 10, an explanation image as a product sample of an online store and a two-dimensional code are displayed on the product shelf.

  When the user captures an image of the two-dimensional code with the terminal device linked to the product transportation cart 10, the product transportation cart 10 reads the two-dimensional code and adds the corresponding product to the product data 15b of the product in the online store.

  Therefore, when the item data 15b is settled, the sum of the amount of the actual item in the basket and the amount of the item in the online store becomes the amount of settlement. The actual products in the basket are handed to the users as they are, and the products in the online store are delivered at a later date. The delivery destination can be registered in advance by the user, for example.

  FIG. 11 is an explanatory diagram of a usage example of data managed by the store management server 20. The store management server 20 manages information about businesses as a business table and manages information about consumers as a consumer table. In addition, personalized information about the consumer can be managed.

  The business entity table is a list of sales information, purchase information, etc. for each product, and includes sales priority, purchase price, selling price, profit margin, inventory quantity, and the like. The consumer table shows the personal belongings of the consumer. Consumer-owned personal belongings can be estimated from the purchase record and expiration date. The personal customization information is attribute information associated with an individual and includes preferences such as likes and dislikes, allergies, family structure, purchase history, menu history, and the like.

  The store management server 20 can accurately analyze the demand of the consumer and the supply of the business by using these pieces of information, and can make a proposal to satisfy both of them at a high level. Specifically, it proposes products and menus to consumers, and proposes purchasing and taking out products to businesses.

  For example, it is possible to present a product that the business wants to sell (a product with sufficient inventory and a high profit margin) to consumers who need it, or to propose a menu that matches the attributes of the consumer using the product that the business wants to sell. it can. In addition, it is possible to make a proposal to predict a purchase target product from a product owned by a consumer and reflect the product to be purchased in a store or put out.

  As described above, in the present embodiment, the product transportation cart 10 recognizes the product placed by the user, recognizes the surrounding state, and uses the product recognition result and the state recognition result to identify the purchase target. The product is specified, and the specified product is settled. Therefore, the product selected by the user can be specified with high accuracy, and the settlement of the product can be made efficient.

  Specifically, the product transportation cart 10 uses image processing to recognize the arrangement of products in the surroundings, the operation of the user placing the product on the product transportation cart, the operation of taking out and returning the product placed on the product transportation cart, and the like. be able to.

  Further, the product transport cart 10 can recognize the product by image processing. Recognition by image processing is performed, for example, by inputting an image of a product to a pre-generated learned model. When the identification information attached to the product is acquired, the identification information is further used to recognize the product.

  Note that, in the present embodiment, the configuration in which the product transport cart 10 specifies the purchase target product is illustrated, but the purchase target product may be specified by an external device (for example, the store management server 20).

  Further, in the present embodiment, the configuration in which the product transport cart 10 performs the payment for the purchase target product is illustrated, but the payment for the purchase target product may be performed by an external device (for example, the cashier terminal device 30).

  Further, in the present embodiment, the configuration in which the product recognition camera 11 and the state recognition camera 12 are provided is illustrated, but one camera may be used for both product recognition and state recognition.

  In addition, each configuration shown in the drawings is a schematic function, and does not necessarily have to be physically configured as shown. That is, the form of distribution / integration of each device is not limited to that shown in the figure, and all or a part of them may be functionally or physically distributed / integrated in arbitrary units according to various loads and usage conditions. Can be configured.

  As described above, the product payment system, the product transportation cart, and the product payment method according to the present invention are suitable for accurately specifying the product selected by the user and improving the efficiency of product payment.

10 Product Transport Cart 11 Product Recognition Camera 12 Status Recognition Camera 13 Communication Unit 14 Card Reader / Writer 15 Storage Unit 15a Learned Model 15b Product Data 15c Payment History Data 16 Control Unit 16a Product Recognition Unit
16b State recognition unit 16c Purchase target product specification unit 16d Payment processing unit 20 Store management server 30 Cash register terminal device 40 In-store camera 50 Backyard terminal device

Claims (11)

Product recognition means for recognizing the product placed on the product transport cart that the user uses to transport the product to be purchased,
A state recognition means for recognizing the state around the product transport cart,
Purchase target product specifying means for specifying a purchase target product using the recognition result by the product recognition means and the recognition result by the state recognition means;
And a payment processing unit that performs payment for the product specified by the purchase target product specifying unit.   The product settlement system according to claim 1, wherein the state recognition means recognizes the arrangement of products around the product transportation cart by image processing.   The product settlement system according to claim 1 or 2, wherein the state recognition unit recognizes an operation in which the user places the product on the product transport cart by image processing.   4. The product settlement system according to claim 1, wherein the state recognition unit recognizes an operation of taking out and returning the product placed on the product transport cart by the user by image processing. ..   The product settlement system according to claim 1, wherein the product recognition unit recognizes the product by image processing.   The product payment system according to claim 5, wherein the product recognition unit inputs the image of the product into a pre-generated learned model to recognize the product.   The product settlement system according to claim 5 or 6, wherein, when the product recognition unit acquires the identification information attached to the product, the product recognition unit further recognizes the product by using the identification information. ..   8. The product settlement system according to claim 1, wherein the purchase target product specifying unit is provided in the product transport cart.   The product payment system according to claim 1, wherein the payment processing means is provided in the product transportation cart. A product transportation cart used by a user to transport products to be purchased,
Product recognition means for recognizing a product placed by the user,
A state recognition means for recognizing the surrounding state,
Purchase target product specifying means for specifying a purchase target product using the recognition result by the product recognition means and the recognition result by the state recognition means;
And a payment processing means for performing payment for the product specified by the purchase target product specifying means. A product recognition step in which a user recognizes a product placed on a product carrying cart used for carrying a product to be purchased,
A state recognition step of recognizing a state around the product carrying cart,
A purchase target product specifying step of specifying a purchase target product using the recognition result of the product recognition step and the recognition result of the state recognition step;
A payment processing step of performing payment for the product specified by the purchase target product specifying step.

Images