JP2014235530A - Self shopping system, portable terminal, computer program, and self shopping method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To automate the settlement of account of the bill of products and shorten the time required for the settlement of account when a purchaser purchases the products displayed in a store.SOLUTION: The self shopping system includes a portable terminal installation rest used for placing a portable terminal 2 in a shopping cart 1; and a space used for photographing products from the portable terminal 2 placed on the portable terminal installation rest. Upon detecting that there are products in the space, the portable terminal 2 transmits, to a management server 3, an image photographed by the own camera. On the basis of the image received from the portable terminal 2, the management server 3 identifies the products purchased by a purchaser and performs the settlement of account of the identified purchased products.

Description

  The present invention relates to a self-shopping system, a portable terminal, a computer program, and a self-shopping method.

  Traditionally, in stores such as supermarkets, shopping centers, and various mass merchandisers, the purchaser takes the products from the product shelves, puts them in shopping carts or shopping carts, and carries them to the cash register located near the exit of the store, where cashiers operate The register pays for the goods.

  On the other hand, in recent years, a self-shopping system that automates the settlement of the price of goods has been studied. For example, in the prior art described in Non-Patent Document 1, an IC tag is affixed to a product, and the products are collectively identified by proximity communication with the IC tag of the product in a shopping cart or shopping basket at a cash register. They pay for the goods without taking them one by one from the shopping cart or basket.

  In addition, a self-shopping system that automates product identification has been studied. For example, in the prior art described in Patent Document 1, a purchaser reads a product barcode using a mobile phone terminal in a store, or identifies a product by image recognition from an image of the product and communicates via the Internet. Place an order for delivery of goods. The purchaser settles the price when the product is delivered to the home.

JP 2003-331024 A

Internet <URL: http://internet.watch.impress.co.jp/cda/news/2004/01/30/1938.html>

  However, the above-described conventional techniques have the following problems.

  In the conventional technique described in Non-Patent Document 1, since close proximity communication with IC tags of products in a shopping cart or a shopping basket is performed at a cash register, when there are a large number of purchasers, the waiting time at the cash register Becomes longer.

  In the prior art described in Patent Document 1, there is an inconvenience that the purchaser cannot use the product until it is delivered to the home. In addition, the products displayed in the store are not always delivered to the home. For example, in the case of fresh food such as vegetables and fish, it is considered that a sample of goods to be delivered is displayed at the store. In addition, it takes time for the store clerk to collect and deliver the ordered products.

  The present invention has been made in view of such circumstances, and when a purchaser purchases a product displayed in a store, it is intended to automate the settlement of the price of the product and shorten the time required for the settlement. It is an object of the present invention to provide a self-shopping system, a portable terminal, a computer program, and a self-shopping method.

(1) A self-shopping system according to the present invention includes a shopping cart or shopping cart used by a purchaser who purchases products displayed in a store, a management server, and a portable terminal that communicates with the management server via a wireless communication line. A mobile terminal installation base on which a mobile terminal is placed, and a space for photographing products from the mobile terminal placed on the mobile terminal installation base. When the mobile terminal detects that there is a product in the space, an image capturing unit that outputs an image captured by a camera, a communication unit that communicates with the management server, the image captured by the image capturing unit is managed by the management unit. A terminal application unit that transmits to a server, and the management server communicates with the mobile terminal. And a server application unit that identifies a purchased product purchased by the purchaser based on an image received from the mobile terminal and performs checkout of the identified purchased product. .

(2) The self-shopping system according to the present invention is a product table in which in the self-shopping system of (1), a product to be photographed from the mobile terminal placed on the mobile terminal installation base is placed in the shopping cart or shopping basket. Is provided.

(3) The self-shopping system according to the present invention is the self-shopping system according to (2), wherein a mirror is provided on the product table, and the mirror is moved from the portable terminal placed on the portable terminal installation table to the product table. It arrange | positions so that the side surface of the goods placed in can be image | photographed.

(4) In the self-shopping system according to the present invention, in the self-shopping system according to any one of (1) to (3), the terminal application unit detects a certain time after detecting that there is a product in the space. An image photographed by the photographing unit is transmitted to the management server after the delay or after determining that the product in the space is stationary.

(5) The self-shopping system according to the present invention is characterized in that, in the self-shopping system according to any one of (1) to (4), the mobile terminal includes a sensor that detects that there is a product in the space. And

(6) In the self-shopping system according to the present invention, in the self-shopping system according to any one of (1) to (5), the server application unit performs image recognition of an image received from the mobile terminal. The purchase item purchased by the purchaser is specified.

(7) In the self-shopping system according to the present invention, in the self-shopping system according to any one of (1) to (5), the server application unit reads a barcode from an image received from the mobile terminal. The purchase item purchased by the purchaser is specified.

(8) In the self-shopping system according to the present invention, in the self-shopping system according to any one of (1) to (7), a pass gate that communicates with the management server is provided, and the server application unit The pass gate is opened to the purchaser who has the mobile terminal that has confirmed that the payment has been made.

(9) In the self-shopping system according to the present invention, in the self-shopping system of (8), an opening operation button is provided on the passing gate, and opening operation information on the opening operation button is transmitted from the passing gate to the passing gate. It transmits to a management server, It is characterized by the above-mentioned.

(10) In the self-shopping system according to the present invention, in the self-shopping system according to (8), a proximity sensor that detects the approach of the purchaser is provided in the passing gate, and the proximity detection information detected by the proximity sensor, It transmits to the management server from the passing gate.

(11) The self-shopping system according to the present invention is the self-shopping system according to (8), wherein a proximity communication device is provided in the passage gate, and information communicated with the portable terminal by the proximity communication device is transmitted from the passage gate. It transmits to the said management server, It is characterized by the above-mentioned.

(12) The self-shopping system according to the present invention is the self-shopping system according to any one of (8) to (11) above, wherein a weight scale is provided near the passing gate, and a purchased product enters before passing the passing gate. The weight of the shopping bag is measured with the weighing scale, and the weight of the measurement result is transmitted from the passage gate to the management server. The server application unit integrates the weight of the measurement result received from the passage gate. When there is a difference of a certain ratio with the expected weight of the purchased product, the passage gate is not opened.

(13) The self-shopping system according to the present invention is the self-shopping system according to any one of the above (1) to (12), wherein the shopping cart is opened with a mouth of a shopping bag so that a product can be put inside. It has a structure to deploy and install.

(14) The self-shopping system according to the present invention is the self-shopping system according to any one of (1) to (13), wherein the shopping cart includes a small battery, and the small battery is placed on the portable terminal installation base. The mobile terminal is configured to be connectable.

(15) A mobile terminal according to the present invention is a shopping cart used by a purchaser who purchases products displayed in a store or a mobile terminal placed on a mobile terminal installation base provided in a shopping basket, and the shopping cart Alternatively, the shopping basket is provided with a space for photographing products from the portable terminal placed on the portable terminal installation table, the portable terminal including a photographing unit that outputs an image photographed by a camera, a management server, Upon detecting that there is a product in the space and a communication unit that communicates, an image taken by the photographing unit is identified based on the image, and a purchase product to be purchased by the purchaser is identified. And a terminal application unit that transmits to the management server.

(16) The computer program according to the present invention performs terminal application processing in a mobile terminal placed on a mobile terminal installation base provided in a shopping cart or shopping basket used by a purchaser who purchases a product displayed in a store. The shopping cart or the shopping cart is provided with a space for photographing a product from the mobile terminal placed on the mobile terminal installation base, and the computer program is stored in the mobile terminal. When it is detected that there is a product in the space and a shooting step of capturing with the camera, the captured image is identified based on the image, and the purchased product purchased by the purchaser is identified. Sending to the management server for checkout, and causing the computer to execute Characterized in that it is a computer program.

(17) A self-shopping method according to the present invention includes a shopping cart or a shopping cart used by a purchaser who purchases merchandise displayed in a store, a management server, and a portable terminal that communicates with the management server via a wireless communication line. A self-shopping method in a self-shopping system comprising: a mobile terminal installation base on which a mobile terminal is placed in the shopping cart or a shopping basket; and a product for photographing a product from the mobile terminal placed on the mobile terminal installation base A shooting step in which the mobile terminal shoots with a camera, and when the mobile terminal detects that there is a product in the space, a step of transmitting the shot image to the management server; The management server purchases the purchaser based on the image received from the mobile terminal. Purchased identifying a product is characterized in that it comprises the steps of: performing a settlement of the purchase item is the identified.

  According to the present invention, when a purchaser purchases a product displayed in a store, it is possible to automate the settlement of the price of the product and to shorten the time required for the settlement.

1 is a schematic configuration diagram of a self-shopping system according to an embodiment of the present invention. It is a block diagram of the portable terminal 2 which concerns on one Embodiment of this invention. It is a block diagram of the management server 3 which concerns on one Embodiment of this invention. It is a lineblock diagram showing an example of shopping cart 1 concerning one embodiment of the present invention. It is a flowchart which shows the procedure of the self shopping which concerns on one Embodiment of this invention. It is a flowchart which shows the procedure of the self shopping which concerns on one Embodiment of this invention. It is a flowchart which shows the procedure of the self shopping which concerns on one Embodiment of this invention. It is a flowchart which shows the procedure of the self shopping which concerns on one Embodiment of this invention. It is a flowchart which shows the procedure of the self shopping which concerns on one Embodiment of this invention. It is a block diagram which shows the other example of the shopping cart 1 which concerns on one Embodiment of this invention.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
FIG. 1 is a schematic configuration diagram of a self-shopping system according to an embodiment of the present invention. In FIG. 1, a shopping cart 1 is a cart for a purchaser to carry a product in a store. The mobile terminal 2 is placed on the mobile terminal installation base of the shopping cart 1. The portable terminal 2 communicates with the management server 3 via a wireless communication line. The passing gate 4 is a gate that allows a purchaser who has finished paying for the goods to pass along with the shopping cart 1. The passage gate 4 communicates with the management server 3 via a wired or wireless communication line.

  FIG. 2 is a configuration diagram of the mobile terminal 2 according to the present embodiment. In FIG. 2, the mobile terminal 2 includes a photographing unit 21, a communication unit 22, a display unit 23, and a terminal application unit 24. The photographing unit 21 has a camera and outputs an image photographed by the camera. The communication unit 22 communicates with the management server 3 via a wireless communication line. The display unit 23 has a display screen and displays information on the display screen. The terminal application unit 24 executes a terminal application (terminal application) of the self-shopping system. In the present embodiment, a touch panel is used as the display unit 23 so that the purchaser can input data by operating the display screen of the touch panel. Moreover, a hard key may be provided in the portable terminal 2 separately from the touch panel so that the purchaser can input data by operating the hard key.

  FIG. 3 is a configuration diagram of the management server 3 according to the present embodiment. In FIG. 3, the management server 3 includes a communication unit 31 and a server application unit 32. The communication unit 31 communicates with the mobile terminal 2 via a wireless communication line. The communication unit 31 communicates with the passing gate 4 via a wired or wireless communication line.

  FIG. 4 is a configuration diagram illustrating an example of the shopping cart 1 according to the present embodiment. In FIG. 4, a mobile terminal installation base 41 for placing the mobile terminal 2 is provided on the top of the shopping cart 1. Further, in the shopping cart 1, a product table 42 for placing the product 110 is provided below the portable terminal installation table 41.

  The purchaser 100 places the mobile terminal 2 on the mobile terminal installation base 41 with the camera surface 211 of the mobile terminal 2 facing the product placement base 42. By placing the mobile terminal 2 on the mobile terminal installation base 41 in this way, the display surface (surface on the display screen side of the mobile terminal 2) 231 on the side opposite to the camera surface 211 can be seen by the purchaser 100.

  The purchaser 100 places the product 110 on the product table 42. The mobile terminal installation base 41 is close to the handle 45 of the shopping cart 1 and is in a position where the purchaser 100 can easily operate the mobile terminal 2. Further, the position of the mobile terminal installation table 41 is a position where the product 110 placed on the product installation table 42 can be easily photographed from the mobile terminal 2 placed on the mobile terminal installation table 41. A space for photographing the product 110 from the mobile terminal 2 placed on the mobile terminal installation base 41 is provided.

  The mobile terminal installation base 41 is configured by a member that transmits visible light so that the camera can be photographed through the mobile terminal installation base 41. A folder for holding the mobile terminal 2 is provided on the mobile terminal installation base 41 so that the mobile terminal 2 placed on the mobile terminal installation base 41 can stably photograph the product 110 placed on the product base 42. Also good.

  The shopping cart 1 is configured so that the product 110 can be placed on the product table 42 from the rear side of the shopping cart 1. The product 110 may be placed on the product table 42 from the left or right side of the rear portion of the shopping cart 1.

  An inner door 43 is provided inside the shopping cart 1. The inner door 43 serves to hold the product 110 on the product table 42. A configuration in which the inner door 43 is opened by pushing the product 110 that the purchaser 100 has decided to purchase forward from the product table 42, and the product 110 enters the shopping bag 44 deployed in the shopping cart 1. Has been. The inner door 43 is a structure that is naturally closed by gravity or is closed by a spring, and has resistance and weight that can be opened by human power.

  Next, the operation of the self-shopping system according to the present embodiment will be described with reference to FIGS. 5 to 9 are flowcharts showing a self-shopping procedure according to the present embodiment.

First, the purchase preparation stage will be described with reference to FIG.
(Step S <b> 11) The purchaser 100 places the mobile terminal 2 on the mobile terminal installation base 41 of the shopping cart 1. The portable terminal 2 may be owned by the purchaser 100 or may be lent from a store.

(Step S12) The purchaser 100 activates the terminal application of the portable terminal 2. Thereby, the terminal application unit 24 of the mobile terminal 2 executes the terminal application.

(Step S13) The terminal application unit 24 communicates with the management server 3 through the communication unit 22, and performs user authentication of the mobile terminal 2 using the ID and password. This ensures the safety of using this system.

Next, the product purchase stage will be described with reference to FIG.
(Step S21) The purchaser 100 takes the product 110 from the product shelf.

(Step S22) The purchaser 100 places the product 110 on the product table 42 of the shopping cart 1.

(Step S <b> 23) The terminal application unit 24 detects that the product 110 is placed on the product table 42 based on the image captured by the imaging unit 21. More specifically, the terminal application unit 24 detects the change in which an object is placed in the empty space of the product table 42 based on the image captured by the image capturing unit 21, so that the product is stored in the product table 42. It is determined that 110 is placed. Note that a sensor that uses sound waves, infrared rays, or the like may be provided in the portable terminal 2 so that the sensor can detect that the product 110 has been placed on the product table 42.

(Step S <b> 24) The terminal application unit 24 images the product 110 placed on the product table 42 by the imaging unit 21. The timing of this photographing is preferably after a certain time delay from the time when it is determined that the product 110 is placed on the product table 42. The reason is that immediately after the product 110 is placed on the product table 42, the product 110 may be moving, so that the product 110 waits until the product 110 is stationary before photographing.
Alternatively, the terminal application unit 24 determines that the product 110 placed on the product table 42 is stationary, and determines that the product 110 is stationary, and then the product placed on the product table 42 by the photographing unit 21. 110 may be photographed. This stillness determination may be made based on an image photographed by the photographing unit 21, or may be made based on an output signal from a sensor using sound waves or infrared rays.

(Step S25) The terminal application unit 24 transmits the image photographed by the photographing unit 21 in step S24 to the management server 3 through the communication unit 22.

(Step S <b> 26) The server application unit 32 of the management server 3 receives the image transmitted from the mobile terminal 2 through the communication unit 31. The server application unit 32 analyzes the received image and identifies the product 110. The product 110 is specified by reading a bar code attached to the product 110. Alternatively, the product 110 may be specified by recognizing the product 110 itself or the identification mark of the product 110 based on the image recognition result.

(Step S <b> 27) The server application unit 32 transmits the product information (price, specifications, etc.) of the specified product 110 to the mobile terminal 2 through the communication unit 31. As a specification of goods, a production place, manufacture date, a shelf life, a kind, a cooking method, etc. are mentioned, for example.

(Step S <b> 28) Upon receiving the product information from the management server 3 through the communication unit 22, the terminal application unit 24 of the portable terminal 2 displays the received product information on the display screen by the display unit 23. On the display screen, a purchase confirmation button for confirming the purchase of the product 110 of the displayed product information is also displayed. The purchaser 100 selects the purchase confirmation button displayed on the display screen when confirming the purchase of the product 110 of the product information displayed on the display screen. The selection of various buttons displayed on the display screen may be performed by operating the display screen, or may be performed by operating a hard key provided on the mobile terminal 2. When the purchase confirmation button displayed on the display screen is selected, the terminal application unit 24 uses the communication unit 22 to manage a purchase confirmation notification message for notifying confirmation of the purchase of the product 110 of the product information displayed on the display screen. Send to server 3. The purchaser 100 opens the inner door 43 from the product table 42 and puts the product 110 whose purchase has been confirmed into the shopping bag 44.

(Step S <b> 29) When the server application unit 32 of the management server 3 receives the purchase confirmation notification message from the mobile terminal 2 through the communication unit 31, the purchaser 100 purchases the price of the product 110 indicated by the received purchase confirmation notification message. Add to the amount. The server application unit 32 transmits the purchase amount after the integration to the mobile terminal 2 through the communication unit 31. When the terminal application unit 24 of the mobile terminal 2 receives the purchase amount from the management server 3 through the communication unit 22, the terminal unit 24 displays the received purchase amount on the display screen using the display unit 23. The display screen also displays a purchase end button for ending the purchase and a purchase continuation button for continuing the purchase. The purchaser 100 selects a purchase end button displayed on the display screen to end the purchase. On the other hand, the purchaser 100 selects the purchase continuation button displayed on the display screen when continuing the purchase.

(Step S <b> 30) When the purchase end button displayed on the display screen is selected, the terminal application unit 24 ends the product purchase stage processing of FIG. 6. On the other hand, when the purchase continuation button displayed on the display screen is selected, the terminal application unit 24 proceeds to a detection process for detecting that the next product 110 is placed on the product table 42.

  The purchaser 100 can replace the product 110 placed on the product placement table 42 until the purchase is confirmed (a purchase confirmation button is selected). In this case, the terminal application unit 24 detects that the product 110 has disappeared from the product table 42 by using an image or a sensor photographed by the photographing unit 21. Then, when the terminal application unit 24 detects again that the product 110 has been placed on the product table 42 by an image or a sensor photographed by the photographing unit 21, the product placed on the product table 42 by the photographing unit 21. 110 is photographed. As described above, the information of the captured image of the product 110 returned without being confirmed is not used in the management server 3 to accumulate the purchase amount.

  Next, the product re-specification step will be described with reference to FIG. The product re-specifying stage is performed when the product 110 cannot be specified in step S26 of FIG.

(Step S <b> 41) The server application unit 32 of the management server 3 transmits a re-installation instruction message to the mobile terminal 2 through the communication unit 31. When the terminal application unit 24 of the portable terminal 2 receives the reinstallation instruction message from the management server 3 through the communication unit 22, the terminal application unit 24 causes the display unit 23 to display the received reinstallation instruction message on the display screen. The reinstallation instruction message is a message that requests the purchaser 100 to reinstall the product 110 by changing the orientation of the product 110 on the product table 42 with respect to the camera surface 211.

(Step S <b> 42) When the terminal application unit 24 detects the re-installation of the product 110 by the image or sensor captured by the imaging unit 21, the terminal 110 captures the product 110 placed on the product placing table 42. The terminal application unit 24 transmits the captured image to the management server 3 through the communication unit 22. When the server application unit 32 of the management server 3 receives the image transmitted from the mobile terminal 2 through the communication unit 31, the server application unit 32 analyzes the received image and identifies the product 110. Thereby, when the product 110 can be specified, the processing of FIG. On the other hand, if the product 110 cannot be specified, the process proceeds to step S43.

(Step S43) The server application unit 32 searches a predetermined product database for a similar image from images received from the mobile terminal 2, and determines whether or not a plurality of product candidates can be presented as a result of this search. If it is determined that a plurality of product candidates can be presented, the process proceeds to step S44. If a plurality of product candidates cannot be presented, the process proceeds to step S46.

(Step S <b> 44) The server application unit 32 transmits the product information of the plurality of product candidates to the mobile terminal 2 through the communication unit 31. When the terminal application unit 24 of the mobile terminal 2 receives each item information of a plurality of item candidates from the management server 3 through the communication unit 22, the terminal unit 24 displays the received item information on the display screen by the display unit 23. The display screen also displays a correct button and a reject button for selecting whether the product 110 of each displayed product information is correct or not. The purchaser 100 selects the correct button of the correct product 110 when there is a correct product 110 among the products 110 of each product information displayed on the display screen. On the other hand, if the correct product 110 is not among the products 110 of each product information displayed on the display screen, the purchaser 100 selects the “No” button for the product 110 of each product information. The terminal application unit 24 transmits to the management server 3 the correct / incorrect selection information of the result of selecting the correct button and the reject button of the product 110 of each product information displayed on the display screen.

(Step S45) When the server application unit 32 of the management server 3 receives the correct / incorrect selection information from the portable terminal 2 through the communication unit 31, the server application unit 32 determines whether or not the product 110 is specified based on the received correct / incorrect selection information. Thereby, when the product 110 can be specified, the processing of FIG. On the other hand, if the product 110 cannot be specified, the process proceeds to step S46.

(Step S46) The server application unit 32 transmits a product specification request message for requesting specification of the product 110 to the person in charge of the store via the communication unit 31, or notifies the product specification that the product 110 cannot be specified. An impossible message is transmitted to the mobile terminal 2. When the terminal application unit 24 of the portable terminal 2 receives the product identification impossibility message from the management server 3 through the communication unit 22, the terminal application unit 24 causes the display unit 23 to display the received product identification impossibility message on the display screen.

Next, the settlement stage will be described with reference to FIG.
(Step S51) When the purchase end button displayed on the display screen of the mobile terminal 2 is selected by the purchaser 100, the terminal application unit 24 of the mobile terminal 2 transmits a purchase end message to the management server 3 by the communication unit 22. To do.

(Step S52) When the server application unit 32 of the management server 3 receives the purchase end message from the portable terminal 2 through the communication unit 31, all the products 110 purchased by the purchaser 100 specified by the received purchase end message are displayed. The accumulated amount obtained by integrating the price is transmitted to the mobile terminal 2 by the communication unit 31. When the terminal application unit 24 of the portable terminal 2 receives the accumulated amount from the management server 3 through the communication unit 22, the terminal unit 24 displays the received accumulated amount on the display screen by the display unit 23. On the display screen, a settlement method menu for selecting a settlement method for the displayed accumulated amount is also displayed. Examples of the settlement method include online payment using electronic money, a credit card, etc., use of a cash payment machine, and the like.

(Step S53) The purchaser 100 selects a settlement method from the settlement method menu, and pays the accumulated amount displayed on the display screen by the selected settlement method. The terminal application unit 24 notifies the management server 3 of the settlement method selected by the purchaser 100 through the communication unit 22.

(Step S54) When the server application unit 32 of the management server 3 receives the settlement method notification from the portable terminal 2 through the communication unit 31, the corresponding accumulated amount has been paid by the settlement method of the received notification. Confirm. This confirmation is performed by receiving a payment completion notice from an online payment institution, a payment completion notice from a cash machine, or the like.

Next, the gate passing stage will be described with reference to FIG.
(Step S61) The purchaser 100 moves to the passing gate 4 together with the shopping cart 1 containing the purchased product 110. The purchaser 100 operates an opening operation button provided on the passing gate 4. The opening operation information on the operation of the opening operation button is transmitted from the pass gate 4 to the management server 3. Alternatively, the proximity sensor provided in the passing gate 4 may detect the approach of the purchaser 100 and transmit the detected approach detection information from the passing gate 4 to the management server 3. When the server application unit 32 of the management server 3 receives the opening operation information or the approach detection information from the passage gate 4 by the communication unit 31, the server application unit 32 performs a control process of the opening operation of the passage gate 4 based on the received information.

(Step S62) The purchaser 100 selects an opening operation button displayed on the display screen of the mobile terminal 2. When the opening operation button is selected, the terminal application unit 24 of the portable terminal 2 transmits opening request information to the management server 3 through the communication unit 22.

(Step S <b> 63) When the server application unit 32 of the management server 3 receives the opening request information from the portable terminal 2 through the communication unit 31, it is checked whether the purchaser 100 specified by the received opening request information has been settled. Check. As a result of the confirmation, the server application unit 32 instructs the passing gate 4 to open the gate when the payment is completed. When the passage gate 4 receives a gate opening instruction from the management server 3, the passage gate 4 opens the gate.

(Step S64) The purchaser 100 passes through the gate together with the shopping cart 1.

  In addition, the purchaser ID is notified to the passing gate 4 using a proximity communication device such as RFID to the portable terminal 2, and the purchaser 100 opens the gate only by approaching the passing gate 4 together with the shopping cart 1. May be. Thereby, the operation of the purchaser 100 is not required, and the purchaser 100 can contribute to smoothly exiting the store.

  Moreover, in order to prevent fraud etc., you may make it use the passage gate 4 in order to pass only a person. In this case, a belt conveyor is provided beside the passing gate 4, the product 110 purchased by the purchaser 100 is placed on the belt conveyor, and the belt conveyor sends the product 110 after the passing gate 4 after the passing gate 4 is opened. To do.

  According to the above-described embodiment, the purchase price can be automatically accumulated until the purchaser 100 comes to the passing gate 4 at the exit of the store. Thereby, the purchaser 100 can finish shopping smoothly without waiting for the turn at the cash register when purchasing the product displayed in the store.

  In addition, the store does not need to prepare a cash register, and it is possible to reduce the labor cost of the store clerk who performs the cash register operation and payment. In addition, since the cash register area is reduced, the operation cost of the store can be reduced.

  Next, examples according to the present embodiment will be described.

  The first embodiment is a modification of the shopping cart 1. FIG. 10 is a schematic configuration diagram for explaining the first embodiment of the shopping cart 1 according to the present invention. FIG. 10 shows a part of the product table 42 of the shopping cart 1 shown in FIG. In FIG. 10, the product table 42 is provided with a mirror 51. The mirror 51 is arranged so that the side surface of the product 110 placed on the product placement table 42 can be photographed from the camera surface 211 of the portable terminal 2 placed on the portable terminal installation table 41.

  It may be difficult to specify the product 110 only with the photographed image of the upper surface of the product 110 (the surface facing the camera surface). In the first embodiment, since the side surface of the product 110 can also be photographed by the mirror 51, the product 110 can be specified using the photographed image on the top surface and the photographed image on the side surface of the product 110. Thereby, the effect which becomes easy to specify the product 110 is obtained. The number of mirrors 51 is not particularly limited, but by providing a plurality of mirrors 51, it is possible to capture the entire side of the product 110.

  The second embodiment is an embodiment of the image recognition process of the server application unit 32 of the management server 3. When the management server 3 recognizes an image of the product 110, the following method or the like can be used as an object matching method for acquiring image analysis data and an identification method for separating the acquired image data.

(Object matching method)
・ SIFT (Scale-invariant feature transform)
・ SURF (Speeded Up Robust Features)
・ HOG (Histogram Oriented Gradient))

(Identification method)
・ SVM (Support Vector Machine)
・ K-means
・ Clustering

  It is also possible to place a plurality of products 110 on the product table 42 of the shopping cart 1 and photograph them together for image recognition. If the resolution of the camera of the mobile terminal 2 is a high resolution such as 10M pixels, the resolution of the image of each product 110 can be secured even if a plurality of products 110 are photographed together. Note that the management server 3 sends a message to the mobile terminal 2 to reduce the number of products or change the direction of the products 110 when the individual products 110 cannot be identified from the images obtained by photographing the plurality of products 110. And sends the message to the purchaser 100.

  Example 3 is a modification of the shopping cart 1. When photographing the product 110 placed on the product placing table 42 of the shopping cart 1 from the camera surface 211 of the portable terminal 2 placed on the portable terminal installation table 41, the product placing table 42 and the inner wall surface of the shopping cart 1 are specified. By coloring the color, the chroma key method can be used for image analysis of the captured image. By this chroma key method, it becomes easy to extract the part of the product 110 from the image excluding the color part. Further, when photographing the product 110 with the mobile terminal 2, lighting with a flash or the like is effective for capturing a clear image.

  Alternatively, the shopping cart 1 may be provided with a small battery, the small battery may be connected to the portable terminal 2 placed on the portable terminal installation base 41 with a USB terminal or the like, and the battery of the portable terminal 2 may be charged.

  The fourth embodiment is a modification regarding gate passage. In self-shopping, it is assumed that the product 110 that has not been settled is illegally put in the shopping bag 44 and taken out. When the payment is confirmed at the passage gate 4, the weight of the shopping bag 44 is measured using a weight meter provided near the passage gate 4, and the weight of the measurement result is transmitted from the passage gate 4 to the management server 3. To do. The server application unit 32 of the management server 3 sets the pass gate 4 when there is a certain percentage difference between the weight of the measurement result received from the pass gate 4 and the estimated weight of the purchased purchased product. Do not open the door. In this case, the person in charge checks the passage gate 4 without opening it. Thereby, fraud can be prevented.

  As a measuring method of the weight, the entire shopping cart 1 containing the product 110 is measured with a weighing scale installed on the floor, or only the shopping bag 44 taken out from the shopping cart 1 is measured with a weighing scale installed on a table, etc. Is mentioned.

  In addition, in order to increase the accuracy of determining fraud by measuring the weight, the shopping bag 44 may be divided into lightweight products and heavy products and the product 110 may be inserted. For example, a structure in which shopping bags 44 of lightweight products and heavy products are arranged on the left and right in the shopping cart 1 can be mentioned.

  The fifth embodiment is a modification of the management server 3. The management server 3 may send various information to the mobile terminal 2 for presentation. For example, the purchaser 100 sends a product list of products 110 that the purchaser wants to purchase to the management server 3 in advance. Then, the management server 3 guides the display location of the product 110 so that the product 110 in the product list can be purchased by the shortest route, and recommends (recommends) the product 110 to the purchaser 100 from the product purchase history. Information such as presenting price transitions, presenting cooking methods, presenting calories, etc. is transmitted to the portable terminal 2 for presentation.

  The sixth embodiment is a modification example regarding the integration of the products 110. In the accumulation stage of the product 110, a paper such as a coupon ticket or a discount ticket held by the purchaser 100 is photographed, and the photographed image is sent from the portable terminal 2 to the management server 3. The server application unit 32 of the management server 3 subtracts the discount amount of the coupon ticket or discount coupon specified by recognizing the captured image from the accumulated amount.

  In addition to paper such as coupons and discount coupons, electronic coupons and discount data are transmitted from the portable terminal 2 to the management server 3. Then, the management server 3 authenticates with an ID and a password as necessary through the network, and subtracts the electronic coupon or the discount amount due to the discount from the accumulated amount. Further, as the history of the purchaser 100, discount points according to the purchase amount may be added in the same manner as the point card.

In addition, the management server 3 may perform the following processing in order to prevent the mobile terminal 2 from being used by a person different from the authorized owner due to theft or the like.
(1) A warning message is sent to the person in charge of the store when the purchase item of the purchaser 100 is significantly different from the usual purchase data.
(2) When a predetermined purchase condition is satisfied, for example, the purchased product is only an expensive product, a warning message is transmitted to the person in charge of the store.

  When the person in charge of the store receives the warning message, the person in charge of the store confirms with the user of the mobile terminal 2 whether or not the user is using the mobile terminal 2 as necessary.

  As mentioned above, although embodiment of this invention was explained in full detail with reference to drawings, the specific structure is not restricted to this embodiment, The design change etc. of the range which does not deviate from the summary of this invention are included.

  In the above-described embodiment, in order to manufacture the shopping cart 1 at a lower cost, the mobile terminal 2 is used instead of providing the shopping cart 1 with advanced functions electronically and mechanically. However, the shopping cart 1 itself may include the photographing unit 21, the communication unit 22, the display unit 23, and the terminal application unit 24 that the mobile terminal 2 has.

  The management server 3 may be installed in a store, or may be realized by cloud computing using a management server function on a communication network via a communication relay point.

  As a wireless communication line between the mobile terminal 2 and the management server 3, LTE of a mobile phone line or WiMAX for broadband communication can be used. Alternatively, a wireless LAN access point may be provided in the store, and WiFi, WiGig, or the like may be used. Alternatively, various types of wireless communication that can be used by the mobile terminal 2 such as infrared wireless communication can be used.

  In the above-described embodiment, the shopping cart 1 is used, but a shopping cart may be used instead of the shopping cart 1.

  In the above-described embodiment, the display unit 23 has a display function and a data input function by using a touch panel for the display unit 23 of the mobile terminal 2, but the present invention is not limited to this. The display unit 23 has only a display function by a display device such as a liquid crystal panel, and an operation unit having an input device such as a hard key is provided separately from the display unit 23 so that the operation unit has a data input function. Also good.

  Further, a computer program for realizing the above-described self-shopping system may be recorded on a computer-readable recording medium, and the program recorded on the recording medium may be read into the computer system and executed. Here, the “computer system” may include an OS and hardware such as peripheral devices.

  “Computer-readable recording medium” means a flexible disk, a magneto-optical disk, a ROM, a writable nonvolatile memory such as a flash memory, a portable medium such as a DVD (Digital Versatile Disk), and a built-in computer system. A storage device such as a hard disk.

  Further, the “computer-readable recording medium” means a volatile memory (for example, DRAM (Dynamic DRAM) in a computer system that becomes a server or a client when a program is transmitted through a network such as the Internet or a communication line such as a telephone line. Random Access Memory)), etc., which hold programs for a certain period of time.

The program may be transmitted from a computer system storing the program in a storage device or the like to another computer system via a transmission medium or by a transmission wave in the transmission medium. Here, the “transmission medium” for transmitting the program refers to a medium having a function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line.
The program may be for realizing a part of the functions described above. Furthermore, what can implement | achieve the function mentioned above in combination with the program already recorded on the computer system, and what is called a difference file (difference program) may be sufficient.

DESCRIPTION OF SYMBOLS 1 ... Shopping cart, 2 ... Mobile terminal, 3 ... Management server, 4 ... Passing gate, 21 ... Shooting part, 22, 31 ... Communication part, 23 ... Display part, 24 ... Terminal application part, 32 ... Server application part, 41 ... mobile terminal installation table, 42 ... product table, 43 ... inner door, 44 ... shopping bag, 51 ... mirror, 100 ... buyer, 110 ... product

Claims (17)

A self-shopping system having a shopping cart or shopping cart used by a purchaser who purchases products displayed in a store, a management server, and a mobile terminal communicating with the management server via a wireless communication line;
The shopping cart or the shopping basket is provided with a mobile terminal installation base for placing a mobile terminal, and a space for photographing products from the mobile terminal placed on the mobile terminal installation base,
The portable terminal is
A shooting unit that outputs images taken with the camera;
A communication unit communicating with the management server;
A terminal application unit that, when detecting that there is a product in the space, transmits an image captured by the imaging unit to the management server,
The management server is
A communication unit communicating with the portable terminal;
A server application unit that identifies a purchased product to be purchased by the purchaser based on an image received from the mobile terminal, and performs settlement of the identified purchased product,
Self-shopping system characterized by that.   2. The self-shopping system according to claim 1, wherein the shopping cart or the shopping basket is provided with a product table on which a product to be photographed from the mobile terminal placed on the mobile terminal installation table is placed. A mirror is provided on the product table,
The mirror is arranged so as to be able to take a picture of a side of a product placed on the product placing table from the portable terminal placed on the portable terminal placing table.
The self-shopping system according to claim 2. The terminal application unit, after detecting that there is a product in the space, after a certain time delay, or after determining that the product in the space is stationary, images taken by the photographing unit, Send to the management server,
The self-shopping system according to any one of claims 1 to 3, wherein   The self-shopping system according to any one of claims 1 to 4, wherein the mobile terminal includes a sensor that detects that there is a product in the space. The server application unit identifies a purchased product purchased by the purchaser by performing image recognition of an image received from the mobile terminal;
The self-shopping system according to any one of claims 1 to 5, characterized in that: The server application unit identifies a purchased product purchased by the purchaser by reading a barcode from an image received from the mobile terminal.
The self-shopping system according to any one of claims 1 to 5, characterized in that: Providing a passing gate for communicating with the management server;
The server application unit opens the passage gate for the purchaser who has the mobile terminal that has confirmed that the purchase product has been settled.
The self-shopping system according to any one of claims 1 to 7, characterized in that: An opening operation button is provided at the passage gate, and opening operation information on the operation of the opening operation button is transmitted from the passage gate to the management server.
The self-shopping system according to claim 8. Proximity sensor for detecting the approach of the purchaser is provided in the pass gate, and the proximity detection information detected by the proximity sensor is transmitted from the pass gate to the management server.
The self-shopping system according to claim 8. Providing a proximity communication device at the pass gate, and transmitting information communicated with the mobile terminal by the proximity communication device from the pass gate to the management server,
The self-shopping system according to claim 8. A weight scale is provided near the passage gate,
The weight of the shopping bag containing the purchased product before passing through the passage gate is measured with the weighing scale, and the weight of the measurement result is transmitted from the passage gate to the management server,
The server application unit does not open the pass gate when there is a certain percentage difference between the weight of the measurement result received from the pass gate and the estimated weight of the purchased purchased product,
The self-shopping system according to any one of claims 8 to 11, characterized in that:   The self-shopping system according to any one of claims 1 to 12, wherein the shopping cart has a structure in which a mouth of a shopping bag is expanded and installed so that merchandise can be put inside. The shopping cart includes a small battery, and the small battery is configured to be connectable to the portable terminal placed on the portable terminal installation base.
The self-shopping system according to any one of claims 1 to 13, characterized in that: A mobile device that is placed on a mobile device installation stand provided in a shopping cart or shopping basket used by a buyer who purchases products displayed in a store,
The shopping cart or the shopping basket is provided with a space for photographing products from the mobile terminal placed on the mobile terminal installation base,
The portable terminal is
A shooting unit that outputs images taken with the camera;
A communication unit that communicates with the management server;
When it is detected that there is a product in the space, the management for identifying the purchased product purchased by the purchaser based on the image taken by the photographing unit and adjusting the specified purchased product A terminal application part to send to the server;
A portable terminal characterized by comprising: A computer program for performing a terminal application process in a mobile terminal placed on a mobile terminal installation base provided in a shopping cart or shopping basket used by a purchaser who purchases a product displayed in a store,
The shopping cart or the shopping basket is provided with a space for photographing products from the mobile terminal placed on the mobile terminal installation base,
The computer program is
A shooting step of shooting with the camera of the mobile terminal;
When it is detected that there is a product in the space, the photographed image is transmitted to a management server that identifies the purchased product purchased by the purchaser based on the image and pays the identified purchased product. Steps,
A computer program for causing a computer to execute. A self-shopping method in a self-shopping system having a shopping cart or shopping cart used by a purchaser who purchases merchandise displayed in a store, a management server, and a mobile terminal communicating with the management server via a wireless communication line ,
The shopping cart or the shopping basket is provided with a mobile terminal installation base for placing a mobile terminal, and a space for photographing products from the mobile terminal placed on the mobile terminal installation base,
A photographing step in which the mobile terminal photographs with a camera;
When the mobile terminal detects that there is a product in the space, the captured image is transmitted to the management server;
The management server identifying a purchased product to be purchased by the purchaser based on an image received from the mobile terminal, and paying out the identified purchased product;
Self-shopping method characterized by including.

Images