JP2022014485A - Remote shopping system and remote shopping method - Google Patents

Abstract

To provide a real-time EC that can eliminate the influence of inventory variance and unnecessary search in a shop without requiring particular preparation, modification, and maintenance of commodity masters.SOLUTION: A remote shopping system 1 deals with commodities displayed in a shop with electronic means, and based on a request through a user terminal 41 from a user via a network 5, presents, to the user terminal 41, an image of a commodity display shelf 23 photographed by a shelf camera 24 provided in a retail shop 2, receives a designation of a desired commodity in the image of the display shelf 23 from the user through the user terminal 41, displays, to a clerk terminal 22, a commodity selection image including an identification mark for identifying the commodity related to the designation, and generates order information based on an operation of a clerk who refers to the commodity selection image including the identification mark.SELECTED DRAWING: Figure 1

Description

The present invention relates to EC (Electronic Commerce) technology, and more particularly to a remote shopping system and a remote shopping method for products actually displayed and sold at retail stores and the like. It is a thing.

With so-called online shopping services becoming widespread, there is an increasing demand for home delivery services that deliver meals and daily necessities to homes, especially among the elderly, residents of depopulated areas, families raising children, and young people living alone. It is thought that it will increase. In this respect, the online mail-order service generally delivers systematically from a distribution center or the like that has a large variety of inventories intensively. Therefore, it may be physically difficult to meet the sudden delivery needs of consumers, such as ordering a small amount of groceries such as rice balls and prepared foods for lunch and delivering them immediately. It is thought that there are many. In addition, there is a need to select such products as if they were in an actual retail store that is used on a daily basis, but the online shopping service should give such a sense of presence. It is difficult.

On the other hand, for example, it is possible for users to refer to the status of products actually displayed and sold by retail stores via the network, and to receive orders from users and deliver them immediately. In addition, a real-time EC mechanism for selling displayed products directly by EC is also being considered.

As a technique related to this, for example, in Japanese Patent Application Laid-Open No. 2005-216005 (Patent Document 1), a webcam is provided in a store to photograph a product displayed in the store, and a communication line is observed while watching a live image of the product. A shopping system that allows merchandise to be ordered from a user terminal connected to is described. Here, the product selected by the user from the video of the webcam is selected by collating the image of the display shelf taken by the webcam with the product database in which the product information and the display coordinate information of the webcam are associated with each other. I am trying to identify it.

Further, although not directly related to EC, regarding grasping the state of the display shelf by photographing the display shelf with a camera, for example, Japanese Patent Application Laid-Open No. 2012-174154 (Patent Document 2) obtains from a real-time camera. When the image of the displayed product display shelf is monitored and a product-free marker provided on the display shelf is detected, the product associated with this is indexed and inventory is being checked at the corresponding position in the display shelf image. An inventory status management device that repeats a message to that effect is described.

Further, in Japanese Patent Application Laid-Open No. 2015-108971 (Patent Document 3), a first image of a bookshelf in which a plurality of books are displayed taken by a camera is compared with a second image taken thereafter. By extracting undisplayed books that exist in one image and not in the second image and displaying the first image in a manner in which the undisplayed books can be identified, the book ordering business in the bookstore is supported. The book management system is described.

Japanese Unexamined Patent Publication No. 2005-216005 Japanese Unexamined Patent Publication No. 2012-174154 JP-A-2015-108971

In order to enable users to order products using mobile terminals such as smartphones and PCs (Personal Computers), the store side posts product information and images of the products to be sold on its website. There is a need. In this regard, retail stores that handle groceries and daily necessities such as convenience stores and supermarkets may target thousands of types of products, and since the revisions and abolition of products are severe, registration and update maintenance of product information The load will be very high. In addition, in the case of ordinary EC, it is basically sufficient to create one website for each business operator, but for example, convenience store chains and franchisee stores sell products that are independently displayed in the stores. In the case of taking the form of this, theoretically, it is necessary to prepare a website for each store, and the maintenance load as a whole becomes very high.

In addition, when posting products on the website, whether or not the products can be purchased because they are in stock in the display shelves, store warehouses, etc. in the store or in the storefront (hereinafter, these may be collectively referred to as "storefront"). Must be shown to the user. Regarding the presence or absence of inventory, the theoretical inventory (= initial inventory + number of deliveries-number of sales) is usually used, but due to some special factors (shoplifting, delivery, inspection mistakes, etc.), the theoretical inventory and the actual inventory do not match ( Inventory shift) can occur frequently. Therefore, even if the order is confirmed based on the theoretical inventory, there may be a situation where there is no actual product to be delivered. Therefore, for example, if the theoretical inventory is less than a predetermined number, inefficient safety measures such as not selling (considered to be out of stock) may be taken.

In addition, when an order is received from a user, the clerk picks the product from the display shelf or the store warehouse while looking at the product list of the ordered contents. At that time, if you are a veteran clerk who is familiar with the product name and the layout in the store, there is no problem, but in the case of a general clerk, you can not quickly find the target product only from the text information of the product list and inside the store The inefficiency of searching can also occur, wasting time and increasing costs.

As described above, in order to realize a home delivery service for over-the-counter products, it is necessary to go through a process that requires a lot of labor and cost, and it is not easy to suppress the cost and realize the service efficiently.

On the other hand, for example, by using the technique described in Patent Document 1 above, it is possible to realize a home delivery service for over-the-counter products. However, in the technique described in Patent Document 1, it is necessary to associate each product information with the display coordinate information of the webcam and store them in the product database in advance. Therefore, when the number of product types increases, the maintenance load on the product database becomes extremely high. Further, it is unrealistic to associate each product information with the display coordinate information of the webcam, for example, for all stores in a chain of convenience stores exceeding 10,000 stores.

Further, the techniques described in Patent Documents 2 and 3 are the same as the techniques described in Patent Document 1 in that the positions and coordinates on the display shelves for each product need to be managed by a database. In particular, the technique described in Patent Document 3 is applicable when products (books) are continuously placed in the same place on the display shelf, but in a retail store where the products are severely revised or abolished, the product display position. And types can change frequently, so there are many situations where it cannot be applied.

Therefore, an object of the present invention is that the preparation, modification, maintenance, etc. of a special product master for realizing real-time EC are not required, that is, the system investment related to these points is not required, and the influence of inventory deviation is not required. , To provide a remote shopping system and a remote shopping method that makes it possible to eliminate unnecessary in-store searches.

The above and other objects and novel features of the invention will become apparent from the description and accompanying drawings herein.

A brief description of the representative inventions disclosed in the present application is as follows.

The remote shopping system according to a typical embodiment of the present invention is a remote shopping system for electronically trading products displayed in a store, via a network and via a user terminal from a user. Based on the request, the image of the display shelf of the product taken by the camera installed in the store is presented to the user terminal, and the image of the display shelf from the user via the user terminal is included. The store clerk who received the designation for the desired product, displayed the product selection image including the identification mark for identifying the product according to the designation on the terminal of the store, and referred to the product selection image including the identification mark. Order information is generated based on the operation of.

The present invention can also be applied to a method executed by an information processing system as described above.

Among the inventions disclosed in the present application, the effects obtained by representative ones are briefly described as follows.

That is, according to a typical embodiment of the present invention, it is not necessary to prepare, modify, or maintain a special product master for realizing real-time EC, that is, system investment related to these points is required. It is possible to realize a remote shopping system and a remote shopping method that can eliminate the influence of inventory deviation and unnecessary in-store search.

It is a figure which showed the outline about the structural example of the remote shopping system which is one Embodiment of this invention. It is a figure which showed the outline of the example of the process flow from ordering of goods to delivery in one Embodiment of this invention. It is a figure which showed the outline of the example which the user in one Embodiment of this invention virtually goes around in a retail store. It is a figure which showed the outline of the example which the user selects the product to purchase on the image of the display shelf in one Embodiment of this invention. It is a figure which showed the outline about the example of the contents of the product basket of the user in one Embodiment of this invention. It is a figure which showed the outline about the example of grasping the store inventory amount of the fryer product in one Embodiment of this invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In addition, in all the drawings for explaining the embodiment, the same parts are in principle the same reference numerals, and the repeated description thereof will be omitted. On the other hand, the parts described with reference numerals in one figure may be referred to with the same reference numerals in the description of other figures, although they are not shown again.

The remote shopping system according to an embodiment of the present invention realizes a home delivery service that accepts orders from users via a network and delivers products that are actually displayed and sold on display shelves at retail stores. It is an information processing system. Users can visually and intuitively select and purchase products through images of the display shelves taken in near real time by a Web camera installed on the display shelves of retail stores.

Further, since the image of the product taken by the Web camera is used as it is for the product identification by the clerk, the work of associating the product image for each product in the product master or the like becomes unnecessary. In addition, since the image of the display shelf almost directly shows the inventory status of each product on display, the influence of inventory deviation can be eliminated. In addition, when the store clerk picks the product, the image of the display shelf when the user places an order is used as it is, so the content and position of the product can be easily grasped, and unnecessary in-store searches can be eliminated. can.

<System configuration>
FIG. 1 is a diagram showing an outline of a configuration example of a remote shopping system according to an embodiment of the present invention. The remote shopping system 1 of the present embodiment includes a store system 21 which is an information processing system installed in each retail store 2 which is a franchisee of a convenience store, and a center system 3 which is operated centrally by the headquarters of the convenience store. , The configuration is connected via a network 5 such as the Internet. A secure network such as VPN (Virtual Private Network) may be used.

The store system 21 is an information processing system having a function of controlling one or more shelf cameras 24 installed in the retail store 2 to acquire image data of the display shelves 23. It also has a function of supporting picking and delivery of products in cooperation with the clerk terminal 22, which is a tablet-type terminal carried by the clerk.

The store system 21 may be configured by using an information processing terminal such as a PC, or may be configured by a server device, a virtual server built on a cloud computing service, or the like. Then, using a CPU (Central Processing Unit) (not shown), middleware such as an OS (Operating System), DBMS (DataBase Management System), and Web server program developed on the memory from a recording device such as an HDD (Hard Disk Drive), and the software thereof. By executing the software running on the above, the above-mentioned various functions in the retail store 2 are realized.

The store system 21 has, for example, each unit such as a center cooperation unit 211, a shelf camera control unit 212, an order processing unit 213, and a cash register processing unit 214 implemented as software. The center cooperation unit 211 has a function of communicating with the center system 3 via the network 5 and exchanging data. For example, the image data taken by the shelf camera 24 acquired by the shelf camera control unit 212, which will be described later, is transmitted to the center system 3. Further, the order instruction data transmitted from the center system 3 is delivered to the order processing unit 213, which will be described later.

The shelf camera control unit 212 controls one or more shelf cameras 24 installed on each display shelf 23 of the retail store 2, photographs the display status of products on the display shelf 23, and uses the acquired image data as a center cooperation unit. It has a function of transmitting to the center system 3 via 211. In the present embodiment, the shelf camera 24 is configured by a Web camera, but it is not particularly limited as long as it is a device / apparatus having a function of capturing an image. The image acquired by the shelf camera 24 may be a moving image or a still image acquired periodically. That is, it does not necessarily have to be a real-time live image, and it may be a quasi-real-time image in which the time lag is suppressed to the extent that the situation of the displayed products does not change significantly.

In the example of FIG. 1, the shelf camera 24 is installed in front of the side plates on both sides of each display shelf 23 with the line of sight facing the product side, but the present invention is not limited to this, and the situation of each product on each display shelf 23 is not limited to this. As long as you can shoot without omission, the installation position and the number of installations do not matter. For example, it may be installed on the top plate of the display shelf 23 and photographed from above, or may be installed on the ceiling or on the opposite side of the display shelf 23. Further, for example, when the display shelf 23 is provided with a door for frozen foods, beverages, etc., the shelf camera 24 may be installed inside the door, or when the inside can be grasped from the outside of the door, the inside of the door may be installed. The shelf camera 24 may be installed on the outside. It may be incorporated in the display shelf 23 and integrally configured. In the present embodiment, all of these various installation methods are also included in "installation on the display shelf 23".

Regardless of which installation method is used, for example, the hand or arm of a customer who is actually standing in front of the display shelf 23 and actually picking up the product may be reflected in the image. Reflection can also occur when the line of sight of the shelf camera 24 shifts due to contact with a person or an object with the shelf camera 24. Depending on the installation position, there is a possibility that parts that can identify an individual, such as the customer's face, body, and clothes, will be reflected. On the other hand, for example, in the shelf camera control unit 212 or the image processing unit 32 of the center system 3 described later, the customer portion may be erased from the image data or a mask such as a mosaic may be applied by image processing. For image processing, for example, a known image processing technique such as a composite method for removing noise such as temporary reflection can be applied.

As will be described later, in the present embodiment, the image taken by the shelf camera 24 is used when a plurality of users select a product in an order. That is, it can be said that the shelf camera 24 is a shared camera used among a plurality of users. Therefore, for example, when moving or enlarging / reducing the captured image of the display shelf 23, the function requiring mechanical control such as "swing" and optical zoom of the shelf camera 24 is a function of the shelf camera 24. It will not be used because it will lead to exclusive use and it will be difficult to control sharing and competition with other users. Processing such as image movement and enlargement / reduction shall be performed as image processing by software.

When a plurality of display shelves 23 are installed in the retail store 2, the display shelves 23 to be referred to can be switched by switching the image of the shelf camera 24 to be referred to based on the instruction from the user, as described later. Switch. As a result, it is possible to realize an interface in which the user wanders around the retail store 2 and virtually moves between the display shelves 23. Further, in order to realize such an interface, for example, in addition to the shelf camera 24 for photographing each display shelf 23, one or more cameras for photographing the aisle and the whole view in the retail store 2 are installed, and these are also installed. It may be controlled at the same time. In the present embodiment, the installation location of the display shelf 23 is not particularly limited, and may include not only the one installed in the retail store 2 but also the one installed in the storefront or the like.

The order processing unit 213 acquires order instruction data from the user received from the center system 3 by the center cooperation unit 211, outputs the order instruction data to the store clerk terminal 22, and picks the target product from the display shelf 23 to the store clerk. Has a function to instruct. Further, regarding the products picked by the clerk from the display shelf 23, the information of the products specifically specified via the cash register processing unit 214, which will be described later, is transmitted to the center system 3 via the center cooperation unit 211 to place an order. It has a function to determine the contents. Based on the confirmed order contents, it may have a function of instructing the clerk to deliver to the home 4 via the clerk terminal 22.

In the present embodiment, when a picking instruction is given, an image of the display shelf 23 by the shelf camera 24 when the user selects a product and information such as an identification mark for identifying the product selected by the user in the image are displayed. Output to the clerk terminal 22. As a result, the clerk can easily visually grasp the contents of the products to be picked and the display position. In addition to this, as will be described later, if possible, the product selected by the user is determined by image processing on the above image data, and text information such as the product name of the product is acquired and displayed together. You may try to do it.

The cash register processing unit 214 has a function of specifying a product purchased by a customer at a retail store 2 by a bar code or the like, inputting product information, recording sales, processing a deposit, and the like. For example, the retail store 2 generally has a function. It can be implemented by a so-called POS (Point of Sales) cash register or POS system that is provided. In addition to processing the products actually purchased by the customers who actually visit the retail store 2, in the present embodiment, the products picked by the clerk based on the contents instructed via the clerk terminal 22 as described above. Similarly, the product is specified by the store clerk reading the product identification information such as the barcode and RFID (Radio Frequency Identification) tag actually attached to the product with the existing POS cash register or the like. That is, in the implementation of this embodiment, no new system investment related to this point is required. The information of the specified product is passed to the above-mentioned order processing unit 213 as the information of the product related to the user's order.

The clerk terminal 22 has a function of receiving and displaying information related to an order instruction and a delivery instruction from the order processing unit 213 via, for example, a wireless LAN (Local Area Network) constructed in the retail store 2. Have. In the present embodiment, information related to instructions such as picking and delivery to the clerk is displayed on the clerk terminal 22, but the present invention is not limited to this, and a display or the like (for example) (for example, not shown) installed in the retail store 2 is not limited to this. , It may be configured to be displayed on a display installed on the wall surface of the store, a display of a POS cash register, or the like).

The center system 3 is an information processing system having a function of acquiring and managing an image of a display shelf 23 taken by a shelf camera 24 in each retail store 2 in cooperation with a store system 21 of each retail store 2 such as a franchisee. be. In addition, there is also a function of acquiring information related to an order from a user using the image from the user terminal 41 via the network 5 and instructing the store system 21 of the target retail store 2 to manage the order information. Have. Using an AI (Artificial Intelligence) engine (not shown), it has a function to recognize images taken by the shelf camera 24, analyze behavioral characteristics based on user's purchase history information, and perform various analyzes such as product sales prediction. You may be doing it.

The center system 3 is composed of, for example, a server device, a virtual server built on a cloud computing service, or the like, and an OS, a DBMS, a Web server program, or the like developed on memory from a recording device such as an HDD by a CPU (not shown). The above-mentioned various functions are realized by executing the middleware of the above and the software running on it. For example, it has a store cooperation unit 31, an image processing unit 32, an order management unit 33, an analysis processing unit 34, and the like implemented as software. It also has a product master database (DB) 35 implemented by a database or the like.

The store cooperation unit 31 has a function of communicating with the store system 21 of each retail store 2 via the network 5 to exchange data. For example, the image data taken by the shelf camera 24 acquired by the shelf camera control unit 212 of the store system 21 is acquired. The image data may be acquired from the shelf camera control unit 212 of the target retail store 2 on demand when there is a request from the user terminal 41, or may be acquired from the shelf camera control unit 212 of each retail store 2. The latest image data may be periodically acquired from 212 and recorded.

Further, the store cooperation unit 31 has a function of transmitting data related to the instruction content of the order received from the user via the user terminal 41 to the order processing unit 213 of the target retail store 2. At this time, as described above, the image data used when the user selects the product and the information such as the identification mark for identifying the selected product may be transmitted together.

The image processing unit 32 performs predetermined image processing and analysis on the image data taken by the shelf camera 24 acquired from the store system 21 of each retail store 2, for example, by AI processing in cooperation with the analysis processing unit 34 described later. Has the function of performing. For example, in the image data of the display shelf 23 taken by the shelf camera 24, when the product image is registered in the product master DB 35 in advance for the portion with the identification mark, the product image is matched with this. , The target product may be specified. The shape, color, design, etc. of the product may be directly extracted and analyzed from the image data by image recognition by AI to specify the product and its number. When the product can be specified from the image data of the display shelf 23, information such as the product name and the sales unit price is acquired from the product master DB 35 as the attribute information of the specified product and used via the user terminal 41. It is also possible to present it to a person.

In the case of a convenience store chain, etc., the image data of each product being developed may already be provided by the central headquarters as a part of the data of the product master DB 35, in which case the product master DB 35 may be provided. It is easy to use the image data of. On the other hand, the remote shopping system 1 of the present embodiment aims to eliminate the influence on the existing product master DB 35, and specifies the product by using the registration information of the product master DB 35 as described above. It is possible to realize a home delivery service by real-time EC without requiring an initial investment in the existing product master DB 35. As a matter of course, it is not necessary to hold the coordinate information of the display position on the display shelf 23 of each product in the product master DB 35 or the like.

In the image processing unit 32, in order to obtain an image to be displayed on the user terminal 41 when the user places an order for a product, as described above, with respect to the image data of the display shelf 23 taken by the shelf camera 24. It may be formed by processing by a predetermined image processing. For example, a known image processing technique such as a composite method may be applied to erase the customer portion reflected in the image data as noise or to apply a mask such as a mosaic. Further, distortion correction processing such as keystone correction may be performed.

The order management unit 33 receives an order instruction from the user via the user terminal 41, and transmits data related to the order contents to the store system 21 of the target retail store 2 via the store cooperation unit 31. Has. The data related to the order contents transmitted to the store system 21 includes image data with an identification mark or the like that identifies the product selected by the user with respect to the image data of the display shelf 23 taken by the shelf camera 24. included. If possible, attribute information such as the product name and sales unit price of the product specified by matching with the image data of the product registered in the product master DB 35 by the image processing unit 32 is included. May be good.

In addition, order information is generated based on the information of the product for which the actual product is actually specified, which is acquired from the store system 21 of the retail store 2, and the order content is approved with the user via the user terminal 41. For the confirmed order, the data including the address information of the user's home 4 registered in advance as account information for the confirmed order is retailed to the target through the store cooperation unit 31. It also has a function of transmitting to the store system 21 of the store 2.

The user terminal 41 owned by each user is, for example, a mobile terminal such as a smartphone or a tablet terminal that can be operated via a touch panel, or an information processing terminal such as a PC. The user accesses the center system 3 via the network 5 using a dedicated application (not shown), a Web browser, or the like, and as will be described later, an image of the display shelf 23 taken by the shelf camera 24 of the target retail store 2. You can directly specify the product to be purchased on the image and place an order while referring to.

<Processing flow>
FIG. 2 is a diagram showing an outline of an example of a processing flow from ordering a product to delivery in the present embodiment. For example, the user first accesses the center system 3 by using a dedicated application, a Web browser, or the like on the user terminal 41, specifies account information, and logs in (S01). At this time, the center system 3 performs user authentication by an account management function (not shown) or the like (S02).

After that, the retail store 2 from which the product is purchased is specified. For example, the user may select from the list of deliverable retail stores 2 presented to the user terminal 41 by the center system 3, the location information of the user terminal 41 and the address information of the home 4 of the delivery destination. The center system 3 may automatically determine the nearest retail store 2 based on the above. Further, instead of specifying the retail store 2 each time the product is purchased, the retail store 2 to be used by default may be set in advance in association with the user's account information. When the retail store 2 is specified, the user terminal 41 displays an image of the display shelf 23 taken by the shelf camera 24 of the target retail store 2 (S03, S04).

At that time, in the store system 21 of the target retail store 2, the shelf camera 24 is controlled by the shelf camera control unit 212 based on the instruction from the center system 3, and the photographed image of the display shelf 23 is acquired ( S05), it is transmitted to the center system 3. Then, in the center system 3, the image processing unit 32 performs predetermined image processing such as noise removal and correction to form an image for display (S04), and the image is displayed on the user terminal 41.

As described above, instead of acquiring the image taken by the shelf camera 24 on demand based on the instruction from the user terminal 41, the latest image acquired periodically is transmitted to the center system 3 in advance. And keep it. When the user operates / instructs the image of the display shelf 23 displayed on the user terminal 41 to display an image of another place of the display shelf 23 or another display shelf 23 (that is, virtual). (When migrating / moving in the retail store 2), the image of the display shelf 23 to be displayed is switched by the same processing as in steps S03 to S05 according to the content.

FIG. 3 is a diagram showing an outline of an example in which a user virtually roams in a retail store. Here, an example of an image of the display shelf 23 and the aisle in the retail store 2 taken by the shelf camera 24 displayed on the screen (touch panel) of the smartphone or the like which is the user terminal 41 is schematically shown. In the part of the aisle, an arrow figure for virtually moving on the aisle is displayed, and by selecting (tapping) this, the image at the position where the user moved on the aisle in that direction is displayed. It shows that it can be switched (moved virtually).

At this time, for example, based on attribute information such as age / gender, occupation, hobby, etc. of the user and information such as purchase history so far, the analysis processing unit 34 analyzes the preference of the user by AI processing. By picking up the recommended products, or by setting the products that the retail store 2 wants to promote or the products that sell well, the place where the products are displayed by the image processing unit 32 is shown in the balloon in the figure. As shown by AR (Augmented Reality) technology, the user may be guided. In this case, it is necessary to set in advance the coordinate information of which place on which display shelf 23 the target product is displayed.

Returning to FIG. 2, the user displays the image of the display shelf 23 on the user terminal 41 at a desired location in the retail store 2, and selects the product to be purchased on the image of the display shelf 23 (S06). .. FIG. 4 is a diagram showing an outline of an example in which a user selects a product to be purchased on an image of a display shelf 23. Here, an example of an image of the display shelf 23 taken by the shelf camera 24 displayed on the screen of the user terminal 41 is shown. The figure on the left shows a state in which an image of a whole view of a specific display shelf 23 is displayed. It is shown that the image can be switched (virtually moved) to the adjacent display shelf 23 by selecting (tapping) the left and right arrow figures at the bottom of the screen.

On the screen shown in the left figure, for example, when the user performs a pinch-out (zoom-in) gesture or operation on the user terminal 41, the image of the display shelf 23 is enlarged and displayed as shown in the right figure. Will be done. This enlarged display may be performed locally by image processing in the application on the user terminal 41, or may be performed remotely by the image processing unit 32 of the center system 3.

Here, when the user selects the vicinity of the product to be purchased (the sandwich in the center in the example of FIG. 4) by an operation such as tapping, an identification mark for identifying the product (in the example of FIG. 4 is a solid ring shape). The figure) is displayed. It should be noted that whether or not some product exists in the vicinity selected by the user may be determined by using a known image processing technique such as contour extraction in the image processing unit 32 of the local or center system 3, for example. The display position and size of the identification mark may be adjusted based on the size and shape of the identified product.

If the user wishes to purchase the selected product, the information about the product is retained in the virtual product basket, and the user can continue to select another product. FIG. 5 is a diagram showing an outline of an example of the contents of a user's product basket according to an embodiment of the present invention. Here, as the contents of the product basket displayed on the screen of the user terminal 41, a product selection image (right side of FIG. 4) in which an identification mark for identifying each of the plurality of products selected by the user is displayed. The image corresponding to the figure of) is displayed. It is also possible for the user to specify the purchase quantity for each product. Information such as a display shelf number, an ID, and an arrangement location that can specify the target display shelf 23 may be associated with each product selection image. As described above, in the present embodiment, it is not necessary to be able to specify what the target product is at this stage, and the clerk grasps the target product and actually picks it from the display shelf 23. It suffices if an image as much as possible can be obtained.

Returning to FIG. 2, when the selection of all the products to be purchased by the user (holding in the virtual product basket) is completed and the user instructs to order these products via the user terminal 41 (S07). , Image data related to the selected product (each product selection image held as the contents of the product basket of FIG. 5) is transmitted to the center system 3. The order management unit 33 of the center system 3 transfers the received information related to the order instruction, that is, the information including the product selection image related to the selected product, to the store system 21 of the target retail store 2 via the store cooperation unit 31. Transfer (S08).

When the order processing unit 213 of the store system 21 acquires the information related to the order instruction via the center cooperation unit 211 (S09), it is displayed on the store clerk terminal 22 (S10). The clerk refers to the product selection image included in the order instruction and the identification mark displayed therein, and picks the target product from the display shelf 23 on which the target product is actually displayed (S11). That is, since the product is picked based on the image of the display shelf 23 displayed on the store clerk terminal 22 (similar to FIG. 5), the target is compared with the case where only the text information of the product name is simply displayed in the list. It is possible for a clerk to visually grasp where in which display shelf 23 in the retail store 2 the product is displayed. When information such as a number that identifies the display shelf 23 is associated with the product selection image, the store clerk may be able to refer to this so that the target display shelf 23 can be easily accessed.

When the product displayed on the display shelf 23 is to be purchased as in the present embodiment, the retail store 2 is between the time when the user selects the product and the time when the clerk picks the product. It is possible that a customer who actually visits the store purchases the target product and the product is out of stock during picking. Therefore, in step S11, the designation of a substitute product or a substitute retail store 2 in the case of out of stock may be separately accepted in advance. For example, when purchasing daily necessities such as groceries, it is considered that there are not so many situations where the user allows only the target product or purchases only at a specific store. Therefore, by accepting the designation of alternative products and alternative stores, it is possible to avoid the loss of sales opportunities and improve the convenience of users.

When the clerk picks all the products, the bar code reader etc. provided by the cash register processing unit 214 (POS cash register etc.) with the bar code etc. attached to the picked product is the same as at the time of normal purchase of the customer who actually visited the store. By performing the cash register process to be read by (S12), the store system 21 specifically identifies the product to be purchased and calculates the charge (S13). As described above, in the present embodiment, the product is specifically specified by the same cash register processing as the normal processing for the customer who visited the store. Therefore, in order to specify the product, for example, the image data of the product is stored in the product master DB 35. It is possible to use the existing product master DB 35 as it is without adding additional information such as the coordinates of the display location or maintaining the latest information, and to the existing product master DB 35. The remote shopping system 1 can be implemented by real-time EC while avoiding the influence of.

Some of the products sold at the retail store 2 do not have a barcode or the like attached to the products, such as fried food prepared foods (flyer products) sold in a case near the cash register. In the present embodiment, for such a product, for example, a clerk can read the bar code of the target product pre-printed on a panel prepared near the cash register, as in the case of normal sales at the store. A product (flyer) that cannot be identified by a bar code or a product shape by an image because the product is specified and input by a means such as a POS cash register button or a manual input by a clerk via a touch panel. It is possible to identify all products, including products, magazines with only a thin spine facing the front, products hidden behind other products and only partly exposed, etc., and target them for remote shopping. be.

It should be noted that, among such fryer products, it is difficult to systematically grasp the amount of in-store inventory, such as how many products are actually displayed in the store. Some products must be disposed of after a certain period of time has passed since they were displayed, and it may be difficult to properly grasp the timing of preparation and replenishment of new products.

FIG. 6 is a diagram showing an outline of an example of grasping the in-store inventory amount of the flyer product. The upper figure schematically shows an example of an image taken by a shelf camera 24 of a part of a case in which fryer products are displayed. In the present embodiment, as shown in the lower figure, for example, contour extraction by image recognition using AI by the image processing unit 32 and the analysis processing unit 34 of the center system 3 for this image (in the lower figure). Judgment of overlap with (dashed line) is performed, and the store inventory amount and its transition are grasped in real time. As a result, it is possible to prepare in advance so that the clerk does not run out of stock of the fryer product, so that it is possible to prevent the stock from running out while maintaining the freshness.

It should be noted that such processing can be applied not only to fryer products but also to other products in general. Further, in the present embodiment, as described above, the store clerk picks the product based on the product image of the display shelf 23 showing the product selected by the user, and reads the bar code or the like by the cash register processing. At that time, the product image of the display shelf 23 is subjected to image recognition using AI as described above to identify the product, and the product information is specified via the clerk terminal 22. May be presented to the clerk to support the clerk's picking of the product, reduce the load, and improve the accuracy of identifying the product.

The identification information (for example, JAN code) of the product to be purchased specified by the cash register processing in the cash register processing unit 214 and the order processing unit 213 of the store system 21 is sent to the center system 3 via the center cooperation unit 211. .. The order management unit 33 of the center system 3 generates order information by acquiring attribute information such as a product name and a sales unit price from the product master DB 35 based on the acquired product identification information (S14), and generates order information (S14). It is presented to the user via the user terminal 41. When the user confirms the order contents and accepts the order (S15), the order management unit 33 of the center system 3 instructs the store system 21 to deliver the order together with the information of the delivery destination specified by the user. Information is transmitted (S16).

In the store system 21 that has received the delivery instruction, the actual delivery process to the user's home 4 is performed (S17). In the store system 21, sales may be recorded at this stage, or sales may be recorded at the registration processing stage in step S12. At this point, the price of the product will be treated as receivable. In the store system 21, in addition to recording sales of each product, delivery status, inventory information, receipt status of receivables, etc. are managed together based on the processing contents of the order processing unit 213 and the cash register processing unit 214. You may. Then, the user receives the product delivered to the home 4 and pays the price to the clerk who delivered the product (S18). The clerk who received the price returns to the retail store 2 and records the receivables on the books by the cash register processing or the like (S19), thereby completing a series of processing related to the transaction.

In the present embodiment, the payment method is configured such that cash on delivery is performed when the product is delivered to the home 4, but the payment method is not particularly limited, and various types such as credit card and point use are used. You can use things as appropriate. Further, in the present embodiment, the delivery of the product to the home 4 is performed by the clerk of the retail store 2, but the present invention is not limited to this, and it is naturally possible to use a specialized staff, an external delivery company, or the like. Is.

As described above, according to the remote shopping system 1 according to the embodiment of the present invention, an order from a user is received via the network 5 and is actually displayed on the display shelf 23 at the retail store 2. It is possible to realize a home delivery service by real-time EC that delivers the products being sold. Then, the user can visually and intuitively select and purchase the product through the image of the display shelf 23 taken in near real time by the shelf camera 24 installed on the display shelf 23 of the retail store 2. ..

Then, the image of the product taken by the shelf camera 24 becomes the product image directly associated with the target product, and the clerk picks the product using this image data as it is as the order information and displays the bar code of the product. The product is specified by the same processing as the normal cash register processing such as reading. As a result, it is not necessary to take measures for the product master DB 35 such as associating a product image for each product, and the existing product master DB 35 can be used as it is, and maintenance work for the product master DB 35 is also unnecessary. That is, when realizing a home delivery service by real-time EC, it is possible to eliminate the need for system investment related to this point.

Further, by adopting such a product specifying method, a method of specifying a product by a matching process between the product image of the display shelf 23 and the product image registered in the product master DB 35 or the like, or a product image of the display shelf 23 It is possible to specify a product even if it is difficult to specify the product by the method of specifying the product by the bar code or the like shown in the above, and all the products in the retail store 2 can be targeted for the home delivery service. For example, products that are displayed on the display shelf 23 and hidden behind other products, products such as magazines lined up on the bookshelf where only a small part of the products are exposed to the front, and flyer products are displayed. Even if the product image on the shelf 23 does not recognize the shape or pattern of the entire product, or if it is difficult, or if the product has a hidden (or no) bar code, it can be targeted for the home delivery service. can.

Further, since the image of the display shelf 23 almost directly shows the inventory status of each of the displayed products, the influence of the inventory deviation can be eliminated. In addition, when the store clerk picks the product, the image data of the display shelf 23 when the user places an order is used as it is as the order information, so that the content of the product and the display position can be easily grasped, and the inside of the store is useless. The search can be ruled out.

In addition, by applying AI to various processes such as various processes for the product image of the display shelf 23, analysis of the sales status of the retail store 2, analysis of attribute information of each user, etc., the home delivery service can be made more efficient and effective. Can be. For example, by identifying a product by AI based on the product image of the display shelf 23 and presenting the product information to the clerk, the load of specifying the product by the clerk can be reduced and the accuracy of specifying the product can be improved. Furthermore, for products such as fryer products whose shape and display status are not constant and do not have a barcode attached, it is possible to grasp the amount of in-store inventory and instruct inventory replenishment at an appropriate timing. In addition, by analyzing the user's tastes and behavioral characteristics by AI, it is possible to recommend the product that the user is likely to purchase or to guide the product to the place where the product is displayed. It is also possible to recommend products to users by analyzing sales, etc., and to efficiently instruct clerk to replenish inventory and prepare fryer products in advance.

Although the invention made by the present inventor has been specifically described above based on the embodiment, the present invention is not limited to the above embodiment and can be variously modified without departing from the gist thereof. Needless to say. For example, the above-described embodiment has been described in detail in order to explain the present invention in an easy-to-understand manner, and is not necessarily limited to the one including all the described configurations. Further, it is possible to add / delete / replace a part of the configuration of the above embodiment with another configuration.

For example, in the above embodiment, the home delivery service by real-time EC is realized for the retail of the product in the retail store 2, but the sales form is not particularly limited, and the wholesale by a wholesaler or the like may be targeted. Further, since it can be widely applied to a service of receiving an order from a user and delivering the target product to home, it is also possible to target a service such as rental that does not involve the sale of the product, for example.

The present invention can be used for a remote shopping system and a remote shopping method for products actually displayed and sold in stores such as retail stores.

1 ... remote shopping system, 2 ... retail store, 3 ... center system, 4 ... home, 5 ... network,
21 ... store system, 22 ... clerk terminal, 23 ... display shelf, 24 ... shelf camera,
31 ... Store cooperation department, 32 ... Image processing department, 33 ... Order management department, 34 ... Analysis processing department, 35 ... Product master DB,
41 ... User terminal,
211 ... Center cooperation unit, 212 ... Shelf camera control unit, 213 ... Order processing unit, 214 ... Cash register processing unit

Claims (6)

A remote shopping system that electronically trades products displayed in stores.
Based on the request from the user via the user terminal via the network, the image of the product display shelf taken by the camera installed in the store is presented to the user terminal, and the user terminal is presented. In response to the designation of a desired product in the image of the display shelf from the user via the above, a product selection image including an identification mark for identifying the product according to the designation is displayed on the terminal of the store, and the product selection image is displayed. A remote shopping system that generates order information based on the operation of a store clerk who refers to a product selection image including an identification mark. In the remote shopping system according to claim 1,
A remote shopping system that identifies a product related to the identification mark in the product selection image by image recognition by artificial intelligence for the product selection image. In the remote shopping system according to claim 1,
By image recognition by artificial intelligence for the image of the display shelf, the inventory quantity of the product actually displayed on the display shelf is determined, and the replenishment instruction of the product to the display shelf is output based on the inventory quantity. , Remote shopping system. In the remote shopping system according to claim 1,
The product recommended for the user is extracted by artificial intelligence analysis of the user's attribute information and / or information including the purchase history, and the recommended product is shown in the image of the display shelf presented to the user terminal. A remote shopping system that displays to the effect that the user is guided to the product to be sold. It is a remote shopping method for electronically trading products displayed in stores.
The information processing system presents to the user terminal an image of a product display shelf taken by a camera installed in the store based on a request from the user via the user terminal via the network. Steps and
A step in which the information processing system accepts a designation for a desired product in an image of the display shelf from the user via the user terminal, and a step.
A step in which the information processing system displays a product selection image including an identification mark for identifying the product according to the designation on the terminal of the store.
A step in which the information processing system generates order information based on an operation of a clerk of the store with reference to a product selection image including the identification mark.
How to do remote shopping. It is a remote shopping method for electronically trading products displayed in stores.
The information processing system presents to the user terminal an image of a product display shelf taken by a camera installed in the store based on a request from the user via the user terminal via the network. Steps and
A step in which the information processing system accepts a designation for a desired product in an image of the display shelf from the user via the user terminal, and a step.
A step in which the information processing system displays a product selection image including an identification mark for identifying the product according to the designation on the terminal of the store.
A step in which a store clerk identifies a product from the display shelf based on the identification mark, and
A step in which the information processing system reads the product identification information attached to the specified product and generates order information.
How to do remote shopping.

Images