JP7145606B2 - Product purchase support system and program - Google Patents

Description

The present invention relates to a merchandise purchase support system and program.

Many food management systems for refrigerators have been proposed, in which an image of the interior of the refrigerator can be remotely viewed using an information terminal. For example, in Patent Document 1, a refrigerator image is periodically transferred to and stored in a server connected to an electric communication line, and an information terminal is operated to acquire the refrigerator interior image from the server and display it on the display device of the information terminal. An output food management device is disclosed.

Also, many electronic commerce (EC) systems have been proposed in which products are purchased using virtual shops installed on the Internet. For example, in Patent Document 2, product information is displayed on an information terminal from a product database, product information and credit card information input from the information terminal are received, and the validity and limit of the user's credit card are determined from the payment database. An e-commerce system for determining and settling is disclosed.

JP-A-2002-236798 JP-A-9-160972

For example, when a user purchases a product at a virtual store according to food stored in a refrigerator, the product information provided by the electronic commerce system is independent of the system that displays the refrigerator interior image and the electronic commerce system. However, it is not necessarily adapted to the purchase tendency of the user, and it may be difficult for the user to purchase the desired product.

SUMMARY OF THE INVENTION The present invention has been made in view of the circumstances as described above, and an object of the present invention is to make product information provided by an electronic commerce system adaptable to user's purchasing tendency .

In order to achieve the above object, the product purchase support system according to the present invention includes an in- fridge information acquisition section, an article detection section, a storage section, a product identification section, and a sales product information supply section. The internal information obtaining unit obtains internal information indicating the internal state of an article storage that stores articles. The article detection unit detects articles stored in the refrigerator from the inside information. The storage unit stores product list information indicating products sold at the actual store or the virtual store. The product identification unit identifies a product corresponding to the product detected by the product detection unit from the products indicated by the product list information, and generates product identification information indicating the identified product. The sales product information supply unit selects, from the product list information stored in the storage unit, products included in past product specification information but not included in the latest product specification information, products repeatedly included in a plurality of product specification information, and products At least one of the products included in the specific information and for which the user has no purchase history at a physical store or a virtual store is selected as a recommended purchase product to be recommended to the user, and sales are made by reflecting the recommended purchase product in the product list information. Product information is supplied to the information terminal used by the user.

According to the present invention, the warehouse management server identifies the sales product of the real store or the virtual shop corresponding to the stocked goods based on the warehouse information acquired from the goods storage, and identifies the sales goods indicating the identified sales goods The information is provided to an e-commerce server that provides users with information on products for sale at the physical or virtual store . Therefore, it becomes possible to adapt the product information provided by the electronic commerce system to the purchasing tendency of the user .

FIG. 1 is a diagram showing a configuration example of a product purchase support system according to Embodiment 1 of the present invention; Diagram to explain the appearance of the goods storage Block diagram showing the functional configuration of the goods storage Block diagram showing the functional configuration of the warehouse management server Block diagram showing the functional configuration of the EC server Block diagram showing the functional configuration of an information terminal Sequence diagram showing an example of the operation of the product purchase support system Flowchart showing the flow of processing for supplying information inside the refrigerator Flowchart showing the flow of product specific information supply processing Flowchart showing the flow of sales product information supply processing Flowchart showing the flow of product purchase support processing Block diagram showing functional configuration of information terminal according to Embodiment 2 (A) is a diagram showing an example of a refrigerator interior confirmation image including a past refrigerator interior image, and (B) is a diagram showing an example of a refrigerator interior confirmation image including the latest refrigerator interior image. (A) is a diagram showing an example of a product category image, (B) is a diagram showing an example of an order image Flowchart showing the flow of product purchase support processing according to the second embodiment A diagram showing an example of a product purchase support image according to the third embodiment Block diagram showing the functional configuration of an EC server according to Embodiment 4 1 is a block diagram showing an example of hardware configuration of an information terminal according to an embodiment of the present invention; FIG.

BEST MODE FOR CARRYING OUT THE INVENTION Below, embodiments of the present invention will be described in detail with reference to the drawings.

(Embodiment 1)
A product purchase support system 1 according to Embodiment 1 of the present invention includes an article storage 100, an internal storage management server 200, an EC server 300, and an information terminal 400, as shown in FIG. The article storage 100, the storage management server 200, the EC server 300, and the information terminal 400 are connected via a network NW so as to be able to communicate with each other. Network NW is, for example, a wide area network such as the Internet. The network NW may include multiple networks such as a local network and the Internet. Although FIG. 1 shows one article storage box 100, one internal storage management server 200, one EC server 300, and one information terminal 400, any number of the article storage boxes 100, internal storage management servers 200, and EC servers are shown. 300 and information terminal 400 are applicable to this embodiment. Further, the in-fridge management server 200 may be a cloud server.

The article storage 100 is, for example, a refrigerator that cools the interior to store articles such as foodstuffs at a low temperature. The article storage 100 has an interior photographing function, captures an interior image of articles stored in the interior, and transmits interior information including the imaged interior image and the imaging date and time via the network NW. It is supplied to the internal management server 200 . Hereinafter, the items stored in the item storage 100 may be referred to as "inventory items".

As shown in FIG. 2, the article storage 100 has three storage compartments, a refrigerating compartment 101, a freezing compartment 102, and a vegetable compartment 103, in order from the top. The refrigerator compartment 101 is partitioned into three storage spaces of an upper stage 101A, a middle stage 101B, and a lower stage 101C by two shelf plates 101P. The front part of the refrigerator compartment 101 is provided with a double-door left door 101L and a right door 101R that can open and close the refrigerator compartment 101 freely. In addition, a camera 101M for capturing an image of the interior of the refrigerator compartment 101 is located near the center of the inner wall surface of the left door 101L on the open/close end side. An open/close sensor 101S for detecting is arranged respectively. A drawer-type door 102A that can freely open and close the freezer compartment 102 is provided on the front part of the freezer compartment 102, and a drawer-type door 102A that can freely open and close the vegetable compartment 103 is provided on the front part of the vegetable compartment 103. A drawer-type door 103A is provided that allows

The in-fridge management server 200 is a server device that manages in-fridge information acquired from the article storage 100 . The in-fridge management server 200 has an in-fridge information database for storing in-fridge information, and accumulates the in-fridge information supplied from the article storage 100 .

The EC server 300 is an electronic commerce server that has a web server function and manages an EC site that operates a virtual store that sells products via the network NW. The EC server 300 generates sales product information for each user based on the product identification information acquired from the warehouse management server 200 , and supplies the information terminal 400 with the sales product information. Also, when an order for a product is received from the information terminal 400, the processing of payment and delivery is executed.

The information terminal 400 is, for example, a smart phone, and performs data communication with the EC server 300 via the network NW. The information terminal 400 incorporates a purchase support program for supporting product purchases on the EC site, accesses the EC server 300 managing the EC site according to user operations, and executes product purchase processing.

Next, functional configuration examples of each of the article storage 100, the storage management server 200, the EC server 300, and the information terminal 400 will be described.

The article storage 100 includes a control unit 110, an imaging unit 120, an open/close detection unit 130, and a communication unit 140, as shown in FIG. These units are electrically connected to each other via a bus line BL.

The control unit 110 centrally controls the operation of each unit of the article storage 100 . The control unit 110 functionally includes an authentication request unit 111 and an in-fridge information supply unit 112 .

The authentication requesting unit 111 transmits an authentication request for authentication of access authority to the in-fridge management server 200 to the in-fridge management server 200 and acquires an authentication result from the in-fridge management server 200 . The authentication request includes identification information of the article storage 100 .

The in-fridge information supply unit 112 supplies in-fridge information to the in-fridge management server 200 . For example, when the refrigerator compartment 101 is closed, that is, when the open/close detection unit 130 detects that both the left door 101L and the right door 101R are closed, the in-fridge information supply unit 112 detects that the image pickup unit 120 is controlled to capture an image of the inside of the storage compartment, and the image of the storage compartment output from the imaging unit 120 and the imaging date and time are associated with the identification information of the article storage compartment 100 to generate the storage compartment information. The in-fridge information supply unit 112 controls the communication unit 140 to transmit the generated in-fridge information to the in-fridge management server 200 . Note that the trigger for the in-fridge information supply unit 112 to generate the in-fridge information and transmit it to the in-fridge management server 200 is not limited to the fact that the refrigerator compartment 101 is closed. The unit 120 may be caused to photograph the inside of the storage room to generate and transmit the inside information. In addition, the article storage 100 is provided with an input device for accepting user operations, and the inside information supply unit 112 indicates that the input device has received an input notifying that the taking in and out of articles has been completed, an instruction to generate inside information, and the like. In-fridge information may be generated and transmitted using this opportunity. The refrigerator interior image included in the refrigerator interior information is compressed by a well-known data compression method such as GIF (Graphics Interchange Format) or JPEG (Joint Photographic Experts Group) method, and is decompressed when the refrigerator interior image is used. may

Under the control of the in-fridge information supply unit 112 , the imaging unit 120 takes an image of the inside of the refrigerator, and outputs the imaged in-fridge image and the imaging date and time to the in-fridge information supply unit 112 . Here, the camera 101M described above in the description of FIG.

The opening/closing detection unit 130 detects opening/closing of a door provided in the storage room. In the present embodiment, opening/closing detection unit 130 detects opening/closing of left door 101L and right door 101R of refrigerator compartment 101 and outputs the detection result to control unit 110 . Here, the opening/closing sensor 101S described above with reference to FIG.

Under the control of the control unit 110, the communication unit 140 performs data communication with the in-fridge management server 200 via the network NW. The communication unit 140 transmits, for example, an authentication request and in-fridge information to the in-fridge management server 200 . The communication unit also receives the authentication result from the refrigerator management server 200 .

The in-fridge management server 200 includes a control unit 210, a storage unit 220, and a communication unit 230, as shown in FIG. These units are electrically connected to each other via a bus line BL.

The control unit 210 centrally controls the operation of each unit of the in-fridge management server 200 . The control unit 210 functionally includes an authentication unit 211 , an in-fridge information acquisition unit 212 , a product list information acquisition unit 213 , an article detection unit 214 , a product identification unit 215 , and a product identification information supply unit 216 .

The authentication unit 211 authenticates the access authority in response to the authentication request from the article storage 100 and the EC server 300, and transmits the authentication result to the request source. The authentication unit 211 collates the identification information of the article storage 100 included in the authentication request from the article storage 100 with the registration information stored in advance in the storage unit 220 , and authorizes access to the storage management server 200 . It authenticates whether or not the article storage 100 is permitted, and transmits the authentication result to the article storage 100.例文帳に追加Also, the identification information of the EC server 300 included in the authentication request from the EC server 300 and the registration information stored in advance in the storage unit 220 are collated, and the EC server permitted to access the in-fridge management server 200 300 and transmits the authentication result to the EC server 300 .

The in-fridge information acquisition unit 212 acquires in-fridge information from the article storage 100 via the communication unit 230 . Further, the in-fridge information acquisition unit 212 stores the in-fridge information acquired from the article storage 100 in the storage unit 220, which will be described later. Note that, when newly storing the in-fridge information in the storage unit 220 , the in-fridge information acquiring unit 212 may store it as difference information indicating a difference from the in-fridge information previously acquired from the article storage 100 .

The product list information acquisition unit 213 acquires, from the EC server 300 via the communication unit 230, product list information indicating products to be sold at the virtual store on the EC site.

The article detection unit 214 detects articles in stock in the article storage 100 . The article detection unit 214 uses a well-known image recognition technique such as deep learning, for example, to recognize an article area in the inside image included in the inside information acquired from the article storage 100, thereby identifying the article. Inventory items in storage 100 are detected.

The product identification unit 215 identifies the EC site sales product corresponding to the stocked product in the product storage 100 . The product identification unit 215, for example, extracts the feature quantity of the inventory product detected by the product detection unit 214, and compares it with the product image of the product list information acquired from the EC server 300 to determine the EC corresponding to the inventory product. Identify the products for sale on the site.

The product identification information supply unit 216 controls the communication unit 230 to transmit to the EC server 300 the product identification information indicating the product sold on the EC site corresponding to the inventory item in the product storage 100 identified by the product identification unit 215. . The product identification information includes identification information of the identified sale product and a specific score indicating the certainty of specifying the sale product.

The storage unit 220 stores various programs and data for controlling the in-fridge management server 200 . The storage unit 220 has, for example, an in-fridge information database 221 and a registration information database 222 . Hereinafter, the database may be referred to as DB.

The in-fridge information DB 221 accumulates and stores in-fridge information acquired from the article storage 100 . Also, the registration information DB 222 stores registration information for authenticating the access authority of each of the article storage 100 and the EC server 300 . The registration information includes, for example, the identification information of the article storage 100, the identification information of the EC server 300, and the like.

Under the control of the control unit 210, the communication unit 230 performs data communication with the article storage 100 and the EC server 300 via the network NW. The communication unit 230 , for example, receives an authentication request and in-fridge information from the article storage 100 and transmits an authentication result in response to the authentication request to the article storage 100 . Also, the communication unit 230 receives, for example, an authentication request from the EC server 300 and transmits an authentication result and product identification information for the authentication request to the EC server 300 .

The EC server 300 includes a control unit 310, a storage unit 320, and a communication unit 330, as shown in FIG.

The control unit 310 centrally controls the operation of each unit of the EC server 300 . The control unit 310 functionally includes an authentication request unit 311 , an authentication unit 312 , a product list information supply unit 313 , a product identification information acquisition unit 314 , a sales product information supply unit 315 and a transaction control unit 316 .

The authentication requesting unit 311 transmits an authentication request for authentication of access authority to the in-fridge management server 200 to the in-fridge management server 200 and acquires the authentication result from the in-fridge management server 200 . The authentication request includes identification information of the EC server 300 .

In response to an authentication request from the information terminal 400 , the authentication unit 312 authenticates whether or not the product purchase support service can be used, and transmits the authentication result to the information terminal 400 . The authentication unit 312 may, for example, authenticate whether or not the product purchase support service can be used based on customer information stored in a customer information DB 323 of the storage unit 320, which will be described later.

The product list information supply unit 313 supplies the product list information to the storage management server 200 . The product list information supplying unit 313 reads out the latest product list information from the product list information DB 321 and controls the communication unit 330 to transmit it to the in-fridge management server 200 .

The product specific information acquisition unit 314 acquires product specific information from the in-fridge management server 200 via the communication unit 330 .

The sales product information supply unit 315 supplies sales product information to the information terminal 400 . The sales product information includes product list information and recommended purchase product information indicating recommended purchase products recommended to the user for purchase. The sales product information supply unit 315 selects a recommended purchase product based on the product specific information stored in the product specific information DB 322 of the storage unit 320 in response to the sales product information transmission request from the information terminal 400 . The sales product information supply unit 315 refers to, for example, the product specifying information of a predetermined period, and repeat purchases of products that have been specified multiple times in the past product specifying information and have not been specified in the latest product specifying information. It is determined that the product is highly likely to be purchased, and is selected as a recommended purchase product. In this way, by chronologically analyzing whether or not the products sold in the virtual store are included in the image of the inside of the refrigerator and selecting recommended products to purchase, the chances of purchasing the products that the user needs can be improved. can be done. The sales product information supply unit 315 generates sales product information including product list information and recommended purchase product information, and controls the communication unit 330 to transmit the generated sales product information to the information terminal 400 . Note that the sales product information supply unit 315 may not consider products whose specific scores included in the product specific information are lower than a preset reference value when selecting recommended products to purchase. Conversely, the sales product information supply unit 315 may select a large number of recommended products to purchase, taking into account products with specific scores lower than the reference value. In addition, the sales product information supply unit 315 may rearrange the order of the products in the product list information so that the recommended purchase products are arranged at the top instead of the recommended purchase product information.

The transaction control unit 316 generally controls processing related to product purchases by the user. When the transaction control unit 316 acquires the order information indicating the product to be purchased from the information terminal 400, the transaction control unit 316 performs order processing including settlement processing, delivery processing, and the like. The transaction control unit 316 refers to the card number, expiration date, usage limit, etc. included in the customer information stored in the customer information DB 323 and performs settlement processing. In addition, the transaction control unit 316 refers to the delivery destination included in the customer information, and carries out delivery processing for the product for which payment has been completed. After executing the order receiving process, the transaction control unit 316 transmits payment information and delivery information to the information terminal 400 . The payment information includes, for example, user ID, purchased product, payment amount, date and time of purchase, and the like. The delivery information includes, for example, information such as a user ID, delivery item, delivery date and time, and delivery status.

The storage unit 320 stores various programs and various data for controlling the EC server 300 . The storage unit 320 has, for example, a product list information DB 321, a product specific information DB 322, a customer information DB 323, and a registration information DB 324.

The product list information DB 321 stores product list information in which information about the product is associated with each product sold at the virtual store. The product list information includes identification information, product images, selling prices, etc. of each product for sale. The product image is an image representing the appearance of the product, and includes the product name attached to the product and the logo of the manufacturer. The product list information is updated at any time as the products sold at the virtual store are changed.

The product specific information DB 322 accumulates and stores product specific information acquired from the in-fridge management server 200 .

The customer information DB 323 stores customer information in which name, address, user ID, password, delivery destination, card number and its expiration date, usage limit, etc. are associated with each EC site user.

The registration information DB 324 stores registration information for authenticating access rights of users who use the EC site from the information terminal 400 . The registration information includes, for example, information such as a user ID, a password, and an identification number of the article storage 100 corresponding to the user.

Under the control of the control unit 310, the communication unit 330 performs data communication with the in-fridge management server 200 and the information terminal 400 via the network NW. The communication unit 330 , for example, transmits an authentication request and product list information to the in-fridge management server 200 and receives from the in-fridge management server 200 an authentication result in response to the authentication request and product identification information. Also, the communication unit 330 receives, for example, an authentication request from the information terminal 400 and transmits the authentication result and sales product information in response to the authentication request to the information terminal 400 .

The information terminal 400 includes a control unit 410, a display operation unit 420, and a communication unit 430, as shown in FIG.

The control unit 410 centrally controls the operation of each unit of the information terminal 400 . The control unit 410 functionally includes an authentication request unit 411 , a sales product information acquisition unit 412 , and a display image control unit 413 .

The authentication requesting unit 411 transmits to the EC server 300 an authentication request requesting authentication as to whether or not the product purchase support service can be used, and acquires an authentication result from the EC server 300 . For example, the authentication request unit 411 instructs the display image control unit 413 to display a user information input image for inputting user information, and controls the communication unit 430 to transmit an authentication request including the user information received by the display operation unit 420. and transmits it to the EC server 300 . User information includes, for example, a user ID and password. In addition to the user ID and password, the user information may be information used for biometric authentication such as fingerprint authentication and face authentication. For example, the POST method of HTTPS (Hypertext Transfer Protocol Secure) communication is used as a transmission method of the authentication request. For example, the authentication requesting unit 411 stores the user information input by the user when using the product purchase support service for the first time, so that when the product purchase support service is used for the second time or later, the input of the user information can be omitted. good.

The sales product information acquisition unit 412 acquires sales product information from the EC server 300 . The sales product information acquisition unit 412 , for example, transmits a sales product information transmission request requesting transmission of sales product information to the EC server 300 according to a user operation, and acquires sales product information from the EC server 300 .

A display image control unit 413 controls a display image displayed on the display operation unit 420 . The display image control unit 413, for example, generates a product purchase support image based on the sales product information acquired from the EC server 300, controls the display operation unit 420, and displays the generated product purchase support image on the display operation unit 420. indicate.

The display operation unit 420 has a display function and a touch operation detection function, and displays various images and receives touch operations by the user. The display operation unit 420 displays various images including a product purchase support image in response to a control signal from the display image control unit 413, for example. In addition, display operation unit 420 accepts a product purchase operation by the user and outputs an operation signal to control unit 410 .

The communication unit 430 performs data communication with the EC server 300 via the network NW under the control of the control unit 410 . The communication unit 430 , for example, transmits an authentication request to the EC server 300 and receives from the EC server 300 an authentication result and sales product information in response to the authentication request.

Next, with reference to FIG. 7, an overview of the operation of product purchase support system 1 according to the present embodiment will be described. Here, transmission and reception of authentication requests and authentication results between devices included in the product purchase support system 1 will be omitted, and transmission and reception of various information between devices and processing executed in each device will be described.

When the article storage 100 captures an image of the inside of the storage room at a predetermined opportunity and generates the storage information including the storage interior image (step S101), it transmits the generated storage information to the storage management server 200 (step S101). S102).

Upon receiving the in-fridge information from the article storage 100 (step S103), the in-fridge management server 200 stores the in-fridge information in the in-fridge information DB 221 of the storage unit 220 (step S104).

After the series of steps S101 to S104 were repeated as needed in the article storage 100 and the in-store management server 200, the product purchase support program was started (step S105A). In response to the user's operation, a sales product information transmission request requesting transmission of sales product information is transmitted to the EC server 300 (step S105).

When the EC server 300 receives a sales product information transmission request requesting transmission of sales product information from the information terminal 400 (step S106), the EC server 300 transmits product list information to the in-fridge management server 200 (step S107).

Upon receiving the product list information from the EC server 300 (step S108), the warehouse management server 200 reads the latest warehouse information of the goods storage 100 associated with the information terminal 400 from the warehouse information DB 221, is detected (step S109). Subsequently, the warehouse management server 200 refers to the product list information acquired from the EC server 300, recognizes the product corresponding to the in-store product (step S110), and sends the product identification information indicating the recognition result to the EC server 300. Send (step S111).

When the EC server 300 receives the product identification information from the warehouse management server 200 (step S112), it refers to the product identification information DB 322 and extracts products that have been recognized in the past but not recognized this time (step S113). Subsequently, the EC server 300 selects the recommended purchase product based on the extraction result (step S114), and transmits sales product information including the recommended purchase product information to the information terminal 400 (step S115).

When the information terminal 400 receives the sales product information from the EC server 300 (step S116), the information terminal 400 displays a product purchase support image based on the sales product information, and transmits the order information indicating the ordered product selected from the sales products by the user to the EC server. 300 (step S117).

When the EC server 300 receives the order information from the information terminal 400 (step S118), it settles the order item (step S119) and performs the delivery procedure for the ordered item (step S120). After that, the EC server 300 transmits order information to the information terminal 400 (step S121).

When the information terminal 400 receives the order information (step S122), it displays it on the display operation unit 420, and ends the product purchase program according to the user's operation (step S122A).

Next, with reference to the flowchart shown in FIG. 8, the internal information supply processing executed by the controller 110 of the article storage 100 will be described. The in-fridge information supply process is a process of generating in-fridge information including an in-fridge image and transmitting it to the in-fridge management server 200 . For example, the control unit 110 starts the internal information supply process when the refrigerator compartment 101 is closed, that is, when the open/close detection unit 130 detects that both the left door 101L and the right door 101R are closed. Start.

When starting the in-fridge information supply process, the control unit 110 first acquires an in-fridge image (step S201). The in-fridge information supply unit 112 of the control unit 110 controls the imaging unit 120 to photograph the inside of the storage chamber, and acquires the in-fridge image and the imaging date and time from the imaging unit 120 .

Subsequently, control unit 110 generates in-fridge information (step S202). The in-fridge information supply unit 112 generates in-fridge information by associating the in-fridge image acquired from the imaging unit 120 and the imaging date and time with the identification information of the article storage 100 .

Next, control unit 110 transmits an authentication request to in-fridge management server 200 (step S203). Authentication requesting section 111 controls communication section 140 to transmit an authentication request including identification information of article storage 100 to internal management server 200 .

After transmitting the authentication request to the refrigerator management server 200, the authentication requesting unit 111 determines whether or not the authentication has succeeded (step S204). Authentication requesting unit 111 determines whether or not an authentication success notification indicating successful authentication has been received from in-fridge management server 200 via communication unit 140 . When it is determined that the authentication has not succeeded (step S204: NO), the authentication requesting unit 111 returns the process to step S203 and repeats transmission of the authentication request until the authentication succeeds.

If it is determined that the authentication has succeeded (step S204: YES), the in-fridge information supply unit 112 controls the communication unit 140 to transmit in-fridge information to the in-fridge management server 200 (step S205). After executing step S205, control unit 110 ends the in-fridge information supply process.

Next, with reference to the flowchart shown in FIG. 9, the product identification information supply process executed by the control unit 210 of the in-fridge management server 200 will be described. The product specific information supply process is a process of generating product specific information based on the internal information acquired from the product storage 100 and supplying the product specific information to the EC server 300 . Upon receiving the authentication request from the EC server 300, the control unit 210 starts the product specifying information supply process.

When starting the product specifying information supply process, the control unit 210 first authenticates the access authority of the EC server 300 in response to an authentication request from the EC server 300 (step S301), and transmits the authentication result to the EC server 300. (step S302). The authentication unit 211 of the control unit 210 collates the identification information of the EC server 300 included in the authentication request from the EC server 300 with the registration information stored in advance in the registration information DB 222 of the storage unit 220, and authenticates the EC server 300. Authenticate access rights. Then, the authentication unit 211 transmits an authentication result notification indicating either authentication success or authentication failure to the EC server 300 according to the authentication result.

After executing the process of step S302, the control unit 210 determines whether or not the product list information has been acquired (step S303). The control unit 210 determines whether or not the product list information has been acquired according to whether or not the product list information has been received from the EC server 300 via the communication unit 230 . If it is determined that the product list information has not been acquired (step S303: NO), the control unit 210 repeats the process of step S303 and waits until the product list information is acquired.

If it is determined that the product list information has been acquired (step S303: YES), the article detection unit 214 of the control unit 210 reads the inside information from the inside information DB 221 of the storage unit 220 (step S304), inventory items are detected (step S305).

Subsequently, the product identification unit 215 of the control unit 210 identifies sales products corresponding to the inventory items detected in step S304 (step S306).

After executing the process of step S306, the product specifying unit 215 generates product specifying information including identification information of the specified sales product and a specific score indicating the certainty of the product, and transmits this product specifying information to the EC server 300. (step S307). After executing step S307, the control unit 210 ends the product specifying information supply process.

Next, sales product information supply processing executed by the control unit 310 of the EC server 300 will be described with reference to the flowchart shown in FIG. The sales product information supply process is a process of generating sales product information based on the product identification information acquired from the storage management server 200 and supplying the information terminal 400 with the information. The control unit 310 starts the sales product information supply process when the sales product information transmission request is received from the information terminal 400 .

When starting the sales product information supply process, the control unit 310 first transmits an authentication request to the in-fridge management server 200 (step S401). The authentication requesting unit 311 controls the communication unit 330 to transmit an authentication request including the identification information of the EC server 300 to the in-fridge management server 200 .

After transmitting the authentication request to the refrigerator management server 200, the authentication requesting unit 311 determines whether or not the authentication is successful (step S402). Authentication requesting unit 311 determines whether or not an authentication success notification indicating successful authentication has been received from in-fridge management server 200 via communication unit 320 . When it is determined that the authentication has not succeeded (step S402: NO), the authentication requesting unit 311 returns the processing to step S401 and repeats transmission of the authentication request until the authentication succeeds.

When determining that the authentication has succeeded (step S402: YES), the control unit 310 transmits the product list information to the in-fridge management server 200 (step S403). Control unit 310 reads the latest product list information from product list information DB 321 of storage unit 320 and controls communication unit 330 to transmit the read product list information to refrigerator management server 200 .

Control unit 310 determines whether or not product identification information has been acquired (step S404). Control unit 310 determines whether or not product identification information has been received from in-fridge management server 200 via communication unit 330 . If it is determined that the product specifying information has not been acquired (step S404: NO), the control unit 310 repeats the process of step S404 and waits until the product specifying information is acquired.

If it is determined that the product identification information has been acquired (step S404: YES), the control unit 310 selects a recommended purchase product (step S405). For example, the sales product information supply unit 315 of the control unit 310 refers to the latest product specific information acquired this time and the past product specific information stored in the product specific information DB 322 of the storage unit 320, and obtains the past product specific information. , but not specified in the latest product specification information, is selected as a recommended purchase product.

Subsequently, the control unit 310 transmits sales product information to the information terminal 400 (step S406). The sales product information supply unit 315 generates sales product information including the latest product list information and recommended purchase product information indicating the recommended purchase product selected in step S405, controls the communication unit 330, and supplies the information terminal 400 Send to After that, the control unit 310 terminates the sale product information supply process.

Next, product purchase support processing executed by control unit 410 of information terminal 400 will be described with reference to the flowchart shown in FIG. The product purchase support process is a process that cooperates with the EC server 300 to assist the user in purchasing a product. Control unit 410 starts product purchase support processing when the product purchase support program is started by the user.

When the product purchase support process is started, the control unit 410 first transmits an authentication request to the EC server 300 (step S501). Authentication request unit 411 of control unit 410 controls display operation unit 420 to display a user information input image on display operation unit 420, prompts the user to input user information, and issues an authentication request including the input user information. to the EC server 300 .

Subsequently, the authentication requesting unit 411 determines whether or not the authentication has succeeded (step S502). The authentication requesting unit 411 determines whether or not an authentication success notification indicating successful authentication has been received from the EC server 300 via the communication unit 430 . If it is determined that the authentication has not succeeded (step S502: NO), the authentication requesting unit 411 returns the process to step S501 and repeats transmission of the authentication request until the authentication succeeds.

If it is determined that the authentication has succeeded (step S502: YES), the sales product information acquisition unit 412 of the control unit 410 transmits a sales product information transmission request requesting transmission of sales product information to the EC server 300 (step S503). ). The sale product information transmission request includes the identification number of the article storage 100 registered in advance by the user of the information terminal 400 .

After transmitting the sale product information transmission request to the EC server 300, the control unit 410 determines whether or not the sale product information has been acquired (step S504). The sales product information acquisition unit 412 of the control unit 410 determines whether sales product information has been received from the EC server 300 via the communication unit 430 . When it is determined that the sales product information has not been acquired (step S504: NO), the sales product information acquisition unit 412 repeats the process of step S504 and waits until the sales product information is acquired.

When determining that the sales product information has been acquired (step S504: YES), the control unit 410 causes the display operation unit 420 to display a product purchase support image (step S505). The sales product information acquisition unit 412 of the control unit 410 instructs the display image control unit 413 to generate and display a product purchase support image based on the sales product information acquired from the EC server 300 . The display image control unit 413 receiving this instruction generates a product purchase support image based on the sales product information, and controls the display operation unit 420 to display the product purchase support image on the display screen of the display operation unit 420. . The user browses the product purchase support image displayed on the display operation unit 420 and selects the desired purchase product.

Subsequently, the control unit 410 determines whether or not the product desired to be purchased has been confirmed (step S506). Control unit 410 determines whether or not display operation unit 420 has received an operation to end the selection of the product desired to be purchased. When determining that the desired purchase product has not been finalized (step S506: NO), the control unit 410 repeats the process of step S506 and waits until the desired purchase product is finalized.

If it is determined that the desired purchase product has been determined (step S506: YES), the control unit 410 transmits order information including the purchase desired product selected by the user and its quantity to the EC server 300 (step S507).

After transmitting the order information, the control unit 410 determines whether or not the order information has been received from the EC server 300 (step S508). If it is determined that the order information has not been received (step S508: NO), the control unit 410 repeats the process of step S508 and waits until the order information is received.

If it is determined that the order information has been received (step S508: YES), the control unit 410 causes the display operation unit 420 to display an order confirmation image (step S509). Control unit 410 generates an order confirmation image based on the received order information, and controls display operation unit 420 to display the order confirmation image on the display screen of display operation unit 420 . The user can view the order confirmation image displayed on the display operation unit 420 to confirm the purchased product. After executing the process of step S509, the control unit 410 ends the product purchase support process.

As described above, according to the product purchase support system 1 of the first embodiment, the warehouse management server 200 detects an inventory item from the image of the interior acquired from the item storage 100, and the detected inventory item The EC server 300 is provided with product specification information indicating the specified product for sale by specifying the corresponding product for sale on the EC site. The EC server 300 selects a recommended purchase product based on the product identification information, and transmits the recommended purchase product information indicating the selected recommended purchase product and the sales product information including product list information indicating products sold on the EC site to the information terminal. 400. This makes it possible to provide product information that matches the user's product purchase tendency.

In the first embodiment, the EC server 300 performs the settlement process, but it may be realized by a settlement server managed by a financial institution or a credit card company.

In Embodiment 1 above, the EC server 300 provides the warehouse management server 200 and the information terminal 400 with product list information indicating products to be sold at the virtual store on the EC site, but the present invention is not limited to this. The EC server 300 may provide, for example, information about products sold by a physical store separately operated by an operator of the EC site to the warehouse management server 200 and the information terminal 400 together with products sold by a virtual store on the EC site. . Then, the refrigerator management server 200 may generate purchase recommendation information for the products sold by the real store as well as for the products sold by the virtual store, and provide the information to the user via the information terminal 400 . As a result, the product purchase support system 1 can improve product purchase opportunities not only at virtual stores on EC sites, but also at actual stores.

(Embodiment 2)
In the second embodiment, an example will be described in which a user can purchase products for sale on an EC site while checking an image of the interior of the goods storage 100 using the information terminal 400 .

Like the product purchase support system 1 according to the first embodiment, the product purchase support system 1 according to the second embodiment includes an article storage 100, an internal management server 200, an EC server 300, and an information terminal 400. They are connected so as to be able to communicate with each other via the network NW. Also, the functional configurations of the article storage 100 and the EC server 300 according to the present embodiment are substantially the same as those of the article storage 100 and the EC server 300 according to the first embodiment.

Registration information DB 222 of storage management server 200 according to the present embodiment stores identification information of article storage 100 and user information of a user who uses the purchase support service on information terminal 400 in association with each other. In addition, in response to an authentication request from information terminal 400, authentication unit 211 collates user information included in the authentication request with user information stored in registration information DB 222, and authorizes access to in-fridge management server 200. Authenticate. In addition, in-fridge management server 200 according to the present embodiment transmits in-fridge information of article storage 100 associated with user information to information terminal 400 in response to an in-fridge information transmission request from information terminal 400. do.

Further, as shown in FIG. 12, the control unit 410 of the information terminal 400 according to the present embodiment is different from the information terminal 400 according to the first embodiment, and functionally further includes an in-fridge information acquisition unit 414. .

The in-fridge information acquisition unit 414 acquires in-fridge information from the in-fridge management server 200 via the communication unit 430 . Specifically, after the product purchase support program is started and authentication of the access authority to the in-fridge management server 200 is completed, the in-fridge information acquisition unit 414 determines that there is unacquired in-fridge information in the in-fridge management server 200. If there is unacquired in-fridge information, the unacquired in-fridge information is acquired from the in-fridge management server 200 .

The display image control unit 413 generates a product purchase support image based on the sales product information acquired by the sales product information acquisition unit 412 and the refrigerator information acquired by the storage information acquisition unit 414, and displays the display operation unit 420. to display the product purchase support image on the display screen.

In the present embodiment, the product purchase support image includes, for example, an interior confirmation image 450 shown in FIGS. 13A and 13B, a product category selection image 460A shown in FIG. including order image 460B shown. Under the control of the display image control unit 413 , these images are displayed on the display screen of the display operation unit 420 by switching them appropriately.

FIG. 13 shows an example in which the in-fridge confirmation image 450 changes over time. FIG. 13A shows an example of a refrigerator interior confirmation image 450 including a past refrigerator interior image 451 at a certain point in time, and FIG. It is an example of an in-fridge confirmation image 450 including an in-fridge image 452 . The past refrigerator interior image 451 is generated, for example, based on the refrigerator interior image included in the refrigerator information obtained before the execution date of the product purchase support program. Also, the latest refrigerator interior image 452 is generated, for example, based on the refrigerator interior image included in the refrigerator information most recently acquired from the refrigerator management server 200 . Further, the refrigerator interior confirmation image 450 is provided with a switching button (not shown) for switching and displaying the latest refrigerator interior image 452 and the past refrigerator interior image 451 . When the user presses the switch button, the display image control unit 413 switches between the past interior image 451 and the latest interior image 452 in the interior confirmation image 450 and displays them on the display screen. Note that the refrigerator interior confirmation image 450 may be provided with a display area indicating the date and time when the previous refrigerator interior image 451 and the latest refrigerator interior image 452 were taken. Further, instead of the switching button, the refrigerator interior confirmation image 450 includes a transition button or a slide bar for switching and displaying a plurality of past refrigerator interior images 451 having different photographing dates and times and the latest refrigerator interior image 452 in order of photographing date and time. may be provided.

As shown in FIGS. 13A and 13B, the past refrigerator interior image 451 and the latest refrigerator interior image 452 correspond to the upper shelf 101A, the middle shelf 101B, and the lower shelf 101C provided in the refrigerator compartment 101. It is divided into three areas: 455A, middle stage 455B, and lower stage 455C. In the past in-fridge image 451, an article 456A is displayed in the upper row 455A and an article 456C is displayed in the lower row 455C, respectively. , an item 456C is displayed respectively. That is, the article 456A currently stored in the upper stage 101A of the refrigerating chamber 101 is gone, replaced by an article 456D, an article 456B is newly stored in the middle section 101B, and an article 456C is continued in the lower section 101C. properly stored. The user uses a switch button, a transition button, or a slide bar (not shown) to switch between and display the past refrigerator interior image 451 and the latest refrigerator interior image 452, and compare the images to determine the interior of the refrigerator compartment 101. It is possible to easily confirm the presence or absence of the article.

Further, in the refrigerator interior confirmation image 450, a display switching image labeled "supermarket" for accepting an operation to switch the display to the product category image 460A is displayed at the lower right of the past refrigerator interior image 451 and the latest refrigerator interior image 452. A button 453 is provided. The user can change the display of the display screen from the refrigerator interior confirmation image 450 to the product category image 460A by pressing the display switching button 453 while the past refrigerator interior image 451 or the latest refrigerator interior image 452 is being displayed. can. In addition, in the refrigerator interior confirmation image 450, a thumbnail of the product category image 460A is provided instead of the display switching button 453, and the display of the display screen is changed from the refrigerator interior confirmation image 450 to the product category image 460A in accordance with the user's tap operation on this thumbnail. You can transition to

The product category image 460A indicates the category classification of products sold at the virtual store on the EC site, and includes a product category selection image 461 and a display switching button 463. Category selection button 461A, category selection button 461B for accepting fish selection operation, category selection button 461C for accepting selection operation for vegetables, category selection button 461D for accepting selection operation for beverages, purchase recommended products recommended for purchase by EC site A category selection button 461E for accepting a selection operation of is shown. The user can select a product category by pressing one of the category selection buttons 461A to 461E. Upon receiving a selection operation by the user, the display image control unit 413 causes the display image to transition from the product category selection image 460A to the order image 460B. In addition, the product category selection image 461 may be provided with category selection buttons for receiving selection operations such as sweets and fruits.

Further, in the product category image 460A, a display switching button 463 labeled "refrigerator" is provided for accepting an operation to switch the display to the refrigerator interior confirmation image 450. FIG. By pressing the display switching button 463 , the user can change the display of the display screen from the product category image 460</b>A to the in-fridge confirmation image 450 . In the product category image 460A, a thumbnail of the inside confirmation image 450 is provided instead of the display switching button 463, and the display of the display screen is changed from the product category image 460A to the inside confirmation image 450 according to the user's tap operation on this thumbnail. You can transition to

The order image 460B displays products included in the category selected in the product category selection image 461, and accepts orders for them. The order image 460 B includes a product selection image 462 , display switching buttons 465 and 466 and a selected product confirmation button 464 . The product selection image 462 includes a product image 462A showing the appearance of each product, a purchase quantity column 462B showing the purchase quantity, purchase quantity increase/decrease buttons 462C and 462D for adjusting the purchase quantity, and a scroll bar 462E at the right end. is provided. The user can browse all the products included in the category selected in the product category selection image 461 by moving the display area of the product selection image 462 using the scroll bar 462E in the product selection image 462 . In addition, the user can press purchase quantity increase/decrease buttons 462C and 462D on the left and right of the desired product image 462A to check the purchase quantity in the purchase quantity column 462B and select the desired purchase product.

In the product selection image 462 shown in FIG. 14B, in response to the selection of the category selection button 461E indicated as "recommended product" in the product category selection image 461 of the product category image 460A, the product category selection image 461E shown in FIG. ) and (B) are displayed assuming the inventory status indicated by the past refrigerator interior image 451 and the latest refrigerator interior image 452 . The recommended purchase product is displayed in the product selection image 462 based on the recommended purchase product information included in the sales product information acquired from the EC server 300 . The sales product information supply unit 315 of the EC server 300 considers the inventory status shown in FIGS. 456A is selected as a recommended product for purchase as a product that is highly likely to be purchased again by the user. In addition, the product for sale information supply unit 315 repeatedly detects and stores the product 456C included in both the past image 451 and the latest image 452 of the inside of the refrigerator by the product detection unit 214 and the product identification unit 215 of the storage management server 200. The specified product is selected as the recommended product for purchase as a product with a high degree of user preference and a high possibility of frequent report purchases. As a result, the product 456A and the product 456C are displayed in the product image 462A of the product selection image 462 shown in FIG. 14B.

Above the product selection image 462, a selected product confirmation button 464 labeled "cart" is provided. The user can display all the desired products selected in the product selection image 462 by pressing the selected product confirmation button 464 . The product category image 460A may also be provided with a selected product confirmation button 464 so that the selected purchase desired product can be confirmed even while the product category image 460A is being displayed.

In addition, a display switching button 465 labeled “refrigerator” is provided at the lower right of the product selection image 462 . The user can change the display of the display screen from the order image 460B to the in-fridge confirmation image 450 by pressing the display switching button 465 . In addition, in the order image 460B, a thumbnail of the inside confirmation image 450 is provided instead of the display switching button 465, and the display of the display screen is changed from the order image 460B to the inside confirmation image 450 according to the user's tap operation on this thumbnail. You may let

A display switching button 466 labeled “return” is provided at the lower left of the product selection image 462 . By pressing the display switching button 466, the user can switch the display of the display screen from the order image 460B to the product category image 460A and select another product category. In the refrigerator interior confirmation image 450, a thumbnail of the product category image 460A is provided instead of the display switching button 466, and the display of the display screen is changed from the order image 460B to the product category image 460A according to the user's tap operation on this thumbnail. You may let

Next, product purchase support processing executed by control unit 410 of information terminal 400 according to the second embodiment will be described with reference to the flowchart shown in FIG. The product purchase support process according to the second embodiment is a process that cooperates with the storage management server 200 and the EC server 300 to assist the user in purchasing a product. Note that the product purchase support processing according to the second embodiment includes the same processing as the product purchase support processing according to the first embodiment. The explanation of the processing is simplified. Control unit 410 starts product purchase support processing when the product purchase support program is started by the user.

When the product purchase support process is started, the control unit 410 first transmits an authentication request to the in-fridge management server 200 and the EC server 300 (step S601). Authentication request unit 411 of control unit 410 controls display operation unit 420 to display a user information input image on display operation unit 420, prompts the user to input user information, and issues an authentication request including the input user information. are transmitted to the in-fridge management server 200 and the EC server 300, respectively.

Subsequently, the authentication requesting unit 411 determines whether or not the authentication has succeeded (step S602). The authentication requesting unit 411 determines whether or not an authentication success notification indicating successful authentication has been received from each of the refrigerator management server 200 and the EC server 300 via the communication unit 430 . If it is determined that the authentication has not succeeded (step S602: NO), the authentication requesting unit 411 returns the processing to step S601 and repeats transmission of the authentication request until the authentication succeeds.

If it is determined that the authentication has succeeded (step S602: YES), the sales product information acquisition unit 412 of the control unit 410 transmits a sales product information transmission request requesting transmission of sales product information to the EC server 300 (step S603). ). The sale product information transmission request includes the identification number of the article storage 100 registered in advance by the user of the information terminal 400 .

Subsequently, control unit 410 determines whether or not sales product information has been acquired (step S604). When it is determined that the sales product information has not been acquired (step S604: NO), the sales product information acquisition unit 412 repeats the process of step S604 and waits until the sales product information is acquired.

If it is determined that the product information for sale has been acquired (step S604: YES), the in-fridge information acquisition unit 414 of the control unit 410 transmits to the in-fridge management server 200 an in-fridge information transmission request requesting transmission of in-fridge information. (step S605). The in-box information transmission request requests transmission of the latest and past in-box information of the article storage 100 associated with the user of the purchase support service. The past in-fridge information may be, for example, the in-fridge information acquired from the article storage 100 on the previous day or the day before.

Subsequently, the in-fridge information acquisition unit 414 determines whether or not in-fridge information has been acquired (step S606). If it is determined that the in-fridge information has not been acquired (step S606: NO), the in-fridge information acquisition unit 414 repeats the process of step S606 and waits until the in-fridge information is acquired.

If it is determined that the in-fridge information has been acquired (step S606: YES), that is, after both the sales product information and the in-fridge information have been acquired, the display image control unit 413 of the control unit 410 generates a product purchase support image. (Step S607). The display image control unit 413 generates a refrigerator interior confirmation image 450 as a product purchase support image based on the refrigerator interior image included in the refrigerator information acquired from the refrigerator interior management server 200, and displays the sales merchandise acquired from the EC server 300. A product category image 460A and an order image 460B are generated based on the information.

After generating the product purchase support image, the display image control unit 413 controls the display operation unit 420 to first display the refrigerator interior confirmation image 450 on the display screen (step S608). In the refrigerator interior confirmation image 450, as described above, the past refrigerator interior image 451 and the latest refrigerator interior image 452 are alternately displayed. After displaying the refrigerator interior confirmation image 450, the display image control unit 413 determines whether or not the user has performed a display switching operation (step S609). The display image control unit 413 determines whether or not the display switching button 453 marked with "super" in the refrigerator interior confirmation image 450 has received a pressing operation by the user. When it is determined that the display switching operation has not been performed (step S609: NO), the display image control unit 413 repeats the processing of step S609 and waits until the display switching operation is performed.

When it is determined that the display switching operation has been performed (step S609: YES), the display image control unit 413 controls the display operation unit 420 to switch from the refrigerator interior confirmation image 450 to the product category image 460A and display it on the display screen. (Step S610). After displaying the product category image 460A, the display image control unit 413 determines whether or not the user has performed a display switching operation (step S611). Display image control unit 413 determines whether or not display switching button 463 marked with "refrigerator" in product category image 460A has received a pressing operation by the user. If it is determined that the display switching operation has been performed (step S611: YES), the display image control unit 413 returns the process to step S608 and displays the refrigerator interior confirmation image 450. FIG.

When determining that the display switching operation has not been performed (step S611: NO), the display image control unit 413 determines whether or not a product category has been selected (step S612). The display image control unit 413 determines whether or not any of the category selection buttons 461A to 461E of the product category selection image 461 has received a pressing operation by the user. When determining that no product category has been selected (step S612: NO), the display image control unit 413 returns the process to step S611.

When it is determined that a product category has been selected (step S612: YES), the display image control unit 413 controls the display operation unit 420 to switch from the product category image 460A to the order image 460B and display it on the display screen (step S612: YES). S613). At this time, a product image 462A showing the appearance of the product included in the product category selected in step S612 is displayed in the product selection image 462 of the order image 460B.

After displaying the order image 460B, the display image control unit 413 determines whether or not the user has performed a display switching operation to the product category image 460A (step S614). The display image control unit 413 determines whether or not the display switching button 466 marked with "return" in the order image 460B has received a pressing operation by the user. If it is determined that the display switching operation to product category image 460A has been performed (step S614: YES), the process returns to step S610 to display product category image 460A.

When it is determined that the display switching operation to product category image 460A has not been performed (step S614: NO), display image control unit 413 further determines whether the display switching operation to refrigerator interior confirmation image 450 has been performed by the user. is determined (step S615). The display image control unit 413 determines whether or not the display switching button 465 marked with "refrigerator" in the order image 460B has received a pressing operation by the user. If it is determined that the display switching operation to the refrigerator interior confirmation image 450 has been performed (step S615: YES), the process returns to step S608, and the refrigerator interior confirmation image 450 is displayed.

If it is determined in step S615 that the display switching operation to the inside confirmation image 450 has not been performed (step S615: NO), the control unit 410 determines whether or not the product desired to be purchased has been selected (step S616). Control unit 410 determines whether purchase quantity increase/decrease buttons 462C and 462D in product selection image 462 have received a pressing operation by the user. If it is determined that the purchase desired product has not been selected (step S616: NO), control unit 410 returns the process to step S614.

On the other hand, if it is determined that the purchase desired product has been selected (step S616: YES), the control unit 410 adds the selected purchase desired product to the cart (step S617). Control unit 410 temporarily stores the selected purchase desired product as a purchase planned product.

Subsequently, control unit 410 determines whether or not the product desired to be purchased has been confirmed (step S618). Control unit 410 determines whether or not display operation unit 420 has received an operation to end the selection of the product desired to be purchased. If it is determined that the desired purchase item has not been confirmed (step S618: NO), control unit 410 returns the process to step S614.

If it is determined that the desired purchase product has been confirmed (step S618: YES), the control unit 410 transmits order information including the purchase desired product selected by the user and its quantity to the EC server 300 (step S619). It is determined whether or not order information has been received from the server 300 (step S620). If it is determined that order information has not been received (step S620: NO), control unit 410 repeats the process of step S620 and waits until order information is received.

If it is determined that the order information has been received (step S620: YES), the control unit 410 controls the display operation unit 420 to display an order confirmation image representing the order details (step S621), and starts the product purchase support process. finish.

As described above, according to the product purchase support system 1 of the second embodiment, the information terminal 400 acquires the refrigerator interior information from the refrigerator interior management server 200, and displays the past refrigerator interior image 451 and the latest refrigerator interior image 451. A refrigerator interior confirmation image 450 including an image 452 is generated and displayed as a product purchase support image for supporting product purchase at the virtual store. In addition to the refrigerator interior confirmation image 450, it is possible to switch the display between the product category image 460A and the order image 460B provided as product purchase support images. As a result, it is possible to provide product information that matches the user's product purchase tendency, and to improve the convenience of product purchase.

It should be noted that the program for acquiring and displaying the in-fridge information from the in-fridge management server 200 may be a separate program from the purchase support program. In this case, regardless of whether or not the purchase support program is used, the user can activate the program for displaying the inside information of the goods storage 100 and confirm the goods in stock.

In the above embodiment, the information terminal 400 alternately displays the past refrigerator interior image 451 and the latest refrigerator interior image 452 in the refrigerator interior confirmation image 450. On the condition that they can be secured, the past refrigerator interior image 451 and the latest refrigerator interior image 452 may be displayed side by side vertically or horizontally at the same time. Further, when the inside information includes not only the inside image of the refrigerator compartment 101 but also the inside images of the freezer compartment 102 and the vegetable compartment 103, the inside images of each storage compartment may be displayed. In this case, on the display screen of the display operation unit 420, the images of the interior of each storage room may be displayed simultaneously, or may be switched and displayed as appropriate according to the user's operation such as swiping or flicking.

(Embodiment 3)
Embodiment 3 will explain an example of improving the convenience of the user's selection of the desired purchase product on the EC site.

Like the product purchase support system 1 according to the first embodiment, the product purchase support system 1 according to the third embodiment includes an article storage 100, an internal management server 200, an EC server 300, and an information terminal 400. They are connected so as to be able to communicate with each other via the network NW. Further, the functional configuration of each of the article storage 100, the internal management server 200, the EC server 300, and the information terminal 400 according to the present embodiment is the same as that of the article storage 100, the internal management server 200, It is generally the same as the EC server 300 and the information terminal 400 .

The registered information DB 222 of the storage management server 200 according to the present embodiment corresponds the identification information of the article storage 100 and the user information of the user who uses the purchase support service on the information terminal 400, as in the second embodiment. memorize it. In addition, in response to an authentication request from information terminal 400, authentication unit 211 collates user information included in the authentication request with user information stored in registration information DB 222, and authorizes access to in-fridge management server 200. Authenticate. In addition, in-fridge management server 200 according to the present embodiment transmits in-fridge information of article storage 100 associated with user information to information terminal 400 in response to an in-fridge information transmission request from information terminal 400. do.

As in the information terminal 400 according to the second embodiment, the control unit 410 of the information terminal 400 according to the present embodiment is functionally provided with an in-fridge information acquisition unit 414 and receives an article storage information from the in-fridge management server 200 . It acquires the inside information of the refrigerator 100 and instructs the display image control unit 413 to generate an inside confirmation image based on the acquired inside information.

When the display image control unit 413 receives an instruction to generate a product purchase support image from the sales product information acquisition unit 412 and an instruction to generate a refrigerator interior confirmation image from the refrigerator interior information acquisition unit 414, the display image control unit 413 captures the refrigerator interior confirmation image. Then, the product purchase support image is generated, and the display operation unit 420 is controlled to display the generated product purchase support image.

Product purchase support image 470 according to Embodiment 3 includes refrigerator interior confirmation image 471, as shown in FIG. The inside confirmation image 471 shows a schematic view of the appearance of the article storage 100 . The user can tap the left door 101L and the right door 101R of the refrigerator compartment 101 to open and close the left door 101L and the right door 101R of the refrigerator compartment 101 in the refrigerator interior confirmation image 471, and can confirm the inside of the refrigerator compartment 101 by opening the left door 101L and the right door 101R. can. In addition, when the user drags the product in the product display frame 472 displaying the products included in the category selected in the category selection frame 473 into the refrigerator compartment 101 of the refrigerator interior confirmation image 471, the user selects the product as the desired purchase product. can be selected. The control unit 410 temporarily stores the product dragged from the product display frame 472 into the refrigerator compartment 101 as the product to be purchased, and when the selected product confirmation button 474 is pressed, all the selected purchase desired products are displayed. Let

As described above, according to the product purchase support system 1 of the third embodiment, the information terminal 400 can display the product purchase support image including the refrigerator interior confirmation image. As a result, it is possible to provide product information that matches the user's product purchase tendency, and to improve the convenience of product purchase.

(Embodiment 4)
In the first embodiment described above, an example of improving the chances of repeat purchases on the EC site operated by the EC server 300 has been described. An example of improving the purchase opportunity of a product that does not have is described.

Like the product purchase support system 1 according to the first embodiment, the product purchase support system 1 according to the fourth embodiment includes an article storage 100, an internal management server 200, an EC server 300, and an information terminal 400. They are connected so as to be able to communicate with each other via the network NW. In addition, the functional configurations of the article storage 100, the internal management server 200, and the information terminal 400 according to the present embodiment are similar to those of the article storage 100, the internal management server 200, and the information terminal 400 according to the first embodiment. It is generally the same.

Unlike the EC server 300 according to the first embodiment, the EC server 300 according to the fourth embodiment, as shown in FIG. It further has a purchase history information DB 325 for storing. The purchase history information includes, for example, a user ID, date and time of purchase, identification information of the purchased product, purchase price, and the like.

In the sales product information supply process of the fourth embodiment, when selecting the recommended purchase product in the process of step S405 in the flowchart shown in FIG. In addition to the selection method described above, the latest product identification information and the purchase history information stored in the purchase history information DB 325 of the storage unit 320 are referred to, and the product identified in the latest product identification information and having no purchase history in the purchase history information To select a product as a recommended purchase product.

As described above, according to the commodity purchase support system 1 of Embodiment 4, the EC server 300 selects a commodity that is specified in the commodity specifying information and has no purchase history as a recommended purchase commodity. As a result, it is possible to increase the chances of purchasing products purchased at another store on the EC site operated by the EC server 300 . In addition, the user can easily understand that the product can be purchased at the EC site operated by the EC server 300, and can narrow down the purchase destination, thereby improving the convenience of product purchase.

The sales product information supply unit 313 of the EC server 300 can be used on the EC site operated by the EC server 300, and the coupon information that reduces the price of the recommended purchase product recommended to the user for purchase is used as the recommended purchase product information. may be added. When the EC server 300 receives the order information for the recommended purchase product by the user from the information terminal 400, the EC server 300 settles the purchase recommended product ordered at the discounted price. As a result, it is possible to provide the user with motivation to purchase the product purchased at another store on the EC site operated by the EC server 300 .

In addition, the sales product information supply unit 313 of the EC server 300 can be used at the actual store managed by the EC server 300, and the coupon information that reduces the price of the recommended purchase product recommended to the user to purchase is used as the recommended purchase product information. may be added. The product purchase support image generated by the display image control unit 413 of the information terminal 400 includes coupon information of the recommended purchase product stating that the purchase price will be discounted if presented at the physical store managed by the EC server 300. . As a result, even users who are not accustomed to purchasing products on the EC site can be motivated to purchase at the actual store managed by the EC server 300 .

It should be noted that in the above-described embodiment, each function provided by the storage management server 200, the EC server 300, and the information terminal 400 may be provided as a function of the product purchase support system 1, and may be provided by another device instead. may For example, the EC server 300 has the functions of the article detection unit 214 and the product identification unit 215 provided in the in-fridge management server 200 in the above embodiment, and the EC server 300 acquires in-fridge information from the in-fridge management server 200. Then, on the basis of this in-store information, it is possible to detect inventory items and identify the products for sale on the EC site corresponding to the inventory items. Further, for example, the in-fridge management server 200 and the EC server 300 may be integrated by providing the EC server 300 with all the functions that the in-fridge management server 200 has.

FIG. 18 is a block diagram showing an example of the hardware configuration of information terminal 400 according to the embodiment of the present invention. The information terminal 400 includes a processor 500, a memory 600, and an interface 700 as hardware configuration. Each function of the control unit 410 of the information terminal 400 is realized by executing various programs stored in the memory 600 by the processor 500 . The memory 600 is, for example, a RAM (Random Access Memory), a Flash ROM (Read Only Memory), or a combination of these. The interface 700 connects the information terminal 400 to the network NW, a communication interface for transmitting and receiving information to and from various devices on the network NW, and a user interface for accepting input from the user and displaying a screen to present it to the user. consists of The communication interface is, for example, a wireless LAN and functions as the communication unit 430 . The user interface is, for example, a touch panel, a push button, a liquid crystal panel, an LED (Light Emitting Diode) indicator, or a combination thereof, and functions as the display operation section 420 . The liquid crystal panel and LED indicator functioning as the display operation unit 420 are the display device of the present invention. Interface 700 may include other types of interfaces as desired. Although FIG. 17 shows an example in which one processor 500 and one memory 600 are configured, a plurality of processors 500 and a plurality of memories 600 may work together to perform each function.

The article storage 100, the storage management server 200, and the EC server 300 also have a processor 500, a memory 600, and an interface 700 as hardware configurations.

In the above-described embodiment, the operation programs executed by the control units 110, 210, 310, and 410 provided in the article storage 100, the storage management server 200, the EC server 300, and the information terminal 400 are pre-stored in ROM and other memory. stored in the device. However, the present invention is not limited to this. It may function as a device corresponding to the article storage 100 , the storage management server 200 , the EC server 300 and the information terminal 400 .

The method of providing such a program is arbitrary, and for example, storing and distributing it in a computer-readable recording medium (flexible disc, CD (Compact Disc)-ROM, DVD (Digital Versatile Disc)-ROM), etc. Alternatively, the program may be stored in storage on a network such as the Internet, and provided by downloading the program.

In addition, when the above processing is performed by an OS (Operating System) and an application program, or by cooperation between the OS and an application program, only the application program may be stored in a recording medium or storage. It is also possible to superimpose a program on a carrier wave and distribute it via a network. For example, the program may be posted on a bulletin board system (BBS) on the network and distributed via the network. Then, this program may be designed to execute the above process by starting up and executing it in the same way as other application programs under the control of the OS.

DESCRIPTION OF SYMBOLS 1... Product purchase support system, 100... Goods storage, 101... Refrigerator, 101A... Upper stage, 101B... Middle stage, 101C... Lower stage, 101L... Left door, 101M... Camera, 101P... Shelf board, 101R... Right door, 101S Opening/closing sensor 102 Freezer compartment 102A Door 103 Vegetable compartment 103A Door 110 Control unit 111 Authentication request unit 112 Internal information supply unit 120 Imaging unit 130 Open/close detection Unit 140 Communication unit 200 In-fridge management server 210 Control unit 211 Authentication unit 212 In-fridge information acquisition unit 213 Product list information acquisition unit 214 Article detection unit 215 Product identification Unit 216 Product identification information supply unit 220 Storage unit 221 In-fridge information DB 222 Registration information DB 230 Communication unit 300 EC server 310 Control unit 311 Authentication request unit 312 Authentication unit 313 Product list information supply unit 314 Product identification information acquisition unit 315 Sales product information supply unit 316 Transaction control unit 320 Storage unit 321 Product list information DB 322 Product identification Information DB 323 Customer information DB 324 Registration information DB 325 Purchase history information DB 330 Communication unit 400 Information terminal 410 Control unit 411 Authentication request unit 412 Sales product information acquisition unit 413 Display image control unit 414 In-fridge information acquisition unit 420 Display operation unit 430 Communication unit 450 In-fridge confirmation image 451 Past in-fridge image 452 Latest in-fridge image 453...Display switching button 455A...Upper stage 455B...Middle stage 455C...Lower stage 456A to D...Goods 460...Product purchase support image 460A...Product category image 460B...Order image 461...Product category selection image 461A ~E... Category selection button, 462... Product selection image, 462A... Product image, 462B... Purchase quantity column, 462C, 462D... Purchase quantity increase/decrease button, 462E... Scroll bar, 463, 465, 466... Display switching button, 464... Selected product confirmation button 470 Product purchase support image 471 Interior confirmation image 472 Product display frame 473 Category selection frame 474 Selected product confirmation button 500 Processor 600 Memory 700 Interface BL... bus line, NW... network

Claims (5)

an internal information acquisition unit for acquiring internal information indicating the internal status of an article storage that stores articles;
an article detection unit that detects an article stored in the storage from the storage information;
a storage unit that stores product list information indicating products sold at a real store or a virtual store;
a product identification unit that identifies a product corresponding to the product detected by the product detection unit from products indicated by the product list information and generates product specification information indicating the specified product;
From the product list information stored in the storage unit, products included in the past product specifying information but not included in the latest product specifying information, products repeatedly included in a plurality of the product specifying information, and the product specifying information at least one of the products included in the physical shop or the virtual shop for which the user has no purchase history, as recommended products to be purchased by the user, and reflecting the recommended purchase products in the product list information. a sales product information supply unit that supplies the sales product information obtained by the user to the information terminal used by the user;
Product purchase support system. The inside information includes an inside image of the inside of the storage room of the article storage,
The article detection unit detects an article stored in the refrigerator by recognizing an article area in the image of the interior of the refrigerator.
The product purchase support system according to claim 1. The product list information includes product images showing the appearance of products sold at the physical store or virtual store,
The product identification unit compares the feature values of the product detected by the product detection unit and the product indicated by the product image, and based on the comparison result, identifies the product corresponding to the product detected by the product detection unit. ,
3. The merchandise purchase support system according to claim 1 or 2. further comprising an in-fridge information supply unit that supplies the in-fridge information acquired by the in-fridge information acquisition unit to an information terminal connected via a network;
The merchandise purchase support system according to any one of claims 1 to 3. the computer,
an internal information acquisition unit for acquiring internal information indicating the internal status of an article storage that stores articles;
an article detection unit that detects an article stored in the storage from the storage information;
a product identification unit that identifies a product corresponding to the product detected by the product detection unit from products indicated by product list information indicating products sold at a real store or a virtual store, and generates product specification information indicating the specified product; and,
From the product list information, products included in the past product specifying information but not included in the latest product specifying information, products repeatedly included in a plurality of the product specifying information, and the actual products included in the product specifying information. At least one of a store or a product for which the user has no purchase history at a virtual store is selected as a recommended purchase product to be recommended to the user, and sales product information is obtained by reflecting the recommended purchase product in the product list information. , a sales product information supply unit that supplies the information terminal used by the user;
A program that acts as a

Images