JP6735888B2 - Product data processing system, product data processing method - Google Patents

Description

Embodiments of the present invention relate to a product data processing system and a product data processing method .

In retail stores such as supermarkets, a checkout system is introduced to allow a customer to carry out a part of the accounting process in order to improve the efficiency of the accounting process by POS (Point Of Sales). For example, a self-checkout system having a self-checkout (self-checkout terminal) has been introduced. In the self-checkout, the customer can read the product code (generally a bar code) of a product to register the product, generate the payment information for the payment process, and the payment information generated by the registration process. The payment process is executed to pay the price.

Further, in recent years, a personal checkout system has been considered in which a customer registers a product for which he or she intends to purchase the product while the customer purchases the product in the store. In the case of this system, it is necessary that the product put in the cart or basket as the purchase plan and the product registered are the same.

Japanese Patent Publication No. 2016-532932

The problem to be solved by the present invention is to provide a product data processing system and a product data processing method for surely registering a product to be purchased by a customer operation.

According to the embodiment, the product data processing system includes a first electronic device and a second electronic device. The first electronic device includes a product information acquisition unit, a display unit , and an output unit. The second electronic device has a storage unit and a transmission unit. The product information acquisition unit acquires product information regarding a product that the customer plans to purchase. The display unit displays the product information acquired by the product information acquisition unit on a display unit. Storage means, Ru stored in the storage unit of the product information acquired by the product information acquiring unit. The transmitting means notifies the first electronic device when a predetermined time set in advance has passed since the product information was stored in the storage unit. The output unit outputs, to the display unit, a guide to the customer that prompts the customer to store the product in a storage unit for storing the product that the customer intends to purchase, in response to the notification from the transmission unit .

1 is a system configuration diagram schematically showing the overall configuration of a system of this embodiment. The block diagram which shows the structure of the electronic device in this embodiment. The figure which shows the usage example of the clip terminal in this embodiment. The figure which shows the external appearance of the clip terminal in this embodiment. 6 is a flowchart for explaining the operation of the electronic device according to the present embodiment. 6 is a flowchart for explaining the operation of the clip terminal according to the present embodiment. 6 is a flowchart for explaining the operation of the clip terminal according to the present embodiment. The figure which shows an example of the goods check list in this embodiment. The figure which shows an example of the message in this embodiment. The figure which shows an example of the image image|photographed by the camera in this embodiment. The figure which shows an example of the object corresponded to the goods detected from the image shown in FIG. The flowchart which shows the registration position determination process which determines whether confirmation is needed based on the registration position in this embodiment. The figure which shows an example of arrangement|positioning of the goods shelf in a store, and the position of a cart.

Hereinafter, the present embodiment will be described with reference to the drawings.
FIG. 1 is a system configuration diagram schematically showing the overall configuration of the system of this embodiment. In the system shown in FIG. 1, a system 10 provided in the headquarters and a store system 12 provided in each of a plurality of stores are connected via a network 14 including a dedicated line or the Internet.

The system 10 provided in the headquarters includes, for example, a headquarters system 10A, a customer system 10B, a settlement system 10C, a sales promotion system 10D, and the like, which are interconnected via a network 10E such as a LAN (Local Area Network).

The headquarters system 10A executes processing for controlling the store system and the like installed in each store. The customer system 10B manages customers who purchase products using the store system. The payment system 10C executes processing (personal authentication, credit payment, etc.) for payment processing using a credit card by a customer in the store system. The sales promotion system 10D manages to provide customers (electronic devices used by the customers) with information such as advertisements of products for sales promotion and in-store guidance through the store system.

The store system 12 provided in the store is realized as, for example, a POS (Point Of Sales) system. The store system 12 is provided with, for example, a store server 20A, a clip terminal management server 20B, a plurality of checkout machines 20C, a plurality of POS terminals 20D, etc., and they are mutually connected via a network 20E such as a LAN.

The store server 20A controls the entire store system 12. The store server 20A stores data about products sold in the store, such as product code, product name, unit price, weight, product position (product shelf position) sold in the store, and product image. The data is stored. In the registration process, the product data corresponding to the product code read by scanning the product is received from the store server 20A and stored, and the payment information (including the amount of money, the product, the number, etc.) for the payment process. Can be generated. In the clip terminal management server 20B, the product (purchased product) that has been registered by using the electronic device 30 as a purchase plan by the customer operation and the product put in the cart 60 or the basket as the purchase plan match. It manages an electronic device 50 (a clip terminal described later) used for checking whether there is any. The accounting machine 20C executes the settlement process for the product for which the registration process is performed by using the electronic device 30 by the operation of the customer. The POS terminal 20D executes a registration process for the customer's purchased product by the operation of the store clerk, a payment process for the product registered by the registration process, and the like.

A plurality of access points (base stations) 20F are connected to the network 20E wired in the store. The access point 20F wirelessly communicates with various electronic devices used in the store by, for example, a wireless LAN. The electronic device used in the store includes, for example, an electronic device 30 used by a customer, an electronic device 40 used by a clerk (attendant), etc., for example, an electronic device 50 used while mounted on a cart 60 (a clip terminal described later). 50) is included.

As the electronic devices 30 and 40, for example, a smartphone, a tablet personal computer (PC), or the like is used. The electronic device 30 performs a registration process for a product to be purchased and a product (purchased product) for which the registration process is performed based on an application program (including a check program) for using the service provided by the store system 12 and purchase. The individual check process for checking whether or not the product put in the cart 60 or the basket as a schedule matches is executed in cooperation with the electronic device 50 (clip terminal). Based on the confirmation program for confirming the registration process executed by the electronic device 30, the electronic device 40 allows the clerk (attendant) to match the product information registered by the registration process with the product placed in the cart 60. Confirmation processing is performed to facilitate confirmation.

The electronic device 30 and the electronic device 50 (clip terminal 50) may communicate with each other via the clip terminal management server 20B (network 20E), or may directly communicate with each other by short-range wireless communication. Furthermore, communication cables may be connected to each other for wired communication.

FIG. 2 is a block diagram showing the configurations of the electronic devices 30 and 50 according to this embodiment. The electronic device 30 illustrated in FIG. 2 is described as being realized by, for example, a smartphone. Further, the electronic device 50 is assumed to be configured as a dedicated clip terminal 50 for executing the individual check process in the store system 12. Hereinafter, the clip terminal 50 will be described. The clip terminal 50 can also be realized by installing a check program in a general-purpose device (computer) in which a camera such as a smartphone is installed.

The electronic device 30 includes a processor 30A, a memory 30B, a storage device 30C, a touch panel 30D, a camera 30E, a communication module 30F, and a battery 30G.
The processor 30A is a circuit that controls each unit in order to implement various processes as the electronic device 30 by executing a program recorded in the memory 30B.

The memory 30B records not only the program executed by the processor 30A but also data used by the processor 30A to execute various processes. The programs recorded in the memory 30B include, for example, a registration processing program for executing registration processing for products to be purchased, products for which registration processing has been performed (purchased products), and products to be put in the cart 60 or basket as purchase plans. A check program is included to execute an individual check process for checking whether and match. The data stored in the memory 30B stores payment information generated based on the product code read in the registration process, product information regarding the product registered in the registration process, and the like.

The storage device 30C is implemented by a hard disk drive (HDD) device, a solid state drive (SSD) device, or the like, and is used by the processor 30A in performing various kinds of processing or generated by processing in the processor 30A. Record the data collected.

The touch panel 30D is a device that is input by a customer, and includes a display device and a touch sensor. On the touch panel 30D, an arbitrary screen such as a GUI (Graphical User Interface) screen associated with the processing of the processor 30A is displayed.
The camera 30E captures an image under the control of the processor 30A.

The communication module 30F controls communication with an external electronic device under the control of the processor 30A. The communication module 30F can control communication such as wireless communication with the access point 20F and short-range wireless communication with other electronic devices such as the clip terminal 50.

The clip terminal 50 includes a processor 50A, a memory 50B, a storage device 50C, a touch panel 50D, a camera 50E, a communication module 50F, a battery 50G, an I/O interface 50H, and a reader/writer 50J.

The processor 50A is a circuit that controls each unit in order to realize various processing as the clip terminal 50 by executing a program recorded in the memory 50B.

The memory 50B records not only the program executed by the processor 50A but also data used by the processor 50A to execute various processes. The program recorded in the memory 50B is a check program that realizes an individual check process for checking whether or not the registered product (purchased product) matches the product put in the cart 60 or basket as a purchase plan. Is included. The check program includes an image that realizes image processing that detects an object corresponding to a product from an image captured by the camera 50E and detects that the product has been put into the cart 60 and that the product has been taken out of the cart 60. A processing program is included. The data stored in the memory 50B includes product information registered by the registration process of the electronic device 30, object management data (see FIG. 11) for managing objects detected by image processing, and products stored in the cart 60. Various data such as a product check list (see FIG. 8) for checking is stored.

The storage device 50C is realized by a hard disk drive (HDD) or a solid state drive (SSD) device, and is used by the processor 50A to perform various kinds of processing, or generated by the processing of the processor 50A. Record the data collected.

The touch panel 50D is a device that is input by a customer, and includes a display device and a touch sensor. On the touch panel 50D, an arbitrary screen such as a GUI (Graphical User Interface) screen associated with the processing of the processor 50A is displayed.
The camera 50E captures an image under the control of the processor 50A.

The communication module 50F controls communication with an external electronic device under the control of the processor 50A. The communication module 50F can control communication such as wireless communication with the access point 20F and short-range wireless communication with other electronic devices such as the electronic device 30.

The I/O interface 50H is an interface for inputting a signal from an external device, and is connected to, for example, a weight sensor 601 (a weighing instrument, a pressure sensitive instrument, a weighing scale, etc.) attached to the basket bottom portion 60C of the cart 60. The weight sensor 601 detects the weight of a product placed (placed) on the cart 60 and outputs a detection signal.

The reader/writer 50J communicates with each product sold in the store, an RF (Radio Frequency) tag attached to a product shelf in the store, and the like. The communication range of the reader/writer 50J is, for example, a range in which communication with the RF tag of the product placed in the cart 60 or a range in which communication with the RF tag attached to the product shelf by moving the cart 60 in the store is possible. To do. The reader/writer 50J is used to detect that a product is put in or taken out from the cart 60, or is used to detect the position of the cart 60 in which the clip terminal 50 is attached in the store. You can

FIG. 3 is a diagram showing a usage example of the clip terminal 50 in the present embodiment, and FIG. 4 is a diagram showing an outer appearance of the clip terminal 50 in the present embodiment.

The clip terminal 50 is used, for example, by being attached to the upper frame 60B of the cart 60 used in the store. As shown in FIG. 4, the clip terminal 50 is provided with a clip 60k, which can be arbitrarily attached and detached. Therefore, it is not necessary to preliminarily attach the clip terminals 50 to all the carts 60 prepared in the store, and only the carts 60 used by the customer who uses the individual check processing service using the clip terminals 50, The clip terminal 50 may be attached as necessary. Therefore, in order to realize the individual check processing service, large-scale investment such as remodeling of the store or exchange of the cart 60 is unnecessary, and the service can be realized at low cost.

In the example shown in FIG. 3, the cart 60 is mounted on the upper frame 60B on the handle 60A side. When mounting the clip terminal 50, the camera 50E provided on the clip terminal 50 is directed toward the basket bottom 60C, and the photographing range is adjusted so that at least all the products put in the cart 60 can be photographed. To be done. The photographing range of the camera 50E preferably includes the upper frame 60B so that the area inside and outside the basket can be clearly divided in the image photographed by the camera 50E. Further, it is desirable that the photographing range of the camera 50E be capable of photographing, for example, the movement of the customer's arm (hand) so that the movement of the customer to take in and out the product can be detected.

As shown in FIG. 4, the clip terminal 50 is attached to the upper frame 60B so that the display surface of the touch panel 50D faces the customer side. As a result, it is easy to confirm the image captured by the camera 50E and the information (message or the like) indicating the check result of the individual checking process using the captured image.

Next, operations of the electronic device 30 and the clip terminal 50 (electronic device) used in the store system 12 of the present embodiment will be described. FIG. 5 is a flowchart for explaining the operation of the electronic device 30 according to this embodiment. 6 and 7 are flowcharts for explaining the operation of the clip terminal 50 in this embodiment.

When the customer performs the registration process using the electronic device 30, the customer activates an application program (check program) for using the service provided by the store system 12. Hereinafter, the processor 30A of the electronic device 30 executes the registration process and the individual check process described below based on the application.

First, the electronic device 30 executes a login process for connecting to the store system 12 (Act 10). In the login process, a bar code including a member code unique to the member attached to the member card is read by the camera 30E by the operation of the customer (image capturing). The processor 30A extracts the member code from the barcode read by the camera 30E, and sends it to the clip terminal management server 20B. Based on the membership code received from the electronic device 30, the clip terminal management server 20B collates with the membership data managed in the customer system 10B to authenticate the membership.

When the electronic device 30 is notified by the clip terminal management server 20B that the member has been authenticated, the electronic device 30 displays a selection screen of services available in the store. Services available in the store include display of leaflet information, display of information on special sale items, display of store information (map), use of clip terminal 50, and the like, in addition to registration processing for purchase schedule processing. By using the clip terminal 50, a message is output from the clip terminal 50 when, for example, a registration process has not been performed for a product placed in the cart 60. Therefore, the customer can prevent forgetting to register the product to be purchased. Also, while the customer is shopping around in the store, the clip terminal 50 checks the products to be purchased, so the burden of confirmation work by the attendant (clerk) after the registration processing by the customer is completed. Can be reduced.

When the customer uses the clip terminal 50, the store system 12 provides the customer with profits (points and the like) (stores data indicating the profits to the customer in the customer system 10B), thereby providing the customer with the profit. Alternatively, the use of the clip terminal 50 may be prompted.

When using the clip terminal 50, the electronic device 30 and the clip terminal 50 are placed in a communicable state. For example, the clip terminal 50 is attached to the cart 60 used by the customer (or the customer uses the cart 60 to which the clip terminal 50 is attached in advance), and the link between the electronic device 30 of the customer and the clip terminal 50 is set. For example, the electronic device 30 and the clip terminal 50 are paired so that they can communicate with each other by near field communication. Alternatively, the unique code of the clip terminal 50 is read by the electronic device 30 and transmitted from the electronic device 30 to the clip terminal management server 20B. The clip terminal management server 20B discriminates between the electronic device 30 and the clip terminal 50 which execute the individual checking process in cooperation with each other, and establishes the clip terminal management server 20B (network 20E) between the electronic device 30 and the clip terminal 50. Allow communication through. It should be noted that as a method of connecting the electronic device 30 and the clip terminal 50, another method can be used.

When the processor 30A of the electronic device 30 is connected to the clip terminal 50 (Act11), the processor 30A makes the product registration process executable. Further, the processor 50A of the clip terminal 50 is connected to the electronic device 30 (Act 40) and starts the product detection process. The product detection process is a process of detecting an object corresponding to a product from an image captured by the camera 50E. The processor 50A detects an object corresponding to a product or the like from an image captured by the camera 50E by existing image processing, and stores the position of each object. For example, when an object that moves inside the cart (inside the cart) from an area corresponding to the outside of the cart (outside the cart) is detected, the processor 50A determines that the customer has performed an operation for taking in and out the product. Based on the change of the object existing in the cart before and after the detection of this operation, it is detected that the product is put in or taken out. Further, even when the movement of the customer is detected, if the number of objects corresponding to the products in the cart has not changed, it is determined that the customer has replaced the products in the cart, for example. The processor 50A stores the position of the replaced object. The processor 50A stores the object detected from the image captured by the camera 50E in the product checklist.

FIG. 8 is a diagram showing an example of the product check list in this embodiment. In the product checklist, as shown in FIG. 8, the data of the object (detected object) detected from the image captured by the camera 50E and the information (registered product) of the product registered by the registration process in the electronic device 30 are displayed. Data (for example, D1 to D4) in which data and data indicating the result of checking the match between the registered product and the product corresponding to the detected object (check results “OK” and “NG”) are stored are stored.

The customer performs product registration by scanning the barcode attached to the product using the camera 30E of the electronic device 30 for the product to be purchased while shopping around the store (Act 12). The processor 30A detects a barcode from the image captured by the camera 30E and extracts a product code from the barcode. The processor 30A acquires the information of the product corresponding to the product code from the store server 20A through the clip terminal management server 20B (network 20E) and displays it on the touch panel 30D. That is, the product name, price, product image, and the like of the product read by the product scan are displayed to be confirmed by the customer. Here, when the customer performs an instruction operation for merchandise registration (for example, a touch of a merchandise registration button displayed on the touch panel 30D) (Act 13, Yes), the processor 30A purchases the merchandise read by the merchandise scan. It is stored as a registered product (Act 14). Further, the processor 30A transmits information on the registered product to the clip terminal 50 (Act15).

On the other hand, when the clip terminal 50 receives the information of the registered product from the electronic device 30 (Act 42, Yes), the clip terminal 50 stores the information of the registered product in the product check list (Act 43). The processor 50A refers to the information of the registered product and detects that it is a large product, that is, when it is determined that the product does not need to be (cannot be) put in the cart 60, the product checklist. The skip flag is set to (Act 45). In the example shown in FIG. 8, a skip flag is set as the data D2 in association with the registered product “product C”. Regarding the registered product “product C” for which the skip flag is set, the processor 50A excludes the registered product and the product placed in the cart 60 from the check target. In addition, the processor 50A transmits a product insertion unnecessary notification to the electronic device 30 (Act 57).

When the notification that the product insertion is not required is received from the clip terminal 50 (Act 18, Yes), the processor 30A of the electronic device 30 displays the registered product information (product name, product image, price, etc.) on the touch panel 30D together with the customer. A message for notifying that it is not necessary to put the product in the cart 60 is displayed. Note that the message notifying that the product is not required may be displayed not only on the electronic device 30 but also on the touch panel 50D of the clip terminal 50.

On the other hand, when the registered product is not a large product, the processor 50A acquires product dictionary data for identifying the registered product by image processing from the clip terminal management server 20B via the network 20E, for example. For example, the processor 50A outputs a transmission request for the product dictionary data corresponding to the registered product to the clip terminal management server 20B when the product dictionary data corresponding to the registered product is not stored. The clip terminal management server 20B transmits the product dictionary data of the corresponding product to the clip terminal 50 in response to the transmission request from the clip terminal 50. When the electronic device 30 and the clip terminal 50 communicate with each other via the clip terminal management server 20B, the clip terminal management server 20B receives the registered product information when receiving the registered product information from the electronic device 30. At the same time, the product dictionary data of the corresponding product may be transmitted to the clip terminal 50. The product dictionary data may be stored in the clip terminal 50 in advance.

Further, when the registered product is stored in the product checklist, the processor 50A starts measuring the elapsed time after storing the registered product. The processor 50A determines whether or not the elapsed time has passed a preset time until it is confirmed that the product corresponding to the registered product has been placed in the cart 60 (Act 46).

In many cases, a customer handles a product to be purchased by a series of operations such as picking up a product from a product shelf or the like, scanning the product using the electronic device 30, and then placing the product on the cart 60. Therefore, when a preset time has passed since the product scan (the storage of the registered product in the clip terminal 50), there is a possibility that the registered product may be left in the cart 60, so that the customer can be notified. ing. When it is determined that the elapsed time of the registered product has passed the preset time (Act 46, Yes), the processor 50A transmits a product insertion request to the electronic device 30 (Act 47).

When the product insertion request is received from the clip terminal 50 (Act 18, Yes), the processor 30A of the electronic device 30 notifies the customer of the registered product information (product name, product image, price, etc.) on the touch panel 30D. On the other hand, a product insertion request message prompting the user to insert the product into the cart 60 is displayed (Act 19). For example, as shown in FIG. 9(A), a message "Please put the registered product in the cart." is displayed. Note that the product putting request message may be displayed not only on the electronic device 30 but also on the touch panel 50D of the clip terminal 50. In addition, when there are a plurality of target products of the product placement request message, information of the plurality of target products is displayed in a list so that the customer can surely recognize the target product. good.

On the other hand, when the processor 50A of the clip terminal 50 detects that the customer has performed an operation for putting in or taking out a product by the image processing on the image taken by the camera 50E, the processor 50A is placed in the cart before and after the operation is detected. Based on changes in existing objects, it is detected that a product has been put in or taken out. It should be noted that the processor 50A not only detects the movement of the goods in and out based on the image, but also determines the movement of the goods in combination with the change in the weight detected by the weight sensor 601 at the timing when the movement is detected. You can also do it.

FIG. 10 shows an example of an image taken by the camera 50E. In the example shown in FIG. 10, for example, three products A, B, and E are put in the cart 60.

The processor 50A detects the position of the upper frame 60B of the cart 60 and the area of the basket bottom 60C from the image captured by the camera 50E. Since the clip terminal 50 is fixed to the cart 60, basically, the areas of the upper frame 60B and the basket bottom 60C in the image do not change during the purchase. When the processor 50A detects an object that moves inside the cart (inside the cart) from the area corresponding to the outside of the cart (outside the cart) with reference to the position of the upper frame 60B, the customer puts the product in and out. Is determined to have been performed.

Further, the processor 50A detects objects corresponding to products or the like from the image captured by the camera 50E by existing image processing, and stores the position of each object as object management data.

FIG. 11 shows an example of the objects OJ-A, OJ-B, and OJ-E corresponding to the products detected from the image shown in FIG. The processor 50A detects that the product has been put in or taken out based on the change in the objects existing in the cart before and after the product has been put in and taken out by the customer.

When the object corresponding to the product is detected from the image (Act 48, Yes), the processor 50A refers to the product check list to determine whether the registered product with the check result “NG” is stored. That is, it is determined whether or not there is a product which has been registered by the electronic device 30 but whose corresponding product has not been confirmed to be put in the cart 60. Here, when the registered product with the check result “NG” is stored (Act 49, Yes), the processor 50A uses the product dictionary data corresponding to the registered product with the check result “NG” to detect the image from the image. Recognition processing is executed for the object (Act 52). The product dictionary data is created so that the product can be recognized based on the characteristics such as the barcode attached to the product, the design of the package, and the characters.

Here, it is determined whether the object detected from the image is a registered product corresponding to the product dictionary data, based on the processing result of the recognition process using the product dictionary data. For example, when the feature of the image represented by the object and the feature represented by the product dictionary data indicate similarity exceeding a preset reference value, it is determined that the product is a registered product corresponding to the product dictionary data.

If it is determined that the product is a registered product (Act 53, Yes), the processor 50A associates the registered product stored in the product check list with the product dictionary data used in the recognition process. , The object detected from the image is stored as a detected object (Act 54). Data indicating the similarity (similarity) in the recognition process may be stored as the data related to the detected object. As a result, it is determined that the product is a registered product corresponding to the product dictionary data, but since the similarity is low, it is possible to prompt the attendant to confirm the product before the settlement process.

Further, the processor 50A sets the check result “OK” corresponding to the set of the registered product and the detection object (Act 55), and stores that the registered product has been confirmed to be put in the cart 60. In this case, the processor 50A transmits a product placement confirmation to the electronic device 30 (Act 56).

When the electronic device 30 receives the product insertion confirmation from the clip terminal 50 (Act 20, Yes), if the product insertion request message is displayed, the electronic device 30 deletes this message from the touch panel 30D (Act 28). As a result, it is possible to notify the customer that the clip terminal 50 has confirmed that the registered merchandise has been placed in the cart 60.

When there are a plurality of registered products with the check result “NG” in the product check list, the processor 50A executes the recognition process on the object using the product dictionary data of each registered product. Further, for an object that is not determined to be a registered product by the recognition process (Act 53, No), the processor 50A stores the product as an unregistered detected object in the product check list as a check result “NG” (Act 50). ). Further, when the object is detected from the image (Act48, Yes) and the registered product of the check result “NG” is not stored (Act49, No), the processor 50A similarly does not register the product. As the detection object, the check result “NG” is stored in the product check list (Act 50).

The data D3 shown in FIG. 8 indicates that the product C has already been registered in the electronic device 30, but the check result “NG” is set, and it has not been confirmed that the product has been placed in the cart 60. There is. Further, the data D4 indicates that the unregistered products E 1 and F 2 have been detected as the objects E and F, respectively, because they have been put into the cart 60 first (check result “NG”).

When the detection object (check result “NG”) in which the product is not registered is stored in the product check list (Act 50), the processor 50A transmits a product registration request to the electronic device 30 (Act 51). When the product registration request is received from the clip terminal 50 (Act 21, Yes), the processor 30A of the electronic device 30 causes the touch panel 30D to display a product registration request message prompting the customer to register the product. For example, as shown in FIG. 9B, a product registration request message "There is a product that has not been registered in the cart. Please register the product." is displayed. The product registration request message may be displayed not only on the electronic device 30 but also on the touch panel 50D of the clip terminal 50 (Act 22).

As described above, the product registration request message displayed on the touch panel 50D is registered in the cart 60 based on the recognition process using the product dictionary data of the registered product after the product is registered. If it is confirmed (Act 49 to 51), it is erased (Act 20, 28).

On the other hand, when returning the merchandise already registered as a purchase plan, the customer scans the bar code attached to the merchandise for the merchandise to be returned using the camera 30E of the electronic device 30 in the same manner as when the merchandise is registered. (Act12). The processor 30A detects a barcode from the image captured by the camera 30E and extracts a product code from the barcode. The processor 30A causes the touch panel 30D to display product information (product name, price, product image, etc.) corresponding to the product code, and prompts the customer to confirm the product to be returned. Here, when the customer performs an instruction operation for product cancellation (for example, touching a product cancellation button displayed on the touch panel 30D) (Act 13, No), the processor 30A selects from the purchase plan of the product read by the product scan. Delete (Act16). In addition, the processor 30A transmits a registration cancellation notification to the clip terminal 50 together with information on the product targeted for cancellation (return) (Act 17).

Upon receiving the registration cancellation notification from the electronic device 30 (Act 60, Yes), the clip terminal 50 determines the corresponding registered product from the product checklist based on the information of the product to be canceled (Act 61), and registers the registered product. Information is deleted (Act 62). The processor 50A transmits a cancellation completion notification to the electronic device 30 (Act 63). When the electronic device 30 receives the cancellation completion notification (Act 25, Yes), if the product putting request message is displayed, the electronic device 30 deletes the product putting request message (Act 28). For example, as described above, the electronic device 30 displays the product insertion request message when the set time elapses without the product being put in the cart 60 after the product registration process. When the customer performs a product cancellation operation on the registered product, the electronic device 30 erases the product insertion request message for the corresponding product.

When the customer puts the product in the cart 60 and the customer takes it out, the processor 50A of the clip terminal 50 indicates that the customer has performed an operation for taking the product in and out based on the image taken by the camera 50E. It is determined and the object corresponding to the product taken out by the existing image processing is detected. Note that, similarly to the case of detecting an object when a product is put in, it is possible to determine that the product has been put out, in combination with a change in weight detected by the weight sensor 601.

When the processor 50A detects that the object (product) is taken out of the cart 60 (Act64, Yes), the registered product corresponding to the detected object is stored in the product checklist (check result). "OK"). Here, when the corresponding registered product is not stored (Act65, No), the processor 50A deletes the detection object corresponding to the taken-out product from the product check list (Act68).

On the other hand, when the corresponding registered product is stored (Act65, Yes), the processor 50A transmits a product cancellation request to the electronic device 30 (Act66). Further, the processor 50A sets the check result “NG” for the detected object corresponding to the taken-out product (Act 67).

When the product cancellation request is received from the clip terminal 50 (Act 23, Yes), the processor 30A of the electronic device 30 causes the touch panel 30D to display a product cancellation request message prompting the customer to cancel the product (Act 24). .. For example, as shown in FIG. 9C, a product cancellation request message of "A product has been taken out of the cart. If you want to return the product, please do a return operation" is displayed. The product cancellation request message may be displayed not only on the electronic device 30 but also on the touch panel 50D of the clip terminal 50.

It should be noted that when a product is taken out of the cart 60, the data stored in the product checklist is immediately updated, but the product is canceled after a predetermined set time (for example, 10 seconds) has passed ( The data may be updated assuming that there is a high possibility that the item will be returned. In this case, the product cancellation request message may be displayed on the electronic device 30 immediately after the product is taken out, or may be displayed after the set time has elapsed.

In this way, the customer mounts the clip terminal 50 on the cart 60 and uses the electronic device 30 to carry out the purchase process while registering, thereby forgetting to register the product to be purchased and forgetting to put the product in the cart 60. Even if there is, it can be easily confirmed by the message displayed on the electronic device 30.

When the registration process for the product to be purchased is completed, the customer gives an instruction to end the product registration by operating the subtotal button, for example. When the customer instructs the end of the product registration (Act 26, Yes), the processor 30A of the electronic device 30 displays a confirmation screen (product list) of the registered products and a payment method (credit, cash, direct debit). , Electronic money, etc.). The processor 30A generates payment information based on the information on the product registered by the registration process.

Further, the processor 30A of the electronic device 30 transmits a notification of the registration processing end to the clip terminal 50 (Act27). When the notification of the completion of the registration process is received from the electronic device 30 (Act 69, Yes), the processor 50A of the clip terminal 50 transmits the product checklist data to the electronic device 30 (Act 70). This data also includes, for example, data indicating the degree of similarity when the detected object is recognized. After transmitting the product checklist data, the processor 50A clears the product checklist data and ends the process (Act 71).

In addition, when the termination of the registration process is instructed, if the data of the check result “NG” remains in the data stored in the product checklist, a message prompting the customer to confirm again is electronically displayed. It is displayed on the touch panel 30D of the device 30 (touch panel 50D of the clip terminal 50).

When the customer gives a final confirmation instruction, the processor 30A of the electronic device 30 touches a code (bar code or two-dimensional code) including payment information and data prompting the attendant included in the product checklist to confirm. Display on 30D.

The attendant uses the camera of the electronic device 40 to scan the code displayed on the touch panel 30D of the electronic device 30. The electronic device 40 displays a confirmation screen according to the payment information included in the code and the data that needs to be confirmed. For example, the electronic device 40 displays a list of products (including product name, price, subtotal amount, etc.) scheduled to be purchased by the customer, and the number of products. Thereby, the attendant can confirm whether the product put in the cart 60 and the registered product match. Further, the electronic device 40 explicitly displays the product corresponding to the detected object having a low degree of similarity in the recognition processing of the clip terminal 50 based on the data that needs to be confirmed. This makes it possible for the attendant to easily recognize a product that needs to be particularly confirmed, thereby reducing the burden of confirmation work.

In the above description, an example of checking the match between the registered product and the product placed in the cart 60 by using the image processing in the clip terminal 50 has been described, but the weight of the product detected by the weight sensor 601 is described. You may also check the product by using the. For example, since the data regarding the product provided from the store server 20A includes the data of the product weight, the weight increased when the product is placed in the cart 60 matches the weight of the registered product within the allowable range. You can check based on crab. In particular, it is effective to use the weight data detected by the weight sensor 601 to make a correct check for a weighing product whose price varies depending on the weight. For example, it is possible to confirm fraud, etc., in which a product which is heavier than a registered product of the same type is put in the cart 60 after the registration process is performed on the product having a small weight. In the product checklist, if there is a weighing product, and if the difference between the weight of the registered product and the weight when put in the cart 60 is larger than the set value, data indicating that confirmation is necessary is stored. I'll do it. As a result, as described above, the attendant can be made aware that the confirmation is necessary.

Further, in the above description, the matching of the product registered by the registration process and the product put in the cart 60 is checked, but the clip terminal 50 may be used to check the registration of other products. It is possible. For example, the position in the store at the timing of product registration in the electronic device 30 or the timing at which the product (detection object) is detected by the clip terminal 50, and the position of the product shelf (product position) in the store where the target product is placed. It is also possible to determine that confirmation is necessary when the difference between and is larger than a preset determination value.

FIG. 12 is a flowchart showing a registration position determination process for determining whether confirmation is necessary based on the registration position in this embodiment. The processor 50A of the clip terminal 50 executes the registration position determination process based on the check program. The registration position determination process is executed in parallel with the individual check process described above.

The processor 50A receives the product information of the registered product from the electronic device 30 (Act 80, Yes), or when it is determined which registered product the detected object corresponds to (check result “OK”) (Act 81). , Yes), the current position is detected (Act 82). The current position can be detected, for example, by communicating with the RF tag attached to each of the product shelves in the store by the reader/writer 50J.

FIG. 13 is a diagram showing an example of the arrangement of the product shelves 70-1 to 70-5 in the store and the position of the cart 60. The clip terminal 50 attached to the cart 60 can communicate with the RF tag 72-5 attached to the product shelf 70-5 by communication within a limited range by the reader/writer 50J, when the product shelf 70 is available. It can be determined that the user is near -5. The detection of the current position of the cart 60 (clip terminal 50) is not limited to the method using the reader/writer 50J (RF tag), but also the difference in the radio field intensity between the communication module 50F and the plurality of access points 20F, and the inside of the store. It is possible to use a method such as image processing for detecting the cart 60 with respect to the image for capturing.

When the current position is detected, the processor 50A acquires data indicating the product position (position of the product shelf) where the registered product is sold next, for example, through the store server 20A or the electronic device 30 (Act 83). The processor 50A determines whether the difference between the current position and the product position (the position of the product shelf) is larger than a preset determination value (Act84). For example, when the product 74 placed on the product shelf 70-1 shown in FIG. 13 is registered or placed in the cart 60 at a position close to the product shelf 70-5, the customer performs an operation different from the normal operation. It is determined that confirmation is necessary (Act, Yes). The clip terminal 50 stores the confirmation data in association with the corresponding registered product in the product check list and the data corresponding to the detected object. As a result, as described above, the attendant can be made aware that the confirmation is necessary.

According to at least one embodiment described above, by using the clip terminal 50, the product information registered by the registration processing by the electronic device 30 and the product to be put in and out of the cart 60 detected by the clip terminal 50 can be obtained. A match with the product corresponding to the corresponding detection object is checked. A message prompting the customer to confirm can be appropriately displayed according to the check result, so that it is possible to surely register the product to be purchased by the operation of the customer.

Although some embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the spirit of the invention. These embodiments and their modifications are included in the scope and gist of the invention, and are also included in the invention described in the claims and the scope of equivalents thereof.

Further, the processing described in the above-described embodiments is a recording medium such as a magnetic disk (flexible disk, hard disk, etc.), optical disk (CD-ROM, DVD, etc.), semiconductor memory, etc., as a program that can be executed by a computer. Can be provided to various devices. It is also possible to transmit the data via a communication medium and provide it to various devices. The computer reads the program recorded in the recording medium or receives the program via the communication medium, and the operation is controlled by the program to execute the above-described processing.

12... Store system, 20A... Store server, 20B... Clip terminal management server, 30, 40... Electronic device, 50... Clip terminal (electronic device), 50A... Processor, 50B... Memory, 50C... Storage device, 50D... Touch panel, 50E... Camera, 50F... Communication module, 50J... Reader/writer, 60... Cart, 601... Weight sensor.

Claims (6)

A product data processing system having a first electronic device and a second electronic device,
The first electronic device is
A product information acquisition means for acquiring product information about a product that the customer plans to purchase;
Anda display means for displaying on the display unit of the product information acquired by the product information acquiring means,
The second electronic device is
Storage means causes the storage unit to receive the product information acquired by the product information acquiring means,
A storage unit for notifying the first electronic device when a predetermined time set in advance has elapsed after storing the product information in the storage unit ,
The first electronic device further includes
A product data processing system having an output unit for outputting, to the display unit, a guide to the customer urging the customer to store the product in a storage unit for storing the product to be purchased by the customer in response to the notification from the transmission unit. .. The second electronic device is
An image information acquisition unit that acquires image information from an imaging unit that images the storage unit,
An object detection unit that detects an object corresponding to a product from the image information,
Further comprising a determining means for determining a match between the product corresponding to the product and the object corresponding to the product information,
The transmission unit, wherein if more item into determining means is determined not to match, product data processing system of claim 1, wherein the transmitting the notification. When the product information indicates a specific product that does not need to be stored in the storage unit , the transmission unit transmits a product insertion unnecessary notification to the first electronic device,
3. The product data processing system according to claim 1 , wherein the output unit outputs a guidance notifying that the storage of the product is unnecessary in the storage unit in response to the notification that the product is not required to be stored. A product data processing method in a product data processing system having a first electronic device and a second electronic device,
The first electronic device is
Obtain product information about the product that the customer plans to purchase,
Display the product information on the display unit,
The second electronic device is
Receiving the product information acquired by the first electronic device and storing it in a storage unit,
When a predetermined time set in advance has passed since the product information was stored in the storage unit, the first electronic device is notified,
The first electronic device is
According to the notification, a product data processing method for outputting, to the display unit, a guide for the customer to store the product in a storage unit for storing the product that the customer intends to purchase. The second electronic device is
Image information is acquired from an image capturing unit that captures the storage unit, and an object corresponding to a product is detected from the image information,
Determine a match with products corresponding to the product and the object corresponding to the product information,
The product data processing method according to claim 4 , wherein the notification is transmitted when it is determined that the products do not match by this determination. The second electronic device is
When the product information indicates a specific product that does not need to be stored in the storage unit, a product insertion unnecessary notification is transmitted to the first electronic device,
The first electronic device is
The product data processing method according to claim 4 or 5 , wherein , in response to the notification that the product is not required, a guide for notifying that the product is not stored in the storage unit is output.