JP7524388B2 - Information processing device, store system, program - Google Patents

Description

The present invention relates to an information processing device , a store system, and a program for use in a store such as a retail store.

As labor shortages worsen, retail companies are finding it difficult to secure store staff. As a result, the type of POS (Point Of Sales) systems used in stores is changing in order to reduce the number of staff required to operate POS terminals and serve customers. Traditionally, POS systems were often used in which store clerks performed everything from scanning products to registering information about products for sale (product registration) to handing over money. In recent years, there has been an increase in self-service POS systems in which customers themselves perform everything from scanning products to processing payments.

However, with self-service POS systems, it was necessary to have store staff known as attendants to assist customers who needed to perform complicated operations and to check that customers were performing the correct operations to purchase products. Therefore, the introduction of self-service POS systems did not result in a substantial reduction in the number of store staff.

JP 2015-109061 A

The embodiments of the present invention have been made in consideration of the above-mentioned circumstances, and their purpose is to provide an information processing device , a store system, and a program that enable customers to easily check whether their operations are being performed correctly without complicating the operations required for product purchases by customers.

In order to solve the above problems, the information processing device in this embodiment has a video receiving means, a receiving means, an identifying means, a counting means , and a first storage means. The video receiving means receives video captured by a camera. The receiving means receives identification data unique to a customer from an electronic device. After receiving the identification data, the identifying means identifies the customer corresponding to the identification data in the video. The counting means counts the number of times the identified customer performs a predetermined action to purchase a product. The first storage means stores the number of times in association with the identification data .

FIG. 1 is an overall configuration diagram of a store system according to an embodiment of the present invention. FIG. 2 is a block diagram showing a main circuit configuration of a store server. FIG. 2 is a block diagram showing the main circuit configuration of the accounting machine. FIG. 2 is a block diagram showing the main circuit configuration of a check-in terminal. FIG. 1 is a block diagram showing a configuration of a main part of an electronic device. FIG. 2 is a schematic diagram showing the main data structure of a member data record registered in a member database of a store server. FIG. 4 is a schematic diagram showing the data structure of transaction detail data. FIG. 4 is a diagram showing a data structure of video analysis data. FIG. 2 is a diagram showing the data structure of video data. FIG. 4 is a diagram showing the data structure of result data. 5 is a flowchart for explaining the operation of a store server in the present embodiment. 4 is a flowchart showing the operation of the electronic device in the embodiment. 5 is a flowchart showing a first check-in process in the present embodiment. 10 is a flowchart showing a second check-in process in the embodiment. FIG. 4 is a diagram showing an example of a product list screen according to the embodiment. FIG. 4 is a diagram showing an example of a product code reading screen in the embodiment. 5 is a flowchart for explaining the operation of a store server in the present embodiment. 5 is a flowchart showing the operation of a check-in terminal in the present embodiment. 4 is a flowchart showing the operation of the accounting machine in the present embodiment.

The following describes an embodiment of the present invention with reference to the drawings.

Figure 1 is an overall configuration diagram of a store system 1 according to this embodiment. Store system 1 is a self-service system that allows customers to perform operations ranging from scanning products to register information about products for sale to processing payments. Product scanning involves reading the product codes attached to the products.

The store system 1 in this embodiment includes a store server 10, multiple payment machines 20 (20-1, ..., 20-K), a check-in terminal 30, an access point (AP) 50, and cameras 60 (60-1, ..., 60-M). For example, multiple payment machines 20 (for example, K units) are installed in one store. It is sufficient that at least one payment machine 20 is installed. Although only one check-in terminal 30 and one access point 50 are shown in FIG. 1, there may be two or more depending on the size of the store. The camera 60 is installed in a position where it can capture images of customers moving around the store.

At least one camera 60 is installed. Basically, the camera 60 is positioned so as to cover the entire store where customers can move around. When a customer moving around the store comes within the camera's shooting range, the camera 60 can capture an image of the customer. The camera 60 is also positioned in a position that can capture an image that can confirm that the customer has taken a specific action to purchase a product. Therefore, it is desirable for the camera 60 to be able to simultaneously capture, for example, a customer and products displayed on a shelf. Furthermore, the cameras 60 may be installed so that one customer can be photographed by multiple cameras 60 at the same time. In other words, by photographing a customer with multiple cameras 60 from different directions, it is possible to reduce blind spots that make it difficult to confirm the customer's actions.

The store server 10, the payment machine 20, the check-in terminal 30, and the access point 50 are connected to a network 70 such as a LAN (Local Area Network).

The store system 1 can incorporate electronic devices 40 (40-1, ..., 40-N) owned by customers into the system. The electronic devices 40 are, for example, smartphones, mobile phones, personal computers, tablet terminals, and portable game terminals. When incorporated into the store system 1, the electronic devices 40 communicate wirelessly with the access point 50. The electronic devices 40 may also be dedicated terminals provided within the store and used to register product information. For example, the electronic devices 40 are attached to carts used within the store and are dedicated terminals for use within the store.

In the store system 1 of this embodiment, the registration of product information may be performed using the electronic device 40, or may be performed in the accounting machine 20. In addition, in the store system 1, the payment process for the registered product information may be performed by the electronic device 40 using an external payment system, or may be performed in the accounting machine 20.

The check-in terminal 30 is installed at the entrance of the store. The check-in terminal 30 has a check-in function that authenticates customers who visit the store and notifies the store server 10. Customer authentication by the check-in terminal 30 can be performed by any one of a number of methods or a combination of a number of methods. For example, a first authentication method is to display a store-specific code (e.g., a two-dimensional code) on the display device (touch panel) of the check-in terminal 30 and have the electronic device 40 read it. A second authentication method is to display a terminal-specific terminal code (e.g., including a member ID that is identification data unique to the customer) on the electronic device 40 and have the check-in terminal 30 read it. A third authentication method is to input the customer's biometric information without using the electronic device 40 and authenticate by comparing it with biometric authentication data recorded in advance at the time of member registration. As biometric information for authenticating a customer (member), for example, data indicating characteristics such as a face image, fingerprint, voice, iris, and vein pattern can be used.

The electronic devices 40 are equipped with a function that allows customers to perform operations such as inputting information about the products they purchase. Each of the multiple electronic devices 40 is assigned unique identification information (terminal ID) and is managed by the store server 10.

The store server 10 controls the entire store system 1. The store server 10 in this embodiment has the function of a POS system for managing the sales of products and the function of a video analysis system for analyzing the video received from the camera 60. The store server 10 collates (matches) data generated by the POS system and the video analysis system, making it possible to easily check whether the customer's operation for purchasing products is performed correctly without complicating the operation for the customer to purchase products. For example, the store server 10 in this embodiment collates the number of product information (number of products) registered by the customer as a purchase target with the number of times the customer performed a predetermined action to purchase products. The number of product information registered by the customer as a purchase target is managed by the POS system. In addition, the number of times the customer performed a predetermined action to purchase products is detected by the video analysis system by analyzing the video captured during the product purchase behavior of one transaction by the customer. When it is determined by the collation (matching process) that the customer may not have performed the correct operation, the store server 10 can store the result of the collation (result data (collation information)). Conditions that may indicate that a customer may not be performing the correct operation include, for example, when the number of times a specific action is performed to purchase an item is greater than the number of items actually registered for purchase.

The store server 10 (POS system) creates transaction detail data including product information of the products purchased by the customer from the product codes entered in the electronic device 40 or the accounting machine 20. The product transaction detail data includes data on the number of registered products (number of product codes entered). When a transaction is made at the accounting machine 20, the store server 10 identifies the customer making the transaction and sends transaction detail data of the products purchased by that customer to the accounting machine 20 used by the customer for the transaction. The transaction detail data may be generated in the electronic device 40 or accounting machine 20 where the product code is input. In this case, the store server 10 receives and stores the transaction detail data from the electronic device 40 or accounting machine 20.

The store server 10 (video analysis system) also receives video captured by the camera 60 and performs various analytical processes based on this video. Examples of analytical processes include identifying customers, tracking customers as they move around the store, detecting customer movements, and identifying the products that customers are trying to purchase.

In customer identification, for example, feature data is detected from a facial image of the customer, and the customer is identified by comparing it with face authentication data (biometric information) showing the features of the facial image registered by the customer in advance. In customer tracking, for example, the customer in the video is identified based on the face image of the customer and the color and shape of the customer's belongings such as the clothes and bag worn by the customer. The store server 10 performs analysis to identify the customer for each of the multiple videos received from the multiple cameras 60. The store server 10 manages each video by associating one customer detected from each video. In detecting customer behavior, various actions are detected, such as picking up a product displayed on a shelf, returning a product taken from the shelf to the shelf, putting a product in a shopping cart, and performing an action to register product information using the electronic device 40. In identifying products, analysis processes are performed, such as reading the product code attached to the product from the video and identifying the product based on the design and shape of the product's package.

For example, a video analysis system counts the number of times a customer performs a specific action to purchase an item while moving around a store for one shopping transaction.

In this embodiment, the video analysis system is realized in the store server 10, but it is also possible to realize the video analysis system by a device (video analysis device) separate from the store server 10. The video analysis device is realized, for example, by a computer connected to the network 70. The video analysis device receives video data from the camera 60 and executes the various analysis processes described above. The video analysis device transmits data indicating the analysis results to the store server 10.

Figure 2 is a block diagram showing the main circuit configuration of the store server 10. The store server 10 comprises a processor 11, a main memory 12, an auxiliary storage device 13, a communication unit 14, and a system transmission path 15. The system transmission path 15 includes an address bus, a data bus, a control signal line, etc. In the store server 10, the processor 11, the main memory 12, the auxiliary storage device 13, and the communication unit 14 are connected to the system transmission path 15. In the store server 10, the processor 11, the main memory 12, the auxiliary storage device 13, and the system transmission path 15 that connects them form a computer.

The processor 11 corresponds to the central part of the computer. The processor 11 controls each part to realize various functions of the store server 10 according to an operating system or application programs. The processor 11 is, for example, a CPU.

The main memory 12 corresponds to the main storage portion of the computer. The main memory 12 includes a nonvolatile memory area and a volatile memory area. The main memory 12 stores an operating system or application programs in the nonvolatile memory area. The main memory 12 may also store data required for the processor 11 to execute processes for controlling each part in a nonvolatile or volatile memory area. The main memory 12 uses the volatile memory area as a work area where data is appropriately rewritten by the processor 11. The nonvolatile memory area is, for example, a ROM. The volatile memory area is, for example, a RAM.

The auxiliary storage device 13 corresponds to the auxiliary memory portion of the computer. For example, the auxiliary storage device 13 can be an HDD or an SSD. The auxiliary storage device 13 stores data used by the processor 11 when performing various processes, or data created by the processes in the processor 11. The auxiliary storage device 13 may also store the above-mentioned application programs.

The auxiliary storage device 13 stores, for example, a product database 61 and a member database 62. The auxiliary storage device 13 also has an area for storing transaction detail data 63 corresponding to multiple transactions, an area for storing video analysis data 64 generated by video analysis, video data 65 received from the camera 60, and result data 66 showing the results of comparing the data generated by the POS system and the video analysis system.

The product database 61 contains product data records created for each product sold in the store. The product data records include items such as product ID, product name, and price. The product ID is a unique code assigned to each product to identify each product individually. The product name and price are the name of the product identified by the product ID and the selling price per unit.

The communication unit 14 controls data communication between the accounting machine 20, the check-in terminal 30, the camera 60, or the electronic device 40 connected via the network 70 in accordance with a communication protocol.

Figure 3 is a block diagram showing the main circuit configuration of the accounting machine 20. The accounting machine 20 comprises a processor 21, a main memory 22, an auxiliary storage device 23, a communication unit 24, a touch panel 25, a printer 26, a reader/writer 27, a human presence sensor 29, a change dispenser 210, a camera 211, and a system transmission path 212. The system transmission path 212 includes an address bus, a data bus, a control signal line, and the like. In the accounting machine 20, the processor 21, the main memory 22, the auxiliary storage device 23, the communication unit 24, the touch panel 25, the printer 26, the reader/writer 27, the human presence sensor 29, and the camera 211 are connected to the system transmission path 212. In the accounting machine 20, a computer is made up of the processor 21, the main memory 22, the auxiliary storage device 23, and the system transmission path 212 that connects them.

The processor 21 corresponds to the central part of the computer. The processor 21 controls each part to realize various functions of the accounting machine 20 according to an operating system or an application program. The processor 21 is, for example, a CPU.

The main memory 22 corresponds to the main storage portion of the computer. The main memory 22 includes a nonvolatile memory area and a volatile memory area. The main memory 22 stores an operating system or application programs in the nonvolatile memory area. The main memory 22 may also store data required for the processor 21 to execute processes for controlling each part in a nonvolatile or volatile memory area. The main memory 22 uses the volatile memory area as a work area where data is appropriately rewritten by the processor 21. The nonvolatile memory area is, for example, a ROM. The volatile memory area is, for example, a RAM.

The auxiliary storage device 23 corresponds to the auxiliary memory portion of the computer. For example, the auxiliary storage device 23 can be an HDD or an SSD. The auxiliary storage device 23 stores data used by the processor 21 when performing various processes, or data created by the processes in the processor 21. The auxiliary storage device 23 may also store the above-mentioned application programs.

The communication unit 24 controls data communication with the store server 10 connected via the network 70 according to a communication protocol.

The touch panel 25 is a device that serves as both an input device and a display device for the accounting machine 20. The touch panel 25 detects the touch position on the displayed image and outputs the touch position information to the processor 21.

The printer 26 issues receipts by printing various characters or images on receipt paper. This type of printer 26 can be, for example, a thermal printer or a dot matrix printer.

The reader/writer 27 has a function of reading data recorded on a recording medium and a function of writing data to the recording medium. The reader/writer 27 may be a contact type reader/writer or a non-contact type reader/writer. The recording medium is a payment card such as a credit card, a debit card, an electronic money card, or a prepaid card. A wireless communication device such as a smartphone having the same function as a payment card may also be used as the recording medium.

The human presence sensor 29 detects the presence of a person standing in front of the payment machine 20, for example, using infrared rays, ultrasound, etc. When the presence of a person is detected, the human presence sensor 29 turns on.

The change machine 210 receives coins and bills that are inserted. The change machine 210 also dispenses coins and bills as change.

The camera 211 captures video (images) for, for example, identifying the customer and the product the customer is about to purchase. When identifying the customer, for example, feature data is detected from a captured facial image of the customer, and the customer is identified by comparing it with facial recognition data (biometric information) that shows the features of the facial image registered in advance by the customer. When identifying the product, analytical processing is performed, such as reading the product code attached to the product from the image and identifying the product based on the design and shape of the product's packaging.

A unique accounting machine ID is set for each accounting machine 20 to identify it. The accounting machine ID is stored, for example, in the auxiliary storage device 23.

Figure 4 is a block diagram showing the main circuit configuration of the check-in terminal 30. The check-in terminal 30 comprises a processor 31, a main memory 32, an auxiliary storage device 33, a communication unit 34, a touch panel 35, a camera 36, a speaker 37, and a system transmission path 39. The system transmission path 39 includes an address bus, a data bus, a control signal line, etc. In the check-in terminal 30, the processor 31, the main memory 32, the auxiliary storage device 33, the communication unit 34, the touch panel 35, the camera 36, and the speaker 37 are connected to the system transmission path 39. In the check-in terminal 30, the processor 31, the main memory 32, the auxiliary storage device 33, and the system transmission path 39 that connects them form a computer.

The processor 31 corresponds to the central part of the computer. The processor 31 controls each part to realize various functions of the check-in terminal 30 according to an operating system or an application program. The processor 31 is, for example, a CPU (Central Processing Unit).

The main memory 32 corresponds to the main storage portion of the computer. The main memory 32 includes a nonvolatile memory area and a volatile memory area. The main memory 32 stores an operating system or application programs in the nonvolatile memory area. The main memory 32 may also store data required for the processor 31 to execute processes for controlling each part in a nonvolatile or volatile memory area. The main memory 32 uses the volatile memory area as a work area where data is appropriately rewritten by the processor 31. The nonvolatile memory area is, for example, a ROM (Read Only Memory). The volatile memory area is, for example, a RAM (Random Access Memory).

The auxiliary storage device 33 corresponds to the auxiliary memory portion of the computer. For example, an HDD (Hard Disc Drive) or an SSD (Solid State Drive) can be the auxiliary storage device 33. The auxiliary storage device 33 stores data used by the processor 31 when performing various processes, or data created by the processes in the processor 31. The auxiliary storage device 33 may also store the above-mentioned application programs.

The communication unit 34 controls data communication with the store server 10 connected via the network 70 in accordance with a communication protocol. The check-in terminal 30 may also be configured to have a wireless communication unit and to perform wireless communication with the access point 50 in accordance with a wireless communication protocol.

The touch panel 35 is a device that serves as both an input device and a display device for the check-in terminal 30. The touch panel 35 detects the touch position on the displayed image and outputs the touch position information to the processor 31.

The camera 36 captures video (images) to authenticate customers who visit the store. For example, the camera 36 captures an image of a terminal-specific terminal code (e.g., a two-dimensional code including member ID data) displayed on the electronic device 40 carried by the customer (second authentication method). In addition, when authenticating a customer based on facial image features as biometric information of the customer, the camera 36 captures an image of the customer's face (third authentication method).

The speaker 37 outputs voice messages to customers who visit the store.

Figure 5 is a block diagram showing the main components of electronic device 40. Electronic device 40 includes a processor 41, a main memory 42, an auxiliary storage device 43, a touch panel 44, a camera unit 45, and a wireless unit 46. Processor 41 is connected to main memory 42, auxiliary storage device 43, touch panel 44, camera unit 45, and wireless unit 46 by a system transmission path 47 including an address bus, a data bus, control signal lines, and the like.

The electronic device 40 constitutes a computer with a processor 41, a main memory 42, an auxiliary storage device 43, and a system transmission path 47 that connects them.

The processor 41 corresponds to the central part of the computer. The processor 41 controls each part to realize various functions of the electronic device 40 according to the operating system and application programs.

The main memory 42 corresponds to the main storage portion of the computer. The main memory 42 includes a nonvolatile memory area and a volatile memory area. The main memory 42 stores the operating system and application programs in the nonvolatile memory area. The main memory 42 may also store data required for the processor 41 to execute processes for controlling each part in the nonvolatile or volatile memory area. The main memory 42 uses the volatile memory area as a work area where data is appropriately rewritten by the processor 41.

The auxiliary storage device 43 corresponds to the auxiliary memory portion of the computer. For example, an HDD, an SSD, or the like may be used as the auxiliary storage device 43. The auxiliary storage device 43 stores data used by the processor 41 in various processes and data generated by the processes in the processor 41. The auxiliary storage device 43 may also store application programs.

The touch panel 44 functions as an input device and a display device for the electronic device 40. Icons for launching application programs are displayed on the touch panel 44.

The camera unit 45 has a resolution capable of capturing an image of a barcode attached to a product that indicates product information. The wireless unit 46 transmits and receives data to and from the access point 60 using wireless communication. The electronic device 40 is, for example, a portable information processing device such as a smartphone, mobile phone, or tablet terminal that has a built-in camera unit 45 and wireless unit 46 and can be used in the store system 1.

The electronic device 40 has the product registration program P1 installed, and operates as an information terminal compatible with the store system 1 through the functions realized by the product registration program P1.

The product registration program P1 controls the processor 41 so that it becomes a terminal that registers product information through customer operation. The product registration program P1 is stored in the auxiliary storage device 43. In addition to the product registration program P1, an area for storing product information and the like is provided in the auxiliary storage device 43. The product registration program P1 also receives from the store server 10 and manages (stores) a member ID unique to the customer that is issued upon member registration (registration of member information) at the store server 10.

Next, we will explain the data stored in the auxiliary memory device 13 of the store system 1.

Figure 6 is a schematic diagram showing the main data structure of a member data record 62R registered in the member database 62 of the store server 10. Figure 6 shows data corresponding to one member. As shown in Figure 6, the member data record 62R registers member information provided by a member in association with a member ID unique to the customer who has registered as a member. The member information includes biometric information and personal information as information for identifying or authenticating the member (customer). Biometric information can be data indicating characteristics such as a facial image, fingerprint, voice, iris, or vein pattern. Personal information can include data indicating, for example, name, address, telephone number, and payment method (e.g., cash, electronic money, credit).

Figure 7 is a diagram showing the data structure of the transaction details data 63 of the store server 10. The transaction details data 63 is product information registered by a customer scanning a product (entering a product code). The transaction details data 63 includes a member ID 63A, a transaction ID 63B, a purchase quantity 63C, and purchased product information 63D. The member ID 63A is set to the member ID of the customer authenticated when visiting the store. The transaction ID 63B is generated to identify one transaction by the authenticated customer, and is set in correspondence with the member ID 63A. The purchased product information 63D is information on the products purchased by the customer identified by the member ID. The purchased product information 63D is registered by reading the product ID, product name, price, etc. corresponding to the detected product code from the product database 61. The purchase quantity 63C is information indicating the number of products for which the purchased product information 63D is registered. When product information is registered in the electronic device 40, the transaction detail data is received from the electronic device 40, and when product information is registered in the accounting machine 20, the transaction detail data is received from the accounting machine 20.

Figure 8 is a diagram showing the data structure of the video analysis data 64 of the store server 10. The video analysis data 64 is data showing the analysis results of the video received from the camera 60. The video analysis data 64 includes data such as a member ID 64A, check-in date and time 64B, number of actions 64C, and check-out date and time 64D. The check-in date and time 64B shows the date and time (indicating the year, month, day, and time) when a customer checked in (came to the store). The number of actions 64C shows the number of times an authenticated customer performed a specific action to purchase a product while the customer was being tracked (during one product purchase transaction). The check-out date and time 64D shows the date and time (indicating the year, month, day, and time) when the customer checked out (end of payment processing).

Furthermore, although not shown, the video analysis data 64 may store data for identifying the camera 60 that captured the video in which a predetermined action for a customer to purchase a product is detected. For example, the camera 60 is installed in association with a sales floor of a specific product. In this way, when a predetermined action by a customer is detected by analyzing the video, the camera 60 that captured the video to be analyzed can be identified, and the sales floor of the product in which the camera 60 is installed can be specified. Therefore, it is possible to determine in which sales floor the customer performed the predetermined action. Furthermore, by determining whether product information of the product sold in the sales floor where the customer performed the predetermined action is registered, it is possible to compare the registered product information with the sales floor where the customer performed the predetermined action. By this comparison, it is possible to determine, for example, that product information of the product sold in the sales floor where the predetermined action was performed is not registered, or that the number of actions in the sales floor where the predetermined action was performed does not match the number of registered products sold in the corresponding sales floor.

Figure 9 is a diagram showing the data structure of video data 65 of the store server 10. Video data 65 includes tracking video data 65B used to track one customer moving around the store. Tracking video data 65B is stored in association with a transaction ID 65A, and includes video data capturing the product purchasing behavior of one customer in one transaction. Typically, tracking video data 65B includes video data captured by multiple cameras 60.

10 is a diagram showing the data structure of the result data 66 of the store server 10. The result data 66 shows the result of matching (matching information) between the data generated by the POS system and the video analysis system. In the result data 66, matching result data 66C indicating that the number of times a predetermined action was performed is greater than the number of products is stored as the result of matching (matching information) in association with the member ID 66A and the transaction ID 66B. The matching result data 66C may include data indicating that the number of products is greater than the number of times the action was performed, as well as data indicating that the number of products is greater than the number of times the action was performed. The matching result data 66C may also include data indicating the difference (number of mismatches) between the number of times the action was performed and the number of registered products. Furthermore, the matching result data 66C may also include data indicating that the location of the camera 60 that captured the video in which the predetermined action was detected (for example, corresponding to a sales floor for a specific product) and the registered product information were determined to be mismatched.

Next, the operation of the store system 1 in this embodiment will be described. First, the operation will be described when the product information of the product to be purchased is registered by the customer himself/herself using the electronic device 40.

Figure 11 is a flowchart for explaining the operation of the store server 10 (processor 11) in this embodiment. Figure 12 is a flowchart for showing the operation of the electronic device 40 (processor 41) in this embodiment. Note that the operations shown in Figures 11 and 12 are merely examples, and the processing procedures and contents are not particularly limited as long as similar results can be obtained.

First, when a customer arrives at the store, he or she operates the electronic device 40 to launch the product registration program P1 and start the product registration process. When the product registration process starts, the processor 41 executes the check-in process based on the product registration program P1 (FIG. 12, Act 21).

Figure 13 is a flowchart showing the first check-in process executed by the electronic device 40 in this embodiment, and Figure 14 is a flowchart showing the second check-in process. The first check-in process identifies the guest using the electronic device 40, and the second check-in process identifies the guest using the guest's biometric information.

In the store system 1, it is assumed that it is predetermined whether to use the first check-in process or the second check-in process. In addition, the first check-in process and the second check-in process may be configured to be used in combination.

First, the first check-in process will be described with reference to the flowchart shown in FIG. 13. When a customer operates the touch panel 44 to instruct an in-store purchase (FIG. 13, YES in Act 41), the processor 41 activates the camera unit 45 and transitions to a store code reading state.

The check-in terminal 30 displays a store-specific store code, such as a two-dimensional code pattern, on the touch panel 35. The customer operates the electronic device 40 so that the camera unit 45 of the electronic device 40 reads the store code displayed on the check-in terminal 30.

When the processor 41 reads the store code (Act 42), it sends a check-in request to the store server 10 together with the member ID pre-stored in the electronic device 40 and the date and time (year, month, date, and time) when the store code was read (Act 43).

The store server 10 receives a check-in request from the electronic device 40 through the access point 50. When the processor 11 of the store server 10 receives the member ID together with the check-in request (YES in Act 1), it writes the member ID to the work area of the main memory 12. The processor 11 then searches the member database 62 using the member ID as a search key, and determines whether or not there is a member data record 62R in which the member ID is written (Act 2).

When the processor 11 confirms that there is no corresponding member data record 62R (NO in Act 3), it sends a check-in denial notification to the electronic device 40 (Act 4 in Fig. 11). When the processor 41 of the electronic device 40 receives a check-in denial notification from the store server 10 (NO in Act 44 in Fig. 13), it outputs a denial notification to notify that check-in has been denied (Act 45). For example, the processor 41 causes the touch panel 44 to display a message indicating that check-in has been denied. In this case, the electronic device 40 ends the first check-in process and ends the product registration process.

On the other hand, when processor 11 confirms that a corresponding member data record 62R exists, it further checks whether data (matching information) indicating a preset matching result is set in result data 66 in association with the corresponding member ID. In other words, it determines whether the customer corresponding to the member ID was determined to have possibly not performed the correct operation when they previously visited the store and made a purchase, in the matching (matching process) described below. For example, if it is determined that the number of times a specified action was performed to purchase an item during a previous purchase was greater than the number of items actually registered for purchase, matching result data 66C is stored, indicating that the number of times the specified action was performed was greater than the number of items.

Here, if the matching result data 66C is stored, the processor 11 transmits a check-in denial notice to the electronic device 40 according to the matching result indicated by the matching result data 66C (Act 4 in FIG. 11). For example, the processor 11 transmits a check-in denial notice indicating that the store cannot be used, that the membership registration has been suspended, or that confirmation by a store clerk is required, according to the matching result indicated by the matching result data 66C. When the processor 41 of the electronic device 40 receives a check-in denial notice from the store server 10 (NO in Act 44 in FIG. 13), it outputs a denial notice to notify that the check-in has been denied, and ends the first check-in process and the product registration process (Act 45).

When matching result data 66C is stored, check-in is not always denied, but rather the decision to permit or deny check-in may be made according to the contents of matching result data 66C. For example, if matching result data 66C includes data indicating the difference (number of mismatches) between the number of times a specific action was taken to purchase a product and the number of registered products, check-in may be denied if the number of mismatches is equal to or greater than a preset number (e.g., 3 or more). Also, check-in may be denied if the same member ID is registered in result data 66 more than a preset number (e.g., 2). Other conditions for permitting check-in may be set in advance.

When a member ID is registered in the result data 66, the store server 10 or an information processing device used by the store clerk may be notified of the relevant member's information. When the member's information has been notified, check-in may be permitted so that the store clerk can assist the customer who has visited the store. In this case, the store clerk will need to assist the customer, but since there is no need to have a store clerk known as an attendant on duty at all times, this does not become an obstacle to reducing the number of store clerks.

If the processor 11 confirms that the member ID received in response to the check-in request is in the member data record 62R and that the corresponding member ID is not registered in the result data 66 (YES in Act 3), it sends a check-in permission notification to the electronic device 40 (Act 5). The processor 11 generates the member ID of the customer whose check-in has been permitted and a transaction ID for identifying one transaction by this customer, and stores them in the auxiliary storage device 13 (Act 6).

When the processor 41 of the electronic device 40 receives a check-in permission notification from the store server 10 (FIG. 13, YES in Act 44), the electronic device 40 is ready to register product information through a customer's operation. The registration of product information will be described later.

Next, the second check-in process will be described with reference to the flowchart shown in FIG. 14. In the second check-in process, the guest's biometric information is used to identify the guest. Here, a case will be described in which, for example, feature data of the guest's facial image is used as the biometric information.

The customer has his/her face photographed by the camera 36 of the check-in terminal 30 and requests authentication processing. The processor 31 of the check-in terminal 30 detects feature data from the face image photographed by the camera 36 and compares it with the biometric information (feature data of the face image) of each member registered in the member database 62 of the store server 10.

When the processor 31 is able to authenticate the member by matching the biometric information, it queries the store server 10 as to whether the member ID of the relevant member is registered in the result data 66, in the same manner as in the first check-in process. Here, if matching result data 66C is stored, the processor 31 sends a notice of check-in denial to the electronic device 40, in the same manner as in the first check-in process. As a method of notifying the authentication result, for example, an authentication code (e.g., a two-dimensional code pattern) indicating the authentication result is displayed on the touch panel 35. In this case, the customer operates the electronic device 40 to have the authentication code read by the camera unit 45 of the electronic device 40, in the same manner as in the first check-in process.

The processor 41 of the electronic device 40 receives the authentication result from the check-in terminal 30 (Act 51 in FIG. 14). If the result is a notification that check-in is not permitted (No in Act 52 in FIG. 14), the processor 41 outputs a notification of denial to notify that check-in is not permitted in the same manner as described above, and ends the second check-in process and the product registration process (Act 55).

In addition, if the check-in terminal 30 determines that check-in is not permitted, a notification that check-in is not permitted may simply be displayed on the touch panel 35 of the check-in terminal 30 (and a message may also be output from the speaker 37).

On the other hand, if the processor 31 is able to authenticate the member by matching the biometric information and the member ID of the relevant member is not registered in the result data 66, it notifies the member of the authentication result indicating that check-in is permitted, in the same manner as described above.

When the processor 41 of the electronic device 40 receives a notification that check-in is permitted (YES in Act 52), it sends a check-in request to the store server 10 together with the member ID and the date and time (year, month, date, and time) when the authentication code was read, which are pre-stored in the electronic device 40 (Act 53).

When a customer operates the touch panel 44 to instruct in-store purchase (YES in Act 54), the processor 41 enters a state in which product information can be registered by the customer's operation. That is, the processor 41 activates the camera unit 45 and transitions to a state in which the product code can be read.

Below, we will explain what happens when a customer has completed check-in.

The processor 11 of the store server 10 identifies the customer who has checked in from the video captured by the camera 60. Features that can be used to identify the customer from the video include, for example, the customer's facial image, and the color and shape of the customer's clothing, bag, and other belongings. Note that customer identification may also be performed using video captured by the camera 36 installed in the check-in terminal 30.

Then, the processor 11 analyzes whether a customer has been captured on video images taken by the multiple cameras 60 placed at various locations based on the customer's characteristics, and tracks the customer moving around the store (Act 7). The processor 11 stores the video data in which the customer to be tracked is identified as video data 65 in association with transaction authentication 65A, as shown in FIG. 9.

On the other hand, when the processor 41 of the electronic device 40 receives the check-in permission notification, it displays a product list screen on the touch panel 44 and waits for an input operation by the customer (Act 22). The processor 41 also displays a message on the touch panel 35 to notify that the device has entered an operational state in which product registration is possible. For example, the processor 41 displays a message such as "Check-in successful. Please start shopping" on the touch panel 35. The electronic device 40 may also output the message by voice.

The processor 41 of the electronic device 40 causes the touch panel 44 to display, for example, a product list screen SC1 as shown in FIG. 15. Note that the product list screen SC1 shown in FIG. 15 shows an example of a state in which some products have already been registered, and no product information is displayed in the initial state.

The product list screen SC1 shown in FIG. 15 includes a registered product list area 70, a product to be registered area 71, a total display area 72, a checkout button 73, and a product registration button 74. The registered product list area 70 is an area for displaying a list of the product names, quantities, unit prices, and amounts for registered products. The product to be registered area 71 is an area for displaying the product name, quantity, and total amount of the most recently registered product. The total display area 72 is an area for displaying the quantity and total amount of registered products.

The checkout button 73 is a button for notifying the store server 10 that product registration has been completed and that the process will proceed to checkout. The product registration button 74 is a button for instructing the user to proceed to product registration mode.

Note that the product list screen SC1 only shows a basic form for displaying various information about registered products, and it is possible to display various information related to product registration other than that shown in FIG. 15.

When the processor 41 detects an operation on the product registration button 74 on the touch panel 44 (Act 23: YES), it activates the camera unit 45 and displays a product code reading screen for reading product information on the touch panel 44 instead of the product list screen SC1 (Act 24).

Figure 16 is a diagram showing an example of the product code reading screen SC2. The product code reading screen SC2 has an image display area 81 and a product list button 83. The image display area 81 is an area for displaying an image captured by the camera unit 45, and displays a line 82 indicating the barcode reading position. The product list button 83 is a button for stopping the reading of product information and instructing the display of the product list screen SC1. The product code reading screen SC2 displays a message such as "Please align the barcode with the line." This allows the customer to easily understand the operation of registering product information using the product code reading screen SC2.

When a customer finds an item to purchase on the sales floor, the customer photographs the item's item code symbol, such as a barcode, attached to the item with the camera unit 45 of the electronic device 40. The image photographed by the camera unit 45 is displayed in the image display area 81. The customer adjusts the display range so that the item code symbol (barcode) is displayed at the position of the line 82.

The processor 41 of the electronic device 40 extracts a code symbol from the image captured by the camera unit 45 and reads a product ID (product code) unique to the product, for example, indicated by the code symbol. When the product ID is read from the image captured by the camera unit 45 (YES in Act 25), the processor 41 transmits product registration data including the product ID and the member ID used in the check-in request to the store server 10 (Act 27).

If the code symbol cannot be extracted from the image for a certain period of time (Act 26, YES), the processor 41 changes the display from the product code reading screen SC2 to the product list screen SC1 (Act 22).

When the processor 11 of the store server 10 receives product registration data from the electronic device 40, it searches the product database 61 using the product ID as a search key. The processor 11 retrieves a product data record that corresponds to the product ID from the product database 61. The product data record includes the product ID, product name, price, etc. The processor 11 transmits the product data record to the electronic device 40.

When the processor 41 of the electronic device 40 receives the product data record from the store server 10 (Act 28), it generates transaction details data and stores it in the auxiliary storage device 43. The processor 41 also switches the display on the touch panel 44 from the product code reading screen SC2 to the product list screen SC1, and updates the display of the registered product list area 70, the product to be registered area 71, and the total display area 72 based on the product name, price, etc. included in the purchased product information of the transaction details data.

After confirming that the product name, price, etc. of the product whose product information (product ID) was read is displayed on the touch panel 44, the customer continues shopping. Then, when he finds the product he wants to purchase, he enters the product ID of that product into the electronic device 40. When the processor 41 of the electronic device 40 detects that the product ID has been entered, it transmits product registration data including the product ID to the store server 10 in the same manner as described above. The processor 41 updates the transaction details data based on the product data record received from the store server 10, and changes the display on the product list screen SC1. The transaction details data includes data indicating the number of products registered for purchase.

A customer moves around the store, finds an item to purchase, removes it from the shelf, and inputs the item's item ID into the electronic device 40, repeating this process. After inputting the item ID into the electronic device 40, the customer places the item in a cart or places it in a bag. In addition, a customer may remove an item from the shelf, check it, then decide not to purchase it and return it to the shelf. Furthermore, a customer may commit fraudulent actions such as placing an item in a bag without inputting the item ID into the electronic device 40.

While tracking a customer moving through the store, the processor 11 of the store server 10 analyzes whether the customer has performed a predetermined action to purchase a product based on the video captured by the camera 60 (Act 8 in FIG. 11). If the processor 11 determines from the analysis of the video that the customer has performed, for example, removing a product from a shelf (Act 9: YES), it counts up (+1) the number of times the customer has performed a predetermined action to purchase a product (Act 10). Also, if the processor 11 determines from the analysis of the video that the customer has performed, for example, removing a product from a shelf and returning it to the shelf (Act 11: YES), it counts down (-1) the number of times the customer has performed a predetermined action to purchase a product (Act 12). That is, the processor 11 of the store server 10 analyzes the actions of the customer to be tracked and counts the number of times the customer has performed a predetermined action to purchase a product until checking out (Acts 7 to 13).

When a customer has finished shopping at the sales floor, the customer touches the checkout button 73 displayed on the touch panel 44 to declare that they wish to carry out the checkout. When the processor 41 of the electronic device 40 detects that the checkout button 73 has been touched (YES in Act 29), it accepts the selection of the checkout method. If a cashless checkout is selected (YES in Act 30), the processor 11 allows the customer to select a payment method (Act 31). Here, for example, a credit card company that the customer can use is selected.

The processor 11 executes payment for the products registered in the transaction details data using the payment method designated by the customer (e.g., a payment system provided by a credit company) (Act 32).

On the other hand, if cashless payment is not specified (NO in Act 30), the processor 11 generates a transaction barcode including the contents of the transaction detail data and displays it on the touch panel 44 (Act 33). The customer, still holding the electronic device 40, moves to the location of the payment machine 20 where the payment will be processed. The payment machine 20 turns on the human presence sensor 29 when the customer approaches. When the human presence sensor 29 turns on, the processor 21 of the payment machine 20 displays a message on the touch panel 25 requesting the input of the transaction barcode. The customer causes the camera 211 of the payment machine 20 to read the transaction barcode displayed on the touch panel 44.

The processor 21 of the payment machine 20 causes the touch panel 25 to display the transaction details indicated by the transaction barcode (product name, price, total, number of products, etc.). The processor 21 also causes the touch panel 25 to display a screen for inputting the payment method designation, and allows the customer to designate the payment method. Here, for example, either cash or electronic money is designated. When the payment method is designated by operating the touch panel 25, the processor 21 executes the payment process using the designated payment method.

For example, when payment is made with cash, the total amount is paid via the change dispenser 210. When payment is made with electronic money, the amount is paid from an electronic money card read by the reader/writer 27. When the payment process is completed, the processor 21 notifies the electronic device 40 that the payment has been completed.

When the processor 41 of the electronic device 40 receives the payment completion notification (FIG. 12, YES in Act 34), it sends transaction detail data for one transaction for which the payment process has been completed to the store server 10 (Act 35). The store server 10 stores the transaction detail data 63 received from the electronic device 40 in the auxiliary storage device 13. The processor 41 also sends a checkout request to the store server 10, which includes the member ID, the date and time when the payment process was completed, and the transaction ID (Act 36).

When the processor 11 of the store server 10 receives a check-out request from the electronic device 40 (FIG. 11, YES in Act 13), it ends tracking of the customer corresponding to the member ID included in the check-out request (Act 14). The processor 11 also stores video analysis data 64 for one transaction for which payment has been completed in the auxiliary storage device 13. As shown in FIG. 8, the video analysis data 64 includes data such as the member ID 64A, check-in date and time 64B, number of actions 64C, and check-out date and time 64D.

Processor 11 uses the transaction ID of the transaction for which the payment process has been completed as a key to identify the number of purchases 63C in the transaction details data 63 and the number of actions 64C registered in the video analysis data 64. Processor 11 compares the number of purchases 63C with the number of actions 64C to execute a matching process to determine whether there is a discrepancy in the numbers (Act 16). That is, processor 11 compares the number of product information items (number of products) registered by the customer as items to be purchased with the number of times the customer performed a specified action to purchase the products (such as removing products from a shelf).

When the processor 11 determines as a result of the matching process that, for example, the number of times a predetermined action was taken to purchase a product is greater than the number of products actually registered as purchase targets, the processor 11 stores matching result data 66C in the auxiliary storage device 13 (result data 66) in association with the member ID 66A and transaction ID 66B. In this case, the matching result data 66C indicates that the number of times the predetermined action was taken is greater than the number of products.

In this way, by storing the result data 66, the store server 10 can call up the tracking image data 65B corresponding to the transaction ID 66B from the image data 65 based on the transaction ID 66B associated with the matching result data 66C. That is, the image captured while the customer is being tracked can be displayed on a display device (not shown) provided in the store server 10, the touch panel 25 of the checkout machine 20, or a display device of another information processing device (personal computer) connected to the network 70. This allows the customer's product purchasing behavior to be confirmed by the image. By this confirmation, the contents of the data stored in the result data 66 can be changed, for example, by the operation of the manager of the store server 10. For example, if it is confirmed that the customer has performed the correct operation, the matching result data 66C can be deleted (or the contents changed) so that check-in is not denied in the check-in process the next time the customer visits the store.

In this way, the store system 1 of this embodiment checks the actions taken by the customer to purchase a product, and cross-checks the number of times the customer performed the specified actions to purchase a product with the number of products actually registered as items to be purchased, making it easy to confirm whether the customer's operations are being performed correctly. Because the customer only needs to perform the actions required for a normal product purchase using the electronic device 40, the operations required for the customer to purchase a product are not complicated.

Next, we will explain the operation when the customer uses the payment machine 20 to register product information for a product to be purchased by themselves. For example, the payment machine 20 allows the customer to register product information for multiple products at once.

Figure 17 is a flowchart for explaining the operation of the store server 10 (processor 11) in this embodiment. Figure 18 is a flowchart for showing the operation of the check-in terminal 30 (processor 31) in this embodiment. Figure 19 is a flowchart for showing the operation of the check-in device 20 (processor 21) in this embodiment. Note that the operations shown in Figures 17 to 19 are merely examples, and the processing procedures and contents are not particularly limited as long as similar results can be obtained.

The check-in terminal 30 uses the biometric information of the guest to manage the guest's check-in. Here, we will explain the case where feature data of the guest's facial image is used as the biometric information, for example.

The customer has his/her face photographed by the camera 36 of the check-in terminal 30 and requests authentication processing. The processor 31 of the check-in terminal 30 detects feature data from the face image photographed by the camera 36 and compares it with the biometric information (feature data of the face image) of each member registered in the member database 62 of the store server 10 (Figure 18, Act 81).

If the processor 31 is unable to authenticate the member by matching the biometric information (NO in Act 82), it may display a notice that check-in is not permitted on the touch panel 35 of the check-in terminal 30 (and may also output a message from the speaker 37) (Act 85).

If the processor 31 is able to authenticate the member by matching the biometric information, it inquires of the store server 10 whether the member ID of the member in question is registered in the result data 66, in the same manner as in the first check-in process. If matching result data 66C is stored, the processor 31 causes the touch panel 35 of the check-in terminal 30 to display a notice that check-in is not permitted (Act 85).

On the other hand, if the processor 31 can authenticate the member by matching the biometric information and the member ID of the relevant member is not registered in the result data 66, it may display a check-in permission notification on the touch panel 35 of the check-in terminal 30 (and may also output a message from the speaker 37) (Act 83). The processor 31 generates the member ID of the customer whose check-in has been permitted and a transaction ID for identifying one transaction by this customer. The processor 31 sends a check-in notification including the member ID and transaction ID to the store server 10 (Act 84).

When the processor 11 of the store server 10 receives a check-in request from the check-in terminal 30 (FIG. 17, YES in Act 61), it stores the member ID and transaction ID included in the check-in notification in the auxiliary memory device 13 (Act 62).

When the customer has completed the check-in process, the store server 10 starts tracking the customer (Act 63). Note that the processes of Acts 63 to 68 shown in FIG. 17 are executed in the same manner as the processes of Acts 7 to 10 shown in FIG. 11, so a description thereof will be omitted.

As customers move around the store, they remove products they wish to purchase from shelves and place them in a cart, for example. When the customer has finished selecting the products they wish to purchase, they move to the location where the payment machine 20 is installed. The payment machine 20 turns on the human presence sensor 29 when the customer approaches. When the human presence sensor 29 turns on, the processor 21 of the payment machine 20 transitions to an operational state for product registration (Figure 19, Act 91). The processor 21 activates the camera 211 to take images of the products placed in the cart or on a reading table, for example. Here, it is assumed that all products are photographed at once, for example.

The processor 21 performs analysis processing such as reading the product code attached to the product from the image captured by the camera 211, or identifying the product based on the design or shape of the product's packaging, to identify the product to be purchased.

The processor 21 transmits product registration data indicating the identified product to the store server 10 and receives the product data record stored in the product database 61 for the relevant product. The processor 21 generates transaction details data based on the product data record received from the store server 10. In addition, the processor 21 displays, for example, a product list screen shown in FIG. 15 on the touch panel 25 according to the contents of the transaction details data.

When the processor 21 detects that the checkout button on the product list screen has been touched (Act 92: YES), the processor 21 accepts the selection of the payment method. The processor 21 determines the payment method according to the customer's operation to select the payment method (Act 93).

Here, the processor 21 performs an authentication process for the customer (Act 94). For example, the processor 21 performs authentication using biometric information (e.g., a facial image) of the customer, in the same manner as the check-in terminal 30 described above.

If the processor 21 is unable to authenticate the member by matching the biometric information (Act 95: NO), it causes the touch panel 25 of the payment machine 20 to display a notification of denial (Act 97). On the other hand, if the processor 21 is able to authenticate the member by matching the biometric information (Act 95: YES), it causes the touch panel 25 of the payment machine 20 to display a notification of permission (Act 96). The processor 21 executes the payment process using the specified payment method (Act 98).

When the payment process is completed, the processor 21 sends a check-out notification to the store server 10 (Act 99), which includes transaction detail data for one transaction for which the payment process has been completed, as well as the member ID, the date and time when the payment process was completed, and the transaction ID. As shown in FIG. 7, the transaction detail data includes data on the number of products (purchased items) whose product information is registered as items to be purchased.

When the processor 11 of the store server 10 receives the check-out notification (YES in Act 69 in FIG. 17), it stores the transaction statement data (including data on the number of items purchased) received from the payment machine 20 in the auxiliary storage device 13. The processor 11 also ends tracking for the customer corresponding to the member ID received from the payment machine 20 (Act 70). The store server 10 executes the processes of Acts 71 to 73 shown in FIG. 17 in the same manner as the processes of Acts 15 to 17 shown in FIG. 11. In addition, in Act 72, it executes a matching process using the number of items purchased (number of items) indicated in the transaction statement data received from the payment machine 20. Below, the explanation of Acts 71 to 73 shown in FIG. 17 is omitted.

In this way, in the store system 1 of this embodiment, even when registering product information using the cash register 20 without using the electronic device 40, it is possible to easily check whether the customer's operations are being performed correctly by cross-checking the number of times a specific action has been taken to purchase a product with the number of products actually registered as items to be purchased.

In the store system 1 of this embodiment, there is no need to assign dedicated store staff for registering product information (product scanning) and checking the bill, which allows for staff reduction. In addition, customer behavior can be checked without placing any burden on the customers, preventing fraudulent behavior by customers. Furthermore, in the store system 1 of this embodiment, customer behavior is analyzed and products are registered using images captured by a camera. In other words, customer behavior can be easily analyzed without the need for many sensors. This makes it possible to reduce the cost of introducing the system.

The transfer of the store server 10 is generally performed with the program stored in a recording medium such as a ROM or HDD. However, this is not limited, and the program may be transferred without being stored in a recording medium. In this case, the program transferred separately from the store server 10 is written to a writable storage device provided in the store server 10 in response to an operation by the user or the like. The program can be transferred by recording it on a removable recording medium or by communication via a network. The form of the recording medium is not important as long as it can store the program, such as a CD-ROM or memory card, and is readable by the device.

Although several embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included in the scope and spirit of the invention, and are included in the scope of the invention and its equivalents described in the claims.
The invention as originally claimed in the present application is set forth below.
[1] An information processing device having an image receiving means for receiving image captured by a camera, a receiving means for receiving identification data unique to a customer, a counting means for counting the number of times the customer has taken a predetermined action to purchase a product based on the captured image of the customer, a comparison means for comparing the number of times with the number of products whose product information is registered as items available for purchase by the customer, and a memory means for storing comparison information indicating the results of the comparison.
[2] The information processing device according to appendix [1], wherein the storage means stores the matching information in association with the identification data.
[3] An information processing device as described in appendix [1], further comprising an output means for outputting a notification corresponding to the matching result when the matching information indicating a predetermined matching result is stored in correspondence with the identification data upon receiving the identification data.
[4] An information processing device according to any one of appendices [1] to [3], further comprising an image storage means for storing the image captured of the customer in correspondence with the identification data.
[5] A store system having a camera for photographing customers, electronic equipment for registering product information of products to be purchased, a counting means for counting the number of times the customer performs a predetermined action to purchase a product based on the image of the customer, a comparison means for comparing the number of times with the number of products for which the product information has been registered by the electronic equipment, and a memory means for storing comparison information indicative of the results of the comparison.

1...store system, 10...store server, 11, 21, 31, 41...processor, 12, 22, 32, 42...main memory, 13, 23, 33, 43...auxiliary storage device, 14, 24, 46...communication unit, 20 (20-1, ..., 20-K)...accounting machine, 25, 35...touch panel, 26...printer, 27...reader/writer, 30 (30-1, ..., 30-M)...information terminal, 34...wireless unit, 36...camera, 37...speaker, 40 (40-1, ..., 40-N)...electronic device, 50...access point, 60...network.

Claims (9)

A video receiving means for receiving a video image captured by a camera;
A receiving means for receiving identification data unique to the customer from the electronic device;
an identification means for identifying a customer in the video corresponding to the identification data after receiving the identification data;
A counting means for counting the number of times the identified customer performs a predetermined action for purchasing a product;
a first storage means for storing the number of times and the identification data in association with each other;
An information processing device having the above configuration. A verification means for verifying the number of products whose product information is registered as purchase targets for the customer with the number of times;
2. The information processing apparatus according to claim 1, further comprising: second storage means for storing collation information indicating a result of said collation in association with said identification data. 3. The information processing device of claim 2, further comprising an output means for outputting a notification that authentication of the customer corresponding to the identification data is not permitted if, at the time of receiving the identification data, the matching information indicating a matching result that the number of products registered in the product information does not match the number of times is stored in correspondence with the identification data. an image storage means for storing the image of the customer in association with transaction identification data corresponding to the identification data;
4. The information processing device according to claim 2, further comprising: a change means for changing the transaction identification data and the collation information corresponding to the identification data after the image corresponding to the transaction identification data is displayed. 4. The information processing apparatus according to claim 1, wherein the counting means stops counting the number of times when a check-out request is received from the electronic device. The information processing apparatus according to claim 1 , wherein the counter counts the number of times by counting up and counting down in response to a predetermined action. A camera to photograph the customers,
An electronic device for registering product information of a product to be purchased;
a receiving means for receiving customer-specific identification data from the electronic device;
A means for identifying a customer corresponding to the identification data in an image captured by the camera after receiving the identification data;
A counting means for counting the number of times the identified customer performs a predetermined action for purchasing a product;
and a means for storing the number of times and the identification data in association with each other . Computer,
A video receiving means for receiving a video image captured by a camera;
A receiving means for receiving identification data unique to the customer from the electronic device;
an identification means for identifying a customer in the video corresponding to the identification data after receiving the identification data;
A counting means for counting the number of times the identified customer performs a predetermined action for purchasing a product;
a program for causing the program to function as a first storage means for storing the number of times and the identification data in association with each other; Computers, and more
A verification means for verifying the number of products whose product information is registered as purchase targets for the customer with the number of times;
9. The program according to claim 8, which functions as a second storage means for storing collation information indicating a result of the collation in association with the identification data.