JP2013238973A - Purchase information management system, merchandise movement detection device and purchase information management method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a purchase information management system, a merchandise movement detection device and a purchase information management method that can simply and highly accurately acquire information required for a purchase process analysis.SOLUTION: A merchandise movement detection device 100 comprises:a merchandise movement detection part 102 that monitors a movement of merchandise via a movement detection sensor 101 and generates merchandise movement information indicative of detection of the merchandise movement; and a merchandise movement person characteristic extraction part 104 that acquires characteristic information on a person moving the merchandise on the basis of imaging information captured by a merchandise movement imaging part 103. A sales management device 200 comprises: a sales information acquisition part 202 that acquires sales information on merchandise; and a merchandise purchase person characteristic extraction part 204 that acquires characteristic information on a person purchasing merchandise on the basis of imaging information captured by a purchaser imaging part 203. A purchase information management device 300 comprises: a purchase information management part 301 that associates information received from the merchandise movement detection device 100 and the sales management device 200 with a device of a transmission source and retains the information as purchase information.

Description

  The present invention relates to a purchase information management system that acquires and analyzes information related to the movement of a product in a store and attention to the product, a product movement detection device and a purchase information management method in the purchase information management system.

  In retail stores such as convenience stores and supermarkets, analysis of purchasing behavior is an important item. Based on the analysis result of the purchase behavior, the retail store can judge the quality of the product purchase quantity, the number of displays, and the location. In retail stores, the appeal of products in stores has a great influence on consumer purchasing behavior. For this reason, there is a demand for determining the effect of product appeal using a display method or advertisement. Therefore, POS (Point Of Sale system) has been introduced in many retail stores, and a mechanism for sequentially managing sales information is in place.

  In addition, there is a demand for quantitative analysis of the product purchase process as more detailed information than just purchase results. At POS, it is only possible to grasp the inventory and sales of products. In other words, analysis of “sold” products is possible, but information is insufficient to analyze “sold” products.

  Techniques for analyzing purchasing processes have been proposed. For example, Patent Document 1 describes a system and method for marketing strategy planning in consideration of customer attributes and regions. The system and method described in Patent Literature 1 detect a line of sight by analyzing an image of a customer's face acquired from a camera, and determine a product that the customer has watched. In addition, based on the time-series data of the product that the customer has watched, the customer attribute in the purchase behavior of the customer is determined. And the product identification data of the product which the customer purchased is acquired, and also the regional data is given to generate the customer purchase data. This makes it possible to acquire area analysis data such as the customer base of the buyer who purchases the product, and to make a marketing strategy that takes into account the customer base and region using the area analysis data.

  Patent Document 2 describes a system and apparatus for analyzing customer interest in unpurchased products. The system and apparatus described in Patent Document 2 captures the inside of a store and acquires customer information from the captured customer video. Then, a product group in which the customer is interested is identified from the customer video, and this is associated with the customer. As a result, it is possible to acquire information on a product that the customer has been interested in in the past, and to effectively promote the product. In Patent Document 2, as a trigger for specifying the degree of customer interest, a product near a point where the residence time is long, a product ahead of the line of sight, and the like are listed.

  Patent Document 3 describes a system for collecting information for estimating the cause of a customer's non-purchase of goods. The system described in Patent Literature 3 gives an IC (Integrated Circuit) chip to a product and acquires information on whether or not the customer has picked up the product. And, information on the movement route in the store, information on the residence time in each place is acquired, combining these information and the product arrangement information of the product display shelf, the product that the customer did not purchase is specified, Analyze why you did not buy. At this time, the movement of the product is determined based on a change in radio field intensity between a plurality of terminals installed in the store and the IC chip. As a result, when there is a product that has been picked up but has not been purchased, it is possible to make a hypothesis that it may have led to the purchase if the missing product in the product display shelf is replenished.

  Patent Document 4 describes a system and method for acquiring customer product selection behavior information from customer movement information and product movement information. The system described in Patent Literature 4 obtains customer movement information by obtaining a customer's in-store position from in-store image information. Moreover, the movement information of goods is acquired from the transmission / reception apparatus attached to goods. Then, finally, the product purchase information purchased by the customer is associated with these pieces of information, information on the product selected, examined, or purchased by the specific customer is calculated and acquired as the product selection behavior information of the customer. . Thereby, a customer's goods selection action information can be collected from customer movement information and goods movement information in a store.

JP 2009-151408 A JP 2009-003701 A JP 2004-348681 A Japanese Patent Laid-Open No. 11-175597

  A consumer psychological process called AIDA is known as a product purchasing process in marketing. AIDA represents an acronym for Attention (Interest), Interest (Interest), Desire (causes desire), and Action (causes purchase behavior), respectively. According to the AIDA model, it is considered that the order of A → I → D → A is followed as a psychological process in consumer purchase. Here, it is considered that the state transition of A → I in the AIDA model appears in the action of “focus on the product” in purchase. The state transition from I to D is considered to appear in the action of “getting a product”. The state transition of D → A is considered to appear in the action of “purchase a product”. Therefore, it is possible to easily and accurately acquire a customer's purchasing process consisting of three phases: “focus on product” → “get product” → “purchase product” for a product. If possible, the ability to analyze purchasing behavior can be dramatically improved. And it can benefit the distribution and retail industry.

  The acquisition of the three phases in the product purchase process can also be applied to measures against goods theft such as shoplifting and analysis. That is, if the user goes out of the store without “purchasing a product” while “getting the product”, it can be assumed that the product may have been stolen. For example, if information that is evidence of theft, such as information for obtaining a product, can be acquired, it is possible to respond with confidence to a customer who has been involved in the theft. Further, if the display information of the stolen product and the product focus information of the customer before and after the theft can be acquired, the store side can be used for analysis and response for theft prevention.

  However, the purchasing behavior analysis system provided to the distribution / retail industry, including the systems described in the above-mentioned documents, pays attention only to a part of the customer purchasing process consisting of these three phases. For this reason, when trying to track all three phases in these systems, the detection accuracy of the phase that was not noticed is insufficient, or between the sensors that differ for each store or product at a large expense in advance. There is a need to link information. Therefore, tracking all three phases in these systems is impractical and impractical. If only a part of the customer's purchasing process consisting of three phases can be obtained accurately, it is impossible to distinguish the state transition of the AIDA model that caused a product to not be sold to the customer. However, effective improvement cannot be made.

  Here, a specific problem in the case of tracking all three phases by the purchasing process analysis method described in each of the above-mentioned documents will be described.

  In the method described in Patent Document 1, “focus on the product” in the purchase process can be acquired, but since the selection of the product and purchase are linked, the phase “get the product” is acquired. I can't.

  The technique described in Patent Document 2 can acquire “focus on products” in the purchasing process. However, Patent Document 2 does not present a specific method for acquiring “getting a product” information. Further, Patent Document 2 suggests that customer information is specified by a video, but does not specifically present a photographing method capable of specifying customer information. In particular, in a technique for identifying customer information using video information, the detection accuracy deteriorates rapidly as the distance or angle increases. Furthermore, in order to understand "who" and "which product" from the video, for example, by associating the customer's location information in the video information with the information on the display location of the product, A model to associate is needed. However, Patent Document 2 does not present a specific model construction method. For example, when associating a person with a product based on location information, it is not possible to accurately convert the position in the video taken by the camera to the display location of the product in the actual 3D model of the store. It's not easy. In addition, each store and camera needs to be adjusted and calibrated, and the model needs to be rebuilt every time the product display or display shelf layout is changed. is there. For this reason, it is difficult for the method described in Patent Document 2 to track all three phases.

  The method described in Patent Document 3 is to tag a product and detect the product movement with a sensor, acquire information on “getting the product”, and then perform settlement at the time of movement in the store and at the cash register. When doing this, customer information is acquired from the IC card, and it is possible to associate “who” and “what product” in hand. However, this method has a big problem that it is not possible to accurately acquire information about “focus on products”. The accuracy of the position obtained by the method based on the communication between the IC card of the customer and the sensor is at most the horizontal position of the shelf and a range of several meters. On the other hand, on the shelf of the retail store, different products are displayed at intervals of several steps vertically and several to several tens of centimeters horizontally. Therefore, with this method, it is difficult to accurately grasp which customer is interested in which product, that is, which has led to Interest.

  Moreover, the method described in Patent Document 3 uses an IC card given to each customer as a method for acquiring customer information. In addition to the premise that the customer holds the IC card, the method using the IC card needs to prepare and issue the IC card for the customer. Therefore, when an IC card is used, the introduction of the system becomes large. Therefore, the technique described in Patent Document 3 is unrealistic in retail industries other than major supermarkets and large-scale chain stores.

  The method described in Patent Document 4 acquires the position of a customer from video information in the same manner as the method described in Patent Document 2 when the customer performs an action “get a product”. And the movement of goods is detected by the tag attached to goods similarly to the technique described in patent documents 3. Therefore, although the detection accuracy of the product movement is increased, in order to associate the customer with “who” and “which product” with the product, for example, in the video information, A model for associating people with products, such as associating location information of customers with information on the display position of products, is required.

  Therefore, the method described in Patent Document 4 is not easy to accurately convert to the display location of the product in the actual three-dimensional model of the store, similar to the method described in Patent Document 2. There are problems such as adjustment and calibration for each store and camera, and restructuring each time the display of products or the layout of display shelves is changed. Therefore, the method described in Patent Document 4 is time consuming and unrealistic. Further, Patent Document 4 does not mention “focus on products” in the purchasing process. For this reason, the method described in Patent Document 4 also cannot acquire all three phases easily and with high accuracy.

  As described above, according to the methods described in each of the above-mentioned documents, a customer's purchasing process consisting of three phases: “focus on a product” → “get a product” → “purchase a product” for a certain product. Cannot be obtained easily and with high accuracy.

  Accordingly, an object of the present invention is to provide a purchase information management system, a commodity movement detection device, and a purchase information management method that can easily and accurately acquire information necessary for a purchase process analysis.

  A purchase information management system according to the present invention includes a product movement detection device installed in the vicinity of a product, a sales management device, and a purchase information management device communicable with the product movement detection device and the sales management device. The device monitors the movement of the product via the product movement imaging unit, the movement detection sensor that detects whether or not the product has moved, and detects the movement of the product when the movement of the product is detected. Product movement detection unit that generates product movement information indicating that the product has moved, and product movement that acquires product movement person feature information that is characteristic information of a person who has moved the product based on the imaging information acquired by the product movement imaging unit The sales management device includes a person feature extraction unit, and purchases a product based on the purchaser imaging unit, a sales information acquisition unit that acquires product sales information, and the imaging information acquired by the purchaser imaging unit. Characteristic of person A product purchase person feature extraction unit that acquires product purchase person feature information as information, and the purchase information management device receives the product movement information and the product movement person feature information from the product movement detection device, and receives the sales information and the product purchase. A personal information is received from the sales management device, and a purchase information management unit that stores each received information as a purchase information in association with the transmission source device is provided.

  The product movement detection device according to the present invention monitors the movement of a product via a product movement imaging unit, a movement detection sensor that detects whether or not the product has moved, and detects the movement of the product when the movement detection is detected. A product movement detection unit that generates product movement information indicating that the movement of the product has been detected, and a product movement person feature that is characteristic information of a person who has moved the product based on the imaging information acquired by the product movement imaging unit A product moving person feature extracting unit for acquiring information is provided.

  A purchase information management method according to the present invention includes a purchase information management system including a product movement detection device installed in the vicinity of a product, a sales management device, and a purchase information management device communicable with the product movement detection device and the sales management device. In the purchasing information management method in the above, the product movement detection device provided with the product movement imaging unit monitors the movement of the product via the movement detection sensor that detects the presence or absence of the movement of the product, and detects the movement of the product. Generates product movement information indicating that the movement of the product has been detected, and acquires product moving person feature information, which is characteristic information of the person who has moved the product, based on the imaging information acquired by the product movement imaging unit. Then, the sales management device including the purchaser imaging unit acquires the sales information of the product, and based on the imaging information acquired by the purchaser imaging unit, the product purchase person feature that is the feature information of the person who purchased the product information The purchase information management device receives the product movement information and the product movement person feature information from the product movement detection device, receives the sales information and the product purchase person feature information from the sales management device, and transmits each received information It is characterized in that it is stored as purchase information in association with the device.

  According to the present invention, information necessary for purchasing process analysis can be acquired easily and with high accuracy.

It is a block diagram which shows the structure of the purchase information management system in the 1st Embodiment of this invention. It is a block diagram which shows the structure of the goods movement detection apparatus in 1st Embodiment. It is a block diagram which shows the structure of the sales management apparatus in 1st Embodiment. FIG. 2 is a block diagram showing a configuration of a purchase information management device in the first embodiment. It is explanatory drawing which shows the mode of arrangement | positioning of the goods movement detection apparatus in 1st Embodiment. It is a flowchart which shows operation | movement of a goods movement detection part. It is a flowchart which shows operation | movement of a sales information acquisition part. It is a flowchart which shows operation | movement of a person feature extraction part. 10 is a flowchart illustrating an example of an operation when the purchase information management unit receives information from the commodity movement detection device 10 or the sales management device 20. It is explanatory drawing which shows an example of the purchase information stored in the database. It is explanatory drawing which shows an example of the statistical information of the purchase action acquired from the purchase information which the purchase information management apparatus in 1st Embodiment hold | maintains. It is a block diagram which shows the structure of the purchase information management system in the 2nd Embodiment of this invention. It is explanatory drawing which shows the relationship between the tracking range of the goods tracking device in 2nd Embodiment, and the detection range of a goods movement detection apparatus. It is a block diagram which shows the structure of the goods tracking device in 2nd Embodiment. It is a flowchart which shows operation | movement of a goods tracking part. It is a flowchart which shows operation | movement of the purchase information management part in 2nd Embodiment. It is explanatory drawing which shows an example of arrangement | positioning when two or more goods tracking apparatuses are installed. It is a block diagram which shows an example of a structure of a goods movement detection apparatus provided with a goods movement detection part and a goods tracking part. It is explanatory drawing which shows an example of arrangement | positioning of a goods movement detection apparatus provided with a goods movement detection part and a goods tracking part. It is a block diagram which shows the structure of the goods movement detection apparatus in the 3rd Embodiment of this invention. It is a flowchart which shows operation | movement of a gaze detection part. It is explanatory drawing which shows an example of the statistical information acquired from the purchase information which the purchase information management apparatus in 3rd Embodiment hold | maintains. It is explanatory drawing which shows a mode that the place where a goods movement detection apparatus and goods are arrange | positioned was seen from upper direction. It is explanatory drawing which shows an example of the image which the imaging part of the goods movement detection apparatus imaged. It is a flowchart which shows operation | movement of the purchase information management part in 4th Embodiment. It is a block diagram which shows the minimum structure of the purchasing information management system by this invention. It is a block diagram which shows the other minimum structure of the purchasing information management system by this invention.

Embodiment 1. FIG.
A first embodiment of the present invention will be described below with reference to the drawings.

  FIG. 1 is a block diagram showing a configuration of a purchase information management system according to the first embodiment of the present invention. As shown in FIG. 1, the purchase information management system 1 includes a commodity movement detection device 10, a sales management device 20, and a purchase information management device 30. Although one merchandise movement detection device is illustrated in FIG. 1, any number of merchandise movement detection devices may be included in the purchase information management system 1. Further, FIG. 1 illustrates one sales management apparatus, but any number of sales management apparatuses may be included in the purchase information management system 1. In the present embodiment, the commodity movement detection device 10 is installed in the vicinity of the commodity 2. Although two products are illustrated in FIG. 1, any number of products may be installed.

  FIG. 2 is a block diagram illustrating a configuration of the commodity movement detection apparatus 10 according to the first embodiment. As illustrated in FIG. 2, the product movement detection device 10 includes a sensor 11, a product movement detection unit 12, an imaging unit 13-1, a person feature extraction unit 14-1, and a communication unit 15-1.

  The sensor 11 converts a physical phenomenon generated in the outside world into an electric signal. The sensor 11 transmits the converted electrical signal to the commodity movement detection unit 12. In the present embodiment, the sensor 11 collects information for detecting the movement of the product from the outside world. Specific examples of the sensor include a vibration sensor, an illuminance sensor, a sound sensor such as a microphone, a pressure sensor, and an imaging device such as a camera, but are not limited thereto. The sensor 11 may be configured by combining a plurality of sensors.

  The commodity movement detection unit 12 is installed in the vicinity of the equipment and the commodity 2, and detects commodity movement based on the electrical signal received from the sensor 11. When the product movement detection unit 12 detects the product movement, the product movement detection unit 12 transmits the product movement information to the purchase information management unit 31 via the communication unit 15-1. The product movement information is information indicating that the movement of the product has been detected. Equipment refers to a facility or a device or apparatus provided in the facility. Specific examples of “in the vicinity of equipment and products” include, but are not limited to, shelves and doors of retail stores.

  The imaging unit 13-1 converts external information into an image signal via a lens and an imaging element. Hereinafter, the information converted into the image signal is referred to as imaging information. The imaging unit 13-1 transmits the converted image signal to the person feature extraction unit 14-1. A specific example of the imaging unit 13-1 is a camera.

  The person feature extraction unit 14-1 acquires person feature information in the imaging range of the imaging unit 13-1 from the image signal received from the imaging unit 13-1. Person characteristic information refers to information in which differences from other persons are recognized. Specific examples of the person characteristic information include face information, clothes color, shape, and physique. When the person feature extraction unit 14-1 acquires the person feature information, the person feature extraction unit 14-1 transmits the person feature information to the purchase information management unit 31 via the communication unit 15-1.

  The communication unit 15-1 transmits / receives information to / from an external device via a network. Specific examples of the network include, but are not limited to, Ethernet (registered trademark), wireless LAN (Local Access Network), Bluetooth (registered trademark), ZigBee (registered trademark), and specific low power wireless. The communication unit 15-1 may use other communication means as long as information can be transmitted and received with other devices.

  The product movement detection unit 12, the person feature extraction unit 14-1, and the communication unit 15-1 are realized by a CPU or the like included in the product movement detection device 10.

  FIG. 3 is a block diagram illustrating a configuration of the sales management apparatus 20 according to the first embodiment. As illustrated in FIG. 3, the sales management device 20 includes a sales information acquisition unit 21, an imaging unit 13-2, a person feature extraction unit 14-2, and a communication unit 15-2. The imaging unit 13-2, the person feature extraction unit 14-2, and the communication unit 15-2 are respectively an imaging unit 13-1, a person feature extraction unit 14-1, and a communication unit 15- included in the commodity movement detection device 10. 1 is performed.

  The sales information acquisition unit 21 acquires the sales information of the product 2. Specific examples of the sales information include information for identifying a product, time information when the product is sold, and the like. A specific example of the sales information acquisition unit 21 is a POS cash register. When the product is sold, that is, when the sales information is acquired, the sales information acquisition unit 21 transmits the sales information to the purchase information management unit 31 via the communication unit 15-2.

  Note that the sales information acquisition unit 21, the person feature extraction unit 14-2, and the communication unit 15-2 are realized by a CPU or the like included in the sales management device 20.

  FIG. 4 is a block diagram showing the configuration of the purchase information management apparatus 30 in the first embodiment. As shown in FIG. 4, the purchase information management apparatus 30 includes a purchase information management unit 31, a database 32, and a communication unit 15-3. Note that the communication unit 15-3 performs the same processing as the communication unit 15-1 included in the commodity movement detection device 10.

  The purchase information management unit 31 stores content information acquired from the product movement detection unit 12, the sales information acquisition unit 21, and the person feature extraction units 14-1 and 14-2 via the communication unit 15 in the database 32. Content information refers to product movement information when the transmission source is a commodity movement detection unit, sales information when the transmission source is a sales information acquisition unit, and person when the transmission source is a person feature extraction unit. Refers to feature information.

  The purchase information management unit 31 stores the received content information in the database 32 as purchase information in association with the transmission source device. In the example illustrated in FIG. 1, the content information of the product movement detection unit 12 and the person feature extraction unit 14-1 is transmitted from the product movement detection device 10. The contents information of the sales information acquisition unit 21 and the person feature extraction unit 14-2 is transmitted from the sales management device 20. The purchase information management unit 31 stores the received content information in the database 32 in association with the transmission source device.

  The purchase information management unit 31 and the communication unit 15-3 are realized by a CPU or the like provided in the purchase information management device 30. The database 32 is realized by a storage device such as an optical disk device, a magnetic disk device, or a memory included in the purchase information management device 30.

  FIG. 5 is an explanatory diagram illustrating an example of an arrangement of the commodity movement detection device according to the first embodiment. As shown in FIG. 5, a commodity movement detection device 10 including a sensor 11 and an imaging unit 13-1 is installed on a commodity display shelf 3 that is equipment. At this time, the commodity movement detection device 10 is installed in the vicinity of the commodity 2. In addition, although the three goods 2 are illustrated in FIG. 5, how many goods may be installed. The sensor 11 detects a change in the state of the commodity display shelf 3 accompanying the movement of the commodity 2. Specific examples of changes in the state of the merchandise display shelf 3 include the generation of sound and vibration, and the change in illuminance accompanying a decrease in merchandise. The imaging unit 13-1 is installed so as to face the wall of the commodity display shelf 3, that is, so that the face of the consumer 4 is reflected in the imaging range 5.

  By arranging the product movement detection device 10 as shown in FIG. 5, the person feature extraction unit 14-1 can accurately acquire the face information of the consumer 4 in the vicinity of the product 2. If the consumer 4 is interested in the product 2, the consumer 4 faces the product 2, and as a result, it is expected that the image appears well within the imaging range of the imaging unit 13-1.

  In this way, the sensor 11, the product movement detection unit 12, the imaging unit 13-1, and the person feature extraction unit 14-1 are included in the same device (the product movement detection device 10) and are arranged in the vicinity of the product 2. . Thereby, the goods movement detection apparatus 10 can acquire the information regarding the trend of the goods 2 without constructing a video structure model. When photographing the product 2 from a remote location, it is necessary to model the relationship between the product arrangement and the building structure in advance. However, by arranging the product movement detection device 10 in the vicinity of the product 2, the acquisition information of each part of the product movement detection device 10 can be regarded as customer behavior information regarding the product 2 as it is.

  Next, the operation of this embodiment will be described.

  FIG. 6 is a flowchart illustrating the operation of the commodity movement detection unit 12. As shown in FIG. 6, when the commodity movement detection unit 12 is activated, the commodity movement detection process is started using the sensor 11 (step P111).

  Examples of the method for detecting the movement of the product by the sensor 11 include a method of detecting vibration generated when the product is picked up by placing it on the same shelf as the product if the sensor 11 is a vibration sensor. Moreover, if the sensor 11 is a sound sensor, the method of installing in the same shelf as goods and detecting the sound which rubs the packaging of goods is mentioned. However, since the commodity movement detection method is determined based on the type of sensor, other methods may be used. Moreover, the goods movement detection part 12 may determine compositely the movement detection of goods from multiple and time-series sensing information.

  When the product movement detection unit 12 detects the product movement (Yes in Step P111), the product movement detection unit 12 generates ID (IdentificationICation) information (detection ID) for identifying the product movement information indicating the product movement (Step P112). ), The product movement information and the detection ID are transmitted to the purchase information management unit 31 (step P113). As an example of information included in the detection ID, in addition to the detection time, information that can uniquely identify the product movement information and information for identifying the product movement detection device 10 may be mentioned. Specific examples of information for identifying the commodity movement detection device 10 include an identification ID assigned in advance to the commodity movement detection device 10, a serial number, a MAC (Media Access Control) address used for communication, IP ( Internet Protocol) address.

  The commodity movement detection unit 12 transmits the commodity movement information and the detection ID, and then transitions again to the commodity movement detection process (the process of Step P111).

  FIG. 7 is a flowchart showing the operation of the sales information acquisition unit 21.

  As shown in FIG. 7, when the sales information acquisition unit 21 is activated, it starts a sales information acquisition process (step P121). When the sales information acquisition unit 21 acquires the sales information, it generates ID information (sales ID) for identifying the sales information (step P122), and transmits the sales information and the sales ID to the purchase information management unit 31 ( Step P123). The sales ID includes the same information as the above-described detection ID.

  After transmitting the sales information and the sales ID, the sales information acquisition unit 21 transitions again to the sales information acquisition process (the process of step P121).

  For example, if the sales information acquisition unit 21 is a POS register, an ID assigned to each product sales sequence in the sales information acquisition process (the process of step P121) by the product sales sequence on the POS register. The generation of information is a sales ID generation process (the process of step P122), and the assignment of ID information and the transmission of merchandise sales sequence information to the store computer are the sales information transmission process to the purchase information management unit 31 (the process of step P123). Respectively.

  FIG. 8 is a flowchart showing the operation of the person feature extraction unit. Here, the operation of the person feature extraction unit 14-1 will be described.

  As illustrated in FIG. 8, when the person feature extraction unit 14-1 is activated, the person feature extraction unit 14-1 performs person feature extraction processing based on the shooting information acquired by the imaging unit 13-1. Specifically, it is confirmed whether or not person feature information can be extracted within the imaging range of the imaging unit 13-1 (step P131). If person feature information is extracted as a result of the person feature extraction process, the person feature extraction unit 14-1 generates ID information (extraction ID) for identifying the person feature information (step P132). Then, the person feature extraction unit 14-1 transmits the person feature information and the extraction ID to the purchase information management unit 31 (step P133). The extraction ID includes the same information as the detection ID described above.

  After transmitting the person feature information and the extraction ID, the person feature extraction unit 14-1 transitions again to the person feature extraction process (the process of step P131).

  In this way, the person feature extracting unit 14-1 acquires the person feature information (product moving person feature information) of the consumer who has moved the product, that is, the consumer who has picked up the product, and purchase information management unit 31. Similarly, the person feature extraction unit 14-2 executes the processes of steps P 131 to P 133, acquires the person feature information (product purchase person feature information) of the consumer who purchased the product, and transmits it to the purchase information management unit 31. The imaging unit 13-2 is installed so that the face of the consumer who purchased the product is reflected in the imaging range.

  In this embodiment, an imaging unit is installed so that a person is in the vicinity of the imaging range and within the line of sight, and the person feature extraction unit extracts the person feature information. The person feature extraction processing of the person feature extraction unit in this case is well known as face recognition and feature point extraction processing in the image processing technology, and thus description thereof is omitted.

  FIG. 9 is a flowchart illustrating an example of an operation when the purchase information management unit 31 receives information from the commodity movement detection apparatus 10 or the sales management apparatus 20.

  As shown in FIG. 9, when activated, the purchase information management unit 31 starts an information reception process (step P141). When the information is received (Yes in Step P141), the ID information and the content information are acquired from the received information (Step P142). Then, the purchase information management unit 31 specifies the transmission source based on the information (hereinafter referred to as transmission source device information) for identifying the transmission source device included in the ID information (step P143). Then, the purchase information management unit 31 stores the content information in the database 32 in association with the transmission source device (step P144). After executing the process of step P144, the purchase information management unit 31 transitions again to the information reception process (process of step P141).

  FIG. 10 is an explanatory diagram showing an example of purchase information stored in the database 32. As illustrated in FIG. 10, the purchase information management unit 31 stores the information received from each unit in the database 32 in association with the commodity movement detection device 10 and the sales management information 20 based on the ID information included in the reception information. . In addition, the purchase information management unit 31 acquires the acquisition time of the information included in the content information or ID information, and stores this in the database 32 in association with the content information.

  As described above, in the present embodiment, the person feature extraction unit 14 is provided in the same device as the product movement detection unit 12 or the sales information acquisition unit 21, and information from the same device is further associated. Therefore, it is possible to simultaneously acquire information on both the product and the consumer who has picked it up, and it is possible to easily associate the purchase process with the person identification. For example, a purchase process for each customer can be easily obtained from the purchase information shown in FIG. 10 using time information and person characteristic information as search keys. As an example, information such as “a male customer wearing a long green sleeve picked up a product“ 490177777 ### ”and purchased it as it is” can be acquired. Therefore, according to the present embodiment, it is possible to acquire all three-phase information necessary for the purchase process analysis with a less troublesome and general-purpose method.

  In the present embodiment, the commodity movement detection device is installed in the vicinity of the commodity to be detected. Accordingly, it is possible to face a consumer who is interested in the product in the vicinity and in the front or the direction close to the front. Therefore, the person information can be acquired with high accuracy without performing a strict and complicated adjustment such as calibration only by placing the product movement detection device in the vicinity of the product.

  In addition, statistical information on purchasing behavior necessary for purchasing process analysis can be acquired from the purchasing information. Specifically, the purchasing information management unit 31 selects one or more pieces of content information based on at least one piece of information indicating the acquired content information and the acquisition time of each piece of content information. Is extracted and processed to obtain statistical information.

  FIG. 11 is an explanatory diagram illustrating an example of purchase behavior statistical information acquired from purchase information held by the purchase information management apparatus 30 according to the first embodiment. As shown in FIG. 11, by classifying the information stored in the database 32 by date, product unit, and gender, the number of product movement detections, that is, the number of times the consumer picked up the product and the number of product sales. That is, it is possible to acquire the number of the products purchased by the consumer. Note that the purchase information management unit 31 may output statistical information to, for example, a display device (POS cash register screen) included in the sales management device 20. According to such a form, statistical information can be presented to a manager of a purchase information management system, a store manager or a store clerk of a retail store, and the like.

  In the present embodiment, the product movement detection unit 12 or the sales information acquisition unit 21 and the person feature extraction unit 14 operate independently, but may be configured to perform a cooperative operation through a trigger. For example, when the product movement detection unit 12 or the sales information acquisition unit 21 acquires information, a trigger is transmitted to the person feature extraction unit in the same device. The person feature extraction unit extracts person feature information before and after the trigger reception timing.

  Accordingly, the person feature extraction unit can transmit only information associated with the purchase process information to the information management apparatus 30. Therefore, the communication load in the commodity movement detection device 10 and the sales management device 20 can be reduced. Moreover, the communication load in the purchase information management part 31 and the processing load accompanying information reception and analysis can be reduced. At this time, the person feature extraction unit may extract the person feature information only when the trigger is received. According to such a form, the person feature extraction process can be started only when the purchase process information is generated, and the load of the person feature extraction process can be reduced.

  Further, the sensor 11 may have a communication function with the IC tag, and the commodity movement detection unit 12 may detect commodity movement based on communication between the sensor 11 and the IC tag attached to the commodity 2. An example of a method for detecting product movement includes a determination method based on whether communication with an IC tag is possible, a determination method based on communication power, and the like, but is not limited thereto. For example, when the product movement detection unit 12 becomes unable to communicate with a specific IC tag or when the communication power becomes lower than a threshold value, the product movement detection unit 12 detects that the product has been moved and detects the product movement.

  According to such a configuration, it is possible to detect product movement with higher accuracy. Since whether or not to communicate with the IC tag and the communication power are purely based on the distance between the product 2 and the sensor 11, compared to other sensing information, it is difficult for false detection and oversight based on factors other than product movement. Therefore, the product movement can be detected with higher accuracy.

  Moreover, according to such a structure, the information which returned the goods to the shelf can be acquired. This is because if the product is returned to the shelf, the distance between the sensor 11 and the IC tag attached to the product 2 is reduced again. At this time, the merchandise movement detection unit 12 detects the timing when the merchandise is returned to the shelf, the person feature extraction unit 14-1 extracts the person feature information at the timing, and the purchase information management unit 31 performs the timing. The person feature information may be managed in association with the product detection device. Thereby, more detailed product movement information can be acquired.

Embodiment 2. FIG.
Hereinafter, a second embodiment of the present invention will be described with reference to the drawings.

  FIG. 12 is a block diagram showing the configuration of the purchase information management system in the second embodiment of the present invention. Here, a case where an IC tag is attached to a product is taken as an example.

  As illustrated in FIG. 12, the purchase information management system 1 a according to the second embodiment includes a product movement detection unit device 10 a, a sales management device 20, a product tracking device 50, and a purchase information management device 30. In the present embodiment, the IC tag 40 is attached to the product 2a.

  The configuration of the commodity movement detection device 10a is the same as the structure of the commodity movement detection device 10 in the first embodiment. However, the commodity movement detection device 10a has a function of detecting commodity movement based on communication with the IC tag 40 attached to the commodity 2a. Specifically, the sensor included in the commodity movement detection device 10a has a communication function with the IC tag 40, and the commodity movement detection unit included in the commodity movement detection device 10a is based on communication with the IC tag 40 attached to the commodity 2a. Detect product movement.

  The product tracking device 50 tracks whether the product stays in a facility such as a retail store for the product whose movement is detected by the product movement detection unit. The product tracking device 50 performs tracking using communication with the IC tag 40. At this time, the product tracking range in the product tracking device 50 is set wider than the detection range in the product movement detection unit of the product movement detection device 10a.

  FIG. 13 is an explanatory diagram showing the relationship between the tracking range of the product tracking device 50 and the detection range of the product movement detection device 10a in the second embodiment. As shown in FIG. 13, the tracking range of the product tracking device 50, that is, the detection range 6-2 of the sensor provided in the product tracking device 50 is wider than the detection range 6-1 of the sensor provided in the product movement detection device 10a. Is set to be Thereby, the goods tracking device 50 can discriminate | determine whether the goods after the movement detection by the goods movement detection apparatus 10a exist in the wide detection range 6-2. In addition, although two goods 2a-1 and 2a-2 are illustrated in FIG. 13, how many goods may exist in a detection range.

  FIG. 14 is a block diagram illustrating a configuration of the commodity tracking device 50 according to the second embodiment. The product tracking device 50 includes a sensor 51, a product tracking unit 52, and a communication unit 15-5.

  The sensor 51 performs communication processing with the IC tag 40 included in the product 2a.

  Based on communication between the sensor 51 and the IC tag 40, the merchandise tracking unit 52 determines whether the merchandise whose movement is detected by the merchandise movement detection apparatus 10a remains within the detection range 6-2. Information about the product to be detected is acquired from the purchase information management apparatus 30 via the communication unit 15-5. In addition, the product tracking unit 52 transmits the detection result to the purchase information management apparatus 30 via the communication unit 15-5.

  The communication unit 15-5 performs the same processing as the communication unit 15-1 included in the commodity movement detection device 10.

  The product tracking unit 52 and the communication unit 15-5 are realized by a CPU or the like provided in the product tracking device 50.

  Next, the operation of this embodiment will be described.

  FIG. 15 is a flowchart showing the operation of the product tracking unit 52.

  As shown in FIG. 15, when the product tracking unit 52 is activated, information on a product that is being moved, that is, a product that has been detected but has not been acquired as sales information from the purchase information management unit 31a (hereinafter referred to as “moving”). Product information) is acquired (step P151). An IC tag is attached to each product. For example, as shown in FIG. 13, IC tags 40-1 and 40-2 are assigned to the products 2a-1 and 2a-2, respectively. Therefore, the product tracking unit 52 can track individual products.

  Upon acquiring the moving product information, the product tracking unit 52 determines whether the moving product is within the detection range 6-2 of the product tracking device 50 based on the acquired moving product information (step P152). ).

  And when all the goods in movement are detected within detection range 6-2 (Yes in Step P153), goods tracking device 50 changes to the state immediately after starting again. In addition, when all the moving products are not detected within the detection range 6-2 (No in Step P153), the product tracking device 50 includes information about the products that could not be detected among the moving products (hereinafter referred to as “following”). , Referred to as undetected moving product information) is notified to the purchase information management unit 31a as a determination result. After the notification, the merchandise tracking apparatus 50 transitions again to the state immediately after activation. In any case, the merchandise tracking apparatus 50 may be changed to a state immediately after activation after a lapse of a certain time after the process of Step P153 is completed.

  FIG. 16 is a flowchart showing the operation of the purchase information management unit 31 in the second embodiment.

  As illustrated in FIG. 16, when the purchase information management unit 31 receives the information (Yes in Step P161), the purchase information management unit 31 confirms whether the information transmission source is the product tracking device 50 (Step P162). When the transmission source is not the commodity tracking device 50 (No in Step P162), the purchase information management unit 31 performs the same processing as Steps P142 to P144 in the first embodiment (Steps P168 to P170).

  When the transmission source is the product tracking device 50 (Yes in Step P162), the purchase information management unit 31 confirms the notification content (Step P163).

  When the notification content is a request for moving product information, the purchase information management unit 31 detects the moving product, that is, the product movement, from the information group stored in the database 32, but the sales information is acquired. Information on the product that has not been acquired is acquired (step P164). Then, the purchase information management unit 31 returns the acquired information (moving product information) to the product tracking device 50 (step P165). After replying, the purchase information management unit 31 transitions again to the information reception process in step P161.

  When the notification content is undetected moving product information, the purchase information management unit 31 acquires the person characteristic information extracted at the time of the latest product movement detection for the product indicated by the undetected moving product information. (Step P166). And the purchase information management part 31 outputs an alert with the acquired person characteristic information (step P167). For example, the purchase information management unit 31 transmits an alert indicating that the product is lost or stolen together with the person characteristic information. As a specific example of the method of sending the alert, the purchase information management system administrator, the store manager or the store clerk, e-mail, POS cash register screen, sales information management system console screen, warning sound, Or a method of transmitting through a plurality of methods. In addition, after sending the alert, the purchase information management unit 31 transitions again to the information reception process of Step P161.

  As described above, according to the present embodiment, it is possible to acquire state information that appears to be abnormal in terms of a product purchase process, such as loss or theft of a product. Further, when the state information is acquired, it is possible to acquire characteristic information of a person involved in the product.

  As an example of a specific use case of the present embodiment, acquisition of related person information in merchandise theft at a retail store can be cited. For example, when the tracking range of the product tracking device 50 is set to be within a retail store, if a moving product is not detected, the possibility that the product has moved out of the tracking range, that is, out of the store, is given to the product. There is a possibility that communication with the IC tag was hindered. The former can be judged as theft of goods. In the latter case, it is considered rare that communication of unpurchased products is interrupted in normal purchase behavior, so that it can be determined as an abnormal state. On the other hand, the person feature information is acquired and held at the timing when the product movement is detected in advance by the product movement detection device 10 for all the products that are the tracking targets of the product tracking device 50. Therefore, at the same time that the product status is acquired in real time, if the purchase information management apparatus 30 determines that the theft or abnormality has occurred, the feature information of the person who has moved the product can be acquired.

  In addition, the product movement detection device 10 holds the video information acquired by the imaging unit 13-1, for example, image data such as a photograph of a person who has moved the product, as person feature information at the time of product movement detection. It may be. According to such a form, the goods movement detection apparatus 10 can be utilized as an incident recorder.

  Alternatively, a plurality of product tracking devices 50 may be installed and each detection range 6-2 may be narrowed. FIG. 17 is an explanatory diagram showing an example of an arrangement when a plurality of product tracking devices 50 are installed. According to the configuration as shown in FIG. 17, it is possible to acquire the position information of a product by determining which product tracking device 50 has detected the product. Therefore, it is possible to acquire the flow line of product movement.

  Moreover, you may make it mount the goods movement detection part 12 and the goods tracking part 52 in a single apparatus. In that case, what is necessary is just to distinguish the detection range of the goods movement detection part 12, and the tracking range of the goods tracking part 52.

  FIG. 18 is a block diagram illustrating an example of the configuration of a product movement detection apparatus 10a 'including a product movement detection unit and a product tracking unit. The sensor information requested by the product movement detection unit 12 and the sensor information requested by the product tracking unit 52 are caused only by a difference in detection range. Therefore, as in the example shown in FIG. 18, the product movement detection unit 12 and the product tracking unit 52 can share sensor information (information from the sensor 11).

  FIG. 19 is an explanatory diagram illustrating an example of the arrangement of the commodity movement detection device 10a ′ including the commodity movement detection unit and the commodity tracking unit. As shown in FIG. 19, the detection range 6-1 of the product movement detection unit 12 is set to the shelf on which the product 2a is displayed and the vicinity thereof, and the detection range 6-2 of the product tracking unit 52 is set to a wide range. Accordingly, it is possible to sequentially acquire which product movement detection device 10a 'is in the vicinity of the product whose movement is detected.

  In the example shown in FIG. 18, the configuration in which the sensor 11 is shared by the product movement detection unit 12 and the product tracking unit 52 is shown, but a configuration may be provided in which each sensor is provided. In addition, the product movement detection unit 12 may detect a product movement based on a method other than communication with the IC tag 40 using a sensor of a different type from the product tracking unit 52.

Embodiment 3. FIG.
Hereinafter, a third embodiment of the present invention will be described with reference to the drawings.

  FIG. 20 is a block diagram showing the configuration of the commodity movement detection apparatus 10b according to the third embodiment of the present invention. The commodity movement detection apparatus 10b in the present embodiment can acquire line-of-sight information on the commodity.

  As illustrated in FIG. 20, the commodity movement detection device 10 b according to the third embodiment includes a sensor 11, a commodity movement detection unit 12, an imaging unit 13-1, a person feature extraction unit 14-1, and a communication unit 15. -1, an infrared light emitting unit 16, an infrared light receiving unit 17, and a line-of-sight detection unit 18.

  The infrared light emitting unit 16 and the infrared light receiving unit 17 are used to acquire eyeball position information for eye gaze detection.

  The line-of-sight detection unit 18 irradiates the eyeball with infrared rays from the infrared light emitting unit 16, receives light reflected from the eyeball at the infrared light receiving unit 17, and is caused by a difference in reflectance between the cornea portion and the sclera portion of the eyeball. The direction of the line of sight is detected using the intensity of the reflected light.

  Note that the line-of-sight detection unit 18 is realized by a CPU or the like included in the commodity movement detection device 10b.

  Next, the operation of this embodiment will be described.

  FIG. 21 is a flowchart showing the operation of the line-of-sight detection unit 18. As shown in FIG. 21, when activated, the line-of-sight detection unit 18 emits infrared light to the human eyeball through the infrared light emitting unit 16 (step P171). Subsequently, the infrared light receiving unit 17 receives infrared light. Specifically, the reflected light from the eyeball of the person is received (step P172). The line-of-sight detection unit 18 performs line-of-sight detection processing based on the eyeball position information received by the infrared light receiving unit 17 (step P173). The detailed operation of the line-of-sight detection process is well known to those skilled in the art, and thus description thereof is omitted.

  When detecting the line of sight (Yes in Step P173), the line-of-sight detection unit 18 acquires information indicating the direction of the detected line of sight (hereinafter referred to as line-of-sight destination information) (Step P174). Then, the line-of-sight detection unit 18 generates ID information (line-of-sight detection ID) (step P175).

  The line-of-sight detection unit 18 transmits the line-of-sight detection ID together with the line-of-sight information to the purchase information management apparatus 30 via the communication unit 15-1 (step P176). The line-of-sight detection ID includes the same information as the above-described detection ID.

  As described above, according to the present embodiment, line-of-sight destination information can be acquired, and more detailed purchase process information can be used. The line-of-sight destination information is an index indicating attention to the product, and corresponds to the first stage of the purchase process, that is, the transition behavior from Attention (with attention) to Interest (with interest) in the AIDA model. Therefore, the purchase process can be analyzed in more detail by acquiring the line-of-sight information in addition to the sales information and the product movement information.

  FIG. 22 is an explanatory diagram illustrating an example of statistical information acquired from purchase information held by the purchase information management apparatus 30 according to the third embodiment. As shown in FIG. 22, the holding information is classified by date / product and sex, and the number of line-of-sight detections, the number of product movements detected, and the number of products sold are acquired.

  For example, when comparing the product “490177721 ###” and “454789441 ####” shown in FIG. 22, “490177721 ###” has a relatively high rate of line-of-sight detection, that is, attention to sales. It can be seen that the purchase ratio of men is particularly high. In addition, since the number of male eyes detected is not so large, there may be room for improvement in the product display location. On the other hand, although “45478441 ###” has a very large number of line-of-sight detections, it can be seen that the ratio of picking up merchandise is small and the ratio leading to sales is very small. From this, it is possible that the product does not meet customer needs, although it works properly from the viewpoint of the exterior and display location of the product. Or the first impression on the product may be negative.

  Thus, by acquiring purchase process information in detail, it becomes possible to perform hypothesis planning and verification result acquisition for determining product display, assortment, and ordering.

  In the present embodiment, the configuration and method using infrared light in visual line detection have been described, but visual line detection may be performed using other configurations and methods. As an example of another method, there is a method of detecting a line of sight based on the orientation of the face and the position of the black eye in the eyeball from the imaging information of the face and the eyeball. In this case, the line-of-sight detection unit 18 performs line-of-sight detection based on video information from the imaging unit 13-1, using the imaging unit 13-1 as an alternative to the infrared light emitting unit 16 and the infrared light receiving unit 17. You can do it.

  Moreover, when the line-of-sight detection unit 18 detects the line of sight, the line-of-sight dwell time may be acquired as line-of-sight destination information. The dwell time of the line of sight is information suitable for analyzing the degree of attention to the product. Therefore, more detailed purchase process information can be acquired and used.

  Further, the product movement detection device 10b may further include an advertisement display unit (not shown) such as digital signage, and may discriminate and acquire the degree of attention to the product and the advertisement based on the line-of-sight information. According to such a configuration, it is possible to easily analyze the degree of attention of advertisement display and the effect of advertisement display on purchasing process behavior. For example, when a consumer's line of sight faces the advertisement display unit, it can be recognized that the consumer is paying attention to the advertisement. After that, when the consumer's line of sight moves to the product advertised in the advertisement, it is possible to recognize that the purchase process of the product has advanced with the advertisement as a trigger.

  Further, according to such a configuration, since the commodity movement detection device itself includes the advertisement display unit, it is not necessary to perform a calibration operation such as associating the line-of-sight destination information with the advertisement display. Therefore, it is possible to easily analyze the degree of attention of advertisement display, including the effect of advertisement, and the effect of advertisement display on purchasing process behavior simply by installing the product movement detection device.

Embodiment 4 FIG.
Hereinafter, a fourth embodiment of the present invention will be described with reference to the drawings.

  Here, a case where the line-of-sight destination information faces the outside of the product movement detection device in an environment where the product movement detection device according to the third embodiment is adjacent is taken as an example.

  FIG. 23 is an explanatory diagram showing a state where the product movement detection device and the place where the product is arranged are viewed from above. The configuration of the merchandise movement detection devices 10c-1 and 10c-2 is the same as that of the merchandise movement detection device 10b in the third embodiment, and a description thereof will be omitted.

  As shown in FIG. 23, the commodity movement detection devices 10 c-1 and 10 c-2 are installed adjacent to the commodity display shelf 3. The product movement detection devices 10c-1 and 10c-2 detect the product movement of the products 2c-1 and 2c-2, respectively. In addition, the commodity movement detection devices 10c-1 and 10c-2 perform person feature extraction and line-of-sight detection for the imaging ranges 5c-1 and 5c-2, respectively.

  Here, the operation of the commodity movement detection device when the consumer 4 shown in FIG. 23 turns the line of sight 7 toward the commodity 2c-1 will be described. FIG. 24 is an explanatory diagram illustrating an example of an image captured by the imaging unit of the commodity movement detection device 10c-2.

  Since the consumer 4 is outside the imaging range 5c-1, the merchandise movement detection device 10c-1 cannot acquire the information of the consumer 4. On the other hand, since the consumer 4 is inside the imaging range 5c-2, the merchandise movement detection device 10c-2 can acquire the personal feature information and the line-of-sight information of the consumer 4, as shown in FIG. However, since the line of sight of the consumer 4 is facing outward, the merchandise movement detection device 10c-2 detects that the consumer 4 is interested in the merchandise grasped by the merchandise movement detection device 10c-2, that is, the merchandise movement detection unit. It is determined that the product 2c-2 is not in the range. For this reason, the purchase information management system cannot acquire purchase process information for the product 2c-1 of the consumer 4.

  Therefore, in this embodiment, the purchase information management unit 31 of the purchase information management device 30 integrates information on adjacent product movement detection devices. FIG. 25 is a flowchart showing the operation of the purchase information management unit 31 in the fourth embodiment. In addition, the purchase information management part 31 shall grasp | ascertain in advance the relative positional relationship of each goods movement detection apparatus. Specifically, information indicating the relative positional relationship between the commodity movement detection devices is stored in advance in a storage unit (not shown) provided in the purchase information management device 30 or the like.

  As illustrated in FIG. 25, when the purchase information management unit 31 receives information (step P181), the purchase information management unit 31 performs the same processing as steps P142 to P143 in the first embodiment. That is, the purchase information management unit 31 acquires ID information and content information from the received information (step P182), and further acquires transmission source device information from the ID information (step P183).

  Here, when the content information is transmitted from the line-of-sight detection unit (Yes in Step P184), the purchase information management unit 31 analyzes the line-of-sight information that is the content information, and the line-of-sight destination is the transmission source device. It is confirmed whether it is out of the range (step P185). If it is out of range, the purchase information management unit 31 sets the transmission source of the line-of-sight destination information as a commodity detection apparatus in the line-of-sight direction, that is, a commodity detection apparatus installed adjacent to the transmission source apparatus. It is considered that there is (step P186). Then, the purchase information management unit 31 stores the line-of-sight information in the database in association with the commodity detection device regarded as the transmission source (step P187). That is, when the consumer's line of sight is facing an adjacent product movement detection device, the purchase information management unit 31 considers that the product grasped by the product movement detection device has been focused, and purchase process information is also Process it by associating it with the product.

  As described above, in this embodiment, when the consumer's line-of-sight destination is facing outward from the product grasped by the self-device, information on the commodity movement detection device that grasps the commodity at the line-of-sight destination is used. I am doing so. Therefore, even if the line of sight indicated by the line-of-sight destination information faces the outside of the product movement detection device, that is, even if the image from the front of the product cannot be acquired, the purchase process information can be acquired easily and with high accuracy. Can do.

  In addition, the purchase information management part 31 may add the same change also about person characteristic information, when transmitting apparatus information is changed in the process of step P186. The person characteristic information is acquired via the imaging unit, and a similar problem is assumed. According to such a form, more appropriate purchase process information can be acquired.

  Further, in the present embodiment, a case has been described in which the purchase information management unit 31 knows in advance which product movement detection device is adjacent, that is, the relative positional relationship of each product movement detection device. However, the purchase information management unit 31 may acquire the relative positional relationship of each product movement detection device based on information from the imaging unit included in the product movement detection device. For example, when a person crosses the front of an adjacent commodity movement detection device, the imaging unit of each device can capture the person with a time difference. Therefore, the purchase information management unit 31 can acquire the arrangement information of the adjacent product movement detection unit by acquiring and analyzing the time difference information. According to such a form, it is possible to obtain relative position information of each device without performing prior tuning, that is, without grasping device information in advance.

  As mentioned above, although each embodiment of this invention was described, the technical scope of this invention is not limited to the range described in each embodiment. In addition, when it is obvious to those skilled in the art that various modifications or improvements can be added to each embodiment, the embodiments added with such modifications or improvements are also included in the technical scope of the present invention. obtain.

  The specific configurations of the system and apparatus according to the present invention are not limited to those described above, and various design changes can be made without departing from the scope of the present invention. For example, among the ID information generated by the product movement detection unit, the sales information acquisition unit, or the person feature extraction unit, the information corresponding to the time is set as the information reception time at the purchase information management unit at the purchase information management unit. You may make it produce | generate based on.

  In addition, the drawings and specific numerical values and contents shown in each embodiment should not be used for a limited interpretation of the invention. In addition, not all combinations of features shown in each embodiment are essential for the solution means of the invention.

  Also, a program for realizing all or part of the functions of the purchase information management system in the present invention is recorded on a computer-readable recording medium, and the program recorded on the recording medium is read into the computer system and executed. By doing so, you may process each part. Here, the “computer system” includes an OS and hardware such as peripheral devices.

  The “computer-readable recording medium” refers to a portable medium such as a magneto-optical disk, ROM, and nonvolatile semiconductor memory, and a storage device such as a hard disk built in the computer system. Further, the “computer-readable recording medium” is a program that dynamically holds a program for a short time, like a communication line when a program is transmitted via a network such as the Internet or a communication line such as a telephone line. In addition, it also includes those that hold a program for a certain period of time, such as a volatile memory inside a computer system serving as a server or client in that case. Further, the program may be for realizing a part of the functions of the purchase information management system, and further, the function of the purchase information management system is realized in combination with a program already recorded in the computer system. It may be possible.

  FIG. 26 is a block diagram showing the minimum configuration of the purchase information management system according to the present invention. FIG. 27 is a block diagram showing another minimum configuration of the purchase information management system according to the present invention.

  As shown in FIG. 26, the purchase information management system (equivalent to the purchase information management system 1 shown in FIG. 1) is a product movement detection device 100 (in the purchase information management system 1 shown in FIG. 1) installed in the vicinity of the product. Communicable with the commodity movement detection apparatus 10), the sales management apparatus 200 (corresponding to the sales management apparatus 20 in the purchase information management system 1 shown in FIG. 1), the commodity movement detection apparatus 100 and the sales management apparatus 200. The product movement detection device 100 includes a purchase information management device 300 (corresponding to the purchase information management device 30 in the purchase information management system 1 shown in FIG. 1), and the product movement detection device 100 (the product movement detection device shown in FIG. 2). 10) and a movement detection sensor 101 that detects whether or not the product has moved (a sensor in the product movement detection device 10 shown in FIG. 2). And a product movement detection unit that generates product movement information indicating that the movement of the product is detected when the movement of the product is monitored via the movement detection sensor 101 and the movement of the product is detected. 102 (corresponding to the product movement detection unit 12 in the product movement detection device 10 shown in FIG. 2) and product movement that is characteristic information of the person who moved the product based on the imaging information acquired by the product movement imaging unit 103 The sales management device 200 includes a product moving person feature extraction unit 104 (corresponding to the person feature extraction unit 14-1 in the product movement detection device 10 shown in FIG. 2). (Corresponding to the imaging unit 13-2 in the sales management device 20 shown in FIG. 3) and a sales information acquisition unit 202 that acquires the sales information of the product (in the sales information acquisition unit 21 in the sales management device 20 shown in FIG. 3). And a product purchase person feature extraction unit 204 (shown in FIG. 3) that acquires product purchase person feature information, which is the feature information of the person who purchased the product, based on the imaging information acquired by the purchaser imaging unit 203. The purchase information management device 300 receives the product movement information and the product movement person feature information from the product movement detection device 100, and receives the sales information and the product. Purchased person characteristic information is received from the sales management apparatus 200, and the received information is associated with the transmission source apparatus and stored as purchase information in the purchase information management unit 301 (purchase information management in the purchase information management apparatus 30 shown in FIG. 4). Equivalent to the unit 31 and the database 32).

  As shown in FIG. 27, the purchase information management system includes a product tracking device 500 (corresponding to the product tracking device 50 in the purchase information management system 1a shown in FIG. 12) in addition to the configuration shown in FIG. The apparatus 500 includes a tracking sensor 501 (corresponding to the sensor 51 in the commodity tracking apparatus 50 shown in FIG. 14) that communicates with the IC tag attached to the commodity, and the commodity movement detection unit 102 among the commodity moves. Based on the communication result between the IC tag attached to the moving product and the tracking sensor, whether or not the moving product that is detected and the sales information management unit 310 has not acquired the sales information is within the tracking range. Product tracking unit 502 (corresponding to product tracking unit 52 in product tracking device 50 shown in FIG. 14).

  In the above embodiment, the following purchase information management system is also disclosed.

(1) The movement detection sensor 101 is a purchase information management system that communicates with an IC tag attached to a product and detects the movement of the product according to the communication result with the IC tag.

  According to such a configuration, it is difficult to cause erroneous detection or oversight based on factors other than product movement. Therefore, the product movement can be detected with higher accuracy.

(2) The tracking sensor 501 that communicates with the IC tag attached to the product and the product movement detection unit 102 among the products, movement is detected, and the sales information management unit 310 has not acquired the sales information. A product tracking unit that determines whether or not the moving product is within the tracking range based on the communication result between the IC tag attached to the moving product and the tracking sensor, and transmits the determination result to the purchase information management apparatus 300 A purchase information management system including a product tracking apparatus 500 (corresponding to the product tracking apparatus 50 in the purchase information management system 1a shown in FIG. 12).

  According to such a configuration, it is possible to acquire state information that appears to be abnormal in terms of the product purchase process, such as product loss or theft.

(3) The product moving person feature extraction unit 104 acquires image data obtained by photographing a person as person feature information, and the sales information management unit 301 uses the determination result received from the product tracking device 500 to move the product. A purchase information management system that extracts person characteristic information corresponding to a product not within the tracking range of the product tracking unit 502 from the purchase information.

  According to such a configuration, when the state information that is considered to be abnormal in view of the product purchase process, such as the loss or theft of the product, is acquired, the feature information of the person involved in the product can be acquired.

(4) The product movement detection device 100 (corresponding to the product movement detection device 10b shown in FIG. 20) includes a line-of-sight detection unit (corresponding to the line-of-sight detection unit 18 in the product movement detection device 10b shown in FIG. 20). The detection unit is a purchase information management system that detects the line of sight of a person photographed by the commodity movement imaging unit 103, acquires line-of-sight information indicating the direction of the line of sight, and transmits the line-of-sight destination information to the purchase information management apparatus 300.

  According to such a configuration, the line-of-sight information can be acquired, and the purchase process can be analyzed in more detail.

(5) A purchase information management system in which the line-of-sight detection unit acquires the stay time of the line of sight as line-of-sight information.

  The dwell time of the line of sight is information suitable for analyzing the degree of attention to the product. Therefore, according to such a configuration, more detailed purchase process information can be acquired and used.

(6) The product movement detection device 100 includes an advertisement display unit, and the line-of-sight detection unit determines whether the line of sight is directed to the advertisement display unit or the product based on line-of-sight destination information.

  According to such a configuration, it is possible to easily analyze the degree of attention of advertisement display and the effect of advertisement display on purchasing process behavior.

(7) When the sales information management unit 301 determines that the line of sight indicated by the line-of-sight destination information is facing outside the detection range of the commodity movement detection device that is the transmission source of the line-of-sight destination information, A purchase information management system that holds a product in the line of sight in association with a product movement detection device that is a detection target.

  According to such a configuration, even if the line of sight indicated by the line-of-sight destination information faces the outside of the product movement detection device, that is, even if the image from the front of the product cannot be acquired, the purchase can be performed easily and with high accuracy. Process information can be acquired.

Moreover, although a part or all of said embodiment may be described also as the following additional remarks, it is not restricted to the following.

(Supplementary Note 1) Product movement detection device (corresponding to product movement detection device 10 in purchase information management system 1 shown in FIG. 1) installed in the vicinity of the product, and sales management device (purchase information management system 1 shown in FIG. 1) And a purchase information management device (corresponding to the purchase information management device 30 in the purchase information management system 1 shown in FIG. 1) that can communicate with the commodity movement detection device and the sales management device. The product movement detection device includes a product movement imaging unit (corresponding to the imaging unit 13 in the product movement detection device 10 shown in FIG. 2), and a movement detection sensor (product movement detection shown in FIG. Corresponding to the sensor 11 in the apparatus 10), and the movement of the commodity is monitored via the movement detection sensor, and when the movement of the commodity is detected, the commodity movement information indicating that the movement of the commodity is detected. Of the person who moved the product based on the image information acquired by the product movement detection unit (equivalent to the product movement detection unit 12 in the product movement detection device 10 shown in FIG. 2) and the product movement imaging unit. A product moving person feature extraction unit (corresponding to the person feature extracting unit 14-1 in the product movement detection device 10 shown in FIG. 2) that acquires product moving person feature information as information. An imaging unit (corresponding to the imaging unit 13-2 in the sales management apparatus 20 shown in FIG. 3) and a sales information acquisition unit (sales information acquisition unit 21 in the sales management apparatus 20 shown in FIG. And a product purchase person feature extraction unit (sales management shown in FIG. 3) that acquires product purchase person feature information, which is feature information of the person who purchased the product, based on the imaging information acquired by the purchaser imaging unit. In the device 20 The purchase information management apparatus receives the merchandise movement information and the merchandise movement person feature information from the merchandise movement detection apparatus, and receives the sales information and the merchandise purchase person characteristic information from the sales management apparatus. Purchasing information management provided with a purchasing information management unit (corresponding to the purchasing information management unit 31 in the purchasing information management device 30 shown in FIG. 4) that receives and stores each received information as a purchasing information in association with the transmission source device. system.

(Supplementary note 2) The purchase information management system according to supplementary note 1, wherein the movement detection sensor communicates with an IC tag attached to the product and detects the movement of the product according to a communication result with the IC tag.

(Appendix 3) A tracking sensor (corresponding to the sensor 51 in the product tracking device 50 shown in FIG. 14) that communicates with the IC tag attached to the product, and the product movement detection unit among the products detects movement. A product that determines whether or not a moving product for which the sales information management unit has not acquired sales information is within the tracking range based on a communication result between an IC tag attached to the moving product and a tracking sensor. Supplementary note 1 including a product tracking device (corresponding to the product tracking device 50 in the purchase information management system 1a shown in FIG. 12) provided with a tracking unit (corresponding to the product tracking unit 52 in the product tracking device 50 shown in FIG. 14). The purchase information management system according to attachment 2.

(Supplementary Note 4) The product moving person feature extracting unit acquires image data obtained by photographing a person as person feature information, and the sales information managing unit is based on the determination result received from the product tracking device, The purchase information management system according to attachment 3, wherein person characteristic information corresponding to a product not within the tracking range of the product tracking unit is extracted from the purchase information.

(Supplementary Note 5) The commodity movement detection device includes a plurality of types of movement detection sensors, and the commodity movement detection unit 102 detects the movement of the commodity based on information acquired from the plurality of kinds of movement detection sensors. The purchase information management system according to any one of 1 to appendix 4.

  According to such a configuration, product movement can be detected with higher accuracy.

(Additional remark 6) A goods movement detection apparatus is provided with a gaze detection part, a gaze detection part detects the gaze of the person image | photographed by the merchandise movement imaging part 103, acquires gaze destination information which shows the direction of a gaze, The purchase information management system according to any one of supplementary notes 1 to 5, which transmits information to a purchase information management device.

(Supplementary note 7) The purchase information management system according to supplementary note 6, wherein the line-of-sight detection unit acquires the stay time of the line of sight as the line-of-sight destination information.

(Supplementary note 8) The commodity movement detection device includes an advertisement display unit, and the line-of-sight detection unit determines whether the line of sight is directed to the advertisement display unit or the commodity based on the line-of-sight destination information. Purchasing information management system.

(Supplementary Note 9) If the sales information management unit determines that the line of sight indicated by the line-of-sight destination information is facing outside the detection range of the commodity movement detection device that is the transmission source of the line-of-sight destination information, 9. The purchase information management system according to any one of appendix 6 to appendix 8, wherein the merchandise in the direction of the line of sight is stored in association with a merchandise movement detection device that is a detection target.

(Supplementary Note 10) The purchase information management unit is based on at least one or more information among product movement person feature information, product purchase person feature information, product movement information, product sales information, and information indicating an acquisition time of each information. The purchase information management system according to any one of appendix 1 to appendix 9, wherein at least one piece of information is extracted and presented from merchandise move person feature information, merchandise purchase person feature information, merchandise move information, and sales information.

  According to such a configuration, statistical information can be presented to a manager of a purchase information management system, a store manager, a store clerk, or the like.

Moreover, although a part or all of said embodiment may be described also as the following additional remarks, it is not restricted to the following.

(Supplementary Note 11) Product movement imaging unit, movement detection sensor for detecting presence or absence of movement of product, movement of product is monitored via movement detection sensor, and movement of product is detected when movement of product is detected. A product movement detection unit that generates product movement information indicating that it has been detected, and a product that acquires product moving person feature information that is characteristic information of a person who has moved the product based on the imaging information acquired by the product movement imaging unit A product movement detection apparatus comprising a moving person feature extraction unit 104.

(Supplementary note 12) The commodity movement detection device according to supplementary note 11, wherein the movement detection sensor communicates with an IC tag attached to the commodity and detects the movement of the commodity according to a communication result with the IC tag.

(Supplementary Note 13) A line-of-sight detection unit (corresponding to the line-of-sight detection unit 18 in the commodity movement detection device 10b shown in FIG. 20) is provided, and the line-of-sight detection unit detects the line of sight of a person photographed by the commodity movement imaging unit. Item 13. The commodity movement detection device according to appendix 11 or appendix 12, which acquires the line-of-sight information indicating a direction and transmits the line-of-sight information to the purchase information management device.

(Supplementary note 14) The commodity movement detection device according to supplementary note 13, wherein the line-of-sight detection unit acquires the stay time of the line of sight as the line-of-sight information.

(Supplementary note 15) The commodity movement detection device includes an advertisement display unit, and the line-of-sight detection unit determines whether the line of sight is directed to the advertisement display unit or the commodity based on the line-of-sight destination information. Product movement detection device.

DESCRIPTION OF SYMBOLS 1, 1a Purchasing information management system 2, 2a, 2a-1, 2a-2 Product 3 Product display shelf 4 Consumer 5 Imaging range 6-1, 6-2 Detection range 7 Line of sight 10, 10a, 10a ', 10b, 10c -1, 10c-2, 100 Product movement detection device 11, 51 Sensor 12, 102 Product movement detection unit 13-1, 13-2 Imaging unit 14-1, 14-2 Person feature extraction unit 15-1, 15-2 , 15-3, 15-5 Communication unit 16 Infrared light emitting unit 17 Infrared light receiving unit 18 Line-of-sight detection unit 20, 200 Sales management device 21, 202 Sales information acquisition unit 30, 300 Purchasing information management device 31, 301 Purchasing information management unit 32 Database 40, 40-1, 40-2 IC tag 50, 500 Merchandise tracking device 52, 502 Commodity tracking unit 101 Movement detection sensor 103 Commodity movement imaging unit 104 Commodity Dynamic personal characteristic extraction section 203 purchaser imaging unit 204 purchase personal characteristic extraction section 501 tracking sensor

Claims (10)

A product movement detection device installed in the vicinity of the product, a sales management device, and a purchase information management device communicable with the product movement detection device and the sales management device,
The commodity movement detection device is:
A commodity moving imaging unit;
A movement detection sensor for detecting the presence or absence of movement of the product;
A product movement detection unit that monitors the movement of the product via the movement detection sensor and generates product movement information indicating that the movement of the product is detected when the movement of the product is detected;
A product moving person feature extracting unit that acquires product moving person feature information that is characteristic information of a person who has moved the product based on the imaging information acquired by the product moving imaging unit;
The sales management device is:
A purchaser imaging unit;
A sales information acquisition unit for acquiring sales information of the product;
A product purchase person feature extraction unit that acquires product purchase person feature information, which is feature information of a person who purchased the product, based on imaging information acquired by the purchaser imaging unit;
The purchase information management device comprises:
The merchandise movement information and the merchandise movement person characteristic information are received from the merchandise movement detection apparatus, the sales information and the merchandise purchase person characteristic information are received from the sales management apparatus, and each received information is sent to a transmission source apparatus. A purchasing information management system characterized by comprising a purchasing information management section that stores information as purchasing information in association with each other. The purchase information management system according to claim 1, wherein the movement detection sensor communicates with an IC tag attached to a product and detects the movement of the product according to a communication result with the IC tag. A tracking sensor that communicates with the IC tag attached to the product;
Of the products, whether or not the product movement detection unit detects the movement and the sales information management unit does not acquire the sales information is within the tracking range, is given to the moving product. The purchase information management according to claim 1, further comprising: a merchandise tracking unit that includes a merchandise tracking unit that makes a determination based on a communication result between the IC tag and the tracking sensor and transmits the determination result to the purchase information management apparatus. system. The product moving person feature extraction unit acquires image data obtained by photographing a person as person feature information,
The sales information management unit extracts, from the purchase information, person characteristic information corresponding to a product that is not within the tracking range of the product tracking unit among the moving products based on the determination result received from the product tracking device. The purchase information management system described. The commodity movement detection device includes a line-of-sight detection unit,
The line-of-sight detection unit detects a line of sight of a person photographed by the commodity movement imaging unit, acquires line-of-sight destination information indicating the direction of the line of sight, and transmits the line-of-sight destination information to a purchase information management device. Item 5. The purchase information management system according to any one of items 4 to 4. The purchase information management system according to claim 5, wherein the line-of-sight detection unit acquires the staying time of the line of sight as line-of-sight information. The commodity movement detection device includes an advertisement display unit,
The purchase information management system according to claim 5 or 6, wherein the line-of-sight detection unit determines whether the line of sight is directed to the advertisement display unit or the product based on line-of-sight destination information. When the sales information management unit determines that the line of sight indicated by the line-of-sight destination information is facing the outside of the detection range of the commodity movement detection device that is the transmission source of the line-of-sight destination information, the line-of-sight destination information The purchase information management system according to any one of claims 5 to 7, wherein a product in the direction of the product is held in association with a product movement detection device as a detection target. A commodity moving imaging unit;
A movement detection sensor for detecting the presence or absence of movement of the product;
A product movement detection unit that monitors the movement of the product via the movement detection sensor and generates product movement information indicating that the movement of the product is detected when the movement of the product is detected;
A product moving person feature extracting unit that acquires product moving person feature information that is characteristic information of a person who has moved the product based on imaging information acquired by the product moving imaging unit; Movement detection device. A purchase information management method in a purchase information management system including a product movement detection device installed in the vicinity of a product, a sales management device, and a purchase information management device communicable with the product movement detection device and the sales management device. And
The product movement detection device provided with a product movement imaging unit,
The movement of the product is monitored via a movement detection sensor that detects the presence or absence of the movement of the product, and when the movement of the product is detected, product movement information indicating that the movement of the product is detected is generated,
Based on the imaging information acquired by the product movement imaging unit, acquires product moving person feature information that is characteristic information of a person who has moved the product,
A sales management device with a purchaser imaging unit
Obtain sales information for the product,
Based on the imaging information acquired by the purchaser imaging unit, to acquire product purchase person feature information that is the feature information of the person who purchased the product,
Purchasing information management device
The merchandise movement information and the merchandise movement person characteristic information are received from the merchandise movement detection apparatus, the sales information and the merchandise purchase person characteristic information are received from the sales management apparatus, and each received information is sent to a transmission source apparatus. A purchase information management method characterized in that the information is stored in association with purchase information.

Images