JP5923962B2 - Information analysis system - Google Patents

Description

  The present invention relates to an information analysis system.

  2. Description of the Related Art Conventionally, an analysis device that recognizes a product that a purchaser is interested in based on the line-of-sight information of the purchaser and performs marketing research is known (for example, see Patent Document 1). According to this analyzer, the results of marketing research can be used for sales promotion of products.

JP 2010-204882 A

  However, this analyzer has a problem in that information collected in marketing research is useful information for the seller, but not useful for the purchaser.

  An object of the present invention is to provide an information analysis system that provides useful information to a purchaser.

The information analysis system according to the present invention includes a first photographing unit that photographs a person who is viewing a product, a second photographing unit that photographs a person who is purchasing the product, and an image photographed by the second photographing unit. A specifying unit for recognizing a person who has purchased the product from image data and identifying a product that the person who has purchased the product was viewing from image data of an image captured by the first imaging unit ; A recording unit that records a product that has not been purchased among the products specified by the specifying unit .

  According to the information analysis system of the present invention, it is possible to provide useful information to the purchaser.

It is a block diagram which shows the structure of the information analysis system which concerns on embodiment. It is a figure which shows the installation division of the product camera in the information analysis system which concerns on embodiment. It is a figure which shows the content of the history file memorize | stored in the information storage part of the information analysis system which concerns on embodiment. It is a flowchart which shows the imaging | photography process of the information analysis system which concerns on embodiment. It is a figure which shows the image of the purchaser's face image | photographed with the product camera in the information analysis system which concerns on embodiment. It is a figure which shows the goods which the purchaser had seen in the information analysis system which concerns on embodiment. It is a line-of-sight detection range in the information analysis system concerning an embodiment. It is a flowchart which shows the purchase process in the information analysis system which concerns on embodiment. It is a flowchart which shows the printing process of the receipt in the information analysis system which concerns on embodiment.

  Hereinafter, an information analysis system according to an embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing a configuration of an information analysis system 2 according to the embodiment. The information analysis system 2 is provided, for example, in a merchandise store such as a supermarket or a convenience store, and includes a control unit 4 that comprehensively controls each unit of the information analysis system 2. The control unit 4 stores a product camera 6 that captures a purchaser who intends to purchase a product, an input unit 8 that inputs image capturing timing of the product camera 6, and image data of an image captured by the product camera 6. The image storage unit 10, the information storage unit 12 that stores history information about the time when the purchaser has been watching the product, the product storage unit 14 that stores information about the product ID of the product placed on the product shelf, Cash register 16, which is a register for accounting for sales of merchandise, a cash register 16, which is arranged in the vicinity of cash register 16, takes a picture of the purchaser's face when performing accounting processing, and images of images taken by cash register camera 18. A registration image storage unit 20 for storing data is connected.

  Here, the product camera 6 includes n cameras, a product camera 6a, a product camera 6b,..., A product camera 6n, and the product camera 6a, the product camera 6b,. It is installed in the display shelf for every certain section. For example, as shown in FIG. 2, a product camera 6a is installed in a section composed of 5 products horizontally and 3 products vertically (products A to E, products F to J, products K to O). A product camera 6b,..., A product camera 6n is installed for each section composed of the number of products.

  Further, the cash register 16 includes m cash registers, that is, a cash register 16a, a cash register 16b,..., A cash register 16m, and each of the cash registers 16a, 16b,. It is installed at the cashier counter. Each cash register includes a bar code scanner (not shown) that reads a product code of a purchased product, and a cash register printer (not shown) that prints the purchase price on a receipt.

  The registration camera 18 includes m cameras, a registration camera 18a, a registration camera 18b,..., And a registration camera 18m. Here, the cash register camera 18a is installed in the vicinity of the cash register 16a. Similarly, the registration camera 18b is installed in the vicinity of the cash register 16b, and the registration camera 18m is installed in the vicinity of the cash register 16m. The cashier camera 18a, cashier camera 18b,..., Cashier camera 18m are installed at positions where the face of the purchaser can be photographed when performing the accounting process.

  Further, the information storage unit 12 stores a history file 30 shown in FIG. The history file 30 includes a history number for distinguishing one history from other histories, face image data of a purchaser's face image taken by the product camera 6, and a purchaser of a purchaser based on the face image data. ID, the product ID of the product viewed by the person having the purchaser ID, the time the person having the purchaser ID was viewing the product with the product ID, the shooting date by the product camera 6, and the purchase result of the product having the product ID Are stored in order of history number.

  FIG. 4 is a flowchart showing a photographing process of the information analysis system 2 according to the embodiment. First, shooting timing for shooting a still image by the product camera 6 is input via the input unit 8 (step S1). For example, when taking a still image with the product camera 6 every 5 seconds, a shooting timing of 5 seconds is input via the input unit 8. In the following flowchart, a shooting process when shooting is performed by the product camera 6a will be described as an example.

  When the photographing timing is input, the control unit 4 performs photographing with the commodity camera 6a (step S2), and stores the image data of the photographed image in the image storage unit 10. Next, the control unit 4 reads the image data from the image storage unit 10 and determines whether or not a human face is included in the image based on the image data (step S3). Here, the detection of the human face is performed by a generally known face recognition technique.

  When the human face is not included in the image based on the image data (step S3: No), the process proceeds to step S11. On the other hand, when a human face is included in the image based on the image data (step S3: Yes), the control unit 4 includes a human face that does not have a purchaser ID in the image based on the image data. It is determined whether or not (step S4). That is, the control unit 4 reads the history file 30 (see FIG. 3) from the information storage unit 12, and based on the face of the person included in the image based on the image data and the face image data stored in the history file 30. By comparing with the face of the purchaser, it is determined whether or not an image based on the image data includes a face of a person who does not have a purchaser ID.

  When a face of a person who does not have a purchaser ID is included in the image based on the image data (step S4: Yes), the control unit 4 selects an image of the face of a person who does not have a purchaser ID from the image data. The face image data is extracted, and the extracted face image data is stored in the history file 30 together with a new purchaser ID (step S5). For example, when the image based on the image data includes a person with the purchaser ID “0001”, a person with the purchaser ID “0002”, and a person who does not have the purchaser ID, the control unit 4 determines from the image data. Face image data of a face image of a person who does not have a purchaser ID is extracted. Then, as shown in FIG. 3, the history of the history number 3 having the extracted image data and the new purchaser ID “0003” is added to the history file 30.

  When the face based on the image data does not include the face of a person who does not have a purchaser ID (step S4: No), or when the process of step S5 is completed, the control unit 4 displays the image based on the image data. The line of sight of the person's face is detected from (step S6), and a line of sight vector indicating the direction of the line of sight is created. For example, as shown in FIG. 5, the image based on the image data includes the face of the person with the purchaser ID “0001”, the face of the person with the purchaser ID “0002”, and the face of the person with the purchaser ID “0003”. Suppose it was. In this case, the control unit 4 first determines the line of sight of the person with the purchaser ID “0001”, the line of sight of the person with the purchaser ID “0002”, and the line of sight of the person with the purchaser ID “0003” from the image based on the image data. To detect. Based on the detected line of sight, a line-of-sight vector 32 of the person with the purchaser ID “0001”, a line-of-sight vector 34 of the person with the purchaser ID “0002”, and a line-of-sight vector 36 of the person with the purchaser ID “0003” are created. To do. Here, the detection of the line of sight is performed by a generally known line-of-sight detection technique.

  Next, the control part 4 specifies the goods which the purchaser is looking based on the produced eyes | visual_axis vector (step S7). For example, as illustrated in FIG. 6, when the tip of the line-of-sight vector 32 is located within the range of the product K, the product viewed by the person with the purchaser ID “0001” is identified as the product K. Similarly, when the tip of the line-of-sight vector 34 is located within the range of the product H, the product viewed by the person with the purchaser ID “0002” is identified as the product H, and the tip of the line-of-sight vector 36 is the product E. If the product is located in the section of the product, the product viewed by the person with the purchaser ID “0003” is identified as the product E.

  Since an error may occur in the line-of-sight vector, a slight error is allowed when specifying a product. For example, as shown in FIG. 7, an auxiliary section 42 is set around the section 40 of the product K, and the person with the purchaser ID “0001” is looking at the previous shooting timing (for example, 5 seconds before). When the product is specified as the product K, if the tip of the line-of-sight vector 32 protrudes from the section 40 at the current photographing timing but is located within the auxiliary section 42, the purchaser ID “0001” is stored. The product viewed by the person is identified as the product K.

  Next, the control part 4 reads the information regarding the goods ID of the goods arrange | positioned at the goods shelf from the goods memory | storage part 14, and specifies the goods ID of the goods detected by step S7 based on the read information. For example, the product ID of the product K viewed by the person with the purchaser ID “0001” is specified as the product ID “A001”. Similarly, the product ID of the product H viewed by the person with the purchaser ID “0002” is identified as the product ID “A002”, and the product ID of the product E viewed by the person with the purchaser ID “0003” is the product ID. Specify "A003".

  Next, the control unit 4 refers to the history file 30 to determine whether or not the product for which the product ID is specified is a product that the same purchaser has seen today (step S8). When the product for which the product ID is specified is a product that the same purchaser has seen today (Yes at Step S8), the control unit 4 inputs the shooting time at the time when the purchaser was viewing the product in the history file 30. The time is added (step S9).

  For example, it is assumed that today's date is December 10, 2011, the photographing timing is 5 seconds, and information of history numbers 1 to 3 is stored in the history file 30 (see FIG. 3). At this shooting timing, the product ID of the product K viewed by the person with the purchaser ID “0001” is identified as the product ID “A001”, and the product H viewed by the person with the purchaser ID “0002” is identified. It is assumed that the product ID is specified as the product ID “A002” and the product ID of the product E viewed by the person with the purchaser ID “0003” is specified as the product ID “A003”. In this case, the control unit 4 adds 5 seconds of the photographing timing to 15 seconds that is the time when the person with the purchaser ID “0001” was looking at the product K with the product ID “A001” in the history with the history number 1. Memorize 20 seconds. Similarly, in the history of the history number 2, the person with the purchaser ID “0002” was watching the product H with the product ID “A002”, added 5 seconds to 10 seconds, and stored 15 seconds. In the history 3, 5 seconds is added to 20 seconds, which is the time when the person with the purchaser ID “0003” was looking at the product E with the product ID “A003”, and 25 seconds is stored.

  On the other hand, if the product with the specified product ID is not a product that the same purchaser has seen today (step S8: No), the control unit 4 adds a history of a new history number to the history file 30 (step S10). ). For example, when the person with the purchaser ID “0001” sees the product with the product ID “A004” for the first time today, the control unit 4 determines that the person with the purchaser ID “0001” has the product ID “A004” as shown in FIG. The history of the new history number 4 is added to the history file 30 with the time of watching the product “5” as 5 seconds.

  Next, the control unit 4 determines whether or not an operation for ending the photographing process of the information analysis system 2 has been performed (step S11). When an operation to end the photographing process of the information analysis system 2 is performed, the control unit 4 ends a series of processes. On the other hand, when the operation for ending the photographing process of the information analysis system 2 has not been performed, the control unit 4 performs photographing with the product camera 6a after five seconds as the photographing timing have elapsed since the previous photographing was performed. Perform (step S2).

  Note that the same processing as that described in step S2 to step S11 is performed even when the product cameras 6b to 6n are photographed.

  FIG. 8 is a flowchart showing purchase processing in the information analysis system 2 according to the embodiment. In the following flowchart, an example in which the accounting process is performed by the cash register 16a and the purchaser is photographed by the cash register camera 18a will be described as an example. First, when the purchaser brings a product to be purchased to the cashier counter and the accounting process by the cash register 16a is started, the control unit 4 photographs the purchaser with the cashier camera 18a (step S21), and the image of the photographed image. The data is stored in the registration image storage unit 20.

  Next, the control unit 4 reads out the image data from the cash register image storage unit 20 and detects a human face (step S22). Next, the control unit 4 reads the history file 30 from the information storage unit 12, and specifies the purchaser ID of the purchaser who has the detected person's face (step S23).

  Next, when the product code of the product purchased by the purchaser is read by the bar code scanner provided in the cash register 16a, the control unit 4 specifies the product ID of the product corresponding to the read product code (step S24) The purchase result indicating that the product has been purchased is stored in the history file 30 for the product with the specified product ID (step S25). For example, if the purchaser who brought the product to the cashier counter is the person with the purchaser ID “0001” and the product read by the barcode scanner is the product with the product ID “A001”, the control unit 4 The purchase result indicating that the product has been purchased is stored in history number 1 in which the person with ID “0001” and the product ID “A001” are stored (see FIG. 3).

  Next, the accounting process by the cash register 16a is completed (step S26), and when the accounting process is finished, the control unit 4 records the history of the products that the purchaser has not purchased from today. The purchase result indicating that the product has not been purchased is stored in the file 30 (step S27). For example, if today's date is December 10, 2011 and the purchaser is a person with purchaser ID “0001”, the product code of the product with product ID “A004” remains unread by the barcode scanner. When the accounting processing end operation is performed, the control unit 4 stores the history number 4 in which the person with the purchaser ID “0001” and the product with the product ID “A004” are stored on the date of December 10, 2011. The purchase result indicating that the product has not been purchased is stored (see FIG. 3).

  Next, the control unit 4 prints the product name and the purchase price of the product purchased by the purchaser on the receipt by the cash register 16a, and purchases the purchaser at the lower end of the receipt. The product name of the product that has not been printed is printed and output (step S28). For example, a person with a purchaser ID “0001” is viewing a product K with a product ID “A001” and a product O with a product ID “A004” today, purchases a product K with a product ID “A001”, and purchases a product ID “A004”. If the product O is not purchased, the control unit 4 prints the product name and price of the product K on the receipt, and prints and outputs the product name of the product O on the lower end portion of the receipt.

  Even when the accounting process is performed by the cash registers 16b to 16m, the same process as described with reference to the flowchart of FIG. 8 is performed.

  According to the information analysis system 2 according to the present embodiment, since the product name of the product that the purchaser has seen but has not purchased is printed on the receipt, the purchaser can refer to it at the next purchase. Can provide useful information.

  In the above-described embodiment, information related to consumable products, information related to a set product, and information related to similar products may be stored in the product storage unit 14. And when a purchaser purchases goods, you may make it perform the printing process of a receipt according to the flowchart shown in FIG.

  For example, it is assumed that the person with the purchaser ID “0001” who purchased the product K is photographed by the cashier camera 18a and the cash register 16a performs the accounting process. In this case, the control unit 4 first refers to the history file 30 and detects the product ID of the product purchased by the person with the purchaser ID “0001” before today. Next, based on the information related to the consumable product stored in the product storage unit 14, among the products with the detected product ID, for example, the product S with the product ID “Z001”, the product T with the product ID “Z002”, and the product The product U with the ID “Z003” is identified as a consumable product such as household goods that is consumed regularly.

  Next, the control unit 4 determines whether there is a consumable product that has not been purchased by the person with the purchaser ID “0001” within a predetermined number of days, for example, five days, among the consumable product S, the consumable product T, and the consumable product U. It is determined whether or not (step S31). If there is a consumable product that has not been purchased by the person with the purchaser ID “0001” within 5 days (step S31: Yes), for example, if the consumable product S has not been purchased within 5 days, the control unit 4 The product name and purchase price of the product K purchased by the purchaser are printed on the receipt by the receipt printer of the cash register 16a, and the product name of the consumable product S is printed on the lower end portion of the receipt and output (step S32). .

  On the other hand, if there is no consumable product that has not been purchased by the person with the purchaser ID “0001” within 5 days (step S31: No), the control unit 4 relates to the set product stored in the product storage unit 14. Based on the information, it is determined whether there is a product to be set with the product K (step S33). When there is a product to be set with the product K (step S33: Yes), for example, when the product K is coffee and coffee and cream are set, the control unit 4 uses the receipt printer of the cash register 16a to make coffee on the receipt. The product name of coffee and the amount of coffee are printed, and the product name of creep is printed on the receipt at the lower end portion of the receipt and output (step S34).

  On the other hand, when there is no product to be set with the product K (step S33: No), the control unit 4 has a product similar to the product K based on the information related to the similar product stored in the product storage unit 14. Is determined (step S35). When there is a product similar to the product K (step S35: Yes), for example, when the product K is coffee and coffee and tea are similar, the control unit 4 uses the receipt printer of the cash register 16a to make a coffee product on the receipt. The name and the amount of coffee are printed, and the product name of tea similar to coffee is printed at the lower end portion of the receipt (step S36). On the other hand, when there is no product similar to the product K (step S35: No), the receipt printer prints and outputs the product name of the product K and the price of the product K (step S37). As a result, it is possible to prevent forgetting to purchase products that have expired, products that are likely to expire, products that are set with purchased products, and products that are similar to purchased products.

  Note that, regardless of the flowchart of FIG. 9, it is determined whether there is a consumable product that has not been purchased by the purchaser within a predetermined number of days (step S31), and whether there is a product that is a set with the product purchased by the purchaser. The determination of whether or not (step S33) and whether or not there is a product that is a set with the product purchased by the purchaser (step S35) may all be performed during the accounting process. Thereby, for example, when the person with the purchaser ID “0001” purchases coffee, the consumable product S is not purchased within 5 days (step S31: Yes), and there is a cream as a product set with coffee. (Step S33: Yes), if there is tea as a product similar to coffee (Step S35: Yes), the name and price of the coffee, the name of the consumable product S, the name of the cream, the tea All product names can be printed, and it is possible to prevent forgetting to purchase products more accurately.

  Further, in the above-described embodiment, among the products that the purchaser has not purchased, the product name of the product that the purchaser has seen over a predetermined time may be printed on the receipt. For example, it is assumed that the person with the purchaser ID “0001” has purchased the product K. Here, the person with the purchaser ID “0001” watched the product V for 10 seconds, the product W for 15 seconds, the product X for 15 seconds, the product Y for 20 seconds, and the product Z for 30 seconds, but did not purchase it. To do. In this case, the product name of the product Y and the product name of the product Z which have been viewed for more than 15 seconds but have not been purchased may be printed on the lower end portion of the receipt. As a result, if there are a large number of products that the buyer has seen but did not purchase, only the product names of the products that the purchaser can truly refer to at the time of the next purchase will be displayed on the receipt. It can be limited to.

  In addition, the product names of the products that the purchaser has seen but did not purchase may be printed on a predetermined number of receipts in the order in which the purchase time is long. For example, it is assumed that the person with the purchaser ID “0001” did not purchase the product V, viewed the product W for 10 seconds, the product W for 15 seconds, the product X for 15 seconds, the product Y for 20 seconds, and the product Z for 30 seconds. . In this case, the product names of the two products, that is, the product name of the product Z and the product name of the product Y may be printed on the lower end portion of the receipt in the order of long viewing time.

  Further, in the above-described embodiment, the information analysis system 2 performs e-mail between the personal information storage unit that stores in advance personal information such as the image data of the purchaser's face, the purchaser's address and e-mail address, and the mail server. There may be provided a wireless communication unit for transmitting and receiving. When the purchaser specified in the purchase process (FIG. 4, step S23) matches the purchaser stored in the personal information storage unit, the control unit 4 purchases via the wireless communication unit and the mail server. An e-mail describing the product name of a product that the customer has seen but did not purchase may be transmitted to the e-mail address of the purchaser. Thereby, useful information can be provided accurately to the purchaser. When the purchaser specified in the purchase process matches the purchaser stored in the personal information storage unit, the product name of the product that the purchaser has seen but did not purchase is printed on the direct mail. May be.

  In the above-described embodiment, recommended information related to a product may be stored in the product storage unit 14. For example, information related to a Hokkaido trip may be stored in relation to a product called crab. And when a purchaser purchases a crab, you may make it print the information regarding Hokkaido travel on the lower end part of a receipt.

  In the above-described embodiment, the cash register 16 may be provided with a display unit that displays the purchase amount and the like so that the purchaser can confirm. Then, when an operation for terminating the accounting process is performed, the control unit 4 may display the product name of the product that the purchaser has viewed but has not purchased on the display unit. Thereby, it is possible to effectively prevent forgetting to purchase a product.

  2 ... Information analysis system, 4 ... Control unit, 6 ... Product camera, 8 ... Input unit, 10 ... Image storage unit, 12 ... Information storage unit, 14 ... Product storage unit, 16 ... Cash register, 18 ... Cash register camera, 20 ... Register image storage

Claims (12)

A first photographing unit for photographing a person watching a product;
A second shooting section for shooting a person who has purchased the product;
A person who purchases the product from the image data of the image captured by the first imaging unit is recognized from the image data of the image captured by the second imaging unit. A specific part that identifies the product you were viewing,
An information analysis system comprising: a recording unit that records a product that has not been purchased among the products specified by the specifying unit . The identifying unit recognizes the face of a person who has purchased the product from the image photographed by the second photographing unit, and the face of the person viewing the product from the image photographed by the first photographing unit The information analysis system according to claim 1, wherein the product that the person who has purchased the product is viewing is identified. The information analysis system according to claim 1, further comprising a notification unit that notifies the product that has not been purchased. A counting unit that counts the time that the person watching the product was watching the product;
The notification unit, according to claim 3, wherein notifying the goods person person looking at the commodity which has been counted by the counting unit is looking at the commodity based on the time looking at products not purchased Information analysis system. 5. The information analysis system according to claim 4 , wherein the notification unit notifies a product that a person watching the predetermined number of products has not purchased in order of a long time that the person watching the product has viewed the product. . 6. The information analysis according to claim 4 or 5 , wherein the notification unit notifies a product that a person watching the product does not purchase when the person watching the product exceeds a predetermined threshold time. system. A history storage unit for storing the purchase history of the product;
A product specifying unit for specifying a purchase history related to a consumable product that is periodically consumed from the purchase history of the product stored in the history storage unit;
A purchase determination unit that determines whether a person viewing the product has purchased the consumable product within a predetermined number of days based on a purchase history related to the consumable product specified by the product specifying unit;
The notification unit, when a person is watching the commodity by the purchase determination unit is determined not to purchase the consumable items within the predetermined number of days, a person looking at the merchandise is within a predetermined number of days The information analysis system according to claim 3 , wherein the consumable product that has not been purchased is notified to a person viewing the product . A product storage unit for storing information about related products;
An extraction unit that extracts information related to the product purchased by the person who has purchased the product from the product storage unit;
The notification unit, the information analysis system according to any one of claims 3-7 for notifying a product associated with the product People are buying the products purchased person has purchased the product. The associated product, according to claim 8 is at least one of the items similar to items a person the person that purchased the goods are purchased items become commodity and set, and the purchase is purchased The information analysis system described. The notification unit, printing unit, and at least one display unit the person looking at the commodity display information about the product that was not purchased for printing information related to the product the person looking at the products did not buy The information analysis system as described in any one of Claims 3-9 provided with one. The information analysis system according to any one of claims 1 to 10 , further comprising: a transmission unit configured to transmit information on a product that the person viewing the product did not purchase to the person viewing the product by e-mail. The information analysis system according to any one of claims 1 to 10 , further comprising: a mail printing unit that prints information related to a product that a person watching the product did not purchase in a direct mail.

Images