JP7337625B2 - Purchase behavior data collection system and purchase behavior data collection program - Google Patents

Description

The embodiments of the present invention relate to a purchasing behavior data collection system and a purchasing behavior data collection program executed by a computer.

Conventionally, there has been known a system that analyzes the purchasing behavior of customers and provides data that can be linked to sales promotion measures on the store side. In conventional systems, for example, the product that the customer is holding is recognized based on the movement of the customer's hand, and the product that the customer is looking at is recognized based on the customer's line of sight. are collecting. Thus, conventional systems can collect data about the products that customers are interested in.

However, it is difficult to estimate on what information about the product the customer's purchasing behavior is based only by collecting data on the product that the customer is interested in.

WO2015/033577

The problem to be solved by the embodiments of the present invention is to provide a purchasing behavior data collection system and a purchasing behavior data collection program capable of collecting data indicating what information customers are interested in regarding products. be.

According to the embodiment, the purchasing behavior data collection system includes a line-of-sight recognition unit, a product recognition unit, a product orientation recognition unit, a product information search unit, a visual recognition information determination unit, and an information recording unit. The line-of-sight recognition unit recognizes the line of sight of the customer. The product recognition unit identifies a product to be visually recognized by the customer. The product posture recognition unit recognizes the posture of the product. The product information search unit searches for package information including a plurality of pieces of information attached to the exterior of the specified product. The visible information determination unit determines information attached to the exterior of the product visually recognized by the customer, based on the line of sight of the customer, the posture of the product, and the package information. The information recording unit records product visual recognition information including information determined to be visually recognized by the customer.

1 is a block diagram showing the configuration of a purchasing behavior data collection system according to this embodiment; FIG. FIG. 2 is a block diagram showing the main circuit configuration of the server according to the embodiment; The figure which shows an example of the data recorded on the camera installation database in this embodiment. FIG. 4 is a diagram for explaining an example of an imaging range of a camera according to the embodiment; FIG. 4 is a diagram showing an example of an image captured by a camera according to the embodiment; The figure which shows an example of a data structure of the goods visual recognition information recorded on the goods visual recognition information database in this embodiment. 4 is a flow chart showing the procedure of main processing executed according to the purchase behavior data collection program in this embodiment. FIG. 4 is a diagram showing area ranges corresponding to a plurality of pieces of information attached to a product indicated by package information; The figure which shows the condition of visual recognition with respect to the goods by a visitor.

Hereinafter, this embodiment will be described with reference to the drawings.

FIG. 1 is a block diagram showing the configuration of the purchasing behavior data collection system of this embodiment. The purchasing behavior data collection system of this embodiment is installed in a store 10 such as a mass retailer, a large electronics store, a home center, or the like. When a customer picks up a product for purchase, etc., the purchasing behavior data collection system determines which of a plurality of pieces of information attached to the exterior of the product the customer pays attention to. collect information about

As shown in FIG. 1, a store 10 consists of a sales floor 11 and a backyard 12 other than that. A plurality of product shelves 13 are arranged in the sales floor 11, and passages between the product shelves 13 and between the product shelves 13 and the wall serve as passages for customers. A customer who has entered the store goes around the aisle of the sales floor 11, picks up a product to be purchased from the product shelf 13, and puts it in a shopping cart or the like. A customer may decide whether or not to purchase a product after confirming various kinds of information written on the exterior (package) of the product. After shopping, the customer pays the bill (settlement process) at a billing counter provided in a part of the sales floor 11. - 特許庁

The store 10 is provided with an image input system 20 for collecting information on products that customers in the sales floor 11 are paying attention to. The image input system 20 inputs image data of images of customers in the store 10. For example, a plurality of cameras 21 (also referred to as ceiling cameras) installed on the ceiling or the like, such as the product shelf 13, are used. a camera controller 23 for controlling the cameras 21 and 22;

The camera 21 (ceiling camera) is installed for each aisle so that, for example, the entire aisle of the sales floor 11 can be covered as an imaging area. A camera 21 shoots an image of a customer who is making a purchase from above. Based on the image captured by the camera 21, the position of the customer (especially the position of the head or face), the movement of the customer to take out the product (arm, hand movement, etc.), the position of the product the customer is holding, etc. can be detected. The number and mounting locations of the cameras 21 are not particularly limited. In short, it is sufficient to use the entire aisle as a photographing area and photograph the situation of customers in the aisle. FIG. 1 shows that one camera 21 is installed in one passage.

The camera 22 (shelf-front camera) is installed from the top of the product shelf 13 toward the aisle so that, for example, the front of the product shelf 13 can be covered as an imaging area. A camera 22 takes an image of a customer looking at a product in front of the product shelf 13. - 特許庁Based on the image taken by the camera 22, the customer's face (face position, position and direction of parts such as eyes, nose, mouth, etc.), eyeball position and pupil position (line of sight), It is possible to detect the position of products that are The number and mounting locations of the cameras 22 are not particularly limited. The camera 22 is installed, for example, in each partition of the product shelf 13 and associated with the product placed in the partition. Although the camera 22 is shown only on one product shelf 13 in FIG.

In this way, by photographing an image of the situation in which the customer is looking at the product with a plurality of cameras 21 and 22, it is possible to confirm that the customer has taken the product from the product shelf 13 and that the customer has the product based on the image. can be detected. Furthermore, based on the position of the customer in the image (face, head, eyes, eyes) and the positional relationship between the product and the installation positions of the cameras 21 and 22 that took the image, the positional relationship between the product and the customer is specified. It is possible to specify which information on the exterior (package) of the product the customer is visually recognizing from the direction of the line of sight.

In this embodiment, based on the images captured by the cameras 21 and 22, it is assumed that the customer has taken out the product from the product shelf 13 and the product that the customer is holding is detected. can also be provided for detection. For example, an infrared sensor, a weight sensor, or the like may be provided on the product shelf 13 to detect that a product has been taken out from the product shelf 13 . For example, the weight sensor can detect weight changes of products placed on the product shelf 13 . That is, when the weight sensor detects that the weight has become lighter, it is determined that the customer has taken the product from the product shelf 13, and the customer has taken the product placed at the installation position of the weight sensor. It can be determined that

In the above description, a configuration example in which the cameras 21 and 22 and sensors (such as a weight sensor) are provided in order to acquire information for determining the customer's situation has been described. You may enable it to acquire various information.

The camera controller 23 connects the cameras 21 and 22 via wired communication or wireless communication and controls their operations. A camera controller 23 receives image signals captured by the cameras 21 and 22 .

The customer recognition device 31 is a device for recognizing customers who have visited the store, and is installed near the entrance of the store 10, for example. The customer recognition device 31 uses face recognition technology, for example, to capture a face image of a customer who visits the store, and identifies the customer by matching it with pre-registered feature data of the customer's face image. Alternatively, the customer recognition device 31 may be a device that reads customer-specific identification data (customer ID) stored in a membership card or mobile device such as a smartphone. The customer recognition device 31 transmits data of the recognized customer to the server 40 .

Based on the image data input through the camera controller 23, the server 40 has a line-of-sight recognition unit that recognizes the customer's line of sight, and a product recognition unit that identifies the product to be viewed by the customer, such as the one the customer is holding in his or her hand. a product posture recognition unit that recognizes the posture of the product when the customer visually recognizes the product; A product information search unit that searches the database 43B, a visual information determination unit that determines information attached to the exterior of the product viewed by the customer based on the customer's line of sight, product posture, and package information; It has a function as a recording unit for recording product visual recognition information including information determined to be viewed by a customer in a product visual recognition information database 43A, which will be described later. The server 40 implements a purchasing behavior data collection program, which will be described later, in order to realize these functions.

Although both the camera controller 23 and the server 40 are shown in FIG. 1 as being provided in the backyard 12 , they may not necessarily be provided in the backyard 12 . For example, the camera controller 23 may be provided in the sales floor 11 as long as it can transmit and receive signals to and from the cameras 21 and 22 . The server 40 may be placed under a cloud computing environment that provides computer resources via a network such as the Internet.

FIG. 2 is a block diagram showing the main circuit configuration of the server 40 in this embodiment. The server 40 includes a processor 41 , a main memory 42 , an auxiliary storage device 43 , a communication device 44 , a first I/O (Input/Output) interface 46 , a second I/O interface 47 and a system transmission line 48 . The system transmission line 48 includes an address bus, a data bus, control signal lines and the like. Server 40 connects processor 41 , main memory 42 , auxiliary storage device 43 , communication device 44 , and first and second I/O interfaces 46 and 47 to system transmission line 48 . In the server 40, a computer is composed of a processor 41, a main memory 42, an auxiliary storage device 43, and a system transmission line 48 connecting them.

The processor 41 corresponds to the central portion of the computer. The processor 41 controls each part to implement various functions of the server 40 according to an operating system or application program. The processor 41 is, for example, a CPU (Central Processing Unit).

The main memory 42 corresponds to the main memory portion of the computer. Main memory 42 includes a non-volatile memory area and a volatile memory area. The main memory 42 stores an operating system or application programs in a non-volatile memory area. The application program includes a purchasing behavior data collection program, which will be described later. The main memory 42 may store data necessary for the processor 41 to control each part in a non-volatile or volatile memory area. The main memory 42 uses a volatile memory area as a work area in which data is appropriately rewritten by the processor 41 . The non-volatile memory area is, for example, ROM (Read Only Memory). A volatile memory area is, for example, a RAM (Random Access Memory).

The auxiliary storage device 43 corresponds to the auxiliary storage portion of the computer. For example, an EEPROM (Electric Erasable Programmable Read-Only Memory), HDD (Hard Disc Drive), SSD (Solid State Drive), or the like can be the auxiliary storage device 43 . The auxiliary storage device 43 stores data used when the processor 41 performs various types of processing, data created by processing in the processor 41, and the like. The auxiliary storage device 43 may store application programs including a purchasing behavior data collection program, which will be described later.

The communication device 44 communicates with external devices through an external network (including the Internet) according to instructions from the processor 41 and receives various information. The communication device 44 is used, for example, when acquiring environmental information such as weather, temperature, and humidity from a website or inputting it from a measuring device.

A first I/O interface 46 transmits and receives data signals to and from the camera controller 23 . A second I/O interface 47 transmits and receives data signals to and from the customer recognition device 31 .

The main memory 42 or the auxiliary storage device 43 of the server 40 stores a commodity visual recognition information database 43A, a package information database 43B, a purchase database 43C, and a camera installation database 43D.

The product visible information database 43A records product visible information including information attached to the exterior (package) and various information related to the same information (see FIG. 6), which is determined to be visually recognized by the customer. The package information database 43B records package information of products sold at the store 10 . The package information includes, for example, a plurality of pieces of information attached to the packaging of the product. The information attached to the outer packaging of the product includes, for example, the product name, product image, ingredient labeling, allergens, precautions, product description, nutritional ingredients, storage method, best-before (consumption) date, and the like. The package information includes surface information indicating the surface (exterior surface) on which each piece of information is printed on the package, area information indicating the area range in which information on the surface is described, and the like. For example, if the product is box-shaped, the package information includes information indicating which information is placed in which area range for each of the six faces.

The package information also includes shape information indicating the shape of the package (outer packaging of the product) (box-shaped, bag-shaped, tray-shaped, etc.) or feature data indicating the external features of the package. Based on the shape information or feature data, the posture of the product can be detected based on the image of the product taken by the cameras 21 and 22 .

Feature data is data that is used for comparison with photographed product images, and is created based on the information (printed matter, letters, etc.) and shape (box shape, bag shape, etc.) attached to the exterior of the product. be. As the feature data, data corresponding to various postures of one product are prepared. For example, if the matching rate (similarity) between the feature data corresponding to any posture and the photographed image of the product is high, the posture corresponding to the feature data is considered to be the current posture of the product. can be discriminated.

In the purchase database 43C, payment information for products purchased by customers (for which payment processing has been completed) is recorded for each customer. The camera installation database 43D stores data indicating correspondence relationships between the plurality of cameras 21 and 22 installed in the store 10 and products arranged at locations corresponding to the installation positions of the cameras 21 and 22 .

FIG. 3 is a diagram showing an example of data recorded in the camera installation database 43D in this embodiment. As shown in FIG. 3, the camera installation database 43D includes identification information (ceiling camera IDs) of a plurality of cameras 21 (ceiling cameras), identification information (ceiling camera IDs) of the cameras 22 (shelf front cameras) installed in the photographing range of the cameras 21 ( shelf-front camera ID) and identification information (product code) of the product placed on the product shelf 13 near the position where the camera 22 (shelf-front camera) is installed are recorded in association with each other.

The ceiling camera ID and shelf front camera ID are unique codes individually assigned to the cameras 21 and 22 arranged in the sales floor 11 . Image data captured by the cameras 21 and 22 and transmitted to the camera controller 23 is assigned a camera ID set for the cameras 21 and 22 . Therefore, it is possible to specify that the product included in the images captured by the cameras 21 and 22, for example, is any of the products indicated by the product code associated with the front camera ID.

The camera installation database 43D shown in FIG. 3 indicates that a plurality of cameras 22 (shelf front cameras) are associated with one camera 21 (ceiling camera). That is, it indicates that a plurality of product shelves 13 are included in the shooting range of one ceiling camera.

A plurality of cameras 21 (ceiling cameras) may be associated with one camera 22 (shelf front camera), or the camera 21 (ceiling camera) and the camera 22 (shelf front camera) are paired. 1 may be associated. Each data is recorded in the camera installation database 43D in accordance with the number of installed cameras 21 and 22 installed in the store 10 and their installation positions.

FIG. 4 is a diagram for explaining an example of an imaging range of the cameras 21 and 22 in this embodiment.

As shown in FIG. 4, the camera 21 (ceiling camera) captures an image of a customer who is making a purchase from above. Therefore, based on the image captured by the camera 21, the position of the customer (especially the position of the head or face), the movement of the customer to take out the product (movement of the arm, hand, etc.), and the position of the product being picked up by the customer. A position etc. can be detected.

Further, as shown in FIG. 4, a camera 22 (shelf-front camera) is attached, for example, to the upper part of the product shelf 13, and by photographing toward the aisle side, the product in front of the product shelf 13 is viewed. Take a picture of the customer. Based on the image taken by the camera 22, the customer's face (face position, position and direction of parts such as eyes, nose, mouth, etc.), eyeball position and pupil position (line of sight), It is possible to detect the position of products that are

FIG. 5 is a diagram showing an example of an image captured by the camera 22 in this embodiment.

As shown in FIG. 5, an area AR1 corresponding to the customer's face, an area AR2 corresponding to the eyeball position, an area S corresponding to the product, and the like can be detected from the image captured by the camera 22. FIG.

FIG. 6 is a diagram showing an example of the data configuration of product visual recognition information recorded in the product visual recognition information database 43A in this embodiment.

The product visual recognition information is generated, for example, for each customer shopping at the store 10 .

In the product visual recognition information, data related to information attached to the product that the customer picks up and visually recognizes while shopping is recorded. For example, the product visual recognition information includes the product code of the product that is the visual target, the visual target information indicating the visual target information attached to the product, and the visual recognition time and the number of visual recognition times for the visual target information. .

In addition, as shown in FIG. 6, the product visual information includes a customer ID for identifying a customer, customer attribute information including various information about the customer corresponding to the customer ID, and when the customer makes a purchase (when the customer purchases the product). environment information (at the time of visual recognition), and purchase information indicating whether or not the product targeted for visual recognition was purchased by the customer. The customer-related information includes, for example, age, sex, purchase history, and the like. Environmental information includes, for example, weather, temperature, humidity, date and time. In addition, other information may be included.

Note that when the product visual recognition information does not include the customer attribute information and the purchase information, it is not necessary to generate it for each customer, and it is not necessary to record the customer ID.

Next, the operation of the server 40 of this embodiment will be described.

FIG. 7 is a flow chart showing the procedure of main processing executed by the processor 41 of the server 40 according to the purchase behavior data collection program in this embodiment. The functions of the server 40 will be specifically described below with reference to FIG. FIG. 7 shows, for example, the procedure of processing for one customer visiting the store, and the same processing is executed in parallel for each of a plurality of customers. In addition, the content demonstrated below is an example. The procedure and contents are not particularly limited as long as a similar function can be obtained.

The customer recognition device 31 executes processing for recognizing a customer who has visited the store, and transmits data of the recognized customer to the server 40 . The processor 41 receives data from the customer recognition device 31 through the second I/O interface 47 and executes customer recognition processing (Act 1). In the customer recognition process, for example, a customer ID for a customer visiting the store is recorded in the merchandise visual recognition information database 43A, and recording of merchandise visual recognition information corresponding to the customer visiting the store is started.

The processor 41 sequentially takes in image data of images shot by the cameras 21 and 22 from the camera controller 23 via the first I/O interface 46 in a time division manner. The processor 41 analyzes the image data every time it takes in the image data and determines whether the customer is taking out the product from the product shelf 13 .

If the processor 41 determines that the customer is making an action to take out the product from the product shelf 13 based on the image data (Act 2, Yes), the processor 41 detects the image captured by the cameras 21 and 22 in which this action is detected. (Act 3). That is, the processor 41 searches the camera installation database 43D based on the ceiling camera ID and shelf front camera ID corresponding to the image of the customer taking out the product, and determines the corresponding product code.

In the camera installation database 43D, when a plurality of product codes are associated with one shelf front camera ID, the plurality of product codes are identified. A sensor such as an infrared sensor, a weight sensor, or the like is provided at each shelf position of the product shelf 13 so that it can be detected from which shelf position of the product shelf 13 the product has been taken out. You may make it discriminate|determine a goods code.

Next, the processor 41 searches the package information database 43B based on the product code determined based on the camera installation database 43D, and acquires the package information of the product indicated by the product code (Act 4).

The processor 41 recognizes the posture of the product held by the customer based on the images of the product captured by the cameras 21 and 22 (Act 5). For example, the processor 41 compares the feature data of the product included in the package information with the image of the product in the photographed image, and the matching rate (similarity) of the feature data corresponding to any posture is high. determine whether

When the processor 41 has acquired package information corresponding to a plurality of product codes, the processor 41 compares each package information with the feature data of the product. As a result, a product corresponding to feature data with a high collation rate (similarity) can be specified as a product owned by the customer.

The processor 41 recognizes the orientation of the product based on the orientation of the product determined based on the matching result and the positional relationship between the installation positions of the cameras 21 and 22 that captured the images. By recognizing the posture of the product, it is possible to identify which side of the product faces the customer, that is, the exterior surface of the product that the customer visually recognizes.

Note that the processor 41 may similarly perform the processing for recognizing the orientation of each of the images captured by the cameras 21 and 22, or may perform the processing on one of the images. may be executed.

Also, the processor 41 may recognize the orientation of the product using shape information indicating the shape of the package (exterior of the product) instead of using the feature data.

Next, the processor 41 recognizes the line of sight of the customer based on the images captured by the cameras 21 and 22 (Act 6). For example, as shown in FIG. 5, the processor 41 detects an area AR1 corresponding to the customer's face and an area AR2 corresponding to the eyeball position from the image of the customer captured by the camera 22. FIG. Further, the processor 41 detects the positions of parts such as the eyes, nose, mouth, and eyebrows of the customer from the image of the area AR1, and determines the direction of the customer's face from the positional relationship with the installation position of the camera 22 that captured the image. presume. Furthermore, the processor 41 detects the position of the pupil of the eyeball from the image of the area AR2, and estimates the line-of-sight direction based on the orientation of the face and the installation position of the camera 22 .

The processor 41 determines the exterior information (visually recognized object information) visually recognized by the customer based on the posture of the product, the position and line-of-sight direction of the customer, and the package information (Act 7).

FIG. 8 is a diagram showing area ranges OJ1 to OJ7 corresponding to a plurality of pieces of information indicated by package information and attached to a certain surface of a product (for example, the back surface of a box-shaped package). On the back of the package, for example, the product name (OJ1), ingredients (OJ2), allergens (OJ3), precautions (OJ4), product description (OJ5), nutritional ingredients (OJ6), storage method (OJ7), etc. Information is provided. In the package information, area information indicating an area range corresponding to each piece of information is defined.

The processor 41 identifies which of the area ranges OJ1 to OJ7 indicated by the area information includes the line-of-sight direction of the customer, measures the viewing time for each area range (viewing target information), and calculates the time. Accumulate and record (Act 8). That is, the processor 41 records the viewing time in association with the surface information and area information indicating the viewing target information. In addition, it may be determined that the line of sight is continuously included in the area for a preset time (for example, 1 second) or more, or simply the line of sight is included in the area. You may make it accumulate time.

As described above, the processor 41 repeatedly executes the recognition of the posture of the product, the recognition of the line of sight, the determination of the information to be viewed, and the recording of the viewing time until it is determined that the viewing of the product has ended (Act 9, No→ Acts 5-8).

FIG. 9 shows how customers visually recognize products. When the customer confirms the information by holding the product in hand, the customer turns the surface to be visually recognized toward the face side, directs the direction of the line of sight to the interesting information described on the surface, and confirms the contents. If the user is interested in multiple pieces of information, he or she changes the line of sight to the target information and confirms the content. In addition, the customer checks the information written on each surface by changing the posture of the product so as to change the surface to be visually recognized, such as the front, back, and side, in addition to the specific surface. The processor 41 executes processing corresponding to such change of the visual recognition target information by the customer, and records the visual recognition time for each visual recognition target information.

In addition, although the recording of the viewing time for the viewing target information has been described in the above description, the number of viewing times for the viewing target information can be recorded in the same manner. For example, the processor 41 counts the number of times of visual recognition when the same product visual recognition information becomes a visual recognition target again after a certain period of time has elapsed after visual recognition of any visual recognition target information. That is, when reviewing the same information many times, it is likely that the information is of particular interest to the customer, so the number of viewing times is recorded together with the viewing time.

When it is determined that the customer has finished viewing the product (Act 9, No), the processor 41 records the corresponding product visual recognition information of the product to be viewed in the product visual recognition information database 43A. The end of the visual recognition of the product is detected when, for example, the customer has moved out of the shooting range of the camera 22, the product has been returned to the product shelf 13, or the product has not been visually recognized for a certain period of time or longer. .

As shown in FIG. 6, the processor 41 associates the product code of the target product, and records the product visibility information, viewing time, and number of times of viewing in association with each other (Act 10). When the customer pays attention to a plurality of pieces of information attached to the product, the viewing time and the number of times of viewing corresponding to each piece of product visual recognition information are recorded in association with each other.

When the processor 41 detects that the customer moves to another product shelf 13 and similarly picks up another product (Act 2, Yes) to check the information recorded on the exterior. (Acts 3 to 9), in the same manner as described above, the product visual recognition information of the target product is recorded in the product visual recognition information database 43A (Act 10).

In this way, by including the viewing time and the number of viewing times for each piece of information attached to the exterior of the product in the product viewing information, it is possible to collect data for estimating the degree of interest of the customer for each piece of information.

When the customer's purchase behavior at the store 10 is completed and accounting (settlement processing) is carried out at the checkout counter, settlement information on the product purchased by the customer is recorded in the purchase database 43C (Act 11, Yes).

The processor 41 matches the product purchased by the customer indicated by the payment information recorded in the purchase database 43C with the product in which the customer is interested and indicated by the product visual recognition information recorded in the product visual recognition information database 43A, and visually recognizes the product. To discriminate which product is actually purchased among products whose information is recorded. The processor 41 adds purchase information indicating that the product has been purchased to the product visual information in which, for example, the product code corresponding to the purchased product determined by matching is recorded (Act 12).

In this way, by adding the purchase information to the merchandise visual information, it is possible to collect data for estimating the relationship between the merchandise for which the customer visually recognizes the information recorded on the exterior and the actually purchased merchandise.

In addition, the processor 41 acquires customer attribute information corresponding to the customer ID recorded in the product visual recognition information from, for example, a member database and adds it to the product visual recognition information. Furthermore, the processor 41 adds environmental information including information on weather, temperature, humidity, etc. and information on date and time acquired from an external device through the communication device 44 to the commodity visual recognition information (Act 13).

In this way, by adding customer attribute information and environmental information to product visibility information, it is possible to collect data that can be used to estimate what kind of customer will pay attention to what information about the product under what kind of environment. becomes.

In the above description, when the customer picks up the product, the product is viewed as the target, but the product may be viewed when the customer does not pick up the product. For example, by specifying the position of the product placed on the product shelf 13, for example, based on the position of each shelf of the product shelf 13 and the shape information of the product, etc., the line-of-sight direction of the customer is detected as described above, It is possible to determine which information of the product is visually recognized.

In this way, the purchasing behavior data collection system in this embodiment recognizes where and how much the customer is looking at the product, and by specifying and recording the information on the package that the customer is interested in, It is possible to collect data indicating what information customers are interested in regarding products. By collecting these data, it becomes possible to more specifically estimate the reasons why each product sells or does not sell.

While several embodiments of the invention have been described, these embodiments have been presented by way of example and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, replacements, and modifications can be made without departing from the scope of the invention. These embodiments and modifications thereof are included in the scope and gist of the invention, and are included in the scope of the invention described in the claims and equivalents thereof.

Further, the processing described in the above-described embodiments can be performed by using a recording medium such as a magnetic disk (flexible disk, hard disk, etc.), an optical disk (CD-ROM, DVD, etc.), a semiconductor memory, etc. as a program that can be executed by a computer. can be written to and provided to various devices. Moreover, it is also possible to transmit it by a communication medium and provide it to various devices. The computer reads a program recorded on a recording medium or receives the program via a communication medium, and executes the above-described processes by being controlled by the program.

DESCRIPTION OF SYMBOLS 10... Store, 11... Sales floor, 12... Backyard, 13... Product shelf, 20... Image input system, 21, 22... Camera, 23... Camera controller, 31... Customer recognition device, 40... Server, 41... Processor, 42 Main memory 43 Auxiliary storage device 43A Merchandise visual recognition information database 43B Package information database 43C Purchasing database 43D Camera installation database 44 Communication device 46, 47 I/O interface.

Claims (6)

a line-of-sight recognition unit that recognizes the line of sight of the customer;
a product recognition unit that identifies a product to be visually recognized by a customer;
a product posture recognition unit that recognizes the posture of a product;
a product information search unit that searches for package information including a plurality of pieces of information attached to the exterior of the specified product;
Based on the customer's line of sight, product orientation, and package information, the visual information discriminating unit discriminates the information attached to the exterior of the product that the customer is visually recognizing, and the information that is discriminated as being visually recognized by the customer. a purchasing behavior data collection system having an information recording unit for recording product visibility information including; The visual recognition information determination unit determines the visual recognition time for each information determined to be visually recognized by the customer,
2. The purchasing behavior data collection system according to claim 1, wherein the information recording unit records the product visual recognition information associated with the information determined to be visually recognized by the customer and the visual recognition time. further comprising customer identification means for identifying the customer,
3. The purchasing behavior data collection system according to claim 1, wherein said information recording unit records said product visual recognition information including customer attribute information relating to said customer. 2. The information recording unit acquires purchase information relating to a product purchased by a customer, and records the product visual recognition information including the purchase information in accordance with the product corresponding to the information included in the product visual recognition information. 4. The purchase behavior data collection system according to any one of 1 to 3. further comprising acquisition means for acquiring environmental information in a situation in which the product is visually recognized by the customer;
5. The purchasing behavior data collection system according to any one of claims 1 to 4, wherein the information recording unit records the product visual recognition information including the environment information. the computer,
a line-of-sight recognition unit that recognizes the line of sight of the customer;
a product recognition unit that identifies a product to be visually recognized by a customer;
a product posture recognition unit that recognizes the posture of a product;
a product information search unit that searches for package information including a plurality of pieces of information attached to the exterior of the specified product;
Based on the customer's line of sight, product orientation, and package information, the visual information discriminating unit discriminates the information attached to the exterior of the product that the customer is visually recognizing, and the information that is discriminated as being visually recognized by the customer. Purchasing behavior data collection program for functioning as an information recording unit for recording product visibility information including;