JP2023120769A - Information processing system, information processing method, and program - Google Patents

Abstract

To allow for generating information indicative of consumer behavior in the process of purchasing products.SOLUTION: An information processing system is provided, comprising an acquisition unit configured to acquire shelf-front behavior information indicative of behavior of consumers near a product shelf having products on display thereon and register-front behavior information indicative of behavior of the consumers purchasing products; and a generation unit configured to generate purchasing behavior information obtained by associating the shelf-front behavior information and the register-front behavior information.SELECTED DRAWING: Figure 2

Description

The present invention relates to an information processing system, an information processing method, and a program.

There is a need to utilize product purchase data for product development and sales promotion. For example, Patent Literature 1 discloses a technology for obtaining approval from consumers and providing purchase data of products settled by a POS (Point Of Sales) terminal to an external trader who is a third party.

JP 2021-108213 A

However, in the process of purchasing a product, many consumers look at the product and consider this or that. For business operators who manufacture products, it is preferable to know the behavior of consumers in the process of purchasing products in order to improve and develop products. In addition, it is expected that distributors will be able to make more precise sales promotion plans based on the behavior of consumers in the process of purchasing products.

The present invention has been made in view of such circumstances, and provides an information processing system, an information processing method, and a program that can generate information that shows consumer behavior in the process of purchasing a product. With the goal.

The information processing system of the present invention provides front-of-shelf behavior information indicating the behavior of a consumer near a product shelf on which products are displayed, and pre-checkout behavior information indicating the consumer's behavior of purchasing products. An acquisition unit that acquires information, and a generation unit that generates purchase behavior information in which the behavior information before the shelf and the behavior information before the checkout are associated with each other.

The information processing method of the present invention is an information processing method performed by a computer that is an information processing server, wherein the acquisition unit indicates the behavior of a consumer in the vicinity of a product shelf on which products are displayed. Information and pre-checkout behavior information indicating the behavior of the consumer to purchase a product are acquired, and a generating unit generates purchase behavior information in which the behavior information before the shelf and the behavior information before the checkout are associated with each other. .

The program of the present invention provides a computer, which is an information processing server, with front-of-shelf behavior information indicating the behavior of a consumer near a product shelf on which products are displayed, and the product purchase behavior of the consumer. is acquired, and purchase behavior information is generated by associating the behavior information before the shelf with the behavior information before the checkout.

According to the present invention, it is possible to generate information that shows consumer behavior in the process of purchasing a product.

It is a figure showing an example of composition of information processing system 1 concerning an embodiment. It is a block diagram showing an example of composition of information processing server 10 concerning an embodiment. It is a block diagram which shows the example of a structure of the customer service terminal 30 which concerns on embodiment. It is a figure which shows the example of the external appearance of the customer service terminal 30 which concerns on embodiment. It is a figure explaining the example of the question displayed on the customer service terminal 30 which concerns on embodiment. 4 is a sequence diagram showing the flow of processing performed by the information processing system 1 according to the embodiment; FIG. 4 is a sequence diagram showing the flow of processing performed by the information processing system 1 according to the embodiment; FIG.

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments of the present invention will be described with reference to the drawings.

FIG. 1 is a block diagram showing a configuration example of an information processing system 1 according to an embodiment. The information processing system 1 includes, for example, an information processing server 10 , a shelf front camera 20 , a customer service terminal 30 , a cash register front camera 40 , a POS terminal 50 , and a business operator server 60 .

The information processing server 10 is a server device managed by a business operator that provides a service of providing purchasing behavior information. Purchase behavior information is consumer purchase data that includes consumer behavior in the process of purchasing a product. The information processing server 10 acquires the front-of-shelf image data captured by the front-of-shelf camera 20, the question-and-answer data acquired by the customer service terminal 30, the front-of-checkout image data captured by the front-of-checkout camera 40, and the POS terminal 50. receive each of the POS data received. The information processing server 10 transmits information associated with the received data to the business operator server 60 as purchasing behavior information. The information processing server 10 performs wireless or wired communication with each of the front camera 20, the customer service terminal 30, the front camera 40, the POS terminal 50, and the provider server 60 via a communication network or the like.

The shelf front camera 20 is a camera that is installed around the product shelf SD and captures an image of a predetermined area in the vicinity of the product shelf SD. The front-of-shelf camera 20 captures, for example, the behavior of a consumer who considers various products while looking at the product in front of the product shelf SD. The shelf front camera 20 may capture a moving image or may capture a still image. The front-of-shelf camera 20 transmits the image information of the captured image to the information processing server 10 as front-of-shelf image data. The shelf front camera 20 performs wireless or wired communication with the information processing server 10 via a communication network or the like.

The customer service terminal 30 is a terminal device installed on the product shelf SD. The customer service terminal 30 asks a prepared question to the consumer in front of the product shelf SD by displaying an image or outputting a sound. The customer service terminal 30 acquires an answer to a question posed to the consumer by, for example, collecting the consumer's voice with a microphone. The customer service terminal 30 transmits information indicating an answer to the acquired question to the information processing server 10 as question answer data. The service terminal 30 performs wireless communication or wired communication with the information processing server 10 via a communication network or the like.

The pre-register camera 40 is a camera that is installed around the register and captures an image of a predetermined area in the vicinity of the register. The pre-checkout camera 40 captures, for example, the behavior of a consumer who purchases a product in front of a cash register. The pre-register camera 40 may capture a moving image or may capture a still image. The pre-checkout camera 40 transmits the image information of the captured image to the information processing server 10 as pre-checkout image data. The front-of-checkout camera 40 performs wireless or wired communication with the information processing server 10 via a communication network or the like.

The POS terminal 50 is a settlement terminal installed at a cash register. The POS terminal 50 acquires sales information of products purchased by consumers via a bar code reader or the like. The POS terminal 50 transmits the acquired product sales information to the information processing server 10 as POS data. The POS terminal 50 performs wireless or wired communication with the information processing server 10 via a communication network or the like.

The provider server 60 is a server device managed by the provider. The business operator here is a business operator that receives purchasing behavior information in a service that provides purchasing behavior information, for example, a product manufacturer or a buyer that purchases products in distribution. The business server 60 receives the purchasing behavior information transmitted from the information processing server 10 . The provider server 60 performs wireless or wired communication with the information processing server 10 via a communication network or the like.

FIG. 2 is a block diagram showing a configuration example of the information processing server 10 according to the embodiment. The information processing server 10 is a computer. As the information processing server 10, for example, a PC (Personal Computer), a server device, a cloud, or the like can be applied.

The information processing server 10 includes, for example, a communication unit 11, a storage unit 12, and a control unit 13. The communication unit 11 communicates with each of the front camera 20 , the customer service terminal 30 , the front camera 40 , the POS terminal 50 , and the company server 60 .

The storage unit 12 is a storage medium such as a HDD (Hard Disk Drive), flash memory, EEPROM (Electrically Erasable Programmable Read Only Memory), RAM (Random Access read/write Memory), ROM (Read Only Memory), or a combination thereof. Consists of The storage unit 12 stores programs for executing various processes of the information processing server 10 and temporary data used when performing various processes.

The control unit 13 includes an acquisition unit 130, a determination unit 131, a generation unit 132, and an output unit 133, for example. Each of these functional units provided in the control unit 13, that is, the acquisition unit 130, the determination unit 131, the generation unit 132, and the output unit 133, loads a program into a CPU (Central Processing Unit) provided as hardware in the information processing server 10. It is realized by executing

Acquisition unit 130 acquires various types of information. For example, the acquisition unit 130 acquires image information captured by the front-of-shelf camera 20 as front-of-shelf image data via the communication unit 11 . The acquisition unit 130 acquires the question-and-answer data acquired by the customer service terminal 30 via the communication unit 11 . The acquiring unit 130 acquires image information captured by the pre-checkout camera 40 as pre-checkout image data via the communication unit 11 . The acquisition unit 130 acquires the POS data acquired by the POS terminal 50 via the communication unit 11 . Acquisition unit 130 causes storage unit 12 to store each of the acquired data.

The determination unit 131 determines whether the person imaged in the front-of-shelf image data and the person indicated in the question-and-answer data (the person who answered the question) are the same person. For example, based on the date and time when the question-and-answer data was obtained, the determination unit 131 determines the front-of-shelf image data captured during a predetermined time period including the same date and time, for example, 10 minutes before and after the date and time when the question-and-answer data was obtained. to the question-and-answer data is stored in the storage unit 12 as shelf behavior information.

The determination unit 131 determines whether the person captured in the pre-checkout image data and the person indicated in the POS data (the person who purchased the product) are the same person. For example, based on the date and time when the POS data was obtained, the determination unit 131 selects pre-checkout image data captured during a predetermined time period including the same date and time, for example, two minutes before and after the date and time when the question-and-answer data was obtained. Information associated with the POS data is stored in the storage unit 12 as pre-checkout behavior information.

The determination unit 131 determines whether the person imaged in the front-of-shelf image data and the person imaged in the pre-register image data are the same person.

For example, the determination unit 131 extracts a purchase scene in which a consumer purchases a product from the pre-checkout imaging data. The determination unit 131 assigns an ID (referred to as a purchase ID) that specifies the purchase scene to each purchase scene extracted from the pre-checkout imaging data.
Next, for each of the extracted purchase scenes, the acquisition unit 130 acquires a time period that is a predetermined time period before the purchase scene, for example, 30 hours before the purchase scene imaging start time until the purchase scene imaging start time. The imaged data in front of the shelf imaged during the time period of is acquired as the determination target scene.
Next, the determination unit 131 identifies a person imaged in the determination target scene, and determines whether or not the identified person and the person purchasing the product in the purchase scene are the same person. The determining unit 131 determines the shape-based features, such as hairstyles, the presence or absence of glasses, clothes, physiques, etc., and the color-based features, such as hair color, skin color, and clothing color. It is determined that they are the same person. On the other hand, the determination unit 131 determines that the person is not the same person when the feature based on the shape or the feature based on the color is not similar.
If the determining unit 131 determines that the person imaged in the purchase scene and the person imaged in the determination target scene are the same person, the determination unit 131 adds an ID associated with the purchase ID assigned to the purchase scene to the determination target scene. (referred to as a determination target ID) is given.
The determination unit 131 causes the storage unit 12 to store information in which the purchase scene is associated with the purchase ID assigned to the purchase scene. In addition, the storage unit 12 stores information in which the determination target scene is associated with the determination target ID given to the determination target scene.

The generating unit 132 generates purchasing behavior information. For example, the generation unit 132 generates, as purchasing behavior information, information in which the same person's behavior information before the shelf and behavior information before the checkout are associated with each other. The front-of-shelf behavior information may be composed of only information on either the front-of-shelf image data or the question-and-answer data. The pre-checkout behavior information may be composed of either pre-checkout imaging data or POS data only.

For example, when generating purchase behavior information that associates behavior information before a shelf and behavior information before a checkout for the same person, the generation unit 132 generates a , information associated with shelf behavior information having a determination target scene to which a determination target ID linked to the purchase ID is assigned, and the generated information is used as purchase behavior information.
As a result, the purchase behavior information in which the same person's behavior information before the shelf and the behavior information before the cash register are associated with each other, that is, the behavior of the same person around the product shelf, the answer obtained from the person, and the purchase behavior information are obtained. Information in which the appearance and sales information of the purchased product are associated can be used as purchasing behavior information.

For example, when generating purchase behavior information in which the same person's front-of-shelf image data and pre-checkout behavior information are associated with each other, the generation unit 132 generates a , and generates information in which the determination target scene to which the determination target ID associated with the purchase ID is assigned is associated, and the generated information is used as purchasing behavior information. As a result, the information in which the behavior of the same person around the product shelf, the state of purchasing the product, and the sales information of the purchased product are associated with each other can be used as the purchasing behavior information.

For example, when generating purchase behavior information in which question-and-answer data and pre-checkout behavior information of the same person are associated, the generation unit 132, based on the purchase ID assigned to the purchase scene included in the pre-checkout behavior information, Information associated with the question-and-answer data included in the front-of-shelf behavior information having the determination target scene to which the determination target ID associated with the purchase ID is assigned is generated, and the generated information is used as purchase behavior information. As a result, information in which an answer obtained from the same person, how the person purchases the product, and sales information of the purchased product are associated with each other can be used as purchasing behavior information.

Similarly, the generating unit 132 may generate, as purchasing behavior information, information in which the front-of-shelf behavior information and the pre-checkout imaging data of the same person are associated with each other. The generation unit 132 may generate, as the purchase behavior information, information in which the front-of-shelf image data and the front-of-the-cash register image data of the same person are associated with each other. The generation unit 132 may generate, as the purchase behavior information, information in which the question-answer data and the pre-checkout imaging data of the same person are associated with each other.
Further, the generation unit 132 may generate, as purchasing behavior information, information in which shelf behavior information and POS data of the same person are associated with each other. The generation unit 132 may generate, as the purchase behavior information, information in which the front-of-shelf image data and the POS data of the same person are associated with each other. The generating unit 132 may generate, as the purchasing behavior information, information in which the question-answer data and the POS data of the same person are associated with each other.

In addition, the generating unit 132 may generate shelf behavior information in accordance with a request from a business operator who provides the shelf behavior information. For example, the generating unit 132 generates, as purchasing behavior information, information that associates the position where the product is displayed on the product shelf with pre-checkout behavior information (which may be either pre-checkout imaging data or POS data). You may make it In this case, the generation unit 132 extracts the display scene in which the product shelf on which the product is displayed is captured from the front-of-shelf image data, and acquires the position where the product is displayed from the extracted display scene. The generation unit 132 associates the acquired position with pre-checkout action information including pre-checkout image data captured on the same day as the display scene was captured and POS data of products purchased on the same day. to generate shelf-front behavior information.

The generating unit 132 may generate purchasing behavior information for each product manufacturer or type. For example, the generation unit 132 generates purchasing behavior information in which the behavior information before the shelf and the behavior information before the checkout are associated with each manufacturer or type of product. Alternatively, the generating unit 132 generates purchasing behavior information by associating the position where the merchandise is displayed on the merchandise shelf with the pre-checkout behavior information for each manufacturer or type of merchandise.

By generating the purchase behavior information for each product maker, it is possible to provide business operators corresponding to the product maker with information that shows the process by which the consumer purchased the product of that maker. By generating purchasing behavior information for each product type, it is possible to provide buyers who purchase products with information that allows them to understand the process of what types of products are being purchased by consumers. .

The generating unit 132 causes the storage unit 12 to store the generated purchasing behavior information.
The output unit 133 outputs the purchase behavior information stored in the storage unit 12 and transmits the output purchase behavior information to the business operator server 60 via the communication unit 11 .

Here, the customer service terminal 30 will be described with reference to FIGS. 3 and 4. FIG. FIG. 3 is a block diagram showing a configuration example of the customer service terminal 30 according to the embodiment. FIG. 4 is a diagram showing an example of the appearance of the customer service terminal 30 according to the embodiment.

The reception terminal 30 is a computer. As the customer service terminal 30, for example, a tablet terminal, a PC, or the like can be applied. Further, as shown in FIG. 4, the customer service terminal 30 may have a shape that allows the consumer to view the screen or listen to the audio guide while being installed on the shelf.
As shown in FIG. 3, the customer service terminal 30 includes, for example, a communication unit 31, a storage unit 32, a control unit 33, a display unit 34, a microphone unit 35, a speaker unit 36, and a printer unit 37. .
The communication unit 31 communicates with the information processing server 10 .
The storage unit 32 is configured by a storage medium such as HDD, flash memory, EEPROM, RAM, ROM, or a combination thereof. The storage unit 32 stores a program for executing various processes of the customer service terminal 30 and temporary data used when performing various processes.

The control unit 33 is implemented by causing a CPU included as hardware in the customer service terminal 30 to execute a program. The control unit 23 controls the communication unit 31 , storage unit 32 , display unit 34 , microphone unit 35 , speaker unit 36 and printer unit 37 .

The display unit 34 includes a display device such as a liquid crystal display, for example, and displays an image under the control of the control unit 23 .
The microphone unit 35 includes, for example, a sound collector, collects surrounding sounds, converts the collected sounds into digital data, and outputs the converted data to the control unit 23 .
The speaker unit 36 includes, for example, a sound output device, and emits a predetermined sound under the control of the control unit 33 .
The printer unit 37 includes, for example, a printing device, and prints and outputs predetermined data under the control of the control unit 33 .

As shown in FIG. 4, the customer service terminal 30 is provided with a display section 34 at an angle so that the consumer can easily see it when it is installed on a shelf. A microphone section 35 , a speaker section 36 and a printer section 37 are provided beside the display section 34 . Since the display unit 34, the microphone unit 35, the speaker unit 36, and the printer unit 37 are provided on the same surface, it is easy for the consumer in front of the shelf to listen to the voice and answer the question while watching the display on the screen. becomes.

FIG. 5 is a diagram illustrating an example of questions displayed on the customer service terminal 30 according to the embodiment. In the example of this figure, an example in which questions are displayed on the display section 34 is schematically shown. Here, an example will be described in which a question is displayed on the customer service terminal 30 provided on a product shelf where lotions are displayed.

For example, when a consumer approaches a product shelf, the speaker unit 36 outputs a voice such as "Daily skin care is important" and displays "We will select a lotion that suits your skin." Please answer the question that was asked." is output. Then, a question having multiple options as shown in FIG. 5 is displayed. In the example shown in this figure, options such as "situation of use of lotion", "condition of skin", and "worries about lotion" are displayed.

According to the displayed contents, the consumer verbally conveys the options corresponding to the matters to be consulted about the lotion. A voice uttered by the consumer is collected by the microphone section 35 and output to the control section 33 . The control unit 33 performs voice recognition of the voice collected by the microphone unit 35 and determines which option the consumer has selected.
Alternatively, the consumer selects an option corresponding to the matter that the consumer wants to consult about the lotion by performing a touch operation on the displayed option. In this case, information indicating options selected by the consumer is output to the control unit 33 via a touch panel (not shown). The control unit 33 determines which option the consumer has selected according to the acquired information.

The control unit 33 causes the display unit 34 to display a screen corresponding to the selected selection according to the determination result. For example, when an option indicating "condition of use of lotion" is selected, a screen with options corresponding to a plurality of conditions of use is displayed so that the consumer can select the corresponding condition of use. When the consumer selects one of the options, the lotion (product) recommended to the consumer is displayed as the selection result.

In this case, the control unit 33 may display the lotion recommended to the consumer and advice regarding the recommended lotion. For example, it is possible to advise a recommended method of use, advise a milky lotion to be combined with lotion as a recommended combination, advise how to select the volume (size) of lotion, and the like.

6 and 7 are sequence diagrams showing the flow of processing performed by the information processing system 1. FIG. FIG. 6 shows the flow of processing for generating shelf-front behavior information. FIG. 7 shows the flow of processing for generating pre-checkout behavior information and further generating pre-shelf behavior information.

As shown in FIG. 6, the front-of-shelf camera 20 captures an image of an area (area in front of the shelf) near the product shelf (step S10). The front-of-shelf camera 20 transmits captured image information to the information processing server 10 as front-of-shelf image data. The information processing server 10 receives the front-of-shelf image data, and stores the received front-of-shelf image data in the storage unit 12 (step S11).

In this case, the shelf front camera 20 may output image information captured by the front camera 20 to the service terminal 30 . In this case, the customer service terminal 30 determines whether or not to start customer service based on the image information received from the shelf front camera 20 (step S12). For example, when the received image information shows an image of a person approaching, the customer service terminal 30 determines that the consumer has approached the product shelf and starts customer service. When starting customer service, the customer service terminal 30 outputs a voice such as "Daily skin care is important" (step S13). Then, the customer service terminal 30 displays a question having a plurality of options as shown in FIG. 5 (step S14). When the consumer answers the question, the customer service terminal 30 acquires a voice or the like indicating the answer (step S15). The customer service terminal 30 transmits information indicating the acquired answer to the information processing server 10 as question answer data. The information processing server 10 receives the question-and-answer data, and stores the received question-and-answer data in the storage unit 12 (step S16).

The information processing server 10 causes the storage unit 12 to store information in which the front-of-shelf image data and the question-and-answer data are associated as front-of-shelf behavior information (step S17).

As shown in FIG. 7, the pre-registration camera 40 captures an area (pre-registration area) in the vicinity of the cash register (step S20). The pre-checkout camera 40 transmits captured image information to the information processing server 10 as pre-checkout image data. The information processing server 10 receives the pre-checkout captured data, and stores the received pre-checkout captured data in the storage unit 12 (step S21).

On the other hand, the POS terminal 50 acquires the sales information of the product purchased at the register (step S22). The POS terminal 50 transmits the acquired sales information to the information processing server 10 as POS data. The information processing server 10 receives the POS data and stores the received POS data in the storage unit 12 (step S23).

The information processing server 10 causes the storage unit 12 to store information in which the pre-checkout imaging data and the POS data are associated as pre-checkout action information (step S24).
The information processing server 10 also generates shelf behavior information (step S25). For example, the information processing server 10 generates, as shelf behavior information, information in which shelf behavior information is associated with cash register behavior information of the same person.

As described above, the information processing system 1 of the embodiment includes the acquisition unit 130 , the storage unit 12 and the generation unit 132 . The acquisition unit 130 acquires behavior information before the shelf and behavior information before the checkout. The front-of-shelf behavior information is information indicating the behavior of consumers in the vicinity of product shelves on which products are displayed. The pre-checkout behavior information is information indicating the consumer's purchase behavior. The storage unit 12 stores behavior information before the shelf and behavior information before the checkout. The generation unit 132 generates purchasing behavior information in which the behavior information before the shelf and the behavior information before the checkout are associated with each other.

Thereby, in the information processing system 1 of the embodiment, it is possible to generate, as purchasing behavior information, information in which the consumer's behavior in the vicinity of the product shelf and the information regarding the purchased product are associated with each other. Therefore, it is possible to generate information that shows the consumer's behavior in the process of purchasing the product.

Further, in the information processing system 1 of the embodiment, the acquisition unit 130 acquires front-of-shelf image data as front-of-shelf behavior information. As a result, in the information processing system 1 of the embodiment, it is possible to acquire the image information in which the behavior of the consumer in the vicinity of the product shelf is captured as the front-of-shelf behavior information. It is possible to generate information that makes it possible to determine whether the user is taking action.

Further, in the information processing system 1 of the embodiment, the acquisition unit 130 acquires pre-checkout imaging data as pre-checkout behavior information. As a result, in the information processing system 1 of the embodiment, it is possible to acquire image information in which a consumer's behavior of purchasing a product is captured as pre-checkout behavior information, and what kind of product the consumer purchased can be obtained. can generate recognizable information.

Further, in the information processing system 1 of the embodiment, the acquiring unit 130 acquires the front-of-shelf image data as the front-of-shelf behavior information, and acquires the front-of-the-checkout image data as the front-of-the-checkout behavior information. The generating unit 132 generates, as purchasing behavior information, information that associates the front-of-shelf image data and the front-of-the-checkout image data in which the same consumer is imaged. As a result, the information processing system 1 of the embodiment can generate information that indicates what kind of product the consumer has purchased and what kind of behavior he/she is taking in front of the product shelf.

Further, in the information processing system 1 of the embodiment, the acquisition unit 130 acquires the question-answer data as the front-of-shelf behavior information. As a result, the information processing system 1 of the embodiment can generate information indicating what kind of interest the consumer in front of the product shelf has in the product.

Further, in the information processing system 1 of the embodiment, the acquisition unit 130 acquires POS data as pre-checkout behavior information. As a result, the information processing system 1 of the embodiment can acquire sales information about the product purchased by the consumer.

Further, in the information processing system 1 of the embodiment, the acquisition unit 130 acquires the position where the product is displayed on the product shelf from the front-of-shelf image data. The generating unit 132 generates, as purchasing behavior information, information in which the position where the product is displayed on the merchandise shelf and the pre-checkout behavior information are associated with each other. As a result, the information processing system 1 of the embodiment can generate information indicating the relationship between the display position of the product and whether or not the product has been purchased.

Further, in the information processing system 1 of the embodiment, the generation unit 132 purchases information that associates the position where the product is displayed on the product shelf with the pre-checkout behavior information for each manufacturer or type of product displayed on the product shelf. Generate as action information. As a result, in the information processing system 1 of the embodiment, it is possible to generate information indicating which manufacturer's product is displayed and where the product is purchased. In addition, it is possible to generate information indicating what type of product is displayed and where the product is purchased.

In the above-described embodiment, the case where the information in which the front-of-the-shelf image data and the front-of-the-cash register image data in which the same person is imaged are associated is generated as the purchasing behavior information has been exemplified and explained. However, it is not limited to this. The generation unit 132 may generate purchase behavior information in which at least the behavior in front of the shelf and the purchase record are associated with each other. For example, the generation unit 132 may generate, as purchasing behavior information, information in which the front-of-shelf image data and the front-of-the-checkout image data of consumers having the same attribute (gender and/or age) are associated with each other.
For example, the determination unit 131 extracts a purchase scene in which a consumer purchases a product from the pre-checkout imaging data. The determination unit 131 estimates an attribute of a consumer who makes a purchase in each purchase scene extracted from the pre-checkout imaging data, and assigns an ID (attribute ID) according to the estimation result. As a method for estimating a consumer's affiliation, an image analysis method such as face recognition technology can be used.
Next, the acquiring unit 130 extracts a front-of-shelf scene in which the consumer is in front of the product shelf from the front-of-shelf image data. The determining unit 131 estimates an attribute of a consumer present in each shelf front scene extracted from the front shelf image data, and assigns an attribute ID according to the estimation result.
Then, the determination unit 131 generates information in which the attribute ID of the person captured in the purchase scene is associated with the front-of-shelf scene to which the same attribute ID is assigned, and uses the generated information as front-of-shelf behavior information.

Further, in the above-described embodiment, a case in which a question prepared in advance is asked to the consumer has been exemplified and explained. However, it is not limited to this. The customer service terminal 30 may ask at least what kind of interest the consumer has in the product or asks a question to understand the consumer's purchasing process. For example, AI (artificial intelligence) that has learned the correspondence between questions from consumers and answers to those questions may be used to interact with consumers by machine learning techniques.

All or part of the information processing system 1, the information processing server 10, and the customer service terminal 30 in the above-described embodiment may be realized by a computer. In that case, a program for realizing this function may be recorded in a computer-readable recording medium, and the program recorded in this recording medium may be read into a computer system and executed. It should be noted that the "computer system" referred to here includes hardware such as an OS and peripheral devices. The term "computer-readable recording medium" refers to portable media such as flexible discs, magneto-optical discs, ROMs and CD-ROMs, and storage devices such as hard discs incorporated in computer systems. Furthermore, "computer-readable recording medium" refers to a program that dynamically retains programs for a short period of time, like a communication line when transmitting a program via a network such as the Internet or a communication line such as a telephone line. It may also include something that holds the program for a certain period of time, such as a volatile memory inside a computer system that serves as a server or client in that case. Further, the program may be for realizing a part of the functions described above, or may be capable of realizing the functions described above in combination with a program already recorded in the computer system, It may be implemented using a programmable logic device such as FPGA.

Although the embodiment of the present invention has been described in detail with reference to the drawings, the specific configuration is not limited to this embodiment, and design and the like are included within the scope of the gist of the present invention.

DESCRIPTION OF SYMBOLS 1... Information processing system 10... Information processing server 130... Acquisition part 131... Judgment part 132... Generation part 20... Shelf front camera 30... Customer service terminal 40... Front cash register camera 50... POS terminal 60... Provider server

Claims (10)

an acquisition unit that acquires front-of-shelf behavior information indicating the behavior of a consumer near a product shelf on which products are displayed, and pre-checkout behavior information indicating the product-purchasing behavior of the consumer;
An information processing system, comprising: a generation unit that generates purchase behavior information in which the behavior information before the shelf and the behavior information before the checkout are associated with each other. The acquisition unit acquires, as the front-of-shelf behavior information, front-of-the-shelf image data in which behavior of a consumer near the product shelf is imaged.
The information processing system according to claim 1. The acquisition unit acquires, as pre-checkout behavior information, pre-checkout imaging data in which the consumer's behavior of purchasing a product is captured.
The information processing system according to claim 1 or 2. The acquisition unit acquires, as the front-of-shelf behavior information, front-of-shelf image data obtained by capturing behavior of a consumer near the product shelf,
The acquisition unit acquires, as pre-checkout behavior information, pre-checkout imaging data in which the consumer's behavior of purchasing a product is captured,
The generation unit generates, as the purchase behavior information, information that associates the front-of-shelf image data and the front-of-the-checkout image data in which the same consumer is imaged.
The information processing system according to claim 1. The acquisition unit acquires, as the front-of-shelf behavior information, question-and-answer data indicating the consumer's answer to a question about the product, which is output from a customer service terminal provided on the product shelf.
The information processing system according to any one of claims 1 to 4. The acquisition unit acquires POS data related to the product purchased by the consumer as the pre-checkout behavior information.
The information processing system according to any one of claims 1 to 5. The acquisition unit acquires the position where the product is displayed on the product shelf from front-of-shelf image data in which the behavior of a consumer near the product shelf is captured,
The generation unit generates information that associates the position with the pre-checkout behavior information as the purchase behavior information.
The information processing system according to any one of claims 1 to 6. The generation unit generates, as the purchase behavior information, information that associates the position with the pre-checkout behavior information for each manufacturer or type of merchandise displayed on the merchandise shelf.
The information processing system according to claim 7. An information processing method performed by a computer that is an information processing server,
An acquisition unit acquires front-of-shelf behavior information indicating the behavior of a consumer near a product shelf on which products are displayed, and pre-checkout behavior information indicating the product-purchasing behavior of the consumer. ,
The generation unit generates purchasing behavior information in which the behavior information before the shelf and the behavior information before the checkout are associated with each other,
Information processing methods. on the computer that is the information processing server,
Acquiring front-of-shelf behavior information showing the behavior of a consumer near a product shelf on which products are displayed, and pre-checkout behavior information showing the consumer's behavior of purchasing a product,
A program for generating purchasing behavior information in which the behavior information before the shelf and the behavior information before the checkout are associated with each other.

Images