JP2017139010A - Information processing apparatus and information processing program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an information processing apparatus that can display an appropriate advertisement according to a user's degree of interest in a commodity.SOLUTION: In an information processing apparatus, an imaging part photographs a customer at a place where an advertisement for a commodity is performed; an extraction part extracts a movement of the customer and a commodity in which the customer has an interest from a result of photographing from the imaging part; a selection part performs selection of advertisement according to the customer's interest in the commodity extracted from the movement of the customer.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an information processing apparatus and an information processing program.

  Patent Document 1 aims to provide a promotion effect determination technology that can verify the effectiveness of advertisements more objectively and accurately, and target customers that meet the targeting conditions are identified as experimental customers to whom advertisements should be passed and advertisements. Control condition calculation means that divides the control customer into a non-passing control customer, and an advertisement output means that outputs advertising information to the experimental customer, and the product purchased by the experimental customer is included in the advertising information A first computing means for determining whether or not the product is an advertising target product, and a second for determining whether or not the product purchased by the control customer is the advertising target product included in the advertising information It is disclosed that a calculation means and a comparison means for comparing calculation results of the first calculation means and the second calculation means are provided.

  Patent Document 2 investigates customer responses to CMs distributed from broadcast stations to customers, with the objective of being able to accurately and in real time totalize the degree of viewer response to CMs being broadcast. A method for investigating a commercial message response, the step of creating data broadcasting content associated with a CM and allowing a customer to input response information indicating the degree of response to the CM, and transmitting the created content to a broadcasting station And receiving the response information for the CM transmitted from the customer according to the data broadcasting content distributed from the broadcasting station to the customer.

  Patent Document 3 has a problem of obtaining a precise and flexible evaluation information collection method for broadcast advertisements, and a research company that collects evaluation information about advertisements can freely access viewers through a network communication network such as the Internet.・ A downloadable website has been established. This website provides viewers with the program information, CM information, and the contents of the broadcast program and the viewer's evaluation and reaction of the CM broadcast accompanying it. By providing program production detailed information, CM production detailed information, premium information, etc., various evaluation information from viewers is collected, organized and analyzed on this homepage, and the research company collects the evaluated and analyzed evaluation information. The broadcasting stations are providing them for a fee, and thus using the widely used network communication networks such as the Internet. A specific assessment of the ad from the audience, that it is possible to easily collect on a large scale has been disclosed for the transmission.

  In Patent Document 4, the promotion information of the supplier for approaching the user (customer), the response information of the user by carrying out the promotion, the verification of the promotion information and the response information, and the verification information of the promotion effect by the analysis are made with low labor. It is a problem to be able to be provided in a short time, and is connected to a promotion information registration means, a sales planning information database managed by a supplier, sales management information data, a user information database, and a user terminal via a network, It has a data server that accepts orders, inquiries, and user registration applications from the user, and provides sales plan information such as promotion plans to the user. The registered promotion information, user response information, and promotional effects It is disclosed that the broadcast service offerings.

  Patent Document 5 aims to provide a technique for reducing the work load such as entry by the viewer himself and enabling the automatic tabulation of the audience quality of the content in the normal viewing mode. By collecting sensing data and recognition data about the viewer obtained by recognizing it, viewer response data indicating the viewer's response to the playback content is collected. When viewing data on the playback content including information on the screen operation history is collected, the collected viewer response data and the collected viewing data are integrated in time synchronization by the user terminal or the information providing device, and the information providing device Based on the integrated data, audience quality data indicating the viewer's qualitative reaction to the playback content is generated and It is disclosed that creates a viewing metadata for distribution of the generated audience quality data.

  It is an object of Patent Document 6 to provide a content evaluation device and a content evaluation method that can more objectively evaluate the effects of advertising media and advertisement programs while protecting the privacy of viewers of advertisements and programs. Detects a moving person detecting unit having a moving region detecting unit that detects a moving region from an image captured by the camera, and a person detecting unit that detects whether or not the moving region is a person, and detects a face of the person And a system server comprising a gaze person detection unit having a face detection processing unit that detects the direction of the face, and a traffic number calculation that calculates the number of people detected by the passer-by detection unit A gaze number calculation unit that calculates the gaze number of content detected by the gaze person detection unit, and a gaze rate calculation unit that calculates a gaze rate of the content from the number of people passing and the number of gazes It is disclosed to have a summary center server equipped.

  In Patent Document 7, it is an object to provide a business system and method for determining advertisement preferences based on the amount of time that a viewer faces a specific advertisement display. A series of images is acquired by a camera in the scene in front of the advertising display, a face is detected from the series of images, and for each detected face, the orientation of the face relative to the advertising display is determined and It is disclosed that preferences for advertisements can be determined.

  Patent Document 8 aims to provide an advertising effectiveness evaluation system and a system, a program applied to the system, and an advertising appeal measurement device that can accurately carry out evaluation of advertising advertisements. A POP device is provided for performing a promotional advertisement for a determined promotional advertisement target product at one or more points in time of the store's business hours, and a person who has viewed the provided advertising advertisement and the viewing time of the person are advertised. Is detected, and the appeal level information is recorded in time series by the appeal level time series data writing means to create time series data of the promotional level, and sales information of the advertisement target product in the store The POS transaction journal writing means of the POS device obtains the time-series data of the POS device, and the advertising effectiveness evaluation means of the store server device determines the time-series data Statistically analyzing evaluate the correlation of the time series data of the advertising appeal degree is disclosed.

  Patent Document 9 has an object to provide an advertising effect measuring device that can calculate the number of high-accuracy attentions and measure it as an advertising effect even when face detection is incomplete, and an image input unit; From a face detection unit, a face posture estimation unit that estimates a parameter value representing the posture of the face, a state determination unit that determines the current frame state of the detected face, and the face state and previous detection state storage of the current frame Based on the detection ID and gaze time passed, a detection ID / gaze time giving unit for giving detection ID and gaze time information of the detected face, face detection time, detection position, size, face state, and detection ID A pre-detection state storage for temporarily storing each information of the gaze time, a focus log storage unit for storing each information of the detection time, detection ID, and gaze time as detection results, and the focus Attention from the log within the specified period And interest The number counting unit that counts, it and a result output unit for outputting the counting result is disclosed.

  It is an object of Patent Document 10 to provide a content evaluation apparatus capable of determining content evaluation that can be easily used for content improvement from a viewing user's biological signal. An acquisition unit that acquires a line-of-sight position, a reaction identification unit that identifies a response type to a user's image content based on the acquired cerebral blood flow, a display area and a provided image of a provided image that is an image included in the image content Refers to the provision information and the provision information storage unit that accumulates the provision information that is information including the display time displayed in the display area, and calculates the degree of gaze on the provision image of the user based on the line-of-sight position The gaze degree calculation unit, the evaluation information accumulation unit that accumulates the evaluation information template, and the evaluation information template are referred to, and the evaluation information corresponding to the gaze degree and the reaction type is displayed as an image container. It is disclosed that includes an evaluation information determination unit for determining as an evaluation of the user for tools.

  Patent Document 11 has an object to provide an advertisement system that evaluates and evaluates the effectiveness of advertisements, and the system is configured to provide advertisement content to potential customers through a display. Including an advertising station and one or more cameras configured to capture an image of the potential customer when the potential customer approaches the advertising station, and further includes a data processing system for analyzing the captured image By doing so, it is disclosed that the line-of-sight direction and body posture direction of the potential customer are determined, and the degree of interest of the potential customer in the advertisement content is determined based on the determined line-of-sight direction and body posture direction.

JP 2000-056720 A JP 2001-357191 A JP 2002-109139 A JP 2003-030404 A JP 2005-142975 A JP 2007-181070 A JP 2008-305379 A JP 2009-140238 A JP 2009-301085 A JP 2012-234405 A JP 2013-050945 A

  An object of this invention is to provide the information processing apparatus and information processing program which can display a suitable advertisement according to the customer's interest degree in goods.

The gist of the present invention for achieving the object lies in the inventions of the following items.
The invention of claim 1 is an image capturing unit that captures a customer in a place where a product advertisement is performed, and an extraction that extracts the customer's movement and the product that the customer is interested in from the image capturing result of the image capturing unit. And a selection unit that selects an advertisement in accordance with the customer's interest in the product extracted from the customer's movement.

  The invention of claim 2 is a computer that shoots a customer in a place where a product is advertised, a movement of the customer and a product that the customer is interested in based on the shooting result of the shooting unit. An information processing program for causing an extraction unit to extract and a selection unit that selects an advertisement in accordance with the customer's interest in the product extracted from the movement of the customer.

  According to the information processing apparatus of the first aspect, it is possible to display an appropriate advertisement according to the degree of interest in the customer's product.

  According to the information processing program of the second aspect, it is possible to display an appropriate advertisement according to the degree of interest in the customer's product.

It is a conceptual module block diagram about the structural example of this Embodiment (information processing (judgment) apparatus). It is a conceptual module block diagram about the structural example of this Embodiment (information processing (learning) apparatus). It is a conceptual module block diagram about the structural example of this Embodiment (information processing (judgment / learning) apparatus). It is explanatory drawing which shows the system configuration example at the time of implement | achieving this Embodiment. It is a flowchart which shows the process example by this Embodiment (information processing (judgment) apparatus). It is explanatory drawing which shows the process example by this Embodiment (monitor). It is explanatory drawing which shows the process example by this Embodiment. It is a flowchart which shows the process example by this Embodiment (information processing (learning) apparatus). It is a related block diagram with the example of a structure in a teacher signal preparation module, and its peripheral module. It is explanatory drawing which shows the example of a data structure of an image feature table. It is explanatory drawing which shows the process example by this Embodiment. It is explanatory drawing which shows the process example by this Embodiment. It is a conceptual module block diagram about the structural example of this Embodiment at the time of adding the 2nd teacher signal preparation module. It is explanatory drawing which shows the example of a data structure of a teacher signal table. It is a conceptual module block diagram about the structural example of this Embodiment at the time of adding the 3rd teacher signal preparation module. It is a block diagram which shows the hardware structural example of the computer which implement | achieves this Embodiment.

Hereinafter, an example of a preferred embodiment for realizing the present invention will be described with reference to the drawings.
FIG. 1 shows a conceptual module configuration diagram of a configuration example of the present embodiment (information processing (judgment) device).
The module generally refers to components such as software (computer program) and hardware that can be logically separated. Therefore, the module in the present embodiment indicates not only a module in a computer program but also a module in a hardware configuration. Therefore, the present embodiment is a computer program for causing these modules to function (a program for causing a computer to execute each procedure, a program for causing a computer to function as each means, and a function for each computer. This also serves as an explanation of the program and system and method for realizing the above. However, for the sake of explanation, the words “store”, “store”, and equivalents thereof are used. However, when the embodiment is a computer program, these words are stored in a storage device or stored in memory. It is the control to be stored in the device. Modules may correspond to functions one-to-one, but in mounting, one module may be configured by one program, or a plurality of modules may be configured by one program, and conversely, one module May be composed of a plurality of programs. The plurality of modules may be executed by one computer, or one module may be executed by a plurality of computers in a distributed or parallel environment. Note that one module may include other modules. Hereinafter, “connection” is used not only for physical connection but also for logical connection (data exchange, instruction, reference relationship between data, etc.). “Predetermined” means that the process is determined before the target process, and not only before the process according to this embodiment starts but also after the process according to this embodiment starts. In addition, if it is before the target processing, it is used in accordance with the situation / state at that time or with the intention to be decided according to the situation / state up to that point. When there are a plurality of “predetermined values”, the values may be different from each other, or two or more values (of course, including all values) may be the same. In addition, the description having the meaning of “do B when it is A” is used in the meaning of “determine whether or not it is A and do B when it is judged as A”. However, the case where it is not necessary to determine whether or not A is excluded.
In addition, the system or device is configured by connecting a plurality of computers, hardware, devices, and the like by communication means such as a network (including one-to-one correspondence communication connection), etc., and one computer, hardware, device. The case where it implement | achieves by etc. is included. “Apparatus” and “system” are used as synonymous terms. Of course, the “system” does not include a social “mechanism” (social system) that is an artificial arrangement.
In addition, when performing a plurality of processes in each module or in each module, the target information is read from the storage device for each process, and the processing result is written to the storage device after performing the processing. is there. Therefore, description of reading from the storage device before processing and writing to the storage device after processing may be omitted. Here, the storage device may include a hard disk, a RAM (Random Access Memory), an external storage medium, a storage device via a communication line, a register in a CPU (Central Processing Unit), and the like.

Terms used in the following description will be explained.
The two-classifier is a device that assigns one of two labels exclusive to the input, for example, “O” or “X”. Examples include support vector machines and boosting.
Learning is to create the above two classifier using learning data. It is also possible to construct a multi-classifier by creating a plurality of two classifiers.
The frequency indicates the number of objects.

In the retail industry and the like, various advertisements (hereinafter also referred to as promotions) such as demonstration sales by humans and provision of campaign videos are provided in order to stimulate customers' willingness to purchase. However, it is very difficult to know if a promotion is really effective. Although it is possible to know through questionnaire surveys, it does not necessarily reflect the actual situation.
In the present embodiment, by creating a learning model using customer image information, it is predicted in advance whether the customer who is watching the promotion will buy the target product or not. In addition, it is determined which part of the promotion contributes to activating the customer's purchasing motivation, and conversely, which part of the promotion reduces the purchasing motivation from the expected results regarding the purchasing motivation of a plurality of customers.

  FIG. 3 is a diagram showing the entire embodiment. The present embodiment is roughly composed of an information processing (learning) apparatus 200 and an information processing (determination) apparatus 100. The information processing (learning) apparatus 200 is an apparatus that constructs a learning model that expresses the relationship between image features and teacher signals. The information processing (determination) device 100 is a device that determines whether or not an image feature currently obtained by using the created learning model is motivated to purchase. Furthermore, it is also possible to select places that are valuable in the promotion (that is, places that increase the customer's willingness to purchase) and places that are not worthwhile or should be corrected.

The information processing (determination) apparatus 100 according to the present embodiment determines a customer's willingness to purchase using a machine-learned learning model. As shown in the example of FIG. 130, an image feature extraction module 140, a customer purchase motivation determination module 150, and a promotion selection module 160. The customer purchase willingness determination module 150 has a learning model storage module 155. The information processing (judgment) device 100 is connected to the promotion device 110 and the imaging device 120.
The information processing (judgment) device 100 informs the store clerk in advance, for example, whether or not the customer who saw the promotion is willing to purchase (that is, before judging whether or not to actually purchase the target product). Used for etc.
Specifically, a moving image of a customer who saw the promotion is acquired, and a learning model is created using whether or not the customer actually purchased the target product as a teacher signal. Then, using a learning model created from the video information of the customer group who saw the promotion, it is determined in advance whether a certain customer who saw the promotion purchases the target product.

The promotion device 110 is connected to the imaging device 120 and the promotion selection module 160 of the information processing (determination) device 100. The promotion device 110 is a device such as a huge monitor or a screen for monitoring demonstration sales of a person (salesperson or the like) or a product promotion video. In addition, it may be human demonstration sales (performance) itself.
The imaging device 120 is connected to the promotion device 110 and the human / face detection module 130 of the information processing (judgment) device 100. The imaging device 120 is a device for imaging a customer who is looking at the promotion device 110. Applicable to video cameras.

The human / face detection module 130 is connected to the imaging device 120 and the image feature extraction module 140. The human / face detection module 130 extracts a human region (or a facial region) from the image data captured by the imaging device 120. Note that the real-time property may or may not be present.
The image feature extraction module 140 is connected to the person / face detection module 130 and the customer purchase motivation determination module 150. The image feature extraction module 140 extracts a feature of a customer's body or face in an image obtained by photographing a place where a product advertisement is performed. Specifically, the image feature extraction module 140 extracts image features from an area determined to be a person (or face). For example, image features related to texture such as Gabor feature, Haar feature, Haar-like feature, and the like are extracted.

The customer purchase willingness determination module 150 is connected to the image feature extraction module 140 and the promotion selection module 160. The customer purchase motivation determination module 150 determines whether or not the target person (customer) is willing to purchase by using the extracted image feature as an input and using a learning model. That is, the customer purchase willingness determination module 150 creates the features extracted by the image feature extraction module 140 by the learning model in the learning model storage module 155 (the learning model creation / save module 270 of the information processing (learning) device 200 described later). Applied to the learning model), the customer's willingness to purchase is determined. This willingness to purchase may be the willingness to purchase the product that is the subject of the advertisement.
The learning model storage module 155 stores the learning model created by the learning model creation / storing module 270 of the information processing (learning) apparatus 200 described later. The customer purchase willingness determination module 150 performs determination processing using the learning model in the learning model storage module 155.

  The promotion selection module 160 is connected to the promotion device 110 and the customer purchase willingness determination module 150. The promotion selection module 160 detects an attention promotion area in units of promotions from the purchase motivation determination results of multiple customers (processing results by the customer purchase motivation determination module 150). When the promotion device 110 is a person's demonstration sale itself, the detected promotion may be presented to the display device such as a liquid crystal display.

FIG. 2 is a conceptual module configuration diagram of an exemplary configuration of the present embodiment (information processing (learning) apparatus 200). The information processing (learning) apparatus 200 creates a learning model so that the customer's willingness to purchase can be determined. As shown in the example of FIG. 2, the person / face detection module 230 and the image feature extraction module 240 , An image feature DB 250, a teacher signal creation module 260, and a learning model creation storage module 270.
The promotion device 210 is connected to the imaging device 220. Promotion device 210 is equivalent to the aforementioned promotion device 110.
The imaging device 220 is connected to the promotion device 210 and the human / face detection module 230 of the information processing (learning) device 200. The imaging device 220 is equivalent to the imaging device 120 described above.
The human / face detection module 230 is connected to the imaging device 220 and the image feature extraction module 240. The person / face detection module 230 is equivalent to the person / face detection module 130 described above.

The image feature extraction module 240 is connected to the person / face detection module 230 and the image feature DB 250. The image feature extraction module 240 is equivalent to the image feature extraction module 140 described above, and extracts a feature of a customer's body or face in an image obtained by photographing a place where a product is advertised.
The image feature DB 250 is connected to the image feature extraction module 240, the teacher signal creation module 260, and the learning model creation / save module 270. The image feature DB 250 is a database that stores image features and the like extracted by the image feature extraction module 240.
The teacher signal creation module 260 is connected to the image feature DB 250 and the learning model creation / save module 270. The teacher signal creation module 260 creates information on whether or not the target person (customer) has purchased the target product.

The learning model creation / save module 270 is connected to the image feature DB 250, the teacher signal creation module 260, and the learning model storage module 155 of the information processing (judgment) device 100. The learning model creation / save module 270 creates a learning model by using, as a teacher signal, information indicating whether or not the customer whose feature has been extracted by the image feature extraction module 240 has been purchased. For example, a neural network support vector machine may be used to create the learning model. Further, a learning model may be created by using, as a teacher signal, information indicating whether or not a customer whose features have been extracted by the image feature extraction module 240 has purchased a product targeted for promotion. Then, the created learning model is stored in the learning model storage module 155 of the information processing (determination) device 100.
Here, regarding the teacher signal, the matching result between the feature of the customer's body or face in the image at the checkout counter or the store exit and the feature extracted by the image feature extraction module 240 is used as the teacher signal. Also good. For example, the customer in the image at the checkout counter may be a purchaser, and the other customers may be those who have not purchased. Further, customers in the image at the store exit, and customers other than the customer in the image at the cashier counter before that may be those who have not purchased. This teacher signal creation processing will be described later with reference to the examples of FIGS.
Further, the learning model creation / storing module 270 may use whether or not the product has been touched by the customer (in hand, held in hand, etc.) as a teacher signal indicating whether or not the product is interested. Good. This teacher signal creation processing will be described later using the example of FIG.

Further, when the advertisement uses video, the following processing may be performed.
The image feature extraction module 240 extracts features in a still image corresponding to each frame of the video.
Then, the learning model creation / save module 270 may generate a learning model based on a teacher signal for the feature in the still image corresponding to each frame.
When the learning model is used in the information processing (determination) device 100 shown in the example of FIG. 1, the promotion selection module 160 of the information processing (determination) device 100 determines that the purchase is based on the still image corresponding to the frame. A frame that is highly likely to be purchased is determined as a frame that has a purchase effect in the advertisement video, or a frame that is less likely to be determined to be purchased in the still image corresponding to the frame is a frame that does not have a purchase effect in the advertisement video You may make it determine as. This determination process will be described later with reference to the examples of FIGS.
With the above learning model, learning is performed based on the still image of the customer, and it is determined whether or not the customer is likely to purchase the product sequentially based on the still image information of the customer who is viewing the promotion. It is determined that the promotion corresponding to the time when many customers shift from the “not purchased” state to the “purchased” state is a good promotion. For example, the portion of the promotion may be emphasized. Conversely, it is determined that the portion of the promotion corresponding to the time to shift from the “purchase” state to the “not purchase” state is a bad portion promotion. For example, the portion of the promotion may be pointed out as an improvement point.

FIG. 3 is a conceptual module configuration diagram of a configuration example of the present embodiment (information processing (determination / learning) device 300).
The information processing (determination / learning) apparatus 300 includes an information processing (determination) apparatus 100 and an information processing (learning) apparatus 200, which are integrated. That is, the information processing (learning) apparatus 200 of the information processing (determination / learning) apparatus 300 creates a learning model, and the information processing (determination) apparatus 100 performs a determination process using the learning model.
The promotion device 110 and the promotion device 210, the imaging device 120 and the imaging device 220, the person / face detection module 130 and the person / face detection module 230, and the image feature extraction module 140 and the image feature extraction module 240, respectively. , Either one may be sufficient. That is, the information processing (determination) apparatus 100 and the information processing (learning) apparatus 200 may share these modules.

FIG. 4 is an explanatory diagram showing an example of a system configuration when the present embodiment is realized.
Store system 400A, store system 400B, store system 400C, store system 400D, store (learning) system 410A, information processing (determination) device 100E, information processing (learning) device 200B, etc. are connected via communication line 290. . The communication line 290 may be wireless, wired, or a combination thereof, for example, the Internet as a communication infrastructure.
The store system 400A includes an information processing (determination) device 100A, a promotion device 110A, and an imaging device 120A. The store system 400B includes an information processing (determination) device 100B, a promotion device 110B, and an imaging device 120B. The store system 400C includes a promotion device 110C and an imaging device 120C. Store system 400D has promotion device 110D and imaging device 120D. The store (learning) system 410A includes an information processing (learning) device 200A, a promotion device 210A, and an imaging device 220A.
For example, each store may have a combination of the promotion device 110, the imaging device 120, and the information processing (determination) device 100 (for example, the store system 400A and the store system 400B). Further, a combination of the promotion device 110 and the imaging device 120 (for example, the store system 400C and the store system 400D) may be included and the information processing (determination) device 100E may make the determination. The determination result may be transmitted to an information processing apparatus such as a personal computer in each store.
In addition, a combination of a promotion device 210, an imaging device 220, and an information processing (learning) device 200 (for example, a store (learning) system 410A) may be used for creating a learning model, or the imaging device 120 in each store. The video may be received from the imaging device 220 and a learning model may be created by the information processing (learning) device 200B. Then, the learning model created by the information processing (learning) apparatus 200A or the information processing (learning) apparatus 200B is transmitted to the information processing (determination) apparatus 100A, the information processing (determination) apparatus 100B, and the information processing (determination) apparatus 100E. .
Note that the information processing (determination / learning) apparatus 300 illustrated in the example of FIG. 3 may be connected to the communication line 290.

FIG. 5 is a flowchart showing an example of processing by the present embodiment (information processing (judgment) device 100).
In step S502, the promotion device 110 starts a promotion. Specifically, it will be described later using the example of FIG.
In step S <b> 504, the imaging device 120 captures a customer in the vicinity of the promotion device 110.
In step S506, the person / face detection module 130 detects a person or face area from the image.
In step S508, the image feature extraction module 140 extracts features in the person or face area.
In step S510, the customer purchase motivation determination module 150 applies the characteristics to the learning model in the learning model storage module 155 to determine the customer's purchase intention.
In step S512, the promotion selection module 160 selects a promotion according to the determination result.
In step S514, the promotion selection module 160 determines whether or not it is the last, and if it is the last, the process ends (step S599), and otherwise returns to step S502.

FIG. 6 is an explanatory diagram showing a processing example according to the present exemplary embodiment (monitor 610).
The monitor 610 is an example of the promotion device 110 and the promotion device 210, and the imaging device 620 is an example of the imaging device 120 and the imaging device 220.
There are various forms of promotions, but here we take an example where a video as a promotion of a target product is played on a large monitor. A case is assumed in which the same promotion video is repeatedly played on the monitor 610. For example, the customer may select a plurality of types of campaign videos using buttons or the like. An imaging device 620 such as a video camera is attached to the monitor 610, and the customer 600 who sees the campaign video can be captured. A plurality of customers 600 to be imaged may be shown at the same time. The image picked up by the image pickup device 620 is sent to the human / face detection module 230.

FIG. 7 is an explanatory diagram illustrating a processing example according to the present exemplary embodiment.
The person / face detection modules 130 and 230 are modules for detecting a person and a person's face from images sent from the imaging devices 120 and 220. The target image may be a still image or a moving image divided for each frame. It is assumed that a method for extracting a person or a person's face uses existing technology. For example, as a face image extraction,
Non-patent information: Viola and MJ. Jones, “Robust Real-Time Face Detection.”, IJCV, 57 (2), 2004.
As human image extraction,
Non-patent information: Dalal and B.M. Triggs, “Histograms of Oriented Gradients for Human Detection.”, CVPR, 2005.
It can be realized by using the technique described in the above. The person area acquired in this way is detected as a rectangular area in the image. FIG. 7 shows an example, FIG. 7A shows an example of face area detection, and FIG. 7B shows an example of human area detection. For example, the face area 712 of the customer 602 and the face area 714 of the customer 604 are extracted from the image 710. A human area 726 of the customer 606 and a human area 728 of the customer 608 are extracted from the image 720.

Next, the image feature extraction modules 140 and 240 extract image features from the selected rectangular area. As this image feature, an image feature used for person detection may be used, or another image feature may be used. Image features are extracted for still image frames, and one image feature is extracted for one rectangular area. For the sake of simplicity, it is assumed that the person / face detection modules 130 and 230 do not have any missing frames while a specific person is shown. It is assumed that the person H is reflected in the frames F1 to Fn, and the person image feature extracted from the frame Fi is fi. generating image features ^{f H} to the customer H from f1 to fn. As a generation method, a statistical value (for example, an average value, a median value, a mode value, etc.) of each fi is taken, or a feature dimension of the image feature fi is represented as d, and an image feature is represented as an nxd-dimensional matrix L. In other words, there is a method of setting f ^H = K * L by creating a dxn-dimensional matrix K. Further, as another form, without creating a f ^H, the n-number of the image feature f1~fn may be directly technique using.
The image feature f ^H extracted by the image feature extraction module 140 is passed to the customer purchase willingness determination module 150. The image feature f ^H extracted by the image feature extraction module 240 is stored in the image feature DB 250.

FIG. 8 is a flowchart showing an example of processing by the present embodiment (information processing (learning) apparatus 200).
In step S802, the promotion device 210 starts promotion.
In step S <b> 804, the imaging device 220 captures a customer in the vicinity of the promotion device 210.
In step S806, the person / face detection module 230 detects a person or face area from the image.
In step S <b> 808, the image feature extraction module 240 extracts features in a person or face area and stores the features in the image feature DB 250.
In step S810, the teacher signal creation module 260 creates a teacher signal.
In step S812, the learning model creation / save module 270 creates a learning model based on the features in the image feature DB 250 and the teacher signal.
In step S814, the learning model creation / storing module 270 stores the learning model in the learning model storage module 155 of each information processing (judgment) device 100.

The teacher signal creation module 260 is a module for obtaining information on whether or not the customer H has actually purchased the target product. Whether or not the target product has been purchased can be determined based on whether or not the customer H has passed the checkout counter. As a method for specifying the customer H, for example, the following may be performed.
(1) Use image features,
(2) Methods using various cards such as point cards and credit cards,
There is.
When using image features, a method using a face matching technique, which is an existing technique, can be used. The face image captured by the imaging device 620 is collated with the face image captured by the cashier counter, and the customer H is specified at the cashier counter. It is also possible to authenticate customer H by keeping track of the person from the promotion device (monitor 610) to the cashier counter.
The example of FIG. 9 shows a processing example when personal authentication is performed using a face image. FIG. 9 is a configuration diagram of a relationship between a configuration example in the teacher signal creation module 260 and peripheral modules. The relationship example of the promotion apparatus 210, the imaging device 220, the face detection module 930, the image feature extraction module 240, the image feature DB 250, and the teacher signal creation module 260 is shown. The same parts as those in the above-described embodiment are denoted by the same reference numerals, and redundant description is omitted. In the example shown in FIG. 9, the human / face detection module 230 in the example shown in FIG. However, the human / face detection module 230 may be a human detection module mainly for human images. In that case, the face detection module 964 and the face verification module 966 may be a human detection module and a human verification module, respectively. Further, in the example of FIG. 9, the face detection module 930 and the image feature DB 250 are connected, but they may be connected via the image feature extraction module 240 as in the example of FIG. The person / face detection module 230 and the image feature DB 250 shown in the example 2 may be connected. That is, the processing result by the person / face detection module 230 may be stored in the image feature DB 250. Hereinafter, an example in the case of using a face image for collation will be shown.

The image feature DB 250 is connected to the face detection module 930, the image feature extraction module 240, and the face collation module 966 of the teacher signal creation module 260. The image feature DB 250 stores a face image region, which is a processing result of the face detection module 930, and a processing result of the image feature extraction module 240 for the face image. The face information is a face image or a face image feature (the face image feature and the image feature f ^H described above may match). The face information can be obtained for each frame, but it may be stored for several frames necessary for collation.
The teacher signal creation module 260 includes a cash register imaging device 962, a face detection module 964, and a face collation module 966. The processing result by the teacher signal creation module 260 is passed to the learning model creation storage module 270.
The cash register imaging device 962 is connected to the face detection module 964. The cash register imaging device 962 captures the area around the cash register counter (or the customer who purchases it).
The face detection module 964 is connected to the cash register imaging device 962 and the face matching module 966. The face detection module 964 performs a process equivalent to the face detection module 930 (a process related to a face image among the functions in the person / face detection module 230) from an image taken by the cash register imaging device 962, and detects a face image To do.
The face collation module 966 is connected to the face detection module 964 and the image feature DB 250. The face matching module 966 performs a matching process between the customer currently at the cashier counter and the data stored in the image feature DB 250. When the image feature of the customer at the checkout counter matches the data in the image feature DB 250, information indicating that the customer has purchased the target product is stored in the image feature DB 250.

The example shown in FIG. 10 shows an example of data stored in the image feature DB 250. FIG. 10 is an explanatory diagram showing an example of the data structure of the image feature table 1000. The image feature table 1000 includes a customer ID column 1010, a face image feature column 1020, a face image column 1030, a promotion type column 1040, an image feature column 1050, and a purchase / non-purchase column 1060. In the present embodiment, customer ID column 1010 stores information (customer ID: IDentification) for uniquely identifying a customer. The face image feature column 1020 stores the face image feature of the customer with the customer ID. The face image column 1030 stores the customer's face image. It may be the face image itself or the file name of the face image. The promotion type column 1040 stores information indicating the type of promotion viewed by the customer. The image feature column 1050 stores image features. The purchase / non-purchase column 1060 stores information indicating purchase / non-purchase for each product to be promoted. In other words, it contains the type of promotion that the customer participated in, its image characteristics, and purchase or non-purchase information, and is blank if purchase / non-purchase has not been confirmed. There may be at least one face image feature or face image, and a plurality of face image features and face images can be registered in order to improve collation accuracy. If the customer does not pass the checkout counter, the teacher signal cannot be given to the image feature DB 250. If a teacher signal is not obtained after a predetermined time after the image feature is stored in the image feature DB 250, a label “not purchased” may be added.
On the other hand, various cards or fingerprint authentication may be used for person verification without using a face image. In this case, the following processing may be performed. Just before the customer sees the presentation, the individual is identified by various authentication methods, and the image feature is associated with the customer. Further, when the checkout is performed at the checkout counter, an individual is identified with a card or the like, and processing such as collating with image features is performed. As a result of such processing, in the image feature DB 250, learning data including image features and purchase / non-purchase pairs is created.

  Next, the learning model creation / storing module 270 creates a learning model (determination model) using the learning data described above, and stores the learning model. When the teacher signal is only purchased or not purchased, a learning model is constructed using a general-purpose two classifier (for example, support vector machine, boosting, etc.). The created learning model is stored in the learning model storage module 155 of the information processing (judgment) device 100. For example, the data is stored in a storage of a server computer or the like, and the customer purchase willingness determination module 150 of the information processing (determination) apparatus 100 downloads the learning model from the storage or the like to the learning model storage module 155. When there are a plurality of target promotions, learning models are created for the number of promotions. The information processing (learning) apparatus 200 has been described above.

Next, the information processing (determination) device 100 will be described. The information processing (judgment) apparatus 100 implements the following two methods.
1) Predicting whether a customer watching a promotion is likely to purchase the target product,
2) Discover good and bad points in the promotion,
The image feature extraction module 140 extracts an image feature of a customer who is watching the target promotion P. The type of image feature used is the same as that used when creating the learning model. If an image feature for the target customer's promotion P already exists, the target customer's data is not used because it is used in the learning model (for example, a customer who visited the same promotion multiple times corresponds). The obtained image feature is input to the customer purchase motivation determination module 150 into the learning model, and the output is obtained. The output value is a label value indicating either “purchase” or “not purchase”. For example, if a customer who saw a promotion determines that he / she does not purchase, the determination result is transmitted to a personal computer in the store, and the store clerk who is notified of the determination result can purchase the product. It is possible to take a different approach.

Next, a method for identifying good and bad points in the promotion will be described. In other words, it is a process performed to identify the time position (such as how many seconds after the start) of the promotion that affects the customer and improve the promotion.
At this time, the image features for creating the learning model are not obtained by converting the output values of a plurality of frames into one, but use image features extracted from each frame. That is, all the image features fi from the image frame for the customer H who saw a certain promotion P are used, and information indicating whether or not the target product has actually been purchased is given as a teacher signal. In other words, all the image features associated with the promotion P and the customer H are given the same label as a teacher signal. Alternatively, not all image features may be used, but the image features for the second half of the target frame considered to be important for the purchase / non-purchase determination may be used as the target of the learning data. Note that the latter half here refers to a period after a predetermined time from the start of the promotion. Using these, the learning model is configured in the same manner as described above.
Next, an image feature of a customer who is looking at a certain promotion P is extracted and used as input data for a learning model, so that an expected label of “purchase” or “non-purchase” is given for each frame. Such data is collected for a plurality of people. The example shown in FIG. 11 is a diagram showing how many images (still images) determined to be “purchase” can be collected on the time axis of the promotion video. Assume that the frequency determined as “purchase” for the image feature for the p-th frame Fp is the largest, and the frequency determined as “non-purchase” for the image feature for the q-th frame Fq is the smallest. At this time, it is determined that the frame around Fp is a good promotion, and the frame around Fq is determined as a bad promotion. In the example of FIG. 11, attention is paid only to the place where the frequency is the maximum and the minimum, but the time axis taking the maximum and the minimum in the frequency distribution of FIG. 11 is extracted, and the frame around the maximum is a good promotion, the minimum periphery The promotion may be determined as a bad promotion. In the example shown in FIG. 12, the maximum values are p1, p2, and p3, and the minimum values are q1 and q2. By determining the good frame and the bad frame around each extreme value, Extract bad promotions. Alternatively, when the total number of images that are determined to be good to see the promotion P divided by the number of frames (the average number that is determined to be “purchase” by looking at a specific frame) is equal to or greater than a predetermined threshold A, the promotion If the promotion itself is good and less than a predetermined threshold B, it may be determined that the promotion is bad. The threshold value A and the threshold value B may be the same value or different values.

In the above explanation, the teacher signal has two choices of “purchase” or “non-purchase”. Information on whether or not the product has shown interest, such as whether or not the target product has actually been picked up, is “important” in terms of purchasing will. Therefore, a teacher signal “not purchased but interested” is created, and three teacher signals “not purchased (not shown interest)”, “not purchased (interested)”, and “purchased (obviously interested)”. An example of creating
FIG. 13 is a conceptual module configuration diagram of a configuration example of the present embodiment when the second teacher signal generation module 1360 is added. That is, it is an explanatory diagram for obtaining a second teacher signal. The teacher signal creation module 1360 is added to the example of FIG.
The teacher signal creation module 1360 includes a product installation location imaging device 1362, a face detection module 1364, and a behavior detection module 1366.
The image feature DB 250 is connected to the face detection module 930, the image feature extraction module 240, the face collation module 966 of the teacher signal creation module 260, the face detection module 1364 of the teacher signal creation module 1360, and the behavior detection module 1366.
The commodity installation location imaging device 1362 is connected to the face detection module 1364 and the behavior detection module 1366. The product installation location imaging device 1362 is installed at a product installation location (product sales location), and images customers at that location. The product here may be a product targeted for promotion.
The face detection module 1364 is connected to the product installation location imaging device 1362, the image feature DB 250, and the face collation module 966 of the teacher signal creation module 260. The face detection module 1364 performs a process equivalent to the face detection module 930 from the image photographed by the product installation location imaging device 1362 to detect a face image.
The behavior detection module 1366 is connected to the product installation location imaging device 1362, the image feature DB 250, and the face matching module 966 of the teacher signal creation module 260. The behavior detection module 1366 detects that the customer touches the target product from the image captured by the product installation location imaging device 1362.

This will be specifically described.
In order to create a “non-purchase (interested)” teacher signal, an imaging device 220 such as a video camera is installed at the target product installation location, and the face of the approaching customer is detected. When only the customer who has seen the promotion device 210 first is targeted, faces that are not stored in the image feature DB 250 may be omitted from the detection target. Alternatively, when a customer who visits the target product installation location before viewing the promotion device 210 is also targeted, information indicating whether or not he / she is interested in the detected face image feature and the target product determined by the behavior detection module 1366 is an image feature. It is stored in the DB 250 and it is determined whether or not the customer has visited the place where the promotion device 210 is installed. The behavior detection module 1366 performs gaze detection and hand detection,
1) Did you touch the target product?
2) How long did you stay at the target product location?
It is determined whether or not the customer is interested in the target product comprehensively from such information.
The example shown in FIG. 14 shows information stored in the image feature DB 250 in addition to the image feature table 1000 shown in the example of FIG. 10 by introducing the teacher signal creation module 1360 as shown in the example of FIG. . FIG. 14 is an explanatory diagram showing an example of the data structure of the teacher signal table 1400. The teacher signal table 1400 includes a customer ID column 1410, a face image feature column 1420, a face image column 1430, a promotion (target product) type column 1440, and an interest / none column 1450. The customer ID column 1410 stores a customer ID. The face image feature column 1420 stores the face image feature of the customer with the customer ID. The face image column 1430 stores the face image of the customer. The promotion (target product) type column 1440 stores the type of promotion (target product). The interest / none column 1450 stores information indicating interest / no interest for each product targeted for promotion.

Of the information from the teacher signal creation module 1360, face information (face image feature field 1420, face image field 1430) is necessary only for matching with face information obtained by the promotion device 210, and face information is face matching. When the collation by the module 966 is completed, the image feature DB 250 may be deleted. At the time of collation, the information that is finally required is the information “interested” among the information related to “interested / not interested”, and the target product is not actually purchased despite being “interested” To get a teacher signal. Alternatively, a customer who does not go to the promotion device 210 within a predetermined time after going to the target product installation location may be deleted from the image feature DB 250.
With the above processing example, a learning model composed of three teacher signals is created in the same manner, and the prediction that “I am interested but will not purchase” is obtained in advance from the image information of the customer watching the promotion. Then, it is possible to narrow down customers who are likely to be interested in the target product from customers expected to be non-purchased. As a result, further countermeasures can be implemented in advance.
As another example of processing, when classifying a customer, not only the image from the promotion device 210 but also information indicating whether or not the target product has been shown is obtained in the same manner as the teacher signal creation module 1360. You may make it raise the determination precision regarding "it seems to be interested" further.
When creating a learning model, there are two so-called one-vs-all classifiers (for example, “purchase” and “other” classifiers, and “other” are further “interested” and “interested”. A learning model can be created by constructing a “none classifier”.

FIG. 15 is a conceptual module configuration diagram of a configuration example of the present embodiment when a third teacher signal generation module 1560 is added. The teacher signal creation module 1560 is added to the example of FIG. An exit imaging device 1562 such as a video camera may be installed at the exit of a store or the like, and a “non-purchased” label may be assigned from the face information of the person exiting the exit without passing through the cashier counter.
The teacher signal creation module 1560 includes an exit imaging device 1562 and a face detection module 1564.
The exit imaging device 1562 is connected to the face detection module 1564. The exit imaging device 1562 is installed at an exit of a store or the like, and images a customer leaving the store or the like.
The face detection module 1564 is connected to the exit imaging device 1562 and the face collation module 966 of the teacher signal creation module 260. The face detection module 1564 performs a process equivalent to the face detection module 930 from the image captured by the exit imaging device 1562 to detect a face image.
The face matching module 966 is connected to the face detection module 964, the image feature DB 250, and the face detection module 1564. In addition to the processing of the face matching module 966 shown in the example of FIG. 9, the face matching module 966 has features of the customer face image detected by the face detection module 1564 and the feature of the customer face image at the checkout counter. If they do not match, the customer detected by the face detection module 1564 stores information indicating that the product has not been purchased in the image feature DB 250.

  A hardware configuration example of the information processing apparatus according to the present embodiment will be described with reference to FIG. The configuration illustrated in FIG. 16 is configured by, for example, a personal computer (PC) or the like, and illustrates a hardware configuration example including a data reading unit 1617 such as a scanner and a data output unit 1618 such as a printer.

  A CPU (Central Processing Unit) 1601 includes various modules described in the above-described embodiments, that is, a person / face detection module 130, an image feature extraction module 140, a customer purchase motivation determination module 150, a promotion selection module 160, a person / person Face detection module 230, image feature extraction module 240, teacher signal creation module 260, learning model creation and storage module 270, face detection module 930, cash register imaging device 962, face detection module 964, face matching module 966, product installation location imaging device 1362 , A computer program describing an execution sequence of each module such as the face detection module 1364, the action detection module 1366, the exit imaging device 1562, the face detection module 1564, etc. It is a control part which performs processing according to.

  A ROM (Read Only Memory) 1602 stores programs, calculation parameters, and the like used by the CPU 1601. A RAM (Random Access Memory) 1603 stores programs used in the execution of the CPU 1601, parameters that change as appropriate during the execution, and the like. These are connected to each other by a host bus 1604 including a CPU bus or the like.

  The host bus 1604 is connected to an external bus 1606 such as a PCI (Peripheral Component Interconnect / Interface) bus through a bridge 1605.

  A keyboard 1608 and a pointing device 1609 such as a mouse are input devices operated by an operator. The display 1610 includes a liquid crystal display device or a CRT (Cathode Ray Tube), and displays various types of information as text or image information.

  An HDD (Hard Disk Drive) 1611 includes a hard disk, drives the hard disk, and records or reproduces a program executed by the CPU 1601 and information. The hard disk stores images, customer person or facial features in the images, teacher signals, learning models, and the like. Further, various computer programs such as various other data processing programs are stored.

  The drive 1612 reads data or a program recorded on a removable recording medium 1613 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, and the data or program is read out from the interface 1607 and the external bus 1606. , A bridge 1605, and a RAM 1603 connected via the host bus 1604. The removable recording medium 1613 can also be used as a data recording area similar to the hard disk.

  The connection port 1614 is a port for connecting an external connection device 1615 and has a connection unit such as USB, IEEE1394. The connection port 1614 is connected to the CPU 1601 and the like via the interface 1607, the external bus 1606, the bridge 1605, the host bus 1604, and the like. A communication unit 1616 is connected to a communication line and executes data communication processing with the outside. The data reading unit 1617 is a scanner, for example, and executes document reading processing. The data output unit 1618 is a printer, for example, and executes document data output processing.

  Note that the hardware configuration of the information processing apparatus shown in FIG. 16 shows one configuration example, and the present embodiment is not limited to the configuration shown in FIG. 16, and the modules described in this embodiment are executed. Any configuration is possible. For example, some modules may be configured with dedicated hardware (for example, Application Specific Integrated Circuit (ASIC), etc.), and some modules are in an external system and connected via a communication line In addition, a plurality of systems shown in FIG. 16 may be connected to each other via communication lines so as to cooperate with each other. Further, it may be incorporated in a copying machine, a fax machine, a scanner, a printer, a multifunction machine (an image processing apparatus having any two or more functions such as a scanner, a printer, a copying machine, and a fax machine).

  Further, in the description of the above-described embodiment, “more than”, “less than”, “greater than”, and “less than (less than)” in a comparison with a predetermined value contradicts the combination. As long as the above does not occur, “larger”, “smaller (less than)”, “more than”, and “less than” may be used.

The program described above may be provided by being stored in a recording medium, or the program may be provided by communication means. In that case, for example, the above-described program may be regarded as an invention of a “computer-readable recording medium recording the program”.
The “computer-readable recording medium on which a program is recorded” refers to a computer-readable recording medium on which a program is recorded, which is used for program installation, execution, program distribution, and the like.
The recording medium is, for example, a digital versatile disc (DVD), which is a standard established by the DVD Forum, such as “DVD-R, DVD-RW, DVD-RAM,” and DVD + RW. Standard “DVD + R, DVD + RW, etc.”, compact disc (CD), read-only memory (CD-ROM), CD recordable (CD-R), CD rewritable (CD-RW), Blu-ray disc ( Blu-ray (registered trademark) Disc), magneto-optical disk (MO), flexible disk (FD), magnetic tape, hard disk, read-only memory (ROM), electrically erasable and rewritable read-only memory (EEPROM (registered trademark)) )), Flash memory, Random access memory (RAM) SD (Secure Digital) memory card and the like.
The program or a part of the program may be recorded on the recording medium for storage or distribution. Also, by communication, for example, a local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN), a wired network used for the Internet, an intranet, an extranet, or a wireless communication It may be transmitted using a transmission medium such as a network or a combination of these, or may be carried on a carrier wave.
Furthermore, the program may be a part of another program, or may be recorded on a recording medium together with a separate program. Moreover, it may be divided and recorded on a plurality of recording media. Further, it may be recorded in any manner as long as it can be restored, such as compression or encryption.

  In the embodiment described above, a learning model is obtained by using, as a teacher signal, information indicating whether or not a customer has purchased a feature of a customer's body or face in an image obtained by photographing a place where a product advertisement is performed. It is also possible to provide an information processing apparatus and an information processing program that are created.

The above-described embodiment may be grasped as follows.
[A1] Extraction means for extracting features of a customer's body or face in an image obtained by photographing a place where a product advertisement is performed;
An information processing apparatus comprising: a creation unit that creates a learning model using, as a teacher signal, information indicating whether or not the customer whose features have been extracted by the extraction unit has been purchased.

[A2] second extraction means for extracting features of a customer's body or face in an image obtained by photographing a place where a product advertisement is performed;
[A1] characterized in that it comprises determination means for determining the customer's willingness to purchase by applying the feature extracted by the second extraction means to the learning model created by the creation means. The information processing apparatus described.

[A3] The creation unit uses a collation result between the feature of the customer's body or face in the image at the checkout counter or the store exit and the feature extracted by the extraction unit as the teacher signal. The information processing apparatus according to [A1] or [A2].

[A4] The advertisement uses video.
The extraction means extracts features in a still image corresponding to each frame of the video,
The creating means generates a learning model based on the teacher signal to a feature in a still image corresponding to each frame,
A frame having a high degree of being determined to be purchased in the still image corresponding to the frame is determined as a frame having a purchase effect in the video of the advertisement, or a degree of being determined to be purchased in the still image corresponding to the frame. The information processing apparatus according to any one of [A1] to [A3], further comprising: a determination unit that determines a low frame as a frame having no purchase effect in the video of the advertisement.

[A5] Any one of [A1] to [A4], wherein the creating means uses whether or not the customer touches the product as a teacher signal indicating whether or not the customer is interested in the product. The information processing apparatus according to item.

[A6]
An extraction means for extracting the characteristics of the body or face of the customer in the image obtained by photographing the place where the product advertisement is performed;
An information processing program for functioning as a creation means for creating a learning model using information indicating whether or not the customer whose features have been extracted by the extraction means has been purchased as a teacher signal.

And the above-mentioned invention has the following effects.
According to the information processing apparatus of [A1], information indicating whether or not the customer has purchased the feature of the body or face of the customer in the image obtained by photographing the place where the product is advertised is used as a teacher signal. A learning model can be created.

  According to the information processing apparatus of [A2], the customer's willingness to purchase is determined by applying, to the learning model, the customer's body or facial features in the image obtained by photographing the place where the product is advertised. Can do.

  According to the information processing apparatus [A3], it is possible to use, as a teacher signal, a matching result of the customer's body or facial features in the image at the checkout counter or the store exit.

  According to the information processing apparatus [A4], it is possible to determine a frame having a purchase effect or a frame having no purchase effect in an advertisement video.

  According to the information processing apparatus of [A5], whether or not a customer touches a product can be used as a teacher signal indicating whether or not there is an interest in the product.

  According to the information processing program of [A6], information indicating whether or not the customer has purchased for the feature of the body or face of the customer in the image obtained by photographing the place where the product is advertised is used as a teacher signal. A learning model can be created.

DESCRIPTION OF SYMBOLS 100 ... Information processing (determination) apparatus 110 ... Promotion apparatus 120 ... Imaging apparatus 130 ... Human / face detection module 140 ... Image feature extraction module 150 ... Customer purchase willingness determination module 155 ... Learning model storage module 160 ... Promotion selection module 200 ... Information Processing (learning) device 210 ... Promotion device 220 ... Imaging device 230 ... Human / face detection module 240 ... Image feature extraction module 250 ... Image feature DB
260 ... Teacher signal creation module 270 ... Learning model creation storage module 290 ... Communication line 300 ... Information processing (judgment / learning) device 400 ... Store system 410 ... Store (learning) system 930 ... Face detection module 962 ... Cash register imaging device 964 ... Face detection module 966 ... Face verification module 1360 ... Teacher signal creation module 1362 ... Product installation location imaging device 1364 ... Face detection module 1366 ... Action detection module 1560 ... Teacher signal creation module 1562 ... Exit imaging device 1564 ... Face detection module

Claims (2)

A filming department that shoots customers in locations where they advertise products,
An extraction unit for extracting the movement of the customer and the product that the customer is interested in from the photographing result of the photographing unit;
A selection unit that selects an advertisement according to the customer's interest in the product extracted from the movement of the customer;
An information processing apparatus. Computer
A filming department that shoots customers in locations where they advertise products,
An extraction unit for extracting the movement of the customer and the product that the customer is interested in from the photographing result of the photographing unit;
An information processing program for functioning as a selection unit that selects an advertisement according to the customer's interest in the product extracted from the customer's movement.

Images