JP6476678B2 - Information processing apparatus and information processing program - Google Patents

Description

  The present invention relates to an information processing apparatus and an information processing program.

  In Patent Document 1, it is an object to recommend content that truly reflects the user's preference based on features extracted from the user's daily behavior tendency, and the PC is a plurality of images extracted from still images or moving images. The personal meta DB is constructed based on the lower meta information and the upper meta information derived from each of the lower meta information, and any personal characteristic information of the user is selected from the personal meta DB and transmitted to the server. The personal feature information similar to the received personal feature information is extracted from the multi-person statistical meta DB constructed by statistically processing the personal meta DB of another PC, and a recommended product or recommendation service related to the personal feature information is extracted. Is selected and transmitted to a PC.

  Patent Document 2 has an object to acquire information useful for marketing from the customer's line of sight, and the product sales apparatus includes a product sample display unit that displays the product being sold in an identifiable manner and at least the face of the customer. A camera that shoots a portion, and an information processing device that acquires and processes video captured by the camera, and the information processing device determines the direction of the customer's face and the position of the pupil included in the video acquired from the camera. Analyzing and detecting the customer's line of sight, and detecting the product sample display section that the detected customer's line of sight gazes, and the customer attribute of the customer based on the product history information indicated by the product sample display section that the customer watches In addition, it generates gaze product data that aggregates the number of gazes for products that the customer did not purchase, and this gaze product data is held in the customer's mind. Is intended to represent the interest of the goods, it is disclosed that is useful information to, such as future product sales plan.

  Patent Document 3 aims to provide a clothing-corresponding service providing apparatus capable of providing a sufficient service to an occupant, and is a clothing-corresponding service providing apparatus mounted on a vehicle. A clothing-corresponding service providing device that includes a clothing pattern recognition unit for recognizing and providing a service to an occupant based on a clothing pattern recognized by the clothing pattern recognition unit. It is disclosed that pattern matching processing and luminance / color matching processing are performed.

  In Patent Document 4, it is an object to be able to track a human motion in a room and estimate the human behavior / intention, and an imaging unit captures a human image in the room and images the captured human image. Is transmitted to the high-speed / low-resolution processing unit and the low-speed / high-resolution processing unit, and the high-speed / low-resolution processing unit, based on the human image captured by the imaging unit, The low-speed / high-resolution processing unit identifies the human gesture using the data accumulated in the knowledge accumulation unit based on the human image captured by the imaging unit, and the action / intention estimation unit However, based on the processing result by the high-speed / low-resolution processing unit and the processing result by the low-speed / high-resolution processing unit, it is disclosed to estimate the human behavior / intention that is the imaging target using Bayesian classification. ing.

JP 2011-215964 A JP 2009-042956 A JP 2008-120289 A JP 2005-242759 A

  An object of the present invention is to provide an information processing apparatus and an information processing program that present a second product to be combined with a product that has been grasped by a customer or a product similar to the product.

The gist of the present invention for achieving the object lies in the inventions of the following items.
According to the first aspect of the present invention, a detecting means for detecting an action in which the positional relationship between the customer's hand or line of sight and the product is within a predetermined range from an image obtained by photographing the customer is detected by the detecting means. Extraction means for extracting the product that is the target of the action , second extraction means for extracting the second product combined with the product extracted by the extraction means or a product similar to the product, and the second extraction A display means for presenting the second product extracted by the means , wherein the detection means detects a plurality of customer actions, and the extraction means is a target of the plurality of customer actions; A plurality of products are extracted, and the second extraction unit extracts a second product combined with the plurality of extracted products or a plurality of products similar to the extracted products. Yes, heavy An information processing apparatus characterized by preferentially selected second product extracted with.

  The invention of claim 2 is characterized in that the second extraction means extracts the second product using combination information of a product published in a magazine or combination information of a product taken up by a social network. The information processing apparatus according to claim 1.

  According to a third aspect of the present invention, the second extracting means uses the weather information, the sales information of the target store, the budget information of the customer, or a combination thereof, and the second product combined with the product. The information processing apparatus according to claim 1, wherein the information processing apparatus is extracted.

  The invention of claim 4 further comprises selection means for selecting an output device to be presented by the presenting means, using the second product and layout information in the target store. It is an information processing apparatus as described in any one of 1-3.

According to a fifth aspect of the present invention, there is provided a detection unit that detects an action in which a positional relationship between a customer's hand or line of sight and a product is within a predetermined range from an image obtained by photographing the customer . Extraction means for extracting the product that is the target of the detected action ; second extraction means for extracting a second product to be combined with the product extracted by the extraction means or a product similar to the product; The second product extracted by the two extraction means is functioned as a presentation means , the detection means detects a plurality of customer actions, and the extraction means is a target of the plurality of customer actions The second extracting means extracts a second product to be combined with the plurality of extracted products or a plurality of products similar to the plurality of extracted products. It is intended to output an information processing program that preferentially selected second product extracted overlap.

According to the information processing apparatus of the first aspect, it is possible to present the second product to be combined with the product that is the target of the action by the customer or the product similar to the product.

  According to the information processing apparatus of the second aspect, it is possible to extract the second product using the combination information of the product published in the magazine or the combination information of the product taken up by the social network.

  According to the information processing apparatus of claim 3, the second product suitable for combining with the product using any one of weather information, sales information of the target store, customer budget information, or a combination thereof. Can be extracted.

  According to the information processing apparatus of the fourth aspect, the output device to be presented by the presenting means can be selected using the second product and the layout information in the target store.

According to the information processing program of the fifth aspect , it is possible to present the second product to be combined with the product that is the target of the action by the customer or the product similar to the product.

It is a conceptual module block diagram about the structural example of this Embodiment. It is explanatory drawing which shows the system configuration example using this Embodiment. It is a flowchart which shows the process example by this Embodiment. It is explanatory drawing which shows the example of a data structure of a customer interest table. It is explanatory drawing which shows the data structure example of an item table. It is explanatory drawing which shows the example of a data structure of an item similarity degree table. It is explanatory drawing which shows the example of a data structure of a coordination table. It is explanatory drawing which shows the example of a data structure of a coordination table. It is a flowchart which shows the process example by this Embodiment. It is a flowchart which shows the process example by this Embodiment. It is a flowchart which shows the process example by this Embodiment. It is a flowchart which shows the process example by this Embodiment. It is a flowchart which shows the process example by this Embodiment. It is a flowchart which shows the process example by this Embodiment. It is a block diagram which shows the hardware structural example of the computer which implement | achieves this Embodiment.

Hereinafter, an example of a preferred embodiment for realizing the present invention will be described with reference to the drawings.
FIG. 1 shows a conceptual module configuration diagram of a configuration example of the present embodiment.
The module generally refers to components such as software (computer program) and hardware that can be logically separated. Therefore, the module in the present embodiment indicates not only a module in a computer program but also a module in a hardware configuration. Therefore, the present embodiment is a computer program for causing these modules to function (a program for causing a computer to execute each procedure, a program for causing a computer to function as each means, and a function for each computer. This also serves as an explanation of the program and system and method for realizing the above. However, for the sake of explanation, the words “store”, “store”, and equivalents thereof are used. However, when the embodiment is a computer program, these words are stored in a storage device or stored in memory. It is the control to be stored in the device. In addition, the modules may correspond to the functions on a one-to-one basis. However, in implementation, one module may be configured by one program, or a plurality of modules may be configured by one program. A module may be composed of a plurality of programs. The plurality of modules may be executed by one computer, or one module may be executed by a plurality of computers in a distributed or parallel environment. Note that one module may include other modules. Hereinafter, “connection” is used not only for physical connection but also for logical connection (data exchange, instruction, reference relationship between data, etc.). “Predetermined” means that the process is determined before the target process, and not only before the process according to this embodiment starts but also after the process according to this embodiment starts. In addition, if it is before the target processing, it is used in accordance with the situation / state at that time or with the intention to be decided according to the situation / state up to that point. When there are a plurality of “predetermined values”, they may be different values, or two or more values (of course, including all values) may be the same. In addition, the description having the meaning of “do B when it is A” is used in the meaning of “determine whether or not it is A and do B when it is judged as A”. However, the case where it is not necessary to determine whether or not A is excluded.
In addition, the system or device includes a plurality of computers, hardware, devices, and the like connected by communication means such as a network (including one-to-one communication connection), etc., and one computer, hardware, The case where it implement | achieves by an apparatus etc. is also included. “Apparatus” and “system” are used as synonymous terms. Of course, the “system” does not include a social “mechanism” (social system) that is an artificial arrangement.
In addition, when performing a plurality of processes in each module or in each module, the target information is read from the storage device for each process, and the processing result is written to the storage device after performing the processing. is there. Therefore, description of reading from the storage device before processing and writing to the storage device after processing may be omitted. Here, the storage device may include a hard disk, a RAM (Random Access Memory), an external storage medium, a storage device via a communication line, a register in a CPU (Central Processing Unit), and the like.

  The information processing apparatus 100 according to the present embodiment presents a second product to be combined with a product (hereinafter also referred to as an item) that has been grasped by a customer or a product similar to the product. As shown in the example of FIG. 1, the customer interest extraction module 110, the magazine item extraction module 130, the social media information extraction module 135, the weather information extraction module 140, the sales history extraction module 145, the customer budget estimation module 150, and the combination creation module 155. A combination storage module 160, a layout recommendation module 165, a layout storage module 170, and a presentation module 175.

The imaging device 105 is connected to the image reception module 115 of the information processing device 100. The photographing device 105 photographs a product and the periphery of the product (also referred to as a product area). In order to purchase the product and confirm the product, the customer enters the product area. For this reason, an image taken by the customer interest extraction module 110 may include a customer along with the product.
Specifically, the photographing device 105 is a 2D camera, a 3D camera, a thermo camera, an omnidirectional camera, or the like for photographing a still image or a moving image. Further, the photographing apparatus 105 may be fixed, or may be capable of changing the angle, position, and the like. Further, the relationship between the product and the photographing device 105 may be one-to-one, one-to-N, N-to-one, or N-to-N. . In the case of N-to-one, as will be described later, the product in the image is recognized, and the behavior of the customer may be determined for each product. In the case of 1 to N, it is only necessary to determine an action with respect to one product or a tag of the product from a plurality of images, and obtain a final determination result from the plurality of determination results using majority vote or the like.

The visit customer interest extraction module 110 includes an image reception module 115, an analysis module 120, and an item extraction module 125, and is connected to the combination creation module 155.
The image reception module 115 is connected to the imaging device 105 and the analysis module 120. The image reception module 115 receives an image captured by the imaging device 105 and passes the image to the analysis module 120.
The analysis module 120 is connected to the image reception module 115 and the item extraction module 125. The analysis module 120 detects an action in which the customer grips the product from an image obtained by photographing the customer. In addition to product recognition as described above, for example, product position, customer body, customer head position, hand (including arm) position, gaze direction, face position, eyes, etc. Extract. For example, the product photographed by the photographing apparatus 105 is predetermined, and the characteristics (color, shape, size, etc.) of the predetermined product are extracted in advance and stored as a dictionary. The product can be recognized from within the image. Further, the product may be recognized by recognizing a tag or the like attached to the product. In addition, the recognition of a person, the recognition of a person's part (hand, head, etc.), the direction of the line of sight, etc. are recognized using existing techniques. In addition, as a position, the position in the coordinate system in an image may be sufficient, and a relative position with respect to goods may be sufficient. Then, when the position of the person's hand and the product is within a predetermined range (including the case where they overlap), it may be detected as an action in which the customer grips the product. Further, it may be detected as an action in which the customer grips the product on condition that the direction of the line of sight is directed to a certain position of the product or a predetermined range including the product.
For techniques such as human detection and face detection from images, see “Yamauchi Akira, Yamashita Takayoshi, Fujiyoshi Hironobu. Human Detection Based on Statistical Learning Techniques from Images. IEICE Transactions D, Vol. J96-D, No. 9, pp. 2017-2040, 2013. ”,“ Kyosei Hiroshi, Osamu Yamaguchi. Application examples of facial image processing (facial image processing and its application examples (first part) where practical use advances) . Information Processing 50 (4), 319-326, 2009. "etc. may be used.

  The item extraction module 125 is connected to the analysis module 120. The item extraction module 125 extracts the product that is the object of gripping detected by the analysis module 120. Then, the extracted product (information indicating) is passed to the combination creation module 155. As described above, when the relationship between the product and the imaging device 105 is 1: 1, when the grasping behavior of the product is detected by the analysis module 120, the product corresponding to 1: 1 may be extracted. In other cases, it is only necessary to recognize the product to be grasped as described above.

The combination creation module 155 includes a visit customer interest extraction module 110, a magazine item extraction module 130, a social media information extraction module 135, a weather information extraction module 140, a sales history extraction module 145, a customer budget estimation module 150, a combination storage module 160, a layout. The recommendation module 165 and the presentation module 175 are connected. The combination creation module 155 extracts a second product to be combined with the product extracted by the item extraction module 125 or a product similar to the product. The combination here is (1) a combination of the product extracted by the item extraction module 125 and the second product, or (2) a product similar to the product extracted by the item extraction module 125 and the second product. It is a combination with a product. In addition, the second product is extracted and combined with the grasped product or a product similar to the product to create a coordinate. The coordination may be a combination of a plurality of products for one customer or a combination of a plurality of products for a plurality of customers. In the latter case, the product is proposed to a plurality of people. For example, if the customer is a couple of men and women, a product that is pair-looked may be selected. Further, the coordinate of the greatest common divisor may be calculated from the preferences of a plurality of customers in a certain area, and the coordinate may be proposed and displayed on a finite medium (signage or store clerk). This will be described more specifically. The analysis module 120 determines the behavior of the plurality of customers (the behavior of gripping a product) from images (a single image or a plurality of images) obtained by photographing a plurality of customers. To detect. Then, the item extraction module 125 extracts a plurality of products held by a plurality of customers. Next, the combination creation module 155 extracts a second product to be combined with a plurality of products extracted by the item extraction module 125 or a plurality of products similar to the extracted products. In general, there are a plurality of second products to be extracted, but the second product extracted in duplicate may be selected with priority. In that case, you may make it select as a goods for combining in order with many extraction frequency | counts.
The product combination table stored in the combination storage module 160 may be used to extract the second product. The combination table stores, for example, a plurality of product IDs (information for uniquely identifying products (ID: IDentification) in the present embodiment) for each row so as to indicate combinations of products suitable as coordinates. It is a thing. The product ID of the grasped product may be searched from the combination table, and the product ID of the product suitable for coordination may be extracted. Furthermore, for each combination of products, a degree of coordination indicating that the combination is suitable is defined in advance, the degree of coordination when the products are combined is extracted, and the value is greater than or equal to a predetermined threshold value. In such a case, the combination may be selected.

In addition, the combination creation module 155 is configured to display combination information (social result of the magazine item extraction module 130) of a product published in a magazine or combination information (social) of a product taken up in a social network (hereinafter also referred to as social media). The second product may be extracted using the processing result by the media information extraction module 135. What is stored in the combination storage module 160 is a predetermined combination of products. The processing result by the magazine item extraction module 130 or the processing result by the social media information extraction module 135 is stored in the combination table. Alternatively, or in addition to the combination table, the second product may be extracted.
The combination storage module 160 is connected to the combination creation module 155. The combination storage module 160 stores the above-described product combination table. Further, information from the magazine item extraction module 130 and the social media information extraction module 135 may be stored.

The magazine item extraction module 130 is connected to the combination creation module 155. The magazine item extraction module 130 reflects the magazine publication information and the magazine publication combination information on the combination storage module 160. These pieces of information are used by the combination creation module 155. For example, you may extract from the server which has memorize | stored the coordination information (combination information of goods) published in the fashion magazine. Further, as described above, products may be extracted from images published in fashion magazines, and a combination of products in one image may be used as coordinate information.
The social media information extraction module 135 is connected to the combination creation module 155. The social media information extraction module 135 extracts a frequency and a combination of products that are taken up by the social media from the social media information, and reflects them in the combination storage module 160. These pieces of information are used by the combination creation module 155. For example, the product A may be detected from text in a social networking service related to a predetermined topic, the other product B described along with the product A may be extracted, and the product A and the product B may be combined information. . Further, as described above, a product may be extracted for an image in a social networking service related to a predetermined topic, and a combination of products in one image may be used as coordinate information. Moreover, the combination of the goods is good also as conditions on having taken up more than a predetermined threshold value.

The combination creation module 155 also includes weather information (processing result by the weather information extraction module 140), sales information of the target store (processing result by the sales history extraction module 145), or customer budget information (by the customer budget estimation module 150). The second product to be combined with the product may be extracted using any one of the processing results) or a combination thereof. For example, when there are a plurality of second products suitable for coordination, a product suitable for combination is selected using these weather information and the like.
For example, when information indicating that it is raining is obtained as weather information, the second product may be combined with priority on umbrellas.
For example, if the sales information is a product with high sales, a product with low sales, etc., the product with high sales (if there is a purpose to further increase the sales of the product), the product with low sales (the purpose of achieving the sales target) Case) or the like may be combined as a second product. Note that “a product with high sales” means that the sales of products are higher than or higher than a predetermined threshold, and “a product with low sales” means that sales of products are less than a predetermined threshold or Both thresholds, which are the following, may be different. Moreover, the threshold value may be predetermined for each product.
Further, for example, product groups that fall within the range of customer budget information (target customer budget) may be preferentially combined.

The weather information extraction module 140 is connected to the combination creation module 155. The weather information extraction module 140 extracts temperature, humidity, weather information, etc., which are weather information in the area of the target store, from the server that measures the weather, and passes them to the combination creation module 155. Moreover, you may make it extract weather information from the weather sensor installed around the shop used as object.
The sales history extraction module 145 is connected to the combination creation module 155. The sales history extraction module 145 extracts sales information for each product from a point-of-sale system (POS) of the target store, and extracts products with high sales, products with low sales, and the like.
The customer budget estimation module 150 is connected to the combination creation module 155. The customer budget estimation module 150 identifies the customer from the face detection of the customer who visits the store (the customer in the image captured by the imaging device 105), and uses the customer's past purchase list to estimate the customer budget. It may be. In addition, the face image is extracted from the image, the age, sex, etc. are extracted from the face image, the clothes part is extracted from the image, the similar customers are searched from the age, sex, etc., or the clothes tendency, and the customer You may make it estimate using the past purchase list. The latter may be limited to new customers.

The layout recommendation module 165 is connected to the combination creation module 155, the layout storage module 170, and the presentation module 175. The layout recommendation module 165 selects the presentation device 195 that is an output device to be presented by the presentation module 175 using the second product and the layout information in the target store. For example, presenting the second product to a digital signage (digital signage) in the vicinity of the product area held by the customer or a terminal possessed by a store clerk in the vicinity of the product area You may make it display. When displaying on the digital signage, the location of the product area where the second product is located (such as the route from the location where the customer is currently located to the product area where the second product is located) may be indicated. Or you may make it display on the terminal etc. which the salesclerk in the vicinity of the goods area with a 2nd goods possesses. In this case, you may make it display the merchandise area in which the customer who is the object further exists.
The layout storage module 170 is connected to the layout recommendation module 165. The layout storage module 170 stores layout information in the target store. Further, a route from one product area to another product area may be stored in advance.
The presentation module 175 is connected to the combination creation module 155, the layout recommendation module 165, and the presentation device 195. The presentation module 175 presents the second product extracted by the combination creation module 155. When the presentation device 195 is digital signage, the customer directly sees the second product (a product suitable for combining with the product being held). When the presentation device 195 is a portable terminal or the like possessed by a store clerk, the store clerk proposes a second product to the customer.
The presentation device 195 is connected to the presentation module 175 of the information processing device 100. The presentation device 195 is a digital signage installed in a target store, a mobile terminal possessed by a store clerk, or the like.

FIG. 2 is an explanatory diagram showing a system configuration example using the present embodiment.
The information processing apparatus 100, the A store system 210A, and the B store system 210B are connected to each other via a communication line 290. The communication line 290 may be wireless, wired, or a combination thereof, for example, the Internet as a communication infrastructure.
The A store system 210A includes an imaging device 105A and a presentation device 195A. An image captured by the imaging apparatus 105A may be transmitted to the information processing apparatus 100 via the communication line 290, and a processing result by the information processing apparatus 100 may be transmitted to the presentation apparatus 195A. You may install the information processing apparatus 100 as what builds cloud computing.
The store B system 210B includes an information processing apparatus 100B, a photographing apparatus 105B1, a photographing apparatus 105B2, a presentation apparatus 195B1, and a presentation apparatus 195B2. Thus, you may make it install the information processing apparatus 100 in each store.

FIG. 3 is a flowchart showing an example of processing according to this embodiment.
In step S302, the imaging device 105 images a customer.
In step S304, the image reception module 115 receives an image.
In step S306, the analysis module 120 analyzes the image.
In step S308, the item extraction module 125 extracts an item taken by the customer. For example, the customer interest table 400 is generated as a processing result by the item extraction module 125. FIG. 4 is an explanatory diagram showing an example of the data structure of the customer interest table 400. The customer interest table 400 has a customer ID column 410, a category column 420, and an item column 430. In the present embodiment, customer ID column 410 stores information (customer ID) for uniquely identifying a customer. The category column 420 stores information indicating the category (tops, pants, etc.) to which the item picked up by the customer belongs. The item column 430 stores information (item ID) indicating an item of an item picked up by the customer.

In step S310, the combination creation module 155 uses the combination storage module 160 to create a coordinate for the target customer. The combination creation module 155 generates the coordinate table 800 from the customer interest table 400 using, for example, the item table 500, the item similarity table 600, and the coordinate table 700. In the combination storage module 160, an item table 500, an item similarity table 600, and a coordination table 700 are stored. FIG. 5 is an explanatory diagram showing an example of the data structure of the item table 500. The item table 500 has an ID column 510, a Family column 520, a price column 530, and a sales column 540. The ID column 510 stores item IDs. The Family column 520 stores the family to which the item belongs. The price column 530 stores the price of the item. Used when generating coordinates using budget information. The sales column 540 stores the (current) sales of the item. Information from the sales history extraction module 145 is stored.
FIG. 6 is an explanatory diagram showing an example of the data structure of the item similarity table 600. The item similarity table 600 has an ID1 column 610, an ID2 column 620, and a similarity column 630. The ID1 column 610 stores an item ID (1). The ID2 column 620 stores an item ID (2) of an item similar to the product with the item ID (1). The similarity column 630 stores the similarity between the items. The combination creation module 155 is used to extract an item similar to the item grasped by the customer. For example, an item group having a similarity that is higher or higher than a predetermined threshold may be determined to be similar. Of course, you may make it show the similarity of three or more items.
FIG. 7 is an explanatory diagram showing an example of the data structure of the coordinate table 700. The coordinate table 700 has an ID1 column 710, an ID2 column 720, an in-store frequency column 730, and a date column 740. The combination of items is memorized. The ID1 column 710 stores an item ID (1). The ID2 column 720 stores an item ID (2) of an item to be combined with the item with the item ID (1). The in-store frequency column 730 stores the frequency at which the combination is proposed (or the frequency at which the combination is purchased). The date column 740 stores a proposed (or purchased) date. The combination creation module 155 searches the items held by the customer from the ID1 column 710 or the ID2 column 720, and extracts items to be combined (ID1 column 710 or ID2 column 720). Further, a column for the frequency of publication in the magazine and the frequency in social media may be provided, and the combination creation module 155 selects the combination when the frequency is greater than or equal to a predetermined threshold. You may make it employ | adopt. Further, a field for storing weather information indicating the weather for which the combination is suitable may be provided, and the combination creation module 155 may select a combination of weather information that matches the current weather information. .

  The combination creation module 155 generates a coordinate table 800 as a processing result. FIG. 8 is an explanatory diagram showing an example of the data structure of the coordinate table 800. The coordination table 800 has a customer ID column 810, a category column 820, and an item column 830. The customer ID column 810 stores a customer ID. The category column 820 stores item categories. The item column 830 stores items proposed to the customer (including items held by the customer). That is, a plurality of items (a group of products combined as coordinates) correspond to one customer.

For example, the combination creation module 155 calculates a coordinate score as in the following equation.
Coordinate score = TOPS (T-001, T-005, T-002) | PANTS (P-022, P-010) | OUTER (O-031) | SHOSE (S-304) | BAG (B-400, B -420)
This equation calculates a coordinate score when items are combined for each category. Tops items (T-001, T-005, T-002 are similar to each other), PANTS items (P-022, P-010 are similar to each other) ), OUTER (outer) item (O-031), SHOSE (shoes) item (S-304), BAG (bag) items (assuming B-400 and B-420 are similar to each other) The coordinate score when combined is shown. Note that similar items have the same score.
The combination creation module 155 extracts a combination having a high score from all the combinations from the coordinate scores of each combination.
Furthermore, an item may be selected from similar items using magazine information, social media information, weather information, sales information, and budget information.

  In step S312, the presentation module 175 presents a coordinate. For example, the presentation module 175 presents an item group to be proposed for each customer from the coordination table 800. As a specific example, as shown in the example of FIG. 14, a coordinate between a recommended item image 1410 (blouson Q) and a recommended item image 1420 (pants R), which is a processing result by the combination creation module 155, is presented in the recommended image 1400. Further, the face image in the recommended image 1400 may be a customer's face image.

FIG. 9 is a flowchart showing an example of processing according to the present embodiment. In this processing example, step S910 is added to the flowchart shown in the example of FIG. 3, and the processing result of step S910 is used in step S912.
In step S902, the photographing apparatus 105 photographs a customer.
In step S904, the image reception module 115 receives an image.
In step S906, the analysis module 120 analyzes the image.
In step S908, the item extraction module 125 extracts items that the customer has picked up.
In step S910, the magazine item extraction module 130 extracts item information published in a magazine related to the handling item of the target store.
In step S912, the combination creation module 155 creates a coordinate with the target customer using the item information extracted in step S910 and the combination storage module 160.
In step S914, the presentation module 175 presents a coordinate.

FIG. 10 is a flowchart showing an example of processing according to this embodiment. In this processing example, step S1010 is added to the flowchart shown in the example of FIG. 3, and the processing result of step S1010 is used in step S1012.
In step S1002, the imaging device 105 images a customer.
In step S1004, the image reception module 115 receives an image.
In step S1006, the analysis module 120 analyzes the image.
In step S1008, the item extraction module 125 extracts items that the customer has picked up.
In step S1010, the social media information extraction module 135 extracts item information taken up by social media related to the handling items of the target store.
In step S1012, the combination creation module 155 creates a coordinate with the target customer using the item information extracted in step S1010 and the combination storage module 160.
In step S1014, the presentation module 175 presents a coordinate.

FIG. 11 is a flowchart showing an example of processing according to the present embodiment. In this processing example, step S1110 is added to the flowchart shown in the example of FIG. 3, and the processing result of step S1110 is used in step S1112.
In step S1102, the photographing apparatus 105 photographs a customer.
In step S1104, the image reception module 115 receives an image.
In step S1106, the analysis module 120 analyzes the image.
In step S1108, the item extraction module 125 extracts an item taken by the customer.
In step S1110, the weather information extraction module 140 extracts current weather information around the target store.
In step S1112, the combination creation module 155 creates a coordinate with the target customer using the weather information extracted in step S1110 and the combination storage module 160.
In step S1114, the presentation module 175 presents a coordinate.

FIG. 12 is a flowchart illustrating a processing example according to the present exemplary embodiment. In this processing example, step S1210 is added to the flowchart shown in the example of FIG. 3, and the processing result of step S1210 is used in step S1212.
In step S1202, the photographing apparatus 105 photographs a customer.
In step S1204, the image reception module 115 receives an image.
In step S1206, the analysis module 120 analyzes the image.
In step S1208, the item extraction module 125 extracts items that the customer has picked up.
In step S1210, the sales history extraction module 145 extracts the sales information of the target store.
In step S <b> 1212, the combination creation module 155 creates a coordinate with the target customer using the sales information extracted in step S <b> 1210 and the combination storage module 160.
In step S1214, the presentation module 175 presents a coordinate.

FIG. 13 is a flowchart illustrating a processing example according to the present exemplary embodiment. In this processing example, step S1310 is added to the flowchart shown in the example of FIG. 3, and the processing result of step S1310 is used in step S1312.
In step S1302, the photographing apparatus 105 photographs a customer.
In step S1304, the image reception module 115 receives an image.
In step S1306, the analysis module 120 analyzes the image.
In step S1308, the item extraction module 125 extracts items that the customer has picked up.
In step S1310, the customer budget estimation module 150 estimates target customer budget information.
In step S1312, the combination creation module 155 creates coordinates for the target customer using the budget information estimated in step S1310 and the combination storage module 160.
In step S1314, the presentation module 175 presents a coordinate.
Note that two or more of the flowcharts shown in the examples of FIGS. 9 to 13 may be combined.

  A hardware configuration example of the information processing apparatus according to the present embodiment will be described with reference to FIG. The configuration shown in FIG. 15 is configured by a personal computer (PC), for example, and shows a hardware configuration example including a data reading unit 1517 such as a scanner and a data output unit 1518 such as a printer.

  A CPU (Central Processing Unit) 1501 includes various modules described in the above-described embodiments, that is, an image reception module 115, an analysis module 120, an item extraction module 125, a magazine item extraction module 130, a social media information extraction module 135, Performs processing according to a computer program that describes the execution sequence of each module such as the weather information extraction module 140, sales history extraction module 145, customer budget estimation module 150, combination creation module 155, layout recommendation module 165, presentation module 175, etc. It is a control part.

  A ROM (Read Only Memory) 1502 stores programs used by the CPU 1501, calculation parameters, and the like. A RAM (Random Access Memory) 1503 stores programs used in the execution of the CPU 1501, parameters that change as appropriate during the execution, and the like. These are connected to each other by a host bus 1504 including a CPU bus or the like.

  The host bus 1504 is connected to an external bus 1506 such as a PCI (Peripheral Component Interconnect / Interface) bus via a bridge 1505.

  A keyboard 1508 and a pointing device 1509 such as a mouse are input devices operated by an operator. The display 1510 includes a liquid crystal display device or a CRT (Cathode Ray Tube), and displays various types of information as text or image information.

  An HDD (Hard Disk Drive) 1511 includes a hard disk, drives the hard disk, and records or reproduces a program executed by the CPU 1501 and information. The hard disk stores an image photographed by the photographing device 105, items extracted by the item extraction module 125, combination results by the combination creation module 155, information extracted by the magazine item extraction module 130, and the like. Further, various computer programs such as various other data processing programs are stored.

  The drive 1512 reads out data or a program recorded on a removable recording medium 1513 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, and the data or program is read out as an interface 1507 and an external bus 1506. , To the RAM 1503 connected via the bridge 1505 and the host bus 1504. The removable recording medium 1513 can also be used as a data recording area similar to the hard disk.

  The connection port 1514 is a port for connecting the external connection device 1515 and has a connection unit such as USB and IEEE1394. The connection port 1514 is connected to the CPU 1501 and the like via an interface 1507, an external bus 1506, a bridge 1505, a host bus 1504, and the like. The communication unit 1516 is connected to a communication line and executes data communication processing with the outside. The data reading unit 1517 is a scanner, for example, and executes document reading processing. The data output unit 1518 is, for example, a printer, and executes document data output processing.

  Note that the hardware configuration of the information processing apparatus shown in FIG. 15 shows one configuration example, and the present embodiment is not limited to the configuration shown in FIG. 15, and the modules described in this embodiment are executed. Any configuration is possible. For example, some modules may be configured with dedicated hardware (for example, Application Specific Integrated Circuit (ASIC), etc.), and some modules are in an external system and connected via a communication line In addition, a plurality of systems shown in FIG. 15 may be connected to each other via communication lines so as to cooperate with each other. Further, it may be incorporated in a copying machine, a fax machine, a scanner, a printer, a multifunction machine (an image processing apparatus having any two or more functions such as a scanner, a printer, a copying machine, and a fax machine).

  Further, in the description of the above-described embodiment, “more than”, “less than”, “greater than”, and “less than (less than)” in a comparison with a predetermined value contradicts the combination. As long as the above does not occur, “larger”, “smaller (less than)”, “more than”, and “less than” may be used.

The program described above may be provided by being stored in a recording medium, or the program may be provided by communication means. In that case, for example, the above-described program may be regarded as an invention of a “computer-readable recording medium recording the program”.
The “computer-readable recording medium on which a program is recorded” refers to a computer-readable recording medium on which a program is recorded, which is used for program installation, execution, program distribution, and the like.
The recording medium is, for example, a digital versatile disc (DVD), which is a standard established by the DVD Forum, such as “DVD-R, DVD-RW, DVD-RAM,” and DVD + RW. Standard “DVD + R, DVD + RW, etc.”, compact disc (CD), read-only memory (CD-ROM), CD recordable (CD-R), CD rewritable (CD-RW), Blu-ray disc ( Blu-ray (registered trademark) Disc), magneto-optical disk (MO), flexible disk (FD), magnetic tape, hard disk, read-only memory (ROM), electrically erasable and rewritable read-only memory (EEPROM (registered trademark)) )), Flash memory, Random access memory (RAM) SD (Secure Digital) memory card and the like.
The program or a part of the program may be recorded on the recording medium for storage or distribution. Also, by communication, for example, a local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN), a wired network used for the Internet, an intranet, an extranet, or a wireless communication It may be transmitted using a transmission medium such as a network or a combination of these, or may be carried on a carrier wave.
Furthermore, the program may be a part of another program, or may be recorded on a recording medium together with a separate program. Moreover, it may be divided and recorded on a plurality of recording media. Further, it may be recorded in any manner as long as it can be restored, such as compression or encryption.

DESCRIPTION OF SYMBOLS 100 ... Information processing apparatus 105 ... Imaging device 110 ... Visit customer interest extraction module 115 ... Image reception module 120 ... Analysis module 125 ... Item extraction module 130 ... Magazine item extraction module 135 ... Social media information extraction module 140 ... Weather information extraction module 145 ... sales history extraction module 150 ... customer budget estimation module 155 ... combination creation module 160 ... combination storage module 165 ... layout recommendation module 170 ... layout storage module 175 ... presentation module 195 ... presentation apparatus 210A ... A store system 210B ... B store system 290 ... communication line

Claims (5)

Detecting means for detecting an action in which the positional relationship between the customer's hand or line of sight and the product is within a predetermined range from an image of the customer photographed;
Extraction means for extracting the product that is the target of the action detected by the detection means;
Second extraction means for extracting a second product to be combined with the product extracted by the extraction means or a product similar to the product;
Presenting means for presenting the second product extracted by the second extracting means ;
The detection means detects a plurality of customer actions,
The extraction means is for extracting a plurality of commodities that are targets of the actions of the plurality of customers,
The second extraction means is configured to extract a second product combined with the plurality of extracted products or a plurality of products similar to the extracted plurality of products. An information processing apparatus characterized by preferentially selecting a product . The second extraction means extracts the second product using combination information of products published in a magazine or combination information of products taken up by a social network. The information processing apparatus described in 1. The second extracting means extracts the second product to be combined with the product by using any one of weather information, sales information of the target store, budget information of the customer, or a combination thereof. The information processing apparatus according to claim 1 or 2. 4. The apparatus according to claim 1, further comprising a selection unit that selects an output device to be presented by the presenting unit using the second product and layout information in the target store. 5. The information processing apparatus according to item. Computer
Detecting means for detecting an action in which the positional relationship between the customer's hand or line of sight and the product is within a predetermined range from an image of the customer photographed;
Extraction means for extracting the product that is the target of the action detected by the detection means;
Second extraction means for extracting a second product to be combined with the product extracted by the extraction means or a product similar to the product;
Function as a presenting means for presenting the second product extracted by the second extracting means ;
The detection means detects a plurality of customer actions,
The extraction means is for extracting a plurality of commodities that are targets of the actions of the plurality of customers,
The second extraction means is configured to extract a second product combined with the plurality of extracted products or a plurality of products similar to the extracted plurality of products. An information processing program that preferentially selects products .

Images