JP6069565B1 - RECOMMENDATION DEVICE, RECOMMENDATION METHOD, AND PROGRAM - Google Patents

Abstract

To a customer who has visited a store, a recommended product based on an item possessed by the customer or a sales product viewed by the customer cannot be easily recommended. A product information storage unit that can store two or more product information related to a product, a photo reception unit that receives a photograph of an article, and owner information related to the owner of the article by recognizing a photograph. An owner information acquisition unit 105 that acquires the product information, and a recommendation unit 106 that acquires, from the product information storage unit, product information of a type of product corresponding to the owner information acquired by the owner information acquisition unit and outputs the product information. The recommendation device 1 can effectively recommend recommended products to customers who have visited the store. [Selection] Figure 2

Description

  The present invention relates to a recommendation device that proposes a product recommended for a customer.

  2. Description of the Related Art Conventionally, a recommendation device that proposes a product recommended for a customer (hereinafter referred to as “recommended product”) in face-to-face sales at an actual store has been proposed (for example, paragraphs [0011] to [0026] in Patent Document 1). FIG. 1).

JP 2009-247926 A

  For example, in a retail store that handles clothing-related products, for example, when a customer chooses a top, the store clerk proposes a bottom that matches the top with consideration for color, overall silhouette, and fashion. Enhancing customer buying motivation is often done.

  The recommendation device described in Patent Document 1 can propose recommended products related to products requested by customers, but with this recommendation device, it is only necessary to propose recommended products with a high purchase probability from the purchase history of customers at the store. As a result, it was not enough as a proposal for recommended products.

  In addition, even in the case of a recommended product proposal by a store clerk, there are many variations depending on the knowledge and skills of the store clerk, and it has been difficult to appropriately propose a recommended product. Further, it has been impossible to select a recommended product using information on products owned by the customer or information on products that the customer is interested in in the store.

  The recommendation device according to the first aspect of the present invention includes a product information storage unit that can store two or more product information about a product, a photo reception unit that receives a photograph of an article, and an owner of the article by recognizing a photograph. An owner information acquisition unit that acquires owner information regarding, and a recommendation unit that acquires and outputs product information of a type of product corresponding to the owner information acquired by the owner information acquisition unit from the product information storage unit, and It is a recommendation device provided.

  With this configuration, it is possible to propose to the customer a type of product corresponding to the owner information of the item shown in the photograph.

  Further, in the recommendation device according to the second aspect of the present invention, in the recommendation device, the recommendation unit indicates that the owner information acquired by the owner information acquisition unit indicates that the owner of the article is the customer side. In the case of owner information, it is a recommendation device that acquires product information related to a product of a type different from the type of product shown in the photograph from the product information storage unit and outputs the product information.

  With this configuration, if the owner of the article shown in the photograph is not the store side, a different type of product from the article can be proposed to the customer.

  Further, in the recommendation device according to the third aspect of the present invention, the recommendation unit is a second item that the owner information acquired by the owner information acquisition unit indicates that the owner of the article is the store side. In the case of owner information, it is a recommendation device that acquires and outputs product information of a product of the same type as the type of product shown in the photograph from the product information storage unit.

  With such a configuration, when the owner of the article shown in the photograph is on the store side, the same type of product as the article can be proposed to the customer.

  The recommendation device according to the fourth aspect of the invention is related to the second or third aspect of the invention, product related information related to the sales product of the store and position information related to the display position of each sales product in the store, From the location product information storage unit using the location information received by the location information receiving unit, the location information receiving unit receiving location information indicating the location of the customer, and the location information receiving unit. A product-related information acquisition unit that acquires product-related information of sales products that are displayed at a customer location in the store, and the recommendation unit also stores the product-related information acquired by the product-related information acquisition unit. It is a recommendation device which uses and determines the product information acquired from a product information storage part.

  With this configuration, if the owner of the item shown in the photo is the store side, it is the same type as the item shown in the photo, and the same type as the product viewed by the customer in the store Products can be proposed to the customer.

  The recommendation device according to the fifth aspect of the invention relates to the recommendation unit according to the fourth aspect of the invention, in which, when the owner information acquired by the owner information acquisition unit is the first owner information, the product related information acquisition unit This is a recommendation device that acquires product information of a product that is the same type of product as the product corresponding to the acquired product-related information and matches the product shown in the photograph from the product information storage unit, and outputs the product information.

  With this configuration, if the owner of the item shown in the photo is the customer side, it is a different type from the item shown in the photo, and the same type as the product seen by the customer in the store, It is possible to propose to the customer a product that matches the item shown in the photograph.

  The recommendation device according to the sixth aspect of the present invention also provides a purchase history storage unit that can store a customer's purchase history in association with terminal identification information, and a customer's mobile phone. A terminal identification information receiving unit for receiving terminal identification information for identifying the terminal device, and the recommendation unit uses the terminal identification information received by the terminal identification information receiving unit to purchase the customer from the purchase history storage unit. It is a recommendation device that acquires a history and determines product information to be acquired from the product information storage unit using the purchase history.

  With this configuration, it is possible to propose to the customer a product that takes into account the purchase history of the customer at the store.

  In addition, the recommendation device of the seventh invention further includes an attribute value acquisition unit that acquires one or more attribute values of the customer shown in the photograph with respect to any one of the first to sixth inventions, A recommendation part is a recommendation apparatus which determines the merchandise information acquired from a merchandise information storage part also using one or more attribute values of a customer.

  With this configuration, a product that takes into account the attribute value of the customer of the store can be proposed to the customer.

  The recommendation device according to the eighth aspect of the invention is directed to any one of the first to seventh aspects, wherein the photo accepting unit accepts a photo input from a photo taking device for taking a photo. The unit is a recommendation device that performs processing of acquiring and outputting product information from the product storage unit in conjunction with a photo shooting operation by the photo shooting device.

With such a configuration, for example, it is possible to propose a recommended product to a customer using a recommendation device, triggered by a store clerk taking a photograph with the photography device.
be able to.

  The recommendation device according to the ninth aspect of the invention differs from the eighth aspect of the invention in that the photographic device includes a motion detection unit that detects the motion of the photographic device, and a motion detection unit that detects the motion of the photographic device. And a camera unit that transitions to a shooting mode in which an article can be taken.

  With such a configuration, for example, it is possible to propose a recommended product to a customer using a recommendation device, triggered by an act such as a store clerk lifting a photography device.

  According to the recommendation device of the present invention, it is possible to propose to the customer a product of a type corresponding to the owner information of the article shown in the photograph.

Conceptual diagram of the recommendation system in the embodiment Block diagram of recommendation device and photo-taking device that composes the recommendation system The flowchart which shows the procedure of the output process of the merchandise information regarding the basic recommended goods of the recommendation apparatus which constitutes the recommendation system The flowchart which shows the procedure of the modified example of the output process of the merchandise information regarding the recommended goods of the recommendation apparatus which comprises the recommendation system. The figure which shows an example of the merchandise information stored in the merchandise information storage part of the recommendation apparatus which comprises the recommendation system The figure which shows an example of the recommended product corresponding table stored in the product information storage part of the recommendation apparatus which comprises the recommendation system The figure which shows an example of the table | surface which shows the relationship between the color of the article | item stored in the product information storage part of the recommendation apparatus which comprises the recommendation system, and the color suitable for the color The figure which shows an example of the list | wrist of the image which shows the shape of the sales goods stored in the position goods information storage part of the recommendation apparatus which comprises the recommendation system The figure which shows an example of the example of a display of the recommended goods displayed on the display screen of the photography apparatus which comprises the recommendation system The figure which shows the other example of a display example of the recommended goods displayed on the display screen of the photography apparatus which comprises the recommendation system The figure which shows an example of the acquisition condition table stored in the product information storage part of the recommendation apparatus which comprises the recommendation system The figure which shows an example of the merchandise information of the sales goods stored in the position merchandise information storage part of the recommendation apparatus which comprises the recommendation system The figure for demonstrating the information regarding the display position of the store where the recommendation system is arrange | positioned The figure which shows an example of the purchase history stored in the purchase history storage part of the recommendation apparatus which comprises the recommendation system The figure for demonstrating the information of the display position corresponding to the position where the customer in the store where the recommendation system is arrange | positioned exists Block diagram when the recommendation system has a single configuration Overview of the computer system for realizing the recommendation device that constitutes the recommendation system Block diagram of the computer system

  Hereinafter, embodiments of a recommendation device and the like according to the present invention will be described with reference to the drawings. In addition, since the component which attached | subjected the same code | symbol in embodiment performs the same operation | movement, description may be abbreviate | omitted again.

(Embodiment 1)
In the present embodiment, a photography apparatus for taking a photograph of an article possessed by a customer who has visited the store, an article worn by the customer, or a sale product displayed at the store, and the photograph Accepts a photo received from the photographing device, acquires owner information about the owner of the article in the photo, and outputs product information on one or more types of products according to the owner information (proposed to the customer) The recommendation system configured with the recommendation device will be described.

  When the recommendation device acquires the product information of the recommended product using the owner information of the article shown in the photograph, the sales product displayed near the location where the customer is in the store, the purchase history of the customer, Recommended products can be determined using information such as customer attribute values.

  In the present embodiment, the recommendation device recommends using, in addition to the owner information, information such as sales products displayed at the customer location in the store, customer purchase history, and customer attribute values. A recommendation system for acquiring and outputting product information related to a product will be described.

  FIG. 1 is a conceptual diagram of a recommendation system.

  The recommendation system shown in FIG. 1 includes a recommendation device 1 and a photography device 2. The recommendation device 1 is a device that outputs product information related to a product proposed or recommended to a customer (hereinafter referred to as “recommended product” as necessary). The recommendation device 1 is usually a server, but may be a so-called cloud server. The photography device 2 takes a photograph of, for example, an article possessed by a customer who has visited the store, an article worn by the customer, or a product sold at the store (a product displayed at the store). And a device for transmitting to the recommendation device 1.

  The photography device 2 may be a camera or a tablet terminal, a notebook computer, a mobile phone, a smartphone, or the like having a camera function. In the following description, an example in which a tablet terminal having a camera function is used as the photography apparatus 2 will be described.

  FIG. 2 is a block diagram of the recommendation device 1 and the photography device 2 that constitute the recommendation system.

  The recommendation device 1 includes a product information storage unit 101, a location product information storage unit 102, a purchase history storage unit 103, a photo reception unit 104, an owner information acquisition unit 105, a recommendation unit 106, a location information reception unit 107, and product related information acquisition. Unit 108, terminal identification information reception unit 109, and attribute value acquisition unit 110.

  The product information storage unit 101 can store two or more product information. The product information is information related to a product to be proposed as a recommended product. For example, a product identifier for identifying the product, a product name, a price (unit price), a product attribute (for example, gender and product target for the product) Age, application, type, color, size, shape, production area, raw materials, etc.), product sample photos or images, inventory information relating to inventory, EC information relating to electronic commerce, and the like. The product identifier may be referred to as a product number or a product ID, for example. The inventory information includes, for example, information for identifying a handling store and the number of the inventory. The EC information is a URL of a web page including product information.

  In addition, the product information storage unit 101 stores a recommended product correspondence table indicating the correspondence between owner information related to the owner of an article shown in the photograph and the recommended product.

  The owner information is, for example, a category for classifying the owner of an article shown in a photograph into a store side and a customer side, and when the article is a product sold by the store (when the owner of the article is a store) Contains information about the owner's category, such as the category that categorizes the product handled by, and the attribute category that classifies the customer if the item is an item worn by the customer (if the item owner is the customer) It is a concept.

  The category for classifying the products handled by the store is a category for classifying the products sold by the store (products owned by the store) into, for example, high-end products and popular products, and normal sales products and limited sales products. The attribute category for classifying customers is, for example, a category in which customers are classified by sex such as male and female, junior (young people), senior (middle and old), silver (old), and children.

  In the following description, a case will be described as an example where the owner information related to the owner of the article in the photograph is information related to a category that classifies the owner of the article into the store side and the customer side.

  The recommended product correspondence table is a product recommended when the owner of the article in the photograph is a customer (hereinafter referred to as “first owner information”) and the owner of the article is a store. This is a table showing recommended products in the case of information (hereinafter referred to as “second owner information”).

  The recommended product when the owner information is the first owner information is, for example, a product different from the type of the article shown in the photograph, for example, a product whose design elements such as color, shape, and pattern are suitable for the article. . In addition, the recommended product when the owner information is the second owner information is, for example, the same product as the type of the article shown in the photograph, for example, a product whose design elements such as color, shape, and pattern match the article It is.

  The design element that matches the article is determined in advance, for example. For example, when the design element is a color, for example, an adjacent color of red or a similar color (for example, pink or magenta) is set to a color that matches the article with respect to red. However, the design element suitable for the article may be dynamically determined based on the purchase history information, the action history information, and the like of the user's product.

  Therefore, the recommended product correspondence table stored in the product information storage unit 101 may be referred to as a correspondence table indicating the correspondence between the category of the owner of the article shown in the photograph and the type of the product.

  Furthermore, the product information storage unit 101 stores an acquisition condition table indicating acquisition conditions for determining the product information acquired from the product information storage unit 101. The acquisition conditions include, for example, conditions for using sales products displayed at the customer's location in the store, conditions for using customer purchase history, and conditions for using customer attribute values. It is.

  The acquisition condition (hereinafter referred to as “first acquisition condition”) when using the sales product displayed at the customer location in the store is, for example, the sales product displayed at the customer location in the store. The condition is that the same type of product is selected as the recommended product.

  In addition, the acquisition condition (hereinafter referred to as “second acquisition condition”) when the customer purchase history is used is, for example, an attribute value (for example, an attribute value such as a color, a size, or a shape) of a product having a large customer purchase history ) Is selected as a recommended product. An attribute value of a product with a large purchase history is, for example, an attribute value of a product whose purchase number is equal to or greater than a threshold.

  An acquisition condition (hereinafter referred to as “third acquisition condition”) in the case of using a customer attribute value is a condition that, for example, a product that matches the customer attribute value is selected as a recommended product. The customer attribute value of the third acquisition condition is, for example, the sex or age group of the customer.

  The position merchandise information storage unit 102 can store one or more sets of merchandise related information related to the sales merchandise in the store and position related information related to the display position of each sales merchandise in the store. The product related information is information that can specify the sales product at the display position, and is information such as the type of the sales product and identification information for identifying the sales product, for example.

  The position related information is information for specifying the position in the store where the sales commodity is displayed. The position related information includes, for example, display space identification information for identifying a display space in which sales merchandise is displayed, information indicating an arrangement area of the display space in the store, and the like.

  The information indicating the arrangement area of the display space in the store is, for example, information in which an XY coordinate system is set for the planar shape of the store and an area occupied by each display space is defined by the XY coordinates of the XY coordinate system. For example, when the shape of the space is rectangular, the information of the arrangement area in the store of the space is the XY coordinates (for example, (x1, y1), (x2, y2)) of the two corner positions of the space or the positions of the four corners. XY coordinates (for example, (x1, y1), (x2, y2), (x3, y3), (x4, y4)).

  The position-related information is not limited to the above-described information, and any content may be used as long as it is information that can specify the position in the store where the sales product is displayed.

  The position product information storage unit 102 displays, for example, display space identification information (identification number) of the position where the sales product is displayed, information (XY coordinates) indicating the display space arrangement area in the store, and product related information of the sales product Are stored in association with each other.

  Further, the position product information storage unit 102 stores one or more images indicating the shape of the product representing each type (the shape in the display state) for each type of the sales product displayed in the store. The position product information storage unit 102 stores type information of sales products in association with each image.

  For example, when a blouse is displayed, the position product information storage unit 102 stores blouse type information in association with an image showing the shape of the blouse in the display state. The image showing the shape of the product for sale is obtained by photographing the product displayed in the store by comparing the shape indicated by each image with the shape of the article shown in the photo received by the photo reception unit 104. It is a reference image (hereinafter referred to as “reference image”) for determining whether or not there is.

  The reference image is, for example, an image of a product that has been previously displayed in a store by the photographic device 2 and is imaged by image processing such as deep learning, edge detection processing, or other image analysis technology from the captured image. Obtained by extracting the contour (shape).

  The purchase history storage unit 103 can store one or more purchase histories of customers in the store. The purchase history is information related to products or services purchased in the past by the customer at the store. The store may be a real store or a virtual store for online sales via the Internet.

  The purchase history includes, for example, customer identification information for identifying a customer, a purchase identification number (a number corresponding to a slip number), purchase date and time, information for identifying a purchase place, the name of a product or service, the number of the product or service, the product or service It includes information such as service attributes (for example, attributes such as type, color, size, application, and sex), price, delivery location, recipient, contact information, delivery method, and payment method. The purchase history is associated with terminal identification information for identifying the customer's mobile terminal device for each purchase identification number.

  The customer identification information is usually an identification number (ID number) given to the customer. You may use for a customer identification information the telephone number, e-mail address, etc. of the portable terminal device (for example, apparatuses, such as a mobile phone, a smart phone, a notebook personal computer) which a customer possesses. The delivery location is usually the customer's address, the recipient is usually the customer's name, and the contact is usually the customer's landline or mobile phone number.

  The photo reception unit 104 receives a photo showing an article. Acceptance is usually reception of information using communication means or broadcast reception means, but reading information stored in a recording medium such as an optical disk, a magnetic disk, or a semiconductor memory from the recording medium, a scanner, This is a concept including inputting image information acquired by an input device such as a camera from the input device.

  In the present embodiment, the photo taking device 2 is configured to take an article worn by a customer or a sale product displayed in a store and transmit the photo to the recommendation device 1. 104 receives the photograph normally transmitted from the photography device 2.

  The customer brings a recording medium such as an optical disk, a magnetic disk, or a semiconductor memory on which data of a photograph of the article is recorded to the store, and the photograph receiving unit 104 receives the data of the photograph read from the recording medium. May be.

  The owner information acquisition unit 105 recognizes the image received by the photo reception unit 104 and acquires owner information of an article shown in the photo. The owner information acquisition unit 105 acquires an image (hereinafter referred to as “article shape image”) indicating the shape of the article in the photograph by, for example, deep learning, edge detection processing, or other image analysis techniques. In addition, the owner information acquisition unit 105 calculates the similarity by comparing the article shape image and the reference image for all the reference images stored in the product information storage unit 101. Then, when the owner information acquisition unit 105 obtains a similarity degree equal to or greater than the threshold, the article in the photograph is a photograph of a sales product that is paired with a reference image having the maximum similarity degree. For example, if the similarity equal to or higher than the threshold value is not obtained, it is determined that the article shown in the photograph is not a photograph of the product sold at the store. In addition, the owner information acquisition unit 105 determines whether or not the article in the photograph is a photograph of the product sold in the store by the photographing mode of the camera that photographed the photograph, not the image recognition of the photograph. You may make it do.

  When the owner information acquisition unit 105 determines that the article shown in the photograph is not a photograph of the store sales product, the first owner indicating that the owner of the article shown in the photograph is a customer. If the information is acquired and it is determined that the item in the photo is a photograph of a product sold in the store, second owner information indicating that the owner of the item in the photo is a store is acquired. To do. The format of the first owner information and the second owner information is not limited. For example, it is preferable to express the first owner information and the second owner information by binary information of “0” and “1”.

  In addition, the owner information acquisition unit 105 performs image processing on the photograph and acquires the type of the article shown in the photograph. When the owner information acquisition unit 105 acquires the second owner information as the item owner information, for example, the owner information acquisition unit 105 obtains the product type information paired with the reference image having the maximum similarity from the position product information storage unit 102. get.

  For example, when the type information “blouse” is associated with the reference image having the maximum similarity, the owner information acquisition unit 105 acquires that the type of the article in the photograph is “blouse”. To do.

  Further, when the owner information acquisition unit 105 acquires the first owner information as the owner information of the article, the owner information acquisition unit 105 determines whether or not the upper body part of the person is reflected in the photograph by the face recognition process. When the owner information acquisition unit 105 detects a human face by face recognition processing, the owner information acquisition unit 105 determines that the upper body part of the person is shown in the photograph.

  When the owner information acquisition unit 105 determines that the upper body part of the person is reflected in the photograph, the image that shows the shape of the upper body part of the person is compared with the article shape image, for example, in the photograph. The positional relationship between the upper body part of the person and the article is calculated. Then, the owner information acquisition unit 105 acquires the type of the article using the calculation result and the type determination based on a predetermined positional relationship.

  For example, the owner information acquisition unit 105 determines that the article is “tops” when the position of the article is on the trunk below the human face. Moreover, the owner information acquisition part 105 acquires the type (types, such as a blouse and a jacket) in tops, for example from the shape of articles | goods.

  Further, when the owner information acquisition unit 105 determines that the upper body part of the person is not shown in the photograph, the owner information acquisition unit 105 determines whether there is an image corresponding to the lower body part of the person from the image indicating the shape of the object. . For example, the owner information acquisition unit 105 determines whether there is an image corresponding to the human foot in the image indicating the shape of the object, and if there is an image corresponding to the human foot, Judge that the lower part of the body is visible. When the owner information acquisition unit 105 determines that the lower body part of the person is included in the photograph, the owner information acquisition unit 105 determines the type of the article from the positional relationship between the image of the part and the article image.

  For example, if the position of the article is at the position of the lower body of the person, the owner information acquisition unit 105 determines that the article is “bottoms”, and further determines the type of the bottoms (skirt, pants, etc.) from the shape of the article. Type). For example, if the shape of the article almost overlaps the shape of the person's leg, the owner information acquisition unit 105 acquires pants as the type of article of the photograph, and the shape of the article covers the person's leg. If so, the owner information acquisition unit 105 acquires a skirt as the type of article of the photograph.

  Note that the owner information acquisition unit 105 does not compare with the reference image, but, for example, a human upper body part or a lower body part in the center of the photograph by image analysis processing such as face recognition processing, deep learning, and edge detection processing. It is also possible to determine whether or not a person is captured, or determine whether or not the whole body of a person is reflected in the center of the photograph, and use these determination results to acquire the owner information.

  For example, if the owner information acquisition unit 105 determines that the upper or lower body part of the person or the whole body of the person is captured in the center of the photograph, the article in the photograph is taken by the customer. The first owner information is acquired. Further, for example, when the owner information acquisition unit 105 determines that none of the upper body, lower body, and whole body of a person is captured in the center of the photograph, the article in the photograph is taken of the customer The second owner information is acquired.

  However, for example, the owner information acquisition unit 105 does not show the upper body, the lower body, or the whole body of the person in the center of the photograph, but for example, an object that is not in the background of the store (for example, a television or a chest) If it is determined that the item is captured, it is determined that the article in the photograph is a photograph of the customer, and the first owner information is acquired. In addition, the owner information acquisition unit 105 can acquire position information by the camera, and if the position information of the shooting location recorded in the photograph is more than a threshold with respect to the position information of the store, the owner information acquisition unit 105 is reflected in the photograph. It may be determined that the article being taken is not a photograph of the product sold at the store, that is, it is determined that the article is a photograph of the customer, and the first owner information may be acquired.

  In the case of this method, when the owner information acquisition unit 105 acquires the first owner information, the owner information acquisition unit 105 acquires the type of the article by comparing the article shape image described above with the image of the person's shape shown in the photograph. do it. Moreover, the owner information acquisition part 105 should just acquire the kind of article | item by the method of comparing the article shape image mentioned above and a reference image, when acquiring 2nd owner information.

  The recommendation unit 106 acquires product information of a type of product corresponding to the owner information acquired by the owner information acquisition unit 105 from the product information storage unit 101 and outputs the product information. The type of product corresponding to the owner information acquired by the owner information acquisition unit 105 is associated with, for example, an article shown in the photograph in the recommended product correspondence table stored in the product information storage unit 101. It is a kind of product. Note that “acquiring and outputting the product information of the type of product corresponding to the owner information from the product information storage unit 101” means that the product information may or may not be output depending on the owner information. Including that there is. For example, when the owner information acquired by the owner information acquisition unit 105 is the first owner information, the recommendation unit 106 acquires the product information of the type of product corresponding to the owner information from the product information storage unit 101. However, when the owner information acquired by the owner information acquisition unit 105 is the second owner information, the recommendation unit 106 may not output the product information.

  The recommendation unit 106 may acquire and output the product information of the type of product corresponding to the owner information acquired by the owner information acquisition unit 105, and the algorithm for acquiring the product information does not matter. After the type of product to be acquired by the recommendation unit 106 is determined, product information can be acquired by a known recommendation algorithm such as collaborative filtering.

  As described above, the merchandise information storage unit 101 recommends an article and an article in the photograph when the owner information of the article is the first owner information and when the owner information of the article is the second owner information. A recommended product correspondence table showing a correspondence relationship with products is stored.

  The recommendation unit 106 uses the article type and owner information in the photograph acquired by the owner information acquisition unit 105 and the recommended product correspondence table stored in the product information storage unit 101 to provide the article of the photograph. The recommended product of the type corresponding to the owner information is determined, and the product information of the recommended product is acquired from the product information storage unit 101.

  For example, it is assumed that the color and type of the article in the photograph acquired by the owner information acquisition unit 105 and the owner information are “red”, “blouse”, and “first owner information”, respectively. Further, as recommended products for “blouse” when the owner information of the item in the recommended product correspondence table of the product information storage unit 101 is the first owner information, for example, the type of “pants” and “skirt”, and the item Assume that “pink” is associated as a matching color.

  The recommendation unit 106 determines “peach-colored pants” and “peach-colored skirt” as recommended products corresponding to “red blouse” from the recommended product correspondence table of the product information storage unit 101, and the “peach-colored pants” and “ Product information of “pink skirt” is acquired from the product information storage unit 101.

  Further, the color and type of the article of the photograph acquired by the owner information acquisition unit 105 and the owner information are “red”, “blouse”, and “second owner information”, respectively. In addition, as the recommended product for “blouse” when the owner information of the article in the recommended product correspondence table of the product information storage unit 101 is the second owner information, for example, the type of “blouse” and the color suitable for the article are “ Assume that “pink” is associated.

  The recommendation unit 106 determines “pink blouse” as a recommended product corresponding to “red blouse” from the recommended product correspondence table of the product information storage unit 101, and stores the product information of the “pink blouse” on the product information storage unit 101.

  Product information output includes display on a display, projection using a projector, printing on a printer, sound output, transmission to an external device, storage on a recording medium, processing to another processing device or other program, etc. It is a concept that includes delivery of results. In addition, for transmission, accumulation, and delivery of processing results, the output target is finally presented to the user.

  In the present embodiment, since the product information acquired by the recommendation device 1 is presented to the customer by the photography device 2, the product information output by the recommendation unit 106 is normally transmitted to the photography device 2. is there.

  The recommendation unit 106 may determine the product information acquired from the product information storage unit 101 using the product related information of the sales product acquired by the product related information acquisition unit 108. A method for acquiring product related information by the product related information acquiring unit 108 will be described later.

  The product related information acquired by the product related information acquisition unit 108 is information related to the sales product displayed at the location where the customer of the store is located. Normally, customers go around the store for window shopping or look for products that they are interested in, so the products sold at the customer's location in the store do window shopping or color. It may be said that it is information related to the sales products.

  The recommendation unit 106 acquires the first acquisition condition from the acquisition condition table stored in the product information storage unit 101, determines a product that also satisfies the first acquisition condition as a recommended product, and sets the product information of the recommended product. It may be acquired from the product information storage unit 101 and output.

  For example, the color and type of the article in the photograph acquired by the owner information acquisition unit 105 and the owner information are “red”, “blouse”, and “first owner information”, respectively, and the product related information acquisition unit 108 The product-related information acquired by is the type of product for sale called “pants”. Further, in the case where the owner information of the article in the recommended product correspondence table of the product information storage unit 101 is the first owner information, for example, the types of “pants” and “skirt”, “pink” and “red” It is assumed that a color matching the “purple” article is associated.

  The recommendation unit 106 satisfies the first acquisition condition of the same type as the sales product among “pants” and “skirt” in the recommended product correspondence table of the product information storage unit 101, and has “pink” or “red purple” “Pink-colored pants” and “red-purple pants” will be recommended products. Then, the recommendation unit 106 acquires product information of “pink pants” and “red purple pants” from the product information stored in the product information storage unit 101 and outputs the product information.

  Further, for example, the color and type of the article of the photograph acquired by the owner information acquisition unit 105 and the owner information are “red”, “blouse”, and “second owner information”, respectively, and the product related information acquisition unit 108 The acquired product-related information is the type of sales product “sweater”. Further, in the case where the owner information of the article in the recommended product correspondence table of the product information storage unit 101 is the second owner information, for example, the type of “sweater”, “pink”, and “purple” Assume that colors that match are associated with each other.

  The recommendation unit 106 determines the “sweater” in the recommended product correspondence table of the product information storage unit 101 as a recommended product, and selects “pink sweater” or “red purple” from the product information stored in the product information storage unit 101. 'S sweater' and output.

  Moreover, the recommendation part 106 may determine the merchandise information acquired from the merchandise information storage part 101 also using the customer's purchase history which the recommendation part 106 acquires. A method for acquiring the purchase history of the customer in the recommendation unit 106 will be described later.

  The recommendation unit 106 acquires a second acquisition condition from the acquisition condition table of the product information storage unit 101, determines a product that also satisfies the second acquisition condition as a recommended product, and stores the product information of the recommended product in the product information storage unit Obtained from 101 and output.

  For example, the color and type of the article in the photograph acquired by the owner information acquisition unit 105 and the owner information are “red”, “blouse”, and “first owner information”, respectively, and the recommendation unit 106 is the customer's The types of products that satisfy the second acquisition condition acquired from the purchase history are “sweaters” and “skirts”. Further, in the case where the owner information of the article in the recommended product correspondence table of the product information storage unit 101 is the first owner information, for example, the types of “pants” and “skirt”, “pink” and “red” It is assumed that a color matching the “purple” article is associated.

  The recommendation unit 106 recommends the “pink skirt” and the “purple skirt” that satisfy the second acquisition condition among the “peach pants” and “peach skirt” in the recommended product correspondence table of the product information storage unit 101. The product information is determined, and the product information of the “pink skirt” and “red purple skirt” is acquired from the product information storage unit 101 and output.

  Further, for example, the color and type of the article of the photograph acquired by the owner information acquisition unit 105 and the owner information are “red”, “blouse”, and “second owner information”, respectively, and the recommendation unit 106 purchases the customer. The types of products satisfying the second acquisition condition acquired from the number of histories are “blouse” and “skirt”. Further, in the case where the owner information of the article in the recommended product correspondence table of the product information storage unit 101 is the second owner information, for example, the type of “blouse” and the article of “pink” and “red purple” Assume that colors that match are associated with each other.

  The recommendation unit 106 determines “peach blouse” and “red purple blouse” in the recommended product correspondence table of the product information storage unit 101 as recommended products, and selects from the product information stored in the product information storage unit 101. Acquires and outputs product information for “Pink Blouse” and “Magenta Blouse”.

  The recommendation unit 106 may also determine the product information acquired from the product information storage unit 101 using the customer attribute information acquired by the attribute information acquisition unit 110. A method of acquiring customer attribute information by the attribute information acquisition unit 110 will be described later.

  The recommendation unit 106 acquires a third acquisition condition from the acquisition condition table of the product information storage unit 101, determines a product that also satisfies the third acquisition condition as a recommended product, and stores the product information of the recommended product in the product information storage unit Obtained from 101 and output.

  For example, the color and type of the article of the photograph acquired by the owner information acquisition unit 105 and the owner information are “red”, “blouse”, and “first owner information”, respectively, and the attribute value acquisition unit 110 acquires Assume that the customer's attribute values that satisfy the three acquisition conditions are “female” and “elderly”. Further, in the case where the owner information of the article in the recommended product correspondence table of the product information storage unit 101 is the first owner information, for example, the types of “pants” and “skirt”, “pink” and “red” It is assumed that a color matching the “purple” article is associated. Further, it is assumed that a deep color is set as a color satisfying the third acquisition condition (a color suitable for the elderly group).

  The recommendation unit 106 is, for example, “a deep pink or deep red-purple skirt” with respect to “peach or red-purple pants” and “pink or red-purple skirt” in the recommended product correspondence table of the product information storage unit 101. “Pants for women with deep pink or deep red purple” are determined as recommended products, and “Deep pink or deep red purple skirt” and “Deep pink or deep skirt” are selected from the product information stored in the product information storage unit 101. Acquire and output the product information of "deep red purple pants".

  The color and type of the article of the photograph acquired by the owner information acquisition unit 105 and the owner information are “red”, “blouse”, and “second owner information”, respectively, and the attribute value acquisition unit 110 acquires Assume that the customer's attribute values satisfying the three conditions are “female” and “elderly”. Further, in the case where the owner information of the article in the recommended product correspondence table of the product information storage unit 101 is the second owner information, for example, the type of “blouse” and the article of “pink” and “red purple” Assume that colors that match are associated with each other.

  The recommendation unit 106 determines “deep pink or deep magenta blouse” as the recommended product for “peach blouse” and “red purple blouse” in the recommended product correspondence table of the product information storage unit 101, The product information of “deep pink or deep red-purple blouse” is acquired from the product information stored in the information storage unit 101 and output.

  The position information receiving unit 107 receives information related to the customer's position. Information on the customer's position is, for example, information on the current position of the photographing apparatus 2 detected by the photographing apparatus 2 using a position detection function such as GPS (hereinafter referred to as “position information”) or a mobile terminal device of the customer. This is position information of the mobile terminal device detected by a position detection function such as GPS.

  Since the customer's portable terminal device is usually possessed by the customer, the position information detected by the portable terminal device can be the customer's position information. In addition, since the store clerk normally holds the photo shoot device 2 when the store clerk recommends recommended products to the customer, the position information of the photo shoot device 2 detected by the photo shoot device 2 is: It can be handled as customer location information.

  The location information receiving unit 107 receives, for example, the location information of the photography device 2 detected by the photography device 2 from the photography device 2 as customer location information.

  The position information is detected by the mobile terminal device possessed by the customer, and wireless communication is performed between the mobile terminal device and the photography device 2 to transfer the location information of the mobile terminal device to the photography device 2. Further, the position information may be transmitted from the photography apparatus 2 to the recommendation apparatus 1. In this case, the position information receiving unit 107 receives the position information (customer position information) of the customer's mobile terminal device from the photography apparatus 2.

  The product related information acquisition unit 108 uses the customer location information received by the location information receiving unit 107 to store product related information on sales products displayed at the location where the customer is located in the store from the location product information storage unit 102 ( For example, sales product identification information or sales product type information) is acquired.

  The product related information acquisition unit 108 includes the customer location information acquired by the location information receiving unit 107, the display space arrangement area information associated with the product related information stored in the location product information storage unit 102, and Are searched for an arrangement area of the display space including the position where the customer is present.

  The arrangement area of the display space including the position of the customer can be considered as the arrangement area of the display space where the customer is closest. Therefore, the sales merchandise displayed in the display space can be estimated, for example, as merchandise for which the customer is doing window shopping.

  When the product related information acquisition unit 108 acquires the arrangement area of the display space including the position of the customer, the product related information acquisition unit 108 acquires the product related information associated with the arrangement area of the display space from the position product information storage unit 102.

  For example, when the type of the product displayed in the display space with the identification number n is “pants” and the identification number corresponding to the arrangement area of the display space including the customer location information is “n”, the product related information acquisition unit 108 obtains the identification number n of the display space including the customer's position information, and stores the product related information associated with the display space of the identification number n from the position product information storage unit 102 ("pants" indicating the type of product for sale). Information).

  The terminal identification information receiving unit 109 receives customer terminal identification information. The terminal identification information is identification information for identifying a mobile terminal device possessed by a customer. The terminal identification information is information such as an ID number and a MAC address stored in the mobile terminal device, for example. The terminal identification information is not limited to an ID number or a MAC address as long as the mobile terminal device can be identified. The terminal identification information may be, for example, a telephone number or a mail address of the mobile terminal device.

  The terminal identification information may be transmitted directly from the customer's mobile terminal device to the recommendation device 1, or once transmitted from the customer's mobile terminal device to the photography device 2 and transferred from the photography device 2 to the recommendation device 1. Also good. Therefore, for example, the terminal identification information receiving unit 109 receives the terminal identification information directly from the mobile terminal device of the customer or indirectly receives the terminal identification information from the photography device 2.

  The attribute value acquisition unit 110 acquires one or more attribute values of a customer when the customer is shown in the photograph. The customer attribute value is an attribute value used for the third acquisition condition, for example, information such as age, sex, body shape, race, and the like. For example, the attribute value acquisition unit 110 performs face detection processing on the photo received by the photo reception unit 104 and determines whether a human (customer) face is included in the photo.

  The attribute value acquisition unit 110 does not perform attribute value acquisition processing when it is determined that a person is not shown in the photo, but if it is determined that a person is shown in the photo, the attribute value used for the third acquisition condition Perform the acquisition process. For example, when the attribute values used for the third acquisition condition are age and gender, the attribute value acquisition unit 110 acquires a face image indicating a face shown in the photograph, performs person attribute processing on the face image, and creates the photograph Get the age and gender attribute values of the person (customer) in the picture. The person attribute process is, for example, a process of extracting facial parts (feature points) such as eyes, nose and mouth from a face image and estimating the age and sex of the face based on the face image and the extracted feature points.

  If the attribute value used for the third acquisition condition includes a body shape, the attribute value acquisition unit 110 may, for example, use the deep learning, edge detection process, or other image analysis technique to show the external shape of the person in the photograph. It is preferable to extract the person shape image indicating the body shape, perform body shape measurement processing using the person shape image, and acquire the attribute value of the customer body shape.

  In addition, an attribute value such as a customer's age, sex, and body shape is input to the photography apparatus 2, the attribute value is transmitted from the photography apparatus 2 to the recommendation apparatus 1, and the attribute value acquisition unit 110 receives the attribute value. You may do it.

  Next, the block configuration of the photography apparatus 2 will be described.

  The photography apparatus 2 includes a motion detection unit 201, a camera unit 202, a photo transmission unit 203, a product information reception unit 204, and an output unit 205.

  The motion detector 201 detects the motion of the photographic device 2. The movement of the photographic device 2 includes the movement of the photographic device 2 when the photographic device 2 changes its posture from a stationary state and the movement itself. The movement of the photographic device 2 is usually a movement when a user (for example, a store clerk) picks up the photographic device 2 from a mounting place such as a charger or a table. The motion detection unit 201 outputs a motion detection signal to the camera unit 202 when detecting the motion of the photography apparatus 2.

  The camera unit 202 takes a photograph of an article possessed by a customer who has visited the store, an article worn by the customer, a sale product displayed at a position where the customer is in the store, and the like. When a motion detection signal for detecting the motion of the photography apparatus 2 is input from the motion detection unit 201, the camera unit 202 transitions to a photography mode in which photography can be performed according to the input of the motion detection signal. When the motion detection signal is input from the motion detection unit 201, the camera unit 202 is activated from, for example, a sleep state and is in a state in which shooting is possible.

  Note that when the motion detection unit 201 inputs a motion detection signal to the camera unit 202, the power of the camera unit 202 may be changed from an off state to an on state so that the camera can be photographed. When the motion detection unit 201 detects a change in the orientation of the photography apparatus 2 (for example, from landscape orientation to portrait orientation, or from portrait orientation to landscape orientation), the camera unit 202 captures a photograph. May be switched to a shooting mode capable of The transition to a photographing mode in which a photograph can be taken is usually activation of a camera application.

  When the store clerk picks up the photographic device 2 from the place of placement to serve the customer, the motion detection unit 201 detects the motion of the photographic device 2, and the camera unit 202 is activated in conjunction with the motion detection. Is done.

  Thereafter, when a store clerk performs a shooting operation (for example, an operation of touching a shutter button displayed on the touch panel of the photography apparatus 2), the camera unit 202 captures a still image. The photo transmission unit 203 transmits the photo taken by the camera unit 202 to the recommendation device 1. Usually, the store clerk adjusts the angle of view so that the article worn by the customer or the sale product displayed at the position where the customer is located is arranged in the center of the shooting screen of the camera unit 202 to take a picture. do. Therefore, the photo transmitted by the photo transmission unit 203 to the recommendation device 1 is a photo in which an article or a product is shown in the center of the screen.

  The merchandise information receiving unit 204 receives merchandise information of recommended merchandise transmitted by the recommendation unit 106 of the recommendation device 1.

  The output unit 205 outputs the product information received by the product information receiving unit 204. The output of the output unit 205 is, for example, that the clerk outputs the recommended product in a manner that allows the recommended product to be recognized visually or audibly, or visually and audibly.

  Further, the output unit 205 converts character information related to the product into audio information for audio output, and outputs the audio information from, for example, a speaker or an earphone output terminal of the photography apparatus 2.

  The product information storage unit 101, the location product information storage unit 102, and the purchase history storage unit 103 are preferably non-volatile recording media, but can also be realized by volatile recording media. Moreover, the process in which predetermined information is memorize | stored in the merchandise information storage part 101 etc. does not ask | require. For example, the predetermined information may be stored in the product information storage unit 101 or the like via a recording medium, a communication line, an input device, or the like.

  The photo accepting unit 104, the position information receiving unit 107, the terminal identification information receiving unit 109, and the product information receiving unit 204 are normally realized by wireless or wired communication means, but may be realized by means for receiving broadcasts. . The photo transmission unit 203 is usually realized by a wireless or wired communication unit, but may be realized by a broadcasting unit.

  The owner information acquisition unit 105, the recommendation unit 106, the product related information acquisition unit 108, and the attribute value acquisition unit 110 can be usually realized by an MPU, a memory, or the like. Also, the processing procedure of the photo accepting unit 104 or the like is usually realized by software, and the software is recorded on a recording medium such as a ROM. The photo accepting unit 104 and the like may be realized by hardware (dedicated circuit).

  The motion detection unit 201 can be realized using, for example, an acceleration sensor. The camera unit 202 can be usually realized by an image sensor (solid-state imaging device) such as a CMOS or CCD, or a camera (digital still camera, digital video camera) using the image sensor.

  Next, the output operation of the product information related to the recommended product of the recommendation device 1 will be described.

  First, the recommendation device 1 determines the recommended product of the type corresponding to the owner information using only the owner information of the article shown in the photograph, and acquires the product information of the recommended product from the product information storage unit 101. The output process will be described with reference to the flowchart of FIG.

  (Step S101) It is determined whether or not the photo accepting unit 102 accepts a photo. Specifically, it is determined whether or not the photograph receiving unit 102 has received a photograph transmitted from the photography apparatus 2. If the photo acceptance unit 102 accepts a photo (S101: Y), the process proceeds to step S102. If the photo acceptance unit 102 does not accept a photo (S101: N), the determination process is continued until a photo is accepted.

  (Step S102) The owner information acquisition unit 105 recognizes an image of the photo received by the photo reception unit 102, and acquires the owner information of the article shown in the photo. For example, the owner information acquisition unit 105 acquires the owner information of the article by determining whether or not the article shown in the photograph is a copy of the product sold in the store.

  The owner information acquisition unit 105 acquires first owner information indicating that the owner of the article is a customer when the article shown in the photograph is not a copy of the product sold in the store. If the article being sold is a copy of the product sold at the store, the owner of the article acquires second owner information indicating that the store is a store.

  (Step S103) The owner information acquisition unit 105 performs image processing on the photograph and acquires the type of the article shown in the photograph. When the owner information acquisition unit 105 acquires the second owner information as the item owner information, for example, the owner information acquisition unit 105 obtains the product type information paired with the reference image having the maximum similarity from the position product information storage unit 102. get.

  In addition, when the owner information acquisition unit 105 acquires the first owner information as the owner information of the article, for example, an image showing the shape of the person's part by deep learning, edge detection processing, or other image analysis technology And an article shape image indicating the shape of the article is extracted. And the owner information acquisition part 105 compares both images, and acquires the kind of article | item from the positional relationship of a person's part and an article | item.

  For example, when the owner information acquisition unit 105 obtains a positional relationship that the position of the article is on the human torso, the owner information acquisition unit 105 acquires, for example, “blouse” as the type of the article from the positional relationship and the shape of the article. Further, when the owner information acquisition unit 105 obtains a positional relationship that the position of the article is on a person's leg, the owner information acquisition unit 105 acquires, for example, “skirt” as the type of the article from the positional relationship and the shape of the article.

  (Step S104) The recommendation unit 106 determines whether the owner information of the article acquired by the owner information acquisition unit 105 is the first owner information. If the owner information acquired by the owner information acquisition unit 105 is the first owner information (S104: Y), the recommendation unit 106 proceeds to step S105, and the owner information acquired by the owner information acquisition unit 105 If it is the second owner information (S104: N), the process proceeds to step S106.

  (Step S <b> 105) The recommendation unit 106 includes the color and type of the article in the photograph acquired by the owner information acquisition unit 105, the first owner information, and the owner information stored in the product information storage unit 101. The recommended product is determined using the recommended product correspondence table in the case of the first owner information.

  The recommendation unit 106 determines a product that is different from the type of the article acquired by the owner information acquisition unit 105 according to the recommended product correspondence table and has a predetermined color that matches the color of the article as the recommended product, and The product information of the recommended product is acquired from the product information storage unit 101.

  (Step S <b> 106) The recommendation unit 106 includes the color and type of the article in the photograph acquired by the owner information acquisition unit 105, the second owner information, and the owner information stored in the product information storage unit 101. The recommended product is determined using the recommended product correspondence table in the case of the second owner information. The recommendation unit 106 determines a product of the same type as that of the article acquired by the owner information acquisition unit 105 based on the recommended product correspondence table and having a predetermined color that matches the color of the article as the recommended product. The product information of the recommended product is acquired from the product information storage unit 101.

  (Step S107) The recommendation part 106 outputs the product information of the acquired recommended product. Specifically, the recommendation unit 106 transmits the acquired product information of the recommended product to the photography apparatus 2.

  In the flowchart of FIG. 3, the process ends when the power is turned off or the process is terminated.

  Next, in addition to the owner information of the article shown in the photograph, the recommendation device 1 also uses information on sales products, customer purchase history, and customer attribute values displayed at the customer location in the store. A process of determining a recommended product, obtaining product information of the recommended product from the product information storage unit 101, and outputting the product information will be described with reference to the flowchart of FIG.

  (Step S201) It is determined whether or not the photograph receiving unit 102 has received a photograph. Specifically, it is determined whether or not the photograph receiving unit 102 has received a photograph transmitted from the photography apparatus 2. If the photo acceptance unit 102 accepts a photo (S201: Y), the process proceeds to step S202. If the photo acceptance unit 102 does not accept a photo (S201: N), the determination process is continued until a photo is accepted.

  (Step S202) The owner information acquisition unit 105 recognizes an image of the photo received by the photo reception unit 102, and acquires owner information of an article shown in the photo. For example, the owner information acquisition unit 105 acquires the owner information of the article by determining whether or not the article shown in the photograph is a copy of the product sold in the store.

  The owner information acquisition unit 105 acquires first owner information indicating that the owner of the article is a customer when the article shown in the photograph is not a copy of the product sold in the store. If the article being sold is a copy of the merchandise sold at the store, second owner information indicating that the owner of the article is a store customer is acquired.

  (Step S <b> 203) The owner information acquisition unit 105 performs image processing on the photograph and acquires the type of the article shown in the photograph. When the owner information acquisition unit 105 acquires the second owner information as the item owner information, for example, the owner information acquisition unit 105 obtains the product type information paired with the reference image having the maximum similarity from the position product information storage unit 102. get.

  Further, when the owner information acquisition unit 105 acquires the first owner information as the owner information of the article, for example, an image showing the shape of the human part by deep learning, edge detection processing, or other image analysis technology And an article shape image indicating the shape of the article is extracted. And the owner information acquisition part 105 compares both images, and acquires the kind of article | item from the positional relationship of a person's part and an article | item.

  For example, when the owner information acquisition unit 105 obtains a positional relationship that the position of the article is on the human torso, the owner information acquisition unit 105 acquires, for example, “blouse” as the type of the article from the positional relationship and the shape of the article. Further, when the owner information acquisition unit 105 obtains a positional relationship that the position of the article is on a person's leg, the owner information acquisition unit 105 acquires, for example, “skirt” as the type of the article from the positional relationship and the shape of the article.

  (Step S204) The position information receiving unit 107 determines, for example, whether or not the photography device 2 has received the location information of the photography device 2 from the photography device 2 as the customer location information. If the location information receiving unit 107 receives the customer location information (S204: Y), the process proceeds to step S205, and if the location information receiving unit 107 does not receive the customer location information (S204: N), the location information receiving unit 107. The determination process is continued until the location information of the customer is received.

  (Step S205) The product related information acquisition unit 108 arranges the display space associated with the customer location information acquired by the location information receiving unit 107 and the product related information stored in the location product information storage unit 102. Using the information of the area, the product related information of the sales product displayed at the position where the customer is present is acquired.

  (Step S206) It is determined whether the terminal identification information receiving unit 109 has received the terminal identification information of the customer's mobile terminal device. For example, when the terminal identification information of the portable terminal device is transmitted from the customer's portable terminal device to the photography device 2, and the photography device 2 transfers the terminal identification information to the recommendation device 1, the terminal identification information receiving unit 109 The terminal identification information is received from the photography apparatus 2.

  Therefore, when the terminal identification information receiving unit 109 receives the terminal identification information from the photography apparatus 2 (S206: Y), the process proceeds to step S207, and if the terminal identification information receiving unit 109 does not receive the terminal identification information (S206: N). ), The determination process is continued until the terminal identification information receiving unit 109 receives the terminal identification information.

  (Step S207) The recommendation unit 106 acquires one or more purchase histories associated with the terminal identification information from the purchase history storage unit 103, using the terminal identification information received by the terminal identification information reception unit 109.

  (Step S208) The attribute value acquisition unit 110 acquires one or more attribute values of the customer shown in the photo using the photo received by the photo reception unit 102. For example, the attribute value acquisition unit 110 performs face detection processing on a photograph to acquire a face image of a person (customer) shown in the photograph. Furthermore, the attribute value acquisition unit 110 performs person attribute processing on the acquired face image, and determines whether or not a person (customer) face is shown in the photograph.

  The attribute value acquisition unit 110 does not perform attribute value acquisition processing when it is determined that a person is not shown in the photo, but if it is determined that a person is shown in the photo, the attribute value used for the third acquisition condition (For example, age group and gender) is acquired.

  (Step S209) The recommendation unit 106 includes the product related information of the sales product acquired by the product related information acquisition unit 108, the customer purchase history acquired by the recommendation unit 106, and the customer attribute value acquired by the attribute value acquisition unit 110, Using the acquisition condition table stored in the product information storage unit 101, the first acquisition condition, the second acquisition condition, and the third acquisition condition for acquiring the product information of the recommended product are acquired.

  In the acquisition condition table, for example, when the condition “product of the same type as the sales product” is described with respect to the type of the sales product displayed at the position where the customer is, the recommendation unit 106 displays the same type as the sales product. A first acquisition condition of determining a product as a recommended product is acquired. For example, when the acquisition condition table describes an acquisition condition of “a product having an attribute value of a product having a large purchase history” with respect to the purchase history of the customer, for example, the recommendation unit 106 indicates that the purchase history number is A second acquisition condition is acquired in which a product having an attribute value (for example, an attribute value such as a color or a shape) of a product equal to or greater than a threshold is determined as a recommended product.

  In addition, when the acquisition condition table describes, for example, the condition of “a product that matches the sex and age group of the customer” for the customer attribute, the recommendation unit 106 matches the sex and age group of the customer, for example. A third acquisition condition for determining a product having an attribute value as a recommended product is acquired.

  (Step S210) The recommendation unit 106 determines whether the owner information of the article acquired by the owner information acquisition unit 105 is the first owner information. If the owner information acquired by the owner information acquisition unit 105 is the first owner information (S210: Y), the recommendation unit 106 proceeds to step S211 and the owner information acquired by the owner information acquisition unit 105 is If it is the second owner information (S210: N), the process proceeds to step S212.

  (Step S211) The recommendation unit 106 includes the color and type of the article in the photograph acquired by the owner information acquisition unit 105, the first owner information, and the first acquisition condition through the first acquisition condition acquired by the recommendation unit 106. The recommended product is determined using the three acquisition conditions and the recommended product correspondence table when the owner information stored in the product information storage unit 101 is the first owner information.

  The recommendation unit 106 determines a product that is different from the type of the article acquired by the owner information acquisition unit 105 according to the product correspondence table and satisfies the first acquisition condition to the third acquisition condition as a recommended product, The product information of the recommended product is acquired from the product information storage unit 101.

  (Step S212) The recommendation unit 106 includes the color and type of the article in the photograph acquired by the owner information acquisition unit 105, the second owner information, and the owner information stored in the product information storage unit 101. A recommended product is determined using the recommended product correspondence table in the case of the second owner information. The recommendation unit 106 determines a recommended product that is a product of the same type as the type of the product acquired by the owner information acquisition unit 105 according to the product correspondence table and satisfies the first acquisition condition to the third acquisition condition. Product information related to the recommended product is acquired from the product information storage unit 101. In addition, when the information acquired by the owner information acquisition unit 105 is the second owner information, the recommendation unit 106 does not have to acquire product information for recommendation. That is, only when the information acquired by the owner information acquisition unit 105 is the first owner information, the recommendation unit 106 may acquire product information for recommendation.

  (Step S213) The recommendation part 106 outputs the product information regarding the acquired recommended product. Specifically, the recommendation unit 106 transmits product information related to the acquired recommended product to the photography apparatus 2.

  In the flowchart of FIG. 4, the process ends when the power is turned off or the process ends.

  Further, in the flowchart of FIG. 4, the first through the second items for obtaining the product information of the recommended product for each information of the sales product displayed at the customer location in the store, the purchase history of the customer, and the attribute value of the customer. Although the three acquisition conditions are acquired, any one or more of the first to third acquisition conditions may be acquired.

  That is, in the flowchart of FIG. 4, any one or two of the processes in steps S204 and S205, the processes in steps S206 and S207, and the process in step S208 may be omitted.

  Next, the process of the photography apparatus 2 will be briefly described.

  For example, a store clerk takes an action such as picking up the photographic device 2 from the placement place in order to propose a recommended product to a customer who has visited the store using the photographic device 2 of the recommendation system. To do. When the posture of the photographic device 2 is changed by this action, the motion detecting unit 201 constituting the photographic device 2 detects the motion of the photographic device 2, and the camera unit 202 is activated in conjunction with the movement detection.

  In order for the clerk to propose a recommended product to the customer, for example, when the product photographed by the photographer 2 is photographed by an article worn by the customer or a product sold at a position where the customer is located, the photograph transmission unit 203 takes a photograph. The photograph taken is transmitted to the recommendation device 1.

  Further, when the store clerk communicates between the photography device 2 and the mobile terminal device of the customer and receives the terminal identification information of the mobile terminal device by the photography device 2, the photography device 2 displays the terminal identification information. It transmits to the recommendation device 1.

  After that, when the recommendation device 1 performs the above-described acquisition processing of the product information of the recommended product, when the product information is transmitted to the photography device 2, the product information receiving unit 204 receives the product information, and the output unit 205 The product information is output.

  The output unit 205 converts, for example, character information related to the product included in the product information and a photograph of the product sample into image information for display, and outputs (displays), for example, on the liquid crystal display of the touch panel of the photography apparatus 2. ). Further, the output unit 205 converts character information related to the product included in the product information into audio information for audio output, and outputs the audio information from, for example, a speaker or an earphone output terminal of the photography apparatus 2.

(Specific example 1)
Next, a specific operation of the recommendation system in the present embodiment will be described.

  In the specific example 1, the recommendation system has the block configuration shown in FIG. 2, but the recommendation apparatus 1 uses only the owner information of the article in the photograph according to the processing procedure shown in FIG. A case where product information relating to a corresponding type of product is output will be described. In the following description, a recommendation operation of the recommendation system in a store that sells women's clothing and clothing will be described.

  The merchandise information storage unit 101 stores a merchandise information management table in which merchandise information regarding a number of recommended merchandise is recorded in a data table format. The product information of each product includes a product identifier that identifies the product, a product name, a price (unit price), product attributes (e.g., sex targeted for the product, target age of the product, usage, type, color, size, shape, Production center, raw materials, etc.), and information such as photographs or images of product samples.

  FIG. 5 is a diagram illustrating an example of a product information management table stored in the product information storage unit 101 of the recommendation device 1.

  FIG. 5 shows information such as “ID”, “product identifier”, “product name”, “price”, “type”, etc., which is a part of product information, and “photograph” which is a product sample.

  One piece of product information is recorded in a record (data for one row) in the product information management table, and a plurality of pieces of product information such as “ID” and “photographs” are recorded side by side in the record. “ID” is identification information for identifying a record in the product information management table. The “product identifier” is a product identifier that identifies a product recorded in the record. In FIG. 5, the product number assigned to the product is described in the “product identifier” column.

  “Product name” is a name for identifying a product. Normally, in the case of merchandise sales, different products with different materials, shapes, colors, etc. are treated as different products even with the same type of products, and different product names using attribute values such as material, shape, etc. are attached. In the “product name” column, for example, a product name distinguished by the attribute of the product is described.

  “Price” is, for example, a retail price of a product, and “Photo” is an image of a photograph of a sample of the product.

  “Type” is, for example, a name given to the smallest category among categories in which products are hierarchically classified in various product classification tables such as “Japanese standard product classification table”. In the product classification table, a large number of products are usually aggregated for each similar product and classified into hierarchical classification items as a product group. The name is used. For example, common names such as “blouse” and “skirt” are used in the upper garment. Therefore, the common name of the product is described in the “type” column.

  FIG. 6 is a diagram illustrating an example of the recommended product correspondence table stored in the product information storage unit 101.

  “ID” is identification information (number) for identifying a record of the recommended product correspondence table, and “type of article” is a type of article shown in the photograph. In the “recommended product” column, attribute values such as recommended product type and color are described.

  “Owner information” is owner information of an article shown in a photograph. Note that “0” in the “owner information” column indicates the first owner information, and “1” indicates the first owner information. “0” may be the second owner information, and “1” may be the first owner information.

  In FIG. 6, for example, ID = *** 11 indicates that the type of the article shown in the photograph is “Blouse” and the owner information of the article is the first owner information. This indicates that “pants” and “skirt” of colors that match the color of “blouse” are selected as recommended products.

  Note that the color that matches the color of the article in the photograph is determined in advance as shown in FIG. 6, for example. In the example of FIG. 7, for example, when the color of the article is “red”, the recommended product color is selected from “white”, “pink”, and “red purple”, and the article color is “blue”. "" Indicates that the recommended product color is selected from "white", "orange", and "dark blue".

  Therefore, if the article in the photo is a “red blouse”, for example, “white pants and skirts”, “peach-colored pants and skirts”, and “red purple pants and skirts” are selected as recommended products. .

  ID = *** 12 is a color of “blouse” in the photograph when the type of the article in the photograph is “blouse” and the owner information of the article is the second owner information. “Blouse” with a color suitable for the product is selected as a recommended product.

  Also in this case, for example, when the article shown in the photograph is a “red blouse”, the color blouse suitable for the article shown in FIG. 6, that is, “white blouse”, “pink blouse”, “red purple” “Blouse” is selected as a recommended product.

  FIG. 8 is a diagram illustrating an example of a list of images (reference images) indicating the shapes of sales products stored in the position product information storage unit 102. Each image in the list is associated with information on the type of product for sale.

  In FIG. 8, “ID” is identification information (number) for identifying a record in the reference image list. Also, in the “Products for sale” column, products previously displayed in the store are photographed with the photography device 2 and the like, and the photographed images are captured by image processing such as deep learning, edge detection, and other image analysis techniques. An image (reference image) obtained by extracting the outline (shape) of the product is posted. “Product type” is the type of product indicated by the shape of the image.

  In FIG. 8, for example, an image with ID = *** 51 indicates that it is an image showing the shape of a “blouse” displayed at the store, and an image with ID = *** 63 is displayed at the store. The image of ID = *** 80 indicates that it is an image indicating the shape of “pants” displayed in the store.

  Next, the operation of the recommendation system will be described.

  When a store clerk approaches a customer who brought a photo taking device 2 and proposes a product such as a bottom that matches the top worn by the customer, the store clerk wears the customer. The upper garment is photographed by the photographic device 2. In addition, when proposing a product related to the sales product displayed at the location where the customer is located in the store (for example, a product such as another product similar to the sales product), the salesclerk will sell the product that the customer is viewing. Is photographed by the photography apparatus 2.

  For example, when a clerk takes a picture of a blouse worn by a customer or a sales product viewed by a customer in the store, the clerk has the entire blouse within a predetermined range in the center of the photographing screen of the photography apparatus 2. Adjust the angle of view so that it enters.

  When the store clerk performs a shooting operation (for example, pressing or touching a shutter button), the camera unit 201 of the photo shooting device 2 takes a blouse photo, and the photo transmission unit 203 transmits the photo to the recommendation device 1.

  In the recommendation device 1, the photo reception unit 102 receives (receives) a photo from the photo shooting device 2. When the photo reception unit 102 receives a photo, the owner information acquisition unit 105 performs image recognition processing on the photo, and acquires owner information regarding the owner of the article in the photo.

  The owner information acquisition unit 105 performs, for example, deep learning, edge detection processing, or other image analysis techniques for extracting the outline of the blouse that is reflected in a predetermined range in the center of the photograph, and extracts the blouse shape image Compared with an image (reference image; see FIG. 8) showing the shape of the sales product stored in the position product information storage unit 102, the similarity is calculated for each reference image.

  And the owner information acquisition part 105 discriminate | determines whether there exists a similarity more than a threshold value, and when there is no similarity more than a threshold value, a photograph is not what image | photographed the sale product of a store (that is, photography And the first owner information indicating that the owner of the article in the photograph is the customer.

  In addition, when there is a similarity equal to or greater than the threshold, the owner information acquisition unit 105 determines that the photograph is a photograph of a store sale product, and the owner of the article in the photograph is the store The second owner information indicating that is acquired.

  Next, the owner information acquisition unit 105 acquires the attribute value of the type and color of the article shown in the photograph. The owner information acquisition unit 105 acquires the color information of the article by deep learning or other image analysis techniques for the part surrounded by the shape of the article in the photograph.

  In addition, when the owner information acquisition unit 105 acquires the second owner information, the photograph shows the type of product associated with the reference image having the maximum similarity from the list of reference images. Acquired as the type of article. For example, in the list shown in FIG. 8, when the reference image with the maximum similarity is an image with ID = *** 51, the owner information acquisition unit 105 is associated with ID = *** 51. The product type “blouse” is acquired as the type of article in the photograph.

  Further, when the owner information acquisition unit 105 acquires the first owner information, the owner information acquisition unit 105 performs face recognition processing on the photo and determines whether or not a human face is reflected in the photo. The owner information acquisition unit 105 extracts a human shape image by performing deep learning, edge detection processing, or other image analysis techniques when a human face is reflected in the photograph, The positional relationship of the article with respect to the person shown in the photograph is calculated by comparing the image shown and the article shape image.

  Then, the owner information acquisition unit 105 estimates the type of the article using the positional relationship. For example, when the article is located in the human torso in the shape of the person, the owner information acquisition unit 105 determines that the article in the photograph is a top (upper), and determines the shape of the article. Estimate the type of tops (type of blouse, jacket, etc.). Further, the owner information acquisition unit 105 determines that the article shown in the photograph is bottoms (underwear) when the article is located on the leg of the person in the shape of the person, and the shape of the article To estimate the type of bottoms (type of pants, skirt, etc.).

  The recommendation unit 106 includes information such as the owner information of the article, the type and color of the article in the photograph acquired by the owner information acquisition unit 105, and the recommended product correspondence table stored in the product information storage unit 101 ( 6), a recommended product of a type corresponding to the owner information of the article of the photograph is determined.

  For example, when the owner information, type, and color of the article in the photograph acquired by the owner information acquisition unit 105 are “first owner information”, “blouse”, and “red”, the recommendation unit 106 The product described in the “recommended product” column of ID = *** 11 is selected as the recommended product. That is, the recommendation unit 106 selects “pants” and “skirts” having white, pink, and magenta colors that are determined as colors suitable for red shown in FIG. 7 as recommended products for the red blouse. .

  If the owner information, type, and color of the article in the photograph acquired by the owner information acquisition unit 105 are “second owner information”, “blouse”, and “red”, respectively, the recommendation unit 106 The product described in the “recommended product” column of ID = *** 12 is selected as the recommended product. That is, the recommendation unit 106 selects, as a recommended product, a “blouse” having white, pink, and magenta colors that are determined as colors suitable for red as shown in FIG. 7 for the red blouse.

  When the recommendation unit 106 determines the recommended product, the recommendation unit 106 acquires the product information of the recommended product from the product information storage unit 101 and transmits the product information to the photography apparatus 2. For example, when the recommended product determined by the recommendation unit 106 corresponds to the skirt of the product information ID = 77 *** 3 shown in FIG. 5, the recommendation unit 106 receives the ID = 77 *** from the product information storage unit 101. 3 product information is read out and transmitted to the photography apparatus 2.

  When the recommendation unit 106 transmits the product information related to the recommended product, the product information receiving unit 204 of the photography apparatus 2 receives the product information of the recommended product, and the output unit 205 outputs the product information. The output unit 205 converts, for example, character information (product name, price, type, color, size, etc.) related to the product included in the product information of the recommended product and a photograph of the product sample into image information for display, For example, the image is output (displayed) on the liquid crystal display of the touch panel of the photography apparatus 2.

  FIG. 9 is a diagram illustrating an example of a display example of recommended products displayed on the display screen 11 of the tablet terminal 1.

  FIG. 9 shows a case where the article shown in the photograph is a blouse worn by a customer (when the owner information of the article is the first owner information), and when the recommended article type is a skirt, An example of output in the photographic device 2 when product information of long skirts and flare skirts that match a predetermined color for the blouse is acquired from the product information storage unit 101 and transmitted to the photographic device 2 Is,

  In FIG. 9, an image G3 in which a blouse image G1 and a long skirt image G2 shown in the photograph are vertically arranged, and an image G5 in which a flare skirt image G4 is vertically arranged in the blouse image G1 are created, The image G3 and the image G5 are displayed side by side on the display screen 11 of the tablet terminal 1. Also, product information such as product names of long skirts and flare skirts are displayed in characters at the bottom of the images G3 and G5.

  Note that the display example of FIG. 9 is an example in which the product information acquired by the recommendation unit 106 of the recommendation device 1 is two pieces. When there are three or more pieces of product information related to recommended products acquired by the recommendation unit 106, for example, by scrolling the display screen 11, the product information about three or more recommended products is displayed in the same display mode as the display mode shown in FIG. 9. It is good to display sequentially.

  FIG. 10 is a diagram illustrating another example of a display example of recommended products displayed on the display screen 11 of the tablet terminal 1.

  FIG. 10 shows a recommendation when the article shown in the photograph is a blouse displayed in a store (when the owner information of the article is the second owner information) and the recommended product type is “blouse”. The unit 106 acquires product information regarding a shirt type blouse (shirt blouse) and a chiffon material blouse (chiffon blouse) having colors that match a predetermined color for the blouse from the product information storage unit 101, It is an output example in the photographic device 2 when transmitted to the photographic device 2.

  In FIG. 10, a shirt blouse image G7 and a chiffon blouse image G8 are displayed on the display screen 11 of the tablet terminal 1 below the blouse image G6 in the photograph. In addition, product information such as product names of shirt blouse and chiffon blouse is displayed in characters below the images G7 and G8, respectively.

  The display example of FIG. 10 is also an example in the case where there are two pieces of product information related to recommended products acquired by the recommendation unit 106 of the recommendation device 1. When there are three or more pieces of product information related to recommended products acquired by the recommendation unit 106, for example, by scrolling the display screen 11, the product information about three or more recommended products is displayed in the same display mode as the display mode shown in FIG. 10. It is good to display with.

(Specific example 2)
In the specific example 2, the recommendation device 1 is shown in the photograph using the sales product displayed at the location where the customer is in the store, the purchase history of the customer, and the attribute value information of the customer according to the processing procedure shown in FIG. The case where the product information regarding the product of the kind according to the owner information of the goods currently being output is demonstrated.

  In the following explanation, the recommendation operation of the recommendation system in a store that sells women's clothing and clothing is also explained.

  The recommendation system in the specific example 2 has the block configuration shown in FIG. 2, and the product information storage unit 101 of the recommendation device 1 has the product information shown in FIG. 5, the recommended product correspondence table shown in FIG. 6, and the article shown in FIG. 7. It is assumed that a correspondence table of colors that match the colors is stored. Further, it is assumed that the position product information storage unit 102 of the recommendation apparatus 1 stores a list of reference images shown in FIG.

  5 to 8 have been described in the specific example 1, the description thereof will be omitted in the specific example 2, and the sales products displayed at the customer's location in the store, the purchase history of the customer, and the attribute values of the customer will be omitted. The configuration for determining the recommended product of the type corresponding to the owner information of the article shown in the photograph using the information and the operation of the recommendation unit 106 will be described.

  FIG. 11A is a diagram illustrating an example of an acquisition condition table stored in the product information storage unit 101. FIG. 5B is a diagram showing an example of a color that is determined in advance as a color that matches the age group of the customer.

The “number” in the acquisition condition table shown in FIG. 11 is a number for identifying a record in the acquisition condition table.
The “item” is an item for which an acquisition condition is set, and the “acquisition condition” is a condition for selecting a recommended product when acquiring product information of the recommended product.

  In the “item” column of the acquisition condition table shown in FIG. 11, sales merchandise displayed at the position where the customer is located in the store is described for number 1, and the purchase history of the customer is stored for number 2. The attribute value of the customer is described for the number 3. The number 1 record in the acquisition condition table corresponds to the first acquisition condition, the number 2 record corresponds to the second acquisition condition, and the number 3 record corresponds to the third acquisition condition.

  The first acquisition condition “product similar to sales product” indicates that a product similar to the sales product is selected as the recommended product. In addition, the second acquisition condition “product having an attribute value of a product with a lot of purchase history” indicates a product having an attribute value whose number of purchases is equal to or greater than a threshold among attribute values such as color and shape of the product purchased by the customer This indicates that the product is selected as a recommended product. The third acquisition condition “product suitable for customer's gender and age group” indicates that a product having an attribute value suitable for the customer's sex and age group is selected as the recommended product.

  In this specific example, since a product having a color that matches the color of the article in the photograph is a recommended product, the attribute values of the second acquisition condition and the third acquisition condition are the color of the product. And about the 3rd acquisition conditions, as shown in FIG.11 (b), the recommended color tone is decided for every sex / age group.

  In FIG. 11 (b), gender / age group fields F1 to F3 and M1 to M3 are layers of generations divided by sex and age group used in marketing. For example, the F1 layer is a woman in the age group of 20 to 34 years old, the F2 layer is a woman in the age group of 35 to 50 years old, and the F3 layer is a woman in the age group of 51 years or older . The M1 layer is a male in the age group of 20 to 34 years old, the M2 layer is a male in the age group of 35 to 50 years old, and the M3 layer is a male in the age group of 51 years or older .

  FIG. 11B shows that when the customer's gender and age group is the F1 layer or the M1 layer, the color of the recommended product determined by the recommended product correspondence table is set to bibit (brilliant color tone). When the age group is F2 layer or M2 layer, it indicates that the color of the recommended product determined by the recommended product correspondence table is light (light color), and the customer's gender and age group is F3 layer or M3 layer , The recommended product color determined by the recommended product correspondence table is set to deep (deep color).

  Therefore, for example, when the recommended product is “red skirt” according to the recommended product correspondence table, if the customer's gender / age group is the F3 layer, a deep color tone will be added as the acquisition condition of the product information. "Deep red skirt" will be selected as a recommended product.

  FIG. 12 is a diagram illustrating an example of product related information stored in the position product information storage unit 102.

  The position product information storage unit 102 stores a position product related correspondence table in which product related information related to sales products in a store is associated with position related information related to the display position of each sales product in the store. ing.

  “ID” in FIG. 12 is identification information for identifying a record in the position product relation table. Further, “product identifier”, “product name”, and “type” are product related information, and “display position SP No.” and “display space area information” are position related information.

  The meanings of the items “ID” to “type” are the same as the meanings of the items having the same name in the product information management table shown in FIG.

  “Display position SPno.” And “region information of display space” are items included in the information related to the display position of the sales product in the store. In the column “Display Position SPno.”, For example, as shown in FIG. 13A, the store S is divided into a plurality of display spaces SP, and each display space SP has an identification number SPNo. (Numbers 000 to 019 in FIG. 13A), the identification number SPNo. Of the display space SP in which the merchandise is displayed. Is described.

  In addition, in the column of “display space area information”, an arrangement area of each display space SP in the store is described. The information indicating the arrangement area in the store of the display space SP is provided with an XY coordinate system orthogonal to the plan view of the store S shown in FIG. 13A, and as shown in FIG. The four corner points a, b, c, b are defined by XY coordinates (Xa, Ya), (Xb, Yb), (Xc, Yc), (Xd, Yd).

  Note that a, b, c, and b are the upper left corner, lower left corner, lower right corner, and upper right corner, respectively. In the example of FIG. 13, the planar shape of the display space SP is rectangular. However, when the planar shape of the display space SP is a polygon, the arrangement area of the display space SP is determined by the XY coordinates of the vertex of the polygon. You can define the information to show.

  FIG. 14 is a diagram illustrating an example of a customer purchase history stored in the purchase history storage unit 103.

  The purchase history storage unit 103 stores a purchase history management table in which product information of products sold by a store to a customer is recorded in a data table format. The purchase history product management table includes products such as “sales date”, “sales location”, “product identifier”, “product name”, “sales price”, “type”, “color”, “size”, “gender”, etc. The products sold are managed by items related to sales and items related to customers such as “customer identification information”.

  In the “sales date and time” column, information on the date and time when the product was sold is described. In the “sales date and time” column, for example, in the case of sales at an actual store, information on the date and time when the store clerk received the price of the product from the customer and completed the predetermined processing related to the sale of the product is described. In addition, when a product is sold at the net shop, information on the date and time when the customer has performed a predetermined checkout procedure on the net shop is described in “sales date and time”.

  In the “sales place” column, information on a sales place such as an actual shop or an online shop where the product is sold is described. Normally, store identification information for identifying the sales location (for example, identification information such as a store number) is attached to the sales location, and the store identification information is described in the “Sales location” column. ing. FIG. 14 illustrates store numbers given to stores.

  In the “product identifier” column, information for identifying the sold product is described. In the “product name” column, the name of the sold product is described. The meanings of “product identifier” and “product name” in FIG. 14 are the same as the meanings of “product identifier” and “product name” in FIG. 5 described above.

  In the “sales price” column, the price at which the product is sold is described. The product is usually sold at a retail price. When the product is sold at a price different from the retail price, the actual sales price is described in the column of “Sale Price”.

  In the “product type” column, the name of the minimum classification category when products are classified hierarchically is described. The type of product is the same as the meaning of “type” in FIG. 5 described above. In the “color” column, the name of the color of the product is described. In addition, the name of the basic color of a hue is used for the classification of the color of goods, for example.

  In the “size” column, information on the size of the product is described. For example, “M” indicates that the product is an M size product in clothing. In the “gender” column, information on the sex of the object of the product is described. For example, “Women” indicates a product for women, and “Men” indicates a product for men. Note that items such as “type”, “color”, “size”, and “gender” are attribute values of the product, and depending on the product, there are items of other attribute values. Is omitted.

  In the “customer identification information” column, information for identifying a customer is described. When the store side gives the customer an identification number, the identification number is used. The example of FIG. 14 is an example in which the telephone number of the customer's mobile terminal device is used as customer identification information.

  When the purchase history management table shown in FIG. 14 is rearranged for each piece of customer identification information, a purchase history table for each customer can be obtained.

  Next, a recommendation operation of the recommendation system in the specific example 2 will be described. From the operation of the store clerk photographing the item worn by the customer or the item sold at the store with the photo taking device 2, the owner information acquisition unit 105 acquires the owner information, type, color, etc. of the item shown in the photo. Since the operation until the information is acquired is the same as that described in the first specific example, the operation after the owner information acquisition unit 105 acquires the owner information of the article will be described below.

  When the owner information acquisition unit 105 acquires the owner information or the like of the article shown in the photograph, the position information receiving unit 107 receives the position information of the photography device 2 from the photography device 2 or the photography device 2 is the customer's information. Position information of the mobile terminal device acquired from the mobile terminal device is received.

  When the position information receiving unit 107 receives the position information, the product related information acquisition unit 108 obtains the position information and the position related information associated with the product related information stored in the position product information storage unit 102. The display area is searched for the display space including the position information.

  For example, when the position information is latitude / longitude information, the merchandise related information acquisition unit 108 converts the latitude / longitude information into coordinates (Xp, Yp) in the XY coordinate system in the layout of the store display space. The product related information acquisition unit 108 includes coordinates (Xp, Yp) indicating the position P of the customer in the store and information indicating the arrangement area of each display space SP in the store (information on the area defined by the XY coordinates at the four corners). And the identification number SPNo. Of the display space SP including the coordinates (Xp, Yp) indicating the position P of the customer. To get.

  For example, as shown in FIG. Information indicating the arrangement area of the display space SP of 003 is a point = (X1, Y1), b point = (X1, Y2), c point = (X2, Y2), d point = (X2, Y1) (X1 < X2, Y2 <Y1), and if X1 <Xp <X2 and Y2 <Yp <Y1, then the customer's position P is the identification number SPNo. Since the product-related information acquisition unit 108 is in the arrangement area of the display space SP of 003, the product related information acquisition unit 108 uses the identification number SPNo. 003 is acquired.

  On the other hand, if X1 <Xp <X2 and Yp <Y2, the position P of the customer is the identification number SPNo. Since it is below the display space SP of 003, the merchandise related information acquisition unit 108 has the identification number SPNo. 004 to SPNo. The same determination is made for the display space SP of 006, and information on the display space SP at the position where the customer in the store S is located (information on identification numbers SPNo. 004 to SPNo. 006) is acquired.

  When the product related information acquisition unit 108 acquires the information of the display space SP at the position where the customer is located, the product related information acquisition unit 108 acquires the product related information of the sales product displayed at the position where the customer is located using the information. The information of the display space SP acquired by the product related information acquisition unit 108 is SPNo. In the case of 003, the product related information acquisition unit 108 stores the sales product in which “SPNo. 003” is written in the “Display position SPNo.” Column from the position product related correspondence table stored in the position product information storage unit 102. Acquire product related information.

  For example, when the position product related correspondence table shown in FIG. 12 is stored in the position product information storage unit 102, the product related information acquisition unit 108 acquires the type of “skirt” associated with “SPNo. 003”. To do. The product related information acquisition unit 108 may acquire “77 *** 3” as the product identifier associated with “SPNo. 003” as the product related information.

  Next, the terminal identification information receiving unit 109 receives, for example, terminal identification information (for example, a telephone number of the portable terminal device) for identifying the customer's portable terminal device from the photography apparatus 2. It is assumed that the store clerk communicates with the customer's mobile terminal device by the photography device 2 and receives the terminal identification information from the mobile terminal device, and recommends the terminal identification information from the photography device 2 to the recommendation. Assume that the data is transmitted to the device 1.

  When the terminal identification information receiving unit 109 receives the telephone number as the terminal identification information of the customer's mobile terminal device, the recommendation unit 106 uses the telephone number to associate the telephone number with the telephone number from the purchase history storage unit 103. One or more purchase histories having a product identification number are acquired.

  For example, when the purchase history management table shown in FIG. 14 is stored in the purchase history storage unit 103 and the telephone number received by the terminal identification information reception unit 109 is “090 ****** 75”, the recommendation unit 106 Information related to the sales products of the product identifiers “77 *** 3” and “77 *** 5” paired with the identification information of “090 ****** 75” from the purchase history storage unit 103 (FIG. 14). Product information regarding “skirts”) as customer purchase history.

  Next, the attribute value acquisition unit 110 acquires one or more attribute values of the customer shown in the photo using the photo received by the photo reception unit 102. For example, the attribute value acquisition unit 110 performs face detection processing on a photograph to acquire a face image of a person (customer) shown in the photograph. Furthermore, the attribute value acquisition unit 110 performs person attribute processing on the acquired face image and acquires attribute values of the age and sex of the person (customer) shown in the photograph.

  Next, the recommendation unit 106 uses the acquisition condition table stored in the product information storage unit 101, the first acquisition condition when the sales product acquired by the product related information acquisition unit 108 is used, and the recommendation unit 106 The second acquisition condition when using the acquired purchase history of the customer and the third acquisition condition when using the attribute value of the customer acquired by the attribute value acquisition unit 110 are acquired.

  For example, when the product information acquisition unit 108 acquires the type of skirt as the product related information, the recommendation unit 106 corresponds to the product related information acquired by the product related information acquisition unit 108 from the acquisition condition table illustrated in FIG. Acquire the first acquisition condition for a recommended product with the same type of skirt as the product type. Further, for example, when the recommendation unit 106 has acquired “red” as the attribute value of the product that the customer often purchases from the purchase history of the customer, the recommendation unit 106 selects the red color from the acquisition condition table illustrated in FIG. 11. A second acquisition condition for acquiring a product as a recommended product is acquired.

  For example, when the attribute value acquisition unit 110 acquires “female” and “senior layer (F3 layer)” as customer attribute values, the recommendation unit 106 determines that the F3 layer from the acquisition condition table illustrated in FIG. A third acquisition condition for acquiring a product having a “deep color tone” as a recommended product as a recommended product is obtained with respect to the color determined in the recommended product correspondence table.

  Next, the recommendation unit 106 includes the owner information of the article, the type of article, the color, and the like that are included in the photograph acquired by the owner information acquisition unit 105 and the recommended product stored in the product information storage unit 101. Using the correspondence table (see FIG. 6) and the acquired first to third acquisition conditions, a recommended product of a type corresponding to the owner information of the photographic article is determined.

  For example, when the owner information, type, and color of the article in the photograph acquired by the owner information acquisition unit 105 are “first owner information”, “blouse”, and “red”, the recommendation unit 106 The product described in the “recommended product” column of ID = *** 11 in the recommended product correspondence table shown in FIG. 6 is selected as the recommended product. In other words, the recommendation unit 106 determines “pants” and “skirts” having colors such as white, pink, and magenta as recommended products.

  Furthermore, for example, the first acquisition condition based on the sales product at the location where the customer is in the store is the recommended product for the skirt, the second acquisition condition based on the customer's purchase history is that the product color is red, the customer When a product that satisfies the third acquisition condition based on the attribute value of F3 is to be a recommended product, the recommendation unit 106 has pants and a skirt having colors such as white, pink, and magenta determined by the recommended product correspondence table. On the other hand, recommended products satisfying the first to third acquisition conditions are determined.

  When the acquisition conditions are combined, it becomes “a skirt having a deep reddish-purple color suitable for senior women”, so the recommendation unit 106 decides, for example, a deep-colored red-purple skirt as a recommended product and stores product information. The product information of the product having the deep reddish purple and skirt attribute values is acquired from the product information stored in the section 101.

  Also, for example, when the owner information, type, and color of the article in the photograph acquired by the owner information acquisition unit 105 are “second owner information”, “blouse”, and “red”, the recommendation unit 106 Selects a product described in the “recommended product” column of ID = *** 12 in the recommended product correspondence table shown in FIG. 6 as the recommended product. That is, the recommendation unit 106 determines a blouse having a color such as white, pink, or magenta as a recommended product.

  Furthermore, for example, the first acquisition condition based on the sales product at the location where the customer is in the store is a blouse recommended product, the second acquisition condition based on the customer's purchase history is that the product color is red, When a product that satisfies the third acquisition condition based on the attribute value of F3 is to be a recommended product, the recommendation unit 106 applies a blouse having a color such as white, pink, or magenta determined in the recommended product correspondence table. Then, a recommended product satisfying the first to third acquisition conditions is determined.

  When the acquisition conditions are summed up, it becomes a “blouse with deep reddish purple suitable for senior women”, so the recommendation unit 106 determines, for example, a deep reddish purple blouse as a recommended product, and the product information storage unit 101 The product information of the product having the deep reddish purple and blouse attribute values is acquired from the product information stored in.

  When the recommendation unit 106 determines the recommended product, the recommendation unit 106 acquires the product information of the recommended product from the product information storage unit 101 and transmits the product information to the photography apparatus 2. For example, when the recommended product determined by the recommendation unit 106 corresponds to the skirt of the product information ID = 77 *** 2 shown in FIG. 5, the recommendation unit 106 receives the ID = 77 *** from the product information storage unit 101. The product information of 2 is read and transmitted to the photography apparatus 2.

  When the recommendation unit 106 transmits the product information, the product information receiving unit 204 of the photography apparatus 2 receives the product information, and the output unit 205 outputs the product information. The output unit 205 converts, for example, character information (information such as product name, price, type, color, size, etc.) and product sample photos included in the product information into image information for display. The information is output (displayed) on the liquid crystal display of the touch panel of the photographing apparatus 2.

  In the specific example 2, the photography apparatus 2 outputs the received product information in the display mode shown in FIGS. The description of the display modes of FIGS. 9 and 10 is given in the first specific example, and the description thereof is omitted here.

  When the recommended product is displayed on the display screen 11 of the tablet terminal 1, the store clerk shows the recommended product to the customer, so that the customer sees a different type of product worn by the customer or in the store. It is possible to easily propose recommended products from the same types of products as sales products.

  In the specific examples 1 and 2 described above, the clothes worn by the customer have been described as an example, but the same method can be applied to accessories such as accessories worn by the customer and small items such as wallets possessed by the customer. Makes it easy to propose recommended products.

  Further, in the above specific example 2, as the conditions for determining the recommended product, the condition based on the product information of the sales product viewed by the customer in the store, the condition based on the purchase history of the customer, and the condition based on the attribute value of the customer Although all of them are used, a recommended product may be determined by combining any one of these conditions or two or more conditions.

  As described above, according to the recommendation device 1 of the present embodiment, a photograph of a product that the customer who visits the store owns or wears or a product that the customer sees in the store is displayed. By simply taking a picture, it is possible to output product information related to the recommended product of the type corresponding to the owner information of the item in the photograph, so it is possible to easily suggest the recommended product to the customer, and the customer's recommendation Increase the willingness to purchase products.

  In particular, if the item in the photo is an item that the customer has or wears, the recommended item is determined from a different type of item and the item related to the recommended item. Since the information is output, it is possible to suitably propose a recommended product of another type of article to be used in accordance with the article that the customer has or wears.

  Moreover, according to the recommendation apparatus 1 of this Embodiment, when the goods reflected in the photograph are the sales goods which the customer sees in the store, the recommended goods are determined from the same type of goods as the goods. Since the product information of the recommended product is output, it is possible to propose a recommended product suitable for a product that the customer is interested in or interested in.

  Moreover, according to the recommendation apparatus 1 of this Embodiment, since a recommended product is determined also using a customer's purchase history, the recommended product according to a customer's liking and preference can be proposed.

  Furthermore, according to the recommendation device 1 of the present embodiment, recommended products are determined using customer attribute values, so it is possible to propose recommended products that match the gender, age group, etc. of the customer.

  In the present embodiment, an example in which a device that outputs product information related to recommended products is configured by the recommendation system configured by the bullet terminal 2 and the recommendation device 1 has been described. However, as shown in FIG. A stand-alone configuration may be used.

  A recommendation device 1 </ b> A shown in FIG. 16 is obtained by adding a motion detection unit 201, a camera unit 202, and an output unit 205 of the photographic device 2 to the configuration of the recommendation device 1. The recommendation device 1A can be realized by a tablet terminal, for example.

  The contents of each part in the recommendation device 1A are as described above in the description of the block diagram of the recommendation device 1 and the photography device 2 shown in FIG.

  Further, the processing in the recommendation device 1 of the present embodiment may be realized by software. The software may be distributed by software download or the like. The software may be recorded on a recording medium such as a CD-ROM and distributed.

  The software that implements the recommendation device 1 in the present embodiment is the following program.

  That is, the computer-accessible recording medium includes a product information storage unit that can store two or more product information about products handled in the store, and the computer shows the article. A photo acceptance unit that accepts a photo, an owner information acquisition unit that recognizes a photo and acquires owner information about the owner of the article, and a type of product according to the owner information acquired by the owner information acquisition unit This is a program for obtaining product information from a product information storage unit and causing it to function as a recommendation unit that outputs the product information.

  In the above program, if the owner information acquired by the owner information acquisition unit is the first owner information indicating that the owner of the item is the customer side, the recommendation unit indicates the type of the item in the photograph and It is preferable to obtain and output the product information of different types of products from the product information storage unit.

  In addition, in the above program, if the recommendation section is the second owner information indicating that the owner information acquired by the owner information acquisition section is the store where the customer came to the store, It is preferable to acquire and output the product information of the same type of product as the type of the article in the product information storage unit.

  In the above program, the computer-accessible recording medium can store the product related information related to the sales product in the store and the location related information related to the display position of each sales product in the store in association with each other. A location product information storage unit is further provided, and the computer stores the location information reception unit that receives location information indicating the location of the customer and the location product information storage unit from the location product information storage unit using the location information received by the location information reception unit. The product related information acquisition unit that acquires the product related information of the sales products displayed at the position where the customer is located, and the recommendation unit also uses the product related information acquired by the product related information acquisition unit, It is preferable to determine the product information acquired from the product information storage unit.

  In the above program, when the owner information acquired by the owner information acquisition unit is the first owner information, the recommendation unit is the same type of product as the sales product corresponding to the product related information acquired by the product related information acquisition unit. Therefore, it is preferable to obtain and output the product information of the product that matches the product shown in the photograph from the product information storage unit.

  In the above program, the computer-accessible recording medium further includes a purchase history storage unit capable of storing the customer purchase history in association with the terminal identification information, and identifies the computer as the mobile terminal device of the customer. Further, the recommendation unit obtains the purchase history of the customer from the purchase history storage unit using the terminal identification information received by the terminal identification information reception unit. It is preferable to obtain and output from the product information storage unit also using the history.

  In the above program, the computer is further caused to function as an attribute value acquisition unit that acquires one or more attribute values of the customer shown in the photograph, and the recommendation unit also uses the one or more attribute values of the customer as the product information. It is preferable to determine the product information acquired from the storage unit.

  FIG. 17 shows the external appearance of a computer that executes the above-described program to realize the recommendation device 1 of the above-described embodiment. The above-described embodiments can be realized by computer hardware and a computer program executed thereon. FIG. 16 is an overview diagram of the computer system 300, and FIG. 18 is a block diagram of the system 300.

  In FIG. 17, a computer system 300 includes a computer 301 including a CD-ROM drive, a keyboard 302, a mouse 303, and a monitor 304.

  In FIG. 18, in addition to the CD-ROM drive 3012, the computer 301 includes an MPU 3013, a bus 3014 connected to the MPU 3013 and the CD-ROM drive 3012, a ROM 3015 for storing a program such as a bootup program, and an MPU 3013. And a RAM 3016 for temporarily storing instructions of the application program and providing a temporary storage space, and a hard disk 3017 for storing the application program, the system program, and data. Although not shown here, the computer 301 may further include a network card that provides connection to a LAN.

  A program that causes the computer system 300 to execute the function of the recommendation device 1 of this embodiment may be stored in the CD-ROM 3101, inserted into the CD-ROM drive 3012, and further transferred to the hard disk 3017. Alternatively, the program may be transmitted to the computer 301 via a network (not shown) and stored in the hard disk 3017. The program is loaded into the RAM 3016 at the time of execution. The program may be loaded directly from the CD-ROM 3101 or the network.

  The program does not necessarily include an operating system (OS), a third party program, or the like that causes the computer 301 to execute the function of the recommendation device according to the above-described embodiment. The program only needs to include an instruction portion that calls an appropriate function (module) in a controlled manner and obtains a desired result. How the computer system 300 operates is well known and will not be described in detail.

  In the above program, in the step of transmitting information and the step of receiving information, processing performed by hardware, for example, processing performed by a modem or an interface card in the transmitting step (only in hardware) Processing not performed) is not included.

  Further, the computer that executes the program may be singular or plural. That is, centralized processing may be performed, or distributed processing may be performed.

  Further, in each of the above embodiments, it goes without saying that two or more communication units existing in one apparatus may be physically realized by one medium.

  In each of the above embodiments, each process and each function may be realized by centralized processing by a single device or a single system, or by distributed processing by a plurality of devices. It may be realized.

  Note that the present invention is not limited to the above-described embodiment, and can be variously modified without departing from the scope of the invention in the implementation stage, and these are also included in the scope of the present invention. Needless to say.

  As described above, the recommendation device according to the present invention is a photograph of a customer who visits a store taking a picture of a product worn by the customer or a sale product displayed at a position where the customer is in the store. Can be used to propose product information related to recommended products of a type corresponding to owner information of articles etc. in the photograph, and is useful as a recommendation device or the like.

DESCRIPTION OF SYMBOLS 1,1A Recommendation apparatus 101 Product information storage part 102 Location product information storage part 103 Purchase history storage part 104 Photo reception part 105 Owner information acquisition part 106 Recommendation part 107 Position information reception part 108 Commodity related information acquisition part 109 Terminal identification information reception Part 110 Attribute value acquisition part 2 Information terminal device (tablet terminal)
201 motion detection unit 202 camera unit 203 photo transmission unit 204 product information reception unit 205 output unit

Claims (11)

A product information storage unit capable of storing two or more product information about the product;
A photo reception unit that accepts a photo showing an article;
An owner information acquisition unit that recognizes an image of the photograph and acquires owner information related to the owner of the article;
A recommendation unit that obtains and outputs the product information of the type of product according to the owner information acquired by the owner information acquisition unit from the product information storage unit, and
A recommendation device comprising: The recommendation part is:
In the case where the owner information acquired by the owner information acquisition unit is first owner information indicating that the owner of the article is on the customer side, a different type of product from the type of article shown in the photograph Obtain and output product information from the product information storage unit,
The recommendation device according to claim 1. The recommendation part is:
If the owner information acquired by the owner information acquisition unit is second owner information indicating that the owner of the product is on the store side, the product of the same type as the type of the product shown in the photo Obtain information from the product information storage unit and output it,
The recommendation device according to claim 1. A position product information storage unit that can store the product related information related to the sales product of the store and the position related information related to the display position of each sales product in the store in association with each other;
A location information receiver that receives location information indicating the location of the customer ;
A product related information acquisition unit that acquires product related information of a sales product displayed at a position where the customer is located in the store from the position product information storage unit using the position information received by the position information receiving unit; ,
Further comprising
The recommendation part is:
Using the product related information acquired by the product related information acquisition unit, determining the product information acquired from the product information storage unit,
The recommendation apparatus of Claim 2 or Claim 3. The recommendation part is:
When the owner information acquired by the owner information acquisition unit is the first owner information, it is a product of the same type as the sales product corresponding to the product related information acquired by the product related information acquisition unit, and is shown in the photograph Obtain and output product information of the product that matches the product from the product information storage unit,
The recommendation device according to claim 4. A purchase history storage unit that can store a customer 's purchase history in association with terminal identification information that identifies the mobile terminal device of the customer ;
Terminal identification information receiving unit for receiving the terminal identification information,
Further comprising
The recommendation part is:
Using the terminal identification information by the terminal identification information receiving unit receives, acquires the customer purchase history from the purchase history storage unit, determines the product information acquired from the product information storage unit also using the purchase history ,
The recommendation apparatus as described in any one of Claims 1-5. Further comprising an attribute value acquisition unit for acquiring one or more attribute values of the customer shown in the photograph;
The recommendation part is:
Determining product information to be acquired from the product information storage unit using one or more attribute values of the customer;
The recommendation apparatus as described in any one of Claims 1-6. The photo acceptance unit is for receiving the photo input from photographing apparatus for photographing a photograph,
The recommendation section, in conjunction with the photographing operation of the photograph by the photographing apparatus acquires the product information from the product information storage unit, performs a process of outputting,
The recommendation apparatus as described in any one of Claims 1-7. The photography apparatus includes:
A motion detector for detecting the motion of the photography apparatus;
In conjunction with detection of the movement of the photographic device by the motion detection unit, a camera unit that transitions to a shooting mode capable of taking a photograph of the article,
Comprising
The recommendation apparatus of Claim 8. The recording medium is
A product information storage unit capable of storing two or more product information about the product;
A product recommendation method realized by a photo reception unit, an owner information acquisition unit, and a recommendation unit,
A photo accepting step in which the photo accepting unit accepts a photo showing an article;
The owner information acquisition unit acquires the owner information related to the owner of the article by recognizing the image of the photograph; and
A product information output step in which the recommendation unit acquires product information of a type of product corresponding to the owner information acquired in the owner information acquisition step from the product information storage unit and outputs the product information. Recommendation method. Computer-accessible recording media
A product information storage unit capable of storing two or more product information related to products handled in a store;
The computer,
A photo reception unit that accepts a photo showing an article;
An owner information acquisition unit that recognizes an image of the photograph and acquires owner information related to the owner of the article;
A recommendation unit that obtains and outputs the product information of the type of product according to the owner information acquired by the owner information acquisition unit from the product information storage unit, and
Program to make it function.

Images