JP2012014544A - Coordinate recommendation apparatus, coordinate recommendation method and program therefor - Google Patents

Abstract

PROBLEM TO BE SOLVED: To recommend suitable coordinates from photographs of fashion items.SOLUTION: A coordinate recommendation apparatus 1 has a magazine full-length photograph feature extraction section 2 for extracting a full-length photograph comprised of a combination of a plurality of kinds of items from a magazine photograph set 61 that is an image set, specifying a region of an item for each kind for each full-length photograph, extracting a feature amount for each region, and storing the feature amount as a magazine full-length photograph feature set 63; an owned photograph feature extraction section 3 for extracting a full-length photograph from an owned photograph set 62 that is an image set, specifying a region of an item for each kind for each full-length photograph, extracting a feature amount for each region, also extracting a feature amount for an image comprised of one type of item in the owned photograph set, and storing the extracted feature amount as an owned photograph feature set 64; and a recommendation section 4 for learning relationships among regions of items using the magazine full-length photograph feature set 63 and the owned photograph feature set 64, and presenting another item for each kind suitable for a combination with an inputted item.

Description

  The present invention relates to a coordination recommendation device, a coordination recommendation method, and a program thereof for recommending appropriate coordination when a photo set of fashion items such as pants and shirts is given.

  Many people are interested in fashion. They regularly check for trends in fashion magazines and use them as reference when buying clothes and making decisions. The fashion magazine contains a lot of photos of the models that I coordinated for my reference. For example, when choosing clothes to wear from among a lot of clothes that you have, or when purchasing clothes at an online store, you can easily achieve ideal coordination by referring to the pictures of such models be able to. Therefore, there is a demand for an apparatus for recommending a desired coordinate from a photograph image of these magazines.

2. Description of the Related Art Conventionally, as a method of handling products in an online store and recommending products to a user, a method of recommending based on the purchase history and score data of the user is known (see Non-Patent Document 1).
Also, in electronic commerce on the Web, there is known a coordinate recommendation method for a plurality of types of items such as trousers, shirts, shoes, and ties using metadata (see Non-Patent Document 2).

Tomoharu Iwata, Takeshi Yamada, Nobuo Ueda, "Collaborative Filtering Using Purchase Order Efficiently" IPSJ Transactions: Mathematical Modeling and Applications, Vol.49, No.SIG4 (TOM20), p.125-134 , 2008 E. Shen, H. Lieberman, F. Lam, “What am I gonna wear ?: Scenario-Oriented Recommendation.”, In IUI '07, p.365-368, 2007

  However, the method of Non-Patent Document 1 has a problem that history data is required. Further, the method of Non-Patent Document 2 has a problem that metadata must be created manually.

  Therefore, the present invention has been made in view of the above problems, and by automatically extracting information on coordination from a coordinated photographic image as a model, it can be combined with a user's desired item. It is an object of the present invention to provide a coordinate recommendation device, a coordinate recommendation method, and a program for recommending a suitable coordinate.

  In order to solve the above problems, a coordination recommendation device according to the present invention is a coordination recommendation device that recommends another type of item suitable for combination with an input item for the coordination of a plurality of types of fashion items. A storage unit for storing a reference photo set predetermined as an exemplary coordinate image set and a recommended photo set predetermined as an image set of recommended items, and the reference photo set A whole body photograph showing an image in which a plurality of types of items are combined is extracted from the inside, each item area is identified in the extracted whole body photograph, and an image feature amount is extracted from each of the identified item areas. , The reference whole body for storing the feature amount extracted for each item in the storage unit as a reference whole body photograph feature set Extracted from the true feature extraction unit and the recommended photo set are a whole body photo showing an image in which a plurality of types of items are combined and a single item photo showing an image consisting of only a single item, which are extracted A region of each item is identified in the whole body photo, and an image feature amount is extracted from the identified item region, and an image feature amount is extracted from the single item photo. A region of each item using the recommendation photo feature extraction unit that stores the feature amount extracted for each item as a recommendation photo feature set in the storage unit, the reference whole body photo feature set, and the recommendation photo feature set The relationship between the regions, and the combination of the input item according to the relationship between the regions and the feature amount of the image of the input item. Search for different types of items which are suitable for office at the recommended photographic set, characterized in that it comprises a recommendation unit that presents as an item to be recommended.

  The coordinate recommendation device having such a configuration stores a reference photo set and a recommendation photo set in the storage unit. The reference photo set is an exemplary coordinated image set, and for example, an image acquired from a fashion magazine or an image acquired from the web can be used. The recommended photo set is an image set of recommended items, and for example, an image of an item owned by a user or an image of an item sold in an online store can be used. Examples of item types include upper garments, lower garments, and shoes. Then, the coordination recommendation device extracts the feature amount of the image for each of the reference photo set and the recommendation photo set stored in the storage unit. Here, the feature amount of the image indicates, for example, a color, a texture, a shape, and the like. Then, the coordination recommendation device learns the relevance between the areas of each item using the set of image feature amounts extracted for the reference photo set and the recommended photo set. Then, the coordination recommendation device searches for and presents another type of item suitable for the combination with the input item using the relationship between the areas of each item. Therefore, according to the coordinate recommendation device having such a configuration, even if the purchase history data of the item and the metadata related to the coordinate are not given, different types of items that can be appropriately coordinated according to the given item are displayed. Can be recommended.

  Further, in the coordinate recommendation device according to the present invention, the recommendation unit uses the reference whole body photo feature set and the recommendation photo feature set, each item in the whole body photo in the reference photo set, The similarity calculation unit for calculating the similarity with each item in the recommended photo set for each item type, and the whole body photo including the item of the same type as the input item and having a high similarity Search from a set of photographs, extract another type of item combined with the same type of item as the input item in the whole body photo image obtained by the search, and the similarity with the other type of item A presentation unit that retrieves a predetermined number of items from the recommendation photo set in descending order and presents the items obtained by the search as recommended items. Rukoto is preferable.

According to the coordinate recommendation device having such a configuration, it is possible to calculate the degree of similarity of items in an image using the feature amount extracted from the image. Thus, the coordination recommendation device first searches the reference photo set for a whole body photo as an image similar to the input item. Thereby, an ideal coordination can be obtained for the input item.
Next, the coordination recommendation device extracts items that are combined in the obtained coordination and that are different from the input item. Then, the coordination recommendation device searches the recommended photograph set for an image similar to the extracted item. Therefore, the coordinate recommendation device recommends another type of item suitable for the combination with the input item with an ideal coordinate by calculating the similarity between the reference photo set and the recommended photo set. Can do.

  Further, in the coordinate recommendation device according to the present invention, the recommendation unit learns the relationship between the areas of each item in the whole body photo in the reference photo set using the reference whole body photo feature set, A model learning unit that learns a topic model that outputs, as a probability value, a ratio of each topic having a potential meaning inherent in the item when the feature amount of the region is given; Using the topic model for the feature quantity of the image of the input item, the input item topic ratio indicating the ratio of the input item having each topic is estimated and input in the recommended photograph feature set. Using the topic model for each feature quantity related to an item of a type different from the selected item, the recommendation value corresponding to the feature quantity is used. A topic estimation unit for estimating a recommendation item topic ratio indicating a ratio of items stored in a true set having each topic, and the input item topic ratio and the recommendation item as different types of items from the input item It is preferable to include a presentation unit that searches a predetermined number of items from the recommendation photo set in descending order of similarity to the item topic ratio, and presents the items obtained by the search as recommended items. .

  According to the coordinate recommendation device having such a configuration, the topic model that outputs the ratio of each topic having each topic as a probability value by learning the relationship between the regions of each item in the whole body photo from the reference whole body photo feature set. learn. By using this topic model, it is possible to obtain a topic as an ideal coordinate feature for the input item feature quantity. This coordination recommendation device first estimates a topic ratio by applying a topic model to each of an input item and different types of items included in the recommended photograph feature set. Here, the different types are different from the types of the input items and are determined in advance. For example, when the type of the input item is an upper garment, a different type can be defined as a lower garment. Next, the coordination recommendation device searches an image of an item having a topic ratio similar to the topic ratio obtained for the input item from the recommendation photo set. Therefore, the coordinate recommendation device can recommend another type of item suitable for the combination with the input item with an ideal coordinate by learning the topic model of the reference photo set.

  In the coordinate recommendation device according to the present invention, the type of the item is an upper garment and a lower garment, and the reference whole body photo feature extraction unit extracts a face area for each image included in the reference photo set. A whole-body photo specifying unit that extracts an image having an area that is wider than the width of the extracted face area and larger than the height of the face area under the face area as the whole-body photograph; Based on the size of the face area, a partial area having a predetermined width and height with respect to the face area existing in the lower part of the face area is defined as an upper clothing item area, and the upper clothing item area is defined below. An area specifying unit that specifies a partial area having a predetermined width and height with respect to the existing face area as an area of an item of a garment, and the recommended photo feature extracting unit includes the recommended photo set Included in For each image, a face area is extracted, and an image in which an area wider than the width of the extracted face area and larger than the height of the face area exists below the face area is extracted as the whole body photograph. A partial area having a predetermined width and height with respect to the face area existing below the face area on the basis of the size of the whole body photo specifying part and the size of the face area, It is preferable to have an area specifying unit that specifies a partial area having a predetermined width and height with respect to the face area existing below the area of the upper garment item as the area of the lower garment item.

  According to the coordinate recommendation device having such a configuration, it is possible to accurately extract the entire photo on the basis of the size of the face area to be extracted and the position in the photo. Then, the coordinate recommendation device sets the area of the upper garment item in the lower part of the face area with the face area in the entire photo as a reference, and further sets the area of the lower garment item in the lower part of the upper garment item area. Set. Therefore, the coordination recommendation device can accurately specify the area of the upper garment item and the area of the lower garment item. Thereby, the feature of the coordination between the item of the upper garment and the item of the lower garment can be accurately extracted.

  Further, in order to solve the above-described problem, the coordinate recommendation method according to the present invention is recommended with a reference photo set predetermined as a coordinate image set as a model for coordinating a plurality of types of fashion items. Coordinated by a coordinate recommendation device that recommends another type of item suitable for a combination with an input item, which includes a storage unit that stores a recommended photo set as an image set of items and a processing unit In the recommendation method, the processing unit accepts an input of an item, and extracts a whole body photograph showing an image in which a plurality of types of items are combined from the reference photograph set, and the extracted whole body Identify the area of each item in the photo, and determine the image features from the identified item area. Each of the extracted feature quantities is stored in a storage unit as a reference whole body photo feature set for each item, and an image in which a plurality of types of items are combined from the recommended photo set is shown. Extract a whole body photo and a single item photo showing an image consisting of only a single item, specify the area of each item in the extracted whole body photo, and extract the feature amount of the image from the specified item area, respectively And extracting the feature amount of the image from the single item photo and storing the feature amount extracted for each item for the whole body photo and the single item photo as a recommended photo feature set in the storage unit; and the reference whole body Using the photo feature set and the recommended photo feature set, the relationship between regions of each item is learned, the relationship between the regions, Searching for another type of item suitable for the combination with the input item according to the feature amount of the image of the input item, and presenting the item. It is characterized by performing.

  Also, in the coordinate recommendation method according to the present invention, in the step of presenting the item, the processing unit uses the reference whole body photo feature set and the recommendation photo feature set in the reference photo set. Calculating the degree of similarity between each item in the whole body photo of each item and each item in the recommended photo set for each item type, and an item of the same kind as the input item and having a high degree of similarity A step of searching for a whole body photo including the reference photo set, and extracting another type of item combined with the same type item as the input item in the whole body photo image obtained by the search, Searching for a predetermined number of items from the recommendation photo set in descending order of similarity to another type of item; The resulting item, it is preferable to perform the step of presenting as an item to be recommended, the.

  Further, in the coordinate recommendation method according to the present invention, in the step in which the processing unit presents the item, the region of each item in the whole body photograph in the reference photograph set using the reference whole body photograph feature set. Topic that learns the relationship between them and outputs the probability that the item with the feature amount has each topic indicating the potential meaning inherent in the item when the feature amount of a certain region is given Learning a model, using the topic model for the feature quantity of the image of the input item, estimating an input item topic ratio indicating a ratio of the input item having each topic, and the recommendation Using the topic model for each feature quantity in the photo feature set for use, storing it in the recommended photo set corresponding to the feature quantity A step of estimating a recommendation item topic ratio indicating a ratio of each item having a topic, and a similarity between the input item topic ratio and the recommendation item topic ratio as an item of a different type from the input item It is preferable to perform a step of searching a predetermined number of items in descending order from the recommended photo set and presenting the items obtained by the search as recommended items.

  In the coordinate recommendation method according to the present invention, the type of the item is an upper garment and a lower garment, and the processing unit stores the reference whole body photo feature set in a storage unit. For each image included in the set, a face area is extracted, and an image in which an area wider than the width of the extracted face area and larger than the height of the face area exists below the face area is described above. Extracted as a whole-body photograph, and using the size of the face area as a reference, a partial area having a predetermined width and height with respect to the face area existing below the face area is defined as an area of the upper clothing item, Identifying a partial area having a predetermined width and height with respect to the face area existing below the item area of the item as an item area of the lower garment, and storing the recommended photograph feature set in the storage unit And For each image included in the recommended photo set, a face area is extracted, and an area that is wider than the width of the extracted face area and larger than the height of the face area exists below the face area. A partial area having a predetermined width and height with respect to the face area existing below the face area on the basis of the size of the face area. It is preferable that a partial area having a predetermined width and height with respect to the face area existing below the area of the upper garment item is specified as the area of the lower garment item.

  The coordinate recommendation program according to the present invention is a program for causing a computer to function as each means constituting the coordinate recommendation device. By being configured in this way, a computer in which this program is installed can realize each function based on this program.

  According to the present invention, it is possible to automatically extract information relating to an ideal coordinate from a reference photo set of coordinate as a model, and recommend a coordinate suitable for a combination with a user's desired item. In addition, according to the present invention, when selecting clothes to be worn from among a large amount of clothes possessed or purchasing clothes at an online store, purchase history data and metadata regarding coordination are not provided. It becomes possible to know the appropriate coordination.

It is a block diagram which shows the structure of the coordination recommendation apparatus which concerns on embodiment of this invention. It is a flowchart which shows the flow of a process by the coordination recommendation apparatus shown in FIG. It is a block diagram which shows the structure of the magazine whole body photograph feature extraction part shown in FIG. It is a block diagram which shows the structure of the possessed photograph feature extraction part shown in FIG. It is a block diagram which shows the 1st structural example of the recommendation part shown in FIG. It is a flowchart which shows the flow of a process by the recommendation part shown in FIG. It is a block diagram which shows the 2nd structural example of the recommendation part shown in FIG. It is a flowchart which shows the flow of a process by the recommendation part shown in FIG. It is a figure which shows an example of the upper and lower garment area | region extracted by the coordination recommendation apparatus which concerns on embodiment of this invention. It is a figure which shows the result of having compared the method based on similarity (Sim), the method based on a topic model (Topic), and the random presentation method (Random). It is a figure which shows the result of having compared the method based on similarity (Sim), the method based on a topic model (Topic), and the random presentation method (Random).

Next, the form for implementing the coordination recommendation apparatus of this invention is demonstrated with reference to drawings.
The coordination recommendation device 1 is for recommending another type of item suitable for combination with an input item for the coordination of a plurality of types of fashion items. In the following, when the coordinate recommendation device 1 is given a photograph of the upper body clothes (upper: tops), it is specifically shown that the photograph of the lower body clothes (lower clothes: bottoms) suitable for the clothes (upper) is presented. A typical task will be described as an example. On the contrary, when a photograph of a lower garment is given, it is a matter of course that a task of presenting a photograph of an upper garment suitable for the clothes (undergarment) can also be executed. It is possible to recommend not only upper and lower garments but also a plurality of items such as hats, bags and shoes. In the following description, it is assumed that clothes to be worn are selected from images of clothes possessed by the user. Select clothes to be worn from a large number of clothes possessed by the user, using a model photograph (magazine photograph image) published in a fashion magazine as a reference photographic image and an image of the clothes possessed by the user as a recommended photographic image. A case will be described.

(First embodiment)
[Outline of configuration of coordination recommendation device]
As shown in FIG. 1, the coordinate recommendation device 1 includes an input unit 7, a magazine whole body photo feature extraction unit (reference whole body photo feature extraction unit) 2, and an owned photo feature extraction unit (recommendation photo feature extraction unit) 3. A recommendation unit 4, an output unit 5, and a storage unit 6. FIG. 1 is a drawing common to the coordinate recommendation device 1B of the second embodiment to be described later, and reference numeral 4B represents a recommendation unit according to the second embodiment.

The input unit 7 includes, for example, a keyboard, a mouse, a disk drive device, and the like, and inputs information on items desired by the user.
The magazine whole body photo feature extraction unit 2, the possessed photo feature extraction unit 3, and the recommendation unit 4 function as a processing unit of the coordination recommendation device 1. The processing unit is a main control device including, for example, a CPU (Central Processing Unit) and a RAM (Random Access Memory). Details of the whole-body photo feature extraction unit 2, the owned photo feature extraction unit 3, and the recommendation unit 4 will be described later.
The output unit 5 is, for example, a graphic board (output interface) and a monitor connected thereto. The monitor is composed of a liquid crystal display or the like, for example, and presents pictures of recommended items.

The storage unit 6 is composed of, for example, a general hard disk device or the like, and stores programs used in the processing unit, processing data used for calculation, calculation processing results, and the like.
In FIG. 1, a magazine photo set 61 and an owned photo set 62 are shown as an example of processing data, and a magazine whole body photo feature set 63 and an owned photo feature set 64 are shown as an example of calculation processing results. _{Incidentally, D mag} shows a magazine systemic photos feature set 63 in the formula (1) to be described _{later, D own} is indicative of ownership photo feature set 64 in the formula (2) to be described later.

The storage unit 6 stores a magazine photo set 61 that is a reference photo set and an owned photo set 62 that is a recommended photo set as processing data before recommending items.
The magazine photo set 61 is a photo set predetermined as an exemplary coordinate image set, and here is a photo set of clothes acquired from a magazine or the like.
The owned photo set 62 is a photo set predetermined as an image set of recommended items, and here is a photo set of clothes owned by the user.

The magazine photo is not limited to a photo published in a magazine, and may be a photo on the web or the like as long as it is a reference for coordination.
In addition, the magazine photo set 61 and the possessed photo set 62 are not only photos showing images in which a plurality of types of items are combined (hereinafter referred to as full-body photos), but single photos such as a photo of only the upper garment and a photo of only the lower garment. A photo including the item (hereinafter referred to as a single item photo) may be included.
In addition, here, since the recommendation of coordination with clothes that are owned is taken as an example, the name is the owned photo set 62. However, in the case of an embodiment that is assumed to be recommended in an online store, the owned photo set 62 is It can be applied by replacing it with a photo collection of clothes sold. The magazine whole body photo feature set 63 and the owned photo feature set 64 will be described later.

  The magazine whole body photo feature extraction unit 2 extracts a whole body photo from the magazine photo set 61, identifies the area of each item in the extracted whole body photo, and extracts the image feature amount from the identified item area, respectively. The feature quantity extracted for each item is stored in the storage unit 6 as a magazine whole body photograph feature set (reference whole body photograph feature set) 63. Details of the magazine whole body photo feature extraction unit 2 will be described later.

  The owned photo feature extraction unit 3 extracts a whole body photo and a single item photo from the owned photo set 62, specifies the area of each item in the extracted whole body photo, and extracts the image from the specified item area. Each feature amount is extracted, the feature amount of the image is extracted from the single item photograph, and the feature amount extracted for each item for the whole body photograph and the single item photograph is stored as an owned photo feature set (recommended photo feature set) 64 6 is stored. Details of the owned photograph feature extraction unit 3 will be described later.

  The recommendation unit 4 uses the magazine whole body photo feature set 63 and the possessed photo feature set 64 to learn the relationship between the regions of each item, and the relationship between the regions and the feature amount of the input item image Accordingly, another type of item suitable for the combination with the input item is searched from the owned photo set 62 and presented as a recommended item. Here, there are two methods for learning the relationship between the areas of each item, which will be described in the first embodiment and the second embodiment, respectively. Details of the recommendation unit 4 will be described later.

[Outline of operation of coordinate recommendation device]
The operation flow of the coordination recommendation device 1 will be described with reference to FIG. 2 (refer to FIG. 1 as appropriate).

  The coordination recommendation device 1 accepts an input of a photograph of an item (for example, an upper garment) that the user desires to find as an item desired by the user through the input unit 7 (step S1).

  The coordinate recommendation device 1 uses the magazine whole body photo feature extraction unit 2 to identify a photograph (whole body photo) that shows the whole body from the magazine photo set 61 that has been read, and in each of the whole body photos, as an area for each item, And the lower garment area are extracted. When recommending not only upper and lower garments but also hats, bags, and shoes, those areas are also extracted here. Then, the magazine whole body photo feature extraction unit 2 extracts a predetermined image feature amount for each of the upper garment region and the lower garment region in the whole body photo, and stores it in the storage unit 6 as a magazine whole body photo feature set 63. Store (step S2).

  The coordinate recommendation device 1 uses the owned photo feature extraction unit 3 to identify the whole body photo from the read owned photo set 62, and in each whole body photo, as an area of each item, an upper clothing area and a lower clothing area. Extract. Then, the possessed photograph feature extraction unit 3 extracts a predetermined image feature amount similar to step S2 for each of the upper garment area and the lower garment area specified in the whole body photograph (step S3).

  Here, it is not necessary to specify an area for a single item photo of only the upper garment or a single item photo of only the lower garment in the possession photo set 62 unlike the whole body photo. Therefore, the possessed photograph feature extraction unit 3 directly extracts the feature amount of the image without specifying an area for the single item photograph. In step S3, the feature amount extracted from the whole body photograph or the single item photograph is stored in the storage unit 6 as the possessed photograph feature set 64. In addition, you may perform the process of above-described step S1-step S3 not only in this order but in random order.

  The recommendation unit 4 of the coordination recommendation device 1 uses the magazine whole body photo feature set 63 extracted by the magazine whole body photo feature extraction unit 2 and the owned photo feature set 64 extracted by the owned photo feature extraction unit 3, The relationship between items is calculated, and using this relationship, another type of item (combination item) suitable for combination with the item given by the input unit 7 is searched (step S4).

Specifically, the coordinate recommendation device 1 intuitively extracts a magazine photo similar to a given upper garment by the recommendation unit 4 and is owned by the user similar to the lower garment of the extracted photo. Present clothes. Or conversely, the coordination recommendation device 1 extracts a magazine photo similar to the given lower garment by the recommendation unit 4 and presents the clothes owned by the user similar to the upper garment of the extracted photo.
Then, the coordination recommendation device 1 presents (recommends) the item (selected item) obtained by the search by the output unit 5 (step S5).

[Details of coordination recommendation device]
Below, the specific process of the magazine whole body photograph feature extraction part 2, the possession photograph feature extraction part 3, and the recommendation part 4 of the coordination recommendation apparatus 1 is demonstrated. For the recommendation unit 4, either a recommendation unit based on a similar region (first embodiment) or a recommendation unit based on a topic model (second embodiment) may be used.

<Magazine Whole Body Photo Feature Extraction Unit 2>
The configuration of the magazine whole body photo feature extraction unit 2 will be described with reference to FIG. In the sub-block diagrams shown in FIGS. 3, 4, 5, and 7, the elements that are not necessary for the main explanation in the sub-block are omitted from the elements shown in FIG.
As shown in FIG. 3, the magazine whole body photo feature extracting unit 2 includes a photo reading unit 21, a whole body photo specifying unit 22, a region specifying unit 23, and a feature extracting unit 24.

First, the magazine photo set 61 is read from the storage unit 6 by the photo reading unit 21.
Then, the whole body photo specifying unit 22 extracts a face area for each photo of the read magazine photo set 61 and specifies a whole body picture based on the size of the face area. Here, the method for extracting the face region is not particularly limited, and a known method can be used. For example, a face region can be extracted by a method described in “Viola, Jones,“ Robust Real-time Object Detection ”IJCV 2001”.

  In addition, as conditions for specifying the whole body photograph, for example, a condition as to whether a face area is included or a condition for selecting an appropriate size and position of the face area is used. Here, the condition for selecting an appropriate size and position of the face area refers to, for example, a condition that there is a space where the upper and lower garments are present. That is, the whole body image specifying unit extracts, as a whole body image, an image in which an area having a width wider than the width of the face area and larger than the height of the face area exists below the face area.

In the present embodiment, as an example, the whole body photo specifying unit 22 extracts, for example, an image satisfying all the following conditions (a) to (h) as a whole body photo for each image included in the magazine photo set 61. did.
(A) The face area exists (b) The width of the face area is 1/3 or less of the image (c) The width of the face area is 1/10 or more of the image (d) Above the image (E) There is a face region on the right side of the 1/4 position of the entire image width measured from the left of the image (e). f) A face area exists on the left side of the position of 3/4 of the entire image width measured from the left of the image. (g) There is a space corresponding to seven heights of the face area below the face area (h) ) The average value of the total values of RGB (Red, Green, Blue) indicating the color at each point in the face area is 600 or less.

Here, as the condition (a), the shape of the face region can be a square, for example.
The condition (g) indicates a condition for selecting an appropriate photograph having a space for the upper body and the lower body. Regarding condition (h), the values of R, G, and B are 0 to 255, respectively. If the sum of RGB values at a certain point is larger than 600, the point is too white as a human face. Is reflected. In other words, if the average of the RGB values for the entire face area is 600 or less, an appropriate photograph can be selected as the color of the human face.

  In a fashion magazine, two people may appear as models in one photo. When such a whole body photo is included in the magazine photo set 61, the above (a) to ( It is also assumed that a plurality of face regions that satisfy all the conditions of h) are extracted. In such a case, the above condition is determined using the image closest to the vertical center line in the height direction of the target image as the face region.

  Next, the area specifying unit 23 specifies the upper garment area and the lower garment area. Here, the shape of the upper garment region and the lower garment region can be, for example, rectangular. At this time, a region having a predetermined size existing below the face region is defined as an upper clothing region, and a region having a predetermined size present below the upper clothing region is defined as a lower clothing region. The size of the upper garment area and the lower garment area is given by a ratio with a predetermined face area. The size of the upper garment area is set to, for example, twice the width of the face area and 2.5 times the height. Also, the size of the lower garment area is set, for example, twice as wide as the face area and 3.5 times as high. Moreover, the arrangement | positioning of each area | region can specify an upper garment area | region continuously in the downward direction of a face area | region, for example, and can specify as a lower garment area | region continuously in the downward direction. In the case where the belt is used as another type of item, the upper garment area and the lower garment area may not be continuous and may be arranged with a predetermined width corresponding to the belt. Moreover, the width | variety and height of the said upper garment area | region or lower garment area | region are not limited to this value, An appropriate value can be set previously. For example, the ratio of the width and height of the face area and the upper garment area (or lower garment area) is calculated based on the data obtained by measuring the size of the face area and the upper and lower body of some people. The value may be used. Also, learning the ratio of the width of the face area to the upper garment area (or the lower garment area) and the ratio of the height when using the whole body photographs of various people as learning data and using the face area as a reference. The value may be used.

  An example of an image satisfying all the conditions (a) to (h) is shown in FIG. FIG. 9 is a diagram showing an example of upper and lower garment region extraction extracted in an example described later. In FIG. 9, reference numeral 101 represents a face area, reference numeral 102 represents an upper garment area, and reference numeral 103 represents a lower garment area. In the case of the example in FIG. 9, since the contrast between the floor of the model standing position and the background wall of the model is large, the height of the lower garment area is set to 4.5 times the height of the face area. did. In addition, when using shoes as another type of item, the height of the lower garment region may be shorter than 3.5 times the height of the face region.

Returning to FIG. 3, the configuration of the magazine whole body photo feature extraction unit 2 will be described.
Next, the feature extraction unit 24 extracts the feature amounts of the images of the upper garment region and the lower garment region in the whole body photograph specified by the region specifying unit 23. As the feature amount, any feature amount such as a color, texture, or shape can be used.

Hereinafter, in order to distinguish each feature amount, the following symbols are introduced for explanation.
The feature amount of ependymal of systemic photos m and u _m. (U: feature value of upper garment area)
Further, the feature amount of the lower garment of the same systemic photo m and l _m. (L: Feature value of the area of the lower garment)
For example, there are a plurality of feature quantities only for color features, and the feature quantities u _m and l _m are expressed by the following expressions (a) and (b) for each feature v.

u _m = {u _mv } _vεV (Equation (a))
l _m = {l _mv } _v∈V Equation (b)
Here, u _mv is the number of times the feature v appears in the upper region of the whole body photo m, l _mv is the number of times the feature v appears in the lower region of the whole body photo m, and V is a feature set.

  Through the above processing, the magazine whole body photo feature extraction unit 2 extracts the feature values of the upper and lower garment regions only for the photos determined to be whole body photos. Then, the magazine whole body photo feature extraction unit 2 stores the feature amounts of the upper and lower garment regions of each photo in the storage unit 6 as a magazine whole body photo feature set 63.

<Owned photo feature extraction unit 3>
The configuration of the possessed photograph feature extraction unit 3 will be described with reference to FIG.
As shown in FIG. 4, the possessed photo feature extracting unit 3 includes a photo reading unit 31, a whole body photo specifying unit 32, a region specifying unit 33, and a feature extracting unit 34.

The photo reading unit 31 reads the owned photo set 62 from the storage unit 6.
The whole body photo specifying unit 32 and the region specifying unit 33 are the whole body photo specifying unit 22 and the region specifying unit 23 in the magazine whole body photo feature extracting unit 2 except that the target photo is not a magazine photo but a possessed photo. Perform the same process as Therefore, detailed description is omitted.

  The feature extraction unit 34 extracts the feature amount of the image of the upper garment region and the lower garment region in the whole body photograph specified by the region specifying unit 33. Since this point is the same as that of the feature extraction unit 24 in the magazine whole body photo feature extraction unit 2 described above, detailed description thereof is omitted. In addition, the feature extraction unit 34 also extracts image feature amounts from single item photos (photos of only the upper garment and photos of only the lower garment) other than the whole body photos stored in the possessed photo set 62. The point is different from the feature extraction unit 24 in the magazine whole body photo feature extraction unit 2 described above. Since the method for extracting the feature amount is the same, detailed description thereof is omitted. The feature amounts extracted from the whole body photograph and the single item photograph by the feature extraction unit 34 are stored in the storage unit 6 as the possessed photograph feature set 64.

<First Embodiment of Recommendation Unit 4 (Recommendation Unit Based on Similar Area)>
The configuration of the recommendation unit 4 based on the similar area will be described with reference to FIG.
As shown in FIG. 5, the recommendation unit 4 includes a similarity calculation unit 41 and a presentation unit 42.

  The similarity calculation unit 41 uses the magazine whole body photo feature set 63 and the owned photo feature set 64 to calculate the similarity between each item in the whole body photo in the magazine photo set 61 and each item in the owned photo set 62. This is calculated for each item type.

  First, the similarity calculation unit 41 reads the magazine whole body photo feature set 63 and the possessed photo feature set 64 stored in the storage unit 6 and stores the magazine photo and the possession for each item type (upper, lower garment). Calculate the similarity to a photo. That is, the similarity between the upper (or lower) of the magazine photo and the upper (or lower) of the possessed photo is calculated. Any similarity such as cosine similarity can be used as the similarity.

  The presentation unit 42 determines a photo to be presented based on the similarity calculated by the similarity calculation unit 41. The presentation unit 42 searches the magazine photo set 61 for a whole body photo that includes an item of the same type as the input item and has a high similarity, and is the same type as the item input in the whole body photo image obtained by the search. The item obtained by extracting another type of item combined with the item of the item, searching for a predetermined number of items from the possession photo set 62 in descending order of similarity to the item of the other type. Are presented as recommended items. The predetermined number is a predetermined integer of 1 or more.

Here, as a premise of the explanation of the expressions (1) and (2) described later, the magazine whole body photo feature set 63 stored in the storage unit 6 is expressed as D _mag . The owned photograph feature set 64 is denoted as _Down . Here, the recommendation when the upper garment is given will be described, but the recommendation when the lower garment is given can be performed in the same manner.

As an example, description will be made with reference to FIG. 6 (refer to FIG. 5 as appropriate) assuming that the input unit 7 is provided with an upper garment having a feature value u _* . The task is to select a garment suitable for this outfit from the owned photo set 62 stored in the storage unit 6.

Similarity calculating unit 41, first, the amount given feature u _*, calculates the feature quantity u _m ependymal region systemic photos m in magazines systemic photo feature set D _{mag (63),} the degree of similarity. Then, the presentation unit 42 acquires the similarity of the upper garment region calculated by the similarity calculation unit 41 (step S11), and obtains a whole body photograph ^ m having a feature quantity most similar to the given feature quantity u _* . A search is made from the magazine photo set 61 in the storage unit 6 (step S12). This means that ^ m satisfying the following expression (1) is obtained. In the present specification, the symbol “^” represents the maximum value of the symbol attached on the right character as shown in the equation (1).

In equation (1), S (u _m , u _* ) is the feature quantity u _m of the upper region of the whole body photograph m in the whole body photograph feature set D _mag (63) of the magazine and the given feature quantity u _* . Represents similarity.

Then, for the magazine whole body photo ^ m specified by the presentation unit 42 in the magazine photo set 61 of the storage unit 6, the similarity calculation unit 41 uses the magazine whole body photo as a feature amount of the lower garment region of this magazine whole body photo ^ m. of the feature quantity _{l ^ m} stored in the feature in the set _D mag (64), and each feature quantity _{l n} of ownership photo feature set _D own (64) base garment area stored in the storage section 6 Each similarity is calculated.

Under The presentation unit 42 obtains the similarity of base garment area calculated by the similarity calculation unit 41 (step S13), and having from within own photo feature set D _own, the feature quantity l _{^ n} that most resembles A predetermined number of photos in order from the clothing photo n are searched from the possessed photo set 62 in the storage unit 6 (step S14). Here, for example, when only the photograph of the lower garment _{^ n} having the most similar feature quantity l _{^ n} is searched from the possessed photograph set 62 of the storage unit 6, the following equation (2) What is necessary is to obtain ^ n satisfying

(Second Embodiment)
<Second Embodiment of Recommendation Unit 4B (Recommendation Unit Based on Topic Model)>
Since the coordination recommendation device 1B of the second embodiment is the same as the first embodiment except that the function of the recommendation unit 4B is different, FIG. 1 is a common drawing. Below, the structure of the recommendation part based on a topic model is demonstrated.

  The topic model is a technique for extracting a topic (latent meaning) included in data, and is applied in various fields such as documents and images. A known method can be used as the topic model. For topic models, see, for example, “Tomoharu Iwata, Kazuaki Saito, Nobuo Ueda,“ Visualization of class structure by parametric embedding ”, IPSJ Journal, vol.46, pp.2337-2346 (2005)” Etc. are described.

Here, the topic is extracted from the whole body photo of the magazine, so that the relationship between the upper garment and the lower garment in the full body coordination published in the magazine etc. is learned and applied to the recommendation. In the topic model, the probabilities of the feature quantities u _m and l _m of the upper garment and the lower garment of one whole body photo m are expressed by Expression (3).

Here, z represents a topic, Z represents a topic set, and θ _mz represents a ratio of the whole body photo m having the topic z.
φ ^u _zv represents the probability that the feature v appears in the upper garment region in the topic z.
φ ^l _zv represents the probability that the feature v appears in the lower garment region in the topic z.
The feature quantities u _m and l _m are expressed as in the above formulas (a) and (b).

  A detailed configuration of the recommendation unit 4B will be described with reference to FIG. As illustrated in FIG. 7, the recommendation unit 4B includes a model learning unit 43, a topic estimation unit 44, and a presentation unit 42B.

  The model learning unit 43 uses the magazine whole body photo feature set 63 to learn the relationship between the regions of each item in the whole body photo in the magazine photo set 61, and when a feature amount of a certain region is given, A topic model is output that outputs, as a probability value, the ratio of items having feature quantities to each topic.

  The topic estimation unit 44 uses the topic model for the feature amount of the image of the input item to estimate the input item topic ratio indicating the ratio of the input item having each topic, and in the possessed photo feature set 64 Using the topic model for each feature amount of an item of a type different from the input item, the recommended item topic proportion indicating the proportion of items stored in the possessed photo set 62 corresponding to the feature amount. Is estimated.

  The presenting unit 42B searches the possessed photo set 62 for a predetermined number of items in descending order of similarity between the input item topic ratio and the recommended item topic ratio as different types of items from the input item. The item obtained by is presented as a recommended item.

Here, as a premise of the description of the expression (4) described later, the magazine whole body photograph feature set 63 stored in the storage unit 6 is expressed as D _mag . The owned photograph feature set 64 is denoted as _Down . Here, the recommendation when the upper garment is given will be described, but the recommendation when the lower garment is given can be performed in the same manner.

As an example, description will be made with reference to FIG. 8 (refer to FIG. 7 as appropriate), assuming that the upper part having the feature value u _* is given to the input unit 7. The task is to select a garment suitable for this outfit from the owned photo set 62 stored in the storage unit 6. Therefore, in the following, the input item topic ratio indicates the topic ratio of the upper garment, and the item topic ratio for recommendation indicates the topic ratio of the lower garment.

First, the model learning unit 43 reads the magazine whole body photo feature set D _mag and sets the topic model Θ = {θ _mz } _{m∈Dmag, z∈Z} , Φ = {φ ^u _zv , φ shown in the above equation (3). ^l _zv } _{Learn zεZ and vεV} (step S21). Accordingly, it is possible to obtain the probability (topic ratio) that each topic appears when a certain feature amount (feature amount of upper or lower garment) is given. Learning can be performed by a method such as an EM algorithm, variational Bayes, or Gibbs sampling.

Next, the topic estimation unit 44 uses the learned topic models Θ and Φ to estimate the topic ratio θ _* of the given upper photograph with respect to the feature value u _* of the given upper photograph (step S22). Here, the topic ratio θ _* indicates the probability of each topic appearing (topic ratio) when the feature amount of the upper garment is given.
Next, the topic estimating unit 44, the topic model Θ learned, using a [Phi, the feature quantity _{l n} in possession photographic feature set _{D own} topic proportion of base garment photographic _{θ n} = {θ _nz} z∈Z Is estimated (step S23). Here, the topic ratio θ _n indicates the probability (topic ratio) that each topic appears when the feature value of the lower garment is given.
Note that the processing order of step S22 and step S23 is arbitrary and may be performed in parallel.

Then, in the presentation unit 42B, a predetermined number of photos are sequentially searched from the owned photo set 62 from the lower garment photos ^ n having a topic rate most similar to the topic rate θ _* of the given upper garment photo. Are presented in order (step S24). Here, for example, in the case where only the underwear photo ^ n having the most similar topic ratio is searched from the owned photo set 62, ^ n satisfying the following expression (4) may be obtained.

Here D ^l _{own (part} of ownership photo feature set 64) represents a characteristic set of photos lower garment owns. As the similarity, any similarity such as negative KL divergence can be used.

  As mentioned above, although embodiment of this invention was described, this invention is not limited to this, It can implement in the range which does not change the meaning. For example, in the first embodiment, it has been described that one undergarment having the highest degree of similarity is selected by the above-described formula (2), but the present invention is not limited to this. For example, instead, a predetermined number of lower garments may be selected from those having a high degree of similarity and presented in order. Also, instead of presenting a full-length photo ^ m of the magazine with the maximum similarity as a reference as in the above formula (1), multiple full-length photos of the magazine are weighted in descending order of similarity, and the weight May be included in the above-described formula (2). A method of presenting a plurality of lower garment candidates by including the weight in the above-described formula (2) and referring to a plurality of whole body photos of the magazine is also conceivable.

  In order to evaluate the present invention, the coordinates corresponding to the items actually input by the coordination recommendation device according to the present embodiment were estimated.

<Region extraction>
In order to evaluate the present invention, an experiment was conducted using data from 32 female magazines. This data includes 14,813 images. This includes not only full-body photos required for the coordinate recommendation device but also images of accessories, symbols, and characters. As a result of extracting the whole body photo and specifying the upper and lower clothing areas using the coordinate recommendation device, the number was reduced to 2,062. When these extracted photographs were judged by humans, 1,502 images (73%) were appropriately extracted. The coordinate recommendation method of the present invention is simple, but it can be said that the region can be extracted with high accuracy. In subsequent experiments, we used 1,475 full-length photographs, which were duplicated photographs that were judged to be appropriate. An example of upper and lower clothing region extraction is as shown in FIG.

<Recommendation>
10% of all photos (1,475) were randomly selected as test photos, and 100 datasets were created using the other photos as learning photos. A color histogram was used as an image feature. For comparison, the method based on the similarity corresponding to the recommendation unit 4 in the first embodiment (Sim) and the method based on the topic model corresponding to the recommendation unit 4B in the second embodiment (Topic) are for comparison. In addition, we conducted a random presentation method (Random) with no particular ingenuity. For these, two types of experiments with different evaluation scales were performed. Here, n-best correct answer rate and similarity were used as different evaluation scales.

  Here, the n-best correct answer rate represents a ratio in which correct answers are included when the upper garment (lower garment) of the test photograph is input and n pieces are recommended. In other words, the n-best correct answer rate represents a ratio in which the lower garment (or upper garment) of the test photo is included when the upper garment (or lower garment) of the test photo is input and n pieces are recommended. Here, the similarity indicates how much the recommended clothes are similar to the correct answer when the upper garment (lower garment) of the test photograph is input and one sheet is recommended. The cosine similarity was used as the similarity.

  The results of comparing the method based on similarity (Sim), the method based on topic model (Topic), and the random presentation method (Random) are shown in FIGS. The result of “Topic” in FIG. 10 is the result of the model when the number of topics is 20. The correct answer rate shown in FIG. 10 is, for example, if the recommended 10 images are included in one set and one correct answer is included in the set (test photo underwear AL), the set is determined to be correct. Calculated. For example, if the correct answer rate is “0.5”, it indicates that the correct answer is determined for 50 sets in 100 sets of experiments.

  Both Sim and Topic show higher performance than Random, and it can be said that both the coordinate recommendation devices according to the first and second embodiments are effective. Above all, Topic is higher than Sim. This is considered because the coordination recommendation device according to the second embodiment can appropriately learn the relationship between the upper and lower garments by extracting the topics. When I looked at the actually recommended clothes, the photos related to the entered clothes were recommended appropriately.

1,1B Coordinate recommendation device 2 Magazine whole body photo feature extraction unit (whole body photo feature extraction unit for reference)
DESCRIPTION OF SYMBOLS 21 Photo reading part 22 Whole body photograph specific part 23 Area | region specific part 24 Feature extraction part 3 Owned photograph feature extraction part (photo feature extraction part for recommendation)
31 Photo reading unit 32 Whole body photo specifying unit 33 Area specifying unit 34 Feature extracting unit 4, 4B Recommending unit 41 Similarity calculating unit 42, 42B Presenting unit 43 Model learning unit 44 Topic estimating unit 5 Output unit 6 Storage unit 61 Magazine photo set (Photo set for reference)
62 Owned Photo Set (Recommended Photo Set)
63 Magazine whole body photo feature set (reference whole body photo feature set)
64 Owned Photo Feature Set (Recommended Photo Feature Set)
7 Input section

Claims (9)

A coordination recommendation device that recommends different types of items suitable for combination with input items for coordination of multiple types of fashion items,
A storage unit for storing a reference photo set predetermined as an exemplary coordinate image set and a recommended photo set set as an image set of recommended items;
A whole body photograph showing an image in which a plurality of types of items are combined is extracted from the reference photograph set, an area of each item is identified in the extracted whole body photograph, and an image of the image is extracted from the identified item area. Each of the feature quantities is extracted, and the feature quantity extracted for each item is stored in the storage section as a reference whole body photograph feature set; and
From the recommended photo set, a whole body photo showing an image in which a plurality of types of items are combined and a single item photo showing an image consisting of only a single item are extracted, and each item in the extracted whole body photo is extracted. And extracting the feature amount of the image from the specified item region, extracting the feature amount of the image from the single item photo, and extracting the feature image and the single item photo for each item. A recommended photo feature extracting unit that stores the quantity in the storage unit as a recommended photo feature set;
Using the reference whole body photo feature set and the recommendation photo feature set, the relationship between the areas of each item is learned, the relation between the areas, and the feature amount of the image of the input item In response, a recommendation unit that searches the recommended photo set for another type of item suitable for combination with the input item and presents it as an item to be recommended,
A coordination recommendation device comprising: The recommendation unit includes:
Using the reference whole body photo feature set and the recommendation photo feature set, the similarity between each item in the whole body photo in the reference photo set and each item in the recommendation photo set is the type of item. A similarity calculation unit to calculate for each,
An item of the same kind as the input item in the image of the whole body photograph obtained by searching a whole body photograph including the item of the same kind as the input item and having a high similarity from the reference photo set. The item of another type combined with is extracted, and a predetermined number of items are searched from the recommendation photo set in descending order of similarity to the item of the other type, and the item obtained by the search is A presenting unit for presenting as recommended items;
The coordinate recommendation device according to claim 1, further comprising: The recommendation unit includes:
Using the reference whole body photo feature set, the relationship between the areas of each item in the whole body photo in the reference photo set is learned, and when a feature amount of a certain region is given, the feature amount is obtained. A model learning unit that learns a topic model that outputs, as a probability value, a proportion of items having potential topics inherent in the item;
Using the topic model for the feature quantity of the image of the input item, the input item topic ratio indicating the ratio of the input item having each topic is estimated and input in the recommended photograph feature set. Using the topic model for each feature amount related to an item of a different type from the item, a recommendation item topic ratio indicating a ratio of items stored in the recommendation photo set corresponding to the feature amount having each topic A topic estimation unit for estimating
As a different type item from the input item, a predetermined number of items are searched from the recommendation photo set in descending order of similarity between the input item topic ratio and the recommendation item topic ratio, and the search A presentation unit that presents the item obtained by the item to be recommended,
The coordinate recommendation device according to claim 1, further comprising: The types of items are upper and lower garments,
The reference whole body photo feature extraction unit is
For each image included in the reference photo set, a face area is extracted, and an area that is wider than the width of the extracted face area and larger than the height of the face area exists below the face area. A whole body photo identifying unit for extracting an image to be taken as the whole body photo;
Based on the size of the face area, a partial area having a predetermined width and height with respect to the face area existing in the lower part of the face area is defined as an upper clothing item area, and a lower part of the upper clothing item area. An area specifying unit that specifies a partial area having a predetermined width and height with respect to the face area existing in
The recommended photographic feature extraction unit includes:
For each image included in the recommended photo set, a face area is extracted, and an area that is wider than the width of the extracted face area and larger than the height of the face area exists below the face area. A whole body photo identifying unit for extracting an image to be taken as the whole body photo;
Based on the size of the face area, a partial area having a predetermined width and height with respect to the face area existing in the lower part of the face area is defined as an upper clothing item area, and a lower part of the upper clothing item area. And a region specifying unit that specifies a partial region having a predetermined width and height with respect to the face region existing in the item as an item region of a garment. Coordinate recommendation device according to item. A memory for storing a reference photo set predetermined as a model image set for coordination and a recommended photo set predetermined as an image set of recommended items for coordinating multiple types of fashion items A coordination recommendation method by a coordination recommendation device that recommends another type of item suitable for a combination with an input item.
The processor is
Receiving the input of the item;
A whole body photograph showing an image in which a plurality of types of items are combined is extracted from the reference photograph set, an area of each item is identified in the extracted whole body photograph, and an image of the image is extracted from the identified item area. Extracting each feature amount and storing the feature amount extracted for each item as a reference whole body photo feature set in a storage unit;
From the recommended photo set, a whole body photo showing an image in which a plurality of types of items are combined and a single item photo showing an image consisting of only a single item are extracted, and each item in the extracted whole body photo is extracted. And extracting the feature amount of the image from the specified item region, extracting the feature amount of the image from the single item photo, and extracting the feature image and the single item photo for each item. Storing the amount in the storage unit as a recommended photo feature set;
Using the reference whole body photo feature set and the recommendation photo feature set, the relationship between the areas of each item is learned, the relation between the areas, and the feature amount of the image of the input item And searching for another type of item suitable for combination with the input item from the recommendation photo set and presenting the item;
Coordination recommendation method characterized by performing including. The processor is
In presenting the item,
Using the reference whole body photo feature set and the recommendation photo feature set, the similarity between each item in the whole body photo in the reference photo set and each item in the recommendation photo set is the type of item. Each calculating step,
Searching the reference photo set for a whole body photo including an item of the same kind as the input item and having a high similarity;
In the whole body photo image obtained by the search, another type of item combined with the same type of item as the input item is extracted, and predetermined in descending order of similarity with the different type of item. Retrieving a number of items from the recommendation photo set;
Presenting the items obtained by the search as recommended items;
The coordinate recommendation method according to claim 5, wherein: The processor is
In presenting the item,
Using the reference whole body photo feature set, the relationship between the areas of each item in the whole body photo in the reference photo set is learned, and when a feature amount of a certain region is given, the feature amount is obtained. Learning a topic model that outputs, as a probability value, a percentage of items having potential topics inherent in the item;
Estimating an input item topic ratio indicating a ratio of the input item having each topic using the topic model for the feature amount of the image of the input item;
Using the topic model for each feature amount related to an item of a type different from the input item in the recommendation photo feature set, the item stored in the recommendation photo set corresponding to the feature amount is a topic. Estimating a recommendation item topic ratio indicating a ratio of having,
As a different type item from the input item, a predetermined number of items are searched from the recommendation photo set in descending order of similarity between the input item topic ratio and the recommendation item topic ratio, and the search Presenting the item obtained by the step as a recommended item;
The coordinate recommendation method according to claim 5, wherein: The types of items are upper and lower garments,
The processor is
In the step of storing the reference whole body photograph feature set in a storage unit,
For each image included in the reference photo set, a face area is extracted, and an area that is wider than the width of the extracted face area and larger than the height of the face area exists below the face area. Extract the image to be taken as the whole body photo,
Based on the size of the face area, a partial area having a predetermined width and height with respect to the face area existing in the lower part of the face area is defined as an upper clothing item area, and a lower part of the upper clothing item area. A partial area having a predetermined width and height with respect to the face area existing in
In the step of storing the recommended photograph feature set in a storage unit,
For each image included in the recommended photo set, a face area is extracted, and an area that is wider than the width of the extracted face area and larger than the height of the face area exists below the face area. Extract the image to be taken as the whole body photo,
Based on the size of the face area, a partial area having a predetermined width and height with respect to the face area existing in the lower part of the face area is defined as an upper clothing item area, and a lower part of the upper clothing item area. A partial area having a predetermined width and height with respect to the face area existing in
The coordinate recommendation method according to any one of claims 5 to 7, wherein:   A coordinate recommendation program for causing a computer to function as each means constituting the coordinate recommendation device according to any one of claims 1 to 4.

Images