JP2015228129A - Coordination recommendation device and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To recommend even a new clothing, and items such as an accessory.SOLUTION: A coordination recommendation device comprises: a model storage part for storing model information which indicates fidelity of a combination of a first feature amount of each of plural areas obtained by dividing a dress, and a first feature amount of at least one of other areas; a candidate item storage part for storing a second feature amount related to each of item candidates forming the dress; a designation acquisition part for acquiring designation information which indicates at least one item forming the dress; a feature amount extraction part for calculating a third feature amount of each of areas corresponding to the items indicated by the designation information; and a recommendation item selection part for selecting the candidate of the item having high fidelity with the item indicated by the designation information, among the candidates of the items, by referring to the third feature amount, candidate item storage part, and model storage part.

Description

  The present invention relates to a technique for recommending coordinates.

Conventionally, there is a system that records daily coordinates together with information such as weather and meeting people, and recommends coordinates by referring to the recorded information according to the weather forecast, schedule, meeting people, etc. Patent Document 1).
Further, when standing in front of a mirror, there is a system in which the coordinates of the day are photographed and recorded, and the coordinates are recommended from the past user's history (see Non-Patent Document 2).

Satoshi Hamada, 4 others, “Asa1-coordinator: Fashion coordination support using history information”, Human Interface Society, Human Interface Symposium 2009, September 2009 Ayaka Sato, 2 others, “Proposal and Evaluation of Fashion Coordination Support System suGATALOG Using Figure”, IPSJ Symposium Series, ISSN1344-0640, 2011.3.10-12, Vol. 2011, No. 3, pp.41 -48

  However, the conventional system has a problem that it is not possible to recommend items such as clothes and accessories that have not been used before.

  In view of the above circumstances, an object of the present invention is to provide a technique that can recommend new items such as clothes and accessories.

  One aspect of the present invention is a model storage unit that stores model information indicating a degree of matching of a first feature amount of each of a plurality of regions obtained by dividing clothes with a first feature amount of at least one other region; A candidate item storage unit that stores a second feature amount relating to each item candidate that constitutes clothing, a designation acquisition unit that obtains designation information indicating at least one item that constitutes clothing, and the plurality of regions, With reference to the feature quantity extraction unit that calculates the third feature quantity of each region corresponding to the item indicated by the designation information, the third feature quantity, the candidate item storage unit, and the model storage unit, It is a coordination recommendation apparatus which has a recommendation item selection part which selects the candidate of an item with a high matching degree with the item which the said designation | designated information shows from the item candidate.

  One aspect of the present invention is the coordination recommendation device described above, wherein the first feature value, the second feature value, and the third feature value are feature values related to a color or a shape of an item. To do.

  One aspect of the present invention is the above-described coordinate recommendation device, wherein a model feature amount extraction unit that calculates the first feature amount for each of the plurality of regions from each of a plurality of learning images, and the model feature amount extraction A model generation unit configured to generate the model information with reference to the first feature amount calculated by the unit and store the model information in the model storage unit.

  One aspect of the present invention is the coordinate recommendation device described above, wherein the model information includes at least one other region of the feature amount of each of the plurality of regions obtained by dividing the clothing based on a reference different from the plurality of regions. It includes information indicating the matching degree of the combination with the feature amount.

  One aspect of the present invention is a program for causing a computer to function as any of the above coordinate recommendation devices.

  According to the present invention, it is possible to recommend new items such as clothes and accessories.

It is a figure which shows the system configuration | structure of the coordination recommendation system in 1st Embodiment of this invention. It is a schematic block diagram showing the functional structure of the terminal device 10 and the coordination recommendation apparatus 20 in the embodiment. It is a figure which shows the specific example which divided | segmented the coordinate image in the embodiment into parts area. It is a table which shows the example of the model information which model DB206 in the embodiment memorizes. It is a table which shows the example of the memory content of candidate item DB208 in the embodiment. It is a flowchart explaining the production | generation process of model DB206 in the embodiment. It is a flowchart explaining the recommendation process of coordination in the same embodiment. It is a schematic block diagram showing the function structure of the terminal device 10 in the 2nd Embodiment of this invention, and the coordination recommendation apparatus 20a. It is a table which shows the example of storage contents of model DB206a in the embodiment. It is a table which shows the example of the memory content of candidate item DB208a in the embodiment.

[First Embodiment]
DESCRIPTION OF EXEMPLARY EMBODIMENTS Hereinafter, a first embodiment of the invention will be described with reference to the drawings.
FIG. 1 is a diagram showing a system configuration of a coordination recommendation system in the present embodiment. The coordinate recommendation system of the present embodiment includes a plurality of terminal devices 10 (10-1 to 10-N: N is an integer equal to or greater than 1), a coordinate recommendation device 20, a network 30, and a shared server 40.
The coordination recommendation system recommends the coordination of clothes to the user. First, the wording used in the following description is defined.
The coordinate image is a learning image for generating a coordinate model.
An item is an element constituting clothing such as clothes, shoes, and accessories.
The possessed item is an item possessed by the user.
The possessed item image is an image of the retained item.
First, a user who wants to request a recommendation of coordination operates the terminal device 10 and transmits to the coordination recommendation device 20 an input image (designation information) of a base coordination and the type of item desired to be recommended. The coordinate recommendation device 20 refers to the coordinate model generated from the coordinate image acquired from the network 30 or from the shared server 40, and is combined with the coordinate as a base represented by the received input image. Is displayed on the terminal device 10.

  The terminal device 10 is configured using an information processing device such as a smartphone, a mobile phone, a PDA (Personal Digital Assistant), a mobile game device, a tablet device, or a personal computer. The terminal device 10 generates recommendation request information from the input image input by the user and the type of item desired to be recommended, and transmits the recommendation request information to the coordination recommendation device 20 via the network 30.

The coordination recommendation device 20 is configured using an information processing device. The coordination recommendation device 20 selects an item to be recommended from the possessed items according to the recommendation request information transmitted from the terminal device 10. Also, the coordination recommendation device 20 transmits information indicating the selected item to the terminal device 10.
The network 30 may be a network configured in any way. For example, the network 30 may be configured using the Internet.

  The shared server 40 is configured using an information processing apparatus. The shared server 40 stores the stored item image transmitted from the terminal device 10 in association with the user ID assigned to the user of the terminal device 10. In the following description, an image stored in the shared server 40 is referred to as “stored image”. The shared server 40 may be constructed as a server that provides an image sharing site, for example.

  FIG. 2 is a schematic block diagram showing functional configurations of the terminal device 10 and the coordination recommendation device 20 in the present embodiment. First, the functional configuration of the terminal device 10 will be described. The terminal device 10 includes a CPU (Central Processing Unit), a memory, an auxiliary storage device, and the like connected by a bus, and executes a search request program. By executing the search request program, the terminal device 10 functions as a device including the input unit 101, the request unit 102, the communication unit 103, and the display unit 104. Note that all or a part of each function of the terminal device 10 may be realized by using hardware such as an application specific integrated circuit (ASIC), a programmable logic device (PLD), or a field programmable gate array (FPGA). The search request program may be recorded on a computer-readable recording medium. The computer-readable recording medium is, for example, a portable medium such as a flexible disk, a magneto-optical disk, a ROM, a CD-ROM, or a storage device such as a hard disk built in the computer system. Further, the search request program may be transmitted / received via a telecommunication line.

  The input unit 101 is configured using an existing input device such as a keyboard, a pointing device (mouse, tablet, etc.), a button, a touch panel, a digital camera, and a memory card reader. The input unit 101 is operated by the user when inputting a user's recommendation request instruction to the terminal device 10. The recommendation request instruction includes an input image representing a base coordinate and designation of an item type for recommending a combination to the coordinate. For example, an input image is input by being captured by a digital camera, and designation of an item type is input by operating a keyboard.

The input unit 101 may be an interface for connecting the input device to the terminal device 10. In this case, the input unit 101 inputs an input signal generated according to a user input in the input device to the terminal device 10.
The request unit 102 generates recommendation request information based on a recommendation request instruction input to the input unit 101.

  The communication unit 103 communicates with the coordination recommendation device 20. For example, the communication unit 103 transmits the recommendation request information generated by the request unit 102 to the coordination recommendation device 20. In addition, the communication unit 103 receives, for example, information indicating the recommended item transmitted from the coordination recommendation device 20. The information indicating the item received by the communication unit 103 is, for example, a possessed item image or a URL (Uniform Resource Locator) to the possessed item image. Note that when the information indicating the item is the URL to the possessed item image, the communication unit 103 acquires the possessed item image indicated by the URL via the network 30.

  The display unit 104 is an image display device such as a CRT (Cathode Ray Tube) display, a liquid crystal display, or an organic EL (Electro Luminescence) display. The display unit 104 displays, for example, a possessed item image. The display unit 104 may be an interface for connecting the image display device to the terminal device 10. In this case, the display unit 104 generates a video signal for displaying the possessed item image, and outputs the video signal to an image display device connected to the display unit 104.

  Next, the functional configuration of the coordination recommendation device 20 will be described. The coordination recommendation device 20 includes a CPU, a memory, an auxiliary storage device, and the like connected by a bus, and executes a distribution program. By executing the distribution program, the coordination recommendation device 20 includes a communication unit 201, a control unit 202, an image acquisition unit 203, a model feature amount extraction unit 204, a model generation unit 205, a model DB (database) 206, and a retained feature amount extraction unit 207. , Function as an apparatus including a candidate item DB 208, a recommended item selection unit 209, a distribution unit 210, and an input feature amount extraction unit 211. Note that all or part of the functions of the coordination recommendation device 20 may be realized by using hardware such as ASIC, PLD, or FPGA. The distribution program may be recorded on a computer-readable recording medium. The computer-readable recording medium is, for example, a portable medium such as a flexible disk, a magneto-optical disk, a ROM, a CD-ROM, or a storage device such as a hard disk built in the computer system. Further, the distribution program may be transmitted / received via a telecommunication line.

The communication unit 201 communicates with the terminal device 10 and the shared server 40. Each unit of the coordination recommendation device 20 communicates with other devices such as the terminal device 10 and the shared server 40 via the communication unit 201. For example, the communication unit 201 receives recommendation request information transmitted from the terminal device 10. Moreover, the communication part 201 transmits the information which shows the item according to the received recommendation request information to the terminal device 10, for example.
The control unit 202 controls each functional unit of the coordination recommendation device 20. For example, the control unit 202 (designation acquisition unit) inputs the recommendation request information received by the communication unit 201 to the input feature amount extraction unit 211, and inputs the coordinated image received by the communication unit 201 to the image acquisition unit 203. The possessed item image received by the communication unit 201 is input to the retained feature amount extraction unit 207.

The image acquisition unit 203 acquires a coordinated image included in the accumulated image stored in the shared server 40 and a coordinated image from the network 30.
The model feature quantity extraction unit 204 extracts the feature quantities (first feature quantities) of the plurality of regions obtained by dividing the clothing from each coordinate image acquired by the image acquisition unit 203. Details of this feature amount extraction will be described later.

The model generation unit 205 generates model information based on the feature amount of the coordinated image extracted by the model feature amount extraction unit 204. The model information is information indicating the degree of suitability of the combination of the feature amount of each of the plurality of regions obtained by dividing the clothing with the feature amount of at least one other region. This fitness is the appearance probability in the coordinated image group acquired by the image acquisition unit 203. Details of the model information and its generation method will be described later.
The model DB 206 is configured using a storage device such as a magnetic hard disk device or a semiconductor storage device. The model DB 206 (model storage unit) stores model information generated by the model generation unit 205.

  The possessed feature amount extraction unit 207 extracts a feature amount (second feature amount) from the retained item image included in the accumulated image stored in the shared server 40. The possessed feature amount extraction unit 207 stores the extracted feature amount in the candidate item DB 208 in association with the URL to the retained item image, the item identification information represented by the retained item image, and the item type represented by the retained item image. . Note that the item identification information and the item type may be stored as tag information in a file of the possessed item image, for example, or may be generated by the retained feature amount extraction unit 207 by image recognition or the like. Good.

The candidate item DB 208 is configured using a storage device such as a magnetic hard disk device or a semiconductor storage device. The candidate item DB 208 (candidate item storage unit) stores the item identification information, URL, item type, and feature amount of the possessed item image in association with each other.
The input feature quantity extraction unit 211 extracts a feature quantity (third feature quantity) from the input image included in the recommendation request information in the same manner as the model feature quantity extraction unit 204.

The recommended item selection unit 209 refers to the feature amount of the input image extracted by the input feature amount extraction unit 211, the candidate item DB 208, and the model DB 206, and selects a possessed item having a high degree of matching with the coordination of the input image. To do.
The distribution unit 210 transmits information indicating the possessed item selected by the recommended item selection unit 209 to the terminal device 10. The distribution unit 210 may transmit the URL of the possessed item to the terminal device 10 as information indicating the retained item, for example.

Next, a method for generating the model DB 206 will be described.
The image acquisition unit 203 acquires a coordinated image designated by the user as a coordinated image to be used as a model from the network 30 or the shared server 40. On the web site on the network 30 (for example, the homepage of a fashion store, etc.), many images of coordinate examples in which products are combined in order for fashion magazines and fashion stores to advertise products are posted. In addition, on the network 30, there are many images of coordinate examples published in fashion brand collections. In addition, ordinary people may post examples of daily coordination on blogs. Therefore, the image acquisition unit 203 can acquire many coordinated images from the network 30. The coordinated image may be collected manually. The coordinated image is preferably a coordinate example including an item to be recommended.

  Next, the model feature amount extraction unit 204 extracts feature amounts from a plurality of coordinate images. Hereinafter, a method for extracting feature amounts will be described in detail. The method for extracting the feature amount includes three steps: a step of extracting a person region from the coordinated image, a step of dividing the extracted person region into finer part regions, and a step of extracting the feature amount for each part region. Executed. Hereinafter, processing of each step will be described. The person area is an area where clothes or a person wearing clothes is imaged. The parts area is an area where each part such as a person's torso, arms, and legs is imaged.

First, processing for extracting a person area from a coordinated image will be described.
FIG. 3 is a diagram illustrating a specific example in which the coordinated image is divided into parts regions. First, the model feature quantity extraction unit 204 extracts a rectangular area surrounding the person area from the coordinated image by using the technique of Reference 1 for the coordinated image. Reference 1 creates a discriminator in which a learning image of a person area and a learning image other than a person area are learned in advance, and selects a feature quantity effective for discrimination from an feature quantity such as an edge by Adaboost or the like. It is a technique. (Reference 1: Science Technique 2012.11.17 "Survey Papers" Human Detection by Statistical Learning Method). In addition, as a specific example of Adaboost, for example, techniques disclosed in the following documents may be applied. IPSJ Research Report 2009 “Efficient reduction of HOG features using real Adaboost for human detection”.

The model feature amount extraction unit 204 thus generates a rectangular area image by extracting a rectangular area surrounding the person area from the coordinated image. Next, the model feature quantity extraction unit 204 separates the foreground area and the background area of the rectangular area image using Graph cut. Thereafter, the model feature quantity extraction unit 204 generates a person area image by extracting the separated foreground area as a person area. The foreground area is an area of an object that is imaged in the foreground in the rectangular area image. A specific example of the foreground area is a person or an area of clothes worn by a person. The background area is an area other than the foreground area in the rectangular area image. Specific examples of the background area include areas such as walls and ceilings. The model feature quantity extraction unit 204 generates a person area image by extracting a person area from the rectangular area image in this way. As a specific example of Graph cut, for example, techniques disclosed in the following documents may be applied. Information Processing Society of Japan Research Report 2007-CVIM-158 “Graph Cut” Hiroshi Ishikawa.
Above, the description about the process which extracts a person area | region from a rectangular area image is complete | finished.

Next, a process for dividing a person area image into parts areas will be described. For example, the technique disclosed in the following document is used as a method of dividing the part area. Mun Wai Lee, Ram Nevatia, “Body Part Detection for Human Pose Estimation and Tracking”, IEEE Workshop on Motion and Video Computing (WMVC'07).
The part region is, for example, the region of the head PA1, the neck PA2, the right arm PA3, the left arm PA4, the torso PA5, the knee upper PA6, the knee lower PA7, and the shoe PA8 shown in FIG.
Above, description about the process which divides | segments a person area image into parts area is complete | finished.

Then, the feature quantity extraction unit 204 extracts a feature quantity for each divided part area.
As the feature quantity to be extracted, a feature quantity that captures fashion-related features such as color, shape, and pattern is preferable. For example, a color histogram may be used as a method for expressing color. Note that the feature amount extraction unit 204 may cluster the color histogram by the k-means method or the like, and use the centroid vector of the cluster as the feature amount. The k-means method is described in the following documents, for example. Yagi Yasushi and Saito Hideo, “Computer Vision Advanced Guide 2”, Adcom Media Co., Ltd., pp65, published in June 2010. Further, as a method for expressing the shape, HOG (Histgrams of Oriented Gradients) feature values may be used. Further, as a method of expressing a pattern, the number of detected straight lines may be used as a pattern feature amount by performing straight line detection. Moreover, you may use what combined these feature-values.
In the following description, it is assumed that the centroid vector of the color histogram is a feature amount.

  The feature amount extraction unit 204 extracts the feature amount from the coordinated image by executing the above-described three steps. The degree of deformation of clothes varies greatly depending on the images taken. The degree of deformation represents the height of the degree of deformation. The higher the degree of deformation, the more the shape is deformed from the original shape. For example, there is a large change in the shape of clothes between an image taken with the elbow extended and an image taken with the elbow bent. As a result, the degree of deformation increases. When the degree of deformation is large, even the same clothes may be determined as different clothes. In particular, the arm has a large movement, and as described above, the degree of deformation of the clothes increases depending on the state of the elbow and the like. Therefore, by dividing the parts into the above-described parts regions, it is possible to select and use the feature amount of the region that is less susceptible to the deformation of the clothes than to extract the feature amount from the coordinated image itself. For this reason, it is possible to reduce erroneous determination due to the degree of deformation.

  The model generation unit 205 constructs model information indicating the suitability of the combination of feature quantities in the part area. The model generation unit 205 regards combinations that appear in more coordinated images as having a high degree of matching, and generates model information based on N-grams. The N-gram is a model based on the assumption that the appearance probability of a certain feature value is affected only by the feature values of the immediately preceding (N-1) regions. N-grams are disclosed in, for example, the following documents. “Voice recognition system made with free software”, Morikita Publishing Co., Ltd., pp164-170.

  For example, if N is 3-gram, the head PA1, neck PA2, right arm PA3, left arm PA4, torso PA5, knee upper PA6, knee lower PA7, and shoe PA8 are divided into three parts. Assume that the probability of appearance is affected. At this time, the alignment may be from the head to the shoes, but in the case of fashion, it is considered that there are many choices from the tops, so the alignment is from the torso to the head and from the torso to the shoes. It is desirable to do. Hereinafter, in this embodiment, the alignment is from the torso to the head and from the torso to the shoes.

  Therefore, in this embodiment, the appearance probability of a certain feature value of the head PA1 is affected by the feature values of the neck PA2 and the right arm PA3, and the appearance probability of the feature value of the neck PA2 is the feature value of the right arm PA3 and the left arm PA4. Model information is generated assuming that Specifically, the model generation unit 205 includes, for each combination of values of the feature amount of the neck PA2 and the feature amount of the right arm PA3, for the feature amounts extracted by the model feature amount extraction unit 204 from a plurality of coordinate images. The appearance probability of a certain feature amount of the head PA1 is counted.

FIG. 4 is a table showing an example of model information generated by the model generation unit 205 and stored in the model DB 206. In the example of FIG. 4, the feature amount of the right arm PA3, the feature amount of the neck PA2, the feature amount of the head PA1, and the appearance probability thereof are associated with each other. For example, when the value of the right arm feature amount that is the feature amount of the right arm PA3 is “RC1” and the value of the neck feature amount that is the feature amount of the neck PA2 is “NC1”, the head that is the feature amount of the head PA1 The appearance probability of the feature value “HC1” is “Ph1”, and the appearance probability of the head feature value “HC2” is “Ph2”. In FIG. 4, only the table related to the appearance probability of the head feature amount is shown, but the model generation unit 205 also counts the appearance probability of other feature amounts to generate model information.
Above, description about the production | generation method of model DB206 is complete | finished.

Next, a method for generating the candidate item DB 208 will be described.
The control unit 202 acquires the possessed item image designated by the user from the network 30 or the shared server 40. This designation is performed from the terminal device 10, for example. The control unit 202 inputs the acquired possessed item image to the retained feature amount extraction unit 207.

  Next, the retained feature amount extraction unit 207 extracts a feature amount from the retained item image. The retained feature amount extraction unit 207 extracts feature amounts from the retained item image in the same manner as the model feature amount extraction unit 204. However, the possessed feature amount extraction unit 207 extracts only the feature amount of the area corresponding to the type of the item from the possessed item image. For example, if the item type is a hat, only the feature amount of the head PA1 is extracted. If the item type is slacks or a long skirt, the feature amount of the knee upper PA6 and the feature amount of the knee lower PA7 are extracted. The possessed feature amount extraction unit 207 associates the extracted feature amount with information related to the possessed item and stores the associated feature amount in the candidate item DB 208.

FIG. 5 is a table showing an example of the contents stored in the candidate item DB 208. The information regarding the possessed item includes the item name, the URL to the possessed item image, and the item type. For example, the item name “hat H1” is associated with the URL “http:...”, The type “hat”, the region “head”, and the feature amount “HC1”. In the case of a hat, since the corresponding region is only “head”, only “HC1” is associated with the information related to the possessed item. On the other hand, for the item name “Slack S1”, the URL “http:. The quantity “LBC1” is associated.
Above, description about the production | generation method of candidate item DB208 is complete | finished.

  FIG. 6 is a flowchart for explaining the generation process of the model DB 206. First, the image acquisition unit 203 acquires a coordinated image group (Sa1). Next, the model feature amount extraction unit 204 divides the coordinated image into parts regions (Sa2). Next, the model feature quantity extraction unit 204 extracts the feature quantity of each part region (Sa3). Next, the model feature quantity extraction unit 204 determines whether or not feature quantity extraction has been completed for all images in the coordinated image group acquired in step Sa1 (Sa4). If it is determined that it has not been completed (Sa4-No), the model feature quantity extraction unit 204 returns to step Sa2 and performs processing on the coordinate image from which feature quantities have not been extracted.

  On the other hand, when it is determined in step Sa4 that the feature amount extraction has been completed for all the images (Sa4-Yes), the model generation unit 205 calculates an N-gram of each part region (Sa5). Next, the model generation unit 205 stores the calculated N-gram as model information in the model DB 206 (Sa6), and ends the process.

  FIG. 7 is a flowchart for explaining coordination recommendation processing. First, the control unit 202 acquires, as recommendation request information, a coordinate input image serving as a base and the type of item to recommend from the terminal device 10 via the communication unit 201 (Sb1). Next, the input feature amount extraction unit 211 divides the input image into parts regions (Sb2). Next, the input feature quantity extraction unit 211 extracts the feature quantity of each part region (Sb3). The processes in steps Sb2 and Sb3 are the same as the processes performed by the model feature amount extraction unit 204 on the coordinate image.

  Next, the recommended item selection unit 209 performs the process of step Sb5 for each possessed item of the recommended item type acquired in step Sb1 (Sb4, Sb6). In step Sb5, the recommended item selection unit 209 reads, from the model DB 206, the appearance probability of the model information corresponding to the feature amount of the part area that is closest to the feature amount of the target possessed item (Sb5). For example, in a loop in the loop of steps Sb4 and Sb6, the target possessed item is the hat H1 in FIG. 5, the right arm feature value extracted in step Sb3 is “RC1”, and the neck feature value is “NC1”. In some cases, the recommended item selection unit 209 determines, from the model DB 206, the appearance probability “Ph1” associated with the right arm feature “RC1”, the neck feature “NC1”, and the head feature “HC1” with the hat H1. "Is read out.

  Also, in a loop in the loop of steps Sb4 and Sb6, when the target possessed item is the slacks S1 in FIG. 5 and the trunk feature amount extracted in step Sb3 is “BC1”, the recommended item selection unit 209 reads out from the model DB 206 the appearance probability associated with the body feature amount “BC1”, the above-knee feature amount “LUC1” and the below-knee feature amount “LBC1” by the slacks S1.

  And when the loop by step Sb4 and Sb6 is complete | finished, the recommended item selection part 209 selects the possessed item with the largest appearance probability read as an item to recommend. The distribution unit 210 distributes the URL of the possessed item selected by the recommended item selection unit 209 as information indicating the recommended item to the terminal device 10 via the control unit 202 and the communication unit 201 (Sb7).

  According to the coordinate recommendation device 20 configured as described above, based on the model information indicating the fitness of the combination of the feature values of each of the plurality of regions obtained by dividing the clothing with the feature values of at least one other region. Recommend items that are highly compatible with the base coordinates. Therefore, when the user designates a base coordinate, an item suitable for the coordinate can be recommended even if the item is new clothes or accessories that the user has never worn.

[Second Embodiment]
Hereinafter, a second embodiment of the present invention will be described with reference to the drawings. The coordinate recommendation system in the present embodiment has the same configuration as the coordinate recommendation system in FIG. 1 except that a coordinate recommendation device 20a is provided instead of the coordinate recommendation device 20. The coordinate recommendation device 20 selects an item to be recommended using the feature amount of the part area, but the coordinate recommendation device 20a is a color model including a base color, an assorted color, and an accent color in addition to the feature amount of the part region. The recommended item is selected using the feature amount.

  FIG. 8 is a schematic block diagram showing functional configurations of the terminal device 10 and the coordination recommendation device 20a in the present embodiment. In the figure, parts corresponding to the respective parts in FIG. The coordination recommendation device 20a includes a CPU, a memory, an auxiliary storage device, and the like connected by a bus, and executes a distribution program. By executing the distribution program, the coordination recommendation device 20a includes a communication unit 201, a control unit 202, an image acquisition unit 203, a model feature amount extraction unit 204a, a model generation unit 205a, a model DB 206a, a retained feature amount extraction unit 207a, and a candidate item DB 208a. , Function as an apparatus including a recommended item selection unit 209a, a distribution unit 210, and an input feature amount extraction unit 211a. Note that all or part of the functions of the coordination recommendation device 20a may be realized using hardware such as an ASIC, PLD, or FPGA. The distribution program may be recorded on a computer-readable recording medium. The computer-readable recording medium is, for example, a portable medium such as a flexible disk, a magneto-optical disk, a ROM, a CD-ROM, or a storage device such as a hard disk built in the computer system. Further, the distribution program may be transmitted / received via a telecommunication line.

  The model feature quantity extraction unit 204a extracts the feature quantity of the part area of the coordinated image in the same manner as the model feature quantity extraction unit 204 of FIG. 2, but further, the feature quantity of the base color, the assorted color, and the accent color of the coordinated image (First feature amount) is extracted. This feature amount is a feature amount based on a color histogram. For example, this feature amount is a centroid vector of a cluster obtained by clustering the color histogram of the corresponding region by the k-means method or the like, similar to the feature amount of the part region.

  The base color is a color that determines the overall appearance atmosphere of the clothes, and is a color that occupies about 70% of the whole. The base color is the color of clothes that occupy a large area, such as suits, dresses, and coats. The assorted color is a color that brings together the entire base color and occupies about 20% to 30% of the whole. The colors of blouse, vest, skirt, pants, shoes, bags, etc. are assorted colors. Accent color is a color that accounts for about 10% of the overall fashion. It is said that the accent color can be sharpened by selecting a color that contrasts with the base color or a conspicuous color. The colors of accessories, ties, scarves, etc. often become accent colors.

  The model feature amount extraction unit 204a extracts a base color feature amount, an assorted color feature amount, and an accent color feature amount from each coordinated image using the SaliencyMap. SaliencyMap represents the degree of attention of a person with respect to an image area, and is calculated by combining feature quantities such as edges and colors. The model feature quantity extraction unit 204a calculates the SalietyMap of the person area of the coordinated image. For example, the technology disclosed in the following document may be applied to the calculation of the SaliencyMap. L.itti, C.Koch, E.Niebur, "A Model of Saliency-Based Visual Attention for Rapid Scene Analysis", IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol.20, No.11, pp.1254-1259, Nov 1998.

  For example, the model feature amount extraction unit 204a sets a region having a Salientity value lower than the threshold value Th1 and a larger area than the threshold value Th2 when the total area is 1 as the base color region. Further, the model feature amount extraction unit 204a sets an area having an Saliency value higher than the threshold Th3 and an area smaller than the threshold Th4 as an accent color area. Further, a region where the Saliency value is higher than the threshold value Th1, lower than the threshold value Th3, the area is larger than the threshold value Th4, and smaller than the threshold value Th2 is set as an assorted color region. The model feature quantity extraction unit 204a extracts a base color feature quantity, an accent color feature quantity, and an assorted color feature quantity for each coordinated image.

  The model generation unit 205a generates model information related to the part area in the same manner as the model generation unit 205 in FIG. 2, but also generates model information related to the base color, assortment color, and accent color. Specifically, the model generation unit 205a determines the number of co-occurrence values of the base color feature value, the assorted color feature value, and the accent color feature value in the coordinated image group. Count and calculate the appearance probability of the combination. The model generation unit 205a stores the calculated appearance probability in the model DB 206a in association with the combination of feature value values.

  The model DB 206a stores model information related to the part area in the same manner as the model DB 206 in FIG. 2, but also stores model information related to the base color, assorted color, and accent color. FIG. 9 is a table showing an example of the contents stored in the model DB 206a. The example shown in FIG. 9 is model information regarding the base color, assorted color, and accent color. In the example of FIG. 9, the base color feature value “BC1”, the assorted color feature value “SC1”, the accent color feature value “AC1”, and the appearance probability “PC1” are associated with each other. ing. Similarly, the base color feature value “BC1”, the assorted color feature value “SC1”, the accent color feature value “AC2”, and the appearance probability “PC2” are associated with each other.

  The possessed feature quantity extraction unit 207a extracts feature quantities related to the part area of the possessed item in the same manner as the possessed feature quantity extraction unit 207 of FIG. 2, and further, a corresponding one of the base color, assort color, and accent color The feature amount related to (second feature amount) is also extracted. In addition, which color classification corresponds to the possessed item among the base color, the assortment color, and the accent color may be determined in advance according to the type of the item, for example. The possessed feature amount extraction unit 207a stores the extracted feature amount in the candidate item DB 208a.

  Similar to the candidate item DB 208 in FIG. 2, the candidate item DB 208 a stores the feature amount relating to the part area of the possessed item and the information relating to the possessed item, and further includes color classification such as base color, assorted color, and accent color. The feature amount is stored. FIG. 10 is a table showing an example of stored contents of the candidate item DB 208a. In the example illustrated in FIG. 10, the candidate item DB 208 a stores the item name, the color classification, and the feature amount in association with each other. For example, the item name “hat H1” stores the color classification “accent color” and the feature amount “AC3” in association with each other. In addition, the item name “jacket J1” corresponding to a plurality of color classifications is associated with the color classification “base color” and the feature quantity “BC2”, the color classification “accent color”, and the feature quantity “AC5”.

The input feature amount extraction unit 211a extracts the feature amount of the part area and the color classification feature amount (third feature amount) from the input image in the same manner as the model feature amount extraction unit 204a.
The recommended item selection unit 209a reads the appearance probability of each possessed item of the type specified by the recommendation request information from the model DB 206a as the first appearance probability in the same manner as the recommended item selection unit 209 in FIG. Further, the recommended item selection unit 209a also determines the appearance probability of each possessed item of the type specified in the recommendation request information for the appearance probability related to the feature amount of the color classification in the same manner as in the part area. Read out from the model DB 206a as a probability. The recommended item selection unit 209a calculates the sum of the first appearance probability and the second appearance probability for each possessed item and sets the sum as the evaluation value of the possessed item. The recommended item selection unit 209a selects the possessed item having the largest evaluation value as the recommended item. The evaluation value may be a product of the first appearance probability and the second appearance probability.

  According to the coordinate recommendation device 20a configured as described above, the fitness of the combination of the feature quantities of each of the plurality of areas obtained by dividing the clothing with a reference different from the part area, with the feature quantities of at least one other area. In addition to the model information by the parts area, the model information indicating is referred to and an item having a high degree of fitness with the base coordinate is recommended. Therefore, it is possible to recommend an item that is surely matched by the coordination as compared with the case where only the model information by the part area is referred to.

[Modification]
In each of the above-described embodiments, the image acquisition unit 203 may classify the acquired coordinated images for each category, and the model generation unit 205 may generate a model for each category. In this case, the user also specifies a category when requesting item recommendation, and the recommended item selection unit 209 refers only to a model of the category specified by the user when referring to the model DB 206.

  As a method of classifying by category, for example, when a tag is attached to each category on the site where the images are collected, the image acquisition unit 203 classifies the acquired coordinated image into a suitable category from each category. In addition, the image acquisition unit 203 may classify the coordinated image for each site from which the coordinated image is acquired, or may classify the image according to the category in accordance with TPO (Time Place Occasion) such as season, weather, and destination. . Further, the category may be classified into a category other than the above-described categories. For example, the category may be a category for each season or usage such as “spring, warm, entrance ceremony”, “spring, chilly, graduation ceremony”, “30s, short stature, feminine”, “40s” , Large, formal, etc. The classification of categories may be performed manually.

  Moreover, although each above-mentioned embodiment demonstrated the possession item as the item which the user possesses, it is not restricted to this. For example, the possessed item is an item possessed by the store, and the coordination recommendation devices 20 and 20a may recommend the item retained by the store to the user for sales promotion.

In each of the above-described embodiments, the coordination recommendation devices 20 and 20a include the image acquisition unit 203, the model feature amount extraction unit 204, and the model generation unit 205. However, the present invention is not limited to this. For example, these may be stored in another device, and the model DB 206 may store model information generated by the other device.
In each of the above-described embodiments, the input image included in the recommendation request information is a coordinated image. However, the input image may not be a coordinated image including all items. For example, if the model is 2-gram, the number of items included in the input image may be one. The input image may include an item of the same type as the recommended item.

In each of the above-described embodiments, N-grams are used as the model information related to the part region. However, the co-occurrence probability P (c ₁ ... C _Q ) represented by Expression (1) may be used. . Note that c _i is a feature amount of the i-th part area, and Q is the number of part areas.
Further, the value of N in the N-gram may be different depending on the part region.

Further, in each of the above-described embodiments, the parts area is an area in which each part such as a person's torso, arms, and legs is imaged, but is not limited thereto. For example, the part area may be an area for each piece of clothing or accessory.
In each of the above-described embodiments, the recommended item selection units 209 and 209a recommend only the possessed item having the highest evaluation value such as the appearance probability, but not only the largest item but also the third item from the largest. A plurality of possessed items may be recommended, for example, items having an evaluation value equal to or greater than a predetermined threshold.
Further, in each of the above-described embodiments, when a plurality of feature amounts (for example, a color histogram and HOG) are calculated as the feature amounts of the part area, in step Sb7 in FIG. The possessed item having the largest vector size may be recommended.

Furthermore, in the above-described second embodiment, when a plurality of feature amounts are calculated as the feature amounts of the part region, each feature amount is calculated with respect to a vector having the first appearance probability of each feature amount as an element. The magnitude of the vector obtained by modifying the vector having the first appearance probability as an element is evaluated by applying the vector having the second appearance probability as an element using the method of ManifoldRanking described in the following document. You may make it use as. “Manifold-Ranking Based Image Retrieval”, MM'04, October 10-16, 2004, NEW YORK. ManifoldRanking is represented by the formula (2). f * is a corrected vector, y is a vector having the first appearance probability of each feature amount as an element, S is a co-occurrence probability matrix regarding the color classification and the part region, and expressed. α is a parameter having a value of [0, 1). In Expression (3), P () is a function defined by the number (1), c _i is a feature amount of the part area corresponding to the item, and c _j is a color corresponding to the item. This is a feature amount of classification. When N feature values are calculated for each of the part region and the color classification, the co-occurrence probability matrix is an N × N square matrix.

  The embodiment of the present invention has been described in detail with reference to the drawings. However, the specific configuration is not limited to this embodiment, and includes designs and the like that do not depart from the gist of the present invention.

DESCRIPTION OF SYMBOLS 10 ... Terminal device 20, 20a ... Coordination recommendation device, 30 ... Network, 40 ... Shared server, 101 ... Input part, 102 ... Request part, 103 ... Communication part, 104 ... Display part, 201 ... Communication part, 202 ... Control 203, image acquisition unit, 204, 204a ... model feature amount extraction unit, 205, 205a ... model generation unit, 206, 206a ... model DB, 207, 207a ... possessed feature amount extraction unit, 208, 208a ... candidate item DB , 209, 209a ... recommended item selection unit, 210 ... distribution unit, 211, 211a ... input feature amount extraction unit

Claims (5)

A model storage unit that stores model information indicating a degree of suitability of a combination of a first feature amount of each of a plurality of regions into which clothes are divided and a first feature amount of at least one other region;
A candidate item storage unit that stores a second feature amount relating to each of the item candidates constituting the clothes;
A designation acquisition unit for acquiring designation information indicating at least one item constituting the clothes;
A feature amount extraction unit that calculates a third feature amount of each region corresponding to the item indicated by the designation information among the plurality of regions;
With reference to the third feature value, the candidate item storage unit, and the model storage unit, an item candidate having a high degree of matching with the item indicated by the designation information is selected from the item candidates. A coordinate recommendation device comprising: a recommended item selection unit.   The coordinate recommendation device according to claim 1, wherein the first feature value, the second feature value, and the third feature value are feature values related to a color or a shape of an item. A model feature amount extraction unit for calculating the first feature amount for each of the plurality of regions from a plurality of learning images;
The model generation unit that generates the model information with reference to the first feature amount calculated by the model feature amount extraction unit and stores the model information in the model storage unit. Coordinate recommendation device described in 1.   The model information includes information indicating a matching degree of a combination of a feature amount of each of a plurality of regions obtained by dividing clothes with a different standard from the plurality of regions and a feature amount of at least one other region. The coordinate recommendation device according to claim 1.   The program for functioning a computer as a coordination recommendation apparatus as described in any one of Claims 1-4.

Images