KR102284148B1 - Method and System of recommending fashion based on vector based deep learning - Google Patents

Abstract

A vector-based fashion recommendation method using deep learning according to the present invention and a fashion recommendation system for performing the same are provided. Meanwhile, the method is performed by a server in a fashion recommendation system that performs the fashion recommendation method, and receives a list of fashion items possessed/preferred by a specific user and a list of fashion items corresponding to a specific weather or situation from a user terminal. List receiving step; a style vector extraction step of extracting a style vector through deep learning for a fashion product image for each item in the fashion item list; and a style vector set output step of outputting the style vector for each specific user and a style vector set corresponding to the specific weather or situation, thereby providing a vector similarity-based fashion recommendation method and system using deep learning.

Description

Vector-based fashion recommendation method and system using deep learning {Method and System of recommending fashion based on vector based deep learning}

The present invention relates to a fashion recommendation method and system. More specifically, it relates to a vector-based fashion recommendation method and system using deep learning.

Recently, due to the development of Internet technology and smart devices, online purchases of various products are being actively made. In particular, customers who want to purchase fashion clothes online access a website that sells fashion clothes, browse through the clothes provided by the website, select clothes that fit their taste, and pay for them. will go through

However, since the operator who operates a site that sells clothing updates the site with at least hundreds of new clothing items every month in addition to the existing clothing items, the range of clothing selection expands from the customer's point of view, and at the same time, the Sometimes it is difficult to choose one.

Therefore, in order to facilitate the customer's selection of fashion clothes, recently, apparel sales sites, etc., provide a service for recommending clothes for customers.

As a method of such a conventional clothing recommendation service, a user-based recommendation method and an item-based recommendation method are used. A user-based recommendation method and an item-based recommendation method are known in the method and the e-commerce recommendation analysis system and recommendation analysis method of Korean Patent Laid-Open No. 10-2014-111225, respectively.

i) The above user-based recommendation method can be said to be a method of recommending clothing selected by other customers with similar physical condition, preferred style, and taste to the corresponding customer, ii) The above item-based recommendation method is a method of recommending to the customer It is a method of recommending clothing with similar physical condition, preferred style and taste to the clothing selected by the customer on the site.

However, such conventional clothing recommendation methods have a disadvantage in that they lack objectivity because the measurement standard for similarity of clothing is not constant. Since the recommendation method can be applied only when additional information, such as the number of clicks, is given, there is a problem in that the management demand of clothing sellers increases.

On the other hand, recently, more and more sites are providing virtual experience services that allow users to match or experience various fashion clothing in advance. You can check to see if it is right for you, but the problem that it takes a lot of time and effort to find product information that suits you because you choose so many styles one by one could not be solved.

Meanwhile, in relation to the fashion recommendation method related to the present invention, a customer segmentation-based recommendation method and a keyword-based recommendation method may be considered. However, such a fashion recommendation method has a problem in that it is not possible to consider specific differences as it is treated uniformly within a segmented group. In addition, the keyword-based recommendation method has a problem in that it is inaccurate for recognizing the characteristics of a corresponding item only with basic format attributes (eg, brand, material, color) of each item.

Accordingly, an object of the present invention is to solve the above problems, and to provide a vector-based fashion recommendation method and system using deep learning.

Another object of the present invention is to provide a vector-based fashion recommendation method and system using deep learning by calculating style vectors for all concepts related to fashion.

To solve the above problems, a vector-based fashion recommendation method using deep learning according to the present invention and a fashion recommendation system for performing the same are provided. Meanwhile, the method is performed by a server in a fashion recommendation system that performs the fashion recommendation method, and receives a list of fashion items possessed/preferred by a specific user and a list of fashion items corresponding to a specific weather or situation from a user terminal. List receiving step; a style vector extraction step of extracting a style vector through deep learning for a fashion product image for each item in the fashion item list; and a style vector set output step of outputting the style vector for each specific user and a style vector set corresponding to the specific weather or situation, thereby providing a vector similarity-based fashion recommendation method and system using deep learning.

According to an embodiment, the method may further include outputting a list of recommended items for outputting a list of one or more recommended items for each specific user based on the set of style vectors.

According to an embodiment, in the step of outputting the style vector set, the fashion items are divided into groups according to the type of the fashion item and the characteristics of the fashion item, and a representative vector is calculated for each group to output the style vector set. can do.

According to an embodiment, in the step of outputting the style vector set, the fashion items are divided into groups according to the type of the fashion item and the characteristics of the fashion item, and a representative vector is calculated for each group to output the style vector set. can do.

According to an embodiment, the type of the fashion item is classified into top, bottom, outer, shoe, and dress, the personality of the fashion item is classified into formal look, casual look, and office look, and in the step of outputting the style vector set A style vector set may be output by calculating a representative vector for each group according to the classification.

According to an embodiment, in the step of outputting the style vector set, a representative value according to an average, a maximum value, and a weighted average is obtained for each dimension with respect to the extracted style vectors, and the representative values for each dimension are combined to form the respective groups. A set of style vectors can be output by calculating a star representative vector.

According to an embodiment, the list of fashion items corresponding to the specific weather or situation is obtained by extracting a list of fashion items used, purchased, or viewed in a specific weather or situation from a usage log of an online fashion service system, and the style vector In the extraction step, the style vector set may be extracted based on the list of fashion items possessed/preferred by each specific user and the list of fashion items used, purchased, or read by the specific user in the specific weather or situation.

According to an embodiment, in the step of receiving the list of fashion items, in addition to the list of fashion items for each specific user and the list of fashion items corresponding to the specific weather or situation, a list of fashion items related to a specific concept by brand, region, and age group may be received. can Meanwhile, in the step of extracting the style vector, the style vector may be extracted for each brand, region, and age group.

According to an embodiment, after the step of extracting the style vector, a weighted sum of a first style vector extracted based on the user ID for each specific user and a second style vector extracted based on the specific weather or situation a query vector generation step of generating a query vector by synthesizing the method; and searching for k vectors having high vector similarity to the generated query vector from the item style DB.

According to an embodiment, after the step of extracting the style vector, a first style vector extracted based on the user ID for each specific user, a second style vector extracted based on the specific weather or situation, and the brand, region and age group a query vector generation step of generating a query vector by synthesizing the extracted third style vector in relation to a specific concept by a weighted sum method; and a vector retrieval step of retrieving k vectors having high vector similarity to the generated query vector from the item style DB.

According to an embodiment, in the vector search step, the extracted first and second style vectors and the vectors stored in the item style DB are input to an artificial neural network, and k recommendations are made in order from the item that calculates the highest score. A list of items can be selected.

According to an embodiment, in the vector search step, the extracted first to third style vectors and vectors stored in the item style DB are input to an artificial neural network, and k recommendations are made in order from the item that calculates the highest score. You can select a list of items.

According to at least one embodiment of the present invention, there is an advantage in that it is possible to provide a vector similarity-based fashion recommendation method and system using deep learning.

In addition, according to at least one embodiment of the present invention, it is possible to provide a vector-based fashion recommendation method and system using deep learning by calculating style vectors for all concepts related to fashion.

In addition, according to at least one embodiment of the present invention, there is an advantage in that various concepts related to fashion can be converted into a vector of a uniform form, and a recommendation result can be calculated through a mutual comparison operation between them.

1 is a block diagram of a vector-based fashion recommendation method and system using deep learning according to the present invention.
2 is an exemplary diagram illustrating mapping of a fashion item to a vector space according to an embodiment of the present invention.
3 shows a user style vector DB construction method according to the present invention.
4 shows a method of constructing a weather/situation style vector DB according to the present invention.
5 shows a method of constructing a style vector DB associated with a list of fashion items for a specific concept according to the present invention.
6 is a flowchart of a vector-based fashion item recommendation method according to the present invention.
7 is a flowchart illustrating a method of determining which item to recommend given weather/situation according to an embodiment of the present invention.
8 is a flowchart illustrating a method of determining which item to recommend to a specific user given weather/situation according to an embodiment of the present invention.
9 is a flowchart of a method of determining which item to recommend when a query concept is given according to the present invention.
10 is a diagram illustrating an extrinsic feature of a fashion item associated with a style vector in a vector-based fashion item recommendation method according to the present invention.
11 shows the intrinsic characteristics of a fashion item associated with a style vector in a vector-based fashion item recommendation method according to the present invention.
12 shows a fashion item type/personality classification example for a vector-based fashion item recommendation method according to the present invention.
13 shows an example of a vector space in which a style vector is disposed in a vector-based fashion item recommendation method according to the present invention.

The features and effects of the present invention described above will become more apparent through the following detailed description in relation to the accompanying drawings, and accordingly, those of ordinary skill in the art to which the present invention pertains can easily implement the technical idea of the present invention. will be able

Since the present invention can have various changes and can have various embodiments, specific embodiments are illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the present invention to a specific embodiment, it should be understood to include all modifications, equivalents and substitutes included in the spirit and scope of the present invention.

In describing each figure, like reference numerals are used for like elements.

Terms such as first, second, etc. may be used to describe various elements, but the elements should not be limited by the terms. The above terms are used only for the purpose of distinguishing one component from another.

For example, without departing from the scope of the present invention, a first component may be referred to as a second component, and similarly, a second component may also be referred to as a first component. and/or includes a combination of a plurality of related listed items or any of a plurality of related listed items.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

Terms such as those defined in a commonly used dictionary should be interpreted as having a meaning consistent with the meaning in the context of the related art, and should not be interpreted in an ideal or excessively formal meaning unless explicitly defined in the present application. shouldn't

The suffix module, block, and part for the components used in the following description are given or mixed in consideration of only the ease of writing the specification, and do not have distinct meanings or roles by themselves.

Hereinafter, a preferred embodiment of the present invention will be described with reference to the accompanying drawings so that a person skilled in the art can easily implement it. In the following description of embodiments of the present invention, if it is determined that a detailed description of a related known function or a known configuration may unnecessarily obscure the gist of the present invention, the detailed description thereof will be omitted.

Hereinafter, a vector-based fashion recommendation method and system using deep learning according to the present invention will be described.

In this regard, FIG. 1 is a block diagram of a vector-based fashion recommendation method and system using deep learning according to the present invention.

The vector-based fashion recommendation system using deep learning of the present invention is configured to include a server 500 interworking with a business item DB 100 and a reference DB 200 .

The business operator item DB 100 includes a query item vector calculation module 110 that extracts a type (part) and a vector of an input user's query item and transmits it to the reference DB 200, and the reference DB ( 200), a recommended item search module 120 that searches for and extracts recommended items having a vector having a high degree of similarity to a vector of reference items inputted from the reference item, and a recommended coordinator configuration that combines a user's query item and recommended items to form recommended coordinators It may be configured to include a module 130 and a recommended coordinates sorting module 140 that sorts the recommended coordinates and transmits them to the user in the sorted order.

Meanwhile, the reference DB 200 includes a similar item search module 210 for searching and extracting similar items having a vector having the highest degree of similarity with the vector of the query item, and searching for reference coordinators including the similar items. A reference coordination search module to extract 220, a reference item extraction module 230 for extracting reference items included in the reference coordinates, respectively, and a business operator item after calculating the type and vector for each of the reference items It may be configured to include a reference item vector calculation module 240 to be transmitted to the DB.

In addition, the reference DB 200 may further include a reference coordination quality evaluation module 250 that searches and extracts reference coordinates in consideration of the quality scores of the coordinates.

The user's query item may be an item that the user is interested in or has a crush on while searching for a fashion item, and selects a query item to inquire about the business item DB 100 from among a plurality of fashion items on the user terminal 300 , or It may be input through a display panel (not shown) connected to the business operator item DB.

In the present invention, the type (part) and vector of the query item), similar item, reference item, and recommendation item can be extracted and used. Here, the type of item (part) may be classified into top, bottom, coat, shoes, bag, and other accessories.

However, in the present invention, it is not limited to the classification of the above item types, and it is also possible to classify the types of items for men and women according to the needs of the invention, i) For men, shirts, T-shirts, suits, tops, and bottoms , cardigan, coat, shoes, bag, waistband, tie, tie pin, watch, hat, etc. ii) For women, blouse, T-shirt, formal top, bottom, one-piece, two-piece, cardigan, overcoat , shoes, bags, scarves, shawls, watches, necklaces, bracelets, hats, etc. The vector extraction and use of the item will be described later in detail.

The similar item preferably has the same type (part) as the query item. For example, if the query item relates to a shirt item among tops, the similar item may also be extracted as a shirt item among tops.

With the reference DB 200, the similar item 420 can extract similar items 420 having the highest similarity by comparing the vector with the vector of the query item 410, and having the highest similarity. In order, k similar items can be extracted.

The recommended item is an item having the same type as the reference item and having a high vector similarity. It may be that the item was extracted.

Since the recommended item is an item having the same type as the reference item, an item of a different type from the type of the query item will be extracted.

The recommended coordination arranging module 140 performs a function of arranging a plurality of recommended coordinations and displaying them to the user. In this case, the alignment criterion is a quality score of a reference coordination, a similarity value of a similar item, or a similarity value of a recommended item. Based on the result obtained by multiplying any two or more factors, it is sorted in descending order and transmitted to the user.

The business item DB 100 and the reference DB 200 may simultaneously perform a function of a database for storing data therein and a function of a server for controlling data flow including various functional modules.

Meanwhile, in the vector-based fashion recommendation method and system using deep learning according to the present invention, the method may be performed by a server performing the fashion recommendation method. In this regard, the server 500 performing a vector-based fashion recommendation method using deep learning may operate as follows.

The server 500 may receive a list of fashion items possessed/preferred by a specific user and a list of fashion items corresponding to a specific weather or situation from the user terminal 300 . Meanwhile, the server 500 may extract a style vector through deep learning for a fashion product image for each item in the fashion item list. Also, the server 500 may output the style vector for each specific user and a set of style vectors corresponding to the specific weather or situation. Accordingly, the server 500 may output a list of one or more recommended items for each specific user based on the set of style vectors.

Meanwhile, a detailed method for a vector-based fashion recommendation method using deep learning according to the present invention will be described in detail with reference to FIGS. 3 to 9 below.

Meanwhile, FIG. 2 is an exemplary diagram illustrating mapping of a fashion item to a vector space according to an embodiment of the present invention.

The business item DB 100 and the reference DB 200 of the present invention may store images and information of various fashion items, and may collect and store images and information of fashion items from domestic and foreign fashion shopping mall sites.

On the other hand, the business item DB 100 and the reference DB 200 may map the image data of the fashion item 510 to a specific location 530 on a preset vector space 520, and from this, the type of the fashion item and vector information. Here, the vector space 520 is not limited to two dimensions, but may be implemented in three dimensions or more according to a plurality of factors considered in a vector-based fashion recommendation method using deep learning.

And, looking at an example of a method of extracting an item having a similar vector characteristic from a specific item, first, the image data of the specific item is mapped to a specific position on a preset vector space to obtain a base vector corresponding to the image data, After obtaining a similar vector by using the obtained basic vector and the vector of a specific item, image data of a fashion item different from the specific item may be obtained based on the obtained vector.

Meanwhile, in the present invention, the vector of a fashion item may be a feature vector extracted from an image of each item, and may be extracted by applying a CNN-based image feature extraction method.

Also, in the present invention, similarity between various items can be calculated by applying a vector similarity calculation method such as cosine similarity and jaccard similarity.

Meanwhile, in relation to the fashion recommendation method related to the present invention, a customer segmentation-based recommendation method, a personalization-based recommendation method, and a keyword-based recommendation method may be considered.

First, the customer segmentation-based recommendation method is a method of segmenting a customer group into 0 to 00 groups based on main attributes, and may recommend each customer based on the characteristics of a group to which they belong.

In this regard, the customer segmentation-based recommendation method has limitations in the following points.

- Within a group, it is treated uniformly, and specific differences cannot be considered.

- It is a structure in which the number of segments cannot be increased indefinitely. In this regard, if the number of groups is increased in a situation where the number of training data is limited, the number of samples in each group decreases, so that there is a problem in that learning is not performed properly.

On the other hand, segmentation does not correspond to personalization. In this regard, two co-workers of the same age from the same school may have completely different tastes in fashion, but traditional customer segmentation methods cannot differentiate between the two. Accordingly, it is difficult to apply segmentation to a fashion recommendation in which likes/dislikes of taste change even with subtle differences.

Next, the keyword-based recommendation method has the following limitations.

- It is inaccurate to recognize the characteristics of each item only with basic formal attributes (eg brand, material, color).

- It expands to the level of tags attached (manually), but it is also difficult to express in terms of keywords, so subtle differences cannot be expressed with keywords alone.

- There are countless items with the same format attribute and tag, but completely different, so the keyword-based recommendation method can produce inaccurate recommendation results.

Therefore, the vector-based fashion recommendation method using deep learning according to the present invention has the following technical characteristics.

1) Image based

- Use the image of each item as the main information source for the image.

- The computer uses deep learning technology to recognize various features of the item in the image.

2) In addition to fashion items, characteristics of entities that influence recommendations such as brand personality and user's fashion taste can be defined based on images.

3) A recommendation framework and methodology that can be used comprehensively can be suggested.

In addition, the vector-based fashion recommendation method using deep learning according to the present invention utilizes the following methodology to achieve the above-described technical characteristics.

1) Express minute features of fashion products (items) as high-dimensional (100 or more) vectors.

2) Express the user and the weather as vectors, respectively.

3) Implement a personalized recommendation considering the situation using a discriminator that takes the combination of these vectors as an input.

In this regard, FIG. 2B is a conceptual diagram of a style vector in a vector-based fashion recommendation method using deep learning according to the present invention. Referring to FIG. 2B , by passing (inference) an image of a fashion item through a deep learning network, the stylistic feature of the corresponding item can be expressed as the strength of hundreds to thousands of individual features.

Accordingly, the vector-based fashion recommendation method using deep learning according to the present invention can implement the following technical features using the aforementioned style vector concept.

1) The stylistic characteristics of fashion items are not only extrinsic characteristics that can be easily expressed with keywords (eg color, shape, button, ribbon, material, etc.), but also implicit characteristics that are not easily expressed in words (eg, feeling, style). etc.) consists of

2) Style Vector: Expresses the stylistic characteristics of each item as a high-dimensional feature vector.

In this regard, each dimension of the vector represents the strength of each feature as a number between 0 and 1.

3) That is, individual fashion items are expressed in as much detail as possible in a form that can be calculated.

4) In this regard, one of the following methods can be used to retrieve the Style Vector.

- Skipgram method of Word2Vec

- Auto-Encoder method

- Siamese Convolutional Neural Network (CNN) method

- Other Image-to-Vector deep learning methods

- A weighted average method of the above methods

On the other hand, in relation to the vector-based fashion recommendation method using deep learning according to the present invention, the user style vector and the weather style vector and the method of determining whether to recommend a specific item to a specific user by generalizing them have the following technical characteristics. have

1) User style vector

- User's preferred stylistic features can also be expressed as style vectors

- Can be obtained using a vector of items that the user owns or prefers (marked 'Like')

- To obtain a single user vector from multiple item vectors, the method of averaging each vector dimension (attribute), taking the maximum value, or averaging the weighted average can be used.

- There is no need for a single user vector, and by grouping user items by type/characteristic, one vector per group can be defined to express users with multiple representative vectors.

2) the style vector of the weather

- Weather is used by classifying the combination of temperature and weather.

- Each type of temperature (very hot, hot, normal, cold, very cold) can be expressed as a style vector

- Items frequently worn by type of temperature can be obtained using vectors

- If the number of items for each type of temperature is too large, the type of temperature may be further subdivided.

- Each type of weather (sunny, cloudy, rain, fog, snow) can be expressed as a style vector

- Items frequently worn by type of weather can be obtained using vectors

- If the number of items for each type of weather is too large, the type of temperature may be further subdivided.

- Items frequently worn by weather type can be identified using the user's clothing log.

- Record of clothes worn by date recorded on online fashion services, etc.

- Use the weather information (temperature and weather) by region and date provided by the Weather Service of the Korea Meteorological Administration, etc.

- By linking this information, we obtain a list of items of clothing that can be worn in a specific temperature and weather.

3) How to decide whether to recommend a specific item to a specific user

- If the proximity of the style vector of the item to the user's style vector is greater than or equal to the standard value, it is judged to be suitable.

- Determination of suitability by learning artificial neural network using two vectors as input

- Learning positively when the user and the user's owned/preferred items are given as inputs

- If an item other than the user and the user's owned/preferred item is given as an input, it is learned negatively

- If it is determined that the product to be recommended is suitable for the user to be recommended by using one of the above two methods or the weighted average of both methods, the product is recommended

4) How to determine whether a particular item is recommended for a particular weather

If the proximity between the style vector of the item and the style vector of the corresponding weather (temperature or weather) is greater than or equal to the reference value, it is judged to be suitable.

Determination of matching by training an artificial neural network using two vectors as input

Learn positively if given as input an item with the weather and a record of wearing it in that weather

Learning negatively when an item other than the weather and an item with a record of wearing it in that weather is given as an input

If it is determined that the product to be recommended is suitable for the weather to be recommended using one of the two methods or the weighted average of both methods, the product is recommended.

5) Recommendation method considering users and weather comprehensively

- When it is determined that a specific item is suitable for the recommended user

- When it is determined that a specific item is suitable for the recommended weather (based on date and region)

- If the above two cases are satisfied at the same time, the item is recommended

- Recommend the item if only one of the above two cases is satisfied

On the other hand, based on the vector-based fashion recommendation system using deep learning of FIG. 1 and the method of mapping the fashion items of FIGS. 2A and 2B to a vector space, a vector-based fashion recommendation method using deep learning is shown in FIGS. 3 to Referring to FIG. 9, the description is as follows. On the other hand, the vector-based fashion recommendation method using deep learning according to the present invention can use the above-described user style vector and weather style vector, and the technical characteristics of a method of determining whether to recommend a specific item to a specific user by generalizing them. In addition, the vector-based fashion recommendation method using deep learning according to the present invention can utilize the technical characteristics of the method of determining whether to recommend the above-mentioned specific item in specific weather and the method of recommending the user and weather in consideration of the user. .

In this regard, Figure 3 shows a user style vector DB construction method according to the present invention. Meanwhile, FIG. 4 shows a method of constructing a weather/situation style vector DB according to the present invention.

3 and 4, the vector-based fashion recommendation method using deep learning according to the present invention includes a fashion item list receiving step (S100), a style vector extraction step (S200), and a style vector set output step (S300). may include Meanwhile, the vector-based fashion recommendation method using deep learning according to the present invention may be performed by the server 500 performing the fashion recommendation method.

In this regard, in the fashion item list receiving step ( S100 ), a list of fashion items possessed/preferred by a specific user and a list of fashion items corresponding to a specific weather or situation may be received from the user terminal. Meanwhile, in the style vector extraction step ( S200 ), a style vector may be extracted through deep learning with respect to a fashion product image for each item in the fashion item list. In addition, in the step of outputting the style vector set ( S300 ), the style vector for each specific user and the style vector set corresponding to the specific weather or situation may be output.

In particular, in the style vector set output step (S300), the fashion items are divided into groups according to the type of the fashion item and the characteristics of the fashion item, and a representative vector is calculated for each group to output the style vector set. there is. Here, the type of the fashion item may be classified into a top, bottom, outerwear, shoes, and dress, and the personality of the fashion item may be classified into a formal look, a casual look, and an office look. Here, the types and characteristics of the selected items are not limited thereto, and may be variously applied to the types and characteristics of the items described above with reference to FIG. 1 .

Meanwhile, in the step of outputting the style vector set ( S300 ), the style vector set may be output by calculating a representative vector for each group according to the classification associated with the type and personality of the selected item.

Meanwhile, in the style vector set output step ( S300 ), representative values according to the average, maximum value, and weighted average of the extracted style vectors are obtained for each dimension, and the representative values for each dimension are combined to represent the respective groups. A vector can be computed to output a set of style vectors.

In more detail, a) A vector that obtains a representative value of the vectors by a method such as average/maximum value/weighted average for each dimension and has it as a value of each dimension may be regarded as a representative vector. Also, b) a clustering algorithm is applied to vectors, and a set of centroid vectors can be regarded as a representative vector set. Also, it is possible to divide item groups according to the type of fashion item (top, bottom, outerwear, shoes, dress, etc.) and personality (formal wear, casual wear, office wear, etc.) and apply the methods a) or b) to each group. there is.

Meanwhile, in relation to the method of constructing the style vector DB for each weather/situation of FIG. 5 , a list of fashion items corresponding to a specific weather or situation in the step S100 of receiving the list of fashion items may be obtained using log information. Specifically, the list of fashion items corresponding to the specific weather or situation may be obtained by extracting a list of fashion items used, purchased, or viewed in a specific weather or situation from a usage log of the online fashion service system. Accordingly, in the style vector extraction step (S200), based on the list of fashion items possessed/preferred by each specific user and the list of fashion items used, purchased, or read by the specific user in the specific weather or situation, the style vector A set can be extracted.

On the other hand, the vector-based fashion recommendation method and system using deep learning according to the present invention is not limited to the style for each specific user and the style for each weather/situation as shown in FIG. 3, but calculates a style vector for all concepts related to fashion. It is possible to generalize in the sense that In this regard, FIG. 5 shows a method of constructing a style vector DB associated with a list of fashion items for a specific concept according to the present invention.

In this regard, the style vector DB construction method associated with the list of fashion items for the specific concept of FIG. 5 together with the style vector DB construction method for each specific user of FIGS. 3 and 4 and the style vector DB construction method for each weather/situation will be performed. may be

Alternatively, without performing the style vector DB construction method for each specific user of FIGS. 3 and 4 and the style vector DB construction method for each weather/situation, the style vector DB construction method associated with the list of fashion items for the specific concept of FIG. 5 is may be performed.

Referring to FIG. 5 , the method for constructing a style vector DB associated with a list of fashion items for a specific concept may include a step of receiving a list of fashion items (S100a), a step of extracting a style vector (S200a), and an step of outputting a set of style vectors (S300a). there is. Meanwhile, the vector-based fashion recommendation method using deep learning according to the present invention may be performed by the server 500 performing the fashion recommendation method.

Meanwhile, in the fashion item list receiving step ( S100a ), a fashion item list may be received for each specific concept.

Specifically, in the fashion item list receiving step (S100a), in addition to the fashion item list for each specific user and the fashion item list corresponding to the specific weather or situation, a list of fashion items related to a specific concept by brand, region and age group may be received. . In this regard, in the fashion item list receiving step (S100a), the list of fashion items for each specific user and the list of fashion items corresponding to the specific weather or situation are not received, but a list of fashion items related to a specific concept for each brand, region, and age group. may only be received.

In the style vector extraction step ( S200a ), the specific concept may be anything that can be expressed as a set of related fashion items in addition to the 'user', 'weather' and 'situation' presented above. Accordingly, as described above, the specific concept can be, for example, 'brand', 'region', 'age', etc., and a style vector DB can be constructed for each concept type. Therefore, the vector-based fashion recommendation method using deep learning according to the present invention has the advantage that it is possible to calculate style vectors for all concepts related to fashion, rather than the two items of user and weather being important.

Accordingly, in the style vector extraction step S200a, the style vector may be extracted for each brand, region, and age group. However, the present invention is not limited thereto, and there is an advantage that style vectors for all concepts related to fashion can be calculated or extracted according to application.

Meanwhile, in the style vector set output step ( S300a ), a style vector set corresponding to each specific concept among all concepts related to fashion may be output. Accordingly, in the step of outputting the style vector set ( S300a ), the style vector set may be output for each brand, region, and age group. In addition, in the style vector set output step ( S300a ), the aforementioned style vector for each specific user and a style vector set corresponding to a specific weather or situation may also be output.

Meanwhile, in the vector-based fashion recommendation method using deep learning according to the present invention, a detailed method of determining which item to recommend to a specific user will be described as follows. In this regard, FIG. 6 is a flowchart of a vector-based fashion item recommendation method according to the present invention. Referring to FIG. 6 , the fashion item recommendation method includes a style vector set output step S300 , a vector search step S400 , and a recommended item list output step S500 .

With respect to the vector-based fashion item recommendation method according to the present invention of FIG. 6 , an input value may be a user ID for recommending a fashion item for each specific user, but is not limited thereto and may be variously changed according to applications. In addition, the input value may further include the type and/or personality of the requested fashion item.

In the style vector set output step ( S300 ), a style vector set for each specific user may be output. Specifically, in the step of outputting the style vector set ( S300 ), the style vector corresponding to the user ID and the requirement may be fetched or output from the user style vector DB.

However, the present invention is not limited thereto, and in the style vector set output step S300 as shown in FIG. 5 , a style vector set corresponding to each specific concept among all concepts related to fashion may be output. To this end, in the style vector set output step ( S300 ), a style vector corresponding to a user ID and a requirement may be fetched from the user style vector DB and outputted.

Meanwhile, in the vector search step S400 , k item style vectors having high vector similarity to the fetched style vector may be searched from the item style DB. In this regard, the vector similarity uses a general vector similarity scale such as cosine similarity, and a fast search can be implemented by applying a technique such as Locality Sensitive Hashing. Meanwhile, the extracted style vector and the vector stored in the item style vector DB are input to the artificial neural network, and the item that yields the highest score can be selected.

In addition, in the step of outputting the list of recommended items ( S500 ), one or more, that is, a list of k recommended items for each specific user may be output based on the set of style vectors.

Meanwhile, FIG. 7 is a flowchart illustrating a method of determining which item to recommend given weather/situation according to an embodiment of the present invention. 8 is a flowchart illustrating a method of determining which item to recommend to a specific user given weather/situation according to an embodiment of the present invention.

First, referring to FIG. 7 , the fashion item recommendation method includes a style vector set output step S300 , a vector search step S400 , and a recommended item list output step S500 .

In relation to the vector-based fashion item recommendation method according to the present invention of FIG. 7 , the input value may be a weather/situation ID for recommending a fashion item for each specific weather/situation, but is not limited thereto and may be variously used depending on the application. you can change it. In addition, the input value may further include the type and/or personality of the requested fashion item.

In the style vector set output step ( S300 ), a style vector set corresponding to a specific weather or situation may be output. Specifically, in the step of outputting the style vector set ( S300 ), the style vector corresponding to the given weather/situation and requirements may be fetched or output from the weather/situation style vector DB.

However, the present invention is not limited thereto, and in the style vector set output step S300 as shown in FIG. 5 , a style vector set corresponding to each specific concept among all concepts related to fashion may be output. To this end, in the style vector set output step ( S300 ), a style vector corresponding to a user ID and a requirement may be fetched from the user style vector DB and outputted.

Meanwhile, in the vector search step S400 , k item style vectors having high vector similarity to the fetched style vector may be searched from the item style DB. In this regard, the vector similarity uses a general vector similarity scale such as cosine similarity, and a fast search can be implemented by applying a technique such as Locality Sensitive Hashing. Meanwhile, the extracted style vector and the vector stored in the item style vector DB are input to the artificial neural network, and the item that yields the highest score can be selected.

In addition, in the step of outputting the list of recommended items ( S500 ), based on the set of style vectors, one or more, that is, a list of k recommended items may be output for each specific user according to the specific weather or situation.

Next, referring to FIG. 8 , the fashion item recommendation method includes a style vector set output step (S300), a query vector generation step (S350), a vector search step (S400), and a recommended item list output step (S500). .

In relation to the vector-based fashion item recommendation method according to the present invention of FIG. 8, input values may be a user ID and a weather/situation ID for recommending a fashion item for each specific user, but are not limited thereto and may vary depending on the application. can be changed to In addition, the input value may further include the type and/or personality of the requested fashion item.

In the style vector set output step ( S300 ), a style vector set for each specific user may be output. Specifically, in the step of outputting the style vector set ( S300 ), the style vector corresponding to the user ID and the requirement may be fetched or output from the user style vector DB. In addition, in the style vector set output step ( S300 ), a style vector set corresponding to a specific weather or situation may be output. Specifically, in the step of outputting the style vector set ( S300 ), the style vector corresponding to the given weather/situation and requirements may be fetched or output from the weather/situation style vector DB.

However, the present invention is not limited thereto, and in the style vector set output step S300 as shown in FIG. 5 , a style vector set corresponding to each specific concept among all concepts related to fashion may be output. To this end, in the style vector set output step ( S300 ), a style vector corresponding to a user ID and a requirement may be fetched from the user style vector DB and outputted.

Meanwhile, in the query vector generation step ( S350 ), the first style vector extracted based on the user ID for each specific user and the second style vector extracted based on the specific weather or situation are synthesized in a weighted sum method. to create a query vector. In this regard, however, the present invention is not limited thereto, and in the query vector generation step ( S350 ), a query vector is generated by synthesizing a plurality of style vectors corresponding to a specific concept among all fashion-related concepts in a weighted sum method. can do.

Meanwhile, in the vector search step S400 , k item style vectors having high vector similarity to the generated query vector may be searched from the item style DB. In this regard, the vector similarity uses a general vector similarity scale such as cosine similarity, and a fast search can be implemented by applying a technique such as Locality Sensitive Hashing. Meanwhile, the generated query vector and the vector stored in the item style vector DB are input to the artificial neural network to select the item that yields the highest score.

In addition, in the recommended item list output step ( S500 ), based on the set of style vectors similar to the generated query vector, one or more, that is, k list of recommended items can be output for each specific user according to the specific weather or situation. there is.

Meanwhile, FIG. 9 is a flowchart of a method of determining which item to recommend when a query concept is given according to the present invention. Referring to FIG. 9 , the method for recommending an existing fashion item includes a style vector set output step (S300a), a query vector generation step (S350a), a vector search step (S400a), and a recommended item list output step (S500a).

In this regard, the core idea of the vector-based fashion item recommendation method according to the present invention is to convert various concepts related to fashion into a vector of a uniform form, and calculate a recommendation result through a mutual comparison operation between them. corresponds to

Meanwhile, the extrinsic and intrinsic features for the vector-based fashion item recommendation method according to the present invention will be described as follows. In this regard, FIG. 10 shows the external characteristics of a fashion item associated with a style vector in the vector-based fashion item recommendation method according to the present invention. On the other hand, FIG. 11 shows the intrinsic characteristics of a fashion item associated with a style vector in the vector-based fashion item recommendation method according to the present invention. 12 shows a fashion item type/personality classification example for a vector-based fashion item recommendation method according to the present invention.

In this regard, since it is an example in which the extrinsic characteristics as shown in FIG. 10 are available, the following classification is not necessarily used. In addition, the present invention does not explicitly use the items listed in FIG. 11 . In this regard, it means that through the results of deep learning feature extraction, it is possible to express various (non-verbally impossible) intrinsic features as follows. In addition, since the present invention is a case that can be classified as shown in FIG. 12, the following classification is not necessarily used.

Meanwhile, in relation to the vector-based fashion item recommendation method according to the present invention, items having a similar style as a result of vector extraction should be placed at a close distance in a vector space. In this regard, FIG. 13 shows an example of a vector space in which a style vector is disposed in a vector-based fashion item recommendation method according to the present invention. Referring to FIG. 13 , a formal dress and high heels are a formal look or an office look, and style vectors are placed close to each other in a vector space. In addition, the style vectors of the punk-style fashion items are placed at a close distance between each other in the vector space.

Accordingly, FIG. 13 shows an example in which items of a similar style are placed at a close distance in a vector space as a result of vector extraction. That is, FIG. 13 is a visualization result of visualizing that the fashion item images are positioned closer to each other as the vector similarity increases.

Meanwhile, in relation to the vector-based fashion item recommendation method according to the present invention, the criteria for judging a similar vector are as follows.

- The scales commonly used to measure the similarity between vectors (ex. Cosine similarity) are used, and the top k items are considered similar by sorting them in descending order of the highest similarity (k is the number of recommended items to be output).

- The reference value can be specified as a threshold, and when the maximum similarity is 1.0, a value of 0.8 to 0.9 is specified as the threshold to judge vector pairs with a similarity higher than this to be similar.

The vector-based fashion item recommendation method according to the present invention will be generalized and described in detail in consideration of the intrinsic and extrinsic characteristics of the above-described fashion item, the classification of the fashion item type/personality, and the vector similarity of the fashion item images.

In this regard, in relation to the vector-based fashion item recommendation method according to the present invention of FIG. 9 , an input value may be Concept_1 ID to Concept_N ID for recommending a fashion item for each specific user. In this regard, Concept_1 ID to Concept_N ID may be IDs of specific concepts for each brand, region, and age group, in addition to the user ID and weather/situation ID. However, the present invention is not limited thereto and may be variously changed according to application. In addition, the input value may further include the type and/or personality of the requested fashion item.

In the style vector set output step ( S300 ), the style vector set of Concept_1 may be output. Specifically, in the step of outputting the style vector set ( S300 ), the style vector corresponding to the user ID and the requirement may be fetched or output from the user style vector DB. Also, in the step of outputting the style vector set ( S300 ), the style vector set corresponding to Concept_2 to Concept_N may be output. Specifically, in the style vector set output step S300 , a style vector corresponding to a given requirement may be fetched or output from the weather/situation style vector DB or a specific concept DB for each brand, region, and age group.

However, the present invention is not limited thereto, and in the style vector set output step S300 as shown in FIG. 5 , a style vector set corresponding to each specific concept among all concepts related to fashion may be output. To this end, in the style vector set output step ( S300a ), the style vector corresponding to the user ID and requirements may be fetched and output from the user style vector DB, the weather/situation DB, and the specific concept DB.

Meanwhile, in the query vector generating step ( S350a ), the query vector may be generated by synthesizing the first to Nth style vectors according to IDs of Concept_1 to Concept_N in a weighted sum method.

For example, in the query vector generating step ( S350a ), a weighted sum method is used for a first style vector extracted based on the user ID for each specific user and a second style vector extracted based on the specific weather or situation. can be combined to create a query vector.

As another example, a first style vector extracted based on the user ID for each specific user, a second style vector extracted based on the specific weather or situation, and a third style vector extracted in relation to a specific concept for each brand, region, and age group A query vector can be generated by synthesizing style vectors using a weighted sum method. In this regard, however, the present invention is not limited thereto, and in the query vector generation step ( S350a ), a query vector is generated by synthesizing a plurality of style vectors corresponding to a specific concept among all fashion-related concepts in a weighted sum method. can do.

Meanwhile, in the vector search step S400a, k item style vectors having high vector similarity to the generated query vector may be searched from the item style DB. Specifically, in the vector search step (S400a), the extracted first to Nth style vectors and vectors stored in the item style DB are input to the artificial neural network, and k recommended items list in order from the item that calculates the highest score. can be selected.

For example, the first and second style vectors extracted in the vector retrieval step (S400a) and the vectors stored in the item style DB are input to the artificial neural network, and list of k recommended items in order from the item that calculates the highest score can be selected.

As another example, the first to third style vectors extracted in the vector search step ( S400a ) and the vectors stored in the item style DB are input to the artificial neural network, and list of k recommended items in order from the item that calculates the highest score can be selected.

In this regard, the vector similarity uses a general vector similarity scale such as cosine similarity, and a fast search can be implemented by applying a technique such as Locality Sensitive Hashing. Meanwhile, the generated query vector and the vector stored in the item style vector DB are input to the artificial neural network to select the item that yields the highest score.

In addition, in the recommended item list output step S500a, based on the set of style vectors similar to the generated query vector, one or more, i.e., k recommended item lists can be output for each specific user according to the specific weather or situation. there is.

In the above, a vector-based fashion recommendation method and system using deep learning according to the present invention has been described. In this regard, the technical effects of the vector-based fashion recommendation method and system using deep learning according to the present invention are as follows.

According to at least one embodiment of the present invention, there is an advantage in that it is possible to provide a vector similarity-based fashion recommendation method and system using deep learning.

In addition, according to at least one embodiment of the present invention, it is possible to provide a vector-based fashion recommendation method and system using deep learning by calculating style vectors for all concepts related to fashion.

In addition, according to at least one embodiment of the present invention, there is an advantage in that various concepts related to fashion can be converted into a vector of a uniform form, and a recommendation result can be calculated through a mutual comparison operation between them.

According to the software implementation, not only the procedures and functions described in this specification but also the design and parameter optimization for each component may be implemented as a separate software module. The software code may be implemented as a software application written in a suitable programming language. The software code may be stored in a memory and executed by a controller or a processor.

Claims (10)

In the vector-based fashion recommendation method using deep learning, the method is performed by a server performing the fashion recommendation method,
a fashion item list receiving step of receiving, from the user terminal, a list of fashion items possessed/preferred by a specific user, a list of fashion items corresponding to a specific weather or situation, and a list of fashion items related to a specific concept by brand, region and age group;
a style vector extraction step of extracting a style vector through deep learning of a Skipgram or Auto-Encoder method for a fashion product image for each item in the fashion item list;
a style vector set output step of outputting the style vector for each specific user and a style vector set corresponding to the specific weather or situation; and
A recommended item list output step of outputting a list of one or more recommended items for each specific user based on the style vector set;
The step of outputting the style vector set,
The fashion item is divided into groups according to the type of the fashion item and the nature of the fashion item, a representative vector is calculated for each group, and a style vector set is output, but the extracted style vectors are averaged for each dimension; Obtaining a representative value according to the maximum value and the weighted average, combining the representative value for each dimension to calculate a representative vector for each group to output a set of style vectors,
After the style vector extraction step,
A first style vector extracted based on the user ID for each specific user, a second style vector extracted based on the specific weather or situation, and a third style vector extracted in relation to a specific concept for each brand, region and age group a query vector generation step of generating a query vector by synthesizing it using a weighted sum method; and
and a vector retrieval step of retrieving k vectors from the item style DB in the order of high vector similarity to the generated query vector.
delete According to claim 1,
The types of the fashion items are classified into tops, bottoms, outers, shoes, and dresses, and the characteristics of the fashion items are classified into formal look, casual look, and office look,
In the step of outputting the style vector set,
A fashion recommendation method for outputting a set of style vectors by calculating a representative vector for each group according to the classification.
delete According to claim 1,
The list of fashion items corresponding to the specific weather or situation is,
Extract and secure a list of fashion items used, purchased, or viewed in a specific weather or situation from the usage log of the online fashion service system;
In the style vector extraction step,
A fashion recommendation method, characterized in that the style vector set is extracted based on the list of fashion items possessed/preferred by each specific user and the list of fashion items used, purchased, or read by the specific user in the specific weather or situation .
delete delete delete delete According to claim 1,
In the vector search step,
Fashion recommendation, characterized in that the extracted first to third style vectors and vectors stored in the item style DB are input to an artificial neural network, and a list of k recommended items is selected in order from the item that calculates the highest score. method.

Images