KR102178961B1 - Artificial neural network apparatus and method for recommending fashion item using user clustering - Google Patents

Abstract

The present invention relates to an apparatus and a method for a fashion item recommendation artificial neural network using user clustering. To this end, the apparatus includes: a fashion item classification module configured to receive fashion item image information from a fashion item detection module, input each fashion item image information to a previously learned artificial neural network, and output fashion item classification information for each fashion item image information; and an embedding module which is connected to an FC layer of the artificial neural network of the fashion item classification module, and outputs an output vector of the FC layer of each artificial neural network for each fashion item image information as an embedding vector for each fashion item image information. According to the present invention, it is possible to automatically cluster fashion contents or fashion items reacted by a user by the embedding vector without an administrator manually tagging a concerned style or classifying users.

Description

Artificial neural network apparatus and method for recommending fashion item using user clustering}

The present invention relates to a fashion item recommendation artificial neural network apparatus and method using user clustering.

As the 4th Industrial Revolution has established itself as a new era, the so-called'style-tech' era is opening up in the fashion industry. 'Styletech' is a concept that encompasses products and services that create new customer experiences by combining information technology (IT) with lifestyle fields such as fashion and beauty. The change in which deep learning or machine learning is applied to recommending fashion items with characteristics such as a specific brand, design, color, material, and category to a specific user can be said to be a representative style technology in the fashion industry. When deep learning or machine learning is applied to recommending fashion items, it becomes possible to automate product recommendation services based on personalization.

The representative algorithm used to recommend fashion items is collaborative filtering. Collaborative filtering in the recommendation system refers to a similar collective intelligence algorithm that predicts purchase possibility or taste based on purchase information/taste information collected from many users. Collaborative filtering can be largely divided into memory-based, model-based, and hybrid, and in the case of memory-based, it can be divided into user-based and item-based as described above. User-based collaborative filtering is an algorithm that finds specific customers with similar purchasing patterns to reference customers, which can be predicted to some extent, and predicts the behavior of specific customers by vectorizing the behavior of reference customers. Item-based collaborative filtering is an algorithm mainly used by Amazon.com.It is an algorithm that recommends items based on the similarity of items, and uses an item-item matrix that determines the correlation between items to provide the latest user data. Based on, the user's preference is inferred. However, as collaborative filtering is an algorithm close to collective intelligence, it is difficult to recommend personalized fashion items.

The most representative place where artificial intelligence algorithms are applied to recommend personalized fashion items is'Stitch Fix'. Founded in 2011, Stitch Fixes, a personalized shopping service, was listed on NASDAQ in 2017 (Ticker: SFIX), achieved sales of $1.58B in 2019, and continues to grow around 20% to 150% annually. As of February 2020, there are 3.5 million active users using Stitch Fix. The key competitive advantage of Stitch Fix in the fashion industry lies in the AI algorithm applied to the recommendation of fashion items. Among Stitch Fix's executives and staff, there are about 125 master's and doctoral data scientists, which have significantly higher AI resources than the 5,100 in-house stylists. AI analyzes the data and picks out clothes that they like, and a professional stylist picks five of them and delivers them to customers. Customers can try on clothes and return them if they don't like it. Stitch Fix's fashion item recommendation algorithm is supervised learning by fashion items that a customer decides to purchase or are returned by customers.

Stitch Fix's successful business virtuous cycle has attracted attention in the fashion market with personal stylists, and recently, Stitch Fix's competitors are increasing considerably. Representatively, Amazon.com also launched'Personal shopper by Prime wardrobe', which is similar to the service model/business model of Stitch Fix.

Republic of Korea Patent Publication 10-2019-0029567, Product recommendation method using style characteristics, Omnious Co., Ltd. US Patent No. 10395301 B2, Method, medium, and system for using a combination of batch-processing and on-demand processing to provide recommendations, Stitch Fix, Inc.

However, collaborative filtering had a difficult problem in recommending personalized fashion items, and the existing artificial intelligence algorithm based on Supervised Learning for recommending fashion items had a problem in clustering users' style tastes and identities.

It was not easy because of the technical difficulty to cluster the styles and identities of users by using the unsupervised learning of log data generated in social networks to optimize fashion commerce. For example, the recommendation of a fashion item with the same style characteristics suggested in Korean Patent Application Laid-Open No. 10-2019-0029567 relates to an algorithm that determines whether it is simply the same shape without taking into account the overall atmosphere or matching harmony, etc. It cannot be used to cluster identities. When users are clustered and used for optimizing fashion commerce, the MD of fashion commerce categorizes the characteristics of users that are difficult to understand, such as a fashion item recommendation service, an advertisement service using a detailed user cluster, and a creator recommendation suitable for brand identity. The effect of optimizing fashion commerce in the form of fashion occurs.

Accordingly, an object of the present invention is to provide an apparatus and method for recommending fashion items using user clustering that recommends fashion items based on the clustered users after clustering the styles or identities of users using log data generated in a social network. Is in.

Hereinafter, specific means for achieving the object of the present invention will be described.

An object of the present invention is to receive fashion content information of interest, which is fashion content information uploaded or input by the specific user, to a social network web server, which is a web server of a specific social network service, through a user client of a specific user, A fashion content receiving module that generates fashion content image information and reaction information included in the fashion content information of interest; The fashion content receiving module receives fashion content image information, detects an area of a fashion item from the fashion content image information to generate a plurality of fashion item area information, and the fashion item area information A fashion item detection module that generates fashion item image information, which is image information of at least part of fashion content image information, as many as the number of fashion items detected in the fashion content image information; The fashion item detection module receives the fashion item image information and is connected to a fully connected layer of a pre-learned artificial neural network using each fashion item image information as input data, and each fashion item image An embedding module for outputting an output vector of the FC layer of the artificial neural network for information as an embedding vector for each fashion item image information; A cluster is created by clustering the embedding vectors of the plurality of fashion content information uploaded through the social network web server by a plurality of users, and the interest is mapped to the embedding vector of the fashion content information of interest to the cluster. A cluster user that generates cluster information, which is information about the cluster to which the embedding vector of fashion content information belongs, and is information about another user to which the embedding vector of the fashion content information of interest is mapped to the cluster included in the cluster information A clustering module that generates information; A purchase item information collection module that collects purchase item information, which is information on purchase items, which are a plurality of fashion items purchased by the other user, included in the cluster user information; And receiving a purchase item embedding vector output by inputting the purchase item information to the embedding module, and among the purchase item embedding vectors based on a distance between the center of the cluster included in the cluster information and the purchase item embedding vector. And an item recommendation module for determining a recommended item embedding vector, which is an embedding vector of the purchase item to be recommended, wherein the loss function of the artificial neural network is for each of the fashion item image information included in one of the fashion content information. Including the distance between the embedding vectors, learning the weight of the artificial neural network to minimize the loss value of the loss function in the learning step of the artificial neural network, and the purchase item information corresponding to the recommended item embedding vector It can be achieved by providing a fashion item recommendation device using user clustering, characterized in that the specific user outputs a recommendation item to the user client.

Another object of the present invention is to provide a fashion content receiving module to a social network web server, which is a web server of a specific social network service, through a user client of a specific user. A fashion content receiving step of receiving content information and generating fashion content image information and reaction information included in the fashion content information of interest; A fashion item detection module receives fashion content image information from the fashion content receiving module, detects an area of a fashion item from the fashion content image information, and generates a plurality of fashion item area information. A fashion item detection step of generating fashion item image information, which is image information of at least part of the fashion content image information with respect to item area information, as many as the number of fashion items detected in the fashion content image information; The embedding module receives the fashion item image information from the fashion item detection module and is connected to a fully connected layer of a pre-learned artificial neural network using each of the fashion item image information as input data. An embedding step of outputting an output vector of the FC layer of the artificial neural network with respect to the fashion item image information as an embedding vector for each fashion item image information; A clustering module creates a cluster by clustering the embedding vectors of the plurality of fashion content information uploaded through the social network web server by a plurality of users, and provides the embedding vector of the fashion content information of interest to the cluster. By mapping to generate cluster information, which is information about the cluster to which the embedding vector of the fashion content information of interest belongs, and to another user to which the embedding vector of the fashion content information of interest is mapped to the cluster included in the cluster information A clustering step of generating cluster user information, which is information; A purchase item information collection step of collecting, by the purchase item information collection module, purchase item information, which is information on purchase items, which are a plurality of fashion items purchased by the other user, included in the cluster user information; And the item recommendation module receives the purchase item embedding vector output by inputting the purchase item information to the embedding module, and based on a distance between the center of the cluster included in the cluster information and the purchase item embedding vector. An item recommendation step of determining a recommended item embedding vector, which is an embedding vector of the purchase item to be recommended, among the purchase item embedding vectors, wherein the loss function of the artificial neural network is each of the fashions included in one of the fashion content information. Including the distance between the embedding vectors for item image information, learning the weight of the artificial neural network to minimize the loss value of the loss function in the learning step of the artificial neural network, and corresponding to the recommended item embedding vector It can be achieved by providing a fashion item recommendation method using user clustering, characterized in that the purchase item information is output to the user client of the specific user as a recommended item.

Another object of the present invention is a memory module for storing a fashion item recommendation program code; And a processing module for processing the fashion item recommendation program code, wherein the fashion item recommendation program code is reacted by the specific user to a social network web server that is a web server of a specific social network service through a user client of a specific user. A fashion content receiving step of receiving fashion content information of interest, which is fashion content information input or uploaded, and generating fashion content image information and reaction information included in the fashion content information of interest; A fashion item image that detects an area of a fashion item from the fashion content image information to generate a plurality of fashion item area information, and is image information of at least a part of the fashion content image information for each of the fashion item area information A fashion item detection step of generating information as much as the number of fashion items detected in the fashion content image information; An embedding step of outputting an output vector of a fully-connected layer of the artificial neural network, which has the fashion item image information as input data, as an embedding vector for each fashion item image information; A cluster is created by clustering the embedding vectors of the plurality of fashion content information uploaded through the social network web server by a plurality of users, and the interest is mapped to the embedding vector of the fashion content information of interest to the cluster. A cluster user that generates cluster information, which is information about the cluster to which the embedding vector of fashion content information belongs, and is information about another user to which the embedding vector of the fashion content information of interest is mapped to the cluster included in the cluster information A clustering step of generating information; A purchase item information collection step of collecting purchase item information, which is information on purchase items, which are a plurality of fashion items purchased by the other user, included in the cluster user information; And a recommended item embedding vector that is an embedding vector of the purchase item to be recommended among the purchase item embedding vectors based on a distance between the center of the cluster included in the cluster information and a purchase item embedding vector that is an embedding vector of the purchase item information. Including a program code for performing an item recommendation step of determining; on a computer, the loss function of the artificial neural network is a distance between the embedding vectors for each of the fashion item image information included in one of the fashion content information Including, In the learning step of the artificial neural network, learning the weight of the artificial neural network to minimize the loss value of the loss function, and the purchase item information corresponding to the recommended item embedding vector is the user client of the specific user It can be achieved by providing a fashion item recommendation device using user clustering, characterized in that outputting the recommended item to the device.

As described above, the present invention has the following effects.

First, according to an embodiment of the present invention, an effect of automatically clustering by an embedding vector occurs without having to manually tag a style of interest or classify users for fashion content or fashion items that users respond to.

Second, according to an embodiment of the present invention, by recommending a product based on fashion content/fashion items uploaded or reacted by a specific user, an effect of improving the conversion rate of a recommendation algorithm or an advertisement algorithm is generated.

Third, according to an embodiment of the present invention, an interest-targeted advertisement of a specific brand is possible by recommending a group of users or creators based on fashion contents/fashion items of a specific brand.

Fourth, according to an embodiment of the present invention, an effect of being able to find fashion items matching each other in various elements including colors is generated by learning of the artificial neural network without separately learning or pre-stored color combinations that are suitable colors. .

The following drawings attached to the present specification illustrate preferred embodiments of the present invention, and serve to further understand the technical idea of the present invention together with the detailed description of the present invention, so the present invention is limited to the matters described in such drawings. And should not be interpreted.
1 is a schematic diagram showing a fashion content embedding apparatus for user clustering according to an embodiment of the present invention;
2 is a schematic diagram showing an operation relationship of the fashion content receiving module 10 according to an embodiment of the present invention;
3 is a schematic diagram showing the application of a smoothing filter by a smoothing filter application module according to an embodiment of the present invention;
4 is a schematic diagram showing a result of generating candidate information of a candidate output module according to an embodiment of the present invention;
5 is a schematic diagram showing a non-candidate removal module according to an embodiment of the present invention;
6 is a schematic diagram showing the operational relationship of the artificial neural network of the fashion item classification module 12 according to an embodiment of the present invention;
7 is a schematic diagram showing an example of an artificial neural network of the fashion item classification module 12 according to an embodiment of the present invention;
8 is a schematic diagram showing an embedding vector mapping in a clustering module according to an embodiment of the present invention;
9 is a schematic diagram showing a fashion item recommendation apparatus using user clustering according to an embodiment of the present invention;
10 is a flowchart of a cluster information request module when a user is introduced according to an embodiment of the present invention;
11 is a schematic diagram showing the generation of an embedding vector by the fashion item classification module 12 according to an embodiment of the present invention;
12 is a schematic diagram showing cluster information and cluster user information generated by the clustering module 14 according to an embodiment of the present invention;
13 is a schematic diagram showing the collection of purchase item information by the purchase item information collection module 21 according to an embodiment of the present invention;
14 is a schematic diagram showing calculation of a recommended item embedding vector according to an embodiment of the present invention;
15 is a schematic diagram showing a cluster importance calculation according to another embodiment of the present invention;
16 is a schematic diagram showing a device for recommending a creator for a fashion brand according to an embodiment of the present invention;
17 is a flowchart of a cluster information request module when brand cluster information is requested according to an embodiment of the present invention;
18 is a schematic diagram showing the generation of an embedding vector for brand content image information by the fashion item classification module 12 according to an embodiment of the present invention;
19 is a schematic diagram showing brand cluster information and cluster user information generated by the clustering module 14 according to an embodiment of the present invention;
20 is a schematic diagram of collection of creator information by a creator information collection module according to an embodiment of the present invention;
21 is a schematic diagram showing calculation of a recommended creator according to an embodiment of the present invention.

Hereinafter, exemplary embodiments in which the present invention can be easily implemented by those of ordinary skill in the art will be described in detail with reference to the accompanying drawings. However, when it is determined that a detailed description of a related known function or configuration may unnecessarily obscure the subject matter of the present invention, the detailed description thereof will be omitted.

In addition, the same reference numerals are used for parts having similar functions and functions throughout the drawings. Throughout the specification, when a specific part is said to be connected to another part, this includes not only the case that it is directly connected, but also the case that it is indirectly connected with another element interposed therebetween. In addition, the inclusion of a specific component does not exclude other components unless specifically stated to the contrary, but means that other components may be further included.

Fashion content embedding device for user clustering

1 is a schematic diagram showing a fashion content embedding apparatus for user clustering according to an embodiment of the present invention. As shown in Fig. 1, the fashion content embedding device 1 for user clustering includes a fashion content receiving module 10, a fashion item detection module 11, a fashion item classification module 12, and an embedding module 13 , May include a clustering module 14. In addition, the fashion content embedding device 1 for user clustering according to an embodiment of the present invention is provided by a processing module of a computing device such as a specific web server, a virtual server such as a cloud server, a smartphone, a tablet PC, and a desktop PC. It can be configured to be processed and stored in a memory module of each device. The fashion content embedding device 1 for user clustering according to an embodiment of the present invention receives the user's reaction information (like, share, comment, etc.) or the fashion content of interest, which is the fashion content uploaded by the user, to a user client. It may be configured to receive directly from 100 or to receive from the social network web server 50 through a social network API.

Fashion content receiving module 10, the user's reaction information (Like, Share, Comment, etc.) from the user client 100 or the social network web server 50 is input or fashion content of interest that is uploaded by the user It is a module that receives 2 is a schematic diagram showing an operation relationship of the fashion content receiving module 10 according to an embodiment of the present invention. As shown in Figure 2, the fashion content receiving module 10 according to an embodiment of the present invention, a specific period for a specific user in the social network application module of the user client 100 or the social network web server 50 Receives fashion content information (fashion content image information, text information, tag information, comment information, like information, view count information, etc.) of interest during the period, and transmits fashion content image information received to the fashion item detection module 11 It is a module.

The fashion item detection module 11 receives fashion content image information from the fashion content receiving module 10, detects a fashion item area from the fashion content image information, and generates fashion item area information, This module generates fashion item image information, which is at least a part of fashion content image information, based on fashion item area information.

The fashion item detection module 11 according to an embodiment of the present invention may use a fashion item detection algorithm fine-tuning YOLO, RCNN, Faster RCNN, and the like. Alternatively, a fashion item detection algorithm that fine-tuned a network such as AlexNet previously learned with ImageNet can be used. Furthermore, it is possible to use existing computer vision algorithms such as Boosting the Haar-like Feature of Viola-jones.

The fashion item detection module 11 according to an embodiment of the present invention may include a smoothing filter application module, a candidate output module, and a non-candidate removal module. The fashion item detection by the fashion item detection module 11 according to an embodiment of the present invention is specifically configured to perform fashion item detection in fashion content image information, and the detected fashion item image information The fashion item expressing as a feature vector is transmitted to the embedding module 13 that performs feature embedding to vectorize each fashion item. In the clustering module 14, the embedding feature vectors are compared to form a cluster between vectors. It may be configured to perform clustering to output cluster information including at least one fashion item image information and user information clustered.

Fashion item detection of the fashion item detection module 11 according to an embodiment of the present invention includes a rectangle (or polygon, oval, or shape) configured to surround at least one fashion item included in the fashion content image information. A candidate output module that outputs candidate information, which is a fashion item box (circular, curved, etc.), non-candidate that outputs non-candidate classification information in which an image in a fashion item box among the candidate information is classified as not a fashion item It may be performed by the removal module, and may be configured to provide the candidate information whose non-candidate classification information is less than or equal to a specific value (excluding candidate information classified as non-candidate).

The smoothing filter application module according to an embodiment of the present invention generates at least one smoothing fashion content image information by applying a smoothing filter to the fashion content image information, and transmits the generated smoothing fashion content image information to the candidate output module. Can be configured. The smoothing filter of the smoothing filter application module may be composed of a smoothing filter such as a Gaussian filter, a bilateral filter, and a median filter, and the fashion content image information is gradually It may be configured to generate a plurality of smoothing fashion content image information by applying a smoothing weight and input it as input information of the artificial neural network of the candidate output module. For example, when a Gaussian filter is configured as a smoothing filter, the value of sigma, which is the standard deviation of the Gaussian kernel in the x and y directions, is gradually applied to a large value (for example, the first smoothing fashion content image It is configured to generate a plurality of smoothing fashion content image information while applying sigma to 1 for information, sigma to 2 for the second smoothing fashion content image information, etc.), and the smoothing fashion to which the smoothing weight of the smoothing filter is gradually applied. It may be configured to input content image information and fashion content image information to which a smoothing filter is not applied to the artificial neural network of the candidate output module. 3 is a schematic diagram illustrating application of a smoothing filter by a smoothing filter application module according to an embodiment of the present invention. As shown in FIG. 3, the smoothing filter application module according to an embodiment of the present invention generates at least one smoothing fashion content image information whose smoothing degree is gradually increased by gradually increasing Sigma for fashion content image information. Can be configured to

The candidate output module according to an embodiment of the present invention includes a pre-learned artificial neural network, and the input information of the artificial neural network of the candidate output module is a smoothing fashion in which a smoothing filter is applied to the fashion content image information and the fashion content image information. Content image information can be configured, and the output information is fashion item classification information (whether it is a fashion item) for a plurality of candidate information and fashion item area information of the candidate information (for example, coordinate information of each vertex, or a center cell). It may be configured to output coordinate information and width/height information). Specifically, the inference session of the previously learned artificial neural network of the candidate output module includes the fashion content image information and smoothing fashion content image information for a specific user who wants to perform user clustering as input information. Fashion item presence information, which is a confidence score for the existence of a fashion item in a specific area, and coordinate information of the corresponding area are output as an output vector. In the training session of the artificial neural network of the candidate output module, a plurality of fashion content image information about a plurality of users previously stored in the database of the social network web server through the social network web server 50 is used as source data. , In the fashion content video information, the area information set for a specific fashion item is used as the ground-truth of artificial neural network learning, and the following loss information is merged to be used as a loss function, and the loss value of the loss function is minimized. The weight of each layer of the artificial neural network is learned.

[Loss 1] The difference between the predicted center coordinates of the area predicted to have a fashion item and the center coordinates of the ground-truth

[Loss 2] The difference between the width/height of the area predicted to have a fashion item and the width/height of the ground-truth

[Loss 3] The difference between the reliability of the area where the fashion item is predicted to exist (information on the presence or absence of fashion items) and the existence of the actual fashion item in the corresponding area of Ground-truth

[Artificial neural network loss function of candidate output module] = loss 1 + loss 2 + loss 3

In addition, in the artificial neural network of the candidate output module according to an embodiment of the present invention, the entire layer may be composed of only a convoulution layer and a pooling layer, and a platen layer or a fully connected layer. Layer may not be configured. Accordingly, since candidate information, which is output information, is configured in three dimensions, no dimensional loss occurs, and a plurality of candidate information is output at a relatively high speed compared to a general convolutional neural network (CNN). For example, the artificial neural network of the candidate output module according to an embodiment of the present invention includes a 3x3 convolution filter and 2x2 max pooling on 12x12x3 input information (fashion content image information or smoothing fashion content image information). A convolution layer that outputs a 5x5x10 vector by applying a filter (Max Pooling Filter), a convolution layer that outputs a 3x3x16 vector by applying a 3x3 convolution filter to a 5x5x10 vector, and a 3x3x16 vector Convolutional layer that outputs a 1x1x32 vector by applying a convolution filter of, and fashion item presence information for a plurality of candidate information output as a 1x1x2 vector by applying a convolution filter to a 1x1x32 vector, and 1x1x4 It may be configured to include fashion item area information of candidate information output as a vector as output information.

The artificial neural network of the candidate output module according to an embodiment of the present invention may use a cross entropy loss as a cost function for learning information about the presence or absence of a fashion item. That is, the weight of the hidden layer of the artificial neural network of the candidate output module is updated so that the cross entropy loss is reduced (optimized) for each candidate information at a specific learning rate. At this time, optimization methods that can be used include a gradient descent method, a momentum method, and the like, and a back propagation algorithm may be used to easily apply gradient descent. In addition, the artificial neural network of the candidate output module according to an embodiment of the present invention can use Euclidean loss as a loss function for coordinate points such as each vertex for learning fashion item area information of each candidate information. have.

The artificial neural network included in the candidate output module according to an embodiment of the present invention may be configured as an artificial neural network including fewer layers than the non-candidate removal module. Accordingly, there is an effect that the output of candidate information proceeds very quickly.

4 is a schematic diagram showing a result of generating candidate information by a candidate output module according to an embodiment of the present invention. As shown in FIG. 4, according to the candidate information generation module according to an embodiment of the present invention, fashion item area information having a relatively small size is generated from fashion content image information, and relatively size is generated from smoothing fashion content image information. It may be configured to generate information about a large fashion item area. According to this, since candidate information is output for each fashion content image information to which a smoothing filter is not applied and a plurality of smoothing fashion content image information to which a smoothing filter is applied, a plurality of different sizes included in one fashion content image information There is an effect of being able to detect all parts of a fashion item. For example, for fashion content image information to which a smoothing filter is not applied, a fashion item portion having the smallest size is detected, and for smoothing fashion content image information to which a smoothing filter is applied, a fashion item portion having a larger size is detected. The effect occurs. This is because the smoothing fashion content video information disappears leaving only features that are relatively larger in scale than the fashion content video information.

In addition, the candidate output module according to an embodiment of the present invention may be configured to remove redundant candidate information, which is redundant candidate information, among the plurality of candidate information output from the artificial neural network. The redundant candidate information removal of the candidate output module may be performed in the following order.

(1) The candidate output module sorts the plurality of candidate information output from the artificial neural network in the order of the highest fashion item presence information (the highest reliability, the highest confidence).

(2) Among candidate information that overlaps with the candidate information having the highest fashion item presence information, candidate information whose ratio of the overlapping area to the total area of the candidate information is more than a certain value is determined to have detected the same fashion item and removed. (For example, if the ratio of the overlapping area to the total area of candidate information is 50% or more, it is removed)

(3) Among the candidate information that was not removed in step 2, step 2 is performed in order of the candidate information with the highest fashion item presence information.

Accordingly, it is possible to prevent duplicated output of candidate information for the fashion content image information or the same fashion item included in the smoothing fashion content image information. This effect generates a chain effect that reduces the computational load of the non-candidate removal module.

The non-candidate removal module according to an embodiment of the present invention is a fashion content image corresponding to the candidate information (fashion item presence information for the candidate information and fashion item area information of the candidate information) output from the candidate output module. It may include a pre-learned artificial neural network that uses image information in the information as input information and non-candidate classification information of the candidate information as output information. In addition, the non-candidate removal module according to an embodiment of the present invention includes candidate information classified as not including a fashion item among candidate information based on the non-candidate classification information of the candidate information (the non-candidate classification information is equal to or greater than a specific value). Candidate information) may be removed to output the remaining candidate information (including fashion item area information, fashion item presence information, and non-candidate classification information) from which the non-candidate candidate has been removed as fashion item image information. 5 is a schematic diagram showing a non-candidate removal module according to an embodiment of the present invention. 5, the non-candidate removal module according to an embodiment of the present invention receives candidate information generated by the candidate output module, and removes candidate information whose reliability of non-candidate classification information is greater than or equal to a specific value among candidate information. Afterwards, it is a module that outputs video information or area information of candidate information remaining without being removed as fashion item video information and transmits it to the fashion item classification module 12. Accordingly, it is possible to remove candidate information classified as not including a fashion item among the candidate information with a high probability.

The fashion item classification module 12 inputs each fashion item image information included in the received fashion content image information into a pre-learned artificial neural network (especially, ConvNet), and stores fashion item classification information for each fashion item image information. It is a module that is configured to output (inference stage of artificial neural network). 6 is a schematic diagram showing an operation relationship of the artificial neural network of the fashion item classification module 12 according to an embodiment of the present invention. As shown in FIG. 6, each fashion item image information included in the fashion content image information received by the fashion item classification module 12 is converted into a pre-learned artificial neural network (especially, ConvNet) of the fashion item classification module 12. It may be configured to input to and output fashion item classification information for each fashion item image information. In the inference stage of the previously learned artificial neural network of the fashion item classification module 12, fashion content image information of a user who wants to perform user clustering is used as input information.

In the learning step, the artificial neural network of the fashion item classification module 12 according to an embodiment of the present invention uses a plurality of fashion content image information of a plurality of users previously stored in the database of the social network web server 50 as source data. The fashion item label information on which fashion item category or fashion item is the fashion item image information is used as the ground-truth of artificial neural network learning, and the following loss information is merged and used as a loss function. The weight of each layer of the artificial neural network is learned to minimize the loss value of the loss function. In this case, the source information (input information) for learning may be a plurality of fashion content image information in a social network, and may include reaction information (like feeling information, comment information, sharing information, etc.) of the fashion content.

[Loss 1] The difference between the fashion item classification information on which label the fashion item video information is classified and the actual fashion item label information on the corresponding fashion item video information of Ground-truth

[Loss 2] The distance between vectors output from the FC layer (Fully Connected Layer) of each artificial neural network for each fashion item video information (if one fashion item is detected in one fashion content video information, the distance is 0 )

[Artificial neural network loss function of fashion item classification module] = loss 1 + loss 2

7 is a schematic diagram showing an example of an artificial neural network of the fashion item classification module 12 according to an embodiment of the present invention. As shown in FIG. 7, the artificial neural network of the fashion item classification module 12 may be constructed as [INPUT-CONV-RELU-POOL-FC]. The fashion item image information, which is input information, may be composed of a matrix having a channel of 32 width, 32 height, and height n, and an input size of [32x32xn]. The CONV layer (Conv. Filter) 101 is connected to a partial area of fashion item image information, and a dot product of the connected area and weight is calculated. The resulting volume will have the same size as [32x32x12]. The RELU layer is an activation function applied to each element, such as max(0,x). The RELU layer does not change the volume size ([32x32x12]). As a result, Activation map 1 (102) is generated. The POOL layer (pooling, 103) performs downsampling on the "horizontal and vertical" dimensions and outputs a reduced volume (Activation map 2, 104) such as [16x16x12]. After the fully-connected (FC) layer 106 connected to the n-th activation map n 105, class scores are calculated, and a volume (output layer, 107) having a size of [m x m x 1] is output. Loss 2 of the loss function of the artificial neural network of the fashion item classification module 12 uses the output vector of the FC layer 106, which is a reduced dimension of fashion item image information, and the FC for each fashion item image information The distance between the output vectors of the layer is composed of the loss value of loss 2.

In particular, the weight of loss 2 may be configured to be proportional to the number of reaction information (like information, comment information, sharing information, etc.) of the corresponding fashion content video information. For example, in the case of fashion content having a lot of response information, loss 2 may have a higher weight than loss 1. That is, in the fashion content having more response information, the distance between the vectors of the FC layer of each artificial neural network with respect to each fashion item image information in the corresponding fashion content image information may be outputted closer.

Regarding the loss 2 of the artificial neural network of the fashion item classification module, for example, when learning the artificial neural network of the fashion item classification module shown in FIG. 6, the artificial neural network and boots for the boot 1 part (fashion item image information) Artificial neural network for part 2, artificial neural network for jeans part, artificial neural network for leather jumper The output vector (embedding vector) of the Fully Connected Layer before the output layer, not each output layer (Boots 1-Boots 2) The artificial neural network is used to minimize the value of the loss function that combines the loss 1 and the loss 2, using the combined value of the distance between embedding vectors, boots 1-the distance between jeans embedding vectors, etc.) You will learn. According to an embodiment of the present invention, fashion items included in one fashion content image information have a distance between the output vectors of the FC layer close to each other, and further, fashion content with more response information is between the output vectors of the FC layer. As the distance of is outputted closer, the effect of being outputted so that matching fashion items have close vector values occurs. According to this, the existing fashion item classification artificial neural network is used to classify the fashion items that match each other when combined with only changing the loss function, without having to tag each fashion item for each fashion item. There is an effect that items can be clustered by style.

The embedding module 13 is a module that is connected to the FC layer of the artificial neural network of the fashion item classification module 12 and receives an output vector of the FC layer as an embedding vector for a corresponding fashion item. For example, as shown in FIG. 6, when there are four fashion item image information in one fashion content image information, the embedding module 13 generates four embedding vectors corresponding to each fashion item image information.

The clustering module 14 generates cluster information by clustering an embedding vector of source data (fashion item image information of at least part of fashion content image information of a social network) previously used for learning, and the embedding module 13 This module maps the embedding vector for each fashion item image information generated in the cluster information. According to the clustering module 14, a plurality of fashion items are detected with respect to at least one fashion content to which a specific user inputs reaction information in a specific social network, and the plurality of fashion items are previously learned through the clustering module 14. Since at least one or more clusters are mapped, at least one or more clusters are mapped for one user, and each cluster represents the style identity of the user, and the number of fashion items included in each cluster is the importance of the style identity ( Cluster importance).

That is, according to the clustering module 14 according to an embodiment of the present invention, the style identity of a specific user, which is a cluster composed of a plurality of fashion items by inputting information on fashion content of interest of a specific user (a plurality of It may be configured to output the fashion item image information). In addition, according to the clustering module 14 according to an embodiment of the present invention, information about fashion contents of interest of each of a plurality of users are input, and a group of users (cluster user information) that groups a plurality of users by style identity is output. Can be configured to Further, according to the clustering module 14 according to an embodiment of the present invention, an individual fashion item or a fashion item of a specific brand is input by the clustering module 14 to output a fashion item corresponding to a cluster such as a corresponding fashion item. It may be configured to input reaction information to a fashion item corresponding to a cluster such as a corresponding fashion item or to output an uploaded user cluster (cluster user information).

8 is a schematic diagram showing embedding vector mapping in a clustering module according to an embodiment of the present invention. As shown in Figure 8, the clustering module 14 according to an embodiment of the present invention may include a plurality of pre-learned cluster information, each of the clusters, such as shoes, tops, bottoms, hats, jewelry, etc. It may include embedding vectors of various categories of fashion items. In addition, the embedding vector of fashion items included in at least one fashion content in which a specific user inputs reaction information in a specific social network may be configured to be mapped to the previously learned clusters, and each cluster to which fashion items are mapped Represents the style identity of the specific user. Using this style identity of a specific user, a recommendation algorithm or an advertisement algorithm can be configured to recommend or advertise other fashion items in the same cluster. Alternatively, an algorithm for recommending creators (based on uploaded fashion content) in the same cluster as the cluster to which the fashion item of a specific brand belongs may be configured. Alternatively, an algorithm for outputting users in the same cluster as the cluster to which a product of a specific brand belongs may be configured in order to compose an advertisement or event targeted to users having similar brand and style identity.

As the clustering algorithm of the clustering module 14 according to an embodiment of the present invention, a general clustering algorithm such as K-means may be used. In addition, in the clustering algorithm according to an embodiment of the present invention, if there are more than a certain number of points within a radius e (epsilon) based on a core point among the embedding vectors for a plurality of fashion item image information, one cluster is formed. It can be configured in a method of forming an arbitrary number of clusters by repeatedly performing the recognition process. Accordingly, there is an effect of automatically finding the number of clusters of fashion item image information without the need to preset the number of clusters, and an effect of preventing deterioration of clustering performance due to outliers is generated. In addition, even when it is not known how many fashion items will appear in the specific fashion content video information, the effect of clustering occurs, and since the FC layer of the fashion item classification module 12 is used as an embedding vector, parameters suitable for clustering are previously The effect of being fixed occurs.

Fashion content embedding method for user clustering

The fashion content embedding method for user clustering according to an embodiment of the present invention may include a fashion content reception step, a fashion item detection step, a fashion item classification step, an embedding step, and a clustering step.

In the fashion content receiving step, the fashion content receiving module 10 includes fashion content information of interest, which is fashion content information that a specific user inputs or uploads response information for a specific period in the user client 100 or the social network web server 50 Fashion content video information, text information, tag information, comment information, like information, view count information, etc.) are received, and the received fashion content video information is transmitted to the fashion item detection module 11.

In the fashion item detection step, the fashion item detection module 11 receives fashion content image information from the fashion content receiving module 10, detects a fashion item area from the fashion content image information, and detects fashion item area information. And generating fashion item image information that is at least a part of fashion content image information based on fashion item area information.

In the fashion item classification step, the fashion item classification module 12 inputs each fashion item image information included in the received fashion content image information into a pre-learned artificial neural network to provide fashion item classification information for each fashion item image information. It is a step that is configured to be output (inference step of the artificial neural network).

In the embedding step, the embedding module 13 is connected to the FC layer of the artificial neural network of the fashion item classification module 12 and receives the output vector of the FC layer as an embedding vector for the fashion item.

In the clustering step, the clustering module 14 generates cluster information by clustering an embedding vector of source data (fashion item image information of at least part of fashion content image information of a social network) previously used for learning, and the embedding module ( This is a step of mapping the embedding vector for each fashion item image information generated in 13) to cluster information.

Apparatus and method for recommending fashion items using user clustering

9 is a schematic diagram illustrating an apparatus for recommending fashion items using user clustering according to an embodiment of the present invention. As shown in Fig. 9, the fashion item recommendation device 2 using user clustering according to an embodiment of the present invention is connected to the fashion content embedding device 1 for user clustering, and the cluster information request module 20 , Purchase item information collection module 21, and item recommendation module 22.

When a specific user enters a specific page (screen, view, layout) of the application module of the user client 100, the cluster information request module 20 generates cluster information for the specific user, which is generated by the clustering module 14. A module that requests in-cluster user information from the clustering module 14 and receives user information of other users included in the cluster including the specific user. 10 is a flowchart of a cluster information request module when a user is introduced according to an embodiment of the present invention. As shown in FIG. 10, when a specific user flows into the application module of the user client 100 according to an embodiment of the present invention, the cluster information request module 20 of the fashion item recommendation device 2 using user clustering In the cluster, the cluster information of the user and the cluster user information on the cluster are requested from the clustering module 14, and user information of other users included in the cluster user information is received from the cluster DB connected to the clustering module 14. According to an embodiment of the present invention, the cluster information request module 20 includes cluster information and cluster information, which are information about user A's clusters a, b, and c generated based on user A's reaction information from the clustering module 14. It may be configured to receive cluster user information, which is information about users A, B, C, and D as users included in a, b, and c.

The clustering module 14 generates at least one cluster information by clustering the embedding vector of fashion item image information of all (or within a specific period) fashion content, and generates cluster user information, which is cluster information to which each user belongs. According to the clustering module 14, a plurality of fashion items are detected with respect to at least one fashion content to which a specific user inputs reaction information in a specific social network, and the plurality of fashion items are previously learned through the clustering module 14. Since at least one or more clusters are mapped, at least one or more clusters are mapped for one user, and each cluster represents the style identity of the user, and the number of fashion items included in each cluster is the importance of the style identity ( Cluster importance). For example, a fashion item to which user A inputs reaction information is mapped to clusters a, b, and c, and fashion items included in each cluster a, b, and c (fashion item image information that user A inputs reaction information The importance of the cluster may be determined according to the number of embedding vectors). According to the clustering module 14 according to an embodiment of the present invention, it is configured to input fashion content information of respective interests of a plurality of users, and to output a user cluster (cluster user information) in which a plurality of users are clustered by style identity. Can be. For example, cluster a may contain users A, B, C, cluster b may contain users A, B, D, E, and cluster c may contain users A and D, and user information of user clusters included in each cluster It can be output as user information.

At this time, the embedding module 13 that generates the embedding vector of fashion item image information is connected to the FC layer of the artificial neural network of the fashion item classification module 12, and receives the output vector of the FC layer as an embedding vector for the fashion item. Is done. The fashion item classification module 12, which is an artificial neural network module, may be configured to use input data as fashion item image information included in specific fashion content image information, and output data to fashion item classification information.

In addition, in the learning session of the fashion item classification module 12, the loss function of the fashion item classification module 12 is calculated for each fashion item image information included in one fashion content image information. A distance between vectors output from a fully connected layer (FC layer) of a neural network may be included, and a loss value of the loss function may be reduced as the distance is closer. In a learning session of the fashion item classification module 12, which is an artificial neural network module, the weight of the fashion item classification module 12 may be updated in a direction in which the loss function is lowered. For example, when a plurality of fashion items are detected in one fashion content image information, the fashion item classification module 12 may be trained so that a distance between embedding vectors for each fashion item is configured within a specific distance. In particular, the specific distance may be configured to be inversely proportional to the number of reaction information (like information, comment information, sharing information, etc.) of the corresponding fashion content image information. For example, each embedding vector for a plurality of fashion items included in fashion content image information with a large number of reaction information such as'like' may be located within a specific distance from each other, and fashion content image information with a small number of reaction information Compared to each embedding vector for the fashion item included in, they may be located closer to each other. That is, in the fashion content having more response information, the distance between the vectors of the FC layer of each artificial neural network with respect to each fashion item image information in the corresponding fashion content image information may be outputted closer. 11 is a schematic diagram illustrating the generation of an embedding vector by the fashion item classification module 12 according to an embodiment of the present invention. As shown in FIG. 11, the fashion item detection module 11 receives fashion content image information, detects fashion items included in the received fashion content image information, and outputs n fashion item image information. The n fashion item image information is input to the fashion item classification module 12, which is an artificial neural network that uses the fashion item image information as input data and the fashion item category information as output data, and the fashion item by the embedding module 13 The FC layer of the classification module 12 outputs an embedding vector for each fashion item image information. At this time, in the learning session of the fashion item classification module 12, the average of each embedding vector of n fashion item image information included in one fashion content image information with respect to all fashion content image information or some fashion content image information. The loss function of the fashion item classification module 12 may be configured such that the distance is configured within a style identity distance k, which is a specific distance calculated according to the reaction information. In particular, when reaction information of the fashion content image information to users is relatively larger than that of other fashion content image information within a specific period or a specific category, the style identity distance k may be configured to decrease. In addition, on the contrary, when reaction information of the fashion content image information for all users is relatively less than that of other fashion content image information within a specific period or a specific category, the specific distance k may be configured to increase. . The loss function of the fashion item classification module 12 of the present invention may be configured as in the following equation.

In Equation 1 above, k is a style identity distance determined according to response information of the fashion content image information, n is the number of fashion item image information included in the fashion content image information, and em _a and em _b are the fashion content image It refers to an embedding vector of fashion item a and fashion item b, which is one of fashion item image information included in the information.

According to an embodiment of the present invention, fashion items included in one fashion content image information have a distance between the output vectors of the FC layer close to each other, and further, fashion content with more response information is between the output vectors of the FC layer. Since the distance of is outputted closer, there is an effect that fashion items matching each other are output to have close vector values. According to this, the existing fashion item classification artificial neural network is used to classify the fashion items that match each other when combined with only changing the loss function, without having to tag each fashion item for each fashion item. There is an effect that items can be clustered by style.

The clustering module 14 collects the same cluster user, which is user information of other users belonging to the cluster user information to which the specific user belongs, at the request of the cluster information request module 20, and sends the cluster information request module 20 Can be configured to transmit. 12 is a schematic diagram showing cluster information and cluster user information generated by the clustering module 14 according to an embodiment of the present invention. In FIG. 12, a gray-filled user image denotes user A, and a gray-filled fashion item image denotes an embedding vector of a fashion item included in fashion content image information to which user A inputs reaction information. As shown in FIG. 12, the cluster information is an embedding vector of each fashion item image information generated based on the reaction information of the user A, which is a specific user targeted for item recommendation, and the fashion content image information inputting the reaction information. This may mean information on mapped clusters (clusters output by clustering embedding vectors of fashion item information for all or at least part of fashion content image information). That is, as shown in FIG. 12, the embedding vector of the fashion item included in the fashion content video information inputted by the user A's response information may be mapped to the clusters a and b, and the cluster information of the user A is Becomes. In addition, the cluster user information may mean information on users corresponding to a corresponding cluster. That is, the cluster user information of the cluster a may include user information of users A, B, C, D, and the like, and the cluster user information of the cluster b may include user information of users A, K, and L.

The purchase item information collection module 21 is a module that collects purchase item information, which is information on fashion items purchased by users belonging to the same cluster user, from a purchase information DB. 13 is a schematic diagram showing the collection of purchase item information by the purchase item information collection module 21 according to an embodiment of the present invention. As shown in FIG. 13, the purchase item information collection module 21 according to an embodiment of the present invention is purchased by users B, C, D, K, L, etc. included in clusters a and b to which user A belongs. It may be configured to receive purchase item information, which is information on fashion items, from a purchase information DB.

The item recommendation module 22 converts the fashion item included in the purchase item information into an application module of the user client 100 of the specific user or other advertisement channels (advertising networks such as banners, video advertisements, offer walls, etc. This module recommends the fashion item to the specific user through an advertisement channel through which advertisements are transmitted through.

The recommendation of the fashion item included in the purchase item information of the item recommendation module 22 may be performed in the following steps.

(1) The item recommendation module 22 receives purchase item image information, which is fashion item image information included in the purchase item information.

(2) The item recommendation module 22 inputs the purchase item image information into the fashion item classification module 12 to output a purchase item embedding vector, which is an embedding vector.

(3) The item recommendation module 22 determines the distance between the center of the cluster containing the purchase item embedding vector and the purchase item embedding vector, and the importance of the cluster (the number of fashion items included in each cluster is the corresponding style identity (cluster)). The recommended item embedding vector based on the importance of

(4) Output fashion item image information corresponding to the recommended item embedding vector to the user client of the specific user

According to this, the purchase conversion rate is increased rather than simply recommending a fashion item corresponding to the same cluster as the user. In addition, compared to CF (Collaborative Filtering), it is possible to recommend items closer to the user's intention. That is, according to an embodiment of the present invention, the advantage of CF (collective intelligence) is included, and the disadvantage of the CF (a problem in which personalized recommendation is difficult) is solved. Furthermore, there is a difference between the fashion contents of ordinary social networks and the fashion items that can be actually purchased.In the existing algorithms such as CF, item recommendations are only made based on the purchase history of fashion items that can be purchased, and because of this difference, the recommendation model There was a limit to its personalized performance. However, since the fashion item recommendation according to an embodiment of the present invention recommends a purchase item based on a reaction to the fashion content of a social network, the personalized recommendation performance is improved compared to the conventional recommendation algorithm such as CF. do.

Regarding the determination of the recommended item embedding vector by the item recommendation module 22, FIG. 14 is a schematic diagram illustrating calculation of a recommended item embedding vector according to an embodiment of the present invention. As shown in FIG. 14, determination of a recommended item embedding vector may be performed in the following steps.

(1) Calculate purchase probability (q) for each cluster information (i) and purchase item embedding vector (j), q _ij = f (distance between j and i centers, ...)

(2) Determine the recommended item embedding vector for each cluster information (i) through binary optimization

(3) Calculate cluster importance

(4) Determine the embedding vector of recommended items to be output to the user client based on the importance of the cluster

The importance of the cluster according to an embodiment of the present invention is the number of embedding vectors of fashion item image information included in the fashion content image information in which the corresponding user inputs reaction information included in each cluster, and the corresponding user for each embedding vector It may be calculated based on at least one or more of a freshness of the time at which the response information is input, and an average distance between embedding vectors included in each cluster.

In particular, according to another embodiment of the present invention, the item recommendation module 22 may further include a cluster importance artificial neural network module that outputs the cluster importance, and the cluster importance artificial neural network module stores input data into the corresponding cluster. At least one or more of the number or ratio of, the latestness of the embedding vectors included in the cluster, and the average distance between the embedding vectors included in the cluster, and the output data may be regarded as the importance of the cluster for the cluster. In addition, in the learning session of the cluster importance artificial neural network module, the ground truth is determined as whether the embedding vector of the fashion item image information that the user inputs reaction information later is mapped to the corresponding cluster {0,1}. A cluster importance artificial neural network module can be trained in the direction of minimizing the difference in ground truth. 15 is a schematic diagram showing a cluster importance calculation according to another embodiment of the present invention. As shown in FIG. 15, the cluster importance artificial neural network module stores input data into the number of embedding vectors of the user included in the cluster (the number of cluster mappings) or the ratio (the occupancy of the cluster), and the embedding of the user included in the cluster. The freshness of the vector (embedding vector freshness, embedding vector time information), the average distance between the embedding vectors of the user included in the cluster (e.g., if the cluster contains embedding vectors a, b, c, , |ab|, |bc|, average of |ac|), and the output data can be set as the cluster importance for the corresponding cluster.

According to an embodiment of the present invention, a recommended item embedding vector to be output to a user client is determined based on the importance of the cluster and the distance to the embedding vector for the fashion item (in which the user inputs reaction information) included in the cluster. Since it is configured to be determined, the loss function of the fashion item classification module 12 allows the user to recommend the fashion item inputted with response information and the purchase item with a high probability of effectively matching one fashion content. .

In addition, according to an embodiment of the present invention, even when the number of fashion contents or fashion items to which a specific user inputs reaction information is not sufficient for training the artificial neural network, it is possible to recommend a purchase item.

Device and method for recommending creators for fashion brands

16 is a schematic diagram illustrating a device for recommending a creator for a fashion brand according to an embodiment of the present invention. As shown in FIG. 16, the device 3 for recommending a creator for a fashion brand according to an embodiment of the present invention is connected to the device 1 for embedding fashion contents for user clustering, and a module 30 for requesting brand cluster information , A creator information collection module 31, and a creator recommendation module 32.

The brand cluster information request module 30 connects the creator according to the request of a specific fashion brand and receives a creator recommendation request from the manager client 300, which is a client of the manager who conducts advertisements such as PPL and plan advertisement of the corresponding fashion brand. , Requesting brand cluster information, which is the cluster information of the fashion brand, to the clustering module 14, and receiving brand cluster information about the cluster to which the brand items, which are items of the fashion brand, belong and cluster user information about the cluster, to the clustering module ( This is a module received from 14). 17 is a flowchart of a cluster information request module when brand cluster information is requested according to an embodiment of the present invention. As shown in FIG. 17, according to an embodiment of the present invention, a specific brand manager or creator manager flows into the browser/application module of the manager client 300, and brand content video information, which is video information about an item of a corresponding fashion brand. When a creator recommendation request is input together, the brand content image information is received by the fashion content embedding device 1 for brand item clustering, the clustering module 14 generates brand cluster information, and the cluster information request module 30 The brand cluster information and cluster user information for the cluster are transmitted from the clustering module 14 to the cluster information requesting module 30 according to the request for brand cluster information of. According to an embodiment of the present invention, the brand cluster information request module 30 includes brand cluster information and cluster a, which are information on clusters a, b, and c of a corresponding fashion brand generated based on the embedding vector of brand content image information. It may be configured to receive cluster user information, which is information about users A, B, C, and D as users included in ,b,c.

The clustering module 14 generates at least one cluster information by clustering the embedding vector of fashion item image information of all (or within a specific period) fashion content, and generates cluster user information, which is cluster information to which each user belongs. In addition, according to the clustering module 14, a plurality of fashion items are detected for at least one fashion content (brand content image information) of a specific fashion brand, and the plurality of fashion items (brand item image information) are previously learned. Since at least one cluster is mapped through the clustering module 14, at least one or more clusters are mapped to one fashion brand (brand cluster information), and each cluster represents the style identity of the corresponding fashion brand, and each cluster The number of brand items included in may be interpreted as indicating the importance of the style identity (cluster importance). For example, brand items of a specific fashion brand are mapped to clusters a, b, and c, and the importance of the cluster may be determined according to the number of brand items included in each cluster a, b, and c. According to the clustering module 14 according to an embodiment of the present invention, it may be configured to output cluster user information, which is information of users included in a cluster of brand cluster information generated by the brand item image information. For example, when the brand cluster information of a specific fashion brand includes clusters a,b,c, cluster a has users A,B,C, cluster b has users A,B,D,E, cluster c has user A, D may be included, and the clustering module 14 may output user information of a user cluster included in each cluster as cluster user information.

At this time, the embedding module 13 that generates the embedding vector of fashion item image information is connected to the FC layer of the artificial neural network of the fashion item classification module 12, and receives the output vector of the FC layer as an embedding vector for the fashion item. Is done. The fashion item classification module 12, which is an artificial neural network module, may be configured to use input data as fashion item image information included in specific fashion content image information, and output data to fashion item classification information.

In addition, in the learning session of the fashion item classification module 12, the loss function of the fashion item classification module 12 is calculated for each fashion item image information included in one fashion content image information. A distance between vectors outputted from a fully connected layer (FC layer) of a neural network may be included, and a loss value of the loss function may be reduced as the distance is closer. In a learning session of the fashion item classification module 12, which is an artificial neural network module, the weight of the fashion item classification module 12 may be updated in a direction in which the loss function is lowered. For example, when a plurality of fashion items are detected in one fashion content image information, the fashion item classification module 12 may be trained so that a distance between embedding vectors for each fashion item is configured within a specific distance. In particular, the specific distance may be configured to be inversely proportional to the number of reaction information (like information, comment information, sharing information, etc.) of the corresponding fashion content image information. For example, each embedding vector for a plurality of fashion items included in fashion content image information with a large number of reaction information such as'like' may be located within a specific distance from each other, and fashion content image information with a small number of reaction information Compared to each embedding vector for the fashion item included in, they may be located closer to each other. That is, in the fashion content having more response information, the distance between the vectors of the FC layer of each artificial neural network with respect to each fashion item image information in the corresponding fashion content image information may be outputted closer. 18 is a schematic diagram showing the generation of an embedding vector for brand content image information by the fashion item classification module 12 according to an embodiment of the present invention. As shown in FIG. 18, the fashion item detection module 11 receives brand content image information, detects brand items included in the received brand content image information, and outputs n number of brand item image information. The n number of brand item image information is input to the fashion item classification module 12, which is an artificial neural network that uses the brand item image information as input data and the fashion item category information as output data, and the fashion item by the embedding module 13 The FC layer of the classification module 12 outputs an embedding vector for each brand item image information.

At this time, in the learning session of the fashion item classification module 12, the average of each embedding vector of n fashion item image information included in one fashion content image information with respect to all fashion content image information or some fashion content image information. The loss function of the fashion item classification module 12 may be configured such that the distance is configured within a style identity distance k, which is a specific distance calculated according to the reaction information. In particular, when reaction information of the fashion content image information to users is relatively larger than that of other fashion content image information within a specific period or a specific category, the style identity distance k may be configured to decrease. In addition, on the contrary, when reaction information of the fashion content image information for all users is relatively less than that of other fashion content image information within a specific period or a specific category, the specific distance k may be configured to increase. .

According to an embodiment of the present invention, fashion items included in one fashion content image information have a distance between the output vectors of the FC layer close to each other, and further, fashion content with more response information is between the output vectors of the FC layer. Since the distance of is outputted closer, there is an effect that fashion items matching each other are output to have close vector values. According to this, the existing fashion item classification artificial neural network is used to classify the fashion items that match each other when combined with only changing the loss function, without having to tag each fashion item for each fashion item. There is an effect that items can be clustered by style.

The clustering module 14 collects user information (same cluster user) of other users belonging to the brand cluster information to which the specific fashion brand belongs at the request of the brand cluster information request module 30, and the brand cluster information request module It can be configured to transmit to 30. 19 is a schematic diagram showing brand cluster information and cluster user information generated by the clustering module 14 according to an embodiment of the present invention. In FIG. 19, a gray-filled fashion item image indicates an embedding vector of a brand item of a corresponding fashion brand. As shown in FIG. 19, the cluster information includes clusters to which the embedding vector of each brand item image information generated based on brand content image information of a specific fashion brand targeted for creator recommendation is mapped (all or at least some fashion This may mean information on a cluster that is output by clustering an embedding vector of fashion item information for content image information. That is, as shown in FIG. 19, the embedding vector of the brand item included in the brand content image information of a specific fashion brand may be mapped to the clusters a and b, and the brand cluster information, which is the cluster information of the corresponding fashion brand, is cluster a, becomes b In addition, the cluster user information may mean information on users corresponding to a corresponding cluster. That is, the cluster user information of the cluster a may include user information of users B, C, D, and the like, and the cluster user information of the cluster b may include user information of users K, L, and the like.

The creator information collection module 31 is a creator (at least, at least), in which users belonging to the same cluster user input reaction information to the content of a specific creator, subscribe to a channel of a specific creator, and watch the content of a specific creator. It is a module that collects creator information, which is information about the influencers who directly or indirectly operate one or more social network channels and have followers for the corresponding channel from the creator information DB. 20 is a schematic diagram of collection of creator information by a creator information collection module according to an embodiment of the present invention. As shown in FIG. 20, the creator information collection module 31 according to an embodiment of the present invention includes users B, C, D, and K included in clusters a, b (brand cluster information) to which a corresponding fashion brand belongs. It may be configured to receive, from the creator information DB, creator information, which is information about a creator to which L or the like (cluster user information) is connected. In particular, the creator information may further include information on the size of the creator channel (the total number of reaction information of the corresponding creator, the number of reaction information for each cluster, the total number of subscribers, the number of subscribers for each cluster, etc.).

The creator recommendation module 32 is a module that recommends the creator information to the manager through the manager client 300.

The creator's recommendation included in the creator information of the creator recommendation module 32 may be performed in the following steps.

(1) The creator recommendation module 32 provides information on the creator channel size, the distance between the center of the cluster including the brand item embedding vector and the distance between the brand item embedding vector, and the cluster importance of the cluster (the number of brand items included in each cluster is Recommended creators are determined based on the importance of the cluster)

(2) Outputting creator information corresponding to the recommended creator to the manager client 300

According to this, when advertising for a specific fashion brand is executed, it is possible to advertise to general consumers (users of the social network, that is, advertisers) through the creator channel that is recognized as producing content that matches the fashion brand. There is an effect that is there. In other words, it is possible to recommend to a fashion brand that is an advertiser a creator who produces content that consumers recognize as similar to the image of the fashion brand, not how much the content produced by the creator matches the fashion brand. The effect of being able to propose an advertisement with reduced image damage and improved advertisement efficiency is generated.

Regarding the determination of the recommended creator by the creator recommendation module 32, FIG. 21 is a schematic diagram illustrating calculation of a recommended creator according to an embodiment of the present invention. As shown in FIG. 21, the recommendation creator may be determined in the following steps.

(1) Calculate advertising efficiency (q) for each brand cluster information (i) and creator information (j), q _ij = f (fashion item embedding vectors reacted by users connected to the center of j and the creator of i) Average distance of, the average distance between the center of j and the brand item embedding vectors, creator channel size information...)

(2) Determine the recommended creator for each brand cluster information (i) through binary optimization

(3) Calculate cluster importance

(4) Determining the recommended creators to be output to the administrator client 300 based on the importance of the cluster

Cluster importance according to an embodiment of the present invention is based on at least one or more of the number of embedding vectors of brand item image information included in each cluster, and the average distance between embedding vectors of brand item image information included in each cluster. Can be calculated as

In particular, according to another embodiment of the present invention, the creator recommendation module 32 may further include a cluster importance artificial neural network module that outputs the cluster importance, and the cluster importance artificial neural network module embeds input data into a corresponding brand cluster. At least one of the number or ratio of vectors and the average distance between embedding vectors included in the corresponding cluster can be used, and the output data can be regarded as the cluster importance for the corresponding cluster.

According to an embodiment of the present invention, an effect of being able to recommend a creator having an audience who responds to fashion items with a high probability of matching effectively with a specific fashion brand is generated.

In addition, according to an embodiment of the present invention, even when the number of fashion content image information and fashion item image information of a specific fashion brand is not sufficient for training the artificial neural network, the creator's recommendation is possible.

In addition, according to an embodiment of the present invention, the fashion item classification module 12 is configured to determine the recommended creator to be output to the manager client based on the importance of the cluster and the embedding vector for the fashion brand included in the cluster. By the loss function of, it is possible to recommend to the manager the creator who is connected with the users who input reaction information to the item with a high probability of matching the brand item of the corresponding fashion brand.

As described above, those skilled in the art to which the present invention pertains will appreciate that the present invention can be implemented in other specific forms without changing the technical spirit or essential features thereof. Therefore, the above-described embodiments are illustrative in all respects and should be understood as non-limiting. The scope of the present invention is indicated by the claims to be described later rather than the detailed description, and all changes or modified forms derived from the meaning and scope of the claims and equivalent concepts should be interpreted as being included in the scope of the present invention.

The features and advantages described herein are not all inclusive, and in particular many additional features and advantages will become apparent to those skilled in the art in view of the drawings, specification, and claims. Moreover, it should be noted that the language used herein has been selected primarily for readability and for teaching purposes, and may not be chosen to describe or limit the subject matter of the invention.

The above description of embodiments of the present invention has been presented for purposes of illustration. It is not intended to be exhaustive or to limit the invention to the precise form disclosed. Those skilled in the art will understand that many modifications and variations are possible in light of the above disclosure.

Therefore, the scope of the invention is not limited by the detailed description, but by any claims in the application on which it is based. Accordingly, the disclosure of the embodiments of the present invention is illustrative and does not limit the scope of the present invention described in the following claims.

1: Fashion content embedding device for user clustering
2: Fashion item recommendation device using user clustering
3: Creator Recommendation Device for Fashion Brands
10: Fashion content receiving module
11: Fashion item detection module
12: Fashion item classification module
13: embedding module
14: clustering module
20: cluster information request module
21: purchase item information collection module
22: Item recommendation module
30: Brand cluster information request module
31: Creator information collection module
32: Creator Recommendation Module
50: social network web server
100: user client
300: admin client

Claims (3)

Receive fashion content information of interest, which is fashion content information uploaded or input by the specific user, to a social network web server, which is a web server of a specific social network service, through a user client of a specific user, and include it in the fashion content information of interest. A fashion content receiving module that generates fashion content image information and response information;
The fashion content receiving module receives fashion content image information, detects an area of a fashion item from the fashion content image information to generate a plurality of fashion item area information, and the fashion item area information A fashion item detection module that generates fashion item image information, which is image information of at least part of fashion content image information, as many as the number of fashion items detected in the fashion content image information;
It is connected to the FC layer (Fully Connected Layer) of the pre-learned fashion item classification artificial neural network, which uses each of the fashion item image information as input data and fashion item classification information as output data, for each fashion item image information. An embedding module for outputting an output vector of the FC layer of the fashion item classification artificial neural network as an embedding vector for each fashion item image information;
A cluster is created by clustering the embedding vectors of the plurality of fashion content information uploaded through the social network web server by a plurality of users, and the interest is mapped to the embedding vector of the fashion content information of interest to the cluster. A cluster user that generates cluster information, which is information about the cluster to which the embedding vector of fashion content information belongs, and is information about another user to which the embedding vector of the fashion content information of interest is mapped to the cluster included in the cluster information A clustering module that generates information;
A purchase item information collection module that collects purchase item information, which is information on purchase items, which are a plurality of fashion items purchased by the other user, included in the cluster user information; And
Receives a purchase item embedding vector output by inputting the purchase item information into the embedding module, and recommends among the purchase item embedding vectors based on the distance between the center of the cluster included in the cluster information and the purchase item embedding vector An item recommendation module for determining a recommended item embedding vector that is an embedding vector of the purchase item to be made;
Including,
The loss function of the fashion item classification artificial neural network includes a function of a distance between the embedding vectors for each of the fashion item image information included in one fashion content information, and the closer the distance is, the more the loss is It is configured to decrease the loss value of the function, and in the learning step of the fashion item classification artificial neural network, the weight of the fashion item classification artificial neural network is learned in a direction in which the loss value decreases,
Outputting the purchase item information corresponding to the recommended item embedding vector to the user client of the specific user as a recommended item,
Fashion item recommendation artificial neural network device using user clustering.
The fashion content receiving module receives fashion content information of interest, which is fashion content information uploaded or input by the specific user, to a social network web server that is a web server of a specific social network service through a user client of a specific user, A fashion content receiving step of generating fashion content image information and reaction information included in the fashion content information of interest;
A fashion item detection module receives fashion content image information from the fashion content receiving module, detects an area of a fashion item from the fashion content image information, and generates a plurality of fashion item area information. A fashion item detection step of generating fashion item image information, which is image information of at least part of the fashion content image information with respect to item area information, as many as the number of fashion items detected in the fashion content image information;
The embedding module is connected to a fully connected layer of a pre-learned fashion item classification artificial neural network that uses each of the fashion item image information as input data and fashion item classification information as output data. An embedding step of outputting an output vector of the FC layer of the fashion item classification artificial neural network for image information as an embedding vector for each fashion item image information;
A clustering module creates a cluster by clustering the embedding vectors of the plurality of fashion content information uploaded through the social network web server by a plurality of users, and provides the embedding vector of the fashion content information of interest to the cluster. By mapping to generate cluster information, which is information about the cluster to which the embedding vector of the fashion content information of interest belongs, and to another user to which the embedding vector of the fashion content information of interest is mapped to the cluster included in the cluster information A clustering step of generating cluster user information, which is information;
A purchase item information collection step of collecting, by the purchase item information collection module, purchase item information, which is information on purchase items, which are a plurality of fashion items purchased by the other user, included in the cluster user information; And
The item recommendation module receives the purchase item embedding vector output by inputting the purchase item information into the embedding module, and the purchase based on the distance between the center of the cluster and the purchase item embedding vector included in the cluster information An item recommendation step of determining a recommended item embedding vector that is an embedding vector of the purchase item to be recommended from among the item embedding vectors;
Including,
The loss function of the fashion item classification artificial neural network includes a function of a distance between the embedding vectors for each of the fashion item image information included in one fashion content information, and the closer the distance is, the more the loss is It is configured to decrease the loss value of the function, and in the learning step of the fashion item classification artificial neural network, the weight of the fashion item classification artificial neural network is learned in a direction in which the loss value decreases,
Outputting the purchase item information corresponding to the recommended item embedding vector to the user client of the specific user as a recommended item,
Fashion item recommendation method using user clustering.
A memory module that stores a fashion item recommendation program code; And
A processing module that processes the fashion item recommendation program code;
Including,
The fashion item recommendation program code,
Receive fashion content information of interest, which is fashion content information uploaded or input by the specific user, to a social network web server, which is a web server of a specific social network service, through a user client of a specific user, and include it in the fashion content information of interest. A fashion content receiving step of generating fashion content image information and reaction information;
A fashion item image that detects an area of a fashion item from the fashion content image information to generate a plurality of fashion item area information, and is image information of at least a part of the fashion content image information for each of the fashion item area information A fashion item detection step of generating information as much as the number of fashion items detected in the fashion content image information;
Pre-learned fashion item classification using each of the fashion item image information as input data and fashion item classification information as output data. The output vector of the fully connected layer of the artificial neural network is used for each fashion item image information. An embedding step of outputting an embedding vector;
A cluster is created by clustering the embedding vectors of the plurality of fashion content information uploaded through the social network web server by a plurality of users, and the interest is mapped to the embedding vector of the fashion content information of interest to the cluster. A cluster user that generates cluster information, which is information about the cluster to which the embedding vector of fashion content information belongs, and is information about another user to which the embedding vector of the fashion content information of interest is mapped to the cluster included in the cluster information A clustering step of generating information;
A purchase item information collection step of collecting purchase item information, which is information on purchase items, which are a plurality of fashion items purchased by the other user, included in the cluster user information; And
A recommended item embedding vector, which is an embedding vector of the purchased item to be recommended, is determined from among the purchase item embedding vectors based on a distance between the center of the cluster included in the cluster information and a purchase item embedding vector that is an embedding vector of the purchase item information An item recommendation step;
Includes program code for executing on a computer,
The loss function of the fashion item classification artificial neural network includes a function of a distance between the embedding vectors for each of the fashion item image information included in one fashion content information, and the closer the distance is, the more the loss is It is configured to decrease the loss value of the function, and in the learning step of the fashion item classification artificial neural network, the weight of the fashion item classification artificial neural network is learned in a direction in which the loss value decreases,
Outputting the purchase item information corresponding to the recommended item embedding vector to the user client of the specific user as a recommended item,
Fashion item recommendation artificial neural network device using user clustering.

Images