KR20220005323A - Apparatus and method for classifying style based on deep learning using fashion attribute - Google Patents

Abstract

Disclosed are an apparatus for classifying styles based on deep learning using fashion attributes, which determines a style group corresponding to an input image by using fashion attributes extracted from the input image, and a method thereof. According to one embodiment of the present invention, the method comprises the following steps: collecting a plurality of first fashion images to which labels for preset fashion attributes are assigned; constructing an attribute extraction model for extracting the fashion attribute from an input image on the basis of the plurality of first fashion images; collecting a plurality of second fashion images to which a label for the fashion attribute is not assigned; acquiring a first feature corresponding to the fashion attribute included in each of the plurality of second fashion images on the basis of the attribute extraction model; constructing a style classification model for classifying an input image into at least one of a plurality of style groups on the basis of the first feature acquired for each of the plurality of second fashion images; and receiving an image to be classified and determining the style group corresponding to the image to be classified by using the style classification model.

Description

Apparatus and method for deep learning-based style classification using fashion attributes {APPARATUS AND METHOD FOR CLASSIFYING STYLE BASED ON DEEP LEARNING USING FASHION ATTRIBUTE}

The present application relates to a deep learning-based style classification apparatus and method using fashion attributes. In particular, the present application relates to a deep learning-based style classification model building technique for determining a style group corresponding to an image by using a fashion attribute extracted from an input image.

In addition to essential functions such as maintaining body temperature and protecting the skin, clothes also have functions as a means of expressing that person. You can protect your body by wearing clothes that can adapt to climate change, and you can express your individuality by wearing clothes that suit you. When the function of clothes in everyday attire is effectively exerted, it is possible to lead a physically comfortable and psychologically satisfactory clothing life.

In this regard, the concept of style related to clothes is an important concept in the fashion domain, but a commonly used standard for quantitative indicators for accurately defining and classifying styles has not been established.

Therefore, until now, it has been common to define or classify styles depending on the experience of experts who have been engaged in the fashion industry for a long time. There was a problem in defining styles or performing style classification with

In other words, even if the concept of each fashion style was defined by experts with abundant experience, it had no choice but to depend on the experience and subjective opinion of each expert rather than being defined based on objective standards.

In this regard, a new methodology is required to define and classify styles by introducing objective criteria such as using quantitative information related to styles in the fashion domain and using computer vision algorithms.

The technology that is the background of the present application is disclosed in Korean Patent Publication No. 10-1913750.

The present application is intended to solve the problems of the prior art described above, and it is possible to extract fashion attributes that can be used as quantitative indicators and objective standards in relation to styles from images, and classify style groups based on the extracted fashion attributes. An object of the present invention is to provide a deep learning-based style classification apparatus and method.

The present application aims to solve the problems of the prior art described above, and to provide a deep learning model for extracting fashion attributes that can objectively function in defining a style and a method for constructing a style classification model using the same.

However, the technical problems to be achieved by the embodiments of the present application are not limited to the technical problems described above, and other technical problems may exist.

As a technical means for achieving the above technical task, the deep learning-based style classification method using fashion attributes according to an embodiment of the present application collects a plurality of first fashion images to which labels for preset fashion attributes are assigned. Step, building an attribute extraction model for extracting the fashion attribute from the input image based on the plurality of first fashion images, collecting a plurality of second fashion images to which a label for the fashion attribute is not assigned , obtaining a first feature corresponding to the fashion attribute included in each of the plurality of second fashion images based on the attribute extraction model; Building a style classification model for classifying an input image into at least one of a plurality of style groups based on the method, receiving a classification target image, and determining the style group corresponding to the classification target image by using the style classification model may include steps.

Also, the fashion attribute may include a color attribute indicating a dominant color of clothes included in the input image and a main attribute indicating a predetermined characteristic reflected in the clothes.

In addition, the building of the attribute extraction model may include constructing the attribute extraction model in which a first extraction process for extracting the color attribute and a second extraction process for extracting the main attribute are separately performed.

In addition, the attribute extraction model performs the first extraction process after removing the background region and the body region so that the color of the background region included in the input image and the color of the body region of the person are not reflected in the color attribute can do.

In addition, the main properties may include clothes type properties (Type of Clothes), parts properties (Garments Parts), textile pattern properties (Textile Pattern), decoration properties (Decorations) and textile finishing properties (Textile Finishing).

In addition, the step of constructing the style classification model includes a data matrix for the first feature through non-negative matrix factorization (NMF), a weight matrix associated with the fashion attribute, and an attribute matrix corresponding to the second fashion image and determining the plurality of style groups by dividing the .

In addition, the deep learning-based style classification method using a fashion attribute according to an embodiment of the present application may include acquiring a second feature associated with an image characteristic for each of the plurality of second fashion images.

In addition, the building of the style classification model may include building the style classification model for classifying the input image into at least one of a plurality of style groups by further considering the second feature.

In addition, the step of constructing the style classification model may include, as a result of classification of the plurality of second fashion images based on the plurality of style groups, the second fashion images included in each of the plurality of style groups are derived. and determining the number of the plurality of style groups based on the similarity between the first features.

Meanwhile, the method for generating a deep learning-based style classification model using fashion attributes according to an embodiment of the present application includes collecting a plurality of first fashion images to which a label for a preset fashion attribute is assigned, the plurality of first fashion images. Building an attribute extraction model for extracting the fashion attribute from an input image based on the image, collecting a plurality of second fashion images to which a label for the fashion attribute is not assigned, based on the attribute extraction model acquiring a first feature corresponding to the fashion attribute included in each of the plurality of second fashion images; The method may include building a style classification model that classifies the style group into at least one of the style groups.

Also, the fashion attribute may include a color attribute indicating a dominant color of clothes included in the input image and a main attribute indicating a predetermined characteristic reflected in the clothes.

In addition, the main properties may include clothes type properties (Type of Clothes), parts properties (Garments Parts), textile pattern properties (Textile Pattern), decoration properties (Decorations) and textile finishing properties (Textile Finishing).

In addition, the step of constructing the style classification model includes a data matrix for the first feature through non-negative matrix factorization (NMF), a weight matrix associated with the fashion attribute, and an attribute matrix corresponding to the second fashion image and determining the plurality of style groups by dividing the .

In addition, the method for generating a deep learning-based style classification model using a fashion attribute according to an embodiment of the present application may include acquiring a second feature associated with an image characteristic for each of the plurality of second fashion images. .

In addition, the building of the style classification model may include building the style classification model for classifying the input image into at least one of a plurality of style groups by further considering the second feature.

On the other hand, the deep learning-based style classification apparatus using a fashion attribute according to an embodiment of the present application collects a plurality of first fashion images to which a label for a preset fashion attribute is given, and based on the plurality of first fashion images an attribute extraction model generating unit that constructs an attribute extraction model that extracts the fashion attribute from the input image, collects a plurality of second fashion images to which a label for the fashion attribute is not assigned, and based on the attribute extraction model A first feature corresponding to the fashion attribute included in each of the plurality of second fashion images is obtained, and an input image is applied to a plurality of styles based on the first feature obtained for each of the plurality of second fashion images. A style classification model generating unit for constructing a style classification model classifying into at least one of the groups and a classification unit receiving a classification target image and determining the style group corresponding to the classification target image by using the style classification model have.

In addition, the style classification model generation unit divides the data matrix for the first feature into a weight matrix associated with the fashion attribute and an attribute matrix corresponding to the second fashion image through non-negative matrix factorization (NMF). The plurality of style groups may be determined.

In addition, the style classification model generation unit acquires a second feature associated with an image characteristic for each of the plurality of second fashion images, and further considers the second feature and converts the input image into at least one of a plurality of style groups. It is possible to build the style classification model to classify.

The above-described problem solving means are merely exemplary, and should not be construed as limiting the present application. In addition to the exemplary embodiments described above, additional embodiments may exist in the drawings and detailed description.

According to the above-described problem solving means of the present application, a deep learning-based method capable of extracting fashion attributes that can be used as quantitative indicators and objective standards in relation to styles from an image, and classifying style groups based on the extracted fashion attributes A style classification apparatus and method may be provided.

According to the above-described problem solving means of the present application, it is possible to provide a deep learning model for extracting a fashion attribute that can objectively function in defining a style and a method for constructing a style classification model using the same.

However, the effects obtainable herein are not limited to the above-described effects, and other effects may exist.

1 is a schematic configuration diagram of a style analysis system 10 according to an embodiment of the present application.
2 is a table showing performance comparison results according to each detailed model type of the attribute extraction model according to an embodiment of the present application.
3 is a conceptual diagram for explaining non-negative matrix factorization (NMF).
4 and 5 are conceptual diagrams for explaining a merge-type style classification model according to an embodiment of the present application.
6 is a chart showing the performance of a style classification model according to an embodiment of the present application compared with a conventional classification model.
7A and 7B are diagrams illustrating fashion attributes and representative images for each of a plurality of style groups generated by a style classification model according to an embodiment of the present application.
8 is a schematic configuration diagram of a deep learning-based style classification apparatus using fashion attributes according to an embodiment of the present application.
9 is an operation flowchart of a method for generating a deep learning-based style classification model using a fashion attribute according to an embodiment of the present application.
10 is an operation flowchart of a deep learning-based style classification method using a fashion attribute according to an embodiment of the present application.

Hereinafter, embodiments of the present application will be described in detail with reference to the accompanying drawings so that those of ordinary skill in the art can easily implement them. However, the present application may be implemented in several different forms and is not limited to the embodiments described herein. And in order to clearly explain the present application in the drawings, parts irrelevant to the description are omitted, and similar reference numerals are attached to similar parts throughout the specification.

Throughout this specification, when a part is "connected" with another part, it is not only "directly connected" but also "electrically connected" or "indirectly connected" with another element interposed therebetween. "Including cases where

Throughout this specification, when it is said that a member is positioned “on”, “on”, “on”, “under”, “under”, or “under” another member, this means that a member is positioned on the other member. It includes not only the case where they are in contact, but also the case where another member exists between two members.

Throughout this specification, when a part "includes" a certain component, it means that other components may be further included, rather than excluding other components, unless otherwise stated.

The present application relates to a deep learning-based style classification apparatus and method using fashion attributes. In particular, the present application relates to a deep learning-based style classification model building technique for determining a style group corresponding to an image by using a fashion attribute extracted from an input image.

1 is a schematic configuration diagram of a style analysis system 10 according to an embodiment of the present application.

Referring to Figure 1, the style analysis system 10 according to an embodiment of the present application, a deep learning-based style classification apparatus 100 using a fashion attribute according to an embodiment of the present application (hereinafter, style classification apparatus 100) ') and the user terminal 200 .

The style classification apparatus 100 and the user terminal 200 may communicate with each other through the network 20 . The network 20 refers to a connection structure in which information exchange is possible between each node, such as terminals and servers, and an example of such a network 20 includes a 3rd Generation Partnership Project (3GPP) network, a long-term LTE (LTE) network. Term Evolution) network, 5G network, WIMAX (World Interoperability for Microwave Access) network, Internet, LAN (Local Area Network), Wireless LAN (Wireless Local Area Network), WAN (Wide Area Network), PAN (Personal Area) Network), a wifi network, a Bluetooth network, a satellite broadcasting network, an analog broadcasting network, a Digital Multimedia Broadcasting (DMB) network, etc. are included, but are not limited thereto.

The user terminal 200 is, for example, a smartphone (Smartphone), a smart pad (SmartPad), a tablet PC and the like and PCS (Personal Communication System), GSM (Global System for Mobile communication), PDC (Personal Digital Cellular), PHS ( Personal Handyphone System), PDA (Personal Digital Assistant), IMT (International Mobile Telecommunication)-2000, CDMA (Code Division Multiple Access)-2000, W-CDMA (W-Code Division Multiple Access), Wibro (Wireless Broadband Internet) terminals The same may be any type of wireless communication device.

The style classification apparatus 100 may collect a plurality of first fashion images to which labels for preset fashion attributes are assigned. For example, a plurality of first fashion images to which a label is assigned are secured in advance in a state in which a label indicating a predefined fashion attribute, which will be described later, is displayed in an area (position) where the corresponding fashion attribute is well exposed in the image. It may mean a trained training data set. For example, the label for the fashion attribute may be displayed in the form of a bounding box. Specifically, according to an embodiment of the present application, the first fashion image may be an image to which a label corresponding to a color attribute is not given among detailed types of fashion attributes to be described later and a label for a fashion attribute corresponding to the main attribute is assigned. have.

In the description of the embodiments of the present application, fashion attributes refer to individual elements used in fashion trend analysis for clothes included in the input image, and are quantitative indicators for determining the style of the input image. It can be understood as an element that can function as

According to an embodiment of the present application, the fashion attribute may include a color attribute indicating a dominant color of clothes included in the input image and a main attribute indicating a predetermined fashion-related characteristic reflected in the clothes included in the input image. In addition, the main properties may include clothes type properties (Type of Clothes), part properties (Garments Parts), textile pattern properties (Textile Pattern), decoration properties (Decorations), and textile finishing properties (Textile Finishing). According to an embodiment of the present application, the fashion attribute may be extracted as 146 including 132 main attributes and 14 color attributes, but is not limited thereto. In particular, styles or fashion trends change with the passage of time. Since it is a relative concept, each type and the number of extracted attributes may be determined differently according to embodiments of the present application.

Specifically, to explain with an example, the main properties corresponding to the Type of Clothes are Outer, Jacket, Top, Pants, Skirt, and Dress. ), etc., may include a fashion attribute that means a category of clothing types appearing in the input image. Also, referring to FIGS. 7A and 7B , which will be described later, the clothing type attribute may include a subdivided fashion attribute even for the same clothing type category. For example, the clothing type attribute corresponding to the jacket may include lower fashion attributes such as a biker jacket (Jacket_Biker) and a blazer jacket (Jacket_Blazer).

In addition, the main attribute corresponding to Garments Parts represents, by way of example, a partial attribute of clothes included in an image. Associated fashion attributes (Turtle_neck, Deep_V_neck, etc.) may be included as an example. Also, the partial attribute may exemplarily include fashion attributes (Long_sleeve, Bell_sleeve, etc.) associated with the characteristic of the sleeve part.

In addition, the main property corresponding to the textile pattern property (Textile Pattern) represents, by way of example, a design pattern, texture, or surface characteristics of clothes included in the image, and with reference to FIGS. 7A and 7B to be described later, Floral, Baroque, It may include fashion attributes expressed as Applique, Solid, Dot, Checked, Washed, Glitter, and the like.

In addition, the main attribute corresponding to the decorations represents, by way of example, the characteristics of decorations additionally attached to clothes included in the image or provided as additional elements in the clothes, which will be described later in FIGS. 7A and 7B . Referring to , it may include fashion attributes related to whether or not a belt is worn, additional elements of clothes such as a pocket, and the presence or absence of decorative accessories.

In addition, the main property corresponding to the textile finishing property is illustratively, and referring to FIGS. 7A and 7B to be described later, Rivet, Ruffle, Pleat, Crochet, Lapel, etc. are identified from the clothes included in the input image. It may include a fashion attribute corresponding to the finishing properties of the fabric.

Each of the subtype categories of the above-mentioned main properties, such as Type of Clothes, Garments Parts, Textile Pattern, Decorations, and Textile Finishing, is a typical It may be set to correspond to each of the six-step processes considered in the fashion design process or the apparel design process. However, the scope of the above-described main attribute may be variously changed according to the embodiment of the present application.

In addition, the style classification apparatus 100 is an attribute extraction model that extracts fashion attributes from the input image when a predetermined image is input based _{on a plurality of collected first fashion images I 1 (refer to FIG. 1 ,} Attribute Detection, M ₁ ) can be built.

According to an embodiment of the present application, the style classification apparatus 100 separates and performs a first extraction process for extracting a color attribute from an input image and a second extraction process for extracting a main attribute (M ₁ ) can be built In addition, according to an embodiment of the present application, in the attribute extraction model M ₁ , the color of the background region included in the input image and the color of the body region of the person (eg, the skin color of the person, etc.) are reflected in the color attribute It may be implemented to perform the first extraction process after removing the background region and the body region from the input image so as not to do so.

_{Specifically, the attribute extraction model M 1} constructed by the style classification apparatus 100 may remove a background region and a body region based on a semantic segmentation technique. Illustratively, the attribute extraction model (M ₁ ) may include an artificial intelligence model based on Mask R-CNN trained based on a data set (eg, COCO data set) secured in advance for semantic domain segmentation. have.

In addition, the attribute extraction model M ₁ built by the style classification apparatus 100 determines the color attribute representing the dominant color of the clothes included in the image from which the background region and the body region are removed based on a preset color extraction algorithm. A first extraction process of extracting may be performed. For example, the color extraction algorithm possessed by the attribute extraction model (M ₁ ) may be linked to the GitHub color extractor, but is not limited thereto, and various color extraction algorithms previously known or developed in the future may be applied. .

2 is a table showing performance comparison results according to each detailed model type of the attribute extraction model according to an embodiment of the present application.

Referring to FIG. 2 , the style classification apparatus 100 may build an _{attribute extraction model M 1 based on a model type of Faster RCNN or RetinaNet.} In addition, referring to FIG. 2 , the attribute extraction model (M ₁ ) may be constructed using ResNET-50 or ResNet-101 as a backbone, but is not limited thereto.

Also, Referring to Figure 2, ResNet-101 to RetinaNet based attribute extraction model for the backbone (Backbone) (M ₁₎ of different model types and the attribute extraction model generated based on a combination of the backbone (M ₁₎ against the precision It can be seen that it shows excellent performance in terms of (Precision), recall (Recall), and F-1 score.

Also, the style classification apparatus 100 may collect _{a plurality of second fashion images I 2} to which a label for the above-described fashion attribute is not applied. Illustratively, it may be desirable to collect the second fashion image I ₂ to include a larger number of images compared to the above-described first fashion image I ₁ , but the present invention is not limited thereto.

According to an embodiment of the present application, the second fashion image may be a runway image collected for a predetermined period. Illustratively, the second fashion image may include 302,772 runway images collected from the 2010 S/S season to the 2019 F/W season, but is not limited thereto.

In addition, the style classification apparatus 100 may obtain a first feature corresponding to a fashion attribute included in each of the plurality of second fashion images I _{2 collected based on the previously constructed attribute extraction model M 1 .} can In other words, the style classification apparatus 100 may extract the fashion attribute, which is the first feature, from the second fashion image I _{2 for each image by using the attribute extraction model M 1 .}

In addition, the style classification apparatus 100 classifies the input image into at least one of a plurality of style groups when at least one image is input based on the first feature obtained for each of the _{plurality of second fashion images I 2 .} A style classification model (refer to FIG. 1 , Style Classification, M ₂ ) may be built.

According to an embodiment of the present application, the style classification apparatus 100 divides the data matrix for the first feature into a weight matrix associated with a fashion attribute and an attribute matrix corresponding to the second fashion image NMF (Non-negative Matrix Factorization) ) to determine multiple style groups.

3 is a conceptual diagram for explaining non-negative matrix factorization (NMF).

Referring to FIG. 3 , NMF in the present application is a non-negative matrix decomposition technique used in an unsupervised learning algorithm, and refers to a matrix parameter-based partitioning technique that expresses the entire data as a product of a weight matrix and an attribute matrix. It can be utilized to search for useful patterns (eg, distinct characteristics of each style group herein). Such NMF may be expressed by Equation 1 below.

[Equation 1]

Here, V denotes the entire data described above, W denotes a weight matrix, and H denotes an attribute matrix, respectively. According to an embodiment of the present application, the style classification apparatus 100 performs the first feature through NMF. It is possible to divide the data matrix V for V into a weight matrix W associated with fashion attributes and an attribute matrix H corresponding to the second fashion image. In this regard, the main fashion attribute of the style group may be confirmed through the weight matrix, and information on the image (the second fashion image) grouped into the corresponding style group may be confirmed through the attribute matrix.

In addition, according to an embodiment of the present application, the style classification apparatus 100 classifies the plurality of second fashion images I ₂ based on the plurality of style groups, and as a result, the second fashion image included in each of the plurality of style groups. The number of a plurality of style groups can be determined based on the adequacy of (I ₂ ) (eg, the degree to which fashion attributes are clearly distinguished between different style groups, the degree of similarity between fashion images classified into the same style group, etc.) have. Exemplarily, the style classification apparatus 100 determines the number of the plurality of style groups based on the similarity in the style group between the first features derived with respect _{to the second fashion image I 2 included in each of the plurality of style groups.} can decide According to an embodiment of the present disclosure, the plurality of style groups may include any one of 25 to 30 style groups, but is not limited thereto.

Also, the style classification apparatus 100 may receive a classification target image. According to an embodiment of the present application, the style classification apparatus 100 may receive a classification target image from the user terminal 200 . As an example to help understanding, the user of the user terminal 200 applies (provides) a specific fashion image to check a style, an image including clothes worn by the user himself, to the style classification apparatus 100 (provides), and the corresponding image It is possible to obtain a style group classification result for .

Also, the style classification apparatus 100 may determine a style group corresponding to the received classification target image based on the _{previously constructed style classification model M 2 .} In addition, the style classification apparatus 100 provides information on the style group corresponding to the determined classification target image (eg, identification information on the style group, detailed information for describing the determined style group, and the style group included in the style group). example image, etc.) may be provided (transmitted) to the user terminal 200 .

Specifically, the style classification apparatus 100 acquires the first feature corresponding to the fashion attribute reflected in the classification target image based on the _{attribute extraction model (M 1 ) from the received classification target image, and the style classification model (M 2} ) It is possible to determine a style group corresponding to the first feature obtained through .

Hereinafter, with reference to FIGS. 4 and 5 , a second feature linked with an image characteristic (a characteristic of the image itself) that can be extracted from an image is learned together with the first feature corresponding to the above-described fashion attribute (Merged) The style classification model (M _{2 ) of the type is described, and the style classification model (M 2} ) that performs style classification based on the first feature described above with reference to FIG. 6 and the merged type style classification model ( Let _{'s compare the performance of M 2} ') with other conventional classification techniques.

4 and 5 are conceptual diagrams for explaining a merge-type style classification model according to an embodiment of the present application.

4 and 5 , the style classification apparatus 100 may acquire a second feature associated with an image characteristic of each of a plurality of second fashion images. In addition, the style classification apparatus 100 considers the acquired second feature _{together with the first feature acquired by the attribute extraction model M 1} to classify the input image into at least one of a plurality of style groups. A style classification model (M ₂ ') of the type can be built.

In particular, referring to FIG. 5 , the merged type style classification model (M ₂ ') according to an embodiment of the present application extracts 2,048 second features based on Fine-tuned Resnet-50, and builds previously A style group for each of a plurality of second fashion images by comprehensively considering 146 first features extracted based on the obtained attribute extraction model (M _{1 ) and transformed based on one-hot encoding through logistic regression} classification can be performed.

6 is a chart showing the performance of a style classification model according to an embodiment of the present application compared with a conventional classification model. In particular, the Our model (Merged) row of FIG. 6 shows the performance of the merged type logistic regression-based style classification model (M ₂ ') disclosed herein, and the Our model (Attribute) row of FIG. 6 shows the performance of the present application. It may indicate the performance of the style classification model (M _{2 ) based on the fashion attribute disclosed in .} In addition, the performance evaluation comparison simulation of the style classification model shown in FIG. 6 may be performed using the conventional FashionStyle 14 data set.

_{6, the performance of the style classification model (M 2} ) based on the fashion attribute according to an embodiment of the present application is improved compared to the conventional classification models such as VGG-19, Xception, Inception v3, VGG-16, , it can be seen that the performance of the merged-type style classification model (M ₂ ') is improved compared to that of the conventional VGG-19, Xception, Inception v3, VGG-16 and ResNet-50.

In this regard, ResNet-50 does not learn the characteristics of fashion attributes in this institute that experts such as designers reflected in the image consider in fashion trend analysis, design process, etc., and a style classification model (M _{2 ) based on fashion attributes.} ) is the classification performance of _{a merge-type style classification model (M 2} ') that learns both the characteristics of the fashion attribute and the attributes of the image itself in that it does not learn the attributes of the image itself, such as the texture characteristics of the image. You can see this is the best. Specifically, it can be seen that the merged-type style classification model (M ₂ ') shows an average classification accuracy of 78%, which improves the performance in terms of classification accuracy of 6%p compared to the conventional ResNet-50.

7A and 7B are diagrams illustrating fashion attributes and representative images for each of a plurality of style groups generated by a style classification model according to an embodiment of the present application.

7A and 7B , the style classification apparatus 100 may operate to perform style classification on an input image based on a plurality of 25 style groups including Group 1 to Group 25 . In particular, FIGS. 7A and 7B may exemplarily show main fashion attributes for each style group and fashion images classified into the corresponding style group.

Also, referring to FIGS. 7A and 7B , the style classification apparatus 100 may determine a style name of each style group based on a fashion attribute corresponding to each style group. According to an embodiment of the present disclosure, the style name of each style group may be determined as a mixed style name including an expression corresponding to each of at least two or more fashion attributes. Exemplarily, the mixed style name may be Autumn Minimal, Floral Chic, and the like. In this regard, considering the ambiguity of the fashion field in which it is difficult to define a style strictly, the mixed style name has an advantage in that it can reflect the needs of experts such as fashion designers and the diversity of fashion styles.

8 is a schematic configuration diagram of a deep learning-based style classification apparatus using fashion attributes according to an embodiment of the present application.

Referring to FIG. 8 , the style classification apparatus 100 may include an attribute extraction model generation unit 110 , a style classification model generation unit 120 , and a classification unit 130 .

The attribute extraction model generation unit 110 may collect _{a plurality of first fashion images I 1} to which a label for a preset fashion attribute is assigned. Also, the attribute extraction model generator 110 may build _{an attribute extraction model M 1} for extracting fashion attributes from the input image based _{on the plurality of collected first fashion images I 1 .}

The style classification model generator 120 may collect _{a plurality of second fashion images I 2} to which a label for a fashion attribute is not applied. Also, the style classification model generation unit 120 may acquire a first feature corresponding to a fashion attribute included in each of the plurality of second fashion images I _{2 collected based on the attribute extraction model M 1 .} have. In addition, the style classification model generation unit 120 classifies the input image into at least one of a plurality of style groups based on the first feature obtained for each of the _{plurality of second fashion images I 2 , the style classification model M 2} ) can be built.

Also, the style classification model generator 120 may acquire a second feature associated with an image characteristic of each of the plurality of second fashion images. Also, the style classification model generator 120 classifies the input image into at least one of a plurality of style groups by considering the first feature corresponding to the fashion attribute and the acquired second feature together. A classification model (M ₂ ') can be built.

The classification unit 130 may receive a classification target image. Also, the classifier 130 may determine a style group corresponding to the classification target image by using the _{style classification model M 2} or the merged type style classification model M _{2 ′.}

Hereinafter, an operation flow of the present application will be briefly reviewed based on the details described above.

9 is an operation flowchart of a method for generating a deep learning-based style classification model using a fashion attribute according to an embodiment of the present application.

The method for generating a deep learning-based style classification model using the fashion attribute shown in FIG. 9 may be performed by the style classification apparatus 100 described above. Accordingly, even if omitted below, the description of the style classification apparatus 100 may be equally applied to the description of the deep learning-based style classification model generation method using the fashion attribute.

Referring to FIG. 9 , in step S11 , the attribute extraction model generation unit 110 may collect _{a plurality of first fashion images I 1 to which a label for a preset fashion attribute is assigned.}

Next, in step S12 , the attribute extraction model generating unit 110 may build _{an attribute extraction model M 1} for extracting fashion attributes from the input image based on the _{plurality of first fashion images I 1 .} have.

Next, in step S13 , the style classification model generation unit 120 may collect _{a plurality of second fashion images I 2 to which a label for a fashion attribute is not assigned.}

Next, in step S14 , the style classification model generating unit 120 , _{on the basis of the attribute extraction model M 1} , a first feature corresponding to the fashion attribute included in each of the plurality of second fashion images I _{2 .} can be obtained.

Next, in step S15 , the style classification model generator 120 may acquire a second feature associated with an image characteristic of each of the _{plurality of second fashion images I 2 .}

Next, in step S16 , the style classification model generating unit 120 generates a plurality of input images based on at least one of the first and second features acquired for each of the _{plurality of second fashion images I 2 .} A style classification model that classifies at least one of the style groups of

Specifically, in step S16, the style classification model generation unit 120 classifies the input image into at least one of a plurality of style groups based on the first feature corresponding to the fashion attribute (M ₂ ) or the first feature. _{A merged type style classification model M 2} ′ that classifies an input image into at least one of a plurality of style groups may be constructed by considering the and second features together.

In the above description, steps S11 to S16 may be further divided into additional steps or combined into fewer steps, according to an embodiment of the present application. In addition, some steps may be omitted if necessary, and the order between steps may be changed.

10 is an operation flowchart of a deep learning-based style classification method using a fashion attribute according to an embodiment of the present application.

The deep learning-based style classification method using the fashion attribute shown in FIG. 10 may be performed by the style classification apparatus 100 described above. Accordingly, even if omitted below, the description of the style classification apparatus 100 may be equally applied to the description of the deep learning-based style classification method using the fashion attribute.

In particular, steps S21 to S26 of the deep learning-based style classification method shown in FIG. 10 correspond to steps S11 to S16 in the deep learning-based style classification model generation method using the fashion attribute described above with reference to FIG. 9, respectively. it could be

Referring to FIG. 10 , in step S21 , the attribute extraction model generating unit 110 may collect _{a plurality of first fashion images I 1 to which a label for a preset fashion attribute is assigned.}

Next, in step S22, the attribute extraction model generation unit 110 may build _{an attribute extraction model M 1} for extracting fashion attributes from the input image based on the _{plurality of first fashion images I 1 .} have.

Next, in step S23 , the style classification model generator 120 may collect _{a plurality of second fashion images I 2 to which a label for a fashion attribute is not assigned.}

Next, in step S24 , the style classification model generation unit 120 _{, based on the attribute extraction model M 1} , performs a first feature corresponding to the fashion attribute included in each of the plurality of second fashion images I _{2 .} can be obtained.

Next, in step S25 , the style classification model generator 120 may acquire a second feature associated with an image characteristic of each of the _{plurality of second fashion images I 2 .}

Next, in step S26 , the style classification model generating unit 120 generates a plurality of input images based on at least one of the first and second features acquired for each of the _{plurality of second fashion images I 2 .} A style classification model that classifies at least one of the style groups of

Next, in step S27 , the classification unit 130 may receive a classification target image.

Next, in step S28 , the classification unit 130 may determine a style group corresponding to the received classification target image using the _{constructed style classification model (eg, M 2} or M _{2 ′).}

In the above description, steps S21 to S28 may be further divided into additional steps or combined into fewer steps, according to an embodiment of the present application. In addition, some steps may be omitted if necessary, and the order between steps may be changed.

The deep learning-based style classification model generation method using fashion attributes and the deep learning-based style classification method using fashion attributes according to an embodiment of the present application are implemented in the form of program instructions that can be performed through various computer means, and are computer-readable media can be recorded in The computer-readable medium may include program instructions, data files, data structures, etc. alone or in combination. The program instructions recorded on the medium may be specially designed and configured for the present invention, or may be known and available to those skilled in the art of computer software. Examples of the computer-readable recording medium include magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic such as floppy disks. - includes magneto-optical media, and hardware devices specially configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine language codes such as those generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the present invention, and vice versa.

In addition, the deep learning-based style classification model generation method using the above-described fashion attribute and the deep learning-based style classification method using the fashion attribute may be implemented in the form of a computer program or application executed by a computer stored in a recording medium. .

The foregoing description of the present application is for illustration, and those of ordinary skill in the art to which the present application pertains will understand that it can be easily modified into other specific forms without changing the technical spirit or essential features of the present application. Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive. For example, each component described as a single type may be implemented in a dispersed form, and likewise components described as distributed may be implemented in a combined form.

The scope of the present application is indicated by the following claims rather than the above detailed description, and all changes or modifications derived from the meaning and scope of the claims and their equivalent concepts should be construed as being included in the scope of the present application.

10: Style classification system
100: Deep learning-based style classification device using fashion attributes
110: attribute extraction model generation unit
120: style classification model generation unit
130: classification unit
200: user terminal
20: network

Claims (15)

In a deep learning-based style classification method using fashion attributes,
collecting a plurality of first fashion images to which labels for preset fashion attributes are assigned;
constructing an attribute extraction model for extracting the fashion attribute from an input image based on the plurality of first fashion images;
collecting a plurality of second fashion images to which a label for the fashion attribute is not applied;
obtaining a first feature corresponding to the fashion attribute included in each of the plurality of second fashion images based on the attribute extraction model;
constructing a style classification model for classifying an input image into at least one of a plurality of style groups based on the first feature obtained for each of the plurality of second fashion images; and
receiving a classification target image, and determining the style group corresponding to the classification target image by using the style classification model;
A style classification method, including. According to claim 1,
The fashion attribute is,
a color property representing a dominant color of the clothes included in the input image and a main property representing a predetermined characteristic reflected in the clothes,
Building the attribute extraction model comprises:
A style classification method of constructing the attribute extraction model that is performed separately from a first extraction process for extracting the color attribute and a second extraction process for extracting the main attribute. 3. The method of claim 2,
The attribute extraction model is
and performing the first extraction process after removing the background region and the body region so that the color of the background region and the color of the body region of the person included in the input image are not reflected in the color attribute. 3. The method of claim 2,
The main property is
A method of classifying a style, which includes Type of Clothes, Garments Parts, Textile Pattern, Decorations, and Textile Finishing. According to claim 1,
The step of constructing the style classification model is,
Determining the plurality of style groups by dividing the data matrix for the first feature into a weight matrix associated with the fashion attribute and an attribute matrix corresponding to the second fashion image through non-negative matrix factorization (NMF); ,
A method of classifying a style that includes. 6. The method of claim 5,
obtaining a second feature associated with an image characteristic for each of the plurality of second fashion images;
further comprising,
The step of constructing the style classification model is,
The style classification method of constructing the style classification model that classifies the input image into at least one of a plurality of style groups by further considering the second feature. 7. The method of claim 6,
The step of constructing the style classification model is,
As a result of classification of the plurality of second fashion images based on the plurality of style groups, based on the similarity between the first features derived with respect to the second fashion image included in each of the plurality of style groups, the plurality of determining the number of style groups;
A method of classifying a style that includes. In a deep learning-based style classification model creation method using fashion attributes,
collecting a plurality of first fashion images to which labels for preset fashion attributes are assigned;
constructing an attribute extraction model for extracting the fashion attribute from an input image based on the plurality of first fashion images;
collecting a plurality of second fashion images to which a label for the fashion attribute is not assigned;
obtaining a first feature corresponding to the fashion attribute included in each of the plurality of second fashion images based on the attribute extraction model; and
constructing a style classification model for classifying an input image into at least one of a plurality of style groups based on the first feature obtained for each of the plurality of second fashion images;
Including, a style classification model generation method. 9. The method of claim 8,
The fashion attribute is,
a color property representing a dominant color of the clothes included in the input image and a main property representing a predetermined characteristic reflected in the clothes,
The main property is,
A method of creating a style classification model, comprising: Type of Clothes, Garments Parts, Textile Pattern, Decorations, and Textile Finishing. 10. The method of claim 9,
The step of constructing the style classification model is,
Determining the plurality of style groups by dividing the data matrix for the first feature into a weight matrix associated with the fashion attribute and an attribute matrix corresponding to the second fashion image through non-negative matrix factorization (NMF); ,
A method for generating a style classification model, comprising: 11. The method of claim 10,
obtaining a second feature associated with an image characteristic for each of the plurality of second fashion images;
further comprising,
The step of constructing the style classification model is,
The method of constructing the style classification model that classifies the input image into at least one of a plurality of style groups by further considering the second feature. In a deep learning-based style classification device using fashion attributes,
Attribute extraction model creation for constructing an attribute extraction model that collects a plurality of first fashion images to which a label for a preset fashion attribute is given, and extracts the fashion attribute from an input image based on the plurality of first fashion images wealth;
Collecting a plurality of second fashion images to which a label for the fashion attribute is not assigned, and obtaining a first feature corresponding to the fashion attribute included in each of the plurality of second fashion images based on the attribute extraction model, , a style classification model generator for constructing a style classification model for classifying an input image into at least one of a plurality of style groups based on the first feature obtained for each of the plurality of second fashion images; and
a classification unit for receiving a classification target image and determining the style group corresponding to the classification target image by using the style classification model;
Including, style classification device. 13. The method of claim 12,
The fashion attribute is,
a color property representing a dominant color of the clothes included in the input image and a main property representing a predetermined characteristic reflected in the clothes,
The main property is,
A style classification device, comprising: Type of Clothes, Garments Parts, Textile Pattern, Decorations, and Textile Finishing. 14. The method of claim 13,
The style classification model generation unit,
Determining the plurality of style groups by dividing the data matrix for the first feature into a weight matrix associated with the fashion attribute and an attribute matrix corresponding to the second fashion image through non-negative matrix factorization (NMF) In, style classification device. 15. The method of claim 14,
The style classification model generation unit,
Obtaining a second feature associated with an image characteristic for each of the plurality of second fashion images, and constructing the style classification model that classifies the input image into at least one of a plurality of style groups by further considering the second feature that is, a style classification device.

Images