KR20230116307A - Method, system and non-transitory computer-readable recording medium for providing style recommendation service - Google Patents

Abstract

The present disclosure relates to a method, system, and non-transitory computer readable recording medium for providing a style recommendation service. According to one aspect of the present disclosure, a method of providing a style recommendation service includes receiving image data including a query product, determining a target category using a style recommendation model, and associated with the query product and corresponding to the target category. A method comprising selecting at least one target product and providing information on the at least one target product is provided. Here, the style recommendation model is learning style coordination recommendation data, and the style coordination recommendation data is generated by detecting a plurality of items included in fashion image data without a separate labeling process for style information.

Description

Method, system and non-temporary computer readable recording medium for providing style recommendation service

The present disclosure relates to a method, system, and non-temporary computer readable recording medium for providing a style recommendation service, and more particularly, using a style recommendation model that learns data about correlations between items detected from fashion image data. Thus, it relates to a method, system, and non-transitory computer readable recording medium for providing product information related to a query product.

The share of the online market in the total retail sales market is rapidly increasing in recent years. Accordingly, attempts to implement a service in which a seller recommends a product to a consumer are actively being made in order to promote product sales in the online market.

For example, in Laid-Open Patent Publication No. 10-2021-0123797, clothing is recommended using an artificial intelligence model based on user body type information, while influencer coordination with a body type or taste similar to that of the user is recommended. A method of recommending clothing with reference is disclosed. In the prior art, there is disclosed a technique of providing recommendation information on one completed costume set in which costume parts are combined based on input information.

In addition, according to the technologies introduced so far, when one clothing product (eg, top) is selected from an Internet shopping site, the other products (eg, Bottoms, outerwear, accessories, etc.), a method of recommending a product for a completed set, or a method of recommending other products similar to the selected product are used.

However, according to this prior art, there is a limit to providing a service that recommends products that match coordination when combined with a product selected by the consumer, in addition to the coordination pre-determined by the seller. That is, according to the prior art, there is a lack of technology for recommending products of different categories to complete one coordination by considering the styles of multiple products regardless of the number of products selected by the consumer, and the user's body shape, taste, and external influence. There is a limit to providing a generalized style recommendation service, such as making a recommendation by using written information together.

In order to generalize the style recommendation service and provide recommended product information according to the product combination selected by the consumer, a large amount of style coordination recommendation data for training the style recommendation model should be secured. However, in order to generate style coordination recommendation data, it is necessary for fashion experts to judge whether each item combination is a suitable combination, requiring a lot of time and cost compared to simple tasks such as image classification. Data is limited in quantity, and there is a problem that the nature or distribution of data is biased toward a specific style depending on the data collection market or labeling method.

Accordingly, the present inventor secures learning data for style coordination recommendation by utilizing the influencer's fashion-related image and market product image, and enables the style recommendation model to learn style characteristics for various style combinations to obtain a product selected by the consumer. We propose a technology that provides a service that recommends other products that can be combined to match.

Publication No. 10-2021-0123797 (2021. 10. 14)

The present disclosure aims to solve all of the problems of the prior art described above.

In addition, the present disclosure aims to detect fashion-related items from fashion image data, secure a large amount of style coordination recommendation data at a low cost, and learn this by a style recommendation model to provide information on products related to a query product. .

In addition, another object of the present disclosure is to improve style recommendation performance by learning a style recommendation model by comprehensively utilizing item attribute information extracted from fashion image data with an item image.

Representative configurations of the present disclosure for achieving the above object are as follows.

A method for providing a style recommendation service according to an embodiment of the present disclosure includes receiving image data including a query product, determining a target category, and using a style recommendation model to determine a target category associated with a query product and corresponding to the target category. It includes selecting at least one target product and providing information on the at least one target product. Here, the style recommendation model is learning style coordination recommendation data, and the style coordination recommendation data is generated by detecting a plurality of items included in fashion image data without a separate labeling process for style information.

According to an embodiment of the present disclosure, the style coordination recommendation data may be generated by detecting the position and type of each of a plurality of items in fashion image data and including a relationship between the plurality of items.

According to an embodiment of the present disclosure, the style coordination recommendation data may be generated by extracting attribute information of each of a plurality of items from fashion image data and including the attribute information of the plurality of items.

According to an embodiment of the present disclosure, the target category may be a category that does not overlap with a category of a query product and can complete one of the recommended styles in combination with the category of a query product.

According to an embodiment of the present disclosure, fashion image data may be image data of wearing a combination of a plurality of fashion items.

According to one embodiment of the present disclosure, information on target products may be provided in a sorted order associated with a query product.

A system for providing a style recommendation service according to an embodiment of the present disclosure determines a target category using a query product image receiving unit receiving image data including a query product, and a style recommendation model, and is associated with the query product. It includes a target product selection unit for selecting at least one target product corresponding to and a target product information providing unit for providing information on the target product. Here, the style recommendation model is learning style coordination recommendation data, and the style coordination recommendation data is generated by detecting a plurality of items included in fashion image data.

In addition to this, another method for implementing the present disclosure, another system, and a non-transitory computer readable recording medium recording a computer program for executing the method are further provided.

According to the present disclosure, a large amount of style coordination recommendation data can be secured at a low cost by utilizing fashion image data, and a style recommendation model can learn this to provide information on recommended products related to a query product.

In addition, according to the present disclosure, an improved style recommendation service can be provided by comprehensively utilizing item attribute information extracted from fashion image data as data for a style recommendation model to learn with the image of the item.

1 is a diagram showing a schematic configuration of a system environment for providing a style recommendation service according to an embodiment of the present disclosure.
2 is a functional block diagram schematically illustrating a functional configuration of a style recommendation server according to an embodiment of the present disclosure.
3 is a diagram exemplarily illustrating items that can be detected from fashion image data according to an embodiment of the present disclosure.
4 is a diagram illustrating attribute information of an item extracted from fashion image data according to an embodiment of the present disclosure by way of example.
5 is an operational flowchart illustrating a process of securing data for learning a style recommendation model according to an embodiment of the present disclosure.
6 is a diagram illustrating a target product record selected by a style recommendation server for a query product image according to an embodiment of the present disclosure by way of example.
7 is an operational flowchart showing a style recommendation process according to an embodiment of the present disclosure.

Hereinafter, with reference to the accompanying drawings, preferred embodiments of the present disclosure will be described in detail so that those skilled in the art can easily implement them.

In order to clearly describe the present disclosure, descriptions of parts not related to the present disclosure have been omitted, and the same reference numerals are assigned to the same components throughout the specification. In addition, specific shapes, structures, and characteristics described in the specification may be changed and implemented from one embodiment to another without departing from the spirit and scope of the present disclosure, and the location or arrangement of individual components may also be implemented without departing from the spirit and scope of the present disclosure. and can be changed without departing from its scope.

Therefore, the detailed description described below is not intended to be performed in a limiting sense, and the scope of the present disclosure should be taken as encompassing the scope claimed by the claims and all scopes equivalent thereto.

1 is a diagram schematically illustrating a system environment for providing a style recommendation service according to an embodiment of the present disclosure.

As shown in FIG. 1 , the style recommendation system 100 according to an embodiment of the present disclosure may include a plurality of user terminals 110 , a communication network 120 , and a style recommendation server 130 .

The user terminal 110 according to an embodiment of the present disclosure is a digital device including a function capable of accessing and communicating with the style recommendation server 130 through the communication network 120 . It may be a portable digital device equipped with a memory unit and equipped with a microprocessor and equipped with an arithmetic capability, such as a smart phone or a tablet PC, and is not limited to a specific form. In this figure, only three user terminals are shown, but the present disclosure is not limited thereto.

According to an embodiment of the present disclosure, various types of user input received on the user terminal 110 may be transmitted to the style recommendation server 130 through the communication network 120 . According to an embodiment of the present disclosure, the user terminal 110 may receive various signals transmitted from the outside (eg, the style recommendation server 130) through the communication network 120.

According to an embodiment of the present disclosure, the user terminal 110 may include an application for supporting functions according to the present disclosure. Such an application may be downloaded from the style recommendation server 130 or an external application distribution server (not shown).

The communication network 120 according to an embodiment of the present disclosure may be configured regardless of communication aspects such as wired communication or wireless communication, and may include a local area network (LAN), a metropolitan area network (MAN), It may be configured with various communication networks such as a wide area network (WAN). Preferably, the communication network 120 referred to in this specification may be the well-known Internet or World Wide Web (WWW). However, the communication network 120 may include a known wired/wireless data communication network, a known telephone network, or a known wired/wireless television communication network at least in part without being limited thereto.

For example, the communication network 120 is a wireless data communication network, and includes WiFi communication, WiFi-Direct communication, Long Term Evolution (LTE) communication, Bluetooth communication (eg, Bluetooth Low Energy ( A conventional communication method such as Bluetooth Low Energy (BLE) communication), infrared communication, or ultrasonic communication may be implemented at least in part.

Style recommendation server 130 according to an embodiment of the present disclosure may provide information on a target product associated with a predetermined product. The style recommendation server 130 may provide information on a target product associated with a predetermined product and belonging to a category different from the predetermined product in order to provide one unified style. As shown, the style recommendation server 130 may communicate with the user terminal 110 through the communication network 120 .

The style recommendation server 130 according to an embodiment of the present disclosure may be a digital device equipped with a memory unit and equipped with a microprocessor to have arithmetic capability. This style recommendation server 130 may be a server system.

2 is a functional block diagram schematically illustrating a functional configuration of a style recommendation server according to an embodiment of the present disclosure.

Referring to FIG. 2, the style recommendation server 130 includes a style recommendation model management unit 202, a query product image receiving unit 204, a target product selection unit 206, a target product information providing unit 208, and a communication unit 210. can include

According to an embodiment of the present disclosure, the style recommendation model management unit 202, the query product image reception unit 204, the target product selection unit 206, the target product information providing unit 208, and the communication unit 210 are at least one of them. Some may be program modules that communicate with external systems. These program modules may be included in the style recommendation server 130 in the form of an operating system, application program module, or other program module, and may be physically stored in various known storage devices. Also, these program modules may be stored in a remote storage device capable of communicating with the style recommendation server 130 . Meanwhile, these program modules include, but are not limited to, routines, subroutines, programs, objects, components, data structures, etc. that perform specific tasks or execute specific abstract data types according to the present disclosure.

According to an embodiment of the present disclosure, the style recommendation model management unit 202 may create, learn, and manage a style recommendation model. Here, the style recommendation model is an artificial intelligence model that recommends a product constituting one style in association with a predetermined query product.

According to an embodiment of the present disclosure, the style recommendation model management unit 202 may generate style coordination recommendation data, which is learning data of the style recommendation model. Here, the style coordination recommendation data is a set of items related to each other. In one embodiment, the style recommendation model management unit 202 detects an item by pre-processing fashion image data including a plurality of items, and generates data (hereinafter referred to as style coordination recommendation data) related to a relationship between the detected items. can Here, the fashion image data is image data including fashion-related items (eg, tops, bottoms, outerwear, hats, handbags, etc.). According to an embodiment of the present disclosure, the style recommendation model management unit 202 may pre-process fashion image data to detect fashion items, determine properties of the fashion items, and generate data related to correlation between the fashion items.

According to an embodiment of the present disclosure, the style recommendation model management unit 202 may vectorize each item information and arrange the vectors on a style space to generate data related to correlation between fashion items. According to an embodiment of the present disclosure, the style recommendation model management unit 202 has a closer distance as the vectors of each item are closer to the so-called recommended style that forms a well-suited combination, and a style recommendation model such that if the vectors have a different style that does not match each other, they are further away. can be learned.

3 is a diagram illustratively illustrating how an item set is detected from fashion image data according to an embodiment of the present disclosure.

3(a) and 3(b) show that each item of the top and bottom is specified and detected as an item set from fashion image data in which the model wears the top and bottom. 3(a) and 3(b) , the style recommendation model management unit 202 selects upper and lower garment items 301 and 305 and lower garment items 303 and 303 from fashion image data in which the model wears upper and lower garments. Item sets (301, 303, 305, 307) can be detected by specifying 307, respectively.

3(c) and 3(d) show that each item of top, bottom, shoes, hat, or bag is specified from fashion image data in which a model wears shoes, hats, or bags other than tops and bottoms together, Indicates detection as a set. As shown in FIGS. 3(c) and 3(d), the style recommendation model management unit 202 identifies tops, bottoms, hats, shoes, and bags from the fashion image data worn by the model, respectively, and sets the items ((( 309, 311, 313, 315), (317, 319, 321), (323, 325, 327)) can be detected.

According to an embodiment of the present disclosure, style coordination recommendation data is a set of items associated with each other, and each item may include attribute information. In one embodiment, when specifying an item, the style recommendation model management unit 202 may store attribute information (eg, category, neckline, texture, fit, print, color, etc.) of the item.

4 is a diagram exemplarily illustrating attribute information of an item extracted from fashion image data according to an embodiment of the present disclosure.

Referring to FIG. 4 , the style recommendation model management unit 202 may store attribute information on an item (eg, casual top) extracted from fashion image data. Here, the attribute information may be at least one of category, length, neckline, texture, style, color, fit, detail, and print items. Similarly, attribute information may also be extracted for bottoms (not shown).

Referring back to FIG. 2 , in one embodiment of the present disclosure, the style recommendation model management unit 202 may generate style coordination recommendation data including items, relationships between items, and attribute information of items in fashion image data.

When a model is trained by generating style coordination recommendation data by synthesizing attribute information, the style recommendation model may provide improved style recommendation considering not only the category or color of an item, but also texture or detail.

In order to provide a style recommendation service with improved performance for various styles, it is necessary to train a style recommendation model by obtaining high-quality and large amount of style coordinator recommendation data. Fashion image data released to the public is limited in quantity and tends to be biased for similar colors, making it difficult to secure high-quality style coordination recommendation data.

In order to secure high-quality style coordination recommendation data, data on matching item combinations is needed. In order to determine 'matching or not', fashion experts judge whether the combination of each item (for example, top, bottom, outerwear, etc.) matches, inputs the information, and provides information on the combination of each item. Collecting takes a lot of time and money.

Accordingly, the present inventors have devised a method of collecting information on a combination of each item without a separate style verification (eg, 'matching or not' determination). According to an embodiment of the present disclosure, style coordination recommendation data may be generated from reliable fashion image data, and a style recommendation service may be provided using a style recommendation model learned therefrom.

According to an embodiment of the present disclosure, fashion image data may be received from the outside. In one embodiment, fashion image data may be received from an influencer's Social Network System (SNS) or an online market. For example, the fashion image data may be an image published on an influencer's SNS or online market. An image of the influencer wearing a fashion item is disclosed on SNS of an influencer in the fashion field, and an item that belongs to a predetermined product or other category and matches the predetermined product in order to sell a predetermined product in the online clothing market. An image wearing a is disclosed. Images published on SNS or online markets of these influencers will show images in which certain products and other products are combined to match, so it can be assumed that the image has a combination of matching items. Therefore, a plurality of items included in a fashion image can be regarded as forming one matching style combination and used as style coordination recommendation data for learning a style recommendation model without separate labeling on whether or not the combination suits well.

5 is an operational flowchart illustrating a process of securing data for learning a style recommendation model according to an embodiment of the present disclosure.

It is an operational flowchart showing a process of generating style coordination recommendation data performed in the style recommendation server 130. Here, the style coordination recommendation data is a set of items related to each other.

In step S501, the style recommendation server 130 may receive fashion image data. Here, the fashion image data is image data including fashion-related items (eg, tops, bottoms, outerwear, hats, handbags, etc.).

In step S503, the style recommendation server 130 may detect an item from fashion image data. According to an embodiment of the present disclosure, the style recommendation server 130 may detect fashion items by pre-processing fashion image data. Here, the fashion items are items related to fashion, such as tops, bottoms, shoes, hats, or bags. In one embodiment, the style recommendation server 130 may determine the type of fashion item.

In step S505, the style recommendation server 130 may determine whether the number of items detected from the fashion image data is two or more. If, as a result of the determination, two or more fashion items are not included, the procedure ends because an association between the items cannot be created.

When the number of items detected by the style recommendation server 130 from the fashion image data is two or more, step S507 is performed. In step S507, the style recommendation server 130 may extract attribute information of each item. For example, attribute information may be extracted using a system that recognizes attributes (eg, category, neckline, texture, fit, print, color, etc.) from an item image and tags the information with text.

In step S509, the style recommendation server 130 may generate style coordination recommendation data including items, relationships between items, and attribute information of items in the fashion image data.

Referring back to FIG. 2 , according to an embodiment of the present disclosure, the query product image receiving unit 204 may receive image data including a predetermined query product. Here, the predetermined query product may be at least one. For example, a predetermined query product may be a top (eg, a hoodie) or may include both a top (eg, a hoodie) and a bottom (eg, shorts).

According to an embodiment of the present disclosure, the target product selection unit 206 determines a target category using a style recommendation model, and is associated with the query product received from the query product image receiving unit 204 and corresponds to the target category. It performs a function of selecting at least one target product.

According to an embodiment of the present disclosure, a target category may be determined not to overlap with a category of a query product. The target category may correspond to a category that can complete one of the recommended styles in combination with the category of the query product. For example, when image data corresponding to blouses, skirts, and boots are received as query products, the style recommendation model determines bags that do not overlap with tops, bottoms, and shoes, which are categories of query products, as a target category, and Among corresponding products, a product that can be combined with a query product to form a related set may be selected as a target product. According to an embodiment of the present disclosure, the target category may include all possible categories. Alternatively, a target product may be selected by receiving information on a target category from a user on the user terminal 110 .

6 is a diagram illustrating a query product image and a target product record selected by a style recommendation server for the corresponding query product image according to an embodiment of the present disclosure by way of example.

Referring to FIG. 6 , the style recommendation server 130 may receive predetermined query product image data and output a predetermined target product image forming one style with a fashion item included in the corresponding image. In one embodiment, the style recommendation server 130 may determine the category of the target product as the category of the most relevant item determined through the style recommendation model. For example, in the record 610 of FIG. 6 , the style recommendation server 130 receives query product image data of outerwear 602a, top 602b, and bottom 602c, and receives the query product and one recommended style. Target products 604a to 604f can be determined in the shoe category that can achieve , and listed in order related to the query product (ie, in order close to the query product on the style space). In another embodiment, the style recommendation server 130 may receive an input of a category of a target product, and determine a target product that may form one recommendation style with a query product within the corresponding category. In this drawing, but shown with respect to the target product when three query product images are input, the present disclosure is not limited thereto.

The target product information providing unit 208 according to an embodiment of the present disclosure performs a function of providing information on at least one target product selected by the target product selection unit 206 . According to an embodiment of the present disclosure, information on target products may be provided to the user terminal 110 in an order associated with a query product.

Next, the communication unit 210 of the style recommendation server 130 according to an embodiment of the present disclosure includes a style recommendation model management unit 202, a query product image receiving unit 204, a target product selection unit 206, and target product information. It performs a function of enabling data transmission/reception from/to the providing unit 208.

7 is an operational flowchart showing a style recommendation process according to an embodiment of the present disclosure.

First, in step S701, the style recommendation server 130 may generate a style recommendation model. The style recommendation model learns style coordination recommendation data. For example, the style recommendation model may be trained so that each item of style coordination recommendation data has a close distance on a vector style space.

Next, in step S703, the style recommendation server 130 receives query product image data. In one embodiment, the style recommendation server 130 may receive query product image data from the user terminal 110 . Here, the query product may include at least one product.

In step S705, the style recommendation server 130 selects a target product constituting one style in association with the query product using the style recommendation model. Specifically, the style recommendation model may first determine a target category corresponding to a category of a query product and select at least one target product corresponding to the target category. The target category does not overlap with the category of the query product, and may be a category that forms a recommendation style in combination with the category of the query product, or may be input from the user terminal 110 .

Finally, in step S707, the style recommendation server 130 may provide information on the selected target product. In one embodiment of the present disclosure, when providing information on a target product, information may be provided by sorting in the order of a product that is close to the query product in the style space.

According to an embodiment of the present disclosure, an internal performance test was conducted using a style recommendation model. When the data made as one recommendation style is given as an input to the model, the Compatibility AUC result of determining whether or not one style is present is 0.98, A high performance of 95.9 was confirmed as a result of the FITB (Fill In The Blank) test, which excludes one product from one style coordinator recommendation data, inputs it as a query product, and selects a product related to the given query products.

In the embodiments of the present disclosure described above with reference to the drawings (and throughout this specification), the user terminal 110 and the style recommendation server 130 are a client-server model, in particular, the client mainly provides only user input and output functions, and Most of the other functions (particularly, many functions related to style recommendation) have been described as being implemented based on a form delegated to the server, but the present disclosure is not limited thereto. According to another embodiment of the present disclosure, it should be noted that the style recommendation system environment may be implemented by evenly distributing its functions between the user terminal and the server, or may be implemented depending more on the application environment installed on the user terminal. . In addition, when the functions of the style recommendation system are distributed and implemented between the user terminal and the server according to an embodiment of the present disclosure, the distribution of each function of the style recommendation system between the client and the server may be implemented differently for each embodiment. should know According to an embodiment of the present disclosure, it should be noted that the main function of the style recommendation server may be implemented and provided to each user terminal 110 instead of the style recommendation server 130 .

In addition, in the above-described embodiment of the present disclosure, it has been described as if a specific module performs predetermined operations for convenience, but the present disclosure is not limited thereto. According to another embodiment of the present disclosure, it should be noted that the operations described as being performed by any specific module in the above description may be performed by separate modules different from it.

Embodiments according to the present disclosure described above may be implemented in the form of program instructions that can be executed through various computer components and recorded in a computer readable recording medium. The computer readable recording medium may include program instructions, data files, data structures, etc. alone or in combination. Program instructions recorded on the computer-readable recording medium may be specially designed and configured for the present disclosure, or may be known and usable to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks and magnetic tapes, optical recording media such as CD-ROM and DVD, and magneto-optical media such as floptical disks. medium), and hardware devices specially configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include high-level language codes that can be executed by a computer using an interpreter or the like as well as machine language codes generated by a compiler. A hardware device may be modified with one or more software modules to perform processing according to the present disclosure and vice versa.

Although the above has been described by specific details and limited embodiments and drawings, such as specific components of the present invention, this is only provided to help a more general understanding of the present disclosure, and the present disclosure is not limited to the above embodiments, and the present disclosure A person with ordinary knowledge in the technical field to which belongs can make various modifications and changes from these descriptions.

Therefore, the spirit of the present disclosure should not be limited to the embodiments described above and should not be defined, and it can be said that not only the claims described below but also all modifications equivalent or equivalent to these claims fall within the scope of the spirit of the present disclosure. will be.

100: style recommendation system
110: user terminal
120: communication network
130: Style recommendation server
202: style recommendation model management unit
204: query product image receiver
206: target product selection unit
208: target product information provider
210: communication department

Claims (8)

As a method of providing a style recommendation service,
Receiving image data including a query product;
Determining a target category, using a style recommendation model to select at least one target product associated with the query product and corresponding to the target category; and
Providing information on the at least one target product
including,
The style recommendation model is learned from style coordination recommendation data, and the style coordination recommendation data is generated by detecting a plurality of items included in fashion image data without a separate labeling process for style information.
method. According to claim 1,
The style coordination recommendation data is generated by detecting the position and type of each of the plurality of items in the fashion image data and including a relationship between the plurality of items,
method. According to claim 1,
The style coordination recommendation data is generated by extracting attribute information of each of the plurality of items from the fashion image data and including the attribute information of the plurality of items.
method. According to claim 1,
The target category is a category that does not overlap with the category of the query product and can complete one of the recommended styles in combination with the category of the query product.
method. According to claim 1,
The fashion image data is image data that is worn by combining a plurality of fashion items,
method. According to claim 1,
Information on the target product is provided in an order related to the query product,
method. A non-temporary computer readable recording medium storing a computer program for executing the method according to claim 1. As a system that provides a style recommendation service,
A query product image receiver receiving image data including a query product;
A target product selection unit that determines a target category and selects at least one target product that is related to the query product and corresponds to the target category using a style recommendation model; and
A target product information providing unit providing information on the target product;
The style recommendation model is learned from style coordination recommendation data, and the style coordination recommendation data is generated by detecting a plurality of items included in fashion image data without a separate labeling process for style information.
system.

Images