KR102285942B1 - A method of providing a fashion item recommendation service to a user - Google Patents

Abstract

The present invention relates to a method of providing a fashion item recommendation service to a user using a service server. Specifically, checking whether a right to acquire an image from a user device exists, and obtaining an image for identifying the user if the right exists; generating a user fashion database by extracting a plurality of fashion items worn by the user from an image capable of identifying the user; Upon receiving a request for a recommendation service for a specific fashion item stored in the generated user fashion database, a style composed of style images from which a style label expressing a human feeling as computer-recognizable data by combining a plurality of fashion items searching a database for items similar to the specific fashion item based on the image similarity; determining an item of a different category from the similar item as a coordination item from the style image in which the similar item is found; and determining, as a recommended product, a product similar to the coordination item based on the image similarity from a product database configured by indexing labels extracted from product contents.

Description

A METHOD OF PROVIDING A FASHION ITEM RECOMMENDATION SERVICE TO A USER}

The present invention relates to a method of providing a fashion item recommendation service to a user, and more particularly, to a service providing method for recommending a fashion item to a user using an image obtained from a user device.

In the background of the recently increased wired and wireless Internet environment, commerce such as public relations and sales using online is being activated. In this regard, when buyers find a product they like while searching for a magazine, blog, or YouTube video on a desktop or mobile terminal connected to the Internet, they search for a product name, etc. and make a purchase. An example is the case where the name of a bag that a famous actress heard at the airport and the name of a childcare product appearing on an entertainment program rank high in real-time search terms on a portal site. However, in this case, the user has to separately open a web page for the search to search for a product name, manufacturer, vendor, etc., and it is inconvenient to not easily search unless clear information about them is already known.

On the other hand, sellers spend a lot of money on media sponsorship and online user recruitment in addition to commercial advertisements to promote their products. This is because word of mouth online recently acts as an important variable in product sales. However, it is often not possible to disclose shopping information, such as product name and vendor, despite the cost of publicity. This is because indirect advertising issues may arise as it is not possible to individually obtain prior approval from media viewers for product name exposure.

As described above, there is a need for both users and sellers to provide shopping information in a more intuitive UI (User Interface) environment for online product images.

Based on the above discussion, hereinafter, it is intended to provide a service for recommending a fashion item to a user using an image obtained from a user device.

The technical problems to be achieved in the present invention are not limited to the above technical problems, and other technical problems not mentioned will be clearly understood by those of ordinary skill in the art to which the present invention belongs from the following description.

In a method of providing a fashion item recommendation service to a user using a service server, which is an aspect of the present invention for solving the above-mentioned problems, it is checked whether an authority to acquire an image from a user device exists, and the obtaining an image capable of identifying the user when authority exists; generating a user fashion database by extracting a plurality of fashion items worn by the user from an image capable of identifying the user; Upon receiving a request for a recommendation service for a specific fashion item stored in the generated user fashion database, a style composed of style images from which a style label expressing a human feeling as computer-recognizable data by combining a plurality of fashion items searching a database for items similar to the specific fashion item based on the image similarity; determining an item of a different category from the similar item as a coordination item from the style image in which the similar item is found; and determining, as a recommended product, a product similar to the coordination item based on the image similarity from a product database configured by indexing labels extracted from product contents.

Furthermore, when a product image linked on a web page is input, the method may further include adding a fashion item for the product image to the user fashion database.

Furthermore, in the user fashion database, information on the plurality of fashion items is sorted and stored based on a user face image extracted from an image capable of identifying the user, and the plurality of fashion items The information on may include at least one of a size of a fashion item, a label expressing a feeling that a person feels in a fashion item as computer-recognizable data, and image information when the user fits. Furthermore, the determined recommended product is reference information so that the user's fit can be predicted based on the label extracted from the product content and information on the plurality of fashion items stored in the user fashion database may be characterized in that it is provided.

Furthermore, when the user's selection of the determined recommended product is received, the method may further include providing a product ordering service through a web page for the recommended product.

Furthermore, in the style database, a style label expressed as computer recognizable data is extracted from at least one fashion item included in the style image, and the style image is clustered based on the extracted style label. It may be characterized in that it consists of at least one style book (style book) sharing the style label of.

According to an embodiment of the present invention, it is possible to efficiently provide a service for recommending a fashion item to a user using an image obtained from a user device.

The effects obtainable in the present invention are not limited to the above-mentioned effects, and other effects not mentioned may be clearly understood by those of ordinary skill in the art to which the present invention belongs from the following description. will be.

BRIEF DESCRIPTION OF THE DRAWINGS The accompanying drawings, which are included as part of the detailed description for helping the understanding of the present invention, provide embodiments of the present invention, and together with the detailed description, explain the technical spirit of the present invention.
1 is a flowchart illustrating a message transmission process transmitted between a service server 101 and a user device 103 according to an embodiment of the present invention.
2 is a reference diagram for explaining a method for a service server to provide a fashion item recommendation service according to an embodiment of the present invention.

The present invention is not limited to the description of the embodiments described below, and it is apparent that various modifications may be made without departing from the technical gist of the present invention. And in describing the embodiments, descriptions of technical contents that are widely known in the technical field to which the present invention pertains and are not directly related to the technical gist of the present invention will be omitted.

Hereinafter, it is assumed that a user device displaying product information is a mobile device, but the present invention is not limited thereto. That is, in the present invention, a user device should be understood as a concept including all types of electronic devices capable of requesting a search and displaying advertisement information, such as a desktop, a smart phone, and a tablet PC.

It should also be noted that the concept of a commodity in this specification is not limited to a tangible good. That is, in the present specification, a product should be understood as a concept including a tangible product as well as an intangible service that can be sold.

Furthermore, as used herein, the term "displayed page in an electronic device" refers to a screen loaded in an electronic device and/or content inside the screen to be immediately displayed on the screen according to the user's scrolling. It can be understood as a concept that includes For example, in the display of the mobile device, the entire execution screen of an application that is extended in a horizontal or vertical direction and displayed according to a user's scrolling may be included in the concept of the page, and a screen in camera roll may also be included in the concept of the page. can

On the other hand, in the accompanying drawings, the same components are represented by the same reference numerals.

And in the accompanying drawings, some components may be exaggerated, omitted, or schematically shown. This is to clearly explain the gist of the present invention by omitting unnecessary descriptions not related to the gist of the present invention.

1 is a flowchart illustrating a message transmission process transmitted between a service server 101 and a user device 103 according to an embodiment of the present invention.

According to an embodiment of the present invention, a user-customized product recommendation service based on an image stored in a user device may be provided. For example, from among images stored in the user device, images that can be identified by a user are image-processed, a fashion item worn by the user is extracted based on the user's face, and a fashion item worn by the user is extracted and sorted, and then based on the corresponding fashion item. can provide recommended products. If a picture of a user A wearing a white bag and a brown suit exists in the user device, 'white bag' and 'brown suit' are sorted in the user's fashion database based on the face of user A. can be saved. Therefore, when a user requests a recommendation service for a fashion item that fits well with a brown suit included in the user's fashion database, the service server retrieves a matching photo of a brown suit and shoes from a style database learned using a fashion magazine. As a basis, coordination items can be recommended.

That is, when the service server 101 receives a request for a style recommendation for a brown suit, it refers to a style database, a product database, and a user fashion database generated in advance, and refers to a shoe category that goes well with the brown suit and matches the user's size. It is possible to recommend items of , and provide online market information of recommended shoes.

More specifically, when a user specifically inquires for a certain fashion item on the user's fashion database, the service server may determine an item similar to the requested item by first searching the style database based on the image similarity of the fashion item object. Thereafter, the service server may check other items matched with the similar item in the image included in the style database, and may determine the coordination item by reflecting user size information among the other items.

Thereafter, the service server may search a product database for the coordination item based on the image similarity, set a priority according to user size information, and determine a recommended product.

In order to provide a fashion item recommendation service according to an embodiment of the present invention, the service server 101 needs to check whether an authority to acquire an image from the user device 103 exists. When the service server does not have the authority to acquire the image from the user device, the service server may request the image authority from the user device (S110). For example, in order to provide a recommendation service, it is possible to request permission for a photo album, an album, etc. stored in the user device.

When the image permission of the service server is requested, the user may proceed with image permission granting for this (S120), and accordingly, the service server provides a recommendation service according to the present invention to the user by using the image permission for the user device. can If the service server already has the authority to acquire the image from the user device, the above-described image authority request step S110 and image authority grant step S120 may be omitted.

The service server 110, when the image right exists for the user device 102, obtains an image that can identify the user (S130). In this case, the service server may acquire all images stored in the user device and use only images identifiable by the user among them. can also

For example, if there are images classified based on the face of the user A in the user device, image processing may be performed by acquiring only the corresponding images. Since unnecessary photos may be included in the user device, the accuracy of the recommendation service can be improved by using the classified image, and unnecessary processing of the server can be prevented.

The service server 101 generates a user fashion database by extracting a plurality of fashion items worn by the user from an image capable of identifying the user. If a product image that can be checked or linked from a user on a web page (eg, an online shopping mall or a clothing brand homepage) is input, a fashion item for the product image may be added to the user's fashion database. For example, user A may capture an image from the online page of a shopping mall where he purchased a bag and add it to the user fashion database. it could be

In addition, the user fashion database may include information on fashion items, such as a size of a fashion item, a label expressing a feeling that a person feels in a fashion item as computer-recognizable data, a picture when the user fits, and the like. . For example, the user's fashion database may include information on the required size of the user's top, bottom, one piece, etc. can make this possible. Alternatively, by storing each of the fashion items to include personal feelings such as #comfort, #tight, and #suitable, the user may refer to it when considering fit when selecting a fashion item later.

Meanwhile, according to another embodiment of the present invention, the user fashion database may include online shopping mall access log data executed by the corresponding user device. For example, when a specific online shopping mall application or webpage is executed on the user device and the user reads or purchases a specific fashion item through this, the corresponding fashion item information, reading time, and purchase time may be stored in the user fashion database. .

For example, the user device 103 may obtain a detailed description URL of a fashion item purchased by the user and transmit the URL data to the service server 101 . Thereafter, the service server may collect text data such as image data and manufacturer, size information, and color of the corresponding fashion item from the URL. Furthermore, the service server may process the image data of the fashion item purchased by the user to extract the label for the content of the fashion item, for example, color, shape, material, pattern, length, shape, etc., and collect it from the URL. Text data, image data, and labels extracted from image data can be stored in a user fashion database. In this case, the data may be managed together with the online shopping mall identification information and purchase date data.

According to a further embodiment of the present invention, the user fashion database may generate taste data by estimating user tastes based on the collected data. For example, if a user frequently purchases a fashion item from a specific manufacturer, the user may presume that he or she prefers that manufacturer, and the color, shape, material, pattern, length, and shape of the fashion item owned or purchased by the user. User preference data can be generated by calculating the similarity of The user preference data may include information on a preferred price range, use, and brand when shopping online. A more detailed description of the user preference information will be described later in the accompanying description of FIG. 2 .

Thereafter, when the service server 101 receives a recommendation service request for a specific fashion item stored in the user fashion database created as described above from the user device 103 ( S140 ), the specific fashion based on the image similarity from the style database Search for items that are similar to an item. When a similar item is found, the service server determines an item of a different category from the similar item as a coordination item from the style image in which the similar item is found.

Accordingly, the service server may determine a product similar to the coordination item as a recommended product from the product database based on the image similarity and provide it to the user (S150). In this case, the service server may provide a predictable fit to the user when a recommended product is purchased based on information on fashion items included in the user's fashion database. If there is a label extracted from the contents of the product in the product database, it may be provided to the user together with information on fashion items included in the user fashion database.

When the user is provided with a recommended product from the service server, the user may choose to purchase it. In this case, the service server may provide a product ordering service through a web page for the recommended product according to the user's selection.

2 is a reference diagram for explaining a method for a service server to provide a fashion item recommendation service according to an embodiment of the present invention.

First, in order to provide a recommendation service according to the present invention, the service server obtains an image that can identify the user from the user device (S210). The method of acquiring the image may acquire the image from the user device, the user may confirm it on a web page (eg, an online shopping mall, a clothing brand homepage), or a linked product image may be input.

The service server creates a user fashion database from the acquired image (S220).

In the present invention, a fashion item extracted from an image may be stored in the user fashion database, and further, the user fashion database includes a fashion item size, a label expressing a feeling that a person feels in a fashion item as computer recognizable data, and an image when the user fits. Information may be included, or information that can estimate the user's taste, such as user's purchase data and reading time data, user's size information, and information on preferred price range, use, and brand when shopping online for fashion items. .

Alternatively, the user fashion database may include user identification information, user behavior information for estimating a user size, a user size estimated from the behavior information, and user size information directly received from the user device.

For example, the service server provides a user device with a query on the user's age, gender, occupation, fashion field of interest, previously owned item, etc., receives a user input for the query, generates user size information, and uses the information on the user's fashion It can be reflected in the database.

The service server includes the time the user reads any style book provided through the application according to the embodiment of the present invention, the item information that created the like tag, the requested item, the fashion item information purchased through the application or other application, and By combining user behavior information for estimating a user's size, such as time information at which the information is generated, taste information for a style that the user is interested in at the time can be generated and reflected in the user's fashion database.

In addition, the service server may generate the user's body type information and reflect it in the user's fashion database. For example, when a user device generates a body image of a user's body from multiple angles and transmits it to a service server, the service server uses a machine learning framework that learns human body features from a large number of body images to model the user's body shape. can create The user's body type model may include not only size information of each part of the user's body, but also information on the proportion and skin tone of each part of the user's body.

According to an additional embodiment of the present invention, the service server may generate preference information for a user's fashion item and reflect it in the user's fashion database. The preference information may include information on the user's preferred price, preferred brand, and preferred use. For example, when a user device reads or purchases a fashion item through an online market, the service server reflects the weight for reading or purchase differently to generate information on the preferred price, preferred brand, and preferred use, and store it in the user fashion database. can be reflected in

In particular, the service server according to an embodiment of the present invention has a characteristic of estimating a user's “taste” corresponding to a human feeling, generating the estimated taste information in a computer-recognizable form, and reflecting it in the user fashion database.

For example, the service server may extract a label for estimating the user's taste from the user's behavior information. The label may be extracted as meanings of fashion items included in the user behavior information, such as a style book read by the user, an item for which a like tag is generated, a requested item, and a purchase item. Furthermore, the label may be generated as information on a look and feel and trend, such as the appearance and feeling of fashion items included in the user behavior information.

The label generated from the user behavior information is weighted according to the user behavior, and the service server may combine them to generate user size information estimating the user size and store it in the user fashion database. User size information, user body type information, and user preference information included in the user fashion database may be used to set an exposure priority for a recommended item or a recommended product.

When the service server receives a request for a recommended service for a specific fashion item stored in the user fashion database, the service server searches for an item similar to the specific fashion item from the style database based on the image similarity ( S230 ). For example, the user may select a photo of a specific fashion item of the user device and transmit a request for requesting product information of the corresponding fashion item or requesting a recommendation for a coordination item suitable for the fashion item to the service server. Here, the user device may first determine whether an object of a preset category is included in the displayed image, and may specify the object as a specific fashion item. When objects are included, it may be operated to specify each object and transmit only a request for the object selected by the user.

In the present invention, the style database may include information on a fashion image that can be referred to for a fashion style and coordination of a plurality of items among images collected on the web. The style database may include, among images collected online, an image in which a plurality of fashion items are combined to fit well (this is referred to as a style image in this specification) and classification information on the style image. A style image according to an embodiment of the present invention is image data generated by a professional or semi-professional by combining a plurality of fashion items in advance, and can be collected on the web: a fashion catalog, a fashion magazine pictorial image, a fashion show shooting image, an idol costume image, and a specific drama. Alternatively, it may be exemplified by a costume image of a movie, an SNS, blog celebrity's costume image, a street fashion image in a fashion magazine, an image coordinated with other items for sale of a fashion item, and the like.

Accordingly, the style image is stored in a style database according to an embodiment of the present invention, and can be used to determine other items that go well with a particular item. According to this, the style image can be used as a reference material that the computer can understand the human feeling that it generally "fits well". Since “matching” with an arbitrary item is about human feeling, machine learning learned about matching multiple fashion items is necessary for a computer to recommend another item that “fits well” for an item without human intervention. You will need a framework. To this end, the service server according to an embodiment of the present invention may collect a style image worn by a person by combining a plurality of fashion items by experts or semi-professionals, and may generate them as a style database. Furthermore, the service server may learn the framework by applying the style database to the machine learning framework. For example, a machine learning framework that has learned a large number of matching style images for a blue shirt and a brown tie will be able to recommend a brown tie as a coordination item for a request for a blue shirt.

Also, in order to configure the style database, the service server may collect style images online. For example, the service server collects a list of web addresses of fashion magazines, fashion brands, drama production companies, celebrity agencies, SNS, online stores, etc. can be collected.

On the other hand, the service server according to an embodiment of the present invention can collect and index images from websites such as fashion magazines, fashion brands, drama production companies, celebrity agencies, SNS, online stores, etc., but images from affiliated companies with index information Information may be provided separately.

Accordingly, the service server may filter out images inappropriate for style recommendation among the collected images. For example, the service server may filter the remaining images while leaving only an image including a human-shaped object and a plurality of fashion items among the collected images.

Since style images are used to determine other items that can be coordinated with the requested item, it is appropriate to filter the image for a single fashion item. Furthermore, constructing a database with images of a person directly wearing a plurality of fashion items may be more useful than an image of the fashion item itself. Therefore, the service server according to the embodiment of the present invention may determine the style image included in the style database by filtering the remaining images, leaving only the image including the human-shaped object and the plurality of fashion items.

Thereafter, the service server may process features of the fashion item object image included in the style image. More specifically, the service server may extract image characteristics of the fashion item object included in the style image, generate the characteristic value of the fashion item object by expressing the characteristic information as a vector value, and structure the characteristic information of the images.

Furthermore, the service server according to an embodiment of the present invention may extract a style label from a style image and cluster the style images based on the style label. It is appropriate that the style label is extracted as the look and feel of the fashion item, such as the look and feel, and the trend. According to a preferred embodiment of the present invention, it is possible to extract a label for a feeling that a person can feel, such as the appearance of a single fashion item or a combination of a plurality of items included in the style image, and use it as a style label. For example, celebrity look, magazine look, summer look, feminine look, sexy look, office look, drama look, Chanel look, etc. can be exemplified as style labels.

According to an embodiment of the present invention, the service server defines a style label in advance, creates a neural network model that learns the characteristics of an image corresponding to the style label, classifies an object in the style image, and extracts a label for the object can do. In this case, the service server may assign a label to an image that matches a specific pattern with random probability through a neural network model that has learned the pattern of the image corresponding to each label.

According to another embodiment of the present invention, the service server learns the characteristics of the image corresponding to each style label to form an initial neural network model, and a large number of style image objects may be applied to extend the neural network model more precisely. there is.

Meanwhile, according to another embodiment of the present invention, the service server may apply the style images to the neural network model formed in a hierarchical structure formed of a plurality of layers without separate learning of the label. Furthermore, weight is given to the characteristic information of the style image according to the request of the corresponding layer, and product images are clustered using the processed characteristic information, and celebrity look, magazine look, summer look, feminine look, and sexy look are added to the clustered image group. , office look, drama look, Chanel look, etc. can be given a label that is interpreted ex post.

Accordingly, the service server may cluster style images using the style label and generate a plurality of style books. This is intended to be provided as a reference to users. The user may browse a specific style book among a plurality of style books provided by the service server and find a favorite item, and may request a product information search for the item.

Meanwhile, the service server may pre-classify items having a very high appearance rate, such as white shirts, jeans, and black skirts. For example, since jeans are a basic item in fashion, their appearance rate in style images is very high. Therefore, even if the user inquires about any item, the probability that jeans will be matched as a coordination item will be significantly higher than that of other items.

Therefore, the service server according to the embodiment of the present invention can pre-classify items with a very high appearance rate in the style image as buzz items, and create style books with different versions of those that include the buzz item and those that do not include the buzz item. there is.

According to another embodiment of the present invention, the buzz item may be classified by reflecting time information. For example, considering the fashion cycle of fashion items, items that are fashionable and disappear for a short time for a month or two, fashionable items that return every season, and items that are continuously popular for a certain period of time may be considered. Accordingly, if time information is reflected in the classification of the buzz item and the appearance rate of a specific fashion item is very high for a certain period, the item may be classified as a buzz item together with the information about the period. When the buzz items are classified in this way, in the subsequent item recommendation step, there is an effect that the recommendation target item can be recommended in consideration of whether the item to be recommended is in vogue or unrelated to the vogue.

That is, the service server according to the present invention may process a specific fashion item object included in the received request and search the style database based on the image similarity. That is, the service server may search for similar items in the style database by processing the image object specified as the search target.

To this end, the service server may extract the characteristics of the image object to be searched and structure specific information of the images for the efficiency of the search.

Furthermore, the service server according to an embodiment of the present invention can extract label and/or category information about the meaning of the search target object image by applying the machine learning technique used to build the product image database to the processed search target object image. there is. The label may be expressed as an abstract value, but may also be expressed in text form by interpreting the abstract value.

For example, the service server according to an embodiment of the present invention may extract labels for women, one-piece, sleeveless, linen, white, and casual look from the requested object image. In this case, the service server uses the labels for women and dresses as category information of the requested object image, and the labels for sleeveless, linen, white, and casual look can be used as label information to describe the characteristics of the object image outside the category. .

Thereafter, the service server may search the style database based on the similarity of the requested object image. This is to search for an item similar to the requested image in the style database, and to check other items that are matched with the similar item in the style image. The similarity of the feature values may be calculated, and an item having a similarity within a preset range may be identified.

Furthermore, the service server according to an embodiment of the present invention processes the feature value of the requested image by reflecting the weights required by a plurality of layers of the artificial neural network model for machine learning configured for the product database, and is within a certain range of the requested image. At least one fashion item group included in the style book having a distance value may be selected, and items belonging to the group may be determined as similar items.

Meanwhile, according to a preferred embodiment of the present invention, the service server determines a similar item by searching the style database based on the similarity of the requested image, and in this case, to increase the accuracy of the image search, the service server uses the label and category information extracted from the image. can

For example, the service server calculates the similarity between the requested image and the feature value of the style database image, and the label and/or category information of the products that are above the similarity within a preset range does not match the label and/or category information of the requested image. Similar items may be determined by excluding products.

As another example, the service server may calculate the item similarity only in the style book having the label and/or category information matching the label and/or category information of the requested image.

For example, the service server according to an embodiment of the present invention may extract a style label from a requested image, and specify a similar item based on the request and image similarity in a style book matching the label. Of course, the service server may specify similar items based on the similarity between the requested image and the image in the style database without extracting a separate label from the requested image.

For example, if there is a leaf pattern dress in the image included in the request, the service server may extract the label tropical from the request. Thereafter, the service server may specify a similar item having a similarity within a preset range to the leaf pattern dress in the clustered style book with the label tropical.

Thereafter, the service server may include the similar item retrieved from the style book and provide a style image in which the similar item is combined with other fashion items to the user device. In the above example with a leaf pattern dress, a style image in which a straw hat, a rattan bag, etc. are combined with the leaf pattern dress may be provided to the user.

When an item similar to a specific fashion item is found in the style database, the service server determines a coordination item by checking a fashion item of another category included in the style image in combination with the similar item (S240).

That is, a specific fashion item inquired by the user may be searched for in the style database based on the image similarity, and in a style image including a similar item, a fashion item of another category matched with the similar item may be considered as a recommended item. This is because the service server according to the embodiment of the present invention has learned that other items matched with the requested item in the style image are well matched.

When the coordination item is determined using the style database, the service server determines a product similar to the coordination item from the product database as a recommended product (S250).

The product database may include detailed product information such as country of origin, size, vendor, and wearing shot of products sold in the online market, and has a characteristic in that product information is configured based on the image of the product.

That is, the service server may collect product information about products sold in any online market as well as product information of a pre-affiliated online market. For example, the service server may include a crawler, a parser, and an indexer to collect web documents of an online store and access text information such as product images and product names and prices included in the web documents.

For example, a crawler can collect a list of web addresses for online stores, check websites and track links, and deliver data related to product information to a service server. In this case, the parser interprets the web documents collected during the crawling process and extracts product information such as product images, product prices, and product names included in the page, and the indexer can index the location and meaning.

Meanwhile, the service server according to an embodiment of the present invention may collect and index product information from websites of any online store, but may also receive product information in a preset format from an affiliate market.

The service server may process the product image. This is to determine a recommended item based on whether a product image is similar, without relying on text information such as a product name or sales category.

According to an embodiment of the present invention, a recommended item may be determined based on whether a product image is similar, but the present invention is not limited thereto. That is, depending on the implementation, the product image as well as the product name or sales category can be used alone or as an auxiliary request.

For example, according to another embodiment of the present invention, based on user preference information, a fashion item that matches at least one of a user's preferred brand, color, pattern, length, or shape over a preset range is selected as a recommended item. may decide In this case, the priority level of the recommended item may be determined in consideration of the user's preferred price range and information on the user's preferred online shopping mall.

Meanwhile, according to a preferred embodiment of the present invention, the service server may extract features of a product image and index feature information of the images for efficiency of search.

More specifically, the service server may detect a feature area of the product images (Interest Point Detection). The feature region refers to a main region from which a descriptor for a feature of an image, ie, a feature descriptor, is extracted for determining whether the images are identical or similar.

According to an embodiment of the present invention, such a feature region is greater than a contour included in an image, a corner such as a corner among contours, a blob distinguished from a peripheral region, a region that is invariant or covariant according to the deformation of the image, or an ambient brightness. It can be a pole with dark or light features, and can target a patch (piece) of an image or the entire image.

Furthermore, the service server may extract a feature descriptor from the feature area (Descriptor Extraction). The feature descriptor expresses the features of an image as vector values.

According to an embodiment of the present invention, such a feature descriptor may be calculated using the position of the feature region in the corresponding image, or brightness, color, sharpness, gradient, scale, or pattern information of the feature region. For example, the feature descriptor may convert a brightness value of a feature region, a change value of brightness, or a distribution value of the feature region into a vector and calculate it.

Meanwhile, according to an embodiment of the present invention, the feature descriptor for an image is a global descriptor, a frequency descriptor, a binary descriptor, or a local descriptor based on the feature region as described above. It can be expressed as a neural network descriptor.

More specifically, the feature descriptor is a global descriptor ( global descriptor).

For example, the feature descriptor includes a frequency descriptor that converts and extracts the number of times that specific descriptors classified in advance are included in an image, the number of times that a global feature such as a conventionally defined color table is included, etc. Learning from the layer of binary descriptor and neural network, which extracts in bits whether it is included or whether the size of each element constituting the descriptor is larger or smaller than a specific value, and converts it into an integer type Alternatively, it may include a neural network descriptor that extracts image information used for classification.

Furthermore, according to an embodiment of the present invention, it is possible to convert the feature information vector extracted from the product image to a lower dimension. For example, feature information extracted through an artificial neural network corresponds to 40,000-dimensional high-dimensional vector information, and it is appropriate to convert it into a low-dimensional vector of an appropriate range in consideration of the resources required for the search.

For the transformation of the feature information vector, various dimensional reduction algorithms such as PCA and ZCA may be used, and the feature information converted into a low dimensional vector may be indexed into a corresponding product image.

Furthermore, the service server according to an embodiment of the present invention may extract a label for the meaning of the image by applying a machine learning technique based on the product image. The label may be expressed as an abstract value, but may also be expressed in text form by interpreting the abstract value.

More specifically, the service server may define a label in advance, create a neural network model that learns the characteristics of the image corresponding to the label, classify an object in the product image, and extract a label for the object. In this case, the service server may assign a label to an image that matches a specific pattern with random probability through a neural network model that has learned the pattern of the image corresponding to each label.

Alternatively, the service server may learn the characteristics of the image corresponding to each label to form an initial neural network model, and apply a large number of product image objects to the neural network model to more elaborately extend. Furthermore, if the corresponding product is not included in any group, the service server may create a new group including the corresponding product.

Accordingly, the service server pre-defines labels that can be used as meta information about products, such as women's bottoms, skirts, dresses, short sleeves, long sleeves, pattern shape, material, color, and abstract feeling (innocence, chic, vintage, etc.) , it is possible to generate a neural network model that learns the characteristics of the image corresponding to the label, and apply the neural network model to the advertiser's product image to extract a label for the advertisement target product image.

Alternatively, the service server may apply the product images to the neural network model formed in a hierarchical structure formed of a plurality of layers without separate learning of the label. Furthermore, weights may be given to the characteristic information of the product image according to the request of the corresponding layer, and the product images may be clustered using the processed characteristic information.

In this case, additional analysis may be required to confirm whether the corresponding images are clustered according to any attribute of the feature value, that is, to connect the clustering result of the images with a concept that can be recognized by a human. For example, when the service server classifies products into three groups through image processing and extracts labels A for the first group feature, B for the second group feature, and C for the third group feature , A, B, and C need to be interpreted ex post to mean, for example, women's tops, blouses, and plaids, respectively.

According to the third embodiment, the service server retrospectively analyzes women's bottoms, skirts, dresses, short sleeves, long sleeves, pattern shapes, materials, colors, and abstract feelings (innocence, chic, vintage, etc.) in the clustered image group. A label that can be interpreted can be given, and labels assigned to an image group to which an individual product image belongs can be extracted as a label of the corresponding product image.

On the other hand, the service server according to the embodiment of the present invention may express the label extracted from the product image as text, and the label in the text form may be used as tag information of the product.

Conventionally, product tag information is subjectively and directly provided by the seller, which is inaccurate and has low reliability. There was a problem in that the product tag subjectively given by the seller acts as a noise and lowers the efficiency of the search.

However, as in the embodiment of the present invention, if label information is extracted based on the product image, the extracted label information is converted into text and used as tag information of the product, the tag information of the product is converted to the image of the product. Since it can be extracted mathematically without human intervention, the reliability of tag information is increased and the accuracy of the search is improved.

Furthermore, the service server may generate category information of the corresponding product based on the product image content. For example, if a label for an arbitrary product image is extracted as women, top, blouse, linen, stripe, long sleeve, blue, or office look, the service server uses the label for women, top, and blouse as category information of the product. Labels for linen, stripe, long sleeves, blue, and office look can be used as label information to describe the characteristics of products outside of the category. Alternatively, the service server may index the product without distinguishing between label and category information. In this case, the category information and/or the label of the product may be used as a parameter to increase the reliability of the image search.

Accordingly, the service server determines, as a recommended item, an item similar to the coordination item from the product database configured by indexing the label extracted from the contents of the above-mentioned product, and provides product information about the recommended item, similar to the recommended item Products can be searched for in the product database.

More specifically, the service server may search the product database based on the image similarity for the coordination item determined using the style database. (Step 190)

To this end, the service server may extract the features of the coordination item object and structure specific information of the images for efficiency of the search.

The service server according to an embodiment of the present invention may search the product database based on the similarity of object images. For example, the service server may calculate a similarity between the recommended item image and feature values of the product image included in the product database, and determine a product having a similarity within a preset range as the recommended product.

Furthermore, the service server according to an embodiment of the present invention processes the feature value of the recommended item image by reflecting the weights required by a plurality of layers of the artificial neural network model for machine learning configured for the product database, and the distance within a certain range At least one product group having a value may be selected, and products belonging to the group may be determined as recommended products.

Furthermore, the service server according to another embodiment of the present invention may specify a recommended product based on a label extracted from the recommended item object.

For example, if the label information of the object extracted from the recommended item image is extracted as women's tops, blouses, white, and stripes, the service server compares the search target object image and The similarity can be calculated.

As another example, the service server may use products having a similarity greater than or equal to a preset range as recommended candidate products, and may exclude products whose sub-category information is not a blouse from the recommended candidate products. In other words, products indexed by sub-category information as blouses may be selected as advertisement items.

As another example, if the label information extracted from the object image of the recommended item is a female top, blouse, long sleeve, lace, or collar neck, the service server has a product having a female top, blouse, long sleeve, lace, and collar neck as labels in the product database It is also possible to calculate the similarity between the recommended item and the image only for the group.

When the recommended product is determined, the service server may determine the priority of exposure by reflecting user preference/size information. For example, when the user's taste information is focused on office look, the priority may be calculated by giving weight to the office look label, and recommended product information may be provided according to the calculated priority.

Embodiments of the present invention published in the present specification and drawings are merely provided for specific examples in order to easily explain the technical contents of the present invention and help the understanding of the present invention, and are not intended to limit the scope of the present invention. It will be apparent to those of ordinary skill in the art to which the present invention pertains that other modifications based on the technical spirit of the present invention can be implemented in addition to the embodiments disclosed herein.

The service of recommending a fashion item to a user using the image obtained from the user device as described above can be applied to various services.

Claims (6)

A method of providing a fashion item recommendation service to a user using a service server, the method comprising:
checking whether a right to obtain an image from a user device exists, and if the authority exists, obtaining an image capable of identifying the user;
generating a user fashion database by extracting a plurality of fashion items worn by the user from an image capable of identifying the user;
A style database composed of style images is created, and style labels expressing a feeling that a person can feel about a combination of a plurality of fashion items included in each style image is extracted from each style image as data that can be recognized by a computer, , clustering the style images based on the extracted style labels to generate the style database to include at least one style book sharing an arbitrary style label;
Upon receiving a request for a recommendation service for a specific fashion item stored in the created user fashion database, a style book having a style label matching the label of the specific fashion item is searched for in the style database, and image similarity is obtained from the searched style book searching for items similar to the specific fashion item based on
determining, as a coordination item, an item of a category different from the similar item in the style image in which the similar item is found; and
A product having a label matching the style label of the coordination item is searched for in a product database configured by indexing a label extracted from the contents of the product, and a product similar to the coordination item is recommended based on the image similarity among the searched products. comprising the step of determining
How to provide a fashion item recommendation service. The method of claim 1,
When a product image linked on a web page is input, the method further comprising the step of adding a fashion item for the product image to the user fashion database,
How to provide a fashion item recommendation service. The method of claim 1,
The user fashion database,
Based on the user face image extracted from the image that can identify the user, information on the plurality of fashion items is sorted and stored,
Information about the plurality of fashion items,
It characterized in that it includes at least one of a fashion item size, a label expressing a feeling that a person feels in a fashion item as computer-recognizable data, and image information when the user fits,
How to provide a fashion item recommendation service. 4. The method of claim 3,
The recommended products determined above are,
Based on the label extracted from the contents of the product and information on the plurality of fashion items stored in the user fashion database, reference information is provided so that the user's fit is predictable,
How to provide a fashion item recommendation service. The method of claim 1,
When the user's selection of the determined recommended product is received, further comprising the step of providing a product ordering service through a web page for the recommended product,
How to provide a fashion item recommendation service. The method of claim 1,
The style book is
A style book including a buzz item and a style book excluding the buzz item,
The buzz item is,
Characterized in that it is a fashion item with a high rate of appearance during a certain period among the style images,
How to provide a fashion item recommendation service.

Images