WO2020141802A2 - Method for providing fashion item recommendation service to user by using date - Google Patents

Abstract

The present invention relates to a method for providing a fashion item recommendation service to a user by using a service server. Specifically, the method comprises the steps of: retrieving, on the reception of a recommendation service request, a specific fashion item from a user fashion database according to a specific date related to the recommendation service request, the specific fashion item being associated with calendar data ranging from predetermined days before to predetermined days after the specific date of the previous year; searching a style database for an item similar to the specific fashion item according to image similarity on the basis of a vogue pattern determined according to the specific date, the style database including style images with style labels extracted therefrom that describe human feelings as data recognizable by computers; searching a product database for a same-category candidate item similar to the similar item according to image similarity, wherein in the product database, labels derived from the details of products are indexed and organized; selecting, as a coordination item, an item different in category from the similar item, from the style image in which the similar item has been found; searching the product database for a different-category candidate item similar to the coordination item according to image similarity; and providing, as a recommendation product, at least one of the same-category candidate item and the different-category candidate item.

Description

How to provide a fashion item recommendation service using a date to a user

The present invention relates to a method of providing a fashion item recommendation service to a user, and more particularly, to a method of providing a fashion item recommendation service using a date.

Background of the Invention With the recent increase in wired/wireless Internet environment, commerce such as online promotion and sales is being activated. In this regard, buyers search for videos of magazines, blogs, or YouTube on a desktop or mobile terminal connected to the Internet, and when they find a product they like, they search for the product name and lead to purchase. An example is when the name of a bag that a famous actress heard at the airport and the name of a childcare product in the entertainment program go up on the portal site's real-time search query ranking. However, at this time, the user has to open a web page for searching separately to search for a product name, a manufacturer, a dealer, and the like. If there is no clear information about them, there is a inconvenience that the user cannot easily search.

Meanwhile, sellers spend a lot of money on media sponsorship and online user recruitment in addition to commercial advertising to promote their products. This is because online word-of-mouth has recently been an important variable in product sales. However, despite spending on promotional expenses, it is often impossible to disclose shopping information such as product names and retailers. This is because indirect advertisement issues may occur because media viewers cannot receive individual approval for product name exposure.

As such, there is a need for both users and sellers to provide shopping information in a more intuitive user interface (UI) environment for product images online.

Based on the discussion as described above, the following is to provide a fashion item recommendation service using a date.

The technical problems to be achieved in the present invention are not limited to the above technical problems, and other technical problems not mentioned will be clearly understood by a person having ordinary knowledge in the technical field to which the present invention belongs from the following description.

A method of providing a fashion item recommendation service to a user using a service server, which is an aspect of the present invention for solving the above-described problem, upon receiving a recommendation service request, on a specific date (date) of the recommendation service request Based on, extracting a specific fashion item associated with calendar data on a predetermined period before and after the specific date of a previous year from a user fashion database; From a style database consisting of style images from which style labels are extracted that express a person's feelings with computer-recognizable data, based on the fashion pattern determined based on the specific date, similar to the specific fashion item based on image similarity Searching for an item; Retrieving the same category recommendation item similar to the similar item based on the image similarity from a product database in which a label extracted from the content of the product is indexed and configured; Determining an item of a different category from the similar item as a coordination item from the style image in which the similar item is searched; Retrieving, from the product database, another category candidate item similar to the coordination item based on image similarity; And providing at least one of the same category candidate item and the other category candidate item as a recommended product.

Furthermore, the predetermined period is pre-specified for fashion item coordination, or is set to one of day, week, or month units input from the user, and the trendy pattern May be determined based on the repetition degree of style labels appearing in style images based on the specific date.

Furthermore, the user fashion database is generated such that fashion items extracted from user images are associated with the user images and calendar data, and the calendar data includes time included in the user images. Based on data and location data, including weather information related to at least one of temperature, humidity, UV index, wind, and fine dust ozone, and obtained through a schedule program installed on a user device in association with time data of the user images It may be characterized in that it comprises user event information, and the step of extracting the specific fashion item is further performed based on an update of whether the fashion item is owned by the user.

Preferably, the step of extracting the specific fashion item is further extracted based on the fashion item corresponding to the calendar data associated with the specific date (data), or for recommending a fashion item based on the specific date (data) In order to do so, it may be characterized by further extracting a fashion item corresponding to the calendar data within one period of a predetermined day, week, or month unit.

Furthermore, the recommendation service request may be characterized by recognizing the voice of the user and requesting a fashion item recommendation service for the specific date.

According to an embodiment of the present invention, a fashion item recommendation service can be efficiently provided using a date.

The effects that can be obtained in the present invention are not limited to the above-mentioned effects, and other effects that are not mentioned can be clearly understood by those skilled in the art from the following description. will be.

BRIEF DESCRIPTION OF THE DRAWINGS The accompanying drawings, which are included as part of the detailed description to aid understanding of the present invention, provide embodiments of the present invention and describe the technical spirit of the present invention together with the detailed description.

1 is a reference diagram for explaining a system for providing a fashion item recommendation service using a date according to an embodiment of the present invention.

2 is a reference diagram for explaining a method of providing a fashion item recommendation service to a user using a date according to an embodiment of the present invention.

The present invention is not limited to the description of the embodiments described below, and it is apparent that various modifications may be made without departing from the technical spirit of the present invention. In describing the embodiments, descriptions of technical contents that are widely known in the technical field to which the present invention pertains and are not directly related to the technical subject matter of the present invention will be omitted.

Hereinafter, it is assumed and described that the user device on which the product information is displayed is a mobile device, but the present invention is not limited thereto. That is, in the present invention, the user device should be understood as a concept including all types of electronic devices capable of requesting search and displaying advertisement information, such as a desktop, a smart phone, and a tablet PC.

It should also be noted that the concept of goods in this specification is not limited to tangible goods. That is, in this specification, a product should be understood as a concept including not only tangible products but also intangible services that can be sold.

Furthermore, in this specification, the term displayed on a user device (displayed page in an electronic device) refers to a screen loaded on an electronic device and/or content inside the screen so that it can be immediately displayed on the screen according to the user's scroll. It can be understood as an inclusive concept. For example, in a display of a mobile device, an entire execution screen of an application that is extended in a horizontal or vertical direction and displayed according to a user's scroll may be included in the concept of the page, and the screen being rolled by the camera may also be included in the concept of the page. Can.

Meanwhile, in the accompanying drawings, the same components are represented by the same reference numerals.

In the accompanying drawings, some components may be exaggerated, omitted, or schematically illustrated. This is to clearly explain the gist of the present invention by omitting unnecessary descriptions not related to the gist of the present invention.

1 is a reference diagram for explaining a system for providing a fashion item recommendation service using a date according to an embodiment of the present invention. The recommended service providing system according to the present invention includes a user device 110, a service server 120, and is connected to a home appliance 130 including an AI speaker 131, a smart TV 133, and the like. Can be configured.

The user device 110 may store user schedules, images, and the like, and a service server through a wired or wireless network. It includes smart phones, smart pads, etc., which can send and receive data to and from other devices in the home appliance, including AI speakers, smart TVs and the like. In the present invention, the user device 110 stores the user's schedule (eg meeting, travel…, etc.) information or weather information application (eg K-weather, Naver weather…, etc.), schedule management application (eg Google Calendar, Naver Calendar… Etc.).

The service server 120 provides a fashion item recommendation service to the user based on time/location information related to the user image received from the user device 110, user schedule information, user event, and the like. Here, the user image may be an image in which image pre-processing is performed through a user device or a service server. For example, the user image may include a certain pattern, such as a room background. By separating the user's head and feet from the captured image, the part excluding the user as the subject is treated as a blank or the user's A process for grasping the overall fit may be performed. In addition, the user image may include and store location data such as time data, such as date, time, and day of the image creation, and GPS coordinates in which the image is generated, in relation to the creation of the image. Further, weather information according to the date of creation of the image may be obtained through the weather information application on the user device based on the time data and the location data of the user image, but has only the time and location data of the user image, and the server or other. Weather information corresponding thereto may be obtained on the device.

The home appliance 130, such as an AI speaker, recognizes a user's voice and processes commands and data, such as a voice recognition device 131, a smart TV capable of providing images, images, etc. to a user, such as a smart pad It may include a display device 133, an image taking device capable of taking a picture, a video, and the like.

The voice recognition device 131 is defined as a device capable of recognizing a user's voice, such as Siri of apple, Google home of Google, NUGU of SKT, Alexa of Amazon, and data processing accordingly, and artificial intelligence (artificial intelligence) intelligence) may be applied to perform data processing according to the user's voice and additional operations in consideration of the user's schedule (eg, a coordination recommendation service according to the weather, a coordination recommendation service considering events such as a future wedding or a dinner).

The display device 133 may display user images or recommended fashion items provided by the service server to the user, and the display device may be a web page (web page) for purchasing fashion items recommended to the user through a wired or wireless network. , eg shopping mall, Soho).

Furthermore, the image photographing module may generate an identifiable digital image by photographing a subject, and as described above, may include and generate time and location data related to the creation of the image.

2 is a reference diagram for explaining a method of providing a fashion item recommendation service to a user using a specific date according to an embodiment of the present invention.

When the service server receives a recommendation service request from the user, the user fashion database displays a specific fashion item associated with calendar data on a predetermined period before or after a specific date in a previous year based on a specific date associated with the recommendation service request. It is extracted from. Here, the predetermined period may be preset through a user or the like for fashion item coordination, or may be set to one of day, week, or month units input from the user. .

Here, the user fashion database is generated such that the fashion items extracted from the user images, the user images and the calendar data (calendar data) are associated, and the calendar data is stored in the time data and location data included in the user images. Based on temperature, humidity, UV index, wind, fine dust, and weather information related to at least one of ozone levels, and user event information obtained through a schedule program installed on a user device in association with time data of user images. Can. For example, if the recommendation service request is related to December 25, 2018, a specific fashion item related to calendar data for a period of one week before and after December 25, 2017 may be extracted. For example, the calendar data may include temperature, humidity, UV index, wind, fine dust, or ozone level information based on time data and location data included in user images, so if the user is in December 2017, An image was created by taking a picture in Myeong-dong, Korea at 20:00 on the 25th. On December 25, 2017, the temperature in Myeong-dong at 20:00 was minus 2 degrees, humidity was 52%, UV index was 25%, and wind was 3m northwest. Weather information such as /s, poor fine dust, and low ozone content can be defined as calendar data.

In addition, the user's schedule and data related to the user's activity, which may be included in the calendar data, may be automatically acquired through a schedule management program installed on the user device in association with the user's time data, or may be manually input from the user. Since the user's schedule is generally different for each individual user, unlike the weather information described above, it is preferable that it is acquired on the user device or added/modified according to the user's input. Therefore, if the schedule management program (for example, Google Calendar, etc.) has a schedule called'date' all day on December 25, 2018, or if the user randomly enters'date with XXX', an event called'date' It can be included in the calendar data.

Accordingly, for example, the service server is associated with the calendar data of 2017, as well as the fashion items worn on the corresponding date (date), each week before and after December 25, 2017 from December 18, 2017 to 2018 You can extract fashion items until January 01. Alternatively, even if there is a request for a fashion item recommendation service related to December 25, 2018, the user may recommend a fashion item for styling considering the schedule, event, weather, etc., before and after January 08, 2018, 2 weeks later, to allow the user It is also possible to connect to shopping malls so that additional purchases can be made in advance.

In addition, the user fashion database may reflect an update on whether or not a fashion item is owned by the user. For example, when a check is made on fashion items discarded by a user, a specific fashion item may be extracted only for fashion items owned by the user.

In more detail with respect to the user database, the service server extracts fashion items from user images obtained from an image capture module or the like, and associates the calendar data related to the user image acquired through the user device to the user fashion database Can generate For example, among images stored in the user device, images that can be identified by the user are image-processed, and the fashion items worn by the user are extracted and sorted based on the user's face, and then the fashion items are sorted. A user fashion database can be generated by associating calendar data. In addition, when calendar data is added/modified from the user with respect to the user fashion database, the user fashion database may be updated.

Therefore, if a user named A wears a white bag and a brown suit exists on the user device, and the photo is associated with calendar data where an event called'meeting' exists, based on the user's face A The'white bag' and the'brown suit' can be stored in the user fashion database along with the associates'meeting'. Accordingly, when a user requests a fashion recommendation service for an event called'conference', the service server searches for fashion items associated with the event'conference' among style databases and user images learned using a fashion magazine or the like. And based on this, a recommended product can be provided.

In addition, when a user can check on a web page (for example, an online shopping mall or a clothing brand homepage) or a linked product image is input, a fashion item for the product image may be added to the user fashion database. For example, a user named A may capture an image from an online page of a shopping mall where a bag is purchased and add it to the user's fashion database. When entering a link to a specific site, the image of the site is added to the user's fashion database. It can be.

The user fashion database may include information on fashion items, such as a size of a fashion item, a label expressing a feeling that a person feels in a fashion item as computer-recognizable data, and a picture when the user fits. For example, the user fashion database may include necessary size information such as a user's top, bottom, and dress, and the appearance when fitting the actual clothes is managed as a photograph, so that the user fits considering his body type You can make this possible. Alternatively, by storing each of the fashion items to include a personal feeling such as #comfort, #tight, #appropriate, etc., the user may refer to the future when considering fit when selecting a fashion item. In addition, the image information when the user is fitted, or information that can estimate the user's preferences, such as the user's purchase data and viewing time data, the user's size information, the preferred price point for online shopping for fashion items, use , May contain information about the brand.

Alternatively, the user fashion database may include user identification information, user behavior information for estimating the user size, user size estimated from the behavior information, and user size information directly received from the user device.

For example, the service server provides a query to the user device for the user's age, gender, occupation, fashion field of interest, reserved items, etc., receives user input for the query, generates user size information, and generates the user fashion. It can be reflected in the database.

The service server is a time when a user browses an arbitrary style book provided through an application according to an embodiment of the present invention, item information generated like a tag, request item, fashion item information purchased through the application or another application, and By combining user behavior information for estimating the user size, such as time information at which the information is generated, preference information for a style in which the user is interested in the corresponding time may be generated and reflected in the user fashion database.

In addition, the service server may generate the user's body shape information and reflect it in the user fashion database. For example, when a user device generates a body image of a user's body from multiple angles and transmits it to a service server, the service server models a user's body model from a machine learning framework that learns human body features from a large number of body images. You can create The user body model may include information about the proportions and skin tones of each part of the user's body as well as the size information of each part of the user's body.

According to a further embodiment of the present invention, the service server may generate preference information for a user's fashion item and reflect it in the user fashion database. The preference information may include information about a user's preferred price, preferred brand, and preferred use. For example, when a user browses or purchases a fashion item through an online market on a user device, the service server reflects different weights for viewing or purchasing to generate information about a preferred price, a preferred brand, and a preferred use, and the user fashion database. Can be reflected in.

In particular, the service server according to an embodiment of the present invention has a feature of estimating a user's “flavor” corresponding to a human feeling, and generating the estimated taste information in a form recognizable by a computer and reflecting it in a user fashion database.

For example, the service server may extract a label for estimating the user's taste from the user's behavior information. The label may be extracted as the meaning of fashion items included in user behavior information, such as a style book viewed by a user, an item that generates a tag like a tag, a request item, or a purchase item. Furthermore, the label may be generated as information about a look and feel, such as the appearance and feel of fashion items included in user behavior information, and trends.

The label generated from the user behavior information is weighted according to the user behavior, and the service server may generate user size information for estimating the user size by combining it and store it in the user fashion database. The user size information, user body shape information, and user preference information included in the user fashion database may be used to set the exposure priority for the recommended item or the recommended product.

Accordingly, the service server may determine a fashion pattern based on a specific date and search for an item similar to a specific fashion item (S220). That is, the service server is based on a specific date related to a request for a recommended service, and a fashion pattern determined based on a specific date from a style database consisting of style images extracted with style labels representing human feelings as data recognizable by a computer. Based on the image similarity, it is possible to search for an item similar to a specific fashion item.

In the present invention, a criterion for determining a fashion pattern may be determined based on a specific date, according to the repetition degree of style labels appearing in style images, or an expert or semi-professional combination of a plurality of fashion items in advance. For example, as of December 25, 2018, related to the request for a recommendation service, style labels such as'party look' and'christmas date look' are repeated more than a certain number of times (eg 5 times), or fashionable patterns by experts/semi-professionals If it is determined, a similar item may be searched based on the image similarity by mixing with a fashion item extracted in association with calendar data of the previous year from the user fashion database.

Alternatively, when the service server receives the recommendation service request, the specific fashion item associated with the calendar data corresponding to the date associated with the recommendation service may be extracted from the user fashion database, and then a similar item may be searched by comparing the fashion pattern. . That is, when a request for a fashion item recommendation service related to an event called'meeting' is received as described above, the fashion items associated with the calendar data called'meeting' among user images are searched, and specific fashion items are extracted based on this. Can. If weather information is used, it is recommended to search for fashion items worn by the user in relation to'image 5 degrees, humidity 20%', and extract specific fashion items to compare fashion patterns based on a specific date. You can also search for similar items for items.

When the service server searches for an item similar to a specific fashion item stored in the user fashion database according to the date data and the fashion pattern, the service server provides at least one recommendation product included in the same category or a different category using the product database and the style database (S230).

For example, a photo of a specific fashion item is extracted, and a recommendation item of the same fashion category can be provided, or another category item can be recommended/provided as a coordination item that goes well with it.

In the present invention, the style database may include information about a fashion image, a fashion image that can be referenced for coordination of a plurality of items, among images collected on the web. The style database may include, among images collected online, images combined with a plurality of fashion items to match well (referred to herein as a style image) and classification information for the style image. The style image according to the embodiment of the present invention is a fashion catalog, a fashion magazine pictorial image, a fashion show shooting image, an idol costume image, a specific drama that can be collected on the web as image data generated by combining a plurality of fashion items in advance by an expert or an expert Or, you can exemplify a movie's costume image, SNS, blog celebrity's costume image, fashion magazine's street fashion image, or an image coordinated with other items to sell fashion items.

Thus, the style image is stored in a style database according to an embodiment of the present invention, and can be used to determine other items that go well with a particular item. According to this, the style image can be used as a reference for a computer to understand the human feeling that it is generally “fits well”. Machine learning learned about matching of multiple fashion items in order for the computer to recommend another item that “matches well” without human intervention for any item, because “being good” with any item is about the human feeling. You will need a framework. To this end, the service server according to the embodiment of the present invention may collect a plurality of fashion items combined by an expert or a semi-expert and collect a style image worn by a person and generate it as a style database. Furthermore, the service server can train the framework by applying the style database to the machine learning framework. For example, a machine learning framework that has learned a large number of style images with matching blue shirts and brown ties could recommend brown ties as a coordination item for requests for blue shirts.

In addition, in order to construct a style database, the service server may collect style images online. For example, the service server collects a list of web addresses such as fashion magazines, fashion brands, drama makers, celebrity planners, SNS, online stores, etc., and checks the website to track the link. Can be collected.

Alternatively, the style database may include a fashion item extracted from the user images described above. In this case, when a link through a web page or a purchase is made to the extracted fashion item, a point for material compensation for a user related to the user image may be set. This is defined as a link point in the present invention, and can be used to compensate a user in various forms such as points and mileage.

On the other hand, the service server according to the embodiment of the present invention can collect and index images from websites such as fashion magazines, fashion brands, drama makers, celebrity planners, SNS, online stores, etc. Information may be provided separately.

Accordingly, the service server may filter images that are not suitable for style recommendation among the collected images. For example, the service server may filter the remaining images, leaving only the images containing the human-shaped objects among the collected images and the plurality of fashion items.

Filtering images for a single fashion item is appropriate because the style image is used to determine the request item and other items that can be coordinated. Furthermore, constructing a database with images of a person wearing a plurality of fashion items directly may be more useful than images of the fashion items themselves. Therefore, the service server according to an embodiment of the present invention may determine the style image included in the style database by filtering the remaining images, leaving only the image including the human-shaped object and the plurality of fashion items.

Thereafter, the service server may process features of the fashion item object image included in the style image. More specifically, the service server may extract image features of the fashion item object included in the style image, express feature information as vector values, generate feature values of the fashion item object, and structure feature information of the images.

Furthermore, the service server according to an embodiment of the present invention may extract a style label from a style image and cluster style images based on the style label. It is appropriate that the style label is extracted with respect to the look and feel of the fashion item's appearance, feel, and trends. According to a preferred embodiment of the present invention, it is possible to extract a label for a feeling that a person can feel from the appearance of a single fashion item included in a style image, a combination of a plurality of items, and use it as a style label. For example, a celebrity look, a magazine look, a summer look, a feminine look, a sexy look, an office look, a drama look, and a Chanel look can be illustrated as style labels.

According to an embodiment of the present invention, the service server defines a style label in advance, generates a neural network model learning the characteristics of the image corresponding to the style label, classifies objects in the style image, and extracts the label for the object can do. At this time, the service server may assign the corresponding label to the image matching the specific pattern with a random probability through the neural network model learning the pattern of the image corresponding to each label.

According to another embodiment of the present invention, the service server may learn characteristics of an image corresponding to each style label to form an initial neural network model, and apply a large number of style image objects to it to expand the neural network model more precisely. have.

Meanwhile, according to another embodiment of the present invention, the service server may apply style images to a neural network model formed of a hierarchical structure formed of a plurality of layers without separate learning of labels. Furthermore, weighting is applied to the feature information of the style image according to the request of the corresponding layer, and the product images are clustered using the processed feature information, and the celebrity look, magazine look, summer look, feminine look, and sexy look to the clustered image group , Labels that are interpreted ex post as office look, drama look, or chanel look can be given.

Accordingly, the service server may cluster style images using a style label and generate a plurality of style books. This is to be provided as a reference to the user. The user may browse a specific style book among a plurality of style books provided by the service server, find a favorite item, and request a product information search for the corresponding item.

Meanwhile, the service server may pre-classify items having a very high appearance rate, such as white shirts, jeans, and black skirts. For example, jeans are very basic items in fashion, so the appearance rate in style images is very high. Therefore, no matter what item the user inquires about, the probability of matching jeans as a coordination item will be significantly higher than other items.

Therefore, the service server according to an embodiment of the present invention can pre-classify an item having a very high appearance rate in a style image as a buzz item, and generate a style book with different versions, including a buzz item and a buzz item. have.

According to another embodiment of the present invention, buzz items may be classified by reflecting time information. For example, considering the fashion cycle of a fashion item, it is possible to consider items that fad and disappear for a month or two, fashion items that return each season, and items that are continuously fashionable for a certain period of time. Therefore, by reflecting time information in the classification of the buzz item, if a specific fashion item has a very high appearance rate during an arbitrary period, the item may be classified as a buzz item along with information on the corresponding period. When the buzz item is classified as described above, in the subsequent item recommendation step, there is an effect that can be recommended in consideration of whether the item to be recommended is fashionable or unrelated to fashion.

That is, the service server according to the present invention can process a specific fashion item object included in the received request, and search a style database based on image similarity. That is, the service server may search for a similar item in the style database by processing an image object specified as a search target.

To this end, the service server may extract characteristics of the image object to be searched and structure specific information of images for efficiency of search.

Furthermore, the service server according to an embodiment of the present invention can extract label and/or category information on the meaning of the object image to be searched by applying the machine learning technique used to construct the product image database to the processed object image to be processed. have. The label may be expressed as an abstracted value, but may also be expressed in text form by interpreting the abstracted value.

For example, the service server according to an embodiment of the present invention may extract labels for women, dresses, sleeveless, linen, white, and casual look from the request object image. In this case, the service server may use labels for women and dresses as category information of the requested object image, and labels for sleeveless, linen, white, and casual look may be used as label information describing characteristics of the object image outside the category. .

Thereafter, the service server may search the style database based on the similarity of the request object image. This is for retrieving items similar to the request image from the style database to identify other items matching the similar items in the style image. For example, the service server may display the request object image and the fashion item object images included in the style image. The similarity of the feature values can be calculated, and an item whose similarity is within a preset range can be checked.

Furthermore, the service server according to the embodiment of the present invention processes the feature values of the request image by reflecting the weights required by the multiple layers of the artificial neural network model for machine learning configured for the product database, and within a certain range with the request image At least one fashion item group included in a style book having a distance value may be selected, and items belonging to the group may be determined as similar items.

Meanwhile, according to a preferred embodiment of the present invention, the service server determines a similar item by searching the style database based on the similarity of the requested image, and uses label and category information extracted from the image to increase the accuracy of the image search. Can.

For example, the service server calculates the similarity between the feature values of the requested image and the style database image, and among the products having a similarity of a predetermined range or higher, the label and/or category information does not match the label and/or category information of the requested image. Similar items can be determined by excluding products.

As another example, the service server may calculate the item similarity only in the style book having label and/or category information matching the label and/or category information of the request image.

For example, the service server according to an embodiment of the present invention may extract a style label from a request image, and specify a similar item based on the request and image similarity in a style book matching the label. Of course, the service server may specify a similar item based on the similarity of the requested image and the image in the style database without extracting a separate label from the requested image.

For example, if there is a leaf pattern dress in the image included in the request, the service server may extract a label of tropical from the request. Thereafter, the service server may identify a similar item having a similarity of a leaf pattern dress and a preset range in a style book clustered with a label of tropical.

Thereafter, the service server includes a similar item retrieved from the style book, and may provide a user device with a style image in which the similar item is combined with other fashion items. In the above example with a leaf pattern dress, a style image in which straw hats, rattan bags, etc. are combined with a leaf pattern dress can be provided to the user.

Accordingly, when an item similar to a specific fashion item is retrieved from the style database, the service server may determine a coordination item by combining the similar items with the similar items to identify fashion items of other categories included in the style image.

That is, a specific fashion item inquired by the user may be searched based on the image similarity in the style database, and a fashion item of another category matched with the similar item in the style image including the similar item may be considered as a recommended item. This is because the service server according to the embodiment of the present invention is learned to match other items that match with the requested item in the style image.

When the coordination item is determined using the style database, the service server may determine a product similar to the coordination item from the product database as a recommended product.

The product database may include product detail information such as origin, size, sales place, and wear shot of products sold in the online market, and is characterized by composing product information based on the image of the product.

That is, the service server may collect product information for products sold in an arbitrary online market as well as product information of an online market affiliated in advance. For example, the service server may include a crawler, a parser, and an indexer, collect web documents of online stores, and access text information such as product images, product names, and prices included in web documents.

For example, a crawler may collect a list of web addresses of an online store, check a website, and track links to deliver data related to product information to a service server. At this time, the parser analyzes web documents collected during the crawling process to extract product information such as product images, product prices, and product names included in the page, and the indexer can index the corresponding location and meaning.

Meanwhile, the service server according to an embodiment of the present invention may collect and index product information from a website of any online store, but may receive product information in a preset format from an affiliate market.

The service server can process the product image. This is for determining a recommended item based on whether the product image is similar, without relying on text information such as a product name or a sales category.

According to an embodiment of the present invention, a recommended item may be determined based on whether the product image is similar, but the present invention is not limited thereto. That is, depending on the implementation, the product image as well as the product name or sales category may be used as a single or secondary request. To this end, the service server may generate a database by structuring text information such as a product name and product category other than the product image.

According to a preferred embodiment of the present invention, the service server can extract the features of the product image and index the feature information of the images for efficiency of search.

More specifically, the service server may detect feature areas of product images (Interest Point Detection). The feature region refers to a main region for extracting a descriptor for a feature of an image, that is, a feature descriptor, to determine whether the images are identical or similar.

According to an embodiment of the present invention, such a feature area may be a contour that an image includes, a corner such as a corner among contours, a blob separated from the surrounding area, an area that is invariant or covariant according to image deformation, or ambient brightness. It can be a pole with dark or light features, and it can be a patch (fragment) of the image or the entire image.

Furthermore, the service server may extract feature descriptors from the feature area. The feature descriptor is a vector value representing features of an image.

According to an embodiment of the present invention, such a feature descriptor can be calculated using the location of the feature region for the corresponding image, or the brightness, color, sharpness, gradient, scale, or pattern information of the feature region. For example, the feature descriptor may calculate the brightness value of the feature region, the change value of the brightness, or the distribution value by converting it to a vector.

On the other hand, according to an embodiment of the present invention, the feature descriptor for the image is a local descriptor based on the feature area as described above, as well as a global descriptor, a frequency descriptor, a frequency descriptor, a binary descriptor, or the like. It can be expressed as a neural network descriptor.

More specifically, the feature descriptor is a global descriptor that converts and extracts the brightness, color, sharpness, gradient, scale, pattern information, etc. of each image or a region where the image is divided by an arbitrary reference, or each feature region into a vector value ( Global descriptor).

For example, the feature descriptor is a frequency descriptor (Frequency Descriptor) that converts and extracts the number of specific descriptors previously included in the image, the number of inclusions of global features such as a previously defined color table, and the like into a vector value. Binary descriptor, which is extracted and used as an integer after extracting in bit units whether the size of each element constituting or including the descriptor is larger or smaller than a specific value, learns from the layer of the neural network Or, it may include a neural network descriptor (Neural Network descriptor) for extracting the image information used for classification.

Furthermore, according to an embodiment of the present invention, it is possible to convert the feature information vector extracted from the product image to a lower dimension. For example, the feature information extracted through the artificial neural network corresponds to 40,000-dimensional high-dimensional vector information, and it is appropriate to convert to a low-dimensional vector in an appropriate range in consideration of resources required for search.

Various feature reduction algorithms such as PCA and ZCA can be used to transform the feature information vector, and feature information converted into a low dimensional vector can be indexed into a corresponding product image.

Furthermore, the service server according to an embodiment of the present invention can extract a label for the meaning of the image by applying a machine learning technique based on the product image. The label may be expressed as an abstracted value, but may also be expressed in text form by interpreting the abstracted value.

More specifically, the service server may define a label in advance, generate a neural network model learning the characteristics of the image corresponding to the label, classify objects in the product image, and extract labels for the corresponding object. At this time, the service server may assign the corresponding label to the image matching the specific pattern with a random probability through the neural network model learning the pattern of the image corresponding to each label.

Alternatively, the service server may learn characteristics of an image corresponding to each label to form an initial neural network model, and apply a large amount of product image objects to it to expand the neural network model more precisely. Furthermore, the service server may create a new group including the product if the product is not included in any group.

Accordingly, the service server predefines a label that can be used as meta information about the product such as female bottoms, skirts, dresses, short sleeves, long sleeves, patterns, materials, colors, and abstract feelings (innocent, chic, vintage, etc.) , A neural network model learning the characteristics of the image corresponding to the label may be generated, and the neural network model may be applied to the advertiser's product image to extract a label for the product image to be advertised.

Alternatively, the service server may apply product images to a neural network model formed of a hierarchical structure formed of a plurality of layers without separate learning of labels. Furthermore, the feature information of the product image may be weighted according to the request of the corresponding layer, and the product images may be clustered using the processed feature information.

In this case, additional analysis may be necessary to determine whether the corresponding images are clustered according to which attribute of the feature value, that is, to connect the clustering results of the images with a concept that can be recognized by a real human. For example, when the service server classifies products into three groups through image processing, and extracts the labels A for characteristics of the first group, B for characteristics of the second group, and C for characteristics of the third group. It needs to be interpreted ex postly, that A, B, and C, respectively, refer to female tops, blouses, and plaids, respectively.

The service server gives the clustered image group a label that can be interpreted ex post with female bottoms, skirts, dresses, short sleeves, long sleeves, patterns, materials, colors, and abstract feelings (pure, chic, vintage, etc.) , Labels assigned to an image group to which an individual product image belongs may be extracted as a label of the corresponding product image.

Meanwhile, the service server according to an embodiment of the present invention may express a label extracted from a product image as text, and a text-type label may be used as tag information of the product.

In the related art, the tag information of the product is directly and subjectively given by the seller, resulting in inaccuracy and poor reliability. The product tag subjectively assigned by the seller has a problem of lowering the efficiency of search by acting as noise.

However, as in the exemplary embodiment of the present invention, when label information is extracted based on a product image, and the extracted label information is converted into text and utilized as tag information of the corresponding product, the tag information of the product is based on the image of the corresponding product. Since it can be extracted mathematically without human intervention, it has the effect of increasing the reliability of the tag information and improving the accuracy of the search.

Furthermore, the service server may generate category information of the corresponding product based on the content of the product image. For example, if a label for an arbitrary product image is extracted as a woman, top, blouse, linen, stripe, long sleeve, blue, office look, the service server sets the label for a woman, top, or blouse as the category information of the product. Labels for linen, stripe, long-sleeved, blue, and office look can be used as label information describing the characteristics of products outside the category. Alternatively, the service server may index the corresponding product without distinguishing the label and category information. At this time, the category information and/or label of the product may be used as a parameter to increase the reliability of the image search.

Accordingly, the service server determines an item similar to the coordination item from the product database configured by indexing the label extracted from the content of the above-mentioned product as a recommended item, and provides product information for the recommended item, similar to the recommended item You can search for products in the product database.

More specifically, the service server may search the product database based on the image similarity for the coordination item determined using the style database. To this end, the service server can extract characteristics of the coordination item object and structure specific information of images for efficiency of search.

The service server according to an embodiment of the present invention may search the product database based on the similarity of the object image. For example, the service server may calculate the similarity between the feature values of the recommended item image and the product image included in the product database, and determine a product whose similarity is within a preset range as a recommended product.

Furthermore, the service server according to an embodiment of the present invention processes the feature values of the recommended item image by reflecting the weights required by multiple layers of the artificial neural network model for machine learning configured for the product database, and the distance within a certain range At least one product group having a value may be selected, and products belonging to the group may be determined as recommended products.

Furthermore, the service server according to another embodiment of the present invention may specify the recommended product based on the label extracted from the recommended item object.

For example, when the label information of the object extracted from the recommended item image is extracted as a female top, blouse, white, and stripes, the service server searches the object image and the object image to be searched only for product groups that have the female top as category information in the product database. Similarity can be calculated.

As another example, the service server may set products having a degree of similarity or higher than a predetermined range as a candidate candidate for recommendation, and may exclude products whose sub-category information is not a blouse from the recommended candidate. In other words, products with sub-category information indexed as blouses may be selected as advertisement items.

As another example, if the label information extracted from the object image of the recommended item is a woman's top, blouse, long sleeve, lace, and collar neck, the service server in the product database has a woman top, blouse, long sleeve, lace, and collar neck as a label. It is also possible to calculate the similarity between the recommended item and the image for the group only.

When the recommended product is determined, the service server may determine the priority of exposure by reflecting user preference/size information. For example, when the user is interested in the office look, the weight of the office look label may be used to calculate the priority and provide recommended product information according to the calculated priority.

The embodiments of the present invention disclosed in the present specification and drawings are merely to provide a specific example to easily explain the technical content of the present invention and to understand the present invention, and are not intended to limit the scope of the present invention. It is obvious to those of ordinary skill in the art to which the present invention pertains that other modifications based on the technical spirit of the present invention can be implemented in addition to the embodiments disclosed herein.

The service for recommending a fashion item to a user using a date related to the request for the recommended service as described above can be applied to various services.

Claims (6)

1. In a method of providing a fashion item recommendation service to a user using a service server,

   Upon receiving a recommendation service request, based on a specific date related to the recommendation service request, a specific fashion item associated with calendar data on a predetermined period before and after the specific date of a previous year from a user fashion database Extracting;

   From a style database consisting of style images from which style labels are extracted that express a person's feelings with computer-recognizable data, based on the fashion pattern determined based on the specific date, similar to the specific fashion item based on image similarity Searching for an item;

   Retrieving a candidate item of the same category similar to the similar item based on an image similarity from a product database configured by indexing a label extracted from the content of the product;

   Determining an item of a different category from the similar item as a coordination item from the style image in which the similar item is searched;

   Retrieving, from the product database, another category candidate item similar to the coordination item based on image similarity; And

   Providing at least one of the same category candidate item and the other category candidate item as a recommended product,

   How to provide fashion item recommendation services.
2. According to claim 1,

   The predetermined period,

   It is designated in advance for fashion item coordination, or is set to one of day, week, or month units input from the user,

   The fashionable pattern,

   Based on the specific date, it is determined according to the repetition degree of style labels appearing in style images,

   How to provide fashion item recommendation services.
3. According to claim 1,

   The user fashion database,

   Fashion items extracted from user images and the user images and calendar data are generated to be associated,

   The calendar data,

   It includes weather information related to at least one of temperature, humidity, UV index, wind, and fine dust ozone based on time data and location data included in the user images, and is associated with the time data of the user images. Contains user event information obtained through the schedule program installed in,

   The step of extracting the specific fashion item,

   Characterized in that it is further performed based on the update (update) as to whether or not to own a fashion item from the user,

   How to provide fashion item recommendation services.
4. The method of claim 3,

   The step of extracting the specific fashion item,

   Characterized in that it is further extracted based on the fashion item corresponding to the calendar data associated with the specific date (date),

   How to provide fashion item recommendation services.
5. The method of claim 3,

   The step of extracting the specific fashion item,

   In order to recommend a fashion item based on the specific date, the fashion item corresponding to the calendar data in one of a predetermined day, week, or month unit is further extracted based on the date. Characterized by,

   How to provide fashion item recommendation services.
6. According to claim 1,

   The recommended service request,

   Recognizing the voice of the user, characterized in that it is determined to request a fashion item recommendation service for the specific date,

   How to provide fashion item recommendation services.

Images