KR102200038B1 - A method of providing a fashion item recommendation service to a user using a date - Google Patents

Abstract

The present invention relates to a method of providing a fashion item recommendation service to a user using a service server. Specifically, upon receiving a recommendation service request, a specific fashion associated with calendar data in a predetermined period before and after a specific date in the previous year from the user fashion database, based on a specific date regarding the recommendation service request. The step of extracting an item, based on a fashion pattern determined on a specific date, from a style database consisting of style images from which style labels expressing human feelings as computer-recognizable data, and based on image similarity Searching for items similar to fashion items, searching for candidate items in the same category similar to similar items based on image similarity from a product database configured by indexing labels extracted from the contents of the product, and searching for similar items from style images Determining an item of a similar item and a different category as a coordination item, searching for another category candidate item similar to the coordination item based on image similarity from the product database, and at least one of the same category candidate item and another category candidate item It characterized in that it comprises the step of providing as a recommended product.

Description

A METHOD OF PROVIDING A FASHION ITEM RECOMMENDATION SERVICE TO A USER USING A DATE}

The present invention relates to a method of providing a fashion item recommendation service to a user, and more particularly, to a method of providing a fashion item recommendation service by using a date.

In the background of the recently increased wired/wireless internet environment, online commerce such as promotion and sale are being activated. In this regard, when buyers find a product they like while searching for a magazine, blog, or YouTube video on a desktop or mobile terminal connected to the Internet, they search for a product name and lead to purchase. For example, the name of a bag that a famous actress heard at the airport and the name of childcare products from entertainment programs are ranked at the top of the portal site's real-time search terms. However, in this case, the user must separately open a web page for search and search for a product name, a manufacturer, a retailer, and the like, and there is an inconvenience in that it is difficult to easily search unless clear information about them is already known.

Meanwhile, sellers spend a lot of money on media sponsorship and online user recruitment in addition to commercial advertisements for product promotion. This is because online word of mouth acts as an important variable in product sales these days. However, in spite of the expenditure of public relations expenses, it is often not possible to disclose shopping information such as product names and vendors. This is because indirect advertising issues may arise because media viewers cannot individually obtain prior approval for product name exposure.

As described above, there is a need for both users and sellers to provide shopping information for online product images in a more intuitive UI (User Interface) environment.

Based on the above discussion, in the following, a fashion item recommendation service will be provided using the date.

The technical problems to be achieved in the present invention are not limited to the above technical problems, and other technical problems that are not mentioned will be clearly understood by those of ordinary skill in the technical field to which the present invention belongs from the following description.

According to an aspect of the present invention for solving the above-described problem, a method of providing a fashion item recommendation service to a user using a service server, when receiving a recommendation service request, is performed on a specific date related to the recommendation service request. Extracting a specific fashion item associated with calendar data on a predetermined period before and after the specific date of a previous year from a user fashion database based on the user fashion database; Similar to the specific fashion item based on image similarity, based on the fashion pattern determined based on the specific date, from a style database composed of style images from which a style label expressing human feeling as data recognizable by a computer Retrieving the item; Searching for a recommendation item in the same category similar to the similar item based on image similarity from a product database configured by indexing labels extracted from the contents of the product; Determining an item of a category different from the similar item as a coordination item from the style image from which the similar item is retrieved; Searching for another category candidate item similar to the coordination item based on image similarity from the product database; And providing at least one of the same category candidate item and the other category candidate item as a recommended product.

Further, the predetermined period is set in advance for fashion item coordination or is set as one of a day, week, or month unit input by the user, and the fashion pattern May be determined according to the degree of repetition of style labels appearing in style images based on the specific date.

Further, the user fashion database is generated to associate fashion items extracted from user images, the user images and calendar data, and the calendar data is a time included in the user images. It includes weather information related to at least one of temperature, humidity, ultraviolet index, wind, and ozone amount of fine dust based on data and location data, and is obtained through a schedule program installed on the user device in association with the time data of the user images. The step of extracting the specific fashion item including user event information may be performed further based on an update on whether the user owns the fashion item.

Preferably, the step of extracting the specific fashion item comprises extracting further based on a fashion item corresponding to calendar data associated with the specific date, or for recommending fashion items based on the specific date. For this, it may be characterized in that the fashion item corresponding to the calendar data within one of a predetermined unit of day, week, or month is further extracted based on the data.

Further, the recommendation service request may be characterized in that it is determined to request a fashion item recommendation service for the specific date by recognizing the user's voice.

According to an embodiment of the present invention, a fashion item recommendation service can be efficiently provided using a date.

The effects that can be obtained in the present invention are not limited to the effects mentioned above, and other effects not mentioned can be clearly understood by those of ordinary skill in the art from the following description. will be.

The accompanying drawings, which are included as part of the detailed description for aiding understanding of the present invention, provide embodiments of the present invention, and together with the detailed description will be described the technical idea of the present invention.
1 is a reference diagram for explaining a system for providing a fashion item recommendation service using a date according to an embodiment of the present invention.
2 is a reference diagram illustrating a method of providing a fashion item recommendation service to a user using a date according to an embodiment of the present invention.

The present invention is not limited to the description of the embodiments to be described below, and it is obvious that various modifications may be made without departing from the technical gist of the present invention. Further, in describing the embodiments, descriptions of technical contents that are widely known in the technical field to which the present invention pertains and are not directly related to the technical gist of the present invention will be omitted.

Hereinafter, it is assumed that the user device on which product information is displayed is a mobile device, but the present invention is not limited thereto. That is, in the present invention, the user device should be understood as a concept including all types of electronic devices capable of requesting search and displaying advertisement information, such as a desktop, a smart phone, and a tablet PC.

In addition, it should be noted that the concept of goods in this specification is not limited to tangible goods. That is, in the present specification, a product should be understood as a concept including not only tangible products but also intangible services that can be sold.

Furthermore, in the present specification, the term displayed page in an electronic device refers to a screen loaded in an electronic device and/or content inside the screen so that it can be immediately displayed on the screen according to the user's scroll. It can be understood as an inclusive concept. For example, on the display of the mobile device, the entire execution screen of the application, which is extended horizontally or vertically and displayed according to the user's scroll, may be included in the concept of the page, and the screen in the camera roll is also included in the concept of the page I can.

Meanwhile, in the accompanying drawings, the same elements are represented by the same reference numerals.

In addition, some components in the accompanying drawings may be exaggerated, omitted, or schematically illustrated. This is to clarify the gist of the present invention by omitting unnecessary description not related to the gist of the present invention.

1 is a reference diagram for explaining a system for providing a fashion item recommendation service using a date according to an embodiment of the present invention. The recommended service providing system according to the present invention includes a user device 110 and a service server 120, and is connected to a home appliance 130 including an AI speaker 131 and a smart TV 133. Can be configured.

The user device 110 may store a user's schedule, image, etc., and a service server through a wired/wireless network. It includes smart phones and smart pads that can transmit and receive data with other devices of home appliances, including AI speakers and smart TVs. In the present invention, the user device 110 stores user's schedule (eg meeting, travel...) information, weather information application (eg K-weather, Naver weather..., etc.), schedule management application (eg Google calendar, Naver calendar... It is desirable to have access to etc.

The service server 120 provides a fashion item recommendation service to a user based on time and location information related to the user image received from the user device 110, user schedule information, and user event. Here, the user image may be an image on which image pre-processing has been performed through a user device or a service server. For example, the user image may include a certain pattern such as a room background, and the head and feet of the user, which are subjects, are separated from the captured image, and the part except the user, which is the subject, is treated as a blank or the user's A processing process may be performed to determine the overall fit. In addition, in relation to the generation of the image, the user image may include time data such as the image creation date, time, and day of the week, and location data such as GPS coordinates at which the image was generated, and stored in the image. In addition, weather information according to the creation date of the image may be obtained from the user device through the weather information application based on the time data and location data of the user image, but only with the time and location data of the user image, the server or other Weather information corresponding thereto may be obtained on the device.

The home appliance 130 includes a voice recognition device 131 capable of recognizing a user's voice and processing commands and data, such as an AI speaker, a smart TV capable of providing images, images, and the like to the user, and a smart pad. It may include a display device 133, an image photographing device capable of photographing a picture, a video, and the like.

The voice recognition device 131 is defined as a device capable of processing data by recognizing a user's voice, such as Siri of Apple, Google Home of Google, NUGU of SKT, and Alexa of Amazon, and artificial intelligence. intelligence) may be applied to process data according to the user's voice and additional operations in consideration of the user's schedule (eg, a coordination recommendation service according to the weather, a coordination recommendation service in consideration of events such as weddings and dinners in the future) can be performed.

The display device 133 may display user images or recommended fashion items provided by a service server to the user, and the display device may be a web page through which a user can purchase fashion items recommended to a user through a wired or wireless network. , eg shopping mall, Soho) and can provide a service.

Further, the image capturing module may photograph a subject to generate an identifiable digital image, and, as described above, may generate the image including time and location data related to the generation of the image.

2 is a reference diagram for explaining a method of providing a fashion item recommendation service to a user using a specific date according to an embodiment of the present invention.

When the service server receives a request for a recommended service from a user, the service server stores a specific fashion item associated with calendar data in a predetermined period before and after a specific date from the previous year based on a specific date related to the recommendation service request. Extract from Here, the predetermined period may be previously designated by a user or the like for fashion item coordination, or may be set as one of a day, week, or month unit input from the user. .

Here, the user fashion database is generated to associate fashion items extracted from user images, user images and calendar data, and the calendar data is included in time data and location data included in the user images. Based, it includes weather information related to at least one of temperature, humidity, ultraviolet index, wind, and ozone amount of fine dust, and includes user event information acquired through a schedule program installed on the user device in association with time data of user images. I can. For example, if the recommendation service request is related to December 25, 2018, a specific fashion item related to calendar data for a week before and after December 25, 2017 may be extracted. For example, calendar data may include temperature, humidity, UV index, wind, fine dust, or ozone amount information, etc., based on time data and location data included in user images. The image was created by taking a picture in Myeong-dong, Korea at 20:00 on the 25th, and at 20:00 on December 25th, 2017, the temperature in Myeong-dong was minus 2 degrees, the humidity was 52%, the UV index was 25%, and the wind was 3m northwest. Weather information such as /s, fine dust is bad, and ozone content is low can be defined as calendar data.

In addition, the user's schedule and data related to the user's activity that may be included in the calendar data may be automatically acquired through a schedule management program installed on the user device in association with the user's time data, or may be manually input from the user. Since the user's schedule is generally different for each individual user, unlike the above-described weather information, it is preferable to be acquired on the user device or added/modified according to the user's input. Therefore, if the schedule management program (e.g., Google Calendar, etc.) has a schedule called'Date' all day on December 25, 2018, or if the user randomly inputs'Date with XXX', an event called'Date' It can be included in the calendar data.

Accordingly, for example, the service server is associated with the 2017 calendar data, not only the fashion items worn on that date, but also one week from December 18, 2017 to 2018 Fashion items until January 01 can be extracted. Alternatively, even if there is a request for a fashion item recommendation service on December 25, 2018, the user can recommend fashion items for styling in consideration of schedules, events, weather, etc., two weeks later, before or after January 08, 2018. It can also be connected to a shopping mall for additional purchases in advance.

In addition, the user fashion database may reflect an update on whether the user owns the fashion item. For example, when a check is made on fashion items discarded by a user, specific fashion items may be extracted only for fashion items owned by the user.

To describe the user database in more detail, the service server extracts fashion items from user images acquired from an image capturing module, etc., and associates calendar data related to the user image acquired through the user device to the user fashion database. Can be created. For example, among images stored in the user device, images that can be identified by the user are image-processed, and fashion items worn by the user are extracted and sorted based on the user's face. You can create a user fashion database by associating calendar data. In addition, when calendar data is added/modified by the user to the user fashion database, the user fashion database may be updated.

Therefore, if a picture of a user A wearing a white bag and a brown suit exists on the user device, and the picture is related to calendar data in which an event called'meeting' exists, the face of the user A is 'White bag' and'brown suit' can be stored in the user's fashion database along with an association called'meeting'. Therefore, when a user requests a fashion recommendation service for an event called'meeting', the service server searches for fashion items associated with an event called'meeting' among the learned style database and user images using fashion magazines. And, based on this, recommended products can be provided.

In addition, when a product image that can be viewed on a web page (for example, an online shopping mall, a clothing brand homepage) or a linked product image is input by a user, a fashion item for the product image may be added to the user fashion database. For example, the user A can capture an image from the online page of the shopping mall where the bag was purchased and add it to the user's fashion database. When a user enters a link to a specific site, the image of that site is added to the user's fashion database. It could be.

The user fashion database may include information on fashion items, and may include the size of the fashion item, a label expressing the feeling that a person feels in the fashion item as computer-recognizable data, and a picture when a user fits it. For example, the user's fashion database may contain necessary size information such as a user's top, bottom, dress, etc., and the appearance when actually fitting clothes is managed as a photo, allowing users to fit their body shape into consideration. You can make this possible. Alternatively, the fashion items are stored so as to include personal feelings such as #comfort, #tight, and #appropriate, respectively, so that the user can refer to it when considering a fit when selecting a fashion item in the future. In addition, image information when the user fits it, information that can estimate the user's taste, such as user's purchase data and browsing time data, user's size information, and preferred price range for online shopping for fashion items, and usage , May contain information about the brand.

Alternatively, the user fashion database may include user identification information, user behavior information for estimating the user size, user size estimated from the behavior information, and user size information directly received from the user device.

For example, the service server provides a query to the user device for the user's age, gender, occupation, fashion field of interest, pre-owned items, etc., receives a user input for the query, generates user size information, and generates user fashion information. It can be reflected in the database.

The service server includes a time when a user browses an arbitrary style book provided through an application according to an embodiment of the present invention, item information for generating a like tag, a request item, information on fashion items purchased through the application or other application, and By combining user behavior information for estimating the user size, such as time information at which the information was generated, preference information for a style that the user is interested in at a corresponding time may be generated and reflected in the user fashion database.

In addition, the service server may generate the user's body shape information and reflect it in the user fashion database. For example, if a user device generates a body image photographing a user's body from multiple angles and transmits it to the service server, the service server uses a machine learning framework that learns human body features from a large amount of body images. Can be created. The user body model may include not only size information of each part of the user's body, but also information about a ratio of each part of the user's body and skin tone.

According to a further embodiment of the present invention, the service server may generate preference information for a user's fashion item and reflect it in the user fashion database. The preference information may include information on a user's preferred price, preferred brand, and preferred use. For example, when a user device browses or purchases a fashion item through an online market, the service server creates information on the preferred price, preferred brand, and preferred purpose by reflecting different weights for the browsing or purchase, and this is the user's fashion database. Can be reflected in.

In particular, the service server according to an embodiment of the present invention has a characteristic of estimating a user's "flavor" corresponding to a human feeling, generating the estimated taste information in a form recognizable by a computer, and reflecting it in a user fashion database.

For example, the service server may extract a label for estimating the user's taste from the user's behavior information. The label may be extracted as the meaning of fashion items included in user behavior information, such as a style book viewed by the user, an item for which a like tag is generated, a request item, and a purchase item. Furthermore, the label may be generated as information on a look and feel such as appearance and feeling of fashion items included in user behavior information, and trends.

The label generated from the user behavior information is applied with a weight according to the user behavior, and the service server may combine it to generate user size information that estimates the user size and store it in the user fashion database. User size information, user body shape information, and user preference information included in the user fashion database may be used to set exposure priority for recommended items or recommended products.

Accordingly, the service server may determine a fashionable pattern based on a specific date and search for an item similar to the specific fashion item (S220). That is, the service server is a fashion pattern determined based on a specific date from a style database composed of style images from which style labels expressing human feeling as computer-recognizable data based on a specific date related to a recommendation service request. Based on, it is possible to search for an item similar to a specific fashion item based on the image similarity.

In the present invention, a criterion for determining a fashion pattern may be determined based on a specific date based on a degree of repetition of style labels appearing in style images, or may be determined by pre-combining a plurality of fashion items by an expert or semi-professional. For example, as of December 25, 2018 related to a recommendation service request, style labels such as'Party Look' and'Christmas Date Look' are repeated a certain number of times (eg 5 times), or a trend pattern by experts/semi-professionals If it is determined as, similar items may be searched based on image similarity by mixing with fashion items extracted in association with calendar data of the previous year from the user fashion database.

Alternatively, when the service server receives a request for a recommendation service, it may extract a specific fashion item related to calendar data corresponding to a date related to the recommendation service from the user fashion database, and then compare it with a fashion pattern to search for similar items. . That is, as described above, upon receiving a request for a fashion item recommendation service related to an event called'meeting', fashion items associated with calendar data called'meeting' among user images are searched, and specific fashion items are extracted based on this. I can. If the weather information is used, it searches for fashion items worn by the user in relation to'image 5 degrees, humidity 20%', extracts specific fashion items from among them, and compares them with popular patterns based on a specific date and recommends them. You can also search for similar items for an item.

When the service server searches for an item similar to a specific fashion item stored in the user fashion database according to date data and fashion patterns, it provides at least one recommended product included in the same category or different categories using the product database and the style database. (S230).

For example, a photo of a specific fashion item may be extracted, and a recommended item of the same fashion category may be provided, or another category item may be recommended and provided as a coordination item suitable for matching with it.

In the present invention, the style database may include information on fashion images that can be referenced for fashion styles and coordination of a plurality of items among images collected on the web. The style database may include an image (referred to herein as a style image) in which a plurality of fashion items are well matched among images collected online, and classification information on the style image. The style image according to the embodiment of the present invention is image data created by combining a plurality of fashion items in advance by an expert or semi-professional, and is a fashion catalog that can be collected on the web, a fashion magazine pictorial image, a fashion show shot image, an idol costume image, a specific drama. Or, the clothes image of a movie, SNS, the costume image of a blog celebrity, a street fashion image in a fashion magazine, an image coordinated with other items for the sale of fashion items, etc. may be exemplified.

Thus, the style image is stored in a style database according to an embodiment of the present invention, and can be used to determine other items that match a particular item well. According to this, a style image can be used as a reference material that a computer can understand about the feeling of a human being that it suits generally. Since “matching well” with an arbitrary item is about human feeling, in order for a computer to recommend another item that “matches well” without human intervention, machine learning learned about matching multiple fashion items You will need a framework. To this end, the service server according to the embodiment of the present invention may collect a style image worn by a person by combining a plurality of fashion items by an expert or a semi-professional, and create a style database. Furthermore, the service server may learn the framework by applying the style database to the machine learning framework. For example, a machine learning framework that learns a large amount of style images matching a blue shirt and a brown tie could recommend a brown tie as a coordination item to a request for a blue shirt.

In addition, in order to configure the style database, the service server may collect style images online. For example, the service server collects a list of web addresses such as fashion magazines, fashion brands, drama production companies, celebrity agencies, SNS, online stores, etc., and checks the website to track links, and image information included on the website Can be collected.

Alternatively, the style database may include fashion items extracted from the above-described user images. In this case, when a link is made through a web page or a purchase is made for the extracted fashion item, points for material compensation for the user related to the user image may be set. This is defined as a link point in the present invention, and can be used to compensate users in various forms such as points and mileage.

Meanwhile, the service server according to the embodiment of the present invention can collect and index images from websites such as fashion magazines, fashion brands, drama production companies, celebrity agencies, SNS, online stores, etc., but images together with index information from affiliated companies Information may be provided separately.

Accordingly, the service server may filter out images unsuitable for style recommendation among the collected images. For example, the service server may filter the remaining images, leaving only images including a person-shaped object among the collected images and including a plurality of fashion items.

Since the style image is used to determine other items that can be coordinated with the requested item, it is appropriate to filter the image for a single fashion item. Furthermore, configuring a database with images of a person wearing a plurality of fashion items directly may be more useful than an image of the fashion item itself. Accordingly, the service server according to the exemplary embodiment of the present invention may determine a style image included in the style database by filtering the remaining images while leaving only images including a person-shaped object and a plurality of fashion items.

Thereafter, the service server may process the characteristics of the fashion item object image included in the style image. More specifically, the service server may extract the image feature of the fashion item object included in the style image and express the feature information as a vector value to generate the feature value of the fashion item object and structure feature information of the images.

Furthermore, the service server according to an embodiment of the present invention may extract a style label from the style image and cluster the style images based on the style label. It is appropriate that the style label is extracted as about the look and feel and trend of the fashion item's appearance and feel. According to a preferred embodiment of the present invention, a label for a feeling that a person can feel from the appearance of a single fashion item included in a style image, a combination of a plurality of items, etc. can be extracted and used as a style label. For example, a celebrity look, a magazine look, a summer look, a feminine look, a sexy look, an office look, a drama look, and a Chanel look may be exemplified as a style label.

According to an embodiment of the present invention, the service server defines a style label in advance, creates a neural network model that learns the characteristics of an image corresponding to the style label, classifies objects in the style image, and extracts a label for the object. can do. In this case, the service server may assign a corresponding label to an image that matches a specific pattern with a random probability through a neural network model that has learned a pattern of an image corresponding to each label.

According to another embodiment of the present invention, the service server may learn characteristics of images corresponding to each style label to form an initial neural network model, and apply a large number of style image objects to it to more elaborately extend the neural network model. have.

Meanwhile, according to another embodiment of the present invention, the service server may apply style images to a neural network model formed in a hierarchical structure formed of a plurality of layers without additional learning about a label. Furthermore, a weight is assigned to the feature information of the style image according to the request of the layer, and product images are clustered using the processed feature information, and the celebrity look, magazine look, summer look, feminine look, and sexy look are added to the clustered image group. , Office look, drama look, Chanel look, etc. can be given a label that is interpreted post-mortem.

Accordingly, the service server clusters style images using the style label and can generate a plurality of style books. This is to be provided as a reference to the user. A user may browse a specific stylebook among a number of stylebooks provided by the service server to find a favorite item, and may request product information search for the item.

Meanwhile, the service server can pre-classify items with a very high appearance rate, such as white shirts, jeans, and black skirts. For example, jeans are a basic item in fashion, so their appearance in style images is very high. Therefore, no matter what item the user inquires about, the probability of matching jeans as a coordination item will be much higher than that of other items.

Therefore, the service server according to the embodiment of the present invention can pre-classify items with a very high appearance rate in the style image as buzz items, and create a stylebook with a different version of one containing a buzz item and one not including a buzz item. have.

According to another embodiment of the present invention, buzz items may be classified by reflecting time information. For example, when considering the fashion cycle of fashion items, items that temporarily disappear after a short period of one or two months, trendy items that return every season, and items that continue to be in fashion for a certain period of time can be considered. Therefore, by reflecting time information in the classification of buzz items, if a specific fashion item has a very high appearance rate during a certain period, the item can be classified as a buzz item together with information about the period. Classifying the buzz items as described above has an effect of being able to recommend the recommended item in consideration of whether the item to be recommended is in fashion or irrelevant to the fashion in a subsequent item recommendation step.

That is, the service server according to the present invention may process a specific fashion item object included in a received request and search a style database based on image similarity. That is, the service server may search for a similar item in the style database by processing the image object specified as a search target.

To this end, the service server may extract features of an image object to be searched and structure specific information of images for search efficiency.

Furthermore, the service server according to the embodiment of the present invention may extract label and/or category information on the meaning of the object image to be searched by applying the machine learning technique used to construct the product image database to the processed object image to be searched. have. The label may be expressed as an abstracted value, but may be expressed in text form by interpreting the abstracted value.

For example, the service server according to an embodiment of the present invention may extract labels for women, dresses, sleeveless, linen, white, and casual look from the request object image. In this case, the service server may use labels for women and dresses as category information of the requested object image, and labels for sleeveless, linen, white, and casual look may be used as label information describing characteristics of object images other than categories. .

Thereafter, the service server can search the style database based on the similarity of the requested object image. This is to search for items similar to the requested image in the style database, and to check other items that match the similar items in the style image.For example, the service server can use the request object image and fashion item object images included in the style image. The similarity of the feature values is calculated, and an item whose similarity is within a preset range can be checked.

Furthermore, the service server according to the embodiment of the present invention processes the feature value of the request image by reflecting the weights required by the plurality of layers of the artificial neural network model for machine learning configured for the product database, and processes the feature value of the request image within a certain range. At least one fashion item group included in the stylebook having a distance value may be selected, and items belonging to the group may be determined as similar items.

Meanwhile, according to a preferred embodiment of the present invention, the service server searches the style database based on the similarity of the requested image to determine a similar item, and at this time, the label and category information extracted from the image are used to increase the accuracy of the image search. I can.

For example, the service server calculates the similarity between the requested image and the feature value of the style database image, and the label and/or category information is not matched with the label and/or category information of the requested image among products having a similarity of more than a preset range. Similar items can be determined by excluding products.

As another example, the service server may calculate the item similarity only in the style book having the label and/or category information matching the label and/or category information of the requested image.

For example, the service server according to an embodiment of the present invention may extract a style label from the requested image and specify a similar item based on the similarity of the request and the image in a style book matching the label. Of course, the service server may not extract a separate label from the requested image, and may specify a similar item based on the similarity of the requested image and the image in the style database.

For example, if there is a leaf pattern dress in the image included in the request, the service server may extract a label of tropical from the request. Thereafter, the service server may specify a similar item having a similarity of a predetermined range with a leaf pattern dress in a style book clustered with a label of tropical.

Thereafter, the service server may provide the user device with a style image in which the similar item retrieved from the style book is included and the similar item is combined with other fashion items. In the above example with a leaf pattern dress, a style image in which a straw hat, a ratan bag, and the like are combined with the leaf pattern dress can be provided to the user.

Accordingly, when an item similar to a specific fashion item is searched in the style database, the service server may determine a coordination item by checking fashion items of other categories included in the style image by combining it with the similar item.

That is, a specific fashion item inquired by the user may be searched based on image similarity in the style database, and fashion items of another category matched together with similar items in a style image including similar items may be considered as recommended items. This is because the service server according to an embodiment of the present invention learns that other items matched with the requested item in the style image are well matched.

When the coordination item is determined using the style database, the service server may determine a product similar to the coordination item as a recommended product from the product database.

The product database may include detailed product information such as origin, size, sales place, and wearing shots of products sold in the online market, and has a characteristic of configuring product information based on the image of the product.

That is, the service server may collect product information not only of product information of a pre-affiliated online market, but also of products sold in an arbitrary online market. For example, the service server may have a crawler, a parser, and an indexer to collect web documents of an online store and access text information such as product images, product names, and prices included in the web documents.

For example, a crawler can deliver product information-related data to a service server by collecting a list of web addresses of online stores, checking websites, and tracking links. At this time, the parser analyzes the web document collected during the crawling process to extract product information such as product images, product prices, and product names included in the page, and the indexer can index the corresponding location and meaning.

Meanwhile, the service server according to the embodiment of the present invention may collect and index product information from a website of an online store, but may receive product information in a preset format from an affiliate market.

The service server can process product images. This is to determine a recommended item based on whether or not the product image is similar, without relying on text information such as a product name or sales category.

According to an embodiment of the present invention, a recommended item may be determined based on whether or not a product image is similar, but the present invention is not limited thereto. That is, depending on the implementation, the product image as well as the product name or sales category may be used alone or as an auxiliary request. To this end, the service server may create a database by structuring text information such as product name and product category in addition to the product image.

According to a preferred embodiment of the present invention, the service server may extract features of a product image and index feature information of the images for search efficiency.

More specifically, the service server may detect characteristic areas of product images (Interest Point Detection). The feature region refers to a main region for extracting a feature descriptor, that is, a feature descriptor of an image for determining whether or not the images are identical or similar.

According to an embodiment of the present invention, such a characteristic area is a contour line included in an image, a corner such as a corner among the contours, a blob that is distinguished from the surrounding area, an area that is invariant or co-variable according to the transformation of the image, or more than the ambient brightness. They can be poles with dark or bright features, and can target patches (fragments) of the image or the entire image.

Furthermore, the service server may extract a feature descriptor from the feature domain. A feature descriptor is a representation of the features of an image as a vector value.

According to an embodiment of the present invention, such a feature descriptor may be calculated by using the location of the feature region for the corresponding image, or brightness, color, sharpness, gradient, scale, or pattern information of the feature region. For example, the feature descriptor may be calculated by converting a brightness value, a brightness change value, or a distribution value of the feature region into a vector.

Meanwhile, according to an embodiment of the present invention, a feature descriptor for an image is not only a local descriptor based on a feature region as described above, but also a global descriptor, a frequency descriptor, a binary descriptor, or It can be expressed as a neural network descriptor.

More specifically, the feature descriptor is a global descriptor that converts and extracts the brightness, color, sharpness, gradient, scale, pattern information, etc. of each of the entire image or the image divided by a random criterion, or Global descriptor) can be included.

For example, a feature descriptor is a frequency descriptor that converts and extracts the number of times that specific descriptors classified in advance are included in an image and the number of global features such as a previously defined color table, etc. Learning in a layer of a binary descriptor or neural network that extracts bitwise whether or not the values of each element constituting a descriptor are larger or smaller than a specific value and converts it to an integer type. Alternatively, it may include a neural network descriptor for extracting image information used for classification.

Furthermore, according to an embodiment of the present invention, a feature information vector extracted from a product image can be converted to a lower dimension. For example, feature information extracted through an artificial neural network corresponds to 40,000-dimensional high-dimensional vector information, and it is appropriate to convert it into a low-dimensional vector in an appropriate range in consideration of the resources required for search.

The feature information vector may be converted using various dimensional reduction algorithms such as PCA and ZCA, and the feature information converted into a lower dimensional vector may be indexed into a corresponding product image.

Furthermore, the service server according to the embodiment of the present invention may extract a label for the meaning of the image by applying a machine learning technique based on the product image. The label may be expressed as an abstracted value, but may be expressed in text form by interpreting the abstracted value.

More specifically, the service server may define a label in advance, generate a neural network model that learns characteristics of an image corresponding to the label, classify objects in a product image, and extract a label for the object. In this case, the service server may assign a corresponding label to an image that matches a specific pattern with a random probability through a neural network model that has learned a pattern of an image corresponding to each label.

Alternatively, the service server may learn characteristics of images corresponding to each label to form an initial neural network model, and apply a large number of product image objects to the neural network model to more elaborately extend the neural network model. Furthermore, if the product is not included in any group, the service server may create a new group including the product.

Accordingly, the service server pre-defines labels that can be used as meta information about products such as women's bottoms, skirts, dresses, short sleeves, long sleeves, patterns, materials, colors, and abstract feelings (innocent, chic, vintage, etc.). , A neural network model having learned features of an image corresponding to the label may be generated, and the neural network model may be applied to a product image of an advertiser to extract a label for an advertisement target product image.

Alternatively, the service server may apply product images to a neural network model formed in a hierarchical structure formed of a plurality of layers without additional learning about the label. Furthermore, a weight may be assigned to the feature information of the product image according to a request of a corresponding layer, and product images may be clustered using the processed feature information.

In this case, additional analysis may be required in order to check whether the corresponding images are clustered according to which attribute of the feature value, that is, to connect the clustering result of the images with a concept that can be recognized by a real human being. For example, when the service server classifies products into three groups through image processing, and extracts labels labeled A for the features of the first group, B for the features of the second group, and C for the features of the third group. It needs to be interpreted posteriorly that, A, B, C mean, for example, women's tops, blouses, and plaids respectively.

The service server assigns labels that can be interpreted posteriorly to the clustered image group as women's bottoms, skirts, dresses, short sleeves, long sleeves, patterns of shapes, materials, colors, and abstract feelings (innocent, chic, vintage, etc.). , Labels assigned to an image group to which an individual product image belongs may be extracted as a label of a corresponding product image.

Meanwhile, a service server according to an embodiment of the present invention may express a label extracted from a product image as text, and a label in a text form may be used as tag information of a product.

In the past, tag information of a product was subjectively directly assigned by a seller, so it was inaccurate and less reliable. Product tags subjectively given by sellers act as noise, thereby lowering the efficiency of search.

However, as in the embodiment of the present invention, when label information is extracted based on a product image, and the extracted label information is converted into text and used as tag information of the product, the tag information of the product is used based on the image of the product. Since it can be extracted mathematically without human intervention, the reliability of tag information is improved and the accuracy of the search is improved.

Furthermore, the service server may generate category information of a corresponding product based on the product image content. For example, if a label for an arbitrary product image is extracted as women, tops, blouses, linen, stripes, long sleeves, blue, and office look, the service server uses labels for women, tops, and blouses as the category information of the product. In addition, labels for linen, stripes, long sleeves, blue, and office look can be used as label information describing the characteristics of products other than categories. Alternatively, the service server may index the product without distinguishing between the label and the category information. In this case, the product category information and/or label may be used as a parameter to increase the reliability of image search.

Accordingly, the service server determines an item similar to the coordination item as a recommended item from a product database configured by indexing the label extracted from the contents of the product, and provides product information on the recommended item. Products can be searched in the product database.

More specifically, the service server may search a product database based on image similarity for the coordination item determined using the style database. To this end, the service server may extract features of the coordination item object and structure specific information of images for efficiency of search.

The service server according to an embodiment of the present invention may search a product database based on the similarity of object images. For example, the service server may calculate a similarity between the recommended item image and feature values of the product image included in the product database, and determine a product having a similarity within a preset range as the recommended product.

Furthermore, the service server according to the embodiment of the present invention processes the feature value of the recommended item image by reflecting the weights required by the plurality of layers of the artificial neural network model for machine learning configured for the product database, and processes a distance within a certain range. At least one product group having a value may be selected, and products belonging to the group may be determined as recommended products.

Furthermore, the service server according to another embodiment of the present invention may specify a recommended product based on a label extracted from the recommended item object.

For example, if the label information of the object extracted from the recommended item image is extracted as women's tops, blouses, whites, and stripes, the service server will search for object images and search targets only for product groups having women's tops as higher category information in the product database. Similarity can be calculated.

As another example, the service server may select products having a similarity greater than or equal to a preset range as recommendation candidate products, and exclude products whose sub-category information is not a blouse from the recommendation candidate products. In other words, products whose sub-category information is indexed by blouses can be selected as advertisement items.

As another example, if the label information extracted from the object image of the recommended item is a female top, blouse, long sleeve, lace, or collar neck, the service server is a product that has a female top, blouse, long sleeve, lace, and collar neck as a label in the product database. It is also possible to calculate the similarity of the recommended item and image for only the group.

When the recommended product is determined, the service server may determine the priority of exposure by reflecting user preference/size information. For example, when the user's preference information is focused on the office look, a weight is assigned to the office look label to calculate the priority, and recommended product information may be provided according to the calculated priority.

The embodiments of the present invention disclosed in the present specification and drawings are provided only to provide specific examples to easily explain the technical content of the present invention and to aid understanding of the present invention, and are not intended to limit the scope of the present invention. It is apparent to those of ordinary skill in the art that other modifications based on the technical idea of the present invention can be implemented in addition to the embodiments posted here.

The service for recommending a fashion item to a user using the date associated with the request for the recommendation service as described above can be applied to various services.

Claims (6)

In a method of providing a fashion item recommendation service to a user using a service server,
Upon receiving the recommendation service request, based on a specific date regarding the recommendation service request, a specific fashion item associated with calendar data in a predetermined period before and after the specific date of the previous year is retrieved from the user fashion database. Extracting;
Similar to the specific fashion item based on image similarity, based on the fashion pattern determined based on the specific date, from a style database composed of style images from which a style label expressing human feeling as data recognizable by a computer Retrieving the item;
Searching for the same category candidate item similar to the similar item based on the image similarity from the product database configured by indexing the labels extracted from the contents of the product;
Determining an item of a category different from the similar item as a coordination item from the style image from which the similar item is retrieved;
Searching for another category candidate item similar to the coordination item based on image similarity from the product database; And
Providing at least one of the same category candidate item and the other category candidate item as a recommended product,
The step of searching for the similar item,
By extracting a style label from the style image, generating at least one style book sharing a random style label in the style database, and having a style label matching the event or weather on a specific date regarding the recommendation service request It characterized in that it comprises the step of searching for an item similar to the specific fashion item based on the image similarity in the book,
How to provide fashion item recommendation service. The method of claim 1,
The predetermined period,
It is specified in advance for fashion item coordination, or is set to one of a day, week, or month unit input by the user,
The fashionable pattern,
Based on the specific date, characterized in that it is determined according to the degree of repetition of style labels appearing in style images,
How to provide fashion item recommendation service. The method of claim 1,
The user fashion database,
Fashion items extracted from user images, and the user images and calendar data are generated to be associated,
The calendar data,
It includes weather information related to at least one of temperature, humidity, ultraviolet index, wind, and ozone amount of fine dust, based on time data and location data included in the user images, and is displayed on the user device in association with time data of the user images. Includes user event information acquired through the schedule program installed in,
The step of extracting the specific fashion item,
It characterized in that it is performed further based on an update as to whether or not the user owns the fashion item,
How to provide fashion item recommendation service. The method of claim 3,
The step of extracting the specific fashion item,
It characterized in that the extraction is further based on the fashion item corresponding to the calendar data associated with the specific date,
How to provide fashion item recommendation service. The method of claim 3,
The step of extracting the specific fashion item,
To further extract fashion items corresponding to calendar data within a period of one of a predetermined day, week, or month unit for fashion item recommendation based on the specific date Characterized in that,
How to provide fashion item recommendation service. The method of claim 1,
The recommended service request,
Recognizing the user's voice, characterized in that it is determined to request a fashion item recommendation service for the specific date,
How to provide fashion item recommendation service.

Images