KR20120085707A - System and method for learning user genres and styles and matching products to user preferences - Google Patents

Abstract

The user's fashion preferences are determined based on the user's interaction with a plurality of fashion item items that individually describe the corresponding fashion item. Recommendations are generated to the user of the fashion product based at least in part on the user's fashion preferences.

Description

TECHNICAL AND METHOD FOR LEARNING USER GENRES AND STYLES AND MATCHING PRODUCTS TO USER PREFERENCES}

The present invention relates to a method and system for learning a user genre and style and matching a product according to a user preference, and more particularly, to recommend a fashion product to a user based on the user's fashion preference.

Digital photography has become a very important consumer application. This provides the convenience of individuals to capture and share digital images. Devices that capture digital images are less expensive, and the ability to transfer images from one place to another is one of the driving forces to expand network bandwidth.

Due to the availability of devices and platforms on which relatively low cost memory and digital images can be displayed, the average consumer can use most digital devices on computer readable devices such as hard drives, CD ROMs, and flash memory. Preserve the images. Although applications have been created to help users organize and view digital images, the use of file folders is a major source of organization.

Online learning is a device learning paradigm in which algorithms learn one instance or sample at a time. While offline learning consists of well-established techniques that are fully analyzed, online algorithms, along with multiple applications that learn complex backgrounds, appearance of models, object detection and classification, and predict and model user behavior, It has received a lot of attention for decades.

Online learning may be the only viable solution for applications that use training data that is not present in the batch but collected concurrently with the decision / classification process, so there is a need to design adaptive learning techniques.

On the other hand, offline or batch paradigms must be retrained if there is new or undiscovered data.

A number of online forms of the most popular offline device learning algorithms have been proposed in the literature. Some approaches try to solve the problem of training Support Vector Machines (SVMs) with large amounts of data. Because training SVMs requires solving two-dimensional programming with multiple coefficients equal to the number of members in the training set, bottlenecks arise in memory requirements, and therefore, online alternatives are needed. . One approach is to find incremental decision tree classifiers, which can be updated or retrieved using newly viewed data cases. In order to extend the famous AdaBoost algorithm to online scenarios, a number of comments are presented with a number of interesting variations ranging from semi-supervised boosting to multiple instance learning. It is becoming.

In fashion, people generally have their own distinctive preferences for the style, color and genre of clothing. Their preference for genre and style develops according to their experience. For example, in a real store, consumers can explain their preferences and styles to a clerk who can recommend to them a precise set of clothing and fashion accessories that match the consumer's preferences. However, there is a problem that the above situation cannot be equally applied in online shopping. In online shopping, consumers must search and scan many products to match styles and preferences.

Accordingly, embodiments described herein describe a computer-implemented method or apparatus that can programmatically determine a user's genre preference for style or fashion.

In addition, embodiments may programmatically categorize and categorize fashion products using images, text, and metadata associated with corresponding fashion product content items.

Further, some embodiments are programmatically related to the top fashion product based on the information determined with respect to the genre preference information of the user and / or the information determined for the genre of the style of the fashion product represented by the content item. The service or system providing the determined recommendation can be implemented.

More specifically, the embodiments described herein include a computer implemented method of determining user preferences for fashion goods. In an embodiment, the user's fashion preferences are determined based on the user's interaction with a plurality of fashion product content items that individually describe the corresponding fashion product. The recommendation is provided to the user of the fashion product, at least in part based on the user's fashion preferences.

According to another embodiment, a fashion product content item is analyzed to determine a set of features of the fashion product that are being depicted in the fashion product content item. Based on the quantitative analysis of the set of determined features, the fashion product is associated with a predefined descriptive category for each of a plurality of descriptive classifications. A predefined descriptive category for each of the merchandise content item and its plurality of descriptive classifications is used to predict or determine user preferences.

In another embodiment, one or more processors (such as those provided in certain computing environments, such as server-clients) are implemented, each processor comprising a set of features of the fashion product depicted in the fashion product content item, respectively. For each fashion product content item, the individual fashion product content item representing the catalog of fashion product is analyzed to confirm. Each fashion product represented by one of the fashion product content items is assigned to a predefined descriptive category for one or more corresponding descriptive categories. Allocation is performed based on quantitative analysis of the set of determined features. One or more fashion product content items that are considered to be of interest to the user are detected. The user's fashion preference is determined through a predefined descriptive category for each of the plurality of descriptive categories for one or more fashion product content items that are considered to be of interest to the user.

Embodiments described herein include: (i) learning a user's or consumer's preference for clothing style, fashion and genre, (ii) predicting a genre of different apparel products and fashion accessories, and / or (iii) (a) the user's known shopping parameters (e.g., user's size information, price preferences, preferences or preferences for certain styles, patterns and colors) and / or (b) the genre predicted for each distinct user And a system and method for suggesting to the consumer the best matched product and accessories, using the style.

 Fashion items include, for example, clothing, accessories, and apparel, and more specifically, blouses, shirts, dresses, shoes, socks, underwear, pants, belts, jewelry (e.g., watches, earrings, necklaces). ), Ties, hats, jackets and coats.

The fashion product content item corresponds to a document or file, which includes visual, textual and / or metadata information for a particular product. The fashion item content items may generally be used as part of an online catalog or e-commerce search engine. Typical aspects of such content items include (i) one or more images of the product, (ii) literal information about the product, including information about the available size and variety of the product, (iii) price information, and / or (iv) In order to purchase the depicted fashion product, the content item includes a link or data component having their viewer.

Some embodiments remain open problems on the computer complexity and response time for all online learning techniques and are limited in applications such as real time object tracking or online shopping scenarios described herein. We recognize that the problem may soon become important. In fact, many high dimensional feature vectors impose strict requirements on the number of operations allowed to meet stringent time requirements.

Some embodiments described herein describe computer implemented techniques for learning user preferences from user interaction with an online interface (eg, an interface provided on a shopping website). By predicting what the user likes, a better search ranking algorithm can be designed, and consequently the user perceives a better experience. In terms of feature selection, the present invention combines heterogeneous cues from visual and literal features, and provides a small but discriminative representation of the user's preferences that cannot be achieved, in particular, traditional features. do. Embodiments also include a learning step that can process relatively large feature vectors in less than a few msecs to avoid compromising the overall user experience.

According to the present invention, a fashion product may be recommended to the user based on the user's fashion preference. In addition, the user's fashion preference may be determined based on the user's interaction with the fashion product content item.

1 illustrates an apparatus that uses visual information to identify a preference of a user's genre and fashion style, in accordance with one or more embodiments.
2 illustrates a method for predicting a user's preference for a particular genre, in accordance with one or more embodiments.
3A depicts an example of a panel that, in an embodiment, may be created to display a set of visual aids to a user to induce a user to enter a response.
3B illustrates a panel that allows a user to select size information for various types of fashion items, such as issues, tops, bottoms and dresses.
3C illustrates a panel that allows a user to indicate or specify a user's preference for feature patterns, colors, and shapes.
4 illustrates, in an embodiment, a method of programmatically predicting a genre or style of a product.
5 illustrates a method of matching a product to consumer preferences, in accordance with one or more embodiments.
6 illustrates a results panel that conveys a user's fashion genre preferences, determined programmatically, according to an embodiment.
7 illustrates a method of determining, in one or more embodiments, descriptive classifications and categories of fashion products provided by fashion product content items.
8 illustrates a system for recommending a fashion product to a user using user activity information and product class / category determination, in accordance with an embodiment.

As used herein, the term “programmatically”, “programmatically” or variations thereof means through code execution, programming, or other logic. Programmatic operations may generally be performed in software, firmware, or hardware without user intervention. However, this action does not have to be automatic because it can be triggered manually.

One or more embodiments described herein may be implemented using programmatic elements, often referred to as modules or components, although other names may be used. Such programmatic components may include programs, subroutines, portions of programs, or software components or hardware components capable of performing one or more defined tasks or functions. As used herein, a module or component resides on a hardware component independently of other modules / components, or a module / component is a component shared by other modules / components, programs or devices. Or process. A module or component may reside on one device, such as a client or server, or the module / component may be distributed among a plurality of devices, such as a plurality of clients or servers. Any system described may be implemented in whole or in part on one server, or as part of a network service. Alternatively, a system as described herein may be implemented in whole or in part on a local computer or terminal. In any case, implementation of the system provided herein may require the use of memory, processors, and network resources (data ports, and signal lines (optical, electrical, etc.), unless otherwise specified). .

Embodiments described in the present invention generally require the use of computers, including processing and memory resources. For example, the systems described herein may be implemented on a server or network service. The user can access and use these servers through networks such as the Internet, or a combination of cellular networks and a network such as the Internet. Alternatively, one or more embodiments described herein may be implemented in whole or in part on computer devices such as desktops, cellular phones, Personal Digital Assistances (PDAs), or laptop computers. Thus, memory, processing, and network resources may all be used in connection with the establishment, use, or execution (including execution of any method or implementation of any system) of any of the embodiments described herein.

In addition, one or more embodiments described herein may be implemented through the use of instructions executable by one or more processors. These instructions may be stored on a computer readable medium. The devices shown in the following figures provide examples for processing resources and a computer readable medium, and instructions for implementing embodiments of the present invention may be stored or executed on the computer readable medium. In particular, many of the devices shown in conjunction with embodiments of the present invention include a processor (s) and various types of memory that retain data and instructions. By way of example, computer readable media includes permanent memory storage devices such as hard drives on personal computers or servers. As another example, computer storage media include portable storage units, such as CD or DVD units, flash memory (eg, embedded in many cell phones and PDAs), and magnetic memory. Computers, terminals, network capable devices (eg, mobile devices such as cell phones) are all examples of devices and devices that use processors, memory, and instructions stored on a computer readable medium.

Steps to learn genre and style preferences of shoppers

1 illustrates a system that uses visual information to identify a genre and fashion style preferences of a user, in accordance with one or more embodiments. As shown in FIG. 1, the system displays an image of a pre-selected fashion item to the individuals, who attempt to determine preferences, preferences, preferences and other user feedback to ascertain the user's style or genre preferences. . Contrary to what is described herein, conventional techniques for calculating the style or genre preferences of shoppers (e.g., users or consumers) are typically based on the genre / style that best represents individuals to their personal preferences for style and genre. This includes asking about. However, the conventional approach is problematic, one of the reasons for which words that specifically capture fashion preferences and states are insufficient. In addition, users do not always know what their preferences are.

Accordingly, embodiments described herein with reference to FIG. 1 include a system for learning user fashion styles and genre preferences using visual aids and pictures in accordance with the program. Accordingly, embodiments described herein in conjunction with FIG. 1 include a system for programmatically learning user fashion styles and genre preferences using visual aids and images. The system 100 can be provided in a variety of computing environments, eg, client-server architectures. For example, system 100 may be implemented in one or more servers (or other computing devices) to provide a service, such as one or more embodiments described below. In such an environment, system 100 may be implemented on a website such as an e-commerce site, search engine, or shopping portal.

System 100 may rely on genre definitions defined by experts or operators. For example, the fashion genre includes 'look', a genre such as chic, street, Boho, urban / hip-hop and conservative, There is no limit to the fashion genre. For example, an expert may select clothing representing a variety of categories and clothing that makes up an ensemble (the number of categories is set by design or selection). In some cases, representative clothing and apparel ensembles form ground truth data or comparison points to (i) determine the user's genre preferences, and (ii) the genre of another item of clothing or apparel. Can be used to predict. In an embodiment, the system 100 depicts an image of a garment that makes up the garment and the ensemble in a worn state. For example, images of people wearing different genres of clothes and accessories (including images of celebrities) may be shown to the user. The user may respond to an individual image to specify whether the depicted clothing is a style or type included in the user's preferences. Thus, the system can learn from the user's selection of images rather than character descriptions of preferences or the user's self-reporting.

More specifically, system 100 includes user interface 110, user database 120, genre score component 130, genre determiner 134, visual aid component 140, and product database. And 150. The user of the system 100 may be a shopper or a consumer of fashion goods. In some embodiments, system 100 is implemented in an online medium. For example, system 100 may be part of an e-commerce site, shopping portal or other web-based or networked environment where individuals are provided with the opportunity to view (potentially purchase) fashion products. Can be implemented. The interface 110 may correspond to, for example, a web page or an interactive feature provided in the web page.

The user of system 100 is associated with the profile of user database 120. For example, a user may receive an operator account for a service that provides system 100. Alternatively, the user may be informed by cookie / computer information, account / login or a single online session of the provider of the system 100 (eg, an e-commerce site). The user is independent of the system 100 that determines the user's style or genre preferences, and the user can interact with the user interface 110 and provide parameters 112 related to the fashion product that a fashion user can wear. can do. Parameters that the user can specify include, for example, user clothing size, preferred price ranges for fashion and clothing items, and preferred brand names. The user may also voluntarily provide information about the visual characteristics of the apparel or apparel that the user prefers or dislikes. For example, a user may specify a preferred color, fabric or material preference information for a certain type of clothing, shoes or apparel of a preferred style, a type of jewelery, aversion or preference for a particular type of pattern. .

The merchandise database 150 holds information from fashion merchandise content items. As described in some embodiments, product database 150 may store information about fashion items depicted in product content items. Such information may be determined programmatically from image, text and metadata analysis of fashion product content items, the fashion product content item being provided from a retailer, producer or other supplier of the fashion product. The information determined programmatically about the depicted fashion items is associated with corresponding product content items, such as electronic catalog pages and sections in the product database 150.

To programmatically determine the user's genre / style preferences, the visual aid component 140 is a device that displays to the user an image or visual aid derived for genre / style preference determination. The visual aid component 140 delivers visuals 152 of fashion products to the user via the user interface 110. In an embodiment, the visuals 152 depict fashion items or an ensemble of fashion items in a worn state (eg, worn on an image of a celebrity or model, a mannequin, a computer-generated person). 2 and 3A illustrate how visuals 152 can be structured for display to a user.

The user may provide input via the user interface 110, which may include (i) a user's preference or dislike for a particular fashion product or ensemble, (ii) the user's preference for one fashion product over another, and And / or (iii) direct evaluation or feedback indicating the level of user's preference or preference for the fashion product. The visual aid component 140 represents a set of visuals 152 that prompts the user to enter a response, which indicates the user's visual preference for the fashion genre depicted by that visual. In addition, as described in the embodiment of FIG. 2 or FIG. 3A, the visual may be displayed to the user in a quiz or game fashion. In the quiz or game fashion, the user may see a panel that individually depicts competing fashion products of different genres. The user can respond to each panel by indicating their preference or preference / unfavorability for one fashion product as compared to the other side of the panel.

Genre score component 130 determines and records genre scores from user input. To score individual classifications of the fashion genre, the genre score component 130 may record a response that is also displayed to the user in visuals 152. Optionally, the genre score component 130, in combination with the visual aid component 140, may score and record users' responses to subcategories of the fashion genre.

Many techniques can be employed to identify the user's fashion genre preferences. In an embodiment, the set of visuals 152 is predesigned to depict images of a plurality of fashion items for each identification genre. To indicate the user's similarity or preference to one fashion genre over another, when the user views images of fashion items, the user simply responds with a preference or preference / unfavorable input. The genre score component 130 maintains a genre score 133 indicating the user's genre preference for the genres represented by the set of visuals 152. The genre score 133 may be associated with a profile from a user and recorded in the user database 120.

Additionally or alternatively, genre determiner 134 may determine one or more preferred genres and / or subcategories of the user (eg, first, second, based, at least in part) on score 133. Third genre). The genre determiner 134 and / or score 133 may use intelligence as probabilistic assumptions that such users with particular genre preferences have particular preferences or preferences for other genres. In this regard, it may affect the visuals 152 output to the user by the visual assistance component 140. For example, a user with a business genre preference may be considered to not like street genre clothing.

Additionally, in one or more embodiments, the fashion items identified in the merchandise database 150 are tagged with genre descriptors 151. Descriptors include programmatically determined genre descriptors, which may be determined by a product genre predictor component (PGPC) 154. In particular, the PGPC 154 analyzes the merchandise content items to obtain information that can be used to determine the genre (s) of the fashion merchandise depicted in the content item. Thus, system 100 may be used to determine the user's genre preferences, and may also be used to predict genre classifications and categories of fashion products. The genre descriptors 151 determined from the PGPC may include genre categories that include sub-genres or second and third genre determinations. For example, many fashion items can share one or more genres. 4 illustrates a method of predicting genre (s) of a fashion product using fashion product content items, in accordance with some embodiments. As described, with the ground truth product set 155 provided by the operator of the system 100, its determination depends on the learning behavior.

In an embodiment, the system 100 also includes a product recommendation engine 170. In accordance with one or more embodiments, product recommendation engine 170 may prompt a user to: (i) user information identifying genre / style preferences and parameters for fashion items that the user can purchase and (ii) fashion product information. On the basis of, recommend fashion goods. User information 172 is provided by user database 120. In particular, user information 172 is provided by genre preferences output from genre score component 133 and / or genre preference information 137. The fashion product information 174 is retrieved from the product database 150. Fashion product information 174 includes programmatically predicted genre categories and / or subcategories associated with individual products. The fashion product information 174 may include information retrieved from a fashion product content item, and a tag (eg, metadata) provided by a fashion product content item or a supplier of the underlying fashion product. Using the user information and fashion product information, the product recommendation engine 170 may recommend individual fashion products, for example, from the products identified in the product database 150. The recommended products 176 may be delivered to the user through the user interface 110.

In addition, embodiments provide a system that can demonstrate its reliability in predicting user genres and styles. As described below, the system 100 includes an interface that allows the user to record known parameters, such as, for example, the user's clothing size, price preference and / or any style, pattern and color. There is their preference / disfavor for. This information is used to match products to user preferences. Since the overall system allows multiple hierarchies in genre prediction, the first or highest level genre predicting a wide genre or style matches, the second predicting multiple fine-grain genres or styles. Or second level genres, third or third level genres that predict multiple domain specific styles, and the like.

2 illustrates a method of predicting a user's preference for a particular genre, in accordance with one or more embodiments. More specifically, the method as described determines the preferences of the user's first, second, and third genres for a particular user. The described method can be implemented using a system as described in FIG. 1. Accordingly, reference may be made to components and numerals in FIG. 1 to describe suitable components and components for performing the steps or sub-steps described below.

The set of images is shown 210 to the user 210. In an embodiment, the visual aid generator 140 selects and displays a set of individual images to the user via the user interface 110. The set of images may be preselected from a range of various genres. In an embodiment, some or all genres are determined through manual definition and selection. Thus, the set of images can be sorted into different genres through manual input for classifying each image into a particular genre. Alternatively, some or all images of the set are programmatically determined to be associated with the genre. For example, programmatic methods may be used to identify similarities between items of clothing, and similarity comparisons may associate clothing with a particular genre.

As each image is displayed individually, the user is prompted to respond (via user interface 110) to provide input, which indicates whether the user likes or dislikes the image. The user's response is recorded (220). In an embodiment, the input is derived from the user as part of the game, and the user can participate in the game via input that indicates whether the user is considering a separate image of the set as a hot-or- sickle (hot-or Hot-or-not games.

Based on the user input, the genre preference of the user is determined 230. Using the reference to the embodiment of FIG. 1, the genre determiner 134 determines a user preference for the genre. In an embodiment, the genre determiner 134 uses an algorithm to determine the user's genre preferences (eg, first, second, and third). In an embodiment, the following algorithm may be used. Can be.

Express the entire set of genres as S = {Si, i = 1 ... n}, where n is the number of genres. Assume that each user has a predetermined set of preferred genres, and denote it as F = {Fj, j = 1 ... m}. It is also assumed that a set of images represented by Ti is provided for each genre Si. Now assume that the user is looking at a pair of images (I1, I2) from different image sets, Ti and Tj. If (i) one of them belongs to F (without loss of generality, this is assumed to be I1), the probability that the user will choose I1 is greater than 0.5. (b) If both images belong to F, or if both images do not belong to F, the user will randomly pick one of the two images with a probability of 0.5.

The algorithm maintains that after every round r, a vector of user's probabilities estimates represented by Q _r = [q _r1 , q _r2 , ... q _rn ] belong to each genre. After each style query in the round, the algorithm updates the user's genre probabilities, and the update can be performed as follows. In the two genres displayed to the user, the one selected by the user is updated via equation (1).

q _{r + 1} = q _r * p (1)

One not selected by the user is updated via equation (2).

q _{r + 1} = q _r * (1-p) (2)

All remaining genres are updated with equation (3).

q _{r + 1} = q _r * 0.5 (3)

After the update, Q _{r + 1} is normalized so that the sum of all probabilities is one.

Depending on the current genre probabilities, one of the following cases is executed.

1. If one of the genre probabilities is above a certain threshold, the algorithm ends the test and returns the best genre to the user.

2. If no genre probabilities of the genre probabilities are above a certain threshold, the algorithm selects two genre images to be shown to the user in the next round.

Different strategies can be used to select two images for the next round of test, for example (i) randomly selecting two genres, and / or (ii) having the highest probabilities. Selecting the top two genres (this strategy helps the probability converge quickly with the correct guess and minimizes the number of image pairs shown to the user).

The algorithm can be generalized to display k images (k> 2) to the user. In this case, the update equations for the formulas (2) and (3) may be the following formulas (4) and (5).

q _{r + 1} = q _r * (1-p) / (k-1) (4)

q _{r + 1} = q _r * 1 / k (5)

Also, the algorithm can be generalized to determine t genres (t> 1). In this case, the criteria for stopping may be modified to check the top t probabilities. The strategy for selecting the next set of images should select images from the top t genres and the remaining genres.

According to an embodiment, the user's response indicating preference or dislike is used to determine preferences for the first, second, and third genres for the user (240). The preferences of the first, second and third genres can be determined simultaneously. One way to implement this is to predict the first, second and third genres in order. However, in order to minimize the number of images shown to the user (and therefore to reduce the amount of users in the user's response), an approximation algorithm can be used. If all images used for the first genre prediction are tagged with the second and third genres, the images selected by the user in the first genre prediction are multiple histograms—one and one for the second genres. Can be used to generate multiples (one per domain) for three genres. Top genres of these histograms can be used to predict the second and third genres.

In order to provide a good user experience, some embodiments provide progress feedback, which indicates the amount of progress that the user proceeds to computer-learning of his genre preferences. In an embodiment, a progress bar may be shown to the user to indicate the progress of genre prediction. The distance between the threshold value, the current best genre probability, the q _ri maximum value, and the like may be used as a progress indicator.

3A depicts an example of a panel, which may be created to display a visual aid 152 (FIG. 1) to a user, in the following embodiment, to induce a user to provide a response. In FIG. 3A, panel 310 is displayed via user interface 110 (FIG. 1). Thus, for example, panel 310 may be formed as a webpage. The panel 310 consists of a pair of images 312 and 314, each depicting a particular genre of clothing (weared by a celebrity or model). The user can select one image over the other to indicate his or her preference for the particular genre depicted by the image (relative to the genre depicted in another image). Therefore, the user's selection of one image compared to other images is an input indicating a genre preferred by the user over another genre. When the user makes a selection, visual aid component 140 displays another panel, which depicts another pair of images (different genres of clothing) to solicit a similar selection from the user. ) To the user. According to an embodiment, the game comparing image pairs may continue for a plurality of rounds, where in each round a user selection providing information related to the user's preferences / preferred preferences for the various genres defined in the system 100. This is accompanied by

3B illustrates a panel 330 that allows a user to select size information for various types of fashion items, such as issues, tops, bottoms and dresses. Parameters such as size recommend fashion items, filter recommendations to users based on the lack of availability for a given size, or skew the user's genre preferences to fit a specific size or individual's body type. It can be used to

In addition, some embodiments provide a step of recording user feedback on genre selection (via competing apparel images), wherein the user may be able to enhance or augment the genre determined by input that specifies some preferences of the user. Provide steps. 3C illustrates a panel 350 that allows a user to specify or indicate a user's preference for feature patterns, colors, and shapes. Features that the user can specify preferences may be specified as a particular type of fashion product. For example, the shape preference that can be specified by the user may be indicated and displayed as a category of shoes or, more specifically, a fashion product for women's shoes.

Associated and ensemble recommendations for clothing, apparel and accessories

According to an embodiment, the online commerce environment (eg, when implemented by the system of FIG. 1) has a recommendation engine that recommends additional clothing, apparel or accessories. Such a recommendation may, for example, consist of providing a fashion ensemble or matching of the apparel / apparel set.

In order to facilitate the recommendation of a garment / apparel or accessory, in one or more embodiments, at least some valid products for the commerce medium are analyzed programmatically to predict the genre and style of the individual product. 4 illustrates a method of programmatically predicting a style or genre of a product under an embodiment.

Product genre prediction combines several different feature types, for example metadata features (based on a character description) and visual features (based on visual words generated from thousands of images).

For individual products in the catalog, programmatic feature extraction uses forms of other features (410). The feature extraction includes metadata extraction 414 and visual feature extraction 418. In metadata extraction, metadata features are identified and represented as a vector, with each word or word pair representing one of the metadata fields (eg, title, description, brand, price, etc.) one in the vector. Represents the dimension of. Visual features can be determined through image analysis and represented as vectors. Here, the vector may represent one global feature calculated over the entire image or a feature based on a visual vocabulary calculated over thousands of images. These visual features include color, shape, and / or texture. The final feature vector can be calculated by combining the metadata and the visual vector, and can, for example, link the metadata feature and the visual feature in order to form one large feature vector V.

Fashion professionals manually tag 420 the first, second, and third genre tags on a set of merchandise to form a ground fact set.

A device learning algorithm (Support Vector Machine or boosting or Bayesian learning) is used to learn to map the extracted feature vector of products to different genres (430). ). For each genre, a given feature vector V, binary classifier, can be learned to determine the probability that a product belongs to or does not belong to that genre.

Genre prediction may be performed first for individual products that are not included in the ground fact set (440). For each product, probabilities for all genres are estimated, and top genres are selected as genre predictions for the product.

In order to estimate all first (444), second (446) and third (448) genres for the commodity, multilevel level classification may be performed, where the second or third genre is the first genre. Subject to The first genre classifier is trained as previously described. Then, with the first genre g ₁ of the product given, a new set of second genre g ₂ and third genre g ₃ is trained for each first genre g1. During the test, the joint probability P (g ₁ g ₂ g ₃ | V) of the first genre and the second genre / third genre for a given feature vector may be calculated as follows.

P (g ₁ g ₂ g ₃ | V) = P (g ₁ | V) * P (g ₂ | g ₁ V) * P (g ₃ | g ₁ V) (6)

Product matching

According to an embodiment, the product recommendations may include (i) identifying the predicted product genres for the products (as shown in FIG. 4), (ii) identifying the user's genre or style preference given to the apparel and apparel. Matching the product to the user using the steps (as shown in FIG. 2) and (iii) (i) and (ii).

In some embodiments, products are pushed up for recommendation by raising products that match a user preference to higher ranks and by reducing weights of products that do not match a user preference to lower ranks. Can be.

As described herein, products that do not match the user preferences are affected as follows: (i) filtering out the unmatched products from the indication presented to the user, or (ii) towards the end of the results. The weight may be de-weighted through a down-weigh step. Matched products (or recommendations) can be shown by the user via periodic automatic emails (eg, sent daily, twice a week, once a week or once a month), or on a website It can also be seen by logging on. Also, depending on how often the product is shown to the user and how often the user clicks on it, the system keeps learning the user's comprehensive genre and domain-specific genre preferences.

More specifically, in accordance with one or more embodiments, FIG. 5 illustrates a method of matching a product to consumer preferences. To describe suitable components for performing the steps or sub-steps described below, reference is made to the components in FIG.

The combination of the first and second / third genres with the highest joint probability may be selected as the genres of the commodity.

For a given user, first, second and third genres of the user are identified 510. For example, the results of the progress as shown in FIG. 2 may be analyzed or retrieved to determine the user's preferred genres. The visual aid component 140 can display the visuals 152 to derive a response from the user. Successive derivations may be requested from the user to specify the relative preferences for the various genres. The score result (determined according to the user's response) is used to determine the user's fashion genre preferences.

Once the user's first, second and third genre preferences are identified, a pool of products that match the user's preferences is identified 520 from the product database 150.

In one embodiment, to identify a subset of fashion products for recommendation to the user, the matched products undergo a process of selection, filtering and weighting (530). For example, selection and filtering may be performed to exclude fashion items that are not available and not the user's size, or the color, pattern, or shape that the user has indicated as unfavorable. In another embodiment, the matched product may be filtered to delete items with colors, brands, or keywords that the user does not like. In addition, the matched product may be weighted, for example, with preference / disfavor fashion products that satisfy the user preferences specified for color, pattern, shape or brand. The matched product may be displayed to the user, for example, as a search or browse list (540). In an embodiment, the products left behind are sorted according to the matching score to determine the order in which they should be sent to the user.

According to an embodiment, the matching score may be calculated as a linear combination of different individual matching scores.

s = w ₁ * a ₁ + w ₂ * a ₂ + ... where (w _i > 0)

The individual matching score includes first, second, or third genre probabilities, age matching score, price preference, and other color, style, or pattern preferences of the product.

Show results

The results of the various processes, algorithms, and system outputs can be provided to the user in various forms, and some embodiments include an interactive tool that a user can use to determine the user's fashion genre preferences. 6 illustrates a results panel for conveying a user's fashion genre preferences determined programmatically. The result panel 610 may be output, for example, in response to participating in a quiz or challenge generated through the visual aid component 140. Through processes, as described by various embodiments, the results panel 610 identifies the user's first genre (sporty) and one or more second (conservative) or third (modern, protective) genres. can do. The result panel 610 may also display fashion items corresponding to user genre / style preferences. Images of fashion products may be preselected based on images considered to represent a particular genre or genre combination. Alternatively, some or all of the fashion items depicted may be selected by the user. For example, parameters such as user specified color preferences are used to indicate some items of clothing or apparel. Equally, if the user prefers a certain style of shoes (e.g., the boots specified by the user via the interface, as shown in Figure 3C), the footwear results panel 610 may be attached to the boots. Can be depicted.

Enhanced feature display

Embodiments described in the present invention may incorporate improved feature representation of descriptive classifications for fashion goods. In particular, descriptive classifications may be defined by human operators, eg, experts, to include multiple categories (or subclasses). According to an embodiment, fashion merchandise content items (eg, a catalog or web image of a garment) are analyzed to extract features from images, text and metadata. The extracted features are then analyzed to associate the fashion product with one or more descriptive classifications (of fashion items), each of which corresponds to an associated descriptive classification.

FIG. 7 illustrates a method for determining descriptive classifications and categories of fashion products provided by fashion product content items, under one or more embodiments.

Descriptive classifications and categories (or subclasses) for fashion products are defined by the human operator (710). In an embodiment, the descriptive classifications include (but are not limited to) shapes or silhouettes, patterns, and colors. For example, the following classifications may be employed.

For individual fashion product content items, a set of primitive visual and character features are extracted from the content item (720). Such features include, for example, color histograms, shape descriptors, texture features, and text description features. To determine the features as described above, image recognition and text analysis (including textual metadata analysis) may be performed on the individual content items.

An analysis of the primitive feature is performed 730 to determine the classification and categorization (or sub classifications) of the products depicted in the content items. The assay can be quantitative. More specifically, in one embodiment, the analysis can be statistical. In addition, a number of methods can be implemented to associate fashion products to the classification. For color classification, a set of cluster centers that are manually labeled with ground facts are created. Each product (or its image) is assigned to the nearest cluster according to its distance in the histogram.

f is a primitive feature vector that encompasses visual and literal information.

c _i ( i = 1,… N ^CFT ) are cluster centroids for color family / classification,

는 컬러 패밀리 고차원(hyper dimension)

의 컴포넌트이다.

Is a color family hyper dimension

Is a component of.

g is the mapping of distances to likelihoods.

A support vector machine (SVM) classifier may be used to associate or assign products to classifications. For each class, human operators (e.g., fashion professionals) have a set of positive examples with properties corresponding to a tag and a set of negative examples without such properties. Choose. As a new (unknown) item occurs, a trained SVM is used to generate decision values from the visual and literal features of the item. The determined value represents the distance between the separating hyperplane and the item. Only values on the positive side of the hyperplane are retained.

는 각 고차원(hyperdimension) T의 각 태그 i 에 대응하여 학습된 SVM 파라미터들이고, 고차원 T 는, T

{GT, ST, PT}이다. 상기한 바와 같이, f 는 아이템의 프리미티브 특징 벡터이고, 트레이닝 세트에서 모든 다른 아이템들은 g_j이다.

는 핵함수(kernel function)이고, τ(x) = xH(x)에서 H(x)는 헤비사이드 함수(Heaviside function)이다.here,

Are the SVM parameters learned for each tag i of each hyperdimension T, and the higher dimension T is T

{GT, ST, PT}. As noted above, f is the primitive feature vector of the item and all other items in the training set are g _j .

Is a kernel function, and τ (x) = xH (x) where H (x) is a heavyside function.

8 illustrates a system for recommending a fashion product to a user using product class / category decisions and user activity information, in accordance with an embodiment. A system as described in the embodiment of FIG. 8 may display modifications or variations to the embodiment described in FIG. 1 and other embodiments of this application. However, the functions and components of FIG. 8 may optionally appear in the form of supplementing or augmenting the system, as described in FIG. 1.

System 800 may be comprised of a user database 120 and a product database 150. As described in another embodiment, the user database 120 may include specific information related to individual users, such as genre preferences (which may be programmatically determined) of users' fashion, and parameters specified by the user ( As shown in Figures 3B and 3C). In addition, the user database 120 may be coupled with a monitor component 810 that monitors or detects user behaviors and related activities related to fashion product content items. The monitor component 810 may detect one or more of the following activities, which may include: i) user interaction with search results, including user selection or otherwise indicating interest in a particular item of search results. ), (ii) user interaction with the online browsing and shopping environment. Information 812 identifying items (eg, merchandise) may be stored in the database 120. In an embodiment of the invention, this information 812 is displayed to the user, and includes items that the user clicked on and items that are displayed to the user but not clicked by the user.

The user monitor 810 can detect the session specific activity or historical activity 814 from the user's past sessions. The historical activity can be extended to search conditions entered by the user, such as a search engine or an e-commerce site. User interaction may be detected via interface 810 or browser or browser data (eg, browser history and cookie information). In some embodiments, historical activity 814 may include queries entered directly by the user, impressions (ie, items retrieved by the search engine and displayed to the user) and purchase clicks. clicks) (ie, items clicked by the user). The set of queries is projected into the fashion-aware feature space described above, and a plurality of positive training samples are obtained.

In the embodiment of the present invention, the product database 150 is connected to the product category / class determination unit 820. The category / class determiner 820 may analyze fashion product content items to determine one or more categories / categories 822 for each product. In an embodiment of the present invention, the category / class determination unit 820 implements the process as described in FIG. In an embodiment, the descriptive classification / categorization results are stored in the commodity database 150.

According to one or more embodiments, user preference profiler 830 generates user profile 832 in accordance with activity information 812 and / or history information 814. Profiler 830 updates user profile 832 for individual users. In the process of creating and updating the user profile 832, the profiler 830 may be configured to (i) user activity information 812 (eg, items selected by the user for viewing during browsing or searching, for example, Identifying the fashion items), (ii) using the merchandise database 150 to determine the classification and categorization of those products (as determined in FIG. 7), and (iii) the user profile 832 In order to evolve, use descriptive classifications and categorization for the products identified from activity information 812. User profile 832 augments, supplements, or otherwise identifies the genre preferences of the user's fashion. Thus, user profile 832 can be combined with, or alternatively used in, the programmatic fashion genre determination described in other embodiments. For example, user profile 832 may be session specific to determine if the user is looking for an event-specific outfit (eg, an evening gown) that may not be included in the user's preferred genre. And lowbust. The user profiler 830 can use the history information 814 to develop the profile 832.

In the present invention, the recommendation engine 180 is configured for a user based at least in part on the identified genre preferences in the user profile 832 and / or genre preferences identified via an aid / score component. The device recommends the selected products 176 data. The recommendation device 170 may include the historical data 814 as a component for determining its recommendation product 176. In response to a user query / search or request for goods from a particular type of fashion items, the recommendation engine 170 may be used to recommend and / or search and / or rerank.

Short term use

Embodiments recognize that in an online scenario, the user's short term preferences may become important. Embodiments also recognize the need for an online algorithm that quickly learns the user's behaviors and immediately improves the user's shopping and searching experience. For example, if a user is shopping for an official holiday party and resort vacation, his long term preferences for color, pattern brand, etc. will rarely be used to enhance the overall shopping experience. Thus, a system that learns the user in real time as the user interacting with the site may send more relevant results.

In an embodiment, in an online system, as a user performs and clicks queries, they are included in a daily user profile. A summary of the preferences is generated through nuclear density estimation and maintained for use in the ranking. As the user enters queries and clicks on item i, feature vectors representing the properties of item i are called (from a pre-calculated table) and the daily user profile is generated according to an online update of kernel density estimation. Efficiently synthesized in a generic model.

The number of clicks of the user's session is n and h is the kernel bandwidth. The function p can be used to score the relevance of the item feature vector x ^{T over} the current session. A quadratic kernel can be used. After the user enters a query, all items associated with the query (visual and text based on relevance) are called from the item database according to corresponding absolute rankings.

conclusion

Although many embodiments for fashion goods have been described, alternatively the embodiments may be extended to other types of goods. In particular, the invention can be extended to products generally classified by the taste and appearance of the individual, for example furniture, carpets (and curtains) and design exteriors.

conclusion

Although illustrative embodiments have been described in detail herein with reference to the accompanying drawings, it should be understood that the specific embodiments described are not limited to the specific examples mentioned. As such, many variations and modifications, including matching features described with one embodiment, enable other embodiments that do not refer to such functionality. In addition, although other features and embodiments do not refer to particular features, certain features described as individual embodiments or as part of an embodiment may be combined separately with other described features or with portions of other embodiments. Can be.

Claims (18)

A computer implemented method of determining user preferences for fashion goods, the method comprising:
Using one or more processors to execute the steps, wherein the steps are:
Programmatically determining a user's fashion preferences based on the user's interaction with a plurality of fashion product content items that individually describe the corresponding fashion product; And
Recommending a fashion product to a user based at least in part on the user's fashion preferences. The method of claim 1,
And individually displaying the plurality of fashion item content items to a user and prompting the user for a response indicating preference or disfavor for the plurality of fashion item content items. Way. The method of claim 1,
Programmatically determining the fashion preferences
Identifying a set of images that individually depict one or more fashion items;
Displaying a sequence to the user, the sequence comprising a plurality of panels, each panel comprising at least two images of the set; And
For each panel, recording the user's response indicating which of the at least two images of the panel are the most preferred or the least preferred of the user. The method of claim 3,
Displaying the sequence
Generating each panel such that each fashion product content item of the individual panels displays a corresponding fashion product, wherein the corresponding fashion fashion product is a corresponding genre different from the fashion product of another fashion product content item of the panel. How to implement a computer. The method of claim 4, wherein
And wherein the fashion product content items of each panel are determined to belong to the corresponding genre by manual input. The method of claim 1,
Directing the user to provide an input specifying one or more known parameters related to the user's fashion preferences. The method of claim 1,
Wherein said one or more known parameters comprise a size preference or a price preference of said user. The method of claim 1,
Recommend the fashion product to the user,
Recommending one or more fashion items based on the user's known parameters and the determined fashion preferences. In a computer implemented method using programmatic descriptors for fashion goods, the method comprises:
Using one or more processors to execute the steps, wherein the steps are:
Analyzing the fashion product content item to determine a set of features of a fashion product depicted in a fashion product content item,
Programmatically associating the fashion product with a descriptive category predefined for each of a plurality of descriptive classifications, based on a quantitative analysis of the determined set of features, and
Using a predefined descriptive category for each of the merchandise content item and its plurality of descriptive categories to predict or determine user preferences. 10. The method of claim 9,
And wherein the plurality of descriptive classifications comprises one or more genre classes, pattern classes, shape classes, or color family classes. 10. The method of claim 9,
Analyzing the fashion product content item comprises performing image analysis on an image portion of the fashion product content item. 10. The method of claim 9,
Programmatically associating the fashion product to the predefined descriptive category comprises determining a probability that the fashion product has a visual feature of each predefined category of one or more descriptive categories. Computer-implemented method. 10. The method of claim 9,
Using a predefined descriptive category for each of the merchandise content item and its plurality of descriptive classifications may include detecting a user's selection or interaction with the fashion merchandise content item, and determining the user preferences. Using a predefined descriptive category for each of the descriptive classifications. The method of claim 13, wherein
Detecting a user's selection or interaction with the fashion merchandise content item may include monitoring which fashion merchandise content items the user has selected to view, so as to define a predefined set of individual descriptive classifications for each product viewed by the user. Based on a descriptive category, identifying a profile for the user. 10. The method of claim 9,
Using a predefined descriptive category for each of the merchandise content item and its plurality of descriptive categories may include identifying a fashion genre or style preference of the user, and a predefined descriptive associated with the fashion product content item. Recommending or not recommending the fashion product based on categories and the user's fashion genre or style preferences. 10. The method of claim 9,
Recording historical information related to a user's online activity on fashion products; And
And identifying, based in part on the history information, the user's genre preferences for fashion products. A computer implemented method of determining user preferences for fashion goods, the method comprising:
Using one or more processors to execute the steps, wherein the steps are:
Analyzing individual fashion product content items representing a catalog of fashion items to determine, for each fashion product content item, a set of features for the fashion product depicted in the fashion product content item;
Based on the quantitative analysis of the determined set of features, each fashion product represented by one of the fashion product content items is predefined for each of a plurality of descriptive classifications in a descriptive category. Programmatically associating with;
Detecting one or more fashion merchandise content items considered to be of interest to the user; And
Determining a fashion preference of the user using a predefined descriptive category for each of a plurality of descriptive categories for one or more fashion product content items deemed of interest of the user. The method of claim 7, wherein
Determining the user's preferences comprises using historical information that includes a search term previously used by the user.

Images