JP2013522719A - Product category classification - Google Patents

Abstract

Product category classification extracts a plurality of product titles from acquired data, divides the title into phrases, determines each score for the phrase, and determines each score determined for the phrase. At least one of the at least partly based phrases is used to construct a first word sequence for a first product of the plurality of products, and the first word sequence is a second of the plurality of products. A first word sequence of the plurality of products to be combined into one product category based on at least in part the comparison with a second word sequence for Including.
[Selection] Figure 1

Description

[Cross-reference of related applications]
This application is named “METHOD AND DEVICE FOR CATEGORIZING DATA”, a method and method for categorizing products, filed on March 9, 2010, which is incorporated herein by reference for all purposes. The priority based on Chinese Patent Application No. 201010122141.2. The application is

  The present invention relates to data processing techniques, and more particularly to a method and system for categorizing product data.

  In an electronic commerce website, generally, various data describing products on the website are stored in the form of text or data tables. Because the amount of data taken up on electronic websites is large, descriptive data about all products forms massive information content. Therefore, there is a question as to how effectively data should be managed, especially for similar products.

  In various electronic commerce websites, it is common to categorize various data of products using clustering technology. A typical clustering technique classifies data about products into categories based on a predetermined set of rules and conditions (eg, similar products are classified into the same category).

  One example of a commonly used clustering method is hierarchical clustering. This clustering hierarchical clustering method refers to a bottom-up policy. In a typical bottom-up policy, each object to be categorized is initially considered a separate atom cluster, and these atom clusters are then either collected until all objects that belong to the same category are grouped together or Until the termination condition is met, it is integrated to form a new cluster at a higher level.

  However, extensive data processing is required to categorize e-commerce website data using the clustering method described above, which may lead to inefficiencies in system resources.

  Various embodiments of the invention are disclosed in the following detailed description and the accompanying drawings.

FIG. 1 illustrates an embodiment of a system for categorizing products.

6 is a flowchart illustrating one embodiment of a process for categorizing products.

5 is a flowchart illustrating another embodiment of a process for categorizing products.

1 illustrates one embodiment of a system for categorizing and using product data. FIG.

  The present invention is provided by a process, apparatus, system, composite, computer program product implemented on a computer readable storage medium, and / or instructions stored in a combined memory and / or combined memory. Can be implemented in a number of forms, including a processor such as a processor configured to execute instructions. In this specification, these implementations, or any other form that the invention may take, may be referred to as techniques. In general, the order of the steps of the disclosed processes is variable within the scope of the present invention. Unless stated otherwise, a component such as a processor or memory that is described as being configured to perform a task is a generic component that is temporarily configured to perform a task at a given time, or to perform a task. Can be implemented as a special component manufactured. The term “processor” as used herein refers to one or more devices, circuits, and / or processing cores configured to process data, such as computer program instructions.

  A detailed description of one or more embodiments of the invention is provided below along with accompanying figures that illustrate the principles of the invention. Although the invention has been described in the context of such embodiments, it is not limited to any embodiment. The scope of the invention is limited only by the claims and the invention encompasses numerous alternatives, modifications and equivalents. In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention. These details are provided for the purpose of example and the invention may be practiced according to the claims without some or all of these details. For the purpose of clarity, technical material that is known in the technical fields related to the invention has not been described in detail so that the invention is not unnecessarily obscured.

  Product categorization is disclosed. In some embodiments, product data is obtained and the titles of the products mentioned in the product data are extracted. In some embodiments, product attribute information is also extracted from the product data. The extracted information is divided into phrases. A score for each phrase is determined based at least in part on the past occurrence frequency of the phrase. For a product, one or more series of phrases are selected and organized into a word sequence. The word sequence configured for each product is compared with the word sequences of the other products. Products with similar word sequences are merged into a set of products under one category.

  In some embodiments, consolidating products with similar word sequences into a set of products under a category also includes associated data for the products in that category (eg, an accompanying description describing the category of the product). Including product data).

  FIG. 1 is a diagram illustrating one embodiment of a system for categorizing products. In the example shown in the figure, the system 100 includes an extraction unit 10, a division unit 11, a selection unit 12, an integration unit 13, and a processing unit 14.

  The system 100 is a hand-held or portable device such as a personal computer, a server computer, or a smartphone, a flat panel device, a multiprocessor system, a microprocessor-based system, a set-top box, a programmable home electronic device, a network PC, or a minicomputer. A large computer, a special purpose device, a distributed computing environment including any of the above systems or devices, or one or more processors and coupled to the processor configured to provide instructions to the processor Can be implemented using any suitable computing device, such as other hardware / software / firmware combinations including

  A unit may be a software component that runs on one or more general purpose processors, as hardware such as a programmable logic device and / or special purpose integrated circuit designed to perform a predetermined function, or It can be implemented as a combination of them. In some embodiments, the unit includes a number of instructions for causing a computing device (such as a personal computer, server, network equipment, etc.) to perform the method described in the embodiments of the present invention and a non-volatile storage medium ( And can be embodied in the form of a software product that can be stored on an optical disk, flash storage device, mobile hard disk, etc. Units can be implemented in one device or distributed across multiple devices. Unit functions can be combined with each other or further divided into multiple subunits.

  The extraction unit 10 is configured to obtain data relating to the product to be categorized. The extraction unit 10 is also configured to extract product titles from the acquired data. In some embodiments, the extraction unit 10 is configured to also extract product attribute information from the acquired data.

  The split unit 11 is configured to split each of the product titles into one or more phrases, each phrase including one or more words. The split unit is further configured to determine a score for each phrase representing the past occurrence frequency of the phrase.

  The selection unit 12 is configured to select phrases having a score that satisfies a predefined condition for each product and consolidate those phrases into a word sequence for that product.

  The integration unit 13 is configured to compare the word sequences configured for that product with each other. In some embodiments, the integration unit 13 is configured to determine which products have similar word sequences and integrate products with similar corresponding word sequences into one product category. . In some embodiments, for products with similar word sequences, the integration unit 13 also integrates related data (eg, attribute information and other descriptive data) for products of the same category (and, for example, that product category). ).

  The processing unit 14 is configured to set and store an identifier corresponding to each of the product categories determined by the integration unit 13.

  FIG. 2 is a flowchart illustrating one embodiment of a process for categorizing products. In some embodiments, process 200 is performed in a guided search system, such as 100 in FIG.

  In step 202, data related to the products to be categorized is obtained and the titles and other attribute information of those products are extracted.

  In some embodiments, at an e-commerce website, data related to the product is manually entered into the website (eg, by the website operator or registered user). For example, a user can access a web page of a website that characterizes a field where the user can enter data related to the product. The web page content can then be transmitted to the user. The server then extracts the title and other attribute information from the content. The server also divides the extracted title into phrases.

  In some embodiments, the product data is periodically and / or automatically to perform product categorization (eg, to update a categorization stored for an e-commerce website). To be acquired. In some embodiments, product data is obtained by a server associated with an e-commerce website (eg, a server that supports the website platform and stores at least a portion of content for the website). Is done. For example, the server can obtain product data after such data is uploaded to a website.

  In various embodiments, since the product title includes keywords that accurately describe the product, it is desirable to extract the product title. Examples of data related to a product include title, price, and other information related to model, year, manufacturer, and the like. For example, the title of the hair dryer product may be “Model D3506 hair dryer by the HairSine brand”.

  In various embodiments, the product attribute information includes a detailed description of the product. For example, the attribute information of a hair dryer may include the time when the product was put on the market, the model and color of the hair dryer, and the evaluation score (evaluation score). In some embodiments, the attribute and the corresponding attribute value are indicated by an identifier that represents the attribute and the corresponding attribute value. In some embodiments, attributes and corresponding attribute values are represented in the form of attribute identifier: attribute value identifier symbol pairs. For example, if the product color attribute is green, it can be written as attribute A: 2000. Here, A is an identifier of a color attribute, and 2000 is an identifier of an attribute value green. In some embodiments, when combining products into one or more groups (eg, each associated with a category), the similarity of both the title and attribute information of different products is Be considered. Thus, in some embodiments, in step 200 both product title and attribute information are extracted.

  In step 204, the product title is divided into phrases.

  In some embodiments, the extracted product title and / or attribute information is divided into one or more phrases, each phrase including at least one word. In some embodiments, the title is divided into one or more phrases based at least on the identifiable meaning of the one or more phrases. In some embodiments, title splitting is performed based on a predefined set of rules that determine which individual words can be considered phrases and which word groups can be considered phrases. For example, the product title “Model D3506 hair dryer by HairSine brand” is divided into “HairSine brand”, “Model D3506”, and “Hair dryer”.

  In some embodiments, dividing the title and / or attribute information into phrases also includes discarding certain phrases. For example, at the end of the splitting process, phrases indicating brand and product type (eg, “HairShine brand” and “Model D3506”) are maintained. Conversely, phrases that tend not to be closely tied to product categorization (eg, “certified products”, “sale”, and “special price”) are eliminated at the end of the segmentation process. In some embodiments, which phrases are discarded are determined using past reference information stored in a database.

  In some embodiments, product titles and product information are divided into phrases using tools implemented in a platform such as a Hadoop distributed computing system. In some embodiments, the Hadoop program is executed in a Hadoop distributed architecture (eg, in a computing cluster comprised of 50 to 300 machines).

  In step 206, the respective score for the phrase is determined. In some embodiments, a score is determined for each phrase that was generated by splitting and was not discarded. In some embodiments, the phrase score represents the past occurrence frequency of the phrase. The past occurrence frequency of the phrase is one of the number of times that the user of the associated electronic commerce website searches for the phrase, the number of times that the phrase is included in the title information input by the user, and the distribution probability. Or two or more.

  In step 208, a word sequence is determined for the product. In some embodiments, the word sequence is formed by phrases that are split for the product. In some embodiments, phrases to be included in the word sequence are selected according to predefined conditions based on their determined score. For example, the default condition is to attribute two phrases with the highest score (one or more) from the product title and five words with the highest score (one or more). You may be required to choose from information.

  In step 210, word sequences corresponding to products are compared. The word sequences constructed for the products in step 206 are compared with each other. In some embodiments, the product word sequence is compared to all other product word sequences in the acquired product data. In some embodiments, each comparison determines a match rate. The match rate determines how similar two word sequences (and their respective products) are. In some embodiments, the two products are considered similar if the comparison match rate is above a certain threshold.

  For example, if two word sequences are identical (eg, each word sequence has exactly the same phrase), the match rate will be 100%. Assuming that the match rate threshold is 95%, the word sequence and each of its two products are considered similar.

  At step 212, based at least in part on the comparison, at least two products are combined into a product category. Based on the comparison of step 210, similar products are combined and classified into the same category. In some embodiments, a product category is a collection of products having word sequences that are similar to each other. These products are considered to be similar to each other because the word sequences of the products are similar to each other. In other words, the word sequence is considered to accurately represent the corresponding product. In some embodiments, product sets that are merged into one category are stored together in one database.

  For example, based on the comparison of step 210, the word sequence of 15 products is considered similar (eg, the word sequence of each product is considered similar to the word sequence of all other products). ). In this example, these 15 products are classified into one category.

  In some embodiments, for products that are merged and classified into the same category, their respective product data is also consolidated (eg, as integral descriptive data) and stored for that product category. For example, integrated product data for products in the same category can be used to describe all products in that category. Products integrated into the same category and their integrated product data can be stored, for example, in the same text file or data table.

  In some embodiments, in managing a product category, the integrated product data for that category is used to characterize that product category. For example, the integrated product data can be used to visually present an associated category of products. Alternatively, the integrated product data can be modified to change the description of the product in the associated category. Also, the integrated product data can be returned in response to a search for products in the associated product category.

  In some embodiments, a unique category identifier is set for each identified product category. Product categories are stored with their identifiers so that they can be searched by their respective unique category identifiers. For example, each unique category identifier can be stored along with a corresponding set of products (eg, using product titles or other product identification information) and their integrated product data.

  FIG. 3 is a flowchart illustrating another embodiment of a process for categorizing products. In some embodiments, steps 302-306 occur following the repetition of process 200 of FIG.

  Process 300 can be implemented to improve the accuracy of the categorization results by process 200. Process 300 merges categories of products that are similar products but classified into different categories in process 200 because the dependent data included different titles for the same product (eg, as entered by the user). Can help. Process 300 can be performed any number of times to improve the overall accuracy of the categorization process.

  The following embodiments of steps 302-306 assume that at least two product categories have been created after the process 200 has been repeated.

  In step 302, word combinations for product categories are determined.

  A word combination for a product category refers to a series of phrases that represent the product category and a respective score determined for the series of phrases. Word combinations can be selected in various ways for product categories. In one example, if all products in a category correspond to the same word sequence, that word sequence is used as the word combination for that category. For example, products that all correspond to word sequences that contain the phrases “HairShine brand”, “Red”, and “DF0753” are categorized in the same category, and thus “HairShine brand, red, and DF0753” It can be understood as a word combination for the product category.

  In another example, not all products in a category correspond to the same word sequence, but all correspond to a word sequence that includes several identical phrases. In such a situation, a series of phrases common to all products in a category can be considered as a word combination for that product category.

  In step 304, the similarity between the two product categories is determined.

In some embodiments, the similarity between two categories is determined using a word combination of the two product categories. For example, the similarity can be determined by the following equation.

In the above equation, TD1 and TD2 represent word combinations of the two product categories, respectively. For example,
TD1 = (Phrase 11, Score 11), (Phrase 12, Score 12), (Phrase 13, Score 13)
TD2 = (Phrase 21, Score 21), (Phrase 22, Score 22), (Phrase 23, Score 23)
Here, “phrase XX” represents a phrase, and “score YY” represents a corresponding score.

  Furthermore, characteristic 2 and characteristic 2 represent values of main attributes corresponding to two product categories. As used herein, primary attributes refer to important attributes of a particular product. For example, the primary attributes of a mobile phone include its brand and model, while its color and weight are general (eg, non-primary) attributes. In some embodiments, key attributes for a particular product are stored and accessed in process 300 to determine which values to use as characteristic 1 and characteristic 2. In some embodiments, the similarity is calculated based on the law of cosine calculation. The greater the calculated similarity, the more similar the two products.

  Furthermore, λ1 and λ2 are coefficients selected to assign weights to titles and attributes. (For example, TD1 and TD2 are formed using phrases divided from the title information, and property 1 and property 2 are attribute values). It represents two coefficients that each indicate which of the title or attribute is more important. For example, when λ1 = 2 and λ2 = 1, this indicates that the importance of the title is twice the importance of the attribute.

から計算される類似度の値をより小さくして調整を行うことができる。これは、２つの製品カテゴリが統合される確率を低くする。 Furthermore, a and b represent predetermined parameters, and n1 and n2 represent the number of products included in the two product categories being compared, respectively. The parameters a and b control the similarity value and thus influence whether the two product categories are integrated. For example, if two product categories both contain a large number of products, the similarity value changes the values of a and b,

The adjustment can be performed by reducing the similarity value calculated from This reduces the probability that the two product categories are integrated.

である。 For example, if a = 50, b = 20, n1 = 100, and n2 = 10,

It is.

  In step 306, it is determined whether the two product categories should be merged by comparing the similarity between the determined two product categories to a predetermined threshold. If the determined similarity exceeds a predetermined threshold, at step 308, the two product categories are merged into one category. If the determined similarity does not exceed a predetermined threshold, the two product categories are not merged.

  In some embodiments, the predetermined threshold is used to determine whether two categories are sufficiently similar to be merged into one category. The predetermined threshold can be stored and accessed for determination in step 304.

  Returning to the above example, assume that the degree of similarity between the two determined product categories is approximately 7%. In this example, if the default threshold for merging the two categories is 97%, the two categories are not merged because the determined similarity is well below the threshold.

  In some embodiments, combining two categories creates a new category identifier, which is used to identify all products in both categories (eg, identifying information about those products) and related product data in both categories. Including remembering with. In some embodiments, combining two categories includes storing all products in both categories and associated product data in both categories along with one of the category identifiers in the two categories.

  FIG. 4 is a diagram illustrating one embodiment of a system for categorizing and using product data. System 400 includes user 402, network 404, and server 406. The network 404 includes various high-speed data networks and / or telecommunications networks. Server 406 is configured to communicate with user 402 over network 404.

  In some embodiments, process 200 is performed using system 400. In some embodiments, process 300 is also performed using system 400. In some embodiments, the units of system 100 (extraction unit 10, split unit 11, selection unit 12, integration unit 13, and processing unit 14) are components of server 406.

  In some embodiments, the server 406 is configured to support a platform for an e-commerce website. For example, the server 406 stores information for the website and provides web pages for the website. In some embodiments, the server 406 is configured to obtain data from a user (eg, user 402) who uploads information (eg, product data) to a website.

  Server 406 is configured to extract the title of the product for which product data was obtained. In some embodiments, the server 406 is also configured to extract product attribute information from the acquired data. For example, the server 406 can extract title and / or attribute information from each of the title and attribute fields of the data uploaded to the website. Server 406 is configured to divide the extracted information (eg, title and / or attribute information) into phrases. For example, product titles can be split based on a set of rules that split a series of alphanumeric words into one or more phrases. Server 406 is configured to determine a score for the phrase. In some embodiments, the score for a phrase is based on the past frequency of occurrence of the phrase (eg, in product data stored on a website). Server 406 is configured to construct a word sequence for the product of acquired data. For example, a word sequence is configured for each product. In some embodiments, the word sequence for a product is determined based on a selected phrase of one of the product's phrases. Phrases can be selected based on predefined conditions (eg, the three phrases with the highest scores are selected). Server 406 is configured to compare a word sequence for a product with a word sequence for other products. In some embodiments, the word sequence for the product is compared to the word sequence for all other products in the acquired data. In some embodiments, the comparison of two word sequences results in whether the word sequences (and their corresponding products) are similar. Server 406 is configured to integrate at least two products into the same category based at least in part on the results of the comparison. In some embodiments, products with word sequences that are considered similar in the comparison are merged into the same category. For example, products that are integrated into the same category are stored under the same category identifier. In some embodiments, product data (eg, title and attribute information) for products in the same category is also stored under the same category identifier.

  In some embodiments, the server 406 is configured to merge product categories. In some embodiments, the server 406 is configured to determine word combinations for product categories. For example, word combinations are determined for each existing product category. The word combination can be selected from a word sequence associated with that category of product. Server 406 is configured to determine the similarity between the two product categories. In some embodiments, the similarity is determined using two categories of word combinations. Server 406 is configured to compare the similarity between the two determined categories to a predetermined threshold to determine whether to merge the categories. If the determined similarity is above a predetermined threshold, the two categories are merged into one category (and, for example, products from both categories are stored with the same category identifier). Otherwise, if the determined similarity is below a predetermined threshold, the two product categories are not merged.

  Server 406 is configured to store and maintain product category information. Such information can be used to represent each category of similar products on an electronic commerce website. For example, a visual representation, i.e., a table that includes product category title and attribute information, can be displayed in response to a user searching for a product in that category. For example, assume that a user puts “mobile phone” in a search box on an electronic commerce website. A server that supports a website will return a set of search results including those related to “mobile phone” among the products sold on that website. The returned search result may display information stored about the product category associated with “mobile phone” in the search result (eg, in the form of product price, model, cost, manufacturer, etc.).

  User 402 is a device through which a user accesses an electronic commerce website. The user 402 is shown as a notebook personal computer in FIG. 4, but the user 402 may be any computer, portable device, or tablet, among others. In some embodiments, the user 402 is configured to allow the user to upload product data to an e-commerce website. In some embodiments, the user 402 is configured to receive search results. In some embodiments, the user 402 is configured to display search results.

  Those skilled in the art will recognize that various modifications and alternatives can be made to the embodiments of the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and alternatives to the embodiments of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention includes all these modifications and alternatives. Is also intended.

  Although the foregoing embodiments have been described in some detail for purposes of clarity of understanding, the invention is not limited to the details provided. There are many alternative ways of implementing the present invention. The disclosed embodiments are illustrative and not restrictive.

Claims (21)

A method for categorizing products,
Extracting multiple product titles from the acquired data;
Dividing the title into phrases;
Determining each score for the phrase;
A first word sequence corresponding to a first product of the plurality of products using at least one of the phrases selected based at least in part on each determined score for the phrase Configuring
Comparing the first word sequence to a second word sequence corresponding to a second product of the plurality of products;
Consolidating the first product and the second product of the plurality of products into a product category based at least in part on the comparison;
A method comprising: The method of claim 1, further comprising:
Determining a similarity between the first product category and the second product category;
Coalescing the first product category with the second product category if the determined similarity is at least commensurate with a coalescence threshold;
A method comprising: The method of claim 1, comprising:
The method wherein determining each score for the phrase is based at least in part on the past occurrence frequency of the phrase. The method of claim 1, further comprising:
Extracting attribute information about the plurality of products from the acquired data, and dividing the attribute information into phrases. The method of claim 1, comprising:
Comparing the first word sequence with a second word sequence for a second product of the plurality of products is whether the first word sequence is similar to the second word sequence Determining. 6. A method according to claim 5, wherein
The method of determining whether the first word sequence is similar to the second word sequence is based at least in part on a match rate. The method of claim 1, comprising:
Integrating the first product and the second product of the plurality of products into one product category means that the first product and the second product of the plurality of products Integrating the data associated with the method. The method of claim 1, comprising:
Integrating the first product and the second product of the plurality of products into one product category means that the first product and the second product of the plurality of products Storing both with a category identifier. The method of claim 2, comprising:
Determining similarity includes calculating a value based on a determined score corresponding to the first product category and a determined score corresponding to the second product category. The method of claim 2, comprising:
Combining the first product category with the second product category includes storing the first product category and the second product category with the same category identifier. A system for categorizing products,
One or more processors;
A memory connected to the one or more processors and configured to provide instructions to the one or more processors;
With
The one or more processors are:
Extract multiple product titles from the acquired data,
Divide the title into phrases,
Determine each score for the phrase,
A first word sequence corresponding to a first product of the plurality of products using at least one of the phrases selected based at least in part on each determined score for the phrase Configure
Comparing the first word sequence to a second word sequence corresponding to a second product of the plurality of products;
Based at least in part on the comparison, such that the first product and the second product of the plurality of products are integrated into a product category;
Configured system. The system of claim 11, comprising:
The one or more processors further include:
Determining the similarity between the first product category and the second product category;
Based on whether the determined similarity exceeds a merge threshold, the first product category is merged with the second product category,
Configured system. The system of claim 11, comprising:
The system wherein the one or more processors are configured to determine each score for the phrase based at least in part on a past occurrence frequency of the phrase. The system of claim 11, comprising:
The system wherein the one or more processors are further configured to extract attribute information about the plurality of products from the acquired data and divide the attribute information into phrases. The system of claim 11, comprising:
The one or more processors include determining whether the first word sequence is similar to the second word sequence, the first word sequence of the plurality of products A system configured to compare with a second word sequence for a second product of which. The system of claim 15, comprising:
The one or more processors are configured to determine whether the first word sequence is similar to the second word sequence based at least in part on a match rate. ,system. The system of claim 11, comprising:
The one or more processors include integrating data associated with the first product and the second product of the plurality of products, the first of the plurality of products. A system configured to consolidate one product and the second product into one product category. The system of claim 11, comprising:
The one or more processors include storing both the first product and the second product of the plurality of products with the same category identifier, and the one of the plurality of products A system configured to integrate a first product and the second product into one product category. The system of claim 11, comprising:
The one or more processors include calculating a value based on a determined score corresponding to the first product category and a determined score corresponding to the second product category; A system that is configured to determine degrees. The system of claim 12, comprising:
The one or more processors include storing the first product category and the second product category together with a category identifier, wherein the first product category is the second product category. A system that is configured to merge. A computer program product for categorizing products, implemented on a computer-readable storage medium,
Computer instructions for extracting multiple product titles from the acquired data;
Computer instructions for dividing the title into phrases;
Computer instructions for determining each score for the phrase;
A first word sequence corresponding to a first product of the plurality of products using at least one of the phrases selected based at least in part on each determined score for the phrase Computer instructions for configuring,
Computer instructions for comparing the first word sequence to a second word sequence corresponding to a second product of the plurality of products;
Computer instructions for consolidating the first product and the second product of the plurality of products into a product category based at least in part on the comparison;
A computer program product comprising:

Images