KR20180111979A - Semantic category classification - Google Patents

Abstract

According to an exemplary embodiment, a large category classification based on sequence semantic embedding and parallel learning is described. As an example, the one or more closest matches may be (i) a publication semantic vector corresponding to at least a portion of a publication, the publication semantic vector being based on a first machine learning model that projects at least a portion of the publication into a semantic vector space And (ii) by comparing a plurality of categories corresponding to each category from a plurality of categories.

Description

Semantic category classification

Cross-reference to related application

This application claims the benefit of US Provisional Application No. 62 / 293,922, filed February 11, 2016, the entirety of which is incorporated herein by reference.

Embodiments of the present invention generally relate to a large category classification and recommendation system (CatReco) based on sequence semantic embedding and parallel learning.

Properly classifying a publication (e.g., a product and / or service) into a publication corpus is important in helping the system provide a recommendation of the publication in response to the user's query. The description of the publication is used to index the publication in the system so that a potential user can locate the publication by querying the user.

The various accompanying drawings are merely representative of the exemplary embodiments of the invention and are not intended to limit the scope of the invention.
1 is a block diagram illustrating a networked system in accordance with some exemplary embodiments.
2 is a block diagram illustrating the listing system of FIG. 1 in greater detail, in accordance with an illustrative embodiment.
Figures 3A and 3B are user interfaces of a listing system used to select a category of listing titles by providing listing titles, in accordance with an illustrative embodiment.
Figure 4 shows a simple example of matching a source semantic vector to a nearest target semantic vector.
5A shows a flow diagram using Sequence Semantic Embedding (SSE) to provide at least one CatReco to a user.
Figure 5B shows a flow diagram of providing a recall set of a leaf category (LeafCat) identifier (ID) to a service using SSE, in accordance with an exemplary embodiment.
6A illustrates a flowchart for performing a runtime process for performing a runtime classification process for a base SSE CatReco service, in accordance with an exemplary embodiment.
6B illustrates a flowchart for performing an offline process for precomputing a target semantic vector for a basic SSE CatReco service, in accordance with an illustrative embodiment.
6C illustrates a flowchart for performing a runtime classification process for a base SSE CatReco service, in accordance with another exemplary embodiment.
FIG. 6D shows a flowchart for performing a basic SSE CatReco service including on-line and off-line components, in accordance with an illustrative embodiment.
7 shows a flow diagram of a method for training an SSE model for a basic SSE CatReco service, in accordance with an exemplary embodiment.
8 shows a flow diagram of a method for deriving labeled training data for training an SSE model used in a basic SSE CatReco service, in accordance with an exemplary embodiment.
Figure 9 shows a flowchart for training an SSE model for a basic SSE CatReco service, in accordance with another exemplary embodiment.
Figure 10 shows a flow diagram for performing SSE-statistical language modeling (SLM) -gradient boosting machine (GBM) runtime processes to generate CatReco, in accordance with an exemplary embodiment.
FIG. 11 illustrates a flow chart for performing an SSE-SLM re-ranking runtime process (SSE-SLM re-ranking runtime process), in accordance with an exemplary embodiment.
12 shows a flow chart for performing a first part of an SSE-SLM-GBM offline training process, in accordance with an exemplary embodiment.
FIG. 13 shows a flow chart for performing a second part of the SSE-SLM-GBM offline training process, according to one exemplary embodiment.
14 is a block diagram illustrating an example of a software architecture that may be installed on a machine, in accordance with some illustrative embodiments.
15 shows a schematic diagram of a machine in the form of a computer system in which a set of instructions may be executed to cause the machine to perform any one or more of the methods described herein, in accordance with an exemplary embodiment.
Figure 16 illustrates an exemplary method for comparing and identifying related categories of a publication.

The subject matter provided herein is for convenience only and does not necessarily affect the scope or meaning of the term used.

The following description includes systems, methods, techniques, instruction sequences, and computing machine program products that implement the exemplary embodiments herein. In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of various embodiments of the subject matter of the present invention. It will be apparent, however, to those skilled in the art that the novel claimed embodiments of the invention may be practiced without these specific details. Generally, known instruction instances, protocols, structures, and techniques may not be described in detail.

In the publication corpus, a very large category was established to fine-tune billions of different publications (product offers). The category classification system is often used to help sellers sort the list of publications based on several title keywords.

Various embodiments may be used to automatically extract labeled data (e.g., billions) of very large size from unsupervised user logs and to use them in parallel for supervised machine learning model training Describe the learning framework.

In an exemplary embodiment, a listing title (e.g., a title keyword of a publication to be listed) and a category tree path may be defined as a source sequence using a sequence semantic embedding (SSE) , The target sequence >. A set of recall candidates can be obtained using the vector distance of the source semantic vector representation and the target semantic vector representation to measure the similarity. The classification recall candidate set may represent a number of LeafCats in the category tree identified by the LeafCat ID.

In another embodiment, a classification recall candidate set is derived from a sentence embedded similarity score (derived using SSE) and a language model perplexity score (SLM (statistical language modeling)) to GBM the gradient model is trained for all categories (e.g., LeafCat) so that it can be reordered with an ensembled signal. The CatReco results produced by this combined SSE-SLM-GBM approach are far superior to the various other approaches. For example, benchmark test results using more than 370,000 samples covering over 19,000 different LeafCats resulted in a 10x improvement in system response time (eg, ~ 200ms to ~ 20ms) and a 24.8% improvement in classification error in the top 1 CatReco As much as 31.12% in the top 3 CatReco and 54.52% in the top 10 CatReco.

The accuracy of CatReco, especially the accuracy of the top 1 referral leaf category (LeafCat), is important because some important information about the publication, such as the seller tag, the listing fee, and the product matching depend on the LeafCat for the publication, Can directly affect the overall experience of the user (e.g., buyer and / or seller). Also, the accuracy of identifying the top 1 recommendation LeafCat is often an obstacle to the Business to Consumer (B2C) automation classification flow. The accuracy of the top 1 CatReco by the issuing system can have a direct impact on gross merchandise volume (GMV), which represents the total sales dollar value of goods sold through a particular marketplace during a particular time period.

Referring to FIG. 1, an exemplary embodiment of a high-level client-server-based network architecture 100 is illustrated. The networked system 102 in the illustrative form of a network-based publishing system or a payment system may provide server-side functionality to one or more client devices 110 via a network 104 (e.g., the Internet or a wide area network (WAN) to provide. Figure 1 illustrates a web client 112 (e.g., a browser such as Internet Explorer developed by Microsoft Corporation of Redmond, Washington) running on client device 110, a client application 114, A programmatic client 116 is shown.

The client device 110 may be a mobile device, a desktop computer, a laptop, a personal digital assistant (PDA), a smartphone, a tablet, an ultrabook, a netbook, a laptop, a multiprocessor system, a microprocessor- Boxes, or any other communication device that a user may use to access the networked system 102. In some embodiments, client device 110 may include a display module (not shown) that displays information (e.g., in the form of a user interface). In another embodiment, the client device 110 may include one or more of a touch screen, an accelerometer, a gyroscope, a camera, a microphone, a Global Positioning System (GPS) device, and the like. Client device 110 may be a user's device used to perform transactions associated with a digital publication within networking system 102. In one embodiment, networking system 102 is responsive to a product listing request to publish a publication containing a list of products available in a network-based marketplace, and to provide a network-based marketplace to be. One or more portions of the network 104 may be an ad hoc network, an intranet, an extranet, a virtual private network (VPN), a local area network (LAN), a wireless local area network (WLAN) A cellular network, a wireless network, a WiFi network, a WiMax network, another type of network, or any other type of network, such as a wireless WAN, a metropolitan area network, a part of the Internet, a part of the Public Switched Telephone Network (PSTN) And may be a combination of two or more of these networks.

Each of the client devices 110 includes one or more applications (also referred to as "apps "), such as a web browser, a messaging application, an email (email) application, a publishing system application (also referred to as a marketplace application) But is not limited to this. In some embodiments, if a publishing system application is included in a given one of the client devices 110, the application may include data or processing capabilities that are not locally available (e.g., a database of publishable publications, , An access for payment method identification), and an application configured to communicate with the networked system 102. In some embodiments, Conversely, if a publishing system application is not included in the client device 110, the client device 110 may use its web browser to access the hosted publishing system (or variations thereof) on the networking system 102 have.

The one or more users 106 may be a person, machine, or other means of interacting with the client device 110. In an exemplary embodiment, the user 106 may interact with the network architecture 100 through the client device 110 or other means, although it is not part of the network architecture 100. For example, a user may provide an input (e.g., a touch screen input or an alphanumeric input) to the client device 110, and the input may be communicated to the networking system 102 via the network 104. In this example, networking system 102 communicates information to client device 110 over network 104 in response to receipt of an input from a user and provides it to the user. In this manner, a user may interact with the networked system 102 using the client device 110. [

An application program interface (API) server 120 and a web server 122 are each coupled to one or more application servers 140 and provide a programmatic and web interface to one or more application servers 140, respectively . The application server 140 may host one or more publishing systems 142 and a payment system 144, each of which may include one or more modules or applications, and each of which may include hardware, software, firmware, As shown in FIG. Application server 140 is shown in turn to be coupled to one or more database servers 124 that facilitate access to one or more information stores or databases 126. In an exemplary embodiment, the database 126 is a storage device that stores information (e.g., a publication or listing) to be posted to the publishing system 120. The database 126 may also store digital publication information in accordance with an exemplary embodiment.

The third party application 132 running on the third party server 130 is also shown as being programmed access to the networked system 102 via the programmatic interface provided by the API server 120. For example, third party application 132 using information obtained from networking system 102 supports one or more features or functions on a web system hosted by a third party. For example, the third party web system provides one or more promotional, marketplace or payment functions supported by the associated application of the networked system 102.

The publishing system 142 may provide a number of publishing functions and services to the user 106 accessing the networking system 102. Likewise, the payment system 144 may provide a number of functions for performing or facilitating payments and transactions. Although the publishing system 142 and the payment system 144 are shown in Figure 1 to form part of the networked system 102, in an alternative embodiment each system 142,144 forms part of a separate payment service And can be distinguished from the networked system 102. In some embodiments, payment system 144 may be formed as part of issuing system 142.

The listing system 150 provides functionality that is operable to perform various aspects of listing a publication for sale using data selected by the user. In various embodiments, the seller may list the publication (using the listing system 150) by providing a title or description of the publication being listed. The title may be referred to as a listing title and provides CatReco for the publication listed and used by the listing system 150 (or other component within the publishing system 142). In another embodiment, the listing system 150 may access user-selected data from the database 126, the third-party server 130, the publishing system 120, and other sources. In some exemplary embodiments, the listing system 150 analyzes user data to perform personalization of user preferences. If more content is added to the category by the user, the listing system 150 may further improve the personalization settings. In some exemplary embodiments, the listing system 150 communicates with the publishing system 120 (e.g., accessing a publication listing) and the payment system 122. [ In another embodiment, listing system 150 is part of issuing system 120.

In addition, although the client-server based network architecture 100 shown in FIG. 1 uses a client-server architecture, the subject matter of the present invention is of course not limited to such a structure and may be, for example, distributed or peer- -peer) Structured systems can also find applications as well. The various publishing systems 142, payment systems 144, and listing system 150 may also be implemented as stand-alone software programs that do not require networking functionality.

Web client 112 may access a variety of publishing and billing systems 142 and 144 via a web interface supported by web server 122. [ Similarly, the programmatic client 116 accesses various services and functions provided by the issuing system and billing system 142, 144 via the programmatic interface provided by the API server 120. The programmatic client 116 allows the merchant to author and manage listings on the networked system 102 in an off-line manner and to enable the merchant to author and manage listings in the bulk mode < RTI ID = 0.0 > (e. g., a Turbo Lister application developed by eBay, Inc. of San Jose, CA) capable of performing batch-mode communications.

The third party application 132 executing on the third party server 130 is also shown as having programmatic access to the networking system 102 via the programmatic interface provided by the API server 120 . For example, a third party application 132 using information obtained from the networked system 102 may support one or more features or functions on a web system hosted by a third party. The third party web system may provide one or more promotional, marketplace or payment functions, for example, supported by the associated application of the networked system 102.

FIG. 2 is a block diagram illustrating the listing system 150 of FIG. 1 in greater detail in accordance with an exemplary embodiment. Here, the listing system 150 includes a listing server 200 operative to perform a back end process associated with the listing of the publication. The listing system 150 includes a CatReco component 202 among other components. The user device 204 may be used directly to list the publication to sell by providing the details of the publication to which the user is to interact with the listing user interface 206 to list. The listing user interface 206 passes this information to the listing server 200. This process can be essentially interoperable. For example, any input by the user via the listing user interface 206 is sent to the listing server 200, at which point the listing server 200 provides feedback, and the user changes the listing information provided It can be added.

For purposes of the present invention, the description will be limited to the CatReco form of the listing server 200 implemented by the CatReco component 202. In one embodiment, the user may initiate a title entry or other text entry via the listing user interface 206, after which a title entry or other text entry may be communicated to the CatReco component 202. The CatReco component 202 may then provide a list of proposed orders of categories for listings of publications that the user may select via the listing user interface 206. [ In another exemplary embodiment, a user (e.g., a B2C merchant) may upload a list of publications to be listed by the listing system 150. The list of publications includes listing titles and categories (based on seller classification schemes) associated with each entry. The CatReco component 202 may then be automatically matched to the category (based on the merchant ' s sales system) to the category for each entry (based on the classification system of the issuing system 142). The seller may provide the seller's taxonomy to the seller's provided list of items (e.g., an entry with the listing title and category), or the seller may provide a copy of the seller's taxonomy for uploading to the publishing system 142 have.

Various embodiments of the CatReco component 202 (a combination of components between the listing system 150 and the publishing system 142) can be used to reorder the SLM to establish accurate, robust, and quick recommendations for the categories of publications listed And leverage SSE with the GBM method.

The listing user interface 206 may take many forms. In one exemplary embodiment, the listing user interface 206 is a web page that is executed by a web browser on the user device 204. In another exemplary embodiment, the listing user interface 206 is a mobile application installed on a mobile device. 3A and 3B illustrate examples of user interfaces created by the listing user interface 206 for listing publications and selecting categories of listing publications.

The listing server 200 may also be accessed by the third party service 208 via the listing API 210. One example of a third party service 208 is a web system that aides the seller in the listing process by listing the publication on behalf of the seller. The listing API 210 is specifically designed to interact with the listing server 202 and may be distributed to a number of third parties 208.

If the user selects a category for listing (at least in part due to the CatReco component 202) or if the listing system automatically matches the category from the seller's classification system to the classification system of the publication system 142, 200 delivers the publication listing to the article management server 212 and the article management server 212 manages the process of publishing the publication listing by storing the publication listing in the listing database 214. [ This can be achieved through a distributed architecture such as Hadoop.

The model server 216 then retrieves from the listing database 214 to perform offline training to create and / or modify the model (including the LeafCat model) used by the CatReco component 202 when recommending the category to the user Information about listings can be obtained. As described above, the language model for all categories (e.g., LeafCat) is based on the assumption that the classification recall candidate set is a GBM from sentence embedded similarity score (derived using SSE modeling) and language model congestion score (derived using SLM) And are trained so that they can be reordered as an ensemble signal. In various embodiments, model server 216 provides the ability to train various models used to compute SSE-SLM-GBM CatReco results. In some embodiments, the model server 216 may obtain information for performing off-line training of the SSE model.

In various embodiments, the SSE is used to encode a sequence of symbols (such as a phrase, sentence or paragraph) into a continuous dimension vector space, where the semantic level similar sequence will have a closer representation in this vector space. This SSE approach automatically captures the deep latent semantic meaning of the listing title and projects the semantic level semantics into a shared multidimensional vector space.

Deep learning has recently shown many possibilities in NLP (Natural Language Processing). NLP researchers in this field are trying various ways to encode a sequence of symbols (eg, phrases, sentences, paragraphs, and documents) into a multidimensional vector space called a semantic space. The semantic level similar sequence will have a closer representation in this multidimensional space. Research in this area has led to the adoption of vector space representations of sentences instead of simple words. In general, phrases and sentences better define contextual information rather than a single word. In various embodiments, the study of sentence embedding is utilized to recommend a category for the publication the seller lists on the issuing system.

In an exemplary embodiment, the SSE is used to embed the deep latent semantic meaning of a given listing title and project it into a shared semantic vector space. A vector space is a set of objects called vectors. Vector spaces can be specified by their dimensions, and their dimensions specify the number of independent directions in the space. A semantic vector space can represent phrases and sentences, and can capture meaning for NLP operations.

Similarly, by using different projection functions, SSE is used to embed in-depth latent semantic meaning (i.e., from top level to leaf level) of a given category tree path and to project it into a shared semantic vector space. This SSE approach captures contextual information and in-depth semantic meaning from the listing title and enables CatRecos to handle many inconsistencies in words such as synonyms, typo, compound words, and split terms.

In an exemplary embodiment, at the system runtime, the entered listing title is projected into a shared semantic vector space, and the listing system retrieves from the offline pre-computed SSE list for the leaf category from the taxonomy classification scheme used by the listing system LeafCat with the closest SSE representation is recommended. In another exemplary embodiment, at the system runtime, the entered listing title is projected into a shared semantic vector space, and the listing system recommends a set of leaf categories used as input to other services of the listing system, Results and scores are generated. For example, other services may include an SLM re-ranking service or a GBM fusion prediction service.

Various in - depth semantic models are trained to project semantic similar phrases to a vector close to each other and to project semantically different phrases into distant vectors. At the time of training, if the listing title T can be classified as LeafCatC ₁ , the projected semantic vector space values for T and C ₁ should be as close as possible, i.e., || SSE (T) - SSE (C ₁ ) || should be minimized. On the other hand, the projected semantic vector space values for any other leaf category C _n should be as far as possible, i.e., || SSE (T) -SSE (C _n ) || should be maximized. During training, the cosine similarity between semantic vectors can be calculated. Therefore, the semantic relation between two vectors can be measured by the degree of cosine similarity.

In various embodiments, machine learning is used to maximize the degree of similarity between the source (X), e.g., the listing title, and the target (Y), a leaf category within the category tree to generate CatRecos. The SSE model may be based on a deep neural network (DNN) and / or a convolutional neural network (CNN). DNN is an artificial neural network with multiple hidden layers between the input and output layers. DNN can apply deep learning structures to recurrent neural networks. The CNN is composed of one or more convolutional layers having a layer completely connected from the upper level (for example, a layer matched with a general artificial neural network). CNN also uses tied weights and pooling layers. Both DNN and CNN can be trained using a standard backpropagation algorithm. Figures 7-9 provide an exemplary flow chart for training SSE models. The trained SSE model is used during runtime by the underlying SSE CatReco service, as shown in Figure 6d.

The SSE model requires a large amount of labeled data for the model training process. Obtaining a huge amount of label data through the manual labeling process is prohibitively expensive. This limitation can be overcome by automatically deriving cleanly labeled learning data using a parallel learning approach that utilizes millions of vendor online behaviors. This parallel learning approach can be implemented using two layers of filters to automatically derive pure training data (pairs of listing titles and leaf categories) for SSE training. Figure 8 illustrates an exemplary method of identifying a pair of labeled training data.

In an exemplary embodiment, the operation of a CatReco given a set of keywords (from a query or listing title provided by a user, such as a seller) may result in a sequential list of associated leaf categories from the category tree representing the classification scheme used by the publishing system Lt; / RTI > Based on a given set of keywords provided by the seller, the issuing system recommends a leaf category associated with a given set of keywords, along with some of the order or scores for each recommendation leaf category. CatReco is often one of the first steps in the consumer sales flow (ie, for listing publications) in the listing system.

According to various embodiments, as described in various embodiments, the SSE is used to sort listing titles into categories used by the issuing system. The categories may represent LeafCats (also referred to as category nodes) in the category tree from the classification scheme used by the publishing system. This approach is scalable, reliable, and inexpensive. There are many advantages to using SSE with CatRecos in the listing system.

First, generating training data automatically through the parallel learning framework for SSR-based CatReco reduces the cost of manually labeling training data. The automatically generated training data is based on a parallel learning framework that ensures high accuracy for the label utilizing the way millions of sellers behave from the issuing system and other off-line available information.

Second, the SSE-based CatReco eliminates dependence on the known KNN (Known Nearest Neighbor) recall set. For example, a KNN recall set can be replaced by an SSE recall set.

Third, out-of-voabulary issues can be addressed by training the SSE model at the subword / character level instead of at the word level. This allows CatReco to process compound words, segmented words, and typo, as well as handle a large number of words. Given that SSE encodes the entire sequence context, modeling at the lower word level does not lose contextual semantic information.

Fourth, the CatReco system allows all semantic spatial vector representations for all category tree paths (e.g., 16,000 leaf categories) to be pre-computed offline in advance, and also for efficient log-level K-dimensional (KD) Can be applied to quickly identify the best matched category tree path, thus providing rapid response during runtime.

Finally, the semantic spatial vector representation for any possible new LeafCat can be computed directly in seconds, without having to retrain any models. This makes SSE-based CatReco systems highly scalable, especially when there are many updates to the category tree.

In various embodiments, the classification scheme used by the publishing system is represented by a category tree. In other embodiments, other classification scheme structures may be used. It will be appreciated that while the above example describes CatRecos generated by a publishing system, various embodiments may be implemented in other types of online systems and are not limited to an issuing system.

In another exemplary embodiment, the SSE may be used to map the source (X) to the target (Y) for other NLP operations to identify one or more leaf categories, and to map the listing title (e.g., source) But is not limited to mapping to a tree (e.g., a target). The following is a table listing examples of various NLP operations with associated sources and targets. In Table 1 below, the source

FIG. 3A illustrates a user interface 300 for listing publications in accordance with an exemplary embodiment. Field 310 is a text field in which the seller provides a title for listing. In the example shown in Fig. 3A, the seller provides the title "Clash of the titans movie" to describe the publication to be listed. The title for the listing is often a general description of the listing (and may include a description of the property associated with the publication). The publishing system may use the title to identify one or more related categories for listing publications. The user interface element 320 presents a seller-related category. In this particular embodiment, the top three categories are presented to the user to select the category in which the user would like to list the publication. Each of the categories represents a leaf of a category within a category tree for an exemplary embodiment. According to Fig. 3A, the first category "DVD & Movie> DVD & Blu-ray Disc" is selected by the seller. FIG. 3B shows an example of a publication listing on the issuing system in the category "DVD & Movie> DVDs & Blu-ray Disc ".

When the SSE is applied to map a particular <source, target> pair, the parameters of the SSE source model and SSE target model are optimized so that the relevant <source, target> pair has a closer vector representation distance. The following formula can be used to calculate the minimum distance.

here,

ScrSeq = source sequence;

TgtSeq = target sequence;

SrcMod = Source SSE model;

TgtMod = target SSE model;

SrcVec = a continuous vector representation of the source sequence (also known as the source semantic vector); And

TgtVec = a continuous vector representation (also referred to as the target semantic vector) for the target sequence.

The source SSE model encodes the source sequence into a continuous vector representation. The target SSE model encodes the target sequence as a continuous vector representation. In an exemplary embodiment, the vectors each have about 100 dimensions.

4 shows an example 400 of a listing title provided by the seller. The listing title 410 shown in FIG. 4 is "hello kitty T-shirt ". In this example, three dimensions are displayed. Two leaf nodes 451 and 452 of the category tree 450 having a root node 453 are also shown. The source SSE model generates a semantic vector of source (X) 420. X is represented by the vector [0.1, 2.3, 3.0]. The target SSE model generates a semantic vector of the targets (Y1 and Y2) 430 and 440. Y1 for the leaf node 451 "Clothing, shoes, accessories> Girls> T-shirt" is represented by the vector [0.1, 2.2, 3.0] Y2 is represented by vector [0.5, 2.6, 2.3]. In this example, depending on the dimension value of the vector, the listing title "Hello Kitty T-shirt" may be larger than the "clothing, shoes, accessories " shirt " of leaf node 452 based on the dimensions of the source and target semantic vector Quot; clothes, shoes, accessories &gt; girl &gt; T-shirt "of the leaf node 451 appear. The example shown in Figure 4 is a very simple example having only three dimensions.

In other embodiments, any number of dimensions may be used. In an exemplary embodiment, the dimensions of the semantic vector are stored in a KD tree structure. The KD tree structure is called a space division data structure for organizing points in the KD space. The KD tree can be used to perform the nearest neighbor lookup. Thus, given a source point in space, the nearest neighbor lookup can be used to identify the point closest to the source point.

5A is a flowchart 500 illustrating a runtime classification process for matching a listing title to a category classification scheme of a system according to an exemplary embodiment. The classification scheme used by the listing system 150 for a category can be represented by a category tree and each leaf in the category tree can represent a category. The flowchart 500 includes operations 510, 520, 530, and 540.

At operation 510, the listing system 150 receives the listing title of the publication. At operation 520, the SSE is used to map the listing title to the category classification scheme used by the listing system 150 listing the publication. At act 530, at least one related category is identified. Identify the relevant category as a category classification scheme used in the publication system for listings of publications. At operation 540, at least one identified related category is provided to the device for presentation to the user.

FIG. 5B is a flowchart 501 illustrating a runtime classification process for matching a listing title to a category classification scheme of a system according to an exemplary embodiment. The leaf category (LeafCat) identifier (ID) is identified using the recall set SSE. The flowchart 501 includes operations 510, 520, 535, and 545. At operation 510, the listing system 150 receives the listing title of the publication. At operation 520, the SSE is used to map the listing title to the category classification scheme used by the listing system 150 listing the publication. At act 535, a set of related categories is identified. Identify the relevant category by the category classification scheme used by the publishing system to list the publication. The related category may indicate the top N category of the received listing title. As an example, N = 50. At operation 545, the service of the listing system 150 is provided with a recall set of LeafCat IDs. For example, the service may be an SLM re-ranking service or a GBM fusion prediction service.

6A is a flow diagram 600 illustrating operation 520 of further mapping the listing title to the category classification scheme used by the issuing system for publication listing, using SSE. Mapping operation 520 includes operations 610, 620, and 630.

At operation 610, a semantically vector of the target Y (i.e., using the target SSE model) is obtained. The pre-computed semantic vector of the target (Y) produces a semantic vector space. In one embodiment, the category system entry of the target system is computed using the target SSE model. The pre-computed semantic vector of the target Y is computed offline and is described in more detail with reference to FIG. 6B. The target SSE model encodes the target sequence as a continuous vector representation.

At operation 620, the semantic vector representation of the source X is projected into the shared semantic vector space. The semantic vector space generated by the semantic vector of the target (Y) is combined with the semantic vector of the source (X) to generate a shared semantic vector space. The source SSE model is used to create a semantic vector representation of the listing title.

At operation 630, a target (Y) semantic vector representation (of the classification entry) having the semantic vector representation closest to the source (X) semantic vector representation (of the listing title) within the shared semantic vector space . The category entry may represent LeafCat. The semantic relevance sim (X, Y) is calculated using the cosine similarity function in the exemplary embodiment. An example of a sub-operation for operations 620 and 630 will be described with reference to Fig. 6C.

As shown in FIG. 6B, the flowchart 601 includes operations 611 - 614. According to FIG. 6B, at operation 610, a pre-computed semantic vector of the target Y is obtained.

At act 611, the target is accessed. In an exemplary embodiment, the target indicates that the path to the category tree of the listing system 150 is being accessed. The path represents the root of LeafCat. In an exemplary embodiment, the category tree path is accessed from a database from the listing system 150.

In operation 612, word hashing is performed on the category tree path from the target. In an exemplary embodiment, word hashing is performed using a letter-trigram. Abbreviated word hashing takes the original phrase (e.g., the leaf path to the root) and identifies the abbreviation after preprocessing (e.g., adding # to empty space). Word hashing can be used to create a compact representation of large vocabularies. For example, you can reduce the vocabulary of 500,000 to an abbreviation of 30,000 characters. The word hashing creates a robust listing system 150 or other system for retrieval, inflections, compound words, segmented words, and the like. In addition, unseen words can be generalized using word hashing.

At operation 613, the target SSE model is used to generate a semantic vector of the target (also referred to as a semantic vector representation).

At operation 614, the semantic vector of the target is stored in the KD tree of the memory device. In an exemplary embodiment, the dimension of the target semantic vector is stored in the KD tree structure. In an exemplary embodiment, the target semantic vector represents a vector for each LeafCat in the category tree. The leaf category can be expressed as a leaf node as shown in Fig. An example of a category tree for listing system 150 includes more than 19,000 category tree paths (e.g., routes to leaves).

By precomputing the target semantic vector, the process of mapping the semantic vector of the source representing the listing title can be computed very quickly. In various embodiments, the target sequence is precalculated prior to runtime (as indicated by operation 601 of FIG. 6B) and compared to the target sequence vector during run-time after the source sequence vector is calculated during run-time. FIG. 6C illustrates an example of combining the offline process and the runtime process for calculating the target sequence vector, so that the listing title is associated with the category classification scheme used by the listing system 150 (using the SSE model and between the source semantic vector and the target semantic vector) (670) that can be mapped by calculating a semantic relevance (e.g., using a cosine function).

6C, the flowchart 670 includes an off-line operation 601 and a run-time operation 610, 620, 630. The offline operation 601 is shown in FIG. 6B. Runtime operations 610, 620 and 630 are shown in Figure 6A. (For projecting the semantic vector representation of the source X into the shared semantic vector space) includes sub-operations 615 and 616. [ (To identify the target (Y) semantic vector representation (of the categorization entry) having the semantic vector representation closest to the source (X) semantic vector representation (of the listing title) in the shared semantic vector space (630) includes sub-actions (617, 618).

As shown in FIG. 6C, the semantic vector representation of the target is computed offline at operation 601. At operation 610, a pre-computed semantic vector of the target is obtained.

At operation 615, word hashing is performed on the source. In an exemplary embodiment, the source represents a listing title for listings. At operation 616, a semantic vector of the source is generated using the source SSE model. In an exemplary embodiment, combined operations 615 and 616 are used to project the semantic vector representation of the source into a shared semantic vector space (as shown at operation 620).

At operation 617, the relevance similarity sim (X, Y) is estimated. In operation 618, an optimal matched category Y (represented as a target semantic vector) having the shortest distance from X (represented as a source semantic vector) is identified. In an exemplary embodiment, combined actions 617 and 618 are used to determine the most likely (as indicated by operation 630) the source (X) semantic vector representation (of the listing title) within the shared semantic vector space (Y) semantic vector representations (of categorization entries) that have near semantic vector representations.

As described above, in various embodiments, the mapping can be performed by learning the semantic relevance sim (X, Y) between the source sequence vector and the target sequence vector. In an exemplary embodiment, the semantic similarity, also referred to as the semantic relevance, can be measured by the cosine similarity function sim (X, Y). In some embodiments, X represents the source sentence sequence (i.e., derived from the seller's title) and Y represents the target sentence sequence (i.e., derived from the category tree of listing system 150). The output of the cosine similarity function represents a shared semantic vector space. Generally, the category that best matches Y has the highest similarity score to X. The source sequence and the target sequence represent the calculated vector sequence, each having multiple dimensions.

FIG. 6D shows a flowchart 680 illustrating an SSE runtime classification process in accordance with an exemplary embodiment. The SSE runtime classification process shown in FIG. 6D is used to sort the listing titles by mapping the listing titles to the category classification scheme of the listing system 150. As described above, the flowchart 680 can be used to perform a number of operations by mapping a source to a target, such as those identified in Table 1 above. The basic SSE runtime classification process shown in FIG. 6D may also be referred to as a basic SSE categorization recommendation (catReco) service 680. The basic SSE categorization service 680 represents an SSE runtime decoding process for obtaining recall sets and similarity scores. In an exemplary embodiment, the recall set represents a set of N upper leaf nodes in a category tree. The recall set and similarity score may be used by the CatReco component 202 (shown in FIG. 2) to generate an SSE-SLM-GBM CatReco result. The generation of the SSE-SLM-GBM CatReco result is described below with reference to FIG.

Flow diagram 680 includes operations 611-618 and 510. [ Operations 611-614 have already been described with reference to Figure 6b. Operations 615-618 have already been described with reference to Figure 6c. Operation 510 has already been described with reference to FIG.

Operation 611-614 describes an offline process used to compute the target sequence semantic vector stored in the KD tree structure. The KD tree may be accessed during runtime so that the sequence semantics vector of the source may be projected into a shared semantic vector space having a sequence semantics vector of the target. The relevance similarity sim (X, Y) is estimated (at step 617) and the best matched category Y with the shortest distance to X is identified (at step 618). The best-matched category Y may be referred to as the top 1 category in the category tree that matches the listing title of the listing. In various embodiments, the upper "N" category is identified such that many N categories can be provided to the user. For example, in FIG. 3A, the top three categories are provided to a user (e.g., a listing seller) in a user interface 300.

In an exemplary embodiment, the target deep-seated SSE model 613A and the source deep-seated SSE model 616A are trained using the SSE model training process described in Figs. 7-9.

In an exemplary embodiment, the CatReco operation is used to sort the listing titles provided by the user into LeafCats. Categorizing listing titles can be difficult if you have a large number of categories. CatReco operations are often used in various sales flows of the listing system 150. In an exemplary embodiment, the listing system 150 may have more than 19,000 different categories in the United States. The listing system often works to improve the accuracy of choosing the most relevant category from more than 19,000 categories based on a set of keywords provided by the user and a response time when generating and presenting CatReco to a listing seller .

7-9 illustrate the training process for the SSE model used by the base SSE CatReco service 680 in accordance with the illustrative embodiment. One embodiment of the SSE runtime classification process is shown in FIG. 6D and may be referred to as a basic CatReco SSE service 580 when used to perform CatReco operations by mapping the listing title to the category tree path of the listing system 150. [ FIG. 7 shows a flow diagram 700 for an SSE training model in accordance with an exemplary embodiment. FIG. 8 shows a flowchart 800 for actively identifying labeled training data pairs used by the SSE training model (shown in FIG. 7) according to an exemplary embodiment. FIG. 9 illustrates an example of an SSE model training process that includes the various operations and components shown in FIGS. 7 and 8. FIG.

Referring to FIG. 7, a source SSE model and a target SSE model are trained. Operations 710A, 720A, 730A, and 740A are used to train the source SSE model. Operations 710B, 720B, 730B, and 740B are used to train the target SSE model. At operation 701, a pair of training data labeled (listing title, category tree path) is provided to train both the source SSE model and the target SSE model. In an exemplary embodiment, the labeled training data pairs are identified using the flowchart 800 shown in FIG.

At act 710A, the source sentence sequence of the source listing title X is received. The source listing title (X) may represent the word sequence provided by the listing seller. At operation 720A, word hashing is performed on the source listing title X. [ In situations with very large large vocabulary words, hashing is performed on a subword basis. In various embodiments, 3-gram word hashing is performed.

In an exemplary embodiment, the convolution layer, the maximum pooling layer, and the semantic layer represent the neural network layer. A plurality of nodes (for example, 500 nodes as shown in Fig. 9) may be configured in these neural network layers. In other embodiments, the number of nodes may vary depending on the data size or may be configured with a different number. At act 730A, keywords and concepts are identified from the source listing title (X) using convolution and maximum pulling.

At operation 740A, a semantic vector representation of the source listing title (X) is extracted using a deep neural network (DNN). The DNN uses one or more neural network layers to project the input sequence into a semantic vector space.

At operation 710B, the source sentence sequence of the target category tree path Y is received. In an exemplary embodiment, the listing system 150 is used to list publications and may include more than 19,000 category tree paths (Y) or CatLeafs. At operation 720B, word hashing may be performed on the target category tree path Y. [ In situations with very large large vocabulary words, hashing is performed on a subword basis. In various embodiments, 3-gram word hashing is performed.

In an exemplary embodiment, the convolution layer, the maximum pooling layer, and the semantic layer represent the neural network layer. A plurality of nodes (for example, 500 nodes as shown in Fig. 9) may be configured in these neural network layers. In other embodiments, the number of nodes may vary depending on the data size or may be configured with a different number. At operation 730B, keywords and concepts are identified from the target category tree path (Y) using convolution and maximum pulling.

At operation 740B, a neural network (DNN) is used to extract a semantic vector representation of the target category tree path (Y). The DNN uses one or more neural network layers to project the input sequence into a semantic vector space.

At operation 750, the semantic vector distance between X and Y is used to measure the similarity between the semantic vector representation of the source listing title X and the semantic vector representation of the target category tree path Y. In an exemplary embodiment, the semantic relevance represented by the function sim (X, Y) is measured by the cosine similarity.

If both the source SSE model and the target SSE model are trained, the semantic vector representations for all of the target category classification scheme entries can be pre-computed in advance using the target SSE model. In addition, if it is necessary to map a new publication listing from the seller, the semantic vector representation of the listing title can be combined with the semantic vector representation of the category tree path from the cataloging system of the listing system 150, Lt; / RTI &gt; In an exemplary embodiment, the correct mapping for a listing title would be a category tree path having a semantic vector representation closest to the semantic vector representation of the listing title.

As described above, when the SSE is applied to map a specific <source sequence, target sequence> pair, the parameters of the SSE source model and the SSE target model are optimized so that the related <source, target> pair has a closer vector expression distance. The following formula can be used to calculate the minimum distance.

here,

ScrSeq = source sequence;

TgtSeq = target sequence;

SrcMod = Source SSE model;

TgtMod = target SSE model;

SrcVec = a continuous vector representation of the source sequence (also known as the source semantic vector); And

TgtVec = a continuous vector representation (also referred to as the target semantic vector) for the target sequence.

The source SSE model encodes the source sequence into a continuous vector representation. The target SSE model encodes the target sequence as a continuous vector representation. In an exemplary embodiment, the vectors each have about 100 dimensions.

Trained SSE modules are used to implement runtime classifications. In various embodiments, the training of the SSE model is performed off-line with training data, e.g., labeled training data pairs. In some embodiments, the labeled training data is derived automatically. In an exemplary embodiment, each labeled training sample is represented by a pair of < source sequence, target sequence >. In an exemplary embodiment, the source sequence represents the title of the publication listing. The target sequence represents LeafCat by the category tree path of the category classification scheme used by the listing system 150.

In general, good natural language processing and machine learning methods require labeled learning data (i.e., map learning). Training SSE modules with millions of labeled training data samples increases the accuracy of the mapping results. In various embodiments, the SSE model is trained using the pre-mounted publication listing in the listing system 150. [ Existing publication listings can quickly train SSE models into relevant data. For example, a company like eBay, located in San Jose, Calif., Has access to billions of pre-installed publication listings with seller's taxonomy information stored in a data warehouse. Preloaded publication listings can be processed to mine, join, and filter these millions of labeled training data based on eBay's previous transaction data.

FIG. 8 shows a flowchart 800 of a method of deriving labeled training data in accordance with an exemplary embodiment. At operation 810, historical data for the listing title stored in the data warehouse of the publishing system 142 is accessed. Historical data associated with previous publication listings registered by the merchant is accessed and history data may be stored in the data warehouse from publishing system 142. [ In various embodiments, the historical data associated with the previous publication listing includes the listing title and category selected by the listing seller during the listing process.

At operation 820, the LeafCat of the category tree stored in the database of the publishing system is accessed. In an exemplary embodiment, LeafCat may include more than 19,000 entries.

Based on the category classification scheme of the listing system 150, the data including the category tree path and the listing title of LeafCat.

At act 830, a listing title for each LeafCat is identified during a particular time period (e. G., Every 8 weeks). Using the data accessed by operations 810 and 820, a listing title for each leaf category is identified.

The training data is then filtered using the filter A in operation 840 and filter B in operation 850. Using filters A and B, the listing system 150 checks whether the seller's category selection matches the first recommendation from the CatReco component 202 of the listing system 150 (act 840). If there is a match, the listing system 150 checks if the miscategorization miscat score is low. If the score is low, it often indicates that the listing publication has been improperly categorized with the wrong LeafCat. An example of a low score could be 50. If the listing publication passes both filters A and B, the pair of (listing title, category tree path) is treated as a pure training sample.

At operation 860, a labeled training data pair (listing title, category tree path) is identified. In various embodiments, this is an automated process that identifies the labeled training pair used by the training process to implement active learning. The method shown in flowchart 800 may be automated to be used to actively train the SSE model process through machine learning, as shown by the flowchart 900 of FIG. 9, and the labeled training data pairs are periodically identified have.

FIG. 9 shows a SSE model training process and is a combination of a flowchart 700 (shown in FIG. 7) and a flowchart 800 (shown in FIG. 8) in an exemplary embodiment. One of the important goals of the SSE model training processor is to obtain an optimized source SSE model and an optimized target SSE model and minimize the distance between the continuous vector representation of the source sequence and the continuous vector representation of the target sequence for all pairs of training samples . In various embodiments, machine learning is used to optimize the source SSE model and the target SSE model to achieve this distance minimization goal.

FIG. 9 illustrates a flow diagram 900 in accordance with an exemplary embodiment. The method shown in FIG. 9 includes operations 710A-740A, 710B-710B, 750 (described with reference to FIG. 7) and operations 810-860 (described with reference to FIG. 8).

According to FIG. 9, a pure training pair (listing title, category tree path) is used to train the source SSE model and the target SSE model. A training pair may be referred to as a pure training pair because the process of generating a training pair (at operations 840 and 850) filters out the improperly classified pair using filters A and B. [ As an example, the listing title is "Video Monitor, Motorola - Wireless Video Baby Monitor - White". The category tree path is "Baby> Baby Safety & Health> Baby Monitor". The listing title of the training pair is provided as input to the source SSE model and the category tree path of the training pair is provided as input to the target SSE model. In an exemplary embodiment, the semantic relevance (of the source semantic vector and the target semantic vector of the listing title of the category tree path in the training data) measured by the cosine similarity is referred to as the similarity score. In an exemplary embodiment, the machine learning system in the CatReco component 202 uses a pure training pair (identified in operation 860) to train the source SSE model and the target SSE model.

In the exemplary embodiment, the basic SSE runtime classification process 680 provided by the base SSE CatReco service (as shown in FIG. 6D) is performed by the SSE model training process 900 (shown in FIG. 9) The target deep-seated SSE model 613A and the source deep-seated SSE model 616A trained by the SSE model training process.

The SSE model training process shown in FIG. 9 is a labeled training pair with the listing title "Video monitor, Motorola-wireless video baby monitor ", and the category tree path is" Tree path) is used to train the source and target SSE models, but other types of labeled training pairs may be trained using the SSE model training process shown in FIG. For example, when performing other types of actions, such as those shown in Table 1 above, the labeled training pair (listing title, product type tree path) or labeled training pair (category tree path, product type tree path) Can be used.

In various embodiments, the CatReco component 202 of the listing system 150 (shown in FIG. 2) may use the SLM provided by the SLM re-ranking service 1110 and the GBM fusion prediction service 1030 You can use the default SSE CatReco service in combination. The SLM re-ranking service 1110 and the GBM fusion prediction service 1030 may also be performed by the listing system 150 in the illustrative embodiment. A high-level block diagram of the SSE-SLM-GBM approach in accordance with the illustrative embodiment is shown in FIG. The flowchart 1000 shown in FIG. 10 illustrates a process for generating an SSE-SLM-GBM CatReco result by score.

In various exemplary embodiments, the SLM is used to improve the accuracy of the recommendations provided by the CatReco component 202. SLM is a data-driven modeling approach that attempts to limit the likelihood of a given text input, such as a sentence, a listing title, or a search query. The SLM can utilize a vast amount of autonomous text data (e.g., text data that is not labeled and has no explicit structure). In an exemplary embodiment, the SLM is used to train a language model for each LeafCat based on the unmanaged listing title, and the sentence log probability (SLP) of the new listing title is used in the appropriate LeafCat language model . This can be repeated for each candidate LeafCat. In various embodiments, the re-ranking process for the ranking of the proposed category is performed after the base SSE CatReco service 680 generates the similarity score and the SSE recall set. The recall set may represent the top N category generated by the basic SSE CatReco service 680.

In particular, in the exemplary embodiment, only the categories listed in the top N leaf category (identified by the default SSE CatReco service 680) are evaluated using the SLM reranking service 1110. This may be more efficient than running the SLM algorithm for all possible categories (e.g., more than 19,000 leaf categories).

In addition, in an exemplary embodiment, GBM is used to fusing various scores and data together, as described below, to combine predictions of some estimators to further refine the proposed category.

According to FIG. 10, the title of the publication listing is received in operation 1001. The title of the publication listing is provided to basic SSE CatReco service 680 and SLM re-ranking service 1110.

A default SSE CatReco service 680 is used to identify the top N LeafCat defined in the SSE recall set 1010 of the LeafCat Id for the listing title. An SSE recall set 1010 of LeafCat Id is provided as input to the SLM re-ranking service 1110. In an exemplary embodiment, the SLM re-ranking service 1110 includes two components: an SLM runtime classification step 1110A (shown in FIG. 11) and an SLM training step 1110B (shown in FIG. 12) do.

Rather than using a K nearest neighbor algorithm for an input text string (e.g., representing the title of a publication listing) to identify a set of leaf categories, the default SSE CatReco service 680 may be used to determine the leaf category That is, a LeafCat of the upper N). The set of leaf categories (defined by the SSE recall set 1010 of LeafCat ID) includes the SLM algorithm performed on the input, the SLM 1232 combined for each LeafCat, the log likelihood probability (LLP) for each LeafCat, (By the SLM re-ranking service 1110) based on the expected congestion and standard deviation (also referred to as the expected PPL and PPL_Std) 1236 for each LeafCat. LLP and PPL will be described in more detail with reference to FIG.

At operation 1030, the GBM Fusion Prediction Service 1030 receives the SSE recall set LeafCat Id 1010, the LLP 1212 for each LeafCat, the expected PPL and PPL_Std 1236 for each LeafCat, As an input, the output from service 1110 (i.e., the set of reordered LeafCats). Then, at operation 1030, the GBM fusion prediction service 1030 is used to fuse the received various inputs and compute the listing order of the recommended LeafCat to the corresponding score. The result of the GBM Fusion prediction is shown at 1040.

11 is a diagram illustrating an SLM runtime classification stage 1110A of an SLM re-ranking service 1110 according to an exemplary embodiment.

According to FIG. 11, a listing title entry 1001 is provided to the basic SSE CatReco service 680. The basic CatReco service 680 generates an SSE recall set 1010 of the LeafCat identifier (ID) provided as input to the SLM runtime classification stage 1110A.

The LLP 1212 for each LeafCat, the combined SLM 1232 for each LeafCat, and the expected PPL and PPL_Std 1236 for each LeafCat are accessed by the SLM runtime classification stage 1110A. More specifically, the LLP 1212 for each LeafCat is pre-calculated offline, stored in a file, and loaded into memory at runtime. The combined SLM 1234 for each LeafCat is the SLM model for each LeafCat, which is pre-trained offline and loaded into memory at runtime. Also, the expected PPL and PPL_STD (1236) for each LeafCat are pre-computed and stored in a file offline during the model training process and loaded into memory at runtime. The precomputation of LLP 1212 for each LeafCat, the combined SLM 1232 for each LeafCat, and the expected PPL and PPL_Std 1236 for each LeafCat are described in more detail with reference to FIG.

In the SLM runtime classification step 1110A, a deep signal is calculated to determine how far away the given listing is from the assigned leaf category. Assuming that the runtime publication listing title is T, the seller places it in category C, and the runtime congestion of the publication is calculated as PP (T), the deviation signal is calculated as:

Where alpha is a parameter that can be fine tuned (set to 2.0 in the exemplary embodiment).

Finally, Mean_PP (C), STD_PP (C), PP (T) and Deviation_PP (C, T) provide the deep features as GBM models along with superficial features such as price, condition, CatReco score, Can be made.

The LLP for the candidate LeafCat ID is identified based on the LLP 1212 for LeafCat in operation 1120. The candidate LeafCat ID is based on the SSE recall set of the LeafCat ID.

The SLP for the candidate LeafCat ID is identified based on the combined SLM 1234 for each LeafCat in operation 1130. [ The candidate LeafCat ID is based on the SSE recall set of the LeafCat ID.

The output of operation 1120 (i.e., the LLP identified for the candidate LeafCat ID) and the output of operation 1130 (i.e., the SLP identified for the candidate LeafCat ID) are used as inputs in operation 1140 to determine the SLM rank score . The SLM ranking score is used as input to operation 1150. At operation 1150, an SLM Vote Score is calculated based on the SLM Rank Score. At act 1150, an SLM ranking score for the listing title is generated.

In an exemplary embodiment, the SLM ranking score (SRS) ranking score for each LeafCat is calculated by adding the (weighted) individual SLP score and the LPP score, such as using the formula SRS = SLP + 1.8 * LPP . In an exemplary embodiment, the SLM vote score is calculated by dividing the difference between the maximum SRS score and the individual SRS score for the leaf category by 1 and the sum of the SRS scores for the leaf category, such as using formula SLM vote score = 1 / (formula 1 + Max_SRS - SRS) .

At act 1160, the predictions PPL and PPL_Std from the SSE recall set of the identified SLP and LeafCat ID for the candidate LeafCat ID are used as inputs to calculate the SLM PPL deviation percentile at operation 1160. At act 1160, an SLM congestion deviation signal for the listing title is generated. The congestion deviation signal can be referred to as a deep feature. In an exemplary embodiment, the SLM PPL deviation percentile is CurPPL / (PPL_Mean + 2 * PPL_Std). CurPPL represents the current congestion and is calculated at runtime. CurPPL represents the PPL value of the new listing title introduced for the candidate LeafCat's SLM model. Referring to the equations provided below, the term "PPL_Mean" may be referred to as meann_PPL, and "PPL_Std" may also be referred to as STD_PP.

During the SLM runtime re-ordering step 1110A, when the SSE generates a recall set of candidate LeafCat Ids, an SLP based on the title of the requested publication listing for the corresponding combined SLM 1232 of each candidate LeafCat for LeafCat , PPL and PPL_Deviation values are calculated at run time. The LLP, PPL, SLP, and PPL_Deviation values are used to re-rank the entire recalled leafCat candidate set.

In an exemplary embodiment, the sentence PPL can be computed as: It is assumed that the sentence S consists of a sequence of N words such as {w ₁ , w ₂ , ..., w _N }. The congestion degree of S is calculated as follows.

For a given LeafCat C, M sentences (from the listing title) can exist as a tuning set. These may be denoted as S ₁ , S ₂ , ..., S _M. For each of these headline sentences, its corresponding congestion can be calculated based on the above equation. The expected congestion value and the associated standard deviation value for a given LeafCat can then be obtained by the following equation (note that all the mean_PP and STD_PP values can be calculated in advance and stored for runtime use) .

12 is a diagram illustrating an SLM training step 1110B in accordance with an exemplary embodiment. In an exemplary embodiment, SLM training step 1110B is part of SLM re-ranking service 1110. [ The SLM training step 1110B accesses a database 1202 that includes publication information, which may include information such as a listing title, a search query, a product name, and the like. Various searches can be performed on this database to identify information related to the specific LeafCat for which the SLM model is created.

Here, (1) the number of listings for LeafCat in the recent X period (e.g., 8 weeks) in operation 1204; (2) the product name of all publications in LeafCat in operation 1206; (3) a query performed on LeafCat of the recent X period in operation 1208; (4) In operation 1210, four searches of the listing title in the recent X period for LeafCat have been specified. The results of each of these searches are used in a different manner. With respect to the number of listings for LeafCat in the last X period accessed in operation 1204, this information is used to generate a log prior probability (LPP) for LeafCat in operation 1212. This process will be described in more detail below.

For product names of all publications in LeafCat accessed in operation 1206, this information is first normalized (e.g., correcting typo or alternative spellings) via text normalization on the corpus in operation 1214, And then used to configure the SLM 1216 for structural data corresponding to the structured data of the leaf category.

For a query performed on LeafCat in the last X period accessed in operation 1208, this information is first normalized (e. G., Typed or alternate spelling corrected) through text normalization on the corpus, And then used to construct the SLM 1220 for LeafCat.

The listing title for the most recent X period for LeafCat is accessed in operation 1210. This information first passes through a filter comprising filter A 1222 and filter B 1224. These filters 1222 and 1224 serve as the most relevant to the listing title. Here, for example, filter A 1222 identifies a listing that includes a merchant category selection that matches the parent CatReco for listing (based on the categorization algorithm). For example, filter B 1224 may classify the inappropriate classification score for each listing by comparing it with a threshold (e.g., 60 of 100, where the highest likelihood of unlisted listings is 300) Which is less likely to fail. In this regard, this process is somewhat recursive because an inappropriate classification score is derived using the runtime process of the SLM re-ranking service 1110 for the leaf category, which is trained at this stage shown in FIG. Text normalization for the corpus may then be performed at operation 1226 to normalize the text of the filtered result. The result of this normalization can be used in two ways. First, an SLM 1228 for each LeafCat title may be generated as part of the training set. Apart from this, the remaining results can be used in tuning sets.

The training SLM 1228 for each of the SLM 1216 (corresponding to structured data in the leaf category), SLM 1220 for each LeafCat, and LeafCat for structured data is then interpolated at operation 1230 To create a combined SLM 1232 for LeafCat.

On the tuning set side, the output of the text normalization for the corpus in combined SLM 1232 and operation 1226 for LeafCat is used in the PPL and PPL_Std evaluation for each listing of LeafCat in operation 1234, It is possible to generate the expected PPL and PPL_Std 1236 for the title. This process is repeated for each leaf category.

13 is a diagram illustrating a GBM training model process 1300 in accordance with an exemplary embodiment. In the off-line autonomous GBM training model process 1300, the set of bootstrap-labeled training data 1320 can be derived in an autonomous manner by checking how the CatRecos was selected and the associated improper classifying scores. Once the labeled training data 1320 is acquired, a GBM characteristic input file 1360 can be prepared based on the output from the SLM re-ranking service 1110 and the output from the basic SSE CatReco service 680. More specifically, SLM re-ranking service 1110 generates SLM congestion deviation signal 1330 for training data and SLM ranking score 1340 for training data and base SSE CatReco service 680 generates training data And generates an SSE similarity score (1350). The GBM training process can then be used to train the GBM model. In operation 1370, a GBM feature file is used for GBM training in operation 1370. [ The GBM training generates the GBM model 1380 by the metadata.

According to FIG. 13, labeled training data 1320 is obtained using operations 1302, 1304, 1306, and 1308. At act 1302, a listing title for the most recent X period for each LeafCat is accessed from database 1301. For example, a recent X period may be referred to as the latest eight weeks in an exemplary embodiment. The two layers of the filter are then applied to this information. At operation 1304, filter A uses the output of operation 1302 and maintains the listing with the seller's category selection that matches the upper selection according to the CatReco algorithm, and then sends the result to the next operation 1306 , Filter B of the next operation filters to keep only those listings that are below a second predetermined threshold (e. G., 35 out of 100, i. E., A low likelihood that the listing will be improperly categorized). A listing title that meets the requirements of these two layer filters A and B is labeled as not improperly classified in operation 1308. [

Congestion level deviation signal 1330 and SLM ranking score 1340 may be derived from SLM re-ranking service 1110 for each portion of labeled training data 1320, as described above. SSE similarity score 1350 may also be derived from the base CatReco service 680 for each portion of the labeled training data 1320.

Modules, components, and logic

Any embodiment is described herein as including logic or a plurality of components, modules, or mechanisms. A module may comprise a software module (e.g., code implemented on a machine readable medium) or a hardware module. A "hardware module" is a unit of a type capable of performing a particular operation and may be configured or arranged in a particular physical manner. In various exemplary embodiments, one or more hardware modules (e.g., a group of processors or processors) of one or more computer systems (e.g., standalone computer systems, client computer systems, or server computer systems) May be configured by software (e.g., an application or application portion) as a hardware module that operates to perform a particular operation as described herein.

In some embodiments, the hardware modules may be implemented mechanically, electronically, or any suitable combination thereof. For example, a hardware module may include dedicated circuitry or logic configured to perform a particular operation. For example, the hardware module may be a special purpose processor such as an FPGA (Field-Programmable Gate Array) or an ASIC (Application Specific Integrated Circuit). The hardware module may also include programmable logic or circuitry that is temporarily configured by software to perform a particular operation. For example, the hardware module may comprise software executed by a general purpose processor or other programmable processor. Once configured by such software, the hardware module becomes a specific machine (or a specific component of the machine) that is uniquely tailored to perform the configured function, and is no longer a general purpose processor. It can be seen that the decision to implement a hardware module in a mechanically, duly and permanently configured circuit or a temporarily configured circuit (e.g., configured by software) can be realized in terms of cost and time.

Thus, the phrase "hardware module" is intended to encompass all types of hardware components, including, but not limited to, physically configured, permanently configured (e.g., hardwired) or temporarily configured (e.g., Quot; is an entity of a type. As used herein, "hardware implementation module" refers to a hardware module. Considering an embodiment in which a hardware module is temporarily configured (e.g., programmed), each of the hardware modules need not be configured or illustrated with respect to time at any one time. For example, if a hardware module includes a general purpose processor configured by software to be a special purpose processor, the general purpose processor may be configured with a different special purpose processor (e.g., including different hardware modules) at different times have. Thus, the software configures a particular processor or processor, e.g., to configure a particular hardware module at a time and to configure different hardware modules at different points in time.

A hardware module may provide information to other hardware modules and may receive information from other hardware modules. Thus, the described hardware modules may be considered to be communicatively coupled. If multiple hardware modules are present at the same time, communication may be accomplished through signal transmission between two or more hardware modules (e.g., via appropriate circuitry and buses). In embodiments where multiple hardware modules are configured or instantiated at different times, communication between such hardware modules may be accomplished through, for example, storage and retrieval of information in a memory structure in which multiple hardware modules are accessed. For example, one hardware module may perform an operation and store the output of the operation in a communicatively coupled memory device. Thereafter, the additional hardware module may access the memory device to acquire and process the stored output. The hardware module may also initiate communication with the input device or output device and operate on the resource (e.g., information collection).

The various operations of the exemplary methods described herein may be performed, at least in part, by one or more processors that are configured temporarily or permanently (e.g., by software) to perform the associated operations. Even if configured temporarily or permanently, such a processor may constitute a processor implementation module that operates to perform one or more of the operations or functions described herein. As used herein, "processor implementation module" refers to a hardware module implemented using one or more processors.

Likewise, the methods described herein may be implemented, at least in part, in a processor, where a particular processor or processor is an example of hardware. For example, the operation of at least some of the methods may be performed by one or more processors or processor implementation modules. In addition, the one or more processors may also be operable to support performance of associated operations in a " cloud computing "environment or as" software as a service " (SaaS). For example, at least some of the operations may be performed in a manner such that the operations are accessible through a network (e.g., the Internet) and one or more appropriate interfaces (e.g., Application Program Interface (API) Can be performed by a group of computers.

The performance of a particular operation may not only be within a single machine, but may also be distributed among processors that are located across multiple machines. In some exemplary embodiments, the processor or processor implementation module may be located in a single geographic location (e.g., in a home environment, an office environment, or a server farm). In other exemplary embodiments, the processor or processor implementation module may be distributed across multiple geographic locations.

Machine and software architecture

The modules, methods, applications, etc., described with reference to Figs. 1-6, are implemented in some embodiments in the context of a machine and associated software architecture. The following sections describe exemplary software structures and machine (e.g., hardware) structures suitable for use in the disclosed embodiments.

A software architecture is used to create machines and devices that are tailored to a particular purpose in relation to the hardware architecture. For example, a particular hardware architecture combined with a particular software architecture would make mobile devices such as mobile phones, tablet devices, and the like. A slightly different hardware and software architecture can yield smart devices to be used in the "internet of things ". Another combination produces a server computer for use within a cloud computing framework. Those skilled in the art will readily understand how to implement the invention in different contexts from the teachings contained herein, and not all combinations of such software and hardware structures are provided herein.

Software architecture

FIG. 14 shows a block diagram 1400 illustrating an exemplary software architecture 1402 that may be used with the various hardware architectures described herein. 14 is only a non-limiting example of a software architecture, and it will be appreciated that many other architectures may be implemented to facilitate the functionality described herein. Software structure 1402 may be executed on hardware such as machine 1500, among others, including processor 1510, memory 1530 and I / O component 1550 in FIG. A representative hardware layer 1404 is shown and may represent, for example, the machine 1500 of FIG. Exemplary hardware layer 1404 includes one or more processors 1406 having associated executable instructions 1408. [ Executable instructions 1408 represent executable instructions of the software architecture 1402, including implementations of the methods, modules, etc. of FIGS. 1-13. The hardware layer 1404 also includes a memory or storage module 1410, which also includes executable instructions 1408. The hardware layer 1404 may also include other hardware, such as 1412, which may represent any other hardware in the hardware layer 1404, such as other hardware depicted as part of the machine 1500 have.

In the exemplary structure of FIG. 14, the software 1402 may be conceptualized as a stack of layers where each layer provides a specific function. For example, the software 1402 may include layers such as an operating system 1414, a library 1416, a framework / middleware 1418, an application 1420 and a presentation layer 1444. The application 1420 or other component in the hierarchy may invoke an API call 1424 through the software stack and return a response, return value, etc. (in response to the API call 1424) Can be received. The described hierarchy is inherently representative and not all software structures have all hierarchies. For example, some mobile or special purpose operating systems may not provide the framework / middleware layer 1418, but other mobile or special purpose operating systems may provide such a layer. Other software architectures may include additional layers or other layers.

The operating system 1414 can manage hardware resources and provide a common service. Operating system 1414 may include, for example, kernel 1428, service 1430, and driver 1432. The kernel 1428 may serve as an abstraction layer between the hardware layer and the software layer. For example, the kernel 1428 may be responsible for memory management, processor management (e.g., scheduling), component management, networking, security settings, and the like. Service 1430 may provide other common services for other software layers. The driver 1432 may be responsible for controlling or interfacing the underlying hardware. For example, driver 1432 is a display driver, a camera driver, Bluetooth ^® drivers, flash memory driver, a serial communication driver, according to the hardware configuration (such as a Universal Serial Bus (g., USB) driver), WI-Fi ^® drivers, Audio drivers, power management drivers, and the like.

Library 1416 may provide a common infrastructure that may be utilized by application 1420 and / or other components and / or layers. Library 1416 typically performs operations in a manner that is easier than other software modules that interface directly with the functionality of base operating system 1414 (e.g., kernel 1428, service 1430, or driver 1432) It provides the ability to Library 1416 may include a system 1434 library (e.g., a C standard library) that may provide functions such as memory allocation, string manipulation, mathematical functions, and the like. The library 1416 may also include a media library (e.g., a library that supports the presentation and manipulation of various media formats, such as MPREG4, H.264, MP3, AAC, AMR, JPG, , An OpenGL framework that can be used to render 2D and 3D in graphics content on the display), a database library (e.g., SQLite that can provide a variety of relational database capabilities), a web library (e.g., And an API library 1436, such as WebKit, The library 1416 may also include various other libraries 1438 to provide many other APIs for the application 1420 and other software components / modules.

Framework 1418 (sometimes referred to as middleware) may provide a higher layer common infrastructure that may be utilized by application 1420 or other software components / modules. For example, the framework 1418 may provide various graphical user interface (GUI) functions, senior resource management, senior location services, and the like. Framework 1418 may provide a wide range of other APIs that may be utilized by application 1420 and / or other software components / modules, some of which may be specific to a particular operating system or platform .

Applications 1420 include built-in applications 1440 and / or third party applications 1442. Exemplary built-in applications 1440 include, but are not limited to, contacts applications, browser applications, book reader applications, location applications, media applications, messaging applications, and / It is not. Third party application 1442 may include any built-in application as well as a wide variety of other applications. As a specific example, a third party application 1442 (e.g., an application developed using an Android trademark or an iOS (trademark) software development kit (SDK) by an entity other than a particular platform vendor) In this example, the third party application 1442 may be the mobile software that is operated by an operating system such as iOS (trademark), Android (trademark), Windows® Phone or other mobile operating system. API call 1424 provided by a mobile operating system, such as operating system 1414,

The application 1420 may include a set of operating system functions such as built-in operating system functionality (e.g., kernel 1428, service 1430 and / or driver 1432), library (e.g., system 1434, API 1436, (E.g., application 1438), and / or framework / middleware 1418 to interact with a user of the system. Alternatively, or in addition, in some systems, interaction with a user may occur through a presentation layer, such as presentation layer 1444. In these systems, the application / module "logic" can be separated in terms of the application / module interacting with the user.

Some software architectures use virtual machines. In the example of FIG. 14, this is shown as a virtual machine 1448. The virtual machine creates a software environment in which the application / module can perform as if it were running on a hardware machine (e.g., the machine of FIG. 15). The virtual machine is hosted by a host operating system (operating system 1414 in FIG. 15), and typically, but not always, has a virtual machine monitor 1446, which not only monitors the operation of the virtual machine, (I.e., operating system 1414). The software architecture is implemented in a virtual machine such as operating system 1450, library 1452, framework / middleware 1454, application 1456 and / or presentation layer 1458. The layers of these software structures implemented in the virtual machine 1448 may be the same as or different from the layers described above.

Exemplary machine structures and machine readable media

FIG. 15 is a block diagram of a component of a machine 1500 according to some illustrative embodiments in which any one or more of the methods described herein may be read from and read from a machine-readable medium (e.g., a machine-readable storage medium) Fig. 15 illustrates a schematic diagram of a machine 1500 of an exemplary form of a computer system and includes instructions 1516 (e.g., software, programs, applications, applets, Code) enables the machine to execute any one or more of the methods described herein. For example, the instruction may cause the machine to execute the flowchart of FIG. Additionally or alternatively, the instructions may implement Figures 5A-13, etc. The instructions translate a general unprogrammed machine into a programmed specific machine so as to perform the functions described and described in the manner described. In an alternative embodiment, the machine 1500 may operate as a stand-alone device or may be coupled (e.g., networked) to another machine. In a network deployment, the machine 1500 may operate as a server machine or client machine in a server-client network environment, or may operate as a peer machine in a peer-to-peer (or distributed) network environment. The machine 1500 may be a server computer, a client computer, a personal computer, a tablet computer, a laptop computer, a netbook, a set top box (STB), an entertainment media system, A mobile device, a wearable device (e.g., a smart watch), a smart home device (e.g., a smart appliance), other smart devices, a web appliance, a network But are not limited to, routers, network switches, network bridges, or any machine capable of executing instructions 1516, sequentially or otherwise. Also, while only one machine 1500 is shown, the term "machine" refers to a machine 1500 that executes instructions 1516 individually or collectively to perform any one or more of the methods described herein. Should also be included.

The machine 1500 may include a processor 1510, a memory 1530 and an I / O component 1550, which may be configured to communicate with one another via a bus 1502. In an exemplary embodiment, a processor 1510 (e.g., a Central Processing Unit (CPU), a Reduced Instruction Set Computing (RISC) processor, a Complex Instruction Set Computing (CISC) processor, a Graphics Processing Unit (GPU) (E.g., a signal processor, an ASIC, a Radio-Frequency Integrated Circuit (RFIC), another processor, or any suitable combination thereof) includes a processor 1512 and a processor 1514, can do. The term "processor" is intended to include a multi-core processor that may include two or more independent processors (sometimes referred to as "cores ") capable of executing instructions concurrently. Although FIG. 15 illustrates multiple processors, machine 1500 may be implemented as a single processor with a single core, a single processor (e.g., multicore process) with multiple cores, multiple processors with a single core, Multiple processors, or any combination thereof. Memory / storage 1530 may include memory 1532 and storage 1536 such as main memory or other memory storage accessible to processor 1510 via bus 1502. [ Storage 1536 and memory 1532 store instructions 1516 that implement any one or more of the methods or functions described herein. The instructions 1516 may also be stored in memory 1532, in storage 1536, in at least one of the processors 1510 (e.g., in a cache memory of the processor), during execution by the machine 1500, Or may be completely or partially within any suitable combination of these. Thus, the memory 1532, the storage 1536, and the memory of the processor 1510 are examples of machine-readable media.

"Machine-readable medium" as used herein refers to a device capable of temporarily or permanently storing instructions and data, and includes a random-access memory (RAM), a read-only memory (ROM) But are not limited to, flash memory, optical media, magnetic media, cache memories, other types of storage devices (e.g., erasable programmable read-only memory (EEPROM) It is not. The term "machine-readable medium" should include a single medium or medium (e.g., a centralized or distributed database, or associated cache and server) capable of storing instructions 1516. The term "machine-readable medium" also refers to any medium or combination of media capable of storing instructions (e.g., instructions 1516) for execution by a machine (e.g., machine 1500) When executed by one or more processors (e.g., processor 1510) of machine 1500, the instructions may cause the machine 1500 to perform any one or more of the methods described herein do.

Thus, "machine readable medium" refers not only to a single storage device or device, but also to a "cloud-based" storage system or storage network that includes multiple storage devices or devices.

The I / O component 1550 can include a wide variety of components for input reception, output provisioning, output generation, information transmission, information exchange, measurement capture, and the like. The particular I / O component 1550 included in a particular machine will depend on the type of machine. For example, a portable machine such as a mobile phone would include a touch input device or other input mechanism, but a headless server machine would not include such a touch input device. It will be appreciated that I / O component 1550 may include many other components not shown in FIG. I / O components 1550 are grouped according to function merely to simplify the following description, and grouping is by no means limiting. In various exemplary embodiments, the I / O component 1550 may include an output component 1552 and an input component 1554. [ The output component 1552 may be a visual component such as a plasma display panel (PDP), a light emitting diode (LED) display, a liquid crystal display (LCD), a projector or a cathode ray tube ), Acoustic components (e.g., speakers), haptic components (e.g., vibration motors, resistance mechanisms), other signal generators, and the like. Input component 1554 may include an alphanumeric input component (e.g., a keyboard, a touch screen configured to receive alphanumeric input, a photo-optical keyboard, or other alphanumeric input component), a point-based input component (E.g., physical buttons, locations and / or taps of a touch and / or touch, such as a mouse, trackball, joystick, motion sensor, or other pointing instrument, Or other touch input components), audio input components (e.g., microphones), and the like.

In another exemplary embodiment, the I / O component 1550 includes a biometric component 1556, a motion component 1558, an environmental component 1560, or a location component 1560, among a number of different components 1562). For example, the biometric component 1556 may be a component that senses an expression (e.g., a hand expression, a facial expression, a voice expression, a gesture or an eye track), a biometric signal (e.g., blood pressure, heart rate, body temperature, Sweat or brain waves), components that identify a person (e.g., voice identification, retinal identification, facial identification, fingerprint identification, or EEG based identification), and the like. The motion component 1558 may include an acceleration sensor component (e.g., an accelerometer), a gravity sensor component, a rotation sensor component (e.g., a gyroscope), and the like. The environmental component 1560 may include, for example, a light sensor component (e.g., a photometer), a temperature sensor component (e.g., one or more thermometers that detect ambient temperature), a humidity sensor component, ), Acoustic sensor components (e.g., one or more microphones that sense ambient noise), proximate angular sensor components (e.g., infrared sensors that detect nearby objects), gas sensors (e.g., Or a gas detection sensor that measures air pollutants), or other components that can provide indications, measurements, or signals corresponding to the surrounding physical environment. The location component 1562 may include a location sensor component (e.g., a GPS receiver component), a height sensor component (e.g., an altimeter or barometer that detects the atmospheric pressure from which altitude may be derived) (E. G., A magnetometer), and the like.

Communication can be implemented using a wide variety of technologies. The I / O component 1550 includes a communication component 1564 that is operable to couple the machine 1500 to the network 104 or the device 1570 via a coupling 1582 and a coupling 1572, respectively. can do. For example, the communication component 1564 may include a network interface component or other suitable device for interfacing with the network 104. As another example, communication component 1564 is a wired communication component, a wireless communications component, a cellular communication component, NFC; g. Component (for example, (Near Field Communication near field communication) component, Bluetooth ^®, Bluetooth Low Energy ( may include a Bluetooth ^® Low Energy), WiFi ^® components and different ways to different communication components to provide communications over. device 1570 is an example, a USB other machines, or any of a wide variety of peripherals (e.g., Lt; / RTI &gt;

The communication component 1564 also includes a component that is capable of detecting the identifier or is operable to detect the identifier. For example, communication component 1564 may include a Radio Frequency Identification (RFID) tag reader component, an NFC smart tag detection component, an optical reader component (e.g., UPC (Universal Product Code) Aztec code, a data matrix, a data matrix, a maxi code, a PDF417, an Ultra code, and the like, such as a one-dimensional bar code and a QR (Quick Response) , Optical sensors that detect UCC RSS-2D bar codes and other optical codes), or acoustic detection components (e.g., a microphone that identifies the tagged audio signal). In addition, various information such as a position by IP (Internet Protocol) spatial position, a position by Wi-Fi (trademark) signal triangulation, a position by detection of NFC beacon signal capable of indicating a specific position, ). &Lt; / RTI &gt;

Transmission medium

In various exemplary embodiments, one or more portions of the network 104 may be part of an ad hoc network, an intranet, an extranet, a VPN, a LAN, a WLAN, a WAN, a WWAN, a MAN, telephone service, existing telephone service), a cellular telephone network, a wireless network, a Wi-Fi network, another type of network, or a combination of two or more of these networks. For example, the network 104 or a portion of the network 104 may include a wireless or cellular network and the coupling 1582 may include a Code Division Multiple Access (CDMA) 0.0 &gt; (GSM) &lt; / RTI &gt; connection or other type of cellular or wireless coupling. In this example, the coupling 1582 may be a Single Carrier Radio Transmission Technology (lxRTT), Evolution-Data Optimized (EVDO) technology, General Packet Radio Service (GPRS) Technology, Enhanced Data Rates for GSM Evolution (EDGE), Third Generation Partnership Project (3GPP) including 3G, Fourth Generation Wireless (4G) network, Universal Mobile (EDGE) technology, including world interoperability for microwave access, 3GPP (3GPP) technology, 3GPP, 3GPP, 3GPP, (WiMAX), Long Term Evolution (LTE) standards, other standards defined by various standards-setting organizations, other long distance protocols, or other data transmission technologies Significance can be implemented in various types of data transmission technology.

Instructions 1516 may be transmitted using a transmission medium through a network interface device (e.g., a network interface component included in communication component 1564) and using a number of known transmission protocols (e.g., The command 1516 may be transmitted or received over the network 104 using any one of the hypertext transfer protocol (HTTP) The term "transmission medium" is used herein to refer to any medium that can store, encode, or otherwise convey instructions 1516 for execution by machine 1500. The term &quot; transmission medium &quot; Digital or analog communications signals or other intangible media that facilitate the communication of such software. Is an example of a possible medium.

Example method

16 illustrates an exemplary method 1600 for identifying related categories of publications. The method 1600 includes an operation 1610 of accessing a request to add a publication to a publication corpus, an operation 1620 of identifying an associated set of categories of a publication, and an operation 1630 of displaying an associated set of categories of a publication ).

At operation 1610, one or more processors access the request from the user device to add the publication to the publication corpus and to identify the relevant set of categories of the publication. For example, in FIG. 2, one or more processors in the server of the listing system 150 access requests from the user device 204. Figure 3B is an example of a publication added by request from a user device.

In operation 1620, a publication semantic vector-publication semantic vector corresponding to (i) at least a portion of a publication is generated by one or more processors in a first machine learning model that projects at least a portion of the publication into a semantic vector space , And (ii) comparing the plurality of category vectors corresponding to each category from the plurality of categories to identify one or more closest matches, wherein the plurality of category vectors are generated by projecting the plurality of categories into a semantic vector space Based on the second machine learning model, a plurality of categories is a classification system of a publication in a publication corpus. Fig. 4 is an example of identifying the closest match.

At operation 1630, one or more nearest matches are displayed at the user device as a related set of categories of publication corpus. For example, in FIG. 2, one or more processors in the server of the listing system 150 are displayed on the user device 204. Fig. 3A shows an example of the closest match.

language

In this specification, a plurality of instances may implement a component, an operation, or a structure described in a single instance. Although separate operations of one or more methods are shown and described as separate operations, one or more separate operations may be performed simultaneously, and operations need not be performed in the order shown. The constitution and function represented as independent components in the exemplary configuration example can be implemented as a combined constitution or component. Likewise, configurations and functions represented as single components may be implemented as discrete components. These and other variations, modifications, additions, and improvements fall within the scope of the subject matter herein.

Although an overview of the subject matter of the present invention has been described with reference to specific exemplary embodiments, various modifications and changes may be made to these embodiments without departing from the broader scope of the embodiments of the present invention. Embodiments of the subject matter of the present invention are not intended to spontaneously limit the scope of the present application to any single disclosure or inventive concept and may be referred to individually or collectively by the term "invention" And is virtually disclosed.

The embodiments described herein have been described in sufficient detail to enable those skilled in the art to practice the disclosed teachings. Other embodiments may be used and derived therefrom so that structural and logical substitutions and changes may be made without departing from the scope of the present application. The detailed description is, therefore, not to be taken in a limiting sense, and the scope of various embodiments is defined only by the appended claims and the full scope of equivalents to which such claims are entitled.

As used herein, the term "or" may be interpreted in a generic or exclusive sense. Also, a plurality of instances may be provided for a resource, operation, or configuration described herein as a single instance.

In addition, the boundaries between various resources, operations, modules, engines, and data stores are somewhat arbitrary and specific operations are described in connection with specific exemplary configurations. Different distributions of functionality are planned and may be within the scope of various embodiments of the present application. In general, the configuration and functions shown as individual resources in the example configuration examples may be implemented as a combined configuration or resource. Similarly, the configuration and function represented as a single resource may be implemented as separate resources. These and other variations, modifications, additions and improvements are within the scope of the embodiments of the present application, which are indicated by the appended claims. Accordingly, the specification and drawings are to be regarded in an illustrative rather than a restrictive sense.

The numbered examples below represent embodiments.

1. A method comprising: (1) accessing a request from a user device by one or more processors, adding a publication to a publication corpus and identifying a set of related categories of publications; A publication semantic vector corresponding to the portion, the publication semantic vector being based on a first machine learning model that projects at least a portion of the publication into a semantic vector space; and (ii) Identifying one or more closest matches between a corresponding plurality of category vectors, wherein the plurality of category vectors are based on a second machine learning model that projects the plurality of categories into the semantic vector space, Category is the classification system of the publication in the publication corpus; And displaying the one or more nearest matches on the user device as a set of related categories of the publication corpus.

2. The method of embodiment 1, wherein said category is a leaf category.

3. The method of embodiment 1 or 2, wherein the category is at least two tree level category paths below the root level in the category tree of the plurality of categories.

4. The method as in any one of the examples 1 to 3, wherein at least a portion of the publication includes the title of the publication.

5. The method of embodiment 1, wherein at least one of the first machine learning model and the second machine learning model is trained for data automatically derived from a previously added publication of the publication corpus.

6. The method of any one of the examples 1 to 5, wherein at least one of the first machine learning model and the second machine learning model is trained at one or more lower word levels and lower character levels, .

7. The method of any one of Examples 1 to 6, further comprising adding the new category to the plurality of categories without re-training the second machine learning model for a new category, The one or more closest matches identified as matches comprise the new category.

8. A computer, comprising: a storage device in which instructions are stored; And one or more hardware processors configured to perform operations by the instructions, the operations comprising: accessing a request from a user device by one or more processors to add a publication to a publication corpus, &Lt; / RTI &gt; (I) a publication semantic vector corresponding to at least a portion of a publication, the publication semantic vector being based on a first machine learning model that projects at least a portion of the publication into a semantic vector space; and (ii) identifying one or more closest matches between a plurality of category vectors corresponding to respective categories from a plurality of categories, the plurality of category vectors being a plurality of categories, 2 machine learning model, the plurality of categories being a classification scheme of the publication in the publication corpus; And causing the one or more closest matches to be displayed on the user device as a set of related categories of the publication corpus.

9. The computer of example 8, wherein said category is a leaf category.

10. The computer of example 8 or 9, wherein the category is at least two tree level category paths below the root level in the category tree of the plurality of categories.

11. The computer as in any one of embodiments 8-10, wherein at least a portion of the publication includes a title of the publication.

12. The computer of any one of embodiments 8-11, wherein at least one of the first machine learning model and the second machine learning model is adapted to perform training on data automatically derived from a previously added publication of the publication corpus, do.

13. The computer of any one of the examples 8-12, wherein at least one of the first machine learning model and the second machine learning model is trained in one or more of a lower word level and a lower character level, Reduce the term.

14. The computer of example 8, wherein the operation further comprises adding the new category to the plurality of categories without re-training the second machine learning model for a new category, The one or more closest matches comprise the new category.

15. A hardware machine readable device having stored thereon instructions for causing the machine to perform operations upon execution by one or more processors of the machine, the operations comprising: accessing, by one or more processors, , Adding a publication to a publication corpus and identifying a set of related categories of publication; (I) a publication semantic vector corresponding to at least a portion of a publication, the publication semantic vector being based on a first machine learning model that projects at least a portion of the publication into a semantic vector space; (ii) identifying one or more closest matches between a plurality of category vectors corresponding to respective categories from a plurality of categories, the plurality of category vectors being a second one of projecting the plurality of categories into the semantic vector space Based on a machine learning model, said plurality of categories being a classification scheme of said publication in said publication corpus; And causing the one or more closest matches to be displayed on the user device as a set of related categories of the publication corpus.

16. The computer of embodiment 15 wherein said category is a leaf category.

17. The computer of example 15 or 16, wherein the category is at least two tree level category paths below the root level in the category tree of the plurality of categories.

18. The computer as in any of the embodiments 15-17, wherein at least a portion of the publication includes a title of the publication.

19. The computer as in any of the embodiments 15-18, wherein at least one of the first machine learning model and the second machine learning model is adapted to perform training on data automatically derived from a publication previously added to the publication corpus do.

20. The computer as in any one of Examples 15 to 19, wherein at least one of the first machine learning model and the second machine learning model is trained at at least one lower word level and a lower character level, Reduce the term.

21. A machine-readable medium that when executed by one or more processors of a machine conveys machine-readable instructions that cause the machine to perform the method of any one of examples 1-7.

Claims (21)

As a method,
Accessing a request from a user device by one or more processors to add a publication to a publication corpus and to identify a set of related categories of the publication;
(I) a publication semantic vector corresponding to at least a portion of the publication, the publication semantic vector comprising at least one of a first machine learning that projects at least a portion of the publication into a semantic vector space, (Ii) comparing a plurality of category vectors corresponding to each category from a plurality of categories to identify one or more closest matches, wherein the plurality of category vectors are associated with the plurality of categories Based on a second machine learning model for projecting into the semantic vector space, the plurality of categories being the classification scheme of the publication in the publication corpus; And
Causing the one or more closest matches to be displayed on the user device as a set of related categories of the publication corpus
Containing
Way.

The method according to claim 1,
Categories are leaf categories.
Way.
The method according to claim 1,
Wherein the category is at least two tree level category paths below the root level in the category tree of the plurality of categories
Way.
The method according to claim 1,
Wherein at least a portion of the publication includes a title of the publication
Way.
The method according to claim 1,
Wherein at least one of the first machine learning model and the second machine learning model is trained for data automatically derived from a previously added publication of the publication corpus
Way.
The method according to claim 1,
At least one of the first machine learning model and the second machine learning model is trained at one or more lower word levels and lower character levels to reduce out-of-vocabulary terms at runtime
Way.
The method according to claim 1,
Further comprising adding the new category to the plurality of categories without re-training the second machine learning model for the new category,
Wherein the one or more closest matches identified as one or more closest matches comprise the new category
Way.
A storage device storing an instruction; And
One or more hardware processors configured to perform operations by the instructions;
/ RTI &gt;
The operation includes:
Accessing a request from a user device by the one or more processors to add a publication to a publication corpus and identify a set of related categories of publication;
(I) a publication semantic vector corresponding to at least a portion of the publication, the publication semantic vector being based on a first machine learning model that projects at least a portion of the publication into a semantic vector space And (ii) comparing a plurality of category vectors corresponding to each category from a plurality of categories to identify one or more closest matches, wherein the plurality of category vectors are generated by comparing the plurality of categories with the semantic vector Based on a second machine learning model projecting into a space, the plurality of categories being the classification scheme of the publication in the publication corpus; And
Causing the one or more closest matches to be displayed on the user device as a set of related categories of the publication corpus
Containing
computer.
9. The method of claim 8,
The category is the leaf category
computer.
9. The method of claim 8,
Wherein the category is at least two tree level category paths below the root level in the category tree of the plurality of categories
computer.
9. The method of claim 8,
Wherein at least a portion of the publication includes a title of the publication
computer.
9. The method of claim 8,
Wherein at least one of the first machine learning model and the second machine learning model is trained for data automatically derived from a previously added publication of the publication corpus
computer.
9. The method of claim 8,
Wherein at least one of the first machine learning model and the second machine learning model is trained at one or more lower word level and lower character level to reduce terms other than vocabulary at runtime
computer.
9. The method of claim 8,
The operation further comprises adding the new category to the plurality of categories without re-training the second machine learning model for the new category,
Wherein the one or more closest matches identified as one or more closest matches comprise the new category
computer.

A hardware machine readable apparatus having stored thereon instructions for causing the machine to perform an operation when executed by one or more processors of the machine,
The operation includes:
Accessing a request from a user device by one or more processors to add a publication to a publication corpus and identify a set of related categories of the publication;
(I) a publication semantic vector corresponding to at least a portion of the publication, the publication semantic vector being based on a first machine learning model that projects at least a portion of the publication into a semantic vector space (Ii) comparing a plurality of category vectors corresponding to each category from a plurality of categories to identify one or more nearest matches, wherein the plurality of category vectors are used to identify the plurality of categories as semantic Based on a second machine learning model projecting into a vector space, wherein the plurality of categories is the classification scheme of the publication in the publication corpus; And
And displaying the one or more closest matches on the user device as a set of related categories of the publication corpus
Hardware machine readable device.

16. The method of claim 15,
The category is the leaf category
computer.
16. The method of claim 15,
Wherein the category is at least two tree level category paths below the root level in the category tree of the plurality of categories
computer.
16. The method of claim 15,
Wherein at least a portion of the publication includes a title of the publication
computer.
16. The method of claim 15,
Wherein at least one of the first machine learning model and the second machine learning model is trained for data automatically derived from a previously added publication of the publication corpus
computer.
16. The method of claim 15,
Wherein at least one of the first machine learning model and the second machine learning model is trained at one or more lower word level and lower character level to reduce terms other than vocabulary at runtime
computer.
Readable instructions that when executed by one or more processors of a machine cause the machine to perform the method of any one of claims 1 to 7.

Images