US8359282B2 - Supervised semantic indexing and its extensions - Google Patents
Supervised semantic indexing and its extensions Download PDFInfo
- Publication number
- US8359282B2 US8359282B2 US12/562,840 US56284009A US8359282B2 US 8359282 B2 US8359282 B2 US 8359282B2 US 56284009 A US56284009 A US 56284009A US 8359282 B2 US8359282 B2 US 8359282B2
- Authority
- US
- United States
- Prior art keywords
- vector
- words
- weight
- grams
- query
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active, expires
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/31—Indexing; Data structures therefor; Storage structures
- G06F16/313—Selection or weighting of terms for indexing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/33—Querying
- G06F16/3331—Query processing
- G06F16/3332—Query translation
- G06F16/3334—Selection or weighting of terms from queries, including natural language queries
Definitions
- the present invention relates to informational retrieval technology and more particularly to systems and methods for text document matching and ranking.
- Ranking text documents given a text-based query is one of the key tasks in information retrieval.
- Classical vector space models use weighted word counts and cosine similarity. This type of model often performs remarkably well, but suffers from the fact that only exact matches of words between query and target texts contribute to the similarity score. It also lacks the ability to adapt to specific datasets since no learning is involved.
- Latent Semantic Indexing and related methods such as probabilistic Latent Semantic Indexing (pLSA), and Latent Dirichlet Allocation (LDA) choose a low dimensional feature representation of “latent concepts”, and hence words are no longer independent.
- pLSA probabilistic Latent Semantic Indexing
- LDA Latent Dirichlet Allocation
- a system and method for determining a similarity between a document and a query includes providing a frequently used dictionary and an infrequently used dictionary in storage memory. For each word or gram in the infrequently used dictionary, n words or grams are correlated from the frequently used dictionary based on a first score. Features for a vector of the infrequently used words or grams are replaced with features from a vector of the correlated words or grams from the frequently used dictionary when the features from a vector of the correlated words or grams meet a threshold value.
- a similarity score is determined between weight vectors of a query and one or more documents in a corpus by employing the features from the vector of the correlated words or grams that met the threshold value.
- a system for determining a similarity between a document and a query includes a memory configured to store a frequently used dictionary and an infrequently used dictionary.
- a processing device is configured to execute a program to correlate n words or grams from the frequently used dictionary based on a first score for each word or gram in the infrequently used dictionary.
- the program is further configured to replace features for a vector of the infrequently used words or grams with features from a vector of the correlated words or grams from the frequently used dictionary when the features from a vector of the correlated words or grams meet a threshold value.
- the processing device is further configured to determine a similarity score between weight vectors of a query and one or more documents in a corpus by employing the features from the vector of the correlated words or grams that met the threshold value.
- FIG. 1 is a block/flow diagram showing a system/method for determining similarity between a query and a document
- FIG. 2 is a block/flow diagram showing a system/method for simplifying dimensions when determining similarity between a query and a document;
- FIG. 3 is a block/flow diagram showing a system/method for correlated feature hashing (CFH) in accordance with one embodiment
- FIG. 4 is a block/flow diagram showing a system/method for determining similarity between a query and a document with a higher order polynomial model
- FIG. 5 is a block diagram showing a system for conducting searches and/or retrieving documents in accordance with one embodiment.
- FIG. 6 is a block/flow diagram showing a more detailed system/method for determining similarity between a query and a document in accordance with the present principles.
- a class of models is provided that are discriminatively trained to directly map from word content in a query-document or document-document pair to a ranking score.
- the present models account for correlations between words (e.g., synonymy, polysemy).
- the present models are trained with a supervised signal (for example, an expert's judgment on whether a document is relevant to a query, or links among documents) directly on the task of interest, which is one reason for superior results.
- a supervised signal for example, an expert's judgment on whether a document is relevant to a query, or links among documents
- the present approach can easily be generalized to other retrieval tasks as well, such as cross-language retrieval.
- An empirical study is provided on a retrieval task based on Wikipedia documents, where we obtain state-of-the-art performance using the present method.
- Embodiments described herein may be entirely hardware, entirely software or including both hardware and software elements.
- the present invention is implemented in software, which includes but is not limited to firmware, resident software, microcode, etc.
- Embodiments may include a computer program product accessible from a computer-usable or computer-readable medium providing program code for use by or in connection with a computer or any instruction execution system.
- a computer-usable or computer readable medium may include any apparatus that stores, communicates, propagates, or transports the program for use by or in connection with the instruction execution system, apparatus, or device.
- the medium can be magnetic, optical, electronic, electromagnetic, infrared, or semiconductor system (or apparatus or device) or a propagation medium.
- the medium may include a computer-readable medium such as a semiconductor or solid state memory, magnetic tape, a removable computer diskette, a random access memory (RAM), a read-only memory (ROM), a rigid magnetic disk and an optical disk, etc.
- q and d are feature vectors of query and document, respectively.
- Each feature in a feature vector could be an individual word, its weight can be, for instance, a TFIDF value.
- W is a weight matrix and T is the transpose function.
- the vectors q and d are sparse so the speed of computation of a single query-document pair involves mn computations of w ij q i d j , where q and d have m and n nonzero terms, respectively. We have found this is reasonable for training, but may be an issue at test time.
- one can compute v q T W once, and then compute vd for each document. This is the same speed as a classical vector space model where the query contains D terms, assuming W is dense.
- W I: if q and d are normalized TFIDF vectors, this is equivalent to using the standard cosine similarity with no learning (and no synonymy or polysemy).
- W D: where D is a diagonal matrix: one learns a re-weighting of TFIDF using labeled data.
- the first term q T W 1 q is useless since it is the same for all target documents. Although this term is not affected by the query at all, it corresponds to ranking a document's “importance” in a static way, which can still be useful. However, in the present case this static ranking is learned using a supervised signal.
- a function t which is a nonlinear neural network based on embedding words
- t ⁇ ( d i ) ⁇ j ⁇ g ⁇ ( w 1 ) j ⁇ g ⁇ ( w 2 ) j .
- n-grams This can be generalized to n-grams, and basically consists of only using some of the polynomial terms. Alternatively, one could hash n-grams.
- V d+ T V w >V d ⁇ T V w train word embedding with document embedding together, where w is in d+, but not in d ⁇ .
- V d+ T f(d+)>V d+ T f(d ⁇ ) train functional embedding with transductive embedding. Again, note V d T ⁇ f(d), instead, they have nonlinear relationship.
- Hashing n-grams One could also use this trick to, e.g., hash n-grams, which otherwise would have no way of fitting in memory. To use less features, one could hash a 3-gram by making it the sum of three 2-grams e.g. “the cat sat” ⁇ g(the,cat)+g(cat,sat)+g(the,sat), where g(•) is the embedding.
- Hashing with Prior Knowledge One does not have to use random hashing functions.
- FIG. 1 a block/flow diagram for determining similarities between a query and documents is illustratively shown in accordance with the present principles.
- a class of models can be trained on a supervised signal to provide rankings of a database of documents (d) 11 given a query (q) 12 .
- W weight matrix
- Block 11 represents a document d
- block 12 represents a query q.
- a TFIDF vector is built for document d
- a TFIDF vector is built for query q.
- blocks 15 and 16 TFIDF vectors Td and Tq are generated.
- This optimization problem is solved through stochastic gradient descent, iteratively, one picks a random tuple and makes a gradient step for that tuple: W ⁇ (qd + ⁇ qd ⁇ ), if 1 ⁇ qd + ⁇ qd ⁇ >0.
- the present method includes a margin ranking perceptron with a particular choice of features. It thus involves a convex optimization problem and is hence related to a ranking SVM, except we have a highly scalable optimizer.
- Training W We enforce sparsity through feature selection.
- a “projection” method can be seen as a generalization of the Recursive Feature Elimination (RFE) feature selection method. This method may include the following: 1) Train the model with a dense matrix W as before. 2) For each row i of W, find the k active elements with the smallest values of W ij . Constrain these elements to equal zero, i.e., make them inactive. 3) Train the model with the constrained W matrix. 4) If W contains more than p non-zero terms in each row go back to 2).
- RFE Recursive Feature Elimination
- feature selection may be applied to supervised semantic indexing (SSI).
- SSI supervised semantic indexing
- a classical vector space model (q′d) and optimal embedding (q′U′Vd) are combined together.
- the first term on the right-hand side (q′U′Vd) can be viewed as a dot product of q and d after transformation of U and V, respectively. This term contains synonym/polysemy information and thus allows fuzzy matching.
- Eq. (6) has fewer parameters to train than Eq. (2), and thus provides for faster training and is less likely to overfit. This can provide better performance.
- Block 31 includes frequent word storage.
- Block 32 includes infrequent word storage.
- Block 33 includes a method to compute correlated features.
- DICE(a,b) occur(a,b)/occur(a)+occur(b). Then, take the m words in (A) with the largest DICE score as the correlated features of a. Examples of correlated features are stored in block 34 .
- a first column includes infrequent words, the other columns are the top 5 words correlated with the words in the first column.
- blocks 37 , 36 , and 35 recomputed features using the correlated words are stored in block 34 .
- Infrequent words in TFIDF vectors in black 37 are replaced with frequent words, based on the correlated hashing features in block 36 .
- correlated feature hashing occurs. This includes, for each word count for a word in (B), replacing the word with its correlated features of weight at least 0.2. Note the weight threshold may be adjusted as needed.
- Word vectors are output in block 35 .
- correlated feature hashing may be introduced.
- we limit the features to the most frequent ones (e.g., the most frequent 30,000 words).
- Any other word, say w is “hashed” into the 5 words (or at least two) that co-occur with w most often.
- the co-occurrence can be measured by a “DICE” score in block 33 , which is the number of co-occurrences of two words divided by the sum of the occurrences of the two words.
- DICE is the number of co-occurrences of two words divided by the sum of the occurrences of the two words.
- N-grams can be hashed into most co-occurred word features.
- Correlated feature hashing can be viewed as a preprocessing stage (e.g., block 201 of FIG. 6 ). Rare words can be removed from word vectors by mapping them to frequent words. Then, the training can be conducted on the resulting vectors. Correlated feature hashing has more meaningful and effective feature reduction then prior art methods.
- Correlated feature hashing includes providing a frequently used dictionary 31 and an infrequently used dictionary 32 in storage memory. For each word or gram in the infrequently used dictionary 32 , correlate n words or grams from the frequently used dictionary 32 based on a first score (e.g., DICE score).
- a first score e.g., DICE score
- features are replaced for a vector of the infrequently used words or grams with features from a vector of the correlated words or grams from the frequently used dictionary 31 when the features from a vector of the correlated words or grams meet a threshold value (e.g., 0.2).
- a similarity score between weight vectors of a query and one or more documents is achieved by employing the features from the vector of the correlated words or grams that met the threshold value.
- a block/flow diagram illustratively depicts a high-order polynomial model in accordance with another embodiment.
- TFIDF vectors are mapped to multiple embedding spaces in block 43 by going through multiple function mappings in block 42 .
- a query TFIDF vector q in block 44 is transformed by function h(q) in block 45 .
- a similarity between q and d is calculated by Eq. 9 in block 46 .
- Eq. (8) provides significant improvements in document ranking.
- Eq. (9) can be viewed as a generalized dot product, which simplifies the superior document ranking capabilities of Eq. (8).
- Ranking is then typically achieved by measuring the distances in the embedding space (Euclidean distance or dot product similarity).
- This method provides a point-to-point correspondence between the input space and the intrinsic space in which the data lie.
- Such methods have multiplied in recent years and are most commonly rooted either in factor analysis (e.g., principal component analysis) or multidimensional scaling and their variants: e.g., kernel PCA, Isomap, Locally Linear Embedding and Laplacian Eigenmaps.
- factor analysis e.g., principal component analysis
- multidimensional scaling e.g., kernel PCA, Isomap, Locally Linear Embedding and Laplacian Eigenmaps.
- transductive learning gives outstanding performance for clustering applications.
- the drawback of this method is that, the query has to be one of the objects y i ⁇ Y for this approach to make sense as only these objects are embedded.
- Out-of-sample extension methods have to be adopted for new samples.
- f(q,d i ) M( ⁇ (q),v i ) (11)
- ⁇ (q) is the functional embedding for new queries
- v i is the nonlinear embedding to give better performance.
- the new framework for half-transductive learning is provided.
- Half of the formulation is functional which handles new sample embedding, while the other half is transductive which provides better performance for existing databases.
- f ⁇ ( q , d ) 1 m ⁇ ⁇ i ⁇ [ 1 , m ] ⁇ ( t ⁇ ( q i ) ) T ⁇ 1 n ⁇ ⁇ j ⁇ [ 1 , n ] ⁇ ( t ⁇ ( d j ) ) .
- a function which is a nonlinear neural network based on embedding words one could use a multi-layer network with sigmoid functions, or could apply a similar process as in the section on polynomials above (but instead to apply a polynomial map to n-grams, within a sliding window)).
- system 100 includes at least one terminal or processing device 104 having a user interface 102 .
- the user can present queries or perform documents searches through interface 102 .
- System 100 includes memory 112 , which may itself include documents or items that can be queried by the user.
- the memory 112 stores a program 114 (or programs), which implement the methods in accordance with the present principles.
- program 114 may learn to rank documents or items using word features.
- Such learning may include learning with a basic model in accordance with the present embodiments. This basic model may be employed in text-image ranking, or in text-text ranking.
- the program 114 may include methods for sparsification of a weight matrix or implement low ranking approximations as described hereinabove.
- Memory 112 and program 114 may be employed to perform correlated feature hashing (CFH).
- Other aspects of the present principles that may be implemented by program 114 may include polynomial modeling of word features, half-transductive modeling, sliding window mapping, and flexible dimensionality for word embedding. It should be understood that the many aspects described in accordance with the present principles can be employed together in various combinations. For example, half-transductive modeling may be employed with sliding window mapping, and correlated feature hashing (CFH) may be employed with any of the embodiments described herein.
- System 100 may interact with local servers 108 and databases 110 or remote servers 108 and databases 110 through a network 106 .
- These databases 110 and servers 108 may include the documents or items to be searched using a user or machine generated query.
- the program 114 may be remotely disposed from system 100 , e.g., located on a server or the like. Training of models may be performed on system 100 or may be performed on other platforms to create models 115 for implementing the present methods.
- a method for determining a similarity between a document and a query is illustratively shown.
- the method may be employed for document ranking, document retrieval or any other analysis that employs similarity scores between a query and documents.
- a preprocessing step may be employed for correlated feature hashing (CFH) as described in FIG. 3 .
- a weight vector for each of a plurality of documents in a corpus of documents is built and stored in memory.
- a weight vector is built for a query input into a document system.
- the weight vectors may include Term Frequency Inverse Document Frequency (TFIDF) weight vectors, although other feature vectors are also contemplated.
- TFIDF Term Frequency Inverse Document Frequency
- the weight vectors for the plurality of documents are mapped to multiple embedding spaces to handle models having polynomial functions of n-order in block 204 .
- functional embedding and transductance embedding are combined to transform weight vectors to model both new queries and improve performance in existing databases in block 205 .
- a sliding window may be employed while embedding words or grams to consider other words or grams in proximity thereof.
- building a weight vector for each of a plurality of documents includes performing a linear embedding to transform the weight vector to embedding space using a matrix U
- building a weight vector for a query includes performing a linear embedding to transform the weight vector to embedding space using a matrix V.
- a weight matrix (W) is generated which distinguishes between relevant documents and lower ranked documents by comparing document/query tuples using a gradient step approach. Other comparison techniques may also be employed.
- the weight matrix generation may include assigning infrequently occurring words to a reduced dimension representation; and mapping the reduced dimension representation to a full dimension representation when the infrequently occurring words are encountered.
- the weight matrix may be sparsified through feature selection in block 212 . Sparification may include finding active elements in the weight matrix with smallest weight values in block 213 by searching the matrix row by row. The active elements in the weight matrix are constrained with smallest weight values set to equal zero in block 214 .
- a model is trained with the weight matrix, and, in block 216 , if the weight matrix includes more than p non-zero terms, returning to the finding step (block 213 ). The result is a sparsely populated weight matrix to improve computation time and reduce memory and computational overhead.
- a similarity score is determined between weight vectors of the query and documents in a corpus by determining a product of a document weight vector, a query weight vector and the weight matrix.
- determining the similarity score includes taking a product between a transformed document weight vector (block 208 ) and a transformed query weight vector (block 209 ) to simplify computations.
- blocks 204 , 205 , 206 , 211 , 212 are all alternative options to decide similarity score f(q,d). Although these techniques can be combined, they may be employed independently or combined in any order.
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Software Systems (AREA)
- Artificial Intelligence (AREA)
- Computational Linguistics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
Description
where q and d are feature vectors of query and document, respectively. Each feature in a feature vector could be an individual word, its weight can be, for instance, a TFIDF value. W is a weight matrix and T is the transpose function. We can see that wij models correlation between the ith feature of query q and jth feature of d.
W=U T V+I
f(q,d)=q′Wd=Σ i,j w ij q′ i d j (1)
where L(•) is a function that computes the loss of the current model given a query q in Q and a labeling Y(q,d) of the relevance of the documents d with respect to q. However, in general any sparsity promoting regularizer or feature selection technique could be used.
f(q,d)=q′Wd=q′U′Vd+q′d (4)
f(q,d)=q′U′Vd (5).
U←U+λV(d + −d −)q′, if 1−f(q,d +)+f(q,d −)>0
V←V+λUq(d + −d −)′, if 1−f(q,d +)+f(q,d −)>0
f(q,d)=Σi,j,k w ijk q i d i d j d k (8)
f(q,d)=ΣjεD h(q), πm-1,k-1 g m(d) (9)
f(q,d i)=M(Wq,v i) (12).
One could choose a function which is a nonlinear neural network based on embedding words, one could use a multi-layer network with sigmoid functions, or could apply a similar process as in the section on polynomials above (but instead to apply a polynomial map to n-grams, within a sliding window)). For example, for 2-grams containing two words w1 and w2 one might want to consider the following mapping. If g(w1) is the embedding of the word with wi, one uses the mapping: t(di)=Σjg(w1)jg(w2)j: This can be generalized to n-grams. This basically includes using only some of the polynomial terms.
Claims (14)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US12/562,840 US8359282B2 (en) | 2009-01-12 | 2009-09-18 | Supervised semantic indexing and its extensions |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US14394209P | 2009-01-12 | 2009-01-12 | |
US12/562,840 US8359282B2 (en) | 2009-01-12 | 2009-09-18 | Supervised semantic indexing and its extensions |
Publications (2)
Publication Number | Publication Date |
---|---|
US20100185659A1 US20100185659A1 (en) | 2010-07-22 |
US8359282B2 true US8359282B2 (en) | 2013-01-22 |
Family
ID=42319750
Family Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US12/562,802 Active 2031-03-06 US8341095B2 (en) | 2009-01-12 | 2009-09-18 | Supervised semantic indexing and its extensions |
US12/562,840 Active 2031-06-23 US8359282B2 (en) | 2009-01-12 | 2009-09-18 | Supervised semantic indexing and its extensions |
Family Applications Before (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US12/562,802 Active 2031-03-06 US8341095B2 (en) | 2009-01-12 | 2009-09-18 | Supervised semantic indexing and its extensions |
Country Status (1)
Country | Link |
---|---|
US (2) | US8341095B2 (en) |
Cited By (12)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20150242387A1 (en) * | 2014-02-24 | 2015-08-27 | Nuance Communications, Inc. | Automated text annotation for construction of natural language understanding grammars |
US9477654B2 (en) | 2014-04-01 | 2016-10-25 | Microsoft Corporation | Convolutional latent semantic models and their applications |
US9519859B2 (en) | 2013-09-06 | 2016-12-13 | Microsoft Technology Licensing, Llc | Deep structured semantic model produced using click-through data |
US9535960B2 (en) | 2014-04-14 | 2017-01-03 | Microsoft Corporation | Context-sensitive search using a deep learning model |
US10089580B2 (en) | 2014-08-11 | 2018-10-02 | Microsoft Technology Licensing, Llc | Generating and using a knowledge-enhanced model |
US10860809B2 (en) | 2019-04-09 | 2020-12-08 | Sas Institute Inc. | Word embeddings and virtual terms |
US10909450B2 (en) | 2016-03-29 | 2021-02-02 | Microsoft Technology Licensing, Llc | Multiple-action computational model training and operation |
US11194878B2 (en) | 2018-12-13 | 2021-12-07 | Yandex Europe Ag | Method of and system for generating feature for ranking document |
US11562292B2 (en) | 2018-12-29 | 2023-01-24 | Yandex Europe Ag | Method of and system for generating training set for machine learning algorithm (MLA) |
US11645540B2 (en) | 2019-08-12 | 2023-05-09 | Nec Corporation | Deep graph de-noise by differentiable ranking |
US11681713B2 (en) | 2018-06-21 | 2023-06-20 | Yandex Europe Ag | Method of and system for ranking search results using machine learning algorithm |
US11928107B2 (en) | 2020-05-22 | 2024-03-12 | International Business Machines Corporation | Similarity-based value-to-column classification |
Families Citing this family (91)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US9195640B1 (en) * | 2009-01-12 | 2015-11-24 | Sri International | Method and system for finding content having a desired similarity |
JP5119288B2 (en) * | 2010-03-15 | 2013-01-16 | シャープ株式会社 | Portable terminal device, information output system, information output method, program, and recording medium |
US7933859B1 (en) | 2010-05-25 | 2011-04-26 | Recommind, Inc. | Systems and methods for predictive coding |
US9298818B1 (en) * | 2010-05-28 | 2016-03-29 | Sri International | Method and apparatus for performing semantic-based data analysis |
US10198431B2 (en) * | 2010-09-28 | 2019-02-05 | Siemens Corporation | Information relation generation |
US8819019B2 (en) * | 2010-11-18 | 2014-08-26 | Qualcomm Incorporated | Systems and methods for robust pattern classification |
US9424351B2 (en) | 2010-11-22 | 2016-08-23 | Microsoft Technology Licensing, Llc | Hybrid-distribution model for search engine indexes |
US8620907B2 (en) * | 2010-11-22 | 2013-12-31 | Microsoft Corporation | Matching funnel for large document index |
US9529908B2 (en) | 2010-11-22 | 2016-12-27 | Microsoft Technology Licensing, Llc | Tiering of posting lists in search engine index |
US8713024B2 (en) | 2010-11-22 | 2014-04-29 | Microsoft Corporation | Efficient forward ranking in a search engine |
US8478704B2 (en) | 2010-11-22 | 2013-07-02 | Microsoft Corporation | Decomposable ranking for efficient precomputing that selects preliminary ranking features comprising static ranking features and dynamic atom-isolated components |
US20120253792A1 (en) * | 2011-03-30 | 2012-10-04 | Nec Laboratories America, Inc. | Sentiment Classification Based on Supervised Latent N-Gram Analysis |
US8892488B2 (en) * | 2011-06-01 | 2014-11-18 | Nec Laboratories America, Inc. | Document classification with weighted supervised n-gram embedding |
US8515964B2 (en) * | 2011-07-25 | 2013-08-20 | Yahoo! Inc. | Method and system for fast similarity computation in high dimensional space |
CN102360372B (en) * | 2011-10-09 | 2013-01-30 | 北京航空航天大学 | Cross-language document similarity detection method |
US8799269B2 (en) * | 2012-01-03 | 2014-08-05 | International Business Machines Corporation | Optimizing map/reduce searches by using synthetic events |
JP2015004996A (en) * | 2012-02-14 | 2015-01-08 | インターナショナル・ビジネス・マシーンズ・コーポレーションInternational Business Machines Corporation | Apparatus for clustering plural documents |
CN104246757B (en) * | 2012-02-22 | 2018-11-06 | 诺基亚技术有限公司 | Predictive service access |
CN103309886B (en) * | 2012-03-13 | 2017-05-10 | 阿里巴巴集团控股有限公司 | Trading-platform-based structural information searching method and device |
US8903813B2 (en) | 2012-07-02 | 2014-12-02 | International Business Machines Corporation | Context-based electronic document search using a synthetic event |
US8898165B2 (en) | 2012-07-02 | 2014-11-25 | International Business Machines Corporation | Identification of null sets in a context-based electronic document search |
US9460200B2 (en) | 2012-07-02 | 2016-10-04 | International Business Machines Corporation | Activity recommendation based on a context-based electronic files search |
US9262499B2 (en) | 2012-08-08 | 2016-02-16 | International Business Machines Corporation | Context-based graphical database |
US8676857B1 (en) | 2012-08-23 | 2014-03-18 | International Business Machines Corporation | Context-based search for a data store related to a graph node |
US8959119B2 (en) | 2012-08-27 | 2015-02-17 | International Business Machines Corporation | Context-based graph-relational intersect derived database |
US9619580B2 (en) | 2012-09-11 | 2017-04-11 | International Business Machines Corporation | Generation of synthetic context objects |
US9251237B2 (en) | 2012-09-11 | 2016-02-02 | International Business Machines Corporation | User-specific synthetic context object matching |
US8620958B1 (en) | 2012-09-11 | 2013-12-31 | International Business Machines Corporation | Dimensionally constrained synthetic context objects database |
US9223846B2 (en) | 2012-09-18 | 2015-12-29 | International Business Machines Corporation | Context-based navigation through a database |
US8782777B2 (en) | 2012-09-27 | 2014-07-15 | International Business Machines Corporation | Use of synthetic context-based objects to secure data stores |
US9020954B2 (en) | 2012-09-28 | 2015-04-28 | International Business Machines Corporation | Ranking supervised hashing |
US8533148B1 (en) * | 2012-10-01 | 2013-09-10 | Recommind, Inc. | Document relevancy analysis within machine learning systems including determining closest cosine distances of training examples |
US9741138B2 (en) | 2012-10-10 | 2017-08-22 | International Business Machines Corporation | Node cluster relationships in a graph database |
US8931109B2 (en) | 2012-11-19 | 2015-01-06 | International Business Machines Corporation | Context-based security screening for accessing data |
US9286379B2 (en) * | 2012-11-26 | 2016-03-15 | Wal-Mart Stores, Inc. | Document quality measurement |
US9229932B2 (en) | 2013-01-02 | 2016-01-05 | International Business Machines Corporation | Conformed dimensional data gravity wells |
US8914413B2 (en) | 2013-01-02 | 2014-12-16 | International Business Machines Corporation | Context-based data gravity wells |
US8983981B2 (en) | 2013-01-02 | 2015-03-17 | International Business Machines Corporation | Conformed dimensional and context-based data gravity wells |
US8856946B2 (en) | 2013-01-31 | 2014-10-07 | International Business Machines Corporation | Security filter for context-based data gravity wells |
US9069752B2 (en) | 2013-01-31 | 2015-06-30 | International Business Machines Corporation | Measuring and displaying facets in context-based conformed dimensional data gravity wells |
US9053102B2 (en) | 2013-01-31 | 2015-06-09 | International Business Machines Corporation | Generation of synthetic context frameworks for dimensionally constrained hierarchical synthetic context-based objects |
US20140236577A1 (en) * | 2013-02-15 | 2014-08-21 | Nec Laboratories America, Inc. | Semantic Representations of Rare Words in a Neural Probabilistic Language Model |
US9110722B2 (en) | 2013-02-28 | 2015-08-18 | International Business Machines Corporation | Data processing work allocation |
US9292506B2 (en) | 2013-02-28 | 2016-03-22 | International Business Machines Corporation | Dynamic generation of demonstrative aids for a meeting |
US9600777B2 (en) * | 2013-03-11 | 2017-03-21 | Georges Harik | Configuring and optimizing computational structure for a machine learning application using a tuple of vectors |
US20140279748A1 (en) * | 2013-03-15 | 2014-09-18 | Georges Harik | Method and program structure for machine learning |
US10152526B2 (en) | 2013-04-11 | 2018-12-11 | International Business Machines Corporation | Generation of synthetic context objects using bounded context objects |
US9195608B2 (en) | 2013-05-17 | 2015-11-24 | International Business Machines Corporation | Stored data analysis |
US9348794B2 (en) | 2013-05-17 | 2016-05-24 | International Business Machines Corporation | Population of context-based data gravity wells |
US9262485B2 (en) * | 2013-08-13 | 2016-02-16 | International Business Machines Corporation | Identifying a sketching matrix used by a linear sketch |
JP6384065B2 (en) * | 2014-03-04 | 2018-09-05 | 日本電気株式会社 | Information processing apparatus, learning method, and program |
WO2015145981A1 (en) * | 2014-03-28 | 2015-10-01 | 日本電気株式会社 | Multilingual document-similarity-degree learning device, multilingual document-similarity-degree determination device, multilingual document-similarity-degree learning method, multilingual document-similarity-degree determination method, and storage medium |
US9330104B2 (en) | 2014-04-30 | 2016-05-03 | International Business Machines Corporation | Indexing and searching heterogenous data entities |
EP3171282A4 (en) * | 2014-11-19 | 2017-12-06 | Informex Inc. | Data retrieval apparatus, program and recording medium |
JP5916974B1 (en) | 2014-11-19 | 2016-05-11 | 株式会社インフォメックス | Data search device, program, and recording medium |
US10318885B2 (en) * | 2015-09-15 | 2019-06-11 | International Business Machines Corporation | Cognitive system virtual corpus training and utilization |
EP3188038B1 (en) * | 2015-12-31 | 2020-11-04 | Dassault Systèmes | Evaluation of a training set |
CN105677873B (en) * | 2016-01-11 | 2019-03-26 | 中国电子科技集团公司第十研究所 | Text Intelligence association cluster based on model of the domain knowledge collects processing method |
AU2017250467B2 (en) * | 2016-04-15 | 2019-12-19 | Solventum Intellectual Properties Company | Query optimizer for combined structured and unstructured data records |
CN106372187B (en) * | 2016-08-31 | 2019-12-17 | 中译语通科技股份有限公司 | Cross-language retrieval method for big data |
CN108269110B (en) * | 2016-12-30 | 2021-10-26 | 华为技术有限公司 | Community question and answer based item recommendation method and system and user equipment |
JP6835204B2 (en) | 2017-03-14 | 2021-02-24 | 日本電気株式会社 | Learning material recommendation method, learning material recommendation device and learning material recommendation program |
US10642875B2 (en) * | 2017-04-28 | 2020-05-05 | International Business Machines Corporation | Accurate relationship extraction with word embeddings using minimal training data |
US10963501B1 (en) * | 2017-04-29 | 2021-03-30 | Veritas Technologies Llc | Systems and methods for generating a topic tree for digital information |
CN107832319B (en) * | 2017-06-20 | 2021-09-17 | 北京工业大学 | Heuristic query expansion method based on semantic association network |
CN107291895B (en) * | 2017-06-21 | 2020-05-26 | 浙江大学 | Quick hierarchical document query method |
CA3116778C (en) * | 2018-07-18 | 2024-06-25 | The Dun & Bradstreet Corporation | Artificial intelligence engine for generating semantic directions for websites for automated entity targeting to mapped identities |
CN109165382B (en) * | 2018-08-03 | 2022-08-23 | 南京工业大学 | Similar defect report recommendation method combining weighted word vector and potential semantic analysis |
US11138278B2 (en) * | 2018-08-22 | 2021-10-05 | Gridspace Inc. | Method for querying long-form speech |
US10891321B2 (en) * | 2018-08-28 | 2021-01-12 | American Chemical Society | Systems and methods for performing a computer-implemented prior art search |
US11087184B2 (en) * | 2018-09-25 | 2021-08-10 | Nec Corporation | Network reparameterization for new class categorization |
CN110968664A (en) * | 2018-09-30 | 2020-04-07 | 北京国双科技有限公司 | Document retrieval method, device, equipment and medium |
US10331713B1 (en) | 2018-10-03 | 2019-06-25 | Gurbaksh Singh Chahal | User activity analysis using word clouds |
WO2020091829A1 (en) * | 2018-10-30 | 2020-05-07 | Google Llc | Automatic hyperlinking of documents |
US11610114B2 (en) | 2018-11-08 | 2023-03-21 | Nec Corporation | Method for supervised graph sparsification |
CN109858015B (en) * | 2018-12-12 | 2022-11-18 | 湖北工业大学 | Semantic similarity calculation method and device based on CTW (computational cost) and KM (K-value) algorithm |
US10970314B2 (en) * | 2018-12-21 | 2021-04-06 | Atlassian Pty Ltd. | Content discovery systems and methods |
CN111382246B (en) * | 2018-12-29 | 2023-03-14 | 深圳市优必选科技有限公司 | Text matching method, matching device, terminal and computer readable storage medium |
US20200233872A1 (en) * | 2019-01-18 | 2020-07-23 | International Business Machines Corporation | System and method for discovering synergistic companies |
CN110674296B (en) * | 2019-09-17 | 2022-11-04 | 上海仪电(集团)有限公司中央研究院 | Information abstract extraction method and system based on key words |
CN111444414A (en) * | 2019-09-23 | 2020-07-24 | 天津大学 | Information retrieval model for modeling various relevant characteristics in ad-hoc retrieval task |
US11176179B2 (en) * | 2019-09-24 | 2021-11-16 | International Business Machines Corporation | Assigning a new problem record based on a similarity to previous problem records |
US11663280B2 (en) | 2019-10-15 | 2023-05-30 | Home Depot Product Authority, Llc | Search engine using joint learning for multi-label classification |
CN111027305B (en) * | 2019-10-29 | 2023-12-19 | 平安科技(深圳)有限公司 | Text matching-based intelligent interview method and device and computer equipment |
US11544569B2 (en) * | 2019-11-21 | 2023-01-03 | Tencent America LLC | Feature map sparsification with smoothness regularization |
CN111144129B (en) * | 2019-12-26 | 2023-06-06 | 成都航天科工大数据研究院有限公司 | Semantic similarity acquisition method based on autoregressive and autoencoding |
CN111104488B (en) * | 2019-12-30 | 2023-10-24 | 广州广电运通信息科技有限公司 | Method, device and storage medium for integrating retrieval and similarity analysis |
US20230072537A1 (en) * | 2020-01-29 | 2023-03-09 | Nippon Telegraph And Telephone Corporation | Learning apparatus, search apparatus, learning method, search method and program |
US11620453B2 (en) | 2020-10-16 | 2023-04-04 | CS Disco, Inc. | System and method for artificial intelligence driven document analysis, including searching, indexing, comparing or associating datasets based on learned representations |
CN114065729A (en) * | 2021-11-16 | 2022-02-18 | 神思电子技术股份有限公司 | Text sorting method based on deep text matching model |
WO2024134784A1 (en) | 2022-12-20 | 2024-06-27 | 株式会社Fronteo | Data analysis device and data analysis program |
Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6137911A (en) * | 1997-06-16 | 2000-10-24 | The Dialog Corporation Plc | Test classification system and method |
US20100082511A1 (en) * | 2008-09-30 | 2010-04-01 | Microsoft Corporation | Joint ranking model for multilingual web search |
Family Cites Families (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US8195669B2 (en) * | 2008-09-22 | 2012-06-05 | Microsoft Corporation | Optimizing ranking of documents using continuous conditional random fields |
US8166032B2 (en) * | 2009-04-09 | 2012-04-24 | MarketChorus, Inc. | System and method for sentiment-based text classification and relevancy ranking |
-
2009
- 2009-09-18 US US12/562,802 patent/US8341095B2/en active Active
- 2009-09-18 US US12/562,840 patent/US8359282B2/en active Active
Patent Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6137911A (en) * | 1997-06-16 | 2000-10-24 | The Dialog Corporation Plc | Test classification system and method |
US20100082511A1 (en) * | 2008-09-30 | 2010-04-01 | Microsoft Corporation | Joint ranking model for multilingual web search |
Non-Patent Citations (19)
Title |
---|
Balasubramanian et al. The Isomap algorithm and topological stability. www.sciencemag.org, Science. vol. 295 . Jan. 2002. (3 Pages) http://www.sciencemag.org/cgi/reprint/295/5552/7a.pdf. |
Belkin et al. Laplacian Eigenmaps for Dimensionality Reduction and Data Representation. Advances in Neural Information Processing Systems 14 . Dec. 2002, pp. 1-28. |
Bengio et al. Out-of-sample extensions for LLE, isomap, MDS, eigenmaps, and spectral clustering. Advances in Neural Information Processing Systems 16. 2004. (8 Pages). |
Blei et al. Latent Dirichlet Allocation. Journal of Machine Learning Research. vol. 3. Jan. 2003, pp. 993-1022. |
Burges et al. Learning to rank using gradient descent. In ICML 2005, in Proceeding of the 22th International Conference on Machine Learning. Aug. 2005. pp. 89-95. |
Collins et al. New ranking algorithms for parsing and tagging: kernels over discrete structures, and the voted perceptron. In Proceedings of the 40th Annual Meeting on Association for Computational Linguistics. Association for Computational Linguistics. Jul. 2002. pp. 263-270. |
Collobert et al. Fast Semantic Extraction Using a Novel Neural Network Architecture. 45th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference. ACL. Jun. 2007. (8 pages). |
Deerwester et al. Indexing by latent semantic analysis. Journal of the American Society for Information Science, vol. 41. 1990. pp. 391-407. |
Grangier et al. A discriminative kernel-based approach to rank Images from text queries. IEEE Transactions on Pattern Analysis and Machine Intelligence. PAMI. vol. 30, Issue: 8. Aug. 2008. pp. 1-14. |
Grangier et al. Inferring document similarity from hyperlinks. In CIKM '05. ACM. Oct. 2005. pp. 359-360. |
Herbrich et al. Large margin rank boundaries for ordinal regression. Advances in Large Margin Classifers. Mar. 2000. pp. 115-132. |
Hofmann. Probabilistic latent semantic Indexing. Proceedings of the Twenty-Second Annual International SIGIR Conference on Research and Development in Information Retrieval. In SIGIR. Aug. 1999. pp. 50-57. |
Joachims. Optimizing search engines using clickthrough data. In ACM SIGKDD '02. International Conference on Knowledge Discovery and Data Mining. Proceedings of the eighth ACM SIGKDD international conference on Knowledge discovery and data mining. 2002. pp. 133-142. (10 Pages). |
Keller et al. A Neural network for Text Representation. In International Conference on Artificial Neural Networks. Sep. 2005. (6 Pages). |
Roweis et al. Nonlinear dimensionality reduction by locally linear embedding. www.sciencemag.org. Science. vol. 290. Dec. 2000. pp. 2323-2326. |
Salakhutdinov et al. Semantic Hashing. In proceedings of the SIGIR Workshop on Information Retrieval and Applications of Graphical Models. vol. 41 No. 2. Dec. 2007. (8 Pages). |
Scholkopf et al. Kernel principal component analysis. Advances in kernel methods: support vector learning. 1999. (6 Pages). |
Shi et al. Hash kernels. In Twelfth International Conference on Artificial Intelligence and Statistics. Apr. 2009. (8 Pages). |
Sun et al. Supervised latent semantic Indexing for document categorization. IEEE Computer Society. In ICDM 2004. ICDM '04: Proceedings of the Fourth IEEE International Conference on Data Mining. Nov. 2004. pp. 535-538. |
Cited By (15)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US9519859B2 (en) | 2013-09-06 | 2016-12-13 | Microsoft Technology Licensing, Llc | Deep structured semantic model produced using click-through data |
US10055686B2 (en) | 2013-09-06 | 2018-08-21 | Microsoft Technology Licensing, Llc | Dimensionally reduction of linguistics information |
US20150242387A1 (en) * | 2014-02-24 | 2015-08-27 | Nuance Communications, Inc. | Automated text annotation for construction of natural language understanding grammars |
US9524289B2 (en) * | 2014-02-24 | 2016-12-20 | Nuance Communications, Inc. | Automated text annotation for construction of natural language understanding grammars |
US9477654B2 (en) | 2014-04-01 | 2016-10-25 | Microsoft Corporation | Convolutional latent semantic models and their applications |
US9535960B2 (en) | 2014-04-14 | 2017-01-03 | Microsoft Corporation | Context-sensitive search using a deep learning model |
US10089580B2 (en) | 2014-08-11 | 2018-10-02 | Microsoft Technology Licensing, Llc | Generating and using a knowledge-enhanced model |
US10909450B2 (en) | 2016-03-29 | 2021-02-02 | Microsoft Technology Licensing, Llc | Multiple-action computational model training and operation |
US11681713B2 (en) | 2018-06-21 | 2023-06-20 | Yandex Europe Ag | Method of and system for ranking search results using machine learning algorithm |
US11194878B2 (en) | 2018-12-13 | 2021-12-07 | Yandex Europe Ag | Method of and system for generating feature for ranking document |
US11562292B2 (en) | 2018-12-29 | 2023-01-24 | Yandex Europe Ag | Method of and system for generating training set for machine learning algorithm (MLA) |
US10860809B2 (en) | 2019-04-09 | 2020-12-08 | Sas Institute Inc. | Word embeddings and virtual terms |
US11048884B2 (en) | 2019-04-09 | 2021-06-29 | Sas Institute Inc. | Word embeddings and virtual terms |
US11645540B2 (en) | 2019-08-12 | 2023-05-09 | Nec Corporation | Deep graph de-noise by differentiable ranking |
US11928107B2 (en) | 2020-05-22 | 2024-03-12 | International Business Machines Corporation | Similarity-based value-to-column classification |
Also Published As
Publication number | Publication date |
---|---|
US20100179933A1 (en) | 2010-07-15 |
US20100185659A1 (en) | 2010-07-22 |
US8341095B2 (en) | 2012-12-25 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US8359282B2 (en) | Supervised semantic indexing and its extensions | |
Alami et al. | Unsupervised neural networks for automatic Arabic text summarization using document clustering and topic modeling | |
US7085771B2 (en) | System and method for automatically discovering a hierarchy of concepts from a corpus of documents | |
Harish et al. | Representation and classification of text documents: A brief review | |
Roussinov et al. | A scalable self-organizing map algorithm for textual classification: A neural network approach to thesaurus generation | |
US7152065B2 (en) | Information retrieval and text mining using distributed latent semantic indexing | |
Kowalski | Information retrieval architecture and algorithms | |
Aliguliyev | Clustering techniques and discrete particle swarm optimization algorithm for multi‐document summarization | |
US8326785B2 (en) | Joint ranking model for multilingual web search | |
Kumar et al. | Analysis of a vector space model, latent semantic indexing and formal concept analysis for information retrieval | |
Seifzadeh et al. | Short-text clustering using statistical semantics | |
Nikolopoulos et al. | High order pLSA for indexing tagged images | |
Xu et al. | A new feature selection method based on support vector machines for text categorisation | |
Devi et al. | A hybrid document features extraction with clustering based classification framework on large document sets | |
Kazi et al. | Towards a new perspective on context based citation index of research articles | |
Rajman et al. | From text to knowledge: Document processing and visualization: A text mining approach | |
AlSumait et al. | Text clustering with local semantic kernels | |
Bouhlel et al. | Hypergraph learning with collaborative representation for image search reranking | |
Aghdam | Automatic extractive and generic document summarization based on NMF | |
Patil | A comprehensive review of recent relevance feedback techniques in CBIR | |
Freeman et al. | Tree view self-organisation of web content | |
Jia et al. | Addressing overgeneration error: An effective and efficient approach to keyphrase extraction from scientific papers | |
Zukuan et al. | An efficient content based image retrieval scheme | |
Zou et al. | Hidden semantic hashing for fast retrieval over large scale document collection | |
Mishra et al. | Extraction techniques and evaluation measures for extractive text summarisation |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: NEC LABORATORIES AMERICA, INC., NEW JERSEY Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:BAI, BING;WESTON, JASON;COLLOBERT, RONAN;AND OTHERS;SIGNING DATES FROM 20090911 TO 20090916;REEL/FRAME:023255/0416 |
|
STCF | Information on status: patent grant |
Free format text: PATENTED CASE |
|
AS | Assignment |
Owner name: NEC CORPORATION, JAPAN Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:NEC LABORATORIES AMERICA, INC.;REEL/FRAME:031998/0667 Effective date: 20140113 |
|
FPAY | Fee payment |
Year of fee payment: 4 |
|
AS | Assignment |
Owner name: NEC CORPORATION, JAPAN Free format text: CORRECTIVE ASSIGNMENT TO CORRECT THE REMOVE 8538896 AND ADD 8583896 PREVIOUSLY RECORDED ON REEL 031998 FRAME 0667. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT;ASSIGNOR:NEC LABORATORIES AMERICA, INC.;REEL/FRAME:042754/0703 Effective date: 20140113 |
|
MAFP | Maintenance fee payment |
Free format text: PAYMENT OF MAINTENANCE FEE, 8TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1552); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY Year of fee payment: 8 |