US20010037324A1 - Multilevel taxonomy based on features derived from training documents classification using fisher values as discrimination values - Google Patents

Multilevel taxonomy based on features derived from training documents classification using fisher values as discrimination values Download PDF

Info

Publication number
US20010037324A1
US20010037324A1 US09777278 US77727801A US2001037324A1 US 20010037324 A1 US20010037324 A1 US 20010037324A1 US 09777278 US09777278 US 09777278 US 77727801 A US77727801 A US 77727801A US 2001037324 A1 US2001037324 A1 US 2001037324A1
Authority
US
Grant status
Application
Patent type
Prior art keywords
documents
terms
document
node
feature
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Abandoned
Application number
US09777278
Inventor
Rakesh Agrawal
Soumen Chakrabarti
Byron Dom
Prabhakar Raghavan
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
International Business Machines Corp
Original Assignee
International Business Machines Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F17/00Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/30Information retrieval; Database structures therefor ; File system structures therefor
    • G06F17/3061Information retrieval; Database structures therefor ; File system structures therefor of unstructured textual data
    • G06F17/30705Clustering or classification
    • G06F17/3071Clustering or classification including class or cluster creation or modification
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y10TECHNICAL SUBJECTS COVERED BY FORMER USPC
    • Y10STECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y10S707/00Data processing: database and file management or data structures
    • Y10S707/99931Database or file accessing
    • Y10S707/99932Access augmentation or optimizing
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y10TECHNICAL SUBJECTS COVERED BY FORMER USPC
    • Y10STECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y10S707/00Data processing: database and file management or data structures
    • Y10S707/99931Database or file accessing
    • Y10S707/99933Query processing, i.e. searching
    • Y10S707/99936Pattern matching access

Abstract

A system, process, and article of manufacture for organizing a large text database into a hierarchy of topics and for maintaining this organization as documents are added and deleted and as the topic hierarchy changes. Given sample documents belonging to various nodes in the topic hierarchy, the tokens (terms. phrases, dates, or other usable feature in the document) that are most useful at each internal decision node for the purpose of routing new documents to the children of that node are automatically detected. Using feature terms, statistical models are constructed for each topic node. The models are used in an estimation technique to assign topic paths to new unlabeled documents. The hierarchical technique, in which feature terms can be very different at different nodes, leads to an efficient context-sensitive classification technique. The hierarchical technique can handle millions of documents and tens of thousands of topics. A resulting taxonomy and path enhanced retrieval system (TAPER) is used to generate context-dependent document indexing terms. The topic paths are used, in addition to keywords, for better focused searching and browsing of the text database.

Description

    PROVISIONAL APPLICATION
  • [0001]
    The present application claims the benefit of U.S. Provisional Application Ser. No. 60/050,611, entitled “USING TAXONOMY, DISCRIMINANTS, AND SIGNATURES FOR NAVIGATING IN TEXT DATABASES”, filed Jun. 24, 1997, by Rakesh Agrawal, et al., attorney's reference number AM9-97-060, which is incorporated herein by reference, in its entirety.
  • BACKGROUND OF THE INVENTION
  • [0002]
    1. Field of the Invention
  • [0003]
    The present invention relates, generally, to a process, system and article of manufacture for organizing and indexing information items such as documents by topic, and in preferred embodiments, to such a process, system and article which employ a topic hierarchy and involve a determination of discriminating terms and stop terms at each internal node in the topic hierarchy.
  • [0004]
    2. Description of Related Art
  • [0005]
    With modem advances in computer technology, modem speeds and network and internet technologies, vast amounts of information have become readily available in homes, businesses and educational and government institutions throughout the world. Indeed, many businesses, individuals and institutions rely on computer-accessible information on a daily basis. This global popularity has further increased the demand for even greater amounts of computer-accessible information. However, as the total amount of accessible information increases, the ability to locate specific items of information within the totality becomes increasingly more difficult.
  • [0006]
    The format with which the accessible information is arranged also affects the level of difficulty in locating specific items of information within the totality. For example, searching through vast amounts of information arranged in a free-form format can be substantially more difficult and time consuming than searching through information arranged in a pre-defined order, such as by topic, date, category, or the like. However, due to the nature of certain on-line systems, such as the internet, much of the accessible information is placed on-line in the form of free-format text. Moreover, the amount of on-line data in the form of free-format text continues to grow very rapidly.
  • [0007]
    Search schemes employed to locate specific items of information among the on-line information content, typically depend upon the presence or absence of key words (words included in the user-entered query) in the searchable text. Such search schemes identify those textual information items that include (or omit) the key words. However, in systems, such as the web, or large intranets, where the total information content is relatively large and free-form, key word searching can be problematic, for example, resulting in the identification of numerous text items that contain (or omit) the selected key words, but which are not relevant to the actual subject matter to which the user intended to direct the search.
  • [0008]
    As text repositories grow in number and size and global connectivity improves, there is a pressing need to support efficient and effective information retrieval (IR), searching and filtering. A manifestation of this need is the recent proliferation of over one hundred commercial text search engines that crawl and index the web, and several subscription-based information multicast mechanisms. Nevertheless, there is little structure on the overwhelming information content of the internet.
  • [0009]
    Common practices for managing such information complexity on the internet or in database structures typically involve tree-structured hierarchical indices. Many internet directories. such as Yahoo!™ (http://www.yahoo.com) and Infoseek (http://www.infoseek.com) are largely manually organized in preset hierarchies. International Business Machine Corporation has implemented a patent database (http://www.ibm.com/patents) which is organized by the US Patent Office's class codes, which form a preset hierarchy. Digital libraries that mimic hardcopy libraries support some form of subject indexing such as the Library of Congress Catalogue, which is also hierarchical. Such topic hierarchies are referred to herein as “taxonomies.” Taxonomies can provide a means for designing vastly enhanced searching, browsing and filtering systems. Querying with respect to a topic can be more reliable than depending only on the presence or absence of specific words in documents. By the same token, multicast systems such as PointCast (http://www.pointcast.com) are likely to achieve higher quality by registering a user profile in terms of classes in a taxonomy rather than key words.
  • [0010]
    The danger in querying or filtering by keywords alone is that there may be many aspects to, and often different interpretations of the key words, and many of these aspects and interpretations are irrelevant to the subject matter that the searcher intended to find.
  • [0011]
    Consider, for example, a situation in which a wildlife researcher is attempting to find information about the running speed of the jaguar, using the conventional Alta Vista™ internet search engine (http://www.altavista.digital.com), with the query “jaguar speed”. In a test search conducted with the above-noted search engine and query, a variety of responses were generated, spanning the car, the Atari™ video game, the football team, and a LAN server, in no particular order. The first page about the animal was ranked 183, and was directed to a fable.
  • [0012]
    To eliminate the responses on cars, the test query was then changed to “jaguar speed−car−auto”. The top response in the generated results read as follows:
  • [0013]
    “If you own a classic Jaguar, you are no doubt aware how difficult it can be to find certain replacement parts. This is particularly true of gearbox parts.”
  • [0014]
    The words car and auto do not occur on this page. There was no cat in the first 50 pages of the generated response. Some search engines such as Alta Vista™ propose additional keywords to refine the query, but, at the time of writing, all of the keyword were related to cars or football.
  • [0015]
    Even the query “jaguar speed+cat” gave unsatisfactory results. The responses included the word “cat”, but were often about automobiles. The 25th page was the first with information about jaguars, but did not contain the desired information.
  • [0016]
    In contrast, if a topic taxonomy such as Yahoo™ is used, there is no problem in insisting that the user seeks documents containing “jaguar” in the topical context of animals, not cars. Unfortunately, it is labor-intensive to maintain Yahoo™ manually as the web changes and grows faster than ever. In our test case, even though the search was easily restricted to within animals, no answer could be found within the relatively small collection returned.
  • [0017]
    Search engines are still an immature technology. Other areas have been researched intensively long before web search engines were devised, and the following discussion surveys the following overlapping areas of related research: Information Retrieval (IR) systems and text databases, data mining, statistical pattern recognition, and machine learning.
  • [0018]
    For data mining, machine learning, and pattern recognition, the supervised classification problem has been addressed in statistical decision theory (both classical, as in Wald, Statistical Decision Functions, 1950, and Bayesian, as in Berger, Statistical Decision Theory and Bayesian Analysis, 1985, each of which is incorporated herein by reference), in statistical pattern recognition (as in Duda and Hart, Pattern Classification and Scene Analysis, 1973 and Fukunaga, An Introduction to Statistical Pattern Recognition, 1990, each of which is incorporated herein by reference), in machine learning (as in Weiss and Kulikowski, Computer Systems that Learn, 1990, Natarajan, Machine Learning: A Theoretical Approach, 1991, and Langley, Elements of Machine Learning, 1996, each of which is incorporated herein by reference).
  • [0019]
    Classifiers can be parametric or non-parametric. Two well-known classes of non-parametric classifiers are decision trees, such as CART (as in Breiman et al, Classification and Regression Trees, 1984, which is incorporated herein by reference) and C4.5 (as in Quinlan, C4.5: Programs for Machine Learning, 1993, which is incorporated herein by reference), and neural networks (as in Hush and Horne, Progress in Supervised Neural Networks, 1993, Lippmann, Pattern Classification using Neural Networks, 1989, and Jain et al, Artificial Neural Networks, 1996, each of which is incorporated herein by reference. For such classifiers, feature sets larger than 100 are considered extremely large. Document classification may require more than 50,000.
  • [0020]
    The most mature ideas in IR systems and text databases, which are also successfully integrated into commercial text search systems such as Verity, ConText, and Alta Vista, involve processing at a relatively syntactic level (e.g., stopword filtering, tokenizing, stemming, building inverted indices, computing heuristic term weights, and computing similarity measures between documents and queries in the vector-space model. as described by Rijsbergen, Information Retrieval, 1979, Salton and McGill, Introduction to Modern Information Retrieval, 1983, or Frakes and Baeza-Yates, Information Retrieval: Data Structures and Algorithms, 1992, each of which is incorporated herein by reference). More recent work includes statistical modeling of documents, unsupervised clustering (where documents are not labeled with topics and the goal is to discover coherent clusters, as described in Anick and Vaithyanathan, Exploiting Clustering and Phrases for Content-based Information Retrieval, 1997, which is incorporated herein by reference), supervised classification (as in Apte et al, Automated Learning of Decision Rules for Text Categorization, 1994, and Cohen and Singer, Context Sensitive Learning Methods for Text Categorization, 1996. each of which is incorporated herein by reference), query expansion (as in Schutze et al, A Comparison of Classifiers and Document Representations for the Routing Problem, 1995, and Voorhees, Using WordNet to Disambiguate Word Senses for text Retrieval, 1993. each of which is incorporated herein by reference).
  • [0021]
    Singular value decomposition on the term-document matrix has been found to cluster semantically related documents together even if they do not share keywords (as discussed in Deerwester et al, Indexing by Latent Semantic Analysis, 1990, and Papadimitriou et al, Latent Semantic Indexing: A Probabilistic Analysis, 1996, each of which is incorporated herein by reference). None of these works address the supervised topic analysis problem in a hierarchy or how to use context-dependent words for indexing, how to automatically and efficiently compute good feature sets, and how to maintain disk data structures as training documents and the topic structure changes with time.
  • [0022]
    Koller and Sahami, Hierarchically Classifying Documents Using Very Few Words, International Conference on Machine Learning, July 1997 and Yang and Pedersen, A comparative study on feature selection in text categorization, International Conference on Machine Learning, July 1997 discuss classification. Koller et al propose a sophisticated feature selection algorithm that uses a Bayesian net to learn interterm dependencies. The complexity in the number of features is supralinear (e.g., quadratic in the number of starting terms and exponential in the degree of dependence between terms). Consequently, the reported experiments have been restricted to a few thousand features and documents. Yang and Pedersen's experiments appear to indicate that much simpler methods suffice, in particular, that the approach of Apte et al of picking a fixed fraction of most frequent terms per class performs reasonably. This fraction may be very sensitive to corpus and methodology (e.g. whether stemming and stopwording is performed). T his is indicated by the poor performance of methods observed in recent work by Mladenic, Feature Subset Selection In Text Learning, 10th European Conference on Machine Learning, 1998.
  • SUMMARY OF THE DISCLOSURE
  • [0023]
    Accordingly, it is an object of the present invention to enable scalable, efficient. reliable, and semi-automatic organization and reorganization of a database of information entities, such as text and hypertext documents, into a topic hierarchy with the express, but not exclusive, purpose of facilitating searching and filtering of documents as per the user's information need.
  • [0024]
    It is an object of further preferred embodiments to provide a system which uses the topic taxonomy to present the user with a series of progressively refined views of document collections in response to queries, and to discover and highlight salient differences between 1:0 two or more collections of documents.
  • [0025]
    It is an object of yet further preferred embodiments to provide such a system which is sufficiently fast, for example, for use in conjunction with a crawler or newswire service.
  • [0026]
    It is yet another object of further preferred embodiments to provide such a system which efficiently updates its knowledge when it makes mistakes and a human intervenes, or when there is a need to modify the topic taxonomy to accommodate new contents.
  • [0027]
    These and other objects are accomplished, according to preferred embodiments of the present invention, with a system called TAPER, standing for Taxonomy And Path Enhanced Retrieval, as described herein. For every internal node in the taxonomy, TAPER separates feature and noise terms by computing the best discriminant terms for that node. This is accomplished, by computing a measure of discrimination capability or power for each term in each document of a training set. In addition, a cut-off point is computed, for which terms having discrimination powers above the cut-off are considered feature (or discriminant) terms and terms having discrimination powers below the cut-off are considered stop terms or noise terms. In a preferred embodiment, a discrimination measure derived from mutual information (for example, as described by T. M. Cover and J. A. Thomas, Elements of Information Theory, John Wiley and Sons, Inc., 1991. incorporated herein by reference) or Fisher's discriminant (for example, as described by R. Duda and P. Hart, Pattern Classification and Scene Analysis, Wiley. New York, 1973, incorporated herein by reference) may be used.
  • [0028]
    Statistical models are constructed for each topic in the taxonomy, using the feature terms determined locally for that topic. In preferred embodiments, a Bernoulli or binary model of text generation may be assumed.
  • [0029]
    When classifying new documents, only the feature terms therein are used. Such feature terms are relatively few in number, so the class models are small and classification may be accomplished relatively quickly, in contrast to existing classifiers that employ a flat set of classes, the feature set changes by context as the classification process proceeds down the taxonomy. As a result, jargon common to lower nodes (or levels) of the taxonomy are filtered out, as stop words, at each node (or level), and the classification accuracy remains high in spite of the reduction in the number of terms and candidate classes inspected.
  • [0030]
    Addition and deletion of documents to given topics, as well as reorganization of the topic hierarchy itself, are easily handled. The text models built at each node also yield a means to summarize a number of documents using a few descriptive keywords, referred to herein as their signature.
  • [0031]
    A taxonomy has been used for illustration purposes for the discussion above. However, the taxonomy may take on various forms, and, in some embodiments of the present invention, is not necessary.
  • BRIEF DESCRIPTION OF THE DRAWINGS
  • [0032]
    [0032]FIG. 1 shows a diagram representing a hardware environment for one preferred embodiment of the present invention.
  • [0033]
    [0033]FIG. 2 shows a tree-shaped topic hierarchy which may be employed with one embodiment of the present invention.
  • [0034]
    [0034]FIG. 3 shows a block diagram representing a classifier training and feature selection system according to a preferred embodiment of the present invention.
  • [0035]
    [0035]FIG. 4 is a flow diagram illustrating the steps performed by the present invention for reorganizing, training, and testing.
  • [0036]
    [0036]FIG. 5 shows a graphical representation of an example of a function of error rate versus number of terms for a single document.
  • [0037]
    [0037]FIG. 6 shows a graphical representation of an example of a function of accuracy versus prefix size.
  • [0038]
    [0038]FIG. 7 shows a block diagram of the main statistics table maintained during training in one preferred embodiment.
  • [0039]
    [0039]FIG. 8 shows an illustration of the data structures involved in the computation of discrimination power of each term during feature selection, in one preferred embodiment.
  • [0040]
    [0040]FIG. 9 shows an illustration of the organization of the table of terms and their discrimination powers in one preferred embodiment.
  • [0041]
    [0041]FIG. 10 shows a diagram showing how indexed statistics are computed after feature selection in one preferred embodiment.
  • DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS
  • [0042]
    Querying in a Taxonomy
  • [0043]
    According to preferred embodiments of the present invention, a search query would not elicit a list of documents, but instead would elicit a list of topic paths. A topic path is a path in a topic taxonomy. For example, the search query “jaguar speed” may elicit a list of topics, as follows:
  • [0044]
    Business_and_Economy:Companies:Automotive
  • [0045]
    Recreation:
  • [0046]
    Automotive
  • [0047]
    Games:Video_Games
  • [0048]
    Sports:Football
  • [0049]
    Science:Biology:Animal_Behavior
  • [0050]
    Thereafter, the user may restrict queries by concept, not by keyword. A concept is more general than a keyword. For example, the concept “animal” together with the keyword “jaguar” is a better query than “jaguar” alone. A keyword is syntactically embedded or implied by a document, whereas a concept is a semantic attribute. A concept could even be functional (e.g., irrespective of syntactic content, a site can be categorized as “commercial” or “academic”). Moreover, the same keyword may induce different concepts (e.g., “jaguar” in the context of “animal” vs. “car”).
  • [0051]
    Upon obtaining a list of topic paths, the user may restrict the search to only one topic path or a few selected topic paths, by selecting one or more topics in the list. Depending upon the depth of the taxonomy, the selection of one or more topic paths may result in a list of further topic paths, representing further levels of the taxonomy.
  • [0052]
    As shown above, designing the query to enforce or forbid additional keywords may not always be as effective as restricting the search to particular topic paths. The ability to restrict searches to particular topic paths may be very useful for multicast channels as well. Additionally, user profiles will be topic paths rather than keywords.
  • [0053]
    Scalable Filtering Using a Topic Taxonomy
  • [0054]
    Another paradigm of information retrieval is filtering, in which a continual stream of documents are generated on-line, as in newsgroups and newsfeed. The system collects interest profiles from users and uses them to implement either content-based or collaborative filtering. i.e., it notifies the user only of those documents they are likely to be interested in.
  • [0055]
    In its simplest form, a profile may be a set of terms and phrases specified explicitly by the user. This has the same problem as querying without topic context as discussed above. A better notion of a profile is the set of documents the user has seen and/or liked, perhaps with scores. This may work well with small systems, but for thousands of users and millions of documents, a system storing this level of detail will not scale. A promising alternative is to characterize profiles, not at the level of individual documents, but at the level of narrow but canonical topics. This can be realized as one embodiment of the present invention.
  • [0056]
    Context Dependent Signatures
  • [0057]
    An exhaustive keyword index employed by such systems as Alta Vista™ is perhaps more of a problem than a solution. The IR literature has advanced further, and now prototypes exist that extract signature terms, which are then used for indexing. These signatures can also be used as summaries or thumbnails. The descriptive power of signatures can often compare favorably with that of arbitrary sentences as extracted by popular search engines. The signatures are also effective for describing a document cluster. Many approaches have been used for signature extraction, and in one common approach, the most frequent terms that are not stopwords are selected.
  • [0058]
    A document abstract or signature as a function of the document alone is of limited utility. In the case of a taxonomy, a useful signature is a function of both the document and the reference node. The signature includes terms that are “surprising” given the path from the root to the reference node. In the above example, car and auto may be good signature terms at the top level, or even at the Recreation level, but not when the user has drilled down to Recreation:Automotive.
  • [0059]
    The following is an illustration of text from a document in Health Nursing (http://www2.best.com/goodnews/practice/faq.htm):
  • [0060]
    “Beware of the too-good-to-be-true baby that is sleeping and sleeping and doesn't want to nurse. Especially monitor the number of wet diapers, as seriously jaundiced babies are lethargic.”
  • [0061]
    The first level classification is Health. The top signature terms are computed with respect to Health as follows:
  • [0062]
    Jaundice, dampen, dehydration, lethargic, hydrate, forcibly, caregiver, laxative, disposable.
  • [0063]
    This indicates that the document is about treating jaundice. The second level classification is Health:Nursing. Shifting the reference class, the new signature is computed to be:
  • [0064]
    Baby, water, breast-feed, monitor, new-born, hormone.
  • [0065]
    This now indicates that the document is about nursing babies. This information comes from both the path and the signatures. Techniques for computing context-sensitive signatures are described herein. Thus, significant improvement in search quality may be possible by maintaining functionally separate indices at each taxonomy node, using only a few signature terms from each document.
  • [0066]
    Context Dependent Term Associations
  • [0067]
    Finding term associations is another application of context-sensitive signatures. The use of phrases for search and classification can potentially boost accuracy. The usual way to find phrases is to test a set of terms for occurrence rate far above that predicted by assuming independence between terms. Unfortunately, associations that are strong for a section of the corpus may not be strong globally and may go unnoticed. For example, the term “precision” may be visibly associated with the term “recall” in a set of documents on IR. but not in a collection also including documents on machine tools. Computing signatures at each node can expose all such associations.
  • [0068]
    Context Dependent Feature Selection
  • [0069]
    Separating feature terms from noise terms is central to preferred embodiments described herein. In the above examples, car and auto should be “stopwords” within Recreation:Automotive and, therefore, should be pruned from the signatures. Feature and noise terms must be determined at each node in the taxonomy.
  • [0070]
    It is tricky to hand-craft the stopwords out of domain knowledge of the language. For example, the term “can” is frequently included in stopword lists. However, that term should not be a stop term for a corpus on waste management. The contents of a stopword list should be highly dependent on the corpus. This issue looms large in searching using categories and clusters. In hierarchical categories, the importance of a search term depends on the position in the hierarchy.
  • [0071]
    Other Applications of Feature Selection
  • [0072]
    Feature selection is also useful in any setting in which salient distinctions are sought between two or more sets of documents. Consider the scenario in which a set of documents (e.g. a keyword query result) has been clustered into subsets, and a user wishes to annotate the clusters using salient keywords. The clusters can be regarded as classes, and feature selection can be used to find these keywords.
  • [0073]
    Another application of feature selection is in the discovery of succinct differences between two sets of documents, such as patents filed by two companies, or by the same company over two different time intervals, to expose interesting trends over time.
  • [0074]
    Hardware Environment
  • [0075]
    As noted above, the present invention relates, generally, to a process, system and article of manufacture for organizing, classifying, and indexing information items by topic, such as text and hypertext documents. In one example embodiment, such information items comprise documents accessible on the internet. However, it will be understood that further embodiments of the invention are applicable to information items accessible in local network environments, dedicated database environments, or the like.
  • [0076]
    An example hardware environment for an internet embodiment is shown in FIG. 1, which includes a user computer 10, a user display 11, and a user interface 12. The display 11 is preferably a visual display device, such as a cathode ray tube monitor, a liquid crystal monitor or other suitable display device. The user interface 12 preferably comprises one or more of a key board, mouse, touch-screen device or other suitable input device. The computer 10 operates in accordance with a software program stored on a computer readable medium. such as a floppy disk 13, hard disk (not shown) or other suitable storage medium.
  • [0077]
    The computer 10 is linked, through an internet connection, and operates in accordance with a suitable software program to access information items stored in at least one information database. In the illustrated embodiment, the information items comprise text documents stored or accessible through one or more server locations. For example, with reference to FIG. 1, a set of text documents on a variety of topics are stored or accessible through the Server 1 location, and a further set of text documents on a variety of topics are stored or accessible through the Server 2 location. Further, server locations (not shown) may store or provide access to additional documents.
  • [0078]
    As described in farther detail herein, preferred embodiments of the present invention include a system comprising a computer, display and user interface, which operate in accordance with a process stored as a program on a computer readable medium, to organize and classify information items. While such information items may be text documents accessible through an internet connection, as shown in FIG. 2, it will be understood that, in further embodiments, the system may operate with information items other than those available on the internet, including, but not limited to information items accessible through local networks, dedicated databases, or the like. However, for purposes of simplifying the present disclosure, embodiments of the present invention are described herein primarily with reference to a search and classification system which operates with information items in the form of text documents that are accessible through an internet connection with the internet.
  • [0079]
    As described in further detail below, embodiments of the invention relate to an automatic process for learning topics from examples and later identifying topics of new documents (also called “test documents”). The process employs a multilevel taxonomy having a plurality of nodes, including a root node, at least one intermediate node associated with and under the root node and a plurality of terminal nodes associated with and under each intermediate node. A different set of feature terms are associated with each intermediate node, which are used to classify test documents. The feature term sets are determined, according to preferred embodiments, during a training procedure.
  • [0080]
    The training procedure, according to preferred embodiments, employs a plurality of training documents that have been pre-assigned manually to various terminal and intermediate nodes in the taxonomy. The training documents are tokenized, and information related to the frequency of terms or tokens is recorded in a database. A discrimination value is determined for each term in the training documents, and a minimum discrimination value is determined. Then, for each intermediate node, a set of feature terms is selected, where the feature terms are those that are in the training documents associated with the intermediate node or any of its descendants and that have discrimination values equal to or above the minimum discrimination value for the intermediate node.
  • [0081]
    In one preferred embodiment, such as related to finding salient differences between two or more sets of documents, the process ends here, after having output the best features.
  • [0082]
    In another embodiment, once the system is trained, test documents are analyzed. During this phase, a text document is first tokenized. Of all the tokens in the document, only those that are also in the feature set of the root topic in the taxonomy are considered useful. The statistics related to these useful terms are retrieved from the database, and the statistics are used to compute a score for each of the children of the root node (nodes comprising the next level connected to the root node). A few children with high scores are then picked for further exploration. If any child is an intermediate node, it has associated with it another feature set. The set of all tokens in the test document is now intersected with this new feature set, and the procedure continues from the child in the same manner.
  • [0083]
    In a related embodiment, the system also computes, for each topic node with suitably high score, the terms in the test document that are significantly more frequent than in the training set for that topic. These are then used for building a term index.
  • [0084]
    In yet another preferred embodiment, the above system can be used to process a search query. The search query is received from the user, for example, through a user input device in the form of keywords. Optionally, the user also restricts the topical context using a suitable selection on the taxonomy. Then a plurality of relevant documents which also adhere to the topic restrictions is retrieved. In a preferred embodiment, each document in the database has been pre-classified using the above system. The user is presented with a suitable display of those portions of the taxonomy where relevant documents were found. The user may then enter a command through the user input device to cause the system to select at least one of the displayed sub-topics. This process is repeated as necessary to refine the query topic until the user's information need is satisfied.
  • [0085]
    Topic Hierarchy
  • [0086]
    Organization and classification of information items, according to embodiments of the present invention, involves a topic hierarchy, or “taxonomy,” preferably having a plurality of levels of nodes. While embodiments of the invention may employ any directed acyclic graph hierarchy structure, embodiments are described herein with reference to a tree-like topic hierarchy.
  • [0087]
    An example of a tree-like topic hierarchy, or taxonomy, for organizing a database of topical documents is shown in FIG. 2. The tree 20 includes a first level comprising a single node 22 titled “All Topics.” A second level of the tree may divide the first level “All Topics” node into several further nodes directed to general topic categories, such as Business and Economy 24, Recreation 26, Science 28, and so forth. Each of the second level nodes may be divided, at the third level, into several further nodes directed to more specific topics within each second level topic. For example, at the third level, the Business and Economy topic 24 may be divided into Companies, Stock Markets, and so forth. Similarly, each of the other second level topics may be divided at the third level to further topics. Also, in a similar fashion, further levels under the third level may be included in the topic hierarchy, or taxonomy. The final level of each path in the taxonomy terminates at a terminal or leaf node, labeled c in the diagram. The taxonomy in the diagram is provided as an example for purposes of simplifying the present disclosure and is not intended to limit the invention to the specific illustration of the taxonomy.
  • [0088]
    According to preferred embodiments of the present invention, an appropriate topic hierarchy, or taxonomy, is provided by the user, based on the material (for example text documents) that are intended to be classified and searched. For example, if the material to be classified and searched includes major topics on the internet, a taxonomy might appear as shown in FIG. 2. As another example, if the material to be classified and searched includes all U.S. patents, a taxonomy which more closely follows the U.S. Patent and Trademark Office classification system might be employed.
  • [0089]
    System Training
  • [0090]
    Before the system can analyze the topics of new documents, the system is provided with examples of documents belonging to given topic(s). Given a topic hierarchy (taxonomy), system training is performed by providing an initial collection of documents for which classifications are known in advance. With reference to the block diagram of a training system 30 of FIG. 3, this may be accomplished, for example, by collecting a number of documents 32. For example, for classification and searching of documents available on the internet, the document collection may be performed with a suitable web crawler. Alternatively, a sample document collection may be provided with the system software 13 (FIG. 1) or manually collected from any suitable source.
  • [0091]
    The sample document collection is divided into two sets. One set of documents is set aside as a testing set 34. The other set is manually classified or otherwise pre-designated as corresponding to a particular class or terminal node (or, in some cases, intermediate node) within the given topic hierarchy and becomes the training set 38. At block 40, the training set of documents 38 is split, preferably randomly, into a statistics collection set of documents 42 and a model validation set of documents 44.
  • [0092]
    In block 46, statistics are collected from the statistics collection set 42, based on terms appearing in those documents and the known classes for those documents. These statistics are used in the determination of the discriminating power of terms in the documents from the collection set 42. The statistics are calculated for each node in the taxonomy, such that, for any one node, the discriminating power is calculated for the terms in all of the documents that are classified in terminal (and intermediate) nodes below that node. That is, the power that each term has to discriminate between classes in the next level below each node is calculated.
  • [0093]
    Thus, with reference to the hierarchy represented in the FIG. 2, statistics are calculated for the “science” node 28, based on the terms in all of the documents d from the collection set 42 that are classified in classes represented by nodes (terminal and intermediate) below the “science” node 28. including the nodes labeled “biology,” “chemistry,” “electronics,” and all children nodes of those nodes. Similarly, statistics are calculated for the terms in all of the documents (from the collection set 42) under each of those intermediate nodes and each of the other intermediate nodes in the hierarchy.
  • [0094]
    The statistics calculated for each intermediate node in the hierarchy includes quantities that enable computing the “discriminating power” of each term found in some training document under the node. Based on these statistics, terms are ordered by decreasing discriminating power and the top discriminating terms (those terms with the highest discriminating power) are selected as feature terms for use in classification, while the remaining terms are characterized as stop terms that have little value in distinguishing between topics in the immediate context. The determination of which terms in the order are feature terms and which terms are stop terms is provided by selecting a cut-off point within the ordering.
  • [0095]
    At block 42 in FIG. 3, the “statistics collection” subset of the training documents are used to collect term frequency information. Then, in block 48, feature terms and stop terms are determined for each internal topic node based on the model validation set 44. Finally, class models are constructed over the chosen features in block 49, preferably as described below in the section titled “Document Models.”
  • [0096]
    The class models and statistical information calculated in block 46 are provided to the classifier 50, for classifying the test documents 34 in a testing mode, as well as new documents when the system is deployed. Classification of test (or new) documents is carried out in the taxonomy, such that each test (or new) document is ultimately classified to correspond to one or more classes, designated by terminal or leaf nodes (or, in some cases, intermediate nodes in the hierarchy).
  • [0097]
    [0097]FIG. 4 is a flow diagram illustrating the steps performed by the present invention for reorganizing, training, and testing. After starting at block 60, in block 62, an action type is determined.
  • [0098]
    If the action type is “reorganize”, then, statistics are updated in block 64 and stored in the raw statistics database in block 70. Reorganization refers to a change in the structure of the topic tree. For example, one type of change involves collapsing two topics into one (e.g., a “science” topic may be collapsed with a “mathematics” topic). Another type of change involves expanding a topic into multiple topics (e.g., a “mathematics” topic may be expanded into a “calculus” topic, a “linear algebra” topic, and other topics). Once statistics are stored in the raw database in block 70, processing continues at block 72.
  • [0099]
    In block 72, whether compaction is needed is determined. As will be discussed under the “Statistics Collection” heading below, compaction refers to merging an unsorted table into the main sorted table. If compaction is needed, then tables are compacted in block 74. If compaction is not needed or after compaction, the processing continues to block 76. In block 76, features are selected. Feature selection will be discussed in more detail below under the “Feature Selection” heading. In block 78, indexed statistics are written to tables for testing. The statistics are stored in the indexed database in block 90.
  • [0100]
    If the action type is “new training document”, then the document is tokenized in block 66. Next, in block 68, statistics are appended. In block 70, the appended statistics are stored in the raw statistics database. Also, the appended statistics are stored in the indexed database in block 90. Then, processing continues at block 72, as discussed above.
  • [0101]
    If the action type is “test”, documents are tokenized in block 92. In block 94, the root topic is selected as a starting point. In block 96, a top topic is picked form the pool (i.e., a topic with a high goodness score). In block 98, using indexed statistics from block 90 (as indicated by the arrow from block 90 to block 98), the children of the picked topic are evaluated and the best ones (i.e., those with high goodness scores) are added to the pool. In block 100, it is determined whether there are enough leaf topics. If there are not enough leaf topics, at block 96, another topic is picked. Otherwise, the context-dependent signatures are computed in block 102. The computation of context-dependent signatures is discussed in more detail under the “Extracting Document Signatures” heading below. These context-dependent signatures are displayed or indexed in block 104.
  • [0102]
    Document Models
  • [0103]
    There have been many proposals for statistical models of text generation. One of the earliest indicators of the power of simple statistical tests on term frequencies is Zipfs law. The models most frequently used in the IR community are Poisson and Poisson mixtures. If X is distributed Poisson with rate μ, denoted X˜P(x), then Pr[X=x]=e−μμx/x!, and if Y is distributed Bernoulli with n trials and mean np, denoted Y˜B(n,p), then Pr [ Y = y ] = ( n y ) p y ( 1 - p ) n - y . As n and p 0 , the distributions B ( n , p ) and P ( np )
    Figure US20010037324A1-20011101-M00001
  • [0104]
    converge to each other.
  • [0105]
    According to preferred embodiments described herein, a Bernoulli or binary model of document generation is assumed. In the Bernoulli model, a document d is generated by first picking a class c. Each class has an associated multi-faced coin, with each face representing a term t and having some success probability θ(c,t). Then a document length n(d) is arbitrarily fixed, and each term is generated by flipping the coin. In the binary model, a document is a set of terms with counts zero or one, and θ(c,t) is an estimate of the fraction of documents in class c that contain term t at least once.
  • [0106]
    Generally, in a binary model, the focus is on whether a term occurs, and so a term is either associated with zero (i.e. occurs) or one (i.e., does not occur). In a Bernoulli model, the focus is on how many times a term occurs, and so the model keeps track of “buckets” for the number of times a term occurs (e.g., once, twice, three times, . . . , n times). However, a variety of other models could be used. For example, in one model, it may be relevant that a term occurred once, twice, three times or four or more times, while it is not relevant that the term occurred specifically four times, five times, . . . , n times. In another model, the “buckets” could be specified to hold terms that occur “once”, “two to three times”, “four to seven times”, etc.
  • [0107]
    Conceptually, as the training text is being scanned, the classifier database will be organized as a three-dimensional table. The first axis is for terms, the second axis is for documents, and the third axis is for classes or topics. The measure maintained along these dimensions, (t,d,c), is called n(t,d,c), which is the number of times t occurs in d∈c. This number is non-zero only when t∈d∈c. t E d means that term t occurs in document d, and d∈c means that d is a training sample for class c. A super-class of c, i.e., an ancestor in the topic tree, inherits all d∈c.
  • [0108]
    Aggregation along some dimensions gives some important statistics about the corpus that is used by the classifier. The following is a list of these statistics:
  • [0109]
    The length of training document d, given by n ( d ) = i n ( t , d , c ) .
    Figure US20010037324A1-20011101-M00002
  • [0110]
    The length of all documents can be found using a GROUP BY on (d,c).
  • [0111]
    The total length of the training documents in class c, denoted n(c).
  • [0112]
    The total number of times that term t occurred over all training documents of class c, denoted n(t,c).
  • [0113]
    The fraction of times, f ( t , d , c ) = n ( t , d , c ) / t n ( τ , d , c ) ,
    Figure US20010037324A1-20011101-M00003
  • [0114]
    that term t occurs in document d. The sum of f and f2 over all documents in a class is needed.
  • [0115]
    The number m(t,c) of training documents in class c that have at least one occurrence of term t. This will be needed for the binary model.
  • [0116]
    The number of training documents in class c, denoted |c|.
  • [0117]
    Assuming the Bernoulli model with parameters θ(c,t), the following equation, Pr [ d c ] = ( n ( d ) { n ( d , t ) } ) t θ ( c , t ) n ( d , t ) , where ( n ( d ) { n ( d , t ) } ) = n ( d , t ) ! n ( d , t 1 ) ! n ( d , t 2 ) !
    Figure US20010037324A1-20011101-M00004
  • [0118]
    is the multinomial coefficient. A corresponding expression can be easily derived for the binary model as well.
  • [0119]
    The Bernoulli model makes the assumption that term occurrences are uncorrelated, which is not accurate. First, given that a term has occurred once in a document it is more likely to occur again when compared to a term about which information is not available. Second, the term frequency distributions are dependent.
  • [0120]
    Our independence assumption leads to what is called a naive Bayes classifier. (A naive Bayes classifier, in essence, builds density functions, which are marginally independent, for each class, and then classifies a data point based on which density function has the maximum value at that point.) In practice, these simple classifiers perform surprisingly well compared to more sophisticated ones that attempt to approximate the dependence between attributes.
  • [0121]
    Recently this phenomenon has been investigated in depth by Friedman in On Bias, Variance, 0/1 Loss, and the Curse-of-dimensionality, Data Mining and Knowledge Discovery, 1(1), pp. 55-77, 1997, incorporated herein by reference). A classifier that uses an estimate of class densities is subject to bias (decision boundaries that are shifted from the “best” position, because the model is inaccurate) and variance (decision boundaries fit to noisy data). Friedman analyzes how the low variance of naive density estimates can mitigate the high bias to give simple classifiers that can often beat more sophisticated ones. It will also be clear from a description of the system that this simplicity enables designing of a system that can handle enormous problem sets.
  • [0122]
    Rare Events and Laws of Succession
  • [0123]
    The average English speaker uses about 20,000 of the 1,000,000 or more terms in an English dictionary. In that sense, many terms that occur in documents are “rare events.” This means that with reasonably small sample sets, there will be zero occurrences of many terms in many classes, and the maximum likelihood estimate θ(c,t)=n(c,t)/n(c) wilt be problematic: a class with θ(c,t)=0 will reject any document containing t even a single time.
  • [0124]
    Finding better estimates of small probabilities, also called laws of succession, has been pursued in classical statistics for centuries. Laplace showed that given the results of n tosses of a L-sided coin, i.e., the number of times each face occurred, n1, . . . , nL, the correct Bayesian estimate for the probability of face i is not ni/n, but n 1 + 1 n + L .
    Figure US20010037324A1-20011101-M00005
  • [0125]
    This is the result of assuming that all possible associated L-component vectors of face probabilities (p1, . . . , pL) are a priori equally likely. This is called the uniform prior assumption. The above value of is obtained by using Bayes rule and evaluating 1 Pr [ n 1 ] 0 1 θ Pr [ n i θ ] θ .
    Figure US20010037324A1-20011101-M00006
  • [0126]
    Alternative priors have been suggested and justified. (E. S. Ristad, A natural law of succession, Research report CS-TR-495-95, Princeton University, July 1995, which is incorporated herein by reference.) However, based on experimentation conducted in connection with the development of the present invention, it was found that Laplace's law provides a few percent better classification accuracy. With this adjustment (and returning to the earlier notation) θ(c,t) is estimated as 1 + n ( c , t ) n ( c ) + L ( c )
    Figure US20010037324A1-20011101-M00007
  • [0127]
    where L(c) is the size of the lexicon (the number of distinct terms found in the training documents) of class c.
  • [0128]
    Hierarchical Classification
  • [0129]
    A classifier inputs a document and outputs a class. If the class is not the one from which the document was generated, the classifier is said to have misclassified that document. In the case of a topic hierarchy, one may wish to give the classifier “partial credit” for correctly finding the first few levels of the “true” topic. This is ignored in the current discussion and will be commented on later. For now the discussion focuses on how to find the best leaf topics.
  • [0130]
    According to preferred embodiments of the present invention, a distinct classifier is associated with each internal node in the taxonomy, including the root. During classifier training, a set of feature terms is generated for each of such nodes. Given a new document d, the goal of the classification process is to find a leaf node c such that the probability that the document d was generated from class c (called the posterior probability Pr[c|d] ) is maximized among all the leaves.
  • [0131]
    Hierarchical classification has the benefit of greatly increased speed of classification. As described next, classification of a test document starts at the taxonomy root by assigning a score to each child of the root. In many cases, it will be possible to eliminate most of the topic sub-trees as unlikely candidates. Thus, large sub-trees in the topic tree can be eliminated forthwith if the score of the root of those sub-trees are very poor. Text database population is not the only application of fast multi-level classification. With increasing connectivity, it will be inevitable that some searches will go out to remote text servers and retrieve results that must then be classified in real time.
  • [0132]
    This benefit of increased speed may be useless if an error is made in choosing a topic early in the process at a shallow level of the tree. Thus, a greedy search for the best leaf may be risky. Let the path to a leaf c from the root c1 be c1,c2, . . . , ck=c. Since the root subsumes all classes, the probability that any document d is within the root class c1 is Pr[c1|d]−1. Thereafter, Pr[ci|d]=Pr[ci=1|d]Pr[ci|ci−1,d], for i=2, . . . , k. Taking logs, log Pr[ci|d]=log Pr[ci−1|d]+log Pr[ci|ci−1,d]. Suppose that, in the taxonomy, the edge (ci−1,ci) is marked with the edge cost−log Pr[ci|ci−1,d]. Then the least-cost path from the root to some leaf is being sought.
  • [0133]
    Computing the one-step conditional probability Pr[ci|ci−1,d] is straight-forward. For notational convenience, name ci−1 as r0 and its children {rj}. Then the probability that the document d belongs to the child node rj, given that it belongs to the parent node r0, is given by Pr[rj|r0,d]=Pr[rj|d]/Pr[r0|d], where Pr [ r 0 d ] = ρ Pr [ ρ d ] ,
    Figure US20010037324A1-20011101-M00008
  • [0134]
    where the sum is over all children ρ of r0. Note that Pr [ r 0 d ] = Pr [ d r j ] / ρ Pr [ d ρ ]
    Figure US20010037324A1-20011101-M00009
  • [0135]
    using Bayes rule, and this can be evaluated from the model parameters. Care is needed here with finite-precision numbers and underflow, because the probabilities are very small and the scaling needed to condition the probability prevents maintaining the numbers always in log-form.
  • [0136]
    Feature and Noise Terms
  • [0137]
    The above application of Bayes rule depended on a document model. This was embedded in the θ(c,t) parameters. These parameters are estimated during the training phase. using sample documents from the statistics collection set 42 (FIG. 3). When building a model for each class from a training set, a determination is made as to whether a term appears only incidentally, or sufficiently consistently to suspect a causal connection; the term is accordingly a noise term (also called a stopword) or a feature term. Only feature terms should be used for classifying new documents.
  • [0138]
    A property of preferred embodiments of the present invention that distinguishes it from prior art (as in Apte, Damerau, and Weiss, 1994) is the use of different feature sets computed separately for each internal node. This prevents the classifier from losing accuracy even though it inspects very few of the classes in the taxonomy to pick the best leaf topics.
  • [0139]
    The challenge is to select suitable feature terms from a lexicon that can be as large as a hundred thousand terms. The selection process is constrained both ways: the highly discriminating terms should not be missed, and every term should not be included, because the frequencies of some terms are noisy and not indicative of content. This is called the feature-selection problem in the statistical pattern recognition literature. In general, there is a need to find a set of terms that minimizes the probability that a document is misclassified, with the understanding that only terms in the intersection of the document and the feature set are used by the classifier.
  • [0140]
    It is not possible to search for the best feature set, because it is not known what the best possible classifier does, and because there are too many terms in the lexicon. Therefore, in practice this is done for a fixed classifier. A heuristic is desired that is essentially linear in the original number of terms and preferably makes only one pass over the training corpus.
  • [0141]
    Therefore, the following approach is carried out: first a merit measure is assigned to each term, and then a prefix of terms with highest merit are selected. In preferred embodiments, the merit measure comprises an index based on mutual information or on Fisher's linear discriminant. Mutual information is a well-known statistical measure of dependence between random variables (see Cover and Thomas). It is straight-forward to apply mutual information to the binary document model, but it is more complicated to apply it to the Bernoulli model, and more expensive to evaluate. Next, the Fisher discriminant measure that was used in the present invention is described, and this measure was found to be more effective than mutual information.
  • [0142]
    Fisher's Discriminant
  • [0143]
    Suppose two sets of points are given in k-dimensional Euclidean space, interpreted as two classes. Fisher's approach finds a direction on which to project all the points so as to maximize (in the resulting one-dimensional space) the relative class separation as measured by the ratio of inter-class to intra-class variance. More specifically, let X and Y be the point sets, and μX, μY be their respective centroids, i.e., μ X = 1 X x X x
    Figure US20010037324A1-20011101-M00010
  • [0144]
    and μ y = 1 y y Y y .
    Figure US20010037324A1-20011101-M00011
  • [0145]
    Further, let the respective covariance matrices be X = 1 X x · X ( x - μ X ) ( x - μ X ) T
    Figure US20010037324A1-20011101-M00012
  • [0146]
    and y = 1 y y Y ( y - μ y ) ( y - μ y ) T .
    Figure US20010037324A1-20011101-M00013
  • [0147]
    Fisher's discriminant approach seeks to find a vector α such that the ratio of the projected difference in means, |αTX−μY)|, to the average variance, ½αTXY)α=αTΣα, is maximized. It can be shown that α=Σ−1X−μY) achieves the extremum, provided Σ−1 exists. Furthermore, when X and Y are drawn from multivariate Gaussian distributions with ΣXY, this is the optimal discriminator, in that computing αTq for a test point q and comparing the result to a suitable threshold is the minimum error classifier.
  • [0148]
    Computing α involves a generalized eigenvalue problem involving the covariance matrices. In applications such as signal processing where Fisher's discriminant is used, the matrix size k is typically a few hundred at most. In the text domain, the matrix size k is typically 50,000 to 100,000, and the covariance matrices may not be suitably sparse for efficient computation. Moreover, it is difficult to interpret a discriminant that is a linear sum of term frequencies, possibly with negative coefficients. A preferred approach, therefore, will be to take the directions α as given, namely, a coordinate axes for each term. A figure of merit is assigned to each term, which is called its Fisher index, based on the variance figures above, which is α T ( μ X - μ Y ) α α T α
    Figure US20010037324A1-20011101-M00014
  • [0149]
    in the two-class case. For each term t, α=ε1, is a unit vector in the direction of t.
  • [0150]
    In general, given a set of two or more classes {c}, with |c| documents in class c, the ratio of the so-called between-class to within-class scatter is computed. Switching back to term frequency notations, this is expressed as Fisher ( t ) = c 1 · c 2 ( μ ( c 1 , t ) - μ ( c 2 t ) ) 2 c 1 c 1 d c ( n ( t , d , c , ) - μ ( c · t ) ) 2 , where μ ( c , t ) = 1 c 1 d c n ( t , d , c ) .
    Figure US20010037324A1-20011101-M00015
  • [0151]
    Selecting a Cut-Off
  • [0152]
    The remaining exercise, having sorted terms in decreasing order of Fisher index, is to pick a suitable number of feature terms starting with those having the highest index. Let F be the list of terms in our lexicon sorted by decreasing Fisher index. A preferred heuristic is to pick from F a prefix Fk of the k most discriminating terms. Fk must include most useful features and exclude most noise terms. A short Fk enables holding a larger taxonomy in memory and hence fast classification. Too large an Fk will degrade not only performance, but also accuracy because of the phenomenon of over-fitting: the classifier will fit the training data very well, but will result in degraded accuracy for test data. There are various techniques for pruning feature sets. The current invention prefers the technique of minimization of the classification error rate on a separate validation set. There exist other approaches in the prior art using the minimum description length principle, resampling or cross validation, but these make too many passes over the text corpus.
  • [0153]
    The pre-classified samples are partitioned, preferably randomly, into the training set T (shown as block 42 in FIG. 3) and the validation set V (shown as block 44 in FIG. 3). The Fisher index of each term based on documents in set T is computed, and then documents in set V are classified using various prefixes Fk. Let Nk be the number of documents incorrectly classified when a prefix of k features is used, then (the smallest) k for which Nk is minimized is sought.
  • [0154]
    For classification using a feature set Fk. the class c is chosen that maximizes the following a posteriori class probability based on the Bernoulli model described above: Pr [ c d ] = f k π ( c ) t d F k ( f ( c · t ) ) n ( d , t ) c π ( c ) t d F k ( f ( c * t ) ) n ( d , t ) ,
    Figure US20010037324A1-20011101-M00016
  • [0155]
    where π is the prior distribution on the classes. Let c.(d) be the “true” class of d∈V, then N k = d N k ( d ) , where N k ( d ) = { 1 , c c * ( d ) : Pr [ c d , F k ] > Pr [ c * ( d ) d , F k ] 0 , otherwise .
    Figure US20010037324A1-20011101-M00017
  • [0156]
    In effect, the overall plot is constructed of the faction of documents incorrectly classified against the number of features used by averaging the per-document function written above. For a fixed document in the model validation set, the class for that document is first estimated based only on the highest Fisher value term in the order. If the class estimate is erroneous, the error is 1 for that document. If the class estimate is correct, then an error rate of 0 (zero) is plotted for the term number 1, for that document.
  • [0157]
    Further terms in the Fisher value order are added, one at a time, to the estimation process, to add points to the plot of Nk(d). Thus, a plot of the error rate versus term number function for any one particular document might appear as shown in FIG. 5. Typically, the document will be erroneously classified until some number m of terms are employed in the class estimation. Accordingly, an error rate of 1 is plotted, up to m terms, upon which the document is correctly classified and the error plot drops to 0. As further terms (in Fisher value order) are added to the class estimation, eventually an added term may be a noise term and may cause misclassification of the document, upon which the error jumps back to 1. Thereafter, further terms may cause fluctuations in the Nk(d) function, between 0 and 1.
  • [0158]
    The process is repeated for each document in the model validation set 44, such that a function of the error rate versus number of terms is generated for each document. The functions for all of the documents in the model validation set are added together to provide an overall error rate versus term number function that appears, for example, as shown in FIG. 6., which is discussed in more detail below under the section titled “Feature Selection”. In preferred embodiments, this aggregate error for all required number of feature terms is computed while scanning each validation document only once through the whole process.
  • [0159]
    The aggregate or average error will typically decrease steeply as terms are initially added to the feature set, reach a minimum, and then show an upward trend. Suppose k* is the smallest number of feature terms for which Nk achieves (close to) its global minimum. Then these k* terms are picked as features for the intermediate node of the taxonomy under discussion. In this manner, a generally distinct set of feature terms are derived for classification at each intermediate node in the taxonomy. However, it will be understood that other means for determining a cut-off point in the order of discriminating powers may be employed, including but not limited to defining a preset number of terms as the cut-off point.
  • [0160]
    Extracting Document Signatures
  • [0161]
    Up to a point, the user can sift a query response based only on the topic paths. However, even the leaf classes are necessarily coarser than individual documents. Support is therefore needed to browse quickly through many documents without looking into the documents in detail. Most search engines attach a few lines from each document. Often these are the title and first few lines, or they are sentences with the most search terms. For many documents, better keyword extraction is needed. Moreover, it would be more advantageous for these signatures to be extracted relative to a node in the taxonomy.
  • [0162]
    Given this reference node c, one approach is to concatenate the training documents vassociated with c into a super document dc and then rank the terms t∈dc in decreasing order of the number of standard deviations that n(d,t) is away from θ(c,t). Here, the previously described simplistic document model may not suffice. As mentioned above, a term that has occurred once in a document is more likely to occur again. Since the Bernoulli model does not take this into account, frequent terms often remain surprising all along the taxonomy path.
  • [0163]
    Matters are improved by moving to another simple model, the binary model. First, consider a single test document d, and consider t∈d. If the observed fraction of training documents in class c containing term t at least once is θ(c,t), all t∈d are sorted in increasing order of θ(c,t) and report the top few. Second, if there are l>1 test documents, then the fraction φ(t) is found that contains t at least once, and the terms are sorted in increasing order of ( θ ( c , t ) - φ ( t ) ) l θ ( c , t ) ( 1 - θ ( c , t ) )
    Figure US20010037324A1-20011101-M00018
  • [0164]
    before presenting the results. Both, in fact, correspond to p-values computed using the normal approximation to the binomial distribution.
  • [0165]
    Data Structures and Pseudocode
  • [0166]
    Preferred embodiments of the current invention have the following distinctive features as an “industrial strength” topic analyzer:
  • [0167]
    Thousands of classes and millions of documents can be handled. The current limits in a prototype implementation are 216 classes in the taxonomy, 232 unique tokens and 232 documents. A reasonable assumption is made that a simple pointer representation of the taxonomy together with a few words of data per class can be held in the computer memory at all times.
  • [0168]
    Training has near real-time response, as needed by crawling and indexing applications. Training makes a single pass over the corpus.
  • [0169]
    The prototype permits efficient incremental updates to a fixed taxonomy with new documents, deletion of documents, or moving documents from one class to another. With some more work, it is also possible to reorganize entire topic sub-trees.
  • [0170]
    System modules, according to preferred embodiments of the invention, include:
  • [0171]
    A mapping from textual terms to numeric term ID's (called TID's), which may be derived using a global counter, or a suitable hash function. A reverse mapping is also maintained.
  • [0172]
    Similar mappings between classes and numeric class ID's (called CID's).
  • [0173]
    A pointer-based tree data structure in memory for storing class-specific information, such as the number of training documents, etc.
  • [0174]
    A module for statistics.
  • [0175]
    A module for feature selection.
  • [0176]
    A module for applying the classifier on a test/new document.
  • [0177]
    The last three modules are described in detail next.
  • [0178]
    Statistics Collection
  • [0179]
    The goal of this module is to collect term statistics from a training document and dispense with it as fast as possible. After simple stopword filtering and stemming while scanning, the document is converted to a sequence of 32-bit TID's (term ID's). The main table, for example, maintained on a computer readable disk, is a frequency table, such as that shown in FIG. 7. TID corresponds to a term that occurs in some document belonging to a class corresponding to KCID (child or kid class ID). The Parent Class ID (PCID) represents the parent of KCID (zero if KCID is the root). The class identifications CID's (KCID's and PCID's) are numbered from one onwards. The explicit presence of PCID is only to simplify the current disclosure. In an actual system, the class tree data structure is preferably always available to map from KCID's to PCID's, and, therefore, PCID does not need to be explicitly stored in the table.
  • [0180]
    In the statistics collection phase the main table is kept sorted on the keys TID and KCID. There is another unsorted table with the same row format. There are four other numeric fields per row. All of these four numbers are additive over documents, so for each document d and term t, a row is appended to the unsorted table, with SMC set to one, SNC set to the number of times t occurred in d, called n(d,t), SF1 set to n(d, t)/Σrn(d,t), and SF2 set to SF12. SMC is used in the binary model, while SNC is needed in the Bernoulli model.
  • [0181]
    This approach trades off space for time, and the unsorted table grows rather quickly, but with a lot of duplicate keys. Depending on how much disk space exists, once in a while the system sorts and aggregates the unsorted table and then merges the result into the main sorted table. Various simple heuristics may be used to decide when to initiate a sort-merge. In one implementation, processing documents stops while the sort-merge is in progress. To meet tough real-time requirements, one can open a new frequency table and fork a thread, perhaps on another processor, to aggregate the last run while more documents continue to be accepted.
  • [0182]
    An indexed access approach could be used instead of the frequency table. As each document is scanned, the system would look up on the (TID, KCID) key and update SMC. SNC, SF1 and SF2. That would result in index lookups and random IO, potentially for every term in the training set. For large corpora, it is far more efficient to append statistics in a logged fashion as in the preferred embodiment. The frequency table is a temporary file and no direct indexed access to it is actually required later. Another benefit is compactness: this is the most space-intensive phase of training, and the storage overheads of indexed access are avoided, while explicit control of compaction is obtained. The space overhead of storing TID redundantly is moderate, as the rest of each row is already 18˜bytes long.
  • [0183]
    Computing Fisher Indices
  • [0184]
    Before beginning feature selection, the frequency table is aggregated one last time, if needed to eliminate all duplicates. The frequency table is rewound and prepared for scanning. At this stage, all rows with the same TID are collected in a contiguous run going through all CID's where that TID occurred (see FIG. 7). Also, preparation takes place to output another file, called the fisher table. For the following description, a format shown in FIG. 8 is assumed. The format includes rows that are keyed by PCID and a floating point number FI (Fisher index), where for each fixed PCID the rows are sorted in decreasing order of FI. The last column is the TID (term ID) whose corresponding PCID and FI are the first and second columns.
  • [0185]
    Because TID is the primary key in the frequency table, as it is scanned, a sequence of runs are obtained, each run having a fixed TID. Associated with each topic node in memory, a few words of statistics are kept (derived from SMC, SNC, etc.). When a run is started for a given TID, these statistics are cleared. As the various KCID's are scanned for the given TID in the frequency table, the node corresponding to the KCID in the taxonomy is located, and these statistics are updated. In a large taxonomy, very few of the nodes will be updated during a run. If a node is updated, its parent will be updated as well. These statistics efficiently can, therefore, be reset after each run.
  • [0186]
    When the run for a given TID completes, exploring only the updated nodes, the Fisher index of that term is computed for every internal node (identified by its PCID) in the taxonomy as described in the section titled “Feature and Noise Terms.” For each of these PCID's. a row is appended to the Fisher table. Next, the Fisher table is sorted on the key (PCID.FI). This collects all PCID's into contiguous segments, and for each PCID, orders terms by decreasing values of FI.
  • [0187]
    Consider now the case in which, for each internal topic c, the number k*(c) of features to pick is specified to TAPER directly. (The next section discusses how k*(c) is determined in one pass over the portion of the training documents set apart earlier for model pruning.) Given k*(c), the sorted Fisher table is scanned while copying the first k*(c) rows for the run corresponding to class c to an output table and discarding the remaining terms. This involves completely sequential IO.
  • [0188]
    As shown in FIG. 10, once feature selection is performed on the Fisher table, both the frequency and fisher table are sorted once again, this time with (PCID,TID) as the key. After these sorts, a merge is performed. Rows of the Fisher table are considered one by one. For each row, once the beginning of a key-matched row of the frequency table is found, the row is read as long as the key remains unchanged, constructing a vector in memory where each element has the form (KCID,SMC,SNC). This buffer is then written into a hash table on disk.
  • [0189]
    Feature Selection
  • [0190]
    Given terms in decreasing Fisher index order for a fixed class c, it is desirable to find a good value for k*, the number of chosen features, as described in the section titled “Selecting a cut-off.” In a preferred embodiment, this is done in only one pass over the validation documents in set V. FIG. 9 shows an illustration of the organization of the table of terms and their discrimination powers in one preferred embodiment. The table of terms is used in the feature selection process.
  • [0191]
    The technique used holds Nk (the number of incorrectly classified documents as a function of k, the number of features used) in memory as against scanning the validation documents for different values of k. The following is pseudocode:
    Naïve pseudocode:
    For all or suitably many values of k
    Prepare models with only k features
    For each validation document d ε V,
    Determine Vk (d)
    IO-efficient pseudocode:
    Compute Fisher ordering and initialize Nk = 0 for k = 0,1,2, . . .
    For each validation document d ε V
    Compute Pr[c′|c, d, F0] for every child c′ using priors only
    If a wrong class has highest Pr[c′| . . .] increment N0
    For k = 1,2, . . .
    Check if the k -th feature occurs in d
    If so, find Pr[c′|c, d, Fk] using Pr[c′|c, d, Fk-1]
    If a wrong class has highest Pr[c′| . . .] increment Nk
  • [0192]
    Even if Nk is stored for every value of k, and the lexicon is of size 500,000 (a sample of 266,000 documents from Yahoo™ required 600,000), only 2MB is needed. As each document d is scanned, Nk (d) is aggregated into the Nk array.
  • [0193]
    Earlier, it was discussed that indexed statistics could be computed, given the optimal number of k* of terms at each node. Here, the discussion is directed to how the k* is determined. The documents in the validation set are considered one by one. Each document d has a pre-assigned “correct” class label c*. The parent c of this class node is located in the topic tree. Recall that c has associated with it a ranked list of terms. These are intersected with the terms in d, and the common terms are sorted by the rank.
  • [0194]
    Now a sequence of progressively more detailed classifiers are constructed at c. The first one has only one feature, the top-ranked term in d. The second classifier has the top two features, etc. For each such classifier, it is checked whether d is correctly routed from c to c*.
  • [0195]
    This gives a sequence of zero-one readings (i.e. zero if correct, and one if not) on a scale indexed by the rank of the last included term. The following is an example sequence for a particular document:
  • [0196]
    0-29 1 (classifiers indexed 0-29 did not place d in c*)
  • [0197]
    29-500 0 (classifiers that used terms ranked 0 through k, for k=29, 30, . . . 500 did place d in c*)
  • [0198]
    500-550 1 (classifiers that included even more terms went wrong again, because they included noise)
  • [0199]
    Attached with each internal node, such as c above, is a file in which such sequences are appended for each validation document d. Once processing of (c, c*) is complete, processing continues up the tree, etc., until the processing of d is completely finished. These sequence files are then sorted on the index and a running sum is computed to generate the error rate curve as illustrated in FIG. 6.
  • [0200]
    Updates to the Database
  • [0201]
    For a batch job, the large frequency and fisher tables can now be deleted, leaving the relatively smaller indexed topic statistics and the much smaller term-to-TID maps. If the system is used to determine where new documents will be added to classes, it is necessary to preserve the frequency table. It continues to be used in the same way as before: rows are appended and occasionally it is compacted to aggregate duplicate keys. Running feature selection integrates the new data into the indexed statistics.
  • [0202]
    Like running statistics generation for a relational server, feature selection is not an interactive operation. For example, on a database with 2000 classes, with an average of 150 documents per class, and an average of 100 terms per document, it may take a couple of hours. So feature selection is invoked only when there is reason to believe that the refreshed statistics will improve classification. In further preferred embodiments, such times to perform feature selection is automatically determined, for example, based on the occurrence of one or more predetermined events.
  • [0203]
    Another issue is deletion of documents and moving of documents from one class to another (perhaps because classification was poor or erroneous for those documents). Since feature selection is always preceded by a frequency table aggregation, negative “correction” entries may be placed in it. That is, a frequency table row is produced, corresponding to each term in the deleted document, and SNIC, SNC, SF1 and SF2 are negated for the class(es) from which the document is being deleted. Here, it cannot be ensured that the document was originally included in the aggregate, but that can be done by preserving ID's for training documents.
  • [0204]
    A more difficult issue is the reorganization of the taxonomy itself. Notice that in TAPER, a parent class inherits, in an additive fashion, the statistics of its children, since each training document generates rows for each topic node from the assigned topic up to the root. The preferred means of reorganization therefore involves reassigning some CID's and writing out a new frequency table with some negative “correction” entries.
  • [0205]
    For example, consider detaching a subtree under node c1 and attaching it elsewhere under node c2. Statistics at or above the least common ancestor c, of c1 and c2 remain unchanged. Negative (respectively, positive) rows are appended to the frequency table corresponding to all classes on the path between cl inclusive and c1 (respectively, c2) exclusive. Finally, the parent and child links have to be modified in the taxonomy tree.
  • [0206]
    Classification
  • [0207]
    The rationale for the data organization described above is now explained by a description of the procedure of classifying a new or test document. When analyzing a new or test document, the taxonomy and associated pre-computed statistics are first loaded, and then the document is submitted to the system in the form of a map from terms to frequencies. In the present model, the probability that the document is generated by the root topic is 1, by definition, and decreases down any path in the taxonomy. Accordingly, the user also specifies a probability cut-off for topics reported back as close matches.
  • [0208]
    Consider the document d at some internal node c0 with children c1,c2, . . . . The system first intersects d with the feature set at c0, then, for each retained term t∈d∩F(c0), and then the system looks up the model parameters for subtopics c1, c2, etc. It is thus best for both space and IO efficiency to index the statistics by (c0, t) and include in the record a vector of statistics for each child ci, for i=1,2, . . . , of node c0. The obvious pseudocode has to be slightly modified to reflect this optimization (i.e., pc denotes log probabilities).
    Naïve index lookup:
    For each child c, of c0, i = 1,2 . . .
    Initialize pc, to 0
    For each term t ε d ∩ F(c0)
    Lookup statistics for term t for class c,
    Update pc,
    Normalize Σ,ep,, to 1 and add pc0 to each pc,.
    Optimized index lookup:
    Initialize all pc, to 0
    For each term t ε d ∩ F(c0)
    Skip if key (c0, t) is not in index
    Otherwise, retrieve record for (c0, t)
    For each c, that appears in the record
    Update pc,
    Normalize etc.
  • [0209]
    Caching and Batch Classification
  • [0210]
    Two additional optimizations may be needed in the above technique to rapidly compute class estimates for test/new documents. First, observe that in practice, most terms in a document will not be found in F(c0). It can be wasteful to frequently look up the index on disk only to discard terms. Since few terms are found useful, it is relatively acceptable to look up these statistics on disk. Thus, a fast cache of feature TID's, even without any attached statistics, just to check if a TID is a feature or not, greatly increases speed.
  • [0211]
    Second, given not one, but several documents to classify, it is desirable to amortize lookup of common terms by using a B-Tree access approach on the statistics (rather than a hash table), pre-sorting the TID's and making indexed scans on the statistics table. It is to be understood that any such locality-based performance optimizations and reordering of accesses to disk data are obvious to one trained in the art.
  • [0212]
    Incomplete Classification
  • [0213]
    With some documents, it may not be possible to narrow down the topic to a leaf node of the taxonomy. For example, there may be strong elements of both “Art:Paintina” and “Arts:Photography” in a document, so that the best classification should be just “Arts.” Various schemes may be proposed for dealing with such situations. In a preferred embodiment the system can trade off between the benefit of narrowing down the topic against the cost of doing so with low confidence.
  • [0214]
    To illustrate this, it will suffice to consider the case of a three-node taxonomy with the root c0 having two children c1,c2, where the goal, for a given document d, is to decide whether to stop at c0, or to pick between c1 and c2. Intuitively, it is pointless to make the finer distinction if it is incorrect too often. This can be quantified in a variety of ways. Suppose the classifier picks c1, the fraction of training documents in that class is π1, and somehow it is known that the classifier is correct in this case with probability p1. One possible estimate is Pr[d∈c1|d∈c0]. This can also be combined in various ways with the fraction of correct classifications over a cross-validation set. Then the “expected” factor by which the scope is narrowed down is p 1 π 1 + ( 1 - p 1 ) .
    Figure US20010037324A1-20011101-M00019
  • [0215]
    If this quantity is smaller than 1, the system stops at c0 in a preferred embodiment. It is to be understood that any other such test of confidence, with the purpose of stopping the classifier prematurely, is within the spirit of the present invention.
  • [0216]
    Performance (Speed and Accuracy) Example
  • [0217]
    We used three data sets for evaluation: the Reuters newswire widely used in the IR community, a portion from the US Patent database, and a sample from Yahoo™.
  • [0218]
    The experimental computers were between 133 and 266 MHz, with 128-256 MB of memory. Once a document is in memory, typical training time is 140ìs and typical testing time was 30ìs. Training and testing on Reuters takes 20 minutes overall. The Yahoo™ sample, with 2118 classes and 266,000 documents, takes 19 hours to train. Bernoulli was found to be superior to binary for all our experiments.
  • [0219]
    The Patent data set had three intermediate nodes below the root and twelve leaf nodes. A few hundred terms out of the lexicon of about 30,000 terms were sufficient to minimize error; the overall accuracy to the leaves was 66% (i.e., 66% patents were correctly classified) and the accuracy at the root node was 75%. For Reuters, the accuracy was 87%, as against the earlier best known technique's accuracy of 81% by Apte et al.
  • [0220]
    The best features in the patent taxonomy for the root and its three children are listed in Table 1 below. Observe how distinct the sets are.
    TABLE 1
    Patent: signal, modulate, motor, receive, antenna, telephone, transmit,
    frequency, demodulate, current, voltage, data . . .
    Communication: Electricity: motor, Electronics: amplifier.
    antenna. telephone, heat, voltage, oscillator, input,
    modulator, transistor, output, output, frequency,
    demodulator, circuit, . . . transistor, . . .
    signal, . . .
  • [0221]
    The feature selection technique is applied to find salient differences between various interesting sets of documents. One application is to find descriptions for clusters in unsupervised document clustering. For example, the query mouse provides hundreds of responses from the US patent database. Clustering the responses and applying TAPER with the clusters treated as classes yields the automatically generated cluster descriptions shown in Table 2 below.
    TABLE 2
    Tissue, thymus, transplanted, hematopoietic, treatment, organ, trypsin, . . .
    Computer, keyboard, hand, edge, top, location, keys, support,
    cleaning, . . .
    Point, select, environment, object, display, correspondence. image, . . .
  • [0222]
    These “cluster digests” clearly identify different contexts of the occurrence of “mouse” and help the user easily refine the query.
  • [0223]
    As yet another example application of the TAPER system, the patent portfolios of two assignees, or one assignee over two different intervals, were compared. For example, a comparison between “Sun Microsystems” and “Silicon Graphics” gave the result shown in Table 3 below.
    TABLE 3
    Sun Microsystems Silicon Graphics
    General purpose Information processing
    programmable digital system organization
    computer systems Data representation,
    Electrical computers and computer graphics
    data processing system Sufrace detail, texture
    Integrated circuit, Adjusting resolution or
    processor level of detail
  • [0224]
    Note how the commonality between the assignees (“UNIX workstations”) is filtered out and the difference brought out.
  • [0225]
    An even more interesting example is comparing Intel patents in 1993-94 with those in 1994-95. TAPER detects a new line of research and patenting in the latter year, as shown in Table 4 below.
    TABLE 4
    Intel, 1993-94 Intel, 1994-95
    General purpose Interactive television
    programmable digital bandwidth reduction
    computer systems system
    Chip, fabrication, counter, Involving difference
    input transmission
    Field or frame difference
  • [0226]
    Note that it is possible to get coherent phrases in this case because the patent meta-data stores these as effectively single terms.
  • [0227]
    In a typical application, a taxonomy will be initially designed by hand, and training documents obtained. Once training is completed, the accuracy of the system can be estimated by comparing the class output of the classifier 50 with the known classifications of the testing documents 34 (FIG. 3). If the accuracy is inadequate, a further training procedure, using a different collection of documents, or a reorganization of the taxonomy, may be carried out to retrain the system. Once satisfactory accuracy is achieved, the system may be used for a number of purposes such as search, filtering, and indexing as described above.
  • [0228]
    Broadly, the current invention can be distinguished from prior work in its use of a context dependent statistics, and its emphasis on scalability and speed in dealing with corpora ranging into tens to hundreds of gigabytes, the use of efficient disk data structures, and efficient update mechanisms. The present invention has focused on techniques that have good statistical foundation while remaining within almost linear time and one pass over the corpus, even when doing feature selection simultaneously for many nodes in a large topic taxonomy.
  • [0229]
    Having thus described exemplary embodiments of the present invention, it should be understood by those skilled in the art that the above disclosures are exemplary only and that various other alternatives, adaptations and modifications may be made within the scope of the present invention. The presently disclosed embodiments are to be considered in all respects as illustrative and not restrictive. The scope of the invention being indicated by the appended claims, rather than the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are, therefore, intended to be embraced therein.

Claims (48)

    What is claimed is:
  1. 1. A process for classifying new documents containing features under nodes defining a multilevel taxonomy, based on features derived from a training set of documents that have been classified under respective nodes of the taxonomy, the process comprising:
    associating a respective set of features with each one of said plurality of nodes, each given set of features comprising a plurality of features that are in at least one training document classified under the associated node; and
    classifying each new document under at least one node, based on the set of features associated with said at least one node.
  2. 2. A process as recited in
    claim 1
    , wherein said step of associating comprises the steps of:
    determining a discrimination value for each term in at least one training document which is classified under each one of a plurality of the nodes of the taxonomy; and
    determining a minimum discrimination value for each of said plurality of nodes;
    wherein the features in each given set of features have discrimination values equal to or above the minimum discrimination value determined for the node associated with the given set of features.
  3. 3. A process as recited in
    claim 1
    , wherein said step of classifying comprises:
    scanning each new document to determine the features in the document; and
    defining, for each of said plurality of said plurality of nodes and for each new document, the probability that the new document is classified under the node, based on the set of features associated with the node and the features in the document.
  4. 4. A process as recited in
    claim 3
    , wherein said step of defining the probability comprises the step of applying a statistical model to define said probability that features in each given new document would occur at the frequency at which they do occur in the given new document.
  5. 5. A process as recited in
    claim 4
    , wherein said statistical model comprises a Bernoulli model.
  6. 6. A process as recited in
    claim 4
    , wherein said statistical model comprises a Poisson model.
  7. 7. A process as recited in
    claim 4
    , wherein said step of classifying further comprises the step of assigning each given new document to at least one respective node in at least one level of the taxonomy, wherein the at least one node to which each given new document is assigned is the node for which the defined probability is above a predefined threshold among all of the nodes at the same level in the taxonomy.
  8. 8. A process as recited in
    claim 7
    , wherein the at least one node to which each given new document is assigned is the node for which the defined probability is maximum among all of the nodes at the same level in the taxonomy.
  9. 9. A process as recited in
    claim 8
    , wherein said step of assigning each given new document to at least one respective node in at least one level of the taxonomy comprises the step of assigning each given new document to at least one respective node in each of a plurality of levels of the taxonomy.
  10. 10. A process as recited in
    claim 2
    , wherein said step of selecting a set of features comprises selecting features that are in a plurality of training documents classified under the associated node and that have discrimination values equal to or above the minimum discrimination value.
  11. 11. A process as recited in
    claim 2
    , wherein said step of selecting a set of features comprises selecting features that are in all of training documents classified under the associated node and that have discrimination values equal to or above the minimum discrimination value.
  12. 12. A process as recited in
    claim 2
    , wherein said step of determining a discrimination value comprises determining a discrimination value for each feature in a plurality of training documents which are classified under each one of a plurality of the nodes of the taxonomy.
  13. 13. A process as recited in
    claim 2
    , wherein said step of determining a discrimination value comprises determining a discrimination value for each feature in all of the training documents which are classified under each one of a plurality of the nodes of the taxonomy.
  14. 14. A process as recited in
    claim 2
    , wherein said step of determining a discrimination value for each feature comprises determining a Fisher value for each feature, based on the equation:
    Fisher ( t ) = c 1 · c 2 ( μ ( c 1 , t ) - μ ( c 2 , t ) ) 2 c 1 / c ( x ( d , t ) - μ ( c , t ) ) 2
    Figure US20010037324A1-20011101-M00020
    where: t represents a term, d represents a document,
    μ(c,t)=1/|c|Σd∈c X(d,t), and
    x(d,t) =the occurrence rate of t in d.
  15. 15. A process as recited in
    claim 1
    , wherein said step of associating a respective set of features with each node comprises the step of determining the number of features to associate with each respective node.
  16. 16. A process as recited in
    claim 15
    , wherein said step of associating a respective set of features with each given node comprises the steps of:
    ranking, by discrimination power, each of a plurality of features that are in at least one training document classified under the each given node;
    providing an optimal number N of features for each given node; and
    defining the set of features associated with a given node as the features ranked highest to the Nth highest in said step of ranking.
  17. 17. A process as recited in
    claim 16
    , wherein said step of providing an optimal number N comprises the step of determining the number N for each given node based on a test set of documents.
  18. 18. A process as recited in
    claim 1
    , further comprising the step of displaying, for given node of a plurality of nodes of the taxonomy, a signature comprising at least one feature associated with the documents classified under the given node.
  19. 19. A process as recited in
    claim 18
    , wherein said signature for each given node comprises a plurality of features associated with the documents classified under the given node.
  20. 20. A process as recited in
    claim 18
    , wherein said signature for each given node comprises a plurality of features that occur in the documents classified under the given node, but which are determined to have a relatively low frequency of occurrence among documents under the given node.
  21. 21. A process for searching for documents relevant to a search query from a group of accessible documents containing terms, comprising the steps of:
    defining a multilevel taxonomy having a plurality of nodes, including a root node, at least one intermediate node associated with and under the root node and a plurality of terminal nodes associated with and under each intermediate node;
    classifying each one of a plurality of training documents with at least one of the terminal and intermediate nodes;
    determining a discrimination value for each term in at least one training document which is classified with each one of a plurality of the terminal and intermediate nodes of the taxonomy;
    determining a minimum discrimination value for each of said plurality of terminal and intermediate nodes;
    selecting a set of feature terms associated with each one of said plurality of terminal and intermediate nodes, said feature terms comprising terms that are in at least one training document classified with the associated node or any node under the associated node and that have discrimination values equal to or above the minimum discrimination value;
    receiving a search query:
    determining a plurality of search documents, each search document comprising one of the accessible document that is relevant to the search query;
    classifying each search document with at least one of the terminal and intermediate nodes of the taxonomy, based on the sets of feature terms associated with the terminal and intermediate nodes of the taxonomy;
    displaying a list of nodes with or under which said search documents are classified;
    selecting at least one of the displayed nodes; and
    displaying at least one search document classified under each selected node.
  22. 22. A process as recited in
    claim 21
    :
    wherein said search query comprises at least one search term; and
    wherein each search document comprises one of the accessible document that contain said at least one search term as one of the terms in the document.
  23. 23. A process as recited in
    claim 21
    :
    wherein, following said step of selecting at least one of the displayed nodes and prior to said step of displaying at least one search document, said process further includes the steps of displaying a second list of further nodes with or under which said search documents are classified; and selecting at least one of the displayed further nodes; and
    wherein said step of displaying at least one search document comprises the step of displaying at least one search document classified under each selected further node.
  24. 24. A process as recited in
    claim 21
    , wherein said step of displaying a list of nodes with or under which said search documents are classified further comprises the step of displaying signature terms associated with said search documents classified with or under each of said nodes in the displayed list.
  25. 25. A process as recited in
    claim 24
    , wherein said signature terms comprise a plurality of the most frequently occurring terms in the search documents that are also feature terms.
  26. 26. A classifier system for classifying new documents containing terms under nodes defining a multilevel taxonomy, based on feature terms derived from a training set of documents which are classified under respective nodes of the taxonomy, the system comprising:
    means for determining a discrimination value for each term in at least one training document which is classified under each one of a plurality of the nodes of the taxonomy;
    means for determining a minimum discrimination value for each of said plurality of nodes;
    means for selecting a set of feature terms associated with each one of said plurality of nodes, said feature terms comprising terms that are in at least one training document classified under the associated node and that have discrimination values equal to or above the minimum discrimination value; and
    means for classifying each new document under at least one node, based on the feature terms associated with said at least one node.
  27. 27. A system as recited in
    claim 26
    , wherein said means for classifying comprises:
    means for scanning each new document to determine the terms in the document; and
    means for defining, for each of said plurality of said plurality of nodes and for each new document, the probability that the new document is classified under the node, based on the feature terms associated with the node and the terms in the document.
  28. 28. A system as recited in
    claim 27
    , wherein said means for defining the probability comprises means for applying a Bernoulli model to define said probability for each of said plurality of nodes.
  29. 29. A system as recited in
    claim 26
    , wherein said means for selecting a set of feature terms comprises means for selecting terms that are in a plurality of training documents classified under the associated node and that have discrimination values equal to or above the minimum discrimination value.
  30. 30. A system as recited in
    claim 26
    , wherein said means for selecting a set of feature terms comprises means for selecting terms that are in all of training documents classified under the associated node and that have discrimination values equal to or above the minimum discrimination value.
  31. 31. A system as recited in
    claim 26
    , wherein said means for determining a discrimination value comprises means for determining a discrimination value for each term in a plurality of training documents which are classified under each one of a plurality of the nodes of the taxonomy.
  32. 32. A system as recited in
    claim 26
    , wherein said means for determining a discrimination value comprises means for determining a discrimination value for each term in all of the training documents which are classified under each one of a plurality of the nodes of the taxonomy.
  33. 33. A system as recited in
    claim 26
    , wherein said means for determining a discrimination value for each term comprises means for determining a Fisher value for each term, based on the equation:
    Fishers ( t ) = c1 , c2 ( μ ( c 1 , t ) - μ ( c 2 , t ) ) 2 c 1 / c ( x ( d , t ) - μ ( c , t ) ) 2
    Figure US20010037324A1-20011101-M00021
    where: t represents a term, d represents a document,
    μ(c,t)=1/|c|Σd∈cx(d,t), and
    x(d,t)=the occurrence rate of t in d.
  34. 34. A system for searching for documents relevant to a search query from a group of accessible documents containing terms, comprising:
    means for defining a multilevel taxonomy having a plurality of nodes, including a root node, at least one intermediate node associated with and under the root node and a plurality of terminal nodes associated with and under each intermediate node;
    means for classifying each one of a plurality of training documents with at least one of the terminal and intermediate nodes;
    means for determining a discrimination value for each term in at least one training document which is classified with each one of a plurality of the terminal and intermediate nodes of the taxonomy;
    means for determining a minimum discrimination value for each of said plurality of terminal and intermediate nodes;
    means for selecting a set of feature terms associated with each one of said plurality of terminal and intermediate nodes, said feature terms comprising terms that are in at least one training document classified with the associated node or any node under the associated node and that have discrimination values equal to or above the minimum discrimination value;
    means for receiving a search query;
    means for determining a plurality of search documents, each search document comprising one of the accessible document that is relevant to the search query;
    means for classifying each search document with at least one of the terminal and intermediate nodes of the taxonomy, based on the sets of feature terms associated with the terminal and intermediate nodes of the taxonomy;
    means for displaying a list of nodes with or under which said search documents are classified;
    means for selecting at least one of the displayed nodes; and
    means for displaying at least one search document classified under each selected node.
  35. 35. A system as recited in
    claim 34
    :
    wherein said search query comprises at least one search term; and
    wherein each search document comprises one of the accessible document that contain said at least one search term as one of the terms in the document.
  36. 36. A system as recited in
    claim 34
    , wherein said means for displaying a list of nodes with or under which said search documents are classified further comprises means for displaying signature terms associated with said search documents classified with or under each of said nodes in the displayed list.
  37. 37. A system as recited in
    claim 34
    , wherein said signature terms comprise a plurality of the most frequently occurring terms in the search documents that are also feature terms.
  38. 38. An article of manufacture comprising a computer program carrier readable by a computer and embodying one or more instructions executable by the computer to perform a process for classifying new documents containing terms under nodes defining a multilevel taxonomy, based on feature terms derived from a training set of documents which are classified under respective nodes of the taxonomy, the process comprising:
    determining a discrimination value for each term in at least one training document which is classified under each one of a plurality of the nodes of the taxonomy;
    determining a minimum discrimination value for each of said plurality of nodes;
    selecting a set of feature terms associated with each one of said plurality of nodes, said feature terms comprising terms that are in at least one training document classified under the associated node and that have discrimination values equal to or above the minimum discrimination value; and
    classifying each new document under at least one node, based on the feature terms associated with said at least one node.
  39. 39. An article as recited in
    claim 38
    , wherein said step of classifying comprises:
    scanning each new document to determine the terms in the document; and
    defining, for each of said plurality of said plurality of nodes and for each new document, the probability that the new document is classified under the node, based on the feature terms associated with the node and the terms in the document.
  40. 40. An article as recited in
    claim 39
    , wherein said step of defining the probability comprises the step of applying a Bernoulli model to define said probability for each of said plurality of nodes.
  41. 41. An article as recited in
    claim 38
    , wherein said step of selecting a set of feature terms comprises selecting terms that are in a plurality of training documents classified under the associated node and that have discrimination values equal to or above the minimum discrimination value.
  42. 42. An article as recited in
    claim 38
    , wherein said step of selecting a set of feature terms comprises selecting terms that are in all of training documents classified under the associated node and that have discrimination values equal to or above the minimum discrimination value.
  43. 43. An article as recited in
    claim 38
    , wherein said step of determining a discrimination value comprises determining a discrimination value for each term in a plurality of training documents which are classified under each one of a plurality of the nodes of the taxonomy.
  44. 44. An article as recited in
    claim 38
    , wherein said step of determining a discrimination value comprises determining a discrimination value for each term in all of the training documents which are classified under each one of a plurality of the nodes of the taxonomy.
  45. 45. An article as recited in
    claim 38
    , wherein said step of determining a discrimination value for each term comprises determining a Fisher value for each term, based on the equation:
    Fishers ( t ) = c1 , c2 ( μ ( c 1 , t ) - μ ( c 2 , t ) ) 2 c 1 / c ( x ( d , t ) - μ ( c , t ) ) 2
    Figure US20010037324A1-20011101-M00022
    where: t represents a term, d represents a document,
    μ(c,t)=1/|c|Σd∈cx(d,t), and
    x(d,t)=the occurrence rate of t in d.
  46. 46. An article of manufacture comprising a computer program carrier readable by a computer and embodying one or more instructions executable by the computer for searching for documents relevant to a search query from a group of accessible documents containing terms, comprising the steps of:
    defining a multilevel taxonomy having a plurality of nodes, including a root node, at least one intermediate node associated with and under the root node and a plurality of terminal nodes associated with and under each intermediate node;
    classifying each one of a plurality of training documents with at least one of the terminal and intermediate nodes;
    determining a discrimination value for each term in at least one training document which is classified with each one of a plurality of the terminal and intermediate nodes of the taxonomy;
    determining a minimum discrimination value for each of said plurality of terminal and intermediate nodes;
    selecting a set of feature terms associated with each one of said plurality of terminal and intermediate nodes, said feature terms comprising terms that are in at least one training document classified with the associated node or any node under the associated node and that have discrimination values equal to or above the minimum discrimination value;
    receiving a search query;
    determining a plurality of search documents, each search document comprising one of the accessible document that is relevant to the search query;
    classifying each search document with at least one of the terminal and intermediate nodes of the taxonomy, based on the sets of feature terms associated with the terminal and intermediate nodes of the taxonomy;
    displaying a list of nodes with or under which said search documents are classified;
    selecting at least one of the displayed nodes; and
    displaying at least one search document classified under each selected node.
  47. 47. An article as recited in
    claim 46
    , wherein said step of displaying a list of nodes with or under which said search documents are classified further comprises the step of displaying signature terms associated with said search documents classified with or under each of said nodes in the displayed list.
  48. 48. An article as recited in
    claim 47
    , wherein said signature terms comprise a plurality of the most frequently occurring terms in the search documents that are also feature terms.
US09777278 1997-06-24 2001-02-05 Multilevel taxonomy based on features derived from training documents classification using fisher values as discrimination values Abandoned US20010037324A1 (en)

Priority Applications (3)

Application Number Priority Date Filing Date Title
US5061197 true 1997-06-24 1997-06-24
US09102861 US6233575B1 (en) 1997-06-24 1998-06-23 Multilevel taxonomy based on features derived from training documents classification using fisher values as discrimination values
US09777278 US20010037324A1 (en) 1997-06-24 2001-02-05 Multilevel taxonomy based on features derived from training documents classification using fisher values as discrimination values

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
US09777278 US20010037324A1 (en) 1997-06-24 2001-02-05 Multilevel taxonomy based on features derived from training documents classification using fisher values as discrimination values

Publications (1)

Publication Number Publication Date
US20010037324A1 true true US20010037324A1 (en) 2001-11-01

Family

ID=26728460

Family Applications (2)

Application Number Title Priority Date Filing Date
US09102861 Expired - Fee Related US6233575B1 (en) 1997-06-24 1998-06-23 Multilevel taxonomy based on features derived from training documents classification using fisher values as discrimination values
US09777278 Abandoned US20010037324A1 (en) 1997-06-24 2001-02-05 Multilevel taxonomy based on features derived from training documents classification using fisher values as discrimination values

Family Applications Before (1)

Application Number Title Priority Date Filing Date
US09102861 Expired - Fee Related US6233575B1 (en) 1997-06-24 1998-06-23 Multilevel taxonomy based on features derived from training documents classification using fisher values as discrimination values

Country Status (1)

Country Link
US (2) US6233575B1 (en)

Cited By (112)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20030101191A1 (en) * 2001-11-28 2003-05-29 Yu-Chwin Lin Publication of commercial entity information and method for classifying commercial entity information
US20030163454A1 (en) * 2002-02-26 2003-08-28 Brian Jacobsen Subject specific search engine
US20030172357A1 (en) * 2002-03-11 2003-09-11 Kao Anne S.W. Knowledge management using text classification
US20030194689A1 (en) * 2002-04-12 2003-10-16 Mitsubishi Denki Kabushiki Kaisha Structured document type determination system and structured document type determination method
US6640224B1 (en) * 1997-12-15 2003-10-28 International Business Machines Corporation System and method for dynamic index-probe optimizations for high-dimensional similarity search
US20030217055A1 (en) * 2002-05-20 2003-11-20 Chang-Huang Lee Efficient incremental method for data mining of a database
US20040049514A1 (en) * 2002-09-11 2004-03-11 Sergei Burkov System and method of searching data utilizing automatic categorization
US20040111419A1 (en) * 2002-12-05 2004-06-10 Cook Daniel B. Method and apparatus for adapting a search classifier based on user queries
US20040117388A1 (en) * 2002-09-02 2004-06-17 Yasuhiko Inaba Method, apparatus and programs for delivering information
US20040153307A1 (en) * 2001-03-30 2004-08-05 Naftali Tishby Discriminative feature selection for data sequences
US20040220901A1 (en) * 2003-04-30 2004-11-04 Benq Corporation System and method for association itemset mining
US20040225645A1 (en) * 2003-05-06 2004-11-11 Rowney Kevin T. Personal computing device -based mechanism to detect preselected data
US20050027723A1 (en) * 2002-09-18 2005-02-03 Chris Jones Method and apparatus to report policy violations in messages
US20050086252A1 (en) * 2002-09-18 2005-04-21 Chris Jones Method and apparatus for creating an information security policy based on a pre-configured template
US20050132046A1 (en) * 2003-12-10 2005-06-16 De La Iglesia Erik Method and apparatus for data capture and analysis system
US20050187892A1 (en) * 2004-02-09 2005-08-25 Xerox Corporation Method for multi-class, multi-label categorization using probabilistic hierarchical modeling
US20060074632A1 (en) * 2004-09-30 2006-04-06 Nanavati Amit A Ontology-based term disambiguation
US20060074907A1 (en) * 2004-09-27 2006-04-06 Singhal Amitabh K Presentation of search results based on document structure
US20060155662A1 (en) * 2003-07-01 2006-07-13 Eiji Murakami Sentence classification device and method
US20060184549A1 (en) * 2005-02-14 2006-08-17 Rowney Kevin T Method and apparatus for modifying messages based on the presence of pre-selected data
US20060195415A1 (en) * 2005-02-14 2006-08-31 France Telecom Method and device for the generation of a classification tree to unify the supervised and unsupervised approaches, corresponding computer package and storage means
US7107254B1 (en) * 2001-05-07 2006-09-12 Microsoft Corporation Probablistic models and methods for combining multiple content classifiers
US20060224589A1 (en) * 2005-02-14 2006-10-05 Rowney Kevin T Method and apparatus for handling messages containing pre-selected data
US20060242190A1 (en) * 2005-04-26 2006-10-26 Content Analyst Comapny, Llc Latent semantic taxonomy generation
US20060242098A1 (en) * 2005-04-26 2006-10-26 Content Analyst Company, Llc Generating representative exemplars for indexing, clustering, categorization and taxonomy
US20060242140A1 (en) * 2005-04-26 2006-10-26 Content Analyst Company, Llc Latent semantic clustering
US20060282442A1 (en) * 2005-04-27 2006-12-14 Canon Kabushiki Kaisha Method of learning associations between documents and data sets
US20070005535A1 (en) * 2005-04-27 2007-01-04 Abdolreza Salahshour System and methods for IT resource event situation classification and semantics
US20070011154A1 (en) * 2005-04-11 2007-01-11 Textdigger, Inc. System and method for searching for a query
US7203669B2 (en) * 2003-03-17 2007-04-10 Intel Corporation Detector tree of boosted classifiers for real-time object detection and tracking
US20070156665A1 (en) * 2001-12-05 2007-07-05 Janusz Wnek Taxonomy discovery
US20070192442A1 (en) * 2001-07-24 2007-08-16 Brightplanet Corporation System and method for efficient control and capture of dynamic database content
US20070203903A1 (en) * 2006-02-28 2007-08-30 Ilial, Inc. Methods and apparatus for visualizing, managing, monetizing, and personalizing knowledge search results on a user interface
US20070234232A1 (en) * 2006-03-29 2007-10-04 Gheorghe Adrian Citu Dynamic image display
US20080059451A1 (en) * 2006-04-04 2008-03-06 Textdigger, Inc. Search system and method with text function tagging
US7363593B1 (en) * 2001-11-30 2008-04-22 Versata Development Group, Inc. System and method for presenting information organized by hierarchical levels
US20080114573A1 (en) * 2006-11-10 2008-05-15 Institute For Information Industry Tag organization methods and systems
US7376635B1 (en) * 2000-07-21 2008-05-20 Ford Global Technologies, Llc Theme-based system and method for classifying documents
WO2008097891A2 (en) * 2007-02-02 2008-08-14 Musgrove Technology Enterprises Llc Method and apparatus for aligning multiple taxonomies
US20080195567A1 (en) * 2007-02-13 2008-08-14 International Business Machines Corporation Information mining using domain specific conceptual structures
US7472114B1 (en) * 2002-09-18 2008-12-30 Symantec Corporation Method and apparatus to define the scope of a search for information from a tabular data source
US20090228499A1 (en) * 2008-03-05 2009-09-10 Schmidtler Mauritius A R Systems and methods for organizing data sets
US20090254540A1 (en) * 2007-11-01 2009-10-08 Textdigger, Inc. Method and apparatus for automated tag generation for digital content
US7650340B2 (en) * 1998-12-21 2010-01-19 Adobe Systems Incorporated Describing documents and expressing document structure
US7657104B2 (en) 2005-11-21 2010-02-02 Mcafee, Inc. Identifying image type in a capture system
US20100030752A1 (en) * 2008-07-30 2010-02-04 Lev Goldentouch System, methods and applications for structured document indexing
US7673344B1 (en) 2002-09-18 2010-03-02 Symantec Corporation Mechanism to search information content for preselected data
US20100076984A1 (en) * 2008-03-27 2010-03-25 Alkis Papadopoullos System and method for query expansion using tooltips
US7689614B2 (en) 2006-05-22 2010-03-30 Mcafee, Inc. Query generation for a capture system
US7730011B1 (en) 2005-10-19 2010-06-01 Mcafee, Inc. Attributes of captured objects in a capture system
US20100162347A1 (en) * 2008-12-22 2010-06-24 Ian Barile Adaptive data loss prevention policies
US20100185577A1 (en) * 2009-01-16 2010-07-22 Microsoft Corporation Object classification using taxonomies
US7774604B2 (en) 2003-12-10 2010-08-10 Mcafee, Inc. Verifying captured objects before presentation
US7814327B2 (en) 2003-12-10 2010-10-12 Mcafee, Inc. Document registration
US7818326B2 (en) * 2005-08-31 2010-10-19 Mcafee, Inc. System and method for word indexing in a capture system and querying thereof
US20100332481A1 (en) * 2002-09-18 2010-12-30 Rowney Kevin T Secure and scalable detection of preselected data embedded in electronically transmitted messages
US7899828B2 (en) 2003-12-10 2011-03-01 Mcafee, Inc. Tag data structure for maintaining relational data over captured objects
US7908260B1 (en) 2006-12-29 2011-03-15 BrightPlanet Corporation II, Inc. Source editing, internationalization, advanced configuration wizard, and summary page selection for information automation systems
US7907608B2 (en) 2005-08-12 2011-03-15 Mcafee, Inc. High speed packet capture
US7930540B2 (en) 2004-01-22 2011-04-19 Mcafee, Inc. Cryptographic policy enforcement
US7949849B2 (en) 2004-08-24 2011-05-24 Mcafee, Inc. File system for a capture system
US7958227B2 (en) 2006-05-22 2011-06-07 Mcafee, Inc. Attributes of captured objects in a capture system
US7962591B2 (en) 2004-06-23 2011-06-14 Mcafee, Inc. Object classification in a capture system
US7962490B1 (en) * 2008-01-07 2011-06-14 Amdocs Software Systems Limited System, method, and computer program product for analyzing and decomposing a plurality of rules into a plurality of contexts
US7996374B1 (en) 2008-03-28 2011-08-09 Symantec Corporation Method and apparatus for automatically correlating related incidents of policy violations
US7996373B1 (en) 2008-03-28 2011-08-09 Symantec Corporation Method and apparatus for detecting policy violations in a data repository having an arbitrary data schema
US8010689B2 (en) 2006-05-22 2011-08-30 Mcafee, Inc. Locational tagging in a capture system
US20110214080A1 (en) * 2010-02-26 2011-09-01 Microsoft Corporation Taxonomy Editor
US20110231411A1 (en) * 2008-08-08 2011-09-22 Holland Bloorview Kids Rehabilitation Hospital Topic Word Generation Method and System
US20110264699A1 (en) * 2008-12-30 2011-10-27 Telecom Italia S.P.A. Method and system for content classification
US8065739B1 (en) 2008-03-28 2011-11-22 Symantec Corporation Detecting policy violations in information content containing data in a character-based language
US20110314024A1 (en) * 2010-06-18 2011-12-22 Microsoft Corporation Semantic content searching
US8122005B1 (en) * 2009-10-22 2012-02-21 Google Inc. Training set construction for taxonomic classification
US20120078969A1 (en) * 2010-09-24 2012-03-29 International Business Machines Corporation System and method to extract models from semi-structured documents
US20120109642A1 (en) * 1999-02-05 2012-05-03 Stobbs Gregory A Computer-implemented patent portfolio analysis method and apparatus
US8205242B2 (en) 2008-07-10 2012-06-19 Mcafee, Inc. System and method for data mining and security policy management
WO2012174640A1 (en) * 2011-06-22 2012-12-27 Rogers Communications Inc. Systems and methods for creating an interest profile for a user
US20130046797A1 (en) * 2005-05-10 2013-02-21 Netseer, Inc. Methods and apparatus for distributed community finding
US8447722B1 (en) 2009-03-25 2013-05-21 Mcafee, Inc. System and method for data mining and security policy management
US8473442B1 (en) 2009-02-25 2013-06-25 Mcafee, Inc. System and method for intelligent state management
US20130166563A1 (en) * 2011-12-21 2013-06-27 Sap Ag Integration of Text Analysis and Search Functionality
US8489538B1 (en) 2010-05-25 2013-07-16 Recommind, Inc. Systems and methods for predictive coding
US8504537B2 (en) 2006-03-24 2013-08-06 Mcafee, Inc. Signature distribution in a document registration system
US8515972B1 (en) 2010-02-10 2013-08-20 Python 4 Fun, Inc. Finding relevant documents
US20130218904A1 (en) * 2012-02-22 2013-08-22 Salesforce.Com, Inc. System and method for inferring reporting relationships from a contact database
US8548170B2 (en) 2003-12-10 2013-10-01 Mcafee, Inc. Document de-registration
US8560534B2 (en) 2004-08-23 2013-10-15 Mcafee, Inc. Database for a capture system
US8645298B2 (en) 2010-10-26 2014-02-04 Microsoft Corporation Topic models
US8656039B2 (en) 2003-12-10 2014-02-18 Mcafee, Inc. Rule parser
US8667121B2 (en) 2009-03-25 2014-03-04 Mcafee, Inc. System and method for managing data and policies
US20140086497A1 (en) * 2012-06-14 2014-03-27 The Board of Trustees for the Leland Stanford, Junior, University Method and System for Optimizing Accuracy-Specificity Trade-offs in Large Scale Visual Recognition
US8700561B2 (en) 2011-12-27 2014-04-15 Mcafee, Inc. System and method for providing data protection workflows in a network environment
US8706709B2 (en) 2009-01-15 2014-04-22 Mcafee, Inc. System and method for intelligent term grouping
US20140172652A1 (en) * 2012-12-19 2014-06-19 Yahoo! Inc. Automated categorization of products in a merchant catalog
US8806615B2 (en) 2010-11-04 2014-08-12 Mcafee, Inc. System and method for protecting specified data combinations
US8826443B1 (en) 2008-09-18 2014-09-02 Symantec Corporation Selective removal of protected content from web requests sent to an interactive website
US8850591B2 (en) 2009-01-13 2014-09-30 Mcafee, Inc. System and method for concept building
US8849716B1 (en) * 2001-04-20 2014-09-30 Jpmorgan Chase Bank, N.A. System and method for preventing identity theft or misuse by restricting access
US8862580B1 (en) * 2004-03-01 2014-10-14 Radix Holdings, Llc Category-based search
US8862619B1 (en) 2008-01-07 2014-10-14 Amdocs Software Systems Limited System, method, and computer program product for filtering a data stream utilizing a plurality of contexts
US8935752B1 (en) 2009-03-23 2015-01-13 Symantec Corporation System and method for identity consolidation
US20150095017A1 (en) * 2013-09-27 2015-04-02 Google Inc. System and method for learning word embeddings using neural language models
US9110985B2 (en) 2005-05-10 2015-08-18 Neetseer, Inc. Generating a conceptual association graph from large-scale loosely-grouped content
US9245029B2 (en) 2006-01-03 2016-01-26 Textdigger, Inc. Search system with query refinement and search method
US9253154B2 (en) 2008-08-12 2016-02-02 Mcafee, Inc. Configuration management for a capture/registration system
US20160140207A1 (en) * 2014-11-14 2016-05-19 Symantec Corporation Systems and methods for aggregating information-asset classifications
US20160182669A1 (en) * 2014-12-22 2016-06-23 Here Global B.V. Optimal Coding Method for Efficient Matching Of Hierarchical Categories In Publish-Subscribe Systems
US9436726B2 (en) 2011-06-23 2016-09-06 BCM International Regulatory Analytics LLC System, method and computer program product for a behavioral database providing quantitative analysis of cross border policy process and related search capabilities
US9443018B2 (en) 2006-01-19 2016-09-13 Netseer, Inc. Systems and methods for creating, navigating, and searching informational web neighborhoods
US20170091590A1 (en) * 2013-03-15 2017-03-30 Sri International Computer vision as a service
US9785634B2 (en) 2011-06-04 2017-10-10 Recommind, Inc. Integration and combination of random sampling and document batching
US9817902B2 (en) 2006-10-27 2017-11-14 Netseer Acquisition, Inc. Methods and apparatus for matching relevant content to user intention

Families Citing this family (354)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP3597697B2 (en) * 1998-03-20 2004-12-08 富士通株式会社 Article summarizing apparatus and method
US6510429B1 (en) * 1998-04-29 2003-01-21 International Business Machines Corporation Message broker apparatus, method and computer program product
US20070294229A1 (en) * 1998-05-28 2007-12-20 Q-Phrase Llc Chat conversation methods traversing a provisional scaffold of meanings
US7711672B2 (en) * 1998-05-28 2010-05-04 Lawrence Au Semantic network methods to disambiguate natural language meaning
US8396824B2 (en) * 1998-05-28 2013-03-12 Qps Tech. Limited Liability Company Automatic data categorization with optimally spaced semantic seed terms
JP3665480B2 (en) * 1998-06-24 2005-06-29 富士通株式会社 Document organize apparatus and method
US6446061B1 (en) * 1998-07-31 2002-09-03 International Business Machines Corporation Taxonomy generation for document collections
US6665837B1 (en) * 1998-08-10 2003-12-16 Overture Services, Inc. Method for identifying related pages in a hyperlinked database
US20040230409A1 (en) * 1998-08-31 2004-11-18 Nutech Solutions, Inc. Method for performing social computation
GB9821787D0 (en) * 1998-10-06 1998-12-02 Data Limited Apparatus for classifying or processing data
US6460029B1 (en) * 1998-12-23 2002-10-01 Microsoft Corporation System for improving search text
US6751606B1 (en) * 1998-12-23 2004-06-15 Microsoft Corporation System for enhancing a query interface
US6601059B1 (en) 1998-12-23 2003-07-29 Microsoft Corporation Computerized searching tool with spell checking
US6721759B1 (en) * 1998-12-24 2004-04-13 Sony Corporation Techniques for spatial representation of data and browsing based on similarity
US6385619B1 (en) * 1999-01-08 2002-05-07 International Business Machines Corporation Automatic user interest profile generation from structured document access information
US7024416B1 (en) * 1999-03-31 2006-04-04 Verizon Laboratories Inc. Semi-automatic index term augmentation in document retrieval
US8275661B1 (en) 1999-03-31 2012-09-25 Verizon Corporate Services Group Inc. Targeted banner advertisements
WO2000058863A1 (en) * 1999-03-31 2000-10-05 Verizon Laboratories Inc. Techniques for performing a data query in a computer system
US8572069B2 (en) 1999-03-31 2013-10-29 Apple Inc. Semi-automatic index term augmentation in document retrieval
US6665681B1 (en) * 1999-04-09 2003-12-16 Entrieva, Inc. System and method for generating a taxonomy from a plurality of documents
US6564197B2 (en) * 1999-05-03 2003-05-13 E.Piphany, Inc. Method and apparatus for scalable probabilistic clustering using decision trees
US6442545B1 (en) * 1999-06-01 2002-08-27 Clearforest Ltd. Term-level text with mining with taxonomies
JP2000348041A (en) * 1999-06-03 2000-12-15 Nec Corp Document retrieval method, device therefor and mechanically readable recording medium
US6711585B1 (en) * 1999-06-15 2004-03-23 Kanisa Inc. System and method for implementing a knowledge management system
US6418434B1 (en) * 1999-06-25 2002-07-09 International Business Machines Corporation Two stage automated electronic messaging system
US7181438B1 (en) * 1999-07-21 2007-02-20 Alberti Anemometer, Llc Database access system
US6718363B1 (en) * 1999-07-30 2004-04-06 Verizon Laboratories, Inc. Page aggregation for web sites
US6578065B1 (en) * 1999-09-23 2003-06-10 Hewlett-Packard Development Company L.P. Multi-threaded processing system and method for scheduling the execution of threads based on data received from a cache memory
US6665656B1 (en) * 1999-10-05 2003-12-16 Motorola, Inc. Method and apparatus for evaluating documents with correlating information
US6651059B1 (en) * 1999-11-15 2003-11-18 International Business Machines Corporation System and method for the automatic recognition of relevant terms by mining link annotations
US8271316B2 (en) * 1999-12-17 2012-09-18 Buzzmetrics Ltd Consumer to business data capturing system
US7181454B1 (en) * 1999-12-29 2007-02-20 International Business Machines Corporation Asset locator
US6397211B1 (en) * 2000-01-03 2002-05-28 International Business Machines Corporation System and method for identifying useless documents
US7644057B2 (en) 2001-01-03 2010-01-05 International Business Machines Corporation System and method for electronic communication management
US6931403B1 (en) * 2000-01-19 2005-08-16 International Business Machines Corporation System and architecture for privacy-preserving data mining
US6701314B1 (en) * 2000-01-21 2004-03-02 Science Applications International Corporation System and method for cataloguing digital information for searching and retrieval
US6704727B1 (en) * 2000-01-31 2004-03-09 Overture Services, Inc. Method and system for generating a set of search terms
US6868525B1 (en) * 2000-02-01 2005-03-15 Alberti Anemometer Llc Computer graphic display visualization system and method
US6502091B1 (en) * 2000-02-23 2002-12-31 Hewlett-Packard Company Apparatus and method for discovering context groups and document categories by mining usage logs
WO2001067225A3 (en) * 2000-03-06 2002-01-17 Kanisa Inc A system and method for providing an intelligent multi-step dialog with a user
US6728701B1 (en) * 2000-04-18 2004-04-27 Claritech Corporation Method and apparatus for database retrieval utilizing vector optimization
US7356604B1 (en) * 2000-04-18 2008-04-08 Claritech Corporation Method and apparatus for comparing scores in a vector space retrieval process
US6742023B1 (en) 2000-04-28 2004-05-25 Roxio, Inc. Use-sensitive distribution of data files between users
US6704728B1 (en) * 2000-05-02 2004-03-09 Iphase.Com, Inc. Accessing information from a collection of data
US8478732B1 (en) 2000-05-02 2013-07-02 International Business Machines Corporation Database aliasing in information access system
US6711561B1 (en) * 2000-05-02 2004-03-23 Iphrase.Com, Inc. Prose feedback in information access system
US6745181B1 (en) * 2000-05-02 2004-06-01 Iphrase.Com, Inc. Information access method
US7310624B1 (en) * 2000-05-02 2007-12-18 International Business Machines Corporation Methods and apparatus for generating decision trees with discriminants and employing same in data classification
US7127450B1 (en) * 2000-05-02 2006-10-24 International Business Machines Corporation Intelligent discard in information access system
US6912525B1 (en) * 2000-05-08 2005-06-28 Verizon Laboratories, Inc. Techniques for web site integration
US6446083B1 (en) * 2000-05-12 2002-09-03 Vastvideo, Inc. System and method for classifying media items
US6697800B1 (en) * 2000-05-19 2004-02-24 Roxio, Inc. System and method for determining affinity using objective and subjective data
US6865600B1 (en) * 2000-05-19 2005-03-08 Napster, Inc. System and method for selecting internet media channels
US6876997B1 (en) * 2000-05-22 2005-04-05 Overture Services, Inc. Method and apparatus for indentifying related searches in a database search system
US7062561B1 (en) 2000-05-23 2006-06-13 Richard Reisman Method and apparatus for utilizing the social usage learned from multi-user feedback to improve resource identity signifier mapping
US7003517B1 (en) * 2000-05-24 2006-02-21 Inetprofit, Inc. Web-based system and method for archiving and searching participant-based internet text sources for customer lead data
US7082427B1 (en) 2000-05-24 2006-07-25 Reachforce, Inc. Text indexing system to index, query the archive database document by keyword data representing the content of the documents and by contact data associated with the participant who generated the document
US7096220B1 (en) 2000-05-24 2006-08-22 Reachforce, Inc. Web-based customer prospects harvester system
US7120629B1 (en) 2000-05-24 2006-10-10 Reachforce, Inc. Prospects harvester system for providing contact data about customers of product or service offered by business enterprise extracting text documents selected from newsgroups, discussion forums, mailing lists, querying such data to provide customers who confirm to business profile data
US7028250B2 (en) * 2000-05-25 2006-04-11 Kanisa, Inc. System and method for automatically classifying text
US6408277B1 (en) 2000-06-21 2002-06-18 Banter Limited System and method for automatic task prioritization
US8290768B1 (en) 2000-06-21 2012-10-16 International Business Machines Corporation System and method for determining a set of attributes based on content of communications
US9699129B1 (en) 2000-06-21 2017-07-04 International Business Machines Corporation System and method for increasing email productivity
US7389208B1 (en) * 2000-06-30 2008-06-17 Accord Solutions, Inc. System and method for dynamic knowledge construction
US6738759B1 (en) * 2000-07-07 2004-05-18 Infoglide Corporation, Inc. System and method for performing similarity searching using pointer optimization
US20020059219A1 (en) * 2000-07-17 2002-05-16 Neveitt William T. System and methods for web resource discovery
US6714941B1 (en) * 2000-07-19 2004-03-30 University Of Southern California Learning data prototypes for information extraction
JP2002041544A (en) * 2000-07-25 2002-02-08 Toshiba Corp Text information analyzing device
US6687696B2 (en) * 2000-07-26 2004-02-03 Recommind Inc. System and method for personalized search, information filtering, and for generating recommendations utilizing statistical latent class models
US6675159B1 (en) * 2000-07-27 2004-01-06 Science Applic Int Corp Concept-based search and retrieval system
US7062488B1 (en) * 2000-08-30 2006-06-13 Richard Reisman Task/domain segmentation in applying feedback to command control
US6647383B1 (en) * 2000-09-01 2003-11-11 Lucent Technologies Inc. System and method for providing interactive dialogue and iterative search functions to find information
US20040100022A1 (en) * 2000-09-25 2004-05-27 Pasquarelli Felice Antonio Puzzle
US20020143524A1 (en) * 2000-09-29 2002-10-03 Lingomotors, Inc. Method and resulting system for integrating a query reformation module onto an information retrieval system
US20050144114A1 (en) * 2000-09-30 2005-06-30 Ruggieri Thomas P. System and method for providing global information on risks and related hedging strategies
US7330850B1 (en) 2000-10-04 2008-02-12 Reachforce, Inc. Text mining system for web-based business intelligence applied to web site server logs
US7043531B1 (en) 2000-10-04 2006-05-09 Inetprofit, Inc. Web-based customer lead generator system with pre-emptive profiling
US7197470B1 (en) * 2000-10-11 2007-03-27 Buzzmetrics, Ltd. System and method for collection analysis of electronic discussion methods
US7185065B1 (en) 2000-10-11 2007-02-27 Buzzmetrics Ltd System and method for scoring electronic messages
US6694337B1 (en) * 2000-10-26 2004-02-17 Intel Corporation Synchronizing databases
US7027974B1 (en) 2000-10-27 2006-04-11 Science Applications International Corporation Ontology-based parser for natural language processing
US7200606B2 (en) * 2000-11-07 2007-04-03 The Regents Of The University Of California Method and system for selecting documents by measuring document quality
US6678694B1 (en) * 2000-11-08 2004-01-13 Frank Meik Indexed, extensible, interactive document retrieval system
US6640228B1 (en) * 2000-11-10 2003-10-28 Verizon Laboratories Inc. Method for detecting incorrectly categorized data
US7295991B1 (en) * 2000-11-10 2007-11-13 Erc Dataplus, Inc. Employment sourcing system
US7363308B2 (en) * 2000-12-28 2008-04-22 Fair Isaac Corporation System and method for obtaining keyword descriptions of records from a large database
US6766316B2 (en) 2001-01-18 2004-07-20 Science Applications International Corporation Method and system of ranking and clustering for document indexing and retrieval
US6892189B2 (en) 2001-01-26 2005-05-10 Inxight Software, Inc. Method for learning and combining global and local regularities for information extraction and classification
US7013289B2 (en) * 2001-02-21 2006-03-14 Michel Horn Global electronic commerce system
US20020133392A1 (en) * 2001-02-22 2002-09-19 Angel Mark A. Distributed customer relationship management systems and methods
US7213069B2 (en) * 2001-02-28 2007-05-01 Microsoft Corporation Category name service able to override the category name based on requestor privilege information
US20040098385A1 (en) * 2002-02-26 2004-05-20 Mayfield James C. Method for indentifying term importance to sample text using reference text
US6584470B2 (en) * 2001-03-01 2003-06-24 Intelliseek, Inc. Multi-layered semiotic mechanism for answering natural language questions using document retrieval combined with information extraction
US6985950B1 (en) 2001-03-06 2006-01-10 Microsoft Corporation System for creating a space-efficient document categorizer for training and testing of automatic categorization engines
US20020129342A1 (en) * 2001-03-07 2002-09-12 David Kil Data mining apparatus and method with user interface based ground-truth tool and user algorithms
US20020138492A1 (en) * 2001-03-07 2002-09-26 David Kil Data mining application with improved data mining algorithm selection
US20020169735A1 (en) * 2001-03-07 2002-11-14 David Kil Automatic mapping from data to preprocessing algorithms
US7136846B2 (en) 2001-04-06 2006-11-14 2005 Keel Company, Inc. Wireless information retrieval
US7155668B2 (en) * 2001-04-19 2006-12-26 International Business Machines Corporation Method and system for identifying relationships between text documents and structured variables pertaining to the text documents
US6947936B1 (en) * 2001-04-30 2005-09-20 Hewlett-Packard Development Company, L.P. Method for a topic hierarchy classification system
US6938025B1 (en) * 2001-05-07 2005-08-30 Microsoft Corporation Method and apparatus for automatically determining salient features for object classification
US7194483B1 (en) 2001-05-07 2007-03-20 Intelligenxia, Inc. Method, system, and computer program product for concept-based multi-dimensional analysis of unstructured information
US6826576B2 (en) * 2001-05-07 2004-11-30 Microsoft Corporation Very-large-scale automatic categorizer for web content
US8589413B1 (en) 2002-03-01 2013-11-19 Ixreveal, Inc. Concept-based method and system for dynamically analyzing results from search engines
US7627588B1 (en) 2001-05-07 2009-12-01 Ixreveal, Inc. System and method for concept based analysis of unstructured data
US7536413B1 (en) 2001-05-07 2009-05-19 Ixreveal, Inc. Concept-based categorization of unstructured objects
US6978266B2 (en) * 2001-05-07 2005-12-20 Microsoft Corporation Determining a rating for a collection of documents
US6970881B1 (en) 2001-05-07 2005-11-29 Intelligenxia, Inc. Concept-based method and system for dynamically analyzing unstructured information
US7024400B2 (en) * 2001-05-08 2006-04-04 Sunflare Co., Ltd. Differential LSI space-based probabilistic document classifier
US20020169872A1 (en) * 2001-05-14 2002-11-14 Hiroshi Nomiyama Method for arranging information, information processing apparatus, storage media and program tranmission apparatus
US6980984B1 (en) 2001-05-16 2005-12-27 Kanisa, Inc. Content provider systems and methods using structured data
US7117215B1 (en) 2001-06-07 2006-10-03 Informatica Corporation Method and apparatus for transporting data for data warehousing applications that incorporates analytic data interface
US7162643B1 (en) 2001-06-15 2007-01-09 Informatica Corporation Method and system for providing transfer of analytic application data over a network
US6820073B1 (en) * 2001-06-20 2004-11-16 Microstrategy Inc. System and method for multiple pass cooperative processing
US7003512B1 (en) * 2001-06-20 2006-02-21 Microstrategy, Inc. System and method for multiple pass cooperative processing
US7043506B1 (en) * 2001-06-28 2006-05-09 Microsoft Corporation Utility-based archiving
US7720842B2 (en) 2001-07-16 2010-05-18 Informatica Corporation Value-chained queries in analytic applications
US20030033263A1 (en) * 2001-07-31 2003-02-13 Reel Two Limited Automated learning system
US20030130993A1 (en) * 2001-08-08 2003-07-10 Quiver, Inc. Document categorization engine
US6609124B2 (en) * 2001-08-13 2003-08-19 International Business Machines Corporation Hub for strategic intelligence
US7257568B2 (en) * 2001-10-16 2007-08-14 Sizatola, Llc Process and system for matching products and markets
US7644102B2 (en) * 2001-10-19 2010-01-05 Xerox Corporation Methods, systems, and articles of manufacture for soft hierarchical clustering of co-occurring objects
US20030084066A1 (en) * 2001-10-31 2003-05-01 Waterman Scott A. Device and method for assisting knowledge engineer in associating intelligence with content
US20030115191A1 (en) * 2001-12-17 2003-06-19 Max Copperman Efficient and cost-effective content provider for customer relationship management (CRM) or other applications
US7206778B2 (en) 2001-12-17 2007-04-17 Knova Software Inc. Text search ordered along one or more dimensions
CA2470864C (en) * 2001-12-18 2014-08-12 Telstra New Wave Pty Ltd Information resource taxonomy
US7197493B2 (en) * 2001-12-21 2007-03-27 Lifestory Productions, Inc. Collection management database of arbitrary schema
US7162480B2 (en) * 2001-12-26 2007-01-09 Sbc Technology Resources, Inc. Usage-based adaptable taxonomy
US6978264B2 (en) * 2002-01-03 2005-12-20 Microsoft Corporation System and method for performing a search and a browse on a query
US7225183B2 (en) * 2002-01-28 2007-05-29 Ipxl, Inc. Ontology-based information management system and method
JP3860046B2 (en) * 2002-02-15 2006-12-20 インターナショナル・ビジネス・マシーンズ・コーポレーションInternational Business Maschines Corporation Program for information processing using a random sample hierarchical structures, systems and recording medium
US6820077B2 (en) 2002-02-22 2004-11-16 Informatica Corporation Method and system for navigating a large amount of data
US7343372B2 (en) * 2002-02-22 2008-03-11 International Business Machines Corporation Direct navigation for information retrieval
US20060004732A1 (en) * 2002-02-26 2006-01-05 Odom Paul S Search engine methods and systems for generating relevant search results and advertisements
US7716207B2 (en) * 2002-02-26 2010-05-11 Odom Paul S Search engine methods and systems for displaying relevant topics
US7340466B2 (en) * 2002-02-26 2008-03-04 Kang Jo Mgmt. Limited Liability Company Topic identification and use thereof in information retrieval systems
US7188107B2 (en) * 2002-03-06 2007-03-06 Infoglide Software Corporation System and method for classification of documents
US7031909B2 (en) * 2002-03-12 2006-04-18 Verity, Inc. Method and system for naming a cluster of words and phrases
KR100457375B1 (en) * 2002-03-19 2004-11-16 (주) 위즈도메인 Method for fast searching and displaying of patent genealogical status from a patent database
US7051009B2 (en) * 2002-03-29 2006-05-23 Hewlett-Packard Development Company, L.P. Automatic hierarchical classification of temporal ordered case log documents for detection of changes
JP4255239B2 (en) * 2002-03-29 2009-04-15 富士通株式会社 Document search method
US20030220917A1 (en) * 2002-04-03 2003-11-27 Max Copperman Contextual search
US20030225763A1 (en) * 2002-04-15 2003-12-04 Microsoft Corporation Self-improving system and method for classifying pages on the world wide web
US7085771B2 (en) * 2002-05-17 2006-08-01 Verity, Inc System and method for automatically discovering a hierarchy of concepts from a corpus of documents
US7167871B2 (en) * 2002-05-17 2007-01-23 Xerox Corporation Systems and methods for authoritativeness grading, estimation and sorting of documents in large heterogeneous document collections
US7296020B2 (en) * 2002-06-05 2007-11-13 International Business Machines Corp Automatic evaluation of categorization system quality
US7165068B2 (en) * 2002-06-12 2007-01-16 Zycus Infotech Pvt Ltd. System and method for electronic catalog classification using a hybrid of rule based and statistical method
US7016884B2 (en) * 2002-06-27 2006-03-21 Microsoft Corporation Probability estimate for K-nearest neighbor
US6990485B2 (en) * 2002-08-02 2006-01-24 Hewlett-Packard Development Company, L.P. System and method for inducing a top-down hierarchical categorizer
US7158983B2 (en) 2002-09-23 2007-01-02 Battelle Memorial Institute Text analysis technique
CN100378713C (en) * 2002-09-25 2008-04-02 微软公司 Method and apparatus for automatically determining salient features for object classification
US7418403B2 (en) * 2002-11-27 2008-08-26 Bt Group Plc Content feedback in a multiple-owner content management system
US8275811B2 (en) * 2002-11-27 2012-09-25 Accenture Global Services Limited Communicating solution information in a knowledge management system
US8572058B2 (en) * 2002-11-27 2013-10-29 Accenture Global Services Limited Presenting linked information in a CRM system
US20050014116A1 (en) * 2002-11-27 2005-01-20 Reid Gregory S. Testing information comprehension of contact center users
US9396473B2 (en) * 2002-11-27 2016-07-19 Accenture Global Services Limited Searching within a contact center portal
US20040100493A1 (en) * 2002-11-27 2004-05-27 Reid Gregory S. Dynamically ordering solutions
US7200614B2 (en) * 2002-11-27 2007-04-03 Accenture Global Services Gmbh Dual information system for contact center users
US7062505B2 (en) * 2002-11-27 2006-06-13 Accenture Global Services Gmbh Content management system for the telecommunications industry
US7769622B2 (en) * 2002-11-27 2010-08-03 Bt Group Plc System and method for capturing and publishing insight of contact center users whose performance is above a reference key performance indicator
US7395499B2 (en) * 2002-11-27 2008-07-01 Accenture Global Services Gmbh Enforcing template completion when publishing to a content management system
US7320000B2 (en) * 2002-12-04 2008-01-15 International Business Machines Corporation Method and apparatus for populating a predefined concept hierarchy or other hierarchical set of classified data items by minimizing system entrophy
US7035838B2 (en) * 2002-12-06 2006-04-25 General Electric Company Methods and systems for organizing information stored within a computer network-based system
US8037496B1 (en) 2002-12-27 2011-10-11 At&T Intellectual Property Ii, L.P. System and method for automatically authoring interactive television content
US20040133560A1 (en) * 2003-01-07 2004-07-08 Simske Steven J. Methods and systems for organizing electronic documents
US6961733B2 (en) * 2003-03-10 2005-11-01 Unisys Corporation System and method for storing and accessing data in an interlocking trees datastore
US8516004B2 (en) * 2003-09-19 2013-08-20 Unisys Corporation Method for processing K node count fields using an intensity variable
US7107264B2 (en) * 2003-04-04 2006-09-12 Yahoo, Inc. Content bridge for associating host content and guest content wherein guest content is determined by search
US7917483B2 (en) * 2003-04-24 2011-03-29 Affini, Inc. Search engine and method with improved relevancy, scope, and timeliness
US8495002B2 (en) 2003-05-06 2013-07-23 International Business Machines Corporation Software tool for training and testing a knowledge base
US6985920B2 (en) * 2003-06-23 2006-01-10 Protego Networks Inc. Method and system for determining intra-session event correlation across network address translation devices
US7644365B2 (en) * 2003-09-12 2010-01-05 Cisco Technology, Inc. Method and system for displaying network security incidents
US7313574B2 (en) 2003-10-02 2007-12-25 Nokia Corporation Method for clustering and querying media items
US7421458B1 (en) 2003-10-16 2008-09-02 Informatica Corporation Querying, versioning, and dynamic deployment of database objects
US7319998B2 (en) * 2003-11-14 2008-01-15 Universidade De Coimbra Method and system for supporting symbolic serendipity
US7444403B1 (en) 2003-11-25 2008-10-28 Microsoft Corporation Detecting sexually predatory content in an electronic communication
US20050114313A1 (en) * 2003-11-26 2005-05-26 Campbell Christopher S. System and method for retrieving documents or sub-documents based on examples
DE10356127A1 (en) * 2003-12-02 2005-07-07 Robert Bosch Gmbh Means for controlling a memory
US7254590B2 (en) * 2003-12-03 2007-08-07 Informatica Corporation Set-oriented real-time data processing based on transaction boundaries
US20050125280A1 (en) * 2003-12-05 2005-06-09 Hewlett-Packard Development Company, L.P. Real-time aggregation and scoring in an information handling system
US7340471B2 (en) 2004-01-16 2008-03-04 Unisys Corporation Saving and restoring an interlocking trees datastore
US20050177599A1 (en) * 2004-02-09 2005-08-11 Microsoft Corporation System and method for complying with anti-spam rules, laws, and regulations
RU2006133549A (en) * 2004-02-20 2008-05-20 ДАУ ДЖОУНС РЕЙТЕРЗ БИЗНЕС ИНТЕРЭКТИВ, Эл Эл Си (US) The system and method of intelligent search and retrieval
US7725414B2 (en) 2004-03-16 2010-05-25 Buzzmetrics, Ltd An Israel Corporation Method for developing a classifier for classifying communications
US7565369B2 (en) * 2004-05-28 2009-07-21 International Business Machines Corporation System and method for mining time-changing data streams
US7660463B2 (en) * 2004-06-03 2010-02-09 Microsoft Corporation Foreground extraction using iterated graph cuts
JP4254623B2 (en) * 2004-06-09 2009-04-15 日本電気株式会社 Topic analysis method and apparatus, and program
US7313575B2 (en) * 2004-06-14 2007-12-25 Hewlett-Packard Development Company, L.P. Data services handler
US7593923B1 (en) 2004-06-29 2009-09-22 Unisys Corporation Functional operations for accessing and/or building interlocking trees datastores to enable their use with applications software
US7428530B2 (en) * 2004-07-01 2008-09-23 Microsoft Corporation Dispersing search engine results by using page category information
US7698333B2 (en) 2004-07-22 2010-04-13 Factiva, Inc. Intelligent query system and method using phrase-code frequency-inverse phrase-code document frequency module
US7383260B2 (en) * 2004-08-03 2008-06-03 International Business Machines Corporation Method and apparatus for ontology-based classification of media content
US20060029275A1 (en) * 2004-08-06 2006-02-09 Microsoft Corporation Systems and methods for image data separation
US8407239B2 (en) * 2004-08-13 2013-03-26 Google Inc. Multi-stage query processing system and method for use with tokenspace repository
US7917480B2 (en) * 2004-08-13 2011-03-29 Google Inc. Document compression system and method for use with tokenspace repository
US7680648B2 (en) 2004-09-30 2010-03-16 Google Inc. Methods and systems for improving text segmentation
US7996208B2 (en) * 2004-09-30 2011-08-09 Google Inc. Methods and systems for selecting a language for text segmentation
WO2006039566A3 (en) * 2004-09-30 2007-07-05 Matthew Francis Hurst Topical sentiments in electronically stored communications
US8051096B1 (en) 2004-09-30 2011-11-01 Google Inc. Methods and systems for augmenting a token lexicon
US7213041B2 (en) * 2004-10-05 2007-05-01 Unisys Corporation Saving and restoring an interlocking trees datastore
US7953723B1 (en) 2004-10-06 2011-05-31 Shopzilla, Inc. Federation for parallel searching
US7716241B1 (en) 2004-10-27 2010-05-11 Unisys Corporation Storing the repository origin of data inputs within a knowledge store
US7908240B1 (en) 2004-10-28 2011-03-15 Unisys Corporation Facilitated use of column and field data for field record universe in a knowledge store
US20060101048A1 (en) * 2004-11-08 2006-05-11 Mazzagatti Jane C KStore data analyzer
US7499932B2 (en) * 2004-11-08 2009-03-03 Unisys Corporation Accessing data in an interlocking trees data structure using an application programming interface
US7348980B2 (en) * 2004-11-08 2008-03-25 Unisys Corporation Method and apparatus for interface for graphic display of data from a Kstore
US7418445B1 (en) 2004-11-08 2008-08-26 Unisys Corporation Method for reducing the scope of the K node construction lock
US20060100845A1 (en) * 2004-11-08 2006-05-11 Mazzagatti Jane C Multiple stream real time data simulation adapted for a KStore data structure
US20060155530A1 (en) * 2004-12-14 2006-07-13 International Business Machines Corporation Method and apparatus for generation of text documents
US9177248B2 (en) 2005-03-30 2015-11-03 Primal Fusion Inc. Knowledge representation systems and methods incorporating customization
US9098575B2 (en) 2011-06-20 2015-08-04 Primal Fusion Inc. Preference-guided semantic processing
US7849090B2 (en) 2005-03-30 2010-12-07 Primal Fusion Inc. System, method and computer program for faceted classification synthesis
US8849860B2 (en) 2005-03-30 2014-09-30 Primal Fusion Inc. Systems and methods for applying statistical inference techniques to knowledge representations
US7606781B2 (en) * 2005-03-30 2009-10-20 Primal Fusion Inc. System, method and computer program for facet analysis
US9235806B2 (en) 2010-06-22 2016-01-12 Primal Fusion Inc. Methods and devices for customizing knowledge representation systems
US7844565B2 (en) 2005-03-30 2010-11-30 Primal Fusion Inc. System, method and computer program for using a multi-tiered knowledge representation model
US9104779B2 (en) 2005-03-30 2015-08-11 Primal Fusion Inc. Systems and methods for analyzing and synthesizing complex knowledge representations
US7596574B2 (en) * 2005-03-30 2009-09-29 Primal Fusion, Inc. Complex-adaptive system for providing a facted classification
US7546294B2 (en) * 2005-03-31 2009-06-09 Microsoft Corporation Automated relevance tuning
US7409380B1 (en) 2005-04-07 2008-08-05 Unisys Corporation Facilitated reuse of K locations in a knowledge store
US7809736B2 (en) * 2005-05-02 2010-10-05 Brown University Importance ranking for a hierarchical collection of objects
US20060253423A1 (en) * 2005-05-07 2006-11-09 Mclane Mark Information retrieval system and method
US7389301B1 (en) 2005-06-10 2008-06-17 Unisys Corporation Data aggregation user interface and analytic adapted for a KStore
US9158855B2 (en) * 2005-06-16 2015-10-13 Buzzmetrics, Ltd Extracting structured data from weblogs
US20060287980A1 (en) * 2005-06-21 2006-12-21 Microsoft Corporation Intelligent search results blending
US7698335B1 (en) * 2005-06-27 2010-04-13 Microsoft Corporation Cluster organization of electronically-stored items
US7403932B2 (en) * 2005-07-01 2008-07-22 The Boeing Company Text differentiation methods, systems, and computer program products for content analysis
CN100585591C (en) 2005-07-15 2010-01-27 国际商业机器公司 Interacting viewing system and method
US20070100779A1 (en) * 2005-08-05 2007-05-03 Ori Levy Method and system for extracting web data
US7882262B2 (en) * 2005-08-18 2011-02-01 Cisco Technology, Inc. Method and system for inline top N query computation
US7774335B1 (en) * 2005-08-23 2010-08-10 Amazon Technologies, Inc. Method and system for determining interest levels of online content navigation paths
US7966395B1 (en) 2005-08-23 2011-06-21 Amazon Technologies, Inc. System and method for indicating interest of online content
US20070073745A1 (en) * 2005-09-23 2007-03-29 Applied Linguistics, Llc Similarity metric for semantic profiling
US7620607B1 (en) * 2005-09-26 2009-11-17 Quintura Inc. System and method for using a bidirectional neural network to identify sentences for use as document annotations
WO2007038713A3 (en) * 2005-09-28 2008-02-14 Epacris Inc Search engine determining results based on probabilistic scoring of relevance
CA2625493C (en) * 2005-10-11 2014-12-16 Intelligenxia Inc. System, method & computer program product for concept based searching & analysis
US7917519B2 (en) * 2005-10-26 2011-03-29 Sizatola, Llc Categorized document bases
US9165039B2 (en) * 2005-11-29 2015-10-20 Kang Jo Mgmt, Limited Liability Company Methods and systems for providing personalized contextual search results
US7502765B2 (en) 2005-12-21 2009-03-10 International Business Machines Corporation Method for organizing semi-structured data into a taxonomy, based on tag-separated clustering
US7747631B1 (en) * 2006-01-12 2010-06-29 Recommind, Inc. System and method for establishing relevance of objects in an enterprise system
US7657522B1 (en) 2006-01-12 2010-02-02 Recommind, Inc. System and method for providing information navigation and filtration
WO2007084616A3 (en) * 2006-01-18 2008-04-17 Behnam Attaran-Rezael System and method for context-based knowledge search, tagging, collaboration, management and advertisement
US7676485B2 (en) * 2006-01-20 2010-03-09 Ixreveal, Inc. Method and computer program product for converting ontologies into concept semantic networks
US20070203865A1 (en) * 2006-02-09 2007-08-30 Hirsch Martin C Apparatus and methods for an item retrieval system
US20070195776A1 (en) * 2006-02-23 2007-08-23 Zheng Danyang R System and method for channeling network traffic
US8019763B2 (en) * 2006-02-27 2011-09-13 Microsoft Corporation Propagating relevance from labeled documents to unlabeled documents
US20070214153A1 (en) * 2006-03-10 2007-09-13 Mazzagatti Jane C Method for processing an input particle stream for creating upper levels of KStore
US7461289B2 (en) * 2006-03-16 2008-12-02 Honeywell International Inc. System and method for computer service security
US20070220069A1 (en) * 2006-03-20 2007-09-20 Mazzagatti Jane C Method for processing an input particle stream for creating lower levels of a KStore
US20080275842A1 (en) * 2006-03-20 2008-11-06 Jane Campbell Mazzagatti Method for processing counts when an end node is encountered
US7734571B2 (en) * 2006-03-20 2010-06-08 Unisys Corporation Method for processing sensor data within a particle stream by a KStore
US7689571B1 (en) 2006-03-24 2010-03-30 Unisys Corporation Optimizing the size of an interlocking tree datastore structure for KStore
US7720848B2 (en) * 2006-03-29 2010-05-18 Xerox Corporation Hierarchical clustering with real-time updating
US8238351B2 (en) * 2006-04-04 2012-08-07 Unisys Corporation Method for determining a most probable K location
US7478075B2 (en) * 2006-04-11 2009-01-13 Sun Microsystems, Inc. Reducing the size of a training set for classification
US20070255732A1 (en) * 2006-04-27 2007-11-01 Moss Barrie J Method and Apparatus for Implementing a Semantic Environment Including Multi-Search Term Storage and Retrieval of Data and Content
US7676330B1 (en) 2006-05-16 2010-03-09 Unisys Corporation Method for processing a particle using a sensor structure
US8233388B2 (en) * 2006-05-30 2012-07-31 Cisco Technology, Inc. System and method for controlling and tracking network content flow
US8370127B2 (en) * 2006-06-16 2013-02-05 Nuance Communications, Inc. Systems and methods for building asset based natural language call routing application with limited resources
US20070294223A1 (en) * 2006-06-16 2007-12-20 Technion Research And Development Foundation Ltd. Text Categorization Using External Knowledge
US7809723B2 (en) * 2006-06-26 2010-10-05 Microsoft Corporation Distributed hierarchical text classification framework
EP1876540A1 (en) * 2006-07-06 2008-01-09 British Telecommunications Public Limited Company Organising and storing documents
US7519619B2 (en) * 2006-08-21 2009-04-14 Microsoft Corporation Facilitating document classification using branch associations
US7660783B2 (en) * 2006-09-27 2010-02-09 Buzzmetrics, Inc. System and method of ad-hoc analysis of data
CN101606152A (en) * 2006-10-03 2009-12-16 Qps技术有限责任公司 Mechanism for automatic matching of host to guest content via categorization
US20080091423A1 (en) * 2006-10-13 2008-04-17 Shourya Roy Generation of domain models from noisy transcriptions
US20080120294A1 (en) * 2006-11-17 2008-05-22 X.Com.Inc Computer-implemented systems and methods for media asset searching and access
US8676802B2 (en) 2006-11-30 2014-03-18 Oracle Otc Subsidiary Llc Method and system for information retrieval with clustering
US20080172380A1 (en) * 2007-01-17 2008-07-17 Wojciech Czyz Information retrieval based on information location in the information space.
US7873583B2 (en) * 2007-01-19 2011-01-18 Microsoft Corporation Combining resilient classifiers
US7603348B2 (en) * 2007-01-26 2009-10-13 Yahoo! Inc. System for classifying a search query
US7853595B2 (en) * 2007-01-30 2010-12-14 The Boeing Company Method and apparatus for creating a tool for generating an index for a document
US7529743B1 (en) * 2007-02-26 2009-05-05 Quintura, Inc. GUI for subject matter navigation using maps and search terms
US7610185B1 (en) * 2007-02-26 2009-10-27 Quintura, Inc. GUI for subject matter navigation using maps and search terms
EP1973045A1 (en) * 2007-03-20 2008-09-24 British Telecommunications Public Limited Company Organising and storing documents
US20080270381A1 (en) * 2007-04-24 2008-10-30 Interse A/S Enterprise-Wide Information Management System for Enhancing Search Queries to Improve Search Result Quality
US20080301033A1 (en) * 2007-06-01 2008-12-04 Netseer, Inc. Method and apparatus for optimizing long term revenues in online auctions
US8935249B2 (en) 2007-06-26 2015-01-13 Oracle Otc Subsidiary Llc Visualization of concepts within a collection of information
US8051073B2 (en) * 2007-06-26 2011-11-01 Endeca Technologies, Inc. System and method for measuring the quality of document sets
US20090028164A1 (en) * 2007-07-23 2009-01-29 Semgine, Gmbh Method and apparatus for semantic serializing
US20090089381A1 (en) * 2007-09-28 2009-04-02 Microsoft Corporation Pending and exclusive electronic mail inbox
US8671104B2 (en) * 2007-10-12 2014-03-11 Palo Alto Research Center Incorporated System and method for providing orientation into digital information
US8073682B2 (en) * 2007-10-12 2011-12-06 Palo Alto Research Center Incorporated System and method for prospecting digital information
US8165985B2 (en) * 2007-10-12 2012-04-24 Palo Alto Research Center Incorporated System and method for performing discovery of digital information in a subject area
US7415460B1 (en) 2007-12-10 2008-08-19 International Business Machines Corporation System and method to customize search engine results by picking documents
US8347326B2 (en) 2007-12-18 2013-01-01 The Nielsen Company (US) Identifying key media events and modeling causal relationships between key events and reported feelings
US20090164418A1 (en) * 2007-12-19 2009-06-25 Valentina Pulnikova Retrieval system and method of searching information in the Internet
US8150869B2 (en) * 2008-03-17 2012-04-03 Microsoft Corporation Combined web browsing and searching
CA2723179C (en) * 2008-05-01 2017-11-28 Primal Fusion Inc. Method, system, and computer program for user-driven dynamic generation of semantic networks and media synthesis
US9378203B2 (en) 2008-05-01 2016-06-28 Primal Fusion Inc. Methods and apparatus for providing information of interest to one or more users
US8676732B2 (en) 2008-05-01 2014-03-18 Primal Fusion Inc. Methods and apparatus for providing information of interest to one or more users
US9361365B2 (en) 2008-05-01 2016-06-07 Primal Fusion Inc. Methods and apparatus for searching of content using semantic synthesis
US20090281900A1 (en) * 2008-05-06 2009-11-12 Netseer, Inc. Discovering Relevant Concept And Context For Content Node
US8082248B2 (en) * 2008-05-29 2011-12-20 Rania Abouyounes Method and system for document classification based on document structure and written style
US20090300009A1 (en) * 2008-05-30 2009-12-03 Netseer, Inc. Behavioral Targeting For Tracking, Aggregating, And Predicting Online Behavior
US8010544B2 (en) * 2008-06-06 2011-08-30 Yahoo! Inc. Inverted indices in information extraction to improve records extracted per annotation
US8145654B2 (en) * 2008-06-20 2012-03-27 Lexisnexis Group Systems and methods for document searching
US20090327210A1 (en) * 2008-06-27 2009-12-31 Microsoft Corporation Advanced book page classification engine and index page extraction
US9268843B2 (en) * 2008-06-27 2016-02-23 Cbs Interactive Inc. Personalization engine for building a user profile
US8214346B2 (en) * 2008-06-27 2012-07-03 Cbs Interactive Inc. Personalization engine for classifying unstructured documents
US20100023319A1 (en) * 2008-07-28 2010-01-28 International Business Machines Corporation Model-driven feedback for annotation
US8010545B2 (en) * 2008-08-28 2011-08-30 Palo Alto Research Center Incorporated System and method for providing a topic-directed search
US20100057577A1 (en) * 2008-08-28 2010-03-04 Palo Alto Research Center Incorporated System And Method For Providing Topic-Guided Broadening Of Advertising Targets In Social Indexing
US20100057536A1 (en) * 2008-08-28 2010-03-04 Palo Alto Research Center Incorporated System And Method For Providing Community-Based Advertising Term Disambiguation
US8209616B2 (en) * 2008-08-28 2012-06-26 Palo Alto Research Center Incorporated System and method for interfacing a web browser widget with social indexing
CN106250371A (en) 2008-08-29 2016-12-21 启创互联公司 Systems and methods for semantic concept definition and semantic concept relationship synthesis utilizing existing domain definitions
US8364663B2 (en) * 2008-09-05 2013-01-29 Microsoft Corporation Tokenized javascript indexing system
US7730061B2 (en) * 2008-09-12 2010-06-01 International Business Machines Corporation Fast-approximate TFIDF
US8417695B2 (en) * 2008-10-30 2013-04-09 Netseer, Inc. Identifying related concepts of URLs and domain names
US8549016B2 (en) * 2008-11-14 2013-10-01 Palo Alto Research Center Incorporated System and method for providing robust topic identification in social indexes
US20100185672A1 (en) * 2009-01-21 2010-07-22 Rising Iii Hawley K Techniques for spatial representation of data and browsing based on similarity
US8239397B2 (en) * 2009-01-27 2012-08-07 Palo Alto Research Center Incorporated System and method for managing user attention by detecting hot and cold topics in social indexes
US8356044B2 (en) * 2009-01-27 2013-01-15 Palo Alto Research Center Incorporated System and method for providing default hierarchical training for social indexing
US8452781B2 (en) * 2009-01-27 2013-05-28 Palo Alto Research Center Incorporated System and method for using banded topic relevance and time for article prioritization
US8521908B2 (en) 2009-04-07 2013-08-27 Verisign, Inc. Existent domain name DNS traffic capture and analysis
US9245243B2 (en) * 2009-04-14 2016-01-26 Ureveal, Inc. Concept-based analysis of structured and unstructured data using concept inheritance
US9292612B2 (en) 2009-04-22 2016-03-22 Verisign, Inc. Internet profile service
US8370504B2 (en) * 2009-07-15 2013-02-05 Verisign, Inc. Method and system for predicting domain name registration renewal probability
US9002857B2 (en) * 2009-08-13 2015-04-07 Charite-Universitatsmedizin Berlin Methods for searching with semantic similarity scores in one or more ontologies
CN102012900B (en) * 2009-09-04 2013-01-30 阿里巴巴集团控股有限公司 An information retrieval method and system
US20110060644A1 (en) * 2009-09-08 2011-03-10 Peter Sweeney Synthesizing messaging using context provided by consumers
US20110060645A1 (en) * 2009-09-08 2011-03-10 Peter Sweeney Synthesizing messaging using context provided by consumers
US9292855B2 (en) * 2009-09-08 2016-03-22 Primal Fusion Inc. Synthesizing messaging using context provided by consumers
EP2488970A4 (en) * 2009-10-15 2016-03-16 Rogers Comm Tnc System and method for grouping multiple streams of data
US9262520B2 (en) 2009-11-10 2016-02-16 Primal Fusion Inc. System, method and computer program for creating and manipulating data structures using an interactive graphical interface
US8392175B2 (en) * 2010-02-01 2013-03-05 Stratify, Inc. Phrase-based document clustering with automatic phrase extraction
EP2531938A1 (en) * 2010-02-05 2012-12-12 FTI Technology LLC Propagating classification decisions
US8392432B2 (en) * 2010-04-12 2013-03-05 Microsoft Corporation Make and model classifier
US9031944B2 (en) 2010-04-30 2015-05-12 Palo Alto Research Center Incorporated System and method for providing multi-core and multi-level topical organization in social indexes
US20110289025A1 (en) * 2010-05-19 2011-11-24 Microsoft Corporation Learning user intent from rule-based training data
US8874727B2 (en) 2010-05-31 2014-10-28 The Nielsen Company (Us), Llc Methods, apparatus, and articles of manufacture to rank users in an online social network
US9600565B2 (en) * 2010-10-15 2017-03-21 Nec Corporation Data structure, index creation device, data search device, index creation method, data search method, and computer-readable recording medium
US8527450B2 (en) * 2011-03-24 2013-09-03 Yahoo! Inc. Apparatus and methods for analyzing and using short messages from commercial accounts
US9183244B2 (en) 2011-08-02 2015-11-10 Cavium, Inc. Rule modification in decision trees
US20130036083A1 (en) 2011-08-02 2013-02-07 Cavium, Inc. System and Method for Storing Lookup Request Rules in Multiple Memories
US9208438B2 (en) * 2011-08-02 2015-12-08 Cavium, Inc. Duplication in decision trees
US8954436B2 (en) * 2012-01-26 2015-02-10 International Business Machines Corporation Monitoring content repositories, identifying misclassified content objects, and suggesting reclassification
US20130238608A1 (en) * 2012-03-07 2013-09-12 Microsoft Corporation Search results by mapping associated with disparate taxonomies
US9069798B2 (en) 2012-05-24 2015-06-30 Mitsubishi Electric Research Laboratories, Inc. Method of text classification using discriminative topic transformation
US20140214835A1 (en) * 2013-01-29 2014-07-31 Richard Thomas Oehrle System and method for automatically classifying documents
US9195939B1 (en) 2013-03-15 2015-11-24 Cavium, Inc. Scope in decision trees
US9298814B2 (en) 2013-03-15 2016-03-29 Maritz Holdings Inc. Systems and methods for classifying electronic documents
US9595003B1 (en) 2013-03-15 2017-03-14 Cavium, Inc. Compiler with mask nodes
US9430511B2 (en) 2013-03-15 2016-08-30 Cavium, Inc. Merging independent writes, separating dependent and independent writes, and error roll back
US9336332B2 (en) 2013-08-28 2016-05-10 Clipcard Inc. Programmatic data discovery platforms for computing applications
US9411905B1 (en) * 2013-09-26 2016-08-09 Groupon, Inc. Multi-term query subsumption for document classification
US9659045B2 (en) 2013-11-08 2017-05-23 Oracle International Corporation Generic indexing for efficiently supporting ad-hoc query over hierarchically marked-up data
US9275336B2 (en) 2013-12-31 2016-03-01 Cavium, Inc. Method and system for skipping over group(s) of rules based on skip group rule
US9544402B2 (en) 2013-12-31 2017-01-10 Cavium, Inc. Multi-rule approach to encoding a group of rules
US9667446B2 (en) 2014-01-08 2017-05-30 Cavium, Inc. Condition code approach for comparing rule and packet data that are provided in portions
US20150347576A1 (en) * 2014-05-28 2015-12-03 Alexander Endert Method and system for information retrieval and aggregation from inferred user reasoning
US9881072B2 (en) * 2014-08-14 2018-01-30 McAFEE, LLC. Dynamic feature set management
US9646512B2 (en) * 2014-10-24 2017-05-09 Lingualeo, Inc. System and method for automated teaching of languages based on frequency of syntactic models
CN104866606A (en) * 2015-06-02 2015-08-26 浙江师范大学 MapReduce parallel big data text classification method

Family Cites Families (21)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH021057A (en) 1988-01-20 1990-01-05 Ricoh Co Ltd Document retrieving device
US4975975A (en) 1988-05-26 1990-12-04 Gtx Corporation Hierarchical parametric apparatus and method for recognizing drawn characters
WO1990016036A1 (en) 1989-06-14 1990-12-27 Hitachi, Ltd. Hierarchical presearch-type document retrieval method, apparatus therefor, and magnetic disc device for this apparatus
US5469354A (en) 1989-06-14 1995-11-21 Hitachi, Ltd. Document data processing method and apparatus for document retrieval
JPH03129472A (en) * 1989-07-31 1991-06-03 Ricoh Co Ltd Processing method for document retrieving device
US5317507A (en) 1990-11-07 1994-05-31 Gallant Stephen I Method for document retrieval and for word sense disambiguation using neural networks
US5325298A (en) 1990-11-07 1994-06-28 Hnc, Inc. Methods for generating or revising context vectors for a plurality of word stems
US5557794A (en) 1991-09-02 1996-09-17 Fuji Xerox Co., Ltd. Data management system for a personal data base
JP3303926B2 (en) 1991-09-27 2002-07-22 富士ゼロックス株式会社 Structured document classification apparatus and method
US5428778A (en) 1992-02-13 1995-06-27 Office Express Pty. Ltd. Selective dissemination of information
US5659724A (en) 1992-11-06 1997-08-19 Ncr Interactive data analysis apparatus employing a knowledge base
DE69330021T2 (en) * 1992-12-18 2001-10-31 Raytheon Co Improved system for pattern recognition for sonar and other applications
JP3175399B2 (en) 1993-05-18 2001-06-11 セイコーエプソン株式会社 Card data management device
US5506984A (en) 1993-06-30 1996-04-09 Digital Equipment Corporation Method and system for data retrieval in a distributed system using linked location references on a plurality of nodes
JP3053153B2 (en) 1993-09-20 2000-06-19 日立西部ソフトウエア株式会社 Application start-up method of document management system
US5576954A (en) 1993-11-05 1996-11-19 University Of Central Florida Process for determination of text relevancy
US5625767A (en) * 1995-03-13 1997-04-29 Bartell; Brian Method and system for two-dimensional visualization of an information taxonomy and of text documents based on topical content of the documents
DE69624755T2 (en) 1995-05-22 2003-04-24 Matsushita Electric Ind Co Ltd Information search apparatus for searching text to retrieve strings that match a keyword
US5675710A (en) * 1995-06-07 1997-10-07 Lucent Technologies, Inc. Method and apparatus for training a text classifier
US5918240A (en) * 1995-06-28 1999-06-29 Xerox Corporation Automatic method of extracting summarization using feature probabilities
US5826260A (en) * 1995-12-11 1998-10-20 International Business Machines Corporation Information retrieval system and method for displaying and ordering information based on query element contribution

Cited By (204)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6640224B1 (en) * 1997-12-15 2003-10-28 International Business Machines Corporation System and method for dynamic index-probe optimizations for high-dimensional similarity search
US7650340B2 (en) * 1998-12-21 2010-01-19 Adobe Systems Incorporated Describing documents and expressing document structure
US9710457B2 (en) * 1999-02-05 2017-07-18 Gregory A. Stobbs Computer-implemented patent portfolio analysis method and apparatus
US20120109642A1 (en) * 1999-02-05 2012-05-03 Stobbs Gregory A Computer-implemented patent portfolio analysis method and apparatus
US7376635B1 (en) * 2000-07-21 2008-05-20 Ford Global Technologies, Llc Theme-based system and method for classifying documents
US20040153307A1 (en) * 2001-03-30 2004-08-05 Naftali Tishby Discriminative feature selection for data sequences
US8849716B1 (en) * 2001-04-20 2014-09-30 Jpmorgan Chase Bank, N.A. System and method for preventing identity theft or misuse by restricting access
US7107254B1 (en) * 2001-05-07 2006-09-12 Microsoft Corporation Probablistic models and methods for combining multiple content classifiers
US20070192442A1 (en) * 2001-07-24 2007-08-16 Brightplanet Corporation System and method for efficient control and capture of dynamic database content
US8380735B2 (en) 2001-07-24 2013-02-19 Brightplanet Corporation II, Inc System and method for efficient control and capture of dynamic database content
US7676555B2 (en) 2001-07-24 2010-03-09 Brightplanet Corporation System and method for efficient control and capture of dynamic database content
US20030101191A1 (en) * 2001-11-28 2003-05-29 Yu-Chwin Lin Publication of commercial entity information and method for classifying commercial entity information
US7363593B1 (en) * 2001-11-30 2008-04-22 Versata Development Group, Inc. System and method for presenting information organized by hierarchical levels
US20070156665A1 (en) * 2001-12-05 2007-07-05 Janusz Wnek Taxonomy discovery
US20030163454A1 (en) * 2002-02-26 2003-08-28 Brian Jacobsen Subject specific search engine
US7949648B2 (en) 2002-02-26 2011-05-24 Soren Alain Mortensen Compiling and accessing subject-specific information from a computer network
WO2003079234A3 (en) * 2002-03-11 2004-06-10 Boeing Co Knowledge management using text classification
US20030172357A1 (en) * 2002-03-11 2003-09-11 Kao Anne S.W. Knowledge management using text classification
US7673234B2 (en) 2002-03-11 2010-03-02 The Boeing Company Knowledge management using text classification
US20030194689A1 (en) * 2002-04-12 2003-10-16 Mitsubishi Denki Kabushiki Kaisha Structured document type determination system and structured document type determination method
US20030217055A1 (en) * 2002-05-20 2003-11-20 Chang-Huang Lee Efficient incremental method for data mining of a database
US20040117388A1 (en) * 2002-09-02 2004-06-17 Yasuhiko Inaba Method, apparatus and programs for delivering information
US20040049514A1 (en) * 2002-09-11 2004-03-11 Sergei Burkov System and method of searching data utilizing automatic categorization
US9515998B2 (en) 2002-09-18 2016-12-06 Symantec Corporation Secure and scalable detection of preselected data embedded in electronically transmitted messages
US7472114B1 (en) * 2002-09-18 2008-12-30 Symantec Corporation Method and apparatus to define the scope of a search for information from a tabular data source
US20100083377A1 (en) * 2002-09-18 2010-04-01 Rowney Kevin T Method and apparatus to define the scope of a search for information from a tabular data source
US20090106205A1 (en) * 2002-09-18 2009-04-23 Rowney Kevin T Method and apparatus to define the scope of a search for information from a tabular data source
US8661498B2 (en) 2002-09-18 2014-02-25 Symantec Corporation Secure and scalable detection of preselected data embedded in electronically transmitted messages
US7673344B1 (en) 2002-09-18 2010-03-02 Symantec Corporation Mechanism to search information content for preselected data
US8595849B2 (en) 2002-09-18 2013-11-26 Symantec Corporation Method and apparatus to report policy violations in messages
US8566305B2 (en) * 2002-09-18 2013-10-22 Symantec Corporation Method and apparatus to define the scope of a search for information from a tabular data source
US8225371B2 (en) 2002-09-18 2012-07-17 Symantec Corporation Method and apparatus for creating an information security policy based on a pre-configured template
US8813176B2 (en) 2002-09-18 2014-08-19 Symantec Corporation Method and apparatus for creating an information security policy based on a pre-configured template
US20050027723A1 (en) * 2002-09-18 2005-02-03 Chris Jones Method and apparatus to report policy violations in messages
US20100332481A1 (en) * 2002-09-18 2010-12-30 Rowney Kevin T Secure and scalable detection of preselected data embedded in electronically transmitted messages
US7996385B2 (en) * 2002-09-18 2011-08-09 Symantec Corporation Method and apparatus to define the scope of a search for information from a tabular data source
US8312553B2 (en) 2002-09-18 2012-11-13 Symantec Corporation Mechanism to search information content for preselected data
US20050086252A1 (en) * 2002-09-18 2005-04-21 Chris Jones Method and apparatus for creating an information security policy based on a pre-configured template
US7886359B2 (en) 2002-09-18 2011-02-08 Symantec Corporation Method and apparatus to report policy violations in messages
US7266559B2 (en) * 2002-12-05 2007-09-04 Microsoft Corporation Method and apparatus for adapting a search classifier based on user queries
US20040111419A1 (en) * 2002-12-05 2004-06-10 Cook Daniel B. Method and apparatus for adapting a search classifier based on user queries
US20070276818A1 (en) * 2002-12-05 2007-11-29 Microsoft Corporation Adapting a search classifier based on user queries
US7203669B2 (en) * 2003-03-17 2007-04-10 Intel Corporation Detector tree of boosted classifiers for real-time object detection and tracking
US20040220901A1 (en) * 2003-04-30 2004-11-04 Benq Corporation System and method for association itemset mining
US8751506B2 (en) 2003-05-06 2014-06-10 Symantec Corporation Personal computing device-based mechanism to detect preselected data
US8041719B2 (en) 2003-05-06 2011-10-18 Symantec Corporation Personal computing device-based mechanism to detect preselected data
US20040225645A1 (en) * 2003-05-06 2004-11-11 Rowney Kevin T. Personal computing device -based mechanism to detect preselected data
US20060155662A1 (en) * 2003-07-01 2006-07-13 Eiji Murakami Sentence classification device and method
US7567954B2 (en) * 2003-07-01 2009-07-28 Yamatake Corporation Sentence classification device and method
US8271794B2 (en) 2003-12-10 2012-09-18 Mcafee, Inc. Verifying captured objects before presentation
US8548170B2 (en) 2003-12-10 2013-10-01 Mcafee, Inc. Document de-registration
US9092471B2 (en) 2003-12-10 2015-07-28 Mcafee, Inc. Rule parser
US7899828B2 (en) 2003-12-10 2011-03-01 Mcafee, Inc. Tag data structure for maintaining relational data over captured objects
US9374225B2 (en) 2003-12-10 2016-06-21 Mcafee, Inc. Document de-registration
US20050132046A1 (en) * 2003-12-10 2005-06-16 De La Iglesia Erik Method and apparatus for data capture and analysis system
US7814327B2 (en) 2003-12-10 2010-10-12 Mcafee, Inc. Document registration
US8301635B2 (en) 2003-12-10 2012-10-30 Mcafee, Inc. Tag data structure for maintaining relational data over captured objects
US7774604B2 (en) 2003-12-10 2010-08-10 Mcafee, Inc. Verifying captured objects before presentation
US8762386B2 (en) 2003-12-10 2014-06-24 Mcafee, Inc. Method and apparatus for data capture and analysis system
US8656039B2 (en) 2003-12-10 2014-02-18 Mcafee, Inc. Rule parser
US7984175B2 (en) 2003-12-10 2011-07-19 Mcafee, Inc. Method and apparatus for data capture and analysis system
US8166307B2 (en) 2003-12-10 2012-04-24 McAffee, Inc. Document registration
US8307206B2 (en) 2004-01-22 2012-11-06 Mcafee, Inc. Cryptographic policy enforcement
US7930540B2 (en) 2004-01-22 2011-04-19 Mcafee, Inc. Cryptographic policy enforcement
US20050187892A1 (en) * 2004-02-09 2005-08-25 Xerox Corporation Method for multi-class, multi-label categorization using probabilistic hierarchical modeling
US7139754B2 (en) * 2004-02-09 2006-11-21 Xerox Corporation Method for multi-class, multi-label categorization using probabilistic hierarchical modeling
US8862580B1 (en) * 2004-03-01 2014-10-14 Radix Holdings, Llc Category-based search
US7962591B2 (en) 2004-06-23 2011-06-14 Mcafee, Inc. Object classification in a capture system
US8560534B2 (en) 2004-08-23 2013-10-15 Mcafee, Inc. Database for a capture system
US7949849B2 (en) 2004-08-24 2011-05-24 Mcafee, Inc. File system for a capture system
US8707008B2 (en) 2004-08-24 2014-04-22 Mcafee, Inc. File system for a capture system
US20060074907A1 (en) * 2004-09-27 2006-04-06 Singhal Amitabh K Presentation of search results based on document structure
US9031898B2 (en) * 2004-09-27 2015-05-12 Google Inc. Presentation of search results based on document structure
US20060074632A1 (en) * 2004-09-30 2006-04-06 Nanavati Amit A Ontology-based term disambiguation
US8011003B2 (en) 2005-02-14 2011-08-30 Symantec Corporation Method and apparatus for handling messages containing pre-selected data
US20060224589A1 (en) * 2005-02-14 2006-10-05 Rowney Kevin T Method and apparatus for handling messages containing pre-selected data
US20060195415A1 (en) * 2005-02-14 2006-08-31 France Telecom Method and device for the generation of a classification tree to unify the supervised and unsupervised approaches, corresponding computer package and storage means
US7584168B2 (en) * 2005-02-14 2009-09-01 France Telecom Method and device for the generation of a classification tree to unify the supervised and unsupervised approaches, corresponding computer package and storage means
US20060184549A1 (en) * 2005-02-14 2006-08-17 Rowney Kevin T Method and apparatus for modifying messages based on the presence of pre-selected data
US20070011154A1 (en) * 2005-04-11 2007-01-11 Textdigger, Inc. System and method for searching for a query
US9400838B2 (en) 2005-04-11 2016-07-26 Textdigger, Inc. System and method for searching for a query
US20060242140A1 (en) * 2005-04-26 2006-10-26 Content Analyst Company, Llc Latent semantic clustering
US20060242190A1 (en) * 2005-04-26 2006-10-26 Content Analyst Comapny, Llc Latent semantic taxonomy generation
US7844566B2 (en) 2005-04-26 2010-11-30 Content Analyst Company, Llc Latent semantic clustering
US20060242098A1 (en) * 2005-04-26 2006-10-26 Content Analyst Company, Llc Generating representative exemplars for indexing, clustering, categorization and taxonomy
US20070005535A1 (en) * 2005-04-27 2007-01-04 Abdolreza Salahshour System and methods for IT resource event situation classification and semantics
US7895137B2 (en) 2005-04-27 2011-02-22 International Business Machines Corporation Rules generation for IT resource event situation classification
US7730007B2 (en) 2005-04-27 2010-06-01 International Business Machines Corporation IT event data classifier configured to label messages if message identifiers map directly to classification categories or parse for feature extraction if message identifiers do not map directly to classification categories
US20090276383A1 (en) * 2005-04-27 2009-11-05 International Business Machines Corporation Rules generation for it resource event situation classification
US20090006298A1 (en) * 2005-04-27 2009-01-01 International Business Machines Corporation It resource event situation classification and semantics
US20060282442A1 (en) * 2005-04-27 2006-12-14 Canon Kabushiki Kaisha Method of learning associations between documents and data sets
US7461044B2 (en) 2005-04-27 2008-12-02 International Business Machines Corporation It resource event situation classification and semantics
US20130046797A1 (en) * 2005-05-10 2013-02-21 Netseer, Inc. Methods and apparatus for distributed community finding
US8825654B2 (en) 2005-05-10 2014-09-02 Netseer, Inc. Methods and apparatus for distributed community finding
US9110985B2 (en) 2005-05-10 2015-08-18 Neetseer, Inc. Generating a conceptual association graph from large-scale loosely-grouped content
US8838605B2 (en) * 2005-05-10 2014-09-16 Netseer, Inc. Methods and apparatus for distributed community finding
US8730955B2 (en) 2005-08-12 2014-05-20 Mcafee, Inc. High speed packet capture
US7907608B2 (en) 2005-08-12 2011-03-15 Mcafee, Inc. High speed packet capture
US8554774B2 (en) 2005-08-31 2013-10-08 Mcafee, Inc. System and method for word indexing in a capture system and querying thereof
US7818326B2 (en) * 2005-08-31 2010-10-19 Mcafee, Inc. System and method for word indexing in a capture system and querying thereof
US8463800B2 (en) 2005-10-19 2013-06-11 Mcafee, Inc. Attributes of captured objects in a capture system
US7730011B1 (en) 2005-10-19 2010-06-01 Mcafee, Inc. Attributes of captured objects in a capture system
US8176049B2 (en) 2005-10-19 2012-05-08 Mcafee Inc. Attributes of captured objects in a capture system
US8200026B2 (en) 2005-11-21 2012-06-12 Mcafee, Inc. Identifying image type in a capture system
US7657104B2 (en) 2005-11-21 2010-02-02 Mcafee, Inc. Identifying image type in a capture system
US9928299B2 (en) 2006-01-03 2018-03-27 Textdigger, Inc. Search system with query refinement and search method
US9245029B2 (en) 2006-01-03 2016-01-26 Textdigger, Inc. Search system with query refinement and search method
US9443018B2 (en) 2006-01-19 2016-09-13 Netseer, Inc. Systems and methods for creating, navigating, and searching informational web neighborhoods
US20070203903A1 (en) * 2006-02-28 2007-08-30 Ilial, Inc. Methods and apparatus for visualizing, managing, monetizing, and personalizing knowledge search results on a user interface
US8843434B2 (en) 2006-02-28 2014-09-23 Netseer, Inc. Methods and apparatus for visualizing, managing, monetizing, and personalizing knowledge search results on a user interface
US8504537B2 (en) 2006-03-24 2013-08-06 Mcafee, Inc. Signature distribution in a document registration system
US20070234232A1 (en) * 2006-03-29 2007-10-04 Gheorghe Adrian Citu Dynamic image display
US8862573B2 (en) 2006-04-04 2014-10-14 Textdigger, Inc. Search system and method with text function tagging
US20080059451A1 (en) * 2006-04-04 2008-03-06 Textdigger, Inc. Search system and method with text function tagging
US8307007B2 (en) 2006-05-22 2012-11-06 Mcafee, Inc. Query generation for a capture system
US8683035B2 (en) 2006-05-22 2014-03-25 Mcafee, Inc. Attributes of captured objects in a capture system
US7958227B2 (en) 2006-05-22 2011-06-07 Mcafee, Inc. Attributes of captured objects in a capture system
US9094338B2 (en) 2006-05-22 2015-07-28 Mcafee, Inc. Attributes of captured objects in a capture system
US7689614B2 (en) 2006-05-22 2010-03-30 Mcafee, Inc. Query generation for a capture system
US8010689B2 (en) 2006-05-22 2011-08-30 Mcafee, Inc. Locational tagging in a capture system
US8005863B2 (en) 2006-05-22 2011-08-23 Mcafee, Inc. Query generation for a capture system
US9817902B2 (en) 2006-10-27 2017-11-14 Netseer Acquisition, Inc. Methods and apparatus for matching relevant content to user intention
US20080114573A1 (en) * 2006-11-10 2008-05-15 Institute For Information Industry Tag organization methods and systems
US7908260B1 (en) 2006-12-29 2011-03-15 BrightPlanet Corporation II, Inc. Source editing, internationalization, advanced configuration wizard, and summary page selection for information automation systems
WO2008097891A3 (en) * 2007-02-02 2008-10-09 Musgrove Technology Entpr Llc Method and apparatus for aligning multiple taxonomies
WO2008097891A2 (en) * 2007-02-02 2008-08-14 Musgrove Technology Enterprises Llc Method and apparatus for aligning multiple taxonomies
US20090037457A1 (en) * 2007-02-02 2009-02-05 Musgrove Technology Enterprises, Llc (Mte) Method and apparatus for aligning multiple taxonomies
US8732197B2 (en) 2007-02-02 2014-05-20 Musgrove Technology Enterprises Llc (Mte) Method and apparatus for aligning multiple taxonomies
US8805843B2 (en) 2007-02-13 2014-08-12 International Business Machines Corporation Information mining using domain specific conceptual structures
US20080243889A1 (en) * 2007-02-13 2008-10-02 International Business Machines Corporation Information mining using domain specific conceptual structures
US20080195567A1 (en) * 2007-02-13 2008-08-14 International Business Machines Corporation Information mining using domain specific conceptual structures
US20090254540A1 (en) * 2007-11-01 2009-10-08 Textdigger, Inc. Method and apparatus for automated tag generation for digital content
US7962490B1 (en) * 2008-01-07 2011-06-14 Amdocs Software Systems Limited System, method, and computer program product for analyzing and decomposing a plurality of rules into a plurality of contexts
US8862619B1 (en) 2008-01-07 2014-10-14 Amdocs Software Systems Limited System, method, and computer program product for filtering a data stream utilizing a plurality of contexts
US20110231412A1 (en) * 2008-01-07 2011-09-22 Amdocs Software Systems Limited System, method, and computer program product for analyzing and decomposing a plurality of rules into a plurality of contexts
US8868563B2 (en) * 2008-01-07 2014-10-21 Amdocs Software Systems Limited System, method, and computer program product for analyzing and decomposing a plurality of rules into a plurality of contexts
US20100262571A1 (en) * 2008-03-05 2010-10-14 Schmidtler Mauritius A R Systems and methods for organizing data sets
US20090228499A1 (en) * 2008-03-05 2009-09-10 Schmidtler Mauritius A R Systems and methods for organizing data sets
US9082080B2 (en) 2008-03-05 2015-07-14 Kofax, Inc. Systems and methods for organizing data sets
US8321477B2 (en) 2008-03-05 2012-11-27 Kofax, Inc. Systems and methods for organizing data sets
US20100076984A1 (en) * 2008-03-27 2010-03-25 Alkis Papadopoullos System and method for query expansion using tooltips
US8255370B1 (en) 2008-03-28 2012-08-28 Symantec Corporation Method and apparatus for detecting policy violations in a data repository having an arbitrary data schema
US7996373B1 (en) 2008-03-28 2011-08-09 Symantec Corporation Method and apparatus for detecting policy violations in a data repository having an arbitrary data schema
US7996374B1 (en) 2008-03-28 2011-08-09 Symantec Corporation Method and apparatus for automatically correlating related incidents of policy violations
US9235629B1 (en) 2008-03-28 2016-01-12 Symantec Corporation Method and apparatus for automatically correlating related incidents of policy violations
US8065739B1 (en) 2008-03-28 2011-11-22 Symantec Corporation Detecting policy violations in information content containing data in a character-based language
US8635706B2 (en) 2008-07-10 2014-01-21 Mcafee, Inc. System and method for data mining and security policy management
US8205242B2 (en) 2008-07-10 2012-06-19 Mcafee, Inc. System and method for data mining and security policy management
US8601537B2 (en) 2008-07-10 2013-12-03 Mcafee, Inc. System and method for data mining and security policy management
US8554800B2 (en) * 2008-07-30 2013-10-08 Portool Ltd. System, methods and applications for structured document indexing
US20100030752A1 (en) * 2008-07-30 2010-02-04 Lev Goldentouch System, methods and applications for structured document indexing
US20110231411A1 (en) * 2008-08-08 2011-09-22 Holland Bloorview Kids Rehabilitation Hospital Topic Word Generation Method and System
US8335787B2 (en) * 2008-08-08 2012-12-18 Quillsoft Ltd. Topic word generation method and system
US9253154B2 (en) 2008-08-12 2016-02-02 Mcafee, Inc. Configuration management for a capture/registration system
US8826443B1 (en) 2008-09-18 2014-09-02 Symantec Corporation Selective removal of protected content from web requests sent to an interactive website
US9118720B1 (en) 2008-09-18 2015-08-25 Symantec Corporation Selective removal of protected content from web requests sent to an interactive website
US8613040B2 (en) 2008-12-22 2013-12-17 Symantec Corporation Adaptive data loss prevention policies
US20100162347A1 (en) * 2008-12-22 2010-06-24 Ian Barile Adaptive data loss prevention policies
US9916381B2 (en) * 2008-12-30 2018-03-13 Telecom Italia S.P.A. Method and system for content classification
US20110264699A1 (en) * 2008-12-30 2011-10-27 Telecom Italia S.P.A. Method and system for content classification
US8850591B2 (en) 2009-01-13 2014-09-30 Mcafee, Inc. System and method for concept building
US8706709B2 (en) 2009-01-15 2014-04-22 Mcafee, Inc. System and method for intelligent term grouping
US20100185577A1 (en) * 2009-01-16 2010-07-22 Microsoft Corporation Object classification using taxonomies
US8275726B2 (en) * 2009-01-16 2012-09-25 Microsoft Corporation Object classification using taxonomies
US8473442B1 (en) 2009-02-25 2013-06-25 Mcafee, Inc. System and method for intelligent state management
US9602548B2 (en) 2009-02-25 2017-03-21 Mcafee, Inc. System and method for intelligent state management
US9195937B2 (en) 2009-02-25 2015-11-24 Mcafee, Inc. System and method for intelligent state management
US8935752B1 (en) 2009-03-23 2015-01-13 Symantec Corporation System and method for identity consolidation
US8667121B2 (en) 2009-03-25 2014-03-04 Mcafee, Inc. System and method for managing data and policies
US8918359B2 (en) 2009-03-25 2014-12-23 Mcafee, Inc. System and method for data mining and security policy management
US8447722B1 (en) 2009-03-25 2013-05-21 Mcafee, Inc. System and method for data mining and security policy management
US9313232B2 (en) 2009-03-25 2016-04-12 Mcafee, Inc. System and method for data mining and security policy management
US8484194B1 (en) 2009-10-22 2013-07-09 Google Inc. Training set construction for taxonomic classification
US8122005B1 (en) * 2009-10-22 2012-02-21 Google Inc. Training set construction for taxonomic classification
US8515972B1 (en) 2010-02-10 2013-08-20 Python 4 Fun, Inc. Finding relevant documents
US20110214080A1 (en) * 2010-02-26 2011-09-01 Microsoft Corporation Taxonomy Editor
US8527893B2 (en) * 2010-02-26 2013-09-03 Microsoft Corporation Taxonomy editor
US8489538B1 (en) 2010-05-25 2013-07-16 Recommind, Inc. Systems and methods for predictive coding
US8554716B1 (en) 2010-05-25 2013-10-08 Recommind, Inc. Systems and methods for predictive coding
US9595005B1 (en) 2010-05-25 2017-03-14 Recommind, Inc. Systems and methods for predictive coding
US20110314024A1 (en) * 2010-06-18 2011-12-22 Microsoft Corporation Semantic content searching
US8380719B2 (en) * 2010-06-18 2013-02-19 Microsoft Corporation Semantic content searching
US20120078969A1 (en) * 2010-09-24 2012-03-29 International Business Machines Corporation System and method to extract models from semi-structured documents
US8645298B2 (en) 2010-10-26 2014-02-04 Microsoft Corporation Topic models
US9794254B2 (en) 2010-11-04 2017-10-17 Mcafee, Inc. System and method for protecting specified data combinations
US8806615B2 (en) 2010-11-04 2014-08-12 Mcafee, Inc. System and method for protecting specified data combinations
US9785634B2 (en) 2011-06-04 2017-10-10 Recommind, Inc. Integration and combination of random sampling and document batching
WO2012174640A1 (en) * 2011-06-22 2012-12-27 Rogers Communications Inc. Systems and methods for creating an interest profile for a user
US9116979B2 (en) 2011-06-22 2015-08-25 Rogers Communications Inc. Systems and methods for creating an interest profile for a user
US9436726B2 (en) 2011-06-23 2016-09-06 BCM International Regulatory Analytics LLC System, method and computer program product for a behavioral database providing quantitative analysis of cross border policy process and related search capabilities
US20130166563A1 (en) * 2011-12-21 2013-06-27 Sap Ag Integration of Text Analysis and Search Functionality
US8700561B2 (en) 2011-12-27 2014-04-15 Mcafee, Inc. System and method for providing data protection workflows in a network environment
US9430564B2 (en) 2011-12-27 2016-08-30 Mcafee, Inc. System and method for providing data protection workflows in a network environment
US20130218904A1 (en) * 2012-02-22 2013-08-22 Salesforce.Com, Inc. System and method for inferring reporting relationships from a contact database
US9477698B2 (en) * 2012-02-22 2016-10-25 Salesforce.Com, Inc. System and method for inferring reporting relationships from a contact database
US20140086497A1 (en) * 2012-06-14 2014-03-27 The Board of Trustees for the Leland Stanford, Junior, University Method and System for Optimizing Accuracy-Specificity Trade-offs in Large Scale Visual Recognition
US9158965B2 (en) * 2012-06-14 2015-10-13 The Board Of Trustees Of The Leland Stanford Junior University Method and system for optimizing accuracy-specificity trade-offs in large scale visual recognition
US9779291B2 (en) * 2012-06-14 2017-10-03 The Board Of Trustees Of The Leland Stanford Junior University Method and system for optimizing accuracy-specificity trade-offs in large scale visual recognition
US20160162731A1 (en) * 2012-06-14 2016-06-09 The Board Of Trustees Of The Leland Stanford Junior University Method and System for Optimizing Accuracy-Specificity Trade-offs in Large Scale Visual Recognition
US20140172652A1 (en) * 2012-12-19 2014-06-19 Yahoo! Inc. Automated categorization of products in a merchant catalog
US20170091590A1 (en) * 2013-03-15 2017-03-30 Sri International Computer vision as a service
US20150095017A1 (en) * 2013-09-27 2015-04-02 Google Inc. System and method for learning word embeddings using neural language models
US20160140207A1 (en) * 2014-11-14 2016-05-19 Symantec Corporation Systems and methods for aggregating information-asset classifications
US20160182669A1 (en) * 2014-12-22 2016-06-23 Here Global B.V. Optimal Coding Method for Efficient Matching Of Hierarchical Categories In Publish-Subscribe Systems

Also Published As

Publication number Publication date Type
US6233575B1 (en) 2001-05-15 grant

Similar Documents

Publication Publication Date Title
Silla et al. A survey of hierarchical classification across different application domains
Maarek et al. Automatically organizing bookmarks per contents
Chakrabarti Data mining for hypertext: A tutorial survey
Etzioni et al. Web-scale information extraction in knowitall:(preliminary results)
Dumais et al. Inductive learning algorithms and representations for text categorization
Sheth et al. Semantics for the semantic web: The implicit, the formal and the powerful
Gauch et al. Ontology-based personalized search and browsing
Li et al. Text document clustering based on frequent word meaning sequences
Xu et al. Document clustering based on non-negative matrix factorization
Nepal et al. Query processing issues in image (multimedia) databases
Gravano et al. GlOSS: text-source discovery over the Internet
Carpineto et al. A lattice conceptual clustering system and its application to browsing retrieval
Gallagher Matching structure and semantics: A survey on graph-based pattern matching
Sclaroff et al. Imagerover: A content-based image browser for the world wide web
US7496567B1 (en) System and method for document categorization
Willett Recent trends in hierarchic document clustering: a critical review
Xue et al. Distributional features for text categorization
Dakka et al. Automatic construction of multifaceted browsing interfaces
Chen et al. Content-based image retrieval by clustering
Roussinov et al. A scalable self-organizing map algorithm for textual classification: A neural network approach to thesaurus generation
US7216129B2 (en) Information processing using a hierarchy structure of randomized samples
Sheikholeslami et al. SemQuery: semantic clustering and querying on heterogeneous features for visual data
US7185001B1 (en) Systems and methods for document searching and organizing
Gravano et al. Categorizing web queries according to geographical locality
US6457028B1 (en) Method and apparatus for finding related collections of linked documents using co-citation analysis