Connect public, paid and private patent data with Google Patents Public Datasets

Information mining using domain specific conceptual structures

Download PDF

Info

Publication number
US20080195567A1
US20080195567A1 US11674601 US67460107A US2008195567A1 US 20080195567 A1 US20080195567 A1 US 20080195567A1 US 11674601 US11674601 US 11674601 US 67460107 A US67460107 A US 67460107A US 2008195567 A1 US2008195567 A1 US 2008195567A1
Authority
US
Grant status
Application
Patent type
Prior art keywords
set
documents
document
information
example
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Abandoned
Application number
US11674601
Inventor
Ying Chen
Jeffrey Thomas Kreulen
James J. Rhodes
William Scott Spangler
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
International Business Machines Corp
Original Assignee
International Business Machines Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRICAL DIGITAL DATA PROCESSING
    • G06F17/00Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/30Information retrieval; Database structures therefor ; File system structures therefor
    • G06F17/3061Information retrieval; Database structures therefor ; File system structures therefor of unstructured textual data
    • G06F17/30705Clustering or classification
    • G06F17/3071Clustering or classification including class or cluster creation or modification
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRICAL DIGITAL DATA PROCESSING
    • G06F17/00Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/30Information retrieval; Database structures therefor ; File system structures therefor
    • G06F17/3061Information retrieval; Database structures therefor ; File system structures therefor of unstructured textual data
    • G06F17/30705Clustering or classification
    • G06F17/30713Clustering or classification including cluster or class visualization or browsing
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06NCOMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N5/00Computer systems utilising knowledge based models
    • G06N5/02Knowledge representation
    • G06N5/022Knowledge engineering, knowledge acquisition

Abstract

A method and analytics tools for information mining incorporating domain specific knowledge and conceptual structures are disclosed, the method including: providing a first set of documents related to a first topic of interest; using a first taxonomy to categorize the first set of documents into a set of categories; providing a second set of documents related to a second topic of interest; categorizing the second set of documents according to the set of categories of the first set of documents; using an element of domain knowledge to re-categorize the first set of documents; and examining a category to identify a document of interest.

Description

    BACKGROUND OF THE INVENTION
  • [0001]
    The present invention relates generally to the field of information analytics tools and methods in data mining and, more particularly, to eliciting and capturing domain knowledge as part of the data mining process.
  • [0002]
    Using text and information mining to find insights in volumes of data is non-trivial. Often endless “googling” (referring to use of the well-known search engine for searching the web) is done to search various kinds of information which might lead to insights. However, such googling is labor-intensive and time-consuming. Furthermore, to make sense of the search results may require significant manual processing. Even so, the results may not be valuable.
  • [0003]
    Better methodologies and tools are needed to help identify insights in the information.
  • SUMMARY OF THE INVENTION
  • [0004]
    In one embodiment of the present invention, a method, for use with 1) a first set of documents related to a first topic of interest and 2) a second set of documents related to a second topic of interest, comprises the steps of: using a first taxonomy to categorize the first set of documents into a set of categories; categorizing the second set of documents according to the set of categories of the first set of documents; and examining a category to identify a document of interest, the document of interest being a representative document within the category.
  • [0005]
    In another embodiment of the present invention, a method, for use with a set of documents related to a first topic of interest, comprises: creating a first set of categories of the set of documents according to an automatically generated taxonomy; creating a second set of categories of the set of documents according to at least one of unstructured data, structured data, and annotations derived from text in the set of documents; constructing a contingency table having the first set of categories along a first axis and the second set of categories along a second axis; and identifying a relationship between at least two different categories using the contingency table.
  • [0006]
    In yet another embodiment of the present invention, a method comprises: extracting a set of documents related to a specified topic from a data warehouse; generating a taxonomy for the set of documents that provides a first partition of the set of documents according to the taxonomy; using domain-specific knowledge to re-partition the set of documents to provide a second partition of the set of documents; and creating a refined taxonomy for the set of documents according to the second partition so that the refined taxonomy incorporates the domain-specific knowledge.
  • [0007]
    In still another embodiment of the present invention, a computer program product, for use with 1) a first set of documents related to a first topic of interest and 2) a second set of documents related to a second topic of interest, comprises a computer useable medium including a computer readable program, wherein the computer readable program when executed on a computer causes the computer to: categorize the first set of documents into a set of categories using a first taxonomy; categorize the second set of documents according to the set of categories of the first set of documents; and examine a category to identify a document of interest, wherein the document of interest typifies the category by most nearly matching a mathematical definition of the category.
  • [0008]
    In a further embodiment of the present invention, a computer program product comprises a computer useable medium including a computer readable program, wherein the computer readable program when executed on a computer causes the computer to: extract a set of documents related to a specified topic from a data warehouse; generate a taxonomy for the set of documents that provides a first partition of the set of documents according to the taxonomy; use domain-specific knowledge to re-partition the set of documents to provide a second partition of the set of documents; and create a refined taxonomy for the set of documents according to the second partition so that the refined taxonomy incorporates the domain-specific knowledge.
  • [0009]
    These and other features, aspects, and advantages of the present invention will become better understood with reference to the following drawings, description, and claims.
  • BRIEF DESCRIPTION OF THE DRAWINGS
  • [0010]
    FIG. 1 is a system block diagram illustrating a system for information mining using domain specific conceptual structures in accordance with an embodiment of the present invention;
  • [0011]
    FIG. 2 is flowchart illustrating phases of a methodology for information mining using domain specific conceptual structures in accordance with an embodiment of the present invention;
  • [0012]
    FIG. 3 is a flowchart illustrating a method of information mining using domain specific conceptual structures in accordance with an embodiment of the present invention;
  • [0013]
    FIG. 4 is an illustration of a graphical user interface for taxonomy generation in the method of FIG. 3; and
  • [0014]
    FIG. 5 is an illustration of a contingency table for a process of the method of FIG. 3.
  • DETAILED DESCRIPTION OF THE INVENTION
  • [0015]
    The following detailed description is of the best currently contemplated modes of carrying out the invention. The description is not to be taken in a limiting sense, but is made merely for the purpose of illustrating the general principles of the invention, since the scope of the invention is best defined by the appended claims.
  • [0016]
    Broadly, embodiments of the present invention provide systems and methods of information mining using domain specific conceptual structures. Embodiments of the present invention provide analytics tools, and methodologies involving those tools, for assisting in finding insights in information by eliciting and capturing domain knowledge as part of the mining process. Embodiments may be used, for example, by businesses for patent portfolio analysis, competitor analysis and white space identification, finding potential licensee markets, identification of experts, and finding potential partnering opportunities.
  • [0017]
    By enabling the use of user domain-specific knowledge to filter information and efficiently and effectively narrow search results, embodiments of the present invention differ from prior art search techniques that do not have a capability to employ such domain-specific knowledge. For example, one embodiment of the present invention goes beyond typical prior art keyword search to find deeper level relationships between collections of documents and to discover important emerging trends and correlations that would otherwise remain hidden. The present invention's incorporation of domain knowledge to capture critical concepts at each stage and to make these domain concepts the focus of the analysis work product stands in contrast to the absence of such in prior art standard text mining techniques.
  • [0018]
    FIG. 1 illustrates information retrieval system 100 in accordance with one embodiment of the present invention. To address the major issues in the information mining process, information retrieval system 100 may utilize a set of analytics tools to allow the use of domain knowledge in the location of information. The analytics tool set may contain a knowledge base data warehouse 102 and a set of analytics tools 104, for identifying information about a specific topic of interest to a user of system 100 (e.g., a topic such as expertise for a group of people) as shown in FIG. 1. The data warehouse 102 may be implemented using computer-readable storage devices 103. Computer-readable storage devices 103 may communicate with a computer system 106 via a network, electronic, or other communication path 108. Computer system 106 may provide an interface for a human user to access data warehouse 102 and may be used to implement and provide a human user interface for analytics tools 104. Data warehouse 102 may contain documents that include, for example, people information as well as other kinds of information that are helpful in analysis. For the example topic of expertise for a group of people, the information could include structured fields on people's titles and ranks. The analytics tools 104 may provide functions including, for example, exploring the data warehouse 102 and analyzing various relationships among documents and people. As indicated in FIG. 1, analytics tools 104 may include capabilities for providing an end user with, for example, taxonomies, name entity extraction, statistical analyses, contingency analyses, and network graph analysis and visualization.
  • [0019]
    Referring now to FIG. 2, a method 200 of one embodiment of the present invention may comprise an investigate phase 201 of processes, a comprehend phase 202 of processes, and an examine phase 203 of processes.
  • [0020]
    The investigate phase 201 may use a search tool, e.g., from the set of analytics tools 104, to extract a set of documents containing information related to a given topic in a specific domain of interest from a data warehouse, e.g., data warehouse 102. Exploration of the data warehouse using the search tool may use structured features, annotations, and unstructured text indexes in combination to select the relevant information for the topic of interest.
  • [0021]
    Then an analytics tool—e.g., from the set of analytics tools 104 and described in more detail below—may convert each document in the extracted document set with a numeric vector that corresponds to the document's dictionary term occurrences (e.g., using the document's word, feature, and structured information content) where the dictionary may be generated based on the frequency of words, phrases, annotations and structured features within each document in the extracted document set, compared to the corpus as a whole. The dictionary may be refined by a user of system 100, if desired.
  • [0022]
    These numeric vectors can then be systematically compared in various ways to determine the similarity of any two documents in the extracted document set to each other. Furthermore, other documents outside the initial extracted document set may be compared with the initial extracted document set via the derived dimensions of the initial extracted document set to determine similarity of the other documents to the initial extracted document set as a whole, or to individual documents within the initial extracted document set.
  • [0023]
    The comprehend phase 202 may use a document classification technology (also called a “taxonomy generation technology”)—e.g., from the set of analytics tools 104 and described in more detail below—to generate naturally occurring categories from the documents of the extracted document set and to classify a set of selected documents from the extracted document set into appropriate categories. Such a taxonomy generation technology may use the numeric vector space and the feature space created for the selected document set. Furthermore, the taxonomy generation technology may use an interactive clustering of the feature space that can help a domain expert (e.g., a user of system 100) refine the categorization if desired.
  • [0024]
    The examine phase 203 may use a contingency table analysis—e.g., from the set of analytics tools 104 and described in more detail below—that compares two taxonomies, or compares a taxonomy against a feature or structured information, such as comparing one taxonomy against a feature over time, or comparing the taxonomy against, e.g., people's titles and ranks (i.e., structured information). The contingency table analysis tool may also enable a detailed category-by-category comparison between two different document sets or two different domain specific conceptual frameworks. Furthermore, a trending tool—e.g., from the set of analytics tools 104 and described in more detail below—that overlays temporal document information on top of document categories may be used to examine the recentness of various aspects of the document information.
  • [0025]
    Referring now to FIG. 3, a method 300 of one embodiment of the present invention may begin with a process 301 of taking a given topic of interest and extracting a set of documents P0 related to the given topic of interest from the data warehouse 102. For example, computer system 106 may be used to extract a set of documents P0 relevant to the topic of interest from a knowledge base (in data warehouse 102) on the given topic using both search operations and query operations, for example, on text, structured fields, and annotations. The topic of interest-related documents P0 may be extracted from the data warehouse 102 using an analytics tool from the set of analytics tools 104. The analytics tool may automatically convert the extracted documents P0 into a numeric vector through words, phrase, and bag of words analysis. For example, computer system 106 through the set of analytics tools 104 may use a method such as that described by U.S. Pat. No. 6,424,971. Such a numeric vector, together with structured information from the documents P0 may then be used for subsequent analysis. It may also be possible that a user of system 100 would only know about one set of initial extracted documents for one or several specific topics so that, as far as the user were concerned, process 301 could be skipped (having been already effectively completed for that user).
  • [0026]
    Continuing with method 300 at process 302, computer system 106 may be used to automatically generate a taxonomy, T1, for the extracted document set P0 using words, bag of words, phrases analysis, and structured and unstructured features.
  • [0027]
    FIG. 4 shows an example of a generated taxonomy 400 for a document set P0, which illustrates the following example. In the example, given a document repository of web pages of companies that includes structured (e.g., dates and URL (uniform resource locater)) and unstructured fields (e.g., web page text), it may be desired to create an analysis of what topics are being discussed and in relation to what companies. To do so, method 300 may (e.g., at process 301) extract a set of relevant documents P0 from the knowledge base (contained, e.g., in data warehouse 102) on a given topic using a query of some kind. For instance, a user of system 100 may be interested in discovering public perception of hybrid electric vehicles. Process 301 may extract the documents P0 related to hybrid electric vehicles from the data warehouse 102. Then (e.g., at process 302) the set of analytics tools 104 may automatically convert the extracted documents P0 into a numeric vector through words, phrase, and bag of words analysis, using a method such as that described in U.S. Pat. No. 6,424,971, referenced above. Such a numeric vector, together with structured information may then be used for subsequent analysis. As above, it may also be possible that end users of system 100 only know about one set of extracted documents on specific topics, which can be taken as the document set P0 so that, as far as such end users were concerned, processes 301 and 302 could be skipped (having been already effectively completed for those users). Using the set of analytics tools 104, users of system 100 can also find the “nearest neighbor”-documents, i.e., the documents P1 most nearly related to P0. Then a combined set of all the documents from both P0 and P1, denoted P0+P1, may need to be examined next.
  • [0028]
    For example, a taxonomy based on the extracted document set P0 can be generated using a method such as that described in U.S. Pat. No. 6,424,971, referenced above. Computer system 106 may be used to automatically classify the document set P0 using automatic taxonomy generation techniques that treat the documents as words, generating a feature space for the document set P0 and using clustering technologies such as K-means and other clustering methods to cluster the documents, as disclosed, for example, by Rasmussen, E., “Clustering Algorithms”, in Frakes, W. B. and Baeza-Yates, R., editors, Information Retrieval, Data Structures and Algorithms, pages 419-442, Prentice Hall, Englewood Cliffs, N.J. (1992). Once the clusters are generated, additional refinements can be made by merging, deleting, or adding classes. The taxonomy may also be used to partition the document set P0, i.e., to divide the document set P0 up into non-overlapping classes.
  • [0029]
    FIG. 4 shows an example of a generated taxonomy 400 for the document set of web pages of companies example given above. Column 401 provides a list of categories (also referred to as “class names” or classifications); column 402 provides, for example, the number of documents of P0 in each category and the percentage the category represents out of the total number of documents of the set P0; and column 403 presents temporal trending information in a graphical format that may show, for example, the relative volume of documents published in each category as a function of time over some pre-determined time-span.
  • [0030]
    At process 303, method 300 may continue by using specific user domain knowledge to locate domain specific concepts, filter out noise and refine the taxonomy T1. In the example illustrated, such user domain knowledge may comprise, for example, some specialized knowledge about hybrid electric vehicles, whether technical, marketing, or regulatory in nature.
  • [0031]
    At process 304, method 300 may continue by classifying the documents of T1 (which may be either the original set or the refined set, e.g., the document set P0 or the document set P0+P1) by structured fields, annotations, or other taxonomies. In other words a second classification or categorization—additional to that of taxonomy T1—may be made of the documents of T1. For example, to link topics with manufacturers, i.e., companies in the illustrative example, a user of system 100 may use analytics tools 104 to build a second taxonomy T2 based either on structured fields of the documents of T1 or on annotations. If the classification is based on a structured field then process 304 may use the URL of the web page to determine the domain name where the web page was originally stored. Another way to make a second classification may be to extract company names using a name-annotation step over the document set of T1 to extract names out of the documents. In either case the result may be a new taxonomy T2 based on companies. The second taxonomy T2 can then be compared to the original taxonomy T1 (shown in FIG. 4) by creating a contingency table, such as contingency table 500 shown in FIG. 5.
  • [0032]
    Method 300 may continue at process 305 by using contingency analysis to generate a contingency table C0 that compares the refined taxonomy T1 and the second classification based on process 304. The contingency table C0— such as contingency table 500 illustrated in FIG. 5—may display a first set of categories (e.g., the companies or automakers of the example) across one axis (e.g., horizontal axis 501 so that each column of the table 500 corresponds to an automaker) and the second set of categories or classifications along the other axis (e.g., vertical axis 502 so that each row of the table 500 corresponds to one of the classifications listed in the “Class” column of table 500). The cells (e.g., cell 503) may indicate the number of documents that occur at the intersection of the first and second classifications, e.g., automaker vs. class of the “Class” column. Each cell—such as cell 503—may have an expected value which can be calculated based on the size 507 of the second category—such as category 504, “fuel economy” for cell 503, and having size 507 equal to 691 in the example—and the total number of documents for the cell, which may also depend on the first (e.g., “automaker”) category for the cell—such as category 505 for cell 503, having the particular value of “honda” in the illustrated example.
  • [0033]
    For example, an expected value percentage for each cell may be calculated as (percent of the class's documents out of the total number for the class)×(percent of the documents in the class out of the total number of documents in all classes). If the expected value is exceeded by the actual value in the cell, then the cell may be shaded. For example, cell 503 is illustrated with a moderate shading that matches moderate affinity shading 508. The degree of shading may indicate the degree of significance of the cell's value, which may be calculated, for example, using a statistical test, such as the well-known chi-squared test (see Press, et al., “Numerical Recipes in C, second edition”, New York, Cambridge University Press, (1992), pages 620-623). Shading of each cell may indicate a significant relationship between the class (second category) corresponding to the cell and the first—or horizontal axis—category corresponding to the cell, and the degree of shading may indicate the degree of significance of the relationship. For example, the moderate affinity shading of cell 503 may indicate a moderate degree (as compared to “very high” 509, “low” 510, and “no” 511 degrees of affinity illustrated in FIG. 5) of significance between class 504 (e.g., “fuel economy”) and category 505 (e.g., “honda”) of cell 503. Each cell may also contain a checkbox, which may be checked—as shown for cell 503—for any desired special purpose, such as to draw specific attention to the particular cell.
  • [0034]
    Method 300 may continue at process 306 by finding significant relationships using contingency tables C0. For example, looking at typical examples within each category along axis 501 (first category) of contingency table C0 500, process 306 may produce the following observations:
  • [0035]
    1. Honda and Toyota are the most frequently discussed, with GM following.
  • [0036]
    2. VW was the least frequently discussed.
  • [0037]
    3. Discussion seems to center around sales of Honda and Toyota models.
  • [0038]
    4. Honda discussions are highly associated with the topic of fuel economy.
  • [0039]
    5. New models and future plans were the frequent topics when GM was discussed.
  • [0040]
    6. Web content that discussed HEV's and SUV's frequently mentioned Ford.
  • [0041]
    Method 300 may continue at process 307 by overlaying trending information on top of document taxonomy T1, and contingency tables C0. For example, FIG. 4 shows trending information 403 overlaid on a taxonomy T1. Trending information—such as trending information 403—may be added to each cell of a contingency table C0. For example, each cell (e.g., cell 503) of contingency table 500 may contain trending information in graphical form, which may be similar in description to that given above for trending information 403. In the illustrative example given, the trend lines in each cell of contingency table 500 may be based on the structured information relating to the date of each web page from the document set P0. The structured information relating to the date of each web page may then be converted into a trend graph showing how the contents of each cell tracks over time. Such trend graphs may help reveal recent categories, e.g., a category with a majority of documents published subsequent to some pre-determined date—such as within the past month.
  • [0042]
    Method 300 may continue at process 308 by identifying recent and most related categories (for example, first categories most related to second categories) using, for example, contingency table 500. “Recent” may defined as desired—for example, as more recent than 5 years, more recent than 1 year, more recent than 3 months, or the like. Similarly, “most related” may be defined as deemed appropriate, using the distance of nearest neighbor methodologies, for example, cosine distance in the feature space, as a measure of most closely related; or, for example, using a statistical correlation or likelihood value from the contingency table C0 (e.g., contingency table 500) as a different measure of “most related”.
  • [0043]
    In addition to identifying categories of interest (e.g., recent or most related), method 300 may also identify a document of interest in a particular category. For example, a document of interest might be one that typifies the category in the sense of being an “average” document. Such an average may be defined, for example, as a centroid of the category using the distance of a nearest neighbor methodology, e.g., cosine distance in the feature space. There may not actually be a “typical” or “average” document that matches the centroid, so the document of interest may be identified, for example, as the document closest to the centroid in the feature space. As an alternative example, the documents of interest may be identified as any documents within a pre-specified distance of the centroid in the feature space. Such a criterion may provide a mathematical definition of the category, and the document of interest may be said to be the document that most nearly matches the mathematical definition of the category. In the same sense, a document of interest may be said to be a document that fits the category model well, or that is a “typical” document within the category, i.e., representative of documents within the category.
  • [0044]
    At process 309, method 300 may repeat processes 304-308 by using different structured fields, features, or other information to compare the document sets in various ways. In other words, by iterating processes 304-308, multiple comparisons between all the different categories and between different types of categories (e.g., first categories and second categories) may be obtained.
  • [0045]
    The invention can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment containing both hardware and software elements. In a preferred embodiment, the invention is implemented in software, which includes but is not limited to firmware, resident software, microcode, etc.
  • [0046]
    Furthermore, the invention can take the form of a computer program product accessible from a computer-usable or computer-readable medium providing program code for use by or in connection with a computer or any instruction execution system. For the purposes of this description, a computer-usable or computer readable medium can be any apparatus that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device.
  • [0047]
    The medium can be an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system (or apparatus or device) or a propagation medium. Examples of a computer-readable medium include a semiconductor or solid state memory, magnetic tape, a removable computer diskette, a random access memory (RAM), a read-only memory (ROM), a rigid magnetic disk and an optical disk. Current examples of optical disks include compact disk-read only memory (CD-ROM), compact disk-read/write (CD-R/W) and DVD.
  • [0048]
    A data processing system suitable for storing and/or executing program code will include at least one processor coupled directly or indirectly to memory elements through a system bus. The memory elements can include local memory employed during actual execution of the program code, bulk storage, and cache memories which provide temporary storage of at least some program code in order to reduce the number of times code must be retrieved from bulk storage during execution.
  • [0049]
    Input/output or I/O devices (including but not limited to keyboards, displays, pointing devices, etc.) can be coupled to the system either directly or through intervening I/O controllers. Network adapters may also be coupled to the system to enable the data processing system to become coupled to other data processing systems or remote printers or storage devices through intervening private or public networks. Modems, cable modem and Ethernet cards are just a few of the currently available types of network adapters.
  • [0050]
    It should be understood, of course, that the foregoing relates to exemplary embodiments of the invention and that modifications may be made without departing from the spirit and scope of the invention as set forth in the following claims.

Claims (2)

1. A method for use with 1) a first set of documents related to a first topic of interest and 2) a second set of documents related to a second topic of interest, the method comprising the steps of:
automatically generating a first taxonomy through a feature space derived from the first set of documents, wherein:
the feature space includes at least one of unstructured data, structured data, and annotations derived from text of the first set of documents; and
the first taxonomy provides a first partition of the set of documents according to the taxonomy;
using domain-specific knowledge to re-partition the first set of documents to provide a second partition of the first set of documents;
creating a refined taxonomy for the first set of documents according to the second partition so that the refined taxonomy incorporates the domain-specific knowledge;
using the refined taxonomy to categorize the first set of documents into a first set of categories;
creating a second set of categories of the first set of documents, independent of the second partition and according to at least one of unstructured data, structured data, and annotations derived from text in the first set of documents;
constructing a contingency table having the first set of categories along a first axis and the second set of categories along a second axis, wherein:
the contingency table includes cells having respective actual values and for which respective expected values are computed; and
the contingency table includes a cell having trending information;
comparing the expected value against the actual value of a cell to identify a category of interest;
computing a degree of significance for the actual value of the cell;
identifying a relationship between at least two different categories using the contingency table;
using the contingency table and trending information to identify a recent category with respect to some pre-determined date;
using an element of domain knowledge to re-categorize the first set of documents;
categorizing the second set of documents according to the first set of categories of the first set of documents, further including categorizing the second set of documents according to a criterion chosen from the group consisting of: text within the second set of documents, structure within the second set of documents, and annotations derived from text within the second set of documents;
examining the first set of categories to identify correlations between categories; and
examining a category of the first set of categories to identify a document of interest, the document of interest being a representative document within the category.
2-20. (canceled)
US11674601 2007-02-13 2007-02-13 Information mining using domain specific conceptual structures Abandoned US20080195567A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US11674601 US20080195567A1 (en) 2007-02-13 2007-02-13 Information mining using domain specific conceptual structures

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US11674601 US20080195567A1 (en) 2007-02-13 2007-02-13 Information mining using domain specific conceptual structures
US12132515 US8805843B2 (en) 2007-02-13 2008-06-03 Information mining using domain specific conceptual structures

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US12132515 Continuation US8805843B2 (en) 2007-02-13 2008-06-03 Information mining using domain specific conceptual structures

Publications (1)

Publication Number Publication Date
US20080195567A1 true true US20080195567A1 (en) 2008-08-14

Family

ID=39686716

Family Applications (2)

Application Number Title Priority Date Filing Date
US11674601 Abandoned US20080195567A1 (en) 2007-02-13 2007-02-13 Information mining using domain specific conceptual structures
US12132515 Active 2029-10-28 US8805843B2 (en) 2007-02-13 2008-06-03 Information mining using domain specific conceptual structures

Family Applications After (1)

Application Number Title Priority Date Filing Date
US12132515 Active 2029-10-28 US8805843B2 (en) 2007-02-13 2008-06-03 Information mining using domain specific conceptual structures

Country Status (1)

Country Link
US (2) US20080195567A1 (en)

Cited By (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090055242A1 (en) * 2007-08-24 2009-02-26 Gaurav Rewari Content identification and classification apparatus, systems, and methods
US20090055368A1 (en) * 2007-08-24 2009-02-26 Gaurav Rewari Content classification and extraction apparatus, systems, and methods
US20100145940A1 (en) * 2008-12-09 2010-06-10 International Business Machines Corporation Systems and methods for analyzing electronic text
US20100287478A1 (en) * 2009-05-11 2010-11-11 General Electric Company Semi-automated and inter-active system and method for analyzing patent landscapes
US20110087646A1 (en) * 2009-10-08 2011-04-14 Nilesh Dalvi Method and System for Form-Filling Crawl and Associating Rich Keywords
US20110113063A1 (en) * 2009-11-09 2011-05-12 Bob Schulman Method and system for brand name identification
US20110112824A1 (en) * 2009-11-06 2011-05-12 Craig Peter Sayers Determining at least one category path for identifying input text
US20110221367A1 (en) * 2010-03-11 2011-09-15 Gm Global Technology Operations, Inc. Methods, systems and apparatus for overmodulation of a five-phase machine
US20110302179A1 (en) * 2010-06-07 2011-12-08 Microsoft Corporation Using Context to Extract Entities from a Document Collection
US8463789B1 (en) * 2010-03-23 2013-06-11 Firstrain, Inc. Event detection
US20140180934A1 (en) * 2012-12-21 2014-06-26 Lex Machina, Inc. Systems and Methods for Using Non-Textual Information In Analyzing Patent Matters
US8782042B1 (en) 2011-10-14 2014-07-15 Firstrain, Inc. Method and system for identifying entities
US8805840B1 (en) 2010-03-23 2014-08-12 Firstrain, Inc. Classification of documents
US8977613B1 (en) 2012-06-12 2015-03-10 Firstrain, Inc. Generation of recurring searches

Families Citing this family (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8041696B2 (en) * 2008-06-13 2011-10-18 International Business Machines Corporation Idea tracking and management
US8825646B1 (en) * 2008-08-08 2014-09-02 Google Inc. Scalable system for determining short paths within web link network
US20100332285A1 (en) * 2009-06-24 2010-12-30 International Business Machines Corporation Intellectual Property Component Business Model for Client Services
US8275646B2 (en) * 2009-07-08 2012-09-25 International Business Machines Corporation Intellectual property assessments based on component business models
US8977620B1 (en) * 2011-12-27 2015-03-10 Google Inc. Method and system for document classification
US9275331B2 (en) * 2013-05-22 2016-03-01 International Business Machines Corporation Document classification system with user-defined rules
US9760592B2 (en) 2014-02-20 2017-09-12 International Business Machines Corporation Metrics management and monitoring system for service transition and delivery management

Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20010037324A1 (en) * 1997-06-24 2001-11-01 International Business Machines Corporation Multilevel taxonomy based on features derived from training documents classification using fisher values as discrimination values
US6397205B1 (en) * 1998-11-24 2002-05-28 Duquesne University Of The Holy Ghost Document categorization and evaluation via cross-entrophy
US6424971B1 (en) * 1999-10-29 2002-07-23 International Business Machines Corporation System and method for interactive classification and analysis of data
US20030084066A1 (en) * 2001-10-31 2003-05-01 Waterman Scott A. Device and method for assisting knowledge engineer in associating intelligence with content
US7117207B1 (en) * 2002-09-11 2006-10-03 George Mason Intellectual Properties, Inc. Personalizable semantic taxonomy-based search agent
US7139754B2 (en) * 2004-02-09 2006-11-21 Xerox Corporation Method for multi-class, multi-label categorization using probabilistic hierarchical modeling
US20070255731A1 (en) * 2001-10-29 2007-11-01 Maze Gary R System and method for locating, categorizing, storing, and retrieving information
US20070294200A1 (en) * 1998-05-28 2007-12-20 Q-Phrase Llc Automatic data categorization with optimally spaced semantic seed terms
US7475018B1 (en) * 2000-03-16 2009-01-06 Swiss Reinsurance Company Method for structuring unstructured domains to create value

Family Cites Families (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6442545B1 (en) * 1999-06-01 2002-08-27 Clearforest Ltd. Term-level text with mining with taxonomies
US7035385B2 (en) 2002-03-29 2006-04-25 Bellsouth Intellectual Property Corporation Method and system for screening calls during voicemail messaging
DE10215495A1 (en) 2002-04-09 2003-10-30 Bayer Ag Computer system and method to research, statistical analysis, and analysis of documents
US7792832B2 (en) 2002-10-17 2010-09-07 Poltorak Alexander I Apparatus and method for identifying potential patent infringement
US20050171948A1 (en) * 2002-12-11 2005-08-04 Knight William C. System and method for identifying critical features in an ordered scale space within a multi-dimensional feature space
GB2401206A (en) 2003-04-29 2004-11-03 Infinite Reason Ltd Object orientated data processing system
US7610313B2 (en) * 2003-07-25 2009-10-27 Attenex Corporation System and method for performing efficient document scoring and clustering
US7505989B2 (en) * 2004-09-03 2009-03-17 Biowisdom Limited System and method for creating customized ontologies
US20060242190A1 (en) * 2005-04-26 2006-10-26 Content Analyst Comapny, Llc Latent semantic taxonomy generation
US7668789B1 (en) * 2006-03-30 2010-02-23 Hewlett-Packard Development Company, L.P. Comparing distributions of cases over groups of categories

Patent Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20010037324A1 (en) * 1997-06-24 2001-11-01 International Business Machines Corporation Multilevel taxonomy based on features derived from training documents classification using fisher values as discrimination values
US20070294200A1 (en) * 1998-05-28 2007-12-20 Q-Phrase Llc Automatic data categorization with optimally spaced semantic seed terms
US6397205B1 (en) * 1998-11-24 2002-05-28 Duquesne University Of The Holy Ghost Document categorization and evaluation via cross-entrophy
US6424971B1 (en) * 1999-10-29 2002-07-23 International Business Machines Corporation System and method for interactive classification and analysis of data
US7475018B1 (en) * 2000-03-16 2009-01-06 Swiss Reinsurance Company Method for structuring unstructured domains to create value
US20070255731A1 (en) * 2001-10-29 2007-11-01 Maze Gary R System and method for locating, categorizing, storing, and retrieving information
US20030084066A1 (en) * 2001-10-31 2003-05-01 Waterman Scott A. Device and method for assisting knowledge engineer in associating intelligence with content
US7117207B1 (en) * 2002-09-11 2006-10-03 George Mason Intellectual Properties, Inc. Personalizable semantic taxonomy-based search agent
US7139754B2 (en) * 2004-02-09 2006-11-21 Xerox Corporation Method for multi-class, multi-label categorization using probabilistic hierarchical modeling

Cited By (22)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090055242A1 (en) * 2007-08-24 2009-02-26 Gaurav Rewari Content identification and classification apparatus, systems, and methods
US20090055368A1 (en) * 2007-08-24 2009-02-26 Gaurav Rewari Content classification and extraction apparatus, systems, and methods
US20100145940A1 (en) * 2008-12-09 2010-06-10 International Business Machines Corporation Systems and methods for analyzing electronic text
WO2010066616A1 (en) * 2008-12-09 2010-06-17 International Business Machines Corporation Systems and methods for analyzing electronic text
US8606815B2 (en) 2008-12-09 2013-12-10 International Business Machines Corporation Systems and methods for analyzing electronic text
US20100287478A1 (en) * 2009-05-11 2010-11-11 General Electric Company Semi-automated and inter-active system and method for analyzing patent landscapes
US8412659B2 (en) * 2009-05-11 2013-04-02 General Electric Company Semi-automated and inter-active system and method for analyzing patent landscapes
US20110087646A1 (en) * 2009-10-08 2011-04-14 Nilesh Dalvi Method and System for Form-Filling Crawl and Associating Rich Keywords
US8793239B2 (en) 2009-10-08 2014-07-29 Yahoo! Inc. Method and system for form-filling crawl and associating rich keywords
US20110112824A1 (en) * 2009-11-06 2011-05-12 Craig Peter Sayers Determining at least one category path for identifying input text
US20110113063A1 (en) * 2009-11-09 2011-05-12 Bob Schulman Method and system for brand name identification
US20110221367A1 (en) * 2010-03-11 2011-09-15 Gm Global Technology Operations, Inc. Methods, systems and apparatus for overmodulation of a five-phase machine
US8805840B1 (en) 2010-03-23 2014-08-12 Firstrain, Inc. Classification of documents
US8463790B1 (en) 2010-03-23 2013-06-11 Firstrain, Inc. Event naming
US8463789B1 (en) * 2010-03-23 2013-06-11 Firstrain, Inc. Event detection
US9760634B1 (en) 2010-03-23 2017-09-12 Firstrain, Inc. Models for classifying documents
US20110302179A1 (en) * 2010-06-07 2011-12-08 Microsoft Corporation Using Context to Extract Entities from a Document Collection
US9251248B2 (en) * 2010-06-07 2016-02-02 Microsoft Licensing Technology, LLC Using context to extract entities from a document collection
US8782042B1 (en) 2011-10-14 2014-07-15 Firstrain, Inc. Method and system for identifying entities
US8977613B1 (en) 2012-06-12 2015-03-10 Firstrain, Inc. Generation of recurring searches
US9292505B1 (en) 2012-06-12 2016-03-22 Firstrain, Inc. Graphical user interface for recurring searches
US20140180934A1 (en) * 2012-12-21 2014-06-26 Lex Machina, Inc. Systems and Methods for Using Non-Textual Information In Analyzing Patent Matters

Also Published As

Publication number Publication date Type
US20080243889A1 (en) 2008-10-02 application
US8805843B2 (en) 2014-08-12 grant

Similar Documents

Publication Publication Date Title
Jansen et al. Determining the informational, navigational, and transactional intent of Web queries
Middleton et al. Ontological user profiling in recommender systems
Hammouda et al. Efficient phrase-based document indexing for web document clustering
US7657522B1 (en) System and method for providing information navigation and filtration
US6687696B2 (en) System and method for personalized search, information filtering, and for generating recommendations utilizing statistical latent class models
Kuhn et al. Semantic clustering: Identifying topics in source code
US20060047649A1 (en) Internet and computer information retrieval and mining with intelligent conceptual filtering, visualization and automation
Losiewicz et al. Textual data mining to support science and technology management
US20070106662A1 (en) Categorized document bases
US20090070322A1 (en) Browsing knowledge on the basis of semantic relations
Aleman-Meza et al. Context-aware semantic association ranking
US6772148B2 (en) Classification of information sources using graphic structures
US20080140644A1 (en) Matching and recommending relevant videos and media to individual search engine results
Yang et al. Text mining and visualization tools–impressions of emerging capabilities
US20060089924A1 (en) Document categorisation system
US20110072047A1 (en) Interest Learning from an Image Collection for Advertising
US20060230033A1 (en) Searching through content which is accessible through web-based forms
US6711585B1 (en) System and method for implementing a knowledge management system
US20090119281A1 (en) Granular knowledge based search engine
US20060173839A1 (en) Method for re-ranking documents retrieved from a multi-lingual document database
US20040002973A1 (en) Automatically ranking answers to database queries
US7013300B1 (en) Locating, filtering, matching macro-context from indexed database for searching context where micro-context relevant to textual input by user
Trippe Patinformatics: Tasks to tools
US20100005061A1 (en) Information processing with integrated semantic contexts
US20090006382A1 (en) System and method for measuring the quality of document sets

Legal Events

Date Code Title Description
AS Assignment

Owner name: INTERNATIONAL BUSINESS MACHINES CORPORATION, NEW Y

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:CHEN, YING;KREULEN, JEFFREY THOMAS;RHODES, JAMES J.;AND OTHERS;SIGNING DATES FROM 20070201 TO 20070202;REEL/FRAME:018887/0316