Classifications

• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
  • G06N5/00—Computing arrangements using knowledge-based models
  • G06N5/02—Knowledge representation; Symbolic representation
  • G06N5/022—Knowledge engineering; Knowledge acquisition
  • G06N5/025—Extracting rules from data
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
  • G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
  • G06F16/35—Clustering; Classification
  • G06F16/353—Clustering; Classification into predefined classes

Definitions

• This invention relates to apparatus for classifying documents.
• US 5,461,488 provides routing based on identification of recipient name.
• classification strategy either profile-based, having a keyword/character profile for each
• the systems also use only a single knowledge acquisition strategy, either statistically
• classifier using the knowledge base, determines a predicted classification for the
• the classifier being switchable between the modes under user control .
• the features are preferably formed into a feature vector for input to the classifier and the
• features preferably comprise classification-associated words or phrases which may appear
• the extracting means may be arranged to provide a measure of the
• the classifier may comprise a supervised ART system, preferably an ARAM system of
• the apparatus may further be operable in knowledge acquisition mode to process a
• the apparatus may further be operable in a rule insertion sub-mode of the knowledge
• the apparatus may further comprising a router arranged to route the document to one of a
• the desc ⁇ bed embodiment provides a document classification apparatus which allows
• the apparatus perform learning of a plurality of cases as a batch. During batch learning, the apparatus learns each case one by one and accumulates the classification information into the
• the apparatus Besides learning from training data, the apparatus also allows rules to be
• the apparatus is furthermore able to determine a confidence that the classification of a
• This confidence value is
• Figure 1 is a schematic diagram illustrating the structure of the described embodiments of
• Figure 2 is a diagram illustrating the document classifier of Figure 1 in a document
• Figure 3 is a diagram illustrating the modes of operation of the embodiments of the invention.
• Figure 4 is a diagram of an ARAM system used as a document classifier in an
• apparatus is operable in a knowledge acquisition mode and a document classification mode.
• knowledge acquisition mode the apparatus learns from training documents and
• a document text file for example a text file derived from a scanned and OCR processed
• the document classifier includes a feature extraction
• ARAM Adaptive Resonance Associative Map
• This classification is associated with a confidence value which, together
• threshold input by a system administrator 50. If the value exceeds the threshold, the threshold
• the document is routed to the system administrator 50 for manual routing via path 60.
• the destinations 52 can also communicate with the system administrator 50 through path
• the acquisition mode two sub-modes are used.
• the first, represented by block 100, is based
• rule insertion is based on rule insertion, represented by block 110.
• rule insertion a feature vector
• the system administrator can access the document classifier directly by via path 70 to
• Such switching may be used, for example, if a mis-directed
• the system administrator may
• the features are extracted and passed to the classifier 30, so that the mis-directed document
• tokens are individual words that have been
• selection is “select”).
• Other “filtering” options based on sentence structure such as
• the keyword-based feature sets can be pre-defined manually or generated automatically
• the algorithm for automatic keyword selection accepts a list of pre-classified (i.e.
• Processing involves the extraction of all nouns (in root form) from each document and
• f rat ⁇ ng a selection rating
• Count (x) Total number of occurrences of keyword in category x
• Count (x) Total number of occurrences of keyword in document x
• the keyword-per-document ratio (f Ratl0 ,) for a category, i, is the total keyword count (C,) for the
• Relative Keyword Count thus gives an indication of the difference between the keyword-
• the keyword occurs at least once.
• a measurement of f DIR for D lsl documents in the top category is given by:
• O lst number of documents in top category in which keyword occurs. The overall ranking of each keyword is therefore simply derived by taking:
• the following example uses a small training set of two categories with 124 relevant
• the categories are business newspaper articles in the first category and
• the algorithm allows for the specification of a minimum number K, of non-zero keyword
• keywords are selected in the manner described above.
• the feature extraction algorithm parses the
• Keyword counts are then normalized such that the maximum score is 1 and the minimum
• the second article for Category 2 (sports, music and
• the Classifier Adaptive Resonance Associative Map (ARAM)
• ARAM is a family of neural network models that performs incremental supervised
• the F, ⁇ field (300) serves as the input field containing the input activity vector
• F2 field (320) contains the activities of categories that are used to encode the patterns.
• ARAM formulates recognition categories of input
• ARAM discovers during learning is compatible with IF-THEN rule-based representation.
• each node in the F 2 field (320) represents a recognition category associating
• node constitute a set of rules that link antecedents to consequents.
• system architecture can be translated into a compact set
• the ART modules used in ARAM can be ART 1 [1] , which categorizes binary patterns,
• ART 2-A [2] ART 2-A [2]
• fuzzy ART [3] which categorize both
• fuzzy ARAM model that is composed of two overlapping
• ARAM learns a set of
• recognition categories or rules by training from pre-labeled document sets.
• ARAM as input A one at a time together with the associated class label input B.
• ARAM Given an input keyword vector A, ARAM first searches for a F 2 recognition category
• templates of the F 2 recognition category is modified to contain the intersection of the
• ARAM learning is stable in the sense that weight values do not oscillate, as
• Input vectors The F, a and F, input vectors are normalized by complement coding that
• Complement coding represents both the on-response and
• the complement coded F, a input vector A is a 2M-
• the complement coded F, b input vector B is a 2N-dimensional vector
• Each F 2 category node j is associated with two adaptive weight templates
• a category node is selected for encoding, it becomes committed.
• Fuzzy ARAM dynamics are determined by the choice parameters ⁇ , > 0 and ⁇ b > 0; the
• T J
• I is defined by
• ⁇ , p, for vectors p and q.
• T j max ⁇ T ; : for all F 2 node j ⁇ .
• Resonance occurs if the match functions, m, ⁇ and m ; , meet the
• mismatch reset occurs in which the value of the choice function T j is set to 0 for the
• J is an uncommitted node, and then take ⁇ a ⁇ 1 and ⁇ b ⁇ 1 after the category node is
• a rule is typically in the IF-THEN format
• the rule insertion algorithm creates a keyword frequent vector in which the frequency
• ARAM first searches
• a recognition category is created to encode a keyword template consisting of "Stock”, “Share ", and
• Rule insertion proceeds in two phases.
• the first phase translates each rule into a 2M-
• N is the number of classes.
• the vector pairs derived from the rules are then used as training patterns to initialize an
• recognition categories are associated through the map field.
• the vigilance parameters p a and p b are each set to 1 to ensure
• a feature extraction module parses the text to derive a
• vector A is then presented to the F, a field.
• ARAM Given an input keyword vector A, ARAM first searches for a F 2 recognition category
• the output class is predicted to be K if b ⁇ > b k for
• output vector y represents a less extreme contrast enhancement of the F, a to F 2 input T, in
• the power rule raises the input T j to the j m F 2 node to a power p and
• the power rule converges toward the choice rule as p becomes large.
• K-max Rule In the spirit of the K Nearest Neighbor (KNN) system, the K-max rule picks
• the F 2 activity values y are
• the output vector B 2 (b,, b 2 ...., b N ) is then read directly from x b .
• the output class is
• voting scores normalized by the number of ARAM provide a prediction score
• v is the number of votes given to and S j is the normalized prediction score for the
• the output class j is the selected predicted
• the system administrator can switch between the classification mode and the knowledge
• the message can be either LEARN, INSERT, or CLASSIFY.
• the document classifier adjusts the input baseline
• the document classifier receives a document text together with a
• the feature extraction module derives the normalized keyword
• the keyword vector is presented as the input vector to
• the ARAM classifier is then run with p_ a ⁇ 1 (typically
• the document classifier receives an IF-THEN rule.
• the ARAM classifier is
• the document classifier receives a document text.
• classification label is then read from the F, b field and returned to the user.
• classifier module has been shown implemented using an ARAM structure, this may be

Landscapes

• Engineering & Computer Science (AREA)
• Theoretical Computer Science (AREA)
• General Engineering & Computer Science (AREA)
• Physics & Mathematics (AREA)
• General Physics & Mathematics (AREA)
• Data Mining & Analysis (AREA)
• Databases & Information Systems (AREA)
• Artificial Intelligence (AREA)
• Computational Linguistics (AREA)
• Evolutionary Computation (AREA)
• Computing Systems (AREA)
• Mathematical Physics (AREA)
• Software Systems (AREA)
• Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Applications Claiming Priority (1)

Publications (1)

Family

ID=20430235

Family Applications (1)

Country Status (2)

Cited By (1)

Families Citing this family (12)

Family Cites Families (8)

• 1999
  • 1999-08-25 WO PCT/SG1999/000089 patent/WO2001014992A1/en active Application Filing
  • 1999-08-25 EP EP99946569A patent/EP1210669A1/de not_active Withdrawn

Non-Patent Citations (1)

Also Published As

Similar Documents

Legal Events