KR20050096912A - Method and apparatus for automatically determining salient features for object classification - Google Patents

Abstract

A method and apparatus for automatically determining salient features (308) for object classification is provided. In accordance with one embodiment, one or more unique features are extracted from a first content group of objects to form a first feature list, and one or more unique features are extracted from a second anti- content group of objects to form a second feature list. A ranked list of features is then created by applying statistical differentiation between unique features of the first feature list and unique features of the second feature list. A set of salient features (308) is then identified from the resulting ranked list of features.

Description

Method and apparatus for automatically determining salient features for object classification {METHOD AND APPARATUS FOR AUTOMATICALLY DETERMINING SALIENT FEATURES FOR OBJECT CLASSIFICATION}

The present invention relates to the field of data processing. More specifically, the present invention relates to the automatic selection of object features used to classify objects into groups.

The World Wide Web provides an important information resource with approximately billions of pages of information available for online viewing and downloading. However, to use this information efficiently, a smart way to navigate these vast data spaces is essential.

In the early days of surfing the Internet, two basic methods were developed to support web browsing. The first approach is to create an indexed database based on the content of web pages collected by automated search engines that "crawl" the web to find new and unique pages. These databases can then be searched using various query techniques, and are often ranked based on the similarity of the query types. The second approach is to group web pages into a categorization hierarchy, typically represented in tree form. The user then descends the hierarchy, making a series of selections using two or more selections at each level that represent a noticeable difference between the subtrees below the decision point, ultimately a page of text and / or multimedia content. Reach leaf nodes containing them.

For example, FIG. 1 shows that a plurality of decision nodes (hereinafter referred to as nodes "nodes") 130-136 may comprise multiple parent nodes and / or child nodes-they Each represents an exemplary prior art subject hierarchy 102 arranged hierarchically, with associated with a unique subject category. For example, node 130 is a parent node of nodes 131, 132, and nodes 131, 132 are child nodes of node 130. Since nodes 131, 132 are both child nodes of the same node (eg, node 130), nodes 131, 132 may be said to be siblings of each other. Additional sibling pairs in the topic hierarchy 102 include nodes 133 and 134 and nodes 135 and 136. Node 130 forms first level 137 of thematic hierarchy 102, nodes 131, 132 form second level 138 of thematic hierarchy 102, and node 133-. It can be seen from FIG. 1 that 136 forms the third level 139 of the topical hierarchy 102. Node 130 is also referred to as the root node of thematic hierarchy 102 in that it is not a child of any other node.

There are many challenges in the process of creating hierarchical categorization of web pages. First, the nature of the hierarchy must be defined. This is typically done manually by a skilled person within a particular subject area, in a manner similar to creating categories of the Dewey Decimal System used in libraries. Then, by providing descriptive labels to these categories, users and categorizers can make appropriate decisions while navigating the hierarchy. Then, the content in the form of individual electronic documents is placed in the categories by manual search through the hierarchy.

Recently, automating the various steps of this process has drawn attention. There is a system for automatically categorizing documents from document material. For example, some systems automatically take advantage of the keywords associated with documents to automatically cluster or group similar documents. Such clusters can be repeatedly grouped into super-clusters to create a hierarchical structure, but such systems require a manual main insertion and create a hierarchical structure that does not have a systematic structure. If a hierarchy is used for manual retrieval, labels must be attached to nodes in the hierarchy by manual inspection of sub-nodes or end documents to identify common feature (s).

Many classification systems use lists of salient words to classify documents. Typically, prominent words are selected or predefined from the document being processed to more accurately describe the characteristics of the documents. Typically these prominent word lists are generated by counting the frequency of occurrence of all words for each document set. The words are then removed from the word lists according to one or more criteria. Often, words that rarely occur in the material are removed because they are so rare that they cannot be reliably distinguished between categories, while words that occur too often occur such words are common in all documents across categories. Because it is considered to be done, it is removed.

In addition, "stop words" and word stems are often removed from the feature list to facilitate determination of salient features. Stopwords include common words in languages such as "a", "the", "his", and "and" that do not convey meaningful content, and the word stem contains "-ing", "-end" suffixes such as "-is", and "-able". Unfortunately, the generation of stopwords and word stem lists is a language-specific task that requires expert knowledge of syntax, grammar, and usage that changes over time. Thus, a more flexible way of determining salient features is desirable.

The invention is described through, but is not limited to, exemplary embodiments shown in the accompanying drawings in which like reference numerals designate like elements.

1 illustrates an exemplary prior art thematic hierarchy comprising a plurality of decision nodes.

2A-2C illustrate the operational flow of a salient feature determination function, in accordance with an embodiment of the invention.

3 illustrates an exemplary application of salient feature determination functions in the present invention, according to one embodiment.

4 illustrates a functional block diagram of the classifier training service of FIG. 3, in accordance with an embodiment of the present invention.

5 illustrates an example computer system suitable for use in determining salient features, in accordance with an embodiment of the present invention.

In the following description, various aspects of the invention will be described. However, it will be apparent to one skilled in the art that the present invention may be implemented in only some or all aspects of the invention. For purposes of explanation, specific numbers, materials, and configurations are set forth in order to provide a thorough understanding of the present invention. However, it will be apparent to one skilled in the art that the present invention may be practiced without the specific details. In other instances, well-known features are omitted or simplified in order not to obscure the present invention.

Portions of the description are terms such as data, storage, selection, determination, calculation, etc., consistent with the manner generally employed by those skilled in the art to convey the gist of their work to others skilled in the art. To the operations executed by the processor-based device. As will be appreciated by those skilled in the art, quantities may take the form of electrical, magnetic or optical signals that can be stored, transmitted, combined and otherwise manipulated through the mechanical and electrical components of the processor-based device; The term processor includes microprocessors, microcontrollers, digital signal processors, and the like, which may be independent, incidental, or embedded.

Various operations will be described, in turn, as a plurality of individual steps, in a manner that may best aid the understanding of the present invention, but the order of description should not be understood to mean that such operations are necessarily order dependent. In particular, these operations need not be performed in the order described. In addition, the description uses the phrase “in one embodiment” repeatedly, which may refer to the same embodiment but does not usually refer to the same embodiment.

According to one embodiment of the invention, one or more unique features are extracted from a first group of objects to form a first feature set, and one or more unique features are extracted from a second group of objects to form a second feature set. . The ranked feature list is then generated by applying statistical differentiation between the unique features of the first feature set and the unique features of the second feature set. The set of salient features is then distinguished from the resulting ranked list of features.

In one embodiment, video sequences, audio, including both proprietary and non-private forms in ultra-large hierarchical classification trees, as well as in non-hierarchical data structures such as flat files. Outstanding features are determined to facilitate efficient classification and categorization of data objects, including but not limited to sequences, image files, text files, and the like. For example, features in a text file may take the form of words that are generally understood to represent a group of characters in which the term "word" has some semantic meaning within a given language. . More generally, the feature may be an N-token gram, where the token is, for example, N-letter grams and N-word grams in English, for example. ), As well as one atomic element of the language, including N-ideogram grams in Asian languages. For example, in audio sequences, notes, intonation, tempo, duration, pitch, volume, etc. may be utilized as features for classifying audio, and in video sequences and still images Various pixel attributes such as color, chrominance and luminance levels can be utilized as features. According to one embodiment of the invention, if a group of features has been identified, for example from a group of electronic documents, then it is determined that a subset of those features stands out for the purpose of classifying a certain group of data objects. The term "electronic document" is widely used herein to describe a family of data objects, such as described above, that includes one or more component features. Although the electronic document may contain text, it may similarly include audio and / or video content instead of or in addition to the text.

Once the feature selection criteria have been determined (ie any of the various text / audio / video attributes are utilized as determinative features within the set of data objects), a prominent feature determination process according to the present invention can be performed. Beginning the process of determining the salient features, the corresponding data objects are divided into two groups. Then, an equation representing " odds of relevance " (see, for example, Equation 1) is applied to the two groups of data objects, where O (d) is a data object. P (R | d) indicates the probability that the data object is a member of the first group, and P (R '| d) indicates the probability that the data object is a member of the second group. Indicates.

Since passive grouping of data objects does not provide the desired probability to calculate the relevant probability, Equation 1 can be maximized to approximate this value. Accordingly, a logarithm function together with Bayes' formula may be applied to both sides of Equation 1 to generate Equation 2.

Assume that the data object consists of the feature set {F _i }, and if X _i is 1 or 0, respectively, if a given feature f _i is present or absent in the data object, then,

Becomes

Since log P (R) and log P (R ') are constant and independent of the features selected to stand out in the data object, the new amount g (d) is defined as:

p _i = P (X _i = 1 | R) indicates the probability that a certain feature f _i will occur in the data object in the first group of data objects, and q _i = P (X _i = 1 | R ') Is assumed to represent the probability that a certain feature f _i will occur in a data object in the second group of data objects, then equation 5 is generated through assignment and simplification.

Since the second summation is not dependent on the feature occurrence in the data objects, Equation 6 is obtained through erasure.

Since the logarithmic function is monotonic,

Finding the maximum of the ratios is enough to find the maximum of the corresponding log value. According to one embodiment of the present invention, Equation 7 is applied to each feature in the combined feature list for two groups of data objects to facilitate the identification of the salient features. To do so, p _i is estimated to represent the number of data objects in the first group of data objects that includes feature f _i at least once divided by the total number of data objects in the first group of data object documents. . Similarly, q _i is estimated to represent the number of data objects in the second group that includes at least once feature f _i divided by the total number of data objects in the second group of data objects.

2A-2C illustrate the operational flow of a salient feature determination function, in accordance with an embodiment of the invention. Beginning, a first set of data objects is examined to generate a feature list consisting of unique features present in one or more data objects from at least the first set of data objects (block 210). For each of the identified unique features, Equation 7 is applied to generate a ranked feature list (block 220), and at least one subset of the ranked feature list is selected as the salient features (block 230). The salient features may include one or more adjacent group (s) or non-adjacent group (s) of elements selected from the ranked feature list. In one embodiment, the first N elements of the ranked feature list are chosen to be prominent, where N may vary depending on the requirements of the system. In an alternative embodiment, the last M elements of the ranked feature list are chosen to be prominent, where M may also change depending on the requirements of the system.

According to one embodiment of the invention, a feature list is generated (block 210), the total number of data objects included in each group of data objects is determined (block 212), and the identified angles in at least the first group of data objects are determined. For the unique feature, the total number of data objects containing the unique feature is also determined (block 214). In addition, the unique feature list may be filtered based on various criteria desired (block 216). For example, the unique feature list may be used to remove features not found in at least a predetermined minimum number of data objects, features shorter than a predetermined set minimum length, and / or features that occur less than an assigned amount. Can be cut.

According to one embodiment of the invention, applying statistical discrimination to obtain a ranked feature list, as described for block 220 of FIG. 2A, further includes the processes described in FIG. 2C. In other words, applying statistical discrimination (ie, represented by equation 7), a determination is made as to which of the unique features identified in the first set of data objects are also present in the second set of data objects. Rather (block 221), a determination is made as to which of the unique features identified in the first set of data objects are not present in the second set of documents (block 222). According to the illustrated embodiment, features determined to be present in one set of data objects but not in another set are assigned a relatively higher rank in the ranked feature list (block 223), so that both of the data objects Features determined to be present in both sets are assigned a relatively lower rank, as determined through statistical distinction (ie, Equation 7) (block 224). Optionally, features may be further ranked in a ranked feature list based on the total number of data objects including each of the individual features.

Example Application

Referring to FIG. 3, shown is an illustration of exemplary application of salient feature determination functions of the present invention, in accordance with an embodiment. As shown, a classifier 300 is provided to include video, including both proprietary and non-private forms, within various data structures, including ultra-large hierarchical classification trees and monolayer file formats. Efficiently classify and categorize data objects, such as, but not limited to, electronic documents, sequences, audio sequences, image files, and text files. Classifier 300 is a classifier as well as a classifier training service 305 that trains classifier 300 to categorize new data objects based on classification rules extracted from previously categorized data hierarchies. A classifier categorization service 315 for categorizing new data objects input to 300.

Classifier training service 305 includes an aggregation function 306, a salient feature determination function 308, and a node characterization function 309 of the present invention. According to the illustrated embodiment, the content from the previously categorized data hierarchy is collected at each node in the hierarchy, for example via a collection function 306, so that the content group and anti-content of the data is collected. Form all of the groups. Features are then extracted from each of these data groups and a subset of those features is determined to be prominent via the salient feature determination function 308. The node characterization function 309 is utilized to describe the characteristics of each node of the previously categorized data hierarchy based on the salient features, and such hierarchical characterizations are further described in the classifier categorization service 315. For use, for example, it stores in data store 310.

Additional information about classifier 300, including classifier training service 305 and classifier categorization service 315, is filed concurrently with the present application and is generally assigned to the assignee of the present application. US Patent Application No. < 51026.P004 " entitled " Very-Large-Scale Automatic Categorizer For Web Content ", which is incorporated herein by reference in its entirety.

Classifier Training Service

4 is a functional block diagram of the classifier training service 305 of FIG. 3, in accordance with an embodiment of the present invention. As shown in FIG. 4, a previously categorized data hierarchy 402 is provided as input to the classifier training service 305 of the classifier 300. The previously categorized data hierarchy 402 represents a set of data objects, such as audio, video and / or text objects, previously categorized and categorized into a topical hierarchy (typically via individual manual input). . The previously categorized data hierarchy 402 may represent one or more sets of electronic documents previously categorized by, for example, a web portal or search engine.

According to the example shown, the gathering function 406 is configured to retrieve content from the previously categorized data hierarchy 402 to increase the distinction between sibling nodes at each level of the hierarchy. Can be grouped into content groups. The salient feature determination function 408 operates to extract features from the content of the data and the anti-content groups and to determine which of the extracted features 409 can be considered salient 409 '.

Also, according to the illustrated example, the node characterization function 309 of FIG. 3 operates to describe the content of the data and the characteristics of the anti-content groups. In one embodiment, the content and anti-content groups of data are characterized based on the determined salient feature. In one embodiment, the feature description is stored in the data store 310, which may be implemented in the form of any number of data structures, such as a database, directory structure, or a simple lookup table. In one embodiment of the invention, the parameters for the classifiers for each node are stored in a hierarchical categorization tree with a file structure that mimics the previous categorized data hierarchy.

Example Computer System

5 illustrates an example computer system suitable for use in determining salient features, in accordance with an embodiment of the present invention. As shown, computer system 500 includes one or more processors 502 and system memory 504. Computer system 500 may also include mass storage 506 (such as diskettes, hard drives, CDROMs, etc.), input / output devices 508 (such as keyboards, cursor controls, etc.), and (such as network interface cards, modems, etc.) Communication interface 510. The components are connected to each other via a system bus 512, which represents one or more buses. If the system bus 512 represents multiple buses, the buses are bridged by one or more bus bridges (not shown).

Each of these components performs a conventional function known in the art. In particular, system memory 504 and mass storage 506 are used to store working copies and permanent copies of programming instructions that implement the categorization system of the present invention. Permanent copies of programming instructions can be loaded into the mass storage 506 at the factory or through a distribution medium (not shown) or through a communication interface 510 (from a distributed server (not shown)), as described above. Can be loaded from. The structure of these components 502-512 is known and therefore will not be described any further.

Conclusion and Epilogue

Thus, as can be seen in the above description, a novel method and apparatus for automatically determining salient features for object classification has been described. Although the present invention has been described through the illustrated embodiments, those skilled in the art will recognize that the present invention is not limited to the described embodiments. The invention may be modified and changed within the spirit and scope of the appended claims. Accordingly, the specification is to be regarded in an illustrative rather than a restrictive sense.

Claims (36)

Extracting one or more unique features from the first content group of data objects to form a first feature list; Extracting one or more unique features from a second anti-content group of data object documents to form a second feature list; Generating a ranked feature list by applying statistical differentiation between the unique features of the first feature list and the unique features of the second feature list; And Identifying a set of salient features from the ranked feature list How to include. The method of claim 1, Each of the first content group of data objects and the second anti-content group of data objects comprises one or more electronic documents. The method of claim 1, Determining a total number of first data objects comprising the first content group of data objects; And Determining a second total number of data objects comprising the second anti-content group of data objects. The method of claim 3, For each of the one or more unique features forming the first feature list, the first content group of data objects including at least one instance of each of the one or more unique features of the first feature list Determining a number of first data objects of the; And For each of the one or more unique features forming the second feature list, of the second anti-content group of data objects comprising at least one instance of each of the one or more unique features of the second feature list. Determining the number of second data objects. The method of claim 4, wherein Generating the ranked list, Identifying unique features of the first feature list that are not in the second feature list as exclusive features; Identifying unique features of the first feature list that are also present in the second feature list as common features; And Ordering the ranked list such that the exclusive features are ranked higher in the ranked list when compared to the common features. The method of claim 5, Applying a probability function to each of the common features to obtain a result vector, the probability function being the first data object number divided by the second data object number divided by the total number of second data objects; Contains the ratio of 1 divided by the total number of data objects; And Ordering the common features in the ranked list based at least in part on the result vector of the probability function. The method of claim 5, The proprietary features are further ranked based on the first data object number. The method of claim 1, Identifying the salient feature set from the ranked feature list comprises selecting the first N neighboring features of the ranked feature list. The method of claim 1, Identifying the salient feature set from the ranked feature list includes selecting the last M adjacent features of the ranked feature list. The method of claim 1, Each of said unique features comprises a grouping of one or more alphanumeric characters. The method of claim 1, Based at least in part on the salient feature set, classifying a new data object as most relevant to one of the first content group of data objects and the second anti-content group of data objects. The method of claim 1, The first content group of data objects comprises a selected node of a topical hierarchy having a plurality of nodes and data objects corresponding to any associated subnodes of the selected node, And the second anti-content group of data objects includes data objects corresponding to any associated sibling nodes of the selected node and any associated subnodes of the sibling nodes. As a way to identify the salient features, Identifying one or more unique features that are members of the first data class; Examining a second data class to identify the one or more unique features that are also members of the second data class, and the one or more unique features that are not members of the second data class; Generating an ordered unique feature list based on a membership of each of the one or more unique features in the second data class; And Identifying one or more of the ranked unique feature list as prominent How to include. The method of claim 13, For each of the ranked unique feature list, determining the number of objects in the first data class that includes each respective unique feature. The method of claim 14, Generating a ranked list includes ranking the unique features in the ranked list that are not members of the second data class above the unique features that are also members of the second data class. How to include more. The method of claim 15, Generating a ranked list may include in the ranked list the unique features belonging to a larger object number of the first data class and the unique features belonging to a smaller object number in the first data class. And further including a higher ranking step. The method of claim 13, Identifying as being salient includes selecting a first set of N consecutive unique features from the ranked unique feature list. The method of claim 13, Identifying as salient comprises selecting the last M consecutive unique features from the ranked unique feature list. Extract one or more unique features from the first content group of data objects to form a first feature list; extract one or more unique features from the second anti-content group of data objects to form a second feature list; Apply a statistical differentiation between the unique features of the first feature list and the unique features of the second feature list to generate a ranked feature list and identify a set of salient features from the ranked feature list. A storage medium that first includes one or more functions for: storing a plurality of programming instructions designed to implement a plurality of functions of a category name service for providing a category name to a data object; And A processor coupled to the storage medium to execute the programming instructions Device comprising a. The method of claim 19, Each of the first content group of data objects and the second anti-content group of data objects comprise one or more data objects. The method of claim 19, The plurality of instructions, Determine a total number of first data objects comprising the first content group of data objects, And determining a second total number of data objects including the second anti-content group of data objects. The method of claim 19, The plurality of instructions, For each of the one or more unique features forming the first feature list, a first of the first content group of data objects comprising at least one instance of each of the one or more unique features of the first feature list Determine the number of data objects, For each of the one or more unique features forming the second feature list, the second anti-content group of data objects comprising at least one instance of each of the one or more unique features of the second feature list. 2 further comprising instructions for determining the number of data objects. The method of claim 20, The plurality of instructions for generating the ranked list, Identify unique features of the first feature list that are not present in the second feature list as exclusive features, Identify unique features of the first feature list that are also present in the second feature list as common features, And ordering the ranked list so that the exclusive features are ranked higher than the common features in the ranked list. The method of claim 23, The plurality of instructions, Apply a probability function to each of the common features to obtain a result vector, the probability function being the first data object number divided by the total number of second data objects divided by the second document object number; Contains the ratio of the value divided by the total number of data objects-, And ordering the common features in the ranked list based at least in part on the result vector of the probability function. The method of claim 23, And the exclusive features are further ranked based on the first data object number. The method of claim 19, And the plurality of instructions identifying the salient feature set from the ranked feature list further includes instructions to select the first N adjacent features of the ranked feature list. The method of claim 19, And the plurality of instructions identifying the salient feature set from the ranked feature list further includes instructions to select the last M adjacent features of the ranked feature list. The method of claim 19, Each of the unique features comprises a grouping of one or more alphanumeric characters. The method of claim 19, The plurality of instructions, Based at least in part on the salient feature set, instructions for classifying a new data object as most relevant to one of the first content group of data objects and the second anti-content group of data objects. The method of claim 19, The first content group of data objects comprises a selected node of a topical hierarchy having a plurality of nodes and data objects corresponding to any associated subnodes of the selected node, And the second anti-content group of data objects includes data objects corresponding to any associated sibling nodes of the selected node and any associated subnodes of the sibling nodes. Identify one or more unique features that are members of a first data class, examine the second data class to be one or more members of the second data class, and the one or more members that are not members of the second data class Identify a unique feature, generate an ordered unique feature list based on a membership of each of the one or more unique features in the second data class, and generate one or more of the ranked unique feature lists as prominent A storage medium storing a plurality of programming instructions designed to implement a plurality of functions first including one or more functions for identifying; And A processor coupled to the storage medium to execute the programming instructions Device comprising a. The method of claim 31, wherein The plurality of instructions, And for each of the ranked unique feature list, instructions for determining the number of objects in the first data class that includes each respective unique feature. 33. The method of claim 32, The plurality of instructions for generating a ranked list rank, in the ranked list, the unique features that are not members of the second data class higher than the unique features that are also members of the second data class. The apparatus further comprises instructions for assigning. The method of claim 33, wherein The plurality of instructions for generating a ranked list further include, in the ranked list, the unique features belonging to a larger number of objects of the first data class that belong to a smaller number of objects in the first data class. And further comprising instructions that rank higher than the intrinsic features. The method of claim 31, wherein The plurality of instructions that identify as salient further include instructions for selecting a first set of N consecutive unique features from the ranked unique feature list. The method of claim 31, wherein And the plurality of instructions that identify as salient include instructions for selecting a final set of M consecutive unique features from the ranked unique feature list.