JP2006501545A - Method and apparatus for automatically determining salient features for object classification - Google Patents

Abstract

A method and apparatus is provided for automatically determining salient features (308) for object classification. According to one embodiment, one or more unique features are extracted from the first content group of the object to form a first feature list, and the one or more unique features are extracted from the second anti-content group of the object. To form a second feature list. Next, a ranked list of features is created by applying statistical discrimination between the unique features of the first feature list and the unique features of the second feature list. A salient feature set (308) is identified from the ranked list of resulting features.

Description

  The present invention relates to the field of data processing. More particularly, the invention relates to automatically selecting object features for use in classifying objects into groups.

  The World Wide Web provides an important source of information with information that estimates an enormous number of pages available for online browsing and downloading. However, in order to effectively use such information, a practical method for navigating this enormous amount of data is necessary.

  In the early days of surfing the Internet, two basic methods were developed to help with web searches. The first approach creates an indexed database based on the content of web pages collected by automated search engines that search for new and unique pages and “crawl” the web. . This database can be searched using various query techniques and is often ranked based on similarity to the form of the query. In the second approach, web pages are grouped into category hierarchies, typically shown in a tree format. The user then descends down the hierarchy with two or more choices at each level that show significant differences between the subtrees below the decision point, and finally to the leaf node that contains the text and / or multimedia content pages. Make a series of choices while reaching.

  For example, FIG. 1 shows a typical prior art subject hierarchy 102. Here, a plurality of decision nodes (hereinafter “nodes”) 130-136 are arranged hierarchically in a plurality of parent and / or child nodes, each of which is associated with its own subject category. ing. For example, the node 130 is a parent node for the nodes 131 and 132, and the nodes 131 and 132 are child nodes for the node 130. Since nodes 131 and 132 are both child nodes for the same parent node (for example, node 130), it can be said that nodes 131 and 132 are siblings of each other. The set of siblings added to the subject hierarchy 102 includes nodes 133 and 134 and nodes 135 and 136. From FIG. 1, node 130 forms a first level 137 of the subject hierarchy 102, nodes 131-132 form a second level 138 of the subject hierarchy 102, and nodes 133-136 represent a third level 139 of the subject hierarchy 102. Can be seen to form. In addition, node 130 is called the root node of subject hierarchy 102 because it is not a child with respect to any other node.

  The process of creating a hierarchical organization for web pages presents a number of challenges. First, the nature of the hierarchy must be defined. This is typically done in a manner similar to creating a category in the library's Dewey decimal format, by the hands of an expert in a particular subject area. These categories are provided with descriptive labels so that users and categorizers can make appropriate decisions while navigating the hierarchy. For example, contents in the form of individual electronic documents are arranged in categories through a hierarchy by manual search.

  In recent years, interest has been directed towards automating the various stages of this process. Systems exist for automatically classifying documents from a document corpus. For example, some systems automatically cluster or group similar documents using keywords associated with the documents. Such clusters can be grouped repeatedly into super clusters, creating a hierarchical structure. However, these systems create a hierarchy without a systematic structure because keywords need to be inserted manually. If such a hierarchy is used for manual retrieval, the nodes in that hierarchy must be labeled by manually examining subnodes or leaf documents to identify common features.

  Many classification systems use a list of headwords to classify documents. Typically, key words are either pre-defined or selected from the document to be processed and more accurately characterize that document. Usually, these main word lists are created by counting the frequency of occurrence of all words used in each of a set of documents. Next, the word is deleted from the word list according to one or more criteria. In many cases, words that appear only occasionally in the document corpus are deleted. This is because such words are very rarely characterized accurately within a category. On the other hand, words that appear too frequently are also deleted. This is because such words are presumed to appear frequently in all documents across categories.

  In addition, “stop words” and stems are often deleted from the feature list to facilitate the determination of salient features. Stop words include words that are common to languages, such as “a”, “the”, “his”, “and”, etc., which are accepted as having no meaningful content. Here, the stem represents a suffix such as “-ing”, “-end”, “-is”, “-able”. Unfortunately, the generation of stopwords and stem lists is a language-specific issue and requires expertise in syntax, grammar, and usage that can vary over time. Therefore, a more flexible method for determining salient features is desired.

  The present invention will be described using the exemplary embodiments shown in the accompanying drawings, but is not limited thereto. In the drawings, like reference numbers indicate like elements.

  In the following description, various aspects of the present invention will be described. However, it will be apparent to those skilled in the art that the present invention may be practiced using only some or all of the embodiments. For purposes of explanation, specific numbers, materials, and configurations are set forth in order to provide a thorough understanding of the present invention, but the present invention may be practiced without such specific details. It will be clear. In other instances, well-known properties have been omitted or simplified so as not to obscure the present invention.

  Some of the following descriptions are presented in terms of operations performed by processor-based devices using terms such as data, storage, selection, determination, calculation, etc., to enable those skilled in the art to give their work summary to others skilled in the art Consistent with the method normally adopted to communicate. As is well understood by those skilled in the art, quantities are quantities of electrical, magnetic, or optical signals that can be stored, transmitted, coupled, and otherwise manipulated through the mechanical and electrical components of processor-based devices. Take shape. The term processor includes stand-alone, auxiliary or embedded microprocessors, microcontrollers, digital signal processors and the like.

  The various operations are described in sequence as a plurality of discrete steps in a way that is most useful for understanding the present invention, but the order of the descriptions is considered to mean that these operations are necessarily dependent on the order. is not. In particular, these operations need not be performed in the order presented. Further, in the following description, the word “one embodiment” is repeatedly used. This may refer to the same embodiment, but usually does not refer to the same embodiment.

  In one embodiment of the invention, one or more unique features are extracted from a first group of objects to form a first feature set, and one or more unique features are extracted from a second group of objects. To form a second feature set. A statistical discrimination is then applied between the unique features of the first feature set and the unique features of the second feature set to create a ranked list of features. A salient feature set is then identified from the ranked list of resulting features.

  In one embodiment, salient features are determined to facilitate efficient classification and categorization of data objects. Data object categorization includes, but is not limited to, text files that contain both proprietary and generic formats within very large hierarchical classification trees and non-hierarchical data structures such as single layer files, Includes image files, audio and video sequences. For example, in a text file, the feature can take the form of a word. Here, it is understood that the term “word” usually refers to a group of characters that have some semantic meaning in a language. More generally, the feature can be an N-tokengram. Here, the token is one atom of a language including, for example, an N-character gram and an N-word gram in English, and an N-ideogram character in an Asian language. For example, in a speech sequence, notes, intonation, tempo, duration, pitch, volume, etc. can be used to classify the speech as a feature. On the other hand, various pixel attributes such as chrominance and luminance level can be used as features in video sequences and still images. In one embodiment of the invention, for example, if a single feature group is identified from a group, such as an electronic document, a subset of these features is prominent for classifying a given group of data objects. It is determined. The term “electronic document” is used broadly herein to describe a group of data objects as described above, including one or more structural name features. An electronic document can include text and can similarly include audio and / or video content in place of or in addition to text.

  Once the feature selection criteria are determined (ie, various text / audio / video attributes are utilized as determining features in the set of data objects), the process of determining salient features according to the present invention is performed. In order to begin the process of determining salient features, the data object is divided into two groups. An expression indicating “fitness odds” is applied to these two groups of data objects (see, for example, Expression 1). Where O (d) represents the odds that a given data object is a member of the first group of data objects, and P (R | d) is the data object is a member of the first group. P (R ′ | d) represents the probability that the data object is a member of the second group.

  Since manual grouping of data objects does not give the desired probability for calculating the odds of fitness, equation (1) can be maximized to approximate this value. Therefore, a logarithmic function using the Bayes formula can be applied to both sides of Equation (1), and Equation (2) is derived.

It is assumed that a data object consists of a set of features {F _i } and if any feature f _i is present in the data object, X _i is 1 and if f _i is not present, X _i is If it is 0,

It becomes. Since logP (R) and logP (R ') are constants and are independent of the features selected as prominent in the data object, a new quantity g (d) is defined as follows:

Let p _i = P (X _i = 1 | R) denote the probability that any feature (f _i ) will occur in a data object in the first group of data objects, and q _i = P (X _i = 1 Assuming that | R ′) represents the probability that any feature (f _i ) will occur in a data object in the second group of data objects, substituting and simplifying the equation yields equation (5) .

  Since the second integration does not depend on the occurrence of features in the data object, it can be deleted, resulting in equation (6).

  Since the log function is a monotone function,

It is a sufficient condition to maximize the corresponding log value. According to one embodiment of the present invention, equation (7) is applied to each feature in the feature list combined for two groups of data objects to facilitate the identification of salient features. To do so, P _i is presumed to indicate the number of data objects in the first group of data objects that include the feature f _i at least once, and the total number of data objects in the first group of data object documents. Divided by. Similarly, q _i is estimated to indicate the number of data objects in two groups that obtain a second result vector that includes feature f _i at least once, and divided by the total number of data objects in the second group of data objects. The

  2A to 2C show an operation flow of the distinctive feature determination function according to one embodiment of the present invention. Initially, a first set of data objects is examined to create a feature list consisting of at least the unique features shown in one or more data objects from the first set of data objects (block 210). For each unique feature identified, equation (7) is applied to generate a ranked feature list (block 220). Then, at least one subset of the ranked feature list is selected as a salient feature (block 230). This salient feature list can include one or more continuous or non-contiguous groups of elements selected from the ranked feature list. In one embodiment, the first N elements of the ranked list of features are chosen as prominent. Here N may vary according to system requirements. In an alternative embodiment, the last M elements of the ranked list of features are chosen as prominent. Here, M can also vary according to the requirements of the system.

  According to one embodiment of the present invention, during the creation of the feature list at block 210, the total number of data objects contained within each group of data objects is determined (block 212), at least in the first group of data objects. For each unique feature identified in, the total number of data objects containing that unique feature is also determined (block 214). Further, the list of unique features may be filtered based on various criteria as desired (block 216). For example, that list of unique features may be truncated, resulting in features that are not found in at least some minimum number of data objects, features that are shorter than some fixed minimum length, and / or less than quota The feature is deleted.

  According to one embodiment of the present invention, applying statistical discrimination to obtain a ranked list of features, as described in block 220 of FIG. 2A, further includes the process shown in FIG. 2C. In other words, in the application of statistical discrimination (ie, as shown by equation (7)), the unique features identified in the first set of data objects are in the second set of data objects. Is also determined (block 221), as well as whether the unique features identified in the first set of data objects are not present in the second set of documents (block 221). Block 222). According to the illustrated embodiment, features that are present in one set of data objects but are not present in another set are assigned a higher relative rank in the ranked list of features. (Block 223), features that are determined to be present in both sets of data objects are determined through statistical discrimination (ie, equation (7)) in the ranked list of features. Assign a lower relative rank (block 224). Depending on the situation, such features may be further ranked in the ranked feature list based on the total number of data objects that contain each individual feature.

Application Example Referring now to FIG. 3, there is shown an application example of means for determining salient features according to one embodiment of the present invention. As shown in the figure, a classifier 300 is provided to efficiently classify and categorize data objects such as electronic documents. Electronic documents include, but are not limited to, text files, image files, audio and video sequences in proprietary and general-purpose formats within various data structures including very large hierarchical classification trees and flat file formats. Classifier 300 includes a classifier training service 305 that trains classifier 300 to categorize new data objects based on classification rules extracted from a pre-categorized data hierarchy. Similarly, a classifier categorization service 315 is included to categorize new data objects input to the classifier 300.

  The classifier training service 305 includes an aggregation function 306, a salient feature determination function 308 of the present invention, and a node characterization function 309. In accordance with the illustrated embodiment, content from pre-categorized data hierarchies is collected at each node in the hierarchy, forming both data content and anti-content groups via, for example, an aggregation function 306. Next, features from each of these groups of data are extracted and a salient feature determination function 308 is used to determine that a subset of these features is salient. A node characterization function 309 is used to characterize each node of the pre-categorized data hierarchy based on salient features, storing such hierarchy characterization in, for example, the data store 310, and Used by the classifier categorization service 315.

  Additional information regarding classifier 300, including classifier training service 305 and classifier categorization service 315, is co-filed with US Patent Application No. 51026. P004, named “Very-Large-Scale Automatic Categorizer For Web Content”, the disclosure of which is incorporated herein by reference.

Classifier Training Service FIG. 4 shows a functional block diagram of the classifier training service 305 of FIG. 3 according to one embodiment of the present invention. As shown in FIG. 4, the pre-categorized data hierarchy 402 is provided as an input to the classifier training service 305 of the classifier 300. The pre-categorized data hierarchy 402 represents a set of data objects, such as audio, video and / or text objects, that have been previously categorized and categorized into a subject hierarchy (typically manually by an individual). The pre-categorized data hierarchy 402 may represent one or more sets of electronic documents that have been pre-categorized, such as by a web portal or search engine.

  According to the illustrated example, the aggregation function 406 aggregates content from the pre-categorized data hierarchy 402 into a data content group and an anti-content group to increase discrimination between sibling nodes at each level of the hierarchy. Let The salient feature determination function 408 operates to extract features from the content group and the anti-content group of the data, and to determine which of the extracted features (409) should be regarded as salient (409 ').

  In addition, according to the illustrated example, the node characterization function 309 of FIG. 3 operates to characterize content groups and anti-content groups of data. In one embodiment, the data content group and the anti-content group are characterized based on the salient features that are determined. In one embodiment, the characterization is stored in the data store 310. The data store 310 is implemented in the form of any number of data structures, such as a database, directory structure, or simple lookup table. In one embodiment of the present invention, the parameters for each node classifier are stored in a hierarchical categorization tree having a file structure that mimics a pre-categorized data hierarchy.

Example Computer System FIG. 5 illustrates an example of a computer system suitable for use in determining salient features according to one embodiment of the present invention. As shown, computer system 500 includes one or more processors 502 and system memory 504. In addition, the computer system 500 includes a mass storage device 506 (diskette, hard drive, CDROM, etc.), an input / output device 508 (keyboard, cursor control, etc.) and a communication interface 510 (network interface card, modem, etc.). Each element is coupled to each other via a system bus 512 representing one or more buses. If the system bus 512 represents multiple buses, they are bridged using one or more bus bridges (not shown).

  Each of these elements performs a known conventional function. Specifically, system memory 504 and mass storage device 506 are used to store working and permanent copies of program instructions that implement the categorization system of the present invention. Permanent copies of program instructions can be stored in the mass storage device at the factory or in the field, as described above, via a distribution medium (not shown) or via a communication interface 510 (from a distribution server (not shown)). 506 is loaded. The structure of these elements 502-512 is well known and will not be described further.

  Thus, it can be seen from the above description that a novel method and apparatus for automatically determining salient features for object classification has been described. Although the present invention has been described using the embodiments shown above, those skilled in the art will recognize that the invention is not limited to the described embodiments. The present invention may be practiced with modification and alteration within the spirit and scope of the appended claims. The description is thus to be regarded as illustrative instead of limiting on the present invention.

FIG. 2 illustrates an exemplary prior art subject hierarchy including multiple decision nodes. It is a flowchart which shows the flow of operation of the remarkable feature determination function concerning one Example of this invention. It is a flowchart which shows the operation flow of the determination function of the remarkable feature concerning one Example of this invention. It is a flowchart which shows the operation flow of the determination function of the remarkable feature concerning one Example of this invention. It is a figure which shows the means to determine the remarkable characteristic concerning one Example of this invention. FIG. 4 is a functional block diagram illustrating the classifier training service of FIG. 3 according to one embodiment of the present invention. FIG. 2 illustrates a computer system suitable for use in determining salient features according to one embodiment of the present invention.

Claims (36)

Extracting one or more unique features from a first content group of data objects to form a first feature list;
Extracting one or more unique features from the second anti-content group of the data object document to form a second feature list;
Creating a ranked list of features by applying statistical discrimination between the unique features of the first feature list and the unique features of the second feature list; and
Identifying a set of salient features from the ranked list of features.   The method of claim 1, wherein each of the first content group of the data object and the second anti-content group of the data object includes one or more electronic documents. Determining a total number of first data objects including a first content group of the data objects; and
The method of claim 1, further comprising: determining a total number of second data objects that include a second anti-content group of the data objects. Determining the number of first data objects in the first content group of the data object for each of the one or more unique features forming the first feature list, the first content group of the data object Includes at least one instance for each of the one or more unique features of the first feature list; and
Determining, for each of the one or more unique features forming the second feature list, a number of second data objects in a second anti-content group of the data object, the second of the data objects The method of claim 3, further comprising: an anti-content group includes at least one instance for each of the one or more unique features of the second feature list. Creating the ranked list is:
Identifying unique features of the first feature list that are not present in the second feature list as exclusive features;
Identifying a unique feature of the first feature list that is also present in the second feature list as a common feature; and
5. The method of claim 4, comprising: ordering the ranked list so that the exclusive feature is ranked higher than the common feature in the ranked list. . Applying a probability function to each of the common features to obtain a result vector, the probability function including a quotient dividing the number of first data objects by the total number of first data objects; Including a ratio to the quotient dividing the number of data objects by the total number of second data objects; and
The method of claim 5, further comprising: ordering the common features at least in part in the ranked list based on a result vector of the probability function.   The method of claim 5, wherein the exclusive features are further ranked based on the number of the first data objects.   Identifying the salient feature set from the ranked list of features includes selecting the first N adjacent features from the ranked list of features. Item 2. The method according to Item 1.   Identifying the salient feature set from the ranked list of features includes selecting the last M adjacent features from the ranked list of features. Item 2. The method according to Item 1.   The method of claim 1, wherein each of the unique features includes one or more alphanumeric character groups.   A new data object as most relevant to one of the first content group of the data object and the second anti-content group of the data object, at least in part based on the set of salient features; The method of claim 1, further comprising classifying. The first content group of data objects includes data objects corresponding to selected nodes of the subject hierarchy having a plurality of nodes and some related subnodes of the selected nodes;
The method of claim 1, wherein the second anti-content group of data objects includes data objects corresponding to some related sibling nodes of the selected node and any related subnodes of the sibling node. A method for identifying salient features, comprising:
Identifying one or more unique features that are members of the first data class;
Examining a second data class to identify the one or more unique features that are also members of the second data class and the one or more unique features that are not members of the second data class;
Generating a ranked list of unique features having an order based on membership of each of the one or more unique features in the second data class; and
Identifying one or more of the ranked list of unique features as prominent.   14. The method of claim 13, further comprising determining, for each ranked list of unique features, the number of objects in the first data class that include the respective unique features. Including methods.   Generating a ranked list causes the unique feature that is not a member of the second data class to be higher in the ranked list than the unique feature that is a member of the second data class. 15. The method of claim 14, comprising ranking.   Generating a ranked list may be configured such that the unique features belonging to an object with a large number of first data classes are compared to the unique features belonging to an object with a small number of first data classes. 16. The method of claim 15, comprising ranking higher in the rendered list.   14. The method of claim 13, wherein identifying as prominent includes selecting an initial set of N consecutive unique features from the ranked list of unique features. .   14. The method of claim 13, wherein identifying as prominent includes selecting the last M consecutive unique features from the ranked list of unique features. A storage medium for internally storing a plurality of program instructions designed to perform a plurality of functions related to a category designation service for giving a category name to a data object,
Extracting one or more unique features from the first content group of the data object to form a first feature list;
Extracting one or more unique features from the second anti-content group of the data object to form a second feature list;
Creating a ranked list of features by applying statistical discrimination between the unique features of the first feature list and the unique features of the second feature list;
A storage medium comprising one or more functions for identifying a set of salient features from the ranked list of features;
And a processor coupled to the storage medium to execute program instructions.   The apparatus of claim 19, wherein each of the first content group of the data object and the second anti-content group of the data object includes one or more data objects. The plurality of instructions are:
Determining a total number of first data objects including the first content group of the data objects;
The apparatus of claim 19, further comprising instructions for determining a total number of second data objects that include the second anti-content group of the data objects. The plurality of instructions are:
Determining the number of first data objects in the first content group of the data object for each of the one or more unique features forming the first feature list, the first content group of the data object Includes at least one instance for each of the one or more unique features of the first feature list; and
Determining, for each of the one or more unique features forming the second feature list, a number of second data objects in a second anti-content group of the data object, the second of the data objects The apparatus of claim 19, wherein the anti-content group further comprises an instruction to include at least one instance for each of the one or more unique features of the second feature list. The plurality of instructions for creating the ranked list is:
Identifying unique features of the first feature list that are not present in the second feature list as exclusive features;
Identifying a unique feature of the first feature list that is also present in the second feature list as a common feature;
21. The apparatus of claim 20, wherein the ranked list is ordered and the exclusive feature includes an instruction that ranks higher in the ranked list than the common feature. The plurality of instructions are:
Applying a probability function to each of the common features to obtain a result vector, the probability function including a quotient dividing the number of first data objects by the total number of first data objects; Including a ratio to the quotient dividing the number of documents by the total number of second data objects; and
24. The apparatus of claim 23, further comprising an instruction to order the common features at least partially in the ranked list based on the result vector of the probability function.   The apparatus of claim 23, wherein the exclusive features are further ranked based on the number of the first data objects.   The plurality of instructions for identifying the salient feature set from the ranked list of features further includes an instruction to select the first N adjacent features from the ranked list of features. The apparatus according to claim 19, characterized in that:   The plurality of instructions identifying the set of salient features from the ranked list of features further comprises an instruction to select the last M adjacent features from the ranked list of features. The apparatus according to claim 19, characterized in that:   The apparatus of claim 19, wherein each of the unique features includes one or more alphanumeric character groups. The plurality of instructions are:
A new data object is most relevant to one of the first content group of the data object and the second anti-content group of the data object, at least in part based on the set of salient features. The apparatus of claim 19 further comprising instructions for classifying. The first content group of data objects includes data objects corresponding to selected nodes of the subject hierarchy having a plurality of nodes and some related subnodes of the selected nodes;
The apparatus of claim 19, wherein the second anti-content group of data objects includes data objects corresponding to some related sibling nodes of the selected node and any related subnodes of the sibling nodes. A storage medium that internally stores a plurality of program instructions designed to perform a plurality of functions,
Identifying one or more unique features that are members of the first data class;
Examining a second data class to identify the one or more unique features that are also members of the second data class and the one or more unique features that are not members of the second data class;
Generating a ranked list of unique features having an order based on membership of each of the one or more unique features in the second data class;
A storage medium comprising one or more functions that identify one or more of the ranked list of unique features as prominent;
And a processor coupled to the storage medium to execute program instructions. The plurality of instructions are:
32. The apparatus of claim 31, comprising instructions for determining a number of objects in the first data class that include a respective unique feature for each of the ranked list of unique features. .   The plurality of instructions for generating a ranked list may cause the unique feature that is not a member of the second data class to be compared to the unique feature that is a member of the second data class. 33. The apparatus of claim 32, comprising instructions that rank higher within.   The plurality of instructions for generating a ranked list may include the unique feature belonging to an object with a large number of the first data class, and the unique feature belonging to an object with a small number of the first data class. 34. The apparatus of claim 33, comprising instructions that rank higher in the ranked list.   32. The plurality of instructions that are identified as prominent include an instruction that selects an initial set of N consecutive unique features from the ranked list of unique features. The device described. 32. The plurality of instructions that are identified as prominent include an instruction that selects a last set of M consecutive unique features from the ranked list of unique features. The device described.

Images