JP2008276571A - Label assignment method, label assignment device, label assignment program and storage medium - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a label assignment technique, capable of reducing determination of "no label" to a content while suppressing the calculation quantity. <P>SOLUTION: The label assignment device is adapted to assign, to a content having at least one of a character and an image, one or more labels showing a type of the content thereof. The device generates a score function using a characteristic vector and a binary classifier, generates an identification function having the score function and a threshold of determining the propriety of label assignment, and determines the propriety of label assignment based on an identification function value calculated by the identification function. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a technique for assigning a label indicating the type of content to content having characters and images.

  In recent years, research and development of technologies for labeling content included in a database have been actively conducted. Here, the content includes text information such as papers, patent gazettes, online news data, and e-mails, text information such as Web data and blog data and link information, and image data. And so on. In this case, it is necessary that the content can be expressed by a feature vector.

  The feature vector is a vector that represents the ratio of appearance frequencies of elements (features) included in the content. For example, when the words (features) “data”, “information”, and “processing” appear (used) in a certain content “3 times”, “1 time”, and “2 times”, the feature vector of the content is It can be expressed as {3, 1, 2} or {3/6, 1/6, 2/6}.

  The label indicates the type of content as described above, and includes “computer”, “sports”, “music”, and “math”. For example, when the content is a patent document, IPC (International Patent Classification), F-term (File Forming Term), FI (File Index), etc. correspond to the label.

  In the label application technique, the label to be used is generally determined in advance. For example, by using statistical information about a set of contents (training data) for which labels to be assigned have already been determined, a multi-label adder (which has a function of inputting a feature vector of contents and outputting an estimated value of label assignment) There is a method of learning one program) and applying one or more labels (multiple label assignment) to unlabeled content (label assignment target content) using the multiple label applicator.

  In the problem of assigning a plurality of labels to content, the multiple label assigner shown in Non-Patent Documents 1 and 2 is a binary classifier that inputs a feature vector of the content for each label and outputs whether or not the label can be given. Designed with. That is, a multiple label applicator that determines the application of a plurality of labels using a plurality of binary classifiers corresponding to each label is realized. In the multiple label applicators shown in Non-Patent Documents 1 and 2, learning of a binary classifier using training data and determination of whether or not labels can be assigned to content using the learned binary classifier are individually It is characterized by being performed independently for each label.

  In the techniques of Non-Patent Documents 1 and 2, a binary classifier is designed for each label. In the binary classifier corresponding to each label, learning using training data and determination of label assignment to content are performed as compared with other labels. Since the binary classifier is designed for each label, the amount of calculation required for learning the multiple label applicator and labeling the content is proportional to the number of labels. This means that multiple labels can be assigned at high speed.

In addition, the multiple label assigners shown in Non-Patent Documents 3 and 4 are realized by a multiple classifier that directly outputs a combination of labels to be assigned to the feature vector of the input content. The multiple label applicators shown in Non-Patent Documents 3 and 4 are characterized by selecting an optimum combination from among label combination candidates for content.
K. Nigam, A. McCallum, S. Thrunand T. Mitchell: Text classification from labeled and unlabeled documents using EM, Machine Learning, 39, 103-134 (2000). T. Joachims: Text categorization with support machines: Learning with many relevant features, Proceeding of 10th European Conference on Machine Learning (ECML-98), 137-142 (1998). Ueda Osamu, Saito Kazuaki: "Probability model of multi-topic text-parametric mixed model-", IEICE Transactions, J87-D-II (3), 872-883, 2004 Hideto Kawasawa, Tomonori Izumiya, Hironori Hira, Hidesaku Maeda, Hideki Amagasaki: “Multiple Labeling Learning Based on the Maximum Margin Principle”, IEICE Transactions, J88-D-II (11), 2246-2259, 2005

  However, the techniques of Non-Patent Documents 1 and 2 have a drawback in that it is often determined not to add a label that should be originally added to content. Generally, in the problem of multiple label assignment, the number of contents to be assigned to individual labels is often overwhelmingly smaller than the number of contents that should not be assigned. In such a case, even if it is determined that a label with a binary classifier is not given to all contents, the classification accuracy of the binary classifier can be high. Therefore, in the problem of multiple label assignment, there is a tendency to acquire a binary classifier that determines not to give a label to a large number of contents by learning using training data. There is a problem that a situation in which it is determined that no label is assigned to the content to be assigned often occurs.

  On the other hand, in the techniques of Non-Patent Documents 3 and 4, as described above, a multiple label applicator for selecting an optimum label combination from label combination candidates for the content is designed. This multiple label applicator can be designed to exclude the case where the optimum label combination of the content does not add a label at all, that is, becomes “no label”. For this reason, the technologies of Non-Patent Documents 3 and 4 can avoid the problem of determining many contents as “no label” caused by the technologies of Non-Patent Documents 1 and 2, and provide more accurate multiple labels. Can be expected.

  However, this type of multiple label applicator has a problem that it is necessary to handle not a single label but a huge number of label combinations (order of power of the number of labels). This means that if the number of labels is large to some extent, a huge amount of calculation is required to add labels to the content. In the techniques of Non-Patent Documents 3 and 4, an approximation algorithm for dealing with this problem is used, but it still requires a calculation amount of the order of the second and third power of the number of labels. That is, the techniques of Patent Documents 3 and 4 have a problem that it may take more time for labeling than the techniques of Non-Patent Documents 1 and 2.

  Therefore, the present invention has been made in view of the above problems, and in the labeling technology, one or more appropriate labels are given to the content with a small amount of calculation, and “no label” for the content is provided. The purpose is to reduce the judgment.

In order to solve the above-described problem, a label applying apparatus that executes a label applying method according to the present invention assigns at least one label indicating the type of content to content having at least one of a character and an image. The label assignment apparatus includes a processing unit including a score function generation unit, an identification function generation unit, and a label addition unit, and a storage unit.
The score function generation unit reads each label using a feature vector obtained by collecting the appearance frequency of each feature of the content read from the storage unit and a binary classifier that is a program designed for each label. A score function is generated for calculating a score for determining whether or not the content can be given for each content to be labeled.
The discriminant function generation unit uses the training data set that is read from the storage unit and includes content for which labels to be assigned have already been determined, and whether or not labels can be assigned using a score function and a score calculated by the score function. A discriminant function having a threshold value for determining
The label assignment unit calculates an identification function value of the content to be labeled for each label using an identification function, and associates the label when the identification function value becomes a predetermined value or more with the content to be labeled Labeling by storing in the storage unit.

  According to this invention, a score function is generated using a feature vector and a binary classifier, an identification function having the score function and a threshold value for determining whether to give a label is generated, and calculated by the identification function Because the discriminating function value determines whether or not labeling is possible, the amount of calculation required for labeling can be suppressed to a time proportional to the number of labels, and the possibility of erroneous determination as “no label” is reduced. Can do.

  In the label assigning apparatus according to the present invention, the score function generation unit uses a classifier based on a probability model that gives a probability of assigning a label to the content as a binary classifier, and the label to be assigned is already determined. If the classifier is learned in advance using a training data set consisting of existing content, the probability and label when it is assumed that a label is attached to the content to be labeled using the binary classifier read from the storage unit The probability of when no label is assigned is calculated, and the logarithm of the value obtained by dividing the probability when the label is assumed by dividing the probability when no label is assigned with the label to the content to be labeled A score function is generated that is calculated as a score for determining whether or not the grant is possible.

  According to this invention, the logarithm of the value obtained by dividing the probability when labeling is assumed by the probability when no label is assumed is used as a score for determining whether or not the labeling target content can be labeled. Thus, the score function generation unit can be appropriately realized. That is, by using this score function generation unit, it is possible to calculate a score suitable for use by the identification function generation unit for calculation with respect to arbitrary content.

  Moreover, in the label assignment apparatus according to the present invention, when the discriminant function generation unit generates the discriminant function, the discriminant function generator calculates from the estimated value of the binary classifier parameter calculated by excluding one piece of training data from the training data set. The combination of labels assigned to the excluded training data using the predicted score value is divided by the total number of labels to be assigned to the training data. To meet the criteria for maximizing the harmonic mean of the value obtained by dividing the number of labels correctly given by using the predicted value by the total number of labels given by using the predicted value, Compute an estimate of the parameters of the discriminant function.

  According to this invention, it is possible to appropriately realize the discriminant function generation unit by calculating the estimated value of the parameter of the discriminant function so that the combination of the labels meets the above-described criterion for maximizing the harmonic average. Can do. That is, it can be expected that the harmonic average is increased by giving a label to a new content by using the discriminant function generation unit.

  In the labeling apparatus according to the present invention, the gradient function is used to search for an estimated value of the parameter of the discriminant function that maximizes the approximate value of the harmonic mean defined using a predetermined logistic function.

  According to this invention, by using a gradient method such as a quasi-Newton method, it is possible to efficiently search for an estimated value of the parameter of the discriminant function described above.

  According to another aspect of the present invention, there is provided a program for causing a computer to execute the labeling method by the labeling device described above.

  According to this invention, it is possible to cause a computer to execute the labeling method.

  A storage medium according to the present invention is a computer-readable storage medium that stores the above-described program.

  According to this invention, the program for the labeling method can be stored.

  According to the present invention, in the label assignment technique, one or more appropriate labels can be assigned to the content with a small amount of calculation, and the determination of “no label” for the content can be reduced.

Hereinafter, the best mode for carrying out the present invention (hereinafter referred to as an embodiment) will be described with reference to the drawings. In addition, it demonstrates in order of the outline | summary of embodiment, the specific example of embodiment, and an Example.
(Outline of the embodiment)
FIG. 1 is a functional block diagram showing the configuration of the multiple label applying apparatus (label applying apparatus) of this embodiment. The multiple label assignment apparatus 1 includes a training data DB 2 (storage unit), a score function generation unit 3 (processing unit), an identification function generation unit 4 (processing unit), a memory 5 (storage unit), and a multiple label addition unit 6 (processing unit). ), And an input unit 7 and an output unit 8.

  Specifically, the multi-label applying device 1 is a computer device, and includes a central processing unit (CPU), a random access memory (RAM), a read only memory (ROM), a hard disk drive (HDD), and the like. For example, the training data DB 2 is realized by an HDD or the like, and the score function generation unit 3, the identification function generation unit 4, and the multiple label assignment unit 6 are realized by the CPU executing various programs stored in the ROM or the HDD. . The memory 5 is realized by a RAM or the like.

  The training data DB 2 stores a training data set in which examples of content having the same format as content to be given a label (hereinafter referred to as “labeling target content”) are collected. This training data set is data for learning an identification function (details will be described later) for determining whether labeling is possible. For example, when the type of content is a patent document, a plurality of patent documents and labels (IPC, F-term, etc.) attached to each of the patent documents are prepared in advance, and a set of content body and label assignment vectors is a training data DB 2. Is remembered. Here, the label addition vector is a representation of whether or not all label candidates can be assigned as a vector (details will be described later), and the user of the multiple label assignment apparatus 1 (hereinafter simply referred to as “user”) or. It is given in advance by the designer of the multi-label applying device 1.

  The score function generation unit 3 generates a score function for calculating the content score for each label from the training data set stored in the training data DB 2. The discriminant function generation unit 4 generates an discriminant function from the training data set for estimating the possibility of labeling the content based on the score. The memory 5 stores the generated discriminant function and the like. The multiple label assigning unit 6 determines whether or not the content can be given a label using the identification function.

  The input unit 7 is an interface used when a user inputs content for which labeling is requested (labeled content), and is, for example, a keyboard or a mouse. The output unit 8 is an interface that displays the result of multiple label assignment by the multiple label assignment device 1 on the screen or outputs it to an external device, for example, a liquid crystal display or a communication device.

  Next, the process of the multiple label applying apparatus 1 will be described. FIG. 2 is a flowchart showing an outline of processing of the multiple label assigning apparatus. FIG. 3 is a flowchart showing a process of calculating a predicted score value (step S103 in FIG. 2). FIG. 4 is a flowchart showing the parameter learning process (step S104 in FIG. 2) of the discrimination function. FIG. 5 is a flowchart showing processing (step S105 in FIG. 2) for determining whether or not a label can be given to content using an identification function.

  As described above, in the multiple label assigning apparatus 1 of the present embodiment, first, based on the technique of the conventional binary classifier, a score function that gives a score indicating the possibility of label assignment for each label is provided. design. Next, an identification function for determining whether or not to give a label is designed from the score obtained for each label by this score function. In the design of the discriminant function, a threshold value for the score that gives the possibility of labeling is set. Then, an optimum threshold value can be obtained by learning the discriminant function so as to maximize the F value defined for the problem of searching for the label to be given to the content.

  That is, according to the multiple label assigning apparatus 1 of the present embodiment, a score function is designed for each label based on the technique of the binary classifier, and the score for each label of the content is calculated using the score function. By determining whether or not labeling can be individually performed using the score, the amount of calculation required for labeling can be suppressed to a time proportional to the number of labels. Further, in the multiple label assigning apparatus 1 of the present embodiment, a score threshold value that gives a label assignment determination is obtained based on maximization of the F value defined for the problem of searching for a label to be assigned. . The F value takes a value from 0 to 1, and is an index that is 0 when content with a label to be attached is erroneously determined as “no label”. Therefore, by using the threshold value acquired based on maximization of the F value, the possibility of determining “no label” can be greatly reduced, and the accuracy of labeling can be improved.

(Specific example of embodiment)
Next, a specific example of the outline of the above-described embodiment will be described. Here, in the multiple label assignment problem in which one or more labels to be given to the content are selected from candidates of K labels {1,..., K,. A specific example in the case of defining and using a score function using a container will be described.

When a feature space composed of words or the like included in the content is T = {t ₁ ,..., T _i ,..., T _V }, the content feature vector x is based on the frequency of t _i included in the content. X = {x ₁ ,..., X _i ,..., X _V }. V represents the number of types of features that may be included in the content. For example, when the content is text data, V represents the total number of vocabulary (word) types that may appear in the content.

  The binary classifier that determines whether or not the label k can be given to the feature vector x of the content includes a naive Bayes (NB) model (Non-patent Document 1) and a maximum entropy (ME) model (K. Nigam, J Lafferty, and A. McCallum: Using maximum entropy for text classification, IJCAI-99 Workshop on Machine Learning for Information Filtering, 61-67 (1999)), classifier based on hybrid of NB model and ME model (hybrid classifier) Existing classifiers such as Support Vector Machine (SVM) (Non-patent Document 2) can be used.

  Note that such algorithm parameter learning and label assignment methods can be realized in the multiple label assignment apparatus 1 which is a computer apparatus by creating a program for executing the flowcharts of FIGS. Furthermore, these programs can be stored in a storage medium such as a hard disk, a flash memory, a CD-ROM (Compact Disk Read Only Memory), or a DVD (Digital Versatile Disk).

(Example)
Next, examples of the present invention will be described. Table 1 shows the multiple label assigning device 1 of the present embodiment described above for the problem of assigning an F term to a patent document included in the Japanese patent document database provided by the NTCIR-6 project sponsored by the National Institute of Informatics. It is a table | surface which shows the result when the case where this application) is applied, and the case where a prior art (comparative example) is applied.

  This database includes patent documents as contents, and each patent document is given an associated F-term. The F-term is prepared for each theme that gives a large classification of patents, and is used as a search key for patent documents. F terms are set from 18 to slightly less than 1000 for each theme.

  In this embodiment, a specific example of an embodiment in which a past patent document to which an F term has already been assigned is used as training data for the problem of automatically assigning an F term to a new patent document having a known theme. The performance of the present invention was evaluated by applying the example multiple label applying apparatus 1. Here, as a past patent document, a document filed in 1993-1997 was used, and as a new patent document, a 21606 document on 108 themes filed in 1998-1999 was used. The number of F-terms set for each theme was 230 on average, 35 on minimum, and 797 on maximum.

  A hybrid classifier disclosed in Japanese Patent Application Laid-Open No. 2006-107354 was used as the binary classifier used in the score function generation unit 3 of the multiple label assigning apparatus 1 of the embodiment. The hybrid classifier is designed by applying the NB model for each component included in the content. In the present embodiment, the “invention name”, “inventor / applicant”, “summary”, “claims”, and “other text (specifications, etc.)” included in the patent document are hybrid components. A classifier was designed. In addition, the values of α and γ set in advance when the discriminant function generation unit 4 calculates the parameter w of the discriminant function are set to 0.5 and 1, respectively.

  The accuracy of F-term assignment and the accuracy of F-term search order (priority) obtained by applying the multiple label assigning apparatus 1 of the specific example of the embodiment to a new patent document is expressed as F value and 11 Evaluation was made using the point average precision (details will be described later).

  The larger the 11-point average relevance rate, the more accurate the search order of F terms given to patent documents. That is, even if the number of correct F terms to be given is the same, the 11-point average precision rate increases as the search order is accurate. Since the F value and the 11-point average relevance rate are evaluation scales calculated for each patent document to which a label is assigned, the average of the F value for all patent documents and 11 points are used for the evaluation of the multiple label assigning apparatus 1. The average value of the average precision was used.

  Each of the method according to the present application and the method according to the comparative example shown in Table 1 uses a hybrid classifier disclosed in JP 2006-107354 A as a binary classifier. In the method according to the comparative example, labeling is performed using only this hybrid classifier, whereas the method according to the present application is different from the method according to the comparative example in that a discrimination function is newly set by the discrimination function generation unit 4.

  As shown in Table 1, the F value is higher in the method according to the present application than the method according to the comparative example, and the 11-point average precision is comparable between the method according to the comparative example and the method according to the present application. Thus, it can be seen that the method according to the present application shows superiority in terms of labeling accuracy and has the same performance in terms of accuracy of label search rank as compared with the method according to the comparative example. . That is, the method according to the present application is less likely to be determined as “no label” than the method according to the comparative example. In addition, although empirical data is not shown here, it is clear from the full text of the present specification that the method according to the present application requires significantly less calculation amount than the methods shown in Non-Patent Documents 3 and 4. .

This is the end of the description of the embodiments, but the aspects of the present invention are not limited to these.
For example, the content may be other documents such as a paper or a novel, image data, or the like as long as the content vector can be expressed even if it is not a patent document. In addition, specific configurations such as labels and flowcharts can be appropriately changed without departing from the spirit of the present invention.

It is a functional block diagram which shows the structure of the multiple label provision apparatus of this embodiment. It is the flowchart which showed the outline | summary of the process of a multiple label provision apparatus. It is the flowchart which showed the process of calculation of the predicted value of a score. It is the flowchart which showed the parameter learning process of an identification function. It is the flowchart which showed the process which determines the possibility of labeling to a content using an identification function.

Explanation of symbols

1 Multi-labeling device 2 Training data DB
3 score function generation unit 4 discriminant function generation unit 5 memory 6 multiple label assignment unit 7 input unit 8 output unit

Claims (7)

It is a labeling method by a labeling device that gives one or more labels representing the type of content to content having at least one of characters and images,
The labeling device includes a processing unit including a score function generation unit, an identification function generation unit, and a label addition unit, and a storage unit,
The score function generation unit uses a feature vector collected from the appearance frequency of each feature of the content read from the storage unit, and a binary classifier that is a program designed for each of the labels And generating a score function for calculating a score for determining whether or not the label can be given for each label target content,
The discriminant function generation unit uses the training data set that is read from the storage unit and includes content for which labels to be assigned have already been determined, and uses the score function and the score calculated by the score function. Generating a discrimination function having a threshold value for determining whether or not labeling is possible;
The label assigning unit calculates an identification function value of the content to be labeled for each label using the identification function, and the label when the identification function value is equal to or greater than a predetermined value A labeling method, characterized in that labeling is performed by storing in the storage unit in association with content. The score function generator is
As the binary classifier, a classifier based on a probability model that gives a probability of assigning a label to content is used, and the classifier learns in advance using a training data set including content for which a label to be assigned is already determined. If
Using the binary classifier read from the storage unit, the probability when it is assumed that a label is given to the content to be labeled and the probability when it is assumed that no label is given are calculated, and the label is calculated. A score function is generated that calculates a logarithm of a value obtained by dividing a probability when it is assumed to be given by a probability when it is assumed that no label is given as a score for determining whether or not the labeling target content can be labeled. The labeling method according to claim 1, wherein: The discriminant function generator generates the discriminant function,
A combination of labels given to the excluded training data using a predicted value of the score calculated from the estimated value of the parameter of the binary classifier calculated by excluding one training data from the training data set. Using the predicted value, the value obtained by dividing the number of labels correctly given by using the predicted value by the total number of labels to be given to the training data, and the number of labels correctly given by using the predicted value 2. The label according to claim 1, wherein an estimated value of the parameter of the discriminant function is calculated so as to meet a criterion for maximizing a harmonic mean of a value divided by the total number of labels given in the first step. Grant method. The discriminant function generator
A logistic function using a difference between a threshold value given by the parameter of the discriminant function and a score function value as a variable when calculating the estimated value of the parameter of the discriminant function so as to meet the criterion for maximizing the harmonic mean The method for assigning a label according to claim 3, wherein an estimated value of the parameter of the discriminant function that maximizes the approximate value of the harmonic mean defined using is searched using a gradient method. A label attaching device that attaches at least one label representing the type of content to content having at least one of characters and images, and includes a processing unit and a storage unit,
The processor is
Using a feature vector that collects the appearance frequency of each feature of the content read from the storage unit, and a binary classifier that is a program designed for each of the labels, and assigns the label to each label A score function generating unit that generates a score function for calculating a score for determining whether or not the content is to be provided for each content to be labeled,
Using a training data set consisting of content that has already been determined to be assigned labels read from the storage unit, whether or not label assignment is possible is determined using the score function and a score calculated by the score function. A discriminant function generator for generating a discriminant function having a threshold for
The identification function value of the labeling target content for each label is calculated using the identification function, and the label when the identification function value is equal to or greater than a predetermined value is associated with the labeling target content A label attaching unit that performs label assignment by storing in a storage unit.   A program for causing a computer to execute the labeling method according to any one of claims 1 to 4.   A computer-readable storage medium storing the program according to claim 6.

Images