JP2005063071A - Document set classification device and its program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To apply a more suitable classification which has excellent exhaustiveness without depending on the characteristics of specific languages. <P>SOLUTION: This document set classification device for applying a classification to a document set being the group of a plurality of documents is provided with: a decision means 101 for deciding whether the theme of the document set is related with a single specific expression or a plurality of specific expressions and further deciding to which specific expression class the specific expressions are belonging; and an output means 102 for outputting information about a classification specified by two factors, that is, whether the specific expressions related with the theme are single or a plurality and to which specific expression class the specific expressions are belonging on the basis of the decision result of the decision means 101. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to a classification apparatus for assigning a classification to a document set including a plurality of documents, and a program therefor.

  Automatic summarization of multiple documents is an area of increasing interest in summary research in recent years. In the US Document Understanding Conference (DUC) and the Japanese Text Summarization Challenge (TSC), multiple document summaries have been added as subjects to evaluate the summarization system. Multiple document summarization is the summarization of a set of documents containing multiple documents collected on a single subject into a single document. More specifically, from a series of reports collected from the series of reports from the beginning to the end of an incident, the contents of the actions and remarks of specific individuals, the damage situation of earthquakes that occurred in various places, etc. Generating a summary on the subject.

In order to improve the accuracy of summarization, it is necessary to correctly grasp the subject of the document set and to select an appropriate summarization method and output format accordingly. There is a research by McKeown et al. Of Columbia University as a prior research for classifying a document set from the viewpoint of multi-document summarization (see Non-Patent Document 1). McKeown et al. As a classification to be assigned to an article set that includes multiple newspaper articles,
(A) Single-Event (a set of articles related to a single event limited to a specific region / period)
(B) Person-centered (A set of articles describing events related to a specific person)
(C) Multi-Event (a set of articles about multiple events across different regions / periods, usually with different actors)
(D) Other (a set of articles that are not related to the above three categories and are vaguely related to each other)
The following four types were defined. And as a clue when classifying article sets, the time span between all articles in the article set, the ratio of articles published on the same day, the frequency of words starting with capital letters, the frequency of personal pronouns such as he, she, etc. Used.
KR McKeown and R. Barzilay and D. Evans and V. Hatzivassilogou and M. Yen Kan and B. Schiffman and S. Teufel, [online], “Columbia Multi-Document Summarization: Approach and Evaluation”, Online Proceedings of DUC2001 <http : //www-nlpir.nist.gov/projects/duc/pubs/2001papers/columbia#redo.pdf>

McKeown et al.'S classification efficiently categorizes the properties that are often found in the set of articles being summarized. However, there are some problems. That is,
・ The article set classified as Other increases. There may be other appropriate classifications for these.
Of the cues used in the algorithm for determining classification, the frequency of words beginning with capital letters and the frequency of personal pronouns such as he and she are specific to English. In order to classify article sets more generally, it is necessary to consider the clues to be used.
The article data that is classified as the non-patent document 1 is created for use in a multiple article summary evaluation workshop. For this reason, it is arranged or biased compared to a general article set.
・ Summary systems such as McKeown apply the same summarization method by equating Multi-Event and Other when actually summarizing multiple articles, and the significance of distinguishing between Multi-Event and Other is lost. ing.

  The present invention has been made in view of the above, and provides a classification device that can define a more suitable classification that does not depend on the characteristics of a specific language and that enhances the comprehensiveness of classification, and can assign a classification based on the classification to a document set. To do.

  In order to solve the above-described problem, in the present invention, classification is given to a document set that is a set of a plurality of documents. As shown in FIG. 1, the subject of the document set is a single named entity (Named Entity). ) Or a plurality of specific expressions, and determination means 101 for determining which specific expression class the specific expression belongs to, and the document based on the determination made by the determination means 101 Output means 102 for outputting information about a classification defined by two elements, ie, a specific expression class to which the specific expression belongs, and whether or not the specific expression related to the theme of the set is single or plural. A document set classifying apparatus is configured.

  In the present invention, the classification is defined based on the unique expression class to which the unique expression related to the subject included in the document set belongs, and the classification apparatus according to the present invention assigns the defined classification to the document set. In such a case, the degree of dependence on a specific language is reduced, and the comprehensiveness of classification is increased. In other words, the possibility that a document set is classified as Other is reduced.

Here, the specific expression means a specific noun such as a person name and an organization name included in a document, a numerical expression such as a date and a monetary amount, and other expressions that are elements of information extraction. The proper expression is important as information, and the expression indicating the content is almost uniquely determined. There are various ways to define a specific expression class to which a specific expression belongs, but as one aspect, a person name (Person), an organization name (Organization), a place name (Location), a facility name (Facility), a unique object name (Product (product name, Legal names etc.)) and class definitions that include six types of event names (Event) can be adopted. When this specific expression class definition is adopted, the classification to be given to the document set is as follows. That is,
(A) Single-Person (Document set for a single person)
(B) Single-Organization (Document set for a single organization)
(C) Single-Location (Document set for a single region)
(D) Single-Facility (Document set for a single facility)
(E) Single-Product (Document set for a single unique product)
(F) Single-Event (document set related to a single event)
(G) Multi-Person (Document set for multiple persons)
(H) Multi-Organization (Document set for multiple organizations)
(I) Multi-Location (Document set for multiple regions)
(J) Multi-Facility (Document set for multiple facilities)
(K) Multi-Product (Document set for multiple unique products)
(L) Multi-Event (document set related to multiple events)
(M) Other
However, the definition of the specific expression class and the definition of the classification of the document set are not limited to this. Therefore, for example, an animal name class (Single / Multi-Animal) can be added, and a person name, an animal name, etc. can be combined into one class, in other words, a “subject name class” that is a class representing an action subject it can.

  The determination unit 101 in the document set classification apparatus normally uses at least one of the frequency of the specific expression or the frequency of the specific expression class appearing in a plurality of documents included in the document set as a material. A determination is made as to whether or not there are a plurality of unique expressions, and a specific expression class to which the specific expression belongs is determined. However, the frequency mentioned here is not limited to the appearance frequency of specific specific expressions that appear in a plurality of documents included in the document set, but the number of documents in which specific specific expressions appear (documents in the document set). Frequency).

  In addition, the determination unit 101 uses as a material at least one of the frequency of a cluster that appears in a plurality of documents included in the document set and the frequency of a unique expression class to which the cluster is related. The determination of the specific expression class to which the specific expression related to the subject belongs may be performed. A cluster is a noun or compound noun that is strongly associated with a particular named entity class. For example, a title such as “Prime Minister” is a cluster of a person name class, and a noun such as “earthquake” is a cluster of an event name class. Incidentally, clusterum is a general noun, unlike proper expression itself.

  In particular, in the case where the creation or publication time of each document included in the document set is known, the determination unit 101 has created or announced each of the plurality of documents included in the document set. With reference to the point-in-time information, the specific expression related to the subject of the article set is single, provided that some or all of these documents are created or published within a predetermined period. In addition, it may be determined that the named entity class to which it belongs is an event name class. At this time, a single-event classification is assigned to the document set.

  As shown in FIG. 2, the above sentence set classification device extracts keywords existing in a given document, calculates the similarity between the keyword of one document and the keyword of another document, and the similarity Given that the document set generation means 103 can further generate at least one document set from a plurality of documents by assigning these documents to the same document set when the degree exceeds a threshold value Processing until a plurality of documents are sorted into one or a plurality of document sets and a classification is given thereto can be executed in a batch. This is useful for constructing a system that automatically generates one or more summaries based on a given document.

  Further, as shown in FIG. 3, the document set summarization apparatus refers to the classification given to the document set, selects a summarization algorithm corresponding to the classification, and summarizes a plurality of documents included in the document set. By configuring the document summarizing apparatus including the summarizing means 201, it is possible to execute summarization of a plurality of documents more appropriately.

  According to the present invention described in detail above, it is possible to assign a more suitable classification to a document set that does not depend on the characteristics of a specific language and that improves the comprehensiveness of classification.

  Hereinafter, an embodiment of the present invention will be described with reference to the drawings. First, the definition of the classification in the present invention and its validity will be described. Here, an article set including a plurality of newspaper articles is experimentally generated as a document set to be analyzed, and this is analyzed to define a classification based on the specific expression class. Then, by applying this classification to another Japanese newspaper article set as test data and the English newspaper article set used in DUC2001, the validity of the classification definition is verified.

In order to avoid the bias of the article set, an article was randomly extracted from the Japanese newspaper article corpus, and articles similar to the article were collected using an information retrieval system to generate an article set. The specific procedure is as follows.
(1) Randomly select one article from articles included in the Japanese newspaper article corpus (2) Extract a keyword string from the selected article; the keyword string is a known Japanese morphological analysis software (Tatsuo Kurohashi) , Makoto Nagao Japanese morphological analysis system Juman version 3.61. Kyoto University, 1999.) From the results of morphological analysis, nouns excluding temporal nouns and adverbial nouns with frequency 2 or higher were extracted (3) The similarity between articles is calculated | required using a keyword row | line | column; A keyword row | line | column is extracted similarly to said (2) about each article | item, and the similarity degree between keywords was calculated | required using Dice's coefficient (coefficient of Dice) (4 A similar article is taken out; articles other than the same article whose Dice coefficient is a predetermined value (for example, 0.5) or more are regarded as similar articles and taken out.

  When an article set including three or more articles is selected from a plurality (50) article sets obtained by repeating the above-described article set generation a plurality of times (for example, 50 times), 26 article sets are obtained. From these, 19 articles were left out if items that were considered to have almost the same content were omitted. The subject of these 19 article sets is shown in FIG.

  As shown in the analysis result of FIG. 4, “Single” is a description in which most articles in an article set describe a single event, a specific expression such as a person name and an organization name. On the other hand, Multi is a description in which articles in an article set describe a plurality of different events, specific expressions such as person names and organization names.

  In the present invention, the subject of the article set as the document set is classified through selecting one of the proper expression classes. The extended class definition (S. Sekine, K. Sudo and C. Nobata, “Extended named entity hierarchy”, In Proceedings of the LREC-2002 conference, 2002.) was adopted as the definition of the proper expression class. Although this definition is hierarchical, the top level hierarchy could be used to categorize almost all article sets. There is only one article set classified as Other without assigning a unique expression class (article set No. 14 in FIG. 4). If we assign a specific expression class to this article set, it will be an animal name (Animal) class, but since there are not many article sets classified into the animal name class, it was set to Other. As described above, as a result of selectively assigning the specific expression class, the 13 types of classifications already mentioned are defined. Incidentally, the analysis result shown in FIG. 4 does not have an article set corresponding to the classification of the location class (Location) class, but an article set concerning a specific country or region can exist and is also important. Included in the set classification definition.

  Subsequently, the test data is classified based on the defined classification. The test data used here is created in the same manner as described above from the Japanese newspaper article corpus. And about 20 article sets created as test data, two subjects gave classification independently. In principle, subjects assigned one category to each set of articles, but for some set of articles, judged that multiple categories were possible and assigned two categories. The number of each classification assigned by two subjects is shown in FIG. The coincidence rate between subjects was 55% when the categories first selected by the subjects were compared, and 85% when the second category was included. There was no article set classified as Other. The number of article sets to which the subject assigned two categories was 6 and 5, respectively.

  Based on the results of classification by the subjects, correct data for test data classification was created. In the 17-article set that is a common classification among the subjects, the assigned classification is regarded as the correct answer, while in the 3-article set in which the classification is divided among the subjects, the correct classification is determined by discussion between the subjects. It should be noted that this correct answer data is used to evaluate the result of an automatic classification experiment performed by a document set classification apparatus described later.

  In order to show that the definition of this classification is valid in other languages, we also tried to assign a classification to the English newspaper article set, which is training data used in the multi-article summary task of DUC2001. As before, the two subjects independently assigned a classification that they judged to be optimal for the article set. Subjects assigned one or two categories to each article set. The number of each classification assigned by two subjects is shown in FIG. There were 4 sets of articles each assigned 2 categories by the subjects. The other category was only one set given by one of the subjects as the second category. The agreement rate between subjects was 80% when the categories selected by the subjects were compared first, and 93.3% when the categories up to the second category were included. This matching rate is higher than that for Japanese article test data. This is probably because the data of DUC2001 is more prepared than the Japanese article test data, and the subject of the article set is intentionally selected.

  Hereinafter, a document set classification apparatus for assigning a defined classification to an article set will be described in detail. The document set classification apparatus according to the present embodiment is configured by installing a predetermined program in the computer 1. For example, as shown in FIG. 7, the computer 1 has a hardware resource such as a processor 1a, a main memory 1b, and an auxiliary storage device 1c represented by a hard disk drive as a controller 1d (that is, a so-called system controller, I / O controller). Etc.) and operate in cooperation with each other. Although not shown, a communication device for exchanging data with the outside via an electric communication line, an input device such as a keyboard and a pointing device for accepting an operation input by a user, a display for displaying information as an image or video, and this It does not prevent the mounting of a display control device (so-called graphics chip or the like) or the like for sending a video signal to the display.

  Normally, a program to be executed by the processor 1a is stored in the auxiliary storage device 1c. When the program is executed, the program is read from the auxiliary storage device 1c into the main memory 1b and decoded by the processor 1a. Then, the hardware resources are operated according to the program so that at least the functions of the determination unit 101 and the output unit 102 are exhibited.

  The determination unit 101 determines whether the subject of an article set (document set) including a plurality of articles (documents) relates to a single specific expression or a plurality of specific expressions, and which of the specific expressions It is determined whether it belongs to the proper expression class. Article data that is an element of an article set given as input is normally stored in advance in a required storage area of the main memory 1b or the auxiliary storage device 1c. Therefore, the processor 1a reads the article data stored in the main memory 1b or the auxiliary storage device 1c according to the program, and makes a determination on the classification to be assigned to the article set based on the read data.

  Based on the determination made by the determination unit 101, the output unit 102 determines whether or not the information about the classification to be given to the article set, that is, whether or not the unique expression related to the subject of the article set is single or plural. Outputs information about a classification defined by two elements, a named entity class to which the expression belongs. Examples of information output modes include display on a display screen, printout using a printer (not shown), transmission to an external computer via a communication device and a telecommunication line, main memory 1b, or auxiliary storage device Write to 1c and others. The specific configuration of the output unit 102 depends on the output mode of information to be added to the article set.

  In the automatic assignment of classification by the document set classification device, the appearance frequency of words and specific expression classes appearing in multiple articles included in the article set, and the article frequency (required words, specific expression classes, etc. appeared in the article set) The number of articles (which is different from the idf value used in information retrieval or the like) is used as a clue. Therefore, it is desirable that the computer 1 can also exhibit the function as the specific expression extracting means shown in FIG. The specific expression extraction unit extracts a specific expression that appears in an article included in the article set, and determines a specific expression class to which the extracted specific expression belongs. The specific expression extraction means enumerates the specific expressions that appear in the article by referring to the analysis result by the morpheme analysis software 104a, for example, referring to the analysis result by the morpheme analysis software 104a by dividing the sentence in the article into words and giving part of speech Can be configured using specific expression extraction software 104b for classifying the specific expressions into specific expression classes. An example of morphological analysis and specific expression extraction is shown in FIG. In the illustrated example, the unique expression is extracted by a pattern-based system. That is, the processor 1a performs morphological analysis on an article given as an input according to a program, and the obtained morpheme analyzed article is a specific expression list 104c (a data group in which specific expressions and their specific expression classes are listed. All of the specific expressions in the article are extracted by checking against the main memory 1b or the required storage area of the auxiliary storage device 1c. After that, there exists a nested relationship of multiple named entities (for example, the named entity “Yoshimoto” belonging to the name class exists in the named entity “Yoshimoto Kogyo” belonging to the organization name class). If there is, the unique expression having a longer character string is preferentially recognized (ie, recognized as “Yoshimoto Kogyo” instead of “Yoshimoto”), the unique expression is uniquely determined, and the result is output. The determination unit 101 can classify the article set with reference to the output from the specific expression extraction unit.

  In addition to proper expressions, clusters that appear in articles can also be used as clues for classification. A cluster is a noun or compound noun that is strongly associated with a particular named entity class. For example, a title such as “Prime Minister” is a cluster of a person name class, and a noun such as “earthquake” is a cluster of an event name class. Therefore, it is desirable that the computer 1 can also exhibit the function as the cluster extraction means shown in FIG. The specific expression extraction unit extracts a cluster that appears in an article included in the article set, and determines a specific expression class to which the extracted cluster is related. The clusterome extraction means, for example, enumerates the morpheme analysis software 104a that divides sentences in an article into words and adds parts of speech, and lists the clusters that appear in the article with reference to the analysis results by the morpheme analysis software 104a. Can be configured using clusterum extraction software 105a for classifying the entity into a proper expression class. An example of morphological analysis and cluster extraction is shown in FIG. Also in the illustrated example, clustering extraction is performed by a pattern-based system in the same manner as the above-described example of extracting the unique expression. That is, the processor 1a performs morphological analysis on an article given as an input according to a program, and the obtained morpheme-analyzed article is a cluster list 105b (a data group in which a cluster and its associated unique expression class are listed. And all the clusters in the article are extracted with reference to the main memory 1b or the required storage area of the auxiliary storage device 1c), and the result is output. The judgment unit 101 can classify the article set with reference to the output from the cluster extraction unit. Incidentally, the cluster list 105b can be created from an existing thesaurus and a manually collected result. The number of clusters created and used by the inventors experimentally is about 16,000 words.

  In addition to proper expressions and clusters, article publication dates and article time spans can also be used as clues for classifying article sets. For example, if most articles included in an article set are published on the same day or within a predetermined short period, it is likely that the article set can be classified as a single-event.

The determination algorithm executed by the determination unit 101 in this embodiment will be described in detail with reference to the flowcharts of FIGS. In the present embodiment, the determination unit 101 classifies article sets using the following four algorithms. First, the determination unit 101 determines whether an article set given as an input is classified as a single-event according to the first algorithm. This determination is made with reference to information regarding the time when each of the plurality of articles included in the article set is created or published. If articles (or all) of a certain percentage or more of the articles included in the article set are created, published, announced, published, etc. within a predetermined period, the judgment means 101 selects the article set. Classify as Single-Event. More specifically, when articles included in an article set are newspaper articles, if most of them are published on the same day or within a predetermined short period, a single-event classification is assigned to the article set. Make a decision to do so. The processing procedure at this time is shown in FIG. The determination means 101 is information relating to the posting date of each article included in the article set given as input (this information is stored in a required storage area of the main memory 1b or the auxiliary storage device 1c in association with article data, for example). Referring to (stored) (step S101), the publication date with the highest article frequency is confirmed, and the number of articles published on this day is counted (step S102). If the number of articles published on this day or the ratio of the number of articles published on this day to the total number of articles included in the article set exceeds _a preset threshold value Ta (step S103), The given article set is classified into Single-Event (step S106). Also, the time span of the posting dates of all articles included in the article set is confirmed (step S104), and if this time span is below a preset threshold T _s (step S105), the given article set is set. Are classified into Single-Event (step S105). That is, the determination is made using the two parameters of the ratio of articles published on a specific day and the maximum value of the time span between articles as materials. When the article set cannot be classified as Single-Event, the process proceeds to the second algorithm. In addition, it does not prevent executing the first algorithm by referring to the date expression appearing in the article.

Next, the determination unit 101 determines whether the given article set is classified into a single-class (any specific expression class) according to the second algorithm. This determination is made by counting the appearance frequency and the article frequency of the unique expression having a high article frequency. If a specific unique expression frequently appears across many articles included in the article set, the determination unit 101 regards the specific expression as representing the subject of the article set, and sets the specific expression class to which the specific expression belongs to the article set. The element class of The processing procedure at this time is shown in FIG. The determination unit 101 refers to the output of the result of the specific expression extraction process performed by the specific expression extraction unit for the given article set (step S201), and the specific expression and the specific expression that appear in the articles included in the article set The appearance frequency is counted for each unique expression class to which the expression belongs (step S202). And the article frequency is counted for each unique expression appearing in the article, and the unique expression having the highest article frequency in the article set is selected (step S203). Here, when there are a plurality of specific expressions having the same article frequency, for example, a specific expression having a higher appearance frequency is selected. Therefore, if the article frequency of the selected unique expression (or the ratio of the article frequency of the selected unique expression to the total number of articles included in the article set) exceeds a preset threshold T _d (step S204), the unique expression class to which this unique expression belongs can be used as the element class of the article set classification. In addition, when classifying an article set into a single-class, it does not prevent adding further judgment processing. That is, as shown in FIG. 11, that the frequency ratio of the named entity class elected frequency / elected unique representation of the named entity belongs exceeds the threshold value T _w a predetermined As a condition (step S207), the specific expression class to which the selected specific expression belongs may be used as the element class of the article set classification (step S208). However, in the present embodiment, it is assumed that the event name class is determined with priority over other specific expression classes from the experimental results of examining the training data. Therefore, when the selected unique expression belongs to the event name class (step S205), the given article set is classified as a single-event as it is (step S206). When the article set cannot be classified into Single-class, the process moves to the third algorithm.

Subsequently, the determination unit 101 determines whether the given article set is classified into Multi-class (any specific expression class) according to the third algorithm. This determination is made by counting the appearance frequency and the article frequency of the unique expression class having a high article frequency. If a specific expression belonging to a specific specific expression class frequently appears across many articles included in the article set, the determination unit 101 regards the specific expression class as representing the subject of the article set, and determines the classification of the article set. Element class. The processing procedure at this time is shown in FIG. The determination unit 101 refers to the output of the result of the specific expression extraction process performed by the specific expression extraction unit for the given article set (step S201), and the specific expression and the specific expression appearing in the articles included in the article set. The appearance frequency is counted for each unique expression class to which the expression belongs (step S202. These processes have already been executed in the second algorithm). The article frequency is counted for each unique expression class to which the unique expression that appears in the article belongs, and the unique expression class with the highest article frequency is selected in the article set (step S301). Here, when there are a plurality of unique expression classes having the same article frequency, for example, a specific expression class having a higher appearance frequency is selected. Therefore, if the article frequency of the selected unique expression class (or the ratio of the article frequency of the selected unique expression class to the total number of articles included in the article set) exceeds a preset threshold value _Td. (Step S302), this specific expression class can be used as an element class of article set classification. In addition, when classifying an article set into Multi-class, it does not prevent adding further judgment processing. That is, as shown in FIG. 12, the frequency of occurrence of elected frequency / total named entity class named entity class (all named entities), that the ratio of exceeds the threshold value T _C of predetermined As a condition (step S305), the selected specific expression class may be used as an element class of article set classification (step S306). However, in this embodiment, the event name class is determined with priority over other specific expression classes, and when the selected specific expression class belongs to the event name class (step S303), the given article set is selected. As it is, it is classified into Multi-Event (step S304). In addition, it does not interfere with setting the threshold value T _C of different from each named entities. For example, regarding the threshold value T _C to be compared with the value of the appearance frequency of the selected specific expression class / the appearance frequency of all the specific expression classes, the selected specific expression class is any of the place name class, the organization name class, and the person name class. In some cases, a stricter or larger threshold value T _C1 is applied, and in a case where the elected named entity class is other than the above, a weaker or smaller threshold value T _C2 (T _C1 > T _C2 ) is applied. it can. When the article set cannot be classified into Multi-class, the process proceeds to the fourth algorithm.

When an appropriate classification to be assigned to the article set cannot be found in the process up to the third algorithm, the determination unit 101 examines the classification to be assigned according to the fourth algorithm. It can be said that the fourth algorithm executes the second algorithm or the third algorithm on the cluster, not the specific expression. In other words, a judgment is made on the specific expression class to which the specific expression related to the subject of the given article set belongs, using at least one of the frequency of the cluster appearing in the article and the frequency of the specific expression class related to the cluster as a material. . Even if a specific cluster frequently appears across many articles in an article set, the classification to be assigned to the article set is preferably a multi-class instead of a single-class. This is because the cluster is a general noun, not a proper noun, and can indicate multiple types of proper expressions. The processing procedure at this time is shown in FIGS. The determination unit 101 refers to the output of the result of the cluster extraction process performed by the cluster extraction unit for the given article set (step S401), and the cluster and cluster appearing in the articles included in the article set. The appearance frequency is counted for each of the unique expression classes related to the event (step S402). And the article frequency is counted about each cluster which appears in an article, and the cluster with the highest article frequency in the said article set is selected (step S403). Here, when there are a plurality of clusters having the same article frequency, for example, a cluster having a higher appearance frequency is selected. Accordingly, if the article frequency of the selected clusterum (or the ratio of the article frequency of the selected clusterum to the total number of articles included in the article set) exceeds a preset threshold value _Td (step In S404, the unique expression class to which this cluster is related can be used as the element class of the article set classification. In addition, when classifying an article set into Multi-class, it does not prevent adding further judgment processing. That is, as shown in FIG. 13, the frequency / elected cluster beam of elected cluster beam is above the frequency of occurrence, the threshold T _w which the ratio of predetermined named entities related classes (Step S407), the specific expression class to which the selected specific expression belongs may be used as the element class of the article set classification (step S408). However, the event name class is determined in preference to the other named entity classes, and when the selected cluster is related to the event name class (step S405), the given article set is directly used as Multi. -Events are classified (step S406). In addition to the above, the article frequency is counted for each unique expression class related to the cluster that appears in the article, and the unique expression class having the highest article frequency in the article set is selected (step S409). When there are a plurality of unique expression classes having the same article frequency, for example, a specific expression class having a higher appearance frequency is selected. Therefore, if the article frequency of the selected unique expression class (or the ratio of the article frequency of the selected unique expression class to the total number of articles included in the article set) exceeds a preset threshold value _Td. (Step S410), this specific expression class can be used as an element class of article set classification. When classifying article sets into multi-classes, additional judgment processing can be added. That is, as shown in FIG. 14, the frequency of occurrence of elected frequency / total named entity class named entity class (all cluster beam), that the ratio of exceeds the threshold value T _C of predetermined As a condition (step S413), the selected specific expression class can be used as an element class of article set classification (step S414). However, the event name class is determined with priority over other specific expression classes. When the selected specific expression class belongs to the event name class (step S411), the given article set is directly used as a Multi-Event. Classification is performed (step S412). Here, as in the third algorithm, it is allowed to set different threshold values T _C1 and T _C2 for each unique expression.

  If no classification can be given using all the above algorithms, the determination unit 101 gives a predetermined default classification to the article set (step S415). The default classification is, for example, Multi-Event or Other.

The results of an automatic classification experiment for assigning a classification to the test data described above using the document set classification apparatus of the present embodiment will be described. Here, each threshold value in the algorithm is manually determined based on the training data. The threshold values were set such that T _a = 0.33, T _s = 150, T _d = 0.90, T _w = 0.40, T _C1 = 0.80, and T _C2 = 0.40. However, it goes without saying that the size of the threshold is not limited to the value shown here. Evaluation of the result of the automatic classification experiment for the test data is shown in FIG. FIG. 15 also shows the evaluation result and the baseline of the classification given by the subject. The evaluation of the subject is a comparative evaluation between the classification given by each subject and the correct answer. The evaluation for the correct answer of both subjects is higher than the concordance rate of 55% between the two subjects, because the correct answer of the classification was created by combining the classification grant results by both subjects. The baseline is the number (and the occupancy) in the test data of the article set of the most frequent classification (in this experiment, Single-Event) in the training data. For the document set classification device, the “match” value indicates the number (and percentage) of article sets in which the classification output by the document set classification device matches the correct answer, and the “partial match” value is output by the document set classification device. The number (and percentage) where the classification matches any of the classifications given by the subject. Both are included regarding the article set which the test subject gave the some classification | category. For the subject, the “match” value indicates the number (and percentage) of article sets where the first category given by the subject matched the correct category, and the “partial match” value is the second category given by the subject. The number of article sets (and percentage) that match the correct answer classification is shown.

  The document set classification apparatus correctly classified nine of the 20 article sets, and the classification results of the two article sets were included in the classification given by the subject. Three of the remaining nine article sets that were not classified correctly were classified as Single-Event while the correct answer was Single-Product. The unique names (Product) that appear in these article sets used in the experiments are specific bills and international treaties, etc., and the articles included in the article set describe the deliberations of the bills and remarks on international treaties. It was a thing. In the current algorithm, Single-Event is given priority for classification, and it is considered that such an error has occurred. However, since bills related to correct classification and unique names of international treaties, etc. appear throughout the articles in the article set, the determination means 101 once gave the single-event classification while performing the subsequent determination process. It may be possible to continue and configure it so that it can be given a single-product classification again.

In another set of 3 articles, the correct classification was Single-Event, but a different classification was assigned. In these article sets, the description corresponding to the event name (Event) was expressed in the form of a phrase or a clause instead of a specific expression. For example, the expression “the suspicion of affair against Monica Luinski, former president of the White House, former President of Clinton” is not a specific expression, but an expression that points to a specific event. In the current specific expression extraction system, such an expression cannot be recognized as one specific expression. Further, since the time span of the article set was larger than the set threshold value T _s (in the above example, one year or more), the determination unit 101 could not assign the single-event classification to the article set. This suggests that it is necessary to use a new clue in addition to the currently used clue to reliably assign the single-event classification.

  In the course of the experiment, it was found that some article sets could inherently be given more than one classification. One of the reasons is that there are many cases where classifications based on other unique expression classes can be assigned to article sets that should be assigned classifications based on event names, such as Single-Event and Multi-Event. Many events are often related to a specific person name, organization name, place name, etc., and focus on specific expressions (other than the event name) related to the event for a set of articles about the event. It is possible. Another reason is that it is difficult to distinguish between Single and Multi in the event. Some events may include several small events. For example, an article set related to the “Sydney Olympics” may be assigned a single-event classification for one sporting event, but this article set includes articles reporting the results of multiple events. Then, it is also possible to give the classification of Multi-Event by paying attention to them. How the event unit is recognized depends on the subject's perspective.

  Incidentally, in the above-described experiment, manual correction is added to the result output by the specific expression extraction means to eliminate errors in the specific expression extraction task. When assigning a classification mechanically using the document set classification apparatus of this embodiment, it is difficult to completely prevent an error from occurring at the stage of the specific expression extraction task (because perfect specific expression extraction software does not exist currently) ). To complement the proper expression extraction process in the proper expression extraction means, co-references and literature (Y. Shin-yama, S. Sekine K. Sudo, and R. Grishman, “Automatic paraphrase acquisition from news articles”, In It may be possible to introduce a paraphrase recognition method for event description as described in the Proceedings of the HLT-2002 conference, 2002.

  In the above description, it is assumed that a plurality of articles (documents) given as input are sorted in advance into article sets (document sets). However, there may be a situation where a plurality of articles are simply given as input. In such a case, it is preferable that the document set classification device can mechanically execute a process of sorting a plurality of given articles into one or a plurality of article sets and adding a classification to the sorted article sets. . That is, it is preferable that the computer 1 constituting the document set classification apparatus can also function as the document set generation unit 103 shown in FIG.

  The document set generation unit 103 is composed mainly of software, extracts keywords existing in a document given as input, calculates the similarity between the keyword of one document and the keyword of another document, When the degree of similarity exceeds a threshold, these documents are assigned to the same document set, thereby generating at least one document set from a plurality of documents. The procedure of processing executed by the document set generation unit 103 is similar to the article set generation method already described. That is, the processor 1a selectively reads one of article data (usually stored in a required storage area of the main memory 1b or the auxiliary storage device 1c) given as an input based on a program, A keyword string is extracted from this article data. Keyword extraction can be performed using morphological analysis software. For example, a noun excluding temporal nouns and adverbial nouns from the result of morphological analysis of article data is extracted as a keyword having a frequency equal to or higher than a predetermined value (for example, 2). After extracting a keyword by the above method for each article data given as input, the processor 1a calculates the similarity between the keyword string related to one article data and the keyword string related to other article data. As an index of similarity calculated by the processor 1a, Dice's coefficient, Jacquard measure, cosine similarity and the like can be adopted. Then, an article set is generated by including a plurality of articles whose similarity index is a predetermined value (for example, 0.5) or more as a similar article in one article set.

  By the way, by referring to the information about the classification of the article set output by the document set classification device, a summary algorithm suitable for the article set can be selected. Therefore, a summary device that summarizes a plurality of articles (documents) contained in an article set (document set) using a summary algorithm corresponding to the classification is constructed by referring to the information about the classification output by the document set classification device. For example, the accuracy of summarization can be improved.

  The document summarization apparatus in this embodiment is constructed by installing a predetermined program on the computer 1 constituting the document set classification apparatus or a computer (not shown) different from the computer 1. Normally, the program is stored in the auxiliary storage device 1c, and when executed, the program is read from the auxiliary storage device 1c into the main memory 1b and decoded by the processor 1a. Then, hardware resources are operated according to the program so as to exhibit the function as the summarizing means 201 shown in FIG.

  The summarizing means 201 refers to the information about the classification given to the article set output from the document set classification device, and converts the articles included in the article set into a single document by the summarization algorithm corresponding to the classification. To summarize. That is, the processor 1a refers to the information about the classification given to the article set based on the program, and selects a summary algorithm corresponding to this classification. Thereafter, the article data included in the article set given as input is read, and a summary is generated. The summarizing means 201 in this embodiment is composed mainly of software. Known multi-article summary software can be applied to the software that is the main body of the summary means 201. Known multi-article summary software generally extracts sentences (or sentences) judged to be important from a plurality of articles included in an article set, and generates a summary based on the extracted sentences. The importance of each sentence is calculated by adding scores related to a plurality of conditions such as the position of the sentence, the length of the sentence, the frequency of words appearing in the sentence, and the similarity to the headline. In calculating the importance, weights are assigned to scores relating to individual conditions (weights are obtained through training using training data). Therefore, the summarizing means 201 in the present embodiment calculates the importance of each sentence in consideration of the score based on the classification given by the document set classification device. As a specific example, if the classification given to the target article set is Single-Person and the specific expression representing the subject is "Koizumi", each article includes "Koizumi" that can be recognized as a personal name. Give a sentence a score. Or when the classification | category provided to the object article set is Multi-Organization, a score is given to the sentence containing an organization name in each article. Thus, by calculating the importance of each sentence in consideration of the score based on the classification, improvement in the accuracy of the generated summary can be expected.

  In addition, the summarizing means 201 calculates a similarity between one sentence and other sentences in a plurality of articles (by referring to the number of common words, etc.) and extracts a plurality of similar sentences. Of the extracted similar sentences, a sentence used for summary generation may be selected. In addition, it is preferable to change the process for selecting a sentence used for summary generation from a plurality of similar sentences according to the classification given to the target article set. For example, if the category assigned to the target article set is Single-class, there is a high possibility that similar sentences represent the same thing. Select only some of the sentences as representatives. In other words, similar sentences are: “An earthquake of seismic intensity 4 that occurred in Kyoto city, three people were lightly injured.” “An earthquake of seismic intensity 4 that occurred in Kyoto city, two people were newly hospitalized, When a plurality of sentences such as “the number of injured persons is five” is extracted, only one of these sentences is selected as an element of the summary article. Which of these sentences is selected can be determined by heuristics such as referring to the importance score of each sentence and selecting the latter sentence in the time series. On the other hand, if the category assigned to the target article set is Multi-class, it is likely that multiple similar sentences represent different things, so it is similar to a sentence with a high importance score. Select some or all of the sentences you want to collect. In other words, similar sentences are: “3 earthquakes with a seismic intensity of 4 that occurred in Kyoto, 3 people were injured.” “5 earthquakes with a seismic intensity of 5 that occurred in Osaka, 5 were hospitalized and 8 When multiple sentences such as “has been injured lightly,” these sentences are considered to describe different events, although they are similar in expression. Therefore, all of these sentences may be selected as elements of the summary article.

  Generally speaking, the summarizing means 201 of the document summarizing apparatus according to the present embodiment is a process of calculating the importance of each sentence by taking into account the importance score based on the classification assigned to the article set, and / or the article set. The process of changing the method of selecting a plurality of similar sentences according to the classification assigned to the can be configured as an addition to the existing multi-article summary software. The summarization process of different summarization algorithms can be executed according to the classification assigned to the target article set. By the way, the biographical description can be summarized and output from the article set with Single-Person classification, or the product name, function, price, etc. can be extracted from the article set with Multi-Product classification. It is also conceivable to output various summaries according to the classification of the article set to the summarizing means 201 such as outputting in the form of a table.

  According to the present embodiment, whether a subject of the document set is related to a single named entity (Named Entity) or a plurality of named entities is to be assigned to a document set that is a set of a plurality of documents. A determination unit 101 for determining which specific expression class the specific expression belongs to, and whether the specific expression related to the subject of the document set is independent based on the determination made by the determination unit 101 The document set classification device is configured to include a plurality of output units 102 that output information about classifications defined by two elements, namely, a specific expression class to which the specific expressions belong. A high classification can be assigned to a document set.

  The determination unit 101 uses as a material at least one of the frequency of the specific expression or the frequency of the specific expression class that appears in a plurality of documents included in the document set, and the specific expression related to the subject of the document set is independent. Depends on the characteristics of a specific language, such as the frequency of words beginning with capital letters and the frequency of personal pronouns such as he and she, in order to determine whether or not there is a plural and the specific expression class to which the specific expression belongs Classification can be performed without any. That is, the article set can be classified more generally.

  The determination unit 101 relates to a subject of the document set using at least one of a frequency of a cluster appearing in a plurality of documents included in the document set and a frequency of a specific expression class related to the cluster as a material. Since the specific expression class to which the specific expression belongs is determined, it is possible to assign an appropriate classification even to a document set that cannot be classified using only the specific expression as a clue.

  In addition, the determination unit 101 refers to information about the time point at which each of the plurality of documents included in the document set is created or published, and part or all of the plurality of documents is within a predetermined period. On the condition that the specific expression related to the subject of the article set is single and the specific expression class to which it belongs is an event name class. Quickly classify class document sets.

  The sentence set classification device extracts keywords existing in a given document, calculates the similarity between the keyword of one document and the keyword of another document, and if the similarity exceeds a threshold, As long as it further includes document set generation means 103 capable of generating at least one document set from a plurality of documents by assigning the documents to the same document set, the given plurality of documents are converted into one or a plurality of documents. Processing until sorting and assigning a classification to a set can be executed in a batch. This is useful for constructing a system that automatically generates one or more summaries based on a given document.

  The document set summarization apparatus further includes a summarizing unit 201 that refers to the classification given to the document set and selects a summarization algorithm corresponding to the classification to summarize a plurality of documents included in the document set. The document summarization apparatus can be configured to enable more appropriate summarization processing of a plurality of documents.

  The present invention is not limited to the embodiment described in detail above. In particular, the article set classification defined in the present invention, the document set classification apparatus for performing the classification, and the program thereof may be applied to applications other than automatic summarization. For example, in information retrieval, re-ranking after retrieval using an upper article in the retrieved result and efficient display of the retrieval result can be performed. Furthermore, it may be used for open domain information extraction. In the conventional information extraction, the domain is limited, and the subject and classification of the article are given as a premise. However, in order to extract information without limiting the domain, it is considered that it is necessary to dynamically classify target domain information, that is, article sets.

  The specific configuration of each part and the processing procedures shown in FIGS. 10 to 14 are not limited to the above embodiment, and various modifications can be made without departing from the spirit of the present invention. Of course, it is possible to configure a document set classification apparatus according to the present invention by installing a program in a personal computer or other general-purpose information processing apparatus, and it is not essential to manufacture a dedicated apparatus. .

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a configuration explanatory diagram of a document set classification apparatus according to the present invention. BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a configuration explanatory diagram of a document set classification apparatus according to the present invention. BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a configuration explanatory diagram of a document set classification apparatus according to the present invention. The figure which illustrates the article set containing a plurality of articles. The figure which shows the result of the article classification | category experiment by two test subjects. The figure which shows the result of the article classification | category experiment by two test subjects. The figure which shows the hardware resource with which the document set classification | category apparatus in one Embodiment of this invention is provided. The figure explaining the specific expression extraction process by a specific expression extraction means. The figure explaining the cluster extraction process by a cluster extraction means. The flowchart which shows the procedure of the judgment process which a judgment means performs. The flowchart. The flowchart. The flowchart. The flowchart. The figure which shows the result of the automatic classification | category experiment by a document set classification | category apparatus.

Explanation of symbols

1 Computer (document set classification device, document summarization device)
DESCRIPTION OF SYMBOLS 101 ... Judgment means 102 ... Output means 103 ... Document set production | generation means 201 ... Summarization means

Claims (12)

A classification is given to a document set that is a collection of a plurality of documents,
Determining means for determining whether the subject of the document set relates to a single specific expression or a plurality of specific expressions, and to determine which specific expression class the specific expression belongs to;
Based on the determination made by the determination means, a classification defined by two elements, that is, a single expression or a plurality of specific expressions related to the subject of the document set, and a specific expression class to which the specific expression belongs A document set classifying apparatus comprising: output means for outputting information about The determination means uses only at least one of the frequency of the specific expression or the frequency of the specific expression class appearing in a plurality of documents included in the document set, and the specific expression related to the subject of the document set is independent. 2. The document set classification apparatus according to claim 1, wherein determination of whether or not there is a plurality and determination of a specific expression class to which the specific expression belongs can be performed. The determination means uses, as a material, at least one of the frequency of a cluster that appears in a plurality of documents included in the document set and the frequency of a unique expression class to which the cluster is related, and the uniqueness related to the subject of the document set 3. The document set classification apparatus according to claim 1, wherein the specific expression class to which the expression belongs can be determined. The determination means refers to information about the time when each of the plurality of documents included in the document set is created or announced, and a part or all of the plurality of documents is created or created within a predetermined period. The document set according to claim 1, 2 or 3, wherein it is determined that the specific expression related to the subject of the article set is single and that the specific expression class to which the article belongs is an event name class on the condition that it is published. Classification device. The keywords existing in a given document are extracted, the similarity between the keyword of one document and the keyword of another document is calculated, and when the similarity exceeds a threshold, these documents are set to the same document set. 5. The document set classification device according to claim 1, further comprising a document set generation unit capable of generating at least one document set from a plurality of documents by assigning to the document set. It is used with the document set classification device according to claim 1, 2, 3, 4 or 5,
A summary for summarizing a plurality of documents included in the document set into a single document by referring to information about the classification assigned to the document set, which is output from the document set summarization apparatus, and using a summary algorithm corresponding to the classification. Document summarizing device comprising means. It is used for constituting the document set classification device according to claim 1, 2, 3, 4 or 5, and comprises at least a computer,
A determination means for determining whether a subject of a document set that is a set of a plurality of documents relates to a single specific expression or a plurality of specific expressions, and to determine to which specific expression class the specific expression belongs; as well as,
Based on the determination made by the determination means, a classification defined by two elements, that is, a single expression or a plurality of specific expressions related to the subject of the document set, and a specific expression class to which the specific expression belongs A program that functions as an output means for outputting information about the. The determination means uses only at least one of the frequency of the specific expression or the frequency of the specific expression class appearing in a plurality of documents included in the document set, and the specific expression related to the subject of the document set is independent. The program according to claim 7, wherein the determination of whether or not there is a plurality and the determination of a specific expression class to which the specific expression belongs is possible. The determination means uses, as a material, at least one of the frequency of a cluster that appears in a plurality of documents included in the document set and the frequency of a unique expression class to which the cluster is related, and the uniqueness related to the subject of the document set 9. The program according to claim 7 or 8, which can execute a determination of a specific expression class to which the expression belongs. The determination means refers to information about the time when each of the plurality of documents included in the document set is created or announced, and a part or all of the plurality of documents is created or created within a predetermined period. 10. The program according to claim 7, 8 or 9, wherein, on the condition that it is published, a determination is made that the unique expression related to the subject of the article set is single and the unique expression class to which the article belongs is an event name class. Further, the computer extracts keywords existing in a given document, calculates the similarity between the keyword of one document and the keyword of another document, and if these similarities exceed a threshold, The program according to claim 7, 8, 9, or 10, which also functions as a document set generation unit capable of generating at least one document set from a plurality of documents by assigning to the same document set. It is used for constituting the document summarizing device according to claim 6, and the computer is at least
As summarization means for referring to information about a classification given to a document set, which is output from the document set classification device, and summarizing a plurality of documents included in the document set into a single document by a summarization algorithm corresponding to the classification A program to function.

Images