JP4349734B2 - News topic component extraction and presentation device - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention also relates to a two-juice topic component extraction presentation equipment.
[0002]
[Summary of Invention]
The present invention relates to a news topic tracking device that performs tracking processing of, for example, a news topic every month extracted by a news topic extraction device (Japanese Patent Application No. 11-65658; unpublished), and a main component constituting the topic. The present invention relates to a news topic component extraction / presentation device that automatically extracts and presents elements, and a broadcast service method using the news topic component extraction / presentation device.
[0003]
The news topic tracking device can implement topic tracking with high accuracy by evaluating the similarity of topics in adjacent months. The news topic component extraction / presentation device can present a topic component that indicates when one topic has occurred and how it has changed in an easy-to-understand natural language sentence by performing language analysis of the tracking result. .
[0004]
Therefore, a broadcast service that presents the previous progress of broadcast news in text or video by using news topic component extracted by the news topic component extraction and presentation device as additional information during television news broadcast. realizable.
[0005]
[Prior art]
When watching a news program, it is difficult to remember everything about when, where, and how the topics covered in the news have occurred.
[0006]
To be able to show viewers when, where, and how a topic covered in a news program has occurred, it is first necessary to perform a news tracking process that extracts relevant topics. It is. Next, it is necessary to perform processing for obtaining related topic titles and components. Naturally, the title and the constituent elements are desired to be natural language sentences.
[0007]
[Problems to be solved by the invention]
The former tracking process has been performed in the past, and there is known a method of calculating whether a news manuscript is similar to a vocabulary used in which news manuscript in the past.
[0008]
However, since the conventional tracking method uses only the word information used in one news manuscript, even the news that deals with the topic about the same incident or phenomenon, the word or subject used over time Changes occur and tracking accuracy is reduced.
[0009]
On the other hand, with regard to the process of obtaining the latter title and components, news topics change in time series, and it is still difficult to grasp when and what is happening from a large amount of news text data. Not proposed. Therefore, how to extract the components of news topics and generate them in natural language sentences becomes a problem.
[0011]
The present invention has been made in view of such circumstances, and a news topic constituent element extraction and presentation device capable of extracting change points of news topic constituent elements and presenting the extracted topic constituent elements in easy-to-understand natural language sentences. It is intended to provide.
[0015]
[Means for Solving the Problems]
In order to achieve the above object, the news topic component extraction / presentation device according to the present invention provides the news related to the news specified by the viewer from a file device that electronically stores a plurality of news articles that change in time series. An extraction means for extracting an article; a calculation means for calculating a difference between a feature vector characterizing the news article extracted by the extraction means; and an average vector indicating an average of the feature vectors; and calculated by the calculation means When the differences are arranged in time series, a change point extraction unit is provided that extracts a minimum value of the difference as a change point of a news topic .
[0016]
According to this configuration, the change point of the topic content can be extracted using the information on the change in the appearance of the word.
[0017]
Further, the calculation means sets the “element” of the feature vector to “word included in a news article”, and the “element value” of the feature vector to “(appearance of a word in a news manuscript (other than a specific pattern portion)” Number) × log2 (number of news manuscripts constituting the topic / news manuscript in which words appear (other than the specific pattern portion)) ”.
Further, the news topic component extraction / presentation device of the present invention, when the change points extracted by the change point extraction means are arranged in time series, the news article between the two change points is a topic component It is characterized by comprising a component extraction means for extracting as follows.
[0018]
According to this structure, between each change point can be extracted as a topic component.
[0019]
Also, the component extracting means, the topic extraction and the calculated word contribution included in news articles were as components, the degree of contribution and Turkey to generate a natural language text using the maximum word It is a feature.
[0020]
According to this configuration, it is possible to generate and present a noun phrase that accurately represents the content of the topic component.
[0023]
Therefore, according to the present invention, it is possible to construct a system capable of presenting to the viewer when and where a topic taken up in a news program has occurred and how it has followed.
[0024]
DETAILED DESCRIPTION OF THE INVENTION
FIG. 1 is an overall configuration diagram of a system that is an embodiment of a news topic tracking device and a news topic component extraction / presentation device according to the present invention.
[0025]
In FIG. 1, a daily news manuscript actually used in a news program includes about 200 articles. In the file device 1, for example, ten years of news manuscripts that change in time series are stored in an electronic form.
[0026]
The news topic extraction device 2 relates to an earlier application of the present applicant (Japanese Patent Application No. 11-65658: unpublished), and as shown in FIG. 2, a morphological analysis unit 21, a syntax analysis unit 22, A word importance calculation unit 23, a similarity comparison unit 24, and a news article classification unit 25 are provided. A clustering process is performed on a news manuscript stored in the file apparatus 1, and similar ones appearing in one month Extract a mass of news topics. The extracted monthly news topics are output to the news topic tracking device 3.
[0027]
The news topic tracking device 3 specifies which month the news topic of the monthly news topic extracted by the news topic extraction device 2 belongs to the news specified by the viewer of the television 4, and the identification The topics related to the news specified by the viewer are collected by evaluating the similarity between the news month and the news topic of the adjacent month, and associating the similar topics, and output them to the news topic component extraction and presentation device 5 .
[0028]
The news topic component extraction / presentation device 5 includes a news topic change point extraction device 51 and a news topic component extraction device 52.
[0029]
The news topic change point extraction device 51 extracts a time when the content of the news topic that has appeared over a long period of time extracted by the news topic tracking device 3 has changed.
[0030]
The news topic component extraction device 52 receives a news topic that has been extracted over a long period of time extracted by the news topic tracking device 3 and a topic change point extracted by the news topic change point extraction device 51, and changes one topic change point. Recognize what kind of content was included in the news in between, and present the topic title and topic components in easy-to-understand natural language sentences.
[0031]
<< Operation Explanation of News Topic Extractor 2 >>
First, the operation of the news topic extraction device 2 will be described with reference to the operation flowchart of the article classification process shown in FIG.
[0032]
One news article read from the file device 1 is subjected to well-known morphological analysis processing and syntax analysis processing in the morphological analysis unit 21 and the syntax analysis unit 22, and a word extracted from the article is a word importance calculation unit 23. Is input. In the word importance calculation unit 23, the similarity comparison unit 24, and the news article classification unit 25, news article classification processing is performed according to the procedure shown in FIG.
[0033]
When a word included in one news article is input (step ST1), the word importance calculating unit 23 calculates, for example, monthly importance of the word included in the article using the χ ² value (step ST1). ST2). Specifically, assuming that the frequency of occurrence of the word W in a certain month is n and the expected value is e, they are substituted into the equation (1) to calculate the word importance Weight (W).
[0034]
[Expression 1]
Weight (w) = (n−e) ² / e... N ≧ e
= 0 n <e (1)
[0035]
Next, the similarity comparison unit 24 and the news article classification unit 25 perform clustering for generating clusters that are collections of articles classified into similar items, using the importance and appearance frequency of words. First, the similarity between an article and a cluster is calculated using the article vector and cluster vector defined as follows (step ST3).
[0036]
The article vector characterizing the article defines a word contained in the article (for example, a word contained in the first sentence of the article) as a vector element and the importance of the word as a value of each vector element. In addition, the cluster vector characterizing the cluster is the word included in the article belonging to the cluster as the vector element, and the product of (importance of each word) and (appearance rate in the cluster) as the value of each vector element Define. Note that the appearance rate is a value obtained by dividing the frequency of appearing articles by the total number of articles included in the cluster. Then, the similarity is obtained by the equation (2).
[0037]
[Expression 2]
Similarity = (sum of common element vectors × 2)
/ (Sum of article and cluster element vector values) (2)
[0038]
In this way, the similarity between the article and the cluster is evaluated (step ST4). If the similarity with the cluster is equal to or higher than a certain threshold (for example, 0.5), the cluster evaluated as the most similar is determined. Integration is performed (step ST5). If the similarity with the cluster is equal to or less than the threshold value, the same evaluation is performed for all clusters (step ST4 → step ST6 → step ST3 → step ST4). As a result, if the similarity with all the clusters is equal to or less than the threshold, a new cluster is constructed with the article (step ST7). By repeating the above processing (step ST8), a cluster having a high degree of similarity is obtained. That is, articles included in the news manuscript are classified with high accuracy.
[0039]
The monthly news topics obtained in this way are output to the news tracking device 3. In addition, although a result changes also with the order of comparison, in the process demonstrated above, the comparison is performed in order of date.
[0040]
<< Operation Explanation of News Topic Tracking Device 3 and News Topic Component Extraction and Presentation Device 5 >>
First, the operation of the news topic tracking device 3 will be described with reference to the flowchart of FIG.
[0041]
In FIG. 4, among the news topics of each month extracted by the clustering process by the news topic extraction device 2, the topic to which the news designated by the viewer of the television 4 belongs is specified (step ST21).
[0042]
Next, for the topic to which the news specified by the user belongs, the similarity to all the topics in the previous month is evaluated, and when the similarity is more than a certain level (for example, 0.2 or more), the same topic is identified. Are associated (steps ST22 to ST24).
[0043]
Here, the similarity used in the similarity comparison process (steps ST22 to ST24) with the topic of the previous month is assumed to be expressed by the following equation (3), for example. The definition formula (3) uses the topic vector because the topic extracted by the news topic extraction device 2 is expressed in vector.
[0044]
[Equation 3]
Similarity = (sum of common element values)
÷ {(sum of element values of two topic vectors) − (sum of values of common elements)} (3)
[0045]
Repeat the similarity comparison process (steps ST22 to ST24) with the topic of the previous month, perform the similarity comparison process with the topic of the previous month, from all the topics of the previous month associated, Association with the topic specified by the viewer is performed.
[0046]
Then, when the similarity comparison processing for all the topics in the previous month is completed and there is no topic in the processing target month to be compared with the topic in the previous month (step ST22: Yes), all of the associated topics The topic is presented as a set of topics related to the news designated by the viewer (step ST25).
[0047]
As described above, since the tracking process targets a group of similar originals collected by the clustering process, the amount of information can be larger than that of one news original, and accurate tracking can be performed.
[0048]
Next, the operation of the news topic component extraction / presentation device 5 will be described with reference to FIGS. FIG. 5 is a flowchart showing a processing procedure of the news topic change point extraction device 51. FIG. 6 is a flowchart showing the procedure of the specific pattern description extraction process executed by the news topic change point extraction device 51. FIG. 7 is an example of news topic change point extraction results. FIG. 8 is a flowchart showing a processing procedure of the news topic component extraction device 52. FIG. 9 is an example of a topic component automatic extraction result.
[0049]
The news topic change point extraction process is performed as follows. In FIG. 5, a news manuscript set constituting the topic to which the news designated by the viewer belongs is acquired from the news topic tracking device 3 (step ST31), and the specific pattern performed in the title extraction process in the news title extraction device (not shown). The description is extracted and excluded from the processing target (step ST32). This is because in the news topic change point extraction process, the specific pattern description that explains the entire topic is determined to be noise.
[0050]
Here, the news topic change point extraction device 51 first executes a specific pattern description extraction process as shown in the flowchart of FIG. That is, in FIG. 6, a news manuscript belonging to a news topic set extracted by the news topic extracting device 2 is extracted from the file device 1, and a specific pattern of the extracted news manuscript is extracted (step ST61). Here, the following specific pattern is used.
[0051]
<< Specific pattern (regular expression) >>
1. ^{∧. *} , (. ^* Incident) [Death]. ^*
2. ^∧ (. ^* Incident) [Death]. ^*
3. ^{∧. *} , (. ^* Accident) [Death]. ^*
4). ^∧ (. ^* Accident) [ ^Death ]. ^*
5. ^{∧. *} , (. ^* Problem) [ ^Death ]. ^*
6). ^∧ (. ^* Problem) [ ^Death ]. ^*
7). ^{∧. * And} (. ^* ). ^*
8). ^{About に つ い て} (. ^* ). ^*
9. ^{∧. *} , (. ^* ) Related. ^*
10. ^{関 連} Related to (. ^* ). ^*
11. ^∧. Regarding ^* and (. ^* ). ^*
12 ^∧ Regarding (. ^* ). ^*
[0052]
Returning to the processing of FIG. 5, the news manuscript from which the specific pattern description portion extracted as described above is removed is expressed by a feature vector (step ST33). In the feature vector, “element” is a word included in a news manuscript (other than a specific pattern portion), and “element value” is expressed by the following equation (6).
[0053]
[Formula 6]
Element value = (Number of occurrences of word Wi in the news manuscript (other than the specific pattern part))
× log2 (number of news manuscripts composing the topic /
News manuscript with the word Wi (except for the specific pattern)) ... (6)
[0054]
Next, an average vector of news original feature vectors (hereinafter referred to as “original vector”) is calculated, and a difference between each original vector and the average vector is calculated (step ST34). Here, for example, for elements whose average vector element value is larger than the document vector element value, the difference is zero, and only those whose document vector element value is large are the difference. This difference is arranged in the order of appearance of the original vector (in order of the date and time when the original was created), and the change point (for example, the minimum value) of the topic is extracted according to the change in value and output (steps ST35 and ST36).
[0055]
For example, FIG. 7 is an example of a news topic change point extraction result of “a case in which seven people including four Japanese mining engineers are abducted by armed forces in Kyrgyz in Central Asia”. In FIG. 7, the horizontal axis represents the document number, and the vertical axis represents the difference value.
[0056]
As shown in FIG. 7, eight dotted lines are minimum values, and are extracted as topic change points. Between the change points is a component of the topic. Therefore, in the news topic change point extraction result example of FIG. 7, it is shown that there are nine topic components. This change point can be extracted in the same manner by changing the number of appearances for each news period.
[0057]
Next, the news topic component extraction process is performed as follows. In FIG. 8, a news manuscript set constituting a topic related to news designated by the viewer is acquired from the news topic tracking device 3, and a topic change point is acquired from the news topic change point extraction device 51 (step ST41). ) Analyzing the topic composition manuscript to extract a specific pattern and a fixed pattern (step ST42). And two noun phrases are produced | generated by the process of step ST43-ST48, and the process of step ST49.
[0058]
That is, in step ST42, the above-described specific pattern description extraction process (see FIG. 6) is performed, and the extracted specific pattern description is excluded from the processing target. This is because, in the news topic component extraction process, the specific pattern explaining the entire topic becomes noise. Focusing on the end-of-sentence expression, “clarify”, “show the idea”, “understand”, “describe”, and “speak” are extracted as fixed pattern descriptions.
[0059]
With respect to the extracted fixed pattern description, the fixed expression is converted according to the conversion rule described later, and a noun phrase explaining the change point is generated (step ST49). On the other hand, in the generation of a noun phrase from a part other than the specific pattern description, first, the contribution degree of the word included in the original between the change points is calculated (step ST43). The contribution of words is defined by the following equation (7).
[0060]
[Expression 7]
Word contribution = (Number of occurrences of word Wi in news manuscript (other than specific pattern part))
× log2 (number of news manuscripts composing the topic /
News manuscript with the word Wi (except for the specific pattern part)
× (number of originals between change points where word Wi appeared) / (total number of originals between change points) (7)
[0061]
Then, a manuscript having a maximum contribution of a word included in the original between the change points is extracted as a representative manuscript between the change points (step ST44), and a sub-variable having a maximum contribution is extracted from the representative manuscript (step ST45). ). At this time, if there is no sub-variable, the noun has the largest contribution. Next, all the words related to the sub-variable are extracted by syntax analysis, and noun phrases are generated using the following conversion rules (steps ST46 and ST47).
[0062]
(Conversion rules)
“Subject” is changed from “to” to “to” and “changed” to “subject” and “to” to “to”.
“Subject” is “Change” → “Subject” of “Subject”
“Subject” changes from “to” to “to” and “changes” to “subject” and “to” to “to”.
“Subject” is “Change” → “Subject” is “Change”
[0063]
On the other hand, when a typical expression is present in the representative manuscript extracted in step ST44 (step ST48: Yes), the following fixed pattern description conversion rule is applied, and a noun that explains the change point from the typical expression. A phrase is generated (step ST49).
[0064]
(Standard pattern description conversion rules)
“Subject” “clarifies” → “subject” “statement”
“Subject” is “Shows an idea” → “Subject” is “Representation”
“Subject” is “understood” → “subject” is “found”
“Subject” is “to describe” → “subject” “interview”
“Subject” is “Speak” → “Subject” “Interview”
(Note that if words such as “conference” and “statement” are used in the manuscript, they will be converted with priority.)
[0065]
Finally, the noun phrase generated from the standard expression and the noun phrase generated from the part other than the specific pattern description are combined with a colon (:) (step ST50). Thereby, a noun phrase representing a topic title and a component corresponding to a change point of the topic is obtained.
[0066]
FIG. 9 shows a result of automatic extraction of news topic components from “an incident in which seven people including four Japanese mining engineers are abducted by armed groups in Kyrgyz in Central Asia” shown in FIG. That is, in FIG. 9, the numbers correspond to the areas shown in FIG. The underlined portion is a portion generated by a fixed expression.
[0067]
Next, FIG. 10 is a diagram illustrating an application example of the news topic component extraction / presentation device 5. In FIG. 10, at the time of television news broadcast, the news topic component extracted by the news topic component extraction and presentation device 5 is used as additional information, so that the previous progress of the broadcast news is presented in text or video. The service model is shown.
[0068]
FIG. 10 (1): While the viewer is watching today's news, he / she is concerned about a certain news item and presses the topic content on the remote control.
[0069]
Then, the history of the selected news so far is presented on the television screen. A viewer selects one news item.
[0070]
FIG. 10 (3); As a result, the viewer can view the selected past news.
[0071]
As described above, in the embodiment described above, when the topic that appears for a long time is recognized as one topic, the similarity of the news topic every month is evaluated. Needless to say, it may be a predetermined cycle such as a unit or a unit of several months.
[0072]
【The invention's effect】
As described above, according to the present invention, it is possible to automatically recognize a topic divided by the current technology as a series by performing tracking processing on a topic that continues for a long time. . Then, by analyzing the series of topics by linguistic analysis, it is possible to extract topic change points and present the constituent elements of the topics between the extracted change points in a natural language. As a result, it is possible to construct a system that can present a history of news desired by the viewer.
[Brief description of the drawings]
FIG. 1 is an overall configuration diagram of a system that is an embodiment of a news topic tracking device and a news topic component extraction / presentation device according to the present invention.
FIG. 2 is a configuration block diagram of a news topic extraction device.
FIG. 3 is a flowchart of an article classification processing operation.
FIG. 4 is a flowchart showing a processing procedure of the news topic tracking device.
FIG. 5 is a flowchart showing a processing procedure of the news topic change point extraction apparatus.
FIG. 6 is a flowchart showing a procedure of specific pattern description extraction processing executed by the news topic change point extraction device.
FIG. 7 is an example of a news topic change point extraction result.
FIG. 8 is a flowchart showing a processing procedure of the news topic component extraction apparatus.
FIG. 9 is an example of a topic component automatic extraction result;
FIG. 10 is a diagram showing an application example of a news topic component extraction / presentation device (a service model that presents previous progress of broadcast news in text or video).
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 1 File apparatus 2 News topic extraction apparatus 3 News topic tracking apparatus 4 Television 5 News topic component extraction presentation apparatus 51 News topic change point extraction apparatus 52 News topic component extraction apparatus

Claims (4)

An extraction means for extracting the news articles related to the news designated by the viewer from a file device that electronically stores a plurality of news articles that change in time series, and
Calculating means for calculating a difference between a feature vector characterizing the news article extracted by the extracting means and an average vector indicating an average of the feature vectors;
When the differences calculated by the calculation means are arranged in time series, a change point extraction means for extracting the minimum value of the difference as a change point of a news topic;
A news topic component extraction and presentation device characterized by comprising: In the news topic constituent element extraction and presentation device according to claim 1,
  The calculation means sets the “element” of the feature vector to “words included in a news article”, and sets the “element value” of the feature vector to “(number of occurrences of words in a news manuscript (other than a specific pattern portion))” × log2 (number of news manuscripts composing a topic / news manuscript in which a word appears (other than a specific pattern part)) ”. In the news topic constituent element extraction and presentation device according to claim 1 ,
When the change points extracted by the change point extraction means are arranged in time series, the news article that is between the two change points is extracted as a topic constituent element. News topic component extraction and presentation device. In the news topic constituent element extraction and presentation device according to claim 3 ,
The constituent element extraction means calculates a contribution degree of a word included in the news article extracted as the topic constituent element, and generates a natural language sentence using a word having the maximum contribution degree. Topic component extraction and presentation device.

Images