JP5528376B2 - Document simplifying apparatus and program - Google Patents

Description

  The present invention relates to natural language processing. In particular, the present invention relates to a document simplification device that automatically simplifies an inputted sentence, and a computer program thereof.

There are cases where it is required to automatically change the expression of a sentence without changing the meaning of the sentence written in a natural language. For example, a system technology that automatically converts difficult sentences into plain sentences has been proposed.
For example, Patent Document 1 describes a technique for obtaining a combination of rules for converting from a dependency state between a predicate or a behavioral noun and another word to a case relationship based on a Japanese parsing result. .

JP 2010-003049 A

  In the prior art, when converting a natural language sentence, only conversion between parts of speech of the same kind was possible, such as converting a noun into a noun or converting a predicate into a predicate. However, for more flexible and broad paraphrasing, it is desirable to be able to automatically convert between different parts of speech. In particular, automatically converting nouns such as behavioral nouns and nouns derived from adjectives to predicates is effective in expanding the range of paraphrasing.

  Conventionally, a system for converting nouns into verbs and other predicates has not been realized. In order to realize this, it is necessary to solve the problem that the grammar in the dependency relations around these words is broken only by simply converting the nouns into predicates (verbs, adjectives, etc.). In other words, it is necessary to convert the modifiers related to the nouns and predicates to be converted in an appropriate form.

  The present invention has been made on the basis of the above problem recognition, and by performing word conversion between different parts of speech from nouns to predicates while also considering modifiers that modify the words to be converted. A document simplification apparatus and a program for simplifying a sentence are provided.

[1] In order to solve the above-described problem, a document simplification apparatus according to an aspect of the present invention stores a simplification rule table that stores simplification rule data in which noun data is associated with prescriptive data corresponding to the noun data. A storage section, a modifier phrase conversion rule table storage section that holds modifier phrase conversion rule data that associates the modifier modifier data with the consecutive modifier phrases data corresponding to the linkage modifier phrase data, the phrase data, and the phrase Co-occurrence relationship between the case modifier table that modifies the data, the case frame table storage unit that stores the predicate data and the appearance frequency data related to the clause modifier data in association with each other, the independent word data, and the independent word data stored as nominal conversion rule data in combination with the selected noun data and particle data is in association with nominal data based on a nominal conversion rule table Read the dependency analysis result data corresponding to the sentence data and the storage unit, and select the applicable simplification rule data from the simplification rule table storage unit based on the noun data included in the dependency analysis result data The simplification rule selection unit, and the combination modification phrase data that modifies the noun data to which the simplification rule data selected by the simplification rule selection unit can be applied are extracted from the dependency analysis result data and extracted. Based on the link modifier phrase data, a modifier phrase conversion rule selection unit that selects the corresponding modifier phrase conversion rule data from the modifier phrase conversion rule table storage unit, and the simplification selected by the simplification rule selection unit Included in the prescriptive data included in the rule data and the modifier phrase conversion rule data selected by the modifier phrase conversion rule selector The appearance frequency data is read out from the case frame table storage unit based on the continuous modification phrase data, and the dependency analysis result data in the dependency analysis result data is selected based on the read appearance frequency data. A case frame matching unit that replaces the linkage modifier data and replaces the noun data in the dependency analysis result data by the selected predicate data, and the noun data before replacement by the case frame matching unit Is extracted from the dependency analysis result data, and based on the extracted independent word data and the particle data, the utterance conversion rule table is used. Select the speech conversion rule data from the storage unit, based on the selected speech conversion rule data, A syntactic conversion rule selecting section that embodies the prescriptive data after replacement by the case frame matching section.

With this configuration, based on the dependency analysis result data of the input sentence, the simplification rule selection unit selects a rule for converting a noun into a predicate, and the modifier phrase conversion rule selection unit modifies the noun. Select the rule (s) that will replace the phrase with the combined modifier, and the case frame matching unit will select a candidate based on the frequency of the actual sentence. The rule selection unit adapts the converted grammar to make the grammar consistent. In other words, the document simplification apparatus having this configuration is grammatically correct, has the same naturalness as many sentences used in reality, and generates plain sentences to simplify the input sentence. it can.
One aspect of the above-described modifier phrase conversion rule table holds a single or plural consecutive modifier phrases corresponding to each type of the modifier modifier phrase.
One aspect of the above-mentioned body data is a combination of a noun and a particle. This noun + particle relates to the above independent word.
In addition to the above configuration, a morphological analysis / dependence analysis unit that reads sentence data, performs morphological analysis processing and dependency analysis processing of the sentence data, and outputs dependency analysis result data corresponding to the sentence data. You may make it provide further.

A program according to an aspect of the present invention causes a computer to function as the document simplification apparatus according to claim 1.

According to the present invention, it is possible to automatically convert a noun in a document into a plain expression. In addition, since each frame is collated using the case frame table and conversion is performed using the result, it is possible to paraphrase into a plain expression in a more natural form considering the context.
Further, according to the present invention, the simplification rule can be automatically created without requiring manual work by the configuration provided with the simplification rule table creation device.
Furthermore, according to the present invention, since the simplification rule table creation device performs context similarity recognition based on the context similarity table, it can be paraphrased into plain expression in a more natural form.

It is a block diagram which shows the function structure of the document simplification apparatus by the 1st Embodiment of this invention. It is the schematic which shows the structure and example of data of the simplification rule table in the embodiment. It is the schematic which shows the structure and data example of a modifier phrase conversion rule table in the embodiment. It is the schematic which shows the structure and data example of a domain dependence case frame table in the embodiment. It is the schematic which shows the structure and example of a data of the body language conversion rule table in the embodiment. It is a flowchart which shows the procedure of the process which the document simplification apparatus by the same embodiment simplifies an input sentence. It is a block diagram which shows the function structure of the document simplification apparatus by the 2nd Embodiment of this invention. It is a block diagram which shows the detailed functional structure of the simplification rule table creation apparatus in the embodiment. It is the schematic which shows the structure and data example of a synonym dictionary table in the embodiment. It is the schematic which shows the structure and data example of the dictionary table in the embodiment. It is the schematic which shows the structure and data example of a thesaurus table in the embodiment. It is the schematic which shows the structural example of the data used as the origin for producing said thesaurus table. It is the schematic which shows the structure and data example of a word difficulty level table in the embodiment. It is the schematic which shows the structure and example of data of a context similarity table in the embodiment. It is a flowchart which shows the procedure of the process which the simplification rule table creation apparatus by the same embodiment produces | generates a simplification rule based on dictionary data etc.

Next, an embodiment of the present invention will be described with reference to the drawings.
[First Embodiment]
FIG. 1 is a block diagram showing a functional configuration of the document simplification apparatus according to the first embodiment. As illustrated, the document simplification apparatus 1 includes an input sentence storage unit 10, a morphological analysis / dependence analysis unit 20, a simplification rule selection unit 30, a modifier phrase conversion rule selection unit 40, and a case frame collation unit. 50, a noun conversion rule selection unit 60, an output sentence storage unit 70, a simplification rule table storage unit 130, a modifier phrase conversion rule table storage unit 140, a domain-dependent case frame table storage unit 150, and a noun conversion rule. And a table storage unit 160.
Each of these units is realized using an electronic circuit. Each storage unit for storing data is realized using a semiconductor memory or a hard disk device.

The input sentence storage unit 10 stores a sentence to be input to the document simplification apparatus 1. The input sentence storage unit 10 may store data of a group of documents composed of a plurality of sentences.
The morpheme analysis / dependence analysis unit 20 reads an input sentence from the input sentence storage unit 10 and performs a morpheme analysis process and a dependency analysis process. Existing technology is used for the morphological analysis and dependency analysis processing itself. For example, a morpheme analyzer “MeCab” can be used for the morpheme analysis process. Further, for example, a dependency analyzer “CaboCha” can be used for dependency analysis processing. Then, the morphological analysis / dependence analysis unit 20 outputs dependency analysis result data.

The simplification rule selection unit 30 reads the dependency analysis result data corresponding to the input sentence data, and converts it into noun data (behavioral nouns, nouns derived from adjectives and adjective verbs) included in the dependency analysis result data. Based on this, applicable simplification rule data is selected from the simplification rule table storage unit 130.
The modifier phrase conversion rule selection unit 40 extracts, from the dependency analysis result data, linkage modifier phrase data that modifies the noun data to which the simplification rule data selected by the simplification rule selection unit 30 can be applied. Then, the corresponding modifier phrase conversion rule data is selected from the modifier phrase conversion rule table storage unit 140 based on the extracted combined modifier phrase data.

  The case frame collation unit 50 includes the predicate data included in the simplification rule data selected by the simplification rule selection unit 30 and the continuous modification included in the modifier phrase conversion rule data selected by the modifier phrase conversion rule selection unit 40. Based on the phrase data, the domain-dependent case frame table storage unit 150 is read. Then, based on the appearance frequency data read from the domain-dependent case frame table storage unit 150, the linkage modifier data in the dependency analysis result data is replaced by the selected linkage modifier data, and the selected statement is used. The noun data in the dependency analysis result data is replaced with the data. At this time, the case frame matching unit 50 selects the candidate having the highest appearance frequency from among a plurality of candidates and performs the above replacement.

The noun conversion rule selection unit 60 extracts the independent word data related to the noun data before being replaced by the case frame matching unit 50 from the dependency analysis result data. Then, based on the extracted independent word data, the word conversion rule data is selected from the word conversion rule table storage unit 160, and after being replaced by the case frame matching unit 50 based on the selected word conversion rule data. Make a prescriptive data. In other words, the noun conversion rule selection unit 60 refers to the noun conversion rule table 160 to apply the noun necessary for the nounization from the particle attached to the noun and the independent word modified by the noun, Convert a predicate to a noun. Then, the syntactic conversion rule selection unit 60 writes the simplified sentence in the output sentence storage unit 70. In other words, the syntactic conversion rule selection unit 60 outputs a simplified sentence.
The output sentence storage unit 70 stores the simplified output sentence.

The simplification rule table storage unit 130 stores a large number of simplification rule data for converting a noun in a sentence into a plain predicate.
The modifier phrase conversion rule table storage unit 140 stores modifier phrase conversion rule data that associates the associated modifier phrase data with the continuous modifier phrase data corresponding to the associated modifier phrase data.
The domain-dependent case frame table storage unit 150 stores prescriptive data, continuous modifier data that modifies the prescriptive data, and appearance frequency data related to the prescriptive data and the joint modifier data. The domain-dependent case frame table storage unit 150 holds the above-described appearance frequency data based on the result of statistically processing a sentence set belonging to a specific domain. A domain corresponds to a classification of sentence content such as news, sports or music. It should be noted that a plurality of different tables may be provided for each of a plurality of domains.
The noun conversion rule table storage unit 160 associates and stores the independent word data and the verbalization data related to the independent word data as the notional conversion rule data. The verbalization data is data for verbalizing the predicate. The verbalization data is data of combinations of nouns and particles. The noun conversion rule table storage unit 160 holds the most appropriate nounization data (noun + particle) for a certain independent word. The most appropriate body data is, for example, an expression in which a noun + a particle relates to an independent word frequently appears in a general sentence.
Details of the data in each table will be described next.

  FIG. 2 is a schematic diagram illustrating the configuration and data example of the simplification rule table. As shown in the figure, the simplification rule table is tabular data, and includes items of an expression before simplification, a part of speech before simplification, an expression after simplification, and a part of speech after simplification. Each row of data in this table represents the relationship between the expression before simplification and the expression after simplification. In the example shown in the figure, in the first line data, the part-of-speech of the expression “depressed” before simplification is a noun (a verb associative noun formed by the conjunctive form of the verb “depressed”), and the corresponding post-simplification expression is It represents the verb “down”. The part of speech after simplification is not limited to a verb, but may be an adjective or an adjective verb. In this way, the simplification rule table holds rules in the form of <noun> → <use>.

  FIG. 3 is a schematic diagram illustrating a configuration of a modifier phrase conversion rule table and a data example. As shown in the figure, the modifier phrase conversion rule table is tabular data and includes items before substitution (continuous modifier phrase) and after substitution (continuous modifier phrase). The item “before replacement” represents the pattern of the continuous modifier phrase before conversion, and the item “after replacement” represents the pattern of the continuous modifier phrase after conversion. In the illustrated example, the data on the first line represents a rule for converting a combination modifier before replacement having a pattern of “related to” to a replacement modifier after replacement having a pattern of “about”. Yes. Further, the data on the fifth line is a rule for converting a combination modifier before replacement having a pattern of “˜˜” into a continuous modifier after substitution having a pattern of “˜˜ ////”. Represents. In addition, the expression "~~ is ////" means that the pattern after replacement is any of "~~", "~~", "~~", or "~~". It represents getting.

  FIG. 4 is a schematic diagram showing the configuration of the domain-dependent case frame table and data examples. As shown in the figure, the domain-dependent case frame table is tabular data, and includes items of noun, particle, precaution, and appearance frequency. One row of data in this table represents one case frame and its appearance frequency value. This data is prepared in advance by parsing a large number of documents, extracting the expression of the pattern of “noun-particle”, and counting the appearance frequency. A large amount of documents can be acquired from, for example, closed caption data of a broadcast program such as a television set, collected from server devices of a large number of websites via the Internet, or acquired from a large number of electronic books. At this time, only documents belonging to a specific domain are collected, a domain-dependent case frame table is created, and applied to the input sentence of the domain. Specific examples of domains are “general news”, “sports”, “entertainment”, and the like. By using such a table, it is possible to perform replacement with natural expressions that are likely to appear in a general document based on the appearance frequency in a specific domain in the processing described later. In the illustrated example, the data in the fourth row indicates that the appearance frequency of the case frame “demand (noun) / ga (particle) / decrease (property)” is 6.

  FIG. 5 is a schematic diagram illustrating a configuration of the syntactic conversion rule table and a data example. As shown in the figure, the syntactic conversion rule table is tabular data, and has nouns, particles, and independent words. One row of data in this table represents a dialect conversion rule. This dialect conversion rule is a rule for applying the noun necessary for the dialectization according to the combination of the particle attached to the noun in the original sentence and the independent word modified by the noun.

  The data of the syntactic conversion rule is created in advance by taking statistics on the co-occurrence relationship of nouns, particles and independent words in a sentence set. For example, the most frequent combination is based on the frequency of appearance of noun data that is expressed in conjunction with verb conjugations extracted from sentences belonging to a sentence set and independent word data that this noun data modifies with particle data. , And stored as body language conversion rule data. For example, in the case of the fourth line shown in the figure, in the co-occurrence relationship between the particle data “ga” and the combination of the independent word data “large”, the most frequent expression is the noun data “degree”. It means that there was.

  For the sake of convenience, the noun data here is referred to as manifestation data. The syntactic conversion rule table associates the independent word data with the nouns (participation data) and particles related to the independent word data and stores them as the syntactic conversion rule data. In other words, the body language conversion rule table holds the noun data selected based on the co-occurrence relationship between the particle data and the combination of the independent word data as the body data. A specific method of applying the body language conversion rule will be described later using an actual example.

Next, a specific processing procedure for simplifying a document will be described based on an example of processing data.
FIG. 6 is a flowchart showing the processing procedure of the document simplification apparatus 1. Hereinafter, it demonstrates along this flowchart. It should be noted that the input sentence storage unit 10 stores in advance a sentence “a demand drop is large” inputted from the outside, and this sentence becomes a target of simplification.

  First, in step S1, the morphological analysis / dependence analysis unit 20 reads an input sentence from the input sentence storage unit 10, and performs a morpheme analysis process and a dependency analysis process. In this example, as a result of the morphological analysis processing, data “demand (noun)” / “no (particle)” / “depression (noun)” / “ga (particle)” / “large (adjective)” is obtained from the input sentence. Is obtained. Here, “/” represents a break between morphemes. For example, the expression “demand (noun)” indicates that the part of speech of the morpheme “demand” is a noun. As a result of the dependency analysis process, the morphological analysis / dependence analysis unit 20 determines that “demand (noun)” / “no (particle)” → “depression (noun)” / “ga (particle)” → “large ( Adjective) ”is output. Here, “→” (right arrow) represents a dependency relationship. For example, “demand” / “no” is related to “depression” / “ga”. The result of dependency analysis can be expressed in a tree structure. In the above example, “demand (noun)” / “no (particle)” and “depression (noun)” / “ga (particle)” and “large ( “Adjectives” ”correspond to the nodes, and“ → ”(right arrow) connecting them corresponds to the edges. The data of the dependency analysis result is expressed by a tree structure or equivalent data. In the case of a Japanese sentence, the rightmost node is the root node in the tree structure of the dependency analysis result.

  Next, in step S2, the simplification rule application unit 30 applies the simplification rule read from the simplification rule table storage unit 130 to the dependency analysis result. Specifically, the simplification rule applying unit 30 performs processing for matching the expression included in the dependency analysis result with the expression before simplification in the simplification rule table. In this example, among the simplification rules illustrated in FIG. 2, the expression “depression (noun)” before simplification matches “depression (noun)” in the dependency analysis result. In other words, a rule for converting the expression “down (noun)” before simplification into the expression “down (verb)” after simplification can be applied to the input sentence. When this rule is applied, an expression (dependency analysis result data) of “demand (noun) / no (particle)” → “lower (verb) / ga (particle)” → “large (adjective)” is obtained. By simply applying the simplification rules, the nouns are replaced by the nouns, so the syntax of the sentence is temporarily broken, but the syntax is normalized in later processing.

  Next, in step S3, the modifier phrase conversion rule selection unit 40 applies the modifier phrase conversion rule read from the modifier phrase conversion rule table storage unit 140 to the dependency analysis result. Specifically, the modifier phrase conversion rule selection unit 40 matches the pattern “demand / no” of the modifier phrase included in the dependency analysis result with the pattern before substitution (combined modifier phrase) in the modifier phrase conversion rule table. To perform the process. In the present example, in the modifier phrase conversion rule shown in FIG. 3, the pattern “˜˜” on the fifth line matches “demand /” obtained from the input sentence side. And, since the pattern after substitution (continuous modifier) in the selected conversion rule is “˜˜ /////”, the original “demand /” is “demand / ga”, “demand” / "," Demand / de ", or" demand / ni "can be converted.

  When this modifier phrase conversion rule is applied, the following four candidates are created. Candidate 1: “demand (noun) / ga (particle)” → “lower (verb) / ga (particle)” → “large (adjective)”. Candidate 2: “Demand (noun) / O (particle)” → “Down (verb) / GA (particle)” → “Large (adjective)”. Candidate 3: “demand (noun) / de (particle)” → “lower (verb) / ga (particle)” → “large (adjective)”. Candidate 4: “Demand (noun) / ni (particle)” → “lower (verb) / ga (particle)” → “large (adjective)”.

Next, in step S4, the case frame matching unit 50 uses the simplified expression obtained by the simplifying rule selecting unit 30 and the candidate for the replaced expression obtained by the modifier conversion rule selecting unit 40. Match case frames. Specifically, the modifier phrase conversion rule selection unit 40 performs matching with the case frame stored in the domain-dependent case frame table storage unit 150, and selects the case frame with the highest appearance frequency. To do. In this example, the simplified expression “down (verb)” obtained in step S2, and the replacement candidates “demand / ga”, “demand / ga”, “demand / ga”, “demand / da” obtained in step S3, By combining with each of “demand / ni”, four candidates of “demand / go / decrease”, “demand / decrease / decrease”, “demand / de / decrease” and “demand / decrease / decrease” are obtained. ing. Then, the case frame matching unit 50 refers to the domain-dependent case frame table storage unit 150 to select “demand / go / fall” with the highest appearance frequency among these four candidates. As already described, the domain-dependent case frame table is created based on a large number of documents in the domain that matches the input sentence, and the case frame matching unit 50 selects from the candidates using the actual appearance frequency. Therefore, it is possible to perform conversion into a plain text that is natural in the domain.
Through the processing up to this step, it has become possible to convert the expression “demand / no / decline” into an expression “demand / no / decline” in the input sentence. That is, the dependency analysis data corresponding to the input sentence is “demand (noun) / ga (particle)” → “down (verb) / ga (particle)” → “large (adjective)”.

  Next, in step S5, the noun conversion rule selection unit 60 refers to the noun conversion rule table 160 to determine the noun necessary for the nounization from the particle attached to the noun and the independent word modified by the noun. Apply and convert predicates to nouns. In this example, “large” in the dependency analysis result corresponding to the input sentence is an independent word that is modified by a noun (depression). In addition, “depression” of “depression / ga” in the dependency analysis result corresponding to the input sentence is a noun, and “ga” is a particle attached to the noun. In the dialect conversion rule table shown in FIG. 5, the rule that matches the independent word “large” and the particle “ga” is the data on the fourth line, and therefore the noun necessary for the sophistication is “degree”. That is, the noun conversion rule selection unit 60 applies the rule “large / large → degree” to the input sentence. As a result, the syntactic conversion rule selection unit 60 obtains the simplified sentence “demand / reduced / decrease / degree / large” and writes the output sentence to the output sentence storage unit 70.

  As described above, the document simplification apparatus 1 receives an input of the sentence “the demand drop is large”, the simplification rule (drop → drop), the modifier conversion rule, the domain-dependent case frame table, In other words, it was rephrased as “the degree of demand is large.” This example shows that the document simplification apparatus 1 can simplify using the natural expression without changing the meaning of the sentence.

[Second Embodiment]
Next, a second embodiment will be described. In addition, description is abbreviate | omitted about the technical matter common to previous embodiment, and the structure peculiar to this embodiment is described here.
In the previous embodiment, the document simplification apparatus 1 converts an input sentence into a plain sentence using data stored in various tables, and outputs it. Here, a new problem is to reduce the trouble of creating the data of the simplification rule table. If the creation of the simplification rule table can be automated, the document simplification apparatus can be efficiently realized at a lower cost.

FIG. 7 is a block diagram showing a functional configuration of the document simplification apparatus according to the second embodiment. As shown in the figure, the document simplification apparatus 2 is different from the document simplification apparatus 1 described in the first embodiment in that a simplification rule table creation apparatus 200 is provided. Other functional blocks in the figure have the same functions as those in the document simplification apparatus 1.
The simplification rule table creation device 200 automatically generates a simplification rule using dictionary data or the like, and writes the created rule in the simplification rule table storage unit 130.

FIG. 8 is a block diagram showing a detailed functional configuration of the simplification rule table creation device 200. As shown in the figure, the simplification rule table creation device 200
Predicate conversion pair creation unit 210, replaceable word pair creation unit 220, simplification rule candidate recognition unit 230, context similarity recognition unit 240, replaceable word pair table storage unit 310, and simplification rule candidate table storage Unit 320, synonym dictionary table storage unit 410, dictionary table storage unit 420 (related word storage unit), thesaurus table storage unit 430 (related word storage unit), word difficulty level table storage unit 440, and context similarities And a table storage unit 450. Each of these units is also realized using an electronic circuit and a storage medium.

When the synonym data read from the synonym dictionary table storage unit 410 is the predicate data, the prescriptive conversion pair creation unit 210 converts the synonym data and the associated noun data into the predicate conversion pair data. Output as.
The replaceable word pair creation unit 220 includes a dictionary table storage unit 420 and a thesaurus table storage unit based on the noun data and the synonym data included in the word conversion pair data output from the predicate conversion pair creation unit 210. 430 is read. Then, the pair of the headword and the related word read from each storage unit is output as replaceable word pair data of the word before replacement and the word after replacement. The replaceable word pair creation unit 220 writes the created replaceable word pair data in the replaceable word pair table storage unit 310.

The simplification rule candidate recognition unit 230 reads the difficulty level data associated with the word data from the word difficulty level table storage unit for the replaceable word pair data output from the replaceable word pair creation unit 220. Based on the difficulty level data, the replaceable word pair data is recognized as the simplification rule candidate data only when the replaced word is easier than the replaced word. The simplification rule candidate recognition unit 230 writes the approved simplification rule candidate data in the simplification rule candidate table storage unit 320.
The context similarity recognition unit 240 refers to the context similarity table storage unit 450 for the simplification rule candidate data recognized by the simplification rule candidate recognition unit 230, so that the word before replacement and the word after replacement are context-similar. The simplification rule candidate data is recognized as simplification rule data only when there is a negative relationship, and the approved simplification rule data is written in the simplification rule table storage unit 130.

The replaceable word pair table storage unit 310 stores replaceable word pair data created by the replaceable word pair creation unit 220.
The simplification rule candidate table storage unit 320 stores simplification rule candidate data certified by the simplification rule candidate certification unit 230.

The synonym dictionary table storage unit 410 associates and stores noun data and synonym data for the noun data.
The dictionary table storage unit 420 stores the headword and the self-supporting word of the final phrase of the word sentence of the headword in association with each other. The dictionary table storage unit 420 functions as a related word storage unit that stores pairs of headwords and related words. In this case, the independent word in the last sentence of the sentence corresponds to the related word.
The thesaurus table storage unit 430 stores a headword and a synonym of the headword or a broader term of the headword in association with each other. The thesaurus table storage unit 430 functions as a related word storage unit that stores pairs of headwords and related words. In this case, the synonym or broader term of the headword corresponds to the related word.
The word difficulty level table storage unit 440 stores word data and difficulty level data representing the difficulty level of the word data in association with each other.
The context similarity table storage unit 450 stores the correspondence between the word data and other words similar in context to the word data. The context similarity is described later in detail.

Next, main data used in the simplification rule table creation device 200 will be described.
FIG. 9 is a schematic diagram illustrating a configuration and data example of the synonym dictionary table. As shown in the figure, the synonym dictionary table is tabular data, and includes items of noun, synonym expression (synonym data), and synonym part-of-speech. One line of data in this table represents the relationship between a noun, another word synonymous with that noun, and its part of speech. The data of the synonym dictionary table is created in advance based on a synonym dictionary or based on a pair of a verb conjunctive noun or the like and a derivation word (predicate). As an example, the first line of the illustrated data example shows the relationship between the noun “command” (verb combined noun) and its synonym “verm” verb and the part of speech “verb” of the synonym. ing. The fourth line represents the relationship between the noun “Kawaisa” (a noun derived from the adjective), the synonym “cute”, and the part of speech “adjective” of the synonym. The synonym dictionary table may store data representing the relationship between the noun derived from the adjective verb and the derivation adjective verb.

  FIG. 10 is a schematic diagram showing the configuration of the dictionary table and data examples. As shown in the figure, the dictionary table is tabular data, and includes headwords and meaning (related words) items. Note that the data in the dictionary table is created in advance based on data in a Japanese (Japanese) dictionary. More specifically, when the Japanese dictionary data includes headword and interpretation data, the independent word included in the last phrase in the interpretation sentence is stored as the meaning in the dictionary table. For example, if there is data of the sentence “save, help.” Corresponding to the heading word “save” in the Japanese dictionary data, the independent word of each final phrase included in this sentence will be “save” And “help”. Therefore, the data of the headword “save” −meaning “save” and the data of the headword “save” −meaning “help” can be included in the dictionary table. In the first line of the data example shown in the figure, the meaning of the headword “command” is “Shiaru”.

  FIG. 11 is a schematic diagram illustrating the configuration of the thesaurus table and data examples. As shown in the figure, the thesaurus table is tabular data, and includes headwords and thesaurus (related words). The thesaurus is, for example, a synonym of a headword or a broader term of a headword. The first line of the illustrated data example indicates that the synonym or broader term of the headword “rising” is “rising”.

  FIG. 12 is a schematic diagram illustrating a configuration example of data serving as a basis for creating the thesaurus table. The illustrated data is a part of tree-structured data, and includes nodes A, B, and C. In this figure, the higher the node position, the higher the word or concept, and the lower the node position, the lower the word or concept. In addition, words in the same node are similar to each other. An example of data representing such a higher / lower relationship is a Japanese word net. Based on the data of this tree structure, a thesaurus table is created by pairing a word with a word in the same node, or pairing a word and a word belonging to a node one level higher than the node to which the word belongs. Can be created. For example, a pair (probability, prediction) shown in the second row of FIG. 11 can be created from the word “prospect” belonging to the node B and the word “prediction” belonging to the upper node A. Further, for example, a pair (probability, expectation) shown in the third line of FIG. 11 can be created from the word “expectation” belonging to node B as well as the word “expectation” belonging to node B.

  FIG. 13 is a schematic diagram illustrating a configuration of a word difficulty level table and data examples. As illustrated, the word difficulty level table is a table in which “word” and “difficulty level” are paired. As an example for determining the degree of difficulty, there is Japanese Language Proficiency Test (JLPT) question standard data. Here, in order from the highest difficulty level, it is out of class (numerical value 0)> first class (numerical value 1)> second class (numerical value 2)> third class (numerical value 3)> fourth class (numerical value 4). In other words, the lower the difficulty level, the higher the difficulty level. In the example shown in the figure, the word “command” has a difficulty level outside the range (numerical value 0), and the word “definite” has a difficulty level 3 (number 3). In other words, “Karuru” is a simpler word than “Commandments”.

FIG. 14 is a schematic diagram illustrating a configuration of the context similarity table and a data example. As shown in the figure, the context similarity table is realized as, for example, tabular data, and includes items of a word and a context similar word corresponding to the word. The context-similar word item holds a list of words as a value. In other words, the context similarity table holds a correspondence relationship between a word and a word (list) of the word and a context similar word. The list stored in the item of the context similar word list is a list of words having a context similarity equal to or higher than a predetermined threshold with the word stored in the word item. The context similarity used here is a general one that does not depend on a specific domain, and a calculation method thereof will be described later. In the illustrated data example, the word “caution” is included in the context similar word list corresponding to the word “command”. Here, “...” Indicates that description of other words in the list is omitted. Further, the context similar word list corresponding to the word “rising” includes the word “rising”.
Note that verb conjunctive nouns such as “command” and “command”, “rise” and “rise” are similar in context to the verbs from which they are derived. Registration in the similar table may be omitted.

Here, the relationship of context similarity between words will be described in detail. In order to calculate the context similarity, a large amount of sentences are collected in advance from the web or the like, and the sentence set is used. When words w ₁ and word w ₂ appears at a given set of sentences, and context of the words w ₁ in sentence word w ₁ appears in the sentence included in the set of sentences, words in the sentence included in the set of sentences Based on the context of the word w ₂ in the sentence in which w ₂ appears, the similarity between both contexts (context similarity) is calculated numerically, and when the similarity is equal to or greater than a predetermined threshold, In the sentence set, the word w ₁ and the word w ₂ are context-similar. As a typical example, the word “red” appears when there are many expressions of “my favorite color is red” and “my favorite color is blue” in a given sentence set. And the word “blue” are similar in context. The context mentioned here is a set of words that co-occur in the sentence with the word w ₁ and the word w ₂ , the appearance frequency distribution of the co-occurrence words, and the dependency relations surrounding the word w ₁ and the word w _2. Etc.

An example of a method for calculating the context similarity will be described. For a given sentence set, the co-occurrence word for the word w (where w∈W) is v (v∈V), and the frequency at which the word w and the word v co-occur is freq (w, v). To do.
(A) When using a dependency relation For each sentence included in the sentence set, a morphological analysis process and a dependency analysis process are performed. The morpheme analysis process and the dependency analysis process itself can be performed using a computer and an existing computer program. Then, based on the result of the dependency analysis process, paying attention to the case particle, a co-starter vector representing the appearance frequency of the co-starter for the noun w is created.
(B) When using intra-sentence co-occurrence For each sentence included in the sentence set, a morphological analysis process and a phrase delimiting process are performed. The phrase delimiting process can also be performed using a computer and an existing computer program. Then, the noun w and the noun v that co-occurs in the sentence are extracted and set as a co-occurrence pair.
The co-occurrence frequency matrix C is created using the dependency relationship or the intra-sentence co-occurrence. Then, using the obtained co-occurrence frequency matrix C, (1) the value of the Jackard coefficient is calculated as the context similarity between the words w ₁ and w ₂ , or (2) the co-occurrence frequency matrix C is The co-occurrence word vector weighted by tf-idf corresponding to each of the words w ₁ and w ₂ is obtained based on the cosine measure between these vectors as the context similarity between the words w ₁ and w ₂ , (3) s (w ₁ ) and s (w ₂ ) are the numbers of sentences in which the words w ₁ and w ₂ appear, respectively, and the number of times the words w ₁ and w ₂ co-occur in the same sentence are s ( A mutual information amount (PMI, Pointwise Mutual Information) is calculated based on w ₁ , w ₂ ) to obtain a context similarity between the words w ₁ and w ₂ .

  In other words, the context similarity between words is the similarity between the context in which a first word appears in a sentence and the context in which a second word appears in a sentence in a given sentence set. It is based on. At this time, the sentence in which the first word appears and the sentence in which the second word appear may be different sentences, and the first word and the second word appear by chance in the same sentence There is also. The similarity of the context is calculated as a numerical value when a sentence set is given. Here, the context is, for example, in a sentence in which a word appears (that is, in the sentence in which the first word appears for the first word, and for the second word A set of other words (called co-occurrence words) that co-occur with the word in the sentence in which the second word appears, the appearance frequency distribution of co-occurrence words, the appearance order of co-occurrence words, Dependency analysis results of sentences in which words appear (this is represented by a dependency analysis tree, equivalent data, etc.) (including the position of the word in the structure), appearance frequency distribution of the structure, etc. It is. Using these exemplified contexts, context similarity between words is calculated by a predetermined process. When the context similarity is equal to or greater than a predetermined threshold, the words are said to be context-similar.

Next, a specific processing procedure of the simplification rule table creation device 200 will be described based on an example of processing data.
FIG. 15 is a flowchart showing the processing procedure of the simplification rule table creation device 200. Hereinafter, it demonstrates along this flowchart. It should be noted that the input sentence storage unit 10 stores in advance a sentence “a demand drop is large” inputted from the outside, and this sentence becomes a target of simplification.

  First, in step S <b> 21, the word conversion pair creation unit 210 creates data of a word conversion pair based on the data read from the synonym dictionary table 410. For example, when reading the first line of the data example of the synonym dictionary table shown in FIG. 9, the prescriptive conversion pair creation unit 210 creates a prescriptive conversion pair of “command to commandment (verb)” Keep in memory.

  Next, in step S22, replaceable word pair creation unit 220 creates replaceable word pair data based on the data read from dictionary table storage unit 420 and thesaurus table storage unit 430. Specifically, the replaceable word pair creation unit 220 reads the word conversion pair created by the word conversion pair creation unit 210 from the memory, and stores the read word conversion pair in the replaceable word pair table storage unit 310. Write. Also, the replaceable word pair creation unit 220 compares each of the nouns and synonym expressions (prescriptions) included in the read prescription conversion pair with the dictionary table and the thesaurus table, and replaces the pair obtained as a result of the collation The possible word pair table storage unit 310 is written.

  For example, regarding the pair “command-> command” passed from the predicate conversion pair creation unit 210, the replaceable word pair creation unit 220 stores the word “command-> command” and the word “command” in a dictionary table ( In FIG. 10), the pair “command-> right” obtained by collating with the first line of FIG. 10 and the pair “command-> caution” obtained by collating with the fourth line in the replaceable word pair table storage unit 310. Write. Further, for example, regarding the pair “increased → increased” passed from the predicate conversion pair creation unit 210, the replaceable word pair creation unit 220 creates a dictionary of the word “increased → increased” and the word “increased”. The pair “increased → rise” obtained by collating with the first line of the table (FIG. 11) is written in the replaceable word pair table storage unit 310. In other words, in the example given here, the replaceable word pair creation unit 220 performs “command-> command”, “command-> appropriate”, “command-> caution”, “increased → increased”, and “increased → Four pairs of “rising” are created as replaceable word pairs and written into the replaceable word pair table storage unit 310.

  Next, in step S23, the simplification rule candidate recognition unit 230 recognizes the simplification rule candidate from the replaceable word pairs created above based on the data read from the word difficulty level table storage unit 440. Specifically, the simplification rule candidate recognition unit 230 compares the difficulty level before replacement with the difficulty level after replacement for each replaceable pair read from the replaceable word pair table storage unit 310. Then, when the difficulty level after replacement is lower than the difficulty level before replacement (the numerical value of the difficulty level is high), the replaceable word pair is recognized as a simplification rule candidate, and the simplification rule candidate table storage unit 320. Write to. For example, when the word difficulty level table shown in FIG. 13 is used, the difficulty level is the same between “command” and “command”, and therefore the simplification rule candidate recognition unit 230 sets the replaceable word pair “command → command”. Is not recognized as a candidate for simplification rule. Moreover, since the difficulty level of “Karuru” after replacement is lower (simpler) between “command” and “Shiaru”, the simplification rule candidate recognition unit 230 makes the replaceable word pair “command → Saru”. Is recognized as a simplification rule candidate and written into the simplification rule candidate table storage unit 320. Similarly, the simplification rule candidate recognition unit 230 recognizes “command to warn” as a simplification rule candidate and writes it in the simplification rule candidate table storage unit 320. Similarly, the simplification rule candidate recognition unit 230 recognizes “rising → rising” and “rising → rising” as simplification rule candidates, and writes them in the simplification rule candidate table storage unit 320.

Next, in step S <b> 24, the context similarity determination unit 240 performs context similarity determination of the simplification rule candidates recognized by the simplification rule candidate recognition unit 230 based on the data read from the context similarity table storage unit 450. Specifically, the context similarity determination unit 240 reads the simplification rule candidates from the simplification rule candidate table storage unit 320, and for each of them, whether or not the word before replacement is context-similar to the word after replacement. to decide.
When the context similarity table shown in FIG. 14 is used, since “command” and “shiru” are not context-similar, “command / right” that is recognized as a simplification rule candidate by the simplification rule candidate recognition unit 230 is It is not recognized as context-similar. In addition, since “command” and “be careful” are context-similar, the context similarity determination unit 240 determines “command-> caution” as context-similar. Although not registered in the context similarity table, as described above, “increased” and “increased” have a derivation relationship and are similar to each other. Therefore, the context similarity determination unit 240 determines “increased → increased”. It is recognized as context similar. Further, since “rising” and “rising” are context-similar, the context similarity recognition unit 240 recognizes “rising → rising” as context-similar.

Next, in step S <b> 25, the context similarity determination unit 240 writes the simplification rules recognized as context similar to the simplification rule table storage unit 130.
In other words, in the above example, the context similarity determination unit 240 writes “command-> caution”, “increased → increased”, and “increased → increased” in the simplification rule table storage unit 130. Then, the context similarity determination unit 240 does not write “command-> right” that has not been recognized as context similarity in the simplification rule table storage unit 130.

  As described above, in the present embodiment, the document simplification apparatus 2 includes the simplification rule table creation apparatus 200 and automatically creates a simplification rule table. That is, it is possible to efficiently generate rules necessary for sentence simplification.

  Note that the functions of the document simplification apparatus and the simplification rule table creation apparatus in the above-described embodiment may be realized by a computer. In that case, the program for realizing the control function may be recorded on a computer-readable recording medium, and the program recorded on the recording medium may be read by a computer system and executed. Here, the “computer system” includes an OS and hardware such as peripheral devices. The “computer-readable recording medium” refers to a storage device such as a flexible medium, a magneto-optical disk, a portable medium such as a ROM and a CD-ROM, and a hard disk incorporated in a computer system. Further, the “computer-readable recording medium” dynamically holds a program for a short time, like a communication line when transmitting a program via a network such as the Internet or a communication line such as a telephone line. It is also possible to include those that hold a program for a certain time, such as a volatile memory inside a computer system serving as a server or client in that case. The program may be a program for realizing a part of the functions described above, and may be a program capable of realizing the functions described above in combination with a program already recorded in a computer system.

  The embodiment of the present invention has been described in detail with reference to the drawings. However, the specific configuration is not limited to this embodiment, and includes designs and the like that do not depart from the gist of the present invention.

  The present invention can generally be used for automatically simplifying and transforming a large amount of text. Further, the present invention can be used for automatically simplifying a large amount of documents, manuscripts, and the like in the fields of broadcasting, reporting, web content production, and the like.

DESCRIPTION OF SYMBOLS 1, 2 Document simplification apparatus 10 Input sentence memory | storage part 20 Morphological analysis and dependency analysis part 30 Simplification rule selection part 40 Modifier phrase conversion rule selection part 50 Case frame collation part 60 Body language conversion rule selection part 70 Output sentence memory | storage part 130 Simplification rule table storage unit 140 Modifier phrase conversion rule table storage unit 150 Domain-dependent case frame table storage unit (case frame table storage unit)
160 Word conversion rule table storage unit 200 Simplification rule table creation device 210 Word conversion pair creation unit 220 Replaceable word pair creation unit 230 Simplification rule candidate recognition unit 240 Context similarity recognition unit 310 Replaceable word pair table storage unit 320 Candidate rule table storage unit 410 synonym dictionary table storage unit 420 dictionary table storage unit (related word storage unit)
430 Thesaurus table storage unit (related word storage unit)
440 Word difficulty table storage unit 450 Context similarity table storage unit

Claims (2)

A simplification rule table storage unit for storing simplification rule data that associates noun data and prescriptive data corresponding to the noun data;
A modifier phrase conversion rule table storage unit that holds modifier phrase conversion rule data that associates the modifier modifier data with the consecutive modifier data corresponding to the modifier modifier data;
A case frame table storage unit that stores predicate data, continuous modifier data that modifies the prescriptive data, and appearance frequency data related to the prescriptive data and the combined modifier data;
A nomenclature conversion rule table storage unit for storing associative conversion rule data in association with independence word data and inscription data that is a combination of noun data and particle data selected based on the co-occurrence relationship of the independence word data When,
Simplification rule selection for reading dependency analysis result data corresponding to sentence data and selecting applicable simplification rule data from the simplification rule table storage unit based on noun data included in the dependency analysis result data And
Extracted from the dependency analysis result data is a modifier data that modifies the noun data to which the simplification rule data selected by the simplification rule selection unit can be applied, and based on the extracted syntactic modifier data A modifier phrase conversion rule selection unit that selects the corresponding modifier phrase conversion rule data from the modifier phrase conversion rule table storage unit,
The prescriptive data included in the simplification rule data selected by the simplification rule selection unit, and the combined modifier data included in the modifier phrase conversion rule data selected by the modifier conversion rule selection unit; The appearance frequency data is read from the case frame table storage unit, and based on the read appearance frequency data, the combination modification phrase data in the dependency analysis result data is replaced with the selected continuous modification phrase data. And a case frame matching unit that replaces the noun data in the dependency analysis result data with the selected prescription data,
The independent word data related to the noun data before the replacement by the case frame matching unit and the particle data for relating to the independent word data are extracted from the dependency analysis result data, and the extracted independent word data and Based on the particle data, the speech conversion rule data is selected from the speech conversion rule table storage unit. Based on the selected speech conversion rule data, the prescriptive data after replacement by the case frame matching unit is selected. A notation conversion rule selection part to be expressed, and
An apparatus for simplifying a document, comprising: A program for causing a computer to function as the document simplifying apparatus according to claim 1.

Images