JP5436484B2 - Factor trivial word acquisition device, method and program - Google Patents

Description

  The present invention relates to a factor trivial word acquisition apparatus, method, and program, and in particular, a factor using a language pattern including an experience word acquired from an appearance position tendency, a part of speech of each word, and a frequency distribution of an immediately preceding particle of each word. The present invention relates to a trivial word acquisition apparatus, method, and program.

  Acquiring two sets of expressions that express some result and expressions that cause the result (acquisition of causality) is important in various fields such as economic forecasting, marketing, and behavioral navigation. Types of causal relationships include causal relationships between events in the world and causal relationships between individual behaviors and their factors. There is a conventional technique for automatically acquiring a large number of articles as an information source (see, for example, Non-Patent Document 1).

  In methods such as Non-Patent Document 1, newspaper articles are used as information sources, and for each article's text, expressions that match a specific language pattern with clues to connection indicators that represent factors such as “for” and “so” are used. , Identify the place where the factor or result is written. Furthermore, each factor and result expression is normalized by word processing, etc., and pair relationships between the factors and results are extracted, and by collecting a large amount of the pair relationships, the strength of the relationship between the factors and results is increased. It is a method to acquire as knowledge.

Inui Takashi, Inui Kentaro, Matsumoto Yuji. "Automatic acquisition of causal knowledge from document collection based on connection sign" for "," IPSJ Transactions, Vol. 45, No. 3, pp. 919-933, 2004 )

  The problems to be solved by the present invention are the following two.

Task 1) Estimate appropriate factors for words that are factors but do not have a language pattern that represents the factors;
Issue 2) Comprehensive acquisition of factor trivial lists in experience article corpus;
A detailed description is given below for each.

1) Appropriate factor estimation for words that are factors but do not have a language pattern representing the factor;
When there is a task to estimate the location of a factor in an experience article, the above method using a language pattern that appears in the vicinity of each expression is a factor but a word that does not have a language pattern that represents the factor. Cannot be estimated to be a factor. For example, in an experience article describing the experience of visiting a restaurant, in the following example sentence 1 and example sentence 2, the word “secondary meeting” is a factor, but since there is no language pattern representing the factor, It cannot be estimated that there is.

２）経験記事コーパスにおける要因自明語リストの網羅的な獲得；
上記の問題を解決するために、経験記事コーパスにおける要因自明語リストを網羅的に獲得する必要がある。要因自明語を自動獲得する方法として、経験記事コーパスに対して、要因を表す接続標識を用いて、結果表現と要因表現の２つ組を大量に獲得し、要因表現を要因自明語とする方法が考えられる。しかしながら現状では、経験記事における経験に対する要因表現だけを選別して取得することができない。なぜなら、様々な結果表現があるとき、それが経験記事における経験か、その他の結果かを区別することができないからである。例として、レストラン訪問を対象とした経験記事コーパスがあるとき、特定の接続標識の前後に着目し、結果表現と要因表現のペアを大量に獲得することができるが、そのうちのレストラン訪問に対する要因表現だけを収集することはできない。従来、レストラン訪問を表す表現のリストが存在しないためである。（選別取得処理ができないと次の例文３、例文４のような表現についても要因自明語として獲得されてしまう。

2) Comprehensive acquisition of factor self-explanatory word list in experience article corpus;
In order to solve the above problems, it is necessary to obtain a comprehensive list of factor trivial words in the experience article corpus. As a method of automatically acquiring factor self-explanatory words, using a connection indicator representing factors for experience article corpus, acquiring a large number of result expressions and factor expressions and making factor expressions self-explanatory words Can be considered. However, at present, it is not possible to select and obtain only factor expressions for experiences in experience articles. This is because when there are various result expressions, it is not possible to distinguish whether it is an experience in an experience article or other results. As an example, when there is an experience article corpus for restaurant visits, it is possible to obtain a large number of pairs of result expressions and factor expressions by focusing on the front and back of a specific connection sign. Can't just collect. This is because, conventionally, there is no list of expressions representing restaurant visits. (If the selection acquisition process cannot be performed, expressions such as the following example sentence 3 and example sentence 4 are also acquired as factor trivial words.

本発明は、上記の点に鑑みなされたもので、要因でありながら、要因を表す言語パターンを伴わない語の適切な要因推定を可能とし、経験記事コーパスにおける要因自明語リストの網羅的な獲得が可能な要因自明語獲得装置及び方法及びプログラムを提供することを目的とする。

The present invention has been made in view of the above points, and enables the appropriate factor estimation of a word that is a factor but does not involve a language pattern representing the factor, and comprehensively obtains a factor trivial word list in the experience article corpus It is an object of the present invention to provide an apparatus, method and program for acquiring a self-evident factor.

To solve the above problems, the present invention is, among the words appearing more than once in the corpus, a factor obvious word acquisition apparatus for extracting factors obvious word is a word representing the factors,
Experience articles that people wrote about their experiences, storage means that stored experience words,
An input means for receiving input of experience words (hereinafter referred to as “experience word seeds”);
The storage means is referred to based on the input experience word seed, and when there is a corpus that is an element of the experience article, the appearance position tendency in each experience article in the corpus of each word appearing at least once , Experience word acquisition means for extracting experience words using the part of speech of each word and the frequency distribution of the immediately preceding particle of each word;
Wherein the experience word acquired in the experience word acquisition means, by using a language pattern including factors particle, by referring to the storage means, cause words to extract a word on a given occurrence times speed than factors evident Language corpus Acquisition means.

Further, the present invention provides the experience word acquisition means according to claim 1,
When determining whether each word that appears more than once is an experience word,
The frequency distribution of the immediately preceding appearing particles of one or more appropriate experience words input in advance by the user, and the type of part of speech of each word appearing more than once in the corpus is “noun-sa-variant connection” or “verb-independence” Includes means for using the similarity with the frequency distribution of the immediately preceding particle of the word corresponding to “” as a determination feature .

Further, the present invention provides the experience word acquisition means according to claim 1,
When determining whether each word that appears more than once is an experience word,
Means for using, as a determination feature, the distribution of the frequency of one or more particles input from the user in advance appearing immediately before each word.

Further, the present invention provides the factor word acquisition means according to claim 1,
For each word in the corpus, there is included means for extracting, as a factor self-explanatory word, a word whose number of matches with a language pattern including a connection indicator representing a factor designated in advance and an experience word is a predetermined number of times or more.

  In the present invention, using the experience words automatically acquired in advance, it is necessary to deal with the above-described problem of acquiring the factor trivial words for the experience in the experience article. We can solve the problem of “exhaustive acquisition”. Previously, in the above factor estimation problem, it was not possible to properly estimate a factor that is a factor in each experience article but not accompanied by a language pattern that represents the factor. Can be appropriately estimated to be a factor, and the accuracy of factor estimation can be improved.

  Specifically, in the above-described example sentence 1 and example sentence 2, if the word “secondary meeting” is included in the factor trivial word list, it is estimated that the word “secondary meeting” in example sentence 1 and example sentence 2 is the factor. Can do.

It is a block diagram of the factor trivial acquisition device in the 1st Embodiment of this invention. It is an example of the experience article each word table in the 1st Embodiment of this invention. It is an example of the experience article in the 1st Embodiment of this invention. It is an example of the experience word table in the 1st Embodiment of this invention. It is an example of the factor trivial word table in the 1st Embodiment of this invention. It is a flowchart of a process of the factor trivial device in the 1st Embodiment of this invention. It is a flowchart of the action expression acquisition process in the 1st Embodiment of this invention. It is an example of a program of the factor trivial word extraction process in the 1st Embodiment of this invention.

  Embodiments of the present invention will be described below with reference to the drawings.

  First, terms used in this specification will be described.

・ Experience articles: Articles that describe people's own experiences:
・ Experience words: Of the words that appear more than once in the experience article corpus, the words used to describe the author's experience:
(Example: In an experience article corpus that describes an experience of visiting a restaurant, words such as “go”, “visit”, and “visit” are usually used to express the experience of visiting a restaurant, and thus are experience words. )
・ Factors: Events, conditions and conditions that had a positive impact on people's decision to implement experiences:
・ Factor word: In each article, each expression expressing the factor:
-Factor self-explanatory words: Of words that appear more than once in the corpus, words that represent factors with high probability without depending on the appearance context:
(Example: The word “ni-kai” is a factor trivial word in the corpus because it has a high probability of representing a factor when the author visits the store in an experience article about a restaurant visit.)
FIG. 1 shows a configuration diagram of a factor trivial word acquisition apparatus according to the first embodiment of the present invention. The apparatus in the figure is roughly composed of an experience word seed input unit 10, an experience word acquisition unit 20, a factor trivial word acquisition unit 30, a recording unit 40, and an external device 50.

  Hereinafter, each table of the recording unit 50 will be described. The recording unit 50 is a storage medium such as a hard disk, and includes an experience article word table 41, an experience word table 42, and a factor trivial word table 43.

<Experience article word table 41>
As shown in FIG. 2, the experience article word table 41 includes an experience article ID field, an order field from the top, a character string field, and a part of speech field. The experience article word table 41 is a table for storing each experience article, and stores each word of each experience article, and further knows the number of words from the top. Further, the part of speech of the word can be understood from the part of speech field. It is assumed that the experience article is an article in which people as shown in FIG. 3 describe their experiences in a specific field (FIG. 3 is an experience in the field of restaurant visits).

<Experience Word Table 42>
As shown in FIG. 4, the experience word table 42 is configured from an experience word field and stores acquired experience words.

<Factor trivial table 43>
As shown in FIG. 5, the factor trivial word table 43 is configured from a factor trivial word field and stores the acquired factor trivial words.

  Below, the process of the factor trivial word acquisition apparatus in said structure is demonstrated.

  FIG. 6 is a flowchart of the process performed by the factor trivial word acquisition apparatus according to the first embodiment of the present invention.

  This apparatus is roughly divided into an experience seed input process S100 by the experience word seed input unit 10, an experience acquisition process S200 by the experience word acquisition unit 20, and a factor trivial word acquisition process S300 by the factor trivial word acquisition unit 30. Each process will be described in detail below.

<Experience Word Seed Input Processing S100>
In the experience word seed input process, the experience word seed input unit 10 receives an input of an experience word from the user. (The number of experience words accepted is assumed to be two or three experience words that the user can conceive.) When the user finishes inputting experience words, the thresholds θ _1, θ _2, used in the experience word acquisition process S200 _. When input of θ _3, θ ₄ is requested and the threshold values are input from the user, the experience word seed input unit 10 receives the experience word list V = {v _i } and θ _1, θ input by the user. _2, θ _3, θ ₄ are output to the experience word acquisition unit 20.

<Experience acquisition process S200)
The experience word acquisition unit 20 inputs the experience word list V (a set of experience word seeds) and θ _1, θ _2, θ _3, θ _4, passed from the experience acquisition process S200, and refers to the recording unit 40, Acquire (output) experience words. Detailed processing is shown below.

  FIG. 7 is a flowchart of the experience word acquisition process in the first embodiment of the present invention.

Step 201) Based on the experience word seed, the experience word acquisition unit 20 refers to the experience article word table 41 and creates a list of all words z _{i that} appear once or more (in this case, each word is a prototype). When a label / part of speech pair is unique, it counts as a unique word.

That is, <go: verb-independence> and <go: verb-independence> are counted as separate words.
Among words z _i , a list of words a _i whose part of speech is “noun-sa-variant connection” or “verb-independence” is created.

Step 202) Calculate the next score p ₁ (a _i ) for all words a _i .

上記の式（１）のPart(a_i)は、語a_iが各経験記事に出現する際の、直前１語に出現する助詞の頻度分布ベクトルである。part(a_i)の例として、『行く』という単語が経験記事コーパスにおいて、『へ』という助詞を直前１語に取る回数が30回、『に』という助詞を直前１語に取る回数が50回、『と』という助詞を直前１語に取る回数が20回であるとき、
Part『行く』=[30,50,20]
(但し、上記ベクトルは『へ』、『に』、『と』という助詞を直前に取る回数が格納されるとする)などとすることができる。

Part (a _i ) in the above equation (1) is a frequency distribution vector of particles that appear in the immediately preceding word when the word a _i appears in each experience article. As an example of part (a _i ), in the experience article corpus, the number of times that the word “Go” is taken as the last word is 30 times, and the number of words “ni” is taken as the last word is 50 Times, when the number of times the particle “to” is taken in the last word is 20 times,
Part “Go” = [30,50,20]
(However, it is assumed that the vector stores the number of times that the particles “h”, “ni”, “to” are taken immediately before).

cossim (x ₁ , x ₂ ) is a function for calculating the cosine similarity of the input vectors x ₁ and x ₂ as follows.

上記式（２）のp₁(a_i)の値が閾値θ₁より高い語b_iのリストを作成する。

A list of words b _{i in which} the value of p ₁ (a _i ) in the above equation (2) is higher than the threshold θ ₁ is created.

Step 203) The following score p ₂ (b _i ) is calculated for all words b _i created in step 203. The following D (b _i ) is an experience article set including the word b _i .

なお、式（４）のd_pos(b_i, d)b_iがd内で出現位置が冒頭に近いほど高いスコアが付与される。これは経験記事について、まず経験を実施するまでの過程を記し、その後に経験語のことを記す順番で記述されるモデルを仮定し、さらに、前者の経験を実施するまでの過程(冒頭に近い部分)に経験語がしばしば出現する性質を仮定しているためである。またの式（４）において、1/2を減算しているのは、単純に出現回数が多い単語のスコアが高くなるのを防ぐためである。

It should be _noted that a higher score is given as d _pos (b _i , d) b _i in Expression (4) is closer to the beginning in d. This is an experience article that first describes the process up to the implementation of the experience, then assumes a model that is described in the order in which the experience words are written, and then the process up to the implementation of the former experience (close to the beginning) This is because it is assumed that the experience words often appear in (part). In Equation (4), the reason why 1/2 is subtracted is to prevent the score of a word having a large number of appearances from increasing.

After calculating the next score p ₂ (b _i ) for all the words b _i , a list of words c _i whose p ₂ (b _i ) value is higher than the threshold θ ₂ is created.

Step 204) Calculate the next score p ₃ (c _i ) for all words c _i .

但し、式（５）のdf(c_i)は、c_iの経験記事コーパスにおける出現ドキュメント数である。

However, df (c _i ) in Expression (5) is the number of documents appearing in the experience article corpus of c _i .

A list of words in which the value of p ₃ (c _i ) is higher than the threshold value θ ₃ is created, stored in the experience word table 42, and the process transitions to the factor trivial word acquisition process (step 300).

  As an output example of the experience word acquisition process S200, when the following experience word seed is given as an input, an experience word seed having the following label can be output.

-Input experience word seed example:
Go, visit ・ Output experience word seed example:
Go, stop, stop, ask, line, line, go, arrive, visit, use, visit, visit <factor trivial word acquisition process S300>
In factors obvious word acquisition unit 30 refers to the experience table 42, and acquires all the experience word e _j.
Further, all the information is acquired by referring to the experience article word table.

Next, in each experience article d, the following processes (1), (2), and (3) are performed.
(1) Experience article For each word that appears in d, all occurrence positions w ₁ (number of words from the beginning) are acquired for words whose original form matches any of each e _j .

(2) Next, for each occurrence position w ₁ , if the word located one word before w ₁ is either {and, as, so, so, therefore}, the appearance position w ₂ Get all.

(3) Next, for each appearance position w ₂ , the word located before the word of w ₂ is recorded in the factor trivial word list in the memory (not shown).

After the above processes (1), (2), and (3) have been performed for all experience articles, all of the words f _i that appear θ ₄ times or more in the factor trivial word list are stored in the factor trivial word table 43.

The processes (1), (2) and (3) can be described in the form of a program as shown in FIG. The program shown in the figure is D for the set of experience articles d, size (d) for the number of words in each experience article d, w (d, g) for the g-th word in experience article d, and E for the set of experience words. , And let the factor particle be j = {and, as, so, and so for}. “Count (w _i )” is processing for counting the number of appearances of the word w _i .

  In the factor trivial word acquisition process, paying attention to language patterns including experience words and factor particle words, the number of occurrences of underlined words is counted when there is the following expression, and the words counted more than a certain number of times are factor trivial words Record as.

要因自明語獲得処理では、誤りを少なく、数多くの要因自明語を獲得するために、経験語および特定の要因助詞語との接続を利用している。

In the factor trivial word acquisition processing, the connection with the experience word and the specific factor particle word is used in order to reduce errors and acquire many factor trivial words.

  As an output example of the factor trivial word acquisition process S300, the following factor trivial words can be output.

・ Examples of self-evident factors:
Cheap, lunch, one person, two people, friends, family, car, secondary party, banquet, boss, date, friends, colleagues, dinner, lunch, good reputation, entertainment, many, thoughts, brunch, wedding, entertainment, families, Year-end party, birthday celebration, dinner, tavern sense, introduction, anniversary, group, friends, couple, 2 people, farewell party, work, takeout, taxi, secondary party, train, morning, banquet, private, joint party, weekday lunch, with children , Celebration, women, birthday, women, dinner, New Year's party, tourism [second embodiment]
In the present embodiment, the process of step 202 in the experience word acquisition process of the first embodiment is changed as follows.

Step 202) First, an input of one or more particle sets Q = {q _i } and a threshold value θ ₅ is received from the user. When the input from the user ends, the process proceeds to the following process.

Score p ₄ (a _i ) is calculated for all words a _i acquired in step 201 using equation (6). Note that tf (a _i ) is the appearance frequency of the word a _i in the corpus.

最後にp₄(a_i)の値が閾値θ₅より高い語b_iのリストを作成する。

Finally, a list of words b _i having a value of p ₄ (a _i ) higher than the threshold θ ₅ is created.

  According to the first and second embodiments described above, the following effects can be obtained.

Effect 1 corresponds to issue 1 and effect 2 corresponds to issue 2.
Effect 1) Appropriate factor estimation for words that are factors but do not have a language pattern representing the factors;
・ Effect 2) Comprehensive acquisition of factor self-explanatory word list in experience article corpus Details are described below.

  As a first effect of the present invention, when there is a task of acquiring an expression representing a factor from each experience article, even a word not necessarily accompanied by a language pattern representing the factor is a factor by using the factor obvious word list. It can be evaluated whether or not.

  For example: When there is the following sentence in an experience article, it is possible in the prior art to evaluate the next underlined word as a word representing a factor. This is because there is a connection indicator indicating the factor.

しかしながら次のような語について要因であると評価することは従来技術においてはできない場合があった。要因を表す言語パターンを伴っていないからである。

However, there are cases where it is not possible in the prior art to evaluate the following words as a factor. This is because there is no language pattern representing the factor.

本発明によって、上記例文３のような場合において、要因であるか否かを判定する際に語の要因らしさ（要因自明語に含まれるか否か）という情報を新たに用いることができ、高精度な要因性判定の実現に寄与することができる。

According to the present invention, in the case of the above-described example sentence 3, when it is determined whether or not it is a factor, it is possible to newly use information on the likelihood of the word (whether or not included in the factor trivial word). This can contribute to the realization of accurate causality determination.

  As a second effect of the present invention, when there is an experience article corpus, it is possible to select and acquire only expressions that surely represent the factors for the experience, and to show the marketer the factors with high accuracy for the experience. Marketers can understand the tendency of experience factors in the entire field of experience, and can use it as knowledge when building various hypotheses and executing strategies in marketing.

  As an example, marketers in the food and beverage industry can grasp the factors that frequently appear in the food and beverage industry by using a set of review articles about the restaurant and this device. An advertising sentence using an ugly phrase including a factor word can be created by the device.

なお、第１の実施の形態における図８において、要因自明語獲得部３０におけるプログラムを示したが、この例に限定されることなく、要因自明語獲得装置の経験語シード入力部１０、経験語獲得部２０の処理についてもプログラムとして構築し、要因自明語獲得装置として利用されるコンピュータにインストールして実行させる、または、ネットワークを介して流通させることが可能である。

In FIG. 8 in the first embodiment, the program in the factor trivial word acquisition unit 30 is shown. However, the present invention is not limited to this example. The processing of the acquisition unit 20 can also be constructed as a program, installed in a computer used as a factor trivial word acquisition device and executed, or distributed via a network.

  The present invention is not limited to the above-described embodiments, and various modifications and applications can be made within the scope of the claims.

DESCRIPTION OF SYMBOLS 10 Experience word seed input part 20 Experience word acquisition part 30 Factor trivial word acquisition part 40 Memory | storage part 41 Experience article word table 42 Experience word table 43 Factor trivial word table

Claims (9)

Among the words appearing more than once in the corpus, a factor obvious word acquisition apparatus for extracting factors obvious word is a word representing the factors,
Experience articles that people wrote about their experiences, storage means that stored experience words,
An input means for receiving input of experience words (hereinafter referred to as “experience word seeds”);
The storage means is referred to based on the input experience word seed, and when there is a corpus that is an element of the experience article, the appearance position tendency in each experience article in the corpus of each word appearing at least once , Experience word acquisition means for extracting experience words using the part of speech of each word and the frequency distribution of the immediately preceding particle of each word;
Wherein the experience word acquired in the experience word acquisition means, by using a language pattern including factors particle, by referring to the storage means, cause words to extract a word on a given occurrence times speed than factors evident Language corpus Acquisition means,
A factor trivial word acquisition device characterized by comprising: The experience word acquisition means includes
When determining whether each word that appears more than once is an experience word,
The frequency distribution of the immediately preceding appearing particles of one or more appropriate experience words input in advance by the user, and the type of part of speech of each word appearing more than once in the corpus is “noun-sa-variant connection” or “verb-independence” The factor trivial word acquisition apparatus according to claim 1, further comprising means for using a similarity with the frequency distribution of the immediately preceding particle of the word corresponding to “ as a feature of determination. The experience word acquisition means includes
When determining whether each word that appears more than once is an experience word,
2. The factor trivial word acquisition apparatus according to claim 1, further comprising means for using, as a determination feature, a distribution of the frequency of one or more particles that are input in advance from a user and appearing immediately before each word. The factor word acquisition means is
The factor self-explanatory claim according to claim 1, further comprising means for extracting, as a factor self-explanatory word, a word whose number of matches with a language pattern including a connection indicator and an experience word representing a pre-designated factor for each word in the corpus Word acquisition device. Among the words appearing more than once in the corpus, a factor obvious word acquisition method of executing extracting the factors obvious word is a word that represents a factor in the computer,
An input step in which the input means receives an input of an experience word (hereinafter referred to as “experience word seed”);
Experience Katarie resulting unit, based on the experience word seed entered, people refers to the storage means for storing the experience articles and experience language noted for their experiences, when there is a corpus experience articles is an element Experience of extracting experience words using the tendency of appearance position in each experience article, the part of speech of each word, and the frequency distribution of the last appearing particle of each word in the corpus of each word appearing more than once A word acquisition step;
Factors word acquisition means, wherein the experience word acquired in the experience language acquisition step, using a language pattern including factors particle, by referring to the storage means, cause obvious words on a given occurrence times speed than from the corpus Factor word acquisition step to extract as words,
A factor trivial acquisition method characterized by In the experience word acquisition step,
When determining whether each word that appears more than once is an experience word,
The frequency distribution of the immediately preceding appearing particles of one or more appropriate experience words input in advance by the user, and the type of part of speech of each word appearing more than once in the corpus is “noun-sa-variant connection” or “verb-independence” The method according to claim 5 , wherein the similarity with the frequency distribution of the immediately preceding particle of the word corresponding to "is used as a determination feature . In the experience word acquisition step,
When determining whether each word that appears more than once is an experience word,
6. The factor trivial word acquisition method according to claim 5, wherein a distribution of the frequency of one or more particles previously input from a user appearing immediately before each word is used as a determination feature. In the factor word acquisition step,
6. The factor trivial word acquisition method according to claim 5, wherein, for each word in the corpus, a word having a predetermined number of times that matches a language pattern including a connection indicator representing a factor designated in advance and an experience word is extracted as a factor trivial word. . Computer
A factor trivial word acquisition program for causing each factor of the factor trivial word acquisition device according to any one of claims 1 to 4 to function.

Images