JPH11175525A - Database device for processing natural language - Google Patents

Abstract

PROBLEM TO BE SOLVED: To allow a user to add and correct a character string desired by the user and the frequency information to a database for processing natural language. SOLUTION: This device is provided with a data base 6 for processing natural language for storing plural sets of a partial character string constituted of prescribed number of characters appearing in a natural language sentence and the absolute or relative frequency information. Also, this device is provided with a user inputting means 1 for receiving a character string to be reflected on the storage content of the data base for processing natural language and the significance, and a data base updating means 7 for adding the character string and the frequency information corresponding to the significance as a set to the data base for processing natural language when the items of one or plural partial character strings constituted of the prescribed number of characters constituting the input character string are not present in the data base, and for updating the frequency information of the partial character string according to the significance when the items of the partial character strings are present in the data base for processing natural language.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a database device for natural language processing, and can be applied to, for example, a morphological analyzer for automatically analyzing morphemes of sentences.

[0002]

2. Description of the Related Art Japanese Patent Laid-Open No. Hei 7-271792 In a device for processing a natural language sentence such as a Japanese sentence (for example, a machine translation device, a question-and-answer device, a computer-assisted educational device, etc.), First, a morphological analysis is performed.

A morphological analyzer generally includes a morphological analyzer (morphological analysis program), a dictionary (word dictionary),
The morphological analysis unit includes an inflectional ending table and a part-of-speech connection table. The morphological analysis unit refers to the dictionary, the inflectional ending table, and the part-of-speech connection table for the input text. In this way, morphological analysis is performed.

On the other hand, recently, instead of using a word dictionary, instead of using a word dictionary, a tag-based (= a large amount of text data) learned from a corpus (= a large amount of text data) which holds morpheme boundaries and part-of-speech information of each morpheme is used. A morphological analyzer using a statistical database (a database for natural language processing) that stores the appearance frequency information of partial character strings has begun to be studied (Japanese Patent Application Laid-Open No. 7-271792 and Japanese Patent Application No. 9-68).
No. 300 and drawings).

In this morphological analysis method, instead of a connection table constructed by a heuristic technique by a developer,
The reason for determining the morpheme boundary is clearer than the conventional method in that a chain probability (appearance frequency information) based on statistical data obtained from the corpus is used. In addition, even if an unknown word is present, it is said that highly accurate analysis can be performed based on a certain standard.

[0006]

However, in the morphological analyzer based on the above-mentioned statistical data, since a dictionary is not used, when a user wants to define a certain morpheme and reflect it in the morphological analysis result, this is realized. There is a problem that means are not provided. That is, since the method of registering in a dictionary cannot be used, there is no means to realize a morpheme when the user wants to define the morpheme and reflect it in a morphological analysis result.

Therefore, there is a need for a natural language processing database device (statistical database device) that can accept input of user-defined morphological data.

[0008]

In order to solve this problem, the present invention stores a plurality of sets of a partial character string consisting of a predetermined number of characters appearing in a natural language sentence and its absolute or relative frequency information. In the natural language processing database device having a natural language processing database used by the natural language processing device main body, (1) a character input by a user and desired to be reflected in the stored content of the natural language processing database (2) if there is no item in the natural language processing database for one or more partial character strings having the predetermined number of characters, which constitutes the character string input by the user, A pair with the frequency information according to the importance is added to the natural language processing database, and a character string input by a user is configured. Database updating means for updating the frequency information of the partial character string in the natural language processing database according to the importance, if one or a plurality of partial character string items having a predetermined number of characters are present in the natural language processing database And characterized in that:

[0009]

DESCRIPTION OF THE PREFERRED EMBODIMENTS (A) First Embodiment (A-1) Configuration of First Embodiment Hereinafter, a database apparatus (statistics database apparatus) for natural language processing according to the present invention is applied to a morphological analyzer. The first embodiment will be described in detail with reference to the drawings.

The morphological analyzer according to the first embodiment has
It is realized by an information processing device such as a workstation or a personal computer having an input / output device, a communication device, an external storage device, or the like as appropriate, and can be functionally shown by the functional block diagram of FIG.

In FIG. 1, a morphological analyzer according to a first embodiment includes an input unit 1, a morphological analyzer 2, and an output unit 3.
And a tagged corpus (tagged corpus storage unit) 4
, A chain probability calculating unit 5, a statistical database 6, a statistical database weight changing unit 7, and a user morphological input unit 8.

Here, the input unit 1, the morphological analysis unit 2 and the output unit 3 constitute a morphological analysis device main body, and include a corpus 4 with a tag, a linkage probability calculation unit 5, a statistical database 6, and a statistical database weight changing unit. 7 and the user morpheme input unit 8 constitute a statistical database device. The tagged corpus 4 and the chain probability calculator 5 form a statistical database 6, and the morphological analysis utilizes the formed statistical database 6. Therefore, in the case of the first embodiment, In this case, the tagged corpus 4 and the chain probability calculator 5 can be omitted.

The input unit 1 is a character string (natural language text).
Is received as an input, and the input character string is sent to the morphological analyzer 2. The input unit 1 may be composed of any means such as a keyboard, a mouse, an OCR (optical character recognition device), a voice recognition device, and the like, and a communication signal from the outside via a communication medium such as a network. May be configured as a means for receiving the message.

The morphological analysis unit 2 converts the input character string into
The morphological analysis is performed using the information of the statistical database 6. The morphological analysis unit 2 includes an extended character string generation unit 2a, a score table 2b, a score calculation unit 2c, and an optimum route search unit 2d having the following functions, although the detailed configuration is not shown.

The extended character string generation unit 2a refers to the statistical database 6 to generate extended characters of the input character string, and from the beginning of the input character string to the end of the sentence, an extended character string (N-gram) composed of N characters. ) Are stored in the score table 2b. Score table 2
b corresponds to the path of all extended character strings (N-gram) from the beginning of the input character string to the end of the sentence and the path of the extended character string obtained based on the partial chain probability information stored in the statistical database 6. It stores the chain probability information of the extended character string. The score calculation unit 2c calculates chain probability information for the path of the extended character string stored in the score table 2b based on the partial chain probability information stored in the statistical database 6. The optimum route search unit 2d converts an extended character string that satisfies an optimal condition (for example, gives maximum value chain probability information) from the chain probability information calculated by the score calculation unit 2c into an optimal expanded character string (morpheme). (Analysis result).

The output unit 3 receives the morphological sequence of the analysis result from the morphological analysis unit 2 and outputs it. For example,
Various display means, printing means, communication means, and the like correspond.

The tagged corpus 4 is a large amount of text data holding tags of morpheme boundaries (or part-of-speech information of each morpheme). FIG. 2 shows an example of data of the corpus 4 with a tag. In the example shown in FIG. 2, each morpheme boundary is displayed by a slash (/), and information of the part of speech, inflection type, and inflection type are listed in square brackets with commas. Note that the tag may be a tagged corpus 4 including only morpheme boundaries.

The chain probability calculating section 5 includes a tagged corpus 4
Is to process the text data held by the database and create the statistical database 6 (referred to as word model estimating means or part-of-speech model estimating means in Japanese Patent Laid-Open No. Hei 7-271792). Specifically, for example, N-g
ram extended string (extended string containing only morpheme boundaries,
Alternatively, it calculates chain probability information of an extended character string including a morpheme boundary and a part of speech.

The statistical database 6 is basically a database of the results calculated by the chain probability calculator 5.
In the case of the first embodiment, the statistical database 6
Can be changed by the statistical database weight changing unit 7.

The statistical database weight changing unit 7 receives morphological information (data of a specific N-gram extended character string) which affects the morphological analysis result from the user morphological input unit 8, and receives the morphological information in the statistical database 6. (N-g
ram extension character string). Specifically, for example, the designated N-g
This is to change the chain probability information of the ram extended character string.

The user morphological input unit 8 receives input of morphological information which affects the morphological analysis result from the user, arranges the information, and sends it to the statistical database weight changing unit 7. The user morpheme input unit 8 according to the first embodiment receives data on the importance of the change and sends the data at the same time. For example, the user inputs the importance level of 0.99 when "I want to surely reflect the morpheme in the analysis result", and when the user wants to reflect the morpheme with minimal influence on other parts, the importance level is 0.
Enter 3.

(A-2) Operation of the First Embodiment The input unit 1 receives an input character string, the morphological analysis unit 2 performs a morphological analysis using the contents stored in the statistical database 6, and The operation of outputting a morpheme sequence is the same as that of a conventional morphological analyzer using the statistical database 6, and a description thereof will be omitted.

The operation of the linkage probability calculation unit 5 for processing the tagged corpus 4 to create the statistical database 6 is the same as that of the conventional apparatus, and therefore the description thereof is omitted.

For example, in the literature “Shin Nagao, Shinsuke Mori,
"How to make n-gram statistics and automatic extraction of words and phrases in large-scale Japanese texts", Information Processing Society of Japan, Natural Language Processing 96
1, July 1993] can be applied.

In the following, the operation of receiving the input of morpheme information from the user at the user morpheme input unit 8 and modifying the data of the statistical database 6 through the statistical database weight changing unit 7 will be described with reference to FIG. .

In the following description, it is assumed that the statistical database 6 relates to character delimiters of N-gram characters. It is also assumed that the data of the tagged corpus 4 has only morpheme delimiters as information (without information such as part of speech).

First, a morpheme information input screen (FIG. 4)
(See step 301).

The portion between square brackets [] in the screen in FIG. 4 is the portion to be input by the user. The morpheme information is input by inserting a morpheme delimiter into a series of character strings. The user is informed of the input method in advance by a manual or the like. This is an input meaning that each partial character string separated by a morpheme delimiter is a morpheme. If there is no delimiter in the entered string,
It is determined that the entire character string is one morpheme. It is assumed that a value between 0.0 and 1.0 is input as the importance. If the value is 0.99 (a value as close to 1 as possible), it means that each input morpheme is surely reflected, and the smaller the value, the less the possibility that the morpheme is reflected. This is a value indicating how much the user tolerates the influence on another morpheme caused by cutting out a certain morpheme.

Next, the statistical database weight changing unit 7, which receives the morpheme sequence input by the user and the importance from the user morpheme input unit 8, converts the input morpheme sequence into an extended character string (step 302). . FIG. 5 shows an example of an extended character string. In FIG. 5, extended characters <ko, 0> and <
The character “ko” in 1> is the character itself that constitutes the morpheme string, and the numbers “0” and “1” respectively indicate whether the back of the character is a morpheme delimiter. <#, 1> is a special extended character, and if the statistical database 6 constitutes the chain probability information for each N-gram extended character string, N -1 is added at a time.

Next, the position pointer is set at the head of the extended character string (step 303), and N extended characters (N-
It is checked whether or not extraction is possible (step 304). For example, when N = 3 and in the case of the extended character string in FIG. 5, the pointer moves from <#, 1> at the beginning to <
It is possible to extract three extended characters when it is between the characters and the character, and it is impossible to extract three characters for the first time when the pointer is located at <#, 1> immediately after the character. Become.

If it is determined in step 304 that extraction is possible, the process proceeds to step 305. If extraction is not possible, a series of processes is terminated.

At step 305, N extended characters are actually extracted, and it is searched whether or not the extracted extended character string exists in the heading of the statistical database 6 (step 306).

FIG. 6 shows an example in which the statistical database 6 has a table configuration. Statistics database 6
As shown in FIG. 6, a heading composed of an N-gram (3-gram) extended character string and a value of the chain probability information corresponding thereto (the probability value itself may be used, or a uniform multiple of the same). (Hereinafter referred to as a weight value). There is no duplicate in the N-gram extension character string of the heading.

Here, the heading is a statistical database 6
If it does not exist (No in step 306), a weight value is created with reference to the value of importance received at the time of morpheme input (step 310). There are various methods for creating values, but the simplest method is to use the value of the importance as it is as the weight. Thereafter, one line of data having the heading is added to the statistical database 6, and a weight value is registered (step 311).

On the other hand, if it is determined in step 306 that the extracted data already exists in the statistical database 6, the current weight value is obtained from the statistical database 6 (step 307). Then, a new weight value is calculated with reference to the current weight value and the numerical value of the importance received at the time of morpheme input (step 308). Then, the weight value of the headline (N-gram extended character string) is changed to the new weight value thus obtained (step 309).

There are various methods for calculating a new weight value. For example, a calculation method represented by the following equation (1) can be given.

(New weight value) = (current weight value) + {1.0− (current weight value)} * (importance) (1) When the calculation method of the equation (1) is applied, If the importance is maximum, the value is almost close to 1.0. Even if the importance is small, a new weight value that is slightly smaller than the current weight value but surely increases can be calculated.

N-gram determined by current position pointer
When the process of adding and changing the storage contents of the statistical database 6 for the extended character string is completed, the pointer is shifted by one character (step 312), and the process returns to step 304 described above.

(A-3) Effects of the First Embodiment According to the first embodiment, the user adds and corrects his desired morpheme (extended character string) and its weight value to the statistical database 6. be able to. As a result, the desired morpheme sequence information can be reflected in the morphological analysis result. That is, the user can change the statistical database 6 so that a desired analysis result can be obtained by a simple input operation, and the usability of the apparatus can be improved as compared with the related art.

(A-4) Modified Embodiment of First Embodiment The statistical database 6 may have any configuration as long as it can respond to addition and correction of items, and the configuration is not limited to a table configuration. Further, since the content is not limited to the N-gram character string, it may be data of N parts of speech arranged as described in JP-A-7-271792.

If the input of the morpheme information is executed in the form having the part of speech information as shown in FIG. 2, the N-gram extended character string having not only the morpheme delimiter information but also the part of speech information is obtained. Items can be added to and modified from the extended statistical database 6.

Further, a negative value may be given to the importance. In that case, statistics database 6
Table value will be smaller than it is now,
It will work to suppress the division of the morpheme.

Further, in the first embodiment,
Although the description has been given of a method in which the user assigns importance to the entire morpheme string, the importance may be specified for each set of N-gram extended characters. Further, when calculating the weight value, a method may be used in which the user is inquired about the importance for each set of N-gram extended characters and interactively processed.

In the case where designation is possible for each set of N-gram extended characters as described above, a weight value (chain probability information) itself may be designated instead of importance. In this case, it is preferable that an existing weight value (chain probability information) is displayed and the correction is designated. When the weight value (chain probability information) itself is designated, it is preferable to allow the designation of 0 as the weight value. In this case, the N-gram extended character string having a weight value of 0 is never reflected in the morphological analysis result. In other words,
This means that an aggressive prohibition condition has been given.

In the first embodiment, the correction of the weight value (chain probability information) is performed on the N-gram extended character string obtained from the morpheme information related to the user input. According to this modification, other N-gr
The weight value (chain probability information) of the am extended character string may be corrected. For example, in general, in the statistical database 6 that stores information on N-gram extended character strings, the weight value (chain) of a plurality of N-gram extended character strings in which the first N-1 extended characters are the same Probability information) is set to a predetermined value (for example, 1),
The weight value (chain probability information) of another N-gram extended character string may be modified so as to keep this condition. Such correction can be executed by changing the weight value (chain probability information) to a frequency value, correcting the frequency value, and then changing the weight value again.

The technical concept of the first embodiment can be applied to a case where the weight value in the statistical database 6 is managed by frequency.

(B) Second Embodiment (B-1) Configuration of Second Embodiment Hereinafter, a second embodiment in which the database device (statistical database device) for natural language processing according to the present invention is applied to a morphological analyzer. The form will be described in detail with reference to the drawings.

The morphological analyzer of the second embodiment also has
It is realized by an information processing device such as a workstation or a personal computer having an input / output device, a communication device, an external storage device, or the like as appropriate, and can be functionally shown by a functional block diagram in FIG. In FIG. 7, the same or corresponding parts as those in FIG. 1 described above are denoted by the same reference numerals.

In FIG. 7, a morphological analyzer according to the second embodiment includes an input unit 1, a morphological analyzer 2, and an output unit 3.
, A tagged corpus 4, a chain probability calculator 5, a statistical database 6, a user morpheme input unit 8, and a corpus adding unit 9. In the case of the second embodiment, the chain probability calculating unit 5, the statistical database 6, the user morphological input unit 8, and the corpus adding unit 9 constitute a statistical database device (a database device for natural language processing).

In FIG. 7, an input unit 1, a morphological analysis unit 2, an output unit 3, a tagged corpus 4, a linkage probability calculation unit 5, a statistical database 6, and a user morphological input unit 8
The third embodiment has the same function as the first embodiment, and a description of the function will be omitted.

The corpus adding unit 9 makes a copy of the morpheme string information sent from the user morphological input unit 8 in a number corresponding to the degree of importance, and adds it to the tagged corpus 4. The corpus adding unit 9 is given an initial value of the data size (for example, the number of sentences, the number of morphemes, the number of characters, and the like) of the tagged corpus 4, and holds the size held when adding data to the tagged corpus 4. Change the value of. The corpus adding unit 9 instructs the chain probability calculating unit 5 to perform recalculation after adding data to the tagged corpus 4.

(B-2) Operation of the Second Embodiment The operation of the conventional device is different from that of the conventional device only in the portions related to the user morphological input unit 8 and the corpus adding unit 9. Therefore,
In the following, input of morphological information from the user is received by the user morphological input unit 8, data is added to the tagged corpus 4 through the corpus adding unit 9, and the chain probability calculating unit 5 is added.
Will be described with reference to the flowchart of FIG. Note that the tagged corpus 4 of the second embodiment is not of the format shown in FIG. 2, but is obtained by removing part of speech information (including inflected and inflected forms) therefrom.

First, a morpheme input screen (FIG.
) And input of morphological information is accepted (step 801). This step is the same as in the first embodiment (step 301 in FIG. 3).

Next, the amount to be added (here, the number of sentences) is determined based on the existing corpus size and importance (step 8).
02). Corpus size data can be obtained from the corpus adding unit 9
Shall be held. For example, the additional amount is determined by the following equation (2). A plurality of formulas may be selectively applied according to the corpus size up to now.

Additional quantity (number of sentences) = corpus size (number of sentences) * 0.01 * importance (however, rounded up to the nearest decimal point) (2) Next, the number of additional sentences determined in step 802 is The input morpheme string information is duplicated and added to the tagged corpus 4 (step 803). At this time, the corpus size is updated based on the number of sentences added (and other data such as the number of morphemes in some cases).

Then, it notifies that the tagged corpus 4 has been updated, and sends a recalculation command to the chain probability calculation unit 5 (step 804). As a result, the chain probability calculation unit 5 performs recalculation using the updated corpus 4 with the tag, and the statistical database 6 is updated.

When a morphological analysis is performed using the updated statistical database 6, an analysis result reflecting the user's input is obtained.

(B-3) Effects of the Second Embodiment According to the second embodiment, the user can add / modify his / her desired morpheme (extended character string) and its weight value to the statistical database 6. And the desired morpheme sequence information can be reflected in the morphological analysis result. That is, the user can change the statistical database 6 so that a desired analysis result can be obtained by a simple input operation, and the usability of the apparatus can be improved as compared with the related art.

In the second embodiment, a method of creating a statistical database 6 is used to reflect a user's desire in the statistical database 6, and the N-garm extended character string is less adversely affected. It is thought that it is enough. In other words, it is only necessary to consider the sentence (morpheme sequence) input by the user as appearing many times in the corpus, and there are few arbitrary operations (= hard to make artificial mistakes). The risk of reflection is small, and the statistical database 6 can be operated slowly.

On the other hand, in the aspect that the degree of reflection of the user information is small and the operation cannot be performed in a direction to reduce the chain probability information of the N-gram extended character string related to the specified morpheme,
The first embodiment is better.

(B-4) Modified Embodiment of Second Embodiment The statistical database 6 may have any configuration as long as it can respond to addition and correction of items, and the configuration is not limited to a table configuration. Further, since the content is not limited to the N-gram character string, it may be data of N parts of speech arranged as described in JP-A-7-271792.

If the input of the morpheme information is executed in a form having the part of speech information as shown in FIG. 2, the N-gram extended character string having not only the morpheme delimiter information but also the part of speech information is obtained. Items can be added to and modified from the extended statistical database 6.

Further, the technical idea of the second embodiment can be applied to the case where the weight value in the statistical database 6 is managed by frequency.

(C) Third Embodiment (C-1) Configuration of Third Embodiment Hereinafter, a third embodiment in which the natural language processing database device (statistical database device) according to the present invention is applied to a morphological analyzer. The form will be described in detail with reference to the drawings.

The morphological analyzer of the third embodiment also has
It is realized by an information processing device such as a workstation or a personal computer having an input / output device, a communication device, an external storage device, or the like as appropriate, and can be functionally shown by a functional block diagram in FIG. In FIG. 9, the same or corresponding parts as those in FIG. 1 described above are denoted by the same reference numerals.

In FIG. 9, the morphological analyzer according to the third embodiment includes an input unit 1, a morphological analyzer 2, and an output unit 3.
A tagged corpus 4, a chain probability calculator 5, a statistical database 6, a statistical database weight changer 7,
It comprises a user morpheme input unit 8, a sentence example search unit 10, a sentence example output unit 11, and an index (index storage unit) 12 of a tagged corpus. In the case of the third embodiment, the linkage probability calculating unit 5, the statistical database 6, the statistical database weight changing unit 7, the user morphological input unit 8, the sentence example searching unit 10, the sentence example output unit 11, and the index 12 are composed of a statistical database device ( (A database device for natural language processing).

In FIG. 9, an input unit 1, a morphological analysis unit 2, an output unit 3, a tagged corpus 4, a linkage probability calculation unit 5, a statistical database 6, and a statistical database weight changing unit 7 are the same as those in the first embodiment. Function.
The description of the function is omitted.

The data held by the tagged corpus 4 is the same as that of the first embodiment, but an index 12 of the stored data is separately prepared. The index 12 of the tagged corpus 4 holds data (plain data) obtained by removing tags (information such as delimiters, parts of speech and inflected forms) of data held by the tagged corpus 4 as headings. Each heading is provided with a pointer to data in the corpus 4 with tags from which the heading is based.

The user morpheme input unit 8 sends the received user morpheme sequence to the sentence example search unit 10 in addition to the operation of the first embodiment.

The sentence example search unit 10 is a user morpheme input unit 8
A partial character string having a fixed number of characters (M characters) (a plurality of characters: as many as possible) is created from a character string obtained by receiving a user morpheme string from and removing a delimiter. Further, each of the partial character strings is sent to the index 12, and if any of the index 12 (plane data) includes the partial character string, a pointer to the original data is obtained. Further, the data of the pointer is sent to the corpus 4 with the tag, and the data (sentence example: a morphological sequence with a tag)
Is acquired and sent to the sentence example output unit 11.

The sentence example output unit 11 outputs the morpheme string (sentence example) sent from the sentence example search unit 10.

(C-2) Operation of the Third Embodiment The operation of the user morpheme input unit 8, the sentence example search unit 10, and the sentence example output unit 11 that perform the characteristic operation in the third embodiment will be described below with reference to FIG. This will be described with reference to the flowchart of FIG.

First, a morpheme input screen (see FIG. 11) is presented to the user, and morpheme input is accepted (step 100).
1) It is determined whether the button pressed at the time of this reception is a sentence example search button or a learning button (step 1002).

As shown in FIG. 11, in the case of the third embodiment, the morpheme input screen displays not only an input field for receiving a morpheme input and an importance input, but also a “learning button” and a “sentence example search button”. It is requested that the operation mode be designated by the "learning button" and the "sentence example search button". Also, a display field for displaying a sentence example obtained by a sentence example search operation started by pressing the “sentence example search button” is prepared in advance.

When the learning button is pressed, the statistical database weight changing unit 7 receives the user morpheme sequence,
In order to perform the same operation as in the first embodiment, the processing illustrated in FIG. 11 ends. When the processing in FIG. 11 is completed, the processing proceeds to the processing in step 302 and subsequent steps in FIG. 3 described above.

On the other hand, if the pressed button is a sentence example search button, the sentence example search unit 10 receives only the user morpheme string (no data of importance is required) and sets the tag (here, only the delimiter). It is removed to create plane data (character string) (step 1003). For example, if the input morpheme string is “here // kimono / w / wipe”, “wipe kimono here” with all the delimiters removed is the plain data.

Next, a partial character string is created from the plane data (step 1004). In this process, assuming that the number of characters for this processing is determined in advance (here, three characters; this number of characters may be irrelevant to the N characters of the N-gram extended character string), Cut out all substrings of that length. Note that the number of characters may be changeable by a method designated at the time of starting the system or by receiving an input from the user. For example, when the plain data is the above example, “here”, “kodani”, “deki”, “kikimo”, “kimono”, “mono”, “nonu”, A total of eight partial character strings "to wipe" are created. here,
If the number of characters of the plane data does not reach the above-defined number of characters, the plane data itself is used as a partial character string.

Next, the leading partial character string among the unprocessed ones is set as a processing target (step 1005), and it is determined whether or not there is a heading in the index including the partial character string. And the sentence example search unit 10 receives all the pointers to the original data of the matched headline (step 1006).

Then, a processed flag is given to the partial character string to be processed (step 1007), and it is checked whether there is another unprocessed partial character string (step 1008).

If there is an unprocessed partial character string, the process returns to step 1005 described above. Step 1 like this
By repeating the processing loop consisting of 005 to 1008, unprocessed partial character strings eventually disappear. When the processing of all the partial character strings is completed in this way, the duplication of the pointer to the received original data is deleted and arranged (step 1009). This is because, when there is plane data including a plurality of partial character strings, original data is obtained for matching processing of each partial character string.

Next, the data is acquired by referring to the data (sentence example) of the corpus 4 with the tag by the pointer (step 101).
0). Then, these sentence examples are sent to the sentence example output unit 11 and output to the screen (Step 1011), and a series of processes is ended.

FIG. 12 shows an example of the output screen. This output screen corresponds to the user morpheme string input screen shown in FIG. 11 described above, and this search result is output below the “sentence example search button”.

Looking at such an output screen, the user can change the degree of importance and press the “learning button” to shift to the operation of adding and changing the statistical database 6 as in the first embodiment. it can.

In other words, when the user adds or corrects his or her desired morpheme and its weight value to the statistical database 6, by searching and displaying a sentence example including the same character string, the user's input can be any other It can be estimated whether it affects the parsing of sentences. Since it is the statistical database 6, it is impossible to absolutely avoid the influence on others. However, by looking at sentence examples, it is possible to change the input morpheme sequence,
You can change the importance.

Corpus 4 with tag to be searched for sentence example
Is an element of the statistical database 6 used for analysis, and thus indirectly expresses the obtained morphological analysis result.
For example, if the tags of those sentences are removed and actually analyzed, the sentence returns to the sentence of the corpus 4 with the tag that holds the morpheme delimiter information as a tag. In that sense, the above estimation can be performed efficiently. Therefore, a completely different corpus (tagged, untagged) may be searched, but in that case,
Although it can be estimated by reference, the effect is somewhat diminished because the direct effect on the current analysis cannot be known.

(C-3) Effect of Third Embodiment According to the third embodiment, the user can add / modify his / her desired morpheme (extended character string) and its weight value to the statistical database 6. And the desired morpheme sequence information can be reflected in the morphological analysis result. That is, the user can change the statistical database 6 so that a desired analysis result can be obtained by a simple input operation, and the usability of the apparatus can be improved as compared with the related art.

Further, according to the third embodiment, the sentence example search unit for searching the sentence example including the same partial character string as the morpheme string input by the user is provided. Can be estimated to adjust the input content.

(C-4) Modified Embodiment of Third Embodiment The third embodiment basically has the configuration of the first embodiment, and is therefore a modification of the above-described first embodiment. The embodiment can be cited as a modified embodiment of the third embodiment.

In the third embodiment, the sentence example search unit 10 and the sentence example output unit 11 are provided in the configuration of the first embodiment. However, the sentence example search unit 10 and the sentence example output unit 11 are provided in the configuration of the second embodiment. The apparatus may be configured by providing the sentence example output unit 11.

Further, in the third embodiment, the example in which the sentence example search unit 10 searches for a sentence example by using the index 12 has been described. However, the index 12 is omitted, and the sentence example search unit 10 searches the corpus 4 with tags. Alternatively, a direct search process may be performed. In this case, the sentence example search unit 10 does not need to remove the tag from the morpheme string input by the user.

(D) Other Embodiments In the description of each of the above embodiments, various modified embodiments have been described, but the following modified embodiments can be further mentioned.

In each of the above embodiments, the morpheme sequence information that the user intends to reflect is input each time. However, the morpheme sequence information output from the output unit 3 is directly or partially corrected, and You may enable it to be input as the input morpheme string information.

In each of the above embodiments, the target natural language is Japanese, but a statistical database device (natural language processing database) storing character strings of a predetermined number of characters relating to other languages. Device)
The present invention can be applied.

In each of the above embodiments, the statistical database device of the present invention is applied to a morphological analyzer. However, the processing using the statistical database device of the present invention is limited to morphological analysis. is not. For example, when a part of a natural language text received by a communication unit is an unknown word due to a burst error, a natural language processing apparatus that estimates a correct character string of the unknown word portion using the contents stored in a statistical database. The statistical database device of the present invention can be applied to a statistical database device. Depending on the use of the statistical database device of the present invention, the statistical database may store only character strings that do not have morpheme delimiter information, part-of-speech information, and the like.

It is to be noted that both the words in the claims and those that do not include those that include morpheme delimiter information or part of speech information are meant.

[0096]

As described above, according to the present invention, a plurality of sets of a partial character string consisting of a predetermined number of characters appearing in a natural language sentence and its absolute or relative frequency information are stored. A user input which receives a character string and importance to be reflected in the content stored in the natural language processing database by a natural language processing database apparatus having a natural language processing database used in the language processing apparatus body If there is no item of one or more partial character strings of a predetermined number of characters constituting the character string input by the user in the database for natural language processing, frequency information according to the importance is set as a set, and If it is added to the database for language processing, and if the item of the substring is in the database for natural language processing, the partial character of the database for natural language processing And a database updating means for updating the frequency information according to the degree of importance, so that the user can add / modify his / her desired character string and its frequency information to the natural language processing database, and The desired character string information can be reflected on the device, and the usability of the device can be improved as compared with the related art.

[Brief description of the drawings]

FIG. 1 is a functional block diagram showing a configuration of a first embodiment.

FIG. 2 is an explanatory diagram showing an example of a corpus with a tag.

FIG. 3 is a flowchart illustrating a characteristic operation of the first embodiment.

FIG. 4 is an explanatory diagram illustrating an example of a morpheme input screen according to the first embodiment.

FIG. 5 is an explanatory diagram showing an example of an extended character string.

FIG. 6 is an explanatory diagram illustrating an example of a statistical database.

FIG. 7 is a functional block diagram illustrating a configuration of a second embodiment.

FIG. 8 is a flowchart illustrating a characteristic operation of the second embodiment.

FIG. 9 is a functional block diagram illustrating a configuration of a third embodiment.

FIG. 10 is a flowchart showing a characteristic operation of the third embodiment.

FIG. 11 is an explanatory diagram illustrating an example of a morpheme input screen according to the third embodiment.

FIG. 12 is an explanatory diagram illustrating an example of a search sentence example output screen according to the third embodiment.

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 1 ... Input part, 2 ... Morphological analysis part, 3 ... Output part, 4 ... Tagged corpus, 5 ... Chain probability calculation part, 6 ... Statistical database, 7 ... Statistical database weight change part, 8 ... User morphological input part, 9 … Corpus addition part, 10… sentence example search part,
11: sentence example output unit, 12: index.

Claims (4)

[Claims] A plurality of sets of a partial character string consisting of a predetermined number of characters appearing in a natural language sentence and its absolute or relative frequency information are stored. Input means for receiving a character string and importance to be reflected in the stored content of the natural language processing database, the character string being input by the user. If the item of the one or more partial character strings having the predetermined number of characters does not exist in the database for natural language processing, it is added to the database for natural language processing as a set with frequency information according to the importance. In addition, the item of one or a plurality of partial character strings having the predetermined number of characters, which constitutes a character string If the use database, natural language processing database device characterized by comprising a database update means for the frequency information of the partial character string of the database for the natural language processing is updated in accordance with the importance. 2. The database updating means separates a character string input from a user into one or more partial character strings having the predetermined number of characters, and searches the natural language processing database using the separated partial character strings. If there is no item of the separated partial character string as a result of the search, the frequency information according to the importance is added as a set to the natural language processing database, and the separated partial character string is searched. 2. The natural language processing database device according to claim 1, wherein if there is an item, the frequency information of the partial character string in the natural language processing database is updated according to the importance. 3. A sentence example storage unit which stores a natural language sentence or a natural language character string used for forming the natural language processing database, wherein the database updating unit stores a character string received by the user input unit. A sentence example adding unit to be added to the sentence example storage unit by at least the number of appearances determined according to the importance, and a database for reconstructing the natural language processing database based on the storage contents of the sentence example storage unit after the addition processing 2. The natural language processing database device according to claim 1, wherein the database device comprises a reconstructing unit. 4. The natural language processing database according to claim 1, further comprising a sentence example storage unit that stores a natural language sentence and a natural language character string used for forming the natural language processing database. In the apparatus, a sentence example extracting character string input means for receiving a sentence example extracting character string input from a user, and having the entirety of the example example extracting character string or a partial character string thereof,
A natural language processing database device further comprising: a sentence example search output unit that extracts and outputs a natural language sentence or a natural language character string stored in the sentence example storage unit.