JP2007080019A - Natural language processing system, natural language processing method and natural language processing program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a natural language processing system or the like capable of appropriately extracting dictionary data to be shared by appropriately selecting a partner to share the dictionary data by the unit of an individual user. <P>SOLUTION: In the natural language processing system 1, a similarity degree calculation means 21 calculates a similarity degree between a first user dictionary and a second user dictionary. A registration candidate extraction means 22 extracts dictionary data included in the first user dictionary and not included in the second user dictionary as a registration candidate for the second dictionary when the similarity degree is a predetermined threshold or higher. A user dictionary registration means 23 registers the dictionary data included in the registration candidate to the second user dictionary. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a technology for processing a natural language using a user dictionary, and more particularly to a natural language processing system and the like that can appropriately extract data to be shared in order to update the user dictionary.

Natural language processing systems such as kana-kanji conversion, machine translation, speech recognition, and speech synthesis basically perform processing based on words stored in the dictionary and the language information assigned thereto, and are stored in the dictionary. It is difficult to properly handle missing words, that is, unknown words. On the other hand, natural languages are changing day by day, and new words are born, and even if they are known words, new usage occurs, so it is difficult to store them all in the dictionary in advance.
Therefore, conventionally, this problem has been addressed by providing a user dictionary function that can individually register words that are required by individual users. However, even for words that are commonly required by a plurality of users, the dictionary registration work must be performed separately for each user, and redundant work has occurred for the entire user. On the other hand, a method of sharing dictionary data registered by individual users among a plurality of users has been proposed.

  For example, Patent Literature 1 describes a method of extracting dictionary data that appears more than a certain number of times in all of a plurality of user dictionaries as candidates for dictionary data to be shared and sharing it with all users. However, in this method, the dictionary data of words related to various organizations and fields are uniformly shared without distinguishing them, so it becomes easy to share the dictionary data of words related to organizations and fields with many users, There arises a problem that useful dictionary data cannot be obtained for users belonging to a minority organization or field. In addition, for individual users, unnecessary word dictionary data related to an organization or field unrelated to him / her is also shared, and the influence of the influence sometimes reduces the accuracy of natural language processing.

As a conventional example for this problem, Patent Document 1 and Patent Document 3 manage individual user dictionaries and dictionary data in association with predetermined fields, and dictionary data candidates to be shared in units of individual fields. The method of extracting and sharing by the unit of field is described.
Patent Document 2 discloses a method of managing individual user dictionaries and dictionary data in association with predetermined organizations, extracting dictionary data candidates to be shared in units of individual organizations, and sharing them in units of organizations Is described.
These methods are intended to prevent the dictionary data from being inappropriately shared across organizations and fields by limiting the partners who share the dictionary data to a range where the organizations and fields are the same.

Japanese Patent No. 3464881 Patent 3556425 JP2003-157257

However, in the above prior art, the definition of an organization or a field varies depending on the user's viewpoint and changes with time, but it is assumed that the organization or field is handled with a predetermined fixed structure. It is said.
For this reason, it is not possible to flexibly cope with differences in viewpoints and changes in time of individual users, and necessary dictionary data and inappropriate dictionary data are extracted as dictionary data to be shared while being mixed. There was a problem.

  Accordingly, an object of the present invention is to provide a natural language processing system or the like that can appropriately extract dictionary data to be shared by appropriately selecting a partner to share dictionary data in units of individual users. To do.

  In the natural language processing system according to the present invention, the similarity calculation means is between the first user dictionary (the user dictionary of the other party sharing the dictionary data) and the second user dictionary (the user dictionary to be registered with the dictionary data). Calculate similarity. The registration candidate extraction unit extracts dictionary data included in the first user dictionary and not included in the second user dictionary as registration candidates for the second dictionary when the similarity is equal to or higher than a predetermined threshold. . The user dictionary registration means registers the dictionary data included in the registration candidates in the second user dictionary (claims 1 to 6).

According to the natural language processing system, the registration candidate extraction unit selects a user dictionary that extracts registration candidates based on the similarity of the user dictionary, that is, a partner user dictionary to which dictionary data should be shared. If the degree of similarity is low, registration candidates are not extracted from the first user dictionary even if the users of the first user dictionary and the users of the second user dictionary belong to the same organization / field.
Therefore, the user dictionary of the other party who shares dictionary data can be selected appropriately for each user.

In the natural language processing system, the similarity is the total number of dictionary data registered in the first user dictionary and the second user dictionary (the number of dictionary data in the first user dictionary and the number of dictionary data in the second user dictionary). Based on the ratio of the number of dictionary data registered in common to the two dictionaries) and the number of dictionary data registered in common in the first user dictionary and the second user dictionary (Claim 2).
In this way, a user dictionary in which many common dictionary data are registered can be selected as a dictionary data sharing partner.

In the natural language processing system, the similarity between the processing target of the natural language processing performed in the past using the first user dictionary and the processing target of the natural language processing performed in the past using the second user dictionary. You may make it calculate a similarity based on a certain process target similarity (Claim 3).
In this way, it is possible to select a user dictionary of a user with a similar processing target to be handled as a sharing partner.

In the natural language processing system, the dictionary data includes classification information, and the first dictionary data set and the second dictionary data that are composed of dictionary data having the same classification information in the dictionary data stored in the first user dictionary. The similarity is calculated based on the dictionary data set similarity, which is the similarity of the second dictionary data set made up of dictionary data having the same classification information among the dictionary data stored in the user dictionary of the first, The dictionary data included in the dictionary data set and not included in the second dictionary data set may be extracted as registration candidates.
In this way, the candidate for registration candidate extraction can be selected in consideration of the type of dictionary data, so that the dictionary data sharing partner can be more appropriately compared with the case where the entire user dictionary is the candidate for registration candidate extraction. Can be selected.

In the natural language processing system, the total number of dictionary data registered in the first dictionary data set and the second dictionary data set and the common registration in the first dictionary data set and the second dictionary data set. The dictionary data set similarity may be calculated based on the ratio to the number of dictionary data.
In this way, a dictionary data set in which many common dictionary data are registered can be selected as a dictionary data sharing partner.

In the natural language processing system, the processing target of the natural language processing performed in the past using the first dictionary data set is similar to the processing target of the natural language processing performed in the past using the second dictionary data set. The dictionary data similarity may be calculated based on the degree (claim 6).
In this way, it is possible to select, as a sharing partner, a part of a user dictionary of users with similar processing targets to be handled in units of dictionary data sets.

  According to the natural language processing method of the present invention, the first user dictionary and the second user dictionary are read from the storage device, and the similarity between the first user dictionary and the second user dictionary is calculated. When the degree is equal to or greater than a predetermined threshold, the dictionary data included in the first user dictionary and not included in the second user dictionary is extracted as a registration candidate for the second user dictionary and extracted. The candidate is recorded in the storage device, the registration candidate is read from the storage device, and dictionary data included in the registration candidate is registered in the second user dictionary (claim 7).

According to the natural language processing method, the user dictionary for extracting registration candidates based on the similarity of the user dictionary, that is, the partner user dictionary to which the dictionary data should be shared is selected. If the degree of similarity is low, registration candidates are not extracted from the first user dictionary even if the users of the first user dictionary and the users of the second user dictionary belong to the same organization / field.
Therefore, the user dictionary of the other party who shares dictionary data can be selected appropriately for each user.

  The natural language processing program of the present invention reads the first user dictionary and the second user dictionary from the storage device and calculates the similarity between the first user dictionary and the second user dictionary on a computer. The dictionary data included in the first user dictionary and not included in the second user dictionary when the similarity is equal to or greater than a predetermined threshold as a registration candidate for the second user dictionary A registration candidate extraction function for extracting and recording the extraction candidates in the storage device and a user dictionary registration function for reading the registration candidates from the storage device and registering dictionary data included in the registration candidates in the second user dictionary are executed. (Claims 8 to 13).

According to the natural language processing program, the computer is caused to select a user dictionary for extracting registration candidates based on the similarity of the user dictionary, that is, a partner user dictionary to which dictionary data should be shared. If the similarity is low, the computer does not extract registration candidates from the first user dictionary even if the user of the first user dictionary and the user of the second user dictionary belong to the same organization / field.
Therefore, it is possible to operate the computer as a natural language processing system and appropriately select a partner user dictionary to share dictionary data for each user.

In the above program, the total number of dictionary data registered in the first user dictionary and the second user dictionary and the number of dictionary data registered in common in the first user dictionary and the second user dictionary The similarity may be calculated based on the ratio (claim 9).
In this way, a user dictionary in which many common dictionary data are registered can be selected as a dictionary data sharing partner.

In the above program, a processing target that is a similarity between a processing target of natural language processing performed in the past using the first user dictionary and a processing target of natural language processing performed in the past using the second user dictionary The similarity may be calculated based on the similarity (claim 10).
In this way, it is possible to select a user dictionary of a user with a similar processing target to be handled as a sharing partner.

In the above program, the dictionary data includes classification information, and the first dictionary data set and the second user dictionary composed of dictionary data having the same classification information in the dictionary data stored in the first user dictionary The similarity is calculated based on the dictionary data set similarity which is the similarity of the second dictionary data set made up of dictionary data having the same classification information in the dictionary data stored in the first dictionary data Dictionary data included in the set and not included in the second dictionary data set may be extracted as registration candidates (claim 11).
In this way, the candidate for registration candidate extraction can be selected in consideration of the type of dictionary data, so that the dictionary data sharing partner can be more appropriately compared with the case where the entire user dictionary is the candidate for registration candidate extraction. Can be selected.

In the above program, the total number of dictionary data registered in the first dictionary data set and the second dictionary data set, and dictionary data registered in common in the first dictionary data set and the second dictionary data set The dictionary data set similarity may be calculated on the basis of the ratio to the number of claims.
In this way, a dictionary data set in which many common dictionary data are registered can be selected as a dictionary data sharing partner.

In the above program, the similarity between the processing target of the natural language processing performed in the past using the first dictionary data set and the processing target of the natural language processing performed in the past using the second dictionary data set. The dictionary data similarity may be calculated based on the processing target similarity.
In this way, it is possible to select, as a sharing partner, a part of a user dictionary of users with similar processing targets to be handled in units of dictionary data sets.

According to the present invention, a user dictionary from which registration candidates are extracted based on the similarity of the user dictionary, that is, a partner user dictionary to which dictionary data should be shared is selected. If the degree of similarity is low, registration candidates are not extracted from the first user dictionary even if the users of the first user dictionary and the users of the second user dictionary belong to the same organization / field.
Therefore, the user dictionary of the other party who shares dictionary data can be selected appropriately for each user.

Next, the configuration and operation of the natural language processing system 10 according to the first embodiment of the present invention will be described in detail with reference to the drawings.
FIG. 1 is a functional block diagram showing the configuration of the natural language processing system 10.
Referring to FIG. 1, a natural language processing system 10 is, for example, a personal computer, and includes an input device 1 such as a keyboard and a microphone, a data processing device 2 that operates under program control, a storage device 3 that stores information, and a display device. And a printing device, and an output device 4 such as a speaker.

The storage device 3 is configured by a hard disk device, for example, and includes a user dictionary storage unit 31 and a registration candidate storage unit 32.
The user dictionary storage unit 31 stores user dictionaries 31A, 31B, 31C of individual users. Here, each user dictionary stores words registered by the user and corresponding language information. Here, the linguistic information is information that the natural language processing means 24 to be described later refers to for the processing, and includes, for example, kana notation, reading, translation, part of speech, and semantic information.
The registration candidate storage unit 32 stores registration candidates for each individual user dictionary, such as a registration candidate 32A corresponding to the user dictionary 31A. The registration candidates include dictionary data candidates to be registered for the corresponding user dictionary. Here, the dictionary data is the smallest unit of information registered in the user dictionary, and is composed of words and corresponding language information.

The data processing device 2 is, for example, a CPU (Central Processing Unit), and includes a similarity calculation unit 21, a registration candidate extraction unit 22, a user dictionary registration unit 23, and a natural language processing unit 24.
The similarity calculation means 21 calculates the similarity between two user dictionaries (the first user dictionary and the second user dictionary) among the user dictionaries stored in the user dictionary storage unit 31. Here, as the degree of similarity between user dictionaries, for example, the degree of coincidence of dictionary data registered in each user dictionary can be used. This degree of coincidence can be defined by the ratio between the total number of dictionary data registered in each user dictionary and the number of dictionary data registered in both user dictionaries.
If the natural language processing performed by the natural language processing means 24 is to process natural language character strings such as kana-kanji conversion processing, machine translation processing, and speech synthesis processing, the similarity between the user dictionaries is set as the past. Alternatively, the similarity between the processing objects of the natural language processing performed using the user dictionary may be used. Here, the similarity between objects to be processed is, for example, a vector space model (G. Salton and MJ McGill, “Introduction to Modern Information Retrieval”, McGraw-Hill, 1983, proposed by Salton et al. ). According to the vector space model, for example, the processing target can be expressed by a feature vector whose component is an independent word contained therein, and the similarity between the processing targets is defined by the cosine of the angle formed by each feature vector. be able to.

  The registration candidate extraction unit 22 takes out a set of user dictionaries (first user dictionary and second user dictionary) whose similarity calculated by the similarity calculation unit 21 is equal to or greater than a threshold value, and includes it in the first user dictionary. Dictionary data that is not included in the second user dictionary is extracted as dictionary data candidates (registration candidates) to be shared. If registration candidates can be extracted, the dictionary data is recorded in the registration candidate storage unit 32 in association with the second user dictionary. For example, if the user dictionary 31B is the second user dictionary and the user dictionary 31A is the first user dictionary, the registration candidate extraction unit 22 is included in the user dictionary 31B and not included in the user dictionary 31A. If there is dictionary data, it is recorded in the registration candidate storage unit 32A as a registration candidate corresponding to the user dictionary 31A.

  The user dictionary registration unit 23 reads registration candidates recorded in the registration candidate storage unit 32 and registers dictionary data included therein in a corresponding user dictionary. Here, the dictionary data included in the read registration candidates may be displayed on the output device 4 and the user may be inquired whether to register the dictionary data. In this case, further, the dictionary data determined to be unnecessary by the user is recorded in the registration candidate storage unit 32 together with information indicating that the user is not registered, and then the registration candidate extraction unit 22 shares the same dictionary data. Even if it is extracted as a candidate for power dictionary data, it may be excluded from being registered.

  The natural language processing unit 24 receives an input from the user, performs natural language processing using the user dictionary of the user stored in the user dictionary storage unit 31, and outputs a processing result. Here, the natural language processing performed by the natural language processing means 24 is, for example, kana-kanji conversion processing, machine translation processing, speech recognition processing, or speech synthesis processing. Here, the kana-kanji conversion process is a process that converts the input kana character string into a kanji-kana mixed character string, and the machine translation process converts the input first language character string into a second language character string. The speech recognition process is a process for converting the input speech signal into a character string, and the speech synthesis process is a process for converting the input character string into a speech signal.

Next, with reference to the flowcharts of FIG. 1 and FIG. 2, an operation of extracting dictionary data candidates to be shared by the natural language processing system 10 as registration candidates will be described in detail.
FIG. 2 is a flowchart showing an operation in which the natural reduction processing system 10 extracts registration candidates. As an outline of the operation, the similarity is calculated for all combinations of two dictionaries selected from the user dictionaries stored in the user dictionary storage unit 31, and the combinations whose similarity is equal to or greater than a threshold are stored in the respective user dictionaries. Corresponding registration candidates are extracted and recorded in the registration candidate storage unit 32.

First, the similarity calculation means 21 takes out a set of two user dictionaries from the user dictionary storage unit 31 (step S101). Next, it is checked whether or not a set of unprocessed user dictionaries can be extracted (step S102), and if it cannot be extracted, the process ends.
When a set of unprocessed user dictionaries can be extracted, the similarity calculation means 21 calculates the similarity between these user dictionaries (step S103).
Next, the registration candidate extraction unit 22 compares the similarity obtained by the similarity calculation unit 21 with a threshold value (step S104). If the obtained similarity is smaller than the threshold value, the process returns to step S101 and continues. If the obtained similarity is equal to or greater than the threshold value, the registration candidate extraction unit 22 compares the dictionary data stored in these user dictionaries, and extracts dictionary data included only in one of the user dictionaries as a registration candidate. (Step S105). If the registration candidate cannot be extracted, the process returns to step S101 and continues (step S106).
When the registration candidate can be extracted, the registration candidate extraction unit 22 stores the registration candidate in the registration candidate storage unit 32 as a registration candidate corresponding to the user dictionary that does not include the registration candidate, and then returns to step S101 for processing. (Step S107).

Here, the similarity between two user dictionaries is calculated in units of user dictionaries and registration candidates are extracted. However, if classification information is set in the dictionary data in the user dictionary, Similarity may be calculated for a set of dictionary data in which the same classification information is set in the user dictionary, and dictionary data candidates to be shared for each unit may be extracted.
For example, a set of dictionary data included in the user dictionary 31A having the same classification information A (second dictionary data set) and a dictionary data included in the user dictionary 31B having the same classification information B The similarity (dictionary data set similarity) between the sets (classification information i and classification information b may be the same or different) is calculated, and this similarity is calculated as the similarity of dictionary data. To do. For example, dictionary data included only in the second dictionary data set is set as registration candidate data for the user dictionary 31A. The similarity of the dictionary data set can be calculated by the same method as the similarity of the user dictionary described above.
Furthermore, in this case, the similarity may be calculated in units of combinations of the obtained dictionary data within the range of the same user dictionary, and dictionary data candidates to be shared may be extracted for each unit.

  Moreover, the process which extracts this registration candidate can take the form operated for every fixed period, for example. Alternatively, when one user dictionary in the user dictionary storage unit 31 is updated, this process is applied only to the combination of the updated user dictionary and other user dictionaries. It can also take the form which processes.

Next, the operation of the natural language processing system 10 for registering the dictionary data included in the registration candidates in the user dictionary will be described in detail with reference to the flowcharts of FIGS. 1 and 3.
FIG. 3 is a flowchart showing an operation of registering the extracted dictionary data in the user dictionary by the natural language processing system 10. As an outline of the operation, dictionary data candidates to be registered are extracted from the registration candidate storage unit 32 and presented to individual users, and each dictionary data is registered in a corresponding user dictionary in accordance with a user instruction.

  First, the user dictionary registration unit 23 extracts registration candidates corresponding to the user dictionary from the registration candidate storage unit 32 (step S111 in FIG. 3). Next, the user dictionary registration means 23 checks whether or not a registration candidate can be extracted from the registration candidate storage unit 32, and if it can be extracted, whether dictionary data included in the registration candidate is not a registration target, and is a dictionary to be registered in the user dictionary. It is checked whether or not there is a data candidate (step S112). Here, when there is no dictionary data candidate to be registered, that is, when the registration candidate could not be extracted, or information indicating that the registration candidate was extracted but all of the dictionary data included in the candidate could not be registered. If so, the process ends.

  If there is a candidate for dictionary data to be registered, the user dictionary registration means 23 displays the dictionary data on the output device 4 and inquires of the user whether or not to register each dictionary data in the user dictionary (step) S113). Thereafter, the user dictionary registration means 23 receives an input from the input device 1 and checks whether there is dictionary data that has been input as non-registered (step S114). If there is dictionary data that is input as non-registration target, the user dictionary registration unit 23 records the dictionary data in the registration candidate storage unit 32 together with information indicating that it is out of registration target (step S115). After that, or when there is no dictionary data input that is not subject to registration in step S113 (NO in step S114), the user dictionary registration means 23 checks whether the input dictionary data exists when it is registered. (Step S116). If there is dictionary data that has been input upon registration, the user dictionary registration means 23 registers the dictionary data in the user dictionary (step S117). After that, or when registered in step S116 and the input dictionary data does not exist, the process ends.

  The process of registering the extracted dictionary data in the user dictionary may take, for example, a form that operates when the user explicitly calls this process or a form that operates when the user calls the natural language processing means 24. it can.

Next, a first specific operation example of the natural language processing system 10 will be described.
In this example, it is assumed that the natural language processing means 24 performs machine translation processing in particular, and each registration candidate is extracted from the two user dictionaries 31A and 31B in FIG. 1, and the dictionaries included in the registration candidates are extracted. An operation for registering data in the user dictionary A and the user dictionary B will be described.
FIG. 4A shows the contents of the user dictionary 31A, and FIG. 4B shows the data contents of the user dictionary 31B. FIG. 4A and FIG. 4B show the contents of two user dictionaries, user dictionary 31A and user dictionary 31B, in tabular form, and one row of each table represents one dictionary data. . For example, the first line in FIG. 4A represents dictionary data in which “chimera” is Japanese, “chimera” is English, and “noun” is a part of speech. In this example, “chimera” is a word, and “chimera” and “noun” are language information.

In the process of extracting registration candidates, the similarity calculation means 21 extracts the user dictionary 31A and the user dictionary 31B from the user dictionary storage unit 31, and calculates the similarity between them.
Ten dictionary data are registered in the user dictionary 31A, and eleven dictionary data are registered in the user dictionary 31B. Of these, nine dictionary data are commonly registered in both of them, such as the “chimera” line. The total number of dictionary data registered in the two dictionaries is 10 + 11−9 = 12.
Based on these numerical values, the similarity calculation means 21 calculates the similarity between the user dictionary 31A and the user dictionary 31B as 9/12 = 0.75.

  Next, the registration candidate extracting unit 22 compares the similarity 0.75 output by the similarity calculating unit 21 with a threshold value. If the threshold is 0.7, the similarity between the user dictionary 31A and the user dictionary 31B is equal to or greater than this value, and the process proceeds to processing for extracting registration candidates from both. Here, first, regarding the user dictionary 31A, the dictionary data of “gene” and “gene diagnosis” that are included in the user dictionary 31B and not included in the user dictionary 31A are extracted as registration candidates, and registration corresponding to the user dictionary 31A is performed. It records in the registration candidate memory | storage part 32A which is a candidate memory | storage area | region (In this case, user dictionary 31A corresponds to a 2nd user dictionary, and user dictionary 31B corresponds to a 1st user dictionary). As for the user dictionary 31B, the dictionary data of “transposon” that is included in the user dictionary 31A and not included in the user dictionary 31B is extracted, and a registration candidate storage unit 32B that is a storage area of registration candidates corresponding to the user dictionary 31B. (In this case, the user dictionary 31A corresponds to the first user dictionary and the user dictionary 31B corresponds to the second user dictionary).

  Examples of the registration candidate storage unit 32A and the registration candidate storage unit 32B obtained as a result of these processes are shown in FIGS. 5 (a) and 5 (b). According to FIG. 5, the registration candidate storage unit 32 has an area for storing information indicating whether or not the dictionary data is not subject to registration in addition to the dictionary data stored in the user dictionary. Here, “−” indicates that the inspection of whether or not the registration target is not completed, and “not target” indicates that the dictionary data has been designated as a registration target in the past. That is, “gene” has not been examined in the dictionary data A, whereas “gene diagnosis” has been extracted as a candidate for dictionary data to be shared before, and is then the owner of the user dictionary 31A. As a result of inquiring the user A about the necessity of registration, it is indicated that there is a history of inputting that registration is not necessary.

  The natural language processing system 10 extracts registration candidates for all user dictionaries by repeating the process of extracting the registration candidates for all two combinations of user dictionaries stored in the user dictionary storage unit 31. The extracted registration candidates are stored in an area corresponding to each user dictionary in the registration candidate storage unit 32.

In the process of registering the dictionary data included in the registration candidate in the user dictionary, the user dictionary registration unit 24 takes out the registration candidate from the registration candidate storage unit 32 and inquires of the user whether to register in the user dictionary.
For example, consider a case where user A who is the owner of user dictionary A explicitly calls this processing. First, the user dictionary registration unit 23 takes out registration candidates from the registration candidate storage unit 32A and inspects the contents. Here, consider the case where the contents of the registration candidate storage unit 32A are as shown in FIG. The user dictionary registration means 24 takes out two dictionary data of “gene” and “gene diagnosis” from here, checks whether these dictionary data are out of registration, and selects “gene” which is not out of registration. It is selected as a final registration candidate to be inquired about whether or not registration is necessary for the user.

Further, the user dictionary registration unit 24 displays registration information corresponding to “gene” on the output device 4 and inquires of the user A whether or not to register. FIG. 6 shows an example of an inquiry screen for the user. In the upper part of the screen, the text “The following dictionary data has been detected as a candidate for registration in the user dictionary. Do you want to register it in the user dictionary?” As a display to ask whether to register the candidate in the user dictionary. A column 61 is displayed. The list box 63 displays registration candidate dictionary data and radio buttons 62a, 62b, and 62c for the user to input an answer to the question. Buttons 64 and 65 for confirming or canceling the input are displayed at the bottom of the screen.
When the user A checks the “Register” radio button 62a of “Gen” and presses the “Execute” button 64, the user dictionary registration unit 23 registers the dictionary data of “Gen” in the user dictionary 31A. Then, the “gene” dictionary data in the registration candidate storage unit 32A is deleted.
When the user A checks the radio button 61b of “not registered” and presses the “execute” button 64, the user dictionary registration unit 23 is not subject to registration of the dictionary data of “gene” in the registration candidate storage unit 32A. Write “Not applicable” information in the column.
When the user A checks the “pending” radio button 62c and presses the “execute” button 64, or presses the “cancel” button 65 whatever the radio button is checked, the user dictionary registration means 23 does nothing.

Although FIG. 6 shows an example in which a radio button is used for a user interface for specifying whether to register, a check box may be used as shown in FIG. In the case of FIG. 7, if the check box 66 for the dictionary data “gene” is checked, the user dictionary registration means 23 registers the dictionary data for “gene” in the user dictionary 31A, and stores it in the registration candidate storage unit 32A. Delete the “Gen” dictionary data.
When the dictionary data “gene” check box 66 is not checked, the user dictionary registration unit 23 writes the information “not subject” to the non-registration column of the dictionary data “gene” in the registration candidate storage unit 32A. .

Next, a second specific operation example of the natural language processing system 10 will be described.
In this example, it is assumed that the natural language processing means 24 performs machine translation processing in particular, and each registration candidate is extracted from the two user dictionaries 31A and 31B in FIG. 1, and the dictionaries included in the registration candidates are extracted. An operation of registering data in the user dictionary 31A and the user dictionary 31B will be described.
FIG. 8A shows the contents of the user dictionary 31A, and FIG. 8B shows the data contents of the user dictionary 31B. The contents of the two user dictionaries, user dictionary 31A and user dictionary 31B, are shown in tabular form as in FIGS. 4 (a) and 4 (b). Further, classification information is recorded for each dictionary data. Has been. Dictionary data set with classification information “gene related” is registered in the user dictionary 31A, and dictionary data set with classification information “gene basis” and classification information “gene application” are registered in the user dictionary 31B. Dictionary data set with is registered.
Here, as the classification information, for example, information set for the purpose of selectively using each dictionary data in the user dictionary according to the scene may be used.

  In the process of extracting registration candidates, the similarity calculation means 21 calculates the similarity between the dictionary data sets included in the user dictionary 31A and the user dictionary 31B. The similarity calculation means 21 divides each user dictionary into a set of dictionary data having the same classification information, and takes out the set of dictionary data sets. Here, from the user dictionary 31A, a “gene related” dictionary data set consisting of ten dictionary data whose classification information is “gene related” can be extracted, and from the user dictionary 31B, the classification information is “gene basis”. It is possible to extract a “gene basis” dictionary data set including nine dictionary data and a “gene application” dictionary data set including two dictionary data whose classification information is “gene application”.

  Next, the similarity calculation means 21 calculates the similarity for each set of dictionary data sets. For example, using the similarity calculation method shown in the first specific operation example, the similarity between the “gene related” dictionary data set and the “gene basis” dictionary data set is 8/11 = 0.73, “gene The similarity between the “related” dictionary data set and the “gene application” dictionary data set is 1/11 = 0.09.

Next, the registration candidate extraction unit 22 compares the similarity output from the similarity calculation unit 21 with a threshold value. If the threshold value is 0.7, the similarity between the “gene related” dictionary data set and the “gene basis” dictionary data set is equal to or greater than this value, and dictionary data to be shared is extracted from this set.
Here, first, for the user dictionary A, the “gene” dictionary data included in the “gene basis” dictionary data set and not included in the “gene related” dictionary data set should be extracted and registered in the user dictionary 31A. It is stored in the registration candidate storage unit 32A, which is a storage area for registration candidates. For the user dictionary 32B, dictionary data of “transposon” included in the “gene related” dictionary data set and not included in the “gene basis” dictionary data set is extracted, and registration candidates to be registered in the user dictionary 32B are extracted. It stores in the registration candidate storage part 32B which is a storage area. 9A and 9B show examples of the registration candidate storage unit 32A and the registration candidate storage unit 32B obtained as a result of these processes. Similar to FIGS. 5A and 5B, the extracted dictionary data and information indicating whether or not the dictionary data is not registered are recorded.

In the process of extracting registration candidates, by repeating this process for all combinations of all dictionary data sets of user dictionaries stored in the user dictionary storage unit 31, dictionary data candidates to be registered in the respective user dictionaries are obtained. The extracted dictionary data is stored in an area corresponding to each of the registration candidate storage units 32.
Note that the process of registering the extracted dictionary data is the same as in the case of the first specific operation example, and thus description thereof is omitted.

As described above, in the natural language processing system 10, the similarity calculating unit 21 calculates the similarity between individual user dictionaries, and the registration candidate extracting unit 22 is to share the dictionary data based on the similarity. Therefore, it is possible to flexibly find an appropriate sharing partner for each user dictionary.
Moreover, since the similarity calculation means 21 calculates the similarity between individual user dictionaries each time, the most suitable sharing partner can be found at that time. For this reason, dictionary data to be shared can be appropriately extracted.

The present invention can also be implemented as a program that causes a computer to execute each function of the natural language processing system 10.
Such an embodiment is shown in FIG.
The computer 8 includes an input device 1, a storage device 3 such as a hard disk device, an output device 4, a main storage device 7 such as a RAM (Random Access Memory), and a CPU 5 having a function for controlling each of the above devices and an arithmetic function. I have.
The natural language processing program 6 stored in the main storage device 7 is read by the CPU 5 and executed by the CPU 5 to control the operation of the computer 8, and the user dictionary storage unit 31 and the registration candidate storage unit 32 are generated in the storage device 3. To do. The natural language processing program 6 causes the CPU 5 to operate as the data processing device 2 in FIG. 1 and causes the computer 8 to operate as the natural language processing system 10.

  According to the present invention, a kana-kanji conversion device that converts an input kana character string into a kanji-kana mixed character string, or a machine translation that converts an input first language character string into a second language character string The present invention can be applied to applications such as an apparatus, a speech recognition device that converts an input speech signal into a character string, and a program for realizing a speech synthesis device that converts an input character string into a speech signal. Further, the present invention can be applied to a use of a program for realizing a dictionary creation support apparatus that supports creation of a dictionary used in natural language processing on a computer.

It is a functional block diagram of the natural language processing system which is 1st Embodiment of this invention. It is a flowchart which shows the registration candidate extraction operation | movement of a natural language processing system. It is a flowchart which shows the dictionary data registration operation | movement of a natural language processing system. It is a figure which shows the example of a data structure of a user dictionary. It is a figure which shows the example of a data structure of a registration candidate memory | storage part. It is a figure which shows an example of the registration confirmation screen of dictionary data. It is a figure which shows an example of the registration confirmation screen of dictionary data. It is a figure which shows the example of a data structure of a user dictionary. It is a figure which shows the example of a data structure of a registration candidate memory | storage part. It is a figure which shows embodiment by a computer program.

Explanation of symbols

1 Input Device 2 Data Processing Device 3 Storage Device 4 Output Device 5 CPU
6 Natural language processing program 7 Main storage device 8 Computer 10 Natural language processing system 21 Similarity calculation means 22 Registration candidate extraction means 23 User dictionary registration means 24 Natural language processing means 31 User dictionary storage section 32 Registration candidate storage section

Claims (13)

In a natural language processing system for performing natural language processing using a user dictionary storing dictionary data including a word and language information corresponding to the word,
Similarity calculating means for calculating the similarity between a first user dictionary that is a user dictionary of a partner to share dictionary data and a second user dictionary that is a user dictionary to which dictionary data is registered;
When the similarity is equal to or higher than a predetermined threshold, dictionary data included in the first user dictionary and not included in the second user dictionary is registered as a registration candidate for the second user dictionary. Registration candidate extraction means for extracting;
A natural language processing system comprising: user dictionary registration means for registering dictionary data included in the registration candidates in the second user dictionary.   The similarity calculation means is registered in common with the total number of dictionary data registered in the first user dictionary and the second user dictionary and in the first user dictionary and the second user dictionary. The natural language processing system according to claim 1, wherein the similarity is calculated based on a ratio to the number of dictionary data.   The similarity calculation means is similar to the processing target of the natural language processing performed in the past using the first user dictionary and the processing target of the natural language processing performed in the past using the second user dictionary. The natural language processing system according to claim 1, wherein the similarity is calculated based on a processing target similarity that is a degree. The dictionary data includes classification information;
The similarity calculation means is stored in the second user dictionary and the first dictionary data set composed of dictionary data having the same classification information in the dictionary data stored in the first user dictionary. Calculating the similarity based on the dictionary data set similarity which is the similarity of the second dictionary data set consisting of dictionary data having the same classification information in the dictionary data;
The registration candidate extraction unit extracts dictionary data included in the first dictionary data set and not included in the second dictionary data set as the registration candidates. Natural language processing system.   The similarity calculation means is common to the total number of dictionary data registered in the first dictionary data set and the second dictionary data set, and to the first dictionary data set and the second dictionary data set. The natural language processing system according to claim 4, wherein the dictionary data set similarity is calculated based on a ratio with the number of dictionary data registered in the above.   The similarity calculation means includes a processing target for natural language processing performed in the past using the first dictionary data set and a processing target for natural language processing performed in the past using the second dictionary data set. The natural language processing system according to claim 4, wherein the dictionary data similarity is calculated based on a processing target similarity that is a similarity of In a natural language processing method for performing natural language processing using a user dictionary storing dictionary data including a word and language information for the word,
A first user dictionary that is a user dictionary of a partner to which dictionary data is shared and a second user dictionary that is a user dictionary to which dictionary data is registered are read from the storage device, and the first user dictionary and the second user dictionary are read out. A similarity calculation step of calculating the similarity between the user dictionary and
Operates when the similarity is greater than or equal to a predetermined threshold, and registers dictionary data included in the first user dictionary and not included in the second user dictionary in the second user dictionary A registration candidate extraction step of extracting as a candidate and recording the registration candidate in a storage device;
A natural language processing method comprising: a user dictionary registration step of reading the registration candidates from the storage device and registering dictionary data included in the registration candidates in the second user dictionary. In a natural language processing program for performing natural language processing using a user dictionary storing dictionary data including a word and language information for the word,
On the computer,
A first user dictionary that is a user dictionary of a partner to which dictionary data is shared and a second user dictionary that is a user dictionary to which dictionary data is registered are read from the storage device, and the first user dictionary and the second user dictionary are read out. A similarity calculation function for calculating the similarity between the user dictionary and
When the similarity is equal to or higher than a predetermined threshold, dictionary data included in the first user dictionary and not included in the second user dictionary is extracted as a registration candidate for the second user dictionary. A registration candidate extraction function for recording the registration candidates in a storage device;
A natural language processing program that reads a registration candidate from the storage device and executes a user dictionary registration function for registering dictionary data included in the registration candidate in the second user dictionary.   The similarity calculation function includes a total number of dictionary data registered in the first user dictionary and the second user dictionary and a common registration in the first user dictionary and the second user dictionary. The natural language processing program according to claim 8, wherein the similarity is calculated based on a ratio to the number of dictionary data.   The similarity calculation function is similar to a processing target of natural language processing performed in the past using the first user dictionary and a processing target of natural language processing performed in the past using the second user dictionary. The natural language processing program according to claim 8, wherein the similarity is calculated based on a processing target similarity that is a degree. The dictionary data includes classification information;
The similarity calculation function is stored in the first dictionary data set consisting of dictionary data having the same classification information in the dictionary data stored in the first user dictionary and in the second user dictionary. Calculating the similarity based on the dictionary data set similarity which is the similarity of the second dictionary data set consisting of dictionary data having the same classification information in the dictionary data;
The registration candidate extraction function is for extracting dictionary data included in the first dictionary data set and not included in the second dictionary data set as the registration candidates. The natural language processing program according to 8.   The similarity calculation function is common to the total number of dictionary data registered in the first dictionary data set and the second dictionary data set, and to the first dictionary data set and the second dictionary data set. 12. The natural language processing program according to claim 11, wherein the dictionary data set similarity is calculated based on a ratio with the number of dictionary data registered in this way.   The similarity calculation function includes a processing target of natural language processing performed in the past using the first dictionary data set and a processing target of natural language processing performed in the past using the second dictionary data set. The natural language processing program according to claim 11, wherein the dictionary data similarity is calculated based on a processing target similarity which is a similarity of

Images