JP2005339043A - Natural language processing method, natural language processing device, natural language processing program, and recording medium with the program recorded thereon - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a natural language processing method allowing extraction of information with expression easily accessible to a user, a natural language processing device, a natural language processing program, and a recording medium with the program recorded thereon. <P>SOLUTION: This natural language processing method for carrying out language analysis on a text to divide it into morphemes for extracting information from the text includes a point increase/reduction procedure for increasing/reducing points about activity of a morpheme and a point storage procedure storing the points of the morpheme increased/reduced by the point increase/reduction procedure. Referring to the points stored in the point storage procedure, information is extracted from the text based on the dimension of the points. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a text mining technique capable of analyzing text data, and in particular, a natural language processing method, a natural language processing apparatus, a natural language processing program, and a natural language processing capable of extracting information with a user-friendly expression. The present invention relates to a recording medium on which a program is recorded.

  Conventionally, text mining for analyzing a large amount of text data such as questionnaire data or data sent to a call center is known. By using text mining, users' complaints and opinions, questionnaire free text responses, and writing on bulletin boards on the Internet can be analyzed.

  When performing analysis using text mining, extracting a characteristic concept included in a document set is one of the major issues. Research on information extraction technology including conceptual information is progressing as a method of finding some knowledge from a large amount of text data. If useful information expressing the subject, topic, writer's intention, and the like can be extracted from the text data, applications such as document search, document analysis, and document analysis based on the subject can be realized.

  In the text mining technology, a document category of analyzed text data is often expressed in terms of representative terms, or keywords of important sentences are extracted, and the results of analysis are often expressed using terms. In such a representative term selection process, statistical information such as the appearance frequency of terms in a document / document collection is generally used mainly by a natural language processing method using a term dictionary such as morphological analysis. is there.

However, even if information is extracted with terms having a high appearance frequency, the extracted information is difficult for some users if the term is a technical term or a term that can be used only in a part. On the other hand, a so-called paraphrasing technique has been proposed as a method for supporting the understanding of the meaning of difficult terms (for example, see Non-Patent Document 1). Paraphrasing technology refers to replacing a term with another term or writing it together. If paraphrasing technology is used, difficult terms can be expressed in different words, so that the extracted information can be easily understood.
http://www.pluto.ai.kyutech.ac.jp/NLP/guide/news/file/525.html“Interactive Paraphrase ”, 7th Annual Workshop of the Language Processing Society of Japan ((March 2001 26 to 30), Ryuichiro Higashinaka, Satoshi Nagao)

  However, since information useful to the user varies from user to user, it is unclear whether or not the paraphrased term is unknown or known to the user or how well known. From the user's point of view, for example, even if it is expressed by unknown terms or terms that are not normally used, it does not seem to be very useful information. Conversely, terms that are frequently used by users are more useful. It can be felt. Therefore, the paraphrasing technique has a problem that the expression may be difficult to understand for the user.

  In view of the above problems, the present invention provides a natural language processing method, a natural language processing apparatus, a natural language processing program, and a recording medium on which a natural language processing program is recorded, which can extract information with a user-friendly expression. With the goal.

  In view of the above problems, the present invention relates to a natural language processing method that linguistically analyzes a text, divides the text into morphemes, and extracts information from the text. And a point storage procedure for storing the points of the morpheme increased or decreased by the step, and by referring to the points stored in the point storage procedure, information is extracted from the text based on the size of the points. According to the present invention, it is possible to provide a natural language processing method capable of extracting information with a user-friendly expression by using a morpheme having a large point related to activity.

  Further, in one form of the natural language processing method of the present invention, there is provided a dictionary that is referred to when text is linguistically analyzed, and a morpheme reference date storage procedure that stores the date and time when the morpheme of the dictionary is referenced for each morpheme. The point increase / decrease procedure compares the first date / time stored by the morpheme reference date / time storage procedure with the second date / time when the previous point increase / decrease is performed, and the first date / time is greater than the second date / time. If new, the morpheme point is increased by a predetermined amount, and if the first date and time is not newer than the second date and time, the morpheme point is decreased by a predetermined amount. ADVANTAGE OF THE INVENTION According to this invention, the point which concerns on the activity of the morpheme which is frequently referred to a dictionary can be enlarged.

  Further, in one form of the natural language processing method of the present invention, there is a synonym information reference procedure for referring to information related to synonyms of morphemes obtained by dividing text by language analysis, and the point increase / decrease procedure refers to synonym information The point of the synonym referred to by the procedure is increased and stored by the point storage procedure. According to the present invention, it is possible to increase points related to activity even in synonyms of morphemes referenced in a dictionary.

  In one embodiment of the natural language processing method of the present invention, the method further comprises a determination threshold value specifying procedure for specifying a determination threshold value for determining whether or not to increase the point of the synonym. The determination threshold value specified by the procedure is compared with the synonym points stored in the point storage procedure, and the point of the synonym is increased when the determination threshold value is smaller. According to the present invention, even when referred to as a synonym, when the point related to activity is lower than the determination threshold, the point is not increased, and thus it is possible to prevent information from being extracted with a morpheme having a small point related to activity. . Note that the determination threshold value may be any value as long as the degree of activity is known, and may be, for example, a date and time when a dictionary is referenced. When the date / time is set as a threshold, it is compared with the first date / time stored by the morpheme reference date / time storing procedure.

  Further, the present invention provides a natural language processing device that linguistically analyzes text, divides it into morphemes, and extracts information from the text, and is increased / decreased by point increasing / decreasing means for increasing / decreasing points related to morpheme activity. Point storage means for storing the points of the morpheme, and referring to the points stored in the point storage means, and extracting information from the text based on the size of the points. ADVANTAGE OF THE INVENTION According to this invention, the natural language processing apparatus which can extract information by an expression easy to understand for a user can be provided by using a morpheme with the big point concerning activity.

  The natural language processing apparatus of the present invention has a dictionary that is referred to when text is linguistically analyzed, and a morpheme reference date storage unit that stores the date and time when the morpheme of the dictionary is referenced for each morpheme, The increase / decrease means compares the first date / time stored in the morpheme reference date / time storage means with the second date / time when the previous point was increased / decreased, and if the first date / time is newer than the second date / time, A morpheme point is increased by a predetermined amount, and if the first date and time is not newer than the second date and time, the morpheme point is decreased by a predetermined amount.

  The natural language processing apparatus of the present invention further includes synonym information reference means for referring to information related to morpheme synonyms obtained by dividing text by language analysis, and the point increase / decrease means is referred to by the synonym information reference means. The points of the synonyms are increased and stored by the point storage means.

  The natural language processing apparatus of the present invention further includes a determination threshold value specifying means for specifying a determination threshold value for determining whether or not to increase the point of the synonym, and the point increase / decrease means is specified by the determination threshold value specifying means. The determined determination threshold value is compared with the synonym point stored in the point storage means, and when the determination threshold value is smaller, the point of the synonym is increased.

  The present invention also provides a natural language processing program that causes a computer to execute the above natural language processing method. The present invention also provides a recording medium on which a natural language processing program for causing a computer to execute a natural language processing method is recorded.

  It is possible to provide a natural language processing method, a natural language processing apparatus, a natural language processing program, and a recording medium on which a natural language processing program is recorded, in which information can be extracted with a user-friendly expression.

  The best mode for carrying out the present invention will be described below with reference to the drawings.

FIG. 1 shows an example of a functional configuration diagram of a natural language processing apparatus. The natural language processing apparatus of FIG. 1 includes text data input means 11, language analysis means 12, term activity calculation means 13, term dictionary 14, term activity storage means 15, term reference time measurement means 16, and term reference time storage means. 17 is configured. Hereinafter, each means in FIG. 1 will be described in detail.

[Text data input means 11]
The text data input unit 11 inputs text data to be subjected to language analysis. It is possible to input using a keyboard, or a file in which text data is already recorded may be input. In addition, you can input by various methods such as scanning, copy and paste. In this embodiment, the minimum unit constituting the text is referred to as a term, and the minimum unit constituting the text divided by linguistic analysis is referred to as a morpheme. Although both are equivalent, in this embodiment, they are referred to as morphemes when divided from text, and otherwise (when stored in a term dictionary, when representing activity, when representing synonyms, etc.) , Terminology.

[Language analysis means 12]
The language analysis means 12 performs language analysis on the input text data. In the natural language processing according to the present embodiment, additional information useful for the user is given regarding morphemes obtained as a result of language analysis of text data. Therefore, the language analysis unit 12 performs language analysis and divides the text data into morphemes using a technique such as morphological analysis or phrase dependency analysis. The language analysis unit 12 includes a term dictionary 14 and performs morphological analysis using the term dictionary 14. In the following embodiment, an arbitrary natural language processing method using the term dictionary 14 is processed, and processing is performed by morphological analysis for a Japanese document. In this embodiment, an example of improving the morpheme analysis algorithm itself is not shown, but a function can be added to the morpheme analysis algorithm itself by a simple application of this embodiment.

  For the sake of simplicity in this embodiment, the function of morpheme analysis is modularized. When an arbitrary character string is input, the identification number of the morpheme constituting the character string is displayed in a list format (one-dimensional list of multiple rows × one column). Format). That is, the language analysis unit 12 has a morpheme analysis module, and the text data is divided into morphemes by the morpheme analysis module.

  The morphological analysis process will be briefly described. The morpheme analysis is a process of dividing a given character string (text data) into morphemes (minimum units of words) based on a prepared term dictionary. For example, FIG. 2 shows a result of morphological analysis performed on a character string “Take off footwear there”. In the result of the morpheme analysis shown in FIG. 2, the symbol “/” indicates a morpheme break. According to the processing result of the morpheme analysis in FIG. 2, the character string “Take off footwear there” is divided into five morphemes. Normally, in morpheme analysis, information such as part of speech and basic type can be obtained for each morpheme. However, in this example, morpheme analysis is based on the basic type of morpheme and the identification number in the dictionary. What is necessary is just to obtain. Also, the internal algorithm and terminology dictionary used are not limited (for details on morphological analysis, see “Information Retrieval and Language Processing”, Takenobu Tokunaga, University of Tokyo Press, etc.).

[Term reference time measuring means 16]
The term reference time measuring means 16 measures the date and time (hereinafter simply referred to as “time”) when the language analyzing means 12 refers to the term dictionary 14. FIG. 3 shows an example of pseudo code of processing for performing morphological analysis as natural language processing and measuring the time when the term reference time measuring means 16 refers to the term dictionary 14. Note that the processing surrounded by “*” (steps S1 and S2) need only be executed once if it is not updated to the term dictionary 14 constituting the morphological analysis.

  A description will be given along the pseudo code of FIG. First, it is assumed that the morphemes constituting the term dictionary 14 and their identification numbers are as shown in FIGS. 4 (a) and 4 (b). For example, the identification number of the morpheme “where” is “1222”, and the identification number of the morpheme “shoes” is “79550”.

  In step S1, the language analysis unit 12 outputs the identification number to the list IDL. As shown in FIG. 3C, the list IDL has the same identification number as that in FIG. That is, all the terms in the term dictionary 14 are output as identification values in the list IDL. Next, in step S2, a list TL having the same number of elements as the list IDL is generated, and an initial value −1 is substituted as shown in FIG.

  Next, in step S3, a character string S “Take off footwear there” is input to the morphological analysis module. Next, in step S4, the morphological analysis module divides the character string S into the morphemes shown in FIG.

  In step 5, the language analysis means 12 outputs the morpheme of the character string S to the list RIDL with the identification number. FIG. 5 shows an example of the list RIDL. In each morpheme of the character string S “Take off footwear there”, the identification number “There” is 1223, “De” is 7866, “Footwear” is 65433, “O” is 7867, and “Take off” is 123405. It becomes.

  In step S6, the term reference time measuring means 16 acquires the time when the analysis of the morpheme is completed in the variable CT. Next, the element value of the list TL corresponding to the identification number of the element value of the list RIDL is replaced with the time stored in the variable CT (steps S7 to S9). Thereby, the time referring to the term dictionary 14 is stored in the list TL. The input time can be in any format, but for example, the arithmetic of the time can be done by using an integer representation of the time as the number of seconds that have elapsed since the reference time, such as that adopted by the UNIX (registered trademark) operating system. Processing becomes easy. For example, if the end time of the morpheme analysis processing as an integer expression is given as CT = 20034567 (April 5, 2003 6:07), 20034567 is stored in a predetermined element of the list TL. In this embodiment, a one-dimensional list is used as the list TL for simplicity. However, by using a multi-dimensional list in which each element can have a plurality of values as the list TL, morpheme reference is performed. It is possible to obtain information having a tendency of the designated time. When processing is completed for all element values of the list RIDL, the list TL has element values as shown in FIG.

  The term reference time storage means 17 stores the reference times of terms included in the term dictionary 14 obtained by processing the pseudo code of FIG. 3 in a format like a list TL. Further, the term reference time measurement storage means 17 refers to a morpheme reference date storage means in the claims of the present application.

[Term activity calculating means 13]
The term activity level calculation means 13 calculates the activity level of the text data term (morpheme) analyzed by the language analysis means 12. First, the term activity will be described. The degree of activity is an index indicating how much contact the user has with the morpheme. The activity is a point related to the activity in the claims of the present application, and the term activity calculation means 13 is a point increase / decrease means.

  As an example, a case will be described in which morpheme activity is given in advance and a morpheme representing a predetermined document set is extracted. The activity of the morpheme is calculated based on information obtained from a document recently created by the user. First, as representative morphemes of a predetermined document set, three morphemes “analysis”, “constraint”, and “document” are obtained, and the normalized appearance frequency in the document set of each morpheme is 0.4, It is assumed that the activity is 0.2, 0.6, and the activity is 0.3, 0.6, 0.2. Furthermore, “restriction”, “restriction”, and “restriction” are given as synonyms for “restriction”, and the respective activities are assumed to be 0.5, 0.1, and 0.2.

  Here, when ordering the morphemes that represent a given document set, the order of "document", "analysis", and "constraint" will be based on the normalization frequency. "," Analysis "," document ". In other words, when ordering is based on the degree of activity, the representative morphemes included in the document set are ordered according to the degree of association with the documents that the user has recently created, and the representative morphemes are selected according to the morphemes that are frequently contacted by the user. Can express. Therefore, by calculating and storing the degree of activity, additional information useful for the user can be given to the processing result of the natural language processing method and higher-level processing including the natural language processing.

  Consider paraphrasing morphemes called "constraints" with synonyms. For example, when taking the procedure of presenting synonyms to the user and letting the user select them, the morphemes (synonyms) are presented in the order of the activity of the obtained morphemes, thereby giving the user assistance during selection. Can do.

In this example, the term activity is defined as follows.
・ The initial value of activity is 0.5.
・ The decrease constant is k. However, k is an integer of 1 or more.
-Decrease the activity by 1 / k for terms that are not referenced in the term dictionary 14 after the previous activity calculation time. However, when the activity is less than 0, it is set to 0.
Add 1 / k to the degree of activity for the term referenced in the term dictionary 14 after the previous calculation time. However, if the activity exceeds 1, set to 1.

  According to the definition of the activity level, the previous activity level calculation time is held in a predetermined storage means, and when the time referenced in the term dictionary 14 is newer than the previous activity level calculation time, The term activity increases by 1 / k. Conversely, when the time referred to in the term dictionary 14 is older than the previous activity level calculation time, the activity level of the term decreases by 1 / k. That is, the activity of a frequently referenced term increases, and the activity of a less frequently referenced term decreases.

  The term activity calculation means 13 will be described. To calculate the activity of the term, the activity of the term already stored in the term activity storage unit 15 described later and the reference time of the term (morpheme) stored in the term reference time storage unit 17 are used. To do. As the term reference time, the reference time measured by the term reference time measuring unit 16 may be used as it is.

  The timing for calculating the degree of activity is not particularly defined, that is, the acquisition of the term reference time and the calculation of the term activity in the term dictionary 14 performed by the term reference time measuring means 16 do not have to be synchronized. Good. Therefore, the activity of the term may be calculated at regular intervals, or may be calculated by an instruction from a predetermined process. In the present embodiment, the term activity is calculated periodically.

  The process for calculating the term activity will be described based on the pseudo code of the operation process of FIG. In the pseudo code of FIG. 6, the processing is continued with reference to the list of FIG. In addition, the process surrounded by * need only be executed once unless the activity of the term is initialized.

  In step S11, the term activity calculation means 13 creates an activity storage list AL having the same length as the list IDL, and inputs an initial value of 0.5. Thereby, the activity storage list AL of FIG. 4F is obtained. In step S12, an appropriate value as the decrease constant k is substituted into the variable DV.

  In step S13, the previous calculation time of activity is read and substituted into the variable PST. Next, the processing after step S14 is repeated for each element of the list TL. First, when the element value of TL is larger (newer) than PST (step S15), 1 / k is added to the corresponding element value of the list AL (step S16). If the result of adding 1 / k is greater than 1, the element value of the list AL is set to 1 (steps S17 and S18).

  When the element value of TL is not larger (older) than PST (step S19), 1 / k is subtracted from the corresponding element value of the list AL (step S20). As a result of subtracting 1 / k, if the value is smaller than 0, the element value of the list AL is set to 0 (steps S21 and S22).

  If the processing of steps S13 to S22 is performed for all elements of the list TL, the current time is held in a predetermined storage means as the activity calculation time in step S23.

  For example, in the pseudo code of FIG. 6, the activity calculation time assigned to the variable PST is the time when the language analysis is started (a predetermined date before 6:07 on April 5, 2003), and the decrease constant k is When the activity is calculated as 10, the result shown in FIG. 4 (g) is obtained. In other words, in the list TL of FIG. 4E, the element value of the list AL corresponding to the morpheme that is not referred to (having an element value of −1) is 0.4, and is referenced (the element of 20034567) The element value of the list AL corresponding to the morpheme (having a value) is 0.6.

  In this embodiment, the degree of activity is calculated by a simple method, but, for example, the tf / idf method (TF (Term Frequency) / IDF (Inverted Document Frequency)) often used in document search or the like. A morpheme that repeatedly appears in the same document. Applying a method that evaluates the importance (activity) of terms) by combining two indicators that represent the importance of morphemes that are important and important because morphemes with a small number of appearing documents are useful for narrowing down documents Thus, a more useful activity can be calculated. In this embodiment, for simplicity, a one-dimensional list is adopted as the list AL. However, by adopting a multidimensional list in which each element can have a plurality of values, the list AL can be activated. Information representing the transition state of the degree can be held.

[Terminology Activity Storage Unit 15]
The term activity storage unit 15 stores the activity of each term included in the term dictionary 14 calculated by the term activity calculation unit 13 in an appropriate format. The term activity storage means 15 is a point storage means in the claims of the present application.

  As described above, according to the present embodiment, it is possible to calculate the activity of terms constituting a document such as text data. The degree of activity is an index indicating how much each morpheme is unknown to the user and how much it is known. By using the activity level, it is possible to order representative morphemes included in a predetermined document set according to the degree of association with a document that the user has recently created. Unlike normalization frequency, the frequency with which each user touches It can be expressed with a representative morpheme that is easy to understand. That is, when the result processed by natural language processing or the like is used by other processing means including the natural language processing, information can be extracted from the document with an arbitrary scale for each user. By extracting information based on how much the morpheme is known to the user, a paraphrasing technique with a high degree of freedom, a representative morpheme process with a high degree of freedom, and the like can be realized.

In the second embodiment, a natural language processing apparatus that calculates the degree of activity for synonyms of morphemes (terms) divided by language analysis will be described. FIG. 7 illustrates an example of a functional configuration diagram of the natural language processing apparatus according to the second embodiment. In FIG. 7, the same components as those in FIG. 1 are denoted by the same reference numerals, and the description thereof is omitted. The natural language processing apparatus of FIG. 7 differs from FIG. 1 in that it includes a synonym threshold value specifying means 21, a synonym extracting means 22, and a synonym selecting means 23. Hereinafter, each unit in FIG. 7 will be described in detail.

[Synonym extraction means 22]
The synonym extraction unit 22 extracts a synonym of the target term. When the synonym is extracted, the term activity calculation means 13 activates the synonym of the target term in the same manner as the target term when calculating the activity of the term referred to in the process of processing the text data. The degree calculation process can be performed. In this embodiment, there are no particular restrictions on the synonym extraction method and the similar dictionary used at that time, and any method can be used as long as the synonym can be extracted appropriately. In this embodiment, the synonym extraction means 22 has a synonym generation module that, when an identification number in the term dictionary 14 of a target term is input, the synonym identification numbers of the term are output as a list. The synonym extraction means 22 is synonym information reference means in the claims of the present application.

[Synonym threshold value specifying means 21, synonym selecting means 23]
The synonym threshold value specifying means 21 provides means for inputting a synonym threshold value. Further, the synonym selecting means 23 compares the activity level of the synonym with the threshold value specified by the synonym threshold value specifying means for the synonym extracted by the synonym extracting means 22, so that the synonym indicates the activity level. Determine if it is a good term.

  Since terms with low activity are considered to be less likely to be recognized by the user, increasing the activity of such terms is thought to result in a decrease in the performance of the indicator of activity. For this reason, the synonym selection means 23 determines whether or not the term may increase activity. Therefore, in this embodiment, the synonym threshold value specifying means 21 specifies the magnitude of the activity as the threshold value. The synonym threshold value specifying means 21 is a determination threshold value specifying means in the claims of the present application.

  The processing of the natural language processing device that extracts synonyms and calculates the activity will be described based on the pseudo code in FIG. The processing similar to that in FIGS. 3 and 6 will be briefly described.

  First, using the text data of FIG. 2 as the text data to be language-analyzed, the processing until the pseudo code processing of FIG. 8 is started will be briefly described. Assume that the morphemes constituting the term dictionary 14 and their identification numbers are as shown in FIGS. 9 (a) and 9 (b). When language analysis is started, the language analysis means 12 acquires the identification number of each term in the list IDL, and the list IDL is input with the same identification number as in FIG. 9B, as shown in FIG. 9C. . Next, a list TL having the same number of elements as the list IDL is generated, and an initial value −1 is substituted as shown in FIG. In the list TL, the time referred to in the term dictionary 14 is input by the term reference time measuring means 16, and the list TL is as shown in FIG. Next, the term activity calculation means 13 creates an activity storage list AL having the same length as the list IDL, and inputs an initial value of 0.5. As a result, the activity storage list AL of FIG. 9F is obtained.

  In step S31, the term activity calculation means 13 reads the synonym threshold value designated by the synonym threshold value designation means 21, and substitutes it into the variable TV. In this embodiment, as an example, 0.3 is substituted for TV as a synonym threshold.

  In step S32, the term activity level calculation means 13 reads the previous activity level calculation time and assigns it to the variable PST. Next, when the element value of the list TL is larger than PST, 1 / k is added to the corresponding element value of the list AL (steps S33 to S35). If the result of adding 1 / k is greater than 1, the element value is set to 1 (steps S36 and S37).

  In order to simplify the relationship of the nested structure, steps S46 to S50 will be described first. That is, if the element value of the list TL is not larger than PST, 1 / k is subtracted from the corresponding element value of the list AL (steps S46 and S47). As a result of subtracting 1 / k, if the value is smaller than 1, the element value is set to 0 (steps S48 and S49). When the processing of steps S31 to S49 has been performed for all elements of the list TL, the current time is stored in a predetermined storage means as the activity calculation time in step S50. The above processing is performed for all elements of the list TL. Note that the activity of a term extracted as a synonym is not reduced by 1 / k even if the element value of the list TL is not larger than PST.

  Steps S38 to S45 perform processing for calculating the activity of synonyms. In step S38, the language analysis unit 12 outputs the identification number stored in the list IDL to the synonym generation module. Note that the identification number output to the synonym generation module is the one whose term reference time is greater than PST as a result of the determination in step S34 (that is, the identification number of each morpheme in the text data in FIG. 2).

  The synonym generation module outputs the identification number of the synonym of the term having the identification number to the synonym selection means 23 (step S39). Thereby, a list SL as shown in FIG. 10 is obtained. If the identification number is known, the corresponding term can be extracted from the term dictionary 14, for example. In the SL list of FIG. 10, “shoes” and “shoes” are extracted as synonyms of “footwear” with respect to the text data of FIG. The terms in parentheses in FIG. 10 are shown for convenience.

  Next, the synonym selecting means 23 refers to the list AL for the term of the identification number shown in the list SL, and determines the magnitude of the activity of the term and TV (0.3). The determination result is output to the term activity calculation means 13 (steps S40 to S42). If the activity of the list AL is greater than TV, the term activity calculation means 13 adds 1 / k to the element value of the list AL corresponding to the synonym of the identification number (step S43). If the result of adding 1 / k is greater than 1, the element value of the list AL is set to 1 (steps S44 and S45). Thereby, the activity of the synonym whose activity is higher than a predetermined value can be selectively increased. When the process is completed for all elements of the list SL, the pseudo code process of FIG. 8 is terminated.

  FIG. 9G shows a list AL that has been processed with the pseudo code of FIG. Compared with FIG. 4G, the activities of “shoes” and “shoes” extracted as synonyms are different. That is, the initial value of the activity of the synonyms “shoes” and “shoes” is 0.5, which is larger than TV (0.3), so 1 / k (0.1) is added, and each 0.6 It has become. The other elements in FIG. 9G are the same as those in FIG.

  In this embodiment, a value corresponding to the degree of activity is adopted as the synonym threshold value, but time may be used as the threshold value. When time is used as the threshold, the value to be compared with the threshold is not the degree of activity, but the reference time is preferable.

  According to the present embodiment, it is possible to calculate the activity of not only a term divided into morphemes but also a synonym of the term. Therefore, in addition to the effects of the first embodiment, when performing paraphrasing, paraphrasing with a synonym having a high degree of activity, natural language processing with a higher degree of freedom such as a representative morpheme processing with a higher degree of freedom can be realized.

In the third embodiment, a program for realizing the natural language processing described in the first or second embodiment (hereinafter referred to as a natural language processing program) and its recording medium will be described. FIG. 11 shows an example of a hardware configuration diagram of a computer that executes a natural language processing program.

  The hardware configuration diagram of FIG. 11 includes a CPU 31, an input / output device 32, a drive device 33, a communication device 34, a main storage device 35, a storage device 36, and a display device 37 that are mutually connected by a bus. Is done.

  The CPU 31 comprehensively controls processing performed by the computer. The input / output device 32 includes a keyboard and a mouse, and is used to input various operation instructions from the user. The communication device 34 is an interface for connecting to a network such as the Internet or a LAN, and includes, for example, a modem or a router. The main storage device 35 is a storage area for temporarily storing an operating system, programs, and data. The storage device 36 is installed with a program that realizes the natural language processing of the first or second embodiment. The drive device 33 can be inserted into a DVD-ROM or CD-ROM, can read programs and data from the recording medium 38, and can write programs and data to the recording medium 38. The display device 37 forms a GUI (Graphical User Interface) screen and displays various windows and data necessary for operation.

  The natural language processing program includes the text data input means 11, the language analysis means 12, the term activity calculation means 13, the term activity storage means 15, the term reference time measurement means 16, and the term reference time storage means 17 shown in FIGS. The synonym threshold value specifying means 21, the synonym extracting means 22, and the synonym selecting means 23 are executed by a computer. As an example, the pseudo code described with reference to FIGS. 3, 6, and 8 is coded in a predetermined program language and compiled.

  The natural language processing program and the term dictionary 14 are provided by a recording medium 38 such as a DVD-ROM. The recording medium 38 on which the natural language processing program and the term dictionary 14 are recorded is set in the drive device 33 and installed in the storage device 36.

  The storage medium 38 on which the natural language processing program and the term dictionary 14 are recorded is information such as a CD-ROM, DVD-ROM, flexible disk, magneto-optical disk (MO), etc. It is possible to use various types of recording media such as a recording medium for recording information, a semiconductor memory for electrically recording information such as a ROM and a flash memory. Further, the natural language processing program and the term dictionary 14 may be downloaded via a network.

  According to the present embodiment, it is possible to provide a natural language processing program executed by a computer and a recording medium on which the natural language processing program is recorded.

It is an example of a functional block diagram of a natural language processing apparatus. It is an example of the result of having performed the morphological analysis with respect to the character string. It is an example of the pseudo code of the process which records the time when the term dictionary was referred. It is a figure which shows an example, such as a term dictionary, term reference time, the point which concerns on activity. It is an example of list RIDL. It is an example of the pseudo code of the process which calculates the point which concerns on the activity of a term. FIG. 10 is an example of a functional configuration diagram of a natural language processing apparatus according to a second embodiment. It is an example of the pseudo code of the process which calculates the point which concerns on activity of a synonym. It is a figure which shows an example in the Example 2, such as a term dictionary, term reference time, the point which concerns on activity. It is an example of list | wrist SL in which the identification number of the synonym was stored. It is an example of the hardware block diagram of the computer which executes a natural language processing program.

Explanation of symbols

DESCRIPTION OF SYMBOLS 11 Text data input means 12 Language analysis means 13 Term activity calculation means 14 Term dictionary 15 Term activity degree memory means 16 Term reference time measurement means 17 Term reference time storage means 21 Synonym threshold value designation means 22 Synonym extraction means 23 Synonym selection means 23

Claims (10)

In a natural language processing method that parses text into morphemes and extracts information from the text,
Point increase / decrease procedure for increasing / decreasing points related to morpheme activity;
A point storage procedure for storing morpheme points increased or decreased by the point increase / decrease procedure;
Referring to the points stored in the point storage procedure and extracting information from the text based on the size of the points;
A natural language processing method characterized by that. A dictionary to be referred to when language analysis of the text;
A morpheme reference date storage procedure for storing the date and time referring to the morpheme of the dictionary for each morpheme,
The point increase / decrease procedure compares the first date / time stored by the morpheme reference date / time storage procedure with the second date / time when the previous point increase / decrease was performed,
When the first date and time is newer than the second date and time, the morpheme points are increased by a predetermined amount, and when the first date and time is not newer than the second date and time, the morpheme points are increased by a predetermined amount. Decrease,
The natural language processing method according to claim 1. A synonym information reference procedure for referring to information related to synonyms of morphemes obtained by dividing text by language analysis,
The point increase / decrease procedure increases the point of the synonym referred to by the synonym information reference procedure, and stores it by the point storage procedure.
The natural language processing method according to claim 1 or 2, wherein A determination threshold value specifying procedure for specifying a determination threshold value for determining whether or not to increase the point of the synonym,
The point increase / decrease procedure compares the determination threshold specified by the determination threshold specification procedure with the synonym point stored in the point storage procedure, and if the determination threshold is smaller, the point of the synonym is Increase,
The natural language processing method according to claim 3. In a natural language processing apparatus that parses text into morphemes and extracts information from the text,
Point increase / decrease means for increasing / decreasing points related to the activity of the morpheme;
Point storage means for storing morpheme points increased or decreased by the point increase / decrease means;
Referring to the points stored in the point storage means, and extracting information from the text based on the size of the points;
A natural language processing apparatus characterized by that. A dictionary to be referred to when language analysis of the text;
Morpheme reference date storage means for storing the date and time referring to the morpheme of the dictionary for each morpheme each time,
The point increase / decrease means compares the first date and time stored by the morpheme reference date and time storage means with the second date and time when the previous point was increased or decreased,
When the first date and time is newer than the second date and time, the morpheme points are increased by a predetermined amount, and when the first date and time is not newer than the second date and time, the morpheme points are increased by a predetermined amount. Decrease,
The natural language processing apparatus according to claim 5. Synonym information reference means for referencing information related to synonyms of morphemes obtained by dividing text by language analysis,
The point increase / decrease means increases the point of the synonym referred to by the synonym information reference means, and stores it by the point storage means.
The natural language processing apparatus according to claim 5, wherein the natural language processing apparatus is a natural language processing apparatus. A determination threshold value specifying means for specifying a determination threshold value for determining whether to increase the point of the synonym;
The point increasing / decreasing means compares the determination threshold specified by the determination threshold specifying means with the synonym point stored in the point storage means, and if the determination threshold is smaller, the point of the synonym is determined. Increase,
The natural language processing apparatus according to claim 7.   A natural language processing program for causing a computer to execute the natural language processing method according to claim 1. A recording medium on which the natural language processing program according to claim 9 is recorded.

Images