JP4985096B2 - Document analysis system, document analysis method, and computer program - Google Patents

Description

  The present invention relates to a document analysis system, a document analysis method, and a computer program. More specifically, for example, a document analysis system and a document analysis method for performing processing for extracting a morpheme that is a classification word as a language unit to be applied to data retrieval or the like from a document in a highly specialized field such as a medical field And a computer program.

  For example, a search key applied in database search, an index as an index of a term dictionary, setting of document classification information, and the like, processing for extracting terms applied in data processing from natural language documents is necessary in various data processing fields. Technology. Language units applied to document processing, such as search keys for document search and indexes used for document classification, are called morphemes.

  Research on extracting morphemes from natural language documents has been conducted. For example, if it is a common document such as [car runs on road], it is possible to extract morphemes such as [car], [road], and [run] by applying a general morphological analysis system. . The morpheme analysis system is known as a system that applies a predetermined morpheme analysis rule to segment a sentence into morphemes that are semantic minimum units and perform a part-of-speech recognition process.

However, it is difficult to divide technical terms in highly specialized fields such as the medical field into appropriate morphemes. For example, a specific disease name
`` Scleroderma ''
If a standard morphological analysis process is performed on the original word representing the disease name, the word is divided into three words (morphemes) “strong (strong)”, “skin”, and “symptom”.

  Morphological analysis processing is a basic technology used in language processing technologies such as document search and document classification. For example, in the document classification, if “scleroderma” is divided into three words, the similarity between two documents including “scleroderma”, the document including “scleroderma”, and “strong” “ The similarity between the documents including the three words “skin” and “symptom” at distant positions becomes equal, leading to a decrease in the accuracy of document classification.

  In order to solve such a problem, it is necessary to prepare a list of technical terms in advance and add the technical term list to the morphological analysis dictionary used at the time of morphological analysis. Patent Document 1 (Japanese Patent Laid-Open No. 2004-151926) is a conventional technique that discloses a typical example of a technique for extracting technical terms from a text set of specialized fields in order to create such a technical term list. .

  In this patent document 1, attention is paid to the fact that technical terms are compound nouns in many cases, and the corrected appearance frequency (of the compound noun itself) calculated based on the appearance frequency of each morpheme (noun or affix) constituting the compound noun. We have proposed a method for determining whether a compound noun is a technical term from the scale of appearance frequency) and the length of the compound noun (number of morphemes constituting the compound noun).

The method disclosed in Patent Document 1, that is, the method for determining whether a compound noun is a technical term using the appearance frequency of the compound noun and the length of the compound noun as two parameters, has the following problems. There is a point.
(Problem 1) Problems when emphasizing the length parameter of a compound noun When a long compound noun is prioritized as a technical term, for example, for medical texts, the following compound noun is It will be extracted as technical terms.
(A) “Chronic diffuse tubular proliferative glomerular lymphoma”
(B) “Chronic tubular proliferative glomerular lymphoma”

  The compound nouns (A) and (B) are both technical terms and are technical terms expressing almost the same disease. However, in the above method, these two technical terms are extracted as two different technical terms. Therefore, for example, in the document classification, the similarity between the document including (A) and the document including (B) is not recognized at all, and the accuracy of the document classification is reduced. That is, a long character string is inappropriately extracted as a morpheme to be registered in the morpheme analysis dictionary.

(Problem 2) Problems when emphasizing the parameters of the appearance frequency of compound nouns When the compound nouns that appear frequently are prioritized as technical terms, long compound nouns rarely appear frequently. The problem 1) does not occur. That is, it is possible to extract a word having an appropriate length for registration in the morphological analysis dictionary. However, in this case, since a compound noun with a short length is likely to appear with a high frequency due to a writer's habit or the like, it is not always possible to extract only technical terms. For example, it happens to be a text set to be extracted,
“I will perform surgery tomorrow.”
"According to the previous inspection ..."
If such an expression appears frequently, “tomorrow's surgery” and “previous execution” will be extracted as technical terms. That is, in this case, the extraction accuracy of technical terms is significantly reduced.
JP 2004-151926 A

  The present invention has been made in view of the above-described problems, and is applied to document processing such as a search key for document search and an index used for document classification by analyzing technical terms in a highly specialized field such as the medical field. An object of the present invention is to provide a document analysis system, a document analysis method, and a computer program that efficiently and accurately extract morphemes as language units.

The first aspect of the present invention is:
A term extraction unit that executes a process for extracting a term in a document, performs an evaluation depending on the length of the term, and executes a term candidate extraction process that emphasizes the length;
Search the document set stored in the document database, count the number of occurrences of the same term as the term candidate extracted by the first term extraction means, and generate a term text including a copy of the term according to the number of occurrences. Configuration means;
A second term extracting means for performing an evaluation depending on the appearance frequency of the term with the term text generated by the term reconfiguring means being analyzed, and selecting a term having a high appearance frequency as an output term;
In a document analysis system characterized by having

  According to this configuration, after extracting the term with emphasis on length, a duplicate text is generated according to the appearance frequency of the term corresponding to the extracted term, and the term emphasizing frequency is executed based on the generated text. Therefore, it becomes possible to extract technical terms with high accuracy.

  Furthermore, in one embodiment of the document analysis system of the present invention, the first term extracting means is configured to execute a term extracting process for analyzing a document in a specific specialized field, for example, a medical field, The configuration means executes a process of counting the number of appearances by searching a document set stored in a document database storing documents of the specific specialized field, and the second term extracting means is a morpheme as a term in the medical field. It is the structure which performs extraction.

  According to this configuration, technical terms as compound nouns frequently found in specialized fields such as the medical field can be extracted with high accuracy.

  Furthermore, in one embodiment of the document analysis system of the present invention, the document analysis system further includes an updated term text obtained by deleting the term extracted by the second term extraction unit from the term text generated by the term reconstruction unit. A term text update unit that generates a term, and the second term extraction unit performs an evaluation depending on the appearance frequency of the term using the updated term text generated by the term text update unit as an analysis target, and a term having a high appearance frequency Is selected as an output term.

  According to this configuration, after length-oriented term extraction, a duplicate text corresponding to the appearance frequency of the term corresponding to the extracted term is generated, and frequency-oriented term extraction is performed based on the generated text. Since the updated term text is generated by deleting the terms extracted by the second term extraction means, the updated term text is selected for analysis, and the frequently occurring term candidates are selected as output terms. Can be obtained.

  Furthermore, in one embodiment of the document analysis system of the present invention, the document analysis system further selects a term having a combination configuration of other extracted terms from the terms extracted by the second term extracting means, and selects the selected terms. When the frequency of occurrence is equal to or less than a predetermined threshold value, term selection means is provided for executing processing for deletion from the output term.

  According to this configuration, after length-oriented term extraction, a duplicate text corresponding to the appearance frequency of the term corresponding to the extracted term is generated, and frequency-oriented term extraction is performed based on the generated text. It is possible to delete an inappropriately long term from the term extracted by the second term extracting means, and a term output having an appropriate length is realized.

Furthermore, the second aspect of the present invention provides
A document analysis method executed in the document analysis system,
A first term extraction step in which a first term extraction means for executing a term extraction process in a document performs an evaluation depending on the term length and executes a term candidate extraction process focusing on length;
A term including a copy of a term corresponding to the number of occurrences, wherein the term reconstructing means searches the document set stored in the document database, counts the number of occurrences of the same term as the term candidate extracted in the first term extraction step A term restructuring step to generate text;
A second term extraction step, wherein the second term extraction means performs an evaluation depending on the appearance frequency of the term with the term text generated in the term reconstruction step as an analysis target, and selects a term having a high appearance frequency as an output term;
The document analysis method is characterized by comprising:

  According to this configuration, after extracting the term with emphasis on length, a duplicate text is generated according to the appearance frequency of the term corresponding to the extracted term, and the term emphasizing frequency is executed based on the generated text. Therefore, it becomes possible to extract technical terms with high accuracy.

  Furthermore, in an embodiment of the document analysis method of the present invention, the first term extraction step is a step of executing a term extraction process for analyzing a document in a specific specialized field, for example, a medical field, The configuration step performs a process of counting the number of occurrences by searching a document set stored in a document database storing documents of the specific specialized field, and the second term extraction step includes the step of morpheme as a medical field term. It is a step for performing extraction.

  According to this configuration, technical terms as compound nouns frequently found in specialized fields such as the medical field can be extracted with high accuracy.

  Furthermore, in one embodiment of the document analysis method of the present invention, the document analysis method further includes a term text update unit that extracts the term extracted in the second term extraction step from the term text generated in the term reconstruction step. A term text update step for generating an updated term text from which the term is deleted, and the second term extraction step performs an evaluation depending on the appearance frequency of the term with the updated term text generated in the term text update step as an analysis target In this step, a term having a high appearance frequency is selected as an output term.

  According to this configuration, after length-oriented term extraction, a duplicate text corresponding to the appearance frequency of the term corresponding to the extracted term is generated, and frequency-oriented term extraction is performed based on the generated text. Since the updated term text is generated by deleting the terms extracted by the second term extraction means, the updated term text is selected for analysis, and the frequently occurring term candidates are selected as output terms. Can be obtained.

  Furthermore, in one embodiment of the document analysis method of the present invention, the document analysis method further selects a term having a combination configuration of other extracted terms from the terms extracted by the term selecting means in the second term extracting step. And a term selection step of executing a process of deleting from the output term when the appearance frequency of the selected term is equal to or less than a predetermined threshold value.

  According to this configuration, after length-oriented term extraction, a duplicate text corresponding to the appearance frequency of the term corresponding to the extracted term is generated, and frequency-oriented term extraction is performed based on the generated text. It is possible to delete an inappropriately long term from the term extracted by the second term extracting means, and a term output having an appropriate length is realized.

Furthermore, the third aspect of the present invention provides
A computer program for executing document analysis processing in a document analysis system;
A first term extraction step for causing a first term extraction means for performing a term extraction process in a document to perform an evaluation depending on a term length and to execute a term candidate extraction process with emphasis on length;
A term including a copy of a term corresponding to the number of occurrences by causing the term reconstructing means to search the document set stored in the document database, count the number of occurrences of the same term as the term candidate extracted in the first term extraction step A term restructuring step to generate text;
A second term extraction step for causing the second term extraction means to perform an evaluation depending on the appearance frequency of the term with the term text generated in the term reconstructing step as an analysis target, and to select a term having a high appearance frequency as an output term; ,
There is a computer program characterized by comprising:

  According to this configuration, after extracting the term with emphasis on length, a duplicate text is generated according to the appearance frequency of the term corresponding to the extracted term, and the term emphasizing frequency is executed based on the generated text. Therefore, it becomes possible to extract technical terms with high accuracy.

  The computer program of the present invention is, for example, a computer program that can be provided by a storage medium or a communication medium provided in a computer-readable format to a general-purpose computer system that can execute various program codes. . By providing such a program in a computer-readable format, processing corresponding to the program is realized on the computer system.

  Other objects, features, and advantages of the present invention will become apparent from a more detailed description based on embodiments of the present invention described later and the accompanying drawings. In this specification, the system is a logical set configuration of a plurality of devices, and is not limited to one in which the devices of each configuration are in the same casing.

  According to the configuration of the present invention, in a configuration for extracting a term in a specialized field such as a medical field, for example, a term as a morpheme, an evaluation depending on the length of the term is performed, and a term candidate extraction process focusing on the length is executed. Search the stored documents in the document database, count the number of occurrences of the same term as the extracted term candidates, generate term text that includes a copy of the term according to the number of occurrences, and use the generated term text as the analysis target The first term extraction means performs an evaluation that depends on the appearance frequency of the extracted term candidates, and selects a candidate word that has a high appearance frequency as the output term. Can be performed efficiently.

  Details of a document analysis system, a document analysis method, and a computer program according to embodiments of the present invention will be described below with reference to the drawings.

[First embodiment]
With reference to FIG. 1, the configuration and processing of a document analysis system according to an embodiment of the present invention will be described. As shown in FIG. 1, a document analysis system 100 according to an embodiment of the present invention includes a first technical term extraction unit 101, a technical term reconstruction unit 102, a technical term text storage unit 103, a second technical term extraction unit 104, Document storage means 121 is included.

  Hereinafter, the configuration and processing of each unit will be described with specific examples. The document analysis system of the present invention is a system that analyzes documents including technical terms in specialized fields. In the following embodiments, the medical field will be described as an example of specialized fields. The present invention can be applied to the analysis of technical terms in various specialized fields such as economics, architecture, and technology.

Each means constituting the document analysis system 100 shown in FIG. 1 will be described.
[Document storage]
The document storage unit 121 is a storage unit that stores a document related to a specialized field, in this embodiment, a Japanese document in the medical field, that is, a document database.

[First terminology extraction means]
The first technical term extraction unit 101 performs processing for extracting a term that is determined to be a technical term from the analysis target document, with the stored document in the document storage unit 121 as an analysis target. The first technical term extraction unit 101 performs an evaluation depending on the length of the term, and executes a term candidate extraction process that emphasizes the length. Specifically, term extraction processing is performed with emphasis on parameters of term length such as compound nouns. For example, perform part-of-speech analysis of a document to be analyzed, select compound nouns composed of multiple nouns as technical terms, set higher evaluation values for terms such as longer compound nouns, etc. Extract terms. Note that the term extraction process emphasizing the length parameter executed by the first technical term extraction means 101 is the length-oriented term extraction disclosed in Patent Document 1 (Japanese Patent Laid-Open No. 2004-151926) described above. It can be executed as a process similar to the process.

Hereinafter, as an example, it is assumed that the first technical term extraction unit 101 extracts compound nouns that are the following technical terms in the term extraction processing in which the stored document of the document storage unit 121 is analyzed.
(A) Chronic diffuse tubular proliferative glomerular lymphoma (b) Diffuse intraglomerular nephritis nephrotic syndrome (c) Acute diffuse mesangial superficial keratitis

[Technical term reconstruction means]
The technical term reconstruction unit 102 counts the number of times each technical term candidate extracted by the first technical term extraction unit 101 appears in all the documents stored in the document storage unit 121, and each technical term Duplicate candidates as many times as they appear. That is, the technical term reconstruction unit 102 searches a document set stored in a document database storing documents of a specific specialized field, and determines the number of appearances of the same term as the term candidate extracted by the first technical term extraction unit 101. Count and generate term text that includes a copy of the term according to the number of occurrences.

For example, in the above example:
The appearance frequency of “chronic diffuse tubular proliferative glomerular lymphoma” is 5 times,
The appearance frequency of “diffuse intraglomerular nephritis nephrotic syndrome” is 4 times,
The appearance frequency of “acute diffuse mesangial superficial keratitis” is 3 times,
If it is these appearance frequency, the technical term reconstruction means 102 produces | generates the following duplicate texts according to the appearance frequency (5, 4, 3, ...) of each term.
Chronic diffuse tubular proliferative glomerular lymphoma Chronic diffuse tubular proliferative glomerular lymphoma Chronic diffuse tubular proliferative glomerular lymphoma Chronic diffuse tubular proliferative glomerular lymphoma Chronic diffuse tubular proliferative glomerular lymphoma Diffuse intraluminal glomeruli Nephritic nephrotic syndrome Diffuse endoglomerular nephritis nephrotic syndrome Diffuse intraglomerular nephritis nephrotic syndrome Diffuse intraglomerular nephritic nephrotic syndrome Acute diffuse mesangial superficial keratitis Acute diffuse mesangial superficial keratitis Acute diffuse mesangial superficial keratitis・

[Technical term text storage means]
The technical term text storage means 103 is a storage means for storing the text created by the technical term reconstruction means.

[Second terminology extraction means]
The second technical term extraction means 104 extracts technical term candidates for the text stored in the technical term text storage means 103. The second technical term extraction means 104 performs term extraction focusing on the parameter of the appearance frequency of compound nouns. The second technical term extraction unit 104 performs an evaluation depending on the appearance frequency of the term using the term text generated by the technical term reconstruction unit 103 as an analysis target, and selects a term having a high appearance frequency as an output term.

  The second technical term extraction unit 104 analyzes only the text stored in the technical term text storage unit 103 and counts the appearance frequency of terms such as compound nouns included in the stored text. A high evaluation value is set for a high term, and terms having high evaluation values are extracted. Note that this appearance frequency-oriented term extraction processing can be executed as the same processing as the frequency-oriented term extraction processing disclosed in the above-mentioned Patent Document 1 (Japanese Patent Application Laid-Open No. 2004-151926).

In the above example, the technical term text storage means 103 includes
(Text a) 5 “chronic diffuse tubular proliferative glomerular lymphoma” (text b) 4 “diffuse intraglomerular nephritis nephrotic syndrome” (text c) 3 “acute diffuse mesangial superficial keratitis” These text data are stored, and the second technical term extraction means 104 extracts the following partial character string from the above text by extraction focusing on the appearance frequency parameter of the compound noun. The obtained top N partial character strings are used as final extraction results. N is a preset threshold value.
(1) Diffuse (2) Glomera ...

When the above texts a to c are analyzed,
“Diffuse” is detected in text a, 5 in text b, and 3 in text c, resulting in a total of 12 occurrences.
In addition, five “globules” are detected in the text “a” and four are detected in the text “b”, and the appearance frequency is 9 times in total.
As described above, the second technical term extraction unit 104 analyzes only the text stored in the technical term text storage unit 103 and counts the appearance frequency of terms such as compound nouns included in the stored text. Then, a high evaluation value is set for terms having a high frequency of appearance, and terms having higher evaluation values are extracted. The extracted terms are registered in the dictionary as morphemes as language units to be applied to document processing, such as a search key for document search and an index used for document classification.

  As described above, in the document analysis system according to the present invention, the first technical term extraction means 101 performs technical term candidate extraction that emphasizes the parameter of the length of a term composed of compound nouns, and the second technical term. The extraction means 104 is configured to extract technical terms with emphasis on the parameter of appearance frequency of terms, and the first technical term extraction means 101 can obtain technical term candidates such as compound nouns with high probability, and only those are targeted. Thus, since the second technical term extraction means 103 extracts technical terms, inappropriate technical terms such as “tomorrow's operation” and “previous execution” are not extracted, and high extraction accuracy can be realized. Furthermore, since the technical terms extracted by the second technical term extraction unit 104 are extracted with emphasis on the appearance frequency parameter, they become morphemes having an appropriate length for registration in the morphological analysis dictionary.

  A sequence of processing executed by the document analysis system according to this embodiment will be described with reference to the flowchart shown in FIG. First, in step S101, the first technical term extraction unit 101 uses an analysis target document, for example, a stored document stored in the document storage unit 121 as an analysis target, and emphasizes a term length parameter including a compound noun from the analysis target document. Execute the extraction process. Specifically, as described above, the part-of-speech analysis of the document is performed, and from terms such as compound nouns composed of multiple nouns, higher terms are set for longer terms, and terms higher in the assessment value are technical terms Extract as a candidate.

  Next, in step S102, the technical term reconstruction unit 102 counts the number of times each technical term extracted in step S101 appears in all the documents stored in the document storage unit 121, and each specialized term is counted. Duplicate the term as many times as it appears. The duplicate text data is stored in the technical term text storage means 103.

  Next, in step S <b> 103, the second technical term extraction unit 104 extracts technical term candidates for the text stored in the technical term text storage unit 103. The second technical term extraction means 104 performs term extraction focusing on parameters of appearance frequency of terms such as compound nouns. Specifically, as described above, the appearance frequency of terms included in the text stored in the technical term text storage means 103 is counted, and a high evaluation value is set for a term having a high appearance frequency, and the higher evaluation value is set. Extract terms.

  With these processes, a highly accurate terminology extraction process with less noise is realized. That is, as a first technical term selection process, technical term candidates based on length are extracted, and all documents stored in a document database (document storage unit 121) storing documents in a specialized field, or a plurality of document sets , The number of occurrences of the same term as the extracted technical term candidate is counted, and duplicate text corresponding to the counted number is generated and temporarily stored, that is, stored in the technical term text storage means 103, In addition, the second technical term extraction unit 104 performs the term extraction focusing on the parameter of the appearance frequency of the term for the text data stored in the technical term text storage unit 103. Specifically, as described above, the appearance frequency of compound nouns included in the text stored in the technical term text storage means 103 is counted, and a high evaluation value is set for the compound noun having a high appearance frequency. The terms above are extracted as output terms.

  In the term-oriented term extraction of the first technical term extraction means 101, technical term candidates corresponding to technical terms are obtained with high probability, and the duplicated texts corresponding to the number of times appearing in the document stored in the database are targeted. Since the second technical term extraction means 103 extracts technical terms with emphasis on frequency, inappropriate technical terms such as “Tomorrow's operation” and “previous execution” are not extracted, and high extraction accuracy can be realized. The terminology extracted by the second terminology extraction unit 104 is executed as an extraction process with emphasis on the appearance frequency parameter, so that a morpheme having an appropriate length is extracted.

[Example 2]
A second embodiment obtained by partially modifying the document analysis system described in the first embodiment will be described with reference to FIG. As shown in FIG. 2, the document analysis system 200 according to the second embodiment of the present invention includes a first terminology extraction unit 101, a terminology reconstructing unit 102, and a terminology similar to the configuration of the example 1 shown in FIG. A technical term text update unit 201 is added to the text storage unit 103, the second technical term extraction unit 104, and the document storage unit 121.

  In the document analysis system 200 according to the second embodiment of the present invention, the first technical term extraction unit 101, the technical term reconstruction unit 102, the technical term text storage unit 103, and the second technical term extraction unit 104 perform first processing. This is the same as the processing in the first embodiment.

  In the second embodiment, the terminology text update unit 201 applies the terminology data generated by the second terminology extraction unit 104, executes the update process of the text stored in the terminology text storage unit 103, and updates it. The second terminology extraction unit 104 performs term extraction processing based on frequency information based on the updated text data stored in the terminology text storage unit 103. Execute. In this configuration, the text update in the technical term text update unit 201 and the term extraction based on the updated text by the second technical term extraction unit 104 are repeatedly executed.

  A specific example will be described. First, it is assumed that the technical term reconstruction unit 102 stores text in the technical term text storage unit 103, assuming that the same processing as in the first embodiment is performed. That is, the number of times each technical term candidate extracted by the first technical term extraction unit 101 appears in all the documents stored in the document storage unit 121 is counted, and each technical term candidate is counted by the number of appearances. Store the duplicated text.

For example, in the same manner as in the first embodiment,
The appearance frequency of “chronic diffuse tubular proliferative glomerular lymphoma” is 5 times,
The appearance frequency of “diffuse intraglomerular nephritis nephrotic syndrome” is 4 times,
The appearance frequency of “acute diffuse mesangial superficial keratitis” is 3 times,
If it is these appearance frequency, the technical term reconstruction means 102 produces | generates the following duplicate texts according to the appearance frequency (5,4,3, ...) of each term, and the technical term text storage means 103. To store.
Chronic diffuse tubular proliferative glomerular lymphoma Chronic diffuse tubular proliferative glomerular lymphoma Chronic diffuse tubular proliferative glomerular lymphoma Chronic diffuse tubular proliferative glomerular lymphoma Chronic diffuse tubular proliferative glomerular lymphoma Diffuse intraluminal glomeruli Nephritic nephrotic syndrome Diffuse endoglomerular nephritis nephrotic syndrome Diffuse intraglomerular nephritis nephrotic syndrome Diffuse intraglomerular nephritic nephrotic syndrome Acute diffuse mesangial superficial keratitis Acute diffuse mesangial superficial keratitis Acute diffuse mesangial superficial keratitis・

The second technical term extraction unit 104 counts the frequency of appearance of terms such as compound nouns included in the text for the text stored in the technical term text storage unit 103, and gives a high evaluation value to terms with high frequency of appearance. To extract terms higher in the evaluation value.
In the above example, the technical term text storage means 103 includes
(Text a) 5 “chronic diffuse tubular proliferative glomerular lymphoma” (text b) 4 “diffuse intraglomerular nephritis nephrotic syndrome” (text c) 3 “acute diffuse mesangial superficial keratitis” These text data are stored, and the second technical term extraction means 104 extracts the following partial character string from the above text by extraction focusing on the parameter of the appearance frequency of terms such as compound nouns. . The obtained top N partial character strings are used as final extraction results. N is a preset threshold value.
(1) Diffuse (2) Glomera ...

[Technical term text update means]
The technical term text update unit 201 deletes the technical term extracted in the second technical term extraction unit 104 from the text stored in the technical term text storage unit 103, and further sets a line break in the deleted portion, and is deleted. Generate data that classifies nouns or groups of nouns. Specifically, in the above example,
(1) Diffuse (2) Glomerulus Since these terms are extracted by the second technical term extraction means 104, the processing is performed by deleting these terms and setting line breaks.

The following text is set as update text data by deleting the extracted terms and sorting processing.
Chronic Tubular Proliferative Lymphoma Chronic Tubular Proliferative Lymphoma Chronic Tubular Proliferative Lymphoma Chronic Tubular Proliferative Lymphoma Chronic Tubular Proliferative Lymphoma Intraductal Nephritis Nephrotic Syndrome Intravascular Nephritic Nephrotic Syndrome Intracellular Nephritic Nephrotic Syndrome Acute Mesangial Surface Sankeratitis Acute Mesangial Surface Sankeratitis Acute mesangial surface keratitis ...

These texts are re-stored in the technical term text storage means 103 as updated text. The second technical term extraction unit 104 executes a term extraction process based on the frequency information based on the updated text data stored in the technical term text storage unit 103. For example, as a frequently occurring term from the above update text data,
Chronic tubular proliferative lymphoma These terms will be selected as new extracted terms. The extracted compound nouns (technical terms) are registered in the dictionary as morphemes as language units to be applied to document processing, such as search keys for document search and indexes used for document classification.

  The text update in the technical term text update unit 201 and the term extraction based on the updated text by the second technical term extraction unit 104 are repeatedly executed as necessary.

  According to the present embodiment, it is possible to extract more morphemes with high accuracy by repeating the process of extracting technical terms again after deleting morphemes that have already been determined as technical terms.

  The sequence of processing executed by the document analysis system according to this embodiment will be described with reference to the flowchart shown in FIG. The flowchart shown in FIG. 4 shows the term update based on the text update in the terminology text update unit 201 and the term extraction based on the updated text in the second term term extraction unit 104 until the number of terms extracted by the second terminology extraction unit 104 becomes M or more. It is a flow that defines a process to be repeatedly executed.

First, in step S201, as an initial setting,
Number of extracted terms: i = 0
Set as follows.
Next, in step S202, the first technical term extraction unit 101 uses the analysis target document, for example, the storage document of the document storage unit 121 as an analysis target, and emphasizes a term length parameter such as a compound noun from the analysis target document. Execute the extraction process. Specifically, as described above, the part-of-speech analysis of a document is performed, a term such as a compound noun composed of a plurality of nouns is selected, and a high evaluation value is set for a longer term, and the technical terms at the top of the evaluation value Extract compound nouns.

  Next, in step S203, the technical term reconstructing unit 102 counts the number of times each of the technical term candidates extracted in step S202 appears in all the documents stored in the document storage unit 121, and Duplicate technical terms as many times as they appear. The duplicate text data is stored in the technical term text storage means 103.

  Next, in step S <b> 204, the second technical term extraction unit 104 extracts technical term candidates for the text stored in the technical term text storage unit 103. The second technical term extraction means 104 performs term extraction focusing on parameters of appearance frequency of terms such as compound nouns. Specifically, as described above, the appearance frequency of compound nouns included in the text stored in the technical term text storage means 103 is counted, and a high evaluation value is set for the compound noun having a high appearance frequency. Extract the compound nouns above.

  In step S205, it is determined whether or not the number of terms extracted by the second technical term extraction unit 104 in step S204 is equal to or larger than a predetermined number [M] set in advance. If it is equal to or greater than [M], the process ends. If it is less than [M], the process proceeds to step S206, and the text stored in the technical term text storage means 103 is updated.

  That is, the technical term text update unit 201 deletes the technical term extracted in the second technical term extraction unit 104 from the text stored in the technical term text storage unit 103, and further sets a line feed in the deleted portion, Generate data that classifies the nouns or noun groups before and after the deletion.

  Further, returning to step S204, the second technical term extraction means 104 performs term extraction focusing on the appearance frequency parameter for the updated text. The processing of steps S204 to S206 is repeated until it is determined in step S205 that the number of terms extracted by the second technical term extraction means 104 is equal to or greater than a predetermined number [M] set in advance. Alternatively, the upper limit number may be set in advance, and the process may be stopped when the upper limit number is reached.

  By these processes, a highly accurate technical term extraction process with less noise is realized, and a predetermined number of technical terms can be extracted in advance.

[Example 3]
Furthermore, a third embodiment of the document analysis system of the present invention will be described with reference to FIG. As shown in FIG. 5, the document analysis system 300 according to the third embodiment of the present invention includes a first technical term extraction unit 101, a technical term reconstruction unit 102, and a technical term similar to the configuration of the first embodiment shown in FIG. A technical term selection unit 301 is added to the text storage unit 103, the second technical term extraction unit 104, and the document storage unit 121.

  In the document analysis system 300 according to the third embodiment of the present invention, the first technical term extraction unit 101, the technical term reconstruction unit 102, the technical term text storage unit 103, and the second technical term extraction unit 104 perform first processing. This is the same as the processing in the first embodiment.

  In the third embodiment, the technical term selection unit 301 performs a process of reselecting data to be registered as the technical term data morpheme extracted by the second technical term extraction unit 104.

  The technical term selection unit 301 extracts all partial character strings X composed of combinations of other N−1 partial character strings from the N partial character strings obtained by the second technical term extraction unit 104. If the appearance frequency of X is equal to or less than a preset threshold, X is deleted from the output term, that is, deleted from the technical term (morpheme) list.

For example, as N partial character strings obtained by the second technical term extraction means 104,
Diffuse tubular Diffuse tubular ...
Assume that these terms are selected as extracted terms.

The technical term selection means 301 is, for example,
"Diffuse tubular"
It is determined to be composed of a combination of other extracted terms [diffuse] and [tubular]
[Diffuse tube] is extracted as a partial character string X composed of a combination of other N-1 partial character strings. The technical term selection unit 301 verifies the appearance frequency of X by using the text stored in the technical term text storage unit 103 as an analysis target. If the appearance frequency of X is equal to or lower than a preset threshold, X Delete from registered technical terms (morphemes).

  By this process, an inappropriately long morpheme that does not appear frequently is deleted from the registration candidates, a more appropriate morpheme registration process is executed, and a morpheme list suitable for morpheme analysis can be obtained.

  Finally, an example of the hardware configuration of the information processing apparatus constituting the document analysis system that executes the above-described processing will be described with reference to FIG. A CPU (Central Processing Unit) 501 is a process corresponding to an OS (Operating System), a first technical term extraction process, a technical term reconstruction process, a second technical term extraction process, a technical term text described in the above-described embodiment. Update processing, technical term selection processing, etc. are executed. These processes are executed according to a computer program stored in a data storage unit such as a ROM or a hard disk of each information processing apparatus.

  A ROM (Read Only Memory) 502 stores programs used by the CPU 501, calculation parameters, and the like. A RAM (Random Access Memory) 503 stores programs used in the execution of the CPU 501, parameters that change as appropriate during the execution, and the like. These are connected to each other by a host bus 504 including a CPU bus.

  The host bus 504 is connected to an external bus 506 such as a PCI (Peripheral Component Interconnect / Interface) bus via a bridge 505.

  A keyboard 508 and a pointing device 509 are input devices operated by the user. The display 510 includes a liquid crystal display device, a CRT (Cathode Ray Tube), or the like, and displays various types of information as text and images.

  An HDD (Hard Disk Drive) 511 includes a hard disk, drives the hard disk, and records or reproduces a program executed by the CPU 501 and information. The hard disk is also used as, for example, document storage means, technical term text, for example, text generated by technical term reconstruction means, update text storage means for technical term text update means, and extracted technical term storage means. . Furthermore, various computer programs such as a data processing program are stored.

  The drive 512 reads data or a program recorded on a removable recording medium 521 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, and the data or program is read out from the interface 507 and the external bus 506 , And supplied to the RAM 503 connected via the bridge 505 and the host bus 504.

  The connection port 514 is a port for connecting the external connection device 522 and has a connection unit such as USB or IEEE1394. The connection port 514 is connected to the CPU 501 and the like via the interface 507, the external bus 506, the bridge 505, the host bus 504, and the like. The communication unit 515 is connected to a network and executes communication with various databases and other information processing apparatuses.

  Note that the hardware configuration example of the information processing apparatus as the document analysis system shown in FIG. 6 is an example of an apparatus configured by applying a PC, and the document analysis system of the present invention is not limited to the configuration shown in FIG. Any configuration capable of executing the processing described in the above-described embodiments may be used.

  The present invention has been described in detail above with reference to specific embodiments. However, it is obvious that those skilled in the art can make modifications and substitutions of the embodiments without departing from the gist of the present invention. In other words, the present invention has been disclosed in the form of exemplification, and should not be interpreted in a limited manner. In order to determine the gist of the present invention, the claims should be taken into consideration.

  The series of processing described in the specification can be executed by hardware, software, or a combined configuration of both. When executing processing by software, the program recording the processing sequence is installed in a memory in a computer incorporated in dedicated hardware and executed, or the program is executed on a general-purpose computer capable of executing various processing. It can be installed and run. For example, the program can be recorded in advance on a recording medium. In addition to being installed on a computer from a recording medium, the program can be received via a network such as a LAN (Local Area Network) or the Internet and can be installed on a recording medium such as a built-in hard disk.

  Note that the various processes described in the specification are not only executed in time series according to the description, but may be executed in parallel or individually according to the processing capability of the apparatus that executes the processes or as necessary. Further, in this specification, the system is a logical set configuration of a plurality of devices, and the devices of each configuration are not limited to being in the same casing.

  As described above, according to the configuration of the present invention, for example, in terms of a configuration in which a term is extracted as a morpheme in a specialized field such as a medical field, evaluation is performed depending on the length of the term, and length is emphasized. The term candidate extraction process is executed, the stored document in the document database is searched, the number of occurrences of the same term as the extracted term candidate is counted, and the term text including a copy of the term according to the number of occurrences is generated and generated The term text that has been analyzed is subjected to evaluation depending on the appearance frequency of the term candidate extracted by the first term extraction means, and the term candidate having a high appearance frequency is selected as the output term. As a result, it is possible to efficiently extract optimal technical terms.

It is a figure which shows the structural example of Example 1 of the document analysis system of this invention. It is a figure which shows the flowchart explaining the process sequence which the document analysis system of Example 1 of this invention performs. It is a figure which shows the structural example of Example 2 of the document analysis system of this invention. It is a figure which shows the flowchart explaining the process sequence which the document analysis system of Example 2 of this invention performs. It is a figure which shows the structural example of Example 3 of the document analysis system of this invention. It is a figure explaining the hardware structural example of the document analysis system which concerns on one Embodiment of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 Document analysis system 101 1st technical term extraction means 102 Technical term reconstruction means 103 Technical term text storage means 104 Second technical term extraction means 121 Document storage means 201 Technical term text update means 301 Technical term selection means 501 CPU (Central Processing) Unit)
502 ROM (Read-Only-Memory)
503 RAM (Random Access Memory)
504 Host bus 505 Bridge 506 External bus 507 Interface 508 Keyboard 509 Pointing device 510 Display 511 HDD (Hard Disk Drive)
512 drive 514 connection port 515 communication unit 521 removable recording medium 522 external connection device

Claims (11)

A term extraction unit that executes a process for extracting a term in a document, performs an evaluation depending on the length of the term, and executes a term candidate extraction process that emphasizes the length;
Search the document set stored in the document database, count the number of occurrences of the same term as the term candidate extracted by the first term extraction means, and generate a term text including a copy of the term according to the number of occurrences. Configuration means;
A second term extracting means for performing an evaluation depending on the appearance frequency of the term with the term text generated by the term reconfiguring means being analyzed, and selecting a term having a high appearance frequency as an output term;
A document analysis system characterized by comprising: The first term extracting means is configured to execute a term extracting process for analyzing a document of a specific specialized field,
The said term reconstruction means is a structure which performs the count process of the frequency | count of appearance by the search of the document set stored in the document database which stored the document of the said specific field of specialization, The structure of Claim 1 characterized by the above-mentioned. Document analysis system. The first term extraction means executes a term extraction process for analyzing a document in the medical field,
The term reconstructing means executes a count process of the number of appearances by searching a document set stored in a document database storing medical field documents,
The document analysis system according to claim 1, wherein the second term extraction unit is configured to extract a morpheme as a term in the medical field. The document analysis system further includes:
A term text updating unit that generates an updated term text from which the term extracted by the second term extracting unit is deleted from the term text generated by the term reconfiguring unit;
The second term extraction unit is configured to perform an evaluation depending on the appearance frequency of the term with the updated term text generated by the term text update unit being analyzed, and to select a term having a high appearance frequency as an output term. The document analysis system according to claim 1, wherein the document analysis system is a document analysis system. The document analysis system further includes:
A term having a combination configuration of other extracted terms is selected from the terms extracted by the second term extracting means, and a process of deleting from the output term is executed when the appearance frequency of the selected term is equal to or lower than a predetermined threshold. 5. The document analysis system according to claim 1, further comprising term selection means. A document analysis method executed in the document analysis system,
A first term extraction step in which a first term extraction means for executing a term extraction process in a document performs an evaluation depending on the term length and executes a term candidate extraction process focusing on length;
A term including a copy of a term corresponding to the number of occurrences, wherein the term reconstructing means searches the document set stored in the document database, counts the number of occurrences of the same term as the term candidate extracted in the first term extraction step A term restructuring step to generate text;
A second term extraction step, wherein the second term extraction means performs an evaluation depending on the appearance frequency of the term with the term text generated in the term reconstruction step as an analysis target, and selects a term having a high appearance frequency as an output term;
A document analysis method characterized by comprising: The first term extraction step is a step of executing a term extraction process for analyzing a document in a specific specialized field.
7. The term reconstructing step is a step of executing a process of counting the number of appearances by searching a document set stored in a document database storing documents of the specific specialized field. Document analysis method. The first term extracting step is a step of executing a term extracting process for analyzing a document in the medical field,
The term reconstructing step is a step of executing a process of counting the number of appearances by searching a document set stored in a document database storing medical field documents,
The document analysis method according to claim 6, wherein the second term extraction step is a step of extracting a morpheme as a term in the medical field. The document analysis method further includes:
The term text update means has a term text update step for generating an updated term text in which the term extracted in the second term extraction step is deleted from the term text generated in the term reconstruction step,
The second term extraction step is a step in which the updated term text generated in the term text update step is subjected to an evaluation depending on the appearance frequency of the term as an analysis target, and a term having a high appearance frequency is selected as an output term. The document analysis method according to claim 6, wherein the document analysis method is a document analysis method. The document analysis method further includes:
Term selection means selects a term having a combination of other extracted terms from the term extracted in the second term extraction step, and deletes it from the output term when the frequency of appearance of the selected term is equal to or lower than a predetermined threshold. The document analysis method according to claim 6, further comprising a term selection step for executing the processing to be performed. A computer program for executing document analysis processing in a document analysis system;
A first term extraction step for causing a first term extraction means for performing a term extraction process in a document to perform an evaluation depending on a term length and to execute a term candidate extraction process with emphasis on length;
A term including a copy of a term corresponding to the number of occurrences by causing the term reconstructing means to search the document set stored in the document database, count the number of occurrences of the same term as the term candidate extracted in the first term extraction step A term restructuring step to generate text;
A second term extraction step for causing the second term extraction means to perform an evaluation depending on the appearance frequency of the term with the term text generated in the term reconstructing step as an analysis target, and to select a term having a high appearance frequency as an output term; ,
A computer program characterized by comprising:

Images