JP2003263451A - Thesaurus formulation support device and thesaurus formulation support method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To solve a problem that, when a thesaurus is formulated, a user himself/herself has to extract again an important word or phrase that ought to have been differentiated in an existing document because character decorating information is lost in the course of processing, in the event that the decorating information exists in the document as an example that a viewpoint of the user is previously reflected in the document. <P>SOLUTION: This device is characterized by being equipped with a tag content extraction means for extracting the kind of tag information and a character string corresponding thereto from a document with the tag information embedded therein, a candidate generation means for generating candidates for registration in a thesaurus from the character string corresponding to the kind of tag extracted by the extracting means by a prescribed document analysis process, and a candidate storing means for storing, for the thesaurus, the candidates for registration generated by the generating means. <P>COPYRIGHT: (C)2003,JPO

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a thesaurus construction support device for extracting a character string in a document and supporting construction of the thesaurus.

[0002]

2. Description of the Related Art When performing a search for a document or automatically classifying a plurality of documents according to a certain viewpoint, background knowledge that defines various associations between words with respect to vocabulary appearing in the document. By giving "," it becomes possible to process documents on the computer in consideration of meaning. In implementing document processing on such a computer, a knowledge base (electronic dictionary) called a thesaurus is often used as background knowledge at present. In this thesaurus, "PC" and "PC" are synonyms,
A synonymous relationship between words and a higher / lower relationship such that "dahlia" and "daisy" belong to a lower concept of "flower" are described.

The thesaurus plays an important role in document processing in consideration of the meaning, but the conventional thesaurus construction method is to create a data list in a format that can be read by the construction system and register them collectively ( JP-A-8-77204 "related word dictionary creation support device") or a method in which a user picks up expressions that he / she wants to store while referring to an existing document, and performs registration (JP-A-8-221427).
"Thesaurus registration support device and thesaurus registration support method").

The former registration method is widely used as a thesaurus construction method. For example, when a large number of vocabularies are newly registered as a user dictionary or a postal code dictionary called an external dictionary in the Kana-Kanji conversion system, It is often the case that the vocabulary list you want to register at once is expressed as general-purpose delimiters such as commas, blanks, and tabs, and given as input.

On the other hand, in most of the systems based on the latter means, the user simultaneously displays the contents of the thesaurus and the reference document in which the vocabulary candidates to be registered in the thesaurus are described on the screen, so that the user can express any expression on the screen. On the other hand, the vocabulary can be registered in the thesaurus simply by specifying the extraction source and the registration destination. There is also a document classification system (Japanese Patent Laid-Open No. 2001-184351 "Document Information Extraction Device and Document Classification Device") that facilitates dictionary maintenance by generating and presenting a list of candidates to be registered in the dictionary.

By the way, as for the vocabulary registered in the thesaurus, many of the vocabulary in the document to be processed that the user considers to be important are usually stored. However, even if a document that is referred to as a vocabulary registration candidate when constructing a thesaurus is conventionally visually emphasized, italicized, or underlined as a result of the user's reflection of importance, such a format is used. It is not possible to perform processing using information, and in terms of processing, it is common to convert and process into a general document format called a so-called plain text format that lacks such format information.

[0007]

However, in such a conventional thesaurus construction system, even if the character decoration information is present in the document as an example in which the user's point of view is reflected in advance in the document, the processing is not performed. Since such character decoration information is lost in the process, the important words that should have been distinguished in the existing document had to be extracted again by the user himself when constructing the thesaurus.

[0008]

SUMMARY OF THE INVENTION The present invention for solving the above-mentioned problems is based on the fact that an arbitrary character string in a document is bold-faced, underlined,
Tag information adding means for adding tag information defining character decoration such as coloring, and a tag for extracting a character string corresponding to the type of tag information from the document in which the tag information is embedded by the tag information adding means Content extraction means, a candidate generation means for generating a thesaurus registration candidate by a predetermined document analysis process from a character string corresponding to the tag type extracted by this tag content extraction means, and this candidate generation means. It is characterized by further comprising a candidate storage means for storing the registration candidate as a thesaurus.

This is a means for constructing a thesaurus from the document given as input by using the tag information as the format information. When extracting information useful for constructing the thesaurus from the document, bold letters, italics, A thesaurus construction device that recognizes format information such as underlining and presents the vocabulary to be registered in the thesaurus or the vocabulary to be referred to as a registration candidate, and is a format for visually characterizing character strings in a document. Extraction can be done based on the defined tags.

[0010]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT FIG. 1 shows an embodiment according to the present invention. The thesaurus construction support device according to the present embodiment is a document creation / browsing unit 101 that is realized by software (word processing software, document editor, etc.) on a computer.
A candidate generation including a tag information adding unit 102 that is one function, an input processing unit 103 for a tagged document, a tag content extracting unit 104 for tag information added to the document, and an extraction candidate specifying unit 106 that receives a user instruction. Unit 105, candidate storage unit 10 for registering the output candidates as a thesaurus
7, and the thesaurus storage unit 108 as a storage destination of the thesaurus.

FIG. 2 shows an example of a character string 210 visually characterized on the document creating / viewing unit 101. Emphasis 21
There are expressions such as 1, colored 212, underline 213, and italic 214.

The tag information adding unit 102 converts the visual characterization performed on the character string by the document creating / viewing unit 101 into a predetermined format and outputs it as a tagged document including format information. Here, an RTF (Rich Text Format) format as shown in FIG. 3 is taken as an example of a tagged document. This RT
The F format is a file format for writing document information including character attributes such as character font and character size, which has been specified by Microsoft Corporation and is open to the public.

The tagging format for documents is R
In addition to TF format, W3C (World Wide Web Consortium
There are formats such as HTML and XML, which are de facto standards on the Web, and whose specifications are similarly published.

FIG. 4 is a tagged document 420 corresponding to the document of FIG. 2, with emphasis 421, red letters 422 and underline 4
23 and italic 424 are highlighted 211, colored 212, and
It is a character string corresponding to the tag information corresponding to the expression of underline 213 and italic 214.

The document tagged by the tag information adding unit 102 is sent to the tag content extracting unit 104. The tag content extraction unit 104 analyzes the input document and extracts a tag type and a character string corresponding to the tag. Candidate generator 10
At 5, in the analyzed tag type, it is determined which tag type corresponds to the character string to be extracted, and the character string corresponding to the specific tag information is output from the input document. The extraction candidate designation unit 106 receives designation of tag type selection from the user as necessary.

The candidate storage unit 107 stores the list output by the candidate generation unit 105 as a thesaurus or presents it as a thesaurus registration candidate. The above is the outline of the system according to the embodiment. Next, details of each processing unit will be described using a flowchart.

First, the flow of processing in the input processing unit 103, the tag content extraction unit 104, and the candidate generation unit 105 will be described with reference to the flowchart of FIG. In the document input process, for example, the text shown in FIG. 4 is received as a document with a tag added (step S502).

Next, in the tag content extraction processing, a tag type (called a control code in the RTF format) in the document data and a character string corresponding to the tagging are recognized and tagged with each tag type. The character string is stored, for example, in the tree structure format shown in FIG. 6 (step S503).

As an example of the storing method, a procedure of storing in a tree structure will be shown here. FIG. 7 is a flowchart showing a procedure for storing in a tree structure. The storage procedure will be described with reference to this flowchart.

First, the storage destination node is initialized, and the initial node depth is set to 0 (step S701). Next, one character is read from the tagged document (step S70).
2). Next, the end determination of the input document is performed (step S
703).

If the data read in step S703 is not the end of the file, it is determined whether the character string is "{" (step S7).
04). Since the symbol "{" means the start of the range affected by the tag, if the read character string is "{", a new node is created and the node depth is incremented (step S707).

Next, the tag information and the corresponding character string subject to the tagging effect are stored in the newly created node (step S708). After the storage, the process returns to step S702 to receive the input from the document again.

On the other hand, if the input character string is not "{" in step S704, it is determined whether the read character string is "}" (step S705). If the read character string is "}", the node depth is decremented and the current storage destination is replaced with the parent node one step higher (step S706). After that, the process moves to step S702 again to read the input character string. If the condition of step S705 is not satisfied, the process proceeds to S702.

Through the document input process in step S502 and the tag content extraction process in step S503, a tree structure storing the tag type and the tagged character string is obtained from the tagged document data.

Next, returning to FIG. 5, the tag type of the data to be output is determined for the analyzed document (step S50).
4). In selecting the tag type to be output here, for example, the following three types can be considered.

(1) A method of fixing the tag type to be extracted in advance and extracting the character string corresponding to the corresponding tag information. This corresponds to the case where there is one extraction type or the extraction target is designated in advance in step S504.

When it is not possible to specify (2-1), the user designates the tag type to be extracted and the corresponding data is extracted. This corresponds to the process of step S505. An example of the user interface is shown in FIG. The interface shown in FIG. 8 displays the tag types appearing in the document after analyzing the document data,
The user can specify extraction candidates by checking the check boxes such as [bold] and [underline] 801 and 802.

(2-2) Selection candidate presentation step S505
As a variation of the above, an interface that enables the extraction candidate designating unit 106 to designate, in addition to the operation of designating the tag type to be extracted, what concept on the thesaurus the character string corresponding to the tag type is labeled Can be considered. Such an extraction candidate designation unit 106
FIG. 9 shows an example of an interface for realizing the above.

As an example, an extraction candidate can be selected by checking check boxes 901 and 902 such as [bold] and [underline], while a text input box 903 corresponding to a check item is displayed in [bold]. By inputting a label name such as [Extended Concept] corresponding to [Underline] in [Corresponding [Basic Concept]] and 904, a user input indicating what kind of concept label on the thesaurus a particular tag type is associated with is received. Provide an interface. The above is the processing of the selection candidate presentation step S505. FIG. 10 is an example of an output data format obtained in the extraction process in the extraction candidate designation unit 106.

In the candidate generation process, the output obtained in this way is used for statistical significance such as grammatical interpretation by morphological analysis that automatically makes segmentation and part-of-speech determination and the frequency of occurrence of character strings in a document. The vocabulary having a low value is removed (step S506), and the obtained output is given as an input in the subsequent candidate storage processing (step S50).
7).

In step S507 for performing the candidate storing process, an interface for registering data (vocabulary) in the thesaurus 108 is provided. In the candidate storage unit 107, the user is presented with the thesaurus contents and an environment in which various text data can be browsed as vocabulary candidates registered in the thesaurus. The data output by the extraction candidate designating unit 106 may be the thesaurus in the candidate storing unit 107, and may be used as a reference document for browsing by the user in the role of presenting registration candidates.

Next, the candidate storing process (step S206)
Will be described with reference to the flowchart of FIG. The flow of processing when the output obtained thus far is presented as a reference document is shown.

First, the output from the candidate designating unit 106 as shown in FIG. 10 is received (step S1101). Subsequently, in order to use the obtained data as a reference document in the candidate storage unit 107, so-called morphological analysis processing is performed on the output data to automatically perform segmentation and part-of-speech estimation on the output data (step S1102). This morphological analysis processing is a widely known method, and there are already various systems on the market regardless of whether they are commercially available or free of charge.

Next, a co-occurrence obtained by extracting two or three words that appear adjacently in a list 1201 or a document obtained by dividing these data subjected to morphological analysis into words shown in FIG. 12 based on a predetermined structure. A list called a list 1202 is generated (step S1103). The list is refined using statistical information such as the frequency of appearance for the purpose of mechanically eliminating the combination of vocabulary that seems to be inappropriate from the list thus generated (step S110).
4). Each list thus obtained is output as a vocabulary registration candidate (step S1105). Through the procedure described above, in the candidate storage processing of step S507, the vocabulary registration candidates for the thesaurus are presented.

While referring to the output vocabulary registration candidate, the user manually registers the candidate in an appropriate place in the thesaurus. An example of the finally constructed thesaurus is shown in FIG. In the area 1303, the user adopts the expressions “excitation”, “co-occurrence”, “stimulation”, “resonance”, and “divergence” presented as registration candidates and registers them on the thesaurus. . The user constructs a thesaurus by giving superordinate concepts such as “plus” and “minus” 1302 and “vocabulary image” 1301 to the presented candidates.

Below, for the document data shown in FIG.
A process of extracting a portion where character decoration is performed and performing actual thesaurus registration in the candidate storage unit 107 will be described. In this embodiment, “stimulation”, “excitation”, and “divergence” emphasized by bold letters
Follow the process by which “resonance” as well as underlined “excited state divergence” are extracted.

First, the document is read by the input processing in step S502. By the tag content extraction processing in step S503, the tag information attached to the document data as shown in FIG. 14 is interpreted and each tag information is stored in the tree structure in the format of FIG. In FIG. 15, reference numeral 1501 denotes a storage location for data “stimulation”, “excitation”, “divergence”, and “resonance” in bold, and 1502 denotes a storage location for underlined “divergence in excited state”.

In this case, since a plurality of decoration types appear, the condition of step S504 is not satisfied, and the user's input is required by the selection candidate presentation process of step S505. Here, as in the input of FIG.
01 and the underlined expression 802 are selected and extracted. Tag information corresponding to the selected expression is searched for,
Corresponding 1501 and 1502 shown in FIG. 15 are output.

Next, in candidate generation processing step S506, morphological analysis processing is performed, and outputs such as the word list 1201 and the co-occurrence list 1202 shown in FIG. 12 are obtained. The list obtained in this way is further used to automatically remove the vocabulary that is considered to have a low utility value for the user, by using a statistical measure such as TF / IDF for the appearance frequency of each word in the document. Can be included.

The data list thus obtained is used as a reference document in the candidate registration process, and the user registers it in the thesaurus. FIG. 13 shows an example of registration in the thesaurus. FIG. 13 is an example in which the concept of three layers is constructed. The output obtained by the candidate generation process corresponds to presenting the vocabulary 1303 appearing in the document as a registration candidate.

As described above, according to the thesaurus construction processing method shown in the present embodiment, when the vocabulary in the existing document data is extracted to construct the thesaurus, underlining, emphasis, By recognizing format information such as italics, it is possible to extract character strings based on those format information, and perform processing that reflects the user's intention for a specific vocabulary applied to existing document data. You can

By providing correspondence between the format information and the character labels in the thesaurus, the thesaurus can be easily constructed from existing document data. Further, by presenting the vocabulary candidates to be registered in the thesaurus, it is possible to reduce the labor required for constructing the thesaurus.

It is also possible to realize the processing in the embodiment of the present invention by a computer-executable program and realize the program as a computer-readable storage medium.

The storage medium in the present invention is a magnetic disk, flexible disk, hard disk, optical disk (CD-ROM, CD-R, DVD).
Etc.), a magneto-optical disk (MO etc.), a semiconductor memory, etc., as long as it is a storage medium which can store a program and can be read by a computer, the storage format may be any form.

Further, an OS (operation system) operating on the computer, database management software, MW (middleware) such as a network, etc., which realizes the present embodiment, based on the instructions of the program installed in the computer from the storage medium. You may perform a part of each process for doing.

Further, the storage medium in the present invention is not limited to a medium independent of a computer, but includes a storage medium in which a program transmitted via a LAN, the Internet or the like is downloaded and stored or temporarily stored.

Further, the number of storage media is not limited to one, and even when the processing in this embodiment is executed from a plurality of media, it is included in the storage media in the present invention, and the structure of the medium may be any one. Good.

The computer according to the present invention is
Each processing is executed in the present embodiment based on a program stored in a storage medium, which has any configuration such as a device such as a personal computer or a system in which a plurality of devices are network-connected. Good.

Further, the computer in the present invention is not limited to a personal computer but also includes an arithmetic processing unit, a microcomputer, etc. included in information processing equipment, and is a general term for equipment and devices capable of realizing the functions of the present invention by a program. is doing.

[0050]

As described above, according to the thesaurus construction processing apparatus of the present invention, when the vocabulary in the existing document data is extracted and the thesaurus is constructed, underlining, emphasis, By recognizing format information such as italics, it is possible to extract character strings based on those format information, and perform processing that reflects the user's intention for a specific vocabulary applied to existing document data. You can

[Brief description of drawings]

FIG. 1 is a configuration block diagram according to an embodiment of the present invention.

[Fig. 2] A concrete example of a vocabulary that is underlined, emphasized, or italicized, which is the extraction target part.

FIG. 3 is an explanatory diagram showing an example of a document format with a tag as an input.

FIG. 4 is a diagram showing tag information corresponding to underlines, emphasized expressions, and italics that are extraction target locations.

FIG. 5 is a flowchart describing an outline of processing.

FIG. 6 is an explanatory diagram illustrating an example of a tree structure as a data storage structure.

FIG. 7 is a flowchart describing a procedure for storing tag information and a corresponding character string in a tree structure.

FIG. 8 is an example of an interface that prompts the user to select an extraction candidate.

FIG. 9 is an example of an interface for inputting extraction candidates and corresponding concept names.

FIG. 10 is an example of an output data format obtained in the extraction process.

FIG. 11 is a flowchart describing processing of a candidate generation unit.

FIG. 12 is a diagram illustrating an example of an output list.

FIG. 13 is an example of a diagram showing an internal structure of a thesaurus.

FIG. 14 is an example of an input document.

FIG. 15 is an example of a tree structure representation when data is extracted based on designation of extraction candidates.

[Explanation of symbols]

101 ... Document creation / viewing unit 102 ... Tag information addition unit 103 ... Input processing unit 104 ... Tag content extraction unit 105 ... Candidate generation unit 106 ... Extraction candidate designation unit 107 ... Candidate storage unit 108 ... Thesaurus storage

   ─────────────────────────────────────────────────── ─── Continued front page    F term (reference) 5B075 ND03 NK35 NR12 QM07 QM08                       UU01                 5B091 AB00 CB24 CC02 CC04 CC16                       DA06

Claims (5)

[Claims] 1. Tag information adding means for adding tag information defining character modifications such as bold, underline, and coloring to an arbitrary character string in a document, and tag information is embedded by the tag information adding means. A tag content extraction unit that extracts a tag information type and a corresponding character string from the document, and a thesaurus registration candidate by a predetermined document analysis process from the character string corresponding to the tag type extracted by the tag content extraction unit. A thesaurus construction support device, comprising: a candidate generating unit for generating a candidate, and a candidate storing unit for storing the registration candidate generated by the candidate generating unit as a thesaurus. 2. The thesaurus construction according to claim 1, wherein the candidate storing means stores a plurality of character strings and a label indicating a higher-level concept common to the character strings as an associated thesaurus. Support device. 3. A thesaurus concept for the combination of a tag type in which a format for visually characterizing an arbitrary character string in a document is defined and a character string corresponding to the tag. 2. The thesaurus construction support device according to claim 1, further comprising a label designating unit for designating a label indicating. 4. The tag content extraction means comprises tag type designation means capable of designating a part of tag types from a plurality of types of tags defining a format for visually characterizing an arbitrary character string in a document, The thesaurus construction support device according to claim 1, wherein the tag type and the character string corresponding to the tag type are extracted from the tagged document for the designated tag. 5. A character string corresponding to the type of tag information from a document in which this tag information is embedded, by adding tag information defining character modifications such as bold, underline, and coloring to any character string in the document. To extract a registration candidate for the thesaurus from a character string corresponding to the extracted tag type by a predetermined document analysis process, and to store the registration candidate created by the candidate creating means as a thesaurus. Characteristic thesaurus construction support method.