JP2006235800A - System for giving invisible information to electronic document and giving method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a system which retrieves a document, without omission, in which a document inputted with external characters and a document inputted with internal characters exist together, and stably gives a versatile external character retrieval function, and also provide a method therefor. <P>SOLUTION: The system is provide with; an external character/variant character conversion table in which external character codes, variant characters of internal characters corresponding to external characters and similarities are stored; a means which retrieves external character codes of the external character/variant character conversion table from external characters included in sentences in an electronic document and retrieves the variant characters of one or more internal characters, whose similarities are at least a certain degree, among the internal characters which correspond to the retrieved external character codes; a means which generates a sentence by replacing external character codes included in an original sentence with the variant characters of one or more internal characters thus retrieved; a means which writes, with a space between words, one or more sentences generated by the replacement with the variant characters; and a means which extracts the character variants of the internal characters corresponding to the external characters, as words, from the words written with a space between them, and embeds the extracted words, as invisible information, into the electronic document. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a system and method for providing invisible information to an electronic document, and more particularly to an electronic document search system including external characters by a computer.

  Generally, an electronic document handled by a computer may require an external character for a person name, a place name, or the like, and input to the electronic document is performed by a kana-kanji conversion system.

Conventionally, when using the Kana-Kanji conversion system, even if the place name and person name are the same, if the electronic document input person is different, one type is entered in internal characters and the other type is entered in external characters. As a result, documents entered in internal characters and documents entered in external characters are mixed in the electronic document search system, and in order to search for place names and personal names that include external characters, internal characters and external characters Both had to be searched.

In addition, when an external character is replaced with an internal variant, it may be necessary to obtain the approval of the person or the manuscript creator, particularly in the case of a person's name.

In an electronic document, an external character may be used as a variant of an internal character, and a method of easily handling an external character includes a method of registering an external character font or a method of embedding it as an image.

In the case of using an external character as an external character font function or an image, the kana-kanji conversion system cannot convert the external character, and the document cannot be searched.

As a method for realizing a search function using external characters, there is a method in which an external character search is realized by a close relationship between a normal internal character search system and an external character search system (see, for example, Patent Documents 1, 2, 3, or 4). ). In these conventional techniques, the internal character search system and the external character search system are closely related to each other. Therefore, it is necessary to develop the entire search system, which requires a great deal of labor. Further, since the internal character search system and the external character search and external character conversion functions operate simultaneously, the system becomes complicated, and it is necessary to operate a plurality of processes and devices at the same time, resulting in a problem that stability and maintainability are deteriorated. Also, since the external character search function is linked with the search system itself, it is difficult to install it in an existing search system.

JP-A-9-232313 JP 2002-41503 A JP 2002-236676 A JP 2003-29736 A

The present invention has been made to solve such a problem, and a first object is to easily and stably give an external character search function to a search system having no external character search function. The second purpose is to perform a search without omission from a document in which a target word is a mixture of a document input in external characters and a document input in internal characters, and has an external character input function and an external character search function. It is an object of the present invention to provide a system and method for providing invisible information to an electronic document, which can provide a general-purpose and stable external character search function to a non-electronic document search system.

The invisible information addition system for an electronic document according to the present invention includes an external character code, an external character / variant character conversion table in which the internal characters corresponding to the external characters are stored in association with the character shape and meaning, and the electronic document including the external character code. Means for retrieving an external character code of the external character / variant character conversion table from external character codes included in the sentence, and searching for one or a plurality of internal character variants, and the search for the sentence in the electronic document A means for creating a sentence replaced with a variant of one or more internal characters, a means for dividing one or more sentences created by replacing with the variant, and a character string from the divided character string into an external character Means for extracting a variant of the corresponding internal character as a character string that is separated and embedding the extracted character string as the information that becomes invisible in the electronic information browsing software in the electronic document; It is characterized by having a.

  The invisible information addition system for an electronic document according to the present invention is a combination of an external character code and an external character code, a reading thereof, and a synonym when the external character / variant character conversion table does not include an internal character corresponding to the external character. It is characterized by setting.

  The system for assigning invisible information to an electronic document according to the present invention has a similarity between an external character shape and an internal variant corresponding to the external character in the external character / variant character conversion table, and the external character code search means has a similarity degree. It is characterized by controlling the number of variants that are used and searched.

  The invisible information imparting system to an electronic document according to the present invention is characterized in that a character string clearly extracted by a search system is extracted by inserting a symbol before and after a character string embedded as invisible information. .

The method for providing invisible information to an electronic document according to the present invention replaces all sentences in the electronic document with internal characters and separates the text replaced with the internal characters with respect to the electronic document composed of external characters and internal characters. The extracted internal character corresponding to the external character portion of the divided character string is extracted from the external character code and the external character / variant character conversion table in which the internal character corresponding to the external character is stored and extracted. Characterized by embedding the character string thus written in the original electronic document as information that becomes invisible with the electronic information browsing software.

As described above, in the present invention, since there is no need to cooperate with the search system and the invisible information is given to the electronic document data, the stability and maintainability of the search system itself is very high. It can also be implemented in a search system.

In addition, in order to eliminate the distinction between internal characters and external characters, it is possible to completely eliminate the leakage of search due to a mixture of internal characters and external characters between different documents due to differences in the input person, etc., even when describing the same target word .

The invisible information addition system for an electronic document according to the present invention stores an external character code, a variant of an internal character corresponding to the external character, and a correspondence between the shape of the external character and the similarity of the internal variant corresponding to the external character The character conversion table is the basis.
In the present invention, for example, in the case of “Watanabe”, the character shape of the external character includes “edge”, “邊”, and “邉”, but these three types have the same meaning. Say.

The present invention is characterized in that the external character / variant character conversion table is used to embed an invisible character string in advance so that it can be handled by the search system in an electronic document used in the search system. This invisible character string may be anywhere in the electronic document as long as the search system can handle it as a search target.

The method of embedding invisible information is to search for internal variants corresponding to external characters using external character / variant character conversion table for sentences composed of external characters and internal characters, and internal variants corresponding to external characters. If you embed a character as invisible information, it cannot be searched as a word when searching. Therefore, replace the sentence that includes the external character with the internal character, write the sentence separately, and use the internal character variant corresponding to the external character part. Are extracted and embedded in the original document as invisible information.

If there is no variant of the internal character corresponding to the external character, the external character can correspond to the reading by setting the reading. Furthermore, the number of search candidates can be expanded by setting synonyms such as a maiden name for a known person and an old name for a land name.

Furthermore, for example, in the case of a large-scale character string set that can handle Chinese kanji, Korean kanji, etc. as Unicode (Unicode) that has been proposed as a character code that handles various languages in a unified manner by international standardization, Japanese kanji Not only Chinese characters and Korean characters can be handled.

In the present invention, words that are treated as symbols in the search system, such as “•” and “:”, are inserted as characters before and after the invisible information word so that the invisible information word is explicitly entered into the search system. It can be recognized as a word.

Also, in the present invention, the similarity between the external characters and the variants of the internal characters corresponding to the external characters is stored in the external characters / variant character conversion table, so that by setting a threshold for similarity on the information disclosure side It is possible to adjust the search accuracy by controlling the number of variants of the internal character corresponding to the external character.

A search system and a system that provides a search function for external characters can be constructed as separate systems by pre-assigning intangible words of internal characters corresponding to external characters to electronic documents, so that searches that do not have an external character search function An external character search function can be easily and stably added to the system.

By assigning a character string of one or more corresponding variants of the internal character as invisible information to one external character, the external character can be searched by using multiple internal variants. This makes it possible to perform a search without omission.

An example of an embodiment of a system and method for providing invisible information to an electronic document according to the present invention will be described with reference to the drawings. FIG. 1 shows one form of an information search system in which the invisible information providing system of the present invention is incorporated into an existing search system, and the present invention is not limited to this.
In the present embodiment, the division refers to dividing a sentence into a certain unit, and the morpheme refers to the smallest unit that does not make sense if it is further reduced.

FIG. 1 is a configuration diagram of an information retrieval system incorporating an invisible information providing system. The invisible information providing system of this embodiment is composed of an invisible information providing unit 5 and an external character / variant character conversion table 6. The existing search system is an information browsing unit 1, an information search unit 2, an electronic document data storage unit 4, an information The search index unit 3 is generated from an electronic document data storage unit for search.

The information browsing unit 1 displays a search screen, inputs a search term, transmits the input search term to the information search unit 2, receives and displays a search result from the information search unit 2, and displays electronic document data. The request is transmitted, the electronic document data is received and displayed, and the information search means 2 is connected.

The information search means 2 connects the information browsing means 1, the search index section 3 and the electronic document data storage section 4, reads electronic document data from the electronic document data storage section 4 in advance, creates a search index, and searches It is stored in the index part 3 for The information search means 2 receives a search word from the information browsing means 1 at the time of search, reads an index from the search index unit 3, performs calculation by a specific algorithm such as weighting by appearance frequency, and searches the information browsing means 1. Returns the result. The information retrieval unit 2 reads electronic document data from the electronic document data storage unit 4 in response to a request from the information browsing unit 1 and sends the data to the information browsing unit 1.

The invisible information adding means 5 is connected to the electronic document data storage unit 4 and the external character / variant character conversion table 6 so that the electronic document data in the electronic document data storage unit 4 can be accessed.

The search index unit 3 is installed to improve the search speed, and there is a search system without the search index unit 3. In this embodiment, the presence or absence of the search index unit 3 does not affect the operation. Since this embodiment is a method of embedding invisible information in electronic data, the connection in FIG. 1 may be in any connection form such as interprocess communication or a network.

In the electronic document data, an external character is given in advance an external character code that can uniquely distinguish the external character, and a document is created and registered in the electronic document data storage unit 4.

In the case of an index creation type information retrieval system, an external character retrieval function is added before the process of creating an index.

FIG. 2 is a diagram showing an external character / variant character conversion table 6 representing the similarity of internal characters to external characters. The external character / variant character conversion table 6 is composed of external character codes, external character images, internal variants corresponding to external characters, and similarities between internal characters corresponding to external characters and external characters, and each data is registered in advance. Keep it.

If one horizontal line in the figure is one record and a search is performed with an external character code 001, a plurality of internal characters to which the external character code 001 is assigned and their similarities are searched. As for an external character image, since there is one external character image for each external character code, the external character image may be defined in a separate table, or may be expressed in a font instead of an image in order to express an external character. You may define a font. Further, the image itself need not be defined as long as it is expressed by being replaced with another character such as “#”.

The similarity between the external characters and the internal variants corresponding to the external characters may be set by human subjectivity, image processing such as pattern matching may be used, and the internal variants that correspond to the external characters and external characters It is only necessary to be able to express similarities such as “S” and “S” in Watanabe, but different in appearance but close in meaning.

FIG. 3 is a flowchart showing a flow of a method for providing a variant character string of an internal character corresponding to an external character. This flowchart shows that an external character search function is given to one electronic document data, and when there are a plurality of electronic document data, the processing is repeated for the electronic document data.

As a setting value for adding the external character function, the similarity of the internal character variant corresponding to the external character to be extracted is set and given in advance to the invisible information adding means as an initial parameter.

The invisible information adding means 5 reads the target electronic document data from the electronic document data storage unit 4 (S1), and uses a symbol such as “.” Or “.” As a sentence delimiter to read a sentence from within the electronic document data. Extract (S2). If the text cannot be extracted, a termination determination (S3) is performed, and the processing of one electronic document data is terminated. When the sentence can be extracted, it is determined whether or not the external character is included in the sentence (S4). When the external character is not included, the next sentence is extracted (S2).

If the text contains an external character, the external character / variant character conversion table is searched using the external character code as a search key (S5), and it is determined whether the external character is registered (S6). If an external character is not registered, a process of registering an external character code and an internal variant of the internal character similar to the external character (S7) is performed.

When an external character code is found, the similarity between the external character and the variant of the internal character corresponding to the external character is referred to, and one or more characters whose similarity is equal to or higher than a preset threshold value are extracted. One character having the highest degree of similarity is extracted even if it is equal to or less than the threshold value, and is replaced with an internal character variant corresponding to one or more external characters extracted in the external character portion of the sentence including the external character. If one variant character is extracted, one sentence is created, and if a plurality of variant characters are extracted, a sentence replaced with a plurality of variant characters is created (step S8).

  When a plurality of external characters appear in one sentence, other than one external character is replaced with an internal character, and the series of processes is performed. For the plurality of sentences obtained as a result, only one character replaced with the inner character is returned to the outer character, and the above-described series of processing is similarly performed. Further, if there are a plurality of external characters, the processing for the number is performed.

For example, suppose that the internal character is X, the external character is A, and B, three variants of the external character A are extracted, a1, a2, and a3, and two variants of the external character B are extracted, b1 and b2. When the sentence is “XXXABXX”, a sentence “XXXAb1XX” in which the external character B is replaced with the variant character b1 is created first, and the above processing is performed. Through the above process, three sentences “XXXa1b1XX”, “XXXa2b1XX”, and “XXXa3b1XX” are obtained. Next, b1 is returned to the external character B for the three sentences obtained, and when the above processing is performed on the three sentences `` XXXa1BXX '', `` XXXa2BX '', and `` XXXa3BXX '', `` XXXa1b1XX '' and `` XXXa1b2X '', You can obtain six sentences “XXXa2b1XX”, “XXXa2b2XX”, “XXXa3b1X”, and “XXXa3b2XX”.

One or more sentences in which an external character is replaced with an internal variant are separated. The division writing process uses a morphological analysis program (S9). In this embodiment, morpheme analysis is used for division, but the unit of division is not a morpheme unit but may be a unit larger than a morpheme. For example, when a word including an external character is a noun and there is a continuous noun including the word, the continuous noun may be extracted as a word giving invisible information.

A word corresponding to the external character portion of the original sentence is extracted from the divided sentence (S10). If one sentence is replaced with a variant by this process, one word is extracted. If a sentence is replaced with a plurality of variants, a plurality of words are extracted.
When there are a plurality of external characters in a sentence, a duplicate word with the same variant may be created. Therefore, if there is a word with the same variant, the duplicate is deleted.

Based on the embodiment described above in detail, for example, a case where there is a sentence “I am the external character image 7 (sai) wisteria in FIG. 2” will be described.

“I am the external character image 7 (sai) wisteria in FIG. 2”. “FIG. 2 external character image 7 (sai)” is a variant character “variant character 8 (sai) in FIG. 2 internal character” and “internal character in FIG. If "I'm a variant of 9" is selected as a variant that exceeds the threshold, "I am a variant of the internal character 8 in Fig. 2" and "I'm a variant of the internal character in Fig. 2." 9 ”is created. Next, these two sentences are divided by using a morphological analysis program for the division processing, and “I am the variant character 8 in FIG. 2” and “I am the character in FIG. "This is a variant 9 (Sai) wisteria". From the text divided by the split-writing process, the variant word corresponding to the outer character part is "Figure 2 intial variant 8 (Sai) wisteria" and "Figure “2 internal variant 9” is extracted.

Before and after the extracted word, symbols such as “•” and “:” that are clearly unrecognized as part of the word are added to the electronic document data as invisible information (S11). .

If the electronic data format is XML, then <p> I am <gaiji code = "123" /> Fuji. </ P> "is defined as" <p> I am <gaiji code = " 123 "> <itaiji> Saito </ itaiji> <itaiji> Saito </ itaiji> </ gaiji> This is Fuji. </ P>". When the electronic data format is converted by XSLT, a definition is made to treat it as non-display when <gaij> is displayed from <gaij>.

Similarly, when the electronic data format is HTML format, “I am <IMG SRC =" sai.jpg "> <span style = 'display: none'> Saito </ span> Fuji.” It is treated as invisible information.

If the search system is of a type that creates an index, it is possible to perform a search by using a variant word of an internal character corresponding to an external character by generating an index after adding invisible information to the electronic document data.

In the full-text search system, since a word can be searched regardless of the position of the document, the position to which the invisible information is added may be anywhere in the document.

If there is no variant of the internal character corresponding to the external character, it can be substituted by registering the reading as a variant.

Also, if the external character is a known one such as a person name or place name, the combination of the external character and the internal character is clearly identified, so the maiden name is used as a synonym search function for a person name and the old name is used for a place name. It becomes possible to do.

For large-scale character strings with Japanese kanji such as Unicode, Chinese kanji, and Korean kanji, Chinese kanji and Korean kanji are also used for Chinese kanji and Korean kanji by using the same method as this example. A search function can be added.

The above embodiment is the case of XML or HTML, but any data format can be used as long as it is an electronic document data format that can be handled by an information search system and can embed invisible information.

Even computers that can handle Unicode do not have fonts that are compatible with all codes. When the information retrieval system and browsing system are separate computers and connected via a network, all the Unicode fonts are registered in the information retrieval system, and the electronic document data is the characters defined in JIS 1st and 2nd standards. Is described. Characters other than the JIS first and second levels in the electronic document are displayed as images, and readings and words are registered as invisible information by this method. Even a computer whose browsing system holds only the JIS first and second levels can perform display and search from reading. Further, even if a browsing system having a font of another JIS first and second level or higher performs a search using characters other than the JIS first and second levels, the search can be performed.

For example, “Shenzhen” of “Shenzhen” that cannot be displayed in the JIS first and second levels is displayed in kana like “Shenzhen”, but can be expressed in Unicode. In the method of the present embodiment, it is possible to hold the Chinese character “Shenzhen” as invisible information, so it is possible to respond to a search request for the Chinese character “Shenzhen”.

In an embodiment of the present invention, a character string including an internal character variant corresponding to the external character is extracted from a sentence including the external character code using a conversion table of the external character code and the internal character variant, and is provided as invisible information. Shows an example that makes it possible to search for a character string including an external character in an internal character string. However, an internal character code is registered in the external character code field of the conversion table, and the internal character and the internal character are different. By associating characters, it is possible to search for a character string including a variant character of the internal character in the character string of the internal character. For example, if the search word is “Saito” and “Saito” and “Sai” as a variant character, it is added as invisible information by the above method. “Saito” can be searched, and it is possible to eliminate the distinction between variants and eliminate the omission of search.

The block diagram of the search system of a present Example is shown. It is a figure which shows the external character and the variant conversion table showing the similarity degree of the variant of the internal character with respect to the external character. It is a flowchart which shows the flow of the provision method of the character string of the variant of the internal character corresponding to an external character.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Information browsing means 2 Information search means 3 Search index part 4 Electronic document data storage part 5 Invisible information provision means 6 External character / variant character conversion table 7 External character image 8, 9 Variant character of internal character

Claims (5)

External character / variant character conversion table in which external character codes, internal characters corresponding to external characters are stored in association with the character shape and meaning, and
Means for retrieving an external character code of the external character / variant character conversion table from an external character code included in a sentence in the electronic document including the external character code, and searching for one or a plurality of internal character variants;
Means for creating a sentence in which the retrieved one or more internal characters are substituted for the sentence in the electronic document;
Means for sharing one or more sentences created by replacing the variant characters;
Means for extracting a variant character of an internal character corresponding to an external character from the character string thus written as a character string, and embedding the extracted character string as information invisible in electronic information browsing software in an electronic document; A system for providing invisible information to an electronic document, comprising: 2. The system for adding invisible information to an electronic document according to claim 1, wherein an external character code, a character combined with the external character code, a reading thereof, and a synonym are set in the external character / variant character conversion table. The external character / variant character conversion table has a similarity between an external character shape and an internal variant corresponding to the external character, and controls the number of characters to be searched using the similarity in the external character code search means. The system for providing invisible information to an electronic document according to claim 1. The electronic document according to claim 1, 2 or 3, wherein the retrieval system extracts the character string clearly divided by inserting symbols before and after the character string embedded as the invisible information. Invisible information grant system. For the text of the electronic document composed of external characters and internal characters, replace all the text of the electronic document with internal characters,
Write down the sentence replaced with the internal character,
Extracting a variant of the internal character corresponding to the external character portion of the divided character string from the external character code and the external character / variant character conversion table storing the internal variant of the internal character corresponding to the external character,
A method for providing invisible information to an electronic document, wherein the extracted character string is embedded in the original electronic document as information that becomes invisible with the electronic information browsing software.

Images