JP3096499B2 - Word segmentation processing method - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a document preparation method for preparing a document in a natural language using alphabet characters, and more particularly to a word division processing method for dividing a word and adding a hyphen at the time of line feed of a sentence. It is.

[0002]

2. Description of the Related Art In recent years, English word processors that can electronically edit and print natural language sentences using alphabet characters, for example, English sentences, have been put to practical use.

In such a conventional document creation device such as an English word processor, when displaying or printing a sentence composed of alphabetic words in a document creation area of a size determined by a predetermined display or printing format, a display line is displayed. Alternatively, a word existing at a line feed position of a print line may need to be divided by a hyphen, and divided and displayed over two upper and lower lines.

In this type of conventional document creation apparatus, in order to perform the above-described word division processing, a memory called a word dictionary that stores a large number of words in association with division position information of the words is stored. Used (Japanese Patent Publication No. 57-323)
No. 82). In such a word dictionary, it is necessary to previously store hyphen position information indicating a dividable position at which a hyphen can be inserted in a word in association with all words.

However, for example, the number of general words required for an English sentence reaches tens of thousands of words, and a large-capacity memory is required to create a word dictionary corresponding to this. In addition to the drawback that the time required for reference processing of the dictionary is increased, there is a disadvantage that it is not possible to obtain division position information for a word that does not exist in the word dictionary.

[0006]

SUMMARY OF THE INVENTION According to the present invention, when a document creation area is restricted by a display or print format and it is necessary to divide a word in accordance with the above-described line feed processing, a division position for each word is set. A word division processing method or a word division apparatus capable of quickly specifying an appropriate word division position for all words without using a large-scale word dictionary having information, and a document creation processing method or a document Provide a creation device.

[0007]

According to the word segmentation method of the present invention, the phoneme code string segmentation unit information determined in consideration of the word segmentable positions of various words and the segmentability by the segmentation unit are determined. By referring to the phoneme pattern table registered in advance in association with the division possibility information shown,
For all the words, the appropriate word division position for pronunciation is specified based on the phoneme code string of the word.

The word segmenting apparatus of the present invention divides phoneme code string segmenting unit information determined in consideration of the word segmentable positions of various words and segmentability information indicating the segmentability by the segmenting unit. It is provided with a phoneme pattern table registered in advance in association with each other, and by referring to this table, for each word, based on the phoneme code string of the word, a division position of a word appropriate for pronunciation is determined. Identify and split words.

According to the document creation processing method of the present invention, the phoneme code string division unit information determined in consideration of the word division possible positions of various words, division possibility information indicating the division possibility by this division unit, and By referring to a phoneme pattern table registered in advance by associating a word with a word recognized as needing to be divided in the input document, based on the phoneme code string of that word, The plurality of dividable positions are specified, and the number of characters in the first half of the word is determined based on the display format set in the input document .
Set the range, the number of characters in the first half of the word is within the range
The word division position is specified so that

[0010] The document creation apparatus of the present invention divides the phoneme code string division unit information determined in consideration of the word division positions of various words and the division possibility information indicating the division possibility by the division unit. A phoneme pattern table registered in advance and associated with it is provided, and by referring to this table,
For words that are recognized as needing to be divided in the input document, based on the phoneme code string of the word, specify a plurality of dividable positions of words that are appropriate for pronunciation, and set them in the input document. The first half of the word based on the display format
Set the range of the number of characters in the part,
The word division position is specified so that the number falls within the above range .

[0011]

According to the present invention, when it is necessary to divide a word due to the limitation of a displayable or printable area, a table in which a phoneme code string of a unit of division and the possibility of division by this phoneme code string are used is used. For a word, it is possible to quickly specify a word division position that is likely to be pronunciationally plausible.

[0012]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS First, the principle of a word division processing method according to the present invention will be described.

When a word is composed of alphabetic characters, there is a sound that can be easily represented for each character of the alphabet. The languages represented by the alphabetic characters include languages of Western European countries such as English, French, and Spanish. There are methods of classifying characters by phonograms such as vowels / consonants. For example, in the case of English, 'a', 'e', 'i', 'o', 'u', '
y ′ and the like are often used to represent vowels. Therefore, a rule is prepared for converting the sound into a sound (phoneme code) that can be easily represented by alphabet characters. FIG. 2 shows an example of such a conversion table.

Next, in the language, a table (phoneme pattern list) in which various types of divideable units of words are phoneme-encoded and their divideability information are associated with each other.
Prepare FIG. 3 shows an example of such a table.
The division possibility information in this case indicates the possibility that a certain phoneme pattern is actually used for dividing a word, and this information may be any information such as a numerical value that indicates the magnitude of the possibility. The setting of such division possibility information may be performed by guesswork based on experience, but the result of statistically measuring the frequency at which phoneme patterns are actually used as word division units using a large-scale word set May be used.

Using this list, matching is performed so that the word to be divided is constituted by the phoneme pattern with the highest division possibility. In the matching, first, a word to be divided is converted into a phoneme code string. Next, several combinations of phoneme patterns are generated so that this phoneme code string is realized by combining phoneme patterns as shown in FIG. However, not all combinations of phoneme patterns are possible, and there are combinations that cannot actually be pronounced. In order to exclude such a combination that is not actually possible, a measure as shown in FIG. 4 is used. Among the combinations of patterns after the impossible combinations have been excluded as described above, the one that is highly likely to divide each of the phoneme patterns that constitute it is selected as the one that indicates the most likely division position in terms of pronunciation. [Structure of Embodiment] Next, the structure of an embodiment to which the principle of the above-described word division processing method of the present invention is applied will be described.

FIG. 1 is a block diagram of an embodiment of an English document creation processing apparatus.

The input device 1 includes a typewriter, a keyboard,
It may be any of a magnetic tape reader, a character recognition device, or another input device.

The memory 2 is constituted by a RAM or the like, and stores coded alphabetic characters generated by the input device 1.

The display device 3 may be a CRT or another display device. This display device 3 is always a memory 2
Display the contents of

Output device 4 may be a printer, a data communication device, or another output device.

An end-of-line word recognizer 5 recognizes an end-of-line word that needs to be divided when the input of characters exceeds the display width of the set document.

The conversion table 7 describes a correspondence table between characters and phoneme codes, an example of which is shown in FIG. By using the table of FIG. 2, 'a', '
e ',' i ',' u ', and' o 'are replaced by the symbol' v ',' y 'is the symbol' a ', and other characters are replaced by the symbol' c '. .

The character-phoneme converter 6 converts an English word into a phoneme code string using the conversion table 7.

The pattern comparator 8 compares the phoneme code string output from the character-phoneme converter 6 with the phoneme pattern table 9, and the phoneme code string is constituted by the phoneme pattern of the phoneme pattern table 9. Generate a combination of patterns. At this time, the combination is generated so as not to meet the conditions described in the comparison condition list 10.

In the phoneme pattern table 9, a representative phoneme pattern and the possibility of actually using the phoneme pattern as a unit of English word division in English (division possibility)
Are recorded in association with each other, an example of which is shown in FIG. In the example of FIG. 3, for each phoneme pattern, a value normalized by setting the phoneme pattern having the highest division possibility to 1.0 is used as division possibility information.

The comparison condition list 10 describes a group of conditions used when generating a phoneme code string by a combination of phoneme patterns in the phoneme pattern table 9, and the comparison condition list 10 is shown in FIG. . In the example of FIG. 4, for example, "" -ed ","-ing ","-th "
Such conditions are described that suffixes such as "do not compare assuming that they can be split" (C1) and "Do not combine phoneme patterns such that four or more consonants continue" (C2).

The division position determiner 11 selects a combination of phoneme patterns having a high possibility of division from the combinations of the phoneme patterns generated by the phoneme pattern comparator 8, and adjusts the English word according to the display width of the document. Is determined. [Processing Operation 1 of Embodiment] Next, a specific process when the English text shown in FIG. 5 is used will be described. FIG. 5 shows a case in which an English sentence is displayed in a fixed format with a display width of 40 characters and left and right margins. (1) Divided Word Recognition Processing The word string input from the input device 1 is read into the memory 2 and displayed on the display device 3 at the same time. FIG. 5 shows the display result of the text on the display device 3 used in the present embodiment.
Since the word "horrible" at the end of the line exists at the line feed position between the display line 12 and the display line 13, the word "horrible" must be displayed in two lines, the display line 12 and the display line 13.
The recognition of the word existing at such a line feed position is performed by the end-of-line word recognizer 5. The display line 14 in FIG. 6 is an output example in the case where the display is performed without depending on the format. However, as shown in this example, the line end word recognizer 5 outputs the word "
In consideration of the fact that a hyphen is added to the divided character string, the first half of the word is composed of three characters ("h
or ").

FIG. 7 shows the flow of processing until this word "horrible" is bisected by the present invention shown in FIG.
And the description of the specific processing will be continued with reference to FIG. (2) Phoneme code conversion processing The input word “horrible” 15 is stored in the conversion table 7
, The character-phoneme converter 6 causes the phoneme sequence "cvcc"
vccv "16. (3) Word segmentation position specifying process Next, the phoneme pattern comparator 8 refers to the phoneme pattern table 9 as shown in FIG. 3 and the comparison condition list 10 as shown in FIG. A combination of phoneme patterns corresponding to No. 16 is generated.

The phoneme pattern combinations 17 to 20 shown in FIG.
Indicates four of the possible combinations of phoneme patterns that have the highest sum of the dividability information values of the respective phoneme patterns that make up the combinations. Numerical values 21 to 24 described corresponding to the combinations 17 to 20 represent the division possibility information values of the phoneme patterns constituting each combination, and numerical values 25 to 2
Numeral 8 indicates the sum of the division possibility information values in each combination. The division position is specified by referring to the phoneme pattern combination 17 having the highest sum of the division possibility information values, and two division positions are specified by three phoneme patterns constituting the phoneme pattern combination 31. (4) Word Division Processing Next, the division position determination unit 11 determines a position at which a word is bisected. Earlier, it was stated that the division had to be done so that the first half of the word would fit within three characters, but this condition is met by dividing the first phoneme pattern that makes up the combination 17 as the first half. Therefore, the input word is divided into character string groups 30 at the positions indicated by the combinations 29, and the character strings are connected by hyphens (-) to generate a divided character string 31.

In the above description of this embodiment, the combination having the highest division possibility information value is selected. However, if only one division position is specified, several possible combinations are selected. Of these, the division position common to many combinations may be selected. [Processing Operation 2 of Embodiment] Next, another embodiment using the format of the table of FIG. 8 will be described. In this example, a format is required in which a sentence is displayed in a frame of a specified width (18 characters) in a table. (1) Divided Word Recognition Processing The word string input from the input device 1 is read into the memory 2 and displayed on the display device 3 at the same time. FIG. 8 shows a display result of the table used in the present embodiment on the display device 3. In this figure, the word “Transylvania” at the end of the display line 32 is displayed between the display line 32 and the display line 33. Display line 32 because it exists at the line feed position
And the display line 33 must be displayed in two lines. The recognition of the word existing at such a line feed position is performed by the end-of-line word recognizer 5. The display line 34 in FIG. 9 is an output example in the case where the display is performed regardless of the format. As shown in this example, the end-of-line word recognizer 5 converts the word “Transylvania” at the end of the line into a character string after division. Considering that a hyphen is added, it is recognized that it is necessary to divide the word so that the first half of the word is within 5 characters (“Trans”).

Hereinafter, this word "Transylvani"
FIG. 10 shows the flow of processing until “a” is divided into two by the present invention in FIG. 1. The specific processing will be continued with reference to FIG. 10. (2) Phoneme code conversion processing Input word “Transylvania” 35 Is a phoneme sequence by the character-phoneme converter 6 according to the conversion table 7.
ccvccaccvcvv "36. (3) Word segmentation position specifying process Next, the phoneme pattern comparator 8 refers to the phoneme pattern table 9 as shown in FIG. 3 and the comparison condition list 10 as shown in FIG. A combination of phoneme patterns corresponding to 36 is generated.

The combination of phoneme patterns 37 to 37 shown in FIG.
Reference numeral 39 indicates three of the possible combinations of phoneme patterns, which have the highest sum of the division possibility information values of the respective phoneme patterns constituting the combination. In addition, each combination 37-39
Numerical values 40 to 42 described in correspondence with the numerical values 43 to 42 represent the dividability information values of the phoneme patterns constituting each combination, respectively.
-45 represent the sum of the division possibility information values in each combination. The division position is specified by the combination 37 of the phoneme patterns having the highest sum of the division possibility information values.
Is performed, and three division positions are specified by four phoneme patterns constituting the combination 37 of phoneme patterns. (4) Word Division Processing Next, the division position determination unit 11 determines a position at which a word is bisected. Earlier, it was stated that the division must be made so that the first half of the word falls within 5 characters, but this condition is satisfied by dividing the first phoneme pattern constituting the combination 32 as the first half. Therefore, the input word is divided into character string groups 40 at the positions indicated by the combination 39, and a divided character string 41 is generated by connecting the character strings with a hyphen.

[0033]

As described above, according to the present invention, a word division position determined by using a word dictionary having large-scale division position information in the prior art is replaced with a word division unit. By using the phoneme information, it is possible to quickly identify the most probable word division position for all words without using a word dictionary with large-scale division position information. Become.

[Brief description of the drawings]

FIG. 1 is a block diagram of a document creation device according to the present invention.

FIG. 2 is a diagram showing a character-phoneme conversion table in FIG. 1;

FIG. 3 is a diagram showing a phoneme pattern table in FIG. 1;

FIG. 4 is a diagram showing a comparison condition list in FIG. 1;

FIG. 5 is a diagram showing a process of dividing a word by the word dividing process.

FIG. 6 is a diagram showing a process of dividing a word by the present word dividing process.

FIG. 7 is a diagram illustrating a process of dividing a word by the present word dividing process.

FIG. 8 is a diagram showing a process of dividing a word by the word dividing process.

FIG. 9 is a diagram showing a process of dividing a word by the present word dividing process.

FIG. 10 is a diagram illustrating a process of dividing a word by the word dividing process.

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 1 Input device 2 Memory 3 Display device 4 Output device 5 End-of-line word recognizer 6 Character-phoneme converter 7 Conversion table 8 Pattern comparator 9 Phoneme pattern table 10 Comparison condition list 11 Division position determiner 12-14 Display line 15 Input word 16 Phoneme Sequence 17-20 Phoneme Pattern Combination 21-24 Dividability Information Value of Each Phoneme Pattern 25-28 Total Dividability Information Value in Each Combination 29 Combination of Input Words 30 Character String Group 31 Split Character String 32- 34 Display Line 35 Input Word 36 Phoneme String 37-39 Combination of Phoneme Patterns 40-42 Dividability Information Value of Each Phoneme Pattern 43-45 Total of Dividability Information Value in Each Combination 46 Combination 47 Character String Group 48 Division String

Claims (4)

(57) [Claims] 1. Registering in advance the unit division information of a phoneme code string determined in consideration of the word division possible positions of various words and the division possibility information indicating the division possibility at the division positions. A phoneme pattern table, and a phoneme code conversion process of converting each alphabetic character constituting a word into a phoneme code to generate a phoneme code string in word units. By constructing the phoneme code string obtained from the phoneme code conversion means for the word obtained from the combination of the most likely division units of the input phoneme code string by the table reference processing, the dividable position of the word is determined. A word division processing method comprising: a specified word division position specifying process. 2. A phoneme code conversion means for converting each alphabetic character constituting a word into a phoneme code to generate a phoneme code string in word units, and a decision is made in consideration of a word-dividable position of various words. A phoneme pattern table pre-registered by associating the division unit information of the phoneme code string with the division possibility information indicating the division possibility at this division position, and a phoneme code conversion means for the input word obtained from the phoneme code conversion means. The input phoneme code string is composed of the most likely combination of the division units based on each of the division unit information of the phoneme pattern table and the division possibility information of the phoneme code string to be divided. A word division device comprising: a word division position specifying unit for specifying a position. 3. A phoneme code string division unit information determined in consideration of word division possible positions of various words and division possibility information indicating division possibility at the division positions are registered in advance in association with each other. And a word recognition process that recognizes whether a word in an input document is a word that needs to be divided, and converts each alphabetic character constituting the word into a phoneme code. And a phoneme code conversion process for generating a phoneme code sequence for each word, and a phoneme code sequence obtained from the phoneme code conversion means for the word recognized in the word recognition process. Based on the division unit information and its division possibility information, the input phoneme code string is composed of a combination of division units having the highest possibility, so that the word A word division position specifying process for specifying a splittable position, and the unit based on a display format set in an input document.
Set the range of the number of characters in the first half of the word,
A word division process for determining a division position of a word recognized in the word recognition process so that the number of characters within the range falls within the range . 4. A document input means for providing a document as an input, a division unit report of a phoneme code string determined in consideration of a word division possible position of various words, and a division possibility at the division position. What is claimed is: 1. A document creation processing apparatus using a phoneme pattern table registered in advance in association with division possibility information, comprising: a word recognition unit for recognizing whether a word in an input document needs to be divided; A phoneme code conversion unit that converts each alphabetic character constituting the word into a phoneme code to generate a phoneme code string in word units, and obtains a word recognized in the word recognition process from the phoneme code conversion unit. The input phoneme code string is set to the most likely set of division units based on each of the division unit information of the phoneme pattern table and its division possibility information. By configuring the Align, the word dividing position specifying means for specifying a dividable position of the word, on the basis of the display format is set in the input document single
Set the range of the number of characters in the first half of the word,
A document creating apparatus comprising: word dividing means for determining a dividing position of a word recognized in the word recognition process so that the number of characters in the minute falls within the range .