JP5336779B2 - Information processing apparatus for performing character string conversion, character string conversion method, program, and information processing system - Google Patents

Description

  The present invention relates to a character string conversion technique, and more specifically, an information processing apparatus, a character string conversion method, a program, and an information processing system for converting a character string described in original characters into a character string of another language such as an alphabet. About.

  In recent years, due to the globalization of economic activities, the development of transportation facilities, the spread of the Internet, etc., it is often necessary to process multiple languages such as Japanese, English, Chinese and Korean simultaneously. There are many tasks that require processing of multiple languages. For example, as for the name of a person, there are many words unique to the name of the person, and the time for determination is often limited. When names in the Indo-European-speaking area such as alphabets are described in alphabets, for example, there is no need for alphabet conversion. However, after a person's name derived from a language other than Indo-European-speaking languages, or after a name of an Indo-European-speaking person is written in another language, such as Japanese, Chinese, or Korean, Various problems arise when generating the alphabet.

  For example, Japanese katakana is a phonetic character, and a case where a character string of “raito” or “light” in kana notation is converted into an alphabet will be considered as an example. For example, for the katakana notation “light”, there are many possible alphabet notations such as “right”, “light”, “write”, “wright” as the same or similar alphabet notation.

  When the name of an Indian / European-speaking person is katakana, for example, “Henry”, an English speaker pronounces “Henry” and is given katakana notation. However, when “Henry” is written in the alphabet, the French speaker pronounces “Henri”, so that different katakana notation is given even though the original alphabet display is the same. When such an alphabet notation is converted into a katakana notation, and when another person converts the katakana notation into an alphabet notation again, a unique conversion is not necessarily given.

  The words used in the language can include place names, coined words, synthesized words, etc. in addition to the names of people. It is also possible to check the spelling of these words by referring to the dictionary each time. However, voice calls such as telephone calls often require processing in real time based on the pronounced words, and it may not be possible to look up the dictionary by quoting each time. In some cases, errors may occur.

Techniques for performing the above-described alphabet conversion are also known. For example, Japanese Patent Laid-Open No. 8-339376 (Patent Document 1) discloses an apparatus and system for efficiently searching for foreign language words registered in a database using katakana words. In Patent Document 1, a phonetic symbol / katakana correspondence table for storing the correspondence between phonetic symbols and katakana characters, and phonetic symbols of registered data composed of foreign words and phonetic symbols input from a registered data input unit, Using phonetic symbol katakana conversion means for converting into katakana words using the katakana correspondence table. The system of Patent Literature 1 calculates a similarity Ri for each katakana word registered in a database of katakana words of search keywords, and uses a foreign language word corresponding to a katakana word having a word similarity Ri equal to or higher than a specified value as a search result. Output.
JP-A-8-339376

  As described above, since the technique described in Patent Document 1 performs katakana foreign language conversion using a conversion table, it requires time and effort for dictionary maintenance. In addition, the accuracy of katakana-alphabet conversion depends on the dictionary accuracy, and word similarity is determined using character comparison between katakana characters. There are cases in which conversion is not possible with sufficient accuracy, such as not being pronounced when converted, that is, the same katakana notation due to the presence of silent characters, etc.

  The inconvenience of the prior art described above is due to the calculation of the degree of similarity by comparison between katakana. In addition, after obtaining the search keyword and calculating the similarity, referring to the pronunciation in response to the similarity and searching for foreign language words, the consumption of hardware resources such as memory allocated for the table In view of the microprocessor occupation time such as the search time and the search accuracy, it cannot be said to be sufficient for realizing the real-time response.

  The present invention has been made in view of the above-described problems of the prior art, and the present invention can be realized by directly associating original character strings such as katakana, hiragana, and hangul with other languages such as alphabets. It is an object of the present invention to provide an information processing apparatus, a character string conversion method, a program, and an information processing system that can convert a character string into a character string corresponding to another language such as an alphabet.

  Furthermore, the present invention relates to an information processing apparatus capable of converting an original character string such as katakana, hiragana or hangul into a character string of another language such as a maximum likelihood alphabet corresponding to the original character string, a character string conversion method, An object is to provide a program and an information processing system.

  Furthermore, the present invention associates phonemes of other languages such as alphabets with original character strings such as katakana, hiragana, and hangul, and enables conversion to a maximum likelihood alphabet character string using a probability model. An object is to provide an information processing apparatus, a character string conversion method, a program, and an information processing system.

  The present invention has been made in view of the above problems of the prior art, and in the present invention, the cost is defined for the correspondence between the phoneme of the original character string and the phoneme of the conversion destination character string, Adopt an alignment cost that characterizes the difference in phoneme characteristics between the original string and the destination string. The alignment cost enables the character string conversion to include a linguistic phoneme characteristic difference between the original character string and the conversion target character string. In the alignment process, the phoneme sequence of the original character string and the phoneme sequence of the conversion destination character string are used to generate a route map around each axis. In the route map, a cell is defined by a unit phoneme of an original character string and a unit phoneme associated with a conversion destination character string. Then, for the route, three unit routes corresponding to three directions, that is, a direction along each axis and an oblique direction with respect to each axis are designated, and different costs are assigned to each. Each cost is determined so as to give the shortest path, that is, the minimum cost path, and a cost model is created.

  The created cost model is then used for automatic alignment of unaligned cases. Automatic alignment of unaligned cases determines the phoneme sequence of the original string, generates multiple path maps from the phoneme sequence of the destination string that may be associated with the phoneme sequence of the original string, and minimizes Search cost path. The searched minimum cost path gives the alignment result of the conversion destination character string with respect to the original character string. In the present invention, the route search may be performed using the Viterbi algorithm.

Furthermore, in the present invention, using the result of the automatic alignment described above, a probability model is generated in which the original character phoneme is an observation sequence and the conversion destination character string is a state transition sequence. The probabilistic model includes a conversion probability π _i of an original character phoneme and a conversion destination phoneme corresponding to the original character, and a conversion probability π _j for converting a subsequent phoneme corresponding to the sequence of the original character string into a conversion destination phoneme. The given transition probability P _ij is generated as a transition probability table in which the original character phoneme or the phoneme of the conversion destination character string is associated.

  When performing character string conversion processing, the original character string is acquired, phoneme decomposed, and the conversion likelihood of the other language character string generated by state transition from the original character string using the result of phoneme decomposition and the probability model. Character string conversion is performed by calculating a degree χ and outputting a character string of another language that gives the maximum conversion likelihood χ as a character string conversion result.

  A plurality of probability models can be used corresponding to different languages, and a language type can be estimated from a specific original character string and a character string can be converted into the estimated language.

  Furthermore, the present invention provides a character string conversion method and program capable of information processing for causing the information processing apparatus to execute the above processing.

  The present invention can be implemented as a web server that provides a character string conversion service to a web client via a network.

  According to the present invention, it is possible to flexibly cope with the difference in linguistic phoneme decomposition between the original character string and the other language of the conversion destination, and further directly convert the phoneme of the original character string and the phoneme of the conversion destination character string. To provide an information processing apparatus, a character string conversion method, a program, and an information processing system that can improve conversion accuracy, flexibly cope with language diversity, and do not waste hardware resources. it can

  Hereinafter, although this invention is demonstrated with embodiment, this invention is not limited to embodiment mentioned later. FIG. 1 is a functional block diagram of the information processing apparatus 100 according to the present embodiment. The information processing apparatus 100 can be implemented as a personal computer, a workstation, or a server dedicated machine.

  When the information processing apparatus 100 is mounted as a server-dedicated machine, the microprocessor may be a microprocessor of CISC architecture such as PENTIUM (registered trademark), a PENTIUM (registered trademark) compatible chip, or POWER PC (registered trademark). A RISC architecture microprocessor can be used, and may be single-core or multi-core. When the information processing apparatus 100 is implemented as a server-dedicated machine, the operating system (OS) can use WINDOWS (registered trademark) 200X, UNIX (registered trademark), LINUX (registered trademark), or the like.

  Further, when the information processing apparatus 100 is implemented as a server-dedicated machine, the CGI, servlet implemented using a programming language such as C ++, JAVA (registered trademark), JAVA (registered trademark) BEANS, PERL, RUBY, etc. A server program such as APACHE or IIS is executed to process various requests via a network (not shown). When the information processing apparatus 100 is implemented as a server dedicated machine, the information processing apparatus 100 can be a web server. In addition, the information processing apparatus 100 can be a dedicated server that enables distributed computing using CORBA (Common Object Resource Broker Architecture).

When the information processing apparatus 100 is implemented using a personal computer or a workstation, the microprocessor (MPU) may include any single-core processor or dual-core processor known so far. In this embodiment, the information processing apparatus 100 may be installed with any operating system such as WINDOWS (registered trademark), UNIX (registered trademark), LINUX (registered trademark), or MAC OS (registered trademark). Further, when the information processing apparatus 100 functions as a web client,
Browser software such as Explorer (registered trademark), Mozilla, Opera, and Netscape (registered trademark) Navigator can be used to access a web server using the HTTP protocol.

  The functional blocks of the information processing apparatus 100 in FIG. 1 will be described in detail below. The information processing apparatus 100 according to the present embodiment uses a phonetic character string including katakana, hiragana, and hangul as an original language. In addition, other languages such as alphabet, Islamic, Hebrew, Slavic, Hindu, etc. can be used. Hereinafter, for the purpose of specifically explaining the present embodiment, the original language will be described as hiragana or kana including katakana, and the other languages will be described as alphabets. Each functional unit described below is realized as a functional unit of the information processing apparatus 100 by developing a program in a memory or the like and causing the CPU or the microprocessor to execute the program.

  The information processing apparatus 100 includes an information processing apparatus main body 110, an input unit 112, and an input / output interface 114. In the embodiment shown in FIG. 1, the input unit 112 can use a keyboard or the like. The input unit 112 also inputs a kana character string as an original character string, and sends the kana character (string) to the kana character acquisition unit 116 that is the original character acquisition unit via the input / output interface 114. In addition, the information processing apparatus 100 shown in FIG. 1 includes a display device, but the description is omitted because it is not related to the gist of the present embodiment.

  The kana character acquisition unit 116 sends the received kana character string to the phoneme decomposition unit 120. The phoneme decomposition unit 120 refers to the phoneme data storage unit 126 and decomposes the received kana character string into phoneme sequences that are used as conversion processing units. For example, when the kana character string is “information”, the original character phonemes are decomposed into “i”, “n”, “fo”, “mae”, “sho”, and “n” phonemes. In the present embodiment, the original character phoneme is decomposed into a unit that can be independent as kana, and when there is a long sound or a prompt sound, the long sound or the prompt sound is combined with the immediately preceding kana character and registered as a phoneme.

  Note that phoneme data associates phonemes that should be the smallest linguistic unit for kana character strings and alphabetic character strings, for example, for alphabetic phonemes that should be classified as phonemes corresponding to the kana phonemes for specific kana phonemes. Can be created as a data structure to be registered, and can be registered in the phoneme data storage unit 126 in advance.

  Further, when another language such as Korean is used as the original character, the original character phoneme can be registered in an appropriate unit from the viewpoint of linguistics. If the original language is not kana and the other languages are not alphabets, the original language phonemes and the other language phonemes may be associated with each other and registered in the phoneme data storage unit 126. Note that the original character phoneme and the conversion destination phoneme are used as a unit for defining a cell in a route search in learning described later.

  When the phoneme decomposition of the kana character string is completed, the phoneme decomposition unit 120 sends the phoneme decomposition result to the conversion likelihood calculation unit 122. The conversion likelihood calculation unit 122 looks up the probability model storage unit 128 and calculates the conversion likelihood. The probability model storage unit 128 stores a probability model, and the probability model registers a conversion probability of a conversion destination phoneme that is a phoneme of a language that performs character conversion corresponding to a specific kana character. The conversion probability is, for example, the probability for associating the kana phoneme “I” with the alphabet phoneme “I”.

  The probability model registers the transition probability between consecutive phonemes. The transition probability is a probability that “N” appears as a subsequent phoneme of the alphabet “I”, for example. In the specific embodiment to be described, the probability π of associating alphabetic phonemes with alphabetic phonemes and the transition probability P of continuous phoneme sequences corresponding to kana characters are registered as, for example, a transition probability table.

The conversion likelihood calculation unit 122 refers to the probability model storage unit 128 and forwards the acquired phoneme sequence of the kana character to acquire the conversion probability π ₁ of the _first kana character, The conversion probability π ₂ and the transition probability P ₁₂ between the first kana character and the subsequent kana character are acquired. Repeat the above process until the phoneme sequence is complete, and use the generated probability value to multiply, multiply, or use other appropriate formulas for kana phoneme-alphabet phoneme sequences from head to tail The conversion likelihood χ until is calculated.

  The maximum likelihood phoneme sequence determination unit 118 receives a notification that the probability calculation of the conversion likelihood calculation unit 122 has been completed, searches the result list created by the conversion likelihood calculation unit 122, and in the embodiment described below, the conversion likelihood The alphabet sequence having the largest χ is determined as the most likely alphabet character string candidate, and is output to the result output unit 130 as a character string conversion result. In the preferred embodiment, the output result is displayed on the desktop screen of the display device, and can be hard-copied by the user as appropriate.

  Note that the phoneme data stored in the phoneme data storage unit 126 and the transition probabilities stored in the probability model storage unit 128 are created by the preprocessor 124 before executing the character conversion process, and the hard disk device (not shown), It can be registered in EEPROM, EPROM or the like. In other embodiments, as long as the phoneme data and the transition probability data can be read as execution data from a hard disk device or the like to the RAM of the information processing device 100 during execution of the program, the information processing device 100 is preprocessord. 124 may not be implemented.

  FIG. 2 shows a functional block configuration 200 of the preprocessor 124 included in the information processing apparatus 100. Aligned cases and unaligned cases are input to the preprocessor 124 by a system administrator or a developer. An aligned case is defined as a data set in which kana phonemes and alphabetic phonemes are already associated. An unaligned case is defined as a data set consisting of an alphabet string and kana string that should be associated with each other, and is used to improve the accuracy of the probability model using a cost model described later. Used as learning data.

  Upon receiving the aligned case 210, the preprocessor 124 passes it to the cost model generation unit 214. The cost model generation unit 214 calculates the kana phoneme-alphabet phoneme association cost using the kana phoneme-alphabet phoneme association relationship of the data of the aligned case 210. Then, the cost model generation unit 214 performs cost calculation on the entire element set of the acquired aligned case 210, and the cost model for associating the kana phoneme and the alphabet phoneme from each aligned case. Is generated and stored in an appropriate storage area.

  In addition, the alignment processing unit 216 performs alignment of the unaligned case 212 using the generated cost model, and calculates the cost between phonemes given by the cost model. Further, the alignment processing unit 216 generates a plurality of different route maps by associating a plurality of alphabetic phonemes that can be associated with kana phonemes, applies a cost model to each route map, and performs the lowest cost alignment. Is determined to generate a learning case by alignment, and the result is stored in the alignment result storage unit 220.

  When the processing of the alignment processing unit 216 is completed, the probability model generation unit 218 extracts the result of the alignment result storage unit 220, counts the number of occurrences of alignment examples, and corresponds to the kana phoneme for the kana phoneme-alphabet phoneme. A conversion probability π of alphabetic phonemes and a transition probability P between sequences of each phoneme are generated. In each of the generated probability values, in the embodiment to be described, the conversion probability π of kana to the alphabet and the transition probability P to the subsequent alphabet are registered in the probability model storage unit 128 in the form of a transition probability table.

  FIG. 3 is a flowchart of an embodiment of the probability model generation process of the present embodiment. The process shown in FIG. 3 corresponds to the process executed by the preprocessor 124 shown in FIG. The process of FIG. 3 starts from step S300, and an alignment example set is acquired in step S301. In step S302, the first aligned case is obtained from the aligned case and the alignment cost is calculated. The alignment cost is calculated using the route cost assigned in consideration of the correspondence between the phonemes in the alphabet and the kana phonemes.

  In the calculation of the alignment cost, a route map is generated for the kana phoneme-alphabet phoneme associated with the aligned case, and a kana phoneme-alphabet phoneme cell is allocated on the route map. Furthermore, the unit route along the cell is assigned under a certain rule, routed along the unit system route from the beginning to the end, and executed by summing the costs of the certain rule appearing on the route. be able to. The alignment cost calculation process will be described later in more detail.

  In step S303, it is determined whether or not the cost has been determined for all elements of the aligned case set. If the cost has not been determined for all elements (no), the process branches to step S302, and the entire aligned case set is determined. The cost calculation is accumulated until the process is completed. The aligned case can be created by the program creator on the program creator side.

  If it is determined in step S303 that cost calculation has been completed for all elements (yes), a cost model for kana-phoneme-alphabet phoneme association is generated with reference to the result of cost calculation in step S304. The cost model provides a measure of validity including the phoneme increase and decrease in the alphabet conversion of the correspondence between successive phonemes.

  In step S305, the preprocessor 124 acquires an unaligned case set. In step S306, the information processing apparatus 100 performs alignment processing using the cost model, generates an automatically aligned set by the information processing apparatus 100, and stores it in an appropriate storage area. More specifically, the automatic alignment process is performed by assigning alphabetical phonemes extracted from aligned cases when generating a cost model to kana phonemes of unaligned cases while referring to the cost model. A route map is generated, and a route cost from the first phoneme to the last phoneme is calculated for each route map. After calculating the path cost for all path maps, it is performed by assigning each alphabetic phoneme that gives the minimum cost for a particular unaligned case.

  In step S307, it is determined whether or not alignment has been completed for all set elements in the unaligned case set. If processing has not been completed for all set elements (no), the process branches to step S306 to automatically Repeat the alignment process. On the other hand, if the alignment has been completed for all the set elements (yes), the conversion probability of kana phoneme-alphabet phoneme and the transition probability between phonemes are calculated in step S308, and a probability model is generated. Various types of probabilistic models can be assumed, but the conversion probabilities and transition probabilities between preceding and succeeding phonemes are registered and generated for kana and alphabetic phonemes registered in the cost model. Can do. In step S309, the generated probability model is registered in the probability model storage unit 128, and in step S310, the probability model generation process ends.

  The above processing is processing executed by the preprocessor 124. In the embodiment in which the preprocessor 124 is not mounted, the probability model is obtained from a CD-ROM, DVD-ROM, or the like as execution data of a program for executing the character conversion processing. At the time of installation, it can be stored in an appropriate storage area of the hard disk device. Then, when the execution of the program is started, the program is read from the hard disk device to a high-speed access memory such as a RAM and used for executing the program. When the information processing apparatus 100 includes the preprocessor 124, the probability model is directly registered in the hard disk device, and is read into a high-speed access memory such as a RAM and used for executing the program when the program is executed. In any embodiment, the phoneme data and the probability model are stored in the RAM or the like of the information processing apparatus 100 and used by the program when the program is called for execution.

  FIG. 4 shows the aligned case set 400 acquired by the preprocessor 124 in step S301 of FIG. 3 together with the data structure of its elements. The aligned case set is collected and selected as basic data for creating execution data for program execution on the program developer side, and is generated by associating kana-alphabets in units of phonemes. In the embodiment shown in FIG. 4, for example, katakana is associated in units of phonemes with alphabetic character strings such as information, george, smith, and clinton. In the embodiment shown in FIG. 4, the association is indicated by parentheses (“”). However, the preprocessor 124 can identify a code such as a space, comma, colon, semicolon, and / or any code other than the kana alphabet. Any separation scheme may be used.

  As the number of elements in the aligned case set shown in FIG. 4 is larger, the accuracy of the cost model to be generated is improved, and more accurate character string conversion is possible. In addition, a character string in which a characteristic cost value appears as much as possible during character string conversion, for example, “ex”, “ox”, “ign”, “kno”, etc., whose phoneme number after conversion changes, alphabetic words It is preferable to select an alphabet character string including a so-called silent character and a kana character string that do not pronounce the character as an aligned case set.

  FIG. 5 is a conceptual diagram of the process for calculating the alignment cost for the aligned case executed in step S302 of the process of FIG. The alignment cost is executed by generating a route map 500 from alphabetic phonemes associated with kana phonemes. Alignment cost calculation using aligned examples is to assign alphabetical phonemes to kana phonemes, especially when the number of phonemes is not matched in character string conversion. This is a process for generating an appropriate cost set.

  More specifically, the case where phonemes cannot be correlated one-to-one between kana and alphabets. For example, “IN” is converted to “i” + “n” in kana phonemes, so the number of phonemes is correlated. There is no excess or deficiency. However, the alphabet string “OX” is pronounced as “O” + “K” + “Su” in Katakana, and the number of Katakana characters increases compared to the number of alphabet characters. Corresponds to what needs to be considered. On the other hand, “FOR” is an alphabetic phoneme, and is “F” + “O” + “R”. However, in Kana phoneme, one phoneme is “fo”, so that the number of phonemes in alphabetic phoneme increases. It corresponds to becoming.

  In order to perform the above-described association, in the present embodiment, a cell 510 defined by a unit phoneme of each character string is defined from the beginning to the end of each character string in the route map 500, and along the cell. Define unit route. In this embodiment, the unit route is shown as each of a diagonal route 520, a vertical route 530, and a horizontal route 540. Cost is assigned as a route search problem that assigns a cost to the traveling mode of the route and gives a minimum cost route. • Process the model.

In the embodiment to be described, the progress mode of the route search is limited to the following three rules in the present embodiment.
(1) When a kana phoneme and an alphabetic phoneme are associated with each other, an oblique path 520 that diagonally crosses the cell is advanced.
(2) If the kana phoneme is shorter than the alphabet phoneme, it proceeds along the vertical path 530 of the cell.
(3) If the kana phoneme is longer than the alphabet phoneme, proceed along the horizontal path 540 of the cell.

In the case of an oblique route, since there is no problem in association, a minimum cost C _min is given. In addition, in the case of mapping that travels along the vertical path, in Indo-European languages, there are cases where silent letters that are not pronounced are present, or when kana phoneme prompts or long sounds are considered due to syllables, etc. Decrease but not a serious mismatch, so assign a higher cost C _med than an exact match. On the contrary, the same phoneme frequently appears in the same word in both alphabets and kana, so the horizontal path is compared to other paths in order to eliminate the mapping of inappropriately separated phonemes. It is preferable to increase the cost. On the other hand, as described above, the kana characters may increase within a reasonable range in the natural language like the alphabet character string “OX”, and it is impossible to completely eliminate the correspondence including the horizontal path. Is appropriate. For this reason, the horizontal path is assigned a higher cost of C _max than the other two. Based on the route search rule described above, the cost is calculated using the following formula (1) for the aligned cases.

上記式（１）中、Ｃ_{ＴＯＴＡＬ}は、合計スコアであり、サフィックスａ、ｂ、ｃは、それぞれセル単位での斜め経路の発生数、垂直経路の発生数、水平経路の発生数である。

In the above formula (1), C _TOTAL is the total score, and suffixes a, b, and c are the number of oblique paths, the number of vertical paths, and the number of horizontal paths in cell units, respectively.

For the elements of the aligned case set, the path is uniquely defined, so the total cost is analyzed for each alignment case, and the total cost of the perfectly matched path divided by the number of phonemes is the highest, and the vertical path is A value obtained by dividing the total cost of the path including the phoneme number gives an intermediate value, and a value obtained by dividing the total cost including the horizontal path by the phoneme number gives a higher maximum value than in the other two modes. In this way, the cost values C _min , C _med , and C _max are set.

  In the above-described embodiment, it has been described that fixed costs are assigned to the vertical, horizontal, and diagonal directions. In another embodiment, a different (horizontal) cost for each alphabet, a different vertical cost for each katakana, and a different (slanted) cost for each alphabet / kana pair may be assigned. In the case of this embodiment, it is possible to use the total score given by the following formula (1 ′) instead of the above formula (1), where Cx is the cost of the side x (vertical, horizontal, or diagonal) on the route. it can

  The route cost calculation will be described with reference to FIG. 5. In the embodiment shown in FIG. 5, the kana character string = information and the alphabet character string = INFORMATION. In the aligned case, since it is pre-aligned as i = I, n = N, fo = FOR, mae = MA, sho = TIO, n = N, “FOR”, “MA”, “TIO” are unambiguous. The vertical path 530 is assigned to the other path, and the slant path 520 is assigned to the other paths.

  On the other hand, in step S305 and step S306 in FIG. 3, the path search is performed using the unaligned case set, and the alignment that gives the minimum value of the total cost is set as the optimum association, and the conversion probability and the transition probability are calculated. To learn. An unaligned case is defined as a data set of {alphabet character string, kana character string} to be associated. For example, a case where the unaligned example is “information” will be described with reference to FIG.

  By referring to the phoneme data for the kana character string = information, a kana phoneme sequence of “i”, “n”, “fo”, “mae”, “sho”, “n” is given. On the other hand, for the alphabet string, the phoneme data storage unit 126 is looked up and registered at the stage where the aligned case is processed, and the phoneme sequence is generated by enumerating the alphabet phonemes associated with the same kana phoneme, Create multiple route maps.

Then, for the given plurality of route maps, the route cost is calculated using the rules described in FIG. For example, in the course of alignment by the route search in step S306, the route 560 and the route 570 are searched in addition to the route 550. At this time, since the calculated C _TOTAL has the minimum path 550, the path 550 is adopted as the optimum path and the paths 560 and 570 are discarded, as in the alignment example.

  The above-described path search for alignment using FIG. 5 can be executed by applying, for example, a Viterbi algorithm in which appropriate conditions are set, in addition to the embodiment described in FIG.

  Furthermore, the probability model generation process of this embodiment will be described. In step S308 of FIG. 3, an alphabetic phoneme allocation probability π corresponding to a kana phoneme and a transition probability P between consecutive phonemes are generated by statistically analyzing learning examples from the kana phoneme-alphabet phoneme correspondence example. FIG. 6 is a conceptual diagram illustrating an embodiment of alignment learning using an unaligned case set. By analyzing the aligned case, the information processing apparatus 100 generates a cost model 650 and stores it in an appropriate storage area. The cost model 650 assigns the kana phoneme = a as an alphabetic leading character = a, e, o, r, u, and if the subsequent characters are e, h, r, the cost is 1, respectively. 2 (corresponding to, for example, aero and our respectively). Similarly, the cost is registered for kana phoneme = i.

  Note that cost = 1 is the cost of an oblique path for the purpose of illustration, cost = 2 is the cost of a vertical path, and cost = 3 is the cost of a horizontal path. The cost model 650 is applied to the alignment result for the unaligned case to calculate the total cost, and among the possible paths, as described above, the path that minimizes the total cost is set as a kana phoneme-alphabet phoneme correspondence. Let them learn. Then, as shown in FIG. 6, the probability model 700 is generated as a transition probability table using the alignment result accumulated as the learning result.

  FIG. 7 shows the data structure of an embodiment of a probability model 700 generated by the alignment learning of FIG. The probability model 700 shown in FIG. 7 associates an alphabetic phoneme, a kana phoneme with respect to the alphabetic phoneme, and a matching probability π, a subsequent alphabetic phoneme, and a conversion probability P thereof. Note that kana phonemes associated with subsequent alphabetic phonemes are omitted in the embodiment shown in FIG.

  As shown in FIG. 7, when the leading alphabet phoneme is “a”, kana phonemes are registered as “a”, “e”, “a”, “ya”, “a”, etc. Corresponding conversion probabilities π are registered in association with each other. In each field on the right-hand side, subsequent alphabetic phonemes are registered as “n”, “l”, “c”, “s”, “li”, etc., and each alphabet after the preceding alphabetic phoneme “a” is registered. A transition probability P at which a phoneme appears is registered.

  Thus, the phoneme data generation and transition probability generation processing executed by the preprocessor 124 shown in FIGS. 1 and 2 is completed. The information processing apparatus 100 executes the kana character string-alphabet character string conversion processing using the probability model 700 generated by the above-described processing. Hereinafter, a character string conversion process executed by the information processing apparatus will be described with reference to FIGS. 1 and 8.

Based on the probability model shown in FIG. 7, the state transition diagram 800 describes the association with alphabetic phonemes as the state transition in the hidden Markov model. As shown in the state transition diagram 800, in the hidden Markov model, the probability that the kana phoneme = “I” as the observation sequence corresponds to the alphabetic phoneme = “I” as the state transition sequence looks up the probability model 700. Is determined as π ₁ . After the association with the first phoneme, the kana phoneme = “n” process is executed, the probability model 700 is looked up for “N” following “a”, and the values of P ₁₂ and π ₂ are calculated. Are acquired and sequentially π ₃ , P ₂₃ ,. . . Is obtained by looking up the probability table 700.

In FIG. 8, P _ij is a probability of causing a transition of S _j in each state S _i , and π _i is a probability of being converted to a conversion destination phoneme x in the state S _i , and Formula (2) can be formulated.

  Thereafter, the conversion likelihood calculation unit 122 uses each acquired probability value, takes the conversion likelihood χ from the beginning to the end of the kana character string, takes the logarithm of each probability, and uses the following equation (2): And calculate.

  Further, the maximum likelihood phoneme sequence determination unit 118 shown in FIG. 1 acquires the conversion likelihood χ calculated by the conversion likelihood calculation unit 122, and converts the alphabet phoneme sequence as a state transition sequence that gives the maximum of the conversion likelihood χ. get. Then, the acquired alphabetic phoneme sequence is passed to the alphabet output unit 130 as a maximum likelihood phoneme sequence, and a series of character string conversion processes is completed.

  FIG. 9 shows an embodiment of an information processing system 900 in which the information processing apparatus 100 according to the present embodiment is implemented as a web server 910 and can handle a plurality of languages of Indo-West European languages. Note that the information processing system 900 in FIG. 9 does not include a function corresponding to the preprocessor 124 and is described as being stored in the hard disk device 990 as execution data together with a program for executing character string conversion.

  The server 910 can also be configured to include a function corresponding to the preprocessor 124. In this case, the server 910 receives an aligned case set and unaligned case set suitable for a specific user via the network, and receives each user. For example, a character string conversion service customized for each specialized field can be provided.

  Hereinafter, the information processing system 900 will be described with reference to FIG. The information processing system 900 provides a web service. The web server 910 includes a network adapter 930, a CGI (Common Gateway Interface) 940 for adapting various requests to the server program type, and a kana character acquisition unit 950. The network adapter 930 includes a network interface card (NIC), receives a request via a network 920 such as the Internet, and transmits a processing result of the web server 910 to a remotely connected web via the network 920. • Returned to client (not shown).

  The web server 910 further manages various databases 990. The database 990 includes a language type estimation morpheme dictionary 990a and a language type probability model storage unit 990b that are used to estimate the language type of Indian / Western European languages expressed by kana character strings.

  The kana character acquisition unit 950 acquires the kana character string included in the character string conversion request from the web client via the CGI 940. The kana character acquisition unit 950 looks up the language type estimation morpheme dictionary 990a, searches for a specific form corresponding to the kana character string, and estimates the language type. In another embodiment, the user can include data specifying the language type in the character string conversion request. In this case, the kana character acquisition unit 950 sends the data to the phoneme decomposition unit 960 or the like. And execute string conversion.

  When the kana character acquisition unit 950 acquires the kana character string and the language type data, the kana character acquisition unit 950 passes the kana character string to the phoneme decomposition unit 960. The phoneme decomposition unit 960 performs phoneme decomposition on the received kana character string by looking up the phoneme data storage unit, and passes the result of phoneme decomposition to the conversion likelihood calculation unit 970. The conversion likelihood calculation unit 970 calls the probability model registered corresponding to the estimated language type from the language type probability model storage unit 990b, and executes the transition probability calculation.

  Maximum likelihood character type sequence determination unit 980 is implemented to include the same function as maximum likelihood alphabet sequence determination unit 118 shown in FIG. 1, and is the maximum value of conversion likelihood χ calculated by conversion likelihood calculation unit 970. Is determined as the maximum likelihood character type sequence. Thereafter, the maximum likelihood character type sequence determination unit 980 returns the result to the web client via the network adapter 930, and returns the character type / sequence corresponding to the kana character string requested by the web client.

  In another embodiment, the maximum likelihood character type sequence determination unit 980 sends a generated character string sequence to a GNA (Grobal Name Analytics) server 995 to execute a person name search when the kana character string is a person name. You can also In addition, about GNA, the system or server described by http://publibfp.boulder.ibm.com/epubs/pdf/c1912860.pdf can be mentioned, for example.

  The information processing system 900 shown in FIG. 9 can convert a kana character string sent from a user into a corresponding Indian / Western European language by identifying its language type, and provides high language versatility. It becomes possible. In addition, in the case of an implementation that implements a preprocessor, it is possible to perform character string conversion customized for the user, and provide an efficient web service without creating a different web server or probability model for each user. be able to.

  Furthermore, if the converted Indo-West European sequence is a person name, it can function as an input interface to the person name search system 995, so it is efficient for important applications such as personal search, name identification, money laundering, etc. Search results can be returned automatically.

  FIG. 10 is a flowchart of an embodiment of character conversion processing executed by the web server 910 of this embodiment. The process shown in FIG. 10 describes a Kana-alphabet character string conversion request and an alignment request (block B) in parallel, but each request can be processed independently.

  The process of FIG. 10 starts from step S1000. In step S1001, the web server 1010 receives a request including a kana character string or both a kana character string and an alphabetic character string. When only a kana character string is received, it is a character string conversion request, and data for estimating the language type may be received simultaneously as described above. Further, the embodiment in which the kana character string and the alphabet character string are received simultaneously is an embodiment that responds to the alignment request, executes the route search shown in FIG. 5, and returns the alignment pair with the minimum total cost as a response. .

  In step S1002, if there is a kana character string or an alphabet character string, the alphabet character string is also phoneme decomposed, and in step S1003, it is determined whether both the kana character string and the alphabet character string exist. If both are present (yes), it is determined that the received request is an alignment request, and the process branches to block B. If it is determined that both the kana character string and the alphabet character string are not included (no), the likelihood probability χ is calculated using a probability model corresponding to the language type in step S1004. In step S1005, a character string having a large likelihood probability χ is obtained. In step S1006, a conversion result is created in an appropriate format for displaying the conversion result, for example, a format such as RSS, table, etc. And the process is terminated in step S1007.

  As described above, in this embodiment, the Kana phoneme and the alphabet phoneme can be directly associated with the transition probability, and the maximum likelihood string conversion can be executed using the hidden Markov model (HMM) method. It makes efficient use of hardware resources and enables more accurate search even more directly. Also, by simply generating a probability model for each language, flexible character string conversion corresponding to the language type becomes possible.

  The process of block B is a process corresponding to the alignment request. In step S1008, a route search is performed using the result of phoneme decomposition, and the minimum alignment with the total cost is determined. Thereafter, a structured document is created and output in a format for displaying the search result in step S1009, and the process ends in step S1007. The process described in block B is the same process as the automatic alignment process in the unaligned case described in FIG.

  The embodiment of the block B described above can be used when the web client wants to check alignment, or when the administrator of the web server 910 or the like wants to check the accuracy of the alignment process. Can be used for.

  A character string conversion process according to this embodiment will be described with reference to FIGS. FIG. 11 shows an embodiment of a graphical user interface (GUI) 1100 displayed on the web client 925 in the character string conversion method of this embodiment. The GUI 1100 indicates that the GUI 1100 performs Katakana / Romaji conversion. Note that the Roman alphabet is a notation system for describing Japanese in alphabets corresponding to its pronunciation, which is designated by the Hebon formula, the decree formula, and the like, and the conversion to the alphabet is substantially executed.

  In the GUI 1100, a field 1110 for inputting kana and a field 1120 for designating the upper limit number of Roman character conversion candidates to be searched are displayed. The user inputs a character string and an upper limit number in each field and then clicks an “OK” button to send a character string conversion request to the web server 910. The web server 910 performs the character conversion process described above, and returns a process result to the web client 925. In the embodiment shown in FIG. 11, kana character string = information.

  FIG. 12 illustrates an embodiment of a GUI 1200 displayed by a web client 925 that has received a processing result from the web server 910. As shown in FIG. 12, the web server 910 creates a conversion result in the RSS format so as to facilitate subsequent reference and search, creates a structured document such as HTML, XML, etc., and sends it to the web client 925. Send it. As shown in FIG. 12, it is shown that conversion target kana = information, and the alphabet conversion result is displayed on the subsequent lines together with the value of the conversion likelihood χ. As shown in FIG. 12, the alphabet string having the highest likelihood is “information”, which indicates that character conversion is being performed with sufficient accuracy.

  FIG. 13 shows a GUI 1300 that displays another embodiment of alphabet conversion for a case where Kana character string = Manchester is used. FIG. 13 shows that the correct result is given with the likelihood χ = 0.22 for the input kana character string = Manchester. In addition, it is shown that the likelihood rank between the “mancester” having the second highest likelihood is sufficiently secured.

  FIG. 14 shows a GUI 1400 according to still another embodiment provided by the web server 910. The embodiment of FIG. 14 is an embodiment of the alignment process of the present embodiment. The embodiment shown in FIG. 14 executes when the web server 910 receives an alignment request that includes both a Kana character string and an alphabetic character string. When the user inputs a kana character string and an alphabetic character string into the GUI 1400 and then clicks an “OK” button, an alignment request including “information” and “information” is sent. When the web server 910 acquires each character string, it performs phoneme decomposition and performs the route search shown in FIG.

  As a result of the route search, the web server 910 finds a kana-phoneme-alphabet phoneme association with the minimum total cost, describes the association in the RSS format as an alignment result, and creates a structured document.

  FIG. 15 is a GUI 1500 that displays the alignment result sent from the web server 910 to the web client 925. As shown in FIG. 15, the alignment result with the minimum total cost is displayed in the RSS type, indicating that good alignment accuracy is obtained. Further, FIG. 16 shows an alignment result generated when “Manchester” and “manchester” are input in the same alignment request. As shown in FIG. 16, Kana phoneme-alphabet phoneme is well associated, and high-precision alphabet character string conversion or It was shown that alphabet string search is possible.

  The functions of this embodiment are described in an object-oriented programming language such as C ++, Java (registered trademark), Java (registered trademark) Beans, Java (registered trademark) Applet, Java (registered trademark) Script, Perl, and Ruby. The program can be realized by a program executable by the apparatus, and the program can be stored in a device-readable recording medium such as a hard disk device, CD-ROM, MO, flexible disk, EEPROM, EPROM, and distributed. It can be transmitted over the network in a possible format.

  Although the present embodiment has been described so far, the present invention is not limited to the above-described embodiment, and other embodiments, additions, changes, deletions, and the like can be conceived by those skilled in the art. It can be changed, and any aspect is within the scope of the present invention as long as the effects and effects of the present invention are exhibited.

The functional block diagram of the information processing apparatus of this embodiment. FIG. 2 is a functional block diagram of the preprocessor shown in FIG. 1. The flowchart of a probability model production | generation process. The figure which showed embodiment of the data structure of the aligned example set. The figure which showed embodiment of a route map. The conceptual diagram explaining embodiment of the alignment learning using an unaligned example set. The figure which showed the data structure of embodiment of the probability model produced | generated by the alignment learning of FIG. State transition diagram describes correspondence to alphabetic phonemes as state transition in hidden Markov model. The functional block diagram of the information processing system of this embodiment. The flowchart of embodiment of the character conversion process which the web server of this embodiment performs. The figure which showed embodiment of GUI displayed on a web client by the character string conversion method of this embodiment. The figure which showed embodiment of GUI which the web client which received the process result about the kana character string = information by a web server displays. The figure which showed GUI which displays other embodiment of alphabet conversion about the case where Kana character string = Manchester is used. The figure which showed GUI of further another embodiment which a web server provides. The figure which showed GUI which displays the alignment result which the web server sent to the web client. The figure which showed GUI which displays the alignment result which the web server sent to the web client.

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 ... Information processing apparatus, 112 ... Input part, 114 ... Input / output interface, 116 ... Kana character acquisition part, 118 ... Maximum likelihood phoneme sequence determination part, 120 ... Phoneme decomposition | disassembly part, 122 ... Conversion likelihood calculation part, 124 ... Preprocessor , 126 ... phoneme data storage unit, 128 ... probability model storage unit, 200 ... preprocessor (functional block), 210 ... aligned case, 212 ... unaligned case, cost model generation unit, 216 ... alignment processing unit, 220 ... alignment result Storage unit, 222 probability model generation unit, 400 ... aligned case set, 500 ... path map, 600 ... unaligned case set, 650 ... cost model, 700 ... probability model, 800 ... state transition diagram, 900 ... information processing system

Claims (14)

An information processing device for converting an original character string into a character string of another language,
An original character acquisition unit for acquiring the original character string;
A phoneme decomposition unit that decomposes the acquired original character string into original character phonemes with reference to the phoneme data of the original character string;
For the original character phoneme sequence generated by the phoneme decomposition unit, refer to the probability model generated by learning corresponding to the original character phoneme, and use the transition probability for successive phoneme sequences to determine the conversion likelihood. A conversion likelihood calculation unit to calculate,
A maximum likelihood phoneme sequence determination unit that determines and outputs a maximum likelihood phoneme sequence of the other language with reference to the conversion likelihood calculated by the conversion likelihood calculation unit;
Including
The probabilistic model generates a cost model by associating the original character phoneme with the conversion destination phoneme using an aligned case in which the original character string and the phonemes of the other languages are aligned, and generating the cost model. A transition probability table for registering transition probabilities when phoneme sequences are generated by learning the alignment of the original character string before alignment according to a model.
Information processing device.   The conversion likelihood calculation unit is a hidden Markov model that uses a conversion probability of a conversion destination phoneme that appears between the beginning and the end of the original character string and a transition probability between successive conversion destination phonemes. The information processing apparatus according to claim 1, wherein the conversion likelihood to a conversion destination phoneme sequence is calculated. The information processing apparatus according to claim 2 , wherein the alignment is learned by a route search along a cell having each phoneme as a unit. In the route search, the Viterbi algorithm or the original character string heads from the beginning to the end, the costs allocated for the unit route along the cell are integrated along the route, and the route having the smallest total cost is searched. The information processing apparatus according to claim 3 . Including a preprocessor for generating the probability model, wherein the preprocessor associates the original character phoneme with the destination phoneme using an aligned case in which the original character string and the phonemes of the other languages are aligned. A cost model generating unit for generating a cost model, an alignment processing unit for learning the alignment of the original character string before alignment according to the cost model and generating the probability model, and the alignment processing The information processing apparatus according to claim 1, further comprising: a probability model generation unit that calculates the transition probability using an output of a unit and registers the transition probability as a transition probability table. A character string conversion method executed by an information processing device for converting an original character string into a character string of another language,
Reading the original character string from a phoneme data storage unit for registering phoneme data of the original character, reading out a probability model corresponding to the original character phoneme and generated by learning from the probability model storage unit;
Obtaining the original character string;
Decomposing the acquired original character string into the original character phonemes with reference to the phoneme data;
The original character phoneme sequence generated by the decomposing step is processed from the beginning to the end of the original character string with reference to the probability model, and the conversion likelihood is calculated using the transition probability for the continuous phoneme sequence. Calculating the degree,
Determining and outputting a maximum likelihood phoneme sequence with reference to the conversion likelihood calculated by the calculating step;
Run
The probability model generates a cost model by associating the original character phoneme with the conversion destination phoneme using an aligned case in which the original character string and each phoneme of the other language are aligned, and generating the cost model A character string conversion method , which is a transition probability table for registering transition probabilities when phoneme sequences are generated, which is generated by learning alignment of the original character string before alignment according to a model . The calculating step includes the conversion destination according to a hidden Markov model using a conversion probability of a conversion destination phoneme that appears between the beginning and the end of the original character string and a transition probability between successive conversion destination phonemes. The character string conversion method according to claim 6 , further comprising: calculating the conversion likelihood to a phoneme sequence. The cost model is generated by associating the original character phoneme with the conversion destination phoneme using an aligned case where the original character string and each phoneme of the other language are aligned, and generating the cost model The character string conversion method according to claim 7 , further comprising the step of learning and registering the alignment of the original character string before alignment according to the model, and generating in advance as a transition probability table. The pre-generating step includes a Viterbi algorithm or a cost assigned to a unit route along a cell in units of each phoneme from the beginning to the end of the original character string. The character string conversion method according to claim 8 , further comprising a step of searching for a route so as to give the route having a minimum total cost by integrating using a number. A program for an information processing apparatus to execute a character string conversion method for converting an original character string into a character string of another language, the information processing apparatus using the program,
A cost model is generated by associating the original character phoneme with the conversion destination phoneme using the aligned example in which the original character string and the phonemes of the other languages are aligned, and the pre-alignment according to the cost model Pre-generating a probability model as a transition probability table for registering transition probabilities when phoneme sequences are continuous, by learning and registering alignment of original character strings;
Reading the original character string from a phoneme data storage unit for registering phoneme data of the original character, reading out a probability model corresponding to the original character phoneme and generated by learning from the probability model storage unit;
Obtaining an original string; and
Decomposing the acquired original character string into the original character phonemes with reference to the phoneme data;
The original character phoneme sequence generated by the decomposing step is processed from the beginning to the end of the original character string with reference to the probability model, and the conversion likelihood is calculated using the transition probability for the continuous phoneme sequence. Calculating the degree,
Performing a step of determining and outputting a maximum likelihood phoneme sequence with reference to the conversion likelihood calculated by the calculating step;
A computer executable program. The calculating step includes the conversion destination according to a hidden Markov model using a conversion probability of a conversion destination phoneme that appears between the beginning and the end of the original character string and a transition probability between successive conversion destination phonemes. The program according to claim 10 , comprising calculating the likelihood of conversion to a sequence of phonemes. The pre-generating step includes a Viterbi algorithm or a cost assigned to a unit route along a cell in units of each phoneme from the beginning to the end of the original character string. The program according to claim 11 , comprising a step of searching for a route so as to accumulate the number using a number and to give the route having a minimum total cost. An information processing system for performing character string conversion via a network, wherein the information processing system includes:
A network adapter for receiving a request including the original string over the network;
An original character string acquisition unit for acquiring the original character string from the request;
A phoneme decomposition unit that decomposes the original character string into original character phonemes;
For the original character phoneme sequence generated by the phoneme decomposition unit, refer to the probability model generated by learning corresponding to the original character phoneme, and use the transition probability for successive phoneme sequences to determine the conversion likelihood. A conversion likelihood calculation unit to calculate,
A maximum likelihood phoneme sequence is determined with reference to the conversion likelihood calculated by the conversion likelihood calculation unit, and is output via the network,
The probability model generates a cost model by associating the original character phoneme with the conversion destination phoneme using an aligned example in which the original character string and each phoneme of another language to be character-string-converted are aligned. A transition probability table for registering transition probabilities in the case of continuous phoneme sequences generated by learning the alignment of the original character string before alignment according to the cost model;
Information processing system. The information processing system further includes a language type estimation morpheme dictionary for estimating a language in which the original character string is to be converted in response to the request, and a language type probability for registering the probability model for each estimated language The information processing system according to claim 13 , further comprising: a model storage unit, wherein the maximum likelihood phoneme sequence determination unit outputs the character string of the language estimated from the request as the original character string.

Images