JP2011008784A - System and method for automatically recommending japanese word by using roman alphabet conversion - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a system and method for automatically recommending Japanese words by using Roman alphabet conversion.SOLUTION: The system for automatically recommending Japanese words includes: a Roman alphabet conversion part for converting pronunciation of a word expressed in a Japanese Hiragana or Katakana mode to Roman alphabets; and a similar word search part for searching for words similar to the word on the basis of the converted Roman alphabets.

Description

    The present invention relates to a system and method for recommending similar words for input Japanese, and more particularly, to a system and method for recommending similar words by converting input Japanese pronunciation into Roman characters.

  The user performs a search by entering a word in a search window of a search engine in order to obtain desired information. At this time, when a user mistakenly inputs a word and a typographical error occurs, there is a problem that the quality of a document (search result) searched by the typographical character is deteriorated or the number of documents to be searched is scarce . In order to solve such a problem, a conventional search engine has a function of determining such a word as a typo and recommending a word (search word) corresponding to the typo actually entered by the user.

  On the other hand, even if the user performs a search by inputting a word, there are only a few cases in which the user has input an optimal word for obtaining a search result desired by the user. For this reason, even if the search engine provides a search result for the word input by the user, the user is dissatisfied with the search result. Therefore, in order to solve such a problem, the conventional search engine aims to improve the accuracy of the search by providing related words or similar words for the word input by the user.

  However, each of the above-described situations has the following problems especially when searching using Japanese. In other words, if the Japanese search term input by the user is judged as a typo and the correct word is presented, or if a similar word for the Japanese input by the user is provided, the accuracy is conventionally guaranteed. could not. Above all, Japanese is expressed in the form of kanji, hiragana, and katakana, and since one word contains these three forms, it is recommended to recommend an appropriate word for the word entered by the user There was a problem that was difficult. Accordingly, there is a strong demand for a method for recommending an appropriate word even when Japanese characters in kanji, hiragana, and katakana are input.

  The present invention relates to a system and method for improving the accuracy of similar word search for Japanese by converting the pronunciation of an input Japanese word into Roman letters and searching for similar words for the word based on the converted Roman letters. The purpose is to provide.

  Another object of the present invention is to determine whether an input Japanese word is a typographical error, and to search for a similar word and provide a correct word when the typographical error is a typographical character, so that the user can search for It is an object of the present invention to provide a system and method for improving the accuracy of a search by recommending an appropriate correct word even if a word) is erroneously input.

  Another object of the present invention is to divide the token into tokens via learning data generated by machine learning and convert the divided tokens into hiragana when the input Japanese word is kanji. An object of the present invention is to provide a system and method capable of performing quick and accurate kanji-hiragana conversion.

  Another object of the present invention is to provide a system that allows a user to perform a more accurate search by searching for and recommending similar words in a form different from the form of Japanese words input by the user. It is to provide a method.

  An automatic Japanese recommendation system according to an embodiment of the present invention includes a romaji conversion unit that converts pronunciation of a word expressed in Japanese hiragana or katakana form into romaji, and the word based on the converted romaji A similar word search unit that searches for similar words for.

  The automatic Japanese recommendation system according to an embodiment of the present invention recommends similar word recommendation by converting the searched similar word into the Japanese form of any one of the hiragana, katakana, or kanji. May further include a portion.

  The automatic Japanese recommendation system according to an embodiment of the present invention may further include a typographical error determination unit that analyzes an input word and determines whether the word is a typo.

  In the Japanese automatic recommendation system according to an embodiment of the present invention, when the input word is a typographical error, based on the similarity score or the edit distance according to the input frequency of the word, A correct word selection unit for selecting a correct word for the word may be further included.

  In the Japanese automatic recommendation system according to an embodiment of the present invention, when an input word is kanji, the word is divided into tokens using token division learning data, and the kanji-Hiragana conversion learning data is used to divide the word. It may further include a Kanji-Hiragana conversion unit for converting into hiragana corresponding to the divided tokens.

  An automatic Japanese recommendation method according to an embodiment of the present invention includes a step of converting a pronunciation of a word expressed in Japanese hiragana form or katakana form into Roman letters, and similarity to the word based on the converted Roman letters Searching for words.

  According to an embodiment of the present invention, the pronunciation of an input Japanese word is converted to Roman characters, and the similar word search accuracy for Japanese is searched by searching similar words for the word based on the converted Roman characters. Can be improved.

  According to an embodiment of the present invention, it is determined whether an input Japanese word is a typographical error. Even if entered, the correct correct word can be recommended to improve the accuracy of the search.

  According to an embodiment of the present invention, when an input Japanese word is a kanji, it is divided into tokens by learning data generated through machine learning, and converted into hiragana for the divided tokens. Quick and accurate Kanji-Hiragana conversion can be performed.

  According to an embodiment of the present invention, it is possible to perform a more accurate search for a user by searching for and recommending similar words in a form different from the form of a Japanese word input by the user. it can.

It is a block diagram which shows the whole structure of the Japanese automatic recommendation system which concerns on one Embodiment of this invention. FIG. 6 is a diagram illustrating a process of automatically recommending Japanese via a romaji conversion for a word input according to an embodiment of the present invention. FIG. 5 is a diagram illustrating a process of converting kanji into hiragana according to an embodiment of the present invention. It is a figure which shows an example converted into a Roman character by one Embodiment of this invention. 3 is a flowchart illustrating an overall process of an automatic Japanese recommendation method according to an embodiment of the present invention.

  Hereinafter, embodiments according to the present invention will be described in detail with reference to the contents described in the accompanying drawings. However, this invention is not restrict | limited or limited by embodiment described below. Moreover, the same referential mark shown in each drawing shows the same member.

  FIG. 1 is a block diagram showing the overall configuration of a Japanese automatic recommendation system according to an embodiment of the present invention.

  The automatic Japanese recommendation system 100 of the present embodiment is configured by a computer, and each unit 101 to 106 described later is realized by a control unit (CPU) included in the computer reading a predetermined program. In addition, as one aspect of the automatic Japanese recommendation system 100 of the present embodiment, a part of a web server that provides a search service or a search server (search engine) that performs a search process using a search word (question word). Or a computer device connected to the web server or the search server in a different configuration.

  In this case, the automatic Japanese recommendation system 100 according to the present embodiment automatically applies one or more similar words, which will be described later, to a search word (question word) of a Japanese word input from a user terminal to a predetermined search window. The extracted similar words (including the correct words extracted from the similar words when the search word is determined to be a typo) are exposed to the search result page for the search word. The automatic Japanese recommendation system 100 can perform a similar word extraction process using Japanese words input to the search window, separately from the process of generating a predetermined search result for the search word performed by the search server. When incorporated as a part of a web server or a search server, the web server or the search server uses a search word input to the search window as a trigger for a search word (question word) of the input Japanese word. One or a plurality of similar words are automatically extracted, a search result page including the extracted similar words is generated and transmitted to the user terminal of the user who requested the search. When the web server or the search server is connected in another configuration, the search word input from the web server or the search server to the search window is received through the network, and the search word (question word) of the received Japanese word is received. ) Is automatically extracted, and the extracted similar words are transmitted to the web server or the search server.

  As shown in FIG. 1, the automatic Japanese recommendation system 100 according to the present embodiment includes an erroneous character determination unit 101, a kanji-hiragana conversion unit 102, a Roman character conversion unit 103, a similar word search unit 104, a similar word recommendation unit 105, and a correct answer A word selection unit 106 is included.

  In Japanese search, the user inputs Japanese for desired information search. At this time, the user can input a Japanese word A107 in each form of kanji, hiragana or katakana. The automatic Japanese recommendation system 100 recommends a more accurate Japanese word B108 by converting the pronunciation of each form of the word 107 input by the user into Roman letters.

  As one embodiment of the present invention, it is determined whether or not a Japanese word input by a user from a predetermined screen by the typographical error determination unit 101 is a typographical error. In the system 100, similar words are extracted by the Kanji-Hiragana conversion unit 102, the Romaji conversion unit 103, the similar word search unit 104, and the similar word recommendation unit 105, and the correct word selection unit 106 selects the correct word (correct word) for the typo. A plurality of similar words extracted for the typo are selected and provided. As another embodiment of the present invention, when it is determined that the Japanese word input by the typographical error determination unit 101 is not a typographical error, or regardless of the determination process of the typographical error determination unit 101, the user is not a typographical error. When an accurate word is input, the automatic Japanese recommendation system 100 may provide a similar word via the kanji-hiragana conversion unit 102, the romaji conversion unit 103, the similar word search unit 104, and the similar word recommendation unit 105. it can. In the following description, the case where the user inputs a typographical character will be mainly described.

  The typographical error determination unit 101 analyzes the word 107 input from the user through the user terminal and determines whether the word 107 is a typo. In this case, the Roman character conversion unit 103 converts the word 107 into a Roman character when it is determined that the word 107 input by the user is an erroneous character.

  As an example, the typo determination unit 101 can determine whether or not the word 107 is a typo based on whether or not the word 107 input by the user is included in preset typo data. Specifically, the typographical error determination unit 101 is input by a user using typographical data that is determined by a word listed in advance, a content DB catalog constructed by a search engine, a manual review, or the like and stored in a predetermined storage area. If the word 107 is included in the typo data, it is determined as a typo.

  As another example, the typographical error determination unit 101 determines that the word 107 is a typo based on whether the input frequency of the word 107 input by the user or the frequency of appearance of the document is lower than a preset reference frequency. It may be determined whether or not.

  At this time, the input frequency of the word 107 means the number of times the word 107 is input by the user. That is, the typo determination unit 101 can determine the word 107 with a low input frequency as a typo. The frequency of document appearance means the number (number of times) of documents extracted as search results when a document is searched using the input word 107, in other words, the number of documents including the word 107 in the document. To do. When the number of documents including the word 107 in the document is smaller than a predetermined reference number, the typographical error determination unit 101 determines that the word appearance frequency is low and determines the word 107 as a typo. In this case, the automatic Japanese recommendation system 100 has a function of counting the number of times of input for each word input in response to the user's word input, and a function of acquiring the number of documents including the word input by the user in the document. In addition, information such as the number of times of input for each word, the number of documents, and a reference frequency (reference number of inputs, number of reference documents) preset for each of these is stored in a predetermined storage area.

  In addition, the typographical error determination unit 101 typographically corrects the corresponding word 107 when the frequency of appearance of the document with respect to the word 107 is lower than the query frequency (the number of times that the query by the word 107 is input, for example, the input frequency of the word 107). It can also be judged as. Further, the typographical error determination unit 101 may be configured to determine, as a typographical error, a continuous word 107 (a continuous word 107 having a low document appearance frequency and a word-to-word connection) although the frequency of document appearance is low. Good.

  As another example, the typo determination unit 101 can determine whether or not the word 107 is a typo based on whether or not the word 107 input by the user is separated into morphemes. At this time, if the input word is separated into morphemes by a morphological analyzer or a part of speech tagger, the typographical error determination unit 101 can determine that the corresponding word 107 is not a typographical error. In other words, if the word is a typographical error, it cannot be separated into morphemes (for example, the smallest unit of a language that has a meaning extracted by dividing it until it no longer makes a language meaning if it is further decomposed). 101, when words are separated into morphemes, the word 107 input by the user can be determined as a normal character (a word that is not a typo).

  When the input word 107 is a Chinese character, the kanji-hiragana conversion unit 102 divides the word into tokens using token division learning data. In addition, the kanji-hiragana conversion unit 102 converts words or characters corresponding to the tokens divided using the kanji-hiragana conversion learning data into hiragana. Note that even if the same kanji is used in Japanese, the reading differs depending on how it is used, so it is important to convert it into an accurate hiragana corresponding to the kanji. However, the detailed processing of the kanji-hiragana conversion unit 102 is illustrated in FIG. The details will be described later with reference to FIG.

  The Romaji conversion unit 103 converts the word 107 expressed in Japanese hiragana form or katakana form into romaji based on the pronunciation. When the word 107 is a kanji, after the word 107 is converted into hiragana by the kanji-hiragana conversion unit 102, the word 107 converted into hiragana is converted into roman characters based on the romaji corresponding to the pronunciation of each hiragana character. The conversion part 103 converts into a Roman character. For example, when the input word is a “movie” in kanji, it is converted into “eiga” by the kanji-hiragana conversion unit 102, and the romaji conversion unit 103 is based on the pronunciation of the word converted into hiragana (eiga). ). An example in which the Romaji conversion unit 103 converts to Romaji will be specifically described later with reference to FIG.

  The similar word search unit 104 searches (extracts) a similar word for the word 107 from a predetermined similar word group based on the Roman character converted by the Roman character conversion unit 103. As an example, the similar word search unit 104 can extract a similar word for the word based on the similarity (similarity score) of the word converted into Romaji. In the hiragana / katakana or kanji character form, measuring the similarity between an input word and a word extracted as a similar word has a very low edit distance resolution and the accuracy is reduced. According to the above, both words are converted into Romaji based on pronunciation and the similarity is measured. For example, rather than comparing “oligon” and “Oricon” directly, the degree of similarity can be compared more accurately by converting this into Romaji and comparing “origon” and “orikon”. .

  At this time, the similarity score is at least one of an input frequency corresponding to the length of the word, an edit distance depending on whether the word includes a long sound, a middle point, a prompt sound or a muddy sound, or a comparison degree of the original state of the word. Is determined (calculated) based on As an example, when the word is a kanji, the similar word search unit 104 compares the kanji converted into romaji (similarity between words after being converted into romaji), and the kanji is converted into hiragana. The similarity score can be determined based on the comparison result of the form (similarity between words after being converted into hiragana) and the comparison result of the kanji form (similarity between words in the kanji form). The similar word search will be specifically described later with reference to FIG.

  The similar word recommendation unit 105 converts the searched similar word into a Japanese word 108 of hiragana, katakana, or kanji and recommends it. The user can search by inputting the recommended word 108. In this case, the similar word recommendation unit 105 performs a process of exposing the word 108 that is a similar word to the input word 107 extracted on a predetermined page or screen. Further, the word 108 that is a similar word is exposed on the search result page where the search result of the search server is exposed, or the word 108 is transmitted to a web server or a search server that generates the search result page.

  As an example, the similar word recommending unit 105 may convert the recommended similar word into a word 108 having a form different from the Japanese form of the word 107 input by the user and recommend it. For example, when the user inputs a hiragana word 107, the similar word recommendation unit 105 may convert the similar word for the input word 107 into a kanji word 108 and recommend it to the user.

  When the word 107 input from the user is a typo, the correct word selection unit 106 corrects the correct word 108 for the word 107 among the searched similar words based on the similarity score or the edit distance based on the word input frequency. Select. That is, when a plurality of similar words are searched for the input word 107 that is a typographical error, the correct word selection unit 106 selects the similar word having the highest similarity score or higher than a predetermined reference value, or the word A similar word whose input frequency is higher than a predetermined reference value can be selected and provided as the correct word 108. The edit distance is a direct basis (reference) for determining the similarity between words, and the similarity is higher when the edit distance is lower. That is, the edit distance based on the word input frequency means, for example, that the higher the input frequency is, the lower the word edit distance is given. In other words, the edit distance according to the input frequency, in other words, the word input Similar words can be selected as correct words based on the frequency.

  FIG. 2 is a diagram illustrating a process of automatically recommending Japanese by Romaji conversion for an input word according to an embodiment of the present invention.

  When a Japanese word is input from the user via the user terminal, the typographical error determination unit 101 determines whether the input word is a typographical error. As described above, the typographical error determination unit 101 determines whether the word is included in the preset typographical data, whether the word input frequency or the document appearance frequency is lower than a preset reference frequency, or Based on whether the word is separated into morphemes, it is determined whether the word is a typo.

  When it is determined that the word input from the user is a typo, the correct word selection unit 106 selects a similar word satisfying a predetermined criterion from the similar words searched for the input word as the correct word. And provide. When it is determined that the input word is not a typo, that is, when it is determined that the input word is a correct character, the correct word selection unit 106 does not operate.

  As shown in FIG. 2, the input Japanese word is in any one of the hiragana form, the katakana form, or the kanji form, and if the inputted word is in the hiragana form or the katakana form, the romaji conversion is performed. The unit 103 converts to a romaji based on pronunciation of a word expressed in Japanese hiragana form or katakana form.

  On the other hand, when the input word is in the kanji form, it is difficult to directly convert the kanji into the romaji. Therefore, the kanji-hiragana conversion unit 102 can normalize the hiragana form. Specifically, the kanji-hiragana conversion unit 102 divides kanji into tokens using token division learning data, and converts words or characters corresponding to the divided tokens into hiragana using kanji-hiragana conversion learning data. be able to. Then, the Roman character conversion unit 103 converts the hiragana converted by the kanji-hiragana conversion unit 102 into a Roman character corresponding to the pronunciation.

  The similar word search unit 104 searches for a similar word for a word from a predetermined group of similar words based on the converted romaji. Specifically, the similar word search unit 104 searches for a similar word for a word based on the similarity score of the word converted into Roman characters.

  As an example, the similarity score is an input frequency according to the length of the word, an edit distance based on whether the word includes a long sound, a middle point, a prompt sound, or a cloudy sound, or a comparison degree of the original state of the word. Determined based on at least one of the following.

Word length, information-information [edit distance, similarity]
Long sound: Hello Work (wrong), Hello Work (wrong), Hello Work (correct)
Midpoint: Pete Rose (typo), Pete Rose (correct)
Semi-turbid sound: Oligon (typo), Oricon (correct answer)
Encouragement sound: BicCamera (typo) Biccamera (correct answer)
Prototype: Dango Final from Flower (Typographical) Boys Final from Flower (Correct)

  Since the word input frequency (input count) increases as the word length is shorter, the similarity search unit 104 can increase the similarity score as the word length is shorter (higher similarity score is given). can do). In other words, the similarity score based on the input frequency according to the word length is based on the relationship between the word length and the input frequency of the word. It is a similarity score given according to the length of the originating word.

  Since the Japanese long sound (-) is easily inserted or deleted compared to other characters, the similar word search unit 104 weights the editing distance to a small degree when the word contains a long sound. The score can be increased. Specifically, when a long sound is included in a word, the editing distance for the long sound becomes large, but since a long sound is easily inserted or deleted compared to other characters, By applying (multiplying) a small weight value (for example, a number greater than or equal to 0 and less than 1), the edit distance of words including long sounds can be adjusted to be small, and the similarity score can be increased. Similarly, since the Japanese midpoint (•) is easily inserted or deleted compared to other characters, the similar word search unit 104 sets the edit distance when the midpoint is included in the word. The similarity score may be increased by applying a small weight. Furthermore, since the Japanese prompt sound (tsu) is often omitted or mistakenly used as a similar pronunciation, the similar word search unit 104 may edit the edit distance when the input word includes the prompt sound. It is also possible to increase the similarity score by weighting.

  Moreover, not only the form converted into the Roman character but the similar word search part 104 can reflect the comparison result of the original state of a word on a similarity score. By comparing the prototype states, it is possible to compensate for errors resulting from searching for similar words in a state normalized to Roman characters. For example, if the input word is “Uton”, the similar word search unit 104 assigns a higher similarity score of “Udon” whose original state is similar than “Uron”, thereby converting the similarity by Romaji conversion. It is possible to supplement the error when judging.

  Further, as an example, when the word is a Chinese character, the similar word search unit 104 is based on the comparison result of the form converted into Romaji, the comparison result of the form converted into Hiragana, and the comparison result of the original form of the Chinese character. A similarity score can also be determined. Specifically, when the word is a Chinese character, the similar word search unit 104 can determine the similarity score according to the following mathematical formula 1.

  Here, q means Japanese (question word) input by the user, and t means a similar word. Moreover, a, b, and c mean constants. At this time, a, b, and c can be derived by a machine learning function or the like.

  When similar words are searched (extracted) through such a process, as shown in FIG. 2, the similar word recommendation unit 105 selects one of the hiragana, katakana, and kanji characters as the searched similar word. Recommend by converting to word form. For example, when the input word is in the hiragana form, the similar word recommendation unit 105 converts the searched similar word into the Japanese form of any one of the hiragana form, the katakana form, or the kanji state, and recommends it. can do. That is, the similar word recommendation unit 105 can convert and recommend the searched similar words into a form different from the Japanese form of the input word.

  Also, as an example, the similar word recommendation unit 105 determines that the difference between the degree of similarity converted to Roman letters and the degree of similarity not converted to Roman letters exceeds a preset criterion, Even if the degree of similarity is high, it is possible not to recommend the corresponding similar word. As another example, the similar word recommendation unit 105 may not recommend a similar word when the input word is used more than the recommended similar word (for the similar word recommended to the user). If the question frequency is compared with the question frequency (input frequency) of the word 107 input by the user, and if the question frequency of similar words recommended to the user is lower than the question frequency of the word 107 input by the user, in other words For example, if the frequency of the word 107 input by the user is higher than the recommended similar word, the similar word that is used less frequently is not recommended).

  When the input word is a typo, the correct word selection unit 106 selects the correct word for the word among the searched similar words based on the similarity score or the edit distance based on the word input frequency. Also good. Specifically, the correct word selection unit 106 can select a correct word with respect to a word having the highest similarity score or a similar word having a high word input frequency and a short editing distance.

  FIG. 3 is a diagram illustrating a process of converting kanji into hiragana according to an embodiment of the present invention.

  The Kanji-Hiragana conversion unit 102 according to the embodiment of the present invention converts the input Kanji into Hiragana. The romaji conversion unit 103 may convert the hiragana converted by the kanji-hiragana conversion unit 102 and the input hiragana and katakana into romaji.

  As an example, the Kanji-Hiragana conversion unit 102 performs token division processing 305 using the token division learning data 302 and divides the input Kanji 304 by token. Then, the kanji-hiragana conversion learning data 303 is used to perform a kanji-hiragana conversion process 306, and the token 305 divided by the token dividing process is converted into a corresponding hiragana 307.

  For example, if the input word is “Me and her way of life”, token division learning data 302 is used to perform token division processing such as “I and her, of life, way of life”. The hiragana state sequence having the maximum probability value is selected from each token bigram. Specifically, it translates to "I-I and her-Kanojo's life-Iki-no-Michi-Michi", and finally to Hiragana form of "I, Kano-jo's Iki-ichi Michi".

  At this time, the learning data creates a hiragana learning document corresponding to the kanji 304 in the Japanese document 301 such as a document posted on a Japanese news or Japanese blog, and based on the learning document, a predetermined machine learning algorithm is used. It can be determined by selecting and combining hiragana according to the input form.

  As an example, the token division learning data 302 can be determined based on a hidden Markov model (HMM) -based descriptive learning algorithm using a corpus that is separated by kanji morpheme tokens. At this time, the token division learning data 302 can be determined based on a syllabic trigram HMM-based segmentation learning algorithm.

  Further, as an example, the kanji-hiragana conversion learning data 303 includes a unigram dictionary 303-1 and a bigram dictionary 303-2 determined based on a corpus-based learning algorithm that separates kanji 304 by morpheme tokens. Can be included. In this case, the unigram dictionary 303-1 can be constructed with the frequency number (token-hiragana) between the token and the hiragana. The bigram dictionary 303-2 can be constructed with the number of frequencies between tokens (token 1-token 2). That is, the Kanji-Hiragana conversion unit 102 converts the Kanji 304 into the Hiragana 307 using the token division learning data 302 and the Kanji-Hiragana conversion learning data 303 determined from the Japanese document 301 based on a predetermined learning process. Can do.

  As another example, the kanji-hiragana conversion unit 102 searches the bigram dictionary 303-2 for every two tokens for tokens divided from the kanji 304 based on the token division learning data 301, and the maximum Tokens with probabilities can be selected. Further, the Kanji-Hiragana conversion unit 102 converts the finally selected token into the Hiragana 307 corresponding to the unigram dictionary 303-1. When the information amount of the bigram dictionary 303-2 is insufficient, the kanji-hiragana conversion unit 102 can select a token having the maximum probability using the unigram dictionary 303-1.

  FIG. 4 is a diagram illustrating an example of converting hiragana or katakana into romaji according to an embodiment of the present invention.

  As shown in the figure, an example is shown in which “a” and “ka” lines are converted to Roman characters. The Romaji conversion unit 103 converts the pronunciation of a word expressed in Japanese hiragana form or katakana form into romaji. At this time, if the input word is kanji, the kanji-hiragana conversion unit 102 converts the kanji into hiragana.

  As shown in the figure, for the “a” line, the Roman alphabet conversion unit 103 converts the hiragana “a” into the Roman alphabet “a”. In addition, the romaji conversion unit 103 converts hiragana “i” into romaji “i”. Similarly, the Romaji conversion unit 103 converts each hiragana “u” into “u”, “e” into “e”, and “o” into “o”. Through such a conversion process, the automatic Japanese recommendation system 100 can search for similar words of the input word more precisely by converting the hiragana or katakana into romaji and using the converted romaji.

  Also, as described above, when searching for similar words using hiragana and katakana as they are, the resolution of the edit distance is low, so in the case of a machine such as a non-human server, “oligon” is distinguished from “oricon”. Difficult to do. In this case, by comparing “Oligon” and “Oricon” with the Roman letters “origon” and “orikon”, it is possible to calculate a more precise similarity score and improve the accuracy of similar word recommendation.

  FIG. 5 is a flowchart showing overall process transition of the automatic Japanese recommendation method according to the embodiment of the present invention.

  Referring to the figure, the automatic Japanese recommendation system 100 determines whether a word input from the user via a predetermined page or screen displayed on the user terminal is a typo (S501). At this time, if the input word is a typo, the Japanese automatic recommendation system 100 selects and provides the correct word from the similar words to the word (S507).

  The automatic Japanese recommendation system 100 can automatically recommend similar words for the input word even when the input word is a correct character instead of a typo. The automatic Japanese recommendation system 100 determines whether the input word is a Chinese character (S502). If it is determined that the word is kanji, the Japanese automatic recommendation system 100 converts the kanji into hiragana (S503), and then performs step S504. If the input word is not kanji, the conversion process in step S504 is not performed.

  Specifically, the automatic Japanese recommendation system 100 divides a word into tokens using token division learning data when it is determined that the word is kanji or when the input word is determined to contain kanji. Further, the kanji-hiragana conversion learning data is used to convert the hiragana corresponding to the divided tokens.

  At this time, the token division learning data can be determined based on a hidden Markov model-based division learning algorithm using a corpus that is separated by kanji morpheme tokens. In addition, the kanji-hiragana conversion learning data may include a bigram dictionary and a unigram dictionary determined by a learning algorithm based on a corpus that is separated for each kanji morpheme token. Here, the bigram dictionary is constructed with the frequency number between tokens, and the unigram dictionary is constructed with the frequency number between tokens and hiragana.

  In this case, the automatic Japanese recommendation system 100 searches the bigram dictionary for the divided tokens, selects a token that shows the maximum probability, and converts the selected token into a hiragana corresponding to the unigram dictionary. .

  The automatic Japanese recommendation system 100 converts the pronunciation of a word expressed in Japanese hiragana form or katakana form into Roman letters (S504). The automatic Japanese recommendation system 100 searches for similar words with respect to the input word based on the converted romaji (S505).

  As an example, the automatic Japanese recommendation system 100 can search for similar words for the input word based on the similarity score of the word converted into Roman characters. At this time, the similarity score is at least one of an input frequency according to the length of the word, an edit distance depending on whether the word includes a long sound, a midpoint, a prompt sound, or a muddy sound, or a comparison of the original state of the word. It can be determined based on one or a combination thereof.

  In addition, the automatic Japanese recommendation system 100 may convert the searched similar word into any one Japanese form of hiragana, katakana, or kanji and recommend it to the user (S506). At this time, the similar word recommendation unit 105 can convert the recommended similar word into a form different from the Japanese form of the input word and recommend it.

  As another example, if the difference between the similarity in the state converted to Romaji and the similarity in the state not converted to Romaji exceeds a preset criterion, the automatic Japanese recommendation system 100 does not recommend a similar word. It can also be configured. As another example, when the input word is used more frequently than the recommended similar word, the automatic Japanese recommendation system may not recommend the similar word.

  If it is determined in step S501 that the input word is a typographical error, the Japanese automatic recommendation system 100 is searched based on the edit distance based on the similarity score or the word appearance frequency (for example, the word input frequency). The correct word for the word is selected from the similar words and provided (S507).

  The description of FIGS. 1 to 4 can be referred to for portions not specifically described in FIG.

  The automatic Japanese recommendation method according to an embodiment of the present invention includes a computer-readable recording medium including program instructions for executing various operations realized by a computer. The recording medium may include program instructions, data files, data structures, etc. alone or in combination, and the recording medium and program instructions may be specially designed and configured for the purposes of the present invention, It may be known and usable by those skilled in the computer software art. Examples of computer-readable recording media include magnetic media such as hard disks, floppy (registered trademark) disks and magnetic tapes, optical recording media such as CD-ROMs and DVDs, and magnetic-lights such as floppy disks. A medium and a hardware device specially configured to store and execute program instructions such as ROM, RAM, flash memory, and the like are included. The recording medium is also a transmission medium such as an optical or metal line or a waveguide including a carrier wave that transmits a signal for storing program instructions, data structures, and the like. Examples of program instructions include not only machine language code generated by a compiler but also high-level language code that can be executed by a computer using an interpreter or the like. The hardware device described above is configured to operate as one or more software modules to perform the operations of the present invention, and vice versa.

  As described above, the preferred embodiments of the present invention have been described with reference to the preferred embodiments, but those skilled in the relevant technical field will not depart from the spirit and scope of the present invention described in the claims. It will be understood that various modifications and changes can be made to the present invention within the scope. In other words, the technical scope of the present invention is defined based on the claims, and is not limited by the best mode for carrying out the invention.

DESCRIPTION OF SYMBOLS 100: Automatic Japanese recommendation system 101: Wrong character judgment part 102: Kanji-Hiragana conversion part 103: Roman character conversion part 104: Similar word search part 105: Similar word recommendation part 106: Correct word selection part

Claims (27)

A romaji conversion unit that converts the pronunciation of words expressed in Japanese hiragana or katakana forms into romaji,
A similar word search unit that searches for similar words for the word based on the converted romaji;
An automatic Japanese recommendation system characterized by including The similar word search unit searches for a similar word for the word based on the similarity score of the word converted into the Roman characters,
The similarity score is at least one of an input frequency according to the length of the word, an edit distance based on whether the word includes a long sound, a midpoint, a prompt sound, or a muddy sound, or a comparison degree of the original state of the word. The automatic Japanese recommendation system according to claim 1, wherein the automatic Japanese recommendation system is determined based on one.   When the word is a Chinese character, the similar word search unit determines a similarity score based on a comparison result of a form converted into a Roman character, a comparison result of a form converted into a hiragana character, and a comparison result of an original form of a Chinese character The automatic Japanese recommendation system according to claim 2.   2. The Japanese language according to claim 1, further comprising a similar word recommendation unit that converts the searched similar word into a Japanese form of any one of the hiragana, katakana, and kanji characters and recommends the same. Automatic recommendation system.   The similar word recommending unit (1) when the difference between the similarity in the state converted into Roman letters and the similarity in the state not converted into Roman letters exceeds a preset criterion, or (2) the word is recommended 5. The automatic Japanese recommendation system according to claim 4, wherein the similar word is not recommended when used more frequently than the similar word.   5. The automatic Japanese recommendation system according to claim 4, wherein the similar word recommendation unit converts the searched similar word into a form different from the Japanese form of the word and recommends the same. A typographical error determination unit that analyzes the input word to determine whether the word is a typo;
2. The automatic Japanese recommendation system according to claim 1, wherein, when the input word is an erroneous character, the Roman character conversion unit converts the word into a Roman character.   The typographical error determination unit determines whether the word is included in preset typographical data, whether the input frequency of the word or the frequency of document appearance is lower than a preset reference frequency, or whether the word is The automatic Japanese recommendation system according to claim 7, wherein whether or not the word is a typo is determined based on whether or not the word is separated into morphemes.   When the word is a typo, it further includes a correct word selection unit that selects a correct word for the word among the searched similar words based on a similarity score or an edit distance based on a word input frequency. The automatic Japanese recommendation system according to claim 7.   If the input word is kanji, the token is divided into tokens using token division learning data, and kanji-hiragana conversion is performed to convert the hiragana corresponding to the divided tokens using kanji-hiragana conversion learning data. The automatic Japanese recommendation system according to claim 1, further comprising a section.   11. The automatic Japanese recommendation system according to claim 10, wherein the token division learning data is determined by a hidden Markov model-based segmentation learning using a corpus that separates the kanji morpheme tokens. The kanji-hiragana conversion learning data includes a bigram dictionary and a unigram dictionary determined by learning based on a corpus that separates kanji morpheme tokens;
The bigram dictionary is built with a frequency number between tokens,
The automatic Japanese recommendation system according to claim 10, wherein the unigram dictionary is constructed with a frequency number between tokens and hiragana.   The Kanji-Hiragana conversion unit searches a bigram dictionary for the divided tokens, selects a token showing the highest probability, and converts the selected token into a hiragana corresponding to a unigram dictionary. The automatic Japanese recommendation system according to claim 12. The steps performed by the computer are
Converting the pronunciation of a word expressed in Japanese hiragana or katakana form into romaji,
Searching for similar words for the word based on the converted romaji;
An automatic Japanese recommendation method characterized by including The step of searching for a similar word with respect to the word searches for the similar word with respect to the word based on the similarity score of the word converted into the Roman character,
The similarity score is at least one of an input frequency according to the length of the word, an editing distance depending on whether the word includes a long sound, a prompt sound, or a muddy sound, or a degree of comparison of the original state of the word. The automatic Japanese recommendation method according to claim 14, wherein the automatic Japanese recommendation method is determined based on the determination.   The step of searching for a similar word for the word is similar based on the comparison result of the form converted to Romaji, the comparison result of the form converted to Hiragana and the comparison result of the original form of the Chinese character when the word is Kanji. The automatic Japanese recommendation method according to claim 15, wherein a score is determined.   15. The automatic Japanese recommendation method according to claim 14, further comprising the step of converting and recommending the retrieved similar word into a Japanese form of any one of the hiragana, katakana, or kanji. .   The step of converting and recommending the searched similar words into the Japanese form of any one of the hiragana, katakana, and kanji is as follows: The similarity word is not recommended when the difference from the similarity of the state exceeds a preset criterion, or (2) when the word is used more than the recommended similarity word. Item 18. The automatic Japanese recommendation method according to Item 17.   The step of converting the recommended similar word into a Japanese form of any one of the hiragana, katakana, and kanji characters and recommending the searched similar word into a form different from the Japanese form of the word 18. The automatic Japanese recommendation method according to claim 17, wherein conversion is recommended. Analyzing the input word to determine whether the word is a typo;
15. The automatic Japanese recommendation method according to claim 14, wherein the step of converting the pronunciation of the word into a Roman character converts the word into a Roman character when the input word is a typographical error.   The step of determining whether or not the word is a typo includes whether or not the word is included in preset typographical data, whether the word input frequency or the document appearance frequency is higher than a preset reference frequency. 21. The automatic Japanese recommendation method according to claim 20, wherein whether or not the word is a typo is determined based on whether the word is low or whether the word is separated into morphemes.   The method may further include selecting a correct word for the word among the searched similar words based on a similarity score or an edit distance based on a word input frequency when the word is a typo. Item 20. The automatic Japanese recommendation method according to Item 20.   If the input word is kanji, the method further includes dividing the words into tokens using token division learning data, and converting the hiragana corresponding to the divided tokens using kanji-hiragana conversion learning data. The automatic Japanese recommendation method according to claim 14.   24. The automatic Japanese recommendation method according to claim 23, wherein the token division learning data is determined by a hidden Markov model-based segmentation learning using a corpus that separates the kanji morpheme tokens. The kanji-hiragana conversion learning data includes a bigram dictionary and a unigram dictionary determined by corpus-based learning that separates kanji morpheme tokens;
The bigram dictionary is built with a frequency number between tokens,
The automatic Japanese recommendation method according to claim 23, wherein the unigram dictionary is constructed with a frequency number between tokens and hiragana. The step of converting into hiragana corresponding to the divided tokens is as follows:
Searching the bigram dictionary for the divided tokens to select a token representing the maximum probability;
Converting the selected token into a hiragana corresponding to a unigram dictionary;
The automatic Japanese recommendation method according to claim 23, comprising:   A computer-readable recording medium recording a program for causing a computer to execute the method according to any one of claims 14 to 26.

Images