WO2019146837A1

WO2019146837A1 - Method and mobile apparatus for performing word prediction

Info

Publication number: WO2019146837A1
Application number: PCT/KR2018/002869
Authority: WO
Inventors: Joo Hyun Kim; Mi Ran Seok
Original assignee: Phill It Co., Ltd.
Priority date: 2018-01-25
Filing date: 2018-03-12
Publication date: 2019-08-01
Also published as: KR20190090646A

Abstract

Disclosed are a method and mobile apparatus for performing word prediction, in which a word succeeding input words is estimated by applying a first N-gram language model and a second N-gram language model to the input words, the first N-gram language model indicating, as a probability value, a frequency of occurrence regarding a word sequence of consecutive words, and the second N-gram language model indicating, as a probability value, a frequency of occurrence regarding a word sequence including undecided words tagged with word attributes.

Description

METHOD AND MOBILE APPARATUS FOR PERFORMING WORD PREDICTION

One or more embodiments relate to a method and mobile apparatus for performing word prediction.

As mobile apparatuses have become widely used, interest in methods of increasing user convenience of mobile apparatuses has risen.

One or more embodiments include a method and mobile apparatus for performing word prediction, in which a word succeeding input words is estimated by applying a first N-gram language model indicating a frequency of occurrence regarding a word sequence of consecutive words as a probability value and a second N-gram language model indicating a frequency of occurrence regarding a word sequence including undecided words having word attributes tagged as a probability value to the input words.

Additional aspects will be set forth in part in the description which follows and, in part, will be apparent from the description, or may be learned by practice of the presented embodiments.

According to one or more embodiments, a method of performing word prediction includes: receiving an input of words; performing first estimation of a word succeeding the input words by applying a first N-gram language model to the input words, wherein the first N-gram language model indicates, as a probability value, a frequency of occurrence regarding a word sequence of consecutive words; when the estimated word fails to satisfy a predetermined condition, performing second estimation of a word succeeding the input words by applying a second N-gram language model to the input words, some of which are replaced with undecided words tagged with word attributes of some of the input words, wherein the second N-gram language model indicates a frequency of occurrence regarding a word sequence of consecutive words including undecided words tagged with word attributes; and recommending a word succeeding the input words, based on a result of estimation using the language models.

According to one or more embodiments, a mobile apparatus for performing word prediction includes: a user interface; a processor; and a memory storing instructions executable by the processor, wherein the processor executes the instructions: to receive an input of words via the user interface; to perform first estimation of a word succeeding the input words by applying a first N-gram language model to the input words, wherein the first N-gram language model indicates, as a probability value, a frequency of occurrence regarding a word sequence of consecutive words; when the estimated word fails to satisfy a predetermined condition, to perform second estimation of a word succeeding the input words by applying a second N-gram language model to the input words, some of which are replaced with undecided words tagged with word attributes of some of the input words, wherein the second N-gram language model indicates a frequency of occurrence regarding a word sequence of consecutive words including undecided words tagged with word attributes; and to recommend a word succeeding the input words, based on a result of estimation using the language models.

These and/or other aspects will become apparent and more readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings in which:

FIG. 1 illustrates a mobile apparatus for performing word prediction and a server for providing a language model, according to an embodiment;

FIG. 2 is a flowchart illustrating a method of building a language model for use in word prediction, according to an embodiment;

FIG. 3 is a detailed flowchart of a process of generating a second N-gram language model, according to an embodiment;

FIG. 4 illustrates an example in which a first N-gram language model and a second N-gram language model are generated when a corpus is provided, according to an embodiment;

FIG. 5 is a flowchart illustrating a method of performing word prediction, according to an embodiment; and

FIG. 6 is a detailed flowchart of a process of second estimation of a word succeeding input words by using a second N-gram language model, according to an embodiment.

Reference will now be made in detail to embodiments, examples of which are illustrated in the accompanying drawings, wherein like reference numerals refer to like elements throughout. In this regard, the present embodiments may have different forms and should not be construed as being limited to the descriptions set forth herein. Accordingly, the embodiments are merely described below, by referring to the figures, to explain aspects of the present description. As used herein, the term "and/or" includes any and all combinations of one or more of the associated listed items. Expressions such as "at least one of," when preceding a list of elements, modify the entire list of elements and do not modify the individual elements of the list.

Hereinafter, one or more embodiments will be described below with reference to accompanying drawings. One or more embodiments described below may be modified variously. In order to describe features of the one or more embodiments more clearly, descriptions about items that are well known to one of ordinary skill in the art to which the one or more embodiments below pertain will be omitted.

Moreover, when it is mentioned that an element is "connected" to another element, it means not only "direct connection" but also "connection" with other elements interposed between the two elements. Throughout the specification, when an element "includes" another element, other elements may be further included, rather than excluding the existence of other elements, unless otherwise described.

One or more embodiments relate to a method and mobile apparatus for performing word prediction. Detailed descriptions about elements well known to one of ordinary skill in the art to which the embodiments herein pertain will be omitted.

FIG. 1 illustrates a mobile apparatus 100 for performing word prediction and a server 200 for providing a language model, according to an embodiment.

The mobile apparatus 100 may be a smartphone, a tablet personal computer (PC), a laptop computer, etc. In one or more embodiments according to the present disclosure, the mobile apparatus 100 may also be a wearable device such as a smartwatch.

Referring to FIG. 1, the mobile apparatus 100 for performing word prediction, according to an embodiment, may include a memory 110, a processor 120, and a user interface 130. One of ordinary skill in the art related to the present embodiment would understand that other universal elements in addition to the elements shown in FIG. 1 may be further included in the mobile apparatus 100.

The mobile apparatus 100 is an electronic apparatus having an operating system (OS) installed and capable of displaying a processing result according to a user input by executing an application installed thereon. The mobile apparatus 100 may be a smartphone, a tablet PC, a laptop computer, a digital camera, etc. The term "application" refers to an application program or a mobile application. A user may select and execute an application from among various kinds of applications installed on the mobile apparatus 100.

The memory 110 may store software and/or a program. For example, the memory 110 may store an application, a program such as an application programming interface (API), and various kinds of data.

The processor 120 may access and use the data stored in the memory 110 or may store new data in the memory 110. Also, the processor 120 may execute the program installed in the memory 110. Also, the processor 120 may install, on the memory 110, an application received from outside.

The processor 120 may include at least one processor. The processor 120 may control other elements included in the mobile apparatus 100 to perform an operation corresponding to the user input received via the user interface 130. The processor 120 may include at least one specialized processor corresponding to each function or may be an integrated-type processor. For example, the processor 120 may execute the program stored in the memory 110, may read data or a file stored in the memory 110, or may store a new file on the memory 110.

The user interface 130 may receive the user input, etc. from the user. The user interface 130 may display information such as a result of executing an application on the mobile apparatus 100, a processing result corresponding to the user input, and a status of the mobile apparatus 100. The user interface 130 may include hardware units for receiving an input from the user or providing an output from the mobile apparatus 100, and may also include an exclusive software module for driving the hardware units. For example, the user interface 130 may include an operation panel such as a touch panel for receiving the user input, a display panel for displaying a screen, etc. The user interface 130 may be a touch screen in which the operation panel and the display panel are coupled to each other, but is not limited thereto.

The memory 110 may store instructions that are executable by the processor 120. The processor 120 may execute the instructions stored in the memory 110. The processor 120 may execute the application installed on the mobile apparatus 100 according to the user input. The processor 120 may display a virtual keyboard and an input field for receiving the user input on a screen of the mobile apparatus 100, via the user interface 130.

The processor 120 may receive an input of words via the user interface 130, and may predict a word succeeding the input words. The processor 120 may estimate the word succeeding the input words by applying a language model to the input words, and may recommend the estimated word to the user via the user interface 130. The language model may be a language model that has learned a rule of constructing a sentence based on the order of words for a word sequence of consecutive words with respect to a number of corpuses collected as samples. The language model may calculate a frequency of occurrence regarding a word sequence of consecutive words as a probability value to learn the rule of constructing a sentence. For example, an N-gram language model may be a probabilistic language model with a series of N words represented by way of probability. The N-gram language model may be a model indicating a probability of appearance of N consecutive words. The N-gram language model may indicate a frequency of occurrence as a probability value according to a statistical method with respect to each of various combinations of N words. The N-gram language model may be used to estimate an N-th word from a sequence of (N-1) words and output a probability thereof.

For example, the processor 120 may estimate a word succeeding input words by using a first N-gram language model indicating, as a probability value, a frequency of occurrence regarding a word sequence of consecutive words. When the estimated word fails to satisfy a predetermined condition, the processor 120 may estimate a word succeeding the input words, some of which are replaced with undecided words tagged with words attributes of some of the input words. The processor 120 may estimate the word succeeding the input words by using a second N-gram language model indicating, as a probability value, a frequency of occurrence regarding a word sequence of consecutive words including undecided words tagged with word attributes. The processor 120 may select some of input words, determine word attributes of the selected words, and replace the selected words with undecided words tagged with the determined word attributes. The processor 120 may apply the second N-gram language model to the input words having the selected words replaced with the undecided words tagged with the determined word attributes. The word attributes may be at least one of a part of speech, a tense, a physical quantity, a place name, an organization's name, and a person's name. The processor 120 may recommend a word succeeding the input words, based on a result of estimation using the language models. The language models may be built in the mobile apparatus 100 and be stored in the memory 110.

The language models may be provided to the mobile apparatus 100 as the mobile apparatus 100 communicates with an external apparatus. To this end, the mobile apparatus 100 may include a communication interface supporting at least one of various wired/wireless communication methods. The mobile apparatus 100 may perform wired/wireless communication with another device or a network. Examples of the wireless communication may include, for example, wireless fidelity (Wi-Fi), Bluetooth, long term evolution (LTE), etc. Examples of the wired communication may include, for example, universal serial bus (USB), high-definition multimedia interface (HDMI), etc. The mobile apparatus 100 may be connected to an external apparatus located outside the mobile apparatus 100 to transmit/receive signals or data. As shown in FIG. 1, the mobile apparatus 100 may communicate with the server 200 for providing a language model. The server 200 may include a memory storing various kinds of databases to build a language model and provide the language model to the mobile apparatus 100, a processor for generating a language model, a communication interface, etc. The mobile apparatus 100 may receive a language model trained in the server 200 for providing a language model from the server 200 and use the language model received from the server 200 as a language model for performing word prediction.

Hereinafter, a process of building a language model and a process of performing word prediction by using a language model will be described in detail.

FIG. 2 is a flowchart illustrating a method of building a language model for use in word prediction, according to an embodiment.

In one or more embodiments according to the present disclosure, the language model for use in word prediction may be preinstalled in the mobile apparatus 100 or may be built in the mobile apparatus 100. However, a case of providing a language model from the server 200 to the mobile apparatus 100 will be hereinafter described as an example. When a language model is built in the mobile apparatus 100, the mobile apparatus 100 may build the language model by the same method as described with reference to FIGS. 2 to 4.

Referring to FIG. 2, in operation S210, with respect to each corpus including a plurality of words, the server 200 may divide a corpus into words.

In operation S220, the server 200 may generate a first N-gram language model indicating, as a probability value, a frequency of occurrence regarding a word sequence of N consecutive words from among a plurality of words constituting a corpus.

In operation S230, the server 200 may generate a second N-gram language model indicating, as a probability value, a frequency of occurrence regarding a word sequence of N consecutive words from among a plurality of words constituting a corpus, the word sequence of N consecutive words having some selected words replaced with undecided words tagged with word attributes of the selected words.

FIG. 3 is a detailed flowchart of a process of generating a second N-gram language model, according to an embodiment.

Referring to FIG. 3, in operation S310, the server 200 may select some words in a word sequence of N consecutive words from among a plurality of words constituting a corpus. The server 200 may use an exceptional word database storing irreplaceable exceptional words to select some words determined as not corresponding to the irreplaceable exceptional words. For example, the exceptional word database may store consecutive words whose relationship is irreplaceable, such as an idiom.

In operation S320, the server 200 may replace the selected words with undecided words tagged with word attributes of the selected words. The server 200 may determine words attributes of the selected words by using a word attribute database storing word attributes of all words, for example, words registered in the dictionary. For example, the word attribute database may store at least one word attribute with respect to each word, and may store a word attribute regarding a part of speech or a tense type of each word, or a word attribute related to a physical quantity, a place name, an organization’s name, or a person’s name.

In operation S330, the server 200 may obtain, as a probability value, a frequency of occurrence regarding a word sequence of N consecutive words of which selected words are replaced with undecided words tagged with word attributes of the selected words.

In operation S340, the server 200 may store the obtained probability value and the word sequence of N consecutive words of which the selected words are replaced with undecided words tagged with word attributes of the selected words, so as to correspond to each other.

Referring to FIG. 2 again, in operation S240, the server 200 may generate an integrated language model of the first N-gram language model and the second N-gram language model. The server 200 may store the integrated language model for use in word prediction or may store each of the first N-gram language model and the second N-gram language model.

FIG. 4 illustrates an example in which a first N-gram language model and a second N-gram language model are generated when a corpus is provided, according to an embodiment.

To build a language model, a frequency of appearance of a word sequence of N consecutive words in each of a plurality of corpuses may be checked, and a probability value may be calculated accordingly. FIG. 4 illustrates a process in which a first N-gram language model and a second N-gram language model are generated when a corpus of "Today we know that between 1948 and 1990" is provided.

The corpus "Today we know that between 1948 and 1990" illustrated in FIG. 4 may be divided into words such as "Today", "we", "know", "that", "between", "1948", "and", and "1990".

The first N-gram language model may generate a 1-gram or unigram list with respect to each word constituting a corpus, and may calculate each frequency of occurrence as a probability value. In addition, the first N-gram language model may generate a 2-gram list with respect to two consecutive words from among words constituting the corpus, and may calculate each frequency of occurrence as a probability value. For example, a list of two consecutive words such as "Today we", "we know", …, and "and 1990" may be generated, and each frequency of occurrence may be calculated as a probability value. In the same manner, for 3-gram, 4-gram, …, and N-gram, respectively, frequencies of occurrence of word sequences of three, four, …, and N consecutive words from among a plurality of words constituting the corpus may be calculated as probability values, and thus, the first N-gram language model may be generated.

The second N-gram language model may indicate, as a probability value, a frequency of occurrence regarding a word sequence of N consecutive words from among a plurality of words constituting a corpus, the word sequence of N consecutive words having some selected words replaced with undecided words tagged with word attributes of the selected words. Since the second N-gram language model selects and replaces K words from among the N words, the second N-gram language model may be referred to as K-skip-N-gram. For example, 1-skip-2-gram may select one word from among a word sequence of two consecutive words from among words constituting a corpus, that is, two consecutive words such as "Today we", "we know", …, and "and 1990" and determine a word attribute of the selected word, and may calculate, as a probability value, a frequency of occurrence regarding a word sequence of which the selected word is replaced with an undecided word tagged with the determined word attribute. With respect to a first word sequence "Today we", the word 'Today' may be selected first, a word attribute of the word 'Today' may be determined as 'noun', and a frequency of occurrence regarding a word sequence "[noun] we" of which the word 'Today' is replaced with an undecided word '[noun]' tagged with the word attribute 'noun' may be calculated as a probability value. In addition, with respect to the word sequence "Today we", the word 'we' may be selected this time, a word attribute of the word 'we' may be determined as 'personal pronoun', and a frequency of occurrence regarding a word sequence "Today [personal pronoun]" of which the word 'we' is replaced with an undecided word '[personal pronoun]' tagged with the word attribute 'personal pronoun' may be indicated as a probability value. In addition, with respect to each word sequence of two consecutive words, such as "we know", …, and "and 1990", one word may be selected, and with respect to each word sequence of which the selected word is replaced with an undecided word tagged with a word attribute of the selected word, a frequency of occurrence may be calculated as a probability value. In the same manner, for 1-skip-3-gram, 1-skip-4-gram, …, and K-skip-N-gram, respectively, frequencies of occurrence regarding word sequences having some K words selected in word sequences of three, four, …, and N consecutive words from among a plurality of words constituting the corpus, the selected words replaced with undecided words tagged with word attributes of the selected words may be calculated as probability values, and thus, the second N-gram language model may be generated.

A language model for word prediction may be an integrated language model of the first N-gram language model and the second N-gram language model.

FIG. 5 is a flowchart illustrating a method of performing word prediction, according to an embodiment.

Referring to FIG. 5, in operation S510, the mobile apparatus 100 may receive an input of words. On the mobile apparatus 100, a user may consecutively input the words via the user interface 130.

In operation S520, the mobile apparatus 100 may estimate a word succeeding the input words by applying a first N-gram language model to the input words, the first N-gram language model indicating, as a probability value, a frequency of occurrence regarding a word sequence of consecutive words. When (N-1) words are input, the mobile apparatus 100 may estimate an N-th word succeeding the input (N-1) words, based on an N-gram list and the probability value. For example, when two consecutive words "Today I" are input, a probability value corresponding to a 3-gram list in the first N-gram language model may be looked up to estimate a third word, and thus, whether there is a word sequence starting with "Today I" in the 3-gram list may be checked, and a third word of the checked word sequence may be estimated as the third word succeeding the input two words.

In operation S530, when the estimated word fails to satisfy a predetermined condition, for example, when there is no word estimated as a result of using the first N-gram language model or when a probability value of a word sequence to which the estimated word belongs is less than a predetermined value, the mobile apparatus 100 may estimate a word succeeding the input words by applying, to the input words of which selected words are replaced with undecided words tagged with word attributes of the selected words, a second N-gram language model indicating, as a probability value, a frequency of occurrence regarding a word sequence of consecutive words including undecided words tagged with word attributes.

Referring to FIG. 6, in operation S610, the mobile apparatus 100 may select some words from among input words. For example, when two consecutive words "Today I" are input, 'Today' or 'I' may be selected.

In operation S620, the mobile apparatus 100 may determine word attributes of the selected words. For example, when two consecutive words "Today I" are input, a word attribute of the selected 'Today' or 'I' may be determined as 'noun' or 'personal pronoun', respectively. In this regard, the same word attribute database used to build a language model may be used.

In operation S630, the mobile apparatus 100 may replace the selected words with undecided words tagged with the determined word attributes. For example, when two consecutive words "Today I" are input, they may be replaced by "Today [personal pronoun]" or "[noun] I".

In operation S640, the mobile apparatus 100 may estimate a word succeeding the input words by applying a second N-gram language model to the input words of which the selected words are replaced with the undecided words tagged with the determined word attributes.

For example, when two consecutive words "Today I" are input, a probability value corresponding to a 1-skip-3-gram list in the second N-gram language model may be looked up to estimate a third word, and thus, whether there is a word sequence starting with "Today [personal pronoun]" or "[noun] I" in the 1-skip-3-gram list may be checked, and a third word of the checked word sequence may be estimated as the third word succeeding the input two words "Today I". In other words, by finding a list including a word sequence "Today [personal pronoun]" or "[noun] I" replaced with undecided words tagged with word attributes of some words in the second N-gram language model instead of finding a list accurately including the input words "Today I" in the first N-gram language model, candidates that may be estimated as a word succeeding the input words may be broadened but with the limitation that at least the input words and the word attributes match each other. When a list including a word sequence "Today [*]" or "[*] I" (where * refers to any word) is found without tagging a word attribute, 'the' or 'a' may be estimated as the third word succeeding the two input words "Today I" from a word sequence such as "Today is the" or "Today is a", and thus, an awkward word may be estimated, and word prediction may be improper.

When a proper word fails to be estimated even by the 1-skip-3-gram list, it may be checked whether there is a word sequence starting with "[personal pronoun]", which is a word attribute of "I" in "Today I", in a 1-skip-2-gram list, and a second word of the checked word sequence may be estimated as the third word succeeding the two input words "Today I". Nevertheless, when a proper word succeeding the input words fails to be estimated, the word may be estimated according to every unigram.

Referring to FIG. 5 again, in operation S540, the mobile apparatus 100 may recommend a word succeeding the input words, based on a result of estimation using the language models.

The embodiments described above may be provided as applications stored on a non-transitory computer-readable storing medium to perform a method of performing word prediction. In other words, the embodiments described above may be provided as applications or computer programs stored on a non-transitory computer-readable storing medium to allow the mobile apparatus 100 to execute each operation of the method of performing word prediction.

In addition, the embodiments described above may be implemented as a non-transitory computer-readable storing medium for storing instructions or data executable by a computer or a processor. At least one of the instructions and the data may be stored in the form of program codes, and when it is executed by a processor, a predetermined program module may be generated to perform a predetermined operation. The non-transitory computer-readable storing medium may be read-only memory (ROM), random-access memory (RAM), flash memory, CD-ROMs, CD-Rs, CD+Rs, CD-RWs, CD+RWs, DVD-ROMs, DVD-Rs, DVD+Rs, DVD-RWs, DVD+RWs, DVD-RAMs, BD-ROMs, BD-Rs, BD-R LTHs, BD-REs, a magnetic tape, a floppy disc, a magneto-optical data storing device, an optical data storing device, a hard disc, a solid-state disk (SSD), and any type of device that is capable of storing instructions or software, related data, data files, and data structures, and providing instructions or software, related data, data files, and data structures to a processor or a computer to allow the processor or the computer to execute the instructions.

It should be understood that embodiments described herein should be considered in a descriptive sense only and not for purposes of limitation. Descriptions of features or aspects within each embodiment should typically be considered as available for other similar features or aspects in other embodiments.

While one or more embodiments have been described with reference to the figures, it will be understood by those of ordinary skill in the art that various changes in form and details may be made therein without departing from the spirit and scope of the disclosure as defined by the following claims.

Claims

A method of performing word prediction, the method comprising:

receiving an input of words;

performing first estimation of a word succeeding the input words by applying a first N-gram language model to the input words, wherein the first N-gram language model indicates, as a probability value, a frequency of occurrence regarding a word sequence of consecutive words;

when the estimated word fails to satisfy a predetermined condition, performing second estimation of a word succeeding the input words by applying a second N-gram language model to the input words, some of which are replaced with undecided words tagged with word attributes of some of the input words, wherein the second N-gram language model indicates a frequency of occurrence regarding a word sequence of consecutive words comprising undecided words tagged with word attributes; and

recommending a word succeeding the input words, based on a result of estimation using the language models.
The method of claim 1, wherein the word attributes comprise at least one of a part of speech, a tense, a physical quantity, a place name, an organization’s name, and a person’s name.
The method of claim 1, wherein the second N-gram language model is generated by:

selecting some words in a word sequence of N consecutive words from among a plurality of words constituting a corpus,

replacing the selected words with undecided words tagged with words attributes of the selected words,

obtaining, as a probability value, a frequency of occurrence regarding a word sequence of the N consecutive words of which the selected words are replaced with the undecided words tagged with the word attributes of the selected words, and

storing the obtained probability value and the word sequence of the N consecutive words of which the selected words are replaced with the undecided words tagged with the word attributes of the selected words, so as to correspond to each other.
The method of claim 3, wherein the selected words are determined as not corresponding to irreplaceable exceptional words by using an exceptional word database storing irreplaceable exceptional words.
The method of claim 3, wherein the word attributes of the selected words are determined by a word attribute database storing word attributes regarding words registered in a dictionary.
The method of claim 1, wherein the performing of the second estimation comprises:

selecting some of the input words;

determining word attributes of the selected words;

replacing the selected words with the undecided words tagged with the determined word attributes; and

estimating the word succeeding the input words by applying the second N-gram language model to the input words of which the selected words are replaced with the undecided words tagged with the determined word attributes.
A mobile apparatus comprising:

a user interface;

a processor; and

a memory storing instructions executable by the processor,

wherein the processor executes the instructions:

to receive an input of words via the user interface;

to perform first estimation of a word succeeding the input words by applying a first N-gram language model to the input words, wherein the first N-gram language model indicates, as a probability value, a frequency of occurrence regarding a word sequence of consecutive words;

when the estimated word fails to satisfy a predetermined condition, to perform second estimation of a word succeeding the input words by applying a second N-gram language model to the input words, some of which are replaced with undecided words tagged with word attributes of some of the input words, wherein the second N-gram language model indicates a frequency of occurrence regarding a word sequence of consecutive words comprising undecided words tagged with word attributes; and

to recommend a word succeeding the input words, based on a result of estimation using the language models.
The mobile apparatus of claim 7, wherein the word attributes comprise at least one of a part of speech, a tense, a physical quantity, a place name, an organization’s name, and a person’s name.
The mobile apparatus of claim 7, wherein the second N-gram language model is provided by a server capable of communicating with the mobile apparatus,

wherein the server

selects some words in a word sequence of N consecutive words from among a plurality of words constituting a corpus,

replaces the selected words with undecided words tagged with word attributes of the selected words and obtains a probability value as a frequency of occurrence regarding a word sequence of the N consecutive words of which the selected words are replaced with the undecided words tagged with the word attributes of the selected words, and

stores the obtained probability value and the word sequence of the N consecutive words of which the selected words are replaced with the undecided words tagged with the word attributes of the selected words, so as to correspond to each other.
The mobile apparatus of claim 9, wherein the selected words are determined as not corresponding to irreplaceable exceptional words by using an exceptional word database storing irreplaceable exceptional words.
The mobile apparatus of claim 9, wherein the word attributes of the selected words are determined by a word attribute database storing word attributes regarding words registered in a dictionary.
The mobile apparatus of claim 7, wherein, when the second estimation is performed, the processor selects some of the input words, determines word attributes of the selected words, replaces the selected words with the undecided words tagged with the determined word attributes, and estimates the word succeeding the input words by applying the second N-gram language model to the input words of which the selected words are replaced with the undecided words tagged with the determined word attributes.