KR20200078454A - Method and system for correcting keyboard typo based on deep learning model - Google Patents

Abstract

Provided are a method and system for correcting a keyboard typo based on a deep learning model. According to one embodiment of the present invention, the method may comprise the steps of: collecting a user log associated with a user input for a virtual keyboard; generating learning data for a deep learning model by processing the collected user log; learning the deep learning model using the learning data; and determining whether there is a grammatical error according to the user input using the learning data by at least one processor.

Description

METHOD AND SYSTEM FOR CORRECTING KEYBOARD TYPO BASED ON DEEP LEARNING MODEL}

The following description is a deep learning model-based keyboard typographic correction method, a computer apparatus for performing the keyboard typographic correction method, and a computer for executing a keyboard typographic correction method according to embodiments of the present invention in combination with a computer apparatus. It relates to a computer program stored on a readable recording medium and the recording medium.

Unlike a typical keyboard, a virtual keyboard used in a touch screen of a smart phone tends to generate a lot of typos due to environmental restrictions. For example, since the size of the touch screen is different depending on the type of the terminal, the size of the virtual keyboard is also different, and since there is no physical boundary between the virtual keys of the virtual keyboard, depending on the thickness of the user's finger or the usage pattern of the user. There is a high possibility of incorrect input.

There are various prior arts related to this typo. For example, Korean Registered Patent No. 10-1858999 relates to an input correction device for a virtual keyboard, stores reference coordinates corresponding to each key constituting the virtual keyboard, and a touch position touched by the virtual keyboard through a user. By deriving the correction candidate key based on the distance of the reference coordinate, and deriving and presenting the correction candidate word by combining a plurality of selected correction candidate keys, the correction guide information is utilized even if the user erases the incorrectly entered word and does not re-enter it manually. In order to solve the input error has been disclosed. However, these prior arts are only related to a technique for helping a user correct a typo directly, and not a technique for correcting a typo.

In addition, there is a conventional technique for correcting a typo using a specific user input pattern. However, in these prior arts, a user's input pattern can be applied only to a specific smartphone virtual keyboard environment and/or a specific user, and thus there is a problem that it cannot be applied to various virtual keyboard environments and/or various users. .

Deep learning model-based keyboard error correction method, a computer device for performing the keyboard error correction method, and a computer readable recording medium in combination with a computer device to execute a keyboard error correction method according to embodiments of the present invention in a computer device It provides a computer program and its recording medium.

A method for correcting keyboard typos in a computer device comprising at least one processor, the method comprising: collecting, by the at least one processor, a user log associated with a user input to a virtual keyboard; Generating learning data for a deep learning model by processing the collected user log by the at least one processor; Training, by the at least one processor, a deep learning model using the training data; And determining, by the at least one processor, whether there is a grammatical error according to the user input using the learned deep learning model.

In combination with a computer device, a computer program stored in a computer-readable recording medium is provided to execute the method for correcting keyboard typos in a computer device.

Provided is a computer-readable recording medium in which a program for executing the method for correcting the keyboard error is recorded on a computer device.

A computer device comprising: at least one processor implemented to execute readable instructions on the computer device, the user log associated with user input to the virtual keyboard is collected by the at least one processor, and the at least one processor By processing the collected user log, and generates training data for a deep learning model by the processor of the, by using the learning data, by using the training data to train the deep learning model, the at least one processor By providing a computer device characterized by determining whether or not a grammatical error according to the user input using the learned deep learning model.

As the data is normalized for typo correction, the keyboard typo can be corrected for various smartphone virtual keyboard environments and/or various users without any difference in performance.

By applying the pre-trained deep learning model directly, it can be used to correct keyboard errors without additional data collection.

1 is a diagram showing an example of a network environment according to an embodiment of the present invention.
2 is a block diagram showing an example of a computer device according to an embodiment of the present invention.
3 is a flowchart illustrating an example of a method for correcting keyboard typos according to an embodiment of the present invention.
4 is a diagram illustrating an example of synthesizing and using the results of a plurality of deep learning models in an embodiment of the present invention.
5 is a diagram illustrating an example of a virtual keyboard in one embodiment of the present invention.
6 is a diagram showing an example of a method of calculating a key rate when a penalty is present in one embodiment of the present invention.
7 is a diagram showing an example of a method of calculating a key rate when there is no penalty in one embodiment of the present invention.
8 is a diagram illustrating an example for describing a key matrix in an embodiment of the present invention.
9 is a diagram illustrating an example of peripheral keys for an input key in an embodiment of the present invention.
10 is a view showing an example of calculating a probability of a next letter using three letters in an embodiment of the present invention.

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings.

A method for correcting keyboard typos based on a deep learning model according to embodiments of the present invention may be implemented through a computer device to be described later. In this case, a computer program according to an embodiment of the present invention may be installed and driven in the computer device, and the computer device may perform the earset control method according to the embodiments of the present invention under the control of the driven computer program. . The above-described computer program can be stored in a computer-readable recording medium in combination with a computer device to execute the earset control method in the computer. According to an embodiment, a method for correcting keyboard typos may be performed through two or more computer devices communicating with each other through a network. For example, training of a deep learning model is performed on a first computer device implementing a server, and a deep learning model trained on a server is mounted on a second computer device device implementing a client to correct keyboard typos in a client. Can be utilized.

1 is a diagram showing an example of a network environment according to an embodiment of the present invention. The network environment of FIG. 1 shows an example including a plurality of electronic devices 110, 120, 130, 140, a plurality of servers 150, 160, and a network 170. 1 is an example for explaining the invention, and the number of electronic devices or the number of servers is not limited as shown in FIG. 1. In addition, the network environment of FIG. 1 is merely an example of one of the environments applicable to the embodiments, and the environment applicable to the embodiments is not limited to the network environment of FIG. 1.

The plurality of electronic devices 110, 120, 130, and 140 may be a fixed terminal or a mobile terminal implemented as a computer device. For example, a plurality of electronic devices (110, 120, 130, 140), smart phones (smart phone), mobile phones, navigation, computers, notebooks, digital broadcasting terminals, PDA (Personal Digital Assistants), PMP (Portable Multimedia Player) ), tablet PC, etc. For example, although the shape of a smartphone is shown as an example of the electronic device 1 110 in FIG. 1, in the embodiments of the present invention, the electronic device 1 110 substantially uses the wireless or wired communication method to connect the network 170. It may mean one of various physical computer devices capable of communicating with other electronic devices 120, 130, 140 and/or servers 150, 160.

The communication method is not limited, and a communication method using a communication network (for example, a mobile communication network, a wired Internet, a wireless Internet, and a broadcasting network) that the network 170 may include may include short-range wireless communication between devices. For example, the network 170 includes a personal area network (PAN), a local area network (LAN), a campus area network (CAN), a metropolitan area network (MAN), a wide area network (WAN), and a broadband network (BBN). , Any one or more of the networks such as the Internet. Further, the network 170 may include any one or more of a network topology including a bus network, a star network, a ring network, a mesh network, a star-bus network, a tree, or a hierarchical network. It is not limited.

Each of the servers 150 and 160 communicates with a plurality of electronic devices 110, 120, 130, and 140 through a network 170 to provide commands, codes, files, contents, services, or the like, or a plurality of computers. It can be implemented with devices. For example, the server 150 is a service (eg, financial service, payment service, social network service, messaging service) to a plurality of electronic devices 110, 120, 130, and 140 accessed through the network 170. It may be a system providing a search service, a mail service, a content providing service, etc.).

2 is a block diagram showing an example of a computer device according to an embodiment of the present invention. Each of the plurality of electronic devices 110, 120, 130, 140 described above, or each of the servers 150, 160 may be implemented by the computer device 200 illustrated through FIG. 2.

2, the computer device 200 may include a memory 210, a processor 220, a communication interface 230, and an input/output interface 240. The memory 210 is a computer-readable recording medium, and may include a non-permanent mass storage device such as random access memory (RAM), read only memory (ROM), and a disk drive. Here, a non-destructive large-capacity recording device such as a ROM and a disk drive may be included in the computer device 200 as a separate permanent storage device separate from the memory 210. In addition, an operating system and at least one program code may be stored in the memory 210. These software components may be loaded into the memory 210 from a computer-readable recording medium separate from the memory 210. Such a separate computer-readable recording medium may include a computer-readable recording medium such as a floppy drive, disk, tape, DVD/CD-ROM drive, and memory card. In other embodiments, software components may be loaded into memory 210 through communication interface 230 rather than a computer-readable recording medium. For example, software components may be loaded into memory 210 of computer device 200 based on a computer program installed by files received over network 170.

The processor 220 may be configured to process instructions of a computer program by performing basic arithmetic, logic, and input/output operations. Instructions may be provided to processor 220 by memory 210 or communication interface 230. For example, the processor 220 may be configured to execute a received command according to program code stored in a recording device such as the memory 210.

The communication interface 230 may provide a function for the computer device 200 to communicate with other devices (eg, the storage devices described above) through the network 170. For example, requests, commands, data, files, etc. generated by the processor 220 of the computer device 200 according to program codes stored in a recording device such as the memory 210 are controlled by the communication interface 230. 170) to other devices. Conversely, signals, commands, data, files, and the like from other devices may be received through the network 170 to the computer device 200 through the communication interface 230 of the computer device 200. Signals, commands, data, etc. received through the communication interface 230 may be transferred to the processor 220 or the memory 210, and files, etc., may be further stored by the computer device 200 (described above) Permanent storage device).

The input/output interface 240 may be a means for interfacing with the input/output device 250. For example, the input device may include a device such as a microphone, keyboard, or mouse, and the output device may include a device such as a display or speaker. As another example, the input/output interface 240 may be a means for interfacing with a device in which functions for input and output are integrated into one, such as a touch screen. The input/output device 250 may be configured as a computer device 200 and a single device.

Further, in other embodiments, the computer device 200 may include fewer or more components than those in FIG. 2. However, there is no need to clearly show most prior art components. For example, the computer device 200 may be implemented to include at least some of the input/output devices 250 described above, or may further include other components such as a transceiver, a database, and the like.

3 is a flowchart illustrating an example of a method for correcting keyboard typos according to an embodiment of the present invention. The keyboard error correction method according to the present exemplary embodiment may be performed by the computer device 200 implementing any one of the plurality of electronic devices 110, 120, 130, and 140 described above. For example, the processor 220 of the computer device 200 may be implemented to execute control instructions according to code of an operating system included in the memory 210 or code of at least one program. Here, the processor 220 is the computer device 200 in accordance with the control command provided by the code stored in the computer device 200 to perform the steps (310 to 340) of the method of FIG. Can be controlled. For example, the at least one program described above may be an application installed in the computer device 200 to correct a keyboard error.

In step 310, the computer device 200 may collect user logs associated with user input to the virtual keyboard. For example, the user log may include an identifier of a key of a virtual keyboard selected by user input and coordinates of the user input. According to an embodiment, the user log is a slope value (eg, yaw, pitch) of the computer device 200 measured through a sensor included in the computer device 200 at the time when the user input occurs. And a value of each roll. Meanwhile, the computer device 200 may know information about the size (width and height) of the virtual keyboard and the center coordinates of each key of the virtual keyboard.

In step 320, the computer device 200 may process the collected user logs to generate training data for a deep learning model. The processing method of the user log for generating the learning data will be described in more detail later.

In step 330, the computer device 200 may train the deep learning model using the training data. For example, the deep learning model may be trained to derive a probability value (key ratio) for a key to be recognized next based on the previous n (n is a natural number) key input. In this case, the deep learning model may be trained to calculate and return a key rate for each of the m (m is a natural number) keys directly related to character input in the virtual keyboard. Here, the key rate can be used as a proximity between a currently recognized key and an adjacent key.

In operation 340, the computer device 200 may determine whether or not a grammatical error is caused by user input using the learned deep learning model. According to an embodiment, the computer device 200 may correct a key input in which a grammatical error occurs through a key rate suggested by the trained deep learning model.

On the other hand, deep learning models exist for deep learning models that correctly match the correct answers (for example, no grammatical errors) depending on the type of learning data, and conversely, deep learning models for correct incorrect answers (for example, there are grammatical errors). do. Therefore, determining whether or not a grammatical error is related to various input data through one deep learning model may lead to low performance for determining whether a relatively low grammatical error is present. Accordingly, according to an embodiment, the computer device 200 may train each of a plurality of deep learning models using learning data. At this time, when a plurality of deep learning models are learned and mounted on the computer device 200, the computer device 200 inputs user input to each of the plurality of deep learning models, and synthesizes the results, such as the existence of a typo. The grammatical error may be finally determined or the recognized key input may be corrected according to the grammatical error.

As a more specific example, three FFNN (Feed Forward Neural Network) models having three layers as a deep learning model without a size limit and three FFNN models having two layers as a deep learning model having a 1 MB size limit may be utilized. Can. The former deep learning model can use different processed data for each model, including language models. The latter deep learning model does not include a language model, and different processed data can be used for each model. In addition to the FFNN model, different deep learning models such as a convolutional neural network (CNN) and a recurrent neural network (RNN) may be used.

4 is a diagram illustrating an example of synthesizing and using the results of a plurality of deep learning models in an embodiment of the present invention. 4 shows an example of calculating the final result by synthesizing the results of each of the three FFNN models.

Also, according to an embodiment, at least one of the steps 310 to 340 included in the method of FIG. 3 may be performed by another computer device implementing any one of the servers 150 and 160 described above. For example, the server 150 may generate training data for a deep learning model by receiving and processing a user log collected from the computer device 200, and learn the deep learning model using the generated training data. Can. At this time, the trained deep learning model may be provided to and mounted on the computer device 200, and the computer device 200 may determine whether or not a grammatical error occurs according to user input in step 340 using the mounted deep learning model, or Recognized keystrokes can be corrected according to grammatical error determination.

Hereinafter, an example of a process of generating learning data will be described. The training data may include various types of data, such as data described later, and at least some of the data described later may be used as learning data.

First, data required for generation of learning data will be described.

1. Normalized coordinates

Coordinates according to user input may be normalized based on the width w and height h of the virtual keyboard. For example, the computer device 200 may calculate (x/w, y/h) as normalized coordinates for one coordinate (x, y). Table 1 below shows the values of the coordinates (x, y) for the input string'ㅇ' (Korean consonant'response'), the width and height of the virtual keyboard, and normalized in an embodiment of the present invention. Coordinates (x/w, y/h) are shown. The values of the normalized coordinates (x/w, y/h) represent the value rounded to the third decimal place.

Input string
/ Identification Recognized coordinates
x/y Width w / height h Normalized coordinates
x/w / y/h ㅇ 12.0 124.50 66.00 37.50 54.00 3.32 1.22 ㅇ 12.0 128.67 98.67 41.40 56.50 3.10 1.74

Since the recognized coordinates (x, y) may be different for the same input string, the normalized coordinates may also vary. Meanwhile, the center coordinates of keys included in the virtual keyboard may also be calculated as coordinates of the normalized coordinate system. .

5 is a diagram illustrating an example of a virtual keyboard in one embodiment of the present invention. 5 shows an example of a virtual keyboard 510 that includes multiple keys. At this time, the center coordinates (a, b) for each of the plurality of keys included in the virtual keyboard 510 may also be expressed as normalized coordinates (a/w, b/h) in the normalized coordinate system, and the computer device 200 may calculate and store these normalized coordinates (a/w, b/h). For example, Table 2 below shows an example of center coordinates for 29 keys directly related to input of characters among keys of the virtual keyboard 510 shown in FIG. 5.

key Center coordinates Heat 'ㅂ' [0.5, 0.5] 1 column 'ㅈ' [1.5, 0.5] 1 column 'ㄷ' [2.5, 0.5] 1 column 'G' [3.5, 0.5] 1 column 'ㅅ' [4.5, 0.5] 1 column 'ㅛ' [5.5, 0.5] 1 column 'ㅕ' [6.5, 0.5] 1 column 'ㅑ' [7.5, 0.5] 1 column 'ㅐ' [8.5, 0.5] 1 column 'ㅔ' [9.5, 0.5] 1 column 'ㅁ' [1, 1.5] 2 rows 'N' [2, 1.5] 2 rows 'ㅇ' [3, 1.5] 2 rows 'L' [4, 1.5] 2 rows 'Hehe' [5, 1.5] 2 rows 'ㅗ' [6, 1.5] 2 rows 'ㅓ' [7, 1.5] 2 rows 'ㅏ' [8, 1.5] 2 rows 'ㅣ' [9.25, 1.5] 2 rows 'C' (Control) [0.5, 2.5] 3 columns 'Lol' [2, 2.5] 3 columns 'T' [3, 2.5] 3 columns 'ㅊ' [4, 2.5] 3 columns 'ㅍ' [5, 2.5] 3 columns 'ㅠ' [6, 2.5] 3 columns 'TT' [7, 2.5] 3 columns 'ㅡ' [8, 2.5] 3 columns 'D' (Delete) [9.25, 2.5] 3 columns 'S' (Space) [5, 3.5] 4 columns

2. The normalized coordinate correction computer apparatus 200 may calculate and accumulate the difference values Diff x, Diff y between the center coordinates of the key and the normalized coordinates for the recognized coordinates. At this time, if the recognized coordinate is on the left of the center coordinate of the key, Diff x may have a positive value, and if it is on the right, Diff x may have a negative value. Similarly, if the recognized coordinate is above the center coordinate of the key, Diff y may have a positive value, and if it is below, Diff y may have a negative value.

In this case, the average of the difference values for each consonant and vowel may be used. For example, Revision x and Revision y, which are the average values of the calculated difference values, are added to the normalized coordinates and used as corrected coordinates (a/w+Revision x, b/h+Revision y).

As an example, Table 3 shows normalized coordinates (a/w, b/h) and difference values Diff x, Diff y for specific input strings.

Input string Normalized coordinates Difference (Diff x, Diff y) 'ㅇ' 2.72 1.76 0.28 -0.26 'ㅏ' 8.32 1.24 0.32 0.26 'N' 1.78 1.46 0.22 0.04 'ㅕ' 6.89 0.33 -0.39 0.17 'ㅇ' 2.88 1.90 0.12 -0.40 'Hehe' 5.11 1.46 -0.10 0.04

In this case, Revision x, the average value of the correction values for consonants, is 0.13((0.28 + 0.22 + 0.12 + (- 0.10)) / 4), and the revision value for consonants is 0.15(((-0.26) + 0.04 + (-0.40) + 0.04) / 4). In this case, the corrected coordinates (a/w+Revision x, b/h+Revision y) when predicting the next consonant may be calculated as Table 4 below as an example.

Input string Normalized coordinates Calibrated coordinates 'ㅇ' 2.86 1.46 2.86 + 0.13 = 2.99 1.46 + 0.15 = 1.61

3. Key Ratio The key ratio is the probability for each of all keys on the keyboard calculated using normalized coordinates. For example, one normalized coordinate for 29 keys described through Table 2 has 29 key rates. The key ratio can be calculated in the form of an array.

For example, Table 5 shows an example of the key rate calculated for one normalized coordinate.

key Height 'ㅂ' 0.00 'ㅈ' 0.00 'ㄷ' 0.00 'G' 0.18 'ㅅ' 0.00 'ㅛ' 0.00 'ㅕ' 0.00 'ㅑ' 0.00 'ㅐ' 0.00 'ㅔ' 0.00 'ㅁ' 0.00 'N' 0.07 'ㅇ' 0.67 'L' 0.86 'Hehe' 0.00 'ㅗ' 0.00 'ㅓ' 0.00 'ㅏ' 0.00 'ㅣ' 0.00 'C' (Control) 0.00 'Lol' 0.00 'T' 0.48 'ㅊ' 0.43 'ㅍ' 0.00 'ㅠ' 0.00 'TT' 0.00 'ㅡ' 0.00 'D' (Delete) 0.00 'S' (Space) 0.00

Such a key rate may be calculated by dividing the input key and the comparison key into the same case and different cases. FIG. 6 shows an example of a method of calculating the key rate in the case where a penalty is present in one embodiment of the present invention. 7 is a diagram showing an example of a method for calculating a key rate when there is no penalty in the embodiment of the present invention.

6 shows an example in which a key rate is calculated by giving a penalty when the input key and the comparison key are the same, and a key rate is calculated without giving a penalty when the input key and the comparison key are different. As shown in FIG. 6, when the input key and the comparison key are the same key (if the key rate is calculated for the character of the currently recognized key), the comparison key, which is the letter key, has a probability value of 0.7, and the control key / The comparison key, which is a delete key, can be calculated to have a probability value of 0.9, and the comparison key, which is a space key, has a relatively low probability value of 0.5. Meanwhile, as shown in FIG. 6, when the input key and the comparison key are different keys, the key rate of the comparison key may be calculated as in Equation 1 below.

Here,'standard' may mean the longest distance between the center of the input key and the comparison key, and'diff' may mean the distance from the center of the key (input key) corresponding to the normalized coordinates.

Meanwhile, FIG. 7 is an example of a method of calculating a key rate when there is no penalty, as described above, and the key rate may be calculated as shown in Equation 1 for all comparison keys. In other words, the method of FIG. 7 may be a method of calculating the key rate without penalty for the same comparison key as the input key.

Meanwhile, the data that the learning data can include may include at least one of the data described below.

(1) Key rate for normalized coordinates

Examples of calculating the key rate for the normalized coordinates have been described in detail through Table 5, FIG. 7 and Equation 1 above.

(2) Key rate with penalty

An example in which a penalty is given to have a low probability value for a letter corresponding to an input key has been described with reference to FIG. 6 above.

(3) Key ratio using corrected coordinates

While the above-described key rate uses normalized coordinates, the corrected coordinates (a/w+Revision x, b/) calculated by reflecting the average value of the correction values (Revision x, Revision y, hereinafter referred to as'Revision' value) to the normalized coordinates. The key rate calculated for h+Revision y) may be included in the training data.

(4) Key matrix

The key matrix may represent a value obtained by dividing the difference between the normalized coordinates and the center coordinates of each letter through the key divided into 9 zones.

8 is a diagram illustrating an example for describing a key matrix in an embodiment of the present invention. FIG. 8 shows an example in which one key is divided into k (k is a natural number) zones, and then an array of k bits corresponding to k zones is generated as a key matrix. At this time, in FIG. 8, k represents 9, and a bit corresponding to ③, an area corresponding to normalized coordinates, may be set to have a value of '1', and all bits corresponding to the remaining areas may be set to '0. It can be set to have a value of'. In this case, the deep learning model can measure the key-biased position of the user's key relative to each key using the key matrix.

(5) Normalized slope values (e.g., yaw, pitch, and roll values respectively)

The slope values can be normalized to have rational values between -1.0 and 1.0.

For example, yaw normalization may be performed through Equation 2 below, pitch normalization through Equation 3 below, and roll normalization through Equation 4 below.

(6) Input string one-hot encoding

One hot encoding of the input string can be obtained as an array of 30 bits including 29 bits for 29 characters and padding bit values for the virtual keyboard 410 described with reference to FIG. 4. At this time, the bit value may be achieved by setting a value of '1' for the bit corresponding to the input string, and setting a value of '0' for the remaining bits. Table 6 shows an example of one-hot encoding for the recognized character string'ㄱ', then the recognized character string '그리고', and then the recognized space.

key Bit value of recognized string'ㄱ' Bit value of recognized string'ㅗ' Bit value of recognized string'space' 'ㅂ' 0 0 0 'ㅈ' 0 0 0 'ㄷ' 0 0 0 'G' One 0 0 'ㅅ' 0 0 0 'ㅛ' 0 0 0 'ㅕ' 0 0 0 'ㅑ' 0 0 0 'ㅐ' 0 0 0 'ㅔ' 0 0 0 'ㅁ' 0 0 0 'N' 0 0 0 'ㅇ' 0 0 0 'L' 0 0 0 'Hehe' 0 0 0 'ㅗ' 0 One 0 'ㅓ' 0 0 0 'ㅏ' 0 0 0 'ㅣ' 0 0 0 'C' (Control) 0 0 0 'Lol' 0 0 0 'T' 0 0 0 'ㅊ' 0 0 0 'ㅍ' 0 0 0 'ㅠ' 0 0 0 'TT' 0 0 0 'ㅡ' 0 0 0 'D' (Delete) 0 0 0 'S' (Space) 0 0 One Padding 0 0 0

The deep learning model can grasp the order and type of user input through a list of one-hot encodings that training data can include. In this case, the deep learning model may be trained to predict and output the key rate (key rate of each of the 29 keys) of the string to appear using the previous n strings. As a more specific example, after dividing a specific sentence into space units, separating the divided word into alphabets, the probability of a character (or string) that the deep learning model will present using the preceding n from the separated alphabet (or string) You can train a deep learning model to output. In this case, the deep learning model calculates the probability value (key rate) of the current character (or character string) for each of the 29 keys using the first n characters (or character string), and calculates the currently entered character (or character string). After leaving only the probability value of the neighboring Jamo (or string) as the center, and changing the remaining probability values to 0, the total data can be divided by the total sum of the remaining values so that the total sum of the probability is 1.

9 is a diagram illustrating an example of peripheral keys for an input key in an embodiment of the present invention. FIG. 8 shows peripheral keys'ㄷ','ㄱ','ㄴ','ㄹ','ㅋ','ㅌ' and'ㅊ' of the peripheral keys of the input key'ㅇ' in the virtual keyboard 910.

On the other hand, Table 7 below shows an example in which the bit value of the input key'ㅇ' and the peripheral key of the input key'ㅇ' are mapped to 1.

key Bit value 'ㅂ' 0 'ㅈ' 0 'ㄷ' One 'G' One 'ㅅ' 0 'ㅛ' 0 'ㅕ' 0 'ㅑ' 0 'ㅐ' 0 'ㅔ' 0 'ㅁ' 0 'N' One 'ㅇ' One 'L' One 'Hehe' 0 'ㅗ' 0 'ㅓ' 0 'ㅏ' 0 'ㅣ' 0 'C' (Control) 0 'Lol' One 'T' One 'ㅊ' One 'ㅍ' 0 'ㅠ' 0 'TT' 0 'ㅡ' 0 'D' (Delete) 0 'S' (Space) 0 Padding 0

For example, in the deep learning model, when the keys having the largest key ratio are the same in the above-described "(3) key ratio using corrected coordinates" and "(1) key ratio for normalized coordinates" (key ratio without penalty) , The bit value of the surrounding keys may be mapped to 1 by using the corresponding key as an input key. FIG. 10 illustrates an example of calculating the probability of a next letter using three letters in an embodiment of the present invention. It is one drawing. In FIG. 10, the first dotted box 1010 shows an example in which the probability value (key rate) of the currently displayed character (or character string) is calculated for each of the 29 keys using n characters (or character strings). In addition, in FIG. 10, the second dotted box 1020 is performed, and only the probability values of neighboring children (or character strings) are centered around the currently entered character (or character string), and the remaining probability values are changed to 0, and then the total sum is Shows an example of the probability value of each of the currently input characters and adjacent characters obtained by dividing the entire data by the average of the values that are currently left to be 1.

As described above, according to embodiments of the present invention, as the data is normalized to use for correcting a typo, it is possible to correct a keyboard typo without a difference in performance for a virtual keyboard environment of various smartphones and/or various users. In addition, the pre-trained deep learning model can be applied directly and used for correction of keyboard errors without additional data collection.

The system or device described above may be implemented as a hardware component, or a combination of hardware components and software components. For example, the devices and components described in the embodiments include, for example, processors, controllers, arithmetic logic units (ALUs), digital signal processors (micro signal processors), microcomputers, field programmable gate arrays (FPGAs). , A programmable logic unit (PLU), microprocessor, or any other device capable of executing and responding to instructions, may be implemented using one or more general purpose computers or special purpose computers. The processing device may perform an operating system (OS) and one or more software applications running on the operating system. Further, the processing device may access, store, manipulate, process, and generate data in response to the execution of the software. For convenience of understanding, a processing device may be described as one being used, but a person having ordinary skill in the art, the processing device may include a plurality of processing elements and/or a plurality of types of processing elements. It can be seen that may include. For example, the processing device may include a plurality of processors or a processor and a controller. In addition, other processing configurations, such as parallel processors, are possible.

The software may include a computer program, code, instruction, or a combination of one or more of these, and configure the processing device to operate as desired, or process independently or collectively You can command the device. Software and/or data may be interpreted by a processing device, or to provide instructions or data to a processing device, of any type of machine, component, physical device, virtual equipment, computer storage medium or device. Can be embodied in The software may be distributed on networked computer systems and stored or executed in a distributed manner. Software and data may be stored on one or more computer-readable recording media.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, or the like alone or in combination. The medium may be a computer that continuously stores executable programs or may be temporarily stored for execution or download. In addition, the medium may be various recording means or storage means in the form of a combination of single or several hardware, and is not limited to a medium directly connected to a computer system, but may be distributed on a network. Examples of the medium include magnetic media such as hard disks, floppy disks and magnetic tapes, optical recording media such as CD-ROMs and DVDs, and magneto-optical media such as floptical disks, And program instructions including ROM, RAM, flash memory, and the like. In addition, examples of other media include an application store for distributing applications, a site for distributing or distributing various software, and a recording medium or storage medium managed by a server. Examples of program instructions include high-level language code that can be executed by a computer using an interpreter, etc., as well as machine language codes produced by a compiler.

As described above, although the embodiments have been described by a limited embodiment and drawings, those skilled in the art can make various modifications and variations from the above description. For example, the described techniques are performed in a different order than the described method, and/or the components of the described system, structure, device, circuit, etc. are combined or combined in a different form from the described method, or other components Alternatively, proper results can be achieved even if replaced or substituted by equivalents.

Therefore, other implementations, other embodiments, and equivalents to the claims are also within the scope of the following claims.

Claims (20)

A method for correcting keyboard typos in a computer device including at least one processor, the method comprising:
Collecting, by the at least one processor, a user log associated with user input to the virtual keyboard;
Generating learning data for a deep learning model by processing the collected user log by the at least one processor;
Training, by the at least one processor, the deep learning model so that the deep learning model outputs a probability value for each of two or more keys included in the virtual keyboard for any key input; And
Determining, by the at least one processor, a grammatical error according to a user input based on a probability value output from the learned deep learning model
Keyboard typo correction method comprising a. According to claim 1,
The step of learning,
The deep learning model is a probability value for the next key input based on the previous n (the n is a natural number) key input included in the learning data, as the two or more keys, corresponding to the next key input in the virtual keyboard And learning the deep learning model to derive for each of the keys and the peripheral keys of the corresponding keys. According to claim 1,
The step of generating the learning data,
Calculating a probability value for coordinates normalized to the virtual keyboard for each of the m keys (where m is a natural number) preset in the virtual keyboard
Keyboard typo correction method comprising a. According to claim 1,
The step of generating the learning data,
Computing the probability value for the normalized coordinates for the virtual keyboard for each of the m (the m is a natural number) keys preset in the virtual keyboard, calculating a probability value so that the key corresponding to the normalized coordinates has a penalty
Keyboard typo correction method comprising a. According to claim 1,
The step of generating the learning data,
Computing a probability value for the corrected coordinates corrected for the normalized coordinates for the virtual keyboard for each of the preset m (where m is a natural number) keys in the virtual keyboard.
Including,
The corrected coordinates are corrected using an average value of difference values for each consonant and vowel,
The difference value comprises a difference value between the coordinates recognized by the user input and the center coordinates of the key recognized by the user input. According to claim 1,
The step of generating the learning data,
For a key divided into k (the k is a natural number) zones, an array of k bits corresponding to k zones is generated as a key matrix, but the bits of the zone corresponding to the normalized coordinates are set to '1'. , Creating an array of bits with the remaining zone bits set to '0'
Keyboard typo correction method comprising a. The method according to any one of claims 3 to 6,
The normalized coordinates include a normalized coordinates based on the width and height of the virtual keyboard based on the coordinates recognized through the user input. According to claim 1,
The step of generating the learning data,
Normalizing a slope value of the computer device output through a sensor included in the computer device
Keyboard typo correction method comprising a. According to claim 1,
The step of generating the learning data,
A bit for each of the m (the m is a natural number) keys preset in the virtual keyboard for one recognized string is included, but the bit for the key corresponding to the recognized string is set to '1', and the rest Creating an array in which bits are all set to '0'
Keyboard typo correction method comprising a. The method of claim 9,
The determining of grammatical errors according to the user input may include:
A key corresponding to the currently recognized coordinates is determined based on a value obtained by dividing all data by an average of probability values calculated for the first key corresponding to the currently recognized coordinates and the adjacent keys of the first key, and the first key and And comparing the determined key to determine whether or not the grammatical error or correcting a typo by changing the first key to the key. According to claim 1,
The step of training the deep learning model using the learning data is,
Each of the plurality of deep learning models are trained by inputting the learning data into a plurality of deep learning models,
The determining of grammatical errors according to the user input may include:
A method for correcting a keyboard typographical error, characterized in that a grammatical error according to the user input is determined using a final result obtained by synthesizing the results output from each of the plurality of deep learning models. A computer program stored in a computer-readable recording medium in combination with a computer device for executing the method of any one of claims 1 to 6 or 8 to 11 on a computer device. A computer-readable recording medium on which a computer program for executing the method of any one of claims 1 to 6 or 8 to 11 is recorded on a computer device. In the computer device,
At least one processor implemented to execute readable instructions on the computer device
Including,
By the at least one processor,
Collect user logs associated with user input to the virtual keyboard,
Processing the collected user log to generate training data for a deep learning model,
Using the training data, the deep learning model trains the deep learning model to output a probability value for each of two or more keys included in the virtual keyboard for any key input,
Determining whether there is a grammatical error according to the user input based on the probability value output from the learned deep learning model
Computer device characterized in that. The method of claim 14,
By the at least one processor,
The deep learning model is a probability value for the next key input based on the previous n (the n is a natural number) key input included in the learning data, as the two or more keys, corresponding to the next key input in the virtual keyboard And learning the deep learning model to derive for each key and a peripheral key of the corresponding key. The method of claim 14,
By the at least one processor,
Calculating the probability value for the coordinates normalized to the virtual keyboard for each of the m keys (where m is a natural number) preset in the virtual keyboard
Computer device characterized in that. The method of claim 14,
By the at least one processor,
Calculating the probability value for the normalized coordinates for the virtual keyboard for each of the m keys (where m is a natural number) preset in the virtual keyboard, but calculating the probability value so that the key corresponding to the normalized coordinate has a penalty.
Computer device characterized in that. The method of claim 14,
By the at least one processor,
The probability value for the corrected coordinates corrected for the normalized coordinates for the virtual keyboard is calculated for each of the preset m keys (where m is a natural number) in the virtual keyboard,
The corrected coordinates are corrected using an average value of difference values for each consonant and vowel,
The difference value includes a difference value between the coordinates recognized by the user input and the center coordinates of the key recognized by the user input.
Computer device characterized in that. The method of claim 14,
By the at least one processor,
A bit for each of the m (the m is a natural number) keys preset in the virtual keyboard for one recognized string is included, but the bit for the key corresponding to the recognized string is set to '1', and the rest Create an array where the bits are all set to '0'
Computer device characterized in that. The method of claim 19,
By the at least one processor,
A key corresponding to the currently recognized coordinates is determined based on a value obtained by dividing all data by an average of probability values calculated for the first key corresponding to the currently recognized coordinates and the adjacent keys of the first key, and the first key and Correcting a typo by comparing the determined key to determine whether or not the grammatical error or changing the first key to the key
Computer device characterized in that.

Images