KR101112624B1 - Character Recognizing method and the system with 3D Accelerometer - Google Patents

Abstract

The present invention relates to a character recognition method and system for recognizing and inputting characters to various terminals by recognizing a writing operation in a space using an acceleration sensor, and more particularly, a signal detected through an acceleration sensor. The present invention relates to a method and system for improved character recognition through preprocessing and pattern recognition.

The present invention has the effect of improving the recognition rate through the amplification, preprocessing, pattern recognition process of the acceleration sensor detection signal during space writing. In other words, the recognition rate is increased by adding and supplementing some preprocessing steps before applying it to the algorithm, and increasing the recognition rate by complementing the algorithm.

Description

Character Recognition Method using Acceleration Sensing and Character Recognition System Using Them {Character Recognizing method and the system with 3D Accelerometer}

The present invention relates to a character recognition method and system for recognizing and inputting characters to various terminals by recognizing a writing operation in a space using an acceleration sensor, and more particularly, a signal detected through an acceleration sensor. The present invention relates to a method and system for improved character recognition through preprocessing and pattern recognition.

Since ancient times, keyboards and mice have been generally used as a means for a person to input information into a terminal such as a computer. In addition, a mobile device such as a mobile phone is also provided with a button corresponding to the keyboard is used to enter information such as letters, numbers.

Such an input method has a standard input means such as a keyboard and a button, and it is necessary to be aware of the rules and rules promised with the computer, and there are some barriers in the free interface with the computer. In recent years, technology for recognizing human handwriting has been proposed, and technology for inputting characters by directly recognizing human handwriting has been commercialized in some PDA terminals or touch pads. However, these touch pad products are inconvenient in that they are not scalable to existing products and most of all, they must be in direct contact with the touch pad.

Therefore, a device that can be universally extended and applied to existing products and can be recognized even in the air, that is, by writing a letter in a three-dimensional space, without having to directly touch a screen or a touch pad, has been proposed.

Three types of methods for detecting motion in three-dimensional space have been attempted. First, there is a method of photographing a moving object using a camera and interpreting the motion through image processing. This method requires a large number of cameras, the user must use an object having a special shape or color for image processing, and the processing is complicated. Second, there is a method using the principle of triangulation. This method allows the user device to transmit or reflect radio waves or sound waves, and based on this, three or more receivers measure the reception time and track the position of the object based on the difference in arrival time. This method also has a problem in that it is necessary to install a plurality of transmitting or receiving devices at a predetermined position and it is very difficult to detect fine movements. Third, there is a method of directly attaching an inertial sensor to a user or a user device and directly obtaining the movement. This method uses three axes of translational motion and three axes of rotational motion to detect the motion of an object in three-dimensional space. Translational and rotational motions can be detected using inertial sensors such as acceleration sensors.

Among them, the present invention relates to a method of using an acceleration sensor, so that information on how the device moves is sensed through an acceleration sensor provided at an end even when writing in the air through a mobile device. In other words, it uses a signal of how much force the end of the device is moved in three directions (X, Y, Z axis), and recognizes letters by movement without physical contact. Attempts to detect the user's movement in the three-dimensional space and input text can extract the trajectory of the user's movement as accurately as possible to obtain a stroke like the user's writing on paper, and a separate character recognition algorithm for this. It was a way to apply them.

This character recognition method has a problem that the recognition rate is lowered due to noise, shaking, etc. of the signal from the sensor. Therefore, there is a need for a method for improving the recognition rate.

The present invention is to solve the above problems, it is an object to improve the recognition rate through the amplification, pre-processing, pattern recognition process of the acceleration sensor detection signal during writing.

The configuration of the present invention for achieving the above object, the step of measuring the acceleration through the acceleration sensor for the operation to write to the virtual plane while the input signal is applied through the input unit; Modulating the sensed signal of the measured acceleration into an amplified and / or digital signal; Preprocessing the modulated digital signal; And pattern recognition using the preprocessed digital signal.

The preprocessing step may include a filtering step, wherein the filtering step includes: removing irregular and non-uniform noise signal waveforms included in the signal and correcting the uniform and continuous signal waveforms.

The preprocessing step may include: a normalizing step, wherein the normalizing step is characterized in that when the amplitude of the signal input by each user is large or small, the normalizing is corrected to a standardized amplitude.

The preprocessing step may include: a regulating step, wherein the regulating step may include: correcting at a standardized speed when the speed of the signal input for each user is fast or slow.

The pattern recognition may include: comparing two signals by stretching and reducing the signal in time and matching the preprocessed signal.

An acceleration sensor for detecting movement; An amplifier for amplifying the signal detected by the acceleration sensor; A modulator for modulating the analog signal into a digital signal; A preprocessing unit performing the one or more preprocessing steps; And a pattern recognition unit for comparing the preprocessed signal with a character through the pattern recognition step to recognize the preprocessed signal.

The present invention has the effect of improving the recognition rate through the amplification, preprocessing, pattern recognition process of the acceleration sensor detection signal during space writing. In other words, the recognition rate is increased by adding and supplementing some preprocessing steps before applying it to the algorithm, and increasing the recognition rate by complementing the algorithm.

Hereinafter, the configuration of the present invention will be described in detail with reference to the embodiments illustrated in the accompanying drawings.

1 is a block diagram of a handwriting input device and a recognition terminal according to the present invention. Referring to FIG. 1, a text recognition system is configured by including the writing input device 100 and the recognition terminal 200. The handwriting input device 100 is a device that is held by a user and moves in the air, and the recognition terminal 200 may correspond to a PC, a PDA, and various other information communication terminals and application devices.

The writing input device 100 includes a three-axis acceleration sensor 110 at the end to detect the movement, the mode conversion unit 120, the input unit 130, the amplifier 140, the modulator 150, The preprocessor 160 and the communication unit 170 are included.

The acceleration sensor 110 is an electromechanical transducer that generates an electrical output when subjected to mechanical shock or vibration, and is widely used for measuring shock and vibration as well as general purpose acceleration.

The acceleration sensor 110 corresponds to a kind of inertial sensor, and may be used as an angular velocity sensor inertial sensor that measures angular velocity in order to calculate a change in rotation angle of an object in addition to an acceleration sensor that measures acceleration in order to calculate a change in position of an object. have.

The acceleration sensor 110 causes a large difference between the calculated position and the actual position due to errors or noise inherent in the acceleration sensor in the process of integrating the measured acceleration with time.

The mode conversion unit 120 is a part for allowing a user to select whether the character to be space-written is English, Korean, or numbers.

The input unit 130 is a part for designating the start and end of the letter writing to be displayed when the user performs the space writing, and does not press the write button of the input unit 130 when moving in the air without any meaning. The write button of the moment input unit 130 of the movement to input the character is turned on, and the write button is turned off the moment the writing of the character is completed to stop the signal detection and recognition.

The amplifying unit 140 amplifies the coordinate values on the x, y, z axes from the acceleration sensor 110 through the OP-Amp to a value having a maximum size range that can be expressed. The amplified analog signal value is converted into a digital value through the modulator 150, and is transmitted to the recognition terminal 200 wirelessly through a communication unit 170 such as a Bluetooth module after processing such as filtering in the preprocessor 160. do.

The recognition terminal 200 includes a pattern recognition unit 220, and in some cases, the preprocessing unit 210 may be included in the recognition terminal.

The acceleration intensity value sensed by the acceleration sensor 110 is transmitted to the analog voltage value through the output terminal, which is amplified through the OP-Amp amplifier 140 and transferred to the modulator 150 (AVR ADC), the specified sampling The digital signal is extracted from the value and transmitted to the recognition terminal 200 through the communication unit 170. The preprocessor 160 may be in the writing input device 100 or 210 or may be provided in the recognition terminal 200. At this time, the transmitted value is only the signal value until the write button is pressed and released from the input unit 130. The signal is transmitted by comparing the transmitted signal value with the pre-stored signal value when the character is written. Similar characters are found.

The signal value transmitted wirelessly through the communication unit 170 such as Bluetooth is mixed by noise due to the slight shaking of the human hand or the influence of the external environment, and the size or writing speed of each person is different, so the input signal is processed before the pattern processing. Increase the preprocessing (filtering, normalizing, regulating) to ensure that the value of is set to an accurate and constant value. The preprocessed data is recognized by the Dynamic Time Warping (DTW) algorithm and its value is shown in the user interface.

2 is a flowchart illustrating a method of recognizing a writing operation. Referring to FIG. 2, when a user writes letters in space using the writing input device 100 recognizing space writing (S110), the acceleration is accelerated by the acceleration sensor 110 mounted in the writing input device 100. Is measured (S120). Then, the user's handwriting is recognized by tracking the handwriting trajectory, that is, the movement trajectory in which the end of the handwriting input device 100 moves in space by using the measured acceleration, and recognizing the shape pattern (S160). As described above with reference to FIG. 1, the writing input apparatus 100 may further include a mode selection step of selecting a character type through the mode converter 120 before writing the text in the space.

In this process, in order to reduce the noise or error of the sensor device and increase the recognition rate, the sensing signal is amplified (S130), modulated into a digital signal (S140), and the preprocessing process is performed (S150). As shown in FIG. 3, the preprocessing process includes any one or more of a filtering process S152, a normalizing process S154, and a regulating process S156.

4 is a graph illustrating a result of a filtering process in a preprocessing process. Referring to FIG. 4, a signal having a waveform like A is smoothly regularized and corrected as shown by B by the filtering process. In order to go through this filtering process, the filtering of the detection signal is performed by the following process and function.

int precedure_filtering (int raw_data_count, coordinate * input_data, processing_data * result_pos);

This function takes as input parameters the number of input data (raw_data_count), an input data structure (coordinate * input_data), and an output data structure (processing_data * result_pos). After receiving x, y, z coordinate value through coordinate structure, calling fun_filter () function to process LowpassFilter and saving the result value in processing_data structure. Is the source.

// structure to receive input data

struct coordinate

{

short x;

short y;

short z;

short reserved;

};

// structure to store the result of calculation of filter, leveling and normalization

typedef struct processing_data

{

double x;

double y;

double z;

} processing_data;

for (i = 0; i <sampling; i ++)

{

data [i] = pos [i] .x; // pos [i] .x is the input value of the coordinate structure

}

// filter the x coordinate

// data: input data, result: output data, raw_data_count: number of input data

fun_filter (data, result, raw_data_count);

// save the filtered x coordinate

for (i = 0; i <sampling; i ++)

outdata [i] .x = result [i]; // oudata.x is a processing_data structure

(skip)

When the filter processing for x, y, and z coordinates is completed, the precedure_filtering () function returns the number of data that has been filtered. Since the basic filter has a delay of (filter factor -1) / 2, the front and rear parts of the result value after the filter processing are not as accurate as the delayed number. Returns the value subtracted by the number of coefficients.

// number of filtered data return

return (sampling-FILTER_LENGTH);

void fun_filter (int * data, double * result, int count);

This function takes as input parameters an input data array, a result data array, and a count of input data. Here is a function to process Low Pass Filter using convolution.

#define FILTER_LENGTH 21 // The number of coefficients in the filter

// filter coefficients

double Filter [FILTER_LENGTH] = {

  0.03283474219465, (short)

};

For basic filter processing, the input value in the time domain is transformed into the frequency domain through Fourier transform and multiplied by the filter coefficient. After the inverse Fourier transform, the result of the filtering process can be obtained. However, if you use the Convolution equation as below, it performs the same function as the product in the frequency domain without going through this process.

........Convolution 식

........ Convolution expression

In other words, you can multiply two functions and sum the products.

The filter processing source using the convolution equation is as follows.

// Convolution by filter coefficient.

for (i = 0; i <sampling; i ++)

{

temp = 0;

if (i <FILTER_LENGTH)

{

input [i] = data [i];

for (j = 0; j <= i; j ++)

temp + = input [i-j] * Filter [j]; // Convolution

}

else

{

for (j = 0; j <FILTER_LENGTH-1; j ++)

input [j] = input [j + 1];

input [FILTER_LENGTH-1] = data [i];

for (j = 0; j <FILTER_LENGTH; j ++)

temp + = input [j] * Filter [j];

}

result [i] = temp;

}

After the convolution is over, there is a delay of FILTER_LENGTH at the beginning of the result array. Shift the value from (FILTER_LENGTH + 1) forward because it has to have the correct signal value. Afterwards, the latter part is duplicated by FILTER_LENGTH, so the precedure_filtering () function returns the number of data after input processing by FILTER_LENGTH.

// Shift by FILTER_LENGTH to remove the delay section earlier.

j = FILTER_LENGTH;

for (i = j; i <sampling; i ++)

{

result [i-j] = result [i];

}

Through the application of the small source and the function expression as described above, the signal value containing a lot of small noise can be filtered smoothly as shown in B of FIG.

5 is a graph showing the results of the normalizing process during the pretreatment process. Referring to FIG. 5, a normalizing process for normalizing the amplitude is necessary because the sensing amplitude when writing is different for each person.

There may be a person who writes so that the amplitude of the acceleration sensor is sensed as small as C of FIG. In normalizing, large or small amplitudes are normalized to constant amplitude E.

To normalize different amplitude values for different people, divide the input value by RMS using the root meas-square (RMS) to obtain an appropriate amplified generalized value. The source and function for this are as follows.

int normalization (int cnt_filtered_data, processing_data * data);

This function takes as input parameters the number of input data (cnt_filtered_data) and the input data structure (processing_data * data). To process data values received through parameters through P_rms () function, each of three coordinates of x, y, z is stored in an array and then P_rms () function is called.

// Receive x, y, z coordinate value and save it for normalization.

for (i = 0; i <sampling; i ++)

{

x [i] = (data + i)-> x;

(skip)

}

// Call the P_rms function to find the root means square.

P_rms (x, cnt_filtered_data);

(skip)

Finally, since the result value of P_rms () function is less than 1, it is amplified to have more than 300 again.

// amplify normalized data

for (i = 0; i <sampling; i ++)

{

(data + i)-> x = x [i] * 300;

(skip)

}

void P_rms (double * input, int cnt_filtered_data);

This function handles RMS. It takes an input data array (input) and the number of input data (cnt_filtered_data) as parameters.

RMS is the arithmetic mean, which is the sum of the squares of each sample, divided by the total number of samples.

n: number of samples, N: total number of samples, A [s]: nth acceleration sample

The following is the part that calculates RMS.

// calculate sum of square value

for (i = 0; i <sampling; i ++)

{

sum_pow = sum_pow + pow (input [i], 2);

}

// RMS calculation P = √ [(∑A [n] ^ 2) / N

rms_val = sqrt (sum_pow / sampling);

// divide each input data by RMS

// normalize, or equalize

// normalization

for (i = 0; i <sampling; i ++)

input [i] = input [i] / rms_val;

Through the application of the sources and functions as described above, the received values of different amplitudes can be corrected to standardized values as shown in E of FIG. 5.

6 is a graph showing the result of the regulating process during the pretreatment. Referring to FIG. 6, in order to correct the point that the writing speed is different for each person, a process of adjusting the speed as H, which is a normalized speed, when writing a letter fast as F or writing a letter late as G to be. If the writing speed is different, the signal length will be different, so you need to match the signal length before pattern recognition.

As the matching method, interpolation and under-sampling methods are used. The principles, sources, and functions for doing this are described below. If you need to match a small data set to 1000 at the time of interpolation, divide the length of the data by 1000 and you will get a mistake. This is called Rate.

Number of data sets Rate 500 2 250 4 126 7.936508

So the data on the original source is relocated at intervals separated by the rate. If the Input Data Set was 500, the Rate would be 2.0. If the first element of the input was at index [0], the second element of Input would be at index [2] and the third element of Input would be at index [4]. Under-Sampling defaults to the minimum acceptable sample rate.

int regulation (int count, processing_data * data, processing_data * final_data);

  This function takes as parameters the number of input signals (count), an input data structure (processing_data * data), and an output data structure (processing_data * final_data). In this case, the rate value is calculated and matched using the value. The lower part is an implementation unit that calculates the rate value.

First, we allocate a dynamic array to calculate the intermediate process. Allocate an output array the length you want to match.

output = (processing_data *) malloc (sizeof (processing_data) * LENGTH);

Initialize the array.

for (i = 0; i <LENGTH; i ++)

{

output [i] .x = 0;

(skip)

}

// Rate calculation: LENGTH / number of input data

// cast both LENGTH and number_of_regulated_data to dobule for decimal calculation

// where number_of_regulated is the number of input data

rate = (double) LENGTH / (double) number_of_regulated_data;

Once the rate is calculated, the start and end points are matched.

output [0] = data [0];

output [LENGTH-1] = data [number_of_regulated_data-1];

Then save the input data in each index according to the calculated rate value. If the output array is 200, that is, when the input data is larger than 200 when the input data is set to the ratio of 200 data to the output data, the undersampling is performed, and when the output data is less than 200, the data is arrayed using interpolation. Then the middle half is filled with data initialized to zero.

for (i = 0; i <number_of_regulated_data; i ++)

{

output [(int (i * rate))]. x = data [i] .x;

(skip)

}

Now we need to initialize the part initialized to 0. After checking whether the current part needs interpolation, call linear_interpol () function to interpolate. The implementation part is as follows.

  before_index examines how old the input data (that is, the non-zero data) is in the data that needs to be interpolated, and after_index examines how long the input data is in the data that needs to be interpolated.

 // get before, after_index

while (1)

{

if (output [before_index] .x! = 0)

break;

before_index--;

}

// Once the index is found, we fill x, y, z.

while (1)

{

if (output [after_index] .x! = 0)

break;

after_index ++;

}

before_value_x = output [before_index] .x;

after_value_x = output [after_index] .x;

(skip)

// Fill in the part that needs to be interpolated with linear_interpol.

output [i] .x =

linear_interpol (before_index, after_index, before_value_x, after_value_x, i);

(skip)

double linear_interpol (int x1, int x2, double y1, double y2, double x);

This function takes as parameters the x and y coordinates of the interval to be interpolated. Interpolation is performed by linear interpolation.

If the equation is a straight line, a straight line passing through two points (x0, f (x0)) and (x1, f (x1)) can be obtained.

(Interpolation formula)

In general, the smaller the interval between data points, the smaller the interval, so it is better to approximate the continuous function in a straight line, so a good approximation can be obtained.

Below is the source to find the length between two points.

result = (y2-y1) / (x2-x1) * (x-x1) + y1;

Through the application of sources and functions as described above, received values of different speeds can be corrected to standardized standard speeds as shown in FIG.

7 and 8 are graphs for explaining dynamic pattern recognition. The signal that has undergone the above preprocessing is matched with characters by dynamic pattern recognition as a dynamic signal.

Patterns that are subject to pattern recognition can be divided into static and dynamic patterns. In general, fixed images such as fingerprints, irises, veins, handwritten numbers, and letters correspond to static patterns, and voices, stock price flows, and robot motion trajectories correspond to dynamic patterns. The input signal of the acceleration sensor, which is an input signal of the present invention, belongs to a dynamic pattern.

The most intuitive way to think about pattern recognition is pattern matching. However, dynamic patterns must be pattern matched in a way that allows them to stretch and shrink in time.

To this end, DTW (Dynamic Time Wraping) algorithm is used, which enables pattern matching for two patterns of different lengths. The DTW algorithm is a pattern matching algorithm that allows nonlinear stretching, ie, stretching and shrinking, on the time axis. Here, nonlinear stretching means that one column must be increased or decreased in order to compare two columns based on one column in two columns of different lengths.

If long A column is compared based on short B column, the mapping function is used. The DTW algorithm first sets the distance measure value for each component in the two rows as cost. It is an algorithm that compares two columns while searching for a mapping function by using an ignition equation that uses a table of two columns and stores the minimum cost cyclically and stores them in the cost table starting from the start component of each column to the end component. .

Finally, the value stored in the table at the end component is the similarity for the two columns. On the other hand, the trajectory of the mapping function saves the path that takes the minimum cost in each step in a separate path table during the search process such as finding the optimal search path of the previous dynamic programming method, and traces back the final minimum cost from the end component. (Backtracking) to find.

The DTW algorithm is a simple pattern matching algorithm for vector strings of different lengths using a problem solving algorithm called dynamic programming. In comparing two signals, the DTW algorithm measures the Euclidean distance between two signals and finds the similarity, or pattern, between the two signals through the sum of the distances.

When comparing signals between D1 data and D2 data as shown in FIG. 7, even if the signals have similar characteristics, the signals may not be mapped to each other. However, through the mapping by the DTW algorithm as shown in FIG. 8, since mapping can be performed while searching for similar features, the two signals can be mapped to each other if they have similar characteristics even though they are at slightly different times.

The DTW algorithm measures the distance value of two signals, and if the distance value is close, determines that it is a similar signal and matches the signal. In order to improve the DTW algorithm itself, in the case of Korean or English, the user inputs a mechanism to correct the spelling or inputs a dictionary database to recognize which letters the user is trying to write. Not only does it display a character or simply select the value closest to the distance from the character, but also displays candidates for secondary selection and correction.

9 is a diagram illustrating an embodiment of a stroke order for character recognition. Referring to FIG. 9, the signals input according to the pre-specified stroke order corresponding to the numbers (a), English (b), and Korean (c) are matched with each character through a preprocessing process and a pattern recognition process using a DTW algorithm. Done.

In terms of the accelerometer, the letters are clearly different from each other, but the letters that give power in the same direction are often matched with each other. This can improve the recognition rate.

In case of character recognition through accelerometer, the recognition rate is not high. However, in addition to filtering, the recognition rate is over 85% as a result of supplementing and modifying to recognize the signal with very low frequency through several preprocessing and using Dynamic Time Warping algorithm. Can be seen. This high recognition rate replaces button terminals or touch pad terminals, thereby enabling the development of non-contact terminals equipped with acceleration sensors.

The present invention described above is capable of various substitutions, modifications, and changes without departing from the spirit of the present invention for those skilled in the art to which the present invention pertains. It is not limited to drawing.

1 is a block diagram of a handwriting input device and a recognition terminal according to the present invention.

2 is a flowchart illustrating a method of recognizing a writing operation.

3 is a flow chart illustrating the pretreatment process in detail.

4 is a graph illustrating a result of a filtering process in a preprocessing process.

5 is a graph showing the results of the normalizing process during the pretreatment process.

6 is a graph showing the result of the regulating process during the pretreatment.

7 and 8 are graphs for explaining dynamic pattern recognition.

9 is a diagram illustrating an embodiment of a stroke order for character recognition.

<Description of Symbols for Main Parts of Drawings>

100: handwriting input device 110: acceleration sensor unit

120: mode conversion unit 130: input unit

140: amplification unit 150: modulator

160: preprocessing unit 170: communication unit

200: recognition terminal 210: preprocessing unit

220: pattern recognition unit

Claims (6)

Receiving a type of character from a mode conversion unit; Measuring an acceleration through an acceleration sensor for writing on a virtual plane while an input signal is applied through the input unit; Amplifying and modulating the sensed signal of the measured acceleration into a digital signal; A filtering step of removing irregular and non-uniform noise signal waveforms included in the signal with respect to the modulated digital signal and correcting them to a uniform and continuous signal waveform; and if the amplitude of the signal input for each user is large or small, A pre-processing step including a normalizing step of leveling and correcting to a standardized amplitude, and a step of adjusting at a standardized speed when the speed of the signal input for each user is fast or slow; And And a pattern recognition step of recognizing a pattern by comparing the reference signal with the reference signal when the preprocessed digital signal is matched with a reference signal according to the selected mode. Character recognition method using sensing. delete delete delete delete A mode converter which receives a type of a character; An input unit including a light button for designating a start and an end of the writing; An acceleration sensor for detecting movement; An amplifier for amplifying the analog signal detected by the acceleration sensor while the light button is pressed; A modulator for modulating the amplified analog signal into a digital signal; A preprocessor performs filtering to remove non-uniform signals included in the digital signal, normalizing to equalize the size of the digital signal, and regulating operations to correct the speed of the digital signal according to the movement speed to a standardized speed. ; And Acceleration sensing characterized in that it comprises a; pattern recognition unit for recognizing characters by comparing the stretched digital signal and the reference signal according to the selected mode while performing the stretching process to increase and decrease the pre-processed digital signal in time; Character recognition system using.

Images