KR20240003779A - Method And Apparatus for Analyzing Live Stroke by using ML for hand writing pen - Google Patents

Abstract

Disclosed is a real-time stroke analysis method and device using machine learning for a writing pen.
In this embodiment, the values obtained using the IMU (Inertial Measurement Unit) sensor including the acceleration sensor and gyro sensor of the electronic pen and the OID (Optical Identification) sensor are used as ANN (Artificial Neural Network). Provides a real-time stroke analysis method and device using machine learning for a writing pen that classifies the strokes input for writing based on the results output by substituting into.

Description

Real-time stroke analysis method and device using machine learning for writing pen {Method And Apparatus for Analyzing Live Stroke by using ML for hand writing pen}

One embodiment of the present invention relates to a real-time stroke analysis method and device using machine learning for a writing pen.

The content described below simply provides background information related to this embodiment and does not constitute prior art.

In general, electronic devices such as computers or mobile communication terminals provide a keyboard input method that matches letters and keyboards for character input. However, with the advent of input devices such as touch pads, a handwriting-type input method in which users input characters by writing them directly is being proposed.

The handwriting input method is a method in which the user directly writes handwriting using an input device such as a touch pad and a handwriting recognition device recognizes it to input characters.

In the general handwriting input method, when characters are written with a tool such as a pen or by hand, the handwriting recognition device recognizes the character that is determined to be closest to the input handwriting as the entered character. Because each person who inputs handwriting has a slightly different handwriting, handwriting recognition devices often recognize the input handwriting as characters different from the user's intent.

Recently, in order for handwriting recognition devices to accurately recognize the word intended by the user, the handwriting recognition device recommends words similar to the initially recognized word, and is increasing the handwriting recognition rate by finding the intended word from the recommended similar words.

The above-mentioned method does not recommend words similar to the user's handwriting, but rather recommends words similar to words that the handwriting recognition device recognizes as the user's handwriting, so the handwriting recognition device effectively determines the user's intention. There is a problem of not being able to find a matching word.

Therefore, in order to accurately recognize words using a writing pen, real-time stroke analysis technology using machine learning is needed.

In this embodiment, the values obtained using an IMU (Inertial Measurement Unit) sensor including an acceleration sensor and a gyro sensor of an electronic pen and an OID (Optical Identification) sensor are used as an ANN (Artificial Neural Network). The purpose is to provide a real-time stroke analysis method and device using machine learning for a writing pen that classifies the strokes input for writing based on the output results.

According to one aspect of this embodiment, in the handwriting recognition improvement device, acceleration a ( ), angular velocity ω( ) After obtaining the acceleration a ( ), the angular velocity ω( ) A process of inputting parameters based on a first ANN (Artificial Neural Network); In the first ANN, the acceleration a ( ), the angular velocity ω( ) A process of outputting a first learning result of learning parameters based on ; A process of obtaining coordinate values (OID 2D position p (x, y)) from the electronic pen in the handwriting recognition improvement device and inputting them into a second ANN; In the second ANN, the acceleration a ( ), the angular velocity ω( A process of outputting a second learning result by learning a parameter based on ) and the coordinate value (OID 2D position p (x, y)); A real-time stroke analysis method comprising: recognizing handwriting input from the electronic pen and classifying strokes in the handwriting recognition improvement device based on the first learning result and the second learning result. provides.

As described above, according to this embodiment, values obtained using an IMU (Inertial Measurement Unit) sensor including an acceleration sensor and a gyro sensor of the electronic pen and an OID (Optical Identification) sensor It has the effect of being able to classify strokes input for writing based on the results output by substituting them into ANN (Artificial Neural Network).

According to this embodiment, there is an effect of recognizing bad handwriting or correcting letters with a low recognition rate by recognizing handwriting using an electronic pen.

1 is a diagram showing the configuration of an electronic pen according to this embodiment.
Figure 2 is a diagram showing the operation of the electronic pen according to this embodiment.
Figure 3 is a diagram showing a real-time stroke analysis system using machine learning for a writing pen according to this embodiment.
Figure 4 is a diagram for explaining a stroke classification method using ANN according to this embodiment.
Figure 5 is a diagram for explaining a stroke classification method reflecting handwriting recognition results according to this embodiment.

Hereinafter, this embodiment will be described in detail with reference to the attached drawings.

1 is a diagram showing the configuration of an electronic pen according to this embodiment.

The electronic pen 110 according to this embodiment includes an IMU sensor 120, a gyro sensor 112, an acceleration sensor 114, and an OID sensor 116. Components included in the electronic pen 110 are not necessarily limited to this.

The electronic pen 110 charges the battery provided inside with USB power. The electronic pen 110 communicates with an external device using Bluetooth.

The IMU sensor 120 is a concept that includes a gyro sensor 112 and an acceleration sensor 114. The IMU sensor 120 is an inertial measurement unit sensor and refers to a 6-axis sensor including 3 axes of gyro and 3 axes of acceleration.

The gyro sensor 112 generates an angular velocity measurement value by sensing the angular velocity, which is the rotational speed of the tip of the pen when the tip moves for writing. The gyro sensor 112 senses the curved movement of dragging, rotational movement, and angular speed when writing the letter ‘S’ or ‘ㄹ’ when writing.

The acceleration sensor 114 is preferably provided at the tip of the pen, but is not necessarily limited thereto. Acceleration sensor 114 includes a three-axis acceleration sensor. The acceleration sensor 114 generates an acceleration measurement value by sensing the acceleration of the center of the pen moving in the pivot orientation for writing. The acceleration sensor 114 senses linear movement in a straight line when writing.

The OID sensor 116 is provided on the pen tip. The OID sensor 116 recognizes the coordinates of points and line data included in the paper when the pen tip moves for writing.

Figure 2 is a diagram showing the operation of the electronic pen according to this embodiment.

The electronic pen 110 uses the IMU sensor 120 to sense the movement, direction, and speed of the pen when writing. When writing, the electronic pen 110 uses the OID sensor 116 to recognize the coordinates of point and line data written on paper when the pen tip moves. The electronic pen 110 recognizes handwriting based on the value sensed using the IMU sensor 120 and the value sensed using the OID sensor 116.

Figure 3 is a diagram showing a real-time stroke analysis system using machine learning for a writing pen according to this embodiment.

The real-time stroke analysis system using machine learning for a writing pen according to this embodiment includes an electronic pen 110, a handwriting recognition improvement device 320, a first ANN 332, and a second ANN 334. The components included in the real-time stroke analysis system using machine learning for writing pens are not necessarily limited to this.

The electronic pen 110 communicates with the handwriting recognition improvement device 320. The electronic pen 110 obtains acceleration a ( ) is transmitted to the handwriting recognition improvement device 320. The electronic pen 110 has an angular velocity ω ( ) is transmitted to the handwriting recognition improvement device 320. The electronic pen 110 transmits the coordinate value (OID 2D position p (x, y)) obtained using the OID sensor 116 to the handwriting recognition improvement device 320.

The handwriting recognition improvement device 320 includes the same hardware module as a typical web server or network server. The handwriting recognition improvement device 320 may be implemented in the form of a web server or network server.

The handwriting recognition improvement device 320 generally communicates with an unspecified number of clients or other servers via an open computer network such as the Internet. The handwriting recognition improvement device 320 refers to a computer system and computer software (web server program) that derives and provides work results corresponding to work performance requests from a client or other web server.

The handwriting recognition improvement device 320 obtains an acceleration value using the acceleration sensor 114 provided in the electronic pen 110 when writing. The handwriting recognition improvement device 320 acquires coordinate values (x, y) using the OID (Optical Identification) sensor 116 provided in the electronic pen 110 when writing. The handwriting recognition improvement device 320 obtains an angular velocity measurement value using the gyro sensor 112 provided in the electronic pen 110 when writing.

The handwriting recognition improvement device 320 calculates velocity information by integrating the acceleration value obtained from the acceleration sensor 114. The handwriting recognition improvement device 320 integrates the speed information to calculate distance information.

The handwriting recognition improvement device 320 performs FFT (Fast Fourier Transform) on the acceleration information to calculate a power spectrum of acceleration value. The handwriting recognition improvement device 320 performs FFT (Fast Fourier Transform) on the angular velocity measurement value obtained from the gyro sensor 112 to calculate a power spectrum of angular velocity. The handwriting recognition improvement device 320 substitutes the power spectrum calculated from the acceleration value, the coordinate value (x, y), and the power spectrum calculated from the angular velocity measurement value into ANN (Artificial Neural Network) and outputs the output result as the stroke input when writing. Classify as

The handwriting recognition improvement device 320 receives acceleration a ( ), angular velocity ω( ) to obtain. The handwriting recognition improvement device 320 has acceleration a ( ), angular velocity ω( ) is input to the first ANN (Artificial Neural Network).

The handwriting recognition improvement device 320 has acceleration a ( ) by integrating the velocity v( ) is calculated. The handwriting recognition improvement device 320 has a speed v( ) is integrated to calculate the distance d(x,y,z).

The handwriting recognition improvement device 320 has acceleration a ( ) to the velocity v( )(velocity v( )) is calculated.

The handwriting recognition improvement device 320 has acceleration a ( ) is input to the input layer of the first ANN (332) as the first input. The handwriting recognition improvement device 320 has a speed v( ) is input to the input layer of the first ANN (332) as the second input. The handwriting recognition improvement device 320 inputs the distance d(x,y,z) as the third input to the input layer of the first ANN 332. The handwriting recognition improvement device 320 has an angular velocity ω ( ) is input to the input layer of the first ANN (332) as the fourth input.

The handwriting recognition improvement device 320 obtains coordinate values (OID 2D position p (x, y)) from the electronic pen 110. The handwriting recognition improvement device 320 inputs coordinate values (OID 2D position p (x, y)) into the second ANN 334.

The handwriting recognition improvement device 320 has an angular velocity ω ( ) by performing FFT (Fast Fourier Transform) on the angular velocity power spectrum ( ) is calculated. The handwriting recognition improvement device 320 has acceleration a ( ) by performing an FFT on the acceleration power spectrum ( ) is calculated.

The handwriting recognition improvement device 320 is an angular velocity power spectrum ( ) is input to the input layer of the second ANN 334 as the first input. The handwriting recognition improvement device 320 is an acceleration power spectrum ( ) is input to the input layer of the second ANN (334) as the second input. The handwriting recognition improvement device 320 inputs the coordinate value (OID 2D position p (x, y)) as a third input to the input layer of the second ANN 334.

The handwriting recognition improvement device 320 uses an AI model to recognize handwriting using a cross-platform API for applications based on coordinate values (OID 2D position p (x, y)) and generates a handwriting recognition result. The handwriting recognition improvement device 320 inputs the handwriting recognition results to the first ANN 332 and the second ANN 334.

The handwriting recognition improvement device 320 recognizes the handwriting input from the electronic pen 110 based on the first learning result of the first ANN 332 and the second learning result of the second ANN 334 and creates a stroke. Classify.

The first ANN 332 classifies raw data in a time series manner.

The first ANN 332 has acceleration a ( ), angular velocity ω( ) outputs the first learning result of learning parameters based on

The first ANN 332 calculates the acceleration a ( ), speed v( ), distance d(x,y,z), angular velocity ω( ) outputs the first learning result of learning.

In the input layer of the first ANN 332, the acceleration a ( ), the speed v( ), distance d(x,y,z), angular velocity ω( ) Save it as a layer with a feature map for each. Acceleration a ( ), speed v( ), distance d(x,y,z), angular velocity ω( ) Use each to obtain the feature map of the layer located in the upper layer, and then learn the features from the feature map. When the output layer of the first ANN (332) obtains a classification result using the features extracted from the hidden layer, the output layer for each GRF (Ground Reaction Force), CoP (Center of Pressure), and CR (Center of Rotation) to be classified Generate a first learning result that maps to a probability value.

When the first ANN 332 receives the handwriting recognition result from the handwriting recognition improvement device 320, the first ANN 332 receives the acceleration a ( ), speed v( ), distance d(x,y,z), angular velocity ω( ), the first learning result obtained by learning the handwriting recognition result is output.

When the input layer of the first ANN 332 receives the handwriting recognition result from the handwriting recognition improvement device 320, the acceleration a ( ), speed v( ), distance d(x,y,z), angular velocity ω( ), and save it as a layer with a feature map for each handwriting recognition result. One or more hidden layers of the first ANN 332 have acceleration a ( ), speed v( ), distance d(x,y,z), angular velocity ω( ), each handwriting recognition result is used to obtain the feature map of the layer located in the upper layer, and then the features are learned from the feature map. When the output layer of the first ANN (332) obtains a classification result using the features extracted from the hidden layer, the output layer for each GRF (Ground Reaction Force), CoP (Center of Pressure), and CR (Center of Rotation) to be classified The first learning result is generated by mapping to a probability value.

The second ANN 334 classifies repetitive movements of the electronic pen 110 when writing. The second ANN 334 extracts the frequency and power spectrum of the signal over time for the electronic pen 110.

The second ANN 334 has acceleration a ( ), angular velocity ω( ) and output the second learning result of learning the coordinate values (OID 2D position p (x, y)) based on .

The second ANN 334 is an angular velocity power spectrum input from the handwriting recognition improvement device 320 ( ), acceleration power spectrum ( ), the second learning result of learning the coordinate values (OID 2D position p (x, y)) is output.

The input layer of the second ANN 334 is the angular velocity power spectrum ( ), acceleration power spectrum ( ), and store it as a layer with a feature map for each coordinate value (OID 2D position p (x, y)). One or more hidden layers of the second ANN 334 include the angular velocity power spectrum ( ), acceleration power spectrum ( ), coordinate values (OID 2D position p (x, y)) are used to obtain the feature map of the layer located in the upper layer, and then the features are learned from the feature map. When the output layer of the second ANN (334) obtains a classification result using the features extracted from the hidden layer, the output for each RF (Repetitive Frequency) and PC (Power Spectrum Characteristics) to be classified is mapped to a probability value. generate results.

When the second ANN 334 receives the handwriting recognition result from the handwriting recognition improvement device 320, the angular velocity power spectrum ( ), acceleration power spectrum ( ), a coordinate value (OID 2D position p (x, y)), and a second learning result obtained by learning the handwriting recognition result are output.

When the input layer of the second ANN 334 receives the handwriting recognition result from the handwriting recognition improvement device 320, the angular velocity power spectrum ( ), acceleration power spectrum ( ), coordinate values (OID 2D position p (x, y)), and handwriting recognition results are stored as a layer with feature maps for each. One or more hidden layers of the second ANN 334 include the angular velocity power spectrum ( ), acceleration power spectrum ( ), coordinate values (OID 2D position p (x, y)), and handwriting recognition results are used to obtain the feature map of the layer located in the upper layer, and then learn the features from the feature map. When the output layer of the second ANN (334) obtains a classification result using the features extracted from the hidden layer, the output for each RF (Repetitive Frequency) and PC (Power Spectrum Characteristics) to be classified is mapped to probability values for second learning. generate results.

Figure 4 is a diagram for explaining a stroke classification method using ANN according to this embodiment.

The electronic pen 110 operates for writing. When the electronic pen 110 moves for writing, it uses the acceleration sensor 114 to detect acceleration a ( )(acceleration a( ))). When the electronic pen 110 moves for writing, the angular velocity ω( )(angular velocity ω( ))). When the electronic pen 110 moves for writing, it acquires coordinate values (OID 2D position p (x, y)) using the OID sensor 116.

When the electronic pen 110 moves for writing, it uses the acceleration sensor 114 to detect acceleration a ( )(acceleration a( ))). The handwriting recognition improvement device 320 receives acceleration a ( ) to obtain. The handwriting recognition improvement device 320 has acceleration a ( ) is input to the input layer of the first ANN (332) as the first input.

The handwriting recognition improvement device 320 has acceleration a ( ) is applied to the adaptive Runge-Kutta 4th order method to obtain the velocity v( )(velocity v( )) is calculated. In other words, the handwriting recognition improvement device 320 increases the acceleration a ( ) by integrating the velocity v( ) is calculated. The electronic pen 110 has a speed v ( ) is input as the second input of the first ANN (332).

The handwriting recognition improvement device 320 has a speed v( ) is applied to the Adaptive Runge-Kutta 4th order method to calculate the distance d (x,y,z). In other words, the handwriting recognition improvement device 320 has a speed v ( ) is integrated to calculate the distance d(x,y,z). The handwriting recognition improvement device 320 inputs the distance d(x,y,z) as the third input of the first ANN 332.

When the electronic pen 110 moves for writing, the angular velocity ω( )(angular velocity ω( ))).

The handwriting recognition improvement device 320 receives an angular velocity ω( ) to obtain. The handwriting recognition improvement device 320 has an angular velocity ω ( ) is input as the fourth input of the first ANN (332).

The first ANN 332 includes an input layer, one or more hidden layers, and an output layer.

The input layer of the first ANN 332 has acceleration a ( )(first input), speed v( )(second input), distance d(x,y,z)(third input), angular velocity ω( ) (fourth input) is input. The input layer of the first ANN 332 has acceleration a ( )(first input), speed v( )(second input), distance d(x,y,z)(third input), angular velocity ω( ) (Fourth input) Save as a layer with a feature map for each.

One or more hidden layers of the first ANN 332 have acceleration a ( )(first input), speed v( )(second input), distance d(x,y,z)(third input), angular velocity ω( ) (fourth input) to obtain the feature map of the layer located in the upper layer. The hidden layer of the first ANN 332 learns features from the obtained feature map.

The output layer of the first ANN (332) obtains classification results using the features extracted from the hidden layer, and outputs each GRF (Ground Reaction Force), CoP (Center of Pressure), and CR (Center of Rotation) to be classified. Map to probability value.

The handwriting recognition improvement device 320 has an angular velocity ω ( ) by performing an FFT on the angular velocity power spectrum ( )(Power spectrum of angular velocity v( )) is calculated. The handwriting recognition improvement device 320 is an angular velocity power spectrum ( ) is input as the first input of the second ANN (334).

The handwriting recognition improvement device 320 has acceleration a ( ) by performing an FFT on the acceleration power spectrum ( )(Power spectrum of acceleration a( )) is calculated. The handwriting recognition improvement device 320 is an acceleration power spectrum ( ) is input as the second input of the second ANN (334).

When the electronic pen 110 moves for writing, it acquires coordinate values (OID 2D position p (x, y)) using the OID sensor 116. The handwriting recognition improvement device 320 obtains coordinate values (OID 2D position p (x, y)) from the electronic pen 110. The handwriting recognition improvement device 320 inputs the coordinate value (OID 2D position p (x, y)) as the third input of the second ANN 334.

The second ANN 334 includes an input layer, one or more hidden layers, and an output layer.

The input layer of the second ANN 334 is the angular velocity power spectrum ( ) (first input), acceleration power spectrum ( ) (second input), coordinate values (OID 2D position p (x, y)) (third input) are input. The input layer of the second ANN 334 is the angular velocity power spectrum ( ) (first input), acceleration power spectrum ( ) (second input), and coordinate values (OID 2D position p (x, y)) (third input) are stored as layers with feature maps for each.

One or more hidden layers of the second ANN 334 include the angular velocity power spectrum ( ) (first input), acceleration power spectrum ( ) (second input) and coordinate values (OID 2D position p (x, y)) (third input) are used to obtain the feature map of the layer located in the upper layer. The hidden layer learns features from the obtained feature maps.

The output layer of the second ANN 334 obtains classification results using the features extracted from the hidden layer, and maps the output for each RF (Repetitive Frequency) and PC (Power Spectrum Characteristics) to be classified into probability values.

Figure 5 is a diagram for explaining a stroke classification method reflecting handwriting recognition results according to this embodiment.

The electronic pen 110 operates for writing. When the electronic pen 110 moves for writing, it uses the acceleration sensor 114 to detect acceleration a ( )(acceleration a( ))). When the electronic pen 110 moves for writing, the angular velocity ω( )(angular velocity ω( ))). When the electronic pen 110 moves for writing, it acquires coordinate values (OID 2D position p (x, y)) using the OID sensor 116.

When the electronic pen 110 moves for writing, it uses the acceleration sensor 114 to detect acceleration a ( )(acceleration a( ))). The handwriting recognition improvement device 320 receives acceleration a ( ) obtain (S510). When the electronic pen 110 moves for writing, the angular velocity ω( )(angular velocity ω( ))). The handwriting recognition improvement device 320 receives an angular velocity ω( ) obtain (S510).

The handwriting recognition improvement device 320 has acceleration a ( ) is input to the input layer of the first ANN 332 as the first input (S512).

The handwriting recognition improvement device 320 has acceleration a ( ) by integrating the velocity v( ) is calculated. In other words, the handwriting recognition improvement device 320 increases the acceleration a ( ) is applied to the adaptive Runge-Kutta 4th order method to obtain the velocity v( )(velocity v( )) is calculated (S514).

The handwriting recognition improvement device 320 has a speed v( ) is integrated to calculate the distance d(x,y,z). In other words, the handwriting recognition improvement device 320 has a speed v ( ) is applied to the adaptive Runge-Kutta 4th order method to calculate the distance d (x,y,z) (S514).

In step S514, the handwriting recognition improvement device 320 uses [Equation 1] to determine acceleration a ( ) based on speed v( ) and the distance d(x,y,z).

The handwriting recognition improvement device 320 has a speed v( ) is input to the input layer of the first ANN 332 as the second input (S516). The handwriting recognition improvement device 320 inputs the distance d(x,y,z) as the third input to the first ANN 332 (S518). The handwriting recognition improvement device 320 has an angular velocity ω ( ) is input as the fourth input of the first ANN (332) (S530).

The first ANN 332 includes an input layer, one or more hidden layers, and an output layer.

The input layer of the first ANN 332 has acceleration a ( )(first input), speed v( )(second input), distance d(x,y,z)(third input), angular velocity ω( ) (fourth input), the handwriting recognition result (fifth input) is input. The input layer of the first ANN 332 has acceleration a ( )(first input), speed v( )(second input), distance d(x,y,z)(third input), angular velocity ω( ) (fourth input), and the handwriting recognition result (fifth input) are stored as layers with feature maps for each.

One or more hidden layers of the first ANN 332 have acceleration a ( )(first input), speed v( )(second input), distance d(x,y,z)(third input), angular velocity ω( ) (fourth input) and the handwriting recognition result (fifth input) are used to obtain the feature map of the layer located in the upper layer. The hidden layer of the first ANN 332 learns features from the obtained feature map.

The output layer of the first ANN (332) obtains classification results using the features extracted from the hidden layer, and outputs each GRF (Ground Reaction Force), CoP (Center of Pressure), and CR (Center of Rotation) to be classified. Map to probability value (S530).

The handwriting recognition improvement device 320 has an angular velocity ω ( ) by performing an FFT on the angular velocity power spectrum ( )(Power spectrum of angular velocity v( )) is calculated. The handwriting recognition improvement device 320 is an angular velocity power spectrum ( ) is input as the first input of the second ANN 334 (S540). The handwriting recognition improvement device 320 has acceleration a ( ) by performing an FFT on the acceleration power spectrum ( )(Power spectrum of acceleration a( )) is calculated. The handwriting recognition improvement device 320 is an acceleration power spectrum ( ) is input as the second input of the second ANN (334) (S540).

In step S540, the handwriting recognition improvement device 320 analyzes repetitive movements of the electronic pen 110 when writing. In other words, the handwriting recognition improvement device 320 extracts the frequency and power spectrum of the signal over time.

When the electronic pen 110 moves for writing, it acquires coordinate values (OID 2D position p (x, y)) using the OID sensor 116. The handwriting recognition improvement device 320 obtains coordinate values (OID 2D position p (x, y)) from the electronic pen 110. The handwriting recognition improvement device 320 inputs the coordinate value (OID 2D position p (x, y)) as the third input of the second ANN 334 (S550).

The handwriting recognition improvement device 320 uses an AI model to recognize handwriting using a cross-platform API for applications based on coordinate values (OID 2D position p (x, y)) and generates a handwriting recognition result (S560 ).

The handwriting recognition improvement device 320 can input the handwriting recognition result as the fifth input of the first ANN 332 and the fourth input of the second ANN 334.

The second ANN 334 includes an input layer, one or more hidden layers, and an output layer.

The input layer of the second ANN 334 is the angular velocity power spectrum ( ) (first input), acceleration power spectrum ( ) (second input), coordinate value (OID 2D position p (x, y)) (third input), and handwriting recognition result (fourth input) are input. The input layer of the second ANN 334 is the angular velocity power spectrum ( ) (first input), acceleration power spectrum ( ) (second input), coordinate value (OID 2D position p (x, y)) (third input), and handwriting recognition result (fourth input) are stored as layers with feature maps for each.

One or more hidden layers of the second ANN 334 include the angular velocity power spectrum ( ) (first input), acceleration power spectrum ( ) (second input), coordinate values (OID 2D position p (x, y)) (third input), and handwriting recognition results (fourth input) are used to obtain the feature map of the layer located in the upper layer. The hidden layer of the second ANN 334 learns features from the obtained feature map.

The output layer of the second ANN (334) obtains classification results using the features extracted from the hidden layer, and maps the output for each RF (Repetitive Frequency) and PC (Power Spectrum Characteristics) to be classified into probability values (S570).

The handwriting recognition improvement device 320 recognizes handwriting and classifies strokes based on the result of step S530 and the result of step S570.

The handwriting recognition improvement device 320 checks whether tuning of the machine learning model (ML Model) is complete as a result of repeatedly performing steps S530 and S570 (S580).

In Figure 5, steps S510 to S580 are described as being sequentially executed, but this is not necessarily limited. In other words, the steps shown in FIG. 5 may be applied by modifying them or executing one or more steps in parallel, so FIG. 5 is not limited to a time-series order.

As described above, the stroke classification method reflecting the handwriting recognition results according to this embodiment shown in FIG. 5 may be implemented as a program and recorded on a computer-readable recording medium. A computer-readable recording medium on which a program for implementing a stroke classification method reflecting the handwriting recognition results according to this embodiment is recorded includes all types of recording devices that store data that can be read by a computer system.

The above description is merely an illustrative explanation of the technical idea of the present embodiment, and those skilled in the art will be able to make various modifications and variations without departing from the essential characteristics of the present embodiment. Accordingly, the present embodiments are not intended to limit the technical idea of the present embodiment, but rather to explain it, and the scope of the technical idea of the present embodiment is not limited by these examples. The scope of protection of this embodiment should be interpreted in accordance with the claims below, and all technical ideas within the equivalent scope should be interpreted as being included in the scope of rights of this embodiment.

110: Electronic pen
320: Handwriting recognition improvement device

Claims (15)

In the handwriting recognition improvement device, acceleration a ( ), angular velocity ω( ) After obtaining the acceleration a ( ), the angular velocity ω( ) A process of inputting parameters based on a first ANN (Artificial Neural Network);
In the first ANN, the acceleration a ( ), the angular velocity ω( ) A process of outputting a first learning result of learning parameters based on ;
A process of obtaining coordinate values (OID 2D position p (x, y)) from the electronic pen in the handwriting recognition improvement device and inputting them into a second ANN;
In the second ANN, the acceleration a ( ), the angular velocity ω( A process of outputting a second learning result by learning a parameter based on ) and the coordinate value (OID 2D position p (x, y));
A process of recognizing handwriting input from the electronic pen and classifying strokes in the handwriting recognition improvement device based on the first learning result and the second learning result;
A real-time stroke analysis method comprising: According to paragraph 1,
In the handwriting recognition improvement device
The acceleration a ( ) by integrating the velocity v( ) is calculated, and the speed v ( A real-time stroke analysis method characterized by calculating the distance d(x,y,z) by integrating ). According to paragraph 2,
In the handwriting recognition improvement device
The acceleration a ( ) to the velocity v ( )(velocity v( )) A real-time stroke analysis method characterized by calculating. According to paragraph 2,
In the handwriting recognition improvement device
The acceleration a ( ) is input as the first input to the input layer of the first ANN,
The speed v( ) is input as a second input to the input layer of the first ANN,
Input the distance d(x,y,z) as a third input to the input layer of the first ANN,
The angular velocity ω( ) is input as a fourth input to the input layer of the first ANN. According to paragraph 4,
In the first ANN
The acceleration a ( ), the speed v( ), the distance d(x,y,z), the angular velocity ω( ) A real-time stroke analysis method characterized in that the first learning result is output. According to clause 5,
Acceleration a ( ), the speed v( ), the distance d(x,y,z), the angular velocity ω( ) Save it as a layer with a feature map for each,
The acceleration a ( ), the speed v( ), the distance d(x,y,z), the angular velocity ω( ) Use each to obtain the feature map of the layer located in the upper layer, and then learn the features from the feature map.
In the output layer of the first ANN, a classification result is obtained using the features extracted from the hidden layer, and the output for each GRF (Ground Reaction Force), CoP (Center of Pressure), and CR (Center of Rotation) to be classified is converted into a probability value. A real-time stroke analysis method characterized by generating the mapped first learning result. According to paragraph 1,
In the handwriting recognition improvement device
The angular velocity ω( ) by performing FFT (Fast Fourier Transform) on the angular velocity power spectrum ( ) is calculated, and the acceleration a ( ) by performing an FFT on the acceleration power spectrum ( ) A real-time stroke analysis method characterized by calculating. In clause 7,
In the handwriting recognition improvement device
The angular velocity power spectrum ( ) is input to the input layer of the second ANN as the first input, and the acceleration power spectrum ( ) is input to the input layer of the second ANN as a second input, and the coordinate value (OID 2D position p (x, y)) is input to the input layer of the second ANN as a third input. Real-time stroke analysis method. According to clause 8,
In the second ANN
The angular velocity power spectrum ( ), the acceleration power spectrum ( ), a real-time stroke analysis method characterized by outputting a second learning result by learning the coordinate value (OID 2D position p (x, y)). According to clause 9,
The angular velocity power spectrum in the input layer of the second ANN ( ), the acceleration power spectrum ( ), stored as a layer with a feature map for each of the coordinate values (OID 2D position p (x, y)),
The angular velocity power spectrum ( ), the acceleration power spectrum ( ), obtain the feature map of the layer located in the upper layer using each of the coordinate values (OID 2D position p (x, y)), and then learn the features from the feature map,
The output layer of the second ANN obtains a classification result using the features extracted from the hidden layer, and the second learning result maps the output for each RF (Repetitive Frequency) and PC (Power Spectrum Characteristics) to be classified into probability values. A real-time stroke analysis method characterized by generating. According to paragraph 1,
In the handwriting recognition improvement device
After generating a handwriting recognition result that recognizes the handwriting using a cross-platform API for an application based on the coordinate value (OID 2D position p (x, y)) using an AI model, the handwriting recognition result is sent to the first A real-time stroke analysis method characterized by input into an ANN and the second ANN. According to clause 11,
In the first ANN
The acceleration a ( ), the speed v( ), the distance d(x,y,z), the angular velocity ω( ), a real-time stroke analysis method characterized by outputting the first learning result obtained by learning the handwriting recognition result. According to clause 12,
The acceleration a ( ), the speed v( ), the distance d(x,y,z), the angular velocity ω( ), stored as a layer with a feature map for each of the handwriting recognition results,
The acceleration a ( ), the speed v( ), the distance d(x,y,z), the angular velocity ω( ), using each of the handwriting recognition results to obtain a feature map of a layer located in a higher layer, and then learning features from the feature map,
In the output layer of the first ANN, a classification result is obtained using the features extracted from the hidden layer, and the output for each GRF (Ground Reaction Force), CoP (Center of Pressure), and CR (Center of Rotation) to be classified is converted into a probability value. A real-time stroke analysis method characterized in that the first learning result is generated by mapping to . According to clause 11,
In the second ANN
The angular velocity power spectrum ( ), the acceleration power spectrum ( ), the coordinate value (OID 2D position p (x, y)), and the second learning result obtained by learning the handwriting recognition result. According to clause 14,
The angular velocity power spectrum in the input layer of the second ANN ( ), the acceleration power spectrum ( ), the coordinate value (OID 2D position p (x, y)), and the handwriting recognition result are stored as a layer having a feature map for each,
The angular velocity power spectrum ( ), the acceleration power spectrum ( ), the coordinate value (OID 2D position p (x, y)), and the handwriting recognition result are used to obtain a feature map of the layer located in the upper layer, and then learn features from the feature map,
In the output layer of the second ANN, a classification result is obtained using the features extracted from the hidden layer, and the output for each RF (Repetitive Frequency) and PC (Power Spectrum Characteristics) to be classified is mapped to a probability value to obtain the second learning result. A real-time stroke analysis method characterized by generating.