RO131754A1 - Method and device for learning calligraphic writing - Google Patents

Abstract

The invention relates to a method designed for learning calligraphic writing in a customized, amusing manner, by means of a tablet-like device. The method, as claimed by the invention, is based on the analysis and the adapted response of a digital tutor and consists, in a first stage, in the acquisition of the letters written by a child on a tactile screen, using the finger and the capacitive pen available as a component of a device to be used, where the writen letters are memorized by determining the coordinates x, y, on angles and rotated letter, of the character-forming points, followed by the evaluation of the characters based on an algorithm, by comparing point by point the tested letter with the model letter, the result of the evaluation being expressed by three rating grades describing the level of acquisition of the writing ability, which are stored and memorized in a data base wherefrom the tutor will calculate an average rating and will take a decision which will be correlated with the affective state of the child, automatically detected by using neural networks, which permits the customization of the response offered, the emotional state of the child being further classified in a data base, and, finally, the results of the cognitive and affective evaluations being used by the tutor for displaying the response by means of a graphic interface, said response being coordinated with animations and video clips existing in the data base.

Description

The present invention relates to a method for learning to calligraphy writing in a personalized, fun form using a tablet device. More precisely, the learning method of the present invention consists of learning the handwriting and writing skills with the simultaneous assessment of the student's affective state. The method is intended for preschool and school children and is developed dynamically through the interaction between student and tutor, which will choose the best pedagogical strategy for the current student in the current situation.

Known electronic products are known, with a sensitive screen that can display visual graphic elements of conventional alphanumeric characters, or drawings, and which can locate conventional elements by scanning tactile sensors, located on the surface of the screen, which can generate associated sound messages, particular and that ensures the simultaneous and correlated transmission of information on visual and / or auditory sensory channels. These products do not provide complete informational content, limiting the use of these products only by persons with major visual impairments.

From the document RO 123209 is known an audio-tactile device consisting of a stratified assembly consisting of a first film on the surface of which there are graphic visual elements and a second film on which sensors are mounted that generate electrical information regarding the position of the associated graphic elements. , sensors connected to an electrical block that decodes the received information, generating pre-recorded audio signals corresponding to the probed graphic elements.

The disadvantage of this device is that only the combination of the image and the graphic symbol is checked.

Document RO 127616 presents a learning method that involves a step of training the subject where he is presented with a combination of images correlated with an associated sound, repetitive so that the subject memorizes those images and sounds, followed by the test stage when the subject i there are several images to choose from

0 15-- 0 D 7 0 ο

9 -09- 2015

good, the errors being registered and based on them is calculated a new distribution of emission of their images / sounds, which is repeated until a certain threshold imposed by errors is reached. This method only checks the image-sound combination, without evaluating the user's emotions.

There are known from documents CN 103824486, CN 201812396 and CN 101840645, machines for learning the painting and calligraphic writing, which are based on the optical transmission of the image projection and its digital analysis, or which performs image processing through a computer. These machines are not based on point coordinates and image processing and do not include a pedagogical approach depending on the child's mood.

From JPH07160832, a method of online character learning of graphic characters / characters is known, which involves selecting characters or graphic characters and recognizing them from a list. This method does not use point coordinates and image processing, without making an assessment of the similarities between graphic characters / characters in relation to a prototype character / graphic sign.

The technical problem proposed by the present invention consists in the training of the writing skills and the automatic evaluation of the written characters, in relation to the affective state of the subject.

The method of learning the calligraphic writing according to the invention consists of the stage of writing the characters of the letters, which are in the next stage memorized by determining the coordinates x, y, on angles and the rotated letter, of the points that form the character, the evaluation of the characters based on a DTW algorithm through point-to-point comparison of the test letter with the model letter, the result of the evaluation being expressed by three qualifications that describe the level of acquisition of the writing ability, which are stored and stored in a database, from which the tutor will calculate an average of the qualifications and will make a decision has to be correlated with the affective state of the child, automatically detected using RBF neural networks and which allows to personalize the offered response, the emotional state of the child being further classified in a database, finally being used by the guardian for displaying the answer through a coordinated graphical interface with graphic animations or video clips exists in the database.

- 1 0 1 5 - - 0 0 7 0 0 2 9 -09- 2015

The learning method is applied using a tablet device that has a processor, memory, camcorder, microphone, speaker, touch screen and capacitive pen, arranged in a simple and attractive design case.

The advantages of the present invention consist in:

learning the skills of writing handwriting through play, spending free time in an educational way; adapts according to the child's preferences; The grades and recommendations of the tutor are positive; interaction between guardian and child

The following is an example of embodiment of the invention in relation to the figures 1 and 2 which represent:

fig.l the scheme of internal operation of the method;

fig.2 view of the device

The learning method according to the present invention consists in the adapted and personalized learning of the handwriting and the training of writing skills, with the simultaneous assessment of the student's affective state.

In the first stage, the letters written by the child are written, which write different characters on a touch screen, using the finger and the capacitive pen provided, as part of the device used. The written characters are stored by the x and y coordinates of the points that make up that character. The following is an automatic character evaluation using a point-to-point algorithm. Four distances are calculated based on the Dynamic Time Warping (DTW) algorithm based on the coordinates of the component points, on the X axis, on the Y axis, on the angles and on the rotated letter. The result of the evaluation of the letter written by the child is expressed by three qualifications that describe the level of acquisition of the writing ability: PURCHASED, NOT BUYED, IN ACQUISITION. These qualifications are only internal, being used only by the tutor during the decision stage, without being made known to the child.

2015-- 00700 ¹ 9 -09- 2015

After each written letter receives such a rating, it is stored in a database to form a history of the child's cognitive assessments. As the child writes letters, the tutor calculates the average of these grades and makes a decision based on it. Also based on the history of cognitive evaluation, the tutor can observe the evolution over time, adapting his answer accordingly.

Simultaneously with the writing stage an automatic recognition of the emotional state of the child takes place, rendered by facial expressions and voice, recorded with the video camera. Emotional recognition of facial expressions is done using a neural network (Radial BasisFunction), and recognition of emotions from the voice is done using a second neural network (Radial BasisFunction). The emotional state is further classified into three classes: POSITIVE, NEGATIVE, NEUTRAL, because the children in the target group do not have the ability to express their emotions very well (according to psychological studies) and, in addition, there is no need for more classes.

If the student is in a POSITIVE state, then the application has reached the purpose of maintaining its curiosity and to want to write beautifully from a calligraphic point of view. in this case he can be allowed to follow the thread of the lesson until the qualification has been purchased for all letters.

If the student is in a NEUTRAL state, the student will be able to move on to the next letter even if he / she has only obtained the ACQUISITION qualification for the current character.

If the student is in a NEGATIVE emotional state, strategies for relaxation / motivation / change of mood will have to be applied.

The history of cognitive and affective evaluations is maintained in the database as part of the student profile, along with identifying information, such as: username, age, gender and color and character preferences. These preferences will be used towards the graphical interface to display the tutor's response in different forms.

Each learning situation is evaluated within the decision-recommendation module, based on an expert system capable of making inferences based on cognitive assessment and affective evaluation. The inferences are based on predefined rules, but also dynamically defined rules during the course of the lesson, as the tutor interacts with the student and learns to learn

^ - 2 0 1 5 - 1 0 7 0 0 ² 9-os-2015 adapt to its needs, style and preferences. Thus the tutor will choose the best pedagogical strategy for the current student in the current situation.

The tutor's answer is provided to the student through the graphical interface. Here will be displayed the actions corresponding to the decision of the tutor, coordinated with animations existing in the database. The tutor can make four recommendations:

a) the student continues to write the current letter;

b) the student can go to the following letter;

c) the student needs help;

d) the student needs to relax.

In the case of recommendation c), the tutor will give the student a clue, showing for example again how to draw the letter beautifully. in the case of recommendation d) the student will receive videos or exercises related to the calligraphy lesson.

The way of providing the answer will always be positive, cheerful, highlighting the student's ability to write beautifully. A beautifully written letter is praised, and a less beautiful letter will only be a reason to persevere.

personalized learning is based on the student model defined both at the beginning of the system and during the course of the lesson, the student profile being composed of the identification data introduced by the parent at the beginning of the application and the history of the cognitive and affective evaluations made during the lesson.

The method of learning to write calligraphy will be implemented by using a tablet device that has a processor, a memory, a camcorder, a microphone, a speaker, a touch screen and a capacitive pen, arranged in a design case. simple and attractive.

Claims (2)

1. Method of learning the handwriting, characterized by the fact that it consists of the stage of writing the characters of the letters, which in the next stage are determined and stored at the x, y coordinates, at angles and the rotated letter, of the points that make up the character, the next stage consisting of an automatic cognitive evaluation of the letter by calculating four distances based on a DTW (Dynamic Time Warping) algorithm and comparing the coordinates of each component point in the letter, on the X axis, on the Y axis, on the angles and on the rotated letter, with the letter model, the result of the evaluation being expressed by three qualifiers that describe the level of acquisition of the writing ability, the next step consisting of memorizing and storing the qualifications in a database, where the tutor will calculate an average of the qualifications and will make a decision which in -a next step will be correlated with the assessment of the affective state of the child, which is automatically detected in in another stage, using two neural networks of the RBF type (Radial BasisFunction) through separate recording of facial expressions and voice and which will allow personalization of the offered response, the emotional or emotional state of the child being in a next stage classified in a database, in a final step of the method, the tutor's answer will be displayed on a graphical interface, based on an expert system capable of making inferences starting from the cognitive evaluation stage and the affective evaluation stage, based on predefined rules. and some dynamically defined rules, an answer that will be coordinated with graphic animations or videos, exist in the database, depending on the pedagogical strategy chosen for the current student in the current situation. 2. A method of learning the calligraphic writing according to claim 1, characterized in that the implementation of the method is performed with a tablet type device comprising a processor, a memory, a camcorder, a microphone, a speaker, a touch screen and a capacitive pen, arranged in a casing with a simple and attractive design.