TR201902975A2 - INTERACTIVE ARTIFICIAL INTELLIGENCE CONTROLLED EDUCATION SYSTEM - Google Patents

Abstract

The invention is an interactive artificial intelligence controlled education system and its feature; A cartoon character that allows cartoon characters to operate on scenarios given to them based on the detection of children's speech and movements with artificial intelligence for the purpose of supporting the education of children with autism, attention deficit, hyperactivity disorder, developmental delay or psychological and neurological problems who require special support in normal development 1), skeleton (2), model (3), child (4), artificial intelligence (5), scenario (6), speech detection (7), motion detection (8) and communication computer with microphone, camera and speaker tablet, phone, etc. device (9).

Description

DESCRIPTION INTERACTIVE AI CONTROLLED EDUCATION SYSTEM TECHNICAL FIELD This invention can be used for autism, attention deficit, hyperactivity disorder, developmental delay or normal Support for the treatment of children with psychological and neurological problems who require special support in development based on the detection of speech and movements of children with artificial intelligence for this purpose. based on the structure of the movie characters to act on the scenarios given to them. It is about an interactive artificial intelligence controlled education system.

CURRENT STATE OF THE TECHNIQUE Today, autism, attention deficit, hyperactivity disorder, developmental delay or normal Children with psychological and neurological problems who require special support in development They receive training individually or as a group by the trainees in their institutions. These children eye movements are tracked with artificial intelligence and pay attention to the instructor or the training material. Detection apps are available. However, there is only one person at home, except for the educational institution or an educator. self-education is not possible.

OBJECTIVE OF THE INVENTION Children with attention deficit and concentration problems relate to their characters. In cases where education is insufficient or education quality An interactive artificial intelligence of children's favorite cartoon characters to increase This invention was created in order to give the relevant education to the children under the control of the child. Children are very interested in their favorite cartoon characters. of cartoon characters They quickly assimilate their movements and behaviors. To their current instructors Children who cannot concentrate can concentrate on the relevant cartoon character. children's eye artificial intelligences that measure attention with their movements, only indicators that show the efficiency of education they produce. They do not give an education to the child. In the related study, direct artificial intelligence of education to children It is aimed to be given by the teacher or by the trainer and artificial intelligence together.

Autism is a communication problem. Children generally continue their physical development until the age of 2. Good breakthroughs in their mental development by observing the environment after the age of 2 they do. They observe their parents, observe their surroundings and educate themselves. Children also do not have much contact with the outside world. These children are isolated from the outside world. a they live in a lantern, they do not observe their surroundings. When the normal child reaches the age of 3, mental He has not brought himself to a level. But children with autism They do not reach the relevant level because they do not train. 3 years old is an age when the autism problem manifests itself. Education starting at age 3 is normal is the treatment process. Education that starts before the age of 3 is an early treatment process and is very successful. Education after the age of 4 is a late education and it is very difficult. What's up with later releases Unfortunately, there are children who are turned away because there is nothing that can be done.

This is a very difficult situation for families. The child has the physical strength of 5 years old, but They cannot express themselves because they cannot speak. They are quite aggressive. physical strength Since it is in place, it is difficult to care for them by their families.

Mothers try to hold on as long as they can. But sometimes they have to give up. Children are sent to nursing homes. They are separated from their families.

In nursing homes, there is not much treatment. Children are given sedatives and the child is half asleep spends his life. The situation is different in normal and early treatment. Educators primarily they try to break the bell around them. They gather around, play games like clap hands they sing songs. The child does not show interest at first. But the trainers continue with patience. The child then claps his hand or shows a reaction. Instructors are very happy cheering they show. They show the child that they are happy. This kid likes it and looks around a little bit. they start watching. To relive the same cheer. This lantern is also a crack and is very it is valuable. Instructors try to destroy the bell about patience, the child's communication with the outside world. they try to establish it. A year passes, the child is no longer isolated from the outside world, communication establishes. However, although he is 4 years old physically, he is 3 years old mentally.

After that, the training is for the child to catch up with his friends. Long short outside and inside many things like this are given by training.

Communication with the outside world is difficult in children with autism, especially in children who are late for education.

They generally live in a lantern. But without the bell, with the outside world There are some moments when they communicate. Children concentrate while watching their favorite cartoons they become. At that time, his senses are clear. When the cartoon character laughs, they smile too. While the cartoon character is crying, they are also sad. There is no fan there.

In our invention, the education of cartoon characters for children with communication difficulties We have ensured that there is an interactive artificial intelligence that will give Let the cartoon characters perceive the movements and speech of children and depending on the scenarios given to them, according to the children's reactions. We made it possible for them to give their training.

These trainings can be used in the treatment of advanced and initial autism. Impact of education The field is not limited to just autism. developmental delay, attention deficit, hyperactivity and It can be used easily in children who have many different problems like this.

In addition, it can be easily used in the education of healthy children who do not have any problems. can be used.

EXPLANATION OF FIGURES Figure 1 Various cartoon characters consisting of a model dressed on a skeleton.

Figure 2 Image of Child's face and eye detection with OpenCv.

Figure 3 With OpenCv, my child's finger and hand movements and movements such as clapping image of the fix.

Figure 4 The cartoon character expects the child to watch him when he sings and and then the scenario flowchart, in which the child expects to join him with his movements.

Figure 5 The child's ability to follow the cartoon character as it speaks or moves. scenario flowchart, in which he is warned by calling his name if it is understood that he is expected and not followed.

Figure 6 When the name of the object is mentioned, the cartoon character allows the child to look at the object and scenario flowchart where he is expected to point with his finger.

Figure 7 The cartoon character tells a short story and who, what, where, scenario flow diagram in which the child asks the questions how, why, what happened, and expects an answer.

Figure 8 Cartoon character questions about self-care, social skills and various skills scenario flowchart, asking for an answer.

Figure 9 A cartoon in which the child is expected to recognize the concepts and showing two glasses. When you ask which character is big and which is short, it is important for the child to answer. scenario flowchart.

Figure 10 Scenario flowchart about the ability to recognize actions.

Figure 11 Scenario flowchart about the ability to recognize and try to imitate actions.

Figure 12 Scenario flowchart where reaction is expected from the child against the nonsense.

Figure 13 The cartoon character tells the child to say hello by saying hello, clapping hands. scenario flowchart that he wants it to circulate.

EXPLANATION OF REFERENCES 1 - cartoon character 2 - Skeleton 3 - Model 4 - Child - Artificial intelligence 6 - Script 7 - Speech detection 8 - Detection of movements, 9 -Computer, tablet, phone with microphone, camera and speaker vb.ayg1t.

DISCLOSURE OF THE INVENTION Components such as the movements of the cartoon character (1), the environment where the cartoon takes place, the background landscape controlled by the systems of this invention.

A cartoon characters(]) consists of 2 basic components skeleton(2) and model(3). skeleton(2) human consists of joints like the skeleton and animations in the cartoon character (1) of these joints achieved by its movements. Dozens of articulation points in the skeleton (2) of a cartoon character (1) is found. Knees and feet are some of these joints. For example, the hand of the cartoon character (l) If it is desired to shake the hand, the joint points in the hand area can be moved to the right and left. movement is performed. The 2nd component of the cartoon character(l) is the model(3). The model(3) is draped over the skeleton(2) a garment is like a Cartoon character(1) and contains the body and clothing components of the skeleton(2).

Model(3) technically consists of polygons. 3D visuality thanks to this polygon structure gains the ability Cartoon characters (l) (like Pepe, spider-man) are models (3) and are placed on a skeleton (2). (Figure 1) The movements are designed on the skeleton (2). skeleton(2) to move when it starts, mode1(3) on the skeleton(2) also moves. Different on a skeleton(2) models(3) can be dressed. For example, if a waving skeleton(2) gesture is designed and If Pepe model is dressed, Pepe waves hand, If Spider-Man model is dressed, Spider-Man hand Interactive communication with the child (4) opposite the cartoon characters(l) in the application is in it. Their own behavior according to the speech and movements of the children in front of them. determines. Detecting the speech and movements of the children (4) in front of them with artificial intelligence (5) is done. Sound processing, voice recognition and similar libraries of the system for the detection of speech(7) and cloud and similar services that provide this service are used. Related voice processing, voice recognition libraries and the cloud and similar services that provide this service, the microphone in which the program works, computer, tablet, phone etc. with camera and speaker, where communication is provided.ayg1tin(9) It interacts with its microphone and makes sense of incoming sounds. Words to be caught It is entered as data to voice processing, voice recognition libraries and cloud and similar services.

Hello, How are you and Let's play a game etc. a piece of code that listens for words are available.

When any of the related words are spoken, the program performs an action according to the word.

The application detects the movements of children along with speech. Detection of movements(8) image processing, image recognition and similar libraries for the system and those providing this service cloud and similar services are used. (Figure 2) Many libraries based on system image processing includes. Image processing, image recognition and similar libraries and cloud and With similar services, face and eye detection can be done, so that the child is in front of the application. and it is detected that it is in communication with the application.

Image processing such as face and eye detection, image recognition and similar libraries and these Objects such as hands, fingers with cloud and similar services that provide service, Also jumping and clapping Detection of movements such as (8) is also done. (Figure 3) The application is based on artificial intelligence(5), speech detection(7) and motion detection(8) of children(4). based on the scenarios (6) given to the cartoon characters(1) it relies on its structure. Cartoon character (l) saying hello to the child (4), clapping hands Flow diagram of a scenario(6) that he wants to multiply 1. (Figure 13) The application waits for the child(4) to say hello. He says to the child(4), let's play clap.

The cartoon character(1) in the app claps his hands. He says to the child (4), now clap your hands. Waits for the child(4) to clap their hands. If the child (4) does not clap his hands, he insists that he clap his hands. If the child (4) claps his hands, he ends the scenario (6) by saying Yasasin clapped his hand.

WH?`@r'ÄPJN'T Education of children with autism(4); Completion of therapy from baseline of therapy can be subdivided into various levels. (Each tier is the same as the previous tier.) It includes more difficult exercises and exercises based on the basics.) -Contact At this stage, it is aimed to establish a basic message with the child (4) and to perform simple commands. It is one of the first actions to look at the child (4) when he is called by his name. (Figure 5)To the child(4) Simple commands such as come to me come here are given and the child(4) has to carry out these commands. is expected. Singing songs, playing simple games and making the child(4) participate in these games. meat is expected. (Figure 4) Games such as ring games, finger games, box-and-box pliers playable child(4) watching these games first; It is expected that he will participate in these games after a while.

The aim at this level is for the child(4) to gain the habit of communicating with the outside world. It is very important to dry eye contact while talking to the child(4). Eye while playing games contact is made. If the child(4) loses eye contact, the game is paused and eye contact is is provided and the game continues. Simple words like mom and dad is meant to be said.

-Recognition of distant objects Helping the child(4) to communicate with the outside world by ensuring that they are interested in distant objects. is intended to be increased. When a distant object (1-2 meters) is asked from the child(4), the child(4) It is expected to bring or show. (Figure 6) (4) is expected to follow. It is also possible to continue a game played at this level for 3-4 rounds. It is expected that sequential games can also be played (in the form of an educator and a child (4)). Your child's other they are expected to imitate the actors (Figure 11). At this level, the child (4) eats the keith himself. eating, taking off clothes unaided, washing hands and face with help, brushing with help, meeting toilet needs with help, on a straight line able to walk and jump on one foot. He is expected to be able to hit and hold the ball on the ground.

It can drain fluids from one container to another. Various shapes using soft material can create -Don't make choices.

The child(4) is able to distinguish and select the desired one among many cards or objects. is expected. Objects in and around the house are recognized. Concepts such as big small long short is learned. (Figure 9) The concept of contrast is learned. Colors are learned. Which one is the food Classifications such as toys are learned. Being able to answer questions such as is this edible, is it sweet? is expected. It is learned to distinguish in questions such as which of the pictures on the table are eaten. from 1 to 10 counts up, it is expected to be able to choose pictures like the same and different. Completes the 4-piece puzzle.

Recognizes body parts such as nose and mouth. Recognizes emotions such as sad joy. Who is what in the story It answers questions like where. Uses nouns and verbs in speech. in conversations uses pronouns, knows the concept of plural. Asked about the daily work he does answers questions. Can tell 2 consecutive events in sequential events. your clothes distinguishes front and back. He collects his own belongings. Participates in group games. queues up waits its turn. Shares own objects. He asks permission from his elders.

-Making sentence, Questions about the functions of objects are expected to be answered. For children (4) scenario (6) When shown the picture cards in order, they are expected to explain the scenario in sentences. Why? responds to questions such as how and whom, fulfills 3- or 4 consecutive instructions(Go to the closet, take the pen, put it on the table). Answers questions about short simple stories, Few sounds He talks about his mistakes, he talks about his experiences. Materials from which some objects are made knows. Distinguish objects of different textures. Establishes the relationship between objects and numbers. Same Matches a number of object groups. Combines two given halves to form a whole. in the space Follows location-related guidelines. Sorts events in the order in which they occur. The object, the person, Answers questions about pictures. Creates a story from the pictures shown, why answers questions that involve a result relationship. Differences in sounds (such as high, deep, thin) - Don't answer the question "What do you want?" Children(4) are expected to express their wishes. My feelings show. of others understands your feelings. It obeys the rules of regular games. Explains the rule of a game to someone else.

He directs the games and his friends. It fulfills its responsibility. a purpose for yourself determines and directs his actions accordingly. A certain event enlivens the situation. Self uses original ways of expressing Protects your rights. He tells about his daily experiences.

He uses predicate appropriate to the subject in his sentences. Asks the meanings of new and unknown words.

Knows the synonyms and opposite meanings of some words. Asks questions such as when, why, how. answers. (Fig. 7) (Fig. 8) Makes simple beards. Chooses clothes suitable for weather conditions. because then it uses conjunctions like

-Giving Opinion Children(4) are expected to use their reasoning abilities. (Figure 10) About stories They are expected to share their thoughts. Children (4) contrasts embedded in stories and they notice nonsense (like swimming in the sea). (Figure 12) What do you see, what do you hear, Questions such as what's in here are answered. Answers to questions such as how do you think this will be? comments are made. Details not told in stories are guessed at by reasoning (if wearing thick like it's cold). They are expected to take action against the conditions (if it is rainy, take an umbrella as he wanted)

Claims (1)

ISTEMLER l - Invention is an interactive artificial intelligence controlled education system. Its feature is; A cartoon character that allows the cartoon characters to act on the scenarios given to them based on the detection of speech and movements of children with artificial intelligence for the purpose of supporting the education of children with autism, attention deficit, hyperactivity disorder, developmental delay or psychological and neurological problems who require special support in normal development. 1), skeleton(2), model(3), child(4), artificial intelligence(5), scenario(6), subject detection(7), motion detection(8) and communication computer with microphone, camera and speaker , tablet, phone, etc. device(9). 2- It is the interactive artificial intelligence-controlled education system in Claim 1, and its feature is; The education of children with autism (4), from the beginner level, which includes more difficult exercises and exercises, based on the foundations of the previous level, to communicate, notice distant objects, make choices, make sentences, answer the question of what do you want and express an opinion. It has various scenarios like (6). 3 - Invention is an interactive artificial intelligence-controlled education system, and its feature is; - controlling the components such as the movements of the cartoon character(l), the environment in which the cartoon takes place, and the background with the systems in the invention, - designing the movements on the skeleton(2) by dressing the cartoon characters(1) on the skeleton(2), -The skeleton(2) ensuring that the model (3) on the skeleton (2) also moves when it starts to move, - providing interactive communication between the cartoon characters (1) and the child in front of them, - the cartoon character(l) according to the speech and movements of the child (4) in front of them. determining their own behaviors, - detecting the speech and movements of the child(4) with artificial intelligence(5), - using the system's voice processing, voice recognition and similar libraries and cloud and similar services that provide this service for the detection of speech(7), - getting caught Ensuring that the desired words are entered as data in voice processing, voice recognition and similar libraries and cloud and similar services that provide this service, Me, voice recognition and similar libraries and cloud and similar services that provide this service, the computer, tablet, phone, etc., where the program is running, with microphone, camera and speaker, etc., by interacting with the microphone of the device(9) and making sense of the incoming sounds - The program performs an operation according to the word when any of the related words are spoken, - The image processing, image recognition and similar libraries belonging to the system containing many libraries based on image processing for the detection of the child(4)'s movements(8) and the cloud and the cloud providing this service. providing face and eye detection with similar services, - detecting that the child(4) is against the application and in communication with the application, - image processing, image recognition and similar libraries belonging to the system and cloud and similar services providing this service and the child's face and eyes Detection of movements such as jumping and clapping as well as eye detection Ensuring that iti(8) is done, - includes the steps of the cartoon characters(l) to act on the scenarios(6) given to them based on the artificial intelligence(5) and the speech detection(7) of the children(7) and the detection(8) of their movements.