WO2024031203A1 - Learning method using artificial intelligence, based on a motion/sound capture and feedback model - Google Patents

Learning method using artificial intelligence, based on a motion/sound capture and feedback model Download PDF

Info

Publication number
WO2024031203A1
WO2024031203A1 PCT/CL2022/050079 CL2022050079W WO2024031203A1 WO 2024031203 A1 WO2024031203 A1 WO 2024031203A1 CL 2022050079 W CL2022050079 W CL 2022050079W WO 2024031203 A1 WO2024031203 A1 WO 2024031203A1
Authority
WO
WIPO (PCT)
Prior art keywords
artificial intelligence
activity
learning
learning method
uses artificial
Prior art date
Application number
PCT/CL2022/050079
Other languages
Spanish (es)
French (fr)
Inventor
Christian BORQUEZ STEINFORT
Victor SAPIAÍN ARAYA
Erving Hidalgo Balboa
Original Assignee
Borquez Steinfort Christian
Sapiain Araya Victor
Erving Hidalgo Balboa
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Borquez Steinfort Christian, Sapiain Araya Victor, Erving Hidalgo Balboa filed Critical Borquez Steinfort Christian
Priority to PCT/CL2022/050079 priority Critical patent/WO2024031203A1/en
Publication of WO2024031203A1 publication Critical patent/WO2024031203A1/en

Links

Classifications

    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/103—Detecting, measuring or recording devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
    • A61B5/11—Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/24—Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
    • A61B5/25—Bioelectric electrodes therefor
    • A61B5/279—Bioelectric electrodes therefor specially adapted for particular uses
    • A61B5/296—Bioelectric electrodes therefor specially adapted for particular uses for electromyography [EMG]
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/24—Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
    • A61B5/316—Modalities, i.e. specific diagnostic methods
    • A61B5/389—Electromyography [EMG]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer

Definitions

  • the invention consists of a method that is based on different Artificial Intelligence techniques, which allows capturing movement and sound, and then packaging said information, processing it and emitting a modified output that allows feedback on variables of the captured activity. These variables are defined a priori by the user.
  • the method is based on a framework that offers a base structure that is capable of capturing, recording and analyzing video and sound images of the subject's motor behaviors by applying different techniques based on artificial intelligence, which delivers signals as output. and technological tools that together are capable of contributing to the learning, modeling and prediction of a certain motor activity.
  • the method converts the captures into electrical activities generated by the muscles when they perform some movement, similar to the captures in electromyography (EMG) sensors, but measured without the use of electrodes. It is also capable of capturing sounds and representing in vector form the positions and orientations of the subject's movements, postures and gestures. This is applicable, among others, to accelerate and massify the learning of instruments musical, accelerate mobility in cases of motor recovery, development of sports skills, motor learning in children, predict patterns for early detection of neuromotor diseases, detection of talents in motor activities, feed other robotic and/or virtual systems, improve motor performance of online game users, evaluate and predict movements of groups of subjects and also emulate techniques and movements of elite athletes or artists.
  • EMG electromyography
  • Learning is a field in which artificial intelligence has begun to make its way, basically with the aim of accelerating the skills of people who are learning a certain task, developing a skill or learning a language, among others.
  • Learning machines are part of artificial intelligence and have become the fundamental tool for making reliable decisions through the analysis of large amounts of data and facts. Machine learning allows the experience to be personalized to each user and allows them to accelerate their learning, reducing not only time, but also effort and frustrations.
  • Document CL 201801536 describes a system to detect people at suicidal risk that comprises: a structured questionnaire with a set of questions defined by a Bayesian predictive model of a person at suicidal risk; a central computer including a database with the responses of people to said structured questionnaire, said central computer is configured to execute said Bayesian predictive model on said database, said predictive model comprises a model with a machine learning and a Bayesian network with its nodes corresponding to the variables of said database, said variables are ranked using Bayesian inference, where said machine learning delivers a Bayesian predictive model with a precision of at least 0.7; and an additional terminal connected to said central computer configured to display structured information, where said structured information indicates an identification of a person at risk of suicide by highlighting a subset of the variable of said database with a higher hierarchy, some with respect to others of a subset of model variables, responses or data complementary of said person in said variables with higher hierarchy of said Bayesian predictive model.
  • a computer-implemented method includes accessing an image of a patient's tissue, accessing the control parameter values of a generator configured to provide power based on the control parameters, processing the tissue image and the values of the control parameters by an artificial intelligence learning system to provide an output related to the settings of the control parameters, providing an indication to a doctor based on the output where the indication indicates whether the values of the control parameters should be maintained , and providing adjusted control parameter values for the generator based on the output of the artificial intelligence learning system, if the indication is not to maintain the control parameter values.
  • US2021224441 A1 describes an apparatus for reducing an error of a physical model using an artificial intelligence algorithm.
  • the apparatus for reducing an error of a physical model includes: a modeling developer configured to derive a physical model of a process that includes error terms representing a modeling error, and a corrector configured to correct the physical model by deriving the error terms. error of the physical model using real data.
  • US2021 17301 1 A1 teaches a device and associated methods that relate to augmenting a device model identified by artificial intelligence, with measurements of physical parameters, iteratively validating and verifying the augmented model until the augmented model satisfies a given quality criterion based on artificial intelligence, and automatically synthesizing an interactive simulation and measurement environment, based on the model.
  • the model can be identified by artificial intelligence based on the measurement of an operating characteristic of the device.
  • the measurements of physical parameters with which the model is augmented can be determined by artificial intelligence, depending on the model.
  • the model can include a component, subsystem, and system model, allowing for validation and verification across multiple levels.
  • Various implementations can automatically generate a measurement scenario that includes communication commands configured to validate and verify the augmented model. Some designs may provide synthesized simulation visualization and measurement results generated based on the validated and verified augmented model.
  • Document US2010279762 A1 teaches an apparatus for adjusting the difficulty level of a game including: an artificial intelligence unit for storing artificial intelligence algorithms; a set of strategy tools to generate metadata for game resources and create game strategies by applying artificial intelligence algorithms using the metadata; and a set of simulation tools for calculating the relative difficulty levels of the game strategies and combinations of the game strategies and applying one of the game strategies and combinations of the game strategies to the game based on the skill level of the user determined during the game.
  • the present invention proposes a method and associated tools to achieve motor learning of a certain activity, massify said activity and detect talents early.
  • the method is based on a framework that offers a base structure that is capable of capturing movements and sounds through different techniques based on artificial intelligence, which delivers as output signals and technological tools that together are capable of contribute to the learning, modeling and prediction of a specific motor activity.
  • the method of the invention enables automatic learning and direct feedback to the user based on the progress that the user is making in his or her learning process.
  • the method will also allow movement predictions to be made, given the learning that will occur from putting the algorithm into practice. This will make it possible to detect talent early.
  • Figure 1 represents a diagram of the algorithm of the invention.
  • Figure 2 represents a general outline of the teaching method by Automatic learning and direct feedback.
  • Figure 3 represents an illustration when the method of the invention is applied in the development of sports skills, in this case in the contextualization of running.
  • Figure 4 represents an illustration when the method of the invention is applied in the development of skills to learn to play instruments, in this case in the contextualization of the activity of playing saxophone.
  • Figure 5 represents an illustration of the scheme for capturing characteristics of a sporting activity.
  • Figure 6 represents an illustration of the scheme for capturing characteristics of a hand mobility physical activity.
  • Figure 7 represents an illustration of the scheme for capturing characteristics of a musical activity playing piano.
  • Figure 8 represents an illustration of interaction of the algorithm of the method of the invention delivering signals to sensory receptors.
  • the method consists of a series of stages with the purpose of helping the learning of a motor activity, which allow an individual to be in a position to master an activity, through the use of artificial intelligence to interact in real time with each individual.
  • the method uses techniques based on Artificial Intelligence to capture, process and deliver feedback to the individual. In order to transfer to a user what the system captures. With the aim of being able to teach a motor activity. As well as being able to predict movements, associated with said activity.
  • the method includes the following stages: a) Motion and sound capture stage through Artificial Intelligence techniques. This stage involves capturing images, video and/or sounds and generating an Electromyogram data set. In certain conditions the visual/auditory medium is segmented to improve its analysis. b) Place a set of images/videos in a context. This identification is automatic and is delivered by an intelligent algorithm. c) Preprocessing, Segmentation and extraction of relevant attributes.
  • the method of the invention identifies the situation in which the individual develops. This condition is necessary to generate the most appropriate teaching steps. The teaching steps are part of the algorithm so it is associated with a library of multiple options of steps and resources that are applied depending on the response. of the individual.
  • the base algorithm is composed of Convolutional, Recurrent and Generative Adversarial Neural Networks. Which process the captures obtained from the activity and generate relevant characteristics of said activity. These characteristics are processed through artificial intelligence algorithms where learning methods more appropriate to the situation are generated. The latter using Machine Learning (ML) techniques e) The learning characteristics are represented by means of electromyographic (EMG) signals. Capturing the electrical signals produced by muscles during a muscle contraction is known as electromyography (EMG).
  • EMG electromyographic
  • stage d) the algorithm includes a series of stages that allow us to delve deeper into the individual's learning techniques and improve their movements to generate greater skills step by step. Namely, the algorithm contemplates for stage d):
  • CNN convolutional neural network
  • RNN recurrent neural network
  • Unimportant objects are discarded. The identification of these objects is through computer vision and they are placed in the context provided by the neural networks of the previous stage. Unimportant movements are those that have no impact on the activity on which the activity is being carried out.
  • EMG electromyogram
  • the learning characteristics are delivered to the user with the purpose of acquiring or reinforcing their knowledge. These learning characteristics are represented through graphical and/or mechanical interfaces such as screen visualization, exoskeletons, virtual reality glasses, among others.
  • the learning method is carried out for the development and improvement of a sports activity.
  • the method contemplates the following stages: a) A person wants to develop better sports skills. For this you need to know the correct way to perform the activity, in this particular case of running. b) A camera captures the movements of the person (user) and transmits them to an artificial vision system. c) An artificial vision system obtains images with sequences of the person's movements, extracts the most important elements from the environment, preprocesses them, packages them and sends them to an intelligent or artificial intelligence algorithm.
  • the AI algorithm recognizes and interprets the information received by the artificial vision system and, if necessary, subdivides the image into segments to contextualize the situation (Person running, playing a musical instrument, muscle rehabilitation, etc.).
  • the information from each segment of the image as well as the context information is processed and translated into electrical signals represented by the movement of the muscles or coordinates that represent the position of the upper and lower extremities.
  • the algorithm finally delivers relevant information (learning characteristics) such as electromyographic (EMG) signals, matrix representations with the posture of their limbs, color representation to indicate whether the movement is correct or optimal, indicators to help understand my performance compared to the optimal.
  • EMG electromyographic
  • the output information or learning characteristics are delivered to the user with the purpose of acquiring or reinforcing their knowledge.
  • step d) the image recognition was segmented into three images ( Figure 2), one of the upper extremity showing the runner's arm in motion with the position of his arm in 45 s facing above, being the left arm which has a built-in motion and heart rate sensor; A second image shows the upper extremity, specifically the runner's shoulder and his right arm at an angle of 45 s downward; and a third image that shows the runner's lower extremity, a leg moving in 45 s towards the front or advanced, where the algorithm recognizes that it is the right leg, the type of shoe and the clothing you wear.
  • the algorithm By analyzing the three images that correspond to segments of the image captured in stage b), the algorithm automatically recognizes that the activity being carried out is a sporting activity and is about running.
  • the algorithm Based on the analysis of the image segments, the algorithm provides a response to the user indicating which postures can be corrected, how to do it and whether the clothing used (shoes and clothing) can influence the performance of the activity.
  • the algorithm is based on a database with multiple kinesiological variables for the configuration and comparison of movements and on multiple variables of types of sports clothing, which is periodically updated with the latest technology in sports clothing that is coming onto the market. . Additionally, the algorithm identifies whether certain sports clothing may affect or not respond in the best way to sports activity, so it also provides feedback that serves as an improvement to the clothing manufacturers. If the algorithm interprets that there is a better clothing that best suits a specific user, it will provide feedback indicating existing options on the market.
  • Physical activity is also constantly monitored during a workout, so that it provides feedback to the user indicating whether they can speed up or slow down their pace in order to improve their performance without affecting their capacity or dosing the effort.
  • Figure 4 shows that from the global image captured by the artificial vision system, the global image captured is segmented into 5 images that They cover the upper area of the user, head and neck, upper left arm, upper right arm, lower left leg and lower right leg. All segments are fed to the algorithm in conjunction with signals sent by heart rate, oxygen level, movement and ambient sound sensors. These variables are analyzed and compared by the algorithm with respect to the database that feeds it and delivers an appropriate response to the particular user.
  • the algorithm engine allows the database to be populated with the information received so that the algorithm also learns as more users use the method of the invention, so that the decision spectrum for recommendations expands, generating an improvement. continuous on itself (Artificial Intelligence), also allowing to improve response time and posture correction and consequently the user's performance in the activity and thus accelerating the learning process.
  • the method is applied to learning to play a musical instrument, in this case a saxophone.
  • the image captured in stage b) is segmented into two images by the artificial vision system.
  • An image with the upper extremity showing the individual blowing and resting the fingers of both hands on the instrument.
  • a second image shows a segment dedicated to the particular instrument where the analysis is carried out on the posture of the fingers of both hands on the instrument.
  • Sound sensors capture the chords emitted by the instrument and additional sound sensors capture the noise that exists in the environment.
  • the agony analyzes the movement of the fingers of both arms, the way the lips are positioned on the mouthpiece to blow and the sound it makes.
  • the algorithm interprets that it is a wind instrument and from the sounds captured it determines that it is a saxophone, so it resorts in real time to the database where the information on the sounds of this type of instrument is loaded. , the techniques used for its execution and the different types of market instruments, thus beginning to make the comparison and providing feedback to the user regarding the positions they are adopting, the way in which they should be executed and the time in which they should do so.
  • oxygenation sensors are used to capture the user's inhalation and exhalation and compare them with the database that has the elements of breathing techniques for the execution of the instrument, so that it provides feedback to the user regarding the way they you are doing it and how you should correct it for better execution.
  • the algorithm uses a database with existing instruments on the market, it will also analyze whether the instrument chosen by the user adapts to the way they breathe or the physiognomy of their mouth, and can suggest everything from changing the mouthpiece to changing of the instrument with one that best suits its execution.
  • the algorithm by incorporating the user's execution parameters, can recommend improvements in the instrument itself, which can be recommended to the manufacturer.
  • the method of the invention is applied for the analysis and correction of the activity posture of a hand.
  • the artificial vision system captures an image of a hand holding a mouse and analyzes the way the hand is performing the action.
  • Associated motion sensors allow you to capture how your fingers, wrist and forearm move. These movements are analyzed physiologically in order to determine if the postures adopted can not only be improved, but also if the way in which the user is performing the activity can lead to trauma problems due to poor posture.
  • the algorithm will analyze the user's movements and compare them with the database and make correction suggestions and eventually generate alarm signals in case an injury is imminently visualized.
  • the therapist will learn the movements of the user's hand and will recommend the best alternative for postural changes in order to reach the appropriate posture for the activity as early as possible and thus avoid injuries or traumatological impediments. And of course achieve the desired learning.
  • the method is applied to the activity of playing a piano.
  • the algorithm will analyze the images of the posture of the hands, in this case on the piano keyboard, how the fingers move, the position of the arms, the pressure exerted on the keys and the sounds emitted by the piano. Based on the data collected, the algorithm analyzes the information and compares it with the data in the database, incorporates the data obtained and provides feedback to the user regarding the postures, the pressure on the keys and how the performance changes depending on the way the instrument is played, suggesting pertinent changes to improve the interpretation.

Landscapes

  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Engineering & Computer Science (AREA)
  • Medical Informatics (AREA)
  • Physics & Mathematics (AREA)
  • Veterinary Medicine (AREA)
  • Biophysics (AREA)
  • Pathology (AREA)
  • Public Health (AREA)
  • Biomedical Technology (AREA)
  • Heart & Thoracic Surgery (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Surgery (AREA)
  • Animal Behavior & Ethology (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Physiology (AREA)
  • Dentistry (AREA)
  • Image Analysis (AREA)

Abstract

The invention relates to a method based on different artificial intelligence techniques, which can be used to capture motion and sound, and then to package this information, process it and issue a modified output that allows feedback on variables of the captured activity. These variables are defined a priori by the user. The method is based on a framework that provides a base structure capable of capturing, recording and analyzing video and sound images of a person's motor behaviors by applying different techniques based on artificial intelligence, outputting signals, and technological tools which together can contribute to learning, modeling and prediction associated with a given motor activity.

Description

MÉTODO DE APRENDIZAJE QUE UTILIZA INTELIGENCIALEARNING METHOD THAT USES INTELLIGENCE
ARTIFICIAL, BASADO EN UN MODELO DE CAPTURA DE MOVIMIENTO/SONIDO Y ENTREGA DE RETROA LIME NT ACIÓN. ARTIFICIAL, BASED ON A MOTION/SOUND CAPTURE MODEL AND LIME NT ATION FEEDBACK DELIVERY.
RESUMEN SUMMARY
La invención consiste en un método que se sustenta en distintas técnicas de Inteligencia Artificial, el cual permite capturar movimiento y sonido, para luego empaquetar dicha información, procesarla y emitir una salida modificada y que permita una retroalimentación sobre variables de la actividad capturada. Estas variables son definidas a priori por el usuario. The invention consists of a method that is based on different Artificial Intelligence techniques, which allows capturing movement and sound, and then packaging said information, processing it and emitting a modified output that allows feedback on variables of the captured activity. These variables are defined a priori by the user.
El método se basa en un marco de trabajo (Framework) que ofrece una estructura base que es capaz de capturar, registrar y analizar imágenes de video y sonido de las conductas motoras del sujeto aplicando distintas técnicas basadas en inteligencia artificial, que entrega como salida señales y herramientas tecnológicas que en su conjunto son capaces de contribuir al aprendizaje, modelamiento y predicción de una determinada actividad motora.The method is based on a framework that offers a base structure that is capable of capturing, recording and analyzing video and sound images of the subject's motor behaviors by applying different techniques based on artificial intelligence, which delivers signals as output. and technological tools that together are capable of contributing to the learning, modeling and prediction of a certain motor activity.
El método convierte las capturas en actividades eléctricas generadas por los músculos cuando realizan algún movimiento, similar a las capturas en sensores de electromiografía (EMG), pero medidos sin el uso de electrodos. También es capaz de capturar sonidos y de representar en forma vectorial las posiciones y orientaciones de los movimientos, posturas y gestos del sujeto. Esto es aplicable, entre otros, para acelerar y masificar el aprendizaje de instrumentos musicales, acelerar la movilidad en casos de recuperación motora, desarrollo de habilidades deportivas, aprendizaje motor en niños, predecir patrones para detección temprana de enfermedades neuromotoras, detección de talentos en actividades motrices, alimentar a otros sistemas robóticos y/o virtuales, mejorar desempeño motor de usuarios de juegos en línea, evaluar y predecir movimientos de grupos de sujetos y también emular técnicas y movimientos de deportistas o artistas de elite. The method converts the captures into electrical activities generated by the muscles when they perform some movement, similar to the captures in electromyography (EMG) sensors, but measured without the use of electrodes. It is also capable of capturing sounds and representing in vector form the positions and orientations of the subject's movements, postures and gestures. This is applicable, among others, to accelerate and massify the learning of instruments musical, accelerate mobility in cases of motor recovery, development of sports skills, motor learning in children, predict patterns for early detection of neuromotor diseases, detection of talents in motor activities, feed other robotic and/or virtual systems, improve motor performance of online game users, evaluate and predict movements of groups of subjects and also emulate techniques and movements of elite athletes or artists.
A R T E P R E V I O A R T E P R E V I O
Cada día es más masivo el uso de inteligencia artificial para facilitar y acelerar las interacciones entre los usuarios de diferentes tipos de plataformas y sistemas y los centros de control y respuesta ante sus requerimientos. Se ha difundido su uso para la intreacción en la web, especialmente con lo que dice relación con recomendaciones personalizadas para los consumidores, basadas, por ejemplo, en sus búsquedas y compras previas o en otros comportamientos en línea. A través del conocimiento que adquiere una determinada red que trabaja bajo el esquema de inteligencia artificial, es posible mejorar no solo la experiencia de compra, sino adecuar dicha experiencia a la reacción que en el tiempo van teniendo los usuarios. Every day the use of artificial intelligence is becoming more widespread to facilitate and accelerate interactions between users of different types of platforms and systems and the control and response centers to their requirements. Its use for interaction on the web has become widespread, especially in relation to personalized recommendations for consumers, based, for example, on their previous searches and purchases or on other online behaviors. Through the knowledge acquired by a certain network that works under the artificial intelligence scheme, it is possible to improve not only the purchasing experience, but also adapt said experience to the reaction that users have over time.
El aprendizaje es un campo en el cual la inteligencia artificial ha comenzado a abrirse espacio, básicamente con la finalidad de acelerar las habilidades de las personas que están aprendiendo una determinada tarea, desarrollando una habilidad o aprendiendo un idioma, entre otros. Las máquinas de aprendizaje (maching learning), son parte de la inteligencia artificial y se han convertido en la herramienta fundamental para la toma de decisiones fiables a través del análisis de grandes cantidades de datos y hechos. El aprendizaje automático permite adecuar la expereincia en forma personalizada a cada usuario y permite acelerar su aprendizaje disminuyendo no solo el tiempo, sino también esfuerzo y frustraciones. Learning is a field in which artificial intelligence has begun to make its way, basically with the aim of accelerating the skills of people who are learning a certain task, developing a skill or learning a language, among others. Learning machines (maching learning), are part of artificial intelligence and have become the fundamental tool for making reliable decisions through the analysis of large amounts of data and facts. Machine learning allows the experience to be personalized to each user and allows them to accelerate their learning, reducing not only time, but also effort and frustrations.
El documento CL 201801536 describe un sistema para detectar personas con riesgo suicida que comprende: un cuestionario estructurado con un conjunto de preguntas definidas mediante un modelo predictivo bayesiano de persona con riesgo suicida; un ordenador central incluyendo una base de datos con las respuestas de personas a dicho cuestionario estructurado, dicho ordenador central es configurado para ejecutar dicho modelo predictivo bayesiano sobre dicha base de datos, dicho modelo predictivo comprende un modelo con un aprendizaje de máquina y una red bayesiana con sus nodos correspondientes a las variables de dicha base de datos, dichas variables son jerarquizadas mediante inferencia bayesiana, en donde dicho aprendizaje de máquina entrega un modelo predictivo bayesiano con una precisión de al menos 0,7; y un terminal adicional conectado a dicho ordenador central configurado para mostrar información estructurada, en donde dicha información estructurada indica una identificación de una persona con riesgo suicida destacando un subconjunto de variable de dicha base de datos con una jerarquía superior, unas respecto a otras de un subconjunto de variables del modelo, las respuestas o datos complementarios de dicha persona en dichas variables con jerarquía superior de dicho modelo predictivo bayesiano. Document CL 201801536 describes a system to detect people at suicidal risk that comprises: a structured questionnaire with a set of questions defined by a Bayesian predictive model of a person at suicidal risk; a central computer including a database with the responses of people to said structured questionnaire, said central computer is configured to execute said Bayesian predictive model on said database, said predictive model comprises a model with a machine learning and a Bayesian network with its nodes corresponding to the variables of said database, said variables are ranked using Bayesian inference, where said machine learning delivers a Bayesian predictive model with a precision of at least 0.7; and an additional terminal connected to said central computer configured to display structured information, where said structured information indicates an identification of a person at risk of suicide by highlighting a subset of the variable of said database with a higher hierarchy, some with respect to others of a subset of model variables, responses or data complementary of said person in said variables with higher hierarchy of said Bayesian predictive model.
El documento WO2021133549 A1 enseña procedimientos quirúrgicos basados en energía. Un método implementado por computadora incluye acceder a una imagen del tejido de un paciente, acceder a los valores de los parámetros de control de un generador configurado para proporcionar energía en función de los parámetros de control, procesar la imagen del tejido y los valores de los parámetros de control por un sistema de aprendizaje de inteligencia artificial para proporcionar una salida relacionada con la configuración de los parámetros de control, proporcionando una indicación a un médico basada en la salida donde la indicación indica si se deben mantener los valores de los parámetros de control, y proporcionando valores de parámetros de control ajustados para el generador basado en la salida del sistema de aprendizaje de inteligencia artificial, si la indicación es no mantener los valores de los parámetros de control. Document WO2021133549 A1 teaches energy-based surgical procedures. A computer-implemented method includes accessing an image of a patient's tissue, accessing the control parameter values of a generator configured to provide power based on the control parameters, processing the tissue image and the values of the control parameters by an artificial intelligence learning system to provide an output related to the settings of the control parameters, providing an indication to a doctor based on the output where the indication indicates whether the values of the control parameters should be maintained , and providing adjusted control parameter values for the generator based on the output of the artificial intelligence learning system, if the indication is not to maintain the control parameter values.
El documento US2021224441 A1 describe un aparato para reducir un error de un modelo físico utilizando un algoritmo de inteligencia artificial. El aparato para reducir un error de un modelo físico incluye: un deñvador de modelado configurado para derivar un modelo físico de un proceso que incluye términos de error que representan un error de modelado, y un corrector configurado para corregir el modelo físico derivando los términos de error del modelo físico utilizando datos reales. El documento US2021 17301 1 A1 enseña un dispositivo y métodos asociados que se relacionan con aumentar un modelo de dispositivo identificado por inteligencia artificial, con mediciones de parámetros físicos, validando y verificando iterativamente el modelo aumentado hasta que el modelo aumentado satisface un criterio de calidad determinado en función de la inteligencia artificial, y sintetizando automáticamente un interactivo entorno de simulación y medida, basado en el modelo. El modelo puede ser identificado por la inteligencia artificial en base a la medición de una característica operativa del dispositivo. Las mediciones de parámetros físicos con las que se aumenta el modelo pueden ser determinadas por la inteligencia artificial, en función del modelo. El modelo puede incluir un modelo de componente, subsistema y sistema, lo que permite la validación y verificación a través de múltiples niveles. Varias implementaciones pueden generar automáticamente un escenario de medición que incluye comandos de comunicación configurados para validar y verificar el modelo aumentado. Algunos diseños pueden proporcionar visualización de simulación sintetizada y resultados de medición generados en función del modelo aumentado validado y verificado. US2021224441 A1 describes an apparatus for reducing an error of a physical model using an artificial intelligence algorithm. The apparatus for reducing an error of a physical model includes: a modeling developer configured to derive a physical model of a process that includes error terms representing a modeling error, and a corrector configured to correct the physical model by deriving the error terms. error of the physical model using real data. US2021 17301 1 A1 teaches a device and associated methods that relate to augmenting a device model identified by artificial intelligence, with measurements of physical parameters, iteratively validating and verifying the augmented model until the augmented model satisfies a given quality criterion based on artificial intelligence, and automatically synthesizing an interactive simulation and measurement environment, based on the model. The model can be identified by artificial intelligence based on the measurement of an operating characteristic of the device. The measurements of physical parameters with which the model is augmented can be determined by artificial intelligence, depending on the model. The model can include a component, subsystem, and system model, allowing for validation and verification across multiple levels. Various implementations can automatically generate a measurement scenario that includes communication commands configured to validate and verify the augmented model. Some designs may provide synthesized simulation visualization and measurement results generated based on the validated and verified augmented model.
El documento US2010279762 A1 enseña un aparato para ajustar el nivel de dificultad de un juego incluye: una unidad de inteligencia artificial para almacenar algoritmos de inteligencia artificial; un conjunto de herramientas de estrategia para generar metadatos para los recursos del juego y crear estrategias de juego mediante la aplicación de algoritmos de inteligencia artificial utilizando los metadatas; y un juego de herramientas de simulación para calcular los niveles de dificultad relativa de las estrategias de juego y combinaciones de las estrategias de juego y aplicar al juego una de las estrategias de juego y las combinaciones de las estrategias de juego basándose en el nivel de habilidad del usuario determinado durante el juego. Document US2010279762 A1 teaches an apparatus for adjusting the difficulty level of a game including: an artificial intelligence unit for storing artificial intelligence algorithms; a set of strategy tools to generate metadata for game resources and create game strategies by applying artificial intelligence algorithms using the metadata; and a set of simulation tools for calculating the relative difficulty levels of the game strategies and combinations of the game strategies and applying one of the game strategies and combinations of the game strategies to the game based on the skill level of the user determined during the game.
Se puede observar que en los documentros citados y en general en todos los que se pueden obtener del estado de la técnica que el uso de inteligencia artificial está aplicado a dar respuesta a las diferentes reacciones que generan los individuos frente a preguntas, estímulos, manejo frente a diversas situaciones y habilidades, las cuales son de alguna forma medidas por un motor de inteligencia artificial que reconoce los razgos específicos de lo que se está midiendo y genera una respuesta adecuada al usuario en cuestión, pudiendo incluso generar señales de alarma en algunos casos. It can be seen that in the cited documents and in general in all those that can be obtained from the state of the art that the use of artificial intelligence is applied to respond to the different reactions that individuals generate when faced with questions, stimuli, management. to various situations and skills, which are in some way measured by an artificial intelligence engine that recognizes the specific features of what is being measured and generates an appropriate response to the user in question, and can even generate alarm signals in some cases.
La presente invención propone un método y herramientas asociadas para lograr el aprendizaje motor de una determinada actividad, masificar dicha actividad y detectar talentos tempranamente. The present invention proposes a method and associated tools to achieve motor learning of a certain activity, massify said activity and detect talents early.
El método se basa en un marco de trabajo (Framework) que ofrece una estructura base que es capaz de capturar movimientos y sonidos a través de distintas técnicas basadas en inteligencia artificial, que entrega como salida señales y herramientas tecnológicas que en su conjunto son capaces de contribuir al aprendizaje, modelamiento y predicción de una determinada actividad motora. The method is based on a framework that offers a base structure that is capable of capturing movements and sounds through different techniques based on artificial intelligence, which delivers as output signals and technological tools that together are capable of contribute to the learning, modeling and prediction of a specific motor activity.
Estas metodologías y herramientas tecnológicas son aplicadas para acelerar y masificar el aprendizaje de instrumentos musicales, acelerar la movilidad en casos de recuperación motora, desarrollo de habilidades deportivas y también emular técnicas y movimientos de deportistas o artistas de elite. These methodologies and technological tools are applied to accelerate and massify the learning of musical instruments, accelerate mobility in cases of motor recovery, development of sports skills and also emulate techniques and movements of elite athletes or artists.
A diferencia de los documentos del estado de la técnica, el método de la invención posibilita el aprendizaje automático y retroalimentación directa al usuario basado en los avances que éste va teniendo en su proceso de aprendizaje. Unlike the documents of the state of the art, the method of the invention enables automatic learning and direct feedback to the user based on the progress that the user is making in his or her learning process.
También el método permitirá realizar predicciones de movimientos, dado el aprendizaje que tendrá por la puesta en práctica del algoritmo. Esto posibilitará detectar talentos tempranamente. The method will also allow movement predictions to be made, given the learning that will occur from putting the algorithm into practice. This will make it possible to detect talent early.
B R EV E D E SC R I PC I O N D E LAS F I G U RAS B R EV E OF SC R I PC I O N OF THE FIGURES
Figura 1 : representa un diagrama del algoritmo de la invención. Figure 1: represents a diagram of the algorithm of the invention.
Figura 2: representa un esquema general del método de enseñanza por aprendizaje Automático y retroalimentación directa. Figure 2: represents a general outline of the teaching method by Automatic learning and direct feedback.
Figura 3: representa una ilustración cuando el método de la invención se aplica en el desarrollo de habilidades deportivas, en este caso en la contextualizacion de correr (running). Figura 4: representa una ilustración cuando el método de la invención se aplica en el desarrollo de habilidades para aprender a tocar instrumentos, en este caso en la contextualización de la actividad de tocar saxofón. Figure 3: represents an illustration when the method of the invention is applied in the development of sports skills, in this case in the contextualization of running. Figure 4: represents an illustration when the method of the invention is applied in the development of skills to learn to play instruments, in this case in the contextualization of the activity of playing saxophone.
Figura 5: representa una ilustración del esquema para la captura de características de una actividad deportiva. Figure 5: represents an illustration of the scheme for capturing characteristics of a sporting activity.
Figura 6: representa una ilustración del esquema para la captura de características de una actividad física movilidad de mano. Figure 6: represents an illustration of the scheme for capturing characteristics of a hand mobility physical activity.
Figura 7: representa una ilustración del esquema para la captura de características de una actividad musical tocar piano. Figure 7: represents an illustration of the scheme for capturing characteristics of a musical activity playing piano.
Figura 8: representa una ilustración de intreacción del algoritmo del método de la invención entregando señales a receptores sensoriales. Figure 8: represents an illustration of interaction of the algorithm of the method of the invention delivering signals to sensory receptors.
D E S C R I P C I O N D E LA I N V E N C I O N D E S C R I P T I O N OF THE I N V E N T I O N
El método consiste en una serie de etapas con la finalidad de ayudar al aprendizaje de una actividad motora, que permiten dejar a un individuo en posición de dominar una actividad, mediante la utilización de inteligencia artificial para interactuar en tiempo real con cada individuo. El método utiliza técnicas basadas en Inteligencia Artificial para captar, procesar y entregar retroalimentación al individuo. Con el fin de transferir a un usuario lo que el sistema capta. Con el objetivo de poder enseñar una actividad motora. Como también poder predecir movimientos, asociados a dicha actividad. The method consists of a series of stages with the purpose of helping the learning of a motor activity, which allow an individual to be in a position to master an activity, through the use of artificial intelligence to interact in real time with each individual. The method uses techniques based on Artificial Intelligence to capture, process and deliver feedback to the individual. In order to transfer to a user what the system captures. With the aim of being able to teach a motor activity. As well as being able to predict movements, associated with said activity.
El método contempla las siguientes etapas: a) Etapa de captura de movimiento y sonido por medio de técnicas de Inteligencia Artificial. Esta etapa implica capturar imágenes, video y/o sonidos y generar una data set de Electromiogramas. En ciertas condiciones el medio visual/auditivo es segmentado para mejorar su análisis. b) Situar un conjunto de imágenes/videos en un contexto. Esta identificación es automática y es entregada por un algoritmo inteligente. c) Preprocesamiento, Segmentación y extracción de atributos relevantes. El método de la invención identifica en la situación que se desenvuelve el individuo. Esta condición es necesaria para generar los pasos de enseñanza más adecuado. Los pasos de enseñanza forman parte del algoritmo por lo que éste está asociado a una biblioteca de multiples opciones de pasos y recursos que se aplican dependiendo de la respuesta del individuo. Con el fin de determinar los pasos más adecuados para un individuo, la imagen global captada es segmentada en otras imágenes, donde se extraen las características más relevantes de cada segmento y donde se categorizan. Este proceso de categohzación continua hasta obtener un contexto general. d) El algoritmo base se compone de Redes Neuronales Convolucionales, Recurrentes y Generativas Antagónicas. Las cuales procesan las capturas obtenidas de la actividad y generan características relevantes de dicha actividad. Estas características son procesadas por medio de algoritmos de inteligencia artificial donde se generan métodos de aprendizaje más adecuados a la situación. Esto último empleando técnicas de Machine Learning (ML) e) Las características de aprendizaje son representadas por medio de señales electromiográficas (EMG). La captación de las señales eléctricas producidas por los músculos durante una contracción muscular se conoce como electromiografía (EMG). Estas señales son generadas por el intercambio de iones a través de las membranas de las fibras musculares debido a una contracción muscular. Los sensores que se utilizan son adosados al individuo y luego conectados al sistema en donde se pone en práctica el método. Estos conformarán las salidas del sistema y los canales para la entrega de retroalimentación. f) Estas características son enviadas al usuario para generar retroalimentación y poder mejorar sus habilidades en la ejecución de la actividad. Estas características suelen ser apoyadas por herramientas tecnológicas. Donde se pueden citar sensores, exoesqueletos, lentes de realidad virtual, etc. The method includes the following stages: a) Motion and sound capture stage through Artificial Intelligence techniques. This stage involves capturing images, video and/or sounds and generating an Electromyogram data set. In certain conditions the visual/auditory medium is segmented to improve its analysis. b) Place a set of images/videos in a context. This identification is automatic and is delivered by an intelligent algorithm. c) Preprocessing, Segmentation and extraction of relevant attributes. The method of the invention identifies the situation in which the individual develops. This condition is necessary to generate the most appropriate teaching steps. The teaching steps are part of the algorithm so it is associated with a library of multiple options of steps and resources that are applied depending on the response. of the individual. In order to determine the most appropriate steps for an individual, the global image captured is segmented into other images, where the most relevant characteristics of each segment are extracted and where they are categorized. This categorization process continues until a general context is obtained. d) The base algorithm is composed of Convolutional, Recurrent and Generative Adversarial Neural Networks. Which process the captures obtained from the activity and generate relevant characteristics of said activity. These characteristics are processed through artificial intelligence algorithms where learning methods more appropriate to the situation are generated. The latter using Machine Learning (ML) techniques e) The learning characteristics are represented by means of electromyographic (EMG) signals. Capturing the electrical signals produced by muscles during a muscle contraction is known as electromyography (EMG). These signals are generated by the exchange of ions across the membranes of muscle fibers due to muscle contraction. The sensors used are attached to the individual and then connected to the system where the method is put into practice. These will make up the outputs of the system and the channels for delivering feedback. f) These characteristics are sent to the user to generate feedback and improve their skills in executing the activity. These features are usually supported by tools technological. Where sensors, exoskeletons, virtual reality glasses, etc. can be mentioned.
En la etapa d) el algoritmo comprende una serie de etapas que permiten profundizar en las técnicas de aprendizaje del individuo y mejorar sus movimientos para ir generando paso a paso mayores habilidades. A saber, al algoritmo contempla para la etapa d): In stage d) the algorithm includes a series of stages that allow us to delve deeper into the individual's learning techniques and improve their movements to generate greater skills step by step. Namely, the algorithm contemplates for stage d):
• que las imágenes capturadas se suavicen, eliminen el ruido y en ciertas condiciones el medio visual/auditivo es segmentado para mejorar su análisis. • that the captured images are smoothed, noise eliminated and in certain conditions the visual/auditory medium is segmented to improve its analysis.
• Para situar un conjunto de imágenes/videos en un contexto se utiliza una red neuronal convolucional (CNN) y red neuronal recurrente (RNN). Estas redes pueden identificar ciertos patrones para relacionarlos a una actividad. Esto en base a la carga de modelos pre entrenados. • To place a set of images/videos in a context, a convolutional neural network (CNN) and recurrent neural network (RNN) are used. These networks can identify certain patterns to relate them to an activity. This is based on the loading of pre-trained models.
• Identificación de objetos más importantes de las imágenes. Se descartan objetos no importantes. La identificación de estos objetos es a través de visión por computadora y se sitúan en el contexto entregado por las redes neuronales de la etapa anterior. Los movimientros no importantes son aquellos que no tienen incidencia en la actividad sobre la cual se está desarrollando la actividad. • Identification of the most important objects in the images. Unimportant objects are discarded. The identification of these objects is through computer vision and they are placed in the context provided by the neural networks of the previous stage. Unimportant movements are those that have no impact on the activity on which the activity is being carried out.
• Generador de poses y gestos sintéticos con redes generativas antagónicas (GAN). Esto es importante para que la caracterización de gestos sea más precisa. • Synthetic pose and gesture generator with generative adversarial networks (GAN). This is important to make gesture characterization more accurate.
• Se generan señales eléctricas representada por los músculos cuando existe una actividad o gesto. Esto se denomina electromiograma (EMG). La EMG se determina por medio de una red convolucional que pronostica cual es la señal asociada a un movimiento y las imágenes generadas por el generador de poses y gestos sintéticos por GAN. En el pronóstico de la EMG se utiliza sólo un sistema de Visión como lo es una cámara de video. • Electrical signals represented by the muscles are generated when there is an activity or gesture. This is called an electromyogram (EMG). The EMG is determined through a convolutional network that predicts the signal associated with a movement and the images generated by the generator of synthetic poses and gestures by GAN. In EMG prognosis, only a Vision system such as a video camera is used.
• Representaciones matñciales con la postura, posición y ángulos de sus extremidades. • Material representations with the posture, position and angles of their extremities.
• Determinar por medio de algoritmo de machine Learning (ML) la postura o gestos óptimos. Esto se realiza en base a la identificación de patrones sobre modelos pre entrenados con base de conocimiento de la actividad. • Determine the optimal posture or gestures through machine learning (ML) algorithms. This is done based on the identification of patterns on pre-trained models based on knowledge of the activity.
• Las características de aprendizaje son entregadas al usuario con la finalidad que adquieran o refuercen sus conocimientos. Estas características de aprendizaje son representadas por medio de interfaz gráfica y/o mecánica como lo son visualización en pantalla, exoesqueletos, lentes de realidad virtual, entre otras. • The learning characteristics are delivered to the user with the purpose of acquiring or reinforcing their knowledge. These learning characteristics are represented through graphical and/or mechanical interfaces such as screen visualization, exoskeletons, virtual reality glasses, among others.
DESCRIPCION DE LAS MODALIDADES PREFERIDAS DE EJECUCION DESCRIPTION OF PREFERRED EXECUTION MODALITIES
Con referencia a la figura 2, el método de aprendizaje se lleva a cabo para el desarrollo y mejoramiento de una actividad deportiva. En esta modalidad el método contempla las siguientes etapas: a) Una persona desea desarrollar mejores habilidades deportivas. Para esto necesita saber cuál es la forma correcta para realizar la actividad, en este caso particular de correr. b) Una cámara capta los movimientos de la persona (usuario) y las transmite hacia un sistema de visión artificial. c) Un sistema de visión artificial obtiene las imágenes con secuencias de movimientos de la persona, extrae los elementos más importantes del ambiente, los preprocesa, empaqueta y los envía hacia un algoritmo inteligente o de inteligencia artificial. d) El algoritmo IA es reconoce e interpreta la información recibida por el sistema de visión artificial y si es necesario subdivide la imagen por segmentos para contextualizar la situación (Persona corriendo, tocando un instrumento musical, rehabilitación muscular, etc.,). e) La información de cada segmento de la imagen como también la información del contexto es procesada y son traducidas en señales eléctricas representadas por el movimiento de los músculos o coordenadas que representan la posición de las extremidades superiores e inferiores. f) El algoritmo finalmente entrega información relevante (características de aprendizaje) como señales electromiográficas (EMG), representaciones matriciales con la postura de sus extremidades, representación en colores para indicar si el movimiento es el correcto u óptimo, indicadores para ayudar a comprender mi rendimiento en comparación al optimo. g) La información de salida o características de aprendizaje son entregadas al usuario con la finalidad que adquieran o refuercen sus conocimientos para una actividad que en este caso es correr. Estas características de aprendizaje pueden ser apoyadas por dispositivos o herramientas tecnológicas como los son exoesqueletos para las extremidades que le indiquen al usuario de qué manera debe de realizar la actividad, lentes de realidad virtual (VR), monitor de imágenes, entre otras. g) Estas características de aprendizaje se van monitoreando en el tiempo con la finalidad de determinar cómo ha sido el avance. La información de progreso es enviada al usuario en forma de retroalimentación o feedback a través de herramientas tecnológicas. Entre estas se pueden citar sensores, exoesqueletos, lentes de realidad virtual, etc. With reference to Figure 2, the learning method is carried out for the development and improvement of a sports activity. In this modality, the method contemplates the following stages: a) A person wants to develop better sports skills. For this you need to know the correct way to perform the activity, in this particular case of running. b) A camera captures the movements of the person (user) and transmits them to an artificial vision system. c) An artificial vision system obtains images with sequences of the person's movements, extracts the most important elements from the environment, preprocesses them, packages them and sends them to an intelligent or artificial intelligence algorithm. d) The AI algorithm recognizes and interprets the information received by the artificial vision system and, if necessary, subdivides the image into segments to contextualize the situation (Person running, playing a musical instrument, muscle rehabilitation, etc.). e) The information from each segment of the image as well as the context information is processed and translated into electrical signals represented by the movement of the muscles or coordinates that represent the position of the upper and lower extremities. f) The algorithm finally delivers relevant information (learning characteristics) such as electromyographic (EMG) signals, matrix representations with the posture of their limbs, color representation to indicate whether the movement is correct or optimal, indicators to help understand my performance compared to the optimal. g) The output information or learning characteristics are delivered to the user with the purpose of acquiring or reinforcing their knowledge. for an activity that in this case is running. These learning characteristics can be supported by technological devices or tools such as exoskeletons for the extremities that tell the user how to perform the activity, virtual reality (VR) glasses, image monitors, among others. g) These learning characteristics are monitored over time in order to determine how progress has been. Progress information is sent to the user in the form of feedback through technological tools. These include sensors, exoskeletons, virtual reality glasses, etc.
Un ejemplo de esto es que la persona en el tiempo mejore su técnica para correr, pero la retroalimentación que entrega el algoritmo IA indica que debe mover sus brazos hacia adelante y hacia atrás, y colocar los brazos en ángulos de 90°, para así mejorar en su técnica. An example of this is that the person improves their running technique over time, but the feedback provided by the AI algorithm indicates that they must move their arms forward and backward, and place their arms at 90° angles, in order to improve. in his technique.
En esta modalidad preferehda de ejecución, en el paso d) el reconocimiento de la imagen se segmentó en tres imágenes (Figura 2), una de la extremidad superior que muestra el brazo del corredor en movimiento con la posición de su brazo en 45s hacia arriba, siendo el brazo izquierdo el cual tiene incorporado un sensor de movimiento y ritmo cardiaco; una segunda imagen enseña la extremidad superior, específicamente el hombro del corredor y su brazo derecho en un ángulo de 45s hacia abajo; y una tercera imagen que muestra la extremidad inferior del corredor, una pierna en movimiento en 45s hacia el frente o avanzado, en donde el algoritmo reconoce que se trata de la pierna derecha, el tipo de zapatilla y la vestimenta que lleva. Al analizar las tres imágenes que corresponden a segmentos de la imagen captada en la etapa b), el algoritmo de forma automática reconoce que la actividad que se está desarrollando es una actividad deportiva y se trata de correr. In this preferred execution modality, in step d) the image recognition was segmented into three images (Figure 2), one of the upper extremity showing the runner's arm in motion with the position of his arm in 45 s facing above, being the left arm which has a built-in motion and heart rate sensor; A second image shows the upper extremity, specifically the runner's shoulder and his right arm at an angle of 45 s downward; and a third image that shows the runner's lower extremity, a leg moving in 45 s towards the front or advanced, where the algorithm recognizes that it is the right leg, the type of shoe and the clothing you wear. By analyzing the three images that correspond to segments of the image captured in stage b), the algorithm automatically recognizes that the activity being carried out is a sporting activity and is about running.
A partir del análisis de los segmentos de imágenes, el algoritmo entrega una respuesta al usuario indicando qué posturas pueden ser corregidas, de qué forma hacerlo y si la indumentaria que utiliza (zapatillas y vestimenta) puede influir en el rendimiento de la actividad. Para ello, el algoritmo se apoya en una base de datos con múltiples variables kinesiológicas para la configuración y comparación de movimientos y en múltiples variables de tipos de indumentaria deportiva, la cual es periódicamente actualizada con la última tecnología en indumentaria deportiva que va saliendo al mercado. Adicionalmente, el algoritmo identifica si cierta indumentaria deportiva puede afectar o no responder de la mejor forma a la actividad deportiva, por lo que también entrega una retroalimentación que sirve de mejora a los fabricantes de la indumentaria. En caso que el algoritmo interprete que hay una mejor indumentaria que se adapte mejor a un usuario específico, lo retroalimentará indicando opciones existentes en el mercado. Based on the analysis of the image segments, the algorithm provides a response to the user indicating which postures can be corrected, how to do it and whether the clothing used (shoes and clothing) can influence the performance of the activity. To do this, the algorithm is based on a database with multiple kinesiological variables for the configuration and comparison of movements and on multiple variables of types of sports clothing, which is periodically updated with the latest technology in sports clothing that is coming onto the market. . Additionally, the algorithm identifies whether certain sports clothing may affect or not respond in the best way to sports activity, so it also provides feedback that serves as an improvement to the clothing manufacturers. If the algorithm interprets that there is a better clothing that best suits a specific user, it will provide feedback indicating existing options on the market.
La actividad física igualmente esta siendo monitoreada constantemente durante un entrenamiento, de modo que retroalimenta al usuario indicando si puede acelerar o disminuir su ritmo con el fin de mejorar su rendimiento sin afectar su capacidad o dosificando el esfuerzo. Physical activity is also constantly monitored during a workout, so that it provides feedback to the user indicating whether they can speed up or slow down their pace in order to improve their performance without affecting their capacity or dosing the effort.
En la figura 4 se observa que a partir de la imagen global captada por sistema de visión artificial, la imagen global captada se segmenta en 5 imágenes que abarcan la zona superior del usuario, cabeza y cuello, extremo superior brazo izquierdo, extremo superior brazo derecho, extremo inferior pierna izquierda y extremo inferior pierna derecha. Todos los segmentos son alimentados al algoritmo en conjunto con las señales enviadas por sensores de ritmo cardiaco, nivel de oxígeno, movimiento y sonido ambiente. Estas variables son analizadas y comparadas por el algoritmo respecto de la base de datos que lo alimenta y entrega una respuesta adecuada al usuario en particular. El motor del algoritmo permite poblar la base de datos con la información recibida de modo que el algoritmo también aprende a medida que más usuarios hacen uso del método de la invención, de manera que el espectro de decisión para las recomendaciones se va ampliando generando una mejora continua sobre sí mismo (Inteligencia Artificial), permitiendo de igual modo mejorar el tiempo de respuesta y la corrección de posturas y consecuentemente el rendimiento del usuario en la actividad y acelerando de esta forma el proceso de aprendizaje. Figure 4 shows that from the global image captured by the artificial vision system, the global image captured is segmented into 5 images that They cover the upper area of the user, head and neck, upper left arm, upper right arm, lower left leg and lower right leg. All segments are fed to the algorithm in conjunction with signals sent by heart rate, oxygen level, movement and ambient sound sensors. These variables are analyzed and compared by the algorithm with respect to the database that feeds it and delivers an appropriate response to the particular user. The algorithm engine allows the database to be populated with the information received so that the algorithm also learns as more users use the method of the invention, so that the decision spectrum for recommendations expands, generating an improvement. continuous on itself (Artificial Intelligence), also allowing to improve response time and posture correction and consequently the user's performance in the activity and thus accelerating the learning process.
Con referencia a la figura 3, el método es aplicado al aprendizaje de ejecutar un instrumento musical, en este caso un saxofón. La imagen captada en la etapa b) es segmentada en dos imágenes por el sistema de visión artificial. Una imagen con la extremidad superior que muetra al individuo soplando y apoyando los dedos de ambas manos en el instrumento. Una segunda imagen muestra un segmento dedicado al instrumento en particular en donde el análisis se efectúa en la postura de los dedos de ambas manos en el instrumento. Sensores de sonido captan los acordes emitidos por el instrumemto y sensores adicionales de sonido captan el ruido que existe en el entorno. El agoñtmo analiza el movimento de los dedos de los dedos de ambos brazos, la forma en que los labios se posicionan en la boquilla para soplar y el sonido que emite. El agoritmo interpreta que se trata de instrumento de viento y por los sonidos captados determina que se trata de un saxofón, por lo que recurre en tiempo real a la base de datos en donde se encuentra cargada la información de los sonidos de este tipo de instrumento, las técnicas utilizadas para su ejecución y los diferentes tipos de instrumentos del mercado, comenzando de esta forma a efectuar la comparación y retroalimenta al usuario respecto a las posturas que está adoptando, la forma en que debiera ejecutarlas y el tiempo en que debe hacerlo. En esta modalidad, sensores de oxigenación son utilizados para captar la inhalación y exhalación del usuario y los compara con la base de datos que posee los elementos de técnicas de respiración para la ejecución del instrumento, de modo que retroalimenta al usuario respecto de la forma que lo está haciendo y cómo debe corregirlo para una mejor ejecución. Adicionalemente, como el algoritmo utiliza una base de datos con los instrumentos existentes en el mercado, también analizará si el instrumento escogido por el usuario se adapta a su forma de respirar o fisonomía de su boca, pudiendo sugerir desde el cambio de boquilla hasta el cambio del instrumento por uno que se adecúe mejor a su ejecucuón. With reference to Figure 3, the method is applied to learning to play a musical instrument, in this case a saxophone. The image captured in stage b) is segmented into two images by the artificial vision system. An image with the upper extremity showing the individual blowing and resting the fingers of both hands on the instrument. A second image shows a segment dedicated to the particular instrument where the analysis is carried out on the posture of the fingers of both hands on the instrument. Sound sensors capture the chords emitted by the instrument and additional sound sensors capture the noise that exists in the environment. The agony analyzes the movement of the fingers of both arms, the way the lips are positioned on the mouthpiece to blow and the sound it makes. The algorithm interprets that it is a wind instrument and from the sounds captured it determines that it is a saxophone, so it resorts in real time to the database where the information on the sounds of this type of instrument is loaded. , the techniques used for its execution and the different types of market instruments, thus beginning to make the comparison and providing feedback to the user regarding the positions they are adopting, the way in which they should be executed and the time in which they should do so. In this modality, oxygenation sensors are used to capture the user's inhalation and exhalation and compare them with the database that has the elements of breathing techniques for the execution of the instrument, so that it provides feedback to the user regarding the way they you are doing it and how you should correct it for better execution. Additionally, as the algorithm uses a database with existing instruments on the market, it will also analyze whether the instrument chosen by the user adapts to the way they breathe or the physiognomy of their mouth, and can suggest everything from changing the mouthpiece to changing of the instrument with one that best suits its execution.
De la misma forma, el algoritmo al ir incorporando los parámetros de ejecucuón del usuario, puede recomendar mejora en el instrumento propiamente tal, lo cual puede ser recomendado al fabricante. Con referencia a la figura 5, el método de la invención es aplicado para el análisis y corrección de la postura de la actividad de una mano. El sistema de visión artificial capta la imagen de una mano sosteniendo un mouse y analiza la forma en que la mano está efectuando la acción. Sensores de movimiento asociados permiten captar cómo se mueven los dedos, la muñeca y el antebrazo. Dichos movimiento son analizados fisiológicamente de modo de determinar si las posturas adoptadas no sólo pueden ser mejoradas, sino además si la manera en que el usuario está realizando la actividad puede derivar en problemas traumatológicos por mala postura. En este caso el algoritmo analizará los movimientos del usuario y los comparará con la base de datos y hará las suguerencias de corrección y eventualmente genererá señales de alarma en caso que una lesión se visualice inmimente. El agoñtmo aprenderá los movimientos de la mano del usuario y recomendará la mejor alternativa de cambios posturales con la finalidad de llegar lo mas temprano posible a la postura adecuada de la actividad y así evitar lesiones o impedimentos traumatológicos. Y por supuesto lograr el aprendizaje deseado. In the same way, the algorithm, by incorporating the user's execution parameters, can recommend improvements in the instrument itself, which can be recommended to the manufacturer. With reference to Figure 5, the method of the invention is applied for the analysis and correction of the activity posture of a hand. The artificial vision system captures an image of a hand holding a mouse and analyzes the way the hand is performing the action. Associated motion sensors allow you to capture how your fingers, wrist and forearm move. These movements are analyzed physiologically in order to determine if the postures adopted can not only be improved, but also if the way in which the user is performing the activity can lead to trauma problems due to poor posture. In this case, the algorithm will analyze the user's movements and compare them with the database and make correction suggestions and eventually generate alarm signals in case an injury is imminently visualized. The therapist will learn the movements of the user's hand and will recommend the best alternative for postural changes in order to reach the appropriate posture for the activity as early as possible and thus avoid injuries or traumatological impediments. And of course achieve the desired learning.
Con referencia a la figura 6, el método es aplicado a la actividad de la ejecución de un piano. Al igual que en el caso del saxofón antes descrito, el algoritmo analizará las imágenes de la postura de las manos, en este caso sobre el teclado del piano, como se mueven los dedos, la posición de los brazos, la presión ejercida sobre las teclas y los sonidos emitidos por el piano. A partir de los datos captados, el algoritmo analiza la información y lo compara con los datos de la base datos, incorpora los datos obtenidos y retroalimenta al usuario en cuanto a las posturas, la presión en las teclas y cómo cambia la ejecucuón dependiendo de la forma en que se toca el instrumento, sugiriendo los cambios pertinentes para mejorar la interpretación. Referring to Figure 6, the method is applied to the activity of playing a piano. As in the case of the saxophone described above, the algorithm will analyze the images of the posture of the hands, in this case on the piano keyboard, how the fingers move, the position of the arms, the pressure exerted on the keys and the sounds emitted by the piano. Based on the data collected, the algorithm analyzes the information and compares it with the data in the database, incorporates the data obtained and provides feedback to the user regarding the postures, the pressure on the keys and how the performance changes depending on the way the instrument is played, suggesting pertinent changes to improve the interpretation.

Claims

R E I V I N D I CA C I O N E S Método de aprendizaje que utiliza inteligencia artificial que capta movimientos y sonidos a través de técnicas de Inteligencia Artificial, que permiten aprender una determinada actividad motora, acortar los tiempos de aprendizaje, que permiten dejar a un individuo en posición de dominar una actividad, CARACTERIZADO porque comprende las siguientes etapas: a. capturar imágenes y audio por medio de técnicas de Inteligencia Artificial; b. situar las imágenes/videos en un contexto de modo de identificar en forma automática la actividad por un algoritmo inteligente, en donde dicho algoritmo genera el almcenamiento de la actividad; c. preprocesar, segmentar y extraer atributos de la actividad identificada; d. procesar las capturas de imágenes y sonidos obtenidas de la actividad y asociar características relevantes de la actividad previamente cargadas en una base datos asociadas al algoritmo; e. representar las características de aprendizaje por medio de señales electromiográficas (EMG) y empaquetar dicha data; f. retroalimentar al usuario para generar información que permite al usuario mejorar sus habilidades en la ejecución de la actividad. g. predecir movimientos a partir del aprendizaje obtenido por el algoritmo. Método de aprendizaje que utiliza inteligencia artificial de acuerdo a la reivindicación 1 , CARACTERIZADO porque la etapa de captura de imágenes y sonidos, generan una respuesta correctiva de una versión mejorada de la captura. Método de aprendizaje que utiliza inteligencia artificial de acuerdo a la reivindicación 2, CARACTERIZADO porque comprende segmentar las imágenes y sonidos capatados para su análisis. Método de aprendizaje que utiliza inteligencia artificial de acuerdo a la reivindicación 1 , CARACTERIZADO porque comprende identificar la situación en que se desenvuelve el individuo y luego generar pasos de enseñanza al usuario. Método de aprendizaje que utiliza inteligencia artificial de acuerdo a la reivindicación 4, CARACTERIZADO porque dichos pasos de enseñanza forman parte de dicho algoritmo, el cual a su vez se asocia a una base de datos que consiste en una biblioteca de múltiples opciones de pasos y recursos que se aplican dependiendo de la respuesta del individuo. Método de aprendizaje que utiliza inteligencia artificial de acuerdo a la reivindicación 2, CARACTERIZADO porque dichos segmentos obtenidos desde una imagen global captada, generar características de cada segmento y las categoñza hasta obtener un contexto general. Método de aprendizaje que utiliza inteligencia artificial de acuerdo a la reivindicación 6, CARACTERIZADO porque dichas características son procesadas por medio de algoritmos de inteligencia artificial donde se generan métodos de aprendizaje adecuados a la situación de la actividad. Método de aprendizaje que utiliza inteligencia artificial de acuerdo a la reivindicación 6, CARACTERIZADO porque comprende representar dichas características de aprendizaje por medio de colores. Método de aprendizaje que utiliza inteligencia artificial de acuerdo a la reivindicación 6, CARACTERIZADO porque comprende representar dichas características de aprendizaje por medio de indicadores de mejoras en la ejecución de una actividad. Método de aprendizaje que utiliza inteligencia artificial de acuerdo a la reivindicación 1 , CARACTERIZADO porque comprende aplicar sensores adosados al usuario y luego conectados a un sistema en donde se pone en práctica el método. Método de aprendizaje que utiliza inteligencia artificial de acuerdo a la reivindicación 10, CARACTERIZADO porque dichos sensores son sensores de ritmo cardiaco Método de aprendizaje que utiliza inteligencia artificial de acuerdo a la reivindicación 10, CARACTERIZADO porque dichos sensores son sensores de nivel de oxigeno en el usuario. Método de aprendizaje que utiliza inteligencia artificial de acuerdo a la reivindicación 10, CARACTERIZADO porque dichos sensores son sensores de movimiento. Método de aprendizaje que utiliza inteligencia artificial de acuerdo a la reivindicación 10, CARACTERIZADO porque dichos sensores son sensores de sonido corporal. Método de aprendizaje que utiliza inteligencia artificial de acuerdo a la reivindicación 1 , CARACTERIZADO porque comprende que dichas imágenes/videos se contextualicen mediante una red neuronal convolucional (CNN) y red neuronal recurrente (RNN). Método de aprendizaje que utiliza inteligencia artificial de acuerdo a la reivindicación 15, CARACTERIZADO porque comprende indentificar de objetos más importantes de las imágenes, en donde se descartan objetos no importantes, por medio de la identificación de estos objetos través de visión por computadora y se sitúan en el contexto entregado por dichas redes neuronales, en donde los movimientros no importantes son aquellos que no tienen incidencia en la actividad sobre la cual se está desarrollando la actividad. Método de aprendizaje que utiliza inteligencia artificial de acuerdo a la reivindicación 16, CARACTERIZADO porque comprende generar poses y gestos sintéticos con redes generativas antagónicas (GAN). CLAIMS Learning method that uses artificial intelligence that captures movements and sounds through Artificial Intelligence techniques, which allow learning a certain motor activity, shorten learning times, which allow an individual to be in a position to master an activity, CHARACTERIZED because it includes the following stages: a. capture images and audio through Artificial Intelligence techniques; b. place the images/videos in a context so as to automatically identify the activity by an intelligent algorithm, where said algorithm generates the storage of the activity; c. preprocess, segment and extract attributes of the identified activity; d. process the image and sound captures obtained from the activity and associate relevant characteristics of the activity previously loaded into a database associated with the algorithm; and. represent learning characteristics through electromyographic (EMG) signals and package said data; F. provide feedback to the user to generate information that allows the user to improve their skills in executing the activity. g. predict movements from the learning obtained by the algorithm. Learning method that uses artificial intelligence according to claim 1, CHARACTERIZED because the image and sound capture stage generates a corrective response of an improved version of the capture. Learning method that uses artificial intelligence according to claim 2, CHARACTERIZED because it comprises segmenting the images and sounds captured for analysis. Learning method that uses artificial intelligence according to claim 1, CHARACTERIZED because it includes identifying the situation in which the individual operates and then generating teaching steps for the user. Learning method that uses artificial intelligence according to claim 4, CHARACTERIZED because said teaching steps are part of said algorithm, which in turn is associated with a database that consists of a library of multiple options of steps and resources which are applied depending on the individual's response. Learning method that uses artificial intelligence according to claim 2, CHARACTERIZED because said segments obtained from a global image captured, generate characteristics of each segment and categorize them until obtaining a general context. Learning method that uses artificial intelligence according to claim 6, CHARACTERIZED because said characteristics are processed by means of artificial intelligence algorithms where learning methods appropriate to the situation of the activity are generated. Learning method that uses artificial intelligence according to claim 6, CHARACTERIZED because it comprises representing said learning characteristics by means of colors. Learning method that uses artificial intelligence according to claim 6, CHARACTERIZED because it comprises representing said learning characteristics through indicators of improvements in the execution of an activity. Learning method that uses artificial intelligence according to claim 1, CHARACTERIZED because it comprises applying sensors attached to the user and then connected to a system where the method is put into practice. Learning method that uses artificial intelligence according to claim 10, CHARACTERIZED because said sensors are heart rate sensors Learning method that uses artificial intelligence according to claim 10, CHARACTERIZED because said sensors are oxygen level sensors in the user . Learning method that uses artificial intelligence according to claim 10, CHARACTERIZED because said sensors are motion sensors. Learning method that uses artificial intelligence according to claim 10, CHARACTERIZED because said sensors are body sound sensors. Learning method that uses artificial intelligence according to claim 1, CHARACTERIZED because it comprises that said images/videos are contextualized by a convolutional neural network (CNN) and recurrent neural network (RNN). Learning method that uses artificial intelligence according to claim 15, CHARACTERIZED because it comprises identifying the most important objects in the images, where unimportant objects are discarded, by identifying these objects through computer vision and placing them in the context delivered by said neural networks, where non-important movements are those that have no impact on the activity on which the activity is being developed. Learning method that uses artificial intelligence according to claim 16, CHARACTERIZED because it comprises generating synthetic poses and gestures with generative adversarial networks (GAN).
PCT/CL2022/050079 2022-08-09 2022-08-09 Learning method using artificial intelligence, based on a motion/sound capture and feedback model WO2024031203A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/CL2022/050079 WO2024031203A1 (en) 2022-08-09 2022-08-09 Learning method using artificial intelligence, based on a motion/sound capture and feedback model

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CL2022/050079 WO2024031203A1 (en) 2022-08-09 2022-08-09 Learning method using artificial intelligence, based on a motion/sound capture and feedback model

Publications (1)

Publication Number Publication Date
WO2024031203A1 true WO2024031203A1 (en) 2024-02-15

Family

ID=89850084

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CL2022/050079 WO2024031203A1 (en) 2022-08-09 2022-08-09 Learning method using artificial intelligence, based on a motion/sound capture and feedback model

Country Status (1)

Country Link
WO (1) WO2024031203A1 (en)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6404925B1 (en) * 1999-03-11 2002-06-11 Fuji Xerox Co., Ltd. Methods and apparatuses for segmenting an audio-visual recording using image similarity searching and audio speaker recognition
US20100106044A1 (en) * 2008-10-27 2010-04-29 Michael Linderman EMG measured during controlled hand movement for biometric analysis, medical diagnosis and related analysis
US20100204616A1 (en) * 2006-01-09 2010-08-12 Applied Technology Holdings, Inc. Apparatus, systems, and methods for gathering and processing biometric and biomechanical data
EP3556429A1 (en) * 2015-06-02 2019-10-23 Battelle Memorial Institute Non-invasive motor impairment rehabilitation system
US20210124420A1 (en) * 2019-10-29 2021-04-29 Hyundai Motor Company Apparatus and Method for Generating Image Using Brain Wave

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6404925B1 (en) * 1999-03-11 2002-06-11 Fuji Xerox Co., Ltd. Methods and apparatuses for segmenting an audio-visual recording using image similarity searching and audio speaker recognition
US20100204616A1 (en) * 2006-01-09 2010-08-12 Applied Technology Holdings, Inc. Apparatus, systems, and methods for gathering and processing biometric and biomechanical data
US20100106044A1 (en) * 2008-10-27 2010-04-29 Michael Linderman EMG measured during controlled hand movement for biometric analysis, medical diagnosis and related analysis
EP3556429A1 (en) * 2015-06-02 2019-10-23 Battelle Memorial Institute Non-invasive motor impairment rehabilitation system
US20210124420A1 (en) * 2019-10-29 2021-04-29 Hyundai Motor Company Apparatus and Method for Generating Image Using Brain Wave

Similar Documents

Publication Publication Date Title
US11367364B2 (en) Systems and methods for movement skill analysis and skill augmentation
US11508344B2 (en) Information processing device, information processing method and program
CN111902077B (en) Calibration technique for hand state representation modeling using neuromuscular signals
US20200000373A1 (en) Gait Analysis Devices, Methods, and Systems
US11521326B2 (en) Systems and methods for monitoring and evaluating body movement
US20220269346A1 (en) Methods and apparatuses for low latency body state prediction based on neuromuscular data
US20150279231A1 (en) Method and system for assessing consistency of performance of biomechanical activity
US20220019284A1 (en) Feedback from neuromuscular activation within various types of virtual and/or augmented reality environments
US20210383714A1 (en) Information processing device, information processing method, and program
Houmanfar et al. Movement analysis of rehabilitation exercises: Distance metrics for measuring patient progress
KR20170129716A (en) A structure, apparatus and method for providing bi-directional functional training content including provision of adaptive training programs based on performance sensor data
KR20220028654A (en) Apparatus and method for providing taekwondo movement coaching service using mirror dispaly
Shi et al. Accurate and fast classification of foot gestures for virtual locomotion
WO2020084351A1 (en) Systems and methods for assessment and measurement of reaction time in virtual/augmented reality
US20220351824A1 (en) Systems for dynamic assessment of upper extremity impairments in virtual/augmented reality
CN116096289A (en) Systems and methods for enhancing neurological rehabilitation
KR20140043174A (en) Simulator for horse riding and method for simulation of horse riding
US20160175646A1 (en) Method and system for improving biomechanics with immediate prescriptive feedback
KR102425481B1 (en) Virtual reality communication system for rehabilitation treatment
JP2021135995A (en) Avatar facial expression generating system and avatar facial expression generating method
WO2024031203A1 (en) Learning method using artificial intelligence, based on a motion/sound capture and feedback model
Vlutters Long short-term memory networks for body movement estimation
Monco From head to toe: body movement for human-computer interaction
KR102510048B1 (en) Control method of electronic device to output augmented reality data according to the exercise motion
US20220365605A1 (en) System and method for learning or re-learning a gesture

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22954184

Country of ref document: EP

Kind code of ref document: A1