WO2024031203A1 - Méthode d'apprentissage qui utilise l'intelligence artificielle, basée sur un modèle de capture de mouvement/son et entrée de rétroalimentation - Google Patents

Méthode d'apprentissage qui utilise l'intelligence artificielle, basée sur un modèle de capture de mouvement/son et entrée de rétroalimentation Download PDF

Info

Publication number
WO2024031203A1
WO2024031203A1 PCT/CL2022/050079 CL2022050079W WO2024031203A1 WO 2024031203 A1 WO2024031203 A1 WO 2024031203A1 CL 2022050079 W CL2022050079 W CL 2022050079W WO 2024031203 A1 WO2024031203 A1 WO 2024031203A1
Authority
WO
WIPO (PCT)
Prior art keywords
artificial intelligence
activity
learning
learning method
uses artificial
Prior art date
Application number
PCT/CL2022/050079
Other languages
English (en)
Spanish (es)
Inventor
Christian BORQUEZ STEINFORT
Victor SAPIAÍN ARAYA
Erving Hidalgo Balboa
Original Assignee
Borquez Steinfort Christian
Sapiain Araya Victor
Erving Hidalgo Balboa
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Borquez Steinfort Christian, Sapiain Araya Victor, Erving Hidalgo Balboa filed Critical Borquez Steinfort Christian
Priority to PCT/CL2022/050079 priority Critical patent/WO2024031203A1/fr
Publication of WO2024031203A1 publication Critical patent/WO2024031203A1/fr

Links

Classifications

    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/103Detecting, measuring or recording devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
    • A61B5/11Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/24Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
    • A61B5/25Bioelectric electrodes therefor
    • A61B5/279Bioelectric electrodes therefor specially adapted for particular uses
    • A61B5/296Bioelectric electrodes therefor specially adapted for particular uses for electromyography [EMG]
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/24Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
    • A61B5/316Modalities, i.e. specific diagnostic methods
    • A61B5/389Electromyography [EMG]
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer

Definitions

  • the invention consists of a method that is based on different Artificial Intelligence techniques, which allows capturing movement and sound, and then packaging said information, processing it and emitting a modified output that allows feedback on variables of the captured activity. These variables are defined a priori by the user.
  • the method is based on a framework that offers a base structure that is capable of capturing, recording and analyzing video and sound images of the subject's motor behaviors by applying different techniques based on artificial intelligence, which delivers signals as output. and technological tools that together are capable of contributing to the learning, modeling and prediction of a certain motor activity.
  • the method converts the captures into electrical activities generated by the muscles when they perform some movement, similar to the captures in electromyography (EMG) sensors, but measured without the use of electrodes. It is also capable of capturing sounds and representing in vector form the positions and orientations of the subject's movements, postures and gestures. This is applicable, among others, to accelerate and massify the learning of instruments musical, accelerate mobility in cases of motor recovery, development of sports skills, motor learning in children, predict patterns for early detection of neuromotor diseases, detection of talents in motor activities, feed other robotic and/or virtual systems, improve motor performance of online game users, evaluate and predict movements of groups of subjects and also emulate techniques and movements of elite athletes or artists.
  • EMG electromyography
  • Learning is a field in which artificial intelligence has begun to make its way, basically with the aim of accelerating the skills of people who are learning a certain task, developing a skill or learning a language, among others.
  • Learning machines are part of artificial intelligence and have become the fundamental tool for making reliable decisions through the analysis of large amounts of data and facts. Machine learning allows the experience to be personalized to each user and allows them to accelerate their learning, reducing not only time, but also effort and frustrations.
  • Document CL 201801536 describes a system to detect people at suicidal risk that comprises: a structured questionnaire with a set of questions defined by a Bayesian predictive model of a person at suicidal risk; a central computer including a database with the responses of people to said structured questionnaire, said central computer is configured to execute said Bayesian predictive model on said database, said predictive model comprises a model with a machine learning and a Bayesian network with its nodes corresponding to the variables of said database, said variables are ranked using Bayesian inference, where said machine learning delivers a Bayesian predictive model with a precision of at least 0.7; and an additional terminal connected to said central computer configured to display structured information, where said structured information indicates an identification of a person at risk of suicide by highlighting a subset of the variable of said database with a higher hierarchy, some with respect to others of a subset of model variables, responses or data complementary of said person in said variables with higher hierarchy of said Bayesian predictive model.
  • a computer-implemented method includes accessing an image of a patient's tissue, accessing the control parameter values of a generator configured to provide power based on the control parameters, processing the tissue image and the values of the control parameters by an artificial intelligence learning system to provide an output related to the settings of the control parameters, providing an indication to a doctor based on the output where the indication indicates whether the values of the control parameters should be maintained , and providing adjusted control parameter values for the generator based on the output of the artificial intelligence learning system, if the indication is not to maintain the control parameter values.
  • US2021224441 A1 describes an apparatus for reducing an error of a physical model using an artificial intelligence algorithm.
  • the apparatus for reducing an error of a physical model includes: a modeling developer configured to derive a physical model of a process that includes error terms representing a modeling error, and a corrector configured to correct the physical model by deriving the error terms. error of the physical model using real data.
  • US2021 17301 1 A1 teaches a device and associated methods that relate to augmenting a device model identified by artificial intelligence, with measurements of physical parameters, iteratively validating and verifying the augmented model until the augmented model satisfies a given quality criterion based on artificial intelligence, and automatically synthesizing an interactive simulation and measurement environment, based on the model.
  • the model can be identified by artificial intelligence based on the measurement of an operating characteristic of the device.
  • the measurements of physical parameters with which the model is augmented can be determined by artificial intelligence, depending on the model.
  • the model can include a component, subsystem, and system model, allowing for validation and verification across multiple levels.
  • Various implementations can automatically generate a measurement scenario that includes communication commands configured to validate and verify the augmented model. Some designs may provide synthesized simulation visualization and measurement results generated based on the validated and verified augmented model.
  • Document US2010279762 A1 teaches an apparatus for adjusting the difficulty level of a game including: an artificial intelligence unit for storing artificial intelligence algorithms; a set of strategy tools to generate metadata for game resources and create game strategies by applying artificial intelligence algorithms using the metadata; and a set of simulation tools for calculating the relative difficulty levels of the game strategies and combinations of the game strategies and applying one of the game strategies and combinations of the game strategies to the game based on the skill level of the user determined during the game.
  • the present invention proposes a method and associated tools to achieve motor learning of a certain activity, massify said activity and detect talents early.
  • the method is based on a framework that offers a base structure that is capable of capturing movements and sounds through different techniques based on artificial intelligence, which delivers as output signals and technological tools that together are capable of contribute to the learning, modeling and prediction of a specific motor activity.
  • the method of the invention enables automatic learning and direct feedback to the user based on the progress that the user is making in his or her learning process.
  • the method will also allow movement predictions to be made, given the learning that will occur from putting the algorithm into practice. This will make it possible to detect talent early.
  • Figure 1 represents a diagram of the algorithm of the invention.
  • Figure 2 represents a general outline of the teaching method by Automatic learning and direct feedback.
  • Figure 3 represents an illustration when the method of the invention is applied in the development of sports skills, in this case in the contextualization of running.
  • Figure 4 represents an illustration when the method of the invention is applied in the development of skills to learn to play instruments, in this case in the contextualization of the activity of playing saxophone.
  • Figure 5 represents an illustration of the scheme for capturing characteristics of a sporting activity.
  • Figure 6 represents an illustration of the scheme for capturing characteristics of a hand mobility physical activity.
  • Figure 7 represents an illustration of the scheme for capturing characteristics of a musical activity playing piano.
  • Figure 8 represents an illustration of interaction of the algorithm of the method of the invention delivering signals to sensory receptors.
  • the method consists of a series of stages with the purpose of helping the learning of a motor activity, which allow an individual to be in a position to master an activity, through the use of artificial intelligence to interact in real time with each individual.
  • the method uses techniques based on Artificial Intelligence to capture, process and deliver feedback to the individual. In order to transfer to a user what the system captures. With the aim of being able to teach a motor activity. As well as being able to predict movements, associated with said activity.
  • the method includes the following stages: a) Motion and sound capture stage through Artificial Intelligence techniques. This stage involves capturing images, video and/or sounds and generating an Electromyogram data set. In certain conditions the visual/auditory medium is segmented to improve its analysis. b) Place a set of images/videos in a context. This identification is automatic and is delivered by an intelligent algorithm. c) Preprocessing, Segmentation and extraction of relevant attributes.
  • the method of the invention identifies the situation in which the individual develops. This condition is necessary to generate the most appropriate teaching steps. The teaching steps are part of the algorithm so it is associated with a library of multiple options of steps and resources that are applied depending on the response. of the individual.
  • the base algorithm is composed of Convolutional, Recurrent and Generative Adversarial Neural Networks. Which process the captures obtained from the activity and generate relevant characteristics of said activity. These characteristics are processed through artificial intelligence algorithms where learning methods more appropriate to the situation are generated. The latter using Machine Learning (ML) techniques e) The learning characteristics are represented by means of electromyographic (EMG) signals. Capturing the electrical signals produced by muscles during a muscle contraction is known as electromyography (EMG).
  • EMG electromyographic
  • stage d) the algorithm includes a series of stages that allow us to delve deeper into the individual's learning techniques and improve their movements to generate greater skills step by step. Namely, the algorithm contemplates for stage d):
  • CNN convolutional neural network
  • RNN recurrent neural network
  • Unimportant objects are discarded. The identification of these objects is through computer vision and they are placed in the context provided by the neural networks of the previous stage. Unimportant movements are those that have no impact on the activity on which the activity is being carried out.
  • EMG electromyogram
  • the learning characteristics are delivered to the user with the purpose of acquiring or reinforcing their knowledge. These learning characteristics are represented through graphical and/or mechanical interfaces such as screen visualization, exoskeletons, virtual reality glasses, among others.
  • the learning method is carried out for the development and improvement of a sports activity.
  • the method contemplates the following stages: a) A person wants to develop better sports skills. For this you need to know the correct way to perform the activity, in this particular case of running. b) A camera captures the movements of the person (user) and transmits them to an artificial vision system. c) An artificial vision system obtains images with sequences of the person's movements, extracts the most important elements from the environment, preprocesses them, packages them and sends them to an intelligent or artificial intelligence algorithm.
  • the AI algorithm recognizes and interprets the information received by the artificial vision system and, if necessary, subdivides the image into segments to contextualize the situation (Person running, playing a musical instrument, muscle rehabilitation, etc.).
  • the information from each segment of the image as well as the context information is processed and translated into electrical signals represented by the movement of the muscles or coordinates that represent the position of the upper and lower extremities.
  • the algorithm finally delivers relevant information (learning characteristics) such as electromyographic (EMG) signals, matrix representations with the posture of their limbs, color representation to indicate whether the movement is correct or optimal, indicators to help understand my performance compared to the optimal.
  • EMG electromyographic
  • the output information or learning characteristics are delivered to the user with the purpose of acquiring or reinforcing their knowledge.
  • step d) the image recognition was segmented into three images ( Figure 2), one of the upper extremity showing the runner's arm in motion with the position of his arm in 45 s facing above, being the left arm which has a built-in motion and heart rate sensor; A second image shows the upper extremity, specifically the runner's shoulder and his right arm at an angle of 45 s downward; and a third image that shows the runner's lower extremity, a leg moving in 45 s towards the front or advanced, where the algorithm recognizes that it is the right leg, the type of shoe and the clothing you wear.
  • the algorithm By analyzing the three images that correspond to segments of the image captured in stage b), the algorithm automatically recognizes that the activity being carried out is a sporting activity and is about running.
  • the algorithm Based on the analysis of the image segments, the algorithm provides a response to the user indicating which postures can be corrected, how to do it and whether the clothing used (shoes and clothing) can influence the performance of the activity.
  • the algorithm is based on a database with multiple kinesiological variables for the configuration and comparison of movements and on multiple variables of types of sports clothing, which is periodically updated with the latest technology in sports clothing that is coming onto the market. . Additionally, the algorithm identifies whether certain sports clothing may affect or not respond in the best way to sports activity, so it also provides feedback that serves as an improvement to the clothing manufacturers. If the algorithm interprets that there is a better clothing that best suits a specific user, it will provide feedback indicating existing options on the market.
  • Physical activity is also constantly monitored during a workout, so that it provides feedback to the user indicating whether they can speed up or slow down their pace in order to improve their performance without affecting their capacity or dosing the effort.
  • Figure 4 shows that from the global image captured by the artificial vision system, the global image captured is segmented into 5 images that They cover the upper area of the user, head and neck, upper left arm, upper right arm, lower left leg and lower right leg. All segments are fed to the algorithm in conjunction with signals sent by heart rate, oxygen level, movement and ambient sound sensors. These variables are analyzed and compared by the algorithm with respect to the database that feeds it and delivers an appropriate response to the particular user.
  • the algorithm engine allows the database to be populated with the information received so that the algorithm also learns as more users use the method of the invention, so that the decision spectrum for recommendations expands, generating an improvement. continuous on itself (Artificial Intelligence), also allowing to improve response time and posture correction and consequently the user's performance in the activity and thus accelerating the learning process.
  • the method is applied to learning to play a musical instrument, in this case a saxophone.
  • the image captured in stage b) is segmented into two images by the artificial vision system.
  • An image with the upper extremity showing the individual blowing and resting the fingers of both hands on the instrument.
  • a second image shows a segment dedicated to the particular instrument where the analysis is carried out on the posture of the fingers of both hands on the instrument.
  • Sound sensors capture the chords emitted by the instrument and additional sound sensors capture the noise that exists in the environment.
  • the agony analyzes the movement of the fingers of both arms, the way the lips are positioned on the mouthpiece to blow and the sound it makes.
  • the algorithm interprets that it is a wind instrument and from the sounds captured it determines that it is a saxophone, so it resorts in real time to the database where the information on the sounds of this type of instrument is loaded. , the techniques used for its execution and the different types of market instruments, thus beginning to make the comparison and providing feedback to the user regarding the positions they are adopting, the way in which they should be executed and the time in which they should do so.
  • oxygenation sensors are used to capture the user's inhalation and exhalation and compare them with the database that has the elements of breathing techniques for the execution of the instrument, so that it provides feedback to the user regarding the way they you are doing it and how you should correct it for better execution.
  • the algorithm uses a database with existing instruments on the market, it will also analyze whether the instrument chosen by the user adapts to the way they breathe or the physiognomy of their mouth, and can suggest everything from changing the mouthpiece to changing of the instrument with one that best suits its execution.
  • the algorithm by incorporating the user's execution parameters, can recommend improvements in the instrument itself, which can be recommended to the manufacturer.
  • the method of the invention is applied for the analysis and correction of the activity posture of a hand.
  • the artificial vision system captures an image of a hand holding a mouse and analyzes the way the hand is performing the action.
  • Associated motion sensors allow you to capture how your fingers, wrist and forearm move. These movements are analyzed physiologically in order to determine if the postures adopted can not only be improved, but also if the way in which the user is performing the activity can lead to trauma problems due to poor posture.
  • the algorithm will analyze the user's movements and compare them with the database and make correction suggestions and eventually generate alarm signals in case an injury is imminently visualized.
  • the therapist will learn the movements of the user's hand and will recommend the best alternative for postural changes in order to reach the appropriate posture for the activity as early as possible and thus avoid injuries or traumatological impediments. And of course achieve the desired learning.
  • the method is applied to the activity of playing a piano.
  • the algorithm will analyze the images of the posture of the hands, in this case on the piano keyboard, how the fingers move, the position of the arms, the pressure exerted on the keys and the sounds emitted by the piano. Based on the data collected, the algorithm analyzes the information and compares it with the data in the database, incorporates the data obtained and provides feedback to the user regarding the postures, the pressure on the keys and how the performance changes depending on the way the instrument is played, suggesting pertinent changes to improve the interpretation.

Landscapes

  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Engineering & Computer Science (AREA)
  • Medical Informatics (AREA)
  • Physics & Mathematics (AREA)
  • Veterinary Medicine (AREA)
  • Biophysics (AREA)
  • Pathology (AREA)
  • Public Health (AREA)
  • Biomedical Technology (AREA)
  • Heart & Thoracic Surgery (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Surgery (AREA)
  • Animal Behavior & Ethology (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Physiology (AREA)
  • Dentistry (AREA)
  • Image Analysis (AREA)

Abstract

La présente invention se rapporte à une méthode qui repose sur diverses techniques d'intelligence artificielle, ce qui permet de capturer le mouvement et le son, pour ensuite mettre en paquets cette information, la traiter et émettre une sortie modifiée et qui permet une rétroalimentation sur des variables de l'activité capturée. Ces variables sont définies à priori par l'utilisateur. La méthode repose sur un cadriciel de travail (Frameowrk) qui offre une structure de base qui est capable de capturer, d'enregistrer et d'analyser des images de vidéo et de son des comportements moteurs du sujet par l'application de diverses techniques basées sur l'intelligence artificielle, qui fournit comme sortie des signaux et des outils technologiques qui dans leur ensemble sont capables de contribuer à l'apprentissage, la modélisation et la prédiction d'une activité motrice déterminée.
PCT/CL2022/050079 2022-08-09 2022-08-09 Méthode d'apprentissage qui utilise l'intelligence artificielle, basée sur un modèle de capture de mouvement/son et entrée de rétroalimentation WO2024031203A1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/CL2022/050079 WO2024031203A1 (fr) 2022-08-09 2022-08-09 Méthode d'apprentissage qui utilise l'intelligence artificielle, basée sur un modèle de capture de mouvement/son et entrée de rétroalimentation

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CL2022/050079 WO2024031203A1 (fr) 2022-08-09 2022-08-09 Méthode d'apprentissage qui utilise l'intelligence artificielle, basée sur un modèle de capture de mouvement/son et entrée de rétroalimentation

Publications (1)

Publication Number Publication Date
WO2024031203A1 true WO2024031203A1 (fr) 2024-02-15

Family

ID=89850084

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CL2022/050079 WO2024031203A1 (fr) 2022-08-09 2022-08-09 Méthode d'apprentissage qui utilise l'intelligence artificielle, basée sur un modèle de capture de mouvement/son et entrée de rétroalimentation

Country Status (1)

Country Link
WO (1) WO2024031203A1 (fr)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6404925B1 (en) * 1999-03-11 2002-06-11 Fuji Xerox Co., Ltd. Methods and apparatuses for segmenting an audio-visual recording using image similarity searching and audio speaker recognition
US20100106044A1 (en) * 2008-10-27 2010-04-29 Michael Linderman EMG measured during controlled hand movement for biometric analysis, medical diagnosis and related analysis
US20100204616A1 (en) * 2006-01-09 2010-08-12 Applied Technology Holdings, Inc. Apparatus, systems, and methods for gathering and processing biometric and biomechanical data
EP3556429A1 (fr) * 2015-06-02 2019-10-23 Battelle Memorial Institute Système non invasif de rééducation de déficience motrice
US20210124420A1 (en) * 2019-10-29 2021-04-29 Hyundai Motor Company Apparatus and Method for Generating Image Using Brain Wave

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6404925B1 (en) * 1999-03-11 2002-06-11 Fuji Xerox Co., Ltd. Methods and apparatuses for segmenting an audio-visual recording using image similarity searching and audio speaker recognition
US20100204616A1 (en) * 2006-01-09 2010-08-12 Applied Technology Holdings, Inc. Apparatus, systems, and methods for gathering and processing biometric and biomechanical data
US20100106044A1 (en) * 2008-10-27 2010-04-29 Michael Linderman EMG measured during controlled hand movement for biometric analysis, medical diagnosis and related analysis
EP3556429A1 (fr) * 2015-06-02 2019-10-23 Battelle Memorial Institute Système non invasif de rééducation de déficience motrice
US20210124420A1 (en) * 2019-10-29 2021-04-29 Hyundai Motor Company Apparatus and Method for Generating Image Using Brain Wave

Similar Documents

Publication Publication Date Title
US11763697B2 (en) User interface system for movement skill analysis and skill augmentation
US20240041349A1 (en) Gait Analysis Devices, Methods, and Systems
US11508344B2 (en) Information processing device, information processing method and program
CN111902077B (zh) 利用神经肌肉信号进行手部状态表示建模的校准技术
US20230078968A1 (en) Systems and Methods for Monitoring and Evaluating Body Movement
US20220269346A1 (en) Methods and apparatuses for low latency body state prediction based on neuromuscular data
CN112005198A (zh) 基于多个输入的手部状态重建
US20150279231A1 (en) Method and system for assessing consistency of performance of biomechanical activity
US20220019284A1 (en) Feedback from neuromuscular activation within various types of virtual and/or augmented reality environments
US20210383714A1 (en) Information processing device, information processing method, and program
Houmanfar et al. Movement analysis of rehabilitation exercises: Distance metrics for measuring patient progress
KR20170128260A (ko) 다양하게 선택가능한 전문지식 내용을 포함하는 쌍방향 기능 훈련 내용의 공급을 가능하게 하는 구조, 장치 및 방법
KR20220028654A (ko) 미러 디스플레이를 이용하여 태권도 동작 코칭 서비스를 제공하는 장치 및 방법
Shi et al. Accurate and fast classification of foot gestures for virtual locomotion
CN116096289A (zh) 用于增强神经康复的系统和方法
WO2020084351A1 (fr) Systèmes et procédés d'évaluation et de mesure d'un temps de réaction dans une réalité virtuelle/augmentée
US20220351824A1 (en) Systems for dynamic assessment of upper extremity impairments in virtual/augmented reality
US20160175646A1 (en) Method and system for improving biomechanics with immediate prescriptive feedback
KR102425481B1 (ko) 재활치료를 위한 가상현실 커뮤니케이션 시스템
JP2021135995A (ja) アバター表情生成システム及びアバター表情生成方法
WO2024031203A1 (fr) Méthode d'apprentissage qui utilise l'intelligence artificielle, basée sur un modèle de capture de mouvement/son et entrée de rétroalimentation
US20190287261A1 (en) System, method, and apparatus to detect bio-mechanical geometry in a scene using machine vision for the application of a virtual goniometer
Vlutters Long short-term memory networks for body movement estimation
Monco From head to toe: body movement for human-computer interaction
KR102510048B1 (ko) 운동 동작에 따른 증강현실 데이터를 출력하는 전자 장치의 제어 방법

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22954184

Country of ref document: EP

Kind code of ref document: A1