FR3137203A1 - SYSTEM AND METHOD OF ASSISTANCE IN PERFORMING PHYSICAL MOVEMENTS - Google Patents

Abstract

SYSTEM AND METHOD FOR ASSISTANCE IN PERFORMING PHYSICAL EXERCISES The invention relates to a method and a system for detecting a physical posture and generating an instruction for assisting in carrying out physical exercises and correcting posture. posture which includes posture detection and the generation of instructions helping a user to perform a physical exercise. Figure for abstract: Fig.1

Description

SYSTEM AND METHOD OF ASSISTANCE IN PERFORMING PHYSICAL MOVEMENTS Field of the invention

The invention relates to the field of methods of assisting the performance of physical movements. More particularly, the invention relates to the field of methods comprising the use of an image or a plurality of images of an individual performing physical exercise in order to analyze the correct performance thereof and to carry out monitoring of user performance.

State of the art

Currently, there is a need to assist people wishing to start physical training without being able to call on a trainer. There is also a need for more experienced athletes wishing to have a guide during sports sessions that they carry out alone.

A problem for people wishing to perform exercises alone or without the supervision of a trainer is that they risk not correctly performing the physical exercises they are going to perform. Several problems can arise from poor performance of physical exercises. This problem arises in the same way in the professional context when an individual does not carry out a manipulation correctly and risks injury, for example in the context of a repetitive movement.

First, it is possible that the user performs a movement that is very little, or even not at all, effective in their physical training or in their work. In this way, the user risks training without obtaining the expected results or carrying out tasks while injuring themselves.

Second, a user who performs incorrect movements risks injury by performing incorrect movements, either by accident during a repetition of the incorrectly performed movement, or by repetition over several sessions of a wrong movement.

There are solutions to visualize these movements, such as setting up a mirror or markings, such as markings to position the feet or hands in certain exercises or manipulations. However, these solutions are bulky, not very flexible and require specific adjustments for each person.

There is therefore a need to provide any individual practicing movements for which a postural and movement reference is known with assistance in detecting the execution of a bad posture or a bad movement which could affect the health of said individual, or even to hurt him in the short or long term.

In particular, there is a need to provide beginners and more experienced athletes with a system and a method allowing the monitoring of their physical activity and their movements in order to secure and increase the effectiveness of their training.

Finally, in the professional context, a movement listed and recommended to accomplish an action can be poorly executed without the individual realizing it.

A problem with current solutions is that they do not allow an analysis based on the analysis of images which take into account the quality of the performance of the physical exercise.

The invention detailed below makes it possible to overcome the aforementioned drawbacks.

• Acquisition d’une séquence de premières images d’un premier individu ;

According to one aspect, the invention relates to a method for detecting a physical posture and generating an instruction, which comprises the following series of steps:

• Acquisition of a sequence of first images of a first individual;

• Traitement des premières images comprenant une extraction automatique d’au moins une zone utile du corps dudit premier individu de chaque première image pour générer une séquence de deuxièmes images de la zone utile ;
• Identification d’un type de mouvement physique par une première fonction d’apprentissage traitant des deuxièmes images de manière à sélectionner une première classe de mouvements parmi une pluralité de classes des mouvements ;
• Première génération d’une séquence de premiers squelettes du premier individu par application d’un algorithme de squelettisation à la séquence de deuxièmes images, chaque premier squelette comportant un ensemble de points caractéristiques du squelette de la zone utile ;
• Extraction de squelettes caractéristiques de la séquence de premiers squelettes par un premier algorithme, les squelettes caractéristiques correspondant à des postures caractéristiques de la première classe de mouvement ;
• Troisième génération d’un premier indicateur d’erreur de posture par comparaison d’un ensemble postures de référence définies pour les postures caractéristiques respectives et de l’ensemble de postures caractéristiques dudit individu ;
• Comparaison d’une valeur du premier indicateur d’erreur avec au moins une valeur seuil prédéterminée ;
• Lorsque la valeur de l’indicateur d’erreur est supérieure ou égale à la valeur seuil, génération automatique d’une première consigne numérique pour afficher une séquence d’images de référence représentative de la première classe de mouvement à une première vitesse inférieure à un seuil de vitesse prédéfini.

the sequence of steps including the following implemented by computer:

• Processing of the first images comprising automatic extraction of at least one useful area of the body of said first individual from each first image to generate a sequence of second images of the useful area;
• Identification of a type of physical movement by a first learning function processing second images so as to select a first class of movements from a plurality of classes of movements;
• First generation of a sequence of first skeletons of the first individual by application of a skeletonization algorithm to the sequence of second images, each first skeleton comprising a set of characteristic points of the skeleton of the useful zone;
• Extraction of characteristic skeletons from the sequence of first skeletons by a first algorithm, the characteristic skeletons corresponding to characteristic postures of the first movement class;
• Third generation of a first posture error indicator by comparison of a set of reference postures defined for the respective characteristic postures and the set of characteristic postures of said individual;
• Comparison of a value of the first error indicator with at least one predetermined threshold value;
• When the value of the error indicator is greater than or equal to the threshold value, automatic generation of a first digital setpoint to display a sequence of reference images representative of the first class of movement at a first speed less than one predefined speed threshold.

The method according to the invention makes it possible, by acquiring and processing a plurality of images of an individual, to process and analyze their actual movements. The method allows the identification of the exercise that the user is carrying out and, by comparing postures of the user during said exercise with reference postures corresponding to the same movement, to generate an error indicator making it possible to evaluate the quality of the movement performed. The method also makes it possible to read a sequence of reference images when the value of the error indicator is greater than a predefined threshold.

Thus, the method according to the invention makes it possible to detect the quality of performance of the exercise by the user and to inform them thereof. In addition, if the movement is performed incorrectly, the method makes it possible to show the user a sequence of images corresponding to the correct execution of the incorrectly performed movement in order to allow him to correct himself. As a result, the method according to the invention makes it possible to prevent the user from performing a movement which is ineffective in their training or which potentially subjects them to a risk of injury.

• Acquisition d’une séquence de premières images d’un premier individu ;

According to another aspect, the invention relates to a method for detecting a physical posture and generating an instruction, which comprises the following series of steps:

• Acquisition of a sequence of first images of a first individual;

• Traitement des premières images comprenant une extraction automatique d’au moins une zone utile du corps dudit premier individu de chaque première image pour générer une séquence de deuxièmes images de la zone utile ;
• Identification d’un type de mouvement physique par une première fonction d’apprentissage traitant des deuxièmes images de manière à sélectionner une première classe de mouvements parmi une pluralité de classes des mouvements ;
• Première génération d’une séquence de premiers squelettes du premier individu par application d’un algorithme de squelettisation à la séquence de deuxièmes images, chaque premier squelette comportant un ensemble de points caractéristiques du squelette de la zone utile ;
• Extraction de squelettes caractéristiques de la séquence de premiers squelettes par un premier algorithme, les squelettes caractéristiques correspondant à des postures caractéristiques de la première classe de mouvement ;
• Troisième génération d’un premier indicateur d’erreur de posture par comparaison d’un ensemble postures de référence définies pour les postures caractéristiques respectives et de l’ensemble de postures caractéristiques dudit individu ;
• Comparaison d’une valeur du premier indicateur d’erreur avec au moins une valeur seuil prédéterminée ;
• Lorsque la valeur de l’indicateur d’erreur est supérieure ou égale à la valeur seuil, génération automatique d’une première consigne numérique pour émettre une notification permettant d’accéder à une séquence d’images de référence représentative de la première classe de mouvement à une première vitesse inférieure à un seuil de vitesse prédéfini.

the sequence of steps including the following implemented by computer:

• Processing of the first images comprising automatic extraction of at least one useful area of the body of said first individual from each first image to generate a sequence of second images of the useful area;
• Identification of a type of physical movement by a first learning function processing second images so as to select a first class of movements from a plurality of classes of movements;
• First generation of a sequence of first skeletons of the first individual by application of a skeletonization algorithm to the sequence of second images, each first skeleton comprising a set of characteristic points of the skeleton of the useful zone;
• Extraction of characteristic skeletons from the sequence of first skeletons by a first algorithm, the characteristic skeletons corresponding to characteristic postures of the first movement class;
• Third generation of a first posture error indicator by comparison of a set of reference postures defined for the respective characteristic postures and the set of characteristic postures of said individual;
• Comparison of a value of the first error indicator with at least one predetermined threshold value;
• When the value of the error indicator is greater than or equal to the threshold value, automatic generation of a first digital instruction to issue a notification allowing access to a sequence of reference images representative of the first movement class at a first speed lower than a predefined speed threshold.

An advantage of this embodiment is that it allows the images to be viewed after a given period of time. This embodiment is particularly interesting in the context of an application of the invention to posture detection in the professional context. In this case, the user can access, for example, at the end of the day all the sequences for which the error indicator was greater than a threshold for one or more classes of movements. Thus, the invention makes it possible to replay a movement at a slower speed to help the user understand and learn how to correct a movement. A technical effect is to avoid injuries, particularly linked to repeated bad postures.

In this example case, the reference postures or movements can be recorded in a memory. The reference movements and postures can be animated or real video sequences of a skeleton or an avatar, respectively of a person allowing gestures and a margin of amplitude of these gestures to be defined.

According to one aspect, the method according to the invention comprises, when the value of the error indicator is lower than the threshold value, a step of generating a second digital instruction to inform the user of a movement instruction associated with a movement execution speed instruction by an output interface. This provision makes it possible to inform the user to carry out the rest of their movement.

• Mesure d’un écart entre un point du squelette caractéristique et un segment d’un squelette de la posture de référence définie pour la posture caractéristique associée au squelette caractéristique ; et/ou
• Mesure d’un écart d’un ou plusieurs segments du squelette caractéristique par rapport à un ou plusieurs segments du squelette de la posture de référence ; et/ou
• Mesure d’un alignement angulaire d’un ou plusieurs segments du squelette caractéristique par rapport à un ou plusieurs segments du squelette d’une des postures de référence et/ou par rapport à un axe horizontal ; et/ou
• Mesure d’un intervalle de temps séparant les instants d’acquisition des deux squelettes caractéristiques, et comparaison de l’intervalle de temps avec un intervalle de temps de référence caractéristique de la classe de mouvements ; et/ou
• Mesure de la trajectoire d’un point du squelette et comparaison de la trajectoire avec la trajectoire d’un ou plusieurs points du squelette de la posture de référence déterminée.

According to one aspect, the third generation of the posture error indicator comprises the following steps, for at least one of the characteristic skeletons:

• Measurement of a difference between a point of the characteristic skeleton and a segment of a skeleton of the reference posture defined for the characteristic posture associated with the characteristic skeleton; and or
• Measurement of a deviation of one or more segments of the characteristic skeleton in relation to one or more segments of the skeleton of the reference posture; and or
• Measurement of an angular alignment of one or more segments of the characteristic skeleton in relation to one or more segments of the skeleton of one of the reference postures and/or in relation to a horizontal axis; and or
• Measurement of a time interval separating the moments of acquisition of the two characteristic skeletons, and comparison of the time interval with a reference time interval characteristic of the class of movements; and or
• Measurement of the trajectory of a point of the skeleton and comparison of the trajectory with the trajectory of one or more points of the skeleton of the determined reference posture.

These arrangements allow a detailed analysis of the user's posture and take into account the position of different parts of the user's body and its comparison with a reference skeleton.

According to one aspect, the sequence of steps is repeated as long as the value of the error indicator is greater than or equal to the threshold value, the sequence of steps comprising, during its repetition, the extraction, in the first movement class of a subsequence of the sequence of first skeletons, the first digital instruction being generated to display the subsequence. This arrangement allows the user to focus on a part of the movement performed for which the execution of the movement is not optimal. As a result, the user repeats the part of the movement to correct to improve.

According to one aspect, the sequence of steps comprises storing the value of the error indicator in a memory, the sequence of steps comprising when repeated, comparing the value of the indicator to at least a value of the indicator stored in the memory during a repetition of the sequence of steps and the generation of a recommendation depending on the comparison. This arrangement makes it possible to compare the different repetitions of the movement and to adapt the recommendation depending on whether the user progresses or regresses in the quality of the movement.

According to one aspect, the first instruction triggers information to the user of a reduction in the speed of execution of the first kinematic sequence via an output interface. This provision makes it possible to instruct the user to reduce their speed of execution of the movement so that they can more easily correct it and progress in their execution.

According to one aspect, the first instruction triggers information to the user of a reduction in the number of repetitions of the movement to be performed. This provision helps reduce the risk of injury when a deterioration in the quality of execution is detected.

According to one aspect, the method according to the invention comprises in real time the display, on a screen, of a first visual error indicator representative of the first error indicator. This arrangement allows the user to view an indicator which is representative of a posture error in his movement. As a result, the user knows what to correct in their movement.

According to one aspect, the method comprises generating the first visual indicator from the error indicator and displaying in real time, on a screen, the first sequence of images or the second superimposed sequence of images to the visual error indicator. The superposition of images with the error indicator allows the user to clearly visualize the areas and parts of their movement that need to be corrected. This arrangement therefore makes it possible to increase the improvement of the user's movement.

• Un indicateur cinématique de translation comprenant une première position, un vecteur et une seconde position, la première position étant représentative d’une position d’une zone du corps de l’utilisateur au début du mouvement et la deuxième représentation étant représentative de la position de la même partie du corps de l’utilisateur à la fin du mouvement ; et/ou
• au moins un des deuxièmes squelettes de la séquence de deuxièmes squelettes ; et/ou
• un avatar généré à partir de la séquence de premiers squelette et/ou de la séquence de deuxièmes squelettes ; et/ou
• une indication d’une zone à atteindre matérialisée par un polygone ou une forme.

According to one aspect, the visual error indicator includes the display of:

• A kinematic translation indicator comprising a first position, a vector and a second position, the first position being representative of a position of a zone of the user's body at the start of the movement and the second representation being representative of the position of the same part of the user's body at the end of the movement; and or
• at least one of the second skeletons of the sequence of second skeletons; and or
• an avatar generated from the sequence of first skeletons and/or the sequence of second skeletons; and or
• an indication of an area to be reached materialized by a polygon or a shape.

This arrangement makes it possible to precisely indicate to the user the gestures they must perform during their movement. It therefore improves learning and correction of movement.

According to one aspect, the method according to the invention comprises the automatic identification of an individual and loading a training profile relating to said individual, said automatic identification step being carried out by facial recognition. This arrangement advantageously makes it possible to associate a user profile with a user to monitor their progress over time and sessions, and the correction of their posture errors. Facial recognition also makes it possible to automate the association.

According to one aspect, the method according to the invention comprises a step of counting the number of repetitions of the first kinematic sequence.

According to one aspect, the method according to the invention comprises counting the number of repetitions of the movement during which the value of the error indicator is less than the threshold value. This arrangement makes it possible to give a score for completing the repetitions of the movement.

• Au moins un dispositif optique configuré pour réaliser l’acquisition d’une pluralité de premières images d’un individu ;
• Au moins un premier calculateur et une mémoire configurés pour effectuer :
  • Un traitement des premières images comprenant une extraction automatique d’au moins une zone utile du corps dudit premier individu de chaque première image pour générer une séquence de deuxièmes images de la zone utile ;
  • Une identification d’un type de mouvement physique par une première fonction d’apprentissage traitant des deuxièmes images de manière à sélectionner une première classe de mouvements parmi une pluralité de classes des mouvements ;
  • Une première génération d’une séquence de premiers squelettes du premier individu par application d’un algorithme de squelettisation à la séquence de deuxièmes images, chaque premier squelette comportant un ensemble de points caractéristiques du squelette de la zone utile ;
  • Une extraction de squelettes caractéristiques de la séquence de premiers squelettes par un premier algorithme, les squelettes caractéristiques correspondant à des postures caractéristiques de la première classe de mouvement ;
  • Une deuxième génération d’un premier indicateur d’erreur de posture par comparaison d’un ensemble postures de référence définies pour les postures caractéristiques respectives et de l’ensemble de postures caractéristiques dudit individu ;
  • Une comparaison d’une valeur du premier indicateur d’erreur avec au moins une valeur seuil prédéterminée ;
  • Lorsque la valeur de l’indicateur d’erreur est supérieure ou égale à la valeur seuil, une génération automatique d’une première consigne numérique pour afficher une séquence d’images de référence représentative de la première classe de mouvement à une première vitesse inférieure à un seuil de vitesse prédéfini
• Au moins un afficheur configuré pour lire la séquence de premières images et/ou la séquence de secondes images ; et
• Au moins un boitier comprenant le dispositif optique.

According to one aspect, the invention also relates to a system for assisting in carrying out physical exercises, characterized in that it comprises:

• At least one optical device configured to acquire a plurality of first images of an individual;
• At least a first calculator and a memory configured to perform:
  • Processing of the first images comprising automatic extraction of at least one useful area of the body of said first individual from each first image to generate a sequence of second images of the useful area;
  • An identification of a type of physical movement by a first learning function processing the second images so as to select a first class of movements from a plurality of classes of movements;
  • A first generation of a sequence of first skeletons of the first individual by application of a skeletonization algorithm to the sequence of second images, each first skeleton comprising a set of characteristic points of the skeleton of the useful zone;
  • An extraction of characteristic skeletons from the sequence of first skeletons by a first algorithm, the characteristic skeletons corresponding to characteristic postures of the first movement class;
  • A second generation of a first posture error indicator by comparison of a set of reference postures defined for the respective characteristic postures and the set of characteristic postures of said individual;
  • A comparison of a value of the first error indicator with at least one predetermined threshold value;
  • When the value of the error indicator is greater than or equal to the threshold value, automatic generation of a first digital instruction to display a sequence of reference images representative of the first class of movement at a first speed less than a predefined speed threshold
• At least one display configured to read the sequence of first images and/or the sequence of second images; And
• At least one box comprising the optical device.

Brief description of the figures

Other characteristics and advantages of the invention will emerge on reading the detailed description which follows, with reference to the appended figures, which illustrate:

: a view of a flowchart showing the steps of the method according to the invention;

: a view of a posture generated by a user by the method according to the invention;

: an illustration of the method of calculating an error indicator according to the invention;

: an illustration of a visual error indicator according to the invention;

: a view of a system according to one embodiment of the invention; And

: a flowchart illustrating the operation of a system according to one embodiment of the invention.

Description of the invention

In the context of the invention, a learning function, or machine learning algorithm, is a process or set of procedures that helps a mathematical model adapt to data according to an objective. A learning algorithm normally specifies how feedback is used to enable the model to learn the appropriate mapping from input to output. The model specifies the matching function and maintains the parameters while the learning algorithm updates the parameters to help the model satisfy the objective.

Learning algorithms can generally be divided into broad categories such as supervised learning, unsupervised learning, and reinforcement learning. Supervised learning involves presenting a machine learning algorithm with training data consisting of inputs and outputs labeled by evaluators, the objective being to train the machine learning algorithm in such a way that 'it learns a general rule of correspondence between inputs and outputs. Unsupervised learning involves presenting the machine learning algorithm with unlabeled data, the goal being for the machine learning algorithm to find hidden structure or patterns in the data. Reinforcement learning involves an algorithm evolving in a dynamic environment, guided only by positive or negative reinforcement.

The models used by learning algorithms include neural networks (including deep learning), decision trees, support vector machines, Bayesian networks and genetic algorithms.

Neural networks, also known as artificial neural networks, are a class of nonlinear models mapping inputs to outputs and composed of layers that can potentially learn representations useful for predicting outputs. Neural networks are typically organized in layers, composed of a number of interconnected nodes that contain activation functions. The models can be presented to the network through an input layer connected to the hidden layers, and processing can be done through the weighted connections of the nodes. The response is then provided by an output layer connected to the hidden layers.

A multilayer perceptron is a class of feed-forward artificial neural networks. A multilayer perceptron consists of at least three layers of nodes: an input layer, a hidden layer, and an output layer. Except for the input nodes, each node is a neuron that uses a nonlinear activation function. A multi-layer perceptron uses a supervised learning technique called backpropagation for training. A multilayer perceptron can distinguish data that is not linearly separable.

A convolutional neural network is a neural network that is a regularized version of a multi-layer perceptron. A convolutional neural network uses convolution instead of general matrix multiplication in at least one layer.

A recurrent neural network is a neural network where the connections between nodes form a directed graph along a temporal sequence. This allows it to exhibit dynamic temporal behavior. Each node in a given layer is connected by a directed (unidirectional) connection to every other node in the next layer. Each node (neuron) has a time-varying real-valued activation. Each connection (synapse) has a real modifiable weight. Nodes are either input nodes (receiving data from outside the network), output nodes (producing results), or hidden nodes (which modify data along the way between input and the exit).

In the following description, each learning function mentioned refers to a function determined by a learning algorithm.

There is a flowchart representing the steps of the method according to the invention.

A first step of the method comprises an acquisition ACQU1 of a plurality of first images IM1 of a first individual. The plurality of first images represents a sequence of first images IM1 of the first individual. The sequence of first images comprises the plurality of first images IM1. The first images are acquired successively in order to represent the individual throughout a movement he is performing. The plurality of first images IM1 corresponding to the individual performing a movement or a physical exercise or a gesture characteristic of handling an object in the professional world. The plurality of first images may include a field of view in which the user is present, but in which other elements may also be present, such as furniture, other people, domestic animals.

The following steps of the method are implemented by computer.

The method then includes a processing step TRAI1 of the first images IM1 of the sequence of first images. This processing step includes automatic extraction of a useful area of the body of the first individual. By useful area of the body of the first individual, we mean an extraction of the area in which the first individual is located. This extraction makes it possible to isolate the areas of the image field occupied by the first individual from the other elements which may appear in each image of the plurality of first images IM1. In this way, the following steps of the process can be carried out on a field which does not include other elements which could disrupt an analysis of the image. Such elements may be other individuals, furniture located in the room in which the first individual is exercising or a gesture characteristic of handling an object, pets, or other. Extracting the useful area on each image of the sequence of first images IM1 makes it possible to generate a sequence of second images IM2. According to one embodiment, the extraction of the useful area is carried out by analysis of the image to extract a skeleton of the individual. A framing is then made around the skeleton of the individual by affixing a rectangular frame circumscribed to said skeleton. Once the rectangular frame is defined, the algorithm follows the movements and changes in position and orientation of the user in order to update the position and size of said rectangular frame at each image of the plurality of first images IM1. Advantageously, this method makes it possible to always center the user in each image of the plurality of second images IM2. According to one example, an algorithm called “HOURGLASS” and/or an algorithm called “MlKit” is used to perform skeleton extraction. According to one embodiment, the skeletonization algorithm for extracting the useful area performs a skeletonization comprising 17 points on said generated skeleton. Such generation of a 17-point skeleton is advantageously fast and inexpensive in computing resources for the calculation.

The method then includes a step of identifying a type of physical movement implementing a first learning function CCN1. The first learning function CCN1 receives the second images IM2 as input. The learning function assigns a first movement class C1 to the sequence of second images IM2. The first movement class C1 belongs to a plurality of movement classes Ci. The plurality of movement classes Ci corresponds to all of the movements taken into account by the method according to the invention. This step of the method makes it possible to determine the type of exercise which is carried out from the plurality of images acquired. Alternatively, it may be a type of manipulative gesture.

The method then comprises a first generation step GEN1 of at least one first skeleton SQ1 of the individual. Preferably, this step comprises the generation of a sequence of first skeletons SQ ₁ . This generation of the first skeleton SQ1 is carried out using a skeletonization algorithm from the sequence of second images IM2. In other words, the skeletonization algorithm processes the sequence of second images IM2 in order to extract the sequence of first skeletons SQ1 of the individual. The first skeleton SQ1 comprises a set of characteristic points of the body of the individual appearing on the sequence of second images IM2. By characteristic points, we mean points which correspond to specific predetermined areas of the anatomy of the individual. For example, the first skeleton SQ1 may include points which correspond to joints of the individual, such as knees, elbows, shoulders. It can also include points corresponding for example to the center of gravity of the individual, that is to say at the level of the navel of the latter.

According to one embodiment, the skeletonization algorithm comprises a kinematic model returning the first skeleton SQ1 in two dimensions or in three dimensions. According to one embodiment, the skeletonization algorithm uses a planar model. The planar model makes it possible to delimit the contours of an area of the user's body. According to one embodiment, the skeletonization algorithm uses a volumetric model in which which provides a skeleton in the form of geometric shapes representing the parts of the user's body. According to one embodiment, an algorithm called “HOURGLASS” and/or an algorithm called “MlKit” is used to perform skeleton extraction. According to one embodiment, an algorithm called “Pixie” and/or an algorithm called “OpenPose” can be used.

According to one embodiment, during the first generation step of a skeleton SQ ₁ , the skeletonization algorithm carries out a skeletonization carrying out a skeletonization comprising 33 points on said generated skeleton. Such generation of a 33-point skeleton is advantageously fast and inexpensive in computing resources for the calculation. It also allows for sufficient precision and definition of the skeleton.

The method then includes a second generation step GEN2 of a set of characteristic postures POS1 of the first movement class C1. This GEN2 generation is carried out by a second ALG2 algorithm from the first SQ1 skeleton of the individual. This generation makes it possible to obtain the set of characteristic postures POS1 which is characteristic of the first movement class C1. In other words, the method selects important postures in the execution of the physical movement by the individual from the first skeleton SQ1 during said movement.

The method then includes a step of extracting characteristic skeletons SQ _R. The characteristic skeletons SQ _R are characteristic of the C1 movement class which has been selected. These characteristic skeletons SQ _R correspond to characteristic postures POS1 of movement class C1. These characteristic postures take into account interesting points of the movement such as peaks of body point trajectories during the repetition of the movement. The extraction is carried out by a first ALG1 algorithm. The first algorithm takes into account the trajectory of the points of the first skeletons SQ ₁ to find the posture corresponding to a vertex or a valley in the trajectory of said point of the skeleton along the sequence of first skeletons.

The method then includes a second generation step GEN2 of a first posture error indicator ERR1. This second generation GEN2 is carried out by comparing the characteristic postures POS1 of the characteristic skeletons SQ _R with reference postures POS1R. The POS1R reference postures are reference postures corresponding to the typical movement corresponding to the selected C1 movement class. Thus, reference postures are compared with those of the individual to generate the first error indicator ERR1. Thus, the first error indicator ERR1 is representative of a deviation of all of the generated postures of the individual compared to all of the reference postures POS1R. In a first case, the reference postures correspond for example to movement postures of a movement class relating to a physical exercise. According to different examples, the movement class can include in a non-restrictive manner: a push-up class, a squat class, an abdominal class, a pull-up class, etc.

In a second case, the reference postures correspond for example to movement postures of a movement class relating to a given gesture characterizing an action in a professional context. According to different examples, the movement class can include in a non-restrictive manner: a class of actuation of a given machine tool, a class of loading or unloading of a package or a load, a class of movement of an action carried out on an industrial production line aimed at sorting elements or affixing a label, etc.

The method then includes a step of comparing COMP1 of a value of the first error indicator ERR1 with at least one threshold value. This step makes it possible to determine whether the difference between the user's postures and the reference postures is significant or not. Therefore, by this comparison with the at least one threshold value, it is determined whether the user is doing the exercise with a deviation in his postures which is problematic or whether this deviation is sufficiently low for the exercise to be considered to be well executed.

The method then comprises, when the value of the error indicator ERR1 is greater than or equal to a threshold value Vs, a step of automatic generation GEN3 of a first digital setpoint. This first digital instruction triggers the display of a sequence of reference images, by a display. This plurality of reference images corresponds to the images of the exercise performed correctly. The sequence of images is preferably displayed at a speed lower than a predefined threshold. According to one embodiment, the predefined threshold is an image broadcast speed which is 50% of the normal speed. According to one embodiment, the sequence of reference images is superimposed on a display of the sequence of first images IM1 or the sequence of second images IM2.

Thus, the method according to the invention allows the analysis of the movements carried out by the user as well as the comparison of these with reference postures. It makes it possible to automatically detect the type of exercise that is carried out by the user during the ID1 identification step. It also helps determine whether the movement is performed correctly or not. Finally, by displaying at low speed the kinematic sequence containing the correctly performed movement, it allows the user to be guided towards correctly carrying out said movement. As a result, the method according to the invention allows control and support of the user in carrying out physical exercises. This control limits the risk of injury to the user in the event of incorrect execution of the movement. The method also allows an improvement in the effectiveness of training by guiding the user towards the realization of correct and efficient movements. The method according to the invention allows the user to progress in carrying out physical movements. Finally, it allows automatic supervision of the user in carrying out the physical movement without resorting to the presence of a trainer guiding the user.

According to one embodiment, the first learning function is determined by training by a learning algorithm. The training of the first learning function is advantageously carried out from a plurality of training image sequences carrying out physical exercises stored in a base so that the function associates each training image sequence with a type of exercise or physical movement performed is associated with the sequence of training images in the base. Such training is similar to supervised training.

Assessment of the quality of the movement performed

According to one embodiment, the method according to the invention comprises, when the value of the error indicator ERR1 is lower than the threshold value Vs, a fourth generation step GEN4 of a second digital setpoint. The second digital instruction is transmitted to a display. The second digital instruction includes a movement instruction and a movement execution speed instruction so that the second instruction triggers the display, by the display, of the movement instruction and the execution speed instruction movement. According to one embodiment, the movement execution speed setpoint is a display of the sequence of reference images at a predetermined speed. According to one embodiment, the execution speed instruction includes the broadcast by a sound broadcast system of audio messages punctuating the different phases of movement class C1. According to one embodiment, the execution speed instruction includes the display by the display of messages punctuating the different phases of movement class C1. Thus, the second digital instruction is generated when the movement is carried out correctly. The movement instruction corresponds to the rest of the physical exercise to be performed by the user. In this way, when the movement is correctly performed by the user, the latter is invited to carry out the rest of his exercise at an execution speed which is specified by the system.

According to one embodiment, the execution speed setpoint depends on the value of the error indicator ERR1. Thus, we can in this way have a speed of execution which is adapted to the quality of the movement carried out by the user. For example, when the value of the error indicator ERR1 is less than the threshold value Vs, the method according to the invention may comprise comparing the value of the error indicator ERR1 to a second threshold value V _S2 lower than the threshold value Vs and defined for the first movement class C1. Advantageously, the second threshold value V _S2 corresponds to optimal execution of the movement of the first movement class C1 carried out. In this way, when the movement is performed sufficiently correctly by the user so that he does not have to start said movement from the beginning to learn it (which corresponds to a value of the error indicator ERR1 greater than the threshold value Vs), the quality of the movement is still evaluated by measuring the proximity of the value of the error indicator ERR1 to the second threshold value V _S2 . In other words, when the value of the error indicator ERR1 is between the first threshold value Vs and the second threshold value Vs2 we can say that the error observed is minor. Thus, the second digital instruction is generated and the display of the movement instruction is given.

Generating the error indicator

According to one embodiment, the posture error indicator is generated from the measurement of a deviation of one or more segments SEG ₁ of the characteristic skeleton SQ _R relative to one or more segments SEG _R1 of the skeleton of the reference posture POS1R. Each segment is obtained by linking characteristic points of the skeleton considered. Thus, according to an example, as represented in , a first segment SEG ₁ of the first characteristic skeleton SQ _R corresponds to the right femur of the user (on the left of the figure). The first corresponding SEG _R1 segment on the skeleton of the reference posture POS1R is also the segment representing the right femur of this skeleton (on the right of the figure). According to this embodiment, a difference is measured between the first segment SEG ₁ of the reference skeleton and the first segment SEG _R1 . This difference is presented on the , which represents the two segments superimposed and magnified in relation to the . The superposition of the segments is obtained by placing the two skeletons in a frame of reference using reference points of said skeletons. For example, we use a point representing the right foot of the skeleton as a reference for the two skeletons. This difference corresponds for example to a minimum distance separating the two segments when the skeletons SQ ₁ and SQ _R are superimposed. This type of determination of the error indicator is particularly advantageous because it makes it possible to detect a discrepancy between a "good posture" and the posture of the user during a physical movement by focusing on one or two areas of interest in it. According to one embodiment, the error indicator comprises the measurement of the difference between several segments of the skeleton SQ _R and the skeleton of the reference posture POS1R.

According to one embodiment, the posture error indicator is generated from the measurement of a difference between one or more points of the characteristic skeleton SQ _R and one or more points of the skeleton of the determined reference posture POS1R. According to one embodiment, the difference is a distance measured between a point of the first skeleton SQ _R and the corresponding point of the skeleton of the reference posture POS1R. Alternatively, the measurement of the difference is the measurement of the distance between a point of the characteristic skeleton SQ _R of the user corresponding to a joint thereof, and a point corresponding to the same joint on the skeleton of the posture reference POS1R. According to one example, the measured difference is the estimated distance between a point of the base of the right femur of the characteristic skeleton SQ _R and the base of the right femur of the skeleton of the reference posture POS1R. The selected point corresponds, according to one embodiment, to a left or right elbow of the skeletons. According to one embodiment, the point corresponds to a left or right shoulder of the skeleton. According to one embodiment, the point corresponds to a left knee or the skeleton. All joints of the body can be taken as points for measuring the differences between the characteristic skeleton SQ _R and the skeleton of the reference posture POS1R. According to one embodiment, the measured deviations are compared to reference lengths to take into account the morphology of the user. This arrangement makes it possible to take into account the variability of morphologies between different users. For this purpose, the measured lengths are compared to a reference length measured on the user's skeleton. The reference length is advantageously chosen from: the width of the pelvis, the width of the shoulders, the size of the torso, the height of the torso, the length of the leg and arms (in a stretched position), and/or the width of the face , for example the ear-nose distance.

According to one embodiment, the posture error indicator is generated from the measurement of an angular alignment of one or more segments of the characteristic skeleton SQ _R relative to one or more segments of the skeleton of the posture of determined reference POS1R. Such an arrangement is shown in , where an angle ANG ₁ is measured. According to this embodiment, an angle between the straight line carrying the first segment SEG ₁ and that carrying the first reference segment SEG _R1 is measured. This measurement is carried out in the plane using two-dimensional skeletons. According to one embodiment, the aforementioned measurement is carried out in space from three-dimensional skeletons. The advantage of measuring this angle is that it makes it possible to compare the orientation in the plane or in three-dimensional space of a member of the user with the orientation of a member of the reference posture. This characteristic therefore represents an effective means of controlling the quality of posture during the physical movement performed.

According to one embodiment, the posture error indicator is generated from the measurement of the time interval separating a first characteristic posture from a second characteristic posture of the first skeleton SQ1. In other words, the generation of the indicator includes the measurement of the time interval separating the instants of acquisition of two images from the sequence of second images IM2 from which the two reference postures are generated. Advantageously, these two postures are characteristic of two important phases of a movement. According to one example, the two postures represent the beginning and the end of a movement phase. According to an example, we measure the duration of raising the arms during a “Jumping jack” type exercise. The time interval measured to generate the posture indicator corresponds in this example to the time separating the acquisition of the start of movement image and the acquisition of the end of movement image. In other words, the time interval corresponds to the time required for the user to raise their arms.

According to one embodiment, the posture error indicator is generated from measuring the trajectory of one or more points of the skeleton SQ1 and comparing this trajectory with the trajectory of one or more points of the skeleton of the reference posture determined POSR1. According to this example, the trajectory formed by the point of the first skeleton SQ1 during the movement is compared with the trajectory formed by at least one point of the skeleton of the reference posture during the same movement. Comparing the trajectory of a point on each skeleton is a good way to estimate the posture error indicator as it allows us to see the quality of the movement. According to the example of the , the trajectory of the point P ₁ corresponding to the right foot of the first skeleton SQ1 is compared with the trajectory of the right foot P ₁ of the skeleton of the reference posture POS1R. A measurement made to compare the two trajectories is the length of the arc formed. For example, in the “jumping jack” movement, the user's feet will form an arc in a frame of reference formed around the person's center of gravity. The length of the arc is a good measure to compare the range of movement performed by the user and compare it with the reference range of movement. According to one embodiment, a mean square error is calculated to estimate the proximity between the two trajectories considered. According to one embodiment, the maximum distance between the two trajectories is measured. This measure is particularly effective because it makes it possible to detect parasitic movements that reduce the efficiency of the overall movement.

According to one embodiment, the error indicator ERR ₁ is calculated by measuring curvature on the trajectory of a point of the skeleton SQ1. According to one embodiment, the error indicator ERR1 is calculated by calculating the curvature of the curve formed by several points of the first skeleton SQ1. According to one embodiment, the curvature is compared to that of the same points taken on the skeleton of the reference posture POS1R. According to one example, the points of the first skeleton SQ1 and of the skeleton of the reference posture used are points corresponding to the curvature of the back on said skeletons. According to one embodiment, a contour analysis is carried out on the plurality of second images in order to extract a curvature of a part of the body, preferably of the back. According to one embodiment, a comparison of the curvature measurements carried out on the skeleton and the curvature measurements carried out from the contours is carried out. This arrangement advantageously makes it possible to limit curvature estimation errors by combining two methods of calculating it. This arrangement makes it possible in particular to limit errors linked to changes in viewing angle during movement.

Generation of recommendations to the user

According to one embodiment, the computer-implemented steps of the method are repeated as long as the value of the error indicator ERR1 is greater than or equal to the predefined threshold value Vs. According to this embodiment, the sequence of repeated steps comprises, during its repetition, the extraction, in the first movement class C1, of a subsequence of first skeletons SQ1. By subsequence is meant a sequence of first skeletons SQ1 which corresponds to a temporal fraction of the sequence of first skeletons SQ1. In other words, the subsequence concerns a small part of the sequence of initial SQ1 first skeletons. This arrangement is particularly advantageous because it allows the analysis to be carried out on a fraction of the movement considered. In this way, the user can focus on a part of the physical movement that is poorly performed. This arrangement improves the correction of the user's movement. According to one embodiment, the first digital instruction is generated to display the subsequence. In this way, the user is guided on his movement on the sub-sequence thanks to the digital instructions. For example, in the case of performing a “jumping jacks” movement, the movement class represents the complete “jumping jack” movement, namely a first jump during which the person raises their arms and spreads their legs, then a second jump during which the user lowers their arms and tightens both legs. According to this example, the subsequence represents the sequence of movement corresponding to the user's first jump. According to one embodiment, the first movement class is broken down into a plurality of subsequences. By plurality we mean two sequences, three sequences, four sequences, five sequences or more. The number of sequences of the plurality of kinematic sequences depends on the movement of the movement class. Thus, those skilled in the art will be able to decide on the number of sequences necessary to decompose the movement class. The usefulness of breaking down the movement into sequences is to be able to use the analysis tools previously mentioned on portions of the overall movement of the movement class.

According to one embodiment, the first digital instruction includes an indication to the user to restart the movement corresponding to the subsequence. This characteristic makes it possible to instruct the user to repeat part of the movement that they performed previously. The first digital instruction is generated according to the comparison of the error indicator ERR ₁ with the at least one threshold value Vs, thus, the instruction makes it possible to give an indication to the user to restart the part of his corresponding movement at the first cinematic sequence. This arrangement is particularly advantageous because it allows the user to work specifically on the part of the movement in which the error was detected. It therefore makes it possible to improve user training. In other words, the iterations of the method relating to the analysis of the subsequence are repeated as long as the quality of the movement performed by the user has not improved. This arrangement is particularly advantageous because it allows the part of the movement to be repeated several times to teach the user to properly perform said part of the movement. In this way, the user focuses on the parts of the movement that are not performed well. This arrangement is particularly advantageous because it limits excessive user fatigue and reduces the risk of injury to the user. According to one embodiment, the first digital instruction includes an indication to stop the exercise.

This characteristic is particularly advantageous depending on the type of error that is detected. Thus, when an error characteristic of a dangerous achievement or a dangerous posture for the user is detected by comparing the postures of the user with the reference postures of the movement class, the method gives the instruction to the user to stop their movement. This arrangement advantageously makes it possible to limit the risk of injury to the user. According to one embodiment, the first digital instruction includes an indication to display the number of repetitions to be performed by the user. In this way, when a degradation in execution quality is detected, the user can receive an instruction to reduce the number of repetitions to be performed. This characteristic is particularly advantageous depending on the type of error that is detected. Thus, when an error characteristic of excessive fatigue is detected by comparison of the user's postures with the reference postures of the movement class, the method instructs the user to reduce the duration of their exercise . The method following these steps makes it possible to analyze the posture and movement of the user on the first sequence SEQ ₁ by isolating it from the rest of the movement. Thus, when the method detects a defect in the user's posture or in their way of carrying out the exercise, the method makes it possible to extract a "sub-movement" which represents a small part of the total movement carried out by the user . Once this sub-movement is isolated, the process allows the analysis to be carried out only on it to focus the user's attention and work on it. Thus, the part of the movement concerning the subsequence is analyzed by skeletonization and comparison of the user's postures with the characteristic postures corresponding to the part of the movement performed. Finally, if the value of the new calculated error indicator ERR1 is greater than the threshold value, a reading of the sequence of images corresponding to the movement class is read. In this way, the user has an example of the part of the movement corresponding to the subsequence. This example allows the user to improve their movement by drawing inspiration from the example. In addition, the broadcast of this sequence also serves to give the user the indication to restart the sequence that he has executed incorrectly. Finally, the sequence of images read is read at a speed lower than the normal speed of execution of the movement considered. In this way, the user can more easily identify the passages of the movement which he has not performed well, and carry out the movement at reduced speed by following the image sequence which is read. In this way, it is put in the best conditions to carry out the movement in the best possible way. According to one embodiment, the sequence of steps comprises storing the successive values of the first error indicator ERR ₁ in a memory. According to this embodiment, the sequence of steps also includes the comparison of the value of the last error indicator ERR ₁ calculated with the values of the error indicator ERR ₁ stored. Finally, the sequence of steps includes the generation of a recommendation depending on said comparison. According to this embodiment, the evolution of the value of the error indicator ERR ₁ is studied to allow adaptation of the instructions according to the evolution of said error indicator. For example, in the event of an increase in the error indicator ERR ₁ , during repetitions of the movement, a recommendation can be generated to display an instruction to the user to stop their movement.

This provision makes it possible to record data relating to the quality of the user's work over the time and sessions that the user carries out. In this way, it makes it possible to monitor the performance and quality of execution of the exercises by the user. According to one embodiment, the value of the first error indicator ERR ₁ is compared with previous values of the first error indicator ERR ₁ and. This arrangement is particularly advantageous because it allows a comparison of the value of the error indicators during the exercise sessions that the user does. According to one embodiment, a recommendation is generated based on the evolution of the first error indicator ERR ₁ . The recommendation includes, according to one embodiment, an instruction to reduce the number of repetitions. According to one embodiment, the instruction includes an indication to stop the exercise. According to one embodiment, the recommendation includes an indication to restart the movement corresponding to the first kinematic sequence. In this way, if a deterioration in the quality of execution of the movement or sub-movement is detected, the user receives an instruction to re-execute the movement or sub-movement in question in order to correct the posture defect which has appeared. during this session. Likewise, if an improvement in the user's posture during the movement is detected, the user can receive an indication of their progress in performing the exercise. In this way, the user is reassured in his good practice. In addition, such information rewards the efforts made by the user, which has the effect of reinforcing the motivation of the user, who is informed of the progress he is making. Strengthening the user's motivation is particularly effective so that the user continues to follow regular training, which increases the user's progress in the long term. According to one embodiment, the information relating to the different values of the error indicator is recorded in a user profile. According to one embodiment, facial recognition is performed to identify the user and associate them with their user profile. According to one embodiment, facial identification is carried out by a facial recognition algorithm applied to the sequence of first images and/or to the sequence of second images. According to one embodiment, the facial recognition algorithm used is an algorithm called “FaceNet”.

Data displayed to the user

According to one embodiment, the first error indicator ERR ₁ and is displayed on a screen in real time. Advantageously, each time a new value of the error indicator ERR ₁ is calculated, it is displayed. This arrangement is particularly advantageous because it allows the user to correct their posture in real time while performing the exercise. According to one embodiment, a score taking into account the value of the first error indicator ERR ₁ is displayed in real time on the screen. The displayed score includes for example a percentage generated from the values of the error indicator ERR ₁ . According to one embodiment, the score takes into account several first error indicators ERR ₁ . In this way, it is possible to display a score taking into account several different data from the user's posture when performing an exercise. In this way, the score takes into account the overall performance of the exercise rather than taking into account a single piece of data from it.

According to one example, the error indicator includes a first repetition indicator. By first repetition indicator, we mean an indicator which is evaluated for each repetition of the movement performed by the user. The first repetition indicator is evaluated as described previously in this description. According to one embodiment, the error indicator includes a movement indicator. The movement indicator is used to evaluate the overall quality of the movement by counting several repetitions of it. Thus, it takes into account the values of the first repetition indicator of each repetition of the movement. The movement indicator advantageously takes into account the number of repetitions of the movement performed. It also takes into account the average time for each repetition of the movement performed. The movement indicator advantageously makes it possible to calculate a score representing the quality of the execution of the user's movement. According to one embodiment, the value of the movement indicator is recorded in a memory. According to one embodiment, this value is associated with the user profile. According to one embodiment, the value of the movement indicator is compared to previous values of the movement indicator. The comparison advantageously makes it possible to deduce a state of fitness of the user during the work session. For example, if the user's score is lower than the scores stored in the user profile during previous sessions, we can deduce that the user's state of fitness is less good than during previous sessions. Advantageously, a recommendation can be displayed to the user to reduce the number of repetitions of the movement to be performed, and/or to reduce the number of repetitions of following movements during the session in order to take into account the state of fatigue of the user. 'user. This arrangement is particularly advantageous because it reduces the risk of injury for an individual who is tired. In the same way, a weak movement indicator makes it possible to evaluate the mastery and good execution of the movement by the user. This arrangement makes it possible in particular to generate a recommendation to the user to enable him to learn to perform the movement better, for example by reducing the speed of execution or by concentrating on a small part of the exercise, as has been described previously.

According to one example, when the user performs a leg bending movement (otherwise called "squat"), an error indicator is calculated by taking into account the sequence of first skeletons SQ1 or the sequence of characteristic skeletons SQR of the orientation of the individual's head. In other words, we check whether the user is looking straight ahead. An error indicator value in this case is the percentage of frames in the sequence that the user is looking at in front of them. According to this example, another error indicator is calculated. This error indicator takes into account on the sequence of characteristic skeletons the angles formed between the bust and the legs at characteristic points of the movement. For example, these angles are estimated at the low point of the movement (when the user is in a chair position) and at the high point of the movement (when the user is in a standing position). According to one embodiment, another error indicator is calculated during the low position of the movement. This indicator is calculated taking into account the curvature of the back on the characteristic skeleton. If the back is not straight (i.e. when the value of the radius of curvature of the curve formed by the points of the back is very large), then a penalty is assigned to the score of the other indicators. If too small a radius of curvature is detected, then an indication to stop movement may be displayed to the user. Advantageously, this arrangement makes it possible to limit the risk of injury when carrying out this movement.

According to one embodiment, the screen displays in real time a video of the user generated from the sequence of first images IM1 and/or the sequence of second images IM2. By video we mean a sequential display of images from the sequence of first images or the sequence of second images. This arrangement advantageously allows the user to see from another point of view the way in which he carries out the exercise. According to one embodiment, the first error indicator ERR ₁ is used to generate a visual error indicator EVR ₁ which is superimposed on the video of the user broadcast when he performs his exercise. For example, in the case where the error indicator ERR ₁ concerns a measurement of the distance of a segment of the skeleton SQ1 of the user with the same segment on the skeleton of the reference posture, the visual indicator d The EVR1 error displays in real time the distance measured between the two skeletons superimposed on the user's video. Advantageously, the visual error indicator is displayed on the area of the user's body over which the distance is measured. According to one embodiment, the visual error indicator EVR ₁ includes an arrow or a segment representing the difference between the user's posture and the reference posture.

According to one embodiment, the visual error indicator EVR ₁ comprises a kinematic translation indicator comprising a first position, a vector and a second position. The first position is representative of a position of an area of the user's body at the start of the movement and the second position is representative of the position of the same part of the user's body at the end of the movement. This arrangement is very advantageous because it makes it possible to represent the position in which the user is, as well as a vector representing the movement that a part of his body must make to reach the correct posture represented by the reference posture, as well as the position of the reference posture. In this way, a visualization of the steps to be carried out for the user to reach the reference posture is displayed to the user. Thus, he can, without having to ask himself any questions, very easily realize the adjustments he must make in his posture.

According to one embodiment, the visual error indicator EVR ₁ includes the display of at least one of the second skeletons SQ2 of the sequence of second skeletons. This arrangement is particularly advantageous because it allows the user to precisely visualize the steps of the exercise or sequence that the user must perform, each second skeleton corresponding to a posture characteristic of movement class C1. It is therefore easy for him to see how to carry out the exercise in a qualitative manner.

According to one embodiment, the visual error indicator EVR ₁ includes the display of an avatar. By avatar we mean a digital representation of the user. The avatar is preferably configured to resemble the user. The avatar tracks the user's movements over time. According to one embodiment, the avatar is generated from the first skeleton SQR ₁ . In this way, the generated avatar follows the user's movements. This arrangement is particularly advantageous because it allows the user to track and evaluate their own movements in real time.

According to one embodiment, the avatar is generated from the second skeleton SQR ₂ of the posture of several postures from the set of reference postures. According to this embodiment, the avatar, which is a virtual representation of the user resembling him, follows the movements of the successive reference posture(s) of the movement class Ci produced. In this way, the user sees in real time a representation of themselves performing a correct and efficient movement. In this way, the user is perfectly encouraged to perform the exercise in the best possible way.

According to one embodiment, the avatar is generated by a third learning function CCN3. According to this embodiment, the avatar is generated by taking as input the first skeleton SQR ₁ and/or the second skeleton SQR ₂ . According to one embodiment, the avatar is also generated based on an image taken of the individual. According to one embodiment, the image of the individual comes from the plurality of first images IM1. According to one embodiment, the avatar takes into account an image previously taken of the individual. Using an image of the individual makes it possible to generate an avatar with physical characteristics that are similar to those of the individual. In this way, the user has an avatar which follows his movements and which resembles him.

According to one embodiment, the third learning function is determined by training by a learning algorithm. The training of the third learning function is advantageously carried out by injecting as input to the learning function skeletons of people performing physical exercises which are associated with images of respective avatars, so that the third function of Driven learning associates each people skeleton with the base avatar image. In this way, supervised learning of the third learning function is carried out. According to one embodiment, the generation of the avatar is carried out by an algorithm generating a skin around the structure defined by said third skeleton.

According to one embodiment, the visual error indicator EVR1 includes the display of an area to be reached. The area to be reached is preferably an area corresponding to the position that a part of the user's body must reach at the end of the movement, or at the end of the movement corresponding to the kinematic sequence. According to one embodiment, the visual error indicator EVR ₁ includes the display of an area in the form of a polygon. In this way, the area to be reached is shown on the display. This arrangement is advantageous because it allows the user to visualize the area he must reach. According to one embodiment, the visual error indicator EVR ₁ is materialized in the form of a circle, or any other simple geometric shape. According to one embodiment, the visual error indicator EVR ₁ also includes a display of the area of the user's body which must move to reach the area to be reached. According to one example, the visual error indicator EVR ₁ comprises a circle, a polygon or any geometric shape surrounding or indicating the area of the user's body which must move towards the area to be reached. In this way, there is no ambiguity for the user to know the actions to perform to correctly perform the physical exercises. Such a provision is presented in , in which two visual error indicators are superimposed on the user's hands during exercise. It can be seen that the left hand indicator indicates an area in which the user's left hand should be placed.

Assistance system

According to one aspect, the invention relates to an assistance system 100 for performing physical exercises. The system comprises at least one optical device 110 configured to perform the acquisition ACQU1 of the plurality of first images IM1 of the individual. By optical device 110 is meant any device comprising optics and a means of acquiring data relating to an optical flow. Such an optical device 110 is for example a camera, a camera, or an infrared sensor. According to one embodiment, the optical device is included in a housing 160.

The assistance system also includes at least a first computer 130 and a memory 132 configured to perform:

- the TRAI1 processing of the images comprising an automatic extraction of at least one useful area of the body of the individual from each image acquired to generate a plurality of second images IM2;

- identification ID1 of a type of physical movement from a first algorithm ALG1 processing as input images processed from a first learning function CCN1 and a first classification CLASS1 so as to select a class of movements This ;

- the generation GEN1 of a first skeleton SQ1 in two dimensions of the individual by application of a skeletonization algorithm on the plurality of acquired images IM1, said first skeleton SQ1 comprising a set of points characteristic of the human body of the processed images ;

- the generation GEN2 of a set of characteristic postures POS1 of the movement class Ci from a second algorithm ALG2 processing as input the first skeleton SQ1 in two dimensions of the individual;

- the generation GEN3 of an error indicator ERR1 of postures from a set of reference postures determined from the class of movements Ci and the set of postures generated by said individual;

- the COMP1 comparison of a value of the error indicator with at least one threshold value;

- the automatic generation GEN4 of a first digital instruction resulting in the reading of a sequence of images of a plurality of reference images corresponding to the movement class Ci selected at a first speed lower than a predefined threshold when the value of the error indicator is greater than the threshold value;

- the GEN5 automatic generation of a second digital setpoint to display a movement instruction associated with a movement execution speed setpoint when the value of the error indicator is lower than the threshold value.

The system further comprises at least one display 150 configured to read the sequence of images and/or the movement instruction. By display 150, we mean any means allowing the display of images or information. According to one embodiment, the display is a screen. According to one embodiment, the display is the terminal of a mobile phone or smartphone.

There presents part of the system according to the invention and a user performing movements while being filmed by two optical devices 110.

There is a diagram illustrating an example of hardware configuration of the configuration assistance system 100 according to the invention.

From a hardware point of view, the configuration assistance system 100 can be seen as a first calculator interacting with a computer program product implementing the method according to the invention.

The system 100 is a computer, for example, a microcomputer, a computer network, an electronic component, a tablet, a smartphone or a personal digital assistant (PDA).

The system 100 comprises a first calculator 130. This first calculator 130 comprises, for example, one or more processors capable of interpreting instructions in the form of a computer program, a programmable logic circuit, such as an application-specific integrated circuit (ASIC), an in situ programmable gate array (FPGA), a programmable logic device (PLD) and programmable logic arrays (PLA), a system on chip (SOC)), an electronic card in which steps of the method according to the invention are implemented in hardware elements. The processing can be executed by a processor, or simultaneously or sequentially, or according to another method, by one or more processors.

The first calculator 130 comprises a data processing module 131 for carrying out calculations, a memory 132, operationally coupled to the data processing circuit 131, a computer-readable medium 134 and optionally a reader 133 adapted to read the computer-readable medium 134.

The system 100 also includes an input device, an output device, and a communication device.

Each function of the system 100 is executed by causing the data processing module 131 to read a predetermined program on hardware such as the memory 132 such that the data processing module 131 executes calculations, controls communications carried out by the communication device and reading and/or writing data in the memory 132 and the computer readable medium 134.

The process is executed on a single computer or on a system distributed between several computers (in particular via the use of cloud computing).

The memory 132 is a computer-readable recording medium, and can be configured with, for example, at least one of the following elements: a read-only memory (ROM), a read-only memory erasable and programmable (EPROM, from English Erasable Programmable Read-Only Memory), a programmable and electrically erasable read-only memory (ÉEPROM, from English Electrically Erasable Programmable Read-Only Memory), a random access memory (RAM, from English Random Access Memory) and another suitable storage medium. Memory 132 can include an operating system and load programs according to the invention. It includes registers suitable for recording parameter variables created and modified during the execution of the aforementioned programs.

The program product may include computer readable recording media 134 which is a tangible device, not being a transient signal per se, may be configured with, for example, at least one of the following: removable media, such as for example, in a non-limiting manner, a magneto-optical disc (for example, a read-only compact disc (CD-ROM, from English Compact Disc Read-Only Memory), a versatile digital disc or DVD (from English Digital Versatile Disc), a removable disk, a hard disk drive, a smart card, a flash memory device (e.g., a card, a key), a magnetic tape, a database, a server, and another suitable storage medium.

Alternatively, program instructions are taken from an external source and downloaded over a network. This is particularly the case for applications. In this case, the computer program product comprises a computer-readable data carrier on which the program instructions are stored or a data carrier signal on which the program instructions are encoded.

A computer program product includes computer readable media 134 containing instructions which, when executed by the circuit, cause the system 100 to implement the step of the method of assisting in the configuration according to the invention.

The form of the program instructions is, for example, a source code form, a computer executable form or any intermediate form between a source code and a computer executable form, such as the form resulting from the conversion of the source code via a interpreter, assembler, compiler, linker or locator. Alternatively, the program instructions are microcode, firmware instructions, state definition data, integrated circuit configuration data (e.g. VHDL), or object code. Program instructions are written in any combination of one or more programming languages, for example, object-oriented programming language (C++, JAVA, Python), procedural programming language (C language for example).

The system 100 further comprises a user interface 120 comprising an input device 121 and an output device 122.

The user interface 120 includes an input device 121 to allow the user to enter data or commands so as to be able to interact with the programs according to the invention. The input device 121 comprises, for example, a keyboard or a pointing interface, such as a mouse, an optical pencil, a touchpad, a remote control, a voice recognition device, a haptic device, a gesture recognition.

The output device 122 is designed to return information to a user, sensory or electrical, such as, for example, visually or audibly. The output interface 122 includes, for example, a graphical interface. The output interface 122 can be the input device, for example, in the case of a touchscreen tablet.

The set of at least one communication device allows communication between the elements of the system 100 and possibly between at least one element of the system and a device external to the system 100. This communication device can establish a physical link between elements of the system. system 100 and/or between an element of the system 100 and a device external to the system 100 and/or a remote (wireless) communication link between elements of the system 100 and/or between an element of the system and a device external to the system 100.

Nomenclature :

ACQU1: First acquisition

ACQU2: second acquisition

IM1: plurality of first images

IM2: plurality of second images

IM3: plurality of third images

IM2: plurality of fourth images

TRAI1: image processing of the plurality of first images

TRAI2: processing of images of the plurality of second images

ALG1: first algorithm

ALG2: second algorithm

CCN1: first learning function

Ci: first class of movement

GEN1, 2, 3, 4, 5, 6, 7, 8, 9: generation steps

ID1: first identification

SQ1: first skeleton of the individual

SQ2: second skeleton of the individual

SQ _R : skeleton of the reference posture

SEG ₁ : first segment

SEG _R1 : first segment of the skeleton of the reference posture

Vs: threshold value

EVR1: visual error indicator

POS1: set of characteristic postures

POS1R: set of reference postures

ERR1: first error indicator

SEQ ₁ : first cinematic sequence

COMP1, COMP2: comparisons of the error indicator with the threshold value

ANG ₁ : angle measured between two segments

P ₁ : skeleton point

100: assistance system for performing physical exercises

110: optical device

120: user interface

121: User interface input device

122: User interface output device

130: first calculator

131: data processing module

132: memory

133: reader

134: computer-readable media

150: display

160: case

Claims (14)

Method for detecting a physical posture and generating an instruction, characterized in that it comprises the following series of steps:

• Acquisition (ACQU1) of a sequence of first images (IM1) of a first individual;

the sequence of steps including the following implemented by computer:

• Processing (TRAI1) of the first images (IM1) comprising automatic extraction of at least one useful area of the body of said first individual from each first image (IM1) to generate a sequence of second images (IM2) of the useful area;
• Identification (ID1) of a type of physical movement by a first learning function (CCN1) processing second images (IM2) so as to select a first class (C1) of movements among a plurality of classes of movements (Ci) ;
• First generation (GEN1) of a sequence of first skeletons (SQ1) of the first individual by application of a skeletonization algorithm to the sequence of second images (IM2), each first skeleton (SQ1) comprising a set of characteristic points of the skeleton of the useful area;
• Extraction (EXT) of characteristic skeletons (SQ _R ) from the sequence of first skeletons by a first algorithm (ALG1), the characteristic skeletons corresponding to characteristic postures (POS1) of the first movement class (C1);
• Second generation (GEN2) of a first posture error indicator (ERR1) by comparison of a set of reference postures (POS1R) defined for the respective characteristic postures and the set of characteristic postures (POS1) of said individual;
• Comparison (COMP1) of a value of the first error indicator (ERR ₁ ) with at least one predetermined threshold value;
• When the value of the error indicator (ERR ₁ ) is greater than or equal to the threshold value (V _S ), automatic generation (GEN3) of a first digital instruction to display a sequence of reference images representative of the first movement class (C1) at a first speed lower than a predefined speed threshold.

Method according to the preceding claim, comprising, when the value of the error indicator (ERR ₁ ) is less than the threshold value (V _S ), a step of generating (GEN4) a second digital instruction to inform the user of a movement instruction associated with a movement execution speed instruction by an output interface. Method according to any one of the preceding claims, in which the third generation of the posture error indicator (ERR₁) includes the following steps, for at least one of the characteristic skeletons:

• Measurement of a difference between a point of the characteristic skeleton (SQ _R ) and a segment of a skeleton of the reference posture (POS1R) defined for the characteristic posture associated with the characteristic skeleton; and or
• Measurement of a deviation of one or more segments of the characteristic skeleton (SQ _R ) in relation to one or more segments of the skeleton of the reference posture (POS1R); and or
• Measurement of an angular alignment of one or more segments of the characteristic skeleton (SQ _R ) in relation to one or more segments of the skeleton of one of the reference postures (POS1R) and/or in relation to a horizontal axis; and or
• Measurement of a time interval separating the moments of acquisition of the two characteristic skeletons (SQ _R ), and comparison of the time interval with a reference time interval characteristic of the movement class (C1); and or
• Measurement of the trajectory of a point of the skeleton (SQ1) and comparison of the trajectory with the trajectory of one or more points of the skeleton of the determined reference posture (POS1R).

Method according to any one of the preceding claims, in which the sequence of steps is repeated as long as the value of the error indicator (ERR ₁ ) is greater than or equal to the threshold value (V _S ), the sequence d the step comprising, during its repetition, the extraction, in the first movement class (C1) of a subsequence of the sequence of first skeletons, the first digital instruction being generated to display the subsequence. Method according to the preceding claim, in which the sequence of steps comprises storing the value of the error indicator in a memory, the sequence of steps comprising when repeated, the comparison of the value of the indicator to at least one value of the indicator stored in the memory during a repetition of the sequence of steps and the generation of a recommendation depending on the comparison. Method according to any one of claims 4 and 5, in which the first instruction triggers information to the user of a reduction in the speed of execution of the first kinematic sequence by an output interface. Method according to any one of claims 4 to 6, in which the first instruction triggers information to the user of a reduction in the number of repetitions of the movement to be performed. Method according to any one of the preceding claims, comprising in real time the display, on a screen, of a first visual error indicator (EVR ₁ ) representative of the first error indicator (ERR ₁ ). Method according to the preceding claim, comprising the generation of the first visual indicator (EVR ₁ ) from the error indicator (ERR ₁ ) and the display in real time, on a screen, of the first sequence of images or of the second sequence of images superimposed on the visual error indicator (EVR1). Method according to the preceding claim, characterized in that the visual error indicator (EVR1) comprises the display of:

• A kinematic translation indicator comprising a first position, a vector and a second position, the first position being representative of a position of a zone of the user's body at the start of the movement and the second representation being representative of the position of the same part of the user's body at the end of the movement; and or
• an avatar generated from the sequence of first skeletons (SQ ₁ ); and or
• an indication of an area to be reached materialized by a polygon or a shape.

Method according to any one of the preceding claims, comprising the automatic identification of an individual and loading a training profile relating to said individual, said automatic identification step being carried out by facial recognition. Method according to any one of the preceding claims, comprising a step of counting the number of repetitions of the first kinematic sequence. Method according to the preceding claim, comprising counting the number of repetitions of the movement during which the value of the error indicator (ERR ₁ ) is less than the threshold value (V _S ). System of assistance for carrying out physical exercises characterized in that it comprises:

• At least one optical device configured to perform the acquisition (ACQU1) of a plurality of first images (IM1) of an individual;
• At least a first calculator and a memory configured to perform:
  • A processing (TRAI1) of the first images (IM1) comprising an automatic extraction of at least one useful area of the body of said first individual from each first image (IM1) to generate a sequence of second images (IM2) of the useful area;
  • An identification (ID1) of a type of physical movement by a first learning function (CCN1) processing second images (IM2) so as to select a first class (C1) of movements among a plurality of classes of movements (Ci ) ;
  • A first generation (GEN1) of a sequence of first skeletons (SQ1) of the first individual by application of a skeletonization algorithm to the sequence of second images (IM2), each first skeleton (SQ1) comprising a set of characteristic points of the skeleton of the useful area;
  • An extraction of characteristic skeletons (SQ _R ) from the sequence of first skeletons by a first algorithm (ALG1), the characteristic skeletons corresponding to characteristic postures (POS1) of the first movement class (C1);
  • A second generation (GEN2) of a first posture error indicator (ERR1) by comparison of a set of reference postures (POS1R) defined for the respective characteristic postures and the set of characteristic postures (POS1) of said individual ;
  • A comparison (COMP1) of a value of the first error indicator (ERR ₁ ) with at least one predetermined threshold value;
  • When the value of the error indicator (ERR ₁ ) is greater than or equal to the threshold value (V _S ), an automatic generation (GEN3) of a first digital instruction to display a sequence of reference images representative of the first movement class (C1) at a first speed lower than a predefined speed threshold
• At least one display configured to read the sequence of first images and/or the sequence of second images; And
• At least one box comprising the optical device.