IT202100016847A1 - Method for generating a pattern of a person's locomotion path - Google Patents

Description

"Method for generating a diagram of a person's locomotion path"

DESCRIPTION

FOUNDATION

Embodiments of the present invention relate to a method of generating a pattern of a person's locomotion path. This method? particularly useful for reconstructing a person's gait. Gait assessments can be used, alone or with additional data, to analyze a person's locomotion and posture, to digitally reconstruct lower limb movements to diagnose risk of falls, asymmetries, and many more gait changes related to aging, frailty and to many other physiological aspects, including, but not limited to, accidents or neurological conditions.

The digital reconstruction of a person's locomotion and movement? known in the state of the art. In particular, ? commonly used not only in medicine and rehabilitation, but also in the field of special effects for film production, as well as? in animation. Indeed, the known system allows a character to be animated by acquiring the movement of an actor using a garment equipped with movement sensors, such as accelerometers. The sensors are in communication with a receiver on a computer, and are placed on the garment in sufficient numbers to be able to reconstruct the movement of the wearer from the movement of the sensors themselves.

The system described above can? also be used to reconstruct a person's gait. The main drawback of such a system, however, is the high cost, cos? as the need? to tailor the garment to each specific wearer. Indeed, the garment must be close to the skin, otherwise the sensors would move on the wearer's body and the system would lose accuracy.

BRIEF DESCRIPTION OF THE INVENTION

A first aspect of the invention ? then directed to a method of generating a pattern of a person's locomotion path. The method includes the steps of:

- provision of a boundary enclosing a walkable area;

- supply of a pair of footwear with detectable foot markers; - arrangement of an imaging device in view within the walkable area and orientation of the same towards the walkable area in order to substantially frame the entire walkable area;

- recording of a person wearing footwear and moving within the walkable area using the imaging device; - processing of the recording in order to extract the position of the foot markers on the walkable area during the movement;

- calculation of a pattern based on the variable position of foot markers in the recording.

A second aspect of the invention relates to a system configured to carry out the method according to the first aspect. The system includes footwear having foot markers; a unit? computer equipped with an image processing neural network and configured to connect to an imaging device configured to acquire a recording.

The above aspects gain the advantage of allowing a person's gait to be reconstructed without the need to to wear sensorized clothing, especially on the legs. Other sensored garments, such as sensored socks, can be used to provide additional or precision data, but without the need to to employ expensive sensor suits or additional external instruments commonly referred to in the industry as laboratory gait instruments.

BRIEF DESCRIPTION OF THE DRAWINGS

Further details and specific embodiments will refer to the attached drawings, where:

- figure 1 ? a schematic representation of a system adapted to carry out a method according to an embodiment of the present invention;

- figure 2 ? a diagram showing a functional representation of part of the method according to the present invention; And

- figure 3 ? a perspective view of a component of the system of figure 1.

DETAILED DESCRIPTION

The following description of exemplary embodiments refers to the accompanying drawings. The same reference numbers in different drawings identify identical or similar elements. The following detailed description does not limit the invention. Rather, the scope of protection of the invention? defined by the attached claims.

Throughout the description, reference to "an embodiment", or "embodiment" means that a particular feature, structure or peculiarity is described in relation to an embodiment ? included in at least one embodiment of the described object. Therefore, the presence of the phrases "in a single embodiment" or "in one embodiment" at various places throughout the specification do not necessarily refer to the same embodiment. Furthermore, the particular characteristics, structures and peculiarities? can be combined in any suitable way in one or more? embodiments.

Referring to the attached figures, verr? now described a method according to an embodiment of the present invention. This method has the aim of generating a diagram and the related digital trace of a person's locomotion path. In particular, this method been adapted to analyze a person's walking gait. However, in different embodiments of the invention, the mode of locomotion can? be of any type such as, for example, running, jogging, skipping, skipping and others.

In greater detail, the pattern generated by the method according to the present invention comprises a series of data relating to a person's gait. Primarily, the pattern includes the position in space and time of each foot.

Optionally, the schema can also include other data and statistics, calculated or inferred from the position of the feet. For example, the following data can be calculated or inferred from the position of the feet:

Parameter Definition

Step counting Step (single stocking x 2 or individual stockings)

Cadence Steps/minute

Center of Pressure Estimated X, Y coordinates of the Center of Pressure Elapsed time between two consecutive steps of the Step Duration

same foot

Elapsed time between the first contact of two Pace Cycles

consecutive strikes of the same foot

Time on the ground (TOG, Time elapsed between the first and last Time On The Ground contact) of the same support

Distance between the heel points of two supports Stride length

consecutive of the same foot

Distance between the current heel strike point Stride length

and the point of the heel resting on the opposite foot Distance between the heel point of the first step and Distance

the heel point of the last support

Support time Time elapsed between the first and last stationary contact with two consecutive support positions on the same foot.

Elapsed time between the last stance contact Current swing time and the first subsequent stance contact on the same foot

Maximum height of

Foot height during swing.

foot

-1 if the landing takes place with the heel first. heel factor

0 if the discharge to the ground ? floor.1 if first mtb.

Speed? (subject

Speed? by subject or by step.

step)

Speed? average Speed? / (leg length 0.5*). normalized Leg length per second (LL/sec).

Elapsed time between the last current support contact and the first subsequent support contact Single support

of the same foot. (== swing time of opposite foot)

Elapsed time between the current first support contact and the last previous support contact, Double support added to the elapsed time between the current last support contact and the next first support contact

Angle between the line of progression and the line that Toe

connects the heel point to the support point in/Toe

front. The corner ? positive for the finger out and out

negative for the finger inside.

Different embodiments of the method according to the invention allow a combination of some or all of the parameters identified above to be extracted.

The method includes a preliminary phase of supplying a border 2 which encloses a walkable area 3. This border 2 can? be of any type that pu? be identified visually as, for example, a line or series of markers drawn on the floor or other walkable surface.

According to one embodiment of the invention, the step of providing a boundary 2 can be performed by supplying a mat 15. This mat 15 can? be equipped with two or more? border markers 51a or a margin covering the whole perimeter of the mat 15 defining the border 2. It should be noted that, in case the mat 15 is sufficiently distinguishable from the surrounding surface (i.e. due to a different colour), Not ? no boundary marker needed.

Furthermore, the method comprises the step of placing an imaging device 4 in view within the walkable area 3. The imaging device 4 is then adjusted so that it faces the walkable area 3. Specifically, the imaging device 4 ? positioned in such a way as to substantially frame the entire walkable area 3. Pu? be used any type of imaging device 4, as long as? it can acquire a sequence of images or, preferably, a video. For example, imaging device 4 can? be a dedicated camera. Alternatively, the imaging device 4 can? be a camera on another device, such as a smartphone, tablet, or computer. The imaging device 4 ? by itself known to the person skilled in the art, therefore it will not be? described in detail in the context of this disclosure.

After the imaging device 4 ? been positioned as discussed above, it is used to acquire at least one image of the walkable area 3, so that the aforementioned border 2 can be traced on the image.

Pi? in detail, the border identification phase 2 on the image comprises the image processing sub-phase through one or more? image filter algorithms. Specifically, an Otsu filter is used to separate the walkway from the background and reduce noise on the image. A Median Disk filter is then applied to clean up the image. The edges are then detected through a canny filter. Finally, a Hough transformation is used to find the shortest lines fit that pass through the edges. Once you find the lines contained in the image, the four most? near the center are selected as boundary 2. The intersections between the lines that define boundary 2 on the image are then identified as the four corners of walkable area 3.

The method according to embodiments of the invention also comprises the steps of providing a pair of footwear 5 having detectable foot markers 6. Preferably, these foot markers 6 are placed on an area which can be easily detected by the imaging device 4 as, for example, the area above the heel, as schematically shown in figure 3.

Preferably, but not exclusively, the footwear 5 is a pair of socks, for example those described in publication WO2017120063A1. Alternatively, footwear 5 may be a pair of shoes.

It should be noted that the footwear 5 comprises pressure sensors 7, arranged on the soles so as to detect a pressure value exerted on the ground. If shoes are used, the pressure sensors can be applied to an insert that can? be placed in each shoe. More details on the use of the pressure sensors 7 will be provided in a later part of the present description.

The method according to the embodiments of the invention also comprises the step of registering a person wearing the footwear 5 and moving within the walkable area 3. This step is performed using the imaging device 4, previously positioned .

In detail, the registration pu? be a sequence of frames or still images 13 or more? preferably, a video recording. The registration ? taken such that the foot markers 6 are visible in most, preferably all, of the images 13.

The recording is then processed in order to extract the position of the foot markers 6 on the walkable area 3. Indeed, the processing step of the recording comprises the sub-step of identifying the foot markers 6 in each image 13.

Pi? in detail, ? You can use a neural network 8 to do this. Specifically, the neural network 8 ? It was built using the transfer learning technique, thus starting from a pre-trained network and with additional layers.

Pi? in detail, a precursor neural network? was taken as a VGG16 trained for the classification task of the 'ImageNet' data set. The neural network 8 is therefore based on the convolutional layers 9 of the precursor neural network. An input layer ? 12 ? positioned before the convolutional layers 9, and ? responsible for the initial processing of each image 13.

The convolutional layers 9 are connected to a flattening layer 10, and are preferably frozen (i.e. untrained) during training of the neural network. The flattening layer 10 ? placed in communication with two completely connected layers 11, which are configured to predict the position of the foot markers 6 on the basis of coordinates extracted in a previously processed image. The fully connected layers 11 of the neural network 8 are trained using the norm L2 of the difference vector between the predicted and actual coordinates. The output of neural network 8 ? then the position 14 of the foot markers 6 on a single image 13. These steps are repeated for each image 13 of the sequence, in order to extract a list of positions 14.

A pattern is calculated based on the variable position 14 of foot markers 6 in the recording. In fact, once the 6-foot markers have been identified on an image, their position in space can be extracted. because ? note the size of the walkable area 3, including the distance between the corners of the border 2, ? It is possible to choose a suitable homography so that the position of the corners of walkable area 3 in the image can be mapped to the actual position of the same corners. So, for example, given:

with ? = 1, 2, 3, 4 the positions of the corners of the boundary 2 in space

effective, and:

with ? = 1, 2, 3, 4 the positions of the corners of the walkway

on the image, the homography pu? be chosen so that:

For example, homography can be calculated using the OpenCV library. The position of the foot markers 6 in the effective space is then determined as

where K represents position 14 of foot marker 6 on the image and k ? the position of foot marker 6 in effective space. The position of each foot marker 6 in space is then listed in sequence according to the original order of the images 13, thus defining the scheme.

Optionally, the method also comprises the step of acquiring at least one pressure value continuously for each foot during the recording. This step is performed through the pressure sensors 7. With reference to figure 3, a pressure sensor 7? placed on the heel and two are placed on the front portion of the foot.

The pressure values can be synchronized with the diagram in order to extract additional data, in particular this allows to better characterize the contact of each foot on the ground, in particular by detecting not only the contact of the foot on the ground as a whole, but also determine the timing of distinct contact points as defined by the pressure sensors 7 on the plantar area of the footwear 5, as well as? to determine the exact amount? of pressure exerted on the individual points of contact, thus determining? better quality? of the gait.

Finally, it should be noted that the steps of registering and/or processing the registration and/or calculating the pattern may be performed sequentially or at least partially at the same time. In fact, in the second case pu? you don't need to permanently store the raw logging, just the schema. Advantageously, in this way the memory requirement for the execution of the method can be reduced.

Another aspect of the invention relates to a system 1 configured to perform the method described above. System 1 comprises footwear 5 having foot markers 6. System 1 also comprises a unit? processing system equipped with the neural network 8 and configured to be connected to an imaging device 4 for acquiring a recording.

Claims (10)

1. Method of generating a scheme of a locomotion path of a person, including the stages of: - provision of a boundary (2) enclosing a walkable area (3); - supply of a pair of shoes (5) having detectable foot markers (6); - arrangement of an imaging device (4) in view inside the walkable area (3) and orientation of the same towards the walkable area (3) in order to substantially frame the whole walkable area (3); - recording of a person wearing footwear (5) and moving within the walkable area (3) using the imaging device (4); - processing of the registration in order to extract the position (14) of the foot markers (6) during the movement; - calculation of a pattern based on the variable position (14) of the foot markers (6) in the recording. Method according to the preceding claim, characterized in that the steps of registering and/or processing the registration and/or calculating the pattern are performed at least partially at the same time. 3. Method according to any one of the preceding claims, characterized in that the registration step comprises the sub-step of acquiring an image sequence (13) with the imaging device (4), the registration processing step comprising the sub-step of identification of foot markers (6) in each image (13). 4. Method according to the preceding claim, characterized in that it further comprises the step of providing an image processing neural network (8) comprising the convolutional layers (9) of a pretrained VGG16 neural network; the step of identifying the foot markers (6) in each image (13) being performed by the neural network (8). 5. Method according to any one of the preceding claims, characterized in that the pair of shoes comprises pressure sensors (7) arranged on the soles so as to detect a pressure value exerted on the ground, the method also comprising the acquisition step in such a way continuous of at least one pressure value for each foot during the recording. 6. Method according to the preceding claim, characterized in that it comprises the step of synchronizing the pressure values with the scheme. 7. Method according to one of the preceding claims, characterized in that the step of supplying a border (2) is performed by supplying a mat (15) having two or more? boundary markers (15a) or having a margin covering the entire perimeter of the mat (15) defining said boundary (2). The method according to the preceding claim, characterized in that the step of providing a boundary (2) comprises the substeps of acquiring at least one image of the walkable area (3) and identifying the boundary (2) on the image. 9. Method according to the preceding claim, characterized in that the step of identifying the border (2) on the image comprises the substep of processing the image through one or more? image filter algorithms. The system (1) configured to perform the method according to any preceding claim, the system (1) comprising footwear (5) having foot markers (6); a unit? equipped with an image processing neural network (8) and configured to be connected to an imaging device (4) configured to acquire a recording.