WO2024162733A1

WO2024162733A1 - Method and system for estimating stride length by using plantar pressure data

Info

Publication number: WO2024162733A1
Application number: PCT/KR2024/001386
Authority: WO
Inventors: 민세동; 김영; 호종갑; 정승민
Original assignee: 순천향대학교 산학협력단
Priority date: 2023-02-03
Filing date: 2024-01-30
Publication date: 2024-08-08
Also published as: KR20240122252A

Abstract

Disclosed is a gait analysis method comprising the steps of: measuring plantar pressure data for each area of a foot by a plurality of sensors; preprocessing the plantar pressure data; deriving vector data of a pressure center point velocity for each area on the basis of the preprocessed data; generating a plantar pressure image in a two-dimensional matrix on the basis of the preprocessed data; estimating a stride length from the vector data of the pressure center point velocity for each area and the plantar pressure image in the two-dimensional matrix by using an artificial intelligence model; and displaying the estimated stride length and a gait analysis result.

Description

Method and system for estimating stride length using plantar pressure data

The following description relates to a method and system for estimating stride length using plantar pressure data, and more specifically, to a method and system for estimating stride length based on plantar pressure data and gait information derived from the data using artificial intelligence.

Walking is the most basic means of human movement and is an act that occurs through complex and organic movements of joints, muscles, and nerves throughout the body. Analyzing walking can help determine the user's motor skills, the possibility of developing specific diseases, and walking health levels. In order to comprehensively analyze walking, dynamic indicators such as walking speed and knee joint angle must be analyzed together with the pressure acting under the foot (plantar pressure) and the power or balance-related indicators that evaluate its balance. In specialized walking analysis institutions such as hospitals and research institutes, both the balance and dynamic indicators of walking can be precisely evaluated using force plates that can measure foot pressure, along with observations by experts, motion capture equipment, and cameras. However, there is the inconvenience of the subject having to visit a specialized institution, and there is also the disadvantage that data is acquired in the measurement environment rather than data measured during real life. To solve these problems, technologies have been developed that aim to collect and evaluate walking data in everyday life non-face-to-face through wearable sensors.

Conventional wearable gait analysis technologies use an insole-type plantar pressure sensor and an IMU (Inertrial Measurement Unit) that can measure data such as acceleration to comprehensively evaluate balance and dynamic indices. However, IMUs are sensitive to shock, so there are restrictions on embedding them in insoles. In addition, there is no consensus or standard for the optimal IMU measurement location for evaluating gait, making it difficult to select a measurement location. In addition, the method of wearing IMUs on the user's knees, calves, thighs, etc. significantly reduces user convenience and compliance, making it unsuitable for monitoring daily gait in the long term.

Accordingly, a technology is required that can predict dynamic indicators such as knee joint angle, stride length, and walking speed using only plantar pressure data.

The present invention aims to provide a method and system for accurately estimating stride length based on vector data derived from plantar pressure data and image data of a two-dimensional matrix by utilizing an artificial intelligence learning model.

As a technical means for achieving the above-described task, a first aspect of the present invention can provide a gait analysis method including a step of measuring plantar pressure data for each region of a foot by a plurality of sensors, a step of preprocessing the plantar pressure data, a step of deriving vector data of center of pressure velocity for each region of the foot based on the preprocessed data, a step of generating a two-dimensional matrix of plantar pressure images based on the preprocessed data, a step of estimating a stride from the vector data of center of pressure velocity for each region and the two-dimensional matrix of plantar pressure images using an artificial intelligence model, and a step of displaying the estimated stride and a gait analysis result.

Additionally, the method may further include a step of estimating a lower extremity joint angle from vector data of a pressure center point velocity by region and a plantar pressure image of the two-dimensional matrix using an artificial intelligence model.

Additionally, the preprocessed data may include heel strike information and toe off information.

Additionally, the foot region may include the phalanges region, the metatarsals region, the mid-foot region, and the heel region.

Additionally, the method may include a step of inputting vector data of the velocity of the center of pressure point by region into a convolutional neural network (CNN), and inputting the plantar pressure image of the two-dimensional matrix into a feed-forward neural network (FNN) to output a lower limb joint angle.

Additionally, the method may further include a step of inputting a two-dimensional matrix of plantar pressure images into the CNN, obtaining features, and inputting the lower extremity joint angles into the FNN to output a stride.

Additionally, the method may include a step of displaying at least one of average stride, walking speed, walking distance, walking cycle, walking count, center of pressure, and plantar pressure distribution information.

A second aspect of the present invention can provide a wearable device including a sensing unit that measures plantar pressure data for each area of the foot, and a communication unit that transmits the plantar pressure data to a server for gait analysis.

A third aspect of the present invention comprises a communication unit which receives plantar pressure data from a wearable device, and a control unit which preprocesses the plantar pressure data, derives vector data of center of pressure velocity for each region of the foot based on the preprocessed data, generates plantar pressure image data of a two-dimensional matrix based on the preprocessed data, and estimates a stride from the vector data of center of pressure velocity for each region and the plantar pressure image of the two-dimensional matrix using an artificial intelligence model, wherein the communication unit can provide a server which transmits the estimated stride and gait analysis results to a mobile device.

A fourth aspect of the present invention can provide a mobile device including a communication unit that receives stride and gait analysis results from a server, and a display unit that displays stride and gait analysis results.

The present invention has a technical effect of more accurately estimating an individual's stride and analyzing walking using only plantar pressure data without an IMU sensor by utilizing artificial intelligence. Based on this, the stride information obtained easily can be utilized in areas such as providing health information in daily life, health monitoring systems, and diagnosing and predicting diseases related to walking.

FIG. 1 is a diagram illustrating an example of a stride estimation system using plantar pressure data according to one embodiment.

FIG. 2 is a flowchart illustrating a method for estimating stride length using plantar pressure data according to one embodiment.

FIG. 3 is a diagram showing an example of preprocessing plantar pressure data according to one embodiment.

FIGS. 4 and 5 are diagrams showing examples of generating a two-dimensional matrix of plantar pressure images from preprocessed data according to one embodiment.

Figure 6 is a diagram showing an example of deriving vector data of the velocity of the center of pressure point by region of the foot according to one embodiment.

FIG. 7 is a diagram illustrating an example of an artificial intelligence model for estimating stride according to one embodiment.

FIG. 8 is a block diagram illustrating a configuration of a wearable device according to one embodiment.

FIG. 9 is a block diagram illustrating the configuration of a server according to one embodiment.

FIG. 10 is a block diagram illustrating a configuration of a mobile device according to one embodiment.

The present invention can be modified in various ways and has various embodiments, and specific embodiments are illustrated in the drawings and described in detail. However, this is not intended to limit the present invention to specific embodiments, but should be understood to include all modifications, equivalents, or substitutes included in the spirit and technical scope of the present invention. In describing each drawing, similar reference numerals are used for similar components.

The terms first, second, A, B, etc. may be used to describe various components, but the components should not be limited by the terms. The terms are only used to distinguish one component from another. For example, without departing from the scope of the present invention, the first component may be referred to as the second component, and similarly, the second component may also be referred to as the first component. The term and/or includes any combination of a plurality of related described items or any item among a plurality of related described items.

When it is said that a component is "connected" or "connected" to another component, it should be understood that it may be directly connected or connected to that other component, but that there may be other components in between. On the other hand, when it is said that a component is "directly connected" or "directly connected" to another component, it should be understood that there are no other components in between.

The terminology used in this application is only used to describe specific embodiments and is not intended to limit the present invention. The singular expression includes the plural expression unless the context clearly indicates otherwise. In this application, the terms "comprises" or "has" and the like are intended to specify the presence of a feature, number, step, operation, component, part or combination thereof described in the specification, but should be understood to not exclude in advance the possibility of the presence or addition of one or more other features, numbers, steps, operations, components, parts or combinations thereof.

Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. Terms defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant art, and will not be interpreted in an idealized or overly formal sense unless expressly defined in this application.

The present embodiments relate to a method and system for estimating stride length using plantar pressure data. Details that are widely known to those skilled in the art to which the embodiments pertain will be omitted. The present invention will be described in detail below with reference to the attached drawings.

Referring to FIG. 1, a stride estimation system according to one embodiment may include a wearable device (100) that senses plantar pressure of a user's foot, a server (200) that estimates stride based on plantar pressure data, and a mobile device (300) that displays stride and gait analysis results received from the server. For example, the wearable device (100) may be a wearable type worn on the sole of the user's foot in the form of an insole or a sock, but is not limited thereto, and may include all types of devices that can sense the user's plantar pressure and transmit plantar pressure data to the server. In one embodiment, the wearable device (100) and the server (200) may be connected wirelessly or by wire through communication units (110, 210) to transmit and receive data.

The server (200) is a computer equipped with a data processing function to estimate stride length from plantar pressure data using artificial intelligence, and is widely known to those skilled in the art, so a detailed description thereof will be omitted. In addition, the server (200) may communicate with a wearable device (100) and a mobile device (300) through a predetermined network to use information necessary for estimating stride length and to display stride and gait analysis results. In this case, the network includes a Local Area Network (LAN), a Wide Area Network (WAN), a Value Added Network (VAN), a mobile radio communication network, a satellite communication network, and a combination thereof, and is a comprehensive data communication network that allows each network component to communicate smoothly with each other, and may include wired Internet, wireless Internet, and mobile radio communication networks. Wireless communications may include, but are not limited to, wireless LAN (Wi-Fi), Bluetooth, Bluetooth low energy, Zigbee, WFD (Wi-Fi Direct), UWB (ultra wideband), infrared communication (IrDA, infrared Data Association), NFC (Near Field Communication), etc.

The mobile device (300) may be a smart phone, a tablet PC, a PC, a smart TV, a mobile phone, a PDA (personal digital assistant), a laptop, a media player, a GPS (global positioning system) device, an e-book reader, a digital broadcasting terminal, a navigation device, a kiosk, an MP3 player, a digital camera, a home appliance, or any other mobile or non-mobile computing device. However, the mobile device (300) is not limited thereto, and may include any type of device capable of receiving stride and gait analysis results from a server and displaying the same.

In step S210, a plurality of sensors may measure plantar pressure data for each region of the foot. In one embodiment, the wearable device (100) may measure plantar pressure data for each region of the foot using a plurality of pressure sensors. For example, the wearable device (100) may be in the form of an insole or a sock, and the plurality of pressure sensors may be arranged in each of the phalanges region, the metatarsal region, the midfoot region, and the heel region of the foot, thereby allowing plantar pressure to be measured more accurately.

In step S220, plantar pressure data can be preprocessed. An example of a server (200) receiving plantar pressure data from a wearable device (100) and preprocessing the plantar pressure data will be described later with reference to FIG. 3.

In step S230, vector data of center of pressure velocity can be derived for each region based on the preprocessed data. In one embodiment, the preprocessed data can include heel strike information and toe off information. An example of the server (200) deriving vector data of center of pressure velocity for each region of the foot will be described later with reference to FIG. 6.

In step S240, a two-dimensional matrix of plantar pressure images can be generated based on the preprocessed data. An example of a server (200) generating a two-dimensional matrix of plantar pressure images will be described later with reference to FIG. 5.

In step S250, a step length can be estimated from vector data of the pressure center point velocity by region and a two-dimensional matrix of plantar pressure images using an artificial intelligence model. Here, the step length can be a step length, which is a distance from one heel to the opposite heel, or a stride length, which is a distance from one heel to the same heel.

In addition, the artificial intelligence model can be used to estimate the lower extremity joint angle from the vector data of the pressure center point velocity by region and the plantar pressure image of the two-dimensional matrix. The lower extremity joint angle can include the hip angle, the knee angle, and the ankle angle. An example of the server (200) estimating the stride using the artificial intelligence model will be described later with reference to FIG. 7.

In step S260, the estimated stride and gait analysis results can be displayed.

In one embodiment, the mobile device (300) may receive and display stride and gait analysis results from the server (200). Additionally, the mobile device (300) may display at least one of average stride, walking speed, walking distance, walking cycle, number of walking steps, center of pressure, and plantar pressure distribution information.

For example, the mobile device (300) can display gait information, gait cycle information, gait index information, etc. derived from the results of estimating stride length and lower extremity joint angle by the server (200). At this time, the gait information derived from the results of estimating stride length and lower extremity joint angle can include stride length, step length, average stride distance, average step distance, gait velocity, hip angle, knee angle, ankle angle, etc.

Gait cycle information may include stride time, step time, stance phase, swing phase, double support time, single support time, and stance/swing ratio.

Gait parameters may include step count, cadence, gait asymmetry, phase coordination index (PCI), coefficient of variance (CV), plantar pressure difference (PPD), center of pressure (COP), and ratio of plantar pressure distribution.

The server (200) can preprocess the plantar pressure data received from the wearable device (100) so that the plantar pressure data obtained can be used for estimating the stride. First, when the server (200) receives raw data from the wearable device (100), it can normalize the data to remove an offset and facilitate signal processing. In addition, the server (200) can remove an outlier element to remove saturated data. Thereafter, the server (200) can pass the data through a low-pass filter (4th butter-worth, cut-off frequency = 3 Hz) to filter high-frequency data to secure complete gait data. In addition, the server (200) can use a derivative filter to obtain slope information and amplify the data and perform Data Squared.

By performing this preprocessing, the point where the pressure rises can be determined as a heel strike through the slope information, and the peak point can be determined as a toe off through the peak detection algorithm.

Fig. 4 shows preprocessed plantar pressure data, heel strike information and toe-off information acquired in the data preprocessing process, and gait cycle phases. Fig. 5 shows a plantar pressure image of a two-dimensional matrix generated by converting the preprocessed plantar pressure data of Fig. 4 and a gait cycle phase. Referring to Fig. 5, the plantar pressure image of the two-dimensional matrix is an image that intuitively displays the location where pressure is applied to the sole of the foot according to the gait cycle phase. This plantar pressure image of the two-dimensional matrix can be used as an input for an artificial intelligence model, which will be described later with reference to Fig. 7. A square in the plantar pressure image of the two-dimensional matrix indicates the location of the pressure, and a lower brightness of the square means a greater magnitude of the applied pressure.

Figure 6 is a diagram showing an example of the center of pressure point velocity by region of the foot according to one embodiment.

Referring to Figure 6, the center of pressure (COP) velocity for each region of the phalanges region, metatarsal region, mid-foot region, and heel region of the foot is depicted by red arrows.

The vector data of the center of pressure velocity by foot area is different for each individual depending on body type, weight, leg length, foot structure, etc., and even for the same person, it is different depending on changes in body type such as before and after childbirth. Therefore, by separating the foot area and obtaining the vector data of the center of pressure velocity to estimate the stride, it is possible to more accurately analyze the walking habits according to the individual's body type.

Referring to the mathematical expression 1 below, the vector data of the velocity of the center of pressure is expressed in x-coordinates and y-coordinates, and the method of obtaining the x-coordinates and y-coordinates of the center of pressure velocity is shown in the mathematical expression 2.

[Mathematical formula 1]

[Mathematical formula 2]

Assuming that the sampling rate for acquiring plantar pressure data in mathematical expression 2 is 100 Hz, 100 plantar pressure data are acquired per second, so 0.01 is used in the denominator, and the numerator is the movement distance of the center of pressure (the difference in coordinates between the current location and the previous location). This vector data of the center of pressure velocity for each area of the foot can be used as an input for an artificial intelligence model, which will be described later with reference to Fig. 7.

Looking at Figure 7, the inputs to the AI model are plantar pressure data, vector data of the center of pressure point velocity by region of the foot derived from the data, and a two-dimensional matrix of plantar pressure images, and the outputs of the AI model are lower extremity joint angles and stride length. At this time, the network that outputs the lower extremity joint angles and the network that outputs the stride length can be used separately, but in order to improve accuracy, the two networks will be used together.

First, the plantar pressure data and the vector data of the velocity of the pressure center point by region are input into the FC layer (fully-connected layer) to extract features, and the two-dimensional matrix of the plantar pressure image is input into the CNN (convolutional neural network) to extract features and combine them to derive the lower extremity joint angle.

Afterwards, the two-dimensional matrix of plantar pressure images from above can be input into CNN to extract features and also the lower limb joint angle derived from above can be input into another FNN to estimate stride length. The results output through the artificial intelligence model can be stored in the memory of the server (200) and can also be transmitted to a mobile device (300) for display to the user.

As illustrated in FIG. 8, a wearable device (100) according to one embodiment may include a communication unit (110) and a sensing unit (120). However, not all of the components illustrated in FIG. 8 are essential components of the wearable device (100). The wearable device (100) may be implemented with more components than the components illustrated in FIG. 8, or may be implemented with fewer components than the components illustrated in FIG. 8.

The communication unit (110) may include one or more components that allow the wearable device (100) to communicate with another device (not shown) and a server (200). The other device (not shown) may be a computing device such as the server (200) or a display device, but is not limited thereto.

In one embodiment, the communication unit (110) may transmit and receive information necessary for estimating plantar pressure data and stride length with another device (not shown) and a server (200). For example, the other device may be another device of the user of the wearable device (100), and the other device of the user may be a mobile device for outputting the user's walking status.

The sensing unit (120) may include, but is not limited to, a plurality of pressure sensors. In one embodiment, the plurality of pressure sensors may be piezoresistive and/or capacitive pressure sensors, and at least one pressure sensor may be positioned in each of the phalanges region, the metatarsal region, the midfoot region, and the heel region of the foot. This has the advantage that plantar pressure data can be measured more accurately in each region.

As illustrated in FIG. 9, a server (200) according to one embodiment may include a communication unit (210), a memory (220), and a control unit (230). However, not all of the components illustrated in FIG. 9 are essential components of the server (200). The server (200) may be implemented with more components than the components illustrated in FIG. 9, or may be implemented with fewer components than the components illustrated in FIG. 9. The above components will be described in turn below.

In one embodiment, the communication unit (210) can transmit and receive information necessary for estimating stride length to and from the wearable device (100), and can transmit and receive stride and gait analysis results to and from the mobile device (300).

The memory (220) can store a program for processing and controlling the control unit (230), and can also store data input to or output from the server (200).

The memory (220) may include at least one type of storage medium among a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (for example, an SD or XD memory, etc.), a RAM (Random Access Memory), a SRAM (Static Random Access Memory), a ROM (Read-Only Memory), an EEPROM (Electrically Erasable Programmable Read-Only Memory), a PROM (Programmable Read-Only Memory), a magnetic memory, a magnetic disk, and an optical disk.

The control unit (230) typically controls the overall operation of the server (200). The control unit (230) may have at least one processor. Depending on its function and role, the control unit (230) may include a plurality of processors or may include one processor in an integrated form. The processor may mainly mean a central processing unit (CPU), an application processor (AP), a graphics processing unit (GPU), etc. In addition, the CPU, AP, or GPU may include one or more cores therein, and the CPU, AP, or GPU may operate using an operating voltage and a clock signal. However, while the CPU or AP may be composed of several cores optimized for serial processing, the GPU may be composed of thousands of smaller and more efficient cores designed for parallel processing.

For example, the control unit (230) can control the user input unit (not shown), the output unit (not shown), the sensing unit (not shown), the communication unit (not shown), the A/V input unit (not shown), etc., in general, by executing the programs stored in the memory (220). In addition, the control unit (230) can cause the server (200) to estimate the stride length and derive the gait analysis result by analyzing the plantar pressure data. In addition, the control unit (230) according to one embodiment can learn the criteria for determining how to estimate the stride length by analyzing the plantar pressure data.

As illustrated in FIG. 10, a mobile device (300) according to one embodiment may include a communication unit (310) and a display unit (320). However, not all of the components illustrated in FIG. 10 are essential components of the mobile device (300). The mobile device (300) may be implemented with more components than the components illustrated in FIG. 10, or the mobile device (300) may be implemented with fewer components than the components illustrated in FIG. 10.

The communication unit (310) may include one or more components that allow the mobile device (300) to communicate with another device (not shown) and the server (200). The other device (not shown) may be a computing device such as the server (200) or a sensing device, but is not limited thereto. In one embodiment, the communication unit (310) may transmit and receive stride and gait analysis results to and from the server (200).

The display unit (320) displays and outputs information received from the mobile device (300). For example, the display unit (320) can display stride and gait analysis results received from the server (200).

Meanwhile, the configurations of the wearable device (100), the server (200), and the mobile device (300) illustrated in FIGS. 8 to 10 are only examples, and each component may be integrated, added, or omitted depending on the specifications of the wearable device (100), the server (200), and the mobile device (300) being implemented. That is, two or more components may be combined into one component, or one component may be divided into two or more components and configured. In addition, the functions performed by each component (or module) are for explaining examples, and the specific operations or devices thereof do not limit the scope of the present invention.

An embodiment may also be implemented in the form of a recording medium containing computer-executable instructions, such as program modules, that are executed by a computer. Computer-readable media can be any available media that can be accessed by a computer and includes both volatile and nonvolatile media, removable and non-removable media. Additionally, computer-readable media can include both computer storage media and communication media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information, such as computer-readable instructions, data structures, program modules, or other data. Communication media typically includes computer-readable instructions, data structures, program modules, or other data in a modulated data signal, such as a carrier wave, or other transport mechanism, and includes any information delivery media.

The above description of the present invention is for illustrative purposes, and those skilled in the art will understand that the present invention can be easily modified into other specific forms without changing the technical idea or essential characteristics of the present invention. Therefore, it should be understood that the embodiments described above are exemplary in all respects and not restrictive. For example, each component described as a single component may be implemented in a distributed manner, and likewise, components described as distributed may be implemented in a combined manner.

The scope of the present invention is indicated by the claims described below rather than the detailed description above, and all changes or modifications derived from the meaning and scope of the claims and their equivalent concepts should be interpreted as being included in the scope of the present invention.

Claims

A step in which multiple sensors measure plantar pressure data for each area of the foot;

A step of preprocessing the measured plantar pressure data;

A step of deriving vector data of the pressure center velocity for each region based on the above preprocessed data;

A step of generating a two-dimensional matrix of plantar pressure images based on the above preprocessed data;

A step of estimating a stride from vector data of the pressure center point velocity by region and the plantar pressure image of the two-dimensional matrix using an artificial intelligence model; and

A step of displaying the estimated stride and gait analysis results;

A gait analysis method comprising:
In the first paragraph,

A step of estimating the lower extremity joint angle from the vector data of the pressure center point velocity by region and the plantar pressure image of the two-dimensional matrix using an artificial intelligence model;

A gait analysis method further comprising:
In the first paragraph,

A gait analysis method, wherein the above preprocessed data includes heel strike information and toe off information.
In the first paragraph,

A gait analysis method, wherein the above foot region includes the phalanges region, the metatarsal region, the mid-foot region, and the heel region.
In the second paragraph,

The step of estimating the above lower limb joint angle is:

A step of inputting vector data of the velocity of the center of pressure point for each region into an FC (fully-connected layer), and inputting the plantar pressure image of the two-dimensional matrix into a CNN (convolutional neural network) to output the lower extremity joint angle;

A gait analysis method comprising:
In paragraph 5,

The step of estimating the above stride is:

A step of inputting the plantar pressure image of the above two-dimensional matrix into the CNN, obtaining features, and inputting the lower limb joint angle into the FNN to output the stride;

A gait analysis method comprising:
In the first paragraph,

The step of displaying the above estimated stride and gait analysis results is:

A gait analysis method, comprising the steps of displaying average stride length, walking speed, walking distance, gait cycle, number of steps, center of pressure, and plantar pressure distribution information.
A sensing unit that measures plantar pressure data for each area of the foot; and

A communication unit that transmits the plantar pressure data to a server for gait analysis;

A wearable device comprising:
A communication unit for receiving plantar pressure data from a wearable device; and

Preprocess the above plantar pressure data,

Based on the above preprocessed data, vector data of the pressure center velocity is derived for each foot region,

Based on the above preprocessed data, a two-dimensional matrix of plantar pressure images is generated, and

A control unit for estimating a stride from vector data of the velocity of the center of pressure point for each region and a plantar pressure image of the two-dimensional matrix using an artificial intelligence model;

Including,

The above communication unit is a server that transmits the estimated stride and gait analysis results to a mobile device.
A communication unit for receiving stride and gait analysis results from a server; and

A display section for displaying the above stride and gait analysis results;

A mobile device, including: