CN115830718A

CN115830718A - Data processing system for predicting rehabilitation training effect based on gait recognition

Info

Publication number: CN115830718A
Application number: CN202310107540.9A
Authority: CN
Inventors: 李灿东; 杨朝阳; 唐志伟; 赖新梅; 周常恩; 辛基梁
Original assignee: Fujian University of Traditional Chinese Medicine
Current assignee: Fujian University of Traditional Chinese Medicine
Priority date: 2023-02-14
Filing date: 2023-02-14
Publication date: 2023-03-21
Anticipated expiration: 2043-02-14
Also published as: CN115830718B

Abstract

The application relates to the field of data processing, and particularly discloses a data processing system for predicting a rehabilitation training effect based on gait recognition. Therefore, the rehabilitation training effect of the patient can be accurately evaluated to track the progress of the disease and accurately judge the recovery condition of the patient.

Description

Data processing system for predicting rehabilitation training effect based on gait recognition

Technical Field

The present application relates to the field of data processing, and more particularly, to a data processing system for predicting a rehabilitation training effect based on gait recognition.

Background

In recent years, chronic diseases are slowly threatening the health of the elderly. According to research, alzheimer's Disease (AD), commonly known as senile dementia, has become one of the third major disease killers of the elderly after heart disease and cancer. Mild Cognitive Impairment (MCI) is an early state of AD, and is particularly important for early assessment and rehabilitation training of MCI because of its irreversibility and the rise in the number of patients year after year.

The mechanism that provides rehabilitation training at present is mostly rehabilitation center, and the mode of adoption is mostly traditional rehabilitation measure. For example, language interactive training is performed with the help of medical staff, and the training is used for recovering social cognitive abilities of the patient, such as memory, logical thinking, judgment and the like; and the limb mechanical training is used for recovering the movement coordination ability of the patient. However, due to the shortage of corresponding medical staff, the time for the patient to get to or get from the rehabilitation center is long, and the nonstandard phenomenon in the operation of the staff causes the efficiency of the rehabilitation work to be low, the rehabilitation training effect of the patient is difficult to evaluate, and the patient misses the best rehabilitation opportunity.

Accordingly, an optimized data processing system for predicting the effectiveness of rehabilitation training is desired.

Disclosure of Invention

The present application is proposed to solve the above-mentioned technical problems. The embodiment of the application provides a data processing system and a data processing method for predicting a rehabilitation training effect based on gait recognition, wherein an artificial intelligence algorithm based on deep learning is adopted to respectively extract multi-scale relevance feature distribution information of each parameter item of a left foot gait parameter and a right foot gait parameter of a patient based on long-distance dependence and medium-short distance dependence, and gait is further analyzed and judged based on the cooperative features of the left foot gait parameter and the right foot gait parameter, so that the rehabilitation training effect is evaluated. Therefore, the rehabilitation training effect of the patient can be accurately evaluated to track the progress of the disease and accurately judge the recovery condition of the patient.

According to an aspect of the present application, there is provided a data processing system for predicting a rehabilitation training effect based on gait recognition, including:

the gait parameter acquisition module is used for acquiring left foot gait parameters and right foot gait parameters of a patient to be evaluated, wherein the left foot gait parameters and the right foot gait parameters comprise a step phase, a landing position, a pressure central point track, a pace speed, a step frequency, a step length, a steering angle, double lower limb symmetry, a swing phase time proportion and a support phase time proportion;

the left-foot-state semantic coding module is used for enabling the left-foot-state parameters to pass through a context coder based on a converter to obtain a plurality of left-foot-state data feature vectors;

the first scale feature extraction module is used for cascading the left-foot gait data feature vectors to obtain a first left-foot gait semantic feature vector;

the second scale feature extraction module is used for inputting the plurality of left-foot gait data feature vectors into the bidirectional long-short term memory neural network model to obtain a second left-foot gait semantic feature vector;

a left-step multi-scale semantic feature fusion module, configured to fuse the first left-step semantic feature vector and the second left-step semantic feature vector to obtain a left-step semantic feature vector;

the right foot gait multiscale semantic feature extraction module is used for obtaining a right foot gait semantic feature vector from the right foot gait parameters through the context encoder based on the converter and the bidirectional long and short term memory neural network model;

the cooperation module is used for calculating a cooperation gait correlation characteristic matrix between the right foot gait semantic characteristic vector and the left foot gait semantic characteristic vector; and

and the data processing result generating module is used for enabling the cooperative gait associated characteristic matrix to pass through a classifier to obtain a classification result, and the classification result is used for representing a grade label of the rehabilitation training effect.

In the data processing system for predicting a rehabilitation training effect based on gait recognition, the left-foot gait semantic coding module includes: the word segmentation unit is used for performing word segmentation processing on the left step state parameter so as to convert the left step state parameter into a word sequence consisting of a plurality of words; an embedding encoding unit for mapping individual words in the sequence of words to word vectors using an embedding layer of the converter-based context encoder to obtain a sequence of word vectors; a context encoding unit for global context-based semantic encoding the sequence of word vectors using a converter of the converter-based context encoder to obtain the plurality of left-foot-state data feature vectors.

In the data processing system for predicting the rehabilitation training effect based on gait recognition, the left-step multi-scale semantic feature fusion module is further configured to: fusing the first left-step semantic feature vector and the second left-step semantic feature vector according to the following formula to obtain a left-step semantic feature vector; wherein the formula is:

wherein

Representing the first left-footed semantic feature vector,

representing the second left-footed semantic feature vector,

representing the left foot state semantic feature directionThe amount of the compound (A) is,

weighting parameters representing the first left-foot-state semantic feature vector and the second left-foot-state semantic feature vector, respectively,

indicating a sum by location.

In the data processing system for predicting the rehabilitation training effect based on gait recognition, the coordination module includes: the initial association feature matrix calculation unit is used for performing association coding on the right foot gait semantic feature vector and the left foot gait semantic feature vector to obtain an initial association feature matrix; a core walking node distribution fusion feature matrix calculation unit, configured to calculate a core walking node distribution fusion feature matrix between the right foot gait semantic feature vector and the left foot gait semantic feature vector, where the core walking node distribution fusion feature matrix is related to a distance matrix between the right foot gait semantic feature vector and the left foot gait semantic feature vector; and the fusion unit is used for fusing the initial correlation characteristic matrix and the graph core wandering node distribution fusion characteristic matrix to obtain the cooperative gait correlation characteristic matrix.

In the data processing system for predicting a rehabilitation training effect based on gait recognition, the initial correlation characteristic matrix calculating unit is further configured to: performing correlation coding on the right foot gait semantic feature vector and the left foot gait semantic feature vector by the following formula to obtain an initial correlation feature matrix;

wherein the formula is:

wherein

Representing the right foot gait semantic feature vector,

representing the left-footed semantic feature vector,

representing the initial correlation characteristic matrix in the initial correlation characteristic matrix,

is the multiplication of a matrix by a vector.

In the data processing system for predicting the rehabilitation training effect based on gait recognition, the graph core wandering node distribution fusion feature matrix calculation unit is further configured to: calculating the graph core wandering node distribution fusion feature matrix between the right foot gait semantic feature vector and the left foot gait semantic feature vector according to the following formula; wherein the formula is:

wherein ,

representing the right foot gait semantic feature vector,

representing the left-footed semantic feature vector,

representing the graph core walk node distribution fusion feature matrix,

is a distance matrix between the right foot gait semantic feature vector and the left foot gait semantic feature vector, and

and

are all column vectors, and are,

an exponential operation representing a matrix that calculates a natural exponent function value raised to a characteristic value of each position in the matrix,

is the vector multiplied by the vector.

In the data processing system for predicting the rehabilitation training effect based on gait recognition, the fusion unit is further configured to calculate the position-based point between the initial association feature matrix and the graph kernel wandering node distribution fusion feature matrix to obtain the cooperative gait association feature matrix.

In the data processing system for predicting the rehabilitation training effect based on gait recognition, the data processing result generating module includes: the unfolding unit is used for unfolding the cooperative gait correlation characteristic matrix into classification characteristic vectors based on row vectors or column vectors; a full-concatenation encoding unit, configured to perform full-concatenation encoding on the classification feature vector using a plurality of full-concatenation layers of the classifier to obtain an encoded classification feature vector; and the classification result generating unit is used for enabling the coded classification feature vector to pass through a Softmax classification function of the classifier to obtain the classification result.

According to another aspect of the present application, there is provided a data processing method for predicting a rehabilitation training effect based on gait recognition, including:

acquiring left foot gait parameters and right foot gait parameters of a patient to be evaluated, wherein the left foot gait parameters and the right foot gait parameters comprise a step phase, a landing position, a pressure center point track, a step speed, a step frequency, a step length, a steering angle, double lower limb symmetry, a swing phase time proportion and a support phase time proportion;

passing the left foot gait parameters through a converter-based context encoder to obtain a plurality of left foot gait data feature vectors;

cascading the plurality of left-foot gait data feature vectors to obtain a first left-foot gait semantic feature vector;

inputting the plurality of left-foot gait data feature vectors into a bidirectional long-short term memory neural network model to obtain a second left-foot gait semantic feature vector;

fusing the first left-foot-state semantic feature vector and the second left-foot-state semantic feature vector to obtain a left-foot-state semantic feature vector;

obtaining a right foot gait semantic feature vector from the right foot gait parameter through the converter-based context encoder and the bidirectional long-short term memory neural network model;

calculating a collaborative gait correlation characteristic matrix between the right foot gait semantic characteristic vector and the left foot gait semantic characteristic vector; and

and passing the cooperative gait correlation characteristic matrix through a classifier to obtain a classification result, wherein the classification result is used for representing a grade label of a rehabilitation training effect.

According to still another aspect of the present application, there is provided an electronic apparatus including: a processor; and a memory having stored therein computer program instructions which, when executed by the processor, cause the processor to perform the data processing method of predicting a rehabilitation training effect based on gait recognition as described above.

According to yet another aspect of the present application, there is provided a computer readable medium having stored thereon computer program instructions which, when executed by a processor, cause the processor to perform the data processing method of predicting a rehabilitation training effect based on gait recognition as described above.

Compared with the prior art, the gait recognition based data processing system and method for predicting the rehabilitation training effect provided by the application can be used for respectively extracting the multi-scale relevance feature distribution information of the long-distance dependence and medium-short distance dependence of each parameter item of the left foot gait parameter and the right foot gait parameter of the patient by adopting the artificial intelligence algorithm based on deep learning, and further carrying out the gait analysis and judgment based on the cooperative features of the long-distance dependence and the medium-short distance dependence so as to evaluate the rehabilitation training effect. Therefore, the rehabilitation training effect of the patient can be accurately evaluated to track the progress of the disease and accurately judge the recovery condition of the patient.

Drawings

The above and other objects, features and advantages of the present application will become more apparent by describing in more detail embodiments of the present application with reference to the attached drawings. The accompanying drawings are included to provide a further understanding of the embodiments of the application and are incorporated in and constitute a part of this specification, illustrate embodiments of the application and together with the description serve to explain the principles of the application. In the drawings, like reference numbers generally represent like parts or steps.

FIG. 1 is a block diagram of a data processing system for predicting a rehabilitation training effect based on gait recognition according to an embodiment of the present application;

FIG. 2 is a system architecture diagram of a data processing system for predicting a rehabilitation training effect based on gait recognition according to an embodiment of the present application;

FIG. 3 is a block diagram of a left-foot gait semantic code module in a data processing system for predicting rehabilitation training effects based on gait recognition according to an embodiment of the present application;

FIG. 4 is a block diagram of a coordination module in a data processing system for predicting rehabilitation training effects based on gait recognition according to an embodiment of the present application;

fig. 5 is a block diagram of a data processing result generation module in the data processing system for predicting the rehabilitation training effect based on gait recognition according to the embodiment of the present application;

FIG. 6 is a flow chart of a data processing method for predicting rehabilitation training effects based on gait recognition according to an embodiment of the present application;

fig. 7 is a block diagram of an electronic device according to an embodiment of the application.

Detailed Description

Hereinafter, example embodiments according to the present application will be described in detail with reference to the accompanying drawings. It should be understood that the described embodiments are only some embodiments of the present application and not all embodiments of the present application, and that the present application is not limited by the example embodiments described herein.

Summary of the application

As for the background technology, due to the shortage of corresponding medical staff, the time for the patient to get to or get from the rehabilitation center is long, and the nonstandard phenomenon in the operation of the staff causes the efficiency of the rehabilitation work to be low, the rehabilitation training effect of the patient is difficult to evaluate, and the patient misses the best rehabilitation opportunity. Accordingly, an optimized data processing system for predicting the effectiveness of rehabilitation training is desired.

At present, deep learning and neural networks have been widely applied in the fields of computer vision, natural language processing, speech signal processing, and the like. In addition, deep learning and neural networks also exhibit a level close to or even exceeding that of humans in the fields of image classification, object detection, semantic segmentation, text translation, and the like.

In recent years, deep learning and development of neural networks provide new solutions and schemes for evaluation of rehabilitation training effects.

In recent years, with the continuous development of gait analysis technology, a faster and more convenient gait analyzer is gradually applied to clinical rehabilitation training. The gait analysis system can objectively and quantitatively evaluate the gait function of a person and the severity of a disease, track the progress of the disease, support the diagnosis of early gait dysfunction and play an important role in judging the operation effect and the postoperative recovery condition. The gait analysis system is characterized in that the miniature sensor is placed in the insole, the insole is worn by a subject to walk, gait parameters are collected, and the gait parameters of the left foot and the right foot are calculated through data processing and comprise: the step phase, the landing position and the pressure central point track of the left foot and the right foot, the step speed, the step frequency, the step length, the steering angle, the symmetry of the two lower limbs, the swing phase time proportion, the support phase time proportion and the like. Finally, the test data is transmitted and stored to the computer terminal. And comparing the test value with a normal reference value range, determining abnormal key influence factors and compensatory changes, and providing suggestions and references for clinical diagnosis, clinical decision making and treatment effect evaluation.

Accordingly, considering that in the conventional gait analysis system, it is not accurate enough to determine the training and diagnosis evaluation of rehabilitation by comparing each test data with the normal reference value range, because there is an association relationship between each gait test data item of the left foot and the right foot, if the analysis and judgment of the rehabilitation training effect is performed only by the respective measurement results, the accuracy of the evaluation result is low. Based on this, in the technical scheme of the application, an artificial intelligence algorithm based on deep learning is adopted to respectively extract the multi-scale relevance feature distribution information of the left foot gait parameter and the right foot gait parameter of the patient based on long-distance dependence and medium-short distance dependence, and further, the gait is analyzed and judged based on the cooperation features of the left foot gait parameter and the right foot gait parameter, so that the rehabilitation training effect is evaluated. Therefore, the rehabilitation training effect of the patient can be accurately evaluated to track the progress of the disease and accurately judge the recovery condition of the patient.

Specifically, in the technical solution of the present application, first, a left foot gait parameter and a right foot gait parameter of a patient to be evaluated are collected by a micro sensor disposed in an insole of the patient to be evaluated, where the left foot gait parameter and the right foot gait parameter include a step phase, a landing position and a pressure center point trajectory, a pace, a step frequency, a step length, a steering angle, a double lower limb symmetry, a swing phase time ratio, and a support phase time ratio.

Then, the relation that each data item in the left-foot-state parameter has relevance is considered, that is, each data item of the left-foot-state parameter has relevance feature distribution representation in a high-dimensional space. Therefore, in order to be able to accurately assess the effectiveness of rehabilitation training, the left gait parameters are further passed through a converter-based context encoder to obtain a plurality of left gait data feature vectors. That is, based on the concept of transform, with the converter being able to capture long-distance dependent association characteristics, global-based context-associated coding is performed on each data item in the left-foot-state parameters to obtain a context global association feature representation with the overall features of the left-foot-state parameters as context, that is, the plurality of left-foot-state data feature vectors. It should be understood that, in the technical solution of the present application, the context-associated feature representation based on long-distance dependence of each data item feature in the left foot-state parameter with respect to the overall feature of the left foot-state parameter can be captured by the converter-based encoder. And then, cascading the plurality of left-foot-state data feature vectors to obtain a first left-foot-state semantic feature vector of each data item with the left-foot-state parameters based on long-distance-dependent global correlation feature distribution information.

Further, it should be understood that the global features of the data items considering the left step parameters have hidden relevance feature distribution information with different scales. That is to say, the global features of the data items of the left step parameters have different degrees of relevance under different data type spans, and therefore, in order to accurately evaluate the rehabilitation effect, feature extraction needs to be performed on relevance feature distributions of the data items under different data type spans. Specifically, in the technical solution of the present application, the plurality of left-foot gait data feature vectors are input into a bidirectional long-short term memory neural network model to obtain a second left-foot gait semantic feature vector. It should be understood that the two-way long-short term memory neural network model (A)

Long Short-Term Memory) enables the weight of the neural network to be updated by adding an input gate, an output gate and a forgetting gate, and the weight scales of different channels can be dynamically changed under the condition that the parameters of the network model are fixed, so that the problem of gradient disappearance or gradient expansion can be avoided. Particularly, the bidirectional long-short term memory neural network model is formed by combining a forward LSTM and a backward LSTM, and the forward LSTM can learn relevance feature distribution information of global features of each data item in a local region before the current data type; and backward LSTM can learn relevance feature distribution information of global features of each data item in subsequent local regions of the current data type. Therefore, the second left-step semantic feature vector obtained through the bidirectional long-short term memory neural network model learns the implicit associated feature information of each data item based on medium-short distance dependence.

Then, feature fusion is carried out on the first left-step-state semantic feature vector and the second left-step-state semantic feature vector, so that multi-scale relevance feature distribution information among the global features of all data items in the left-step-state parameters, namely semantic understanding features of the left-step gait, is obtained, and accordingly the left-step-state semantic feature vector is obtained.

Similarly, the right foot gait parameters are processed through the context encoder based on the converter and the bidirectional long and short term memory neural network model, so as to extract the multi-scale relevance feature distribution information of the global features of each data in the different parameters of the right foot based on long-distance dependence and medium and short-distance dependence, namely the semantic understanding features of the right foot gait, and obtain the right foot gait semantic feature vector.

And further, calculating a collaborative gait correlation characteristic matrix between the right foot gait semantic characteristic vector and the left foot gait semantic characteristic vector to represent correlation characteristic distribution information between the right foot and the left foot gait of the patient, and taking the correlation characteristic distribution information as a classification characteristic matrix to perform classification processing in a classifier so as to obtain a classification result of a grade label for representing the rehabilitation training effect. Thus, the rehabilitation training effect of the patient can be evaluated to track the progress of the disease and judge the recovery condition of the patient.

In particular, in the technical solution of the present application, the collaborative gait associative feature matrix is obtained by calculating a correlation value between positions of the right foot gait semantic feature vector and the left foot gait semantic feature vector, so that the collaborative gait associative feature matrix can express a correlation feature of a position granularity between the right foot gait semantic feature vector and the left foot gait semantic feature vector, but at the same time, it is still desirable that the collaborative gait associative feature matrix can express a feature correlation of a vector granularity between the right foot gait semantic feature vector and the left foot gait semantic feature vector.

Therefore, the applicant of the present application further calculates a graph core wandering node distribution fusion feature matrix between the right foot gait semantic feature vector and the left foot gait semantic feature vector, which is expressed as:

for the right foot gait semantic feature vector

And the left step semantic feature vector

A matrix of distances between, i.e.

，

Is a distance matrix

The value of the ith row and the jth column of (1),

is composed of

And

the value of the distance between the two,

is composed of

The value of the (i) th of (c),

is composed of

A j-th value of (a), and

and

are all column vectors, and are,

is the vector multiplied by the vector.

The idea of simulating a graph kernel (graph kernel) by the graph kernel migration node distribution fusion characteristic matrix is to use the right foot gait semantic characteristic vector

And the left step semantic feature vector

Respectively viewed as nodes in a graph and based on the right foot gait semantic feature vector

And the left step semantic feature vector

Is run on the distance topological graph to generalize the topological node to semantic feature vector relative to the right foot gait

And the left step semantic feature vector

Under the scene that the class feature distribution has continuous high-dimensional class space attributes, thereby representing the right foot gait semantic feature vector serving as a topological node

And said left foot gait languageSemantic feature vector

Local distribution information in a high-dimensional feature space of the fusion features to realize feature association of vector granularity between the right foot gait semantic feature vector and the left foot gait semantic feature vector.

Further, feature fusion is carried out on the graph core wandering node distribution fusion feature matrix and the cooperative gait correlation feature matrix, so that feature expression of the cooperative gait correlation feature matrix is optimized. Accordingly, in a specific example of the present application, a position-based point between the graph core walking node distribution fusion feature matrix and the cooperative gait correlation feature matrix may be calculated to obtain an optimized cooperative gait correlation feature matrix. And then, the optimized cooperative gait correlation characteristic matrix is used for evaluating and detecting the rehabilitation training effect through a classifier. Therefore, the rehabilitation training effect of the patient can be accurately evaluated to track the progress of the disease and accurately judge the recovery condition of the patient.

Based on this, the present application provides a data processing system for predicting a rehabilitation training effect based on gait recognition, which includes: the gait parameter acquisition module is used for acquiring left foot gait parameters and right foot gait parameters of a patient to be evaluated, wherein the left foot gait parameters and the right foot gait parameters comprise a step phase, a landing position, a pressure center point track, a pace speed, a step frequency, a step length, a steering angle, double lower limb symmetry, a swing phase time proportion and a support phase time proportion; the left-foot-state semantic coding module is used for enabling the left-foot-state parameters to pass through a context coder based on a converter to obtain a plurality of left-foot-state data feature vectors; the first scale feature extraction module is used for cascading the left-foot gait data feature vectors to obtain a first left-foot gait semantic feature vector; the second scale feature extraction module is used for inputting the plurality of left-foot gait data feature vectors into the bidirectional long-short term memory neural network model to obtain a second left-foot gait semantic feature vector; a left-foot-state multi-scale semantic feature fusion module, configured to fuse the first left-foot-state semantic feature vector and the second left-foot-state semantic feature vector to obtain a left-foot-state semantic feature vector; the right foot gait multiscale semantic feature extraction module is used for obtaining a right foot gait semantic feature vector from the right foot gait parameters through the context encoder based on the converter and the bidirectional long and short term memory neural network model; the cooperation module is used for calculating a cooperation gait correlation characteristic matrix between the right foot gait semantic characteristic vector and the left foot gait semantic characteristic vector; and the data processing result generation module is used for enabling the cooperative gait associated feature matrix to pass through a classifier to obtain a classification result, and the classification result is used for representing a grade label of a rehabilitation training effect.

Having described the basic principles of the present application, various non-limiting embodiments of the present application will now be described with reference to the accompanying drawings.

Exemplary System

Fig. 1 is a block diagram of a data processing system for predicting a rehabilitation training effect based on gait recognition according to an embodiment of the present application. As shown in fig. 1, a data processing system 300 for predicting rehabilitation training effect based on gait recognition according to an embodiment of the present application includes: a gait parameter acquisition module 310; a left step semantic encoding module 320; a first scale feature extraction module 330; a second scale feature extraction module 340; a left-footed gait multi-scale semantic feature fusion module 350; a right foot gait multiscale semantic feature extraction module 360; a collaboration module 370; and a data processing result generation module 380.

The gait parameter acquisition module 310 is configured to acquire left foot gait parameters and right foot gait parameters of a patient to be evaluated, which are acquired by a micro sensor disposed in an insole of the patient to be evaluated, where the left foot gait parameters and the right foot gait parameters include a step phase, a landing position and a pressure center point trajectory, a pace, a step frequency, a step length, a steering angle, two lower limb symmetry, a swing phase time proportion and a support phase time proportion; the left-foot-state semantic encoding module 320 is configured to pass the left-foot-state parameters through a context encoder based on a converter to obtain a plurality of left-foot-state data feature vectors; the first scale feature extraction module 330 is configured to cascade the plurality of left-step data feature vectors to obtain a first left-step semantic feature vector; the second scale feature extraction module 340 is configured to input the plurality of left-step-state data feature vectors into a bidirectional long-short term memory neural network model to obtain a second left-step-state semantic feature vector; the left-step multi-scale semantic feature fusion module 350 is configured to fuse the first left-step semantic feature vector and the second left-step semantic feature vector to obtain a left-step semantic feature vector; the right foot gait multiscale semantic feature extraction module 360 is configured to obtain a right foot gait semantic feature vector from the right foot gait parameter through the converter-based context encoder and the bidirectional long-short term memory neural network model; the cooperation module 370, configured to calculate a cooperation gait correlation feature matrix between the right foot gait semantic feature vector and the left foot gait semantic feature vector; and the data processing result generating module 380 is configured to pass the cooperative gait associated feature matrix through a classifier to obtain a classification result, where the classification result is a grade label representing a rehabilitation training effect.

Fig. 2 is a system architecture diagram of a data processing system for predicting rehabilitation training effects based on gait recognition according to an embodiment of the present application. As shown in fig. 2, in the network architecture, first, the gait parameter collecting module 310 obtains a left foot gait parameter and a right foot gait parameter of a patient to be evaluated, which are collected by a micro sensor disposed in an insole of the patient to be evaluated, where the left foot gait parameter and the right foot gait parameter include a step phase, a landing position, a pressure center point track, a step speed, a step frequency, a step length, a steering angle, a double lower limb symmetry, a swing phase time ratio, and a support phase time ratio; secondly, the left step semantic encoding module 320 passes the left step parameters acquired by the gait parameter acquisition module 310 through a context encoder based on a converter to obtain a plurality of left step data feature vectors; the first scale feature extraction module 330 concatenates the left-foot gait data feature vectors obtained by the left-foot gait semantic encoding module 320 to obtain a first left-foot gait semantic feature vector; the second scale feature extraction module 340 inputs the plurality of left-foot gait data feature vectors obtained by the left-foot gait semantic encoding module 320 into a bidirectional long-short term memory neural network model to obtain a second left-foot gait semantic feature vector; then, the left-step multi-scale semantic feature fusion module 350 fuses the first left-step semantic feature vector obtained by the first scale feature extraction module 330 and the second left-step semantic feature vector obtained by the second scale feature extraction module 340 to obtain a left-step semantic feature vector; the right foot gait multiscale semantic feature extraction module 360 obtains a right foot gait semantic feature vector from the right foot gait parameters acquired by the gait parameter acquisition module 310 through the converter-based context encoder and the bidirectional long-short term memory neural network model; then, the cooperation module 370 calculates a cooperation gait correlation feature matrix between the right foot gait semantic feature vector obtained by the right foot gait multi-scale semantic feature extraction module 360 and the left foot gait semantic feature vector obtained by the left foot gait multi-scale semantic feature fusion module 350; further, the data processing result generating module 380 passes the cooperative gait associated feature matrix through a classifier to obtain a classification result, where the classification result is used as a grade label representing a rehabilitation training effect.

Specifically, in the running process of the data processing system 300 for predicting the rehabilitation training effect based on gait recognition, the gait parameter collecting module 310 is configured to acquire a left foot gait parameter and a right foot gait parameter of a patient to be evaluated, which are collected by a micro sensor disposed in an insole of the patient to be evaluated, wherein the left foot gait parameter and the right foot gait parameter include a step phase, a landing position and a pressure center point trajectory, a step speed, a step frequency, a step length, a steering angle, a double lower limb symmetry, a swing phase time proportion and a support phase time proportion. It should be understood that the gait analysis system can objectively and quantitatively evaluate the gait function of a person and the severity of a disease, track the progress of the disease, support the diagnosis of early gait dysfunction, and play an important role in judging the operation effect and the postoperative recovery condition. The gait analysis system is characterized in that the miniature sensor is placed in the insole, the insole is worn by a subject to walk, gait parameters are collected, and the gait parameters of the left foot and the right foot are calculated through data processing and comprise: the step phase, the landing position and the pressure central point track of the left foot and the right foot, the step speed, the step frequency, the step length, the steering angle, the symmetry of the two lower limbs, the swing phase time proportion, the support phase time proportion and the like. Finally, the test data is transmitted and stored to the computer terminal. And comparing the test value with the normal reference value range, determining abnormal key influence factors and compensatory changes, and providing suggestions and references for clinical diagnosis, clinical decision making and treatment effect evaluation. Thus, in one particular example of the present application, left and right foot gait parameters of a patient to be assessed may be collected by micro sensors deployed within an insole of the patient to be assessed, where the left and right foot gait parameters include phase, landing position and pressure center point trajectory, pace, step frequency, step size, steering angle, bipedal symmetry, swing phase time ratio and support phase time ratio.

Specifically, during the operation of the gait recognition based rehabilitation training effect prediction data processing system 300, the left step semantic encoding module 320 is configured to pass the left step parameters through a context encoder based on a converter to obtain a plurality of left step data feature vectors. The left step state parameter is considered to have a relationship with an association between the data items, that is, the data items of the left step state parameter have an associated feature distribution representation in a high-dimensional space. Therefore, in order to be able to accurately evaluate the effectiveness of rehabilitation training, the left-foot-state parameters are further passed through a converter-based context encoder to obtain a plurality of left-foot-state data feature vectors. That is, based on the concept of transform, with the converter being able to capture long-distance dependent association characteristics, global-based context-associated coding is performed on each data item in the left-foot-state parameters to obtain a context global association feature representation with the overall features of the left-foot-state parameters as context, that is, the plurality of left-foot-state data feature vectors. It should be understood that, in the technical solution of the present application, the context-associated feature representation based on long-distance dependence of each data item feature in the left foot-state parameter with respect to the overall feature of the left foot-state parameter can be captured by the converter-based encoder. More specifically, said passing said left-foot-state parameters through a converter-based context encoder to obtain a plurality of left-foot-state data feature vectors comprises: firstly, performing word segmentation processing on the left step state parameter to convert the left step state parameter into a word sequence consisting of a plurality of words; mapping each word in the sequence of words to a word vector using an embedding layer of the converter-based context encoder to obtain a sequence of word vectors; in turn, global context-based semantic encoding is performed on the sequence of word vectors using a converter of the converter-based context encoder to obtain the plurality of left-foot gait data feature vectors. Wherein said globally context-based semantic-coding the sequence of word vectors using a converter of the converter-based context encoder to obtain the plurality of left-foot gait data feature vectors comprises: one-dimensional arrangement is carried out on the word vector sequence to obtain a global word feature vector; calculating a product between the global word feature vector and a transposed vector of each word vector in the sequence of word vectors to obtain a plurality of self-attention correlation matrices; respectively carrying out standardization processing on each self-attention correlation matrix in the self-attention correlation matrixes to obtain a plurality of standardized self-attention correlation matrixes; obtaining a plurality of probability values by passing each normalized self-attention correlation matrix in the plurality of normalized self-attention correlation matrices through a Softmax classification function; and weighting each word vector in the word vector sequence by taking each probability value in the probability values as a weight to obtain the left step data feature vectors.

Fig. 3 is a block diagram of a left-step semantic-coding module in a data processing system for predicting rehabilitation training effects based on gait recognition according to an embodiment of the present application. As shown in fig. 3, the left-step semantic encoding module 320 includes: a word segmentation unit 321, configured to perform word segmentation processing on the left step parameter to convert the left step parameter into a word sequence composed of multiple words; an embedding encoding unit 322 for mapping individual words in the sequence of words to word vectors using an embedding layer of the converter-based context encoder to obtain a sequence of word vectors; a context encoding unit 323 for performing a global context-based semantic encoding of the sequence of word vectors using a converter of the converter-based context encoder to obtain the plurality of left-foot-state data feature vectors.

Specifically, during the operation of the data processing system 300 for predicting the rehabilitation training effect based on gait recognition, the first scale feature extraction module 330 is configured to cascade the plurality of left foot gait data feature vectors to obtain a first left foot gait semantic feature vector. Namely, the left-foot-state data feature vectors are cascaded to obtain a first left-foot-state semantic feature vector of each data item with the left-foot-state parameters based on long-distance-dependent globally relevant feature distribution information. In one technical solution of the present application, the cascading the plurality of left-foot gait data feature vectors to obtain a first left-foot gait semantic feature vector includes: cascading the left-step data feature vectors according to the following formula to obtain a first left-step semantic feature vector; wherein the formula is:

wherein ,

representing the plurality of left foot gait data feature vectors,

a function of the cascade is represented as,

representing the first left-footed semantic feature vector.

Specifically, during the operation of the gait recognition based rehabilitation training prediction effect data processing system 300, the second scale feature extraction module 340 is configured to input the plurality of left step data feature vectors into a bidirectional long-short term memory neural network model to obtain a second left step semantic feature vector. It should be understood that the global features of the data items considering the left-step parameters have hidden relevance feature distribution information with different scales. That is to say, the global features of the data items of the left step parameters have different degrees of relevance under different data type spans, and therefore, in order to accurately evaluate the rehabilitation effect, feature extraction needs to be performed on relevance feature distributions of the data items under different data type spans. Specifically, in the technical solution of the present application, the plurality of left-foot gait data feature vectors are input into a bidirectional long-short term memory neural network model to obtain a second left-foot gait semantic feature vector. It should be understood that, the bidirectional long-short term memory neural network model (a)

Long Short-Term Memory) through adding an input gate, an output gate and a forgetting gate, the weight of the neural network can be updated by itself, and under the condition that the parameters of the network model are fixed, the weight scales of different channels can be changed dynamically, so that the problem of gradient disappearance or gradient expansion can be avoided. Particularly, the bidirectional long-short term memory neural network model is formed by combining a forward LSTM and a backward LSTM, and the forward LSTM can learn relevance feature distribution information of global features of each data item in a local region before the current data type; and backward LSTM may learn relevance feature distribution information for the global features of the respective data items for subsequent local regions of the current data type. Therefore, the second left-step semantic feature vector obtained by the bidirectional long-short term memory neural network model learns the respective numbersAccording to the implicit association characteristic information based on medium and short distance dependence of the items. More specifically, the inputting the plurality of left-foot gait data feature vectors into a bidirectional long-short term memory neural network model to obtain a second left-foot gait semantic feature vector comprises: passing the plurality of left-foot gait data feature vectors through the bidirectional long-short term memory neural network model to obtain a plurality of context left-foot gait data feature vectors; and cascading the plurality of context left-step data feature vectors to obtain the second left-step semantic feature vector.

Specifically, during the operation of the gait recognition based rehabilitation training prediction effect data processing system 300, the left-step multi-scale semantic feature fusion module 350 is configured to fuse the first left-step semantic feature vector and the second left-step semantic feature vector to obtain a left-step semantic feature vector. In the technical scheme of the application, the first left-step semantic feature vector and the second left-step semantic feature vector are subjected to feature fusion so as to obtain multi-scale relevance feature distribution information among global features of all data items in the left-step parameters, namely semantic understanding features of the left-step gait, and thus the left-step semantic feature vector is obtained. In a specific example of the present application, the right foot gait semantic feature vector and the left foot gait semantic feature vector are subjected to correlation coding by the following formula to obtain an initial correlation feature matrix; wherein the formula is:

wherein

Representing the right foot gait semantic feature vector,

representing the left-footed semantic feature vector,

is a multiplication of a matrix and a vector.

Specifically, during the operation of the gait recognition based rehabilitation training prediction effect data processing system 300, the right foot gait multiscale semantic feature extraction module 360 is configured to obtain a right foot gait semantic feature vector from the right foot gait parameters through the context encoder based on the converter and the bidirectional long-short term memory neural network model. In the technical scheme of the application, the right foot gait parameters are processed through the context encoder based on the converter and the bidirectional long and short term memory neural network model, so as to extract multi-scale relevance feature distribution information of global features of each data in different parameters of the right foot based on long-distance dependence and medium and short-distance dependence, namely semantic understanding features of the right foot gait, and accordingly a right foot gait semantic feature vector is obtained. More specifically, the right foot gait parameter is subjected to word segmentation processing so as to convert the right foot gait parameter into a word sequence consisting of a plurality of words; mapping each word in the sequence of words to a word vector using an embedding layer of the converter-based context encoder to obtain a sequence of word vectors; performing global context-based semantic encoding on the sequence of word vectors using a translator of the translator-based context encoder to obtain the plurality of right foot gait data feature vectors. Then, passing the plurality of right foot gait data feature vectors through the bidirectional long-short term memory neural network model to obtain a plurality of context right foot gait data feature vectors; and cascading the plurality of context right foot gait data feature vectors to obtain the right foot gait semantic feature vector.

Specifically, during the operation of the data processing system 300 for predicting the rehabilitation training effect based on gait recognition, the cooperation module 370 is configured to calculate a cooperation gait correlation feature matrix between the right foot gait semantic feature vector and the left foot gait semantic feature vector. It should be understood that a collaborative gait correlation feature matrix between the right foot gait semantic feature vector and the left foot gait semantic feature vector is calculated to represent the correlation feature distribution information between the right foot and the left foot of the patient, and is used as a classification feature matrix to be classified in a classifier, so as to obtain a classification result of a grade label for representing the rehabilitation training effect. In particular, in the technical solution of the present application, the collaborative gait associative feature matrix is obtained by calculating a correlation value between positions of the right foot gait semantic feature vector and the left foot gait semantic feature vector, so that the collaborative gait associative feature matrix can express a correlation feature of a position granularity between the right foot gait semantic feature vector and the left foot gait semantic feature vector, but at the same time, it is still desirable that the collaborative gait associative feature matrix can express a feature correlation of a vector granularity between the right foot gait semantic feature vector and the left foot gait semantic feature vector. Therefore, the applicant of the present application further calculates a graph core wandering node distribution fusion feature matrix between the right foot gait semantic feature vector and the left foot gait semantic feature vector, which is expressed as:

for the right foot gait semantic feature vector

And the left step semantic feature vector

A matrix of distances between, i.e.

And is and

and

are all column vectors, and are,

is the vector multiplied by the vector. The idea of simulating a graph kernel (graph kernel) by the graph kernel migration node distribution fusion characteristic matrix is to use the right foot gait semantic characteristic vector

And the left step semantic feature vector

And the left step semantic feature vector

And the left step semantic feature vector

And the left step semantic feature vector

Local distribution information in a high-dimensional feature space of the fusion features to realize feature association of vector granularity between the right foot gait semantic feature vector and the left foot gait semantic feature vector. Further, the feature matrix of the distribution and fusion of the graph core wandering nodes and the collaborative gait relationAnd performing feature fusion on the feature matrix so as to optimize the feature expression of the cooperative gait correlation feature matrix. Accordingly, in a specific example of the present application, the optimized cooperative gait correlation feature matrix may be obtained by calculating a point-by-point position between the core-walk node distribution fusion feature matrix and the cooperative gait correlation feature matrix. And then, the optimized cooperative gait correlation characteristic matrix is used for evaluating and detecting the rehabilitation training effect through a classifier. Therefore, the rehabilitation training effect of the patient can be accurately evaluated to track the progress of the disease and accurately judge the recovery condition of the patient.

Fig. 4 is a block diagram of a coordination module in a data processing system for predicting rehabilitation training effect based on gait recognition according to an embodiment of the present application. As shown in fig. 4, the coordination module 370 includes: an initial correlation feature matrix calculation unit 371, configured to perform correlation coding on the right foot gait semantic feature vector and the left foot gait semantic feature vector to obtain an initial correlation feature matrix; a core walking node distribution fusion feature matrix calculation unit 372, configured to calculate a core walking node distribution fusion feature matrix between the right foot gait semantic feature vector and the left foot gait semantic feature vector, where the core walking node distribution fusion feature matrix is related to a distance matrix between the right foot gait semantic feature vector and the left foot gait semantic feature vector; and a fusion unit 373, configured to fuse the initial association feature matrix and the graph core wandering node distribution fusion feature matrix to obtain the cooperative gait association feature matrix. Wherein the initial associated feature matrix calculating unit 371 is further configured to: performing correlation coding on the right foot gait semantic feature vector and the left foot gait semantic feature vector by the following formula to obtain an initial correlation feature matrix; wherein the formula is:

, wherein

Representing the right foot gaitA semantic feature vector is generated by the semantic feature vector,

representing the left-footed semantic feature vector,

representing the initial correlation characteristic matrix. More specifically, the graph core walk node distribution fusion feature matrix calculation unit 372 is further configured to: calculating the distribution and fusion feature matrix of the graph core wandering node between the right foot gait semantic feature vector and the left foot gait semantic feature vector according to the following formula; wherein the formula is:

wherein ,

representing the right foot gait semantic feature vector,

representing the left-footed semantic feature vector,

representing the distribution and fusion feature matrix of the graph core wandering node,

and

are all column vectors, and are,

an exponential operation representing a matrix for computing a natural exponential function value raised to a characteristic value at each position in the matrix，

Is the vector multiplied by the vector.

Specifically, during the operation of the data processing system 300 for predicting the rehabilitation training effect based on gait recognition, the data processing result generating module 380 is configured to pass the cooperative gait associated feature matrix through a classifier to obtain a classification result, where the classification result is used as a grade label representing the rehabilitation training effect. That is, the classification feature matrix is subjected to classification processing through a classifier to obtain a classification result of a grade label for representing the rehabilitation training effect. Thus, the rehabilitation training effect of the patient can be evaluated to track the progress of the disease and judge the recovery condition of the patient. In a specific example of the present application, the classifier is used to process the cooperative gait related feature matrix with the following formula to obtain a classification result, where the formula is:

, wherein

Representing the projection of the co-gait correlation characteristic matrix as a vector,

to

Is a weight matrix of the fully connected layers of each layer,

to

Representing the bias vectors of the fully connected layers of each layer,

output information for representing Softmax layer, i.e. classification of class labels for representing rehabilitation training effectAnd (4) obtaining the result. "

The multiple dot numbers in are used to indicate

To

Is omitted and

to

Where n is a symbol indicating that the classifier has n fully connected layers, the input information of the first layer is a classification feature vector, the output information of the previous layer is input information of the next layer, and the output information of the last layer is input to the classifier

The function outputs a classification result representing a class label representing the rehabilitation training effect, wherein,

to

The weight matrices of the first layer to the nth layer respectively,

to

Are the bias vectors of the first layer to the nth layer. In particular, the classifier includes a plurality of fully-connected layers and a Softmax layer cascaded with a last fully-connected layer of the plurality of fully-connected layers. In the classification processing of the classifier, the cooperative gait related feature matrix is first projected as a vector, for example, in a specific example, the cooperative gait related feature matrix is expanded as a classification feature vector along a row vector or a column vector; then, the segments are divided using a plurality of fully connected layers of the classifierCarrying out multiple full-connection coding on the class characteristic vector to obtain a coding classification characteristic vector; in turn, the encoded classification feature vector is input into a Softmax layer of the classifier, i.e., the encoded classification feature vector is classified by using the Softmax classification function to obtain a classification result of a grade label for representing rehabilitation training effect.

Fig. 5 is a block diagram of a data processing result generation module in the data processing system for predicting the rehabilitation training effect based on gait recognition according to the embodiment of the application. As shown in fig. 5, the data processing result generating module 380 includes: an expansion unit 381 configured to expand the cooperative gait association feature matrix into classification feature vectors based on row vectors or column vectors; a full-concatenation encoding unit 382, configured to perform full-concatenation encoding on the classification feature vector using a plurality of full-concatenation layers of the classifier to obtain an encoded classification feature vector; and a classification result generating unit 383, configured to pass the encoded classification feature vector through a Softmax classification function of the classifier to obtain the classification result.

In summary, the data processing system 300 for predicting the rehabilitation training effect based on gait recognition according to the embodiment of the present application is illustrated, which performs evaluation of the rehabilitation training effect by extracting the multi-scale relevance feature distribution information based on long-distance dependence and medium-short distance dependence of each parameter item of the left foot gait parameter and the right foot gait parameter of the patient respectively by using the artificial intelligence algorithm based on deep learning, and further performing analysis and judgment of the gait based on the synergistic features of the two. Therefore, the rehabilitation training effect of the patient can be accurately evaluated to track the progress condition of the disease and accurately judge the recovery condition of the patient.

As described above, the data processing system for predicting the rehabilitation training effect based on gait recognition according to the embodiment of the present application can be implemented in various terminal devices. In one example, the data processing system 300 for predicting rehabilitation training effect based on gait recognition according to the embodiment of the present application can be integrated into a terminal device as a software module and/or a hardware module. For example, the data processing system 300 for predicting the rehabilitation training effect based on gait recognition may be a software module in the operating system of the terminal device, or may be an application developed for the terminal device; of course, the data processing system 300 for predicting the rehabilitation training effect based on gait recognition can also be one of numerous hardware modules of the terminal device.

Alternatively, in another example, the data processing system 300 for predicting the rehabilitation training effect based on the gait recognition and the terminal device may be separate devices, and the data processing system 300 for predicting the rehabilitation training effect based on the gait recognition may be connected to the terminal device through a wired and/or wireless network and transmit the mutual information according to the agreed data format.

Exemplary method

Fig. 6 is a flowchart of a data processing method for predicting rehabilitation training effect based on gait recognition according to an embodiment of the present application. As shown in fig. 6, the data processing method for predicting the rehabilitation training effect based on gait recognition according to the embodiment of the present application includes the steps of: s110, acquiring left foot gait parameters and right foot gait parameters of a patient to be evaluated, wherein the left foot gait parameters and the right foot gait parameters comprise a step phase, a landing position, a pressure center point track, a step speed, a step frequency, a step length, a steering angle, double lower limb symmetry, a swing phase time proportion and a support phase time proportion, and the left foot gait parameters and the right foot gait parameters are acquired by a miniature sensor deployed in an insole of the patient to be evaluated; s120, enabling the left-foot step state parameters to pass through a context encoder based on a converter to obtain a plurality of left-foot step state data characteristic vectors; s130, cascading the left-step data feature vectors to obtain a first left-step semantic feature vector; s140, inputting the plurality of left-foot gait data feature vectors into a bidirectional long-short term memory neural network model to obtain a second left-foot gait semantic feature vector; s150, fusing the first left-step semantic feature vector and the second left-step semantic feature vector to obtain a left-step semantic feature vector; s160, obtaining a right foot gait semantic feature vector from the right foot gait parameters through the context encoder based on the converter and the bidirectional long and short term memory neural network model; s170, calculating a cooperative gait correlation characteristic matrix between the right foot gait semantic characteristic vector and the left foot gait semantic characteristic vector; and S180, enabling the cooperative gait correlation characteristic matrix to pass through a classifier to obtain a classification result, wherein the classification result is used for representing a grade label of a rehabilitation training effect.

In one example, in the data processing method for predicting the rehabilitation training effect based on gait recognition, the step S120 includes: performing word segmentation processing on the left step state parameters to convert the left step state parameters into a word sequence consisting of a plurality of words; mapping each word in the sequence of words to a word vector using an embedding layer of the converter-based context encoder to obtain a sequence of word vectors; performing global context-based semantic encoding on the sequence of word vectors using a converter of the converter-based context encoder to obtain the plurality of left-foot gait data feature vectors.

In one example, in the data processing method for predicting the rehabilitation training effect based on gait recognition, the step S150 includes: fusing the first left-step semantic feature vector and the second left-step semantic feature vector according to the following formula to obtain a left-step semantic feature vector;

wherein the formula is:

wherein

Representing the first left-footed semantic feature vector,

representing the second left-footed semantic feature vector,

representing the left-footed semantic feature vector,

indicating a sum by position.

In an example, in the data processing method for predicting the rehabilitation training effect based on gait recognition, the step S170 includes: performing correlation coding on the right foot gait semantic feature vector and the left foot gait semantic feature vector to obtain an initial correlation feature matrix; calculating a core wandering node distribution fusion feature matrix between the right foot gait semantic feature vector and the left foot gait semantic feature vector, wherein the core wandering node distribution fusion feature matrix is related to a distance matrix between the right foot gait semantic feature vector and the left foot gait semantic feature vector; and fusing the initial correlation characteristic matrix and the graph core wandering node distribution fusion characteristic matrix to obtain the cooperative gait correlation characteristic matrix. Wherein the performing the association coding on the right foot gait semantic feature vector and the left foot gait semantic feature vector to obtain an initial association feature matrix comprises: performing correlation coding on the right foot gait semantic feature vector and the left foot gait semantic feature vector by the following formula to obtain an initial correlation feature matrix; wherein the formula is:

, wherein

Representing the right foot gait semantic feature vector,

representing the left-footed semantic feature vector,

representing the initial correlation feature matrix. More specifically, the calculating the right foot gaitA graph core wandering node distribution fusion feature matrix between the semantic feature vector and the left step semantic feature vector, comprising: calculating the distribution and fusion feature matrix of the graph core wandering node between the right foot gait semantic feature vector and the left foot gait semantic feature vector according to the following formula; wherein the formula is:

wherein ,

representing the right foot gait semantic feature vector,

representing the left-footed semantic feature vector,

representing the graph core walk node distribution fusion feature matrix,

and

are all column vectors, and are,

is the vector multiplied by the vector.

In one example, in the data processing method for predicting the rehabilitation training effect based on gait recognition, the step S180 includes: expanding the cooperative gait association feature matrix into classification feature vectors based on row vectors or column vectors; performing full-join encoding on the classification feature vectors using a plurality of full-join layers of the classifier to obtain encoded classification feature vectors; and passing the encoding classification feature vector through a Softmax classification function of the classifier to obtain the classification result.

In summary, the data processing method for predicting the rehabilitation training effect based on gait recognition according to the embodiment of the application is clarified, and the evaluation of the rehabilitation training effect is performed by extracting the multi-scale relevance feature distribution information of the long-distance dependence and the medium-short distance dependence of each parameter item of the left foot gait parameter and the right foot gait parameter of the patient based on the artificial intelligence algorithm based on deep learning, and further performing the gait analysis and judgment based on the synergistic features of the long-distance dependence and the medium-short distance dependence. Therefore, the rehabilitation training effect of the patient can be accurately evaluated to track the progress of the disease and accurately judge the recovery condition of the patient.

Exemplary electronic device

Next, an electronic apparatus according to an embodiment of the present application is described with reference to fig. 7.

FIG. 7 illustrates a block diagram of an electronic device in accordance with an embodiment of the application.

As shown in fig. 7, the electronic device 10 includes one or more processors 11 and memory 12.

The processor 11 may be a Central Processing Unit (CPU) or other form of processing unit having data processing capabilities and/or instruction execution capabilities, and may control other components in the electronic device 10 to perform desired functions.

Memory 12 may include one or more computer program products that may include various forms of computer-readable storage media, such as volatile memory and/or non-volatile memory. The volatile memory may include, for example, random Access Memory (RAM), cache memory (cache), and/or the like. The non-volatile memory may include, for example, read Only Memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer readable storage medium and executed by the processor 11 to implement the functions in the gait recognition based data processing system for predicting the rehabilitation training effect of the various embodiments of the application described above and/or other desired functions. Various content such as a left-foot gait data feature vector may also be stored in the computer readable storage medium.

In one example, the electronic device 10 may further include: an input device 13 and an output device 14, which are interconnected by a bus system and/or other form of connection mechanism (not shown).

The input device 13 may include, for example, a keyboard, a mouse, and the like.

The output device 14 can output various information including the classification result to the outside. The output devices 14 may include, for example, a display, speakers, a printer, and a communication network and its connected remote output devices, among others.

Of course, for simplicity, only some of the components of the electronic device 10 relevant to the present application are shown in fig. 7, and components such as buses, input/output interfaces, and the like are omitted. In addition, the electronic device 10 may include any other suitable components depending on the particular application.

Exemplary computer program product and computer-readable storage Medium

In addition to the above-described methods and apparatus, embodiments of the present application may also be a computer program product comprising computer program instructions that, when executed by a processor, cause the processor to perform the steps in the functions of the data processing method for predicting a rehabilitation training effect based on gait recognition according to various embodiments of the present application described in the "exemplary systems" section of this specification above.

The computer program product may be written with program code for performing the operations of embodiments of the present application in any combination of one or more programming languages, including an object oriented programming language such as Java, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device, or entirely on the remote computing device or server.

Furthermore, embodiments of the present application may also be a computer readable storage medium having stored thereon computer program instructions which, when executed by a processor, cause the processor to perform steps in functions of a data processing method for predicting a rehabilitation training effect based on gait recognition according to various embodiments of the present application described in the above-mentioned "exemplary systems" section of this specification.

The computer readable storage medium may take any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may include, for example, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

The foregoing describes the general principles of the present application in conjunction with specific embodiments, however, it is noted that the advantages, effects, etc. mentioned in the present application are merely examples and are not limiting, and they should not be considered essential to the various embodiments of the present application. Furthermore, the foregoing disclosure of specific details is for the purpose of illustration and description and is not intended to be limiting, since the foregoing disclosure is not intended to be exhaustive or to limit the disclosure to the precise details disclosed.

The block diagrams of devices, apparatuses, systems referred to in this application are only given as illustrative examples and are not intended to require or imply that the connections, arrangements, configurations, etc. must be made in the manner shown in the block diagrams. These devices, apparatuses, devices, systems may be connected, arranged, configured in any manner, as will be appreciated by those skilled in the art. Words such as "including," "comprising," "having," and the like are open-ended words that mean "including, but not limited to," and are used interchangeably therewith. The words "or" and "as used herein mean, and are used interchangeably with, the word" and/or, "unless the context clearly dictates otherwise. The word "such as" is used herein to mean, and is used interchangeably with, the phrase "such as but not limited to".

It should also be noted that in the devices, apparatuses, and methods of the present application, the components or steps may be decomposed and/or recombined. These decompositions and/or recombinations are to be considered as equivalents of the present application.

The previous description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other aspects without departing from the scope of the application. Thus, the present application is not intended to be limited to the aspects shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

The foregoing description has been presented for purposes of illustration and description. Furthermore, the description is not intended to limit embodiments of the application to the form disclosed herein. While a number of example aspects and embodiments have been discussed above, those of skill in the art will recognize certain variations, modifications, alterations, additions and sub-combinations thereof.

Claims

1. A data processing system for predicting a rehabilitation training effect based on gait recognition, comprising:

2. The gait recognition-based rehabilitation training effect prediction data processing system according to claim 1, wherein the left step semantic coding module comprises:

the word segmentation unit is used for performing word segmentation processing on the left step state parameter so as to convert the left step state parameter into a word sequence consisting of a plurality of words;

an embedding encoding unit for mapping individual words in the sequence of words to word vectors using an embedding layer of the converter-based context encoder to obtain a sequence of word vectors;

a context encoding unit for globally context-based semantic-coding the sequence of word vectors using a converter of the converter-based context encoder to obtain the plurality of left-foot-state-data feature vectors.

3. The gait recognition-based data processing system for predicting rehabilitation training effects of claim 2, wherein the left-step dynamic multi-scale semantic feature fusion module is further configured to: fusing the first left-step semantic feature vector and the second left-step semantic feature vector according to the following formula to obtain a left-step semantic feature vector;

wherein the formula is:

wherein

Representing the first left-footed semantic feature vector,

representing the second left-footed semantic feature vector,

representing the left-footed semantic feature vector,

weighted parameters representing the first left-footed semantic feature vector and the second left-footed semantic feature vector, respectivelyThe number of the first and second groups is,

indicating a sum by position.

4. The data processing system for predicting rehabilitation training effect based on gait recognition of claim 3, wherein the coordination module comprises:

the initial association feature matrix calculation unit is used for performing association coding on the right foot gait semantic feature vector and the left foot gait semantic feature vector to obtain an initial association feature matrix;

a core walking node distribution fusion feature matrix calculation unit, configured to calculate a core walking node distribution fusion feature matrix between the right foot gait semantic feature vector and the left foot gait semantic feature vector, where the core walking node distribution fusion feature matrix is related to a distance matrix between the right foot gait semantic feature vector and the left foot gait semantic feature vector; and

and the fusion unit is used for fusing the initial correlation characteristic matrix and the graph core wandering node distribution fusion characteristic matrix to obtain the cooperative gait correlation characteristic matrix.

5. The gait recognition-based rehabilitation training effect prediction data processing system according to claim 4, wherein the initial correlation feature matrix calculation unit is further configured to: performing correlation coding on the right foot gait semantic feature vector and the left foot gait semantic feature vector by the following formula to obtain an initial correlation feature matrix;

wherein the formula is:

wherein

representing the left-footed semantic feature vector,

is a multiplication of a matrix and a vector.

6. The gait recognition-based rehabilitation training effect prediction data processing system according to claim 5, wherein the graph kernel walking node distribution fusion feature matrix calculation unit is further configured to: calculating the distribution and fusion feature matrix of the graph core wandering node between the right foot gait semantic feature vector and the left foot gait semantic feature vector according to the following formula;

wherein the formula is:

wherein ,

represent the right foot gait semantic feature vector,

representing the left-footed semantic feature vector,

and

are all column vectors, and are,

an exponential operation representing a matrix that calculates a natural exponential function value raised to a characteristic value of each position in the matrix,

is the vector multiplied by the vector.

7. The gait recognition-based rehabilitation training effect prediction data processing system according to claim 6, wherein the fusion unit is further configured to calculate the position-point-based sum of the initial correlation feature matrix and the kernel walking node distribution fusion feature matrix to obtain the cooperative gait correlation feature matrix.

8. The gait recognition-based rehabilitation training effect prediction data processing system according to claim 7, wherein the data processing result generation module comprises:

the unfolding unit is used for unfolding the cooperative gait correlation characteristic matrix into classification characteristic vectors based on row vectors or column vectors;

a full-concatenation encoding unit, configured to perform full-concatenation encoding on the classification feature vector using a plurality of full-concatenation layers of the classifier to obtain an encoded classification feature vector; and

a classification result generating unit, configured to pass the encoded classification feature vector through a Softmax classification function of the classifier to obtain the classification result.