CN115830718B

CN115830718B - Gait recognition-based data processing system for predicting rehabilitation training effect

Info

Publication number: CN115830718B
Application number: CN202310107540.9A
Authority: CN
Inventors: 李灿东; 杨朝阳; 唐志伟; 赖新梅; 周常恩; 辛基梁
Original assignee: Fujian University of Traditional Chinese Medicine
Current assignee: Fujian University of Traditional Chinese Medicine
Priority date: 2023-02-14
Filing date: 2023-02-14
Publication date: 2023-04-25
Anticipated expiration: 2043-02-14
Also published as: CN115830718A

Abstract

The application relates to the field of data processing, and particularly discloses a gait recognition prediction rehabilitation training effect-based data processing system, which is used for estimating rehabilitation training effect by extracting multi-scale relevance characteristic distribution information of each parameter item of left foot step parameters and right foot step parameters of a patient based on long-distance dependence and middle-short distance dependence by adopting an artificial intelligence algorithm based on deep learning, and further carrying out gait analysis and judgment based on cooperative characteristics of the two. Thus, the rehabilitation training effect of the patient can be accurately evaluated to track the progress of the disease, and the recovery condition of the patient can be accurately judged.

Description

Gait recognition-based data processing system for predicting rehabilitation training effect

Technical Field

The present application relates to the field of data processing, and more particularly, to a data processing system for predicting rehabilitation training effects based on gait recognition.

Background

In recent years, chronic diseases are slowly threatening the health of the elderly. According to research, alzheimer's Disease (AD) is commonly called senile dementia, and has become one of the third biggest "disease killers" after heart disease and cancer in middle-aged and elderly people. Mild cognitive impairment (Mild Cognition Impairment, MCI) is an early state of AD, which is particularly important for early assessment and rehabilitation training of MCI, as AD is irreversible and the number of patients rises year by year.

At present, most mechanisms for providing rehabilitation training are rehabilitation centers, and most adopted modes are traditional rehabilitation measures. For example, the method is used for carrying out language interaction training with the help of medical staff and carrying out social cognitive ability rehabilitation such as memory, logic thinking, judgment and the like on patients; the mechanical training of limbs is used for recovering the coordination ability of the movement of the patient. However, due to the shortage of corresponding medical staff, time is consumed for patients to go to and from a rehabilitation center, and the phenomenon of non-standardization in staff operation makes the efficiency of rehabilitation work not high, the rehabilitation training effect of the patients is difficult to evaluate, and the patients miss the optimal rehabilitation opportunity.

Accordingly, an optimized data processing system for predicting rehabilitation training effects is desired.

Disclosure of Invention

The present application has been made in order to solve the above technical problems. The embodiment of the application provides a data processing system and a method based on gait recognition prediction rehabilitation training effect, which are used for respectively extracting multi-scale relevance characteristic distribution information of left foot step parameters and right foot step parameters of a patient based on long-distance dependence and middle-short distance dependence by adopting an artificial intelligence algorithm based on deep learning, and further carrying out gait analysis and judgment based on cooperative characteristics of the left foot step parameters and the right foot step parameters, so as to evaluate the rehabilitation training effect. Thus, the rehabilitation training effect of the patient can be accurately evaluated to track the progress of the disease, and the recovery condition of the patient can be accurately judged.

According to one aspect of the present application, there is provided a gait recognition-based predictive rehabilitation training effect data processing system, comprising:

the gait parameter acquisition module is used for acquiring left foot step parameters and right foot step parameters of a patient to be evaluated, wherein the left foot step parameters and the right foot step parameters comprise a step phase, a ground position, a pressure center point track, a step speed, a step frequency, a step length, a steering angle, double lower limb symmetry, a swing phase time proportion and a support phase time proportion, which are acquired by micro sensors deployed in insoles of the patient to be evaluated;

the left foot step semantic coding module is used for enabling the left foot step parameters to pass through a context encoder based on a converter to obtain a plurality of left foot step data feature vectors;

the first scale feature extraction module is used for cascading the plurality of left-foot step data feature vectors to obtain a first left-foot step semantic feature vector;

the second scale feature extraction module is used for inputting the plurality of left foot step data feature vectors into a two-way long-short-term memory neural network model to obtain a second left foot step semantic feature vector;

the left-foot-state multi-scale semantic feature fusion module is used for fusing the first left-foot-state semantic feature vector and the second left-foot-state semantic feature vector to obtain a left-foot-state semantic feature vector;

The right foot step multi-scale semantic feature extraction module is used for obtaining right foot step semantic feature vectors from the right foot step parameters through the context encoder based on the converter and the two-way long-short-term memory neural network model;

the cooperative module is used for calculating a cooperative gait correlation feature matrix between the right foot step semantic feature vector and the left foot step semantic feature vector; and

the data processing result generation module is used for enabling the collaborative gait correlation feature matrix to pass through a classifier to obtain a classification result, and the classification result is used for representing a grade label of a rehabilitation training effect.

In the above data processing system based on gait recognition prediction rehabilitation training effect, the left foot step semantic coding module includes: the word segmentation unit is used for carrying out word segmentation processing on the left step parameters so as to convert the left step parameters into word sequences composed of a plurality of words; an embedded encoding unit for mapping each word in the word sequence to a word vector using an embedded layer of the converter-based context encoder to obtain a sequence of word vectors; and the context coding unit is used for performing global context semantic coding on the sequence of the word vectors by using a converter of the context coder based on the converter so as to obtain the plurality of left-foot step data feature vectors.

In the above data processing system based on gait recognition prediction rehabilitation training effect, the left foot gait multiscale semantic feature fusion module is further configured to: fusing the first left-foot step semantic feature vector and the second left-foot step semantic feature vector to obtain a left-foot step semantic feature vector by the following formula; wherein, the formula is:

wherein

Representing the first left-foot step semantic feature vector,>

representing the second left-foot step semantic feature vector,>

representing the left-foot step semantic feature vector, < >>

Weighting parameters respectively representing the first left-foot step semantic feature vector and the second left-foot step semantic feature vector,/a>

Representing the sum by location.

In the above data processing system based on gait recognition prediction rehabilitation training effect, the cooperation module includes: the initial association feature matrix calculation unit is used for carrying out association coding on the right foot step semantic feature vector and the left foot step semantic feature vector to obtain an initial association feature matrix; the kernel walking node distribution fusion feature matrix calculation unit is used for calculating a kernel walking node distribution fusion feature matrix between the right foot step semantic feature vector and the left foot step semantic feature vector, wherein the kernel walking node distribution fusion feature matrix is related to a distance matrix between the right foot step semantic feature vector and the left foot step semantic feature vector; and a fusion unit, configured to fuse the initial association feature matrix and the kernel walking node distribution fusion feature matrix to obtain the collaborative gait association feature matrix.

In the above data processing system based on gait recognition prediction rehabilitation training effect, the initial association feature matrix calculation unit is further configured to: performing association coding on the right foot step semantic feature vector and the left foot step semantic feature vector by using the following formula to obtain an initial association feature matrix;

wherein, the formula is:

wherein

Representing the right foot step semantic feature vector, < >>

Representing the left-foot step semantic feature vector,

representing the initial association feature matrix, +.>

For matrix multiplication with vectors.

In the above data processing system based on gait recognition prediction rehabilitation training effect, the kernel walking node distribution fusion feature matrix calculation unit is further configured to: calculating the kernel walking node distribution fusion feature matrix between the right foot step semantic feature vector and the left foot step semantic feature vector according to the following formula; wherein, the formula is:

wherein ,

representing the right foot step semantic feature vector, < >>

Representing the left-foot step semantic feature vector,

representing the distribution fusion feature matrix of the core wandering nodes>

Is a distance matrix between the right foot step semantic feature vector and the left foot step semantic feature vector, and +. >

and />

Are column vectors, +.>

An exponential operation representing a matrix representing the calculation of a natural exponential function value raised to a power by a characteristic value at each position in the matrix,/v>

Is the multiplication of the vector by the vector.

In the above data processing system based on gait recognition prediction rehabilitation training effect, the fusion unit is further configured to calculate a per-position point between the initial association feature matrix and the kernel walking node distribution fusion feature matrix to obtain the collaborative gait association feature matrix.

In the above data processing system based on gait recognition prediction rehabilitation training effect, the data processing result generating module includes: the unfolding unit is used for unfolding the collaborative gait correlation feature matrix into a classification feature vector based on a row vector or a column vector; the full-connection coding unit is used for carrying out full-connection coding on the classification characteristic vectors by using a plurality of full-connection layers of the classifier so as to obtain coded classification characteristic vectors; and the classification result generating unit is used for enabling the coding classification feature vector to pass through a Softmax classification function of the classifier to obtain the classification result.

According to another aspect of the present application, there is provided a data processing method for predicting rehabilitation training effects based on gait recognition, including:

Acquiring left foot step parameters and right foot step parameters of a patient to be evaluated, wherein the left foot step parameters and the right foot step parameters comprise a step phase, a ground position, a pressure center point track, a step speed, a step frequency, a step length, a steering angle, double lower limb symmetry, a swing phase time proportion and a support phase time proportion, which are acquired by micro sensors deployed in insoles of the patient to be evaluated;

passing the left foot step parameters through a context encoder based on a converter to obtain a plurality of left foot step data feature vectors;

cascading the plurality of left-foot step data feature vectors to obtain a first left-foot step semantic feature vector;

inputting the plurality of left foot step data feature vectors into a two-way long-short term memory neural network model to obtain a second left foot step semantic feature vector;

fusing the first left-foot step semantic feature vector and the second left-foot step semantic feature vector to obtain a left-foot step semantic feature vector;

obtaining right-foot-state semantic feature vectors from the right-foot-state parameters through the context encoder based on the converter and the two-way long-short-term memory neural network model;

calculating a collaborative gait correlation feature matrix between the right foot step semantic feature vector and the left foot step semantic feature vector; and

And the collaborative gait correlation feature matrix passes through a classifier to obtain a classification result, wherein the classification result is used for representing a grade label of the rehabilitation training effect.

According to still another aspect of the present application, there is provided an electronic apparatus including: a processor; and a memory having stored therein computer program instructions that, when executed by the processor, cause the processor to perform a data processing method of predicting rehabilitation training effects based on gait recognition as described above.

According to yet another aspect of the present application, there is provided a computer readable medium having stored thereon computer program instructions which, when executed by a processor, cause the processor to perform a data processing method of predicting rehabilitation training effects based on gait recognition as described above.

Compared with the prior art, the data processing system and the method based on gait recognition prediction rehabilitation training effect provided by the application are characterized in that the artificial intelligence algorithm based on deep learning is adopted to extract the multi-scale relevance characteristic distribution information of each parameter item of the left foot step parameter and the right foot step parameter of a patient based on long-distance dependence and middle-short distance dependence, and further, the analysis and judgment of gait are further carried out based on the cooperative characteristics of the two parameters, so that the rehabilitation training effect is evaluated. Thus, the rehabilitation training effect of the patient can be accurately evaluated to track the progress of the disease, and the recovery condition of the patient can be accurately judged.

Drawings

The foregoing and other objects, features and advantages of the present application will become more apparent from the following more particular description of embodiments of the present application, as illustrated in the accompanying drawings. The accompanying drawings are included to provide a further understanding of embodiments of the application and are incorporated in and constitute a part of this specification, illustrate the application and not constitute a limitation to the application. In the drawings, like reference numerals generally refer to like parts or steps.

FIG. 1 is a block diagram of a data processing system for predicting rehabilitation training effects based on gait recognition according to embodiments of the present application;

FIG. 2 is a system architecture diagram of a data processing system based on gait recognition predictive rehabilitation training effects according to an embodiment of the present application;

FIG. 3 is a block diagram of a left foot step semantic coding module in a gait recognition prediction rehabilitation training effect-based data processing system according to an embodiment of the present application;

FIG. 4 is a block diagram of collaboration modules in a data processing system based on gait recognition predictive rehabilitation training effects in accordance with an embodiment of the present application;

FIG. 5 is a block diagram of a data processing result generation module in a data processing system based on gait recognition prediction rehabilitation training effects according to an embodiment of the present application;

FIG. 6 is a flow chart of a method of data processing based on gait recognition predictive rehabilitation training effects according to an embodiment of the present application;

fig. 7 is a block diagram of an electronic device according to an embodiment of the present application.

Detailed Description

Hereinafter, example embodiments according to the present application will be described in detail with reference to the accompanying drawings. It should be apparent that the described embodiments are only some of the embodiments of the present application and not all of the embodiments of the present application, and it should be understood that the present application is not limited by the example embodiments described herein.

Summary of the application

As the corresponding medical staff are short in the prior art, the time for the patient to go to and from the rehabilitation center is consumed, and the phenomenon of non-standardization in staff operation is caused, so that the efficiency of rehabilitation work is low, the rehabilitation training effect of the patient is difficult to evaluate, and the patient misses the optimal rehabilitation opportunity. Accordingly, an optimized data processing system for predicting rehabilitation training effects is desired.

At present, deep learning and neural networks have been widely used in the fields of computer vision, natural language processing, speech signal processing, and the like. In addition, deep learning and neural networks have also shown levels approaching and even exceeding humans in the fields of image classification, object detection, semantic segmentation, text translation, and the like.

In recent years, the development of deep learning and neural networks has provided new solutions and solutions for the assessment of rehabilitation training effects.

In recent years, with the continuous development of gait analysis technology, a quicker and more convenient gait analyzer is gradually applied to clinical rehabilitation training. The gait analysis system can objectively and quantitatively evaluate the gait function and the severity of the disease of the person, track the progress of the disease, and support the diagnosis of early walking dysfunction, and has important roles in judging the surgical effect and the postoperative recovery condition. Gait analysis system is through placing miniature sensor in the shoe-pad, and the testee dresses the shoe-pad and walks, carries out the collection of gait parameter, and through data processing, the calculation left foot and right foot step parameter includes: the walking phase, the grounding position, the pressure center point track, the walking speed, the walking frequency, the step length, the steering angle, the symmetry of the double lower limbs, the swing phase time proportion, the support phase time proportion and the like of the left foot and the right foot. And finally, the test data are transmitted to and stored in the computer terminal. Comparing the test value with a normal reference value range, determining abnormal key influence factors and compensatory changes, and providing suggestions and references for clinical diagnosis, clinical decision making and treatment effect evaluation.

Accordingly, considering that in the conventional gait analysis system, the determination of the rehabilitation training diagnosis evaluation by comparing each test data with the normal reference value range is not accurate enough, because each gait test data item of the left foot and the right foot has an association relationship, the accuracy of the evaluation result is low if the analysis and judgment of the rehabilitation training effect is performed only by the respective measurement results. Based on the above, in the technical scheme of the application, an artificial intelligence algorithm based on deep learning is adopted to extract each parameter item of the left foot step parameter and the right foot step parameter of a patient respectively, and based on multi-scale relevance feature distribution information of long-distance dependence and middle-short-distance dependence, further, based on cooperative features of the two, gait analysis and judgment are carried out, so that rehabilitation training effect evaluation is carried out. Thus, the rehabilitation training effect of the patient can be accurately evaluated to track the progress of the disease, and the recovery condition of the patient can be accurately judged.

Specifically, in the technical scheme of the application, firstly, the left foot step parameter and the right foot step parameter of a patient to be evaluated are collected through a micro sensor deployed in an insole of the patient to be evaluated, wherein the left foot step parameter and the right foot step parameter comprise a step phase, a ground position, a pressure center point track, a step speed, a step frequency, a step length, a steering angle, double lower limb symmetry, a swing phase time proportion and a support phase time proportion.

Next, the relationship that each data item in the left-foot step parameter has a relevance is considered, that is, each data item of the left-foot step parameter has a relevant feature distribution representation in a high-dimensional space. Therefore, in order to be able to accurately evaluate the effect of the rehabilitation training, the left-foot step parameters are further passed through a context encoder based on a converter to obtain a plurality of left-foot step data feature vectors. That is, based on the transform concept, the converter is used to capture the correlation characteristic that depends on a long distance, and the global context correlation encoding is performed on each data item in the left-foot step parameters to obtain a context global correlation feature representation with the global feature of the left-foot step parameters as the context, that is, the plurality of left-foot step data feature vectors. It should be appreciated that in the solution of the present application, the context-dependent feature representation of each data item feature in the left-foot step parameter with respect to the overall feature of the left-foot step parameter may be captured by the converter-based encoder. And then cascading the plurality of left-foot step data feature vectors to obtain a first left-foot step semantic feature vector of each data item with the left-foot step parameters based on global associated feature distribution information which depends on a long distance.

Further, it should be appreciated that there are different scales of hidden relevance feature distribution information between the global features of the individual data items taking into account the left foot step parameters. That is, the global features of the individual data items of the left foot step parameters have different degrees of relevance under different spans of data types, and therefore, in order to be able to accurately evaluate the rehabilitation effect, it is necessary to correlate the individual data items under different spans of data typesAnd extracting the characteristic by the characteristic distribution. Specifically, in the technical scheme of the application, the plurality of left-foot step data feature vectors are input into a two-way long-short-term memory neural network model to obtain a second left-foot step semantic feature vector. It should be understood that the two-way long-short term memory neural network model #

Long Short-Term Memory) to enable the weight of the neural network to be updated by adding an input gate, an output gate and a forgetting gate, and the weight scale of different channels can be dynamically changed under the condition of fixed network model parameters, so that the problem of gradient disappearance or gradient expansion can be avoided. In particular, the two-way long-short-term memory neural network model is formed by combining a forward LSTM and a backward LSTM, and the forward LSTM can learn the association characteristic distribution information of the global characteristics of the local area before the current data type and related to each data item; the relevant feature distribution information about the global features of the individual data items of the subsequent local region of the current data type can then be learned towards the LSTM. Therefore, the second left-foot step semantic feature vector obtained through the two-way long-short term memory neural network model learns implicit associated feature information based on middle-short distance dependence of each data item.

And then, carrying out feature fusion on the first left foot step semantic feature vector and the second left foot step semantic feature vector so as to obtain multi-scale relevance feature distribution information among global features of all data items in the left foot step parameters, namely semantic understanding features of the left foot step, and further obtaining the left foot step semantic feature vector.

And processing the right foot step parameters in the context encoder based on the converter and the two-way long-short-term memory neural network model, so as to extract the multi-scale relevance feature distribution information of the global features of each data in the different right foot parameters based on long-distance dependence and medium-short-distance dependence, namely the semantic understanding features of the right foot gait, and further obtain a right foot step semantic feature vector.

Further, the correlation feature distribution information between the right foot and the left foot of the patient is represented by calculating the cooperative gait correlation feature matrix between the right foot step semantic feature vector and the left foot step semantic feature vector, and the correlation feature distribution information is used as a classification feature matrix to be subjected to classification processing in a classifier, so that a classification result of the grade label for representing the rehabilitation training effect is obtained. Thus, the rehabilitation training effect of the patient can be evaluated to track the progress of the disease, and the recovery condition of the patient can be judged.

In particular, in the technical solution of the present application, the collaborative gait correlation feature matrix is obtained by calculating the correlation value between each position of the right foot step semantic feature vector and the left foot step semantic feature vector, so that the collaborative gait correlation feature matrix expresses the correlation feature of the position granularity between the right foot step semantic feature vector and the left foot step semantic feature vector, but at the same time, it is still expected that the collaborative gait correlation feature matrix can express the feature correlation of the vector granularity between the right foot step semantic feature vector and the left foot step semantic feature vector.

Therefore, the applicant of the present application further calculates a kernel walking node distribution fusion feature matrix between the right-foot step semantic feature vector and the left-foot step semantic feature vector, expressed as:

for the right foot step semantic feature vector +.>

And said left-foot-state semantic feature vector +.>

Distance matrix between, i.e.)>

，/>

For distance matrix->

The value of the ith row and jth column of (c),

is->

And->

Distance value between>

Is->

I of the value of (i),>

is->

Is the j-th value of>

and />

Are column vectors, +.>

Is the multiplication of the vector by the vector.

The graph kernel walk node distributes and fuses ideas of feature matrix simulation graph kernel (graph kernel), and the right foot step semantic feature vector is obtained

And said left-foot-state semantic feature vector +.>

Respectively regarded as nodes in the graph, based on the right-footstep semantic feature vector +.>

And said left-foot-state semantic feature vector +.>

Is walked on the distance topology to generalize the topology nodes to +.>

And said left-foot-state semantic feature vector +.>

In a scenario with continuous high-dimensional class spatial properties, representing said right-foot-step semantic feature vector as a topological node>

And said left-foot-state semantic feature vector +.>

And fusing local distribution information in a high-dimensional feature space of the features to realize feature association of vector granularity between the right-foot step semantic feature vector and the left-foot step semantic feature vector.

Further, feature fusion is carried out on the kernel walking node distribution fusion feature matrix and the collaborative gait correlation feature matrix, so that feature expression of the collaborative gait correlation feature matrix is optimized. Accordingly, in a specific example of the present application, the coordinated gait correlation feature matrix after the coordinated gait correlation feature matrix is optimized according to the location points between the kernel walking node distribution fusion feature matrix and the coordinated gait correlation feature matrix may be calculated. And then, the optimized collaborative gait correlation characteristic matrix passes through a classifier to evaluate and detect the rehabilitation training effect. Thus, the rehabilitation training effect of the patient can be accurately evaluated to track the progress of the disease, and the recovery condition of the patient can be accurately judged.

Based on this, the application proposes a data processing system based on gait recognition prediction rehabilitation training effect, which includes: the gait parameter acquisition module is used for acquiring left foot step parameters and right foot step parameters of a patient to be evaluated, wherein the left foot step parameters and the right foot step parameters comprise a step phase, a ground position, a pressure center point track, a step speed, a step frequency, a step length, a steering angle, double lower limb symmetry, a swing phase time proportion and a support phase time proportion, which are acquired by micro sensors deployed in insoles of the patient to be evaluated; the left foot step semantic coding module is used for enabling the left foot step parameters to pass through a context encoder based on a converter to obtain a plurality of left foot step data feature vectors; the first scale feature extraction module is used for cascading the plurality of left-foot step data feature vectors to obtain a first left-foot step semantic feature vector; the second scale feature extraction module is used for inputting the plurality of left foot step data feature vectors into a two-way long-short-term memory neural network model to obtain a second left foot step semantic feature vector; the left-foot-state multi-scale semantic feature fusion module is used for fusing the first left-foot-state semantic feature vector and the second left-foot-state semantic feature vector to obtain a left-foot-state semantic feature vector; the right foot step multi-scale semantic feature extraction module is used for obtaining right foot step semantic feature vectors from the right foot step parameters through the context encoder based on the converter and the two-way long-short-term memory neural network model; the cooperative module is used for calculating a cooperative gait correlation feature matrix between the right foot step semantic feature vector and the left foot step semantic feature vector; and the data processing result generation module is used for enabling the collaborative gait correlation feature matrix to pass through a classifier to obtain a classification result, and the classification result is used for representing a grade label of the rehabilitation training effect.

Having described the basic principles of the present application, various non-limiting embodiments of the present application will now be described in detail with reference to the accompanying drawings.

Exemplary System

FIG. 1 is a block diagram of a data processing system for predicting rehabilitation training effects based on gait recognition according to embodiments of the present application. As shown in fig. 1, a gait recognition-based predictive rehabilitation training effect data processing system 300 according to an embodiment of the present application includes: a gait parameter acquisition module 310; a left foot step semantic coding module 320; a first scale feature extraction module 330; a second scale feature extraction module 340; a left footstep multi-scale semantic feature fusion module 350; a right footstep multi-scale semantic feature extraction module 360; a collaboration module 370; and a data processing result generation module 380.

The gait parameter acquisition module 310 is configured to acquire a left foot step parameter and a right foot step parameter of a patient to be evaluated, which are acquired by a micro sensor deployed in an insole of the patient to be evaluated, wherein the left foot step parameter and the right foot step parameter include a step phase, a landing position, a pressure center point track, a pace speed, a step frequency, a step length, a steering angle, a double lower limb symmetry, a swing phase time proportion and a support phase time proportion; the left-foot step semantic coding module 320 is configured to pass the left-foot step parameters through a context encoder based on a converter to obtain a plurality of left-foot step data feature vectors; the first scale feature extraction module 330 is configured to concatenate the plurality of left-foot step data feature vectors to obtain a first left-foot step semantic feature vector; the second scale feature extraction module 340 is configured to input the plurality of left-foot step data feature vectors into a two-way long-short term memory neural network model to obtain a second left-foot step semantic feature vector; the left-foot gait multi-scale semantic feature fusion module 350 is configured to fuse the first left-foot gait semantic feature vector and the second left-foot gait semantic feature vector to obtain a left-foot gait semantic feature vector; the right-foot-stage multi-scale semantic feature extraction module 360 is configured to obtain a right-foot-stage semantic feature vector from the right-foot-stage parameter through the context encoder based on the converter and the two-way long-short-term memory neural network model; the collaboration module 370 is configured to calculate a collaborative gait correlation feature matrix between the right foot step semantic feature vector and the left foot step semantic feature vector; and the data processing result generating module 380 is configured to pass the collaborative gait correlation feature matrix through a classifier to obtain a classification result, where the classification result is used as a class label for representing a rehabilitation training effect.

Fig. 2 is a system architecture diagram of a data processing system based on gait recognition predictive rehabilitation training effects according to an embodiment of the present application. As shown in fig. 2, in the network architecture, first, acquiring, by the gait parameter acquisition module 310, left and right foot step parameters of a patient to be evaluated, wherein the left and right foot step parameters include a step phase, a ground location and a pressure center point track, a pace, a step frequency, a step length, a steering angle, a dual lower limb symmetry, a swing phase time proportion, and a support phase time proportion, by a micro sensor deployed in an insole of the patient to be evaluated; secondly, the left foot step semantic coding module 320 obtains a plurality of left foot step data feature vectors by passing the left foot step parameters obtained by the gait parameter collection module 310 through a context encoder based on a converter; the first scale feature extraction module 330 concatenates the plurality of left-foot step data feature vectors obtained by the left-foot step semantic coding module 320 to obtain a first left-foot step semantic feature vector; the second scale feature extraction module 340 inputs the plurality of left-foot step data feature vectors obtained by the left-foot step semantic coding module 320 into a two-way long-short term memory neural network model to obtain a second left-foot step semantic feature vector; next, the left-foot gait multi-scale semantic feature fusion module 350 fuses the first left-foot step semantic feature vector obtained by the first scale feature extraction module 330 and the second left-foot step semantic feature vector obtained by the second scale feature extraction module 340 to obtain a left-foot step semantic feature vector; the right-foot-state multi-scale semantic feature extraction module 360 obtains a right-foot-state semantic feature vector from the right foot-state parameters acquired by the gait parameter acquisition module 310 through the context encoder based on the converter and the two-way long-short-term memory neural network model; then, the collaboration module 370 calculates a collaboration gait correlation feature matrix between the right foot gait multi-scale semantic feature vector obtained by the right foot gait multi-scale semantic feature extraction module 360 and the left foot gait multi-scale semantic feature fusion module 350; further, the data processing result generating module 380 passes the collaborative gait correlation feature matrix through a classifier to obtain a classification result, where the classification result is used as a class label for representing the rehabilitation training effect.

Specifically, during operation of the data processing system 300 for predicting rehabilitation training effect based on gait recognition, the gait parameter acquisition module 310 is configured to acquire a left foot step parameter and a right foot step parameter of a patient to be evaluated, which are acquired by a micro sensor deployed in an insole of the patient to be evaluated, wherein the left foot step parameter and the right foot step parameter include a walking phase, a landing position, a pressure center point track, a walking speed, a walking frequency, a step length, a steering angle, a dual lower limb symmetry, a swing phase time proportion, and a support phase time proportion. It should be understood that the gait analysis system can objectively and quantitatively evaluate the gait function and the severity of the disease, track the progress of the disease, and support the diagnosis of early walking dysfunction, and play an important role in judging the surgical effect and the postoperative recovery. Gait analysis system is through placing miniature sensor in the shoe-pad, and the testee dresses the shoe-pad and walks, carries out the collection of gait parameter, and through data processing, the calculation left foot and right foot step parameter includes: the walking phase, the grounding position, the pressure center point track, the walking speed, the walking frequency, the step length, the steering angle, the symmetry of the double lower limbs, the swing phase time proportion, the support phase time proportion and the like of the left foot and the right foot. And finally, the test data are transmitted to and stored in the computer terminal. Comparing the test value with a normal reference value range, determining abnormal key influence factors and compensatory changes, and providing suggestions and references for clinical diagnosis, clinical decision making and treatment effect evaluation. Thus, in one specific example of the present application, left and right foot parameters of a patient to be evaluated may be acquired by miniature sensors deployed within the insole of the patient to be evaluated, where the left and right foot parameters include a stride phase, a footprint and center-of-pressure point trajectory, a stride rate, a stride frequency, a stride length, a steering angle, dual lower limb symmetry, a swing phase time ratio, and a support phase time ratio.

Specifically, during the operation of the gait recognition-based predictive rehabilitation training effect data processing system 300, the left-foot step semantic coding module 320 is configured to pass the left-foot step parameters through a context encoder based on a converter to obtain a plurality of left-foot step data feature vectors. The relation that each data item in the left foot step parameter has relevance is considered, that is, each data item of the left foot step parameter has relevance characteristic distribution representation in high-dimensional space. Therefore, in order to be able to accurately evaluate the effect of the rehabilitation training, the left-foot step parameters are further passed through a context encoder based on a converter to obtain a plurality of left-foot step data feature vectors. That is, based on the transform concept, the converter is used to capture the correlation characteristic that depends on a long distance, and the global context correlation encoding is performed on each data item in the left-foot step parameters to obtain a context global correlation feature representation with the global feature of the left-foot step parameters as the context, that is, the plurality of left-foot step data feature vectors. It should be appreciated that in the solution of the present application, the context-dependent feature representation of each data item feature in the left-foot step parameter with respect to the overall feature of the left-foot step parameter may be captured by the converter-based encoder. More specifically, the passing the left-foot step parameters through a converter-based context encoder to obtain a plurality of left-foot step data feature vectors includes: firstly, word segmentation processing is carried out on the left step parameters so as to convert the left step parameters into word sequences composed of a plurality of words; mapping each word in the word sequence to a word vector using an embedding layer of the converter-based context encoder to obtain a sequence of word vectors; further, the sequence of word vectors is globally based context semantic encoded using a translator of the translator-based context encoder to obtain the plurality of left-foot step data feature vectors. Wherein the performing, by the converter of the converter-based context encoder, global-based context semantic coding on the sequence of word vectors to obtain the plurality of left-foot step data feature vectors comprises: one-dimensional arrangement is carried out on the sequence of the word vectors to obtain global word feature vectors; calculating the product between the global word feature vector and the transpose vector of each word vector in the sequence of word vectors to obtain a plurality of self-attention association matrices; respectively carrying out standardization processing on each self-attention correlation matrix in the plurality of self-attention correlation matrices to obtain a plurality of standardized self-attention correlation matrices; obtaining a plurality of probability values by using a Softmax classification function through each normalized self-attention correlation matrix in the normalized self-attention correlation matrices; and weighting each word vector in the sequence of word vectors by taking each probability value in the plurality of probability values as a weight so as to obtain the plurality of left-foot step data feature vectors.

Fig. 3 is a block diagram of a left foot step semantic coding module in a data processing system based on gait recognition prediction rehabilitation training effects according to an embodiment of the present application. As shown in fig. 3, the left-foot step semantic coding module 320 includes: a word segmentation unit 321, configured to perform word segmentation processing on the left step parameter to convert the left step parameter into a word sequence composed of a plurality of words; an embedded encoding unit 322 for mapping each word in the word sequence to a word vector using an embedded layer of the converter-based context encoder to obtain a sequence of word vectors; a context coding unit 323, configured to perform global context semantic coding on the sequence of word vectors using a converter of the converter-based context encoder to obtain the plurality of left-foot step data feature vectors.

Specifically, during the operation of the data processing system 300 based on gait recognition prediction rehabilitation training effect, the first scale feature extraction module 330 is configured to concatenate the plurality of left-foot step data feature vectors to obtain a first left-foot step semantic feature vector. That is, the plurality of left-foot step data feature vectors are cascaded to obtain a first left-foot step semantic feature vector of each data item with the left-foot step parameters based on global associated feature distribution information which depends on a long distance. In one technical solution of the present application, the concatenating the plurality of left-foot step data feature vectors to obtain a first left-foot step semantic feature vector includes: cascading the plurality of left-foot step data feature vectors with the following formula to obtain a first left-foot step semantic feature vector; wherein, the formula is:

wherein ,

representing the plurality of left foot step data feature vectors,/->

Representing a cascade function->

Representing the first left-foot step semantic feature vector.

Specifically, during the operation of the data processing system 300 based on gait recognition prediction rehabilitation training effect, the second scale feature extraction module 340 is configured to input the plurality of left foot step data feature vectors into a two-way long-short term memory neural network model to obtain a second left foot step semantic feature vector. It should be appreciated that there are different scales of hidden relevance feature distribution information between the global features of the individual data items taking into account the left foot step parameters. That is, the global features of the individual data items of the left foot step parameters have different degrees of relevance over different spans of data types, so that feature extraction is required for the relevance feature distribution of the individual data items over different spans of data types in order to be able to evaluate the rehabilitation effect accuratelyTaking. Specifically, in the technical scheme of the application, the plurality of left-foot step data feature vectors are input into a two-way long-short-term memory neural network model to obtain a second left-foot step semantic feature vector. It should be understood that the two-way long-short term memory neural network model #

Long Short-Term Memory) to enable the weight of the neural network to be updated by adding an input gate, an output gate and a forgetting gate, and the weight scale of different channels can be dynamically changed under the condition of fixed network model parameters, so that the problem of gradient disappearance or gradient expansion can be avoided. In particular, the two-way long-short-term memory neural network model is formed by combining a forward LSTM and a backward LSTM, and the forward LSTM can learn the association characteristic distribution information of the global characteristics of the local area before the current data type and related to each data item; the relevant feature distribution information about the global features of the individual data items of the subsequent local region of the current data type can then be learned towards the LSTM. Therefore, the second left-foot step semantic feature vector obtained through the two-way long-short term memory neural network model learns implicit associated feature information based on middle-short distance dependence of each data item. More specifically, the inputting the plurality of left-foot step data feature vectors into the two-way long-short term memory neural network model to obtain a second left-foot step semantic feature vector includes: passing the plurality of left foot step data feature vectors through the two-way long-short term memory neural network model to obtain a plurality of context left foot step data feature vectors; and cascading the plurality of context left-foot step data feature vectors to obtain the second left-foot step semantic feature vector.

Specifically, during the operation of the data processing system 300 based on gait recognition prediction rehabilitation training effect, the left-foot gait multi-scale semantic feature fusion module 350 is configured to fuse the first left-foot step semantic feature vector and the second left-foot step semantic feature vector to obtain a left-foot step semantic feature vector. In the technical scheme of the application, feature fusion is carried out on the first left foot step semantic feature vector and the second left foot step semantic feature vector, so that multi-scale relevance feature distribution information among global features of all data items in the left foot step parameters is obtained, namely semantic understanding features of the left foot step are obtained, and therefore the left foot step semantic feature vector is obtained. In a specific example of the application, the right-foot step semantic feature vector and the left-foot step semantic feature vector are subjected to association coding according to the following formula to obtain an initial association feature matrix; wherein, the formula is:

wherein

Representing the right foot step semantic feature vector, < >>

Representing the left-foot step semantic feature vector,

representing the initial association feature matrix, +.>

For matrix multiplication with vectors.

Specifically, during the operation of the gait recognition prediction rehabilitation training effect-based data processing system 300, the right-foot-stage multi-scale semantic feature extraction module 360 is configured to obtain a right-foot-stage semantic feature vector from the right-foot-stage parameters through the context encoder based on the converter and the two-way long-short-term memory neural network model. In the technical scheme of the application, the right foot step parameters are processed through the context encoder based on the converter and the two-way long-short-term memory neural network model, so that global characteristic of each data in the different right foot parameters is extracted based on multi-scale relevance characteristic distribution information of long-distance dependence and medium-short-distance dependence, namely semantic understanding characteristics of the right foot step, and a right foot step semantic characteristic vector is obtained. More specifically, word segmentation processing is performed on the right foot step parameters so as to convert the right foot step parameters into word sequences composed of a plurality of words; mapping each word in the word sequence to a word vector using an embedding layer of the converter-based context encoder to obtain a sequence of word vectors; the sequence of word vectors is globally context-based semantic encoded using a translator of the translator-based context encoder to obtain the plurality of right-foot step data feature vectors. Then, the right foot step data feature vectors pass through the two-way long-short-term memory neural network model to obtain a plurality of context right foot step data feature vectors; and cascading the context right-foot step data feature vectors to obtain the right-foot step semantic feature vector.

Specifically, during the operation of the data processing system 300 based on gait recognition prediction rehabilitation training effect, the collaboration module 370 is configured to calculate a collaborative gait correlation feature matrix between the right foot step semantic feature vector and the left foot step semantic feature vector. It should be understood that, the correlation feature distribution information between the right foot and the left foot of the patient is represented by calculating the collaborative gait correlation feature matrix between the right foot step semantic feature vector and the left foot step semantic feature vector, and the correlation feature distribution information is used as a classification feature matrix to be classified in a classifier, so as to obtain a classification result of the class label for representing the rehabilitation training effect. In particular, in the technical solution of the present application, the collaborative gait correlation feature matrix is obtained by calculating the correlation value between each position of the right foot step semantic feature vector and the left foot step semantic feature vector, so that the collaborative gait correlation feature matrix expresses the correlation feature of the position granularity between the right foot step semantic feature vector and the left foot step semantic feature vector, but at the same time, it is still expected that the collaborative gait correlation feature matrix can express the feature correlation of the vector granularity between the right foot step semantic feature vector and the left foot step semantic feature vector. Therefore, the applicant of the present application further calculates a kernel walking node distribution fusion feature matrix between the right-foot step semantic feature vector and the left-foot step semantic feature vector, expressed as:

For the right foot step semantic feature vector +.>

And said left-foot-state semantic feature vector +.>

Distance matrix between, i.e.)>

And->

and />

Are column vectors, +.>

Is the multiplication of the vector by the vector. The graph kernel walk node distributes and fuses ideas of feature matrix simulation graph kernel (graph kernel), and the right foot step semantic feature vector is +.>

And said left-foot-state semantic feature vector +.>

And the left foot stepSemantic feature vector +.>

Is walked on the distance topology to generalize the topology nodes to +.>

And said left-foot-state semantic feature vector +.>

And said left-foot-state semantic feature vector +.>

And fusing local distribution information in a high-dimensional feature space of the features to realize feature association of vector granularity between the right-foot step semantic feature vector and the left-foot step semantic feature vector. Further, feature fusion is carried out on the kernel walking node distribution fusion feature matrix and the collaborative gait correlation feature matrix, so that feature expression of the collaborative gait correlation feature matrix is optimized. Accordingly, in a specific example of the present application, the coordinated gait correlation feature matrix after the coordinated gait correlation feature matrix is optimized according to the location points between the kernel walking node distribution fusion feature matrix and the coordinated gait correlation feature matrix may be calculated. And then, the optimized collaborative gait correlation characteristic matrix passes through a classifier to evaluate and detect the rehabilitation training effect. Thus, the rehabilitation training effect of the patient can be accurately evaluated to track the progress of the disease, and the recovery condition of the patient can be accurately judged.

FIG. 4 is a block diagram of collaboration modules in a data processing system based on gait recognition predictive rehabilitation training effects according to an embodiment of the application. As shown in fig. 4, the collaboration module 370 includes:an initial association feature matrix calculation unit 371, configured to perform association encoding on the right-foot step semantic feature vector and the left-foot step semantic feature vector to obtain an initial association feature matrix; a kernel walking node distribution fusion feature matrix calculation unit 372, configured to calculate a kernel walking node distribution fusion feature matrix between the right-foot-state semantic feature vector and the left-foot-state semantic feature vector, where the kernel walking node distribution fusion feature matrix relates to a distance matrix between the right-foot-state semantic feature vector and the left-foot-state semantic feature vector; and a fusion unit 373, configured to fuse the initial association feature matrix and the kernel walking node distribution fusion feature matrix to obtain the collaborative gait association feature matrix. Wherein the initial correlation feature matrix calculation unit 371 is further configured to: performing association coding on the right foot step semantic feature vector and the left foot step semantic feature vector by using the following formula to obtain an initial association feature matrix; wherein, the formula is:

, wherein />

Representing the right foot step semantic feature vector, < >>

Representing the left-foot step semantic feature vector, < >>

Representing the initial association feature matrix. More specifically, the kernel walking node distribution fusion feature matrix computing unit 372 is further configured to: calculating the kernel walking node distribution fusion feature matrix between the right foot step semantic feature vector and the left foot step semantic feature vector according to the following formula; wherein, the formula is:

wherein ,

representing the right foot step semantic feature vector, < >>

Representing the left-foot step semantic feature vector,

Is a distance matrix between the right foot step semantic feature vector and the left foot step semantic feature vector, and +.>

and />

Are column vectors, +.>

Is the multiplication of the vector by the vector.

Specifically, during the operation of the data processing system 300 for predicting rehabilitation training effect based on gait recognition, the data processing result generating module 380 is configured to pass the collaborative gait correlation feature matrix through a classifier to obtain a classification result, where the classification result is used for a class label for representing rehabilitation training effect. That is, the classification feature matrix is subjected to classification processing in the classifier to obtain a classification result of the class label for representing the rehabilitation training effect. Thus, the rehabilitation training effect of the patient can be evaluated to track the progress of the disease, and the recovery condition of the patient can be judged. In a specific example of the present application, the classifier is used to process the collaborative gait correlation feature matrix to obtain a classification result according to the following formula:

, wherein />

Representing the projection of the collaborative gait correlation feature matrix as a vector,>

to->

Weight matrix for all connection layers of each layer, < ->

To->

Bias vector representing all connected layers of each layer, < ->

Output information for representing the Softmax layer, i.e., classification results of the class labels for representing the rehabilitation training effect. "/>

The multiple dot numbers in "are used to indicate +.>

To->

Omission and +.>

To->

The symbol n is the omission of the classifier with n fully connected layers, the first layerThe input information is classified feature vector, the output information of the upper layer is used as the input information of the lower layer, and the output information of the last layer is input into +.>

Outputting classification result of the grade label representing rehabilitation training effect in the function, wherein +_>

To->

Weight matrix of the first layer to the n-th layer respectively,>

to->

Is the bias vector of the first layer through the nth layer. Specifically, the classifier includes a plurality of fully connected layers and a Softmax layer cascaded with a last fully connected layer of the plurality of fully connected layers. In the classification process of the classifier, the collaborative gait correlation feature matrix is projected as a vector, for example, in a specific example, the collaborative gait correlation feature matrix is expanded along a row vector or a column vector to be a classification feature vector; then, performing multiple full-connection coding on the classification feature vectors by using multiple full-connection layers of the classifier to obtain coded classification feature vectors; further, the coding classification feature vector is input into a Softmax layer of the classifier, that is, the coding classification feature vector is subjected to classification processing using the Softmax classification function to obtain a classification result of a class label for representing the rehabilitation training effect.

Fig. 5 is a block diagram of a data processing result generating module in a data processing system based on gait recognition prediction rehabilitation training effect according to an embodiment of the present application. As shown in fig. 5, the data processing result generating module 380 includes: a developing unit 381 for developing the collaborative gait correlation feature matrix into a classification feature vector based on a row vector or a column vector; a full-connection encoding unit 382, configured to perform full-connection encoding on the classification feature vector by using multiple full-connection layers of the classifier to obtain an encoded classification feature vector; and a classification result generating unit 383, configured to pass the encoded classification feature vector through a Softmax classification function of the classifier to obtain the classification result.

In summary, the data processing system 300 based on gait recognition prediction rehabilitation training effect according to the embodiment of the application is illustrated, wherein the evaluation of rehabilitation training effect is performed by extracting the multi-scale relevance feature distribution information of each parameter item of the left foot step parameter and the right foot step parameter of the patient based on long-distance dependence and middle-short-distance dependence by adopting an artificial intelligence algorithm based on deep learning, and further performing gait analysis and judgment based on the cooperative features of the two. Thus, the rehabilitation training effect of the patient can be accurately evaluated to track the progress of the disease, and the recovery condition of the patient can be accurately judged.

As described above, the gait recognition-based data processing system for predicting rehabilitation training effects according to the embodiments of the present application may be implemented in various terminal devices. In one example, the gait recognition-based predictive rehabilitation training effect data processing system 300 according to embodiments of the present application may be integrated into the terminal device as a software module and/or hardware module. For example, the gait recognition based predictive rehabilitation training effect based data processing system 300 may be a software module in the operating system of the terminal device or may be an application developed for the terminal device; of course, the gait recognition based predictive rehabilitation training effect based data processing system 300 can also be one of the hardware modules of the terminal device.

Alternatively, in another example, the gait recognition-based predictive rehabilitation training effect data processing system 300 and the terminal device may be separate devices, and the gait recognition-based predictive rehabilitation training effect data processing system 300 may be connected to the terminal device through a wired and/or wireless network and transmit interactive information in a contracted data format.

Exemplary method

Fig. 6 is a flowchart of a data processing method for predicting rehabilitation training effects based on gait recognition according to an embodiment of the present application. As shown in fig. 6, a data processing method based on gait recognition prediction rehabilitation training effect according to an embodiment of the present application includes the steps of: s110, acquiring left foot step parameters and right foot step parameters of a patient to be evaluated, wherein the left foot step parameters and the right foot step parameters comprise a step phase, a ground position, a pressure center point track, a pace speed, a step frequency, a step length, a steering angle, double lower limb symmetry, a swing phase time proportion and a support phase time proportion, which are acquired by micro sensors deployed in insoles of the patient to be evaluated; s120, the left foot step parameters pass through a context encoder based on a converter to obtain a plurality of left foot step data feature vectors; s130, cascading the plurality of left-foot step data feature vectors to obtain a first left-foot step semantic feature vector; s140, inputting the plurality of left foot step data feature vectors into a two-way long-short term memory neural network model to obtain a second left foot step semantic feature vector; s150, fusing the first left-foot step semantic feature vector and the second left-foot step semantic feature vector to obtain a left-foot step semantic feature vector; s160, obtaining a right-foot-state semantic feature vector from the right-foot-state parameter through the context encoder based on the converter and the two-way long-short-term memory neural network model; s170, calculating a collaborative gait correlation feature matrix between the right foot step semantic feature vector and the left foot step semantic feature vector; and S180, the collaborative gait correlation feature matrix is passed through a classifier to obtain a classification result, and the classification result is used for representing a grade label of the rehabilitation training effect.

In one example, in the above data processing method based on gait recognition prediction rehabilitation training effect, the step S120 includes: word segmentation processing is carried out on the left step parameters so as to convert the left step parameters into word sequences composed of a plurality of words; mapping each word in the word sequence to a word vector using an embedding layer of the converter-based context encoder to obtain a sequence of word vectors; the sequence of word vectors is globally context-based semantic encoded using a translator of the translator-based context encoder to obtain the plurality of left-foot step data feature vectors.

In one example, in the above data processing method for predicting rehabilitation training effect based on gait recognition, the step S150 includes: fusing the first left-foot step semantic feature vector and the second left-foot step semantic feature vector to obtain a left-foot step semantic feature vector by the following formula;

wherein, the formula is:

wherein

Representing the first left-foot step semantic feature vector,>

representing the second left-foot step semantic feature vector,>

representing the left-foot step semantic feature vector, < > >

Representing the sum by location.

In one example, in the above data processing method based on gait recognition prediction rehabilitation training effect, the step S170 includes: performing association coding on the right foot step semantic feature vector and the left foot step semantic feature vector to obtain an initial association feature matrix; calculating the right foot step semantic feature vectorAnd a kernel walking node distribution fusion feature matrix between the left-foot step semantic feature vectors, wherein the kernel walking node distribution fusion feature matrix is related to a distance matrix between the right-foot step semantic feature vector and the left-foot step semantic feature vector; and fusing the initial association feature matrix and the kernel walking node distribution fusion feature matrix to obtain the collaborative gait association feature matrix. The performing association coding on the right-foot step semantic feature vector and the left-foot step semantic feature vector to obtain an initial association feature matrix includes: performing association coding on the right foot step semantic feature vector and the left foot step semantic feature vector by using the following formula to obtain an initial association feature matrix; wherein, the formula is:

, wherein />

Representing the right foot step semantic feature vector, < >>

Representing the left-foot step semantic feature vector, < >>

Representing the initial association feature matrix. More specifically, the calculating the kernel walking node distribution fusion feature matrix between the right-foot step semantic feature vector and the left-foot step semantic feature vector includes: calculating the kernel walking node distribution fusion feature matrix between the right foot step semantic feature vector and the left foot step semantic feature vector according to the following formula; wherein, the formula is:

wherein ,

representing the right sideFoot-step semantic feature vector,/->

Representing the left-foot step semantic feature vector,

and />

Are column vectors, +.>

Is the multiplication of the vector by the vector.

In one example, in the above data processing method based on gait recognition prediction rehabilitation training effect, the step S180 includes: expanding the collaborative gait correlation feature matrix into classification feature vectors based on row vectors or column vectors; performing full-connection coding on the classification feature vectors by using a plurality of full-connection layers of the classifier to obtain coded classification feature vectors; and passing the coding classification feature vector through a Softmax classification function of the classifier to obtain the classification result.

In summary, the data processing method based on gait recognition prediction rehabilitation training effect according to the embodiment of the application is clarified, wherein the artificial intelligence algorithm based on deep learning is adopted to extract the multi-scale relevance characteristic distribution information of each parameter item of the left foot step parameter and the right foot step parameter of a patient based on long-distance dependence and middle-short distance dependence, and further based on the cooperative characteristics of the two, gait analysis and judgment are carried out, so that the rehabilitation training effect is evaluated. Thus, the rehabilitation training effect of the patient can be accurately evaluated to track the progress of the disease, and the recovery condition of the patient can be accurately judged.

Exemplary electronic device

Next, an electronic device according to an embodiment of the present application is described with reference to fig. 7.

Fig. 7 illustrates a block diagram of an electronic device according to an embodiment of the present application.

As shown in fig. 7, the electronic device 10 includes one or more processors 11 and a memory 12.

The processor 11 may be a Central Processing Unit (CPU) or other form of processing unit having data processing and/or instruction execution capabilities, and may control other components in the electronic device 10 to perform desired functions.

Memory 12 may include one or more computer program products that may include various forms of computer-readable storage media, such as volatile memory and/or non-volatile memory. The volatile memory may include, for example, random Access Memory (RAM) and/or cache memory (cache), and the like. The non-volatile memory may include, for example, read Only Memory (ROM), hard disk, flash memory, and the like. One or more computer program instructions may be stored on the computer readable storage medium that can be executed by the processor 11 to implement the functions in the gait recognition-based predictive rehabilitation training effect data processing system and/or other desired functions of the various embodiments of the present application described above. Various contents such as left-foot step data feature vectors may also be stored in the computer-readable storage medium.

In one example, the electronic device 10 may further include: an input device 13 and an output device 14, which are interconnected by a bus system and/or other forms of connection mechanisms (not shown).

The input means 13 may comprise, for example, a keyboard, a mouse, etc.

The output device 14 may output various information including the classification result and the like to the outside. The output means 14 may include, for example, a display, speakers, a printer, and a communication network and remote output devices connected thereto, etc.

Of course, only some of the components of the electronic device 10 that are relevant to the present application are shown in fig. 7 for simplicity, components such as buses, input/output interfaces, etc. are omitted. In addition, the electronic device 10 may include any other suitable components depending on the particular application.

Exemplary computer program product and computer readable storage Medium

In addition to the methods and apparatus described above, embodiments of the present application may also be a computer program product comprising computer program instructions which, when executed by a processor, cause the processor to perform steps in the functions of the data processing method of predicting rehabilitation training effects based on gait recognition described in the above section of the "exemplary system" of the present application.

The computer program product may write program code for performing the operations of embodiments of the present application in any combination of one or more programming languages, including an object oriented programming language such as Java, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device, partly on a remote computing device, or entirely on the remote computing device or server.

Furthermore, embodiments of the present application may also be a computer-readable storage medium, having stored thereon computer program instructions, which when executed by a processor, cause the processor to perform steps in the functions of the data processing method for predicting rehabilitation training effects based on gait recognition according to the various embodiments of the present application described in the above-mentioned "exemplary systems" section of the present specification.

The computer readable storage medium may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may include, for example, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium would include the following: an electrical connection having one or more wires, a portable disk, a hard disk, random Access Memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

The basic principles of the present application have been described above in connection with specific embodiments, however, it should be noted that the advantages, benefits, effects, etc. mentioned in the present application are merely examples and not limiting, and these advantages, benefits, effects, etc. are not to be considered as necessarily possessed by the various embodiments of the present application. Furthermore, the specific details disclosed herein are for purposes of illustration and understanding only, and are not intended to be limiting, as the application is not intended to be limited to the details disclosed herein as such.

The block diagrams of the devices, apparatuses, devices, systems referred to in this application are only illustrative examples and are not intended to require or imply that the connections, arrangements, configurations must be made in the manner shown in the block diagrams. As will be appreciated by one of skill in the art, the devices, apparatuses, devices, systems may be connected, arranged, configured in any manner. Words such as "including," "comprising," "having," and the like are words of openness and mean "including but not limited to," and are used interchangeably therewith. The terms "or" and "as used herein refer to and are used interchangeably with the term" and/or "unless the context clearly indicates otherwise. The term "such as" as used herein refers to, and is used interchangeably with, the phrase "such as, but not limited to.

It is also noted that in the apparatus, devices and methods of the present application, the components or steps may be disassembled and/or assembled. Such decomposition and/or recombination should be considered as equivalent to the present application.

The previous description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other aspects without departing from the scope of the application. Thus, the present application is not intended to be limited to the aspects shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

The foregoing description has been presented for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of the application to the form disclosed herein. Although a number of example aspects and embodiments have been discussed above, a person of ordinary skill in the art will recognize certain variations, modifications, alterations, additions, and subcombinations thereof.

Claims

1. A data processing system for predicting rehabilitation training effects based on gait recognition, comprising:

2. The gait recognition-based predictive rehabilitation training effect data processing system of claim 1, wherein the left foot step semantic coding module comprises:

the word segmentation unit is used for carrying out word segmentation processing on the left step parameters so as to convert the left step parameters into word sequences composed of a plurality of words;

an embedded encoding unit for mapping each word in the word sequence to a word vector using an embedded layer of the converter-based context encoder to obtain a sequence of word vectors;

and the context coding unit is used for performing global context semantic coding on the sequence of the word vectors by using a converter of the context coder based on the converter so as to obtain the plurality of left-foot step data feature vectors.

3. The gait recognition-based predictive rehabilitation training effect data processing system of claim 2, wherein the left foot gait multiscale semantic feature fusion module is further configured to: fusing the first left-foot step semantic feature vector and the second left-foot step semantic feature vector to obtain a left-foot step semantic feature vector by the following formula;

wherein, the formula is:

wherein

Representing the first left-foot step semantic feature vector,>

representing the second left-foot step semantic feature vector,>

representing the left-foot step semantic feature vector, < >>

Representing the sum by location.

4. The gait recognition-based predictive rehabilitation training effect data processing system of claim 3, wherein the collaboration module comprises:

the initial association feature matrix calculation unit is used for carrying out association coding on the right foot step semantic feature vector and the left foot step semantic feature vector to obtain an initial association feature matrix;

the kernel walking node distribution fusion feature matrix calculation unit is used for calculating a kernel walking node distribution fusion feature matrix between the right foot step semantic feature vector and the left foot step semantic feature vector, wherein the kernel walking node distribution fusion feature matrix is related to a distance matrix between the right foot step semantic feature vector and the left foot step semantic feature vector; and

And the fusion unit is used for fusing the initial association characteristic matrix and the kernel walking node distribution fusion characteristic matrix to obtain the collaborative gait association characteristic matrix.

5. The gait recognition-based predictive rehabilitation training effect data processing system of claim 4, wherein the initial correlation feature matrix calculation unit is further configured to: performing association coding on the right foot step semantic feature vector and the left foot step semantic feature vector by using the following formula to obtain an initial association feature matrix;

wherein, the formula is:

wherein

Representing the right foot step semantic feature vector, < >>

Representing the left-foot step semantic feature vector, < >>

Representing the initial association feature matrix, +.>

For matrix multiplication with vectors.

6. The gait recognition-based predictive rehabilitation training effect data processing system according to claim 5, wherein the kernel walking node distribution fusion feature matrix calculation unit is further configured to: calculating the kernel walking node distribution fusion feature matrix between the right foot step semantic feature vector and the left foot step semantic feature vector according to the following formula;

Wherein, the formula is:

wherein ,

representing the right foot step semantic feature vector, < >>

Representing the left-foot step semantic feature vector, < >>

and />

Are column vectors, +.>

Is the multiplication of the vector by the vector.

7. The gait recognition prediction rehabilitation training effect-based data processing system according to claim 6, wherein the fusion unit is further configured to calculate a per-position point between the initial correlation feature matrix and the kernel walking node distribution fusion feature matrix to obtain the collaborative gait correlation feature matrix.

8. The gait recognition-based predictive rehabilitation training effect data processing system of claim 7, wherein the data processing result generation module comprises:

the unfolding unit is used for unfolding the collaborative gait correlation feature matrix into a classification feature vector based on a row vector or a column vector;

The full-connection coding unit is used for carrying out full-connection coding on the classification characteristic vectors by using a plurality of full-connection layers of the classifier so as to obtain coded classification characteristic vectors; and

and the classification result generation unit is used for passing the coding classification feature vector through a Softmax classification function of the classifier to obtain the classification result.