CN110046675A - A kind of the exercise ability of lower limbs appraisal procedure based on improved convolutional neural networks - Google Patents
A kind of the exercise ability of lower limbs appraisal procedure based on improved convolutional neural networks Download PDFInfo
- Publication number
- CN110046675A CN110046675A CN201910337860.7A CN201910337860A CN110046675A CN 110046675 A CN110046675 A CN 110046675A CN 201910337860 A CN201910337860 A CN 201910337860A CN 110046675 A CN110046675 A CN 110046675A
- Authority
- CN
- China
- Prior art keywords
- feature
- gait
- neural networks
- convolutional neural
- lower limbs
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 70
- 238000013527 convolutional neural network Methods 0.000 title claims abstract description 50
- 210000003141 lower extremity Anatomy 0.000 title claims abstract description 33
- 230000005021 gait Effects 0.000 claims abstract description 77
- 210000000629 knee joint Anatomy 0.000 claims abstract description 27
- 238000005457 optimization Methods 0.000 claims abstract description 24
- 230000009467 reduction Effects 0.000 claims abstract description 24
- 238000001914 filtration Methods 0.000 claims abstract description 17
- 238000011156 evaluation Methods 0.000 claims abstract description 8
- 239000000284 extract Substances 0.000 claims abstract description 8
- 230000002146 bilateral effect Effects 0.000 claims abstract description 7
- 238000006243 chemical reaction Methods 0.000 claims abstract description 7
- 238000004458 analytical method Methods 0.000 claims abstract description 6
- 239000011159 matrix material Substances 0.000 claims description 21
- 230000008569 process Effects 0.000 claims description 14
- 230000004927 fusion Effects 0.000 claims description 12
- 238000010606 normalization Methods 0.000 claims description 12
- 210000000544 articulatio talocruralis Anatomy 0.000 claims description 10
- 206010054197 Gait deviation Diseases 0.000 claims description 9
- 210000004394 hip joint Anatomy 0.000 claims description 9
- 208000012661 Dyskinesia Diseases 0.000 claims description 7
- 230000036541 health Effects 0.000 claims description 6
- 230000000007 visual effect Effects 0.000 claims description 6
- 238000011478 gradient descent method Methods 0.000 claims description 3
- 210000003127 knee Anatomy 0.000 claims description 3
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 claims description 3
- 208000001613 Gambling Diseases 0.000 claims description 2
- 101000911772 Homo sapiens Hsc70-interacting protein Proteins 0.000 claims description 2
- 210000003423 ankle Anatomy 0.000 claims 1
- 238000005259 measurement Methods 0.000 description 5
- PCHJSUWPFVWCPO-UHFFFAOYSA-N gold Chemical compound [Au] PCHJSUWPFVWCPO-UHFFFAOYSA-N 0.000 description 4
- 239000010931 gold Substances 0.000 description 4
- 229910052737 gold Inorganic materials 0.000 description 4
- 238000010220 Pearson correlation analysis Methods 0.000 description 3
- 230000007812 deficiency Effects 0.000 description 3
- 238000005520 cutting process Methods 0.000 description 2
- 238000000605 extraction Methods 0.000 description 2
- 210000003414 extremity Anatomy 0.000 description 2
- 238000013139 quantization Methods 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 238000012549 training Methods 0.000 description 2
- 241000066478 Aspidiotus cryptomeriae Species 0.000 description 1
- 206010017577 Gait disturbance Diseases 0.000 description 1
- 230000002159 abnormal effect Effects 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 230000007423 decrease Effects 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 201000010099 disease Diseases 0.000 description 1
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 230000001537 neural effect Effects 0.000 description 1
- 238000004321 preservation Methods 0.000 description 1
- 238000000513 principal component analysis Methods 0.000 description 1
- 230000008439 repair process Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
- G06F18/253—Fusion techniques of extracted features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Computational Linguistics (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a kind of the exercise ability of lower limbs appraisal procedures based on improved convolutional neural networks, comprising the following steps: gait video image and skeleton joint position information during acquisition subject's gait；Depth data is generated to gait video image and binary conversion treatment is carried out by bilateral filtering method, to obtain gait contour images；Knee joint angle is calculated using space vector method；The gait contour feature of gait contour images is extracted using improved convolutional neural networks；Gait contour feature is connected into and is normalized with knee joint angle, then Feature Dimension Reduction is carried out to feature with core principle component analysis method, resettles the exercise ability of lower limbs evaluation index, the exercise ability of lower limbs assessment is carried out to subject.This method greatly reduces complexity, improves assessment accuracy using joined spatial pyramid pond layer in traditional convolutional neural networks and the improved convolutional neural networks of COCOB optimization algorithm automatically extract video image characteristic.
Description
Technical field
It is specifically a kind of based under improved convolutional neural networks the present invention relates to image procossing and deep learning field
Limb locomitivity appraisal procedure.
Background technique
With the continuous improvement of living condition and medical level, the health index of people is being improved, while the service life of people also gets over
It more grows, causes the endowment problem increasingly serious.With advancing age, the elderly can be degenerated due to physical function and disease causes
Apparent variation, movement speed decline occur for serious dyskinesia, gait feature, and joint angles scope of activities reduces, stands
With walking stability deficiency.So for handicapped the elderly, it can be by dressing some specific exoskeleton devices
Restore ability to act to a certain extent.But how system, perfect assessment state are carried out to the exercise ability of lower limbs of the elderly
It is inside and outside to also lack corresponding research.
Currently, most of to locomitivity assessment both at home and abroad is to use scale, such as the MDS (Minimum Data Set) in the U.S.
Scale, " elder's health and household care assessment " system of " Easy Care " scale of Britain and Hong-Kong etc., mostly all base
In " international function, deformity and health classification " (international classification of functioning,
Disability and health, ICF) standard evolution, improves assessment content according to national conditions and carries out grade classification.But
It is strong, complicated tediously long to assess content subjectivity, and evaluates ability and experience that accuracy rate relies primarily on doctor, there is no a kind of visitors
It sees, quantization, perfect appraisal procedure.
In terms of the exercise ability of lower limbs assessment, most commonly Gillette gait index (gillette gait index,
GGI), gait deviations index (gait deviation index, GDI) and gait overview score (gait profile
Score, GPS).GGI based on principal component analysis defines the distance between 16 discrete gait parameters and average value.GDI and
GPS using discrete variable, does not consider 15 motion sequences of entire gait cycle, and tends to provide more fully
Gait deviations measurement.The disadvantage is that measurement process is complicated, artificial feature of extracting is cumbersome, and it is big to calculate dimension.Traditional convolutional Neural net
The input of network is fixed picture size, and which limits the length and width of input picture and zoom scale.When encountering arbitrary dimension
Image when, be all first by image by cut or deformation be converted into fixed dimension.But the loss that will lead to information is cut, deformation
It will lead to the distortion of location information, just will affect the precision of identification.Therefore, currently without a kind of simple, objective, quantization, perfect
The exercise ability of lower limbs appraisal procedure.
Summary of the invention
In view of the deficiencies of the prior art, the technical issues of present invention intends to solve is to provide a kind of based on improved convolution mind
The exercise ability of lower limbs appraisal procedure through network.
The technical solution that the present invention solves the technical problem is to provide a kind of based under improved convolutional neural networks
Limb locomitivity appraisal procedure, it is characterised in that method includes the following steps:
Gait video image and skeleton joint position information during step 1, acquisition subject's gait；Subject
Including middle aged group, old group and without the young healthy group of any dyskinesia；Depth data is generated to gait video image
And binary conversion treatment is carried out by bilateral filtering method, to obtain gait contour images；By the hip joint of frame every in walking process,
The three dimensional space coordinate of the location information of knee joint and ankle-joint saves, and space vector method is recycled to calculate knee joint angle；
Step 2, the gait contour feature that gait contour images are extracted using improved convolutional neural networks；
Improved convolutional neural networks are the convolutional Neural nets that spatial pyramid pond layer and COCOB optimization algorithm is added
Network；Three convolutional layers, the convolutional neural networks of two pond layers the last one convolutional layer after a spatial pyramid is added
Pond layer, for adapting to various image sizes；Optimization is trained to gait contour images in combination with COCOB optimization algorithm to come
Extract the gait contour feature of gait contour images；
Step 2 is obtained gait contour feature and knee joint angle that step 1 obtains is connected into fusion feature square by step 3
Battle array simultaneously is normalized fusion feature matrix to obtain normalization matrix, then with core principle component analysis method to the spy of normalization matrix
Sign carries out Feature Dimension Reduction, the feature after obtaining dimensionality reduction；Establish the exercise ability of lower limbs evaluation index according to the feature after dimensionality reduction, to by
Examination person carries out the exercise ability of lower limbs assessment.
Compared with prior art, the beneficial effects of the invention are that:
(1) this method automatically extracts video image characteristic using improved convolutional neural networks, greatly reduces complexity,
Improve assessment accuracy.Improved convolutional neural networks are to joined spatial pyramid pond in traditional convolutional neural networks
Change layer and COCOB optimization algorithm.Spatial pyramid pond layer can eliminate traditional convolutional neural networks for picture size
The loss problem of image information, inputs the image of network-adaptive arbitrary dimension in limitation and cutting process.COCOB optimization is calculated
Method does not need setting learning rate, can save the process of regularized learning algorithm rate during random optimization, attempts optimal in every one-step prediction
The convergence of learning rate, the optimizer than tradition based on momentum is many fastly, realizes fast convergence, makes that accuracy rate is trained significantly to mention
It rises, keeps the gait feature extracted more accurate.
(2) this method is complicated for traditional gait Index Assessment method measurement process, and artificial feature of extracting is cumbersome, calculates
The big problem of dimension has comprehensively considered using improved convolutional neural networks autonomous learning gait image feature, has abandoned artificial
Extract the complexity and deficiency of feature.
(3) strong for the assessment content subjectivity of scale, complicated tediously long, and evaluate the ability that accuracy rate relies primarily on doctor
And experience, this method establish the exercise ability of lower limbs evaluation index, make assess enhanced convenience, using SPSS software to WGS with
GAS carries out Pearson correlation analysis, illustrates that the two has very strong linear relationship, illustrates GAS in the exercise ability of lower limbs
Assess the validity and objectivity of aspect.
(4) it selects Kinect sensor as gait data acquisition equipment, extracts the three-dimensional coordinate in lower limb skeletons joint, into
And use space vector method calculates acquisition knee joint angle, it not only can be to avoid complicated manikin extraction process, to clothing
Clothes, illumination, background environment do not have sensibility yet, and the gait feature vector data extracted are very accurate, alleviate tested
The discomfort of person's walking process.
(5) in view of factors such as method complexity and Edge preservations, bilateral filtering method is selected；Bilateral filtering method passes through lower two
The difference of the differences of a distance between pixels and pixel color calculates weight, can guarantee the details of profile well, reflection is true
Information is also convenient for subsequent convolutional neural networks training.
Detailed description of the invention
Fig. 1 is the experimental situation figure of the acquisition gait data of an embodiment of the present invention；
Fig. 2 is the structure chart of the improved convolutional neural networks of an embodiment of the present invention；
Fig. 3 is the structure chart of the spatial pyramid pond layer network of an embodiment of the present invention；
Fig. 4 is the knee joint angle curve graph of a gait cycle of an embodiment of the present invention；
Fig. 5 is the result bar chart of the exercise ability of lower limbs evaluation index of an embodiment of the present invention；
Fig. 6 is the Pearson correlation analysis figure of the GAS and WGS of an embodiment of the present invention.
Specific embodiment
Specific embodiments of the present invention are given below.Specific embodiment is only used for that present invention be described in more detail, unlimited
The protection scope of the claim of this application processed.
The present invention provides a kind of the exercise ability of lower limbs appraisal procedure (the abbreviation side based on improved convolutional neural networks
Method), it is characterised in that method includes the following steps:
Step 1 utilizes the gait video image and skeleton joint position during Kinect acquisition subject's gait
Information；
It is that can take human hip, knee joint and ankle-joint position that two Kinect, which are placed on apart from elevation of water,
The position of confidence breath (it is at 1.0m that the present embodiment, which is apart from elevation of water)；Kinect II is for obtaining human body during gait
(in the present embodiment, Kinect II is disposed on face subject direction of travel and apart from subject skeletal joint location information
Position at 4.5m)；Kinect I for obtain complete gait video image during gait (in the present embodiment, Kinect I
It is disposed on the position for being 3.5m with the vertical range of subject's walking path)；
System building finishes, and from the position that Kinect II can take to the walking of Kinect II, (the present embodiment is subject
As shown in figure 1 shown in arrow direction, run to from 0.5m straight from the position of II 4.5m of distance Kinect towards Kinect II), this
Kinect I obtains complete gait video image during gait in walking process, and Kinect II obtains human body during gait
Skeletal joint location information；Subject includes 20 20-39 years old young healthy group (young for not having any dyskinesia
Healthy group, YG), 10 40-59 years old middle aged group (middle-aged group, MG) and 10 60-70 years old old
Year group (elderly group, EG)；
Kinect I generates depth data to gait video image and carries out binary conversion treatment by bilateral filtering method, thus
Obtain gait contour images；Depth data totally 16, latter 3 be measure the visual field in human body index, i.e., when in the visual field someone and
Nobody when respectively 1 and 0 (the present embodiment be as someone in the visual field is 1, in the visual field nobody when be 0)；Therefore gait wheel is obtained
Wide image only need to indicate whether that nobody is 0 by judging latter 3 of depth data；By first 13 range information data conversions
At 8, the depth data of background and human body is set to 0 (for black) and 1 (for white) respectively, and (the present embodiment is by the depth of background
According to being set to 0,1) depth data of human body is set to degree；
II automatic capture of Kinect enters human body within the vision, and the hip joint of frame every in walking process, knee are closed
The three dimensional space coordinate of the location information of section and ankle-joint is saved and (can be stored in the list data structure of Python)；
The acquisition speed of Kinect II is 30Hz；Again location information is carried out eliminating shake and gaussian filtering process, specific method is:
Xn (2)=β Xn (1)+(1-β)(Xn-1 (2)+bn-1) (2)
bn=γ (Xn (2)-Xn-1 (1))+(1-γ)bn-1 (3)
Formula 1) -3) in: α is to inhibit noise coefficient；XnFor the artis information before filtering；Subscript (1) and subscript (2) are respectively
Represent two stages of filtering；Xn-1For the output of a upper filtering stage；β is to inhibit noise coefficient；bnFor biasing；bn-1It is previous
Filtering stage biasing；γ is control biasing coefficient；
Then knee joint angle θ is calculated using space vector method, to avoid the boundary condition of interspace analytic geometry: counting
Before calculating joint angles, Kinect coordinate system is mapped in mathematical coordinates system, the position of hip joint, knee joint and ankle-joint is passed through
The three dimensional space coordinate of confidence breath obtains hip joint, knee joint and the coordinate of ankle-joint mathematical coordinates system and will seek knee angle
Degree, which is converted to, calculates space vector angle；For hip joint H (Hx, Hy, Hz), knee joint K (Kx, Ky, Kz) and ankle-joint A (Ax,
Ay, Az), the formula for solving knee joint angle θ is as follows:
The measurement result is compared with the result of the Position and attitude sensor measurement in existing method, accuracy with higher；
Spatial pyramid pond layer and COCOB optimization are added using improved convolutional neural networks (PC-CNN) for step 2
The convolutional neural networks of algorithm extract the gait contour feature of gait contour images；
Currently, the input of convolutional neural networks (CNN) is fixed picture size, which limits the length of input picture
Wide and zoom scale；When encountering the image of arbitrary dimension, need that image is first converted into fixed dimension by cutting or deforming；But
The loss that will lead to information is cut, deformation will lead to the distortion of location information, influence the precision of identification；Spatial pyramid pond is added
Problem above can be eliminated by changing (SPP) layer；Spatial pyramid pond layer can be in the size and ratio for not considering input picture
In the case of, generate the feature vector of fixed size；In addition to this, it is mentioned since a characteristic pattern is carried out feature from different angles
It takes and polymerize again, it is shown that the robust property of algorithm, and improve accuracy of identification；Being trained using the image of various sizes can
To improve scaling invariance and reduce over-fitting；
The convolutional layer of convolutional neural networks receives the input of arbitrary size, so output is also all size, and convolution is refreshing
Classifier or full articulamentum through network need the input vector of fixed size；In order to make convolutional neural networks adapt to any ruler
Very little image input, in last of traditional convolutional neural networks (i.e. the network structures of three convolutional layers, two pond layers)
A spatial pyramid pond layer is added after a convolutional layer, obtains improved convolutional neural networks, it is big for adapting to various images
It is small；Stochastic gradient descent method (SGD) is replaced with COCOB optimization algorithm, gait contour images are carried out in conjunction with COCOB optimization algorithm
The gait contour features of gait contour images is extracted in training optimization, can save regularized learning algorithm rate size during random optimization
Process；
Regularized learning algorithm rate is still main bottleneck during random optimization.The purpose of learning rate is determined to low gradient side
To mobile step-length.From the perspective of learning rate, stake is given as security to direction and the gradient in gradient based on the optimizer of momentum
In size；If the prediction of optimizer is correctly, to be recompensed；If prediction be it is wrong, will be penalized and self repair
Just.
COCOB optimization algorithm, which is one, does not need the algorithm that setting learning rate is developed for stochastic gradient descent, it is every
One-step prediction variable optimal learning rate is able to achieve fast convergence based on the optimizer of momentum than tradition；COCOB optimization algorithm is random
Gradient descent method is reduced to stake on the process of one piece of coin front and back sides: initial capital ε, ε > 0；It can be any amount of gambling
Gold is given as security in front or back, but cannot be by any additional money；The stake of i-th wheel is defined as ωi, symbology is to gamble just
Face (+) or reverse side (-)；I-th wheel throws the result g of coini∈ { -1,1 }, 1 indicates front, and -1 indicates reverse side；If defeated, just
Lose stake；If won, just retain stake, while obtaining bonus Reward identical with stake；Triumph is advantageous in that money
The increase of gold, can be more for next iteration bet；Wealth is defined as wealth, bonus Reward is wealth and initial money
The difference of gold, then the Wealth and Reward such as formula (7) and formula (8) after t takes turns are shown:
COCOB optimization algorithm is to bet in next iteration:
ωt=βtWealtht-1 (9)
Formula 9) in, βtIndicate that the next next iteration bet of optimization accounts for the percentage of current wealth；
The specific structure of improved convolutional neural networks is with parameter: connecting entirely including 3 convolutional layers, 3 pond layers and 1
Connect layer (as shown in Figure 2)；Convolutional layer and pond layer are alternately distributed, and a spatial pyramid is added in the last one convolutional layer (C5) afterwards
Pond layer SPP6 is full articulamentum FC7 after the layer of spatial pyramid pond；
Successively sequence are as follows: C1 layers are convolutional layer, the filter for being 3 × 3 comprising 32 convolution kernel sizes, step-length 1, input
For gait contour images；S2 layers are pond layer, and maximum pond, step-length 2 are done in 2 × 2 window；C3 layers are convolutional layer, packet
The filter for being 3 × 3 containing 32 convolution kernel sizes, step-length 1；S4 layers are pond layer, and maximum pond is done in 2 × 2 window,
Step-length is 2；C5 layers are convolutional layer, the filter for being 3 × 3 comprising 64 convolution kernel sizes, step-length 1；SPP6 layers are space gold
The extraction of word tower basin layer, characteristic block includes 1 × 1,2 × 2 and 4 × 4 three kinds of scales, and each characteristic block is all made of maximum Chi Huafa
(maximum pond being done in 1 × 1,2 × 2 and 4 × 4 window, as shown in Figure 3), step-length 1；Spatial pyramid pond layer
Output is k × M dimensional vector, and M is the quantity of window, and k is the convolution nuclear volume of the last one convolutional layer；FC7 layers are Quan Lian
Connect layer, the input of full articulamentum is spatial pyramid pond layer as a result, the output node number of full articulamentum is set as 64, exports 1
A 64 dimensional vector；
Data are to be instructed using three kinds of methods to the gait contour feature of the gait contour images of 40 subjects in table 1
Practice, obtains the penalty values and accuracy rate of three kinds of methods.40 subjects include 20 20-39 years old for not having any dyskinesia
Young healthy group (young healthy group, YG), 10 40-59 years old middle aged group (middle-aged group,
) and 10 60-70 years old old group (elderly group, EG) MG.Using MG and EG as experimental group (test group), YG
(control group) as a control group.Three kinds of methods are convolutional neural networks (CNN), spatial pyramid pond layer are added
Convolutional neural networks (P-CNN) and the convolutional neural networks (PC- that spatial pyramid pond layer and COCOB optimization algorithm is added
CNN).PC-CNN method proposed by the present invention is better than other methods as can be seen from Table 1.
1 young people of table
Step 2 is obtained gait contour feature and is connected into the knee joint angle feature that step 1 obtains to merge spy by step 3
Sign matrix simultaneously is normalized fusion feature matrix to obtain normalization matrix, then with core principle component analysis (KPCA) method to normalizing
The feature for changing matrix carries out Feature Dimension Reduction, the feature after obtaining dimensionality reduction；
The PC-CNN gait contour feature extracted and knee joint angle θ feature are connected into fusion feature matrix M=[m1,
m2,...,mk]l×k, l is the dimension after gait contour feature and knee joint angle Fusion Features, and k is the number of experimental subjects；So
Fusion feature matrix is normalized afterwards to obtain normalization matrix MG, calculate the mean value and variance of each subject；
Formula 10) in, miFor the fusion feature of i-th of subject；μ represents mean value；σiRepresent the variance of i-th of subject；
Feature Dimension Reduction is carried out using feature of the KPCA method to normalization matrix again, the feature [c after obtaining dimensionality reduction1,c2,...,
ck]r×k, r indicate dimensionality reduction after characteristic dimension；Feature Dimension Reduction is particular by the definition threshold value E and variance contribution degree in KPCA method
VAF determines the characteristic dimension after dimensionality reduction, removes normalization matrix MGRedundancy, extract effective information and carry out feature drop
Dimension；
Establish the exercise ability of lower limbs evaluation index (gait ability score, GAS) according to the feature after dimensionality reduction, to by
Examination person carries out the exercise ability of lower limbs assessment, and assessment result >=100 then show that locomitivity is normal；Otherwise score is lower shows to transport
Dynamic obstacle is more serious；
Establishing the exercise ability of lower limbs evaluation index is specifically: the feature c after the dimensionality reduction of any subject ααWith young healthy
The average value of feature after group dimensionality reductionDeviation are as follows:
Formula 11) in, | | | | indicate norm；
Calculate gait deviations index:
To all subjects, it is quantitatively evaluated by formula (13):
Obtain the assessment result of all subjects；Formula 13) in,For young healthy group gait deviations
The average value of index；For the standard deviation of young healthy group gait deviations index；
Assessment result >=100 then show that subject motion's ability is normal；Otherwise score is lower shows that dyskinesia is tighter
Weight.
Go out three other GAS of group (as shown in Figure 5) using the feature calculation after dimensionality reduction.Sample T inspection has been done to any two groups
It tests, as shown in table 2.For YG-MG group, equal value difference is 22.362, and standard deviation (sd) value is 4.902, and 95% confidence interval is
(12.322,32.403) illustrate YG and two groups of the MG differences (P < 0.01) with highly significant.For MG-EG group, equal value difference is
27.391, standard deviation 4.096,95% confidence interval is (18.84,35.941), and it is very aobvious to illustrate that MG and two groups of EG have
The difference (P < 0.01) of work.For YG-EG group, equal value difference is 49.753, standard deviation 5.077, and 95% confidence interval is
(39.354,60.152) illustrate YG and two groups of the EG differences (P < 0.01) with highly significant.By analyzing above, it was demonstrated that GAS
The exercise ability of lower limbs can be quantified.
Table 2
Subject is assessed by winconsin gait scale (wisconsin gait scale, WGS), WGS is
The half active assessment method for being used to evaluate abnormal gait based on walking video, total score 14-45 points, score is higher, shows gait
It is abnormal more serious.For the consistency of verifying GAS and medical practitioner assessment, medical practitioner comments MG group and EG group by WGS
Estimate, utilizes SPSS (Statistical Product and Service Solutions, statistical product and service solution)
Software carries out Pearson correlation analysis (as shown in Figure 6) to WGS and GAS.Show GAS and medical practitioner proposed by the present invention
WGS between related coefficient be 0.916, both illustrate that there is very strong linear relationship, also illustrate proposed by the present invention
Validity and objectivity of the GAS in terms of the exercise ability of lower limbs assessment.
The present invention does not address place and is suitable for the prior art.
Claims (9)
1. a kind of the exercise ability of lower limbs appraisal procedure based on improved convolutional neural networks, it is characterised in that this method include with
Lower step:
Gait video image and skeleton joint position information during step 1, acquisition subject's gait；Subject includes
Middle aged group, old group and the young healthy group without any dyskinesia；Depth data is generated to gait video image and is led to
It crosses bilateral filtering method and carries out binary conversion treatment, to obtain gait contour images；The hip joint of frame every in walking process, knee are closed
The three dimensional space coordinate of the location information of section and ankle-joint saves, and space vector method is recycled to calculate knee joint angle；
Step 2, the gait contour feature that gait contour images are extracted using improved convolutional neural networks；
Improved convolutional neural networks are the convolutional neural networks that spatial pyramid pond layer and COCOB optimization algorithm is added；?
Three convolutional layers, two pond layers convolutional neural networks the last one convolutional layer after a spatial pyramid pond is added
Layer, for adapting to various image sizes；Optimization is trained to gait contour images in combination with COCOB optimization algorithm to extract
The gait contour feature of gait contour images；
Step 2 is obtained gait contour feature and knee joint angle that step 1 obtains is connected into fusion feature matrix simultaneously by step 3
Be normalized to obtain normalization matrix to fusion feature matrix, then with core principle component analysis method to the feature of normalization matrix into
Row Feature Dimension Reduction, the feature after obtaining dimensionality reduction；The exercise ability of lower limbs evaluation index is established according to the feature after dimensionality reduction, to subject
Carry out the exercise ability of lower limbs assessment.
2. the exercise ability of lower limbs appraisal procedure according to claim 1 based on improved convolutional neural networks, feature
It is in step 1, the gait video image and skeleton joint position letter during subject's gait is obtained using Kinect
Breath；It is that can take human hip, knee joint and ankle-joint location letter that two Kinect, which are placed on apart from elevation of water,
The position of breath；System building finishes, and subject walks from the position that Kinect II can take to Kinect II, this walking process
Middle Kinect I obtains the gait video image during gait, and Kinect II obtains skeleton joint position during gait
Information.
3. the exercise ability of lower limbs appraisal procedure according to claim 2 based on improved convolutional neural networks, feature
It is in step 1, Kinect I generates depth data to gait video image and carries out binary conversion treatment by bilateral filtering method,
To obtain gait contour images；Depth data totally 16, latter 3 be measure the visual field in human body index, as someone in the visual field
With when nobody respectively 1 and 0；Therefore obtaining gait contour images only need to indicate whether nothing by judging latter 3 of depth data
People；By first 13 range information data conversions at 8, the depth data of background and human body is set to 0 and 1 respectively.
4. the exercise ability of lower limbs appraisal procedure according to claim 2 based on improved convolutional neural networks, feature
It is in step 1, the location information obtained of Kinect II is carried out eliminating shake and gaussian filtering process, specific method are:
Xn (2)=β Xn (1)+(1-β)(Xn-1 (2)+bn-1) (2)
bn=γ (Xn (2)-Xn-1 (1))+(1-γ)bn-1 (3)
Formula 1) -3) in: α is to inhibit noise coefficient；XnFor the artis information before filtering；Subscript (1) and subscript (2) respectively represent
Two stages of filtering；Xn-1For the output of a upper filtering stage；β is to inhibit noise coefficient；bnFor biasing；bn-1For previous filtering
Stage biasing；γ is control biasing coefficient.
5. the exercise ability of lower limbs appraisal procedure according to claim 1 based on improved convolutional neural networks, feature
It is in step 1, is using the method that space vector method calculates knee joint angle θ: Kinect coordinate system is mapped to mathematics and is sat
In mark system, the three dimensional space coordinate of the location information of hip joint, knee joint and ankle-joint is converted to hip joint, knee joint and ankle
The coordinate of joint mathematical coordinates system and will seek knee joint angle be converted to calculate space vector angle；For hip joint H (Hx,
Hy, Hz), knee joint K (Kx, Ky, Kz) and ankle-joint A (Ax, Ay, Az), the formula for solving knee joint angle θ is as follows:
6. the exercise ability of lower limbs appraisal procedure according to claim 1 based on improved convolutional neural networks, feature
It is in step 2, COCOB optimization algorithm is that stochastic gradient descent method is reduced to stake on the process of one piece of coin front and back sides: just
Beginning fund is ε, ε > 0；Any amount of stake can be given as security in front or back, but cannot be by any additional money；The
The stake of i wheel is defined as ωi, symbology is gambling front or reverse side；I-th wheel throws the result g of coini∈ { -1,1 }, 1 indicates
Front, -1 indicates reverse side；If defeated, stake is just lost；If won, just retain stake, while obtaining identical with stake
Bonus Reward；Winning is advantageous in that the increase of fund, can be more for next iteration bet；Wealth is defined as wealth
Richness, bonus Reward are the difference of wealth and initial capital, then Wealth and Reward such as formula (7) and formula after t takes turns
(8) shown in:
COCOB optimization algorithm is to bet in next iteration:
ωt=βtWealtht-1 (9)
Formula 9) in, βtIndicate that the next next iteration bet of optimization accounts for the percentage of current wealth.
7. the exercise ability of lower limbs appraisal procedure according to claim 1 based on improved convolutional neural networks, feature
It is in step 2, the specific structure of improved convolutional neural networks is: including the full connection of 3 convolutional layers, 3 pond layers and 1
Layer；Convolutional layer and pond layer are alternately distributed, and a spatial pyramid pond layer SPP6, space are added after the last one convolutional layer C5
It is full articulamentum FC7 after the layer of pyramid pond；
Successively sequence are as follows: C1 layers are convolutional layer, include the filter that 32 convolution kernel sizes are 3 × 3, step-length 1 inputs as step
State contour images；S2 layers are pond layer, and maximum pond, step-length 2 are done in 2 × 2 window；C3 layers are convolutional layer, include 32
The filter that a convolution kernel size is 3 × 3, step-length 1；S4 layers are pond layer, and maximum pond, step-length are done in 2 × 2 window
It is 2；C5 layers are convolutional layer, the filter for being 3 × 3 comprising 64 convolution kernel sizes, step-length 1；SPP6 layers are spatial pyramid
Pond layer does maximum pond, step-length 1 in 1 × 1,2 × 2 and 4 × 4 window；The output of spatial pyramid pond layer is one
A k × M dimensional vector, M are the quantity of window, and k is the convolution nuclear volume of the last one convolutional layer；FC7 layers are full articulamentum, Quan Lian
The input for connecing layer is spatial pyramid pond layer as a result, the output node number of full articulamentum is set as 64, export 1 64 tie up to
Amount.
8. the exercise ability of lower limbs appraisal procedure according to claim 1 based on improved convolutional neural networks, feature
It is in step 3, gait contour feature and knee joint angle is connected into fusion feature matrix M=[m1,m2,...,mk]l×k, l
For gait contour feature and the fused dimension of knee joint angle, k is the number of experimental subjects；Then to fusion feature matrix into
Row normalization obtains normalization matrix MG, calculate the mean value and variance of each subject；
Formula 10) in, miFor the fusion feature of i-th of subject；μ represents mean value；σiRepresent the variance of i-th of subject；
Feature Dimension Reduction is carried out using feature of the core principle component analysis method to normalization matrix again, the feature [c after obtaining dimensionality reduction1,
c2,...,ck]r×k, r indicate dimensionality reduction after characteristic dimension；Feature Dimension Reduction is particular by the definition threshold in core principle component analysis method
Value E and variance contribution degree VAF determines the characteristic dimension after dimensionality reduction, removes normalization matrix MGRedundancy, extract effective
Information carries out Feature Dimension Reduction.
9. the exercise ability of lower limbs appraisal procedure according to claim 1 based on improved convolutional neural networks, feature
It is in step 3, establishing the exercise ability of lower limbs evaluation index is specifically: the feature c after the dimensionality reduction of any subject ααWith youth
The average value of feature after health group dimensionality reductionDeviation are as follows:
Formula 11) in, | | | | indicate norm；
Calculate gait deviations index:
To all subjects, it is quantitatively evaluated by formula (13):
Formula 13) in,For the average value of young healthy group gait deviations index；For year
The standard deviation of spry and light health group gait deviations index；
Obtain the assessment result of all subjects；Assessment result >=100 then show that subject motion's ability is normal；Score is got over
It is low to show that dyskinesia is more serious.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910337860.7A CN110046675A (en) | 2019-04-25 | 2019-04-25 | A kind of the exercise ability of lower limbs appraisal procedure based on improved convolutional neural networks |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910337860.7A CN110046675A (en) | 2019-04-25 | 2019-04-25 | A kind of the exercise ability of lower limbs appraisal procedure based on improved convolutional neural networks |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110046675A true CN110046675A (en) | 2019-07-23 |
Family
ID=67279321
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910337860.7A Pending CN110046675A (en) | 2019-04-25 | 2019-04-25 | A kind of the exercise ability of lower limbs appraisal procedure based on improved convolutional neural networks |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110046675A (en) |
Cited By (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110895818A (en) * | 2019-10-16 | 2020-03-20 | 南京大学 | Knee joint contour feature extraction method and device based on deep learning |
| CN111680608A (en) * | 2020-06-03 | 2020-09-18 | 长春博立电子科技有限公司 | Intelligent sports auxiliary training system and training method based on video analysis |
| CN112233084A (en) * | 2020-10-13 | 2021-01-15 | 深圳度影医疗科技有限公司 | Ultrasonic image processing method, ultrasonic image processing apparatus, and computer-readable storage medium |
| CN112401834A (en) * | 2020-10-19 | 2021-02-26 | 南方科技大学 | Movement-obstructing disease diagnosis device |
| CN112472531A (en) * | 2020-12-17 | 2021-03-12 | 大连理工大学 | Gait smoothing algorithm of lower limb exoskeleton robot for medical rehabilitation and assisted walking |
| CN112842261A (en) * | 2020-12-30 | 2021-05-28 | 西安交通大学 | Intelligent evaluation system for three-dimensional spontaneous movement of infant based on complex network |
| CN112906673A (en) * | 2021-04-09 | 2021-06-04 | 河北工业大学 | Lower limb movement intention prediction method based on attention mechanism |
| CN112998694A (en) * | 2021-02-03 | 2021-06-22 | 上海橙捷健康科技有限公司 | Method and system for measuring angle of human body lower limb joint |
| CN113537164A (en) * | 2021-09-15 | 2021-10-22 | 江西科技学院 | Real-time action time sequence positioning method |
| WO2021217542A1 (en) * | 2020-04-30 | 2021-11-04 | 深圳大学 | Gait identification method and device, and terminal and storage medium |
| CN111369626B (en) * | 2020-03-04 | 2023-05-16 | 刘东威 | Mark point-free upper limb movement analysis method and system based on deep learning |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106651830A (en) * | 2016-09-28 | 2017-05-10 | 华南理工大学 | Image quality test method based on parallel convolutional neural network |
| CN107633513A (en) * | 2017-09-18 | 2018-01-26 | 天津大学 | The measure of 3D rendering quality based on deep learning |
| CN108022248A (en) * | 2016-11-03 | 2018-05-11 | 北京航空航天大学 | A kind of lower limb gait rehabilitation assessment system of view-based access control model collecting device |
-
2019
- 2019-04-25 CN CN201910337860.7A patent/CN110046675A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106651830A (en) * | 2016-09-28 | 2017-05-10 | 华南理工大学 | Image quality test method based on parallel convolutional neural network |
| CN108022248A (en) * | 2016-11-03 | 2018-05-11 | 北京航空航天大学 | A kind of lower limb gait rehabilitation assessment system of view-based access control model collecting device |
| CN107633513A (en) * | 2017-09-18 | 2018-01-26 | 天津大学 | The measure of 3D rendering quality based on deep learning |
Non-Patent Citations (4)
| Title |
|---|
| FRANCESCO ORABONA ET AL.: "Training Deep Networks without Learning Rates Through Coin Betting", 《COMPUTER SCIENCE》 * |
| 张燕 等: "基于TF-CNN 与KECA 的下肢运动能力评价方法", 《仪器仪表学报》 * |
| 梅礼晔 等: "基于空间金字塔池化的深度卷积神经网络多聚焦图像融合", 《云南大学学报》 * |
| 赵玉婷 等: "卷积神经网络用于关节角度识别与姿势评估", 《计算机工程与应用》 * |
Cited By (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110895818A (en) * | 2019-10-16 | 2020-03-20 | 南京大学 | Knee joint contour feature extraction method and device based on deep learning |
| CN111369626B (en) * | 2020-03-04 | 2023-05-16 | 刘东威 | Mark point-free upper limb movement analysis method and system based on deep learning |
| WO2021217542A1 (en) * | 2020-04-30 | 2021-11-04 | 深圳大学 | Gait identification method and device, and terminal and storage medium |
| CN111680608A (en) * | 2020-06-03 | 2020-09-18 | 长春博立电子科技有限公司 | Intelligent sports auxiliary training system and training method based on video analysis |
| CN111680608B (en) * | 2020-06-03 | 2023-08-18 | 长春博立电子科技有限公司 | Intelligent sports auxiliary training system and training method based on video analysis |
| CN112233084A (en) * | 2020-10-13 | 2021-01-15 | 深圳度影医疗科技有限公司 | Ultrasonic image processing method, ultrasonic image processing apparatus, and computer-readable storage medium |
| CN112401834A (en) * | 2020-10-19 | 2021-02-26 | 南方科技大学 | Movement-obstructing disease diagnosis device |
| CN112472531A (en) * | 2020-12-17 | 2021-03-12 | 大连理工大学 | Gait smoothing algorithm of lower limb exoskeleton robot for medical rehabilitation and assisted walking |
| CN112842261B (en) * | 2020-12-30 | 2021-12-28 | 西安交通大学 | Intelligent evaluation system for three-dimensional spontaneous movement of infant based on complex network |
| CN112842261A (en) * | 2020-12-30 | 2021-05-28 | 西安交通大学 | Intelligent evaluation system for three-dimensional spontaneous movement of infant based on complex network |
| CN112998694A (en) * | 2021-02-03 | 2021-06-22 | 上海橙捷健康科技有限公司 | Method and system for measuring angle of human body lower limb joint |
| CN112998694B (en) * | 2021-02-03 | 2022-12-09 | 上海橙捷健康科技有限公司 | Method and system for measuring angles of joints of lower limbs of human body |
| CN112906673A (en) * | 2021-04-09 | 2021-06-04 | 河北工业大学 | Lower limb movement intention prediction method based on attention mechanism |
| CN113537164A (en) * | 2021-09-15 | 2021-10-22 | 江西科技学院 | Real-time action time sequence positioning method |
| CN113537164B (en) * | 2021-09-15 | 2021-12-07 | 江西科技学院 | Real-time action time sequence positioning method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110046675A (en) | A kind of the exercise ability of lower limbs appraisal procedure based on improved convolutional neural networks | |
| CN106384093B (en) | A kind of human motion recognition method based on noise reduction autocoder and particle filter | |
| CN107341786B (en) | The infrared and visible light image fusion method that wavelet transformation and joint sparse indicate | |
| CN101807245B (en) | Artificial neural network-based multi-source gait feature extraction and identification method | |
| CN105787439B (en) | A kind of depth image human synovial localization method based on convolutional neural networks | |
| Pollick et al. | Estimating the efficiency of recognizing gender and affect from biological motion | |
| CN109902558B (en) | CNN-LSTM-based human health deep learning prediction method | |
| CN109886986A (en) | A kind of skin lens image dividing method based on multiple-limb convolutional neural networks | |
| CN106295555A (en) | A kind of detection method of vital fingerprint image | |
| CN109730637A (en) | A kind of face face-image quantified system analysis and method | |
| Datcu et al. | Noncontact automatic heart rate analysis in visible spectrum by specific face regions | |
| CN107330249A (en) | A kind of Parkinsonian symptoms area of computer aided method of discrimination based on KINECT skeleton datas | |
| CN109635618A (en) | Visible images vein developing method based on convolutional neural networks | |
| CN110532850B (en) | Fall detection method based on video joint points and hybrid classifier | |
| CN109544518A (en) | A kind of method and its system applied to the assessment of skeletal maturation degree | |
| CN106203256A (en) | A kind of low resolution face identification method based on sparse holding canonical correlation analysis | |
| CN111728590A (en) | Individual cognitive ability prediction method and system based on dynamic function connection | |
| CN109766838A (en) | A kind of gait cycle detecting method based on convolutional neural networks | |
| CN115024706A (en) | Non-contact heart rate measurement method integrating ConvLSTM and CBAM attention mechanism | |
| CN109063643A (en) | A kind of facial expression pain degree recognition methods under the hidden conditional for facial information part | |
| CN107967941A (en) | A kind of unmanned plane health monitoring method and system based on intelligent vision reconstruct | |
| CN110427987A (en) | A kind of the plantar pressure characteristic recognition method and system of arthritic | |
| Gray et al. | A perceptron reveals the face of sex | |
| CN115346272A (en) | Real-time tumble detection method based on depth image sequence | |
| CN113768471A (en) | Parkinson disease auxiliary diagnosis system based on gait analysis |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190723 |