CN109271918B - Method for distinguishing people with balance ability disorder based on gravity center shift model - Google Patents

Method for distinguishing people with balance ability disorder based on gravity center shift model Download PDF

Info

Publication number
CN109271918B
CN109271918B CN201811052076.3A CN201811052076A CN109271918B CN 109271918 B CN109271918 B CN 109271918B CN 201811052076 A CN201811052076 A CN 201811052076A CN 109271918 B CN109271918 B CN 109271918B
Authority
CN
China
Prior art keywords
gravity
image
human body
center
barycentric
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201811052076.3A
Other languages
Chinese (zh)
Other versions
CN109271918A (en
Inventor
金海燕
肖聪
肖照林
蔡磊
李秀秀
石俊飞
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Xian University of Technology
Original Assignee
Xian University of Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Xian University of Technology filed Critical Xian University of Technology
Priority to CN201811052076.3A priority Critical patent/CN109271918B/en
Publication of CN109271918A publication Critical patent/CN109271918A/en
Application granted granted Critical
Publication of CN109271918B publication Critical patent/CN109271918B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/20—Movements or behaviour, e.g. gesture recognition
    • G06V40/23—Recognition of whole body movements, e.g. for sport training
    • G06V40/25—Recognition of walking or running movements, e.g. gait recognition
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G06F18/2411—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on the proximity to a decision surface, e.g. support vector machines

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Data Mining & Analysis (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Evolutionary Computation (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Artificial Intelligence (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Evolutionary Biology (AREA)
  • General Health & Medical Sciences (AREA)
  • Psychiatry (AREA)
  • Social Psychology (AREA)
  • Human Computer Interaction (AREA)
  • Multimedia (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)

Abstract

The invention discloses a method for distinguishing people with balance ability disorder based on a gravity center shift model, which comprises the following steps: firstly, acquiring human body walking posture videos of normal people and abnormal people in a built virtual reality scene from 45 degrees; then, loading the collected videos of the normal person and the abnormal person, extracting the video images into pictures, and then processing the pictures to respectively obtain the gravity center coordinates of the normal person and the abnormal person; and finally, according to the obtained gravity center data, extracting gravity center included angle data and the mean value and variance of the upper and lower gravity centers of the human body, classifying the extracted gravity center data through an SVM classifier, and then quickly judging the crowd with the balance ability obstacle by combining the gravity center included angle data and the mean value and variance of the upper and lower gravity centers of the human body. The method disclosed by the invention solves the problems that the traditional subjective method is too rough, and the scale evaluation method is too complex and too costly, and the accuracy of final classification is over 85% by processing the video and the image through multiple steps.

Description

Method for distinguishing people with balance ability disorder based on gravity center shift model
Technical Field
The invention belongs to the technical field of computer digital image processing, and relates to a method for distinguishing people with balance ability disorder based on a gravity center shift model.
Background
The human body balance ability refers to the ability of the human body to maintain self stability and resist balance damage, including the ability to maintain a certain posture or the ability to regulate and control the body to maintain balance when being subjected to external force, and is one of the important physiological functions of the human body. The main factors influencing the balance ability include the factors of support area, height of center of gravity, weight and the like, and are also influenced by the factors of vision, body organs, reaction of a sensing system and the like. Has good balance capability, is beneficial to improving the functions of motor organs and vestibular organs, and improves the regulating function of the central nervous system to muscle tissues and internal organs, thereby ensuring the smooth proceeding of physical activities and improving the capability of adapting to complex environment and self-protection capability.
At present, the traditional subjective human balance ability detection methods mainly comprise a Romberg test method, an enhanced Romberg test method and a single-leg upright test method (OLST). Although the traditional subjective observation method is simple to operate, the traditional subjective observation method is too rough and subjective, lacks objectivity and unified standards, cannot clearly and intuitively judge the degree of balance disorder, and can only be used for clinically performing preliminary tests on patients with suspected balance disorder. Other methods such as scale evaluation methods, including Berg balance scale, Tinetti gait and balance scale, activity balance confidence scale, Brunel balance scale, etc., require complex equipment support, and a large number of patient tests, and obtain corresponding activity data through different posture activities such as continuous unsupported standing and sitting, standing-to-sitting movement, bed-chair transfer, standing-up to take articles from the ground, etc., to judge the balance ability of the human body. Compared with the traditional subjective detection, the method improves the reliability, but the detection method is too complicated, the realization cost is too high, and the method is not beneficial to being implemented in actual situations.
For the improvement of the prior art, the detection of the human body balance ability does not need to be carried out by using the traditional subjective observation method or scale evaluation method. Therefore, the VR system based on virtual reality is greatly utilized, relates to technologies such as computer graphics, man-machine interaction technology, sensing technology and artificial intelligence, and is expected to produce great economic and social benefits. The computer is utilized to generate vivid three-dimensional visual, auditory, olfactory and other senses, so that the participants naturally experience and interact with the virtual world, and the feeling of being personally on the scene is generated through accurate 3D world images. Different virtual scenes are simulated by the VR system, so that the participants can react and interact according to the corresponding scenes, and meanwhile, the computer can quickly judge the quality of the human body balance capacity according to data obtained by the reaction and a scientific basis and data measurement and calculation method. Has higher accuracy and reliability, and higher efficiency.
Disclosure of Invention
The invention aims to provide a method for distinguishing people with balance ability disorder based on a gravity center shift model, and solves the problems that a traditional subjective method is too rough, and a scale evaluation method is too complex and too costly.
The invention adopts the technical scheme that the method for distinguishing the crowd with balance ability disorder based on the gravity center shift model comprises the following specific operation steps:
step 1, a lens of a camera is right opposite to the position between the front side and the side surface of a human body, namely, human body walking posture videos of normal people and abnormal people in a built virtual reality scene are collected from 45 degrees;
step 2, loading the two collected human body walking posture videos, respectively extracting video images into pictures, and then processing the pictures to respectively obtain the gravity center coordinates of the two types of human bodies;
and 3, extracting included angle of gravity data and mean square deviation of upper and lower centers of gravity of the human body according to the obtained coordinate data of the center of gravity, classifying the extracted included angle of gravity data through a Support Vector Machine (SVM) classifier, and judging the crowd with balance disorder by combining the included angle of gravity data and the mean square deviation of the upper and lower centers of gravity of the human body.
Yet another feature of the present invention is that,
the operation process of the step 2 is as follows:
step 2.1, reading the two collected human body posture videos by using a cvLoadImage function, setting the starting time and the ending time of the read-in videos, resetting the starting time and the ending time of the videos if the capture function does not read the videos containing the portrait, and carrying out the next step if the capture function reads the videos containing the portrait;
step 2.2, frames of the two groups of extracted videos are respectively removed through a CvCapture function in OpenCV, firstly, blank scenes in the two groups of videos are respectively extracted for picture storage, and then, one frame of the shot human body posture video is taken every two seconds and stored as a picture;
2.3, carrying out differential processing on the two groups of stored human body posture pictures and the blank scene respectively to obtain pictures only with human body postures;
step 2.4, carrying out image denoising on the image after the difference;
step 2.5, further carrying out image corrosion on the denoised picture to finally obtain a black and white image only with a human body image;
step 2.6, performing edge extraction on the image obtained after corrosion, processing the corroded image by using a Canny edge detection operator to obtain a connected region of the image, and performing convolution on the image by using a Gaussian filter to reduce the obvious noise influence on an edge detector; then, calculating the gradient strength and direction of each pixel point in the image, and applying non-maximum value inhibition to eliminate stray response caused by edge detection; finally, determining real and potential edges by using double-threshold detection, finally finishing edge detection by inhibiting isolated weak edges, and extracting a human body contour image by using findContours;
and 2.7, calculating the moment of the human body contour image, and calculating the barycentric coordinate of the human body through the moment of the contour image.
The specific process of the differential processing in step 2.3 is as follows:
firstly, carrying out binarization processing on two groups of stored human body posture pictures and blank scene pictures to enable all images to be black and white images;
then, carrying out difference processing on the two groups of binarized human body posture photos and blank scene pictures, and setting the image containing the human body extracted at the kth moment as xkThe image of the blank scene is xjAnd differentiating the two images to obtain a differential image delta xk：Δxk＝xk-xj。
The image denoising process of the step 2.4 is as follows: and (2) replacing the pixel value of one point in the differential processing image by using a median filtering method to obtain a denoised image by using the pixel median of each point in the neighborhood of the point, wherein the specific process is that f (x, y) and g (x, y) are respectively the image subjected to differential processing and the image subjected to denoising, and the median filtering output is g (x, y) ═ med { f (x-k, y-l), (k, l ∈ W) }, wherein W is a two-dimensional template, and k and l are respectively rows and columns of the image.
The specific operation steps of step 2.5 are as follows: and corroding the image subjected to difference and denoising, defining the size of a corrosion window by a getStructuringElement function, selecting a rectangular window MORPH-RECT, selecting the size of a corrosion kernel by 3 multiplied by 3, and then performing corrosion operation by using an erode function through the corrosion window MORPH-RECT to obtain the image only containing the portrait.
The calculation method of the body weight center coordinates in step 2.7 is as follows:
firstly, calculating the moment of a human body contour image, taking a human body in a picture as a planar object, taking a pixel value of each point as the density of the point, taking an expected value of the point as the moment of the point, and calculating the barycentric coordinate of the human body by adopting the first moment of the image, as shown in formulas 1-3:
Figure BDA0001794829990000041
Figure BDA0001794829990000042
the coordinates of the center of gravity of the human body image are:
Figure BDA0001794829990000051
wherein, V (i, j) represents the gray value of the human body image at the point (i, j), when the image is a binary image and V (i, j) only has black and white, namely two values of 0 and 1, then M is00Expressed as the sum of white areas in the human body image, i.e. the area of the binary image, M10Representing the accumulation of the horizontal coordinate values of the white area in the image; in the same way, M01Representing the accumulation of ordinate values of white areas in the image, xcRepresents the abscissa of the center of gravity, ycRepresenting the ordinate of the center of gravity.
The operation steps of step 3 are as follows:
step 3.1, classifying the obtained gravity center seat as input data through an SVM classifier to obtain gravity center coordinate data of normal people and abnormal people with marks;
step 3.2, respectively extracting the gravity center included angle of the normal person and the abnormal person;
step 3.3, calculating the mean square error of the gravity centers of the normal person and the abnormal person respectively;
and 3.4, combining the steps 3.1-3.3 to obtain the barycentric coordinates, barycentric included angles and barycentric mean square deviations of the human body, and distinguishing the crowd with balance ability disorder.
The process of classification by the SVM classifier in step 3.1 is as follows:
firstly, loading a training data set and a testing data set, wherein the training data set comprises training data, training labels, testing data and testing labels, namely human body barycentric coordinate data and correct labels, dividing the training data and the testing data into two parts, obtaining the optimal parameters of current data through an SVMcgForRegress parameter optimizing function, and obtaining a trained model through the optimized parameters and training data through an svmtrain function; and finally, testing by using an svmpredict function to obtain barycentric coordinate data with a mark of '1' or '-1'.
The specific calculation process of step 3.2 is as follows:
by calculating the three barycentric coordinates of the upper, middle and lower parts of the human body, using the atan2 function and converting the barycentric coordinates into angles, the included angle of the upper, middle and lower barycentric coordinates can be calculated: let the coordinate of the upper center of gravity P1 be (x)1,y1) Center of gravity coordinates P2 (x)2，y2) The lower center of gravity P3 is represented by the coordinate (x)3，y3) When the angle of gravity center is theta, the radian of the angle between P1P2 and the positive direction of the x-axis is atan2 (y)2-y1,x2-x1) Angle of inclination theta1Is atan2 (y)2-y1,x2-x1) 180/pi, and the same principle, the angle theta between P3P2 and the positive direction of the x axis2Is atan2 (y)3-y2,x3-x2) 180/pi, and the angle of gravity theta is represented as theta1+θ2As formula 7, the included angle of the center of gravity of the normal person and the abnormal person is calculated respectively:
Figure BDA0001794829990000061
wherein n is the number of the extracted included angles of the center of gravity, p1.y、p2.y、p3.yIs the ordinate of three gravity points, p1.x、p2.x、p3.xThe abscissa of the three gravity points.
The specific process of step 3.3 is as follows:
extracting one frame of picture every two seconds from the human body walking video, wherein the 20 frames of pictures are total, and calculating the mean value and the variance of the vertical coordinates of the gravity centers of the upper half body and the lower half body extracted after calculation by using a formula 8:
Figure BDA0001794829990000062
wherein n is the number of experimental objects, and t is the element [1, n ]]，
Figure BDA0001794829990000063
Represents the values of the upper and lower body barycentric ordinates at time t,
Figure BDA0001794829990000064
representing the mean value of the ordinate of the center of gravity, and CGS is the mean square error of the center of gravity;
step 3.4 the method for distinguishing people with balance ability disorder comprises the following steps:
judging according to the mark on the gravity center data, if the output balance capability is marked as '1', distinguishing as a normal person, and showing that the balance capability is good; if the output balance ability is marked as "-1", the person is distinguished as an abnormal person, which indicates that the balance ability is obstructed; the included angle value of the gravity center is relatively large, the balance capability is good, the included angle value of the gravity center is relatively small, and the balance capability is obstructed; the relatively large variance value of the center of gravity indicates that the larger and more unstable the fluctuation of the center of gravity of the human body, the worse the balance ability.
The method has the beneficial effects that the method for distinguishing the crowd with balance ability disorder based on the gravity center shift model solves the problems that the traditional subjective method is too rough, and the scale evaluation method is too complex and too costly. The method has the advantages that the gravity center data are obtained by processing the human body walking video, people with balancing obstacle ability are distinguished through the gravity center offset model without any balance measuring instrument, the balancing ability of the people or other people is objectively judged, the video and the image are processed through multiple steps, and the accuracy rate of final classification is guaranteed to be more than 85%.
Drawings
FIG. 1 is a flow chart of the operation of the method of the present invention for distinguishing persons with balance impairment based on a center of gravity shift model;
FIG. 2 is an overall flow chart of the method of the present invention for distinguishing persons with balance impairment based on a center of gravity shift model;
FIG. 3 is a flow chart of balance ability determination and analysis of the method for distinguishing persons with balance ability disorders based on a center of gravity shift model according to the present invention;
FIG. 4 is an abnormal human frontal pose model;
FIG. 5 is an abnormal human lateral pose model;
FIG. 6 is a normal human frontal pose model;
FIG. 7 is a normal human lateral pose model;
FIG. 8 is a diagram of the difference result of walking images of a human body, the left side is a diagram of the difference result of walking images of a normal person, and the right side is a diagram of the difference result of walking images of an abnormal person;
FIG. 9 is a graph of the difference image erosion denoising result, with the left side being a normal person and the right side being an abnormal person;
FIG. 10 is a barycentric coordinate extraction diagram with a normal person on the left and an abnormal person on the right;
FIG. 11 is a chart of centroid angle analysis;
FIG. 12 is a diagram of upper body weight and mind ANOVA;
fig. 13 is a lower body center of gravity variance analysis diagram.
Detailed Description
The present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.
The method for distinguishing people with balance ability disorder based on the center-of-gravity shift model, disclosed by the invention, is shown in fig. 1 and fig. 2, and comprises the following specific operation steps:
when the camera lens is opposite to the front of the human body, the collection angle is defined to be 0 degrees, when the camera lens is opposite to the side face of the human body, the collection angle is defined to be 90 degrees, the camera lens is selected to be opposite to the position between the front and the side face of the human body, and the collection angle at the moment is defined to be 45 degrees.
Step 1, collecting human body walking posture videos of normal people and abnormal people in a built virtual reality scene from 45 degrees;
step 2, loading the two collected human body walking posture videos, respectively extracting video images into pictures, and then processing the pictures to respectively obtain the gravity center coordinates of the two types of human bodies;
and 3, extracting barycentric included angle data and mean square deviations of upper and lower barycentrics of the human body according to the obtained barycentric coordinate data, classifying the extracted barycentric data through an SVM classifier, and judging the crowd with balance ability obstacle by combining the barycentric included angle data and the mean square deviations of the upper and lower barycentrics of the human body.
The operation process of the step 2 is as follows:
step 2.1, reading posture videos of normal people and abnormal people by using a cvLoadImage function, setting the starting time and the ending time of the read-in video, resetting the starting time and the ending time of the video if the capture function does not read the video containing the portrait, and carrying out the next step if the capture function reads the video containing the portrait;
step 2.2, frames of the two groups of extracted videos are respectively removed through a CvCapture function in OpenCV, firstly, blank scenes in the two groups of videos are respectively extracted for picture storage, and then, one frame of the shot human body posture video is taken every two seconds and stored as a picture;
2.3, carrying out differential processing on the two groups of stored human body posture pictures and the blank scene respectively to obtain pictures only with human body postures;
step 2.4, carrying out image denoising on the image after the difference;
step 2.5, further carrying out image corrosion on the denoised picture to finally obtain a black and white image only with a human body image;
step 2.6, performing edge extraction on the image obtained after corrosion, processing the corroded image by using a Canny edge detection operator to obtain a connected region of the image, and performing convolution on the image by using a Gaussian filter to reduce the obvious noise influence on an edge detector; then, calculating the gradient strength and direction of each pixel point in the image, and applying non-maximum value inhibition to eliminate stray response caused by edge detection; finally, determining real and potential edges by using double-threshold detection, finally finishing edge detection by inhibiting isolated weak edges, and extracting a human body contour image by using findContours;
and 2.7, calculating the moment of the human body contour image, and calculating the barycentric coordinate of the human body through the moment of the contour image.
The specific process of the differential processing in step 2.3 is as follows:
firstly, carrying out binarization processing on two groups of stored human body posture pictures and blank scene pictures to enable all images to be black and white images;
then, carrying out difference processing on the two groups of binarized human body posture photos and blank scene pictures, and setting the image containing the human body extracted at the kth moment as xkThe image of the blank scene is xjAnd differentiating the two images to obtain a differential image delta xk：
Δxk＝xk-xj
The image denoising process of the step 2.4 is as follows: and (2) replacing the pixel value of one point in the differential processing image by using a median filtering method to obtain a denoised image by using the pixel median of each point in the neighborhood of the point, wherein the specific process is that f (x, y) and g (x, y) are respectively the image subjected to differential processing and the image subjected to denoising, and the median filtering output is g (x, y) ═ med { f (x-k, y-l), (k, l ∈ W) }, wherein W is a two-dimensional template, and k and l are respectively rows and columns of the image.
The specific operation steps of step 2.5 are as follows: and corroding the image subjected to difference and denoising, defining the size of a corrosion window through a getStructuringElement function, selecting a rectangular window MORPH-RECT, selecting the size of a corrosion kernel by 3 multiplied by 3, and performing corrosion operation through the corrosion window by using an anode function through the corrosion window MORPH-RECT to obtain the image only containing the portrait.
The calculation method of the body weight center coordinates in step 2.7 is as follows:
firstly, calculating the moment of a human body contour image, taking a human body in a video picture as a planar object, taking a pixel value of each point as the density of the point, taking an expected value of the point as the moment of the point, and calculating the barycentric coordinate of the human body by adopting the first moment of the image, as shown in formula 1-3:
Figure BDA0001794829990000101
Figure BDA0001794829990000102
the coordinates of the center of gravity of the human body image are:
Figure BDA0001794829990000103
v (i, j) represents the gray value of the human body image at the point (i, j), and when the image is a binary image and the V (i, j) only has black and white, namely two values of 0 and 1, M is obtained00Expressed as the sum of white areas in the human body image, i.e. the area of the binary image, M10Representing the accumulation of the horizontal coordinate values of the white area in the image; in the same way, M01Representing the accumulation of ordinate values of white areas in the image, xcRepresents the abscissa of the center of gravity, ycRepresenting the ordinate of the center of gravity.
The operation steps of step 3 are as follows:
step 3.1, classifying the acquired barycentric coordinates as input data by an SVM classifier to obtain barycentric coordinate data of normal persons and abnormal persons with marks;
step 3.2, respectively extracting the gravity center included angle of the normal person and the abnormal person;
step 3.3, calculating the mean square error of the gravity centers of the upper and lower half bodies of the normal person and the abnormal person respectively;
and 3.4, combining the steps 3.1-3.3 to obtain the barycentric coordinates, barycentric included angles, barycentric mean values and variances of the human body, and classifying the pictures of normal people and abnormal people.
The process of classification by the SVM classifier in step 3.1 is as follows:
firstly, loading a training data set and a testing data set, wherein the training data set comprises training data, training labels, testing data and testing labels, namely, barycentric coordinate data and correct labels of a human body, dividing the training data set into two parts, namely training data and testing data, acquiring optimal parameters of current data through an SVMcgForRegress parameter optimizing function, and acquiring trained model through svmtrain function training data by using the acquired optimized parameters; and finally, testing by using an svmpredict function to obtain barycentric coordinate data with a mark of '1' or '-1'.
The basic principle of the SVM classifier is as follows: the support vector machine is a supervised learning method and is widely applied to statistical classification and regression analysis. In the experiment, the barycentric coordinates of the human body are mapped to a high-dimensional space, and a hyperplane with the maximum interval is searched for in the high-dimensional space
Figure BDA0001794829990000111
The coordinates x of the center of gravity of the human body of the experimental groupiAnd the coordinates x of the center of gravity of the human body in the contrast groupiSeparated from each other. Where W represents the hyperplane normal separating the characteristic vectors of the barycentric coordinates of the experimental group from the barycentric coordinates of the control group, and γ is the displacement interval added for the flexibility of the method. Make the characteristic vector of the barycentric coordinate of the human body in the contrast group satisfy
Figure BDA0001794829990000112
Characteristic vector satisfaction of human body barycentric coordinates of experimental group
Figure BDA0001794829990000113
When there are W and γ satisfying such a condition, such a feature vector is said to be divisible. In practical problems, it may not be possible to completely separate all feature vectors, and the best hyperplane is selected as the optimal solution. 1/omega is
Figure BDA0001794829990000114
And
Figure BDA0001794829990000115
finding the optimal hyperplane translates to minimizing ω. The mathematical expression is
Figure BDA0001794829990000121
Constraint of yi[wxi+γ]1, i ═ 1, 2. By utilizing the mature theory of convex quadratic programming, when the characteristic vector of the barycentric coordinate of the human body is not completely separable, a corresponding kernel function K can be selected, and a characteristic space formed by the barycentric coordinate of the input human body is implicitly mapped to a high-dimensional space, wherein the barycentric coordinate of the input human body is linearly separable in the high-dimensional space. According to the dual condition, the optimization problem can be converted into a corresponding dual problem to be solved.
The specific calculation process of step 3.2 is as follows:
by calculating the three barycentric coordinates of the upper, middle and lower parts of the human body, using the atan2 function and converting the barycentric coordinates into angles, the included angle of the upper, middle and lower barycentric coordinates can be calculated: let the coordinate of the upper center of gravity P1 be (x)1,y1) Center of gravity coordinates P2 (x)2，y2) The lower center of gravity P3 is represented by the coordinate (x)3，y3) When the angle of gravity center is theta, the radian of the angle between P1P2 and the positive direction of the x-axis is atan2 (y)2-y1,x2-x1) Angle of inclination theta1Is atan2 (y)2-y1,x2-x1) 180/pi, and the same principle, the angle theta between P3P2 and the positive direction of the x axis2Is atan2 (y)3-y2,x3-x2) 180/pi, and the angle of gravity theta is represented as theta1+θ2As formula 7, the included angle of the center of gravity of the normal person and the abnormal person is calculated respectively:
Figure BDA0001794829990000122
wherein n is the number of the extracted included angles of the center of gravity, p1.y、p2.y、p3.yIs the ordinate of three gravity points, p1.x、p2.x、p3.xThe abscissa of the three gravity points.
The specific calculation process of step 3.3 is as follows:
one frame of picture is extracted every two seconds from the human body walking video, 20 frames of pictures are totally extracted, and mean square deviations of the vertical coordinates of the gravity centers of the upper half body and the lower half body extracted after calculation are respectively calculated by using a formula 8:
Figure BDA0001794829990000123
wherein n is the number of pictures, t is the [1, n ]]，
Figure BDA0001794829990000124
Represents the values of the upper and lower body barycentric ordinates at time t,
Figure BDA0001794829990000131
representing the mean value of the ordinate of the center of gravity, and CGS is the mean square error of the center of gravity;
step 3.4 the method for distinguishing people with balance ability disorder comprises the following steps:
as shown in fig. 3, the judgment is performed according to the output mark of the barycentric coordinate data, and after classification is performed by the SVM classifier, if the output balance capability is marked as "1", the classification is performed to be a normal person, which indicates that the balance capability is good; if the output balance ability is marked as "-1", the person is distinguished as an abnormal person, which indicates that the balance ability is obstructed;
the judgment is carried out according to the included angle of the gravity center, which is the posture control capability reflecting proprioception and refers to the size of the included angle between the upper, middle and lower gravity center points of the human body in the motion process. The relatively large included angle indicates that the balance capability is better; the smaller the included angle is, the weaker the human posture control capability is, the worse the balance capability is;
the judgment is carried out according to the gravity center variance, the gravity center variance reflects the discrete degree of variables in the group, and the larger the variance is, the larger and the more unstable the fluctuation of the gravity center of the human body is, and the poorer the balance capability is.
The invention designs a human body balance posture model based on the gravity center according to the shaking of the human body, as shown in figures 4-7, wherein P is1、P2、P3If the balance ability of a normal person is good, the included angle theta of the three gravity centers is larger, and the distance L of the gravity center to the middle axis is smaller. When the included angle of the center of gravity of a person is large, and the distance from the center of gravity to the central axis is small, the posture of the person tends to be a line, and the person is judged to be a normal person and has good balance capability. Factors influencing the balance of the human body are the included angle of the center of gravity, the mean value of the upper and lower centers of gravity and the variance of the upper and lower centers of gravity.
The included angle of the gravity centers of normal people in the virtual environment for balance training is larger than that of abnormal people, and the included angle of the gravity centers of normal people in the virtual environment for training is also higher than that of abnormal people. Meanwhile, due to the weak balance capability, the gravity center fluctuation of the training of the special person in the virtual environment is larger than that of a normal person, and the body shaking is obviously better than that of the normal person. Therefore, the mean value of the center of gravity of the upper and lower half bodies of a normal person is higher than that of an abnormal person in the training process, and the variance of the center of gravity of the upper and lower half bodies is generally smaller than that of a special person. Therefore, various decisions are integrated to measure the balance ability of the human body.
The specific implementation mode is as follows:
the implementation process of the method for classifying people with balance disorder based on the center-of-gravity shift model is described below by extracting the center of gravity and the included angle from a group of human walking videos.
TABLE 1 two-class test data and tags
Figure BDA0001794829990000141
Table 2 SVM test output label
Figure BDA0001794829990000142
Table 35 centre of gravity angle for normal person video analysis
Object A Object B Object C Object D Object E
Angle
1 165.193 143.47 168.943 163.919 159.417
Angle 2 117.495 168.986 144.627 148.402 149.246
Angle 3 127.619 164.707 148.088 157.817 167.652
Angle 4 164.163 166.13 133.002 124.572 134.112
Angle 5 124.437 164.973 148.754 163.538 146.965
Angle 6 120.294 142.347 156.048 142.739 129.503
Angle 7 130.256 140.419 132.031 139.283 145.452
Angle 8 121.957 122.756 143.362 119.525 143.944
Angle 9 122.95 167.096 160.831 135.912 142.712
Angle 10 118.602 157.364 123.537 157.193 159.853
Angle 11 126.105 177.533 132.831 146.615 135.736
Angle 12 159.543 152.654 167.266 152.064 151.242
Angle 13 154.525 146.652 132.284 163.148 148.865
Angle 14 125.446 128.54 128.439 141.804 137.409
Angle 15 122.481 118.704 142.858 148.155 132.536
Angle 16 119.33 153.988 131.764 129.525 146.693
Angle 17 131.288 131.373 131.764 142.731 139.557
Angle 18 118.496 152.749 156.048 155.912 153.752
Angle 19 116.124 119.282 129.324 149.381 171.355
Angle 20 125.705 120.03 135.617 135.485 128.542
TABLE 45 Special person video analysis barycenter angle
Object A Object B Object C Object D Object E
Angle
1 81.6148 111.057 116.515 118.419 121.318
Angle 2 109.372 112.516 113.043 105.832 128.467
Angle 3 120.73 109.153 114.948 127.089 112.709
Angle 4 100.799 107.415 104.563 105.07 102.759
Angle 5 109.668 108.88 101.023 124.502 106.706
Angle 6 72.7893 105.285 118.588 107.603 125.628
Angle 7 114.999 101.03 108.091 122.447 97.4247
Angle 8 104.504 104.219 112.477 106.48 115.254
Angle 9 112.66 109.335 95.686 106.073 124.882
Angle 10 133.608 123.346 105.898 114.945 116.532
Angle 11 122.385 118.258 107.626 103.139 128.328
Angle 12 103.523 119.35 106.692 123.779 113.849
Angle 13 86.4591 119.693 112.627 123.904 98.7091
Angle 14 131.69 104.884 120.48 126.066 122.961
Angle 15 121.503 107.016 119.959 125.433 118.74
Angle 16 109.578 110.361 116.628 96.6743 122.095
Angle 17 123.69 109.257 110.754 121.265 125.428
Angle 18 115.133 118.679 111.425 106.873 111.713
Angle 19 109.111 109.078 114.059 116.375 120.102
Angle 20 102.369 121.584 108.353 106.241 125.246
(1) Firstly, shooting three conditions of an empty scene, normal characters and abnormal characters from the positions of 0 degrees, 45 degrees and 90 degrees of included angles between a camera and a target respectively, and extracting pictures from a video; (2) differentiating the pictures, wherein the left side is a normal person, and the right side is an abnormal person, and the obtained result is shown in fig. 8; (3) performing median filtering and corrosion denoising on the differential picture, wherein the result is shown in fig. 9; (4) extracting the coordinates of the barycenter of the person as shown in fig. 10; (5) classifying the gravity coordinates by using an SVM, wherein the correct label of the test data is shown in table 1, the gravity data of a normal person is marked as 1, the gravity data of an abnormal person is marked as-1, and the label of the test data after SVM classification is shown in table 2; (6) calculating and analyzing the included angles of the centers of gravity of 5 normal persons and 5 abnormal persons, as shown in tables 3, 4 and 11, the included angles of the centers of gravity of the normal persons are all larger than 120 degrees, and the included angles of the centers of the abnormal persons are relatively smaller; (7) the mean value of the center of gravity and the variances of the upper and lower half bodies are analyzed and compared, as shown in fig. 12 and 13, the variances of the upper and lower centers of gravity of a normal person are generally smaller than those of an obstacle group in the walking process, which shows that the centers of gravity of three points of the normal person are close to the central axis, the body is not obviously shaken in the walking process, and the balance capability is good.

Claims (7)

1. The method for distinguishing the crowd with balance ability disorder based on the center-of-gravity shift model is characterized by comprising the following specific operation steps of:
step 1, a lens of a camera is right opposite to the position between the front side and the side surface of a human body, namely, human body walking posture videos of normal people and abnormal people in a built virtual reality scene are collected from 45 degrees;
step 2, loading the two collected human body walking posture videos, respectively extracting video images into pictures, and then processing the pictures to respectively obtain upper, middle and lower barycentric coordinates of the two types of human bodies;
step 3, according to the obtained barycentric coordinate data, barycentric included angle data and mean square deviations of upper and lower barycenters of the human body are extracted, the extracted barycentric data are classified through an SVM classifier, and then the crowd with balance ability disorder is judged according to the barycentric included angle data and the mean square deviations of the upper and lower barycenters of the human body; the operation steps are as follows:
step 3.1, classifying the acquired barycentric coordinates as input data by an SVM classifier to obtain barycentric coordinate data of normal persons and abnormal persons with marks;
step 3.2, respectively extracting the gravity center included angle of the normal person and the abnormal person; the specific calculation process is as follows:
by calculating the three barycentric coordinates of the upper, middle and lower parts of the human body, using the atan2 function and converting the barycentric coordinates into angles, the included angle of the upper, middle and lower barycentric coordinates can be calculated: let the coordinate of the upper center of gravity P1 be (x)1,y1) Center of gravity coordinates P2 (x)2,y2) The lower center of gravity P3 is represented by the coordinate (x)3,y3) When the angle of gravity center is θ, the radian of the angle between P1P2 and the positive direction of x-axis is a tan2 (y)2-y1,x2-x1) Angle of inclination theta1Is alpha tan2 (y)2-y1,x2-x1) 180/pi, and the same principle, the angle theta between P3P2 and the positive direction of the x axis2Is a tan2 (y)3-y2,x3-x2) 180/pi, and the angle of gravity theta is represented as theta1+θ2As formula 7, the included angle of the center of gravity of the normal person and the abnormal person is calculated respectively:
Figure FDA0003197242470000021
wherein N is the number of the extracted included angles of the center of gravity, p1.y、p2.y、p3.yIs the ordinate of three gravity points, p1.x、p2.x、p3.xThe abscissa of the three gravity points;
step 3.3, calculating the mean square error of the gravity centers of the normal person and the abnormal person respectively; the specific process is as follows:
extracting one frame of picture every two seconds from the human body walking video, wherein the 20 frames of pictures are total, and calculating the mean value and the variance of the vertical coordinates of the gravity centers of the upper half body and the lower half body extracted after calculation by using a formula 8:
Figure FDA0003197242470000022
wherein n is the number of experimental objects, and t is the element [1, n ]]，
Figure FDA0003197242470000023
Represents the values of the upper and lower body barycentric ordinates at time t,
Figure FDA0003197242470000024
representing the mean value of the ordinate of the center of gravity, and CGS is the mean square error of the center of gravity;
step 3.4, combining the steps 3.1-3.3 to obtain the barycentric coordinates, barycentric included angles and barycentric mean square deviations of the human body, and distinguishing people with balance ability disorder; the method for distinguishing the people with balance disorder comprises the following steps:
judging according to the mark on the gravity center data, if the output balance capability is marked as '1', distinguishing as a normal person, and showing that the balance capability is good; if the output balance ability is marked as "-1", the person is distinguished as an abnormal person, which indicates that the balance ability is obstructed; the included angle value of the gravity center is relatively large, the balance capability is good, the included angle value of the gravity center is relatively small, and the balance capability is obstructed; the relatively large variance value of the center of gravity indicates that the larger and more unstable the fluctuation of the center of gravity of the human body, the worse the balance ability.
2. The method for distinguishing people with balance impairment based on the center-of-gravity shift model according to claim 1, wherein the step 2 is performed by the following steps:
step 2.1, reading the two collected human body posture videos by using a cvLoadImage function, setting the starting time and the ending time of the read-in videos, resetting the starting time and the ending time of the videos if the capture function does not read the videos containing the portrait, and carrying out the next step if the capture function reads the videos containing the portrait;
step 2.2, frames of the two groups of extracted videos are respectively removed through a CvCapture function in OpenCV, firstly, blank scenes in the two groups of videos are respectively extracted for picture storage, and then, one frame of the shot human body posture video is taken every two seconds and stored as a picture;
2.3, carrying out differential processing on the two groups of stored human body posture pictures and the blank scene respectively to obtain pictures only with human body postures;
step 2.4, carrying out image denoising on the image after the difference;
step 2.5, further carrying out image corrosion on the denoised picture to finally obtain a black and white image only with a human body image;
step 2.6, performing edge extraction on the image obtained after corrosion, processing the corroded image by using a Canny edge detection operator to obtain a connected region of the image, and performing convolution on the image by using a Gaussian filter to reduce the obvious noise influence on an edge detector; then, calculating the gradient strength and direction of each pixel point in the image, and applying non-maximum value inhibition to eliminate stray response caused by edge detection; finally, determining real and potential edges by using double-threshold detection, finally finishing edge detection by inhibiting isolated weak edges, and extracting a human body contour image by using findContours;
and 2.7, calculating the moment of the human body contour image, and calculating the barycentric coordinate of the human body through the moment of the contour image.
3. The method for distinguishing people with balance disabilities based on center-of-gravity shift model according to claim 2, wherein the specific process of the difference processing in step 2.3 is as follows:
firstly, carrying out binarization processing on two groups of stored human body posture pictures and blank scene pictures to enable all images to be black and white images;
then, carrying out difference processing on the two groups of binarized human body posture photos and blank scene pictures, and setting the image containing the human body extracted at the kth moment as xkThe image of the blank scene is xjAnd differentiating the two images to obtain a differential image delta xk：Δxk＝xk-xj。
4. The method for distinguishing people with balance ability impairment based on the center-of-gravity shift model as claimed in claim 2, wherein the image denoising process of the step 2.4 is: and (2) replacing the pixel value of one point in the differential processing image by using a median filtering method to obtain a denoised image by using the pixel median of each point in the neighborhood of the point, wherein the specific process is that f (x, y) and g (x, y) are respectively the image subjected to differential processing and the image subjected to denoising, and the median filtering output is g (x, y) ═ med { f (x-k, y-l), (k, l ∈ W) }, wherein W is a two-dimensional template, and k and l are respectively rows and columns of the image.
5. The method for distinguishing people with balance impairment based on the center-of-gravity shift model according to claim 2, wherein the specific operation steps of step 2.5 are as follows: and corroding the image subjected to difference and denoising, defining the size of a corrosion window by a getStructuringElement function, selecting a rectangular window MORPH-RECT, selecting the size of a corrosion kernel by 3 multiplied by 3, and then performing corrosion operation by using an erode function through the corrosion window MORPH-RECT to obtain the image only containing the portrait.
6. The method for distinguishing people with balance impairment based on the center-of-gravity shift model according to claim 2, wherein the calculation method of the body weight center coordinates in the step 2.7 is:
firstly, calculating the moment of a human body contour image, taking a human body in a picture as a planar object, taking a pixel value of each point as the density of the point, taking an expected value of the point as the moment of the point, and calculating the barycentric coordinate of the human body by adopting the first moment of the image, as shown in formulas 1-3:
Figure FDA0003197242470000051
Figure FDA0003197242470000052
the coordinates of the center of gravity of the human body image are:
Figure FDA0003197242470000053
wherein, V (i, j) represents the gray value of the human body image at the point (i, j), when the image is a binary image and V (i, j) only has black and white, namely two values of 0 and 1, then M is00Expressed as the sum of white areas in the human body image, i.e. the area of the binary image, M10Representing the accumulation of the horizontal coordinate values of the white area in the image; in the same way, M01Representing the accumulation of ordinate values of white areas in the image, xcRepresents the abscissa of the center of gravity, ycRepresenting the ordinate of the center of gravity.
7. The method for distinguishing people with balance ability impairment based on the center-of-gravity shift model according to claim 1, wherein the classification process using the SVM classifier in the step 3.1 is as follows:
firstly, loading a training data set and a testing data set, wherein the training data set comprises training data, training labels, testing data and testing labels, namely human body barycentric coordinate data and correct labels, dividing the training data and the testing data into two parts, obtaining the optimal parameters of current data through an SVMcgForRegress parameter optimizing function, and obtaining a trained model through the optimized parameters and training data through an svmtrain function; and finally, testing by using an svmpredict function to obtain barycentric coordinate data with a mark of '1' or '-1'.
CN201811052076.3A 2018-09-10 2018-09-10 Method for distinguishing people with balance ability disorder based on gravity center shift model Active CN109271918B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201811052076.3A CN109271918B (en) 2018-09-10 2018-09-10 Method for distinguishing people with balance ability disorder based on gravity center shift model

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201811052076.3A CN109271918B (en) 2018-09-10 2018-09-10 Method for distinguishing people with balance ability disorder based on gravity center shift model

Publications (2)

Publication Number Publication Date
CN109271918A CN109271918A (en) 2019-01-25
CN109271918B true CN109271918B (en) 2021-11-16

Family

ID=65187690

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201811052076.3A Active CN109271918B (en) 2018-09-10 2018-09-10 Method for distinguishing people with balance ability disorder based on gravity center shift model

Country Status (1)

Country Link
CN (1) CN109271918B (en)

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110555394A (en) * 2019-08-19 2019-12-10 西安理工大学 Fall risk assessment method based on human body shape characteristics
CN110705367A (en) * 2019-09-05 2020-01-17 西安理工大学 Human body balance ability classification method based on three-dimensional convolutional neural network
JP6813206B1 (en) * 2019-11-20 2021-01-13 株式会社Ｔａｏｓ研究所 Biological condition determination system, biological condition determination method and biological condition determination program
CN110931131B (en) * 2019-12-30 2023-04-28 华中科技大学鄂州工业技术研究院 Balance capability evaluation method and device
CN113017571A (en) * 2021-03-16 2021-06-25 西南交通大学 Balance capability evaluation method and system based on image recognition and balance beam test

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2009093631A1 (en) * 2008-01-23 2009-07-30 Panasonic Electric Works Co., Ltd. Device for evaluating center of gravity balancing
CN106491088A (en) * 2016-11-01 2017-03-15 吉林大学 A kind of balanced ability of human body appraisal procedure based on smart mobile phone
CN108309236A (en) * 2018-01-15 2018-07-24 新绎健康科技有限公司 Total balance of the body appraisal procedure and system

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2009093631A1 (en) * 2008-01-23 2009-07-30 Panasonic Electric Works Co., Ltd. Device for evaluating center of gravity balancing
CN106491088A (en) * 2016-11-01 2017-03-15 吉林大学 A kind of balanced ability of human body appraisal procedure based on smart mobile phone
CN108309236A (en) * 2018-01-15 2018-07-24 新绎健康科技有限公司 Total balance of the body appraisal procedure and system

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
《Three-dimensional, virtual reality vestibular rehabilitation for chronic imbalance problem caused by Ménière"s disease: a pilot study》;Hsu, Su-Yi et al;《Disability & Rehabilitation》;20171231;第39卷(第16期);全文 *
《基于体感交互设备的人体重心计算方法》;杨再兴;《中国优秀硕士学位论文全文数据库 基础科学辑》;20180215(第2期);第A002-793页 *

Also Published As

Publication number Publication date
CN109271918A (en) 2019-01-25

Similar Documents

Publication Publication Date Title
CN109271918B (en) Method for distinguishing people with balance ability disorder based on gravity center shift model
US10417775B2 (en) Method for implementing human skeleton tracking system based on depth data
CN105022982B (en) Hand motion recognition method and apparatus
CN109949341B (en) Pedestrian target tracking method based on human skeleton structural features
Zeng et al. Silhouette-based gait recognition via deterministic learning
CN105335725B (en) A kind of Gait Recognition identity identifying method based on Fusion Features
CN105160310A (en) 3D (three-dimensional) convolutional neural network based human body behavior recognition method
CN110532850B (en) Fall detection method based on video joint points and hybrid classifier
CN109345508A (en) A kind of Assessing Standards For Skeletal method based on two stages neural network
CN112395977B (en) Mammalian gesture recognition method based on body contour and leg joint skeleton
CN110991274B (en) Pedestrian tumbling detection method based on Gaussian mixture model and neural network
CN106650619A (en) Human action recognition method
CN104715238A (en) Pedestrian detection method based on multi-feature fusion
CN110032932B (en) Human body posture identification method based on video processing and decision tree set threshold
CN108648211A (en) A kind of small target detecting method, device, equipment and medium based on deep learning
Yun et al. Human fall detection via shape analysis on Riemannian manifolds with applications to elderly care
CN109558797B (en) Method for distinguishing human body balance disorder based on gravity center area model under visual stimulation
KR20120089948A (en) Real-time gesture recognition using mhi shape information
CN106023256B (en) State observation method towards augmented reality auxiliary maintaining System planes intended particle filter tracking
CN115346272A (en) Real-time tumble detection method based on depth image sequence
CN110490049A (en) The method for distinguishing total balance of the body obstacle based on multiple features and SVM
Liu et al. Adaptive recognition method for VR image of Wushu decomposition based on feature extraction
CN110555394A (en) Fall risk assessment method based on human body shape characteristics
JP7087969B2 (en) Pretreatment device, pretreatment method and pretreatment program
Li Research on camera-based human body tracking using improved cam-shift algorithm

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant