CN110675936A - Fitness compensation assessment method and system based on OpenPose and binocular vision - Google Patents
Fitness compensation assessment method and system based on OpenPose and binocular vision Download PDFInfo
- Publication number
- CN110675936A CN110675936A CN201911035478.7A CN201911035478A CN110675936A CN 110675936 A CN110675936 A CN 110675936A CN 201911035478 A CN201911035478 A CN 201911035478A CN 110675936 A CN110675936 A CN 110675936A
- Authority
- CN
- China
- Prior art keywords
- coordinate system
- compensation
- human body
- obtaining
- rotation
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H20/00—ICT specially adapted for therapies or health-improving plans, e.g. for handling prescriptions, for steering therapy or for monitoring patient compliance
- G16H20/30—ICT specially adapted for therapies or health-improving plans, e.g. for handling prescriptions, for steering therapy or for monitoring patient compliance relating to physical therapies or activities, e.g. physiotherapy, acupressure or exercising
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/20—Movements or behaviour, e.g. gesture recognition
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
- G16H50/30—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for calculating health indices; for individual health risk assessment
Landscapes
- Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- General Health & Medical Sciences (AREA)
- Public Health (AREA)
- Primary Health Care (AREA)
- Epidemiology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physical Education & Sports Medicine (AREA)
- Data Mining & Analysis (AREA)
- Biomedical Technology (AREA)
- Psychiatry (AREA)
- Biophysics (AREA)
- Databases & Information Systems (AREA)
- Pathology (AREA)
- Life Sciences & Earth Sciences (AREA)
- Social Psychology (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)
Abstract
The invention discloses a fitness compensation assessment method based on OpenPose and binocular vision, belonging to the field of fitness compensation assessment, and the method comprises the following steps: constructing a human body sport item-sport mode-compensation health basic library; capturing skeleton information of a human body by using two common RGB cameras and an OpenPose human skeleton model library to obtain two-dimensional coordinates of each joint point of the human body in different coordinate systems; calibrating the two RGB cameras, so as to realize the conversion and fusion of human skeleton data under the two cameras and obtain three-dimensional motion data of the two cameras under a world coordinate system; and extracting abnormal motion characteristic values according to a human body kinematics model with known kinematics parameters, and obtaining a fitness compensation evaluation result comprising three dimensions of freedom compensation, activity compensation and activity coordination compensation according to a matching result of the abnormal motion characteristic values and the standard motion characteristic values. The invention effectively reduces the subjectivity and the randomness of the evaluation process and improves the comprehensiveness and the scientificity of the compensation evaluation.
Description
Technical Field
The invention belongs to the field of fitness compensation assessment, and particularly relates to a fitness compensation assessment method and system based on OpenPose and binocular vision.
Background
Body-building compensation means that when a certain part of the limbs of a human body are damaged, the body mobilizes other undamaged or slightly damaged limbs to replace or compensate the functions of the severely damaged limbs. Many studies show that the body-building compensation can greatly influence the rehabilitation of the injured limb, seriously influence the reconstruction of the normal motion mode of the human body, and is extremely unfavorable for the motion stability, symmetry and coordination of the human body, and even can cause the further reduction of the function of the injured limb. Therefore, how to correctly evaluate and inhibit the body-building compensation is a key problem for the medical rehabilitation of stroke.
At present, the evaluation of the body health compensation mainly depends on the manual practical experience, a unified evaluation standard does not exist, the traceability and the reproducibility of the compensation evaluation based on the manual experience are poor, the subjective cognition degrees of different individuals in the evaluation process can influence the consistency of the evaluation result, and the occurrence rate of the compensation can be greatly improved under the condition of lack of real-time supervision and guidance. However, in the research of body health compensation by the existing scientific research institutions, sensors such as electromagnetism, myoelectricity, angle and the like need to be worn on the human body, the cost of data acquisition is too high, and the method is not suitable for practical application, so the existing compensation evaluation method only exists in experiments. In addition, most of the existing compensation evaluation methods only consider the degrees of freedom and joint motion degrees involved in movement, and the characterization signals of abnormal movement patterns are not only the joint angles, but also the movement sequence, the movement stationarity, the coordination and the like of the joints are very important factors.
In general, how to reduce the subjectivity of compensation assessment and provide a more convenient and universal body-building compensation assessment method has important significance.
Disclosure of Invention
Aiming at the defects or improvement requirements of the prior art, the invention provides a body health compensation assessment method and system based on OpenPose and binocular vision, and aims to solve the technical problems that the traditional compensation assessment method depends on manual experience judgment, is high in subjectivity, and is poor in assessment accuracy and effectiveness.
To achieve the above object, according to an aspect of the present invention, there is provided a method for evaluating fitness compensation based on openpos and binocular vision, including:
(1) constructing a human body sport item-sport mode-compensation health basic library;
the basic library comprises a human body movement project, a movement mode corresponding to the movement project and a compensatory health-care body corresponding to the movement mode; the compensation health-care body is a human body part which causes abnormal movement in the process of carrying out normal movement;
(2) collecting and calibrating human motion data;
collecting human body motion information by using uncalibrated visual sensors distributed at different positions, inputting the collected motion information into an OpenPose human body skeleton model to obtain skeleton information of a human body, and obtaining two-dimensional coordinates of each joint point of the human body under different coordinate systems;
calibrating the two vision sensors to realize the conversion and fusion of each joint point of the human body under different coordinate systems to obtain three-dimensional motion data of the human body under a world coordinate system;
(3) constructing a human body kinematics model;
respectively establishing human body kinematics models corresponding to different training items for different human bodies according to the basic library in the step (1); the human body kinematics model comprises a change mode of joint angles, joint angular velocities and angular accelerations along with the motion of a human body;
obtaining a kinematic parameter corresponding to a kinematic model capable of representing a motion mode of a human body according to three-dimensional motion data of the human body under a world coordinate system;
(4) evaluating body health compensation;
extracting an abnormal motion characteristic value according to a human body kinematics model with known kinematics parameters, and obtaining a body health compensation evaluation result according to a matching result of the abnormal motion characteristic value and a standard motion characteristic value; the fitness compensation evaluation result comprises three dimensions of freedom compensation, activity compensation and activity coordination compensation.
Further, the step (1) of constructing the human body training item-motion pattern-compensation health basic library specifically comprises the following steps:
(1.1) performing kinematic analysis on upper limbs and a trunk of a human body when different movement projects are generated, and refining the movement of the upper limbs and the trunk into flexion and extension movement of each joint around a coronal axis, expansion and contraction movement around a sagittal axis and rotation movement around a vertical axis;
and (1.2) taking the kinematic joints corresponding to different sports items and the generated compensated health as the basic library.
Further in pairs, the vision sensor is an RGB camera.
Further, the step (3) of respectively establishing human body kinematics models corresponding to different training items for different human bodies specifically includes:
(01) establishing a local coordinate system;
respectively establishing a world coordinate system Og-XgYgZgTorso coordinate system Ot-XtYtZtUpper arm coordinate system Ou-XuYuZuForearm coordinate system Of-XfYfZfAnd palm coordinate system Oh-XhYhZh；
(02) Calculating a rotation column vector of each local coordinate system;
obtaining a rotation column vector of the trunk coordinate system according to the unit vector of each coordinate axis of the trunk coordinate system; obtaining a rotation row vector of the upper arm coordinate system according to the unit vector of each coordinate axis of the upper arm coordinate system; obtaining a rotation column vector of the forearm coordinate system according to the unit vector of each coordinate axis of the forearm coordinate system; obtaining a rotation column vector of the palm coordinate system according to the unit vector of each coordinate axis of the palm coordinate system;
(03) calculating joint angles of different joints at all times;
according to the rotating column vector of the trunk coordinate system, a first rotating matrix of the trunk coordinate system relative to a world coordinate system is obtained, and a trunk angle is obtained according to the first rotating matrix;
according to the rotation row vector of the trunk coordinate system and the rotation row vector of the upper arm coordinate system, obtaining a second rotation matrix of the upper arm coordinate system relative to the trunk coordinate system, and obtaining a shoulder joint angle according to the second rotation matrix;
according to the rotation row vector of the upper arm coordinate system and the rotation row vector of the forearm coordinate system, obtaining a third rotation matrix of the forearm coordinate system relative to the upper arm coordinate system, and obtaining an elbow joint angle according to the third rotation matrix;
and obtaining a fourth rotation matrix of the palm coordinate system relative to the forearm coordinate system according to the rotation column vector of the forearm coordinate system and the rotation column vector of the palm coordinate system, and obtaining the wrist joint angle according to the fourth rotation matrix.
(04) And calculating the angular velocity and the angular acceleration of the corresponding joint according to the joint angles of different joints at various times.
According to another aspect of the present invention, there is provided an openpos and binocular vision based fitness compensation assessment system, comprising:
the basic library construction module is used for constructing a basic library comprising a human body movement project, a movement mode corresponding to the movement project and a compensatory health-care body corresponding to the movement mode; the compensation health-care body is a human body part which causes abnormal movement in the process of carrying out normal movement;
the motion data acquisition module is used for acquiring human motion information by using uncalibrated visual sensors distributed at different positions, and inputting the acquired motion information into an OpenPose human skeleton model to obtain two-dimensional coordinates of each joint point of the human body under different coordinate systems; calibrating the two visual sensors simultaneously to realize the conversion and fusion of all joint points of the human body under different coordinate systems and obtain three-dimensional motion data of the joint points under a world coordinate system;
the human body kinematics model building module is used for respectively building human body kinematics models corresponding to different training items for different human bodies by the basic library and obtaining kinematics parameters corresponding to the kinematics models capable of representing the movement modes of the different human bodies according to three-dimensional movement data of the different human bodies in a world coordinate system; the human body kinematics model comprises a change mode of joint angles, joint angular velocities and angular accelerations along with the motion of a human body;
the body-building compensation evaluation module is used for extracting an abnormal motion characteristic value according to a human body kinematics model with known kinematics parameters and obtaining a body-building compensation evaluation result according to a matching result of the abnormal motion characteristic value and a standard motion characteristic value; the fitness compensation evaluation result comprises three dimensions of freedom compensation, activity compensation and activity coordination compensation.
Further, the base library building module comprises:
the motion analysis unit is used for performing kinematic analysis on the upper limbs and the trunk of the human body when different motion items are generated, and the motion of the upper limbs and the trunk is refined into flexion and extension motion of each joint around a coronal axis, expansion and contraction motion around a sagittal axis and rotation motion around a vertical axis;
and the motion information storage unit is used for taking the motion joints corresponding to different motion items and the generated compensated health as the basic library.
Further, the vision sensor is an RGB camera.
Further, the establishing of the human kinematics models corresponding to different training items for different human bodies respectively specifically includes:
(01) establishing a local coordinate system;
respectively establishing a world coordinate system Og-XgYgZgTorso coordinate system Ot-XtYtZtUpper arm coordinate system Ou-XuYuZuForearm coordinate system Of-XfYfZfAnd palm coordinate system Oh-XhYhZh；
(02) Calculating a rotation column vector of each local coordinate system;
obtaining a rotation column vector of the trunk coordinate system according to the unit vector of each coordinate axis of the trunk coordinate system; obtaining a rotation row vector of the upper arm coordinate system according to the unit vector of each coordinate axis of the upper arm coordinate system; obtaining a rotation column vector of the forearm coordinate system according to the unit vector of each coordinate axis of the forearm coordinate system; obtaining a rotation column vector of the palm coordinate system according to the unit vector of each coordinate axis of the palm coordinate system;
(03) calculating joint angles of different joints at all times;
according to the rotating column vector of the trunk coordinate system, a first rotating matrix of the trunk coordinate system relative to a world coordinate system is obtained, and a trunk angle is obtained according to the first rotating matrix;
according to the rotation row vector of the trunk coordinate system and the rotation row vector of the upper arm coordinate system, obtaining a second rotation matrix of the upper arm coordinate system relative to the trunk coordinate system, and obtaining a shoulder joint angle according to the second rotation matrix;
according to the rotation row vector of the upper arm coordinate system and the rotation row vector of the forearm coordinate system, obtaining a third rotation matrix of the forearm coordinate system relative to the upper arm coordinate system, and obtaining an elbow joint angle according to the third rotation matrix;
and obtaining a fourth rotation matrix of the palm coordinate system relative to the forearm coordinate system according to the rotation column vector of the forearm coordinate system and the rotation column vector of the palm coordinate system, and obtaining the wrist joint angle according to the fourth rotation matrix.
(04) And calculating the angular velocity and the angular acceleration of the corresponding joint according to the joint angles of different joints at various times.
In general, compared with the prior art, the above technical solution contemplated by the present invention can achieve the following beneficial effects:
(1) the invention provides a fitness compensation evaluation method taking kinematic parameters as main characteristic values, which does not depend on subjective judgment of human experience any more, and effectively reduces the subjectivity and the randomness of the evaluation process; the invention simultaneously constructs evaluation indexes covering multiple dimensions such as freedom degree, joint activity degree, motion sequence, motion stability and the like, and improves the comprehensiveness and scientificity of compensation evaluation.
(2) The invention utilizes the non-calibrated visual sensor to acquire the human motion data, eliminates the physical and psychological loads on the human body and the inconvenience of the acquisition process caused by the traditional contact type sensing equipment such as electromagnetism, myoelectricity, angle and the like, and ensures that the evaluation process is more convenient and more practical.
Drawings
Fig. 1 is a flowchart of a fitness compensation assessment method based on openpos and binocular vision according to the present invention.
FIG. 2 is a view of the basic plane and the basic axis of a human body;
FIG. 3 is a diagram of the basic movement pattern of the human joint;
FIG. 4 is a diagram of a human world coordinate system and local coordinate systems of segments;
fig. 5 is a schematic diagram illustrating establishment of a fitness compensation assessment index.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. In addition, the technical features involved in the embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.
Referring to fig. 1, an embodiment of the present invention provides a fitness compensation assessment method based on openpos and binocular vision, including:
(1) constructing a human body sport item-sport mode-compensation health basic library; the basic library comprises a human body movement project, a movement mode corresponding to the movement project and a compensatory health-care body corresponding to the movement mode; the compensation health care body is a human body part which causes abnormal movement in the process of carrying out normal movement;
specifically, the kinematics of the upper limbs and the trunk of the human body which may generate movement is analyzed, the movement is refined into the flexion and extension movement of each joint around the coronal axis, the extension and retraction movement around the sagittal axis and the rotation movement around the vertical axis, the position relationship among the coronal axis, the sagittal axis, the vertical axis and the human body is shown in fig. 2, and the refined result of various movements of the human body is shown in fig. 3; then enumerating common movement joints and movement modes of related movement items, and obtaining a compensatory body building causing abnormal movement through experimental observation, thereby constituting the basic library, as shown in table 1;
TABLE 1
(2) Collecting and calibrating human motion data;
the embodiment of the invention utilizes two uncalibrated visual sensors distributed at different positions to acquire the motion information of a human body, and inputs the acquired motion information into an OpenPose human skeleton model to acquire skeleton information of the human body, so as to obtain two-dimensional coordinates of each joint point of the human body under different coordinate systems; specifically, the present embodiment employs a general RGB camera as a visual sensor.
Calibrating the two visual sensors to realize the conversion and fusion of each joint point of the human body under different coordinate systems to obtain three-dimensional motion data of the human body under a world coordinate system;
(3) constructing a human body kinematics model;
respectively establishing human body kinematics models corresponding to different training items for different human bodies according to the basic library established in the step (1); the human body kinematics model comprises a change mode of joint angles, joint angular velocities and angular accelerations along with the motion of a human body; obtaining kinematic parameters corresponding to a kinematic model capable of representing a motion mode of the human body according to three-dimensional motion data of different human bodies in a world coordinate system;
specifically, the step (3) of respectively establishing human body kinematics models corresponding to different training items for different human bodies specifically includes:
(01) establishing a local coordinate system;
as shown in FIG. 4, a world coordinate system O is established respectivelyg-XgYgZgTorso coordinate system Ot-XtYtZtUpper arm coordinate system Ou-XuYuZuForearm coordinate system Of-XfYfZfAnd palm coordinate system Oh-XhYhZh；
(02) Calculating a rotation column vector of each local coordinate system;
obtaining a rotation column vector of the trunk coordinate system according to the unit vector of each coordinate axis of the trunk coordinate system;
more specifically, the unit vectors of the coordinate axes of the trunk coordinate system are respectively represented by etz、etx、etyThis means that there are:
the rotated column vector of the torso coordinate system is:
obtaining a rotation row vector of the upper arm coordinate system according to the unit vector of each coordinate axis of the upper arm coordinate system;
more specifically, the unit vectors of the coordinate axes of the upper arm coordinate system are respectively represented by euz、eux、euyThis means that there are:
the rotated row vector of the upper arm coordinate system is:
obtaining a rotation row vector of the forearm coordinate system according to the unit vector of each coordinate axis of the forearm coordinate system;
more specifically, the unit vectors of the coordinate axes of the forearm coordinate system are represented by efz、efx、efyThis means that there are:
the rotated column vector of the forearm coordinate system is:
obtaining a rotation row vector of the palm coordinate system according to the unit vector of each coordinate axis of the palm coordinate system;
more specifically, the unit vectors of the coordinate axes of the palm coordinate system are represented by ehz、ehx、ehyThis means that there are:
the rotated column vector of the palm coordinate system is:
(03) calculating joint angles of different joints at all times;
specifically, according to the coordinate system defined in fig. 4, the joint motion can be described by the local coordinate systems of the adjacent segments, and the local coordinate system rotation matrices at different times of each segment are respectively calculated, so that the joint angles at different times of different joints can be obtained. Further specifically, a first rotation matrix of the trunk coordinate system relative to the world coordinate system is obtained according to the rotation column vector of the trunk coordinate system, and a trunk angle is obtained according to the first rotation matrix; the first rotation matrix of the torso coordinate system relative to the world coordinate system is:
according to the rotation column vector of the trunk coordinate system and the rotation column vector of the upper arm coordinate system, obtaining a second rotation matrix of the upper arm coordinate system relative to the trunk coordinate system, and obtaining a shoulder joint angle according to the second rotation matrix; the second rotation matrix of the upper arm coordinate system relative to the torso coordinate system is:
obtaining a third rotation matrix of the forearm coordinate system relative to the upper arm coordinate system according to the rotation column vector of the upper arm coordinate system and the rotation column vector of the forearm coordinate system, and obtaining an elbow joint angle according to the third rotation matrix; the third rotation matrix of the forearm coordinate system relative to the upper arm coordinate system is:
and obtaining a fourth rotation matrix of the palm coordinate system relative to the forearm coordinate system according to the rotation column vector of the forearm coordinate system and the rotation column vector of the palm coordinate system, and obtaining the wrist joint angle according to the fourth rotation matrix. The fourth rotation matrix of the palm coordinate system relative to the forearm coordinate system is:
after rotation matrixes of each local coordinate system relative to the other local coordinate system are respectively obtained, different Euler angles can be obtained due to different rotation sequences, the rotation sequence is defined to rotate around three X-Y-Z axes in sequence, and the Euler angles obtained by the known rotation matrixes R (alpha, beta, gamma) have the following formula:
wherein r isijIs the matrix element corresponding to the index position.
(04) And calculating the angular velocity and the angular acceleration of the corresponding joint according to the joint angles of different joints at various times.
Specifically, the angular acceleration
Expressed in a scalar form, in a rigid body local coordinate system, the joint angular velocity of the known joint angle obtained by the method has the following Euler equation:
accordingly, in the rigid local coordinate system, the angular acceleration is:
(4) evaluating body health compensation;
extracting an abnormal motion characteristic value according to a human body kinematics model with known kinematics parameters, and obtaining a body health compensation evaluation result according to a matching result of the abnormal motion characteristic value and a standard motion characteristic value; the fitness compensation evaluation result comprises three dimensions of freedom compensation, activity compensation and activity coordination compensation.
The embodiment of the invention also provides a health compensation evaluation system based on OpenPose and binocular vision, which comprises: the basic library construction module is used for constructing a basic library comprising a human body movement project, a movement mode corresponding to the movement project and a compensatory health-care body corresponding to the movement mode; the compensation health-care body is a human body part which causes abnormal movement in the process of carrying out normal movement; the motion data acquisition module is used for acquiring human motion information by using uncalibrated visual sensors distributed at different positions, and inputting the acquired motion information into an OpenPose human skeleton model to obtain two-dimensional coordinates of each joint point of the human body under different coordinate systems; calibrating the two visual sensors simultaneously to realize the conversion and fusion of all joint points of the human body under different coordinate systems and obtain three-dimensional motion data of the joint points under a world coordinate system; the human body kinematics model building module is used for respectively building human body kinematics models corresponding to different training items for different human bodies by the basic library and obtaining kinematics parameters corresponding to the kinematics models capable of representing the movement modes of the different human bodies according to three-dimensional movement data of the different human bodies in a world coordinate system; the human body kinematics model comprises a change mode of joint angles, joint angular velocities and angular accelerations along with the motion of a human body; the body-building compensation evaluation module is used for extracting an abnormal motion characteristic value according to a human body kinematics model with known kinematics parameters and obtaining a body-building compensation evaluation result according to a matching result of the abnormal motion characteristic value and a standard motion characteristic value; the fitness compensation evaluation result comprises three dimensions of freedom compensation, activity compensation and activity coordination compensation.
Wherein, the vision sensor is an RGB camera; the basic library building module comprises: the motion analysis unit is used for performing kinematic analysis on the upper limbs and the trunk of the human body when different motion items are generated, and the motion of the upper limbs and the trunk is refined into flexion and extension motion of each joint around a coronal axis, expansion and contraction motion around a sagittal axis and rotation motion around a vertical axis; and the motion information storage unit is used for taking the motion joints corresponding to different motion items and the generated compensated health as the basic library.
The method for establishing the human body kinematics model corresponding to different training items for different human bodies comprises the following steps:
(01) establishing a local coordinate system; respectively establishing a world coordinate system Og-XgYgZgTorso coordinate system Ot-XtYtZtUpper arm coordinate system Ou-XuYuZuForearm coordinate system Of-XfYfZfAnd palm coordinate system Oh-XhYhZh；
(02) Calculating a rotation column vector of each local coordinate system; obtaining a rotation column vector of the trunk coordinate system according to the unit vector of each coordinate axis of the trunk coordinate system; obtaining a rotation row vector of the upper arm coordinate system according to the unit vector of each coordinate axis of the upper arm coordinate system; obtaining a rotation row vector of the forearm coordinate system according to the unit vector of each coordinate axis of the forearm coordinate system; obtaining a rotation row vector of the palm coordinate system according to the unit vector of each coordinate axis of the palm coordinate system;
(03) calculating joint angles of different joints at all times; according to the rotating column vector of the trunk coordinate system, a first rotating matrix of the trunk coordinate system relative to the world coordinate system is obtained, and a trunk angle is obtained according to the first rotating matrix; according to the rotation row vector of the trunk coordinate system and the rotation row vector of the upper arm coordinate system, a second rotation matrix of the upper arm coordinate system relative to the trunk coordinate system is obtained, and the shoulder joint angle is obtained according to the second rotation matrix; obtaining a third rotation matrix of the forearm coordinate system relative to the upper arm coordinate system according to the rotation column vector of the upper arm coordinate system and the rotation column vector of the forearm coordinate system, and obtaining an elbow joint angle according to the third rotation matrix; and obtaining a fourth rotation matrix of the palm coordinate system relative to the forearm coordinate system according to the rotation column vector of the forearm coordinate system and the rotation column vector of the palm coordinate system, and obtaining the wrist joint angle according to the fourth rotation matrix.
(04) And calculating the angular velocity and the angular acceleration of the corresponding joint according to the joint angles of different joints at various times.
The implementation of each module of the system corresponds to the above method, and the present invention is not described herein again. It will be understood by those skilled in the art that the foregoing is only a preferred embodiment of the present invention, and is not intended to limit the invention, and that any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the scope of the present invention.
Claims (8)
1. A fitness compensation assessment method based on OpenPose and binocular vision is characterized by comprising the following steps:
(1) constructing a human body sport item-sport mode-compensation health basic library;
the basic library comprises a human body movement project, a movement mode corresponding to the movement project and a compensatory health-care body corresponding to the movement mode; the compensation health-care body is a human body part which causes abnormal movement in the process of carrying out normal movement;
(2) collecting and calibrating human motion data;
collecting human body motion information by using uncalibrated visual sensors distributed at different positions, inputting the collected motion information into an OpenPose human body skeleton model to obtain skeleton information of a human body, and obtaining two-dimensional coordinates of each joint point of the human body under different coordinate systems;
calibrating the two vision sensors to realize the conversion and fusion of each joint point of the human body under different coordinate systems to obtain three-dimensional motion data of the human body under a world coordinate system;
(3) constructing a human body kinematics model;
respectively establishing human body kinematics models corresponding to different training items for different human bodies according to the basic library in the step (1); the human body kinematics model comprises a change mode of joint angles, joint angular velocities and angular accelerations along with the motion of a human body;
obtaining a kinematic parameter corresponding to a kinematic model capable of representing a motion mode of a human body according to three-dimensional motion data of the human body under a world coordinate system;
(4) evaluating body health compensation;
extracting an abnormal motion characteristic value according to a human body kinematics model with known kinematics parameters, and obtaining a body health compensation evaluation result according to a matching result of the abnormal motion characteristic value and a standard motion characteristic value; the fitness compensation evaluation result comprises three dimensions of freedom compensation, activity compensation and activity coordination compensation.
2. The fitness compensation assessment method based on OpenPose and binocular vision according to claim 1, wherein the step (1) of constructing a human body training project-motion pattern-compensation fitness base library specifically comprises:
(1.1) performing kinematic analysis on upper limbs and a trunk of a human body when different movement projects are generated, and refining the movement of the upper limbs and the trunk into flexion and extension movement of each joint around a coronal axis, expansion and contraction movement around a sagittal axis and rotation movement around a vertical axis;
and (1.2) taking the kinematic joints corresponding to different sports items and the generated compensated health as the basic library.
3. The openpos and binocular vision based fitness compensation assessment method according to claim 1 or 2, wherein the vision sensor is an RGB camera.
4. The fitness compensation assessment method based on OpenPose and binocular vision according to any one of claims 1-3, wherein the step (3) of establishing human kinematics models corresponding to different training items for different human bodies respectively comprises:
(01) establishing a local coordinate system;
respectively establishing a world coordinate system Og-XgYgZgTorso coordinate system Ot-XtYtZtUpper arm coordinate system Ou-XuYuZuForearm coordinate system Of-XfYfZfAnd palm coordinate system Oh-XhYhZh；
(02) Calculating a rotation column vector of each local coordinate system;
obtaining a rotation column vector of the trunk coordinate system according to the unit vector of each coordinate axis of the trunk coordinate system; obtaining a rotation row vector of the upper arm coordinate system according to the unit vector of each coordinate axis of the upper arm coordinate system; obtaining a rotation column vector of the forearm coordinate system according to the unit vector of each coordinate axis of the forearm coordinate system; obtaining a rotation column vector of the palm coordinate system according to the unit vector of each coordinate axis of the palm coordinate system;
(03) calculating joint angles of different joints at all times;
according to the rotating column vector of the trunk coordinate system, a first rotating matrix of the trunk coordinate system relative to a world coordinate system is obtained, and a trunk angle is obtained according to the first rotating matrix;
according to the rotation row vector of the trunk coordinate system and the rotation row vector of the upper arm coordinate system, obtaining a second rotation matrix of the upper arm coordinate system relative to the trunk coordinate system, and obtaining a shoulder joint angle according to the second rotation matrix;
according to the rotation row vector of the upper arm coordinate system and the rotation row vector of the forearm coordinate system, obtaining a third rotation matrix of the forearm coordinate system relative to the upper arm coordinate system, and obtaining an elbow joint angle according to the third rotation matrix;
and obtaining a fourth rotation matrix of the palm coordinate system relative to the forearm coordinate system according to the rotation column vector of the forearm coordinate system and the rotation column vector of the palm coordinate system, and obtaining the wrist joint angle according to the fourth rotation matrix.
(04) And calculating the angular velocity and the angular acceleration of the corresponding joint according to the joint angles of different joints at various times.
5. A fitness compensation assessment system based on OpenPose and binocular vision, comprising:
the basic library construction module is used for constructing a basic library comprising a human body movement project, a movement mode corresponding to the movement project and a compensatory health-care body corresponding to the movement mode; the compensation health-care body is a human body part which causes abnormal movement in the process of carrying out normal movement;
the motion data acquisition module is used for acquiring human motion information by using two uncalibrated visual sensors distributed at different positions, and inputting the acquired motion information into an OpenPose human skeleton model to obtain two-dimensional coordinates of each joint point of the human body under different coordinate systems; calibrating the two visual sensors simultaneously to realize the conversion and fusion of all joint points of the human body under different coordinate systems and obtain three-dimensional motion data of the human body under a world coordinate system;
the human body kinematics model building module is used for respectively building human body kinematics models corresponding to different training items for different human bodies by the basic library and obtaining kinematics parameters corresponding to the kinematics models capable of representing the movement modes of the different human bodies according to three-dimensional movement data of the different human bodies in a world coordinate system; the human body kinematics model comprises a change mode of joint angles, joint angular velocities and angular accelerations along with the motion of a human body;
the body-building compensation evaluation module is used for extracting an abnormal motion characteristic value according to a human body kinematics model with known kinematics parameters and obtaining a body-building compensation evaluation result according to a matching result of the abnormal motion characteristic value and a standard motion characteristic value; the fitness compensation evaluation result comprises three dimensions of freedom compensation, activity compensation and activity coordination compensation.
6. The openpos and binocular vision based fitness compensation assessment system of claim 5, wherein the base library construction module comprises:
the motion analysis unit is used for performing kinematic analysis on the upper limbs and the trunk of the human body when different motion items are generated, and the motion of the upper limbs and the trunk is refined into flexion and extension motion of each joint around a coronal axis, expansion and contraction motion around a sagittal axis and rotation motion around a vertical axis;
and the motion information storage unit is used for taking the motion joints corresponding to different motion items and the generated compensated health as the basic library.
7. The openpos and binocular vision based fitness compensation assessment system of claim 5, wherein the vision sensor is an RGB camera.
8. The system for evaluating fitness compensation according to any one of claims 5-7, wherein the system for respectively establishing human kinematics models corresponding to different training items for different human bodies comprises:
(01) establishing a local coordinate system;
respectively establishing a world coordinate system Og-XgYgZgTorso coordinate system Ot-XtYtZtUpper arm coordinate system Ou-XuYuZuForearm coordinate system Of-XfYfZfAnd palm coordinate system Oh-XhYhZh；
(02) Calculating a rotation column vector of each local coordinate system;
obtaining a rotation column vector of the trunk coordinate system according to the unit vector of each coordinate axis of the trunk coordinate system; obtaining a rotation row vector of the upper arm coordinate system according to the unit vector of each coordinate axis of the upper arm coordinate system; obtaining a rotation column vector of the forearm coordinate system according to the unit vector of each coordinate axis of the forearm coordinate system; obtaining a rotation column vector of the palm coordinate system according to the unit vector of each coordinate axis of the palm coordinate system;
(03) calculating joint angles of different joints at all times;
according to the rotating column vector of the trunk coordinate system, a first rotating matrix of the trunk coordinate system relative to a world coordinate system is obtained, and a trunk angle is obtained according to the first rotating matrix;
according to the rotation row vector of the trunk coordinate system and the rotation row vector of the upper arm coordinate system, obtaining a second rotation matrix of the upper arm coordinate system relative to the trunk coordinate system, and obtaining a shoulder joint angle according to the second rotation matrix;
according to the rotation row vector of the upper arm coordinate system and the rotation row vector of the forearm coordinate system, obtaining a third rotation matrix of the forearm coordinate system relative to the upper arm coordinate system, and obtaining an elbow joint angle according to the third rotation matrix;
and obtaining a fourth rotation matrix of the palm coordinate system relative to the forearm coordinate system according to the rotation column vector of the forearm coordinate system and the rotation column vector of the palm coordinate system, and obtaining the wrist joint angle according to the fourth rotation matrix.
(04) And calculating the angular velocity and the angular acceleration of the corresponding joint according to the joint angles of different joints at various times.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911035478.7A CN110675936B (en) | 2019-10-29 | 2019-10-29 | Fitness compensation assessment method and system based on OpenPose and binocular vision |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911035478.7A CN110675936B (en) | 2019-10-29 | 2019-10-29 | Fitness compensation assessment method and system based on OpenPose and binocular vision |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110675936A true CN110675936A (en) | 2020-01-10 |
| CN110675936B CN110675936B (en) | 2021-08-03 |
Family
ID=69084719
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911035478.7A Active CN110675936B (en) | 2019-10-29 | 2019-10-29 | Fitness compensation assessment method and system based on OpenPose and binocular vision |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110675936B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113409913A (en) * | 2021-06-25 | 2021-09-17 | 黄富表 | Microsoft Kinect-based method for evaluating upper limb motor function of stroke patient |
| WO2022088290A1 (en) * | 2020-10-28 | 2022-05-05 | 中国科学院深圳先进技术研究院 | Motion assessment method, apparatus and system, and storage medium |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102567638A (en) * | 2011-12-29 | 2012-07-11 | 无锡微感科技有限公司 | Interactive upper limb rehabilitation system based on micro-sensor |
| US20170000386A1 (en) * | 2015-07-01 | 2017-01-05 | BaziFIT, Inc. | Method and system for monitoring and analyzing position, motion, and equilibrium of body parts |
| CN109243572A (en) * | 2018-11-08 | 2019-01-18 | 中科数字健康科学研究院（南京）有限公司 | A kind of accurate locomotion evaluation and rehabilitation training system |
-
2019
- 2019-10-29 CN CN201911035478.7A patent/CN110675936B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102567638A (en) * | 2011-12-29 | 2012-07-11 | 无锡微感科技有限公司 | Interactive upper limb rehabilitation system based on micro-sensor |
| US20170000386A1 (en) * | 2015-07-01 | 2017-01-05 | BaziFIT, Inc. | Method and system for monitoring and analyzing position, motion, and equilibrium of body parts |
| CN109243572A (en) * | 2018-11-08 | 2019-01-18 | 中科数字健康科学研究院（南京）有限公司 | A kind of accurate locomotion evaluation and rehabilitation training system |
Non-Patent Citations (3)
| Title |
|---|
| VALDÉS B A, SCHNEIDER A N, HFM V D L: "Reducing Trunk Compensation in Stroke Survivors: A Randomized Crossover Trial Comparing Visual vs. Force Feedback Modalities", 《ARCHIVES OF PHYSICAL MEDICINE & REHABILITATION》 * |
| YING XUAN ZHI;MICHELLE LUKASIK;MICHAEL H. LI: "Automatic Detection of Compensation During Robotic Stroke Rehabilitation Therapy", 《 IEEE JOURNAL OF TRANSLATIONAL ENGINEERING IN HEALTH AND MEDICINE》 * |
| 陈学梅: "基于人体三维姿态的动作评价系统", 《中国优秀硕士学位论文全文数据库 信息科技辑》 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2022088290A1 (en) * | 2020-10-28 | 2022-05-05 | 中国科学院深圳先进技术研究院 | Motion assessment method, apparatus and system, and storage medium |
| CN113409913A (en) * | 2021-06-25 | 2021-09-17 | 黄富表 | Microsoft Kinect-based method for evaluating upper limb motor function of stroke patient |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110675936B (en) | 2021-08-03 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Fantozzi et al. | Assessment of three-dimensional joint kinematics of the upper limb during simulated swimming using wearable inertial-magnetic measurement units | |
| Robertson et al. | Research methods in biomechanics | |
| CN107616898B (en) | Upper limb wearable rehabilitation robot based on daily actions and rehabilitation evaluation method | |
| CN107349570A (en) | Rehabilitation training of upper limbs and appraisal procedure based on Kinect | |
| Gil-Agudo et al. | A novel motion tracking system for evaluation of functional rehabilitation of the upper limbs | |
| CN110675936B (en) | Fitness compensation assessment method and system based on OpenPose and binocular vision | |
| CN103692454A (en) | Exoskeleton wearable data glove | |
| WO2001073689A2 (en) | Method and system for viewing kinematic and kinetic information | |
| Chèze | Kinematic analysis of human movement | |
| CN107256390B (en) | Hand function evaluation device and method based on change of each part of hand in three-dimensional space position | |
| Bumacod et al. | Image-processing-based digital goniometer using OpenCV | |
| Yahya et al. | Accurate shoulder joint angle estimation using single RGB camera for rehabilitation | |
| CN203680324U (en) | Outer framework wearing-type data glove | |
| CN111369626B (en) | Mark point-free upper limb movement analysis method and system based on deep learning | |
| CN108538362B (en) | Tendon anisotropic stress injury early warning analysis method with real-time acquisition of motion data | |
| CN110766985A (en) | Wearable motion sensing interactive teaching system and motion sensing method thereof | |
| Lin et al. | Using hybrid sensoring method for motion capture in volleyball techniques training | |
| CN109887570B (en) | Robot-assisted rehabilitation training method based on RGB-D camera and IMU sensor | |
| Saggio et al. | Dynamic measurement assessments of sensory gloves based on resistive flex sensors and inertial measurement units | |
| Chen et al. | An inertial-based human motion tracking system with twists and exponential maps | |
| CN116740618A (en) | Motion video action evaluation method, system, computer equipment and medium | |
| Xu et al. | A Low-Cost Wearable Hand Gesture Detecting System Based on IMU and Convolutional Neural Network | |
| WO2019152566A1 (en) | Systems and methods for subject specific kinematic mapping | |
| Du et al. | Data fusion of multiple kinect sensors for a rehabilitation system | |
| JP2014117409A (en) | Method and apparatus for measuring body joint position |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |