CN113180645A - Multi-segment foot kinematics analysis system and method based on dynamic point cloud segmentation - Google Patents

Multi-segment foot kinematics analysis system and method based on dynamic point cloud segmentation Download PDF

Info

Publication number
CN113180645A
CN113180645A CN202110591048.4A CN202110591048A CN113180645A CN 113180645 A CN113180645 A CN 113180645A CN 202110591048 A CN202110591048 A CN 202110591048A CN 113180645 A CN113180645 A CN 113180645A
Authority
CN
China
Prior art keywords
point cloud
data
foot
segment
dynamic
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202110591048.4A
Other languages
Chinese (zh)
Inventor
蒋东港
陈文明
谷彦颉
钱乐文
王晨
马昕
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Fudan University
Zhuhai Fudan Innovation Research Institute
Original Assignee
Fudan University
Zhuhai Fudan Innovation Research Institute
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Fudan University, Zhuhai Fudan Innovation Research Institute filed Critical Fudan University
Priority to CN202110591048.4A priority Critical patent/CN113180645A/en
Publication of CN113180645A publication Critical patent/CN113180645A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/103—Detecting, measuring or recording devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
    • A61B5/11—Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb
    • A61B5/1116—Determining posture transitions
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/103—Detecting, measuring or recording devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
    • A61B5/11—Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb
    • A61B5/1118—Determining activity level
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/103—Detecting, measuring or recording devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
    • A61B5/11—Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb
    • A61B5/112—Gait analysis
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes

Abstract

The invention provides a multi-segment foot kinematics analysis system and method based on dynamic point cloud segmentation, wherein the system comprises a 3D point cloud acquisition part, a calculation processing part and a server, and the 3D point cloud acquisition system comprises a physical platform, a plurality of supporting cloud platforms and a plurality of depth sensor modules; the physical platform consists of a support frame and an upper plate; the method comprises the following steps: step 1, checking whether the system needs to be calibrated again, if so, entering step 2, and if not, entering step 3; step 2, calibrating the system; and 3, acquiring data. The invention can measure the multi-segment foot kinematic parameters of the testee which are closer to the real gait because no marker is needed; the precision of the whole system can be calibrated by using a calibration object; meanwhile, the end-to-end processing function can be realized, the acquisition efficiency is high, and the large-batch acquisition and analysis of multi-segment foot kinematic data can be supported.

Description

Multi-segment foot kinematics analysis system and method based on dynamic point cloud segmentation
Technical Field
The invention belongs to the field of dynamic point cloud sequence segmentation and the field of kinematics analysis, and particularly relates to a multi-segment foot kinematics analysis system and method based on dynamic point cloud segmentation.
Background
Biomechanical analysis of the foot during walking or other movements has been challenging for many years. As scientific evidence is not yet sufficient, the concept and paradigm of current foot function is often based on qualitative explanations. The inability to measure In Vivo Foot Kinematics (In Vivo Foot Kinematics) has been a particularly challenging problem. The human foot is a complex anatomical structure. It contains 26 bones and 33 joints, and each joint has 6 degrees of freedom (DoF), and their fine movements make various movement tasks efficient. The joint synergy in foot function is quite complex. The foot functions primarily to absorb shock and exert force, however, these two conflicting functions can be observed in a variety of athletic tasks, including the basic walking task. Therefore, modeling of the foot is a key problem in gait analysis, so as to grasp the internal motion laws of the foot, such as Pronation (progression) and Supination (Supplication), in detail. This is extremely important for assessing foot function. By analyzing the internal motion of the foot, the typical motion pattern of the pathological foot can be known, and the pathological change of the foot can be diagnosed in an auxiliary way. For example, pathological mechanisms such as special motion patterns of feet of patients with stiff toes (Hallux Rigidus), gait differences of people with Flat feet (Flat Foot) relative to people with normal arches (normalar arch), abnormal Joint mobility of feet and ankles in the gait process of Degenerative Joint Diseases (DJD) can be researched by adopting multi-segment Foot kinematics analysis.
Currently, the methods that can be used for multi-segment foot kinematics Measurement are mainly Bone Pins (Bone Pins), Fluoroscopy (fluorocopy), optical marker points and Inertial Measurement Unit (IMU). Among them, the highest accuracy is: bone screw-based methods, which are highly invasive; fluoroscopy, which is invasive and usually only quasi-static measurements can be made. The most widely used method is an optical marking point method, which has relatively high accuracy, but Soft Tissue Artifact (STA), namely deformation of Soft tissues such as skin and muscle, can cause relative movement between a cursor point attached to the skin and an internal skeleton, so that inherent measurement errors are brought, and the measurement precision is reduced; secondly, the method has subjectivity in cursor point attachment, so that the measurement result has artificial deviation; finally, the method uses expensive experimental equipment, has long experimental flow and is difficult to separate from the laboratory environment, so the data acquisition cost is high and the efficiency is low. The IMU-based method has the advantages of low cost and portability, but the system precision is difficult to calibrate, the attaching of the mark points has subjectivity, the obtained data is sparse, and the accuracy is low. The above methods all belong to (except for fluoroscopy) foot kinematics measurement methods with marks, and the use of the marks is likely to cause an influence on the gait of a subject which is difficult to ignore; meanwhile, the data acquisition efficiency of the methods is generally low, and large-scale foot kinematics data acquisition is difficult.
Disclosure of Invention
The present invention is made to solve the above problems, and an object of the present invention is to provide a multi-segment foot kinematics analysis system and method based on dynamic point cloud segmentation.
The invention provides a multi-segment foot kinematics analysis system based on dynamic point cloud segmentation, which is characterized by comprising the following steps: the 3D point cloud acquisition part comprises a physical platform for a subject to walk, a plurality of supporting cloud platforms arranged on the outer sides of the physical platform, and a plurality of depth sensors which are respectively arranged on the supporting cloud platforms and the ground and are used for acquiring 3D data; the calibration object is placed on the physical platform and used for calibrating the 3D point cloud acquisition part; the calculation processing part is connected with the depth sensor through a data transmission line and is used for receiving and processing the 3D data to obtain 3D point cloud, and then kinematic parameter data are obtained through calculation; and the server is in communication connection with the calculation processing part through a network and is used for receiving and storing the kinematic parameter data.
In the multi-segment foot kinematics analysis system based on dynamic point cloud segmentation provided by the invention, the system can also have the following characteristics: wherein, the physics platform is including placing in the support frame on ground and placing in the upper plate of support frame top, the upper plate has the ya keli board that central point put and surrounds the bakelite board of ya keli board, the quantity of supporting the cloud platform is not less than 4, every supports the cloud platform including placing in the base on ground, connect and be used for adjusting the first telescopic link of vertical direction height on the base, pass through the second telescopic link that is used for adjusting horizontal direction length that the connecting rod is connected with first telescopic link and pass through the sensor that ball twist is connected and is used for centre gripping depth sensor with the second telescopic link and draw the clamp.
In the multi-segment foot kinematics analysis system based on dynamic point cloud segmentation provided by the invention, the system can also have the following characteristics: wherein, the physical platform is embedded in the ground below, and the examinee directly walks on the ground without an upper plate.
In the multi-segment foot kinematics analysis system based on dynamic point cloud segmentation provided by the invention, the system can also have the following characteristics: wherein, depth sensor's quantity is not less than 5, when the physical platform is located subaerially, 1 at least depth sensor is fixed in the subaerial of upper plate below for obtain sole 3D data, 4 at least depth sensors draw through the sensor and press from both sides the centre gripping, be used for obtaining foot surface 3D data, inlay in subaerial time when the physical platform, 1 at least depth sensor sets up in the sunken space of physical platform below, be used for obtaining sole 3D data, 4 at least depth sensors draw through the sensor and press from both sides the centre gripping, be used for obtaining foot surface 3D data.
In the multi-segment foot kinematics analysis system based on dynamic point cloud segmentation provided by the invention, the system can also have the following characteristics: wherein the calibration object has at least 5 corner points, and each corner point is an intersection point of at least 3 mutually non-parallel planes.
In the multi-segment foot kinematics analysis system based on dynamic point cloud segmentation provided by the invention, the system can also have the following characteristics: the calculation processing part comprises an acquisition module, a fine registration module, a point cloud conversion module, a point cloud segmentation module and a kinematics analysis module, wherein the acquisition module is used for receiving 3D data and a 6D pose of a depth sensor set by a user, calculating a corresponding target transformation matrix according to the 6D pose and rendering the 3D point cloud, the fine registration module is used for calculating a final transformation matrix of each depth sensor according to the target transformation matrix, the point cloud conversion module is used for synchronizing the 3D data and converting the 3D data into a single-frame 3D point cloud, then registering the 3D point cloud of each single frame into a complete dynamic 3D point cloud, the point cloud segmentation module is used for intercepting effective data frames in the dynamic 3D point cloud and segmenting the effective data frames of each frame to obtain a multi-segment foot 3D point cloud, and the kinematics analysis module is used for calculating the kinematics of the same segment of the multi-segment foot 3D point cloud between adjacent data frames And counting the data.
In the multi-segment foot kinematics analysis system based on dynamic point cloud segmentation provided by the invention, the system can also have the following characteristics: wherein the 3D data comprises dynamic 3D data of the acquisition area walked by the subject and static 3D data of the calibration object, and the kinematic parameter data comprises relative displacement and relative rotation between the segments.
The invention also provides an analysis method adopting the multi-segment foot kinematics analysis system based on dynamic point cloud segmentation, which is characterized by comprising the following steps: step 1, checking whether a 3D point cloud acquisition part needs to be calibrated again, entering step 2 when the 3D point cloud acquisition part needs to be calibrated again, and directly entering step 3 when the 3D point cloud acquisition part does not need to be calibrated again; step 2, firstly, placing a calibration object near the center of a measurement area, inputting target 6D poses of all depth sensors to obtain target transformation matrixes corresponding to all the depth sensors, then converting 3D data acquired by the depth sensors into 3D point clouds, rendering, then adjusting the 6D poses of the depth sensors and observing the rendered 3D point clouds of the calibration object, so that a connecting line between the center of the rendered 3D point clouds of the calibration object and the origin of a global coordinate system is approximately vertical, meanwhile, the acquired 3D point clouds of the calibration object realize approximate registration, then recording static 3D data of the calibration object for several seconds by adopting an acquisition module, ensuring that each depth sensor acquires at least 1 frame of effective data, and then starting a precise registration module to calculate and obtain a final transformation matrix of each depth sensor; and 3, after the preparation of the subject is finished, acquiring dynamic 3D data of the subject in a walking and reciprocating acquisition area, synchronizing the 3D data, converting the 3D data into 3D point clouds and registering the 3D point clouds into complete dynamic 3D point clouds, then intercepting effective dynamic 3D point clouds of feet, segmenting the point clouds into multi-segment 3D point clouds of feet frame by frame, calculating kinematic parameter data of corresponding segments of all adjacent frames, storing the kinematic parameter data to the local, uploading the kinematic parameter data to a database of a server through a network, judging whether the acquisition needs to be continued, acquiring the dynamic 3D data again and entering a cycle when the acquisition needs to be performed, and closing the system if the acquisition does not need to be performed.
The analysis method adopting the multi-segment foot kinematics analysis system based on dynamic point cloud segmentation provided by the invention can also have the following characteristics: wherein, the calibration conditions of the 3D point cloud collecting part in the step 1 are as follows: after the first start or last calibration, the physical platform is moved or the relative pose of the depth sensor and the physical platform is changed.
The analysis method adopting the multi-segment foot kinematics analysis system based on dynamic point cloud segmentation provided by the invention can also have the following characteristics: wherein, the global coordinate system in step 2 is a right-hand rectangular coordinate system, which specifically comprises: using the center of the upper surface of the upper plate of the physical platform as an origin OGThe upper surface is an xz plane of a global coordinate system, the x axis is perpendicular to the long edge of the upper surface of the upper plate, the y axis is vertical upwards, and the z axis direction is determined through a right hand rule and the directions of the x axis and the y axis.
Action and Effect of the invention
According to the multi-segment foot kinematics analysis system and method based on dynamic point cloud segmentation, the 3D point cloud collection part is provided with the physical platform, so that the system can be used for a subject to walk; the 3D point cloud acquisition part is provided with the supporting cloud deck, so that the physical platform can be fixed, and a subject can walk stably; the 3D point cloud acquisition part is provided with a depth sensor, so that 3D data can be acquired; the 3D point cloud acquisition part can be calibrated due to the calibration object placed on the physical platform; the depth sensor is provided with a calculation processing part connected with the depth sensor through a data transmission line, so that 3D data can be calculated and processed, and kinematic parameter data can be obtained; since the server is connected to the calculation processing unit through network communication, the server can receive and store the kinematics parameter data.
In addition, the multi-segment foot kinematics analysis system and method based on dynamic point cloud segmentation can solve the problems that the existing multi-segment foot kinematics analysis system is difficult to realize measurement without marking points, the precision can be calibrated and the dynamic measurement is difficult to realize simultaneously, and can also solve the problems that the existing multi-segment foot kinematics analysis method depends on the marking points, the acquisition efficiency is low and the like.
Therefore, the analysis method is a label-free method, and cannot directly influence the gait of the testee, so that the multi-segment foot kinematic parameters of the testee which are closer to the real gait can be measured, and the precision of the whole system can be calibrated by using a calibration object; meanwhile, the calculation processing part can realize the end-to-end processing function, has high acquisition efficiency and can support the large-batch acquisition of multi-segment foot kinematic data.
Drawings
FIG. 1 is a system block diagram of a multi-segment foot kinematics analysis system based on dynamic point cloud segmentation in an embodiment of the invention;
FIG. 2 is a schematic structural diagram of a 3D point cloud collecting part in an embodiment of the invention;
FIG. 3 is an illustration of a 3D model of a calibration object in an embodiment of the invention;
FIG. 4 is a flow chart of a multi-segment foot kinematics analysis method based on dynamic point cloud segmentation in an embodiment of the invention;
FIG. 5 is a detailed process diagram of step 2 of the multi-segment foot kinematics analysis method based on dynamic point cloud segmentation according to the embodiment of the invention;
FIG. 6 is a diagram illustrating a detailed process of step 3 of the multi-segment foot kinematics analysis method based on dynamic point cloud segmentation according to the embodiment of the present invention;
FIG. 7 is a flowchart of the operation of the kinematic analysis module in the multi-segment foot kinematic analysis method based on dynamic point cloud segmentation in an embodiment of the present invention;
FIG. 8 is an illustration of segment segmentation in a multi-segment foot kinematics analysis method based on dynamic point cloud segmentation in an embodiment of the invention;
fig. 9 is a schematic diagram of a foot 3D point cloud local coordinate system and a segment 3D point cloud local coordinate system in an embodiment of the invention.
Detailed Description
In order to make the technical means and functions of the present invention easy to understand, the present invention is specifically described below with reference to the embodiments and the accompanying drawings.
Example (b):
as shown in fig. 1, the present embodiment provides a multi-segment foot kinematics analysis system 100 based on dynamic point cloud segmentation, including: a 3D point cloud collection unit 10, a calibration object 20, a calculation processing unit 30, and a server 40.
As shown in fig. 2, the 3D point cloud collecting unit 10 includes a physical platform 11 for a subject to walk, a plurality of supporting holders 12 disposed outside the physical platform 11, and a plurality of depth sensors 13 disposed on the supporting holders 12 and on the ground respectively for acquiring 3D data, wherein the physical platform 11 is disposed on the ground or embedded below the ground, the number of the supporting holders 12 is not less than 4, the number of the depth sensors 13 is not less than 5, and when the physical platform 11 is disposed on the ground, at least 1 depth sensor 13 is fixed on the ground below the upper plate for acquiring 3D data of sole, at least 4 depth sensors 13 are clamped by a sensor clamp 1204 for acquiring 3D data of foot surface, and when the physical platform 11 is embedded below the ground, at least 1 depth sensor 13 is disposed in a recessed space below the physical platform for acquiring 3D data of sole, at least 4 depth sensors 13 are held by a sensor pull clip 1204 for acquiring 3D data from the surface of the foot.
In this embodiment, the physical platform 11 is placed on the ground, and includes a support frame 1101 placed on the ground and an upper plate placed above the support frame 1101, the upper plate has an acrylic plate 1102 at the center position and an bakelite plate 1103 surrounding the acrylic plate, where the support frame is 2 identical stainless steel brackets, and the bakelite plate 1103 is fixed on the support frame 1101 by screws.
In this embodiment, the number of the supporting holders 12 is 4, and each supporting holder 12 includes a base 1201 placed on the ground, a first telescopic rod 1202 connected to the base 1201 through a screw thread for adjusting the height in the vertical direction, a second telescopic rod 1203 connected to the first telescopic rod 1202 through a connecting rod for adjusting the length in the horizontal direction, and a sensor pulling clamp 1204 connected to the second telescopic rod 1203 through a ball hinge and used for clamping the depth sensor 13.
In this embodiment, the number of the depth sensors 13 is 5, which are respectively marked as s0, s1, s2, s3 and s4, the depth sensors s0 to s3 are clamped by the sensor pull clamps 1204, the centers of rectangles formed by the depth sensors s0 to s3 are right above the center of the upper surface of the upper plate of the physical platform 11, and 4 sides of the rectangles are respectively parallel to 4 sides of the upper plate of the physical platform 11; the depth sensor s4 is fixed on the ground, the projection of the depth sensor s4 on the plane of the depth sensors s 0-s 3 is located at the midpoint of the connecting line of the depth sensor s0 and the depth sensor s3, the lens of the depth sensor s4 points to the center of the upper plate, and the projection of the lens on the upper plate in the direction perpendicular to the long side of the upper plate.
In this embodiment, the 3D data includes dynamic 3D data of the acquisition area and static 3D data of the calibration object, and the dynamic 3D data includes plantar 3D data and foot surface 3D data.
The calibration object 20 is placed on the physical platform 11 for calibrating the 3D point cloud collection unit 10, and the calibration object 20 can be obtained by using 3D modeling and 3D printing technology, or other technologies such as injection molding and other modes can be used, and the material is also variable, as long as the surface of the calibration object 20 can be clearly captured by the depth sensor 13, and the calibration object 20 has at least 5 corner points, and each corner point is an intersection point of at least 3 planes which are not parallel to each other.
In this embodiment, the calibration object 20 is a barrel-shaped object obtained by using 3D modeling and 3D printing techniques, and has only 1 bottom surface, and the interior is hollow, and it contains 16 corner points in total, wherein 8 corner points are intersection points of 3 non-parallel planes, and the other 8 corner points are intersection points of 4 non-parallel planes, as shown in fig. 3.
The calculation processing unit 30 is connected to the depth sensor through a data transmission line, and is configured to receive and process the 3D data to obtain a 3D point cloud, and further obtain kinematic parameter data through calculation.
In this embodiment, the calculation processing unit 30 includes an acquisition module, a fine registration module, a point cloud conversion module, a point cloud segmentation module, and a kinematics analysis module, where the acquisition module is configured to receive 3D data and a 6D pose of a depth sensor set by a user, calculate a corresponding target transformation matrix from the 6D pose, and render a 3D point cloud, the fine registration module is configured to calculate a final transformation matrix of each depth sensor according to the target transformation matrix, the point cloud conversion module is configured to synchronize the 3D data and convert the 3D data into a single-frame 3D point cloud, and then register the single-frame 3D point cloud into a complete dynamic 3D point cloud, the point cloud segmentation module is configured to intercept effective data frames in the dynamic 3D point cloud and segment each effective data frame, thereby obtaining a multi-segment foot 3D point cloud, and the kinematics analysis module is configured to calculate a motion of a same segment of the multi-segment foot 3D point cloud between adjacent data frames And (4) learning parameter data.
In this embodiment, the kinematic parameter data includes relative displacement and relative rotation between the segments.
In this embodiment, the acquisition module, the fine registration module, the point cloud conversion module, the point cloud segmentation module, and the kinematics analysis module are all corresponding programs pre-stored in the calculation processing unit 30.
The server 40 is connected to the calculation processing unit 30 through network communication, and receives and stores the kinematic parameter data.
As shown in fig. 4 to 7, the analysis method of the multi-segment foot kinematics analysis system based on dynamic point cloud segmentation of the present embodiment is as follows:
step 1, checking whether the 3D point cloud acquisition part 10 needs to be recalibrated, entering step 2 when recalibration is needed, and directly entering step 3 when recalibration is not needed.
The calibration conditions of the 3D point cloud collection unit 10 are: after the first start or last calibration, the physical platform 11 is moved or the relative pose of the depth sensor and the physical platform 11 is changed.
Step 2, firstly, placing the calibration object 20 near the center of the measurement area, then inputting the target 6D poses of all depth sensors to obtain a target transformation matrix corresponding to each depth sensor 13, then converting the 3D data acquired by the depth sensors 13 into a 3D point cloud, rendering, then adjusting the 6D poses of the depth sensors 13 and observing the rendered calibration object 3D point cloud, so that the connecting line of the center of the rendered calibration object 3D point cloud and the origin of the global coordinate system is approximately vertical, meanwhile, the acquired calibration object 3D point cloud realizes approximate registration, then, an acquisition module is adopted to record static 3D data of the calibration object for several seconds, each depth sensor is ensured to acquire at least 1 frame of effective data, and then, a precise registration module is started to calculate and obtain the final transformation matrix of each depth sensor.
In this embodiment, the valid data refers to that the 3D point cloud rendered by the data has a smooth and flat surface, and there is no obvious void on the corresponding surface of the 3D point cloud and the calibration object; the purpose of performing the fine registration is to obtain accurate transformation matrices of the depth sensors 13, so that the system accuracy after the registration is as high as possible, the fine registration is performed by taking a target transformation matrix as a coarse registration result, and the fine registration process uses standard 3D point cloud of a calibration object as a target; the final transformation matrix is a matrix that transforms 3D data recorded by the corresponding depth sensor 13 into a 3D point cloud, and then transforms the 3D point cloud into a global coordinate system.
And 3, after the preparation of the subject is finished, acquiring dynamic 3D data of the subject in a walking and reciprocating acquisition area, synchronizing the 3D data, converting the 3D data into 3D point clouds and registering the 3D point clouds into complete dynamic 3D point clouds, then intercepting effective dynamic 3D point clouds of feet, segmenting the point clouds into multi-segment 3D point clouds of feet frame by frame, calculating kinematic parameter data of corresponding segments of all adjacent frames, storing the kinematic parameter data to the local, uploading the kinematic parameter data to a database of the server 40 through a network, judging whether the acquisition needs to be continued or not, acquiring the dynamic 3D data again and entering a cycle when the acquisition needs to be performed, and closing the system if the acquisition does not need to be performed.
In this example, subject preparation refers to: the subject enters a barefoot state, exposes the lower leg, and stands on one end of the upper plate of the physical platform 11.
Further, the subject round trip acquisition region refers to: the testee walks from one end of the upper plate of the physical platform to the other end with normal gait, turns around and returns; in the forward and backward process, at least 1 complete footprint needs to be ensured on the transparent material at the center of the upper plate of the physical platform 11, i.e. the acrylic plate.
Furthermore, the multi-segment foot 3D point cloud in step 3 is defined in a local coordinate system of the foot, where the local coordinate system is a right-hand rectangular coordinate system defined as: calculating the 3D coordinate mean value of all points under the global coordinate system, and taking the mean value as the origin of the local coordinate system; calculating characteristic values and characteristic vectors of a foot 3D point cloud through Principal Component Analysis (PCA), and taking 3 characteristic vectors corresponding to the characteristic values of the first 3 in descending order; taking the minimum module value in the 3 characteristic vectors, wherein the direction of the minimum module value is the x axis; respectively calculating included angles between the remaining 2 eigenvectors and an xz plane of the global coordinate system, and taking the eigenvector corresponding to the smaller included angle, wherein the direction of the eigenvector is a z axis; the direction of the last remaining 1 eigenvector is the y-axis.
Further, each 1 frame in the effective dynamic 3D point cloud of the foot in step 3 includes the following feature points: medial malleolus MmLateral malleolus MlProximal talus TpProximal scaphoid bone NpDistal calcaneus Cd5 th proximal metatarsal Mtp55 th metatarsal head Mth51 st metatarsal head Mth1Root of great toe Hb. The definition of the feature points under the local coordinate system of the foot 3D point cloud is as follows: mmDefined as the point in the medial malleolus bulge region where the value of x is minimal; mlThe point in the lateral malleolus bulge region where the value of x is the greatest; t ispIs MmAnd MlThe connecting line translates towards the positive direction of the z axis and is in point cloud with the foot in 3DA tangent point at the outer side; n is a radical ofpIs MmThe point of maximum curvature in the raised area closest to the heel, between the sole and the foot; mtp5Is the outside of the foot, CdThe point with the maximum curvature in the convex area with the minimum z value is towards the positive direction of the z axis; mth5The point with the maximum curvature in the convex area with the maximum z value is the negative direction of the lateral side of the foot and the far end of the 5 th toe to the z axis; mth1The point with the maximum curvature in the convex area with the maximum z value is the negative direction of the inner side of the foot and the far end of the big toe to the z axis; hbIs the saddle point in the heel area of the big toe.
Further, the multi-segment foot 3D point cloud in step 3 contains 5 segments: shank, hind foot, middle foot, forefoot, and big toe. Under the local coordinate system of the foot 3D point cloud, the boundary of the section is defined by the characteristic points: shank-Mm、Ml、TpCan determine 1 plane p0The 3D point cloud on the upper part of the p0, namely the positive side of the y axis of the global coordinate system, is the shank; hind foot-3D Point cloud with shank removed, Tp、NpDetermining 1 plane p1Making p1 perpendicular to xz plane of foot 3D point cloud local coordinate system, and dividing part of 3D point cloud containing heel after p1 as hindfoot; middle foot-3D point cloud for removing shank and hind foot, line segment is taken
Figure BDA0003089560850000141
Midpoint H ofNM1With HNM1、Mtp5Determining 1 plane p2Let p be2Xz plane, p, perpendicular to the foot 3D point cloud local coordinate system2The part which does not contain toes after being divided is the midfoot; forefoot-3D Point cloud with shank, hindfoot, midfoot removed, in Mth1、Mth5Determining 1 plane p3Let p be3Xz plane, p, perpendicular to the foot 3D point cloud local coordinate system3The part which does not contain toes after being divided is the midfoot; thumb-3D Point cloud with Calf, hindfoot, midfoot, forefoot removed, at Mth1Determining 1 plane p4Let p be4Parallel to the xy plane of the foot 3D point cloud local coordinate system, and then HbDetermining 1 plane p5Let p be5Local sitting parallel to foot 3D point cloudYz plane of the system, p4And p5The part including the big toe after being divided is the big toe.
Further, the kinematic parameters include relative displacement and relative rotation between the segments. The relative displacement and relative rotation between the segments is defined in the local coordinate system of the segments, which is a right-hand rectangular coordinate system defined as: calculating the 3D coordinate mean value of all points under the global coordinate system, and taking the mean value as the origin of the local coordinate system; calculating the eigenvalue and eigenvector of segment 3D point cloud by PCA, and taking the eigenvalue lambda of 3 before descending order0、λ1、λ2Corresponding 3 eigenvectors xi0、ξ1、ξ2(ii) a Xi is reduced0、ξ1、ξ2Respectively normalized to unit feature vector
Figure BDA0003089560850000142
Respectively calculate
Figure BDA0003089560850000143
3 unit basis vectors of local coordinate system of 3D point cloud of feet
Figure BDA0003089560850000144
Dot product between; taking and
Figure BDA0003089560850000145
the unit characteristic vector with the largest dot product, wherein the direction of the unit characteristic vector is the x axis; in the remaining 2 unit feature vectors, and
Figure BDA0003089560850000146
the direction of the unit feature vector with the largest dot product is the z axis; the direction of the last remaining 1 unit feature vector is the y-axis.
Further, the kinematic parameters in step 3 include relative displacement and relative rotation between the segments. Is expressed in terms of euler angles, which combine The local coordinate system of The segments and The Standardization and terminology committee of The International Society of Biomechanics (ISB)y Committee, STC) was calculated: dorsiflexion (rotation angle is positive), plantarflexion (rotation angle is negative) are positive (medial-lateral direction) rotation around x-axis, abduction (rotation angle is positive), adduction (rotation angle is negative) are positive (vertical direction) rotation around y-axis, eversion (rotation angle is positive), and inversion (rotation angle is negative) are positive rotation around z-axis. Wherein, the calculation of the kinematic parameters among 2 segments can be converted into 2 6D poses Pose0And Pose1Relative displacement and relative rotation between, i.e. Pose0And Pose1Relative displacement and relative rotation between the represented local coordinate systems; by o0And
Figure BDA0003089560850000151
denotes Pose0Local coordinate system of, using1And
Figure BDA0003089560850000152
denotes Pose1The local coordinate system of (1), then Pose1Relative to Pose0Relative displacement t and relative rotation theta ofx、θy、θzThe calculation method of (2) is as follows.
Figure BDA0003089560850000153
t＝o1-o0
Figure BDA0003089560850000154
Figure BDA0003089560850000155
Figure BDA0003089560850000156
As shown in fig. 8, according to the multi-segment foot 3D point cloud partitioning of the embodiment, 5 segments can be obtained: shank, hind foot, middle foot, forefoot, and big toe. Fig. (a) and (b) are respectively a plot of the division results for 2 different viewing angles, each viewing angle containing 5 segments, illustrated in grayscale, and a segmented forefoot 3D point cloud, illustrated in black.
As shown in fig. 9, according to the local coordinate system calculation method of the embodiment, a foot 3D point cloud local coordinate system and a segment 3D point cloud local coordinate system can be obtained. The local coordinate system of the entire foot is shown, denoted x _ local, y _ local, z _ local, respectively, and the local coordinate system of the midfoot, denoted x _ mf, y _ mf, z _ mf, respectively.
Effects and effects of the embodiments
According to the multi-segment foot kinematics analysis system and method based on dynamic point cloud segmentation, the 3D point cloud collection part is provided with the physical platform, so that the system and method can be used for a subject to walk; the 3D point cloud acquisition part is provided with the supporting cloud deck, so that the physical platform can be fixed, and a subject can walk stably; the 3D point cloud acquisition part is provided with a depth sensor, so that 3D data can be acquired; the 3D point cloud acquisition part can be calibrated due to the calibration object placed on the physical platform; the depth sensor is provided with a calculation processing part connected with the depth sensor through a data transmission line, so that 3D data can be calculated and processed, and kinematic parameter data can be obtained; since the server is connected to the calculation processing unit through network communication, the server can receive and store the kinematics parameter data.
Further, the upper plate of the physical platform of the embodiment has the transparent acrylic plate at the center and the opaque bakelite plate surrounding the acrylic plate, so that the lower depth sensor can conveniently acquire the 3D data of the sole.
Further, the supporting platform of the embodiment has the first telescopic rod, the second telescopic rod and the sensor pull clamp of the clamping degree sensor, so that the position of the depth sensor can be changed.
Further, the calibration object of the present embodiment has at least 5 corner points, and each corner point is an intersection point of at least 3 mutually nonparallel planes, so that the registration accuracy of the fine registration can be improved, and the misalignment of the registration result can be reduced.
In summary, the multi-segment foot kinematics analysis system and method based on dynamic point cloud segmentation in the embodiment can solve the problems that the existing multi-segment foot kinematics analysis system is difficult to realize measurement without mark points, can calibrate precision and dynamically measure, and can also solve the problems that the existing multi-segment foot kinematics analysis method depends on mark points, is low in acquisition efficiency and the like.
Therefore, the analysis method of the embodiment is a label-free method, and does not directly influence the gait of the subject, so that the multi-segment foot kinematic parameters of the subject closer to the real gait can be measured, and the precision of the whole system can be calibrated by using a calibration object; meanwhile, the calculation processing part can realize the end-to-end processing function, has high acquisition efficiency and can support the large-batch acquisition of multi-segment foot kinematic data.
The above embodiments are preferred examples of the present invention, and are not intended to limit the scope of the present invention.
For example, in the above embodiments, the physical platform includes a support frame placed on the ground and an upper plate placed above the support frame, the upper plate has a transparent plate at the center and an opaque plate surrounding the transparent plate, so that the physical platform can be used for the examinee to walk, and the depth sensor can conveniently acquire the 3D data of the sole of the foot, but in the present invention, the physical platform can also be embedded into the ground, at this time, the upper plate is directly replaced by the ground, the depth sensor on the original ground is placed in the underground concave space, but it is still required that the examinee has at least 1 complete foot print on the transparent material during the reciprocating process, in addition, the bakelite plate can also be replaced by other opaque materials with good strength and rigidity, such as aluminum alloy plate, transparent acrylic plate can also be replaced by other materials with good light transmittance, such as tempered glass, and the physical platform can also be used for the examinee to walk, meanwhile, the depth sensor is convenient to acquire the 3D data of the sole.
Further, in the above-described embodiment, the number of the depth sensors is 5 and the number of the support heads is 4, but in the present invention, the number of the depth sensors and the number of the support heads may be increased.
In addition, in the above embodiments, the support platform includes a base, a first telescopic rod for adjusting the height in the vertical direction, a second telescopic rod for adjusting the length in the horizontal direction, and a sensor clamp for clamping the depth sensor, so that the position of the depth sensor can be adjusted, thereby facilitating data acquisition.

Claims (10)

1. A multi-segment foot kinematics analysis system based on dynamic point cloud segmentation, comprising:
the 3D point cloud acquisition part comprises a physical platform for a subject to walk, a plurality of supporting cloud platforms arranged on the outer sides of the physical platform, and a plurality of depth sensors which are respectively arranged on the supporting cloud platforms and the ground and are used for acquiring 3D data;
the calibration object is placed on the physical platform and used for calibrating the 3D point cloud acquisition part;
the calculation processing part is connected with the depth sensor through a data transmission line and is used for receiving and processing the 3D data to obtain 3D point cloud, and then kinematic parameter data are obtained through calculation; and
and the server is in communication connection with the calculation processing part through a network and is used for receiving and storing the kinematic parameter data.
2. The multi-segment foot kinematics analysis system according to claim 1, wherein:
wherein the physical platform comprises a support frame placed on the ground and an upper plate placed above the support frame,
the upper plate has a transparent plate at a central position and an opaque plate surrounding the transparent plate,
the number of the supporting cloud platforms is not less than 4, and each supporting cloud platform comprises a base placed on the ground, a first telescopic rod connected to the base and used for adjusting the height in the vertical direction, a second telescopic rod connected with the first telescopic rod through a connecting rod and used for adjusting the length in the horizontal direction, and a sensor pull clamp connected with the second telescopic rod through a ball hinge and used for clamping the depth sensor.
3. The multi-segment foot kinematics analysis system according to claim 1, wherein:
wherein, the physical platform is embedded in the ground below, and the examinee directly walks on the ground without an upper plate.
4. The multi-segment foot kinematics analysis system according to claim 1, wherein:
wherein the number of the depth sensors is not less than 5,
when the physical platform is positioned on the ground, at least 1 depth sensor is fixed on the ground below the upper plate and used for acquiring 3D data of the sole, at least 4 depth sensors are clamped by the sensor pull clamps and used for acquiring 3D data of the surface of the foot,
when the physics platform inlays in the below ground, 1 at least depth sensor set up in the sunken space of physics platform below for acquire sole 3D data, 4 at least depth sensor draws through the sensor and presss from both sides the centre gripping, is used for acquiring foot surface 3D data.
5. The multi-segment foot kinematics analysis system according to claim 1, wherein:
wherein the calibration object has at least 5 corner points, and each of the corner points is an intersection of at least 3 mutually non-parallel planes.
6. The multi-segment foot kinematics analysis system according to claim 1, wherein:
wherein the calculation processing part comprises an acquisition module, a fine registration module, a point cloud conversion module, a point cloud segmentation module and a kinematics analysis module,
the acquisition module is used for receiving the 3D data and the 6D pose of the depth sensor set by a user, calculating a corresponding target transformation matrix according to the 6D pose and rendering the 3D point cloud,
the fine registration module is used for calculating a final transformation matrix of each depth sensor according to the target transformation matrix,
the point cloud conversion module is used for synchronizing the 3D data, converting the 3D data into single-frame 3D point clouds, registering the 3D point clouds of the single frames into complete dynamic 3D point clouds,
the point cloud segmentation module is used for intercepting effective data frames in the dynamic 3D point cloud and segmenting the effective data frames of each frame so as to obtain a multi-segment foot 3D point cloud,
the kinematic analysis module is used for calculating kinematic parameter data of the same segment of the multi-segment foot 3D point cloud between adjacent data frames.
7. The multi-segment foot kinematics analysis system according to claim 1, wherein:
wherein the 3D data comprises dynamic 3D data of the subject walking to and from the acquisition area and static 3D data of the calibration object,
the kinematic parameter data includes relative displacement and relative rotation between segments.
8. An analysis method using the multi-segment foot kinematics analysis system based on dynamic point cloud segmentation according to claim 1, comprising the steps of:
step 1, checking whether the 3D point cloud acquisition part needs to be calibrated again, entering step 2 when the 3D point cloud acquisition part needs to be calibrated again, and directly entering step 3 when the 3D point cloud acquisition part does not need to be calibrated again;
step 2, firstly, a calibration object is placed near the center of a measurement area, then the 6D poses of the targets of all the depth sensors are input to obtain a target transformation matrix corresponding to each depth sensor, then converting the 3D data collected by the depth sensor into a 3D point cloud and rendering the 3D point cloud, then adjusting the 6D pose of the depth sensor and observing the rendered calibration object 3D point cloud, so that the connection line of the center of the rendered calibration object 3D point cloud and the origin of the global coordinate system is approximately vertical, and the acquired calibration object 3D point cloud realizes approximate registration, then, an acquisition module is adopted to record static 3D data of the calibration object for a plurality of seconds, each depth sensor is ensured to acquire at least 1 frame of effective data, then starting a fine registration module to calculate and obtain a final transformation matrix of each depth sensor;
and 3, after the preparation of the subject is finished, acquiring dynamic 3D data of a region acquired by the subject walking back and forth, synchronizing the 3D data, converting the 3D data into 3D point clouds and registering the 3D point clouds into complete dynamic 3D point clouds, then intercepting effective dynamic 3D point clouds of feet, segmenting the point clouds into multi-segment 3D point clouds of feet frame by frame, calculating kinematic parameter data of corresponding segments of all adjacent frames, storing the kinematic parameter data to the local, uploading the kinematic parameter data to a database of the server through a network, judging whether the acquisition is needed to be continued, re-acquiring the dynamic 3D data and entering a cycle when the acquisition is needed, and closing the system if the acquisition is not needed.
9. The analysis method using a multi-segment foot kinematics analysis system based on dynamic point cloud segmentation according to claim 8, wherein:
wherein, the calibration conditions of the 3D point cloud collecting part in the step 1 are as follows: after the first start or last calibration, the physical platform is moved or the relative pose of the depth sensor and the physical platform is changed.
10. The analysis method using a multi-segment foot kinematics analysis system based on dynamic point cloud segmentation according to claim 8, wherein:
wherein, the global coordinate system in the step 2 is a right-hand rectangular coordinate system, which specifically includes: using a center of an upper surface of the upper plate of the physical platform as an origin OGThe upper surface is an xz plane of a global coordinate system, the x axis is perpendicular to the long edge of the upper surface of the upper plate, the y axis is vertical upwards, and the z axis direction is determined through a right hand rule and the directions of the x axis and the y axis.
CN202110591048.4A 2021-05-28 2021-05-28 Multi-segment foot kinematics analysis system and method based on dynamic point cloud segmentation Pending CN113180645A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110591048.4A CN113180645A (en) 2021-05-28 2021-05-28 Multi-segment foot kinematics analysis system and method based on dynamic point cloud segmentation

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110591048.4A CN113180645A (en) 2021-05-28 2021-05-28 Multi-segment foot kinematics analysis system and method based on dynamic point cloud segmentation

Publications (1)

Publication Number Publication Date
CN113180645A true CN113180645A (en) 2021-07-30

Family

ID=76986395

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110591048.4A Pending CN113180645A (en) 2021-05-28 2021-05-28 Multi-segment foot kinematics analysis system and method based on dynamic point cloud segmentation

Country Status (1)

Country Link
CN (1) CN113180645A (en)

Similar Documents

Publication Publication Date Title
KR101902551B1 (en) Malalignment syndrome diagnosis apparatus based on plantar pressure and body movement and method thereof
Leardini et al. A new anatomically based protocol for gait analysis in children
CN102018527B (en) Vertebral column three-dimensional dynamic measurement and analysis system and method
CN111358471A (en) Body posture detection device and detection method
CN108186021A (en) A kind of gait zero-speed detection method and system based on multimodal information fusion
CN201879707U (en) Detector for three-dimensional parameters of foot under different loads
Leardini et al. ISB recommendations for skin-marker-based multi-segment foot kinematics
Kandasamy et al. Posture and back shape measurement tools: A narrative
Bisi et al. Anatomical calibration for wearable motion capture systems: Video calibrated anatomical system technique
CN217365854U (en) Point cloud collection device of multi-segment foot kinematics analysis system
CN113180645A (en) Multi-segment foot kinematics analysis system and method based on dynamic point cloud segmentation
Schallig et al. The Amsterdam Foot Model: a clinically informed multi-segment foot model developed to minimize measurement errors in foot kinematics
CN115778333B (en) Method and device for visually positioning pulse acupoints on cun, guan and chi
Newton et al. Three-dimensional quantification of human standing posture
EP2178441A1 (en) Radiographic imaging method and apparatus
Lidstone et al. Concurrent validity of an automated footprint detection algorithm to measure plantar contact area during walking
CN113712535A (en) Ankle form comprehensive information acquisition system and method
RU75146U1 (en) COMPLEX FOR DIAGNOSTIC OF FUNCTIONAL STATUS OF STOP AND IDENTIFICATION OF THE PATHOLOGY OF THEIR DEFORMATION AT MASS SCREENING EXAMINATIONS
KR20090059095A (en) Method for three-dimensional biomechanical data and parameter analysis and apparatus using the same method
CN115018977A (en) Semi-automatic registration method based on biplane X-ray and joint three-dimensional motion solving algorithm
Gastaldi et al. Technical challenges using magneto-inertial sensors for gait analysis
Varga et al. Reliability of measuring morphology of the paediatric foot using the Artec Eva hand held scanner
Trujillo-Hernández et al. Development of an integrated podometry system for mechanical load measurement and visual inspection
CA2595167A1 (en) Method for calibrating sensor positions in a human movement measurement and analysis system
RU2705232C1 (en) Method and device for diagnosing and correcting locomotor dysfunction

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination