CN116524124A - Spinal three-dimensional dynamic reconstruction method and system - Google Patents
Spinal three-dimensional dynamic reconstruction method and system Download PDFInfo
- Publication number
- CN116524124A CN116524124A CN202310462721.3A CN202310462721A CN116524124A CN 116524124 A CN116524124 A CN 116524124A CN 202310462721 A CN202310462721 A CN 202310462721A CN 116524124 A CN116524124 A CN 116524124A
- Authority
- CN
- China
- Prior art keywords
- spine
- joint
- point
- corr
- pelvis
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 47
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 22
- 210000000988 bone and bone Anatomy 0.000 claims abstract description 20
- 238000004458 analytical method Methods 0.000 claims abstract description 15
- 238000001914 filtration Methods 0.000 claims abstract description 13
- 238000004364 calculation method Methods 0.000 claims abstract description 9
- 210000004197 pelvis Anatomy 0.000 claims description 57
- 238000012937 correction Methods 0.000 claims description 46
- 210000000115 thoracic cavity Anatomy 0.000 claims description 31
- 238000012545 processing Methods 0.000 claims description 28
- 230000006870 function Effects 0.000 claims description 20
- 101150060298 add2 gene Proteins 0.000 claims description 18
- 210000004394 hip joint Anatomy 0.000 claims description 16
- 101150005267 Add1 gene Proteins 0.000 claims description 13
- 238000005259 measurement Methods 0.000 claims description 13
- 239000011436 cob Substances 0.000 claims description 9
- 238000013461 design Methods 0.000 claims description 8
- 238000009499 grossing Methods 0.000 claims description 7
- 230000003068 static effect Effects 0.000 claims description 7
- 101100108191 Vibrio parahaemolyticus serotype O3:K6 (strain RIMD 2210633) add gene Proteins 0.000 claims description 6
- 238000005457 optimization Methods 0.000 claims description 5
- 238000002474 experimental method Methods 0.000 claims description 4
- 230000005540 biological transmission Effects 0.000 claims description 3
- 238000012360 testing method Methods 0.000 claims description 3
- 210000001624 hip Anatomy 0.000 claims 4
- 210000000038 chest Anatomy 0.000 claims 1
- 206010039722 scoliosis Diseases 0.000 abstract description 17
- 230000008901 benefit Effects 0.000 abstract description 5
- 230000005865 ionizing radiation Effects 0.000 abstract description 5
- 238000012216 screening Methods 0.000 abstract description 4
- 238000012549 training Methods 0.000 abstract description 4
- 238000011156 evaluation Methods 0.000 abstract description 3
- 230000036961 partial effect Effects 0.000 abstract description 2
- 230000000007 visual effect Effects 0.000 abstract description 2
- 238000005516 engineering process Methods 0.000 description 13
- 238000010586 diagram Methods 0.000 description 11
- 238000004590 computer program Methods 0.000 description 7
- 238000011282 treatment Methods 0.000 description 7
- 230000033001 locomotion Effects 0.000 description 6
- 239000011159 matrix material Substances 0.000 description 5
- 230000008569 process Effects 0.000 description 5
- 230000005855 radiation Effects 0.000 description 5
- 238000002591 computed tomography Methods 0.000 description 4
- 238000003384 imaging method Methods 0.000 description 4
- 210000001503 joint Anatomy 0.000 description 4
- 210000004705 lumbosacral region Anatomy 0.000 description 4
- 230000002829 reductive effect Effects 0.000 description 4
- 230000007704 transition Effects 0.000 description 4
- 230000002159 abnormal effect Effects 0.000 description 3
- 238000005452 bending Methods 0.000 description 3
- 230000008859 change Effects 0.000 description 3
- 238000003759 clinical diagnosis Methods 0.000 description 3
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 3
- 238000002595 magnetic resonance imaging Methods 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 238000003860 storage Methods 0.000 description 3
- 238000012800 visualization Methods 0.000 description 3
- 206010023509 Kyphosis Diseases 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- 238000003745 diagnosis Methods 0.000 description 2
- 201000010099 disease Diseases 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 230000000670 limiting effect Effects 0.000 description 2
- 210000000323 shoulder joint Anatomy 0.000 description 2
- 206010010356 Congenital anomaly Diseases 0.000 description 1
- 208000007623 Lordosis Diseases 0.000 description 1
- 206010058907 Spinal deformity Diseases 0.000 description 1
- 208000020307 Spinal disease Diseases 0.000 description 1
- 230000009471 action Effects 0.000 description 1
- 229940060587 alpha e Drugs 0.000 description 1
- 230000004075 alteration Effects 0.000 description 1
- 210000003484 anatomy Anatomy 0.000 description 1
- 238000000429 assembly Methods 0.000 description 1
- 230000000712 assembly Effects 0.000 description 1
- 208000035475 disorder Diseases 0.000 description 1
- 239000000428 dust Substances 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 210000002310 elbow joint Anatomy 0.000 description 1
- 238000007667 floating Methods 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 230000005484 gravity Effects 0.000 description 1
- 210000002414 leg Anatomy 0.000 description 1
- 230000003902 lesion Effects 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000005312 nonlinear dynamic Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000036285 pathological change Effects 0.000 description 1
- 231100000915 pathological change Toxicity 0.000 description 1
- 230000001575 pathological effect Effects 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000012827 research and development Methods 0.000 description 1
- 230000000284 resting effect Effects 0.000 description 1
- 239000000523 sample Substances 0.000 description 1
- 210000004872 soft tissue Anatomy 0.000 description 1
- 239000000126 substance Substances 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 238000011269 treatment regimen Methods 0.000 description 1
- 210000000689 upper leg Anatomy 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H20/00—ICT specially adapted for therapies or health-improving plans, e.g. for handling prescriptions, for steering therapy or for monitoring patient compliance
- G16H20/30—ICT specially adapted for therapies or health-improving plans, e.g. for handling prescriptions, for steering therapy or for monitoring patient compliance relating to physical therapies or activities, e.g. physiotherapy, acupressure or exercising
Abstract
The invention provides a three-dimensional dynamic spine reconstruction method and system, which are characterized in that firstly, the human body tracking function of Azure Kinect is integrated in Unity software, partial joint points and additional points are selected as control points, the control points are corrected through the relative positions among the joint points, the joint point data are smoothed by adopting a Kalman filtering algorithm based on bone length constraint, then, a spine curve is generated by interpolation fitting of the control points by adopting Catmull-Rom spline, a general 3D spine model is established and registered on the fitted spine line to form an automatically calibrated personalized 3D spine model, and the spine parameter analysis and calculation are performed. The invention has the advantages of establishing a real-time visual three-dimensional dynamic model of the spine and accurately evaluating the related parameters of the spine, having no any ionizing radiation, low cost, convenience and easiness in deployment, and being suitable for scoliosis screening and spine rehabilitation training evaluation scenes.
Description
Technical Field
The invention belongs to the technical field of medical auxiliary equipment, and particularly relates to a three-dimensional dynamic spine reconstruction method and system.
Background
Three-dimensional dynamic spine reconstruction is an important medical technology for three-dimensional reconstruction of the human spine and tracking the motion thereof in real time. The technology has wide application prospect in diagnosis and treatment of spinal diseases, can provide detailed three-dimensional anatomical structure information of the spinal column, and helps doctors to accurately position lesion positions and formulate reasonable treatment schemes. However, the existing three-dimensional spine reconstruction technology still has some defects, which limit the application of the technology in clinical practice.
The three-dimensional spine reconstruction technology is mainly based on the following five aspects: firstly, based on the traditional two-dimensional X-ray film, the alignment and fusion of the biplane X-ray film are carried out by shooting multi-angle X-ray films and carrying out image processing so as to generate a three-dimensional model of the spine. The technique can visualize the shape and structure of the spine, but the use of X-ray photography can cause the patient and medical staff to suffer from radiation hazard, and the two-dimensional X-ray film can not provide complete three-dimensional space information; second, based on Computed Tomography (CT), a large number of cross-sectional images are generated using a CT scanner, and then the images are combined into a three-dimensional model using computer software. The technology can provide higher image resolution and can display complete three-dimensional structure information of the spine, but has the defects of larger radiation quantity which is hundreds of times of X-rays; thirdly, based on Magnetic Resonance Imaging (MRI), which is an imaging technology based on magnetic fields and radio waves, detailed spine soft tissue structure images can be generated, but the resolution of the MRI images is lower and the imaging time is longer; and fourthly, based on a three-dimensional laser scanner, the technology captures the back indication information of the human body through the laser scanner or a depth camera, and obtains a three-dimensional spine model by utilizing complex data processing and image processing. The technology can actually and safely reconstruct the spine three-dimensionally without radiation, but due to the limited imaging depth of the laser scanning technology and the individual difference of people, the established three-dimensional model can not completely reflect the true deep structure of the spine, and the technology only visualizes the static three-dimensional spine. Fifth, grating three-dimensional imaging is represented by a DIERS spine analysis system, the system is a commercial spine dynamic reconstruction system, three-dimensional dynamic data of a spine are obtained and posture analysis is carried out by the system through a structured light projection and three-dimensional vision technology, but the system can only carry out spine assessment of small-range motion in a dark environment, is not applicable to complicated spine diseases and conditions of large-range motion or rapid change, and has a certain limitation in clinical application because of high price.
In summary, the existing three-dimensional spine reconstruction technology mainly has the problems of radiation hazard, low reliability, high cost and poor universality, so that the low-cost three-dimensional spine dynamic reconstruction system which is safe, reliable and convenient to use is researched and has great application value.
The applicant has previously filed a chinese patent for a method and system for CN 202211600511.8-spine 3D modeling, which primarily solves the above-mentioned problems, but the applicant has found in further research that there is room for further adjustment and improvement in this design.
Disclosure of Invention
Therefore, aiming at the actual improvement requirement of the prior art, the invention provides a three-dimensional dynamic spine reconstruction method and system, wherein firstly, the human body tracking function of an AzureKinect sensor is integrated in Unity software, the coordinate data of original articulation points are captured, partial articulation points and additional points added in design are selected as control points, the control points are corrected through the relative positions among the articulation points, and the control point data is smoothed by adopting a Kalman filtering algorithm based on bone length constraint; secondly, interpolating the control points by adopting Catmull-Rom splines to obtain a fitted spine curve, establishing a general three-dimensional spine model in a Unity scene, and registering the general three-dimensional spine model on the fitted spine curve to obtain a personalized three-dimensional dynamic spine model; finally, automatically calculating the Cobb angle, the pelvic tilt angle, the trunk unbalance angle and the thoracic and lumbar vertebra posterior lobe and the trunk tilt angle of the sagittal plane of the human body according to the curvature analysis algorithm of the fit spinal line and the position analysis algorithm of the joint points. The invention has the advantages of establishing a real-time visual three-dimensional dynamic model of the spine and accurately evaluating the related parameters of the spine, having no any ionizing radiation, low cost, convenience and easiness in deployment, and being suitable for scoliosis screening and spine rehabilitation training evaluation scenes.
The technical scheme adopted for solving the technical problems is as follows:
a three-dimensional dynamic spine reconstruction method, which is characterized in that: firstly integrating the human body tracking function of AzureKinect in Unity software, selecting part of articulation points and additional points as control points, correcting the control points through the relative positions among the articulation points, smoothing the articulation point data by adopting a Kalman filtering algorithm based on bone length constraint, then interpolating and fitting the control points by adopting Catmull-Rom spline to generate a spine curve, establishing a general 3D spine model, registering the spine model on the fitted spine line to form an automatically calibrated personalized 3D spine model, and performing spine parameter analysis and calculation.
Further, 6 control points of Neck, spineChest, spineNaval, pelvis, add1 and Add2 are adopted to act to generate a spinal curve; the method comprises the following steps of taking Neck, spineChest, spineNaval and Pelvis as basic control points, and taking the vertical leg from a Neck joint point to a connection line of the LeftShoulder and the RightShoulder as an additional control point Add1; a ray passing through the Pelvis node and oriented in the direction perpendicular to the line connecting the LeftHip and the RightHip is taken, and a point is taken on the ray, so that the distance from the point to the Pelvis node is 50% of the distance from the LeftHip node to the RightHip node, and the point is taken as an additional control point Add2.
Further, the correction of the control point is specifically:
let the three-dimensional coordinates of the neg joint be: neck x ，Neck y ，Neck z The three-dimensional coordinates of the Add1 point are: add1 x ，Add1 y ，Add1 z The three-dimensional coordinates of the spinehest articulation point are: spineCHest x ，SpineChest y ，SpineChest z ；
Calculating to obtain a correction factor 1:
corr 1 ＝[(Neck x -Add1 x )(Neck y -SpineChest y )]/(Neck y -Add1 y )
let the height of the subject be h, then correction factor 2 be:
corr 2 ＝|1-(Neck y -SpineChest y )/(0.3h)|,corr 2 ∈[0,1]
coordinate spineConst of spineCongest joint in X-axis direction x Applying correction factor 1 and correction factor 2 yields:
SpineChest x ＝SpineChest x +corr 1 (1+corr 2 )
further, correction factor 3 is obtained:
corr 3 ＝[(Neck z -Add1 z )(Neck y -SpineChest y )]/(Neck y -Add1 y )
coordinate spineCHest of spineCHest joint point in Z-axis direction z Applying correction factor 3 yields:
SpineChest z ＝SpineChest z +corr 3
the final three-dimensional coordinates obtained after the spineCHest joint point is corrected are as follows: spineCHest x +corr 1 (1+corr 3 )，SpineChest y ，SpineChest z +corr 2 ；
Let the three-dimensional coordinates of the spinenval joint be: spinenval x ，SpineNaval y ，SpineNaval z The three-dimensional coordinates of the Pelvis joint are: pelvis (Pelvis) x ，Pelvis y ，Pelvis z The three-dimensional coordinates of the Add2 point are: add2 x ，Add2 y ，Add2 z ；
Obtaining correction factor 4:
corr 4 ＝[(Pelvis x -Add2 x )(Pelvis y -SpineNaval y )]/(Pelvis y -Add2 y )
the correction factor 5 is:
corr 5 ＝|1-(SpineNaval y -Pelvis y )/(0.2h)|,corr 5 ∈[0,1]
for the coordinate spinenval of the spinenval joint point in the X-axis direction x Correction is performed by applying correction factor 4 and correction factor 5 to obtain:
SpineNaval x ＝SpineNaval x +corr 4 (1+corr 5 )
and calculating to obtain a correction factor 6:
corr 6 ＝[(Pelvis z -Add2 z )(Pelvis y -SpineNaval y )]/(Pelvis y -Add2 y )
for the coordinate spinenval of the spinenval joint point in the Z-axis direction z Applying correction factor 6 yields:
SpineNaval z ＝SpineNaval z +corr 6
the final three-dimensional coordinates obtained after the spinenval node is corrected are as follows: spinenval x +corr 4 (1+corr 5 )，SpineNaval y ，SpineNaval z +corr 6 。
Further, the specific method for generating the control point comprises the following steps: the Kalman filtering based on the bone length constraint specifically comprises the following steps:
for the four nodes of LeftShoulder, rightShoulder, leftHip, rightHip,
let the expected coordinates of the node point be B (x, y, z), and the predicted value obtained by applying the Kalman filter beIts parent joint coordinates are A (x 1 ,y 1 ,z 1 ) The length of the corresponding skeleton of the joint obtained by static measurement experiments is L, and the joint point B is on a spherical surface with the radius L taking the father joint A as the center, so as to establish a bone length constraint equation:
(x-x 1 ) 2 +(y-y 1 ) 2 +(z-z 1 ) 2 ＝L 2
establishing a spatial linear equation between the node A and the point P:
and solving a simultaneous constraint equation and a space linear equation to obtain:
and selecting a solution of the point with the smallest distance from the point P as the optimized joint coordinate.
Further, for the control points obtained after the data processing, a Catmull-Rom spline function is generated by adopting a Catmull-Rom interpolation algorithm to serve as a fitted spine curve, and the Catmull-Rom spline line obtained by interpolation of the control points is used for representing a central contour curve of the spine so as to reconstruct a spine vertebra model in a three-dimensional manner.
Further, the parameter analysis calculation of the spine includes: calculating and evaluating coronal plane parameters and sagittal plane parameters; the coronal parameters include Cobb angle, pelvic tilt angle, and torso imbalance angle; the sagittal plane parameters include the posterior thoracic lobe, anterior lumbar lobe, and torso tilt angle.
Further, the method for estimating the pelvic tilt angle comprises the following steps: let the three-dimensional coordinates of the right hip joint point be: rightHip x ，RightHip y ，RightHip z The three-dimensional coordinates of the left hip joint point are: leftHip x ，LeftHip y ，LeftHip z The method comprises the steps of carrying out a first treatment on the surface of the The pelvis tilt angle is calculated by the relative positions of the left hip joint and the right hip joint obtained after data processing:
PelvicTilt＝arctan((RightHip y -LeftHip y )/(RightHip x -LeftHip x ))；
the trunk unbalanced angle is calculated through the relative positions of the neck joint and the pelvis joint obtained after data processing:
TrunkImbalance＝arctan((Pelvic x -Neck x )/(Pelvic y -Neck y ))。
further, a posterior thoracic lobe (TK) is defined as an included angle between an upper end vertebra of the fourth thoracic vertebra (T4) and a tangent line of a lower end vertebra of the twelfth thoracic vertebra (T12), and an anterior Lumbar Lobe (LL) is defined as an included angle between an upper end vertebra of the first lumbar vertebra (L1) and a tangent line of an upper end vertebra of the coccyx (S1);
calculating the slope k of the position of the T4 center point in the established 3D spine model through projection interpolation fitted spine curve on the sagittal plane 4 Slope k of the position of the T12 center point 5 Slope k of the location of the L1 center point 6 Slope k of the location of the L5 center point 7 Resulting in a thoracic posterior lobe (TK) and a lumbar anterior lobe (LL):
further, the torso tilt angle is obtained by calculating the relative positions of the cervical and pelvic nodes:
TrunkInclination＝arctan((Pelvic x -Neck x )/(Pelvic y -Neck y ))。
and a system for three-dimensional dynamic reconstruction of the spine, characterized by: the three-dimensional dynamic spine reconstruction method comprises the following steps:
the system comprises a data capturing module, a data processing optimization module and a Unity scene modeling module;
the data capturing module is used for configuring an AzureKinect use environment in Unity, calling built-in functions in AzureKinectSDK and AzureKinectBodyTrackingSDK, and inputting joint data captured by the sensor into Unity in a queue form;
the data optimization processing module is used for correcting the original control points and the additional control points added by design, and then smoothing the original control points and the additional control points by using a Kalman filtering algorithm based on bone length constraint to reduce errors, so as to generate control points for interpolation fitting;
the Unity scene modeling module is used for carrying out Catmull-Rom spline interpolation on control points obtained by data processing to obtain a fitted spine curve, registering and aligning an imported 3D universal spine model with the fitted spine curve to obtain a personalized 3D spine model, evaluating parameters of the fitted spine curve through an analysis algorithm of a sagittal plane and a coronal plane of a human body, automatically calculating spine parameters, and finally refreshing the Unity scene and continuously updating according to data transmission of AzureKinect.
The invention and the preferable scheme thereof can be applied to the fields of scoliosis screening and spine rehabilitation evaluation training, have the advantages of no ionizing radiation, high precision, automatic calculation of spine parameters, spine three-dimensional dynamic visualization, low cost, convenient use and the like, and can replace the existing spine three-dimensional modeling method adopting X-ray films, CT, MRI, a three-dimensional laser scanner, a grating three-dimensional imager and the like. Has the following advantages and uses:
1. no ionizing radiation. The AzureKinect depth camera is used for measuring depth information based on the principle of time difference ranging (TOF) of infrared light, does not have any ionizing radiation to a body, and can detect repeatedly.
2. The algorithm accuracy is high. The high-precision depth sensor AzureKinect is adopted to capture the joint points, the correction coefficient is calculated through the relative positions among the joint points to correct, and the Kalman filtering algorithm constrained by bone length is adopted to reduce data errors, so that the result is accurate and reliable, and the method is suitable for scoliosis screening and detecting scenes.
3. Realizing three-dimensional dynamic visualization of the spine. The three-dimensional structure and the movement condition of the spine can be intuitively presented, so that doctors and patients can intuitively know the movement and the deformation of the spine, and the pathological condition can be presumed, thereby better assisting in diagnosis and treatment planning, and being applicable to assisting in clinical diagnosis of spine diseases and rehabilitation training guidance scenes.
4. Spinal parameters are automatically calculated. Parameters such as the spine Cobb angle, the pelvis inclination angle, the trunk inclination angle and the like are automatically measured through an algorithm and software, the accuracy and the efficiency of measurement are improved, the manual measurement error is reduced, and the method is suitable for guiding clinical diagnosis scenes.
5. Low cost and convenient use. The invention only needs one AzureKinect depth sensor, a camera tripod and a common notebook computer, and has lower cost compared with a system using equipment such as a three-dimensional ultrasonic probe, a three-dimensional scanner and the like; and the angle parameters can be automatically calculated and the 3D spine visualization can be realized by only installing the Unity software in the notebook computer and importing the project package, building the tripod and connecting the sensor to the computer running system, so that the manual complicated operation is not needed, the product deployment performance is strong, and the universality is realized.
Drawings
The invention is described in further detail below with reference to the attached drawings and detailed description:
FIG. 1 is a flow chart of a method and system for three-dimensional dynamic reconstruction of the spine in accordance with an embodiment of the present invention;
FIG. 2 is a flowchart of a method for detecting and processing a joint point according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of AzureKinect nodes and additional points according to an embodiment of the present invention;
FIG. 4 is a schematic diagram of a bone length constraint model according to an embodiment of the present invention;
FIG. 5 is a schematic view of a local coordinate system of a vertebral model according to an embodiment of the present invention;
FIG. 6 is a schematic diagram of coronal plane parameter calculation according to an embodiment of the present invention: wherein: (a) spinal parameters, (b) fitting spinal lines;
FIG. 7 is a schematic diagram of a sagittal plane parameter calculation in accordance with an embodiment of the present invention;
FIG. 8 is a block diagram of a method and system for three-dimensional dynamic reconstruction of the spine in accordance with an embodiment of the present invention.
Detailed Description
In order to make the features and advantages of the present patent more comprehensible, embodiments accompanied with figures are described in detail below:
it should be noted that the following detailed description is illustrative and is intended to provide further explanation of the present application. Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs.
It is noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of example embodiments in accordance with the present application. As used herein, the singular is also intended to include the plural unless the context clearly indicates otherwise, and furthermore, it is to be understood that the terms "comprises" and/or "comprising" when used in this specification are taken to specify the presence of stated features, steps, operations, devices, components, and/or combinations thereof.
The invention solves the problems of radiation hazard, low reliability and high cost and poor universality existing in the prior art, and the specific technical scheme is as follows: the method comprises the steps of selecting a low-cost and non-radiative depth camera AzureKinect as a research and development basis, integrating a human body tracking function of the AzureKinect in Unity software, selecting part of joint points and designing additional points added as control points, correcting the control points through relative positions among the joint points, smoothing joint point data by adopting a Kalman filtering algorithm based on bone length constraint, performing interpolation fitting on the control points by adopting Catmull-Rom spline to generate a spine curve, establishing a general 3D spine model, registering the spine model on the fitted spine line to form an automatically calibrated personalized 3D spine model, and finally performing spine parameter analysis calculation on the three-dimensional spine model. The flow chart is shown in fig. 1.
1. Control point generation and preprocessing
And configuring an AzureKinect environment in Unity, initializing a human body tracking function, arranging original joint point data into a queue for input according to the form of an AzureKinect data stream, generating control points according to the original joint points and joint points added in design to accurately position important nodes of the spine, and processing the obtained control point data by adopting a Kalman filtering algorithm based on bone length constraint to improve the precision, wherein a flow chart is shown in figure 2.
1.1 control Point Generation
The AzureKinect human body tracking automatic tracking joint points are shown in fig. 3, and four points related to the spine are selected: neck, spineChest, spineNaval, pelvis, as a basic control point, but there may be some abnormal curvature of the human spine, and the basic control point is insufficient to accurately restore the true morphology of the spine, so additional control points need to be added to improve the accuracy of spine morphology recognition.
Considering the structural stability of the front half section of the thoracic vertebra of the spine, the connecting line of the thoracic vertebra is perpendicular to the connecting line of the left and right shoulder joints, and the characteristics of the shoulder joints and the Neck joints are obvious and the recognition rate is high, so that the drop foot from the Neck joint point to the connecting lines of the LeftShoulder and the RightShoulder is used as an additional control point Add1. The control point is formed by the combined action of three joint points, and can reflect the transverse deviation characteristic of the thoracic spine of the spine when a scoliosis subject with high and low shoulder characteristics is tested. The lumbar structure of the spine is more unstable than the thoracic structure, the inclination of the Pelvis causes lateral deflection of the lumbar spine, and since the posterior half of the lumbar spine is perpendicular to the line connecting the Left and right iliac crest, a ray is made from the Pelvis joint point and in the direction perpendicular to the line connecting the LeftHip and the RightHip toward the Y axis, a point is taken on the ray, the distance from the Pelvis joint point is 50% of the distance from the Left Hip joint point to the RightHip joint point, and the point is taken as an additional control point Add2. The control point corrects the position of the control point in real time through the relative position of the two hip joint points, and can feed back the change of the pelvis height under the motion condition and the transverse offset characteristic of the lumbar vertebra of the spine caused by congenital pelvic inclination. In summary, the invention adopts 6 control points of Neck, spineChest, spineNaval, pelvis, add1 and Add2 to act to generate the spine curve, which can restore the true shape of the spine more accurately, improve the recognition and tracking precision of the spine posture, and further be better applied to the technical field of medical auxiliary equipment.
1.2 control Point correction
The spinecast and SpineNaval joint points are approximate positions of the thoracic and lumbar spine segments estimated by an internal algorithm of the AzureKinect sensor relative to joint points such as Neck, leftShoulder and rightholder, and the like, and the joint points are not explicitly corresponding to the human joint positions, however, the spinecast and SpineNaval joint points are critical to the overall structure of the spine, so that correction is required.
Because the estimated positions of the Neck and Add1 joint points are relatively accurate, the spineCongest node is corrected by the relative positions of these two points and the spineCongest node. Let the three-dimensional coordinates of the neg joint be: neck x ，Neck y ，Neck z The three-dimensional coordinates of the Add1 point are: add1 x ，Add1 y ，Add1 z spineCHest switchThe three-dimensional coordinates of the nodes are: spineCHest x ，SpineChest y ，SpineChest z 。
In the coronal plane of the human body, the correction factor 1 is calculated by the relative positions of the three articulation points because the high-low shoulder characteristics of the scoliosis subject can cause the lateral offset of the thoracic vertebrae of the spine:
corr 1 ＝[(Neck x -Add1 x )(Neck y -SpineChest y )]/(Neck y -Add1 y )
for subjects with severe scoliosis, correction according to the above method may not be sufficient to accurately describe the change, as excessive spinal deformity may result in an increase in lateral offset distance of the thoracic spine. By analyzing the human body structure, the height of the backbone thoracic vertebrae of a normal person is about 0.3 times of the height, and the backbone bending of a patient with scoliosis leads to the transverse deflection of the backbone thoracic vertebrae, so that the longitudinal height of the backbone is reduced. Thus, assuming the height of the subject is h, the correction factor 2 is:
corr 2 ＝|1-(Neck y -SpineChest y )/(0.3h)|,corr 2 ∈[0,1]
coordinate spineConst of spineCongest joint in X-axis direction x Applying correction factor 1 and correction factor 2 yields:
SpineChest x ＝SpineChest x +corr 1 (1+corr 2 )
in the sagittal plane of the human body, since there is thoracic kyphosis in the spine, and AzureKinect human body tracking ignores this, correction factor 3 is obtained by calculating the relative positions of the three joints as well:
corr 3 ＝[(Neck z -Add1 z )(Neck y -SpineChest y )]/(Neck y -Add1 y )
coordinate spineCHest of spineCHest joint point in Z-axis direction z Applying correction factor 3 yields:
SpineChest z ＝SpineChest z +corr 3
SpineChethe final three-dimensional coordinates obtained after the st joint point is corrected are as follows: spineCHest x +corr 1 (1+corr 3 )，SpineChest y ，SpineChest z +corr 2 。
Similarly, let the three-dimensional coordinates of the spinenval joint be: spinenval x ，SpineNaval y ，SpineNaval z The three-dimensional coordinates of the Pelvis joint are: pelvis (Pelvis) x ，Pelvis y ，Pelvis z The three-dimensional coordinates of the Add2 point are: add2 x ，Add2 y ，Add2 z 。
In the coronal plane of the human body, since the pelvic tilt feature of scoliosis patients may cause lateral offset of the lumbar spine, correction factor 4 is obtained by calculating the relative positions of the three above-mentioned articulation points:
corr 4 ＝[(Pelvis x -Add2 x )(Pelvis y -SpineNaval y )]/(Pelvis y -Add2 y )
for a patient with severe scoliosis, the transverse offset distance of the spine lumbar vertebra is increased due to excessive deformation of the spine, the spine lumbar vertebra height of a normal person is about 0.2 times of the height through analysis of human body structures, and the longitudinal height of the spine is reduced due to the transverse offset of the spine lumbar vertebra caused by spine bending of the patient with scoliosis. Thus, correction factor 5 is:
corr 5 ＝|1-(SpineNaval y -Pelvis y )/(0.2h)|,corr 5 ∈[0,1]
coordinate spinenval of spinenval node in X-axis direction x The correction is performed by applying correction factor 4 and correction factor 5 to obtain the following components:
SpineNaval x ＝SpineNaval x +corr 4 (1+corr 5 )
in the sagittal plane of the human body, since lumbar lordosis exists in the spine, correction factor 6 is obtained by calculating the relative positions of the three nodes of Pelvis, add2 and SpineNaval as described above:
corr 6 ＝[(Pelvis z -Add2 z )(Pelvis y -SpineNaval y )]/(Pelvis y -Add2 y )
coordinate spinenval of spinenval node in Z-axis direction z Applying correction factor 6 yields:
SpineNaval z ＝SpineNaval z +corr 6
the final three-dimensional coordinates obtained after the spinenval node is corrected are as follows: spinenval x +corr 4 (1+corr 5 )，SpineNaval y ，SpineNaval z +corr 6 。
1.3 Kalman Filter based on bone Length constraints
Although AzureKinect's human body tracking has achieved higher accuracy, it is necessary to process the joint points using a smoothing filter algorithm because its depth camera is based on the time of flight ranging (TOF) principle, which is inevitably affected by dust, light and obstruction in the environment, so that errors may occur in the joint points.
First, noise of data is smoothed using a kalman filter. Taking the state vector quantity as the real 3D coordinate and speed of the skeleton joint, and expressing the state vector quantity as the real 3D coordinate and speed of the skeleton jointThe measurement vector is taken as the real three-dimensional coordinates of the skeleton joint and the sound source angle, expressed as y= [ x, Y, z, arctan (x/z)] T . The state transition process is modeled as a linear dynamic system and the measurement is modeled as a nonlinear dynamic system, where the next state of time instance k+1 is represented by the previous state of the kth instance and is mathematically represented as:
X k+1 ＝FX k +Q k
Y k ＝h(X k )+R k
wherein X is k+1 And Y k State vector and measurement vector, respectively, at time instant k, Q k And R is k Process noise and measurement noise, respectively, F is the state transition matrix and h is the state transition function.
The state transition matrix F is as follows:
linearizing H by adopting an extended Kalman filter, replacing a matrix H in the filter with a jacobian matrix of H, and calculating under the current state estimation as follows:
kalman filter with measurement vector knowledge Y k X of (2) k EstimationThe method is divided into two steps of prediction and updating, and the standard Kalman filtering prediction step can be written as follows:
wherein the method comprises the steps ofIs +.>Is associated with the prediction of (c) and is expressed as:
the updated state based on the measurement is expressed as:
wherein K is k Is a kalman gain matrix. Kalman filter order estimationAnd true X k The mean square error between them is minimized, providing smoother coordinates.
Second, from the analysis of the upper section, leftShoulder, rightShoulder, leftHip, rightHip four joints are critical to the generation of control points, and in practice the bone length of the human body is constant, and this error can be reduced by limiting the distance between the two joints. Therefore, the invention carries out static measurement experiments on the lengths of the left upper arm, the right upper arm and the left thigh, and continuously collects the data with smaller fluctuation in multi-frame data to calculate the average value under the condition that the subject keeps the upright posture for a plurality of seconds so as to establish the bone length model.
Taking right shoulder rightholder as an example, let the expected coordinates of the rightholder node be B (x, y, z), and the predicted value obtained after applying the kalman filter beThe parent joint right elbow joint coordinates are a (x 1 ,y 1 ,z 1 ) The length of the upper arm obtained by static measurement experiment is L. As shown in fig. 4, the node B should establish a bone length constraint equation on a sphere of radius L centered on its parent joint a:
(x-x 1 ) 2 +(y-y 1 ) 2 +(z-z 1 ) 2 ＝L 2
establishing a spatial linear equation between the node A and the point P:
and solving a simultaneous constraint equation and a space linear equation to obtain:
and selecting a solution of the point with the smallest distance from the point P as the optimized RightShoulder joint coordinate, and performing secondary treatment on the other three joint points. According to the method, the independent joint point information estimated by the AzureKinect is combined through the rigid constraint of the bone length of the human body, so that the estimated human body structure is more complete, and the position of the spine can be reflected more truly and accurately.
2. Interpolation fitting of spinal curves
The biggest difference between the Catmull-Rom spline and the Bessel spline is that each of its control points is located on the spline, which allows each vertebra to move and rotate more accurately according to the transformation of the control points when the spinal model is registered to the Catmull-Rom spline. Therefore, for the control points obtained after the data processing, a Catmull-Rom spline function is generated by adopting a Catmull-Rom interpolation algorithm to serve as a fitted spine curve.
Catmull-Rom spline interpolation is essentially a piecewise polynomial interpolation, with four points per segment respectively named P i-2 、P i-1 、P i 、P i+1 Then according to the coordinates of four points, at P i-1 、P i Constructing a cubic polynomial curve between each two to divide eachThe spline curve of the segment is spliced to construct a complete continuous curve. Cubic polynomial for each segment:
P(t)＝C 0 +C 1 α+C 2 α 2 +C 3 α 3
wherein: floating point coordinates alpha E [0,1 ]]Spline function coefficient C 0 ～C 3 From the constraint of P (t) at endpoints α=0, α=1:
where T is called a shape factor, and the interpolation fitting effect is optimal when T is set to 0.5. Then, the coefficient C is obtained from 8 0 ～C 3 Then substituting into 7 to obtain interpolation point P i-1 、P i The Catmull-Rom curve equation between:
3. generic 3D spinal model creation and registration
3.1 general 3D spinal model creation
In order to judge the scoliosis condition more intuitively, a Catmull-Rom spline obtained by interpolation of control points is used for representing a central profile curve of the spine so as to reconstruct a spine vertebra model in three dimensions. The structure of the spine is complex and consists of 7 cervical vertebrae, 12 thoracic vertebrae, 5 lumbar vertebrae, sacrum and coccyx. According to studies, scoliosis is shown to occur mainly in the thoracic (T1-T12) and lumbar (L1-L5) vertebrae.
Thus, a 3D model including 17 vertebrae in total of thoracic and lumbar vertebrae was pre-built according to the geometric proportion of vertebrae counted in the existing literature, and the built model was imported into Unity software. Wherein the relative percent heights of the center point height of each vertebra relative to the spinal column minus the disc height are shown in table 1.
Table 1 vertebral height relative percentages
In practice, intervertebral disc spaces exist between vertebrae of the human spinal column, and studies have shown that the height between all of the intervertebral discs is approximately 25% -30% of the total spinal column length. Thus, the disc height should be considered in creating a generic 3D spinal model, and the percentage of the height between the thoracic and lumbar discs is calculated from literature as shown in table 2.
Table 2 disc height percent
The actual percent height for each vertebra is calculated in combination with consideration of the vertebral height and the disc height as shown in table 3.
Table 3 actual vertebral height percentages
3.2 general 3D spinal model registration
(1) Vertebral model position registration
After the universal vertebral models are imported into Unity, a local coordinate system shown in fig. 5 is established for each vertebral model, the axes of the coordinate system are positioned at the center of the vertebrae (the junction of the vertebral body and the pedicle), the left is a sagittal plane view, the right is a coronal plane view, and the Z axis and the Y axis are respectively perpendicular and parallel to the upper plane of the vertebrae.
In the three-dimensional space, the position of an object can be fixed by determining the coordinate axes of the object and the directions of two coordinate axes. The proportional relationship of each vertebra in Table 3 relative to the entire spinal length was calculated based on the vertebra model parameters in tables 1 and 2 to obtain the percent position of each vertebra on the fitted spinal curve. The axes of each vertebral model are fixed to points at a percentage of the corresponding fitted spinal line, and the Z-axis direction is set to be tangential to the points on the fitted spinal line, and the Y-axis direction is set to be normal to the points on the fitted spinal line and toward the camera plane, thereby registering each vertebra to the fitted spinal line.
(2) Automatically scaling vertebral models
The length of the spine line corresponding to the established general spine model is denoted as L normal The length of the fitted spine curve calculated for each test of the different subjects is denoted as L, and the scaling factor of the vertebral model is denoted as:
ScaleFactor＝L normal ÷L
the obtained scaling factors are applied to the scaling assemblies of each vertebra model, so that the vertebra models can be automatically scaled according to the fitting spinal curve characteristics of different people, and finally the personalized 3D vertebra model is obtained.
4. Spinal parameter assessment
4.1 coronal plane parameters
The coronal plane of the human body refers to a plane perpendicular to the anterior-posterior direction and parallel to the ground, and the coronal plane parameters of the spine are as shown in fig. 6, and mainly comprise three parts of Cobb angle, pelvic tilt angle (pelvicTilt), and trunk imbalance angle (trunk imbalance).
4.1.1Cobb Angle
The Cobb angle of the spine is the most commonly used measurement index for judging the scoliosis degree at present, and is defined as the maximum included angle between the upper edge plane of the upper end vertebra and the lower edge plane of the lower end vertebra of the human coronal plane spine. According to the method, the Cobb angle is calculated according to the generated fitting spinal curve, so that the position with the maximum bending degree of the spinal line can be automatically positioned, and meanwhile, the influence caused by abnormal rotation of a single vertebra in the traditional method is avoided. Because the positions of the upper and lower vertebrae are the points with the largest gradient in the spine curve, and the slope is the extreme point of the slope of the curve, the extreme point of the slope can be obtained by calculating the point with zero second derivative of the fitted spine curve, so as to position the positions of the upper and lower vertebrae.
According to the invention, the Catmull-Rom spline line is adopted to fit the spine curve, and as the Catmull-Rom spline function has C2 continuity, namely the second derivative function is continuous, the slope can be calculated by solving the first derivative of the Catmull-Rom spline function according to 9:
P′(α)＝(-0.5P i-2 +0.5P i )+(2P i-2 -5P i-1 +4P i -P i+1 )α
+(-1.5P i-2 +4.5P i-1 -4.5P+1.5P i+1 )α 2
calculating extreme points by solving second derivative:
P″(α)＝(2P i-2 -5P i-1 +4P- i P i+1 )+(-3P i-2 +9P i-1 -9P+3P i+1 )α
obtaining extreme point (x) of the slope of the coronal plane 1 ,z 1 )、(x 2 ,z 2 )、(x 3 ,z 3 ) And the slope k corresponding to these points 1 、k 2 、k 3 . Cobb angle size is calculated according to equation 13:
/>
the largest of the two angles is defined as a scoliosis Cobb angle, and the degree of coronal curvature is evaluated and judged according to the angle.
4.1.2 pelvic tilt angle
Pelvic tilt refers to the angle of inclination of the pelvic bowl in the anterior-posterior direction, an important feature of scoliosis patients. The resting pelvic tilt angle is typically within 2 degrees and varies periodically with small amplitude, typically between 5 and 8 degrees, in the walking state, beyond which there may be a risk of scoliosis. Therefore, the invention estimates the human pelvis inclination angle under static and dynamic conditions and has clinical application reference value.
Let the three-dimensional coordinates of the right hip joint point be: rightHip x ，RightHip y ，RightHip z The three-dimensional coordinates of the left hip joint point are: leftHip x ，LeftHip y ，LeftHip z 。
The pelvis tilt angle is calculated by the relative positions of the left hip joint and the right hip joint obtained after data processing:
PelvicTilt＝arctan((RightHip y -LeftHip y )/(RightHip x -LeftHip x ))
4.1.3 torso imbalance angles
The coronal torso imbalance angle of a person is the right and left inclination angle of the torso with respect to the gravity line when standing or sitting. The coronary face trunk unbalanced angle is an index for measuring the trunk posture, can be used for evaluating the balance state of the trunk in the aspect of tilting left and right, helps doctors to know whether the posture of the spine of a patient has the left and right inclination of the spine, and has important significance for clinical diagnosis and treatment decision. According to the invention, the body unbalanced angle is obtained through calculating the relative positions of the neck joint and the pelvis joint obtained after data processing:
TrunkImbalance＝arctan((Pelvic x -Neck x )/(Pelvic y -Neck y ))
4.2 sagittal plane parameters
The sagittal plane of the human body refers to a plane that divides the human body in left and right halves along the anterior-posterior direction, and the sagittal parameters evaluated by the present invention are shown in fig. 7, and include the thoracic posterior lobe (thoracic kyphosis, TK), lumbar anterior lobe (LL), and torso tilt angle (Trunk Inclination).
4.2.1 thoracic and lumbar anterior lobes
The posterior thoracic lobe (TK) is defined as the included angle between the upper end vertebra of the fourth thoracic vertebra (T4) and the tangent line of the lower end vertebra of the twelfth thoracic vertebra (T12), and the anterior Lumbar Lobe (LL) is defined as the included angle between the upper end vertebra of the first lumbar vertebra (L1) and the tangent line of the upper end vertebra of the coccyx (S1). Assessing the posterior and anterior lumbar lobes helps determine the physiological status of the spine and whether there is a deformity or pathological change, thereby aiding in the selection of clinical treatment regimens and management of spinal-related disorders.
The invention calculates the slope k of the position of the T4 center point in the established 3D spine model by projecting the interpolation fitted spine curve on the sagittal plane 4 Slope k of the position of the T12 center point 5 Slope k of the location of the L1 center point 6 Slope k of the location of the L5 center point 7 TK and LL are obtained:
4.2.2 torso Tilt Angle
The trunk inclination angle of the spine refers to the inclination angle of the trunk of a human body on a front-back plane relative to a vertical line, whether the spine is abnormally bent front and back can be determined by measuring the trunk inclination angle, whether the abnormal condition can cause the problems of gesture pain and the like is analyzed, and corresponding correction treatment is found and carried out early. The invention obtains the trunk inclination angle by calculating the relative positions of the neck joint and the pelvis joint:
TrunkInclination＝arctan((Pelvic x -Neck x )/(Pelvic y -Neck y ))
5. system design
The system design of the invention mainly comprises three parts, namely a data capturing module, a data processing optimizing module and a Unity scene modeling module, as shown in fig. 8.
The data capturing module is used for configuring an AzureKinect use environment in the Unity, calling built-in functions in the AzureKinectSDK and the AzureKinectBodyTrackingSDK and inputting joint data captured by the sensor into the Unity in a queue mode. The data optimization processing module corrects the original control points and the additional control points added by design, and then uses a Kalman filtering algorithm based on bone length constraint to carry out smoothing processing so as to reduce errors, so that control points for interpolation fitting are generated. And the Unity scene modeling module carries out Catmull-Rom spline interpolation according to control points obtained by data processing to obtain a fitted spine curve, registers and aligns the imported 3D universal spine model with the fitted spine curve to obtain a personalized 3D spine model, evaluates parameters of the fitted spine curve through an analysis algorithm of a sagittal plane and a coronal plane of a human body, automatically calculates spine parameters, and finally refreshes the Unity scene and continuously updates the spine curve according to data transmission of AzureKinect.
The application process of the product of the embodiment is specifically as follows:
1. horizontally placing a camera tripod and adjusting the camera tripod to a height of 1.4-1.6 meters, and installing an AzureKinect sensor;
2. the subject stands upright and faces the sensor, and the distance is 1.0-2.0 meters;
3. starting Unity software, opening 3D spine modeling and system projects, clicking a play button to run a system, enabling a subject to stand upright for about 5 seconds, collecting static data of the subject, exporting the data to a database, and clicking the play button again to finish collection;
4. clicking a play button, so that the subject can freely walk, and the system can monitor the three-dimensional shape of the spine and evaluate the angle parameters in real time;
and clicking the play button again to finish the system, thereby completing the acquisition.
It will be appreciated by those skilled in the art that embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
The above description is only a preferred embodiment of the present invention, and is not intended to limit the invention in any way, and any person skilled in the art may make modifications or alterations to the disclosed technical content to the equivalent embodiments. However, any simple modification, equivalent variation and variation of the above embodiments according to the technical substance of the present invention still fall within the protection scope of the technical solution of the present invention.
The present patent is not limited to the above-mentioned best mode, any person can obtain other various types of three-dimensional dynamic reconstruction methods and systems for the spine under the teaching of the present patent, and all equivalent changes and modifications made according to the claims of the present application shall be covered by the present patent.
Claims (10)
1. A three-dimensional dynamic spine reconstruction method, which is characterized in that: firstly integrating the human body tracking function of Azure Kinect in Unity software, selecting part of joint points and additional points as control points, correcting the control points through the relative positions among the joint points, smoothing the joint point data by adopting a Kalman filtering algorithm based on bone length constraint, then interpolating and fitting the control points by adopting Catmull-Rom spline to generate a spine curve, establishing a general 3D spine model, registering the spine model on the fitted spine line to form an automatically calibrated personalized 3D spine model, and performing spine parameter analysis and calculation.
2. The method for three-dimensional dynamic reconstruction of the spine according to claim 1, wherein: the Spine curve is generated by adopting 6 control points of Neck, spine test, spine Naval, pelvis, add1 and Add 2; the method comprises the steps of taking Neck, spine test, spine Naval and Pelvis as basic control points, and taking the foot of a connection line from a Neck joint point to Left holder and Right holder as an additional control point Add1; and (3) taking a ray which passes through the Pelvis joint point and is oriented to the Y axis in the direction perpendicular to the connection line of the Left Hip and the Right Hip, and taking a point on the ray, wherein the distance from the point to the Pelvis joint point is 50% of the distance from the Left Hip joint point to the Right Hip joint point, and the point is taken as an additional control point Add2.
3. The method for three-dimensional dynamic reconstruction of the spine according to claim 2, wherein: the correction of the control point is specifically:
let the three-dimensional coordinates of the neg joint be: neck x ，Neck y ，Neck z The three-dimensional coordinates of the Add1 point are: add1 x ，Add1 y ，Add1 z The three-dimensional coordinates of the Spine Chest articulation point are: spineCHest x ，SpineChest y ，SpineChest z ；
Calculating to obtain a correction factor 1:
corr 1 ＝[(Neck x -Add1 x )(Neck y -SpineChest y )]/(Neck y -Add1 y )
let the height of the subject be h, then correction factor 2 be:
corr 2 ＝|1-(Neck y -SpineChest y )/(0.3h)|,corr 2 ∈[0,1]
coordinate spineConst of spineCongest joint in X-axis direction x Applying correction factor 1 and correction factor 2 yields:
SpineChest x ＝SpineChest x +corr 1 (1+corr 2 )
further, correction factor 3 is obtained:
corr 3 ＝[(Neck z -Add1 z )(Neck y -SpineChest y )]/(Neck y -Add1 y )
coordinate spineCHest of spineCHest joint point in Z-axis direction z Applying correction factor 3 yields:
SpineChest z ＝SpineChest z +corr 3
the final three-dimensional coordinates obtained after the spineCHest joint point is corrected are as follows: spineCHest x +corr 1 (1+corr 3 )，SpineChest y ，SpineChest z +corr 2 ；
Let the three-dimensional coordinates of the spinenval joint be: spinenval x ，SpineNaval y ，SpineNaval z The three-dimensional coordinates of the Pelvis joint are: pelvis (Pelvis) x ，Pelvis y ，Pelvis z The three-dimensional coordinates of the Add2 point are: add2 x ，Add2 y ，Add2 z ；
Obtaining correction factor 4:
corr 4 ＝[(Pelvis x -Add2 x )(Pelvis y -SpineNaval y )]/(Pelvis y -Add2 y )
the correction factor 5 is:
corr 5 ＝|1-(SpineNaval y -Pelvis y )/(0.2h)|,corr 5 ∈[0,1]
coordinate spinenval of spinenval node in X-axis direction x Correction is performed by applying correction factor 4 and correction factor 5 to obtain:
SpineNaval x ＝SpineNaval x +corr 4 (1+corr 5 )
and calculating to obtain a correction factor 6:
corr 6 ＝[(Pelvis z -Add2 z )(Pelvis y -SpineNaval y )]/(Pelvis y -Add2 y )
coordinate spinenval of spinenval node in Z-axis direction z Applying correction factor 6 yields:
SpineNaval z ＝SpineNaval z +corr 6
the final three-dimensional coordinates obtained after the spinenval node is corrected are as follows: spinenval x +corr 4 (1+corr 5 )，SpineNaval y ，SpineNaval z +corr 6 。
4. The method for three-dimensional dynamic reconstruction of the spine according to claim 1, wherein: the specific method for generating the control point comprises the following steps: the Kalman filtering based on the bone length constraint specifically comprises the following steps:
for the four nodes Left hand holder, right hand holder, left Hip, right Hip,
let the expected coordinates of the node point be B (x, y, z), and the predicted value obtained by applying the Kalman filter beIts parent joint coordinates are A (x 1 ,y 1 ,z 1 ) The length of the corresponding skeleton of the joint obtained by static measurement experiments is L, and the joint point B is on a spherical surface with the radius L taking the father joint A as the center, so as to establish a bone length constraint equation:
(x-x 1 ) 2 +(y-y 1 ) 2 +(z-z 1 ) 2 ＝L 2
establishing a spatial linear equation between the node A and the point P:
and solving a simultaneous constraint equation and a space linear equation to obtain:
and selecting a solution of the point with the smallest distance from the point P as the optimized joint coordinate.
5. The method for three-dimensional dynamic reconstruction of the spine according to claim 4, wherein: and for the control points obtained after the data processing, generating a Catmull-Rom spline function by adopting a Catmull-Rom interpolation algorithm to serve as a fitted spine curve, and adopting the Catmull-Rom spline line obtained by interpolation of the control points to represent the central contour curve of the spine so as to carry out three-dimensional reconstruction on the spine vertebra model.
6. The method for three-dimensional dynamic reconstruction of the spine according to claim 5, wherein: the parameter analysis and calculation of the spine comprises: calculating and evaluating coronal plane parameters and sagittal plane parameters; the coronal parameters include Cobb angle, pelvic tilt angle, and torso imbalance angle; the sagittal plane parameters include the posterior thoracic lobe, anterior lumbar lobe, and torso tilt angle.
7. The method for three-dimensional dynamic reconstruction of the spine according to claim 6, wherein:
the pelvic tilt angle estimation method comprises the following steps: let the three-dimensional coordinates of the right hip joint point be: rightHip x ，
RightHip y ，RightHip z The three-dimensional coordinates of the left hip joint point are: leftHip x ，LeftHip y ，LeftHip z ；
The pelvis tilt angle is calculated by the relative positions of the left hip joint and the right hip joint obtained after data processing:
PelvicTilt＝arctan((RightHip y -LeftHip y )/(RightHip x -LeftHip x ))；
the trunk unbalanced angle is calculated through the relative positions of the neck joint and the pelvis joint obtained after data processing:
TrunkImbalance＝arctan((Pelvic x -Neck x )/(Pelvic y -Neck y ))。
8. the method for three-dimensional dynamic reconstruction of the spine according to claim 6, wherein:
the posterior thoracic lobe (TK) is defined as an included angle between the upper end vertebra of the fourth thoracic vertebra (T4) and the tangent line of the lower end vertebra of the twelfth thoracic vertebra (T12), and the anterior Lumbar Lobe (LL) is defined as an included angle between the upper end vertebra of the first lumbar vertebra (L1) and the tangent line of the upper end vertebra of the coccyx (S1);
calculating the slope k of the position of the T4 center point in the established 3D spine model through projection interpolation fitted spine curve on the sagittal plane 4 Slope k of the position of the T12 center point 5 Slope k of the location of the L1 center point 6 Slope k of the location of the L5 center point 7 Resulting in a thoracic posterior lobe (TK) and a lumbar anterior lobe (LL):
9. the method for three-dimensional dynamic reconstruction of the spine according to claim 6, wherein:
the torso tilt angle is obtained by calculating the relative positions of the cervical and pelvic nodes:
TrunkInclination＝arctan((Pelvic x -Neck x )/(Pelvic y -Neck y ))。
10. a system for three-dimensional dynamic reconstruction of the spine, characterized by: the method of three-dimensional dynamic reconstruction of the spine according to any one of claims 1-9, comprising:
the system comprises a data capturing module, a data processing optimization module and a Unity scene modeling module;
the data capturing module is used for configuring an Azure Kinect use environment in the Unity, calling built-in functions in the Azure Kinect SDK and Azure Kinect BodyTracking SDK, and inputting the joint data captured by the sensor into the Unity in a queue;
the data optimization processing module is used for correcting the original control points and the additional control points added by design, and then smoothing the original control points and the additional control points by using a Kalman filtering algorithm based on bone length constraint to reduce errors, so as to generate control points for interpolation fitting;
the Unity scene modeling module is used for carrying out Catmull-Rom spline interpolation on control points obtained by data processing to obtain a fitted spine curve, registering and aligning an imported 3D universal spine model with the fitted spine curve to obtain a personalized 3D spine model, evaluating parameters of the fitted spine curve through an analysis algorithm of a sagittal plane and a coronal plane of a human body, automatically calculating spine parameters, and finally refreshing the Unity scene and continuously updating according to data transmission of Azure Kinect.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310462721.3A CN116524124A (en) | 2023-04-26 | 2023-04-26 | Spinal three-dimensional dynamic reconstruction method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310462721.3A CN116524124A (en) | 2023-04-26 | 2023-04-26 | Spinal three-dimensional dynamic reconstruction method and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116524124A true CN116524124A (en) | 2023-08-01 |
Family
ID=87395404
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310462721.3A Pending CN116524124A (en) | 2023-04-26 | 2023-04-26 | Spinal three-dimensional dynamic reconstruction method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116524124A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117338240A (en) * | 2023-09-15 | 2024-01-05 | 国家体育总局运动医学研究所 | Intelligent spine function detection device |
-
2023
- 2023-04-26 CN CN202310462721.3A patent/CN116524124A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117338240A (en) * | 2023-09-15 | 2024-01-05 | 国家体育总局运动医学研究所 | Intelligent spine function detection device |
| CN117338240B (en) * | 2023-09-15 | 2024-04-02 | 国家体育总局运动医学研究所 | Intelligent spine function detection device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US7899220B2 (en) | Time-dependent three-dimensional musculo-skeletal modeling based on dynamic surface measurements of bodies | |
| Aubert et al. | Toward automated 3D spine reconstruction from biplanar radiographs using CNN for statistical spine model fitting | |
| Pazos et al. | Accuracy assessment of human trunk surface 3D reconstructions from an optical digitising system | |
| US20170273614A1 (en) | Systems and methods for measuring and assessing spine instability | |
| Aubin et al. | Reliability and accuracy analysis of a new semiautomatic radiographic measurement software in adult scoliosis | |
| US20070038070A1 (en) | Determining patient-related information on the position and orientation of MR images by the individualisation of a body model | |
| US20080193904A1 (en) | Systems and Methods for Simulation of Organ Dynamics | |
| CN110891488B (en) | Sagittal rotation determination | |
| Cheriet et al. | Towards the self-calibration of a multiview radiographic imaging system for the 3D reconstruction of the human spine and rib cage | |
| JP2020074876A (en) | Vertebra posture estimation device | |
| CN116524124A (en) | Spinal three-dimensional dynamic reconstruction method and system | |
| Kandasamy et al. | Posture and back shape measurement tools: A narrative | |
| Ćuković et al. | Supporting diagnosis and treatment of scoliosis: using augmented reality to calculate 3D spine models in real-time-ARScoliosis | |
| CN115908717A (en) | Method and system for 3D modeling of spine | |
| CN114092447B (en) | Method, device and equipment for measuring scoliosis based on human body three-dimensional image | |
| Cerveri et al. | Non-invasive approach towards the in vivo estimation of 3D inter-vertebral movements: methods and preliminary results | |
| Bousigues et al. | 3D reconstruction of the scapula from biplanar X-rays for pose estimation and morphological analysis | |
| Chang et al. | A Photogrammetric system for 3D reconstruction of a scoliotic torso | |
| Groisser et al. | 3D Reconstruction of Scoliotic Spines from Stereoradiography and Depth Imaging | |
| Berthonnaud et al. | Geometric structure of 3D spinal curves: plane regions and connecting zones | |
| Cukovic et al. | Development of 3D kinematic model of the spine for idiopathic scoliosis simulation | |
| Xing et al. | Design and validation of depth camera-based static posture assessment system | |
| Wakaiki et al. | Individualization of musculoskeletal model for analyzing pelvic floor muscles activity based on gait motion features | |
| Min | Measurement of Spine Curvature using Flexicurve Integrated with Machine Vision | |
| US20230310084A1 (en) | Spinal surgery planning software |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |