WO2015142291A1 - Computer-aided planning of craniomaxillofacial and orthopedic surgery - Google Patents
Computer-aided planning of craniomaxillofacial and orthopedic surgery Download PDFInfo
- Publication number
- WO2015142291A1 WO2015142291A1 PCT/SG2015/050045 SG2015050045W WO2015142291A1 WO 2015142291 A1 WO2015142291 A1 WO 2015142291A1 SG 2015050045 W SG2015050045 W SG 2015050045W WO 2015142291 A1 WO2015142291 A1 WO 2015142291A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- model
- bone fragments
- restored
- property
- ones
- Prior art date
Links
- 238000001356 surgical procedure Methods 0.000 title claims abstract description 53
- 230000000399 orthopedic effect Effects 0.000 title claims abstract description 21
- 210000000988 bone and bone Anatomy 0.000 claims abstract description 221
- 238000000034 method Methods 0.000 claims abstract description 102
- 210000003625 skull Anatomy 0.000 claims abstract description 83
- 239000012634 fragment Substances 0.000 claims description 195
- 238000002591 computed tomography Methods 0.000 claims description 20
- 230000009466 transformation Effects 0.000 claims description 11
- 210000004279 orbit Anatomy 0.000 claims description 9
- 230000003247 decreasing effect Effects 0.000 claims description 8
- 238000002278 reconstructive surgery Methods 0.000 claims description 7
- 230000002980 postoperative effect Effects 0.000 claims description 3
- 238000012795 verification Methods 0.000 claims description 3
- 238000004364 calculation method Methods 0.000 claims description 2
- 210000004197 pelvis Anatomy 0.000 abstract description 23
- 206010017076 Fracture Diseases 0.000 description 13
- 208000014674 injury Diseases 0.000 description 6
- 208000027418 Wounds and injury Diseases 0.000 description 4
- 230000006378 damage Effects 0.000 description 4
- 238000013179 statistical model Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 3
- 206010034246 Pelvic fractures Diseases 0.000 description 2
- 230000002146 bilateral effect Effects 0.000 description 2
- 238000007796 conventional method Methods 0.000 description 2
- 238000001514 detection method Methods 0.000 description 2
- 230000001815 facial effect Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012549 training Methods 0.000 description 2
- 230000008733 trauma Effects 0.000 description 2
- 208000025978 Athletic injury Diseases 0.000 description 1
- 208000028771 Facial injury Diseases 0.000 description 1
- 206010019196 Head injury Diseases 0.000 description 1
- 206010039203 Road traffic accident Diseases 0.000 description 1
- 230000002159 abnormal effect Effects 0.000 description 1
- 210000003484 anatomy Anatomy 0.000 description 1
- 210000000784 arm bone Anatomy 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 210000002532 foramen magnum Anatomy 0.000 description 1
- 210000004373 mandible Anatomy 0.000 description 1
- 239000011159 matrix material Substances 0.000 description 1
- 210000003205 muscle Anatomy 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000001028 reflection method Methods 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 210000001694 thigh bone Anatomy 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Images
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B6/00—Apparatus or devices for radiation diagnosis; Apparatus or devices for radiation diagnosis combined with radiation therapy equipment
- A61B6/50—Apparatus or devices for radiation diagnosis; Apparatus or devices for radiation diagnosis combined with radiation therapy equipment specially adapted for specific body parts; specially adapted for specific clinical applications
- A61B6/501—Apparatus or devices for radiation diagnosis; Apparatus or devices for radiation diagnosis combined with radiation therapy equipment specially adapted for specific body parts; specially adapted for specific clinical applications for diagnosis of the head, e.g. neuroimaging or craniography
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
- A61B34/10—Computer-aided planning, simulation or modelling of surgical operations
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
- A61B34/10—Computer-aided planning, simulation or modelling of surgical operations
- A61B2034/101—Computer-aided simulation of surgical operations
- A61B2034/105—Modelling of the patient, e.g. for ligaments or bones
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
- A61B34/25—User interfaces for surgical systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10072—Tomographic images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30008—Bone
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30196—Human being; Person
- G06T2207/30201—Face
Definitions
- the present invention relates to surgery.
- it relates to computer-aided planning of craniomaxillofacial and orthopedic surgery.
- CMF craniomaxillofacial
- orthopedic reconstructive surgeries are performed. These surgeries involve complex operations on the skull or pelvis. In addition to the procedural complexity, difficulties of performing such surgeries also result from the variation of patients’ anatomical structures and the severity of the deformation. Therefore, careful pre-operative planning of surgery is crucial for the success of the operation.
- Restoration seeks to restore a deformed skull or pelvis back to its normal state by repositioning the fractured bones, which directly results in a feasible surgery plan.
- Reconstruction derives an estimation of the normal shape from the deformed skull or pelvis by shape similarity.
- the detailed shapes of the bones in the reconstructed model are not necessarily the same as those in the patient’s skull or pelvis. It may not be possible to obtain the reconstructed model by repositioning fractured bones in the deformed model.
- the surgeon needs to manually work out how to reposition the patient’s fractured bones to match a reference as given by the reconstructed model.
- Some existing computer-aided CMF or orthopedic surgery systems can reconstruct estimations of the skull or pelvis before injury. But, there is no existing planning system that generates a restored skull or pelvis from a patient’s deformed skull or pelvis by repositioning fractured bones.
- Conventional methods for solving the restoration and reconstruction problems include manual manipulation, symmetry-based reconstruction, geometric reconstruction, statistical reconstruction and fracture surface matching.
- Manual manipulation methods display a three-dimensional (3D) model of the patient’s skull in the computer monitor and allow the surgeon to manually cut and reposition the bones in the 3D model to generate a restored model.
- 3D three-dimensional
- Such methods are relatively easy to implement, and give the surgeon maximum control over how the bones in the deformed model should be repositioned.
- bone repositioning is planned by the surgeon manually, such manual manipulation methods can be very tedious, time-consuming, and inaccurate.
- Symmetry-based reconstruction methods reflect the healthy parts of a deformed model about a symmetry plane and use the reflected parts as estimates of the normal shapes of the fractured parts. While the symmetry-based reconstruction methods do not require any reference model, they do require the presence of healthy symmetric parts to reconstruct normal shape on the fractured side of the skull. When both sides of the skull are fractured, which is common in impact injuries, these methods are inapplicable.
- Geometric reconstruction methods register a reference model to the healthy parts of a deformed model and use the registered reference model to estimate the normal shapes of the fractured parts. These methods do not require the presence of healthy symmetric parts. However, they use a single reference model to generate the reconstructed model and their reconstruction accuracy depends on the similarity between the reference model and the normal model, as well as the degree of the correlation between the healthy parts and deformed parts.
- Statistical reconstruction methods overcome the limitation of geometric reconstruction by matching a statistical reference model to the healthy parts of a deformed model and using the matched statistical model to infer the normal shapes of the fractured parts of the deformed model. As such methods require a statistical model, statistical reconstruction methods require a large amount of training samples of healthy skull models to construct the statistical model.
- Fracture surface matching methods reposition fractured bones by determining their correct relative positions and orientations based on shape complementarity of adjacent fracture surfaces. They do not require any reference models or the presence of healthy parts in the deformed model. Instead, they produce the restored model by computing the correct positions and orientations of the fractured bone fragments. These methods, however, require the features of the fracture surfaces to be well captured in the deformed model. In the case of impact injuries where the fracture surfaces abrade each other destroying shape features, fracture surface matching methods become inaccurate.
- Computer-aided systems have also been developed for planning CMF surgery and orthopedic surgery. Some of these conventional systems allow the surgeon to manually reposition the bones and then predict the facial appearance. However, as described above, manual repositioning of bones is tedious, time-consuming and highly inaccurate. Some other conventional systems generate reconstructed models that can be used to guide manual planning, such as generating reconstructed models using symmetry-based method. These methods, however, are not applicable to patients with bilateral fractures, i.e., fractures on both sides of the skull. Further, the output of these methods is typically a reconstructed model instead of a restored model, which can only be used as a reference model for surgery (i.e., the surgeon still needs to manually work out how to reposition the bones according to the reference).
- Another conventional software package provides manual surgery planning service to surgeons where the planning can be performed manually by the surgeon’s staff. This procedure, however, shares the same limitation as manual restoration including the limitation that it is tedious, time-consuming and inaccurate.
- Several other software systems can generate surgery plans for mandible restoration using a fractured surface matching method. Yet, this method is also not reliable because it incorporates the limitations of the fractured surface matching method used.
- This invention relates to a procedure for computer-aided planning of craniomaxillofacial and orthopedic surgery and a scheme for generating a restored skull or pelvis model from a patient’s fractured skull or pelvis model for computer-aided surgery planning.
- a novel algorithm for generating a restored model from a patient’s deformed model by bone repositioning is presented along with a computer-aided procedure for assisting a surgeon in deriving a CMF or orthopedic surgery plan.
- a method for reconstructive modeling of a deformed model includes the steps of initializing a restored model as the deformed model and registering a reference model to the restored model by plane-fitting registration.
- the method also includes identifying corresponding points between the reference model and the restored model, wherein a corresponding point on the restored model for each point on the reference model is the closest point on the restored model, and computing a mid-sagittal plane (MSP) and a Frankfurt plane (FP) of the restored model.
- MSP mid-sagittal plane
- FP Frankfurt plane
- the method further includes associating one of a first property and a second property with fragments of the restored model and calculating a restoration order of fragments of the restored model having the second property associated therewith.
- the method includes generating an estimated normal surface around the fragment, registering the fragment to an estimated surface proximate thereto by a surface continuity-constrained registration, and associating the first property with fragments repositioned.
- the embodiment includes repeating all steps of the method except the initializing step until convergence of all of the fragments of the restored model.
- the method includes selecting the reference model from a plurality of reference models in response to a similarity between the reference model and the deformed model.
- the step of selecting the reference model includes registering each of the plurality of reference models to the deformed model by the plane-fitting registration, calculating mean surface distances to the fragments of the restored model having the first property associated therewith, and selecting one of the plurality of the reference models as the reference model in response to the one of the plurality of the reference models having a smallest mean surface distance over the fragments of the restored model having the first property associated therewith.
- the plane-fitting registration includes finding a correspondence between the reference model and the restored model by defining a corresponding point of a point on the restored model as a closest point on the reference model and defining a corresponding point of a landmark on the restored model as a perpendicular projection of a corresponding landmark on the mid-sagittal plane (MSP) of the reference model, defining inlier points as an inlier subset of corresponding points in response to small distances between the corresponding points, calculating a weighted similarity transformation in response to a correspondence of the inlier points wherein larger weights are given to correspondences generated from the corresponding landmarks, applying the calculated similarity transformation on all points of the reference model, and repeating these steps until convergence of all the points on the restored model.
- MSP mid-sagittal plane
- the method of computing the mid-sagittal plane (MSP) and Frankfurt plane (FP) of the restored model includes transferring the MSP of the reference model to the restored model and fitting the FP to FP landmarks identified by the user while constraining it to be orthogonal to the MSP by projecting the FP landmarks to the MSP orthogonally, fitting a line to the projected FP landmarks by minimizing a least squared distance from the line to the projected FP landmarks, and computing a plane that is orthogonal to the MSP and passes through the fitted line, wherein an origin is any point on the fitted line and a normal is a cross product of a normal of the plane and a direction of the fitted line.
- the fragments of the restored model includes bone fragments
- the first property comprises a confident property
- the second property comprises a nonconfident property
- the bone fragments comprise fixed bone fragments and movable bone fragments.
- the step of associating the first property and the second property with the bone fragments of the restored model includes associating the confident property with the fixed bone fragments and associating the nonconfident property with the movable bone fragments.
- the step of calculating the restoration order includes computing a repositioning order of the movable bone fragments in response to weighted sums of the following for each of the movable bone fragments:
- This step of computing the repositioning order of the movable bone fragments in response to the weighted sums of each of the movable bone fragments includes computing the repositioning order of the movable bone fragments in response to a decreasing value of the weighted sums of each of the movable bone fragments.
- This step of computing the repositioning order of the movable bone fragments in response to a decreasing value of the weighted sums of each of the movable bone fragments may also include identifying ones of the movable bone fragments that contain teeth, computing the repositioning order of the movable bone fragments in response to a decreasing value of the weighted sums of each of the movable bone fragments, and moving the ones of the movable bone fragments that contain teeth to the end of the repositioning order so that the ones of the movable bone fragments that contain teeth are repositioned last.
- the step of generating an estimated normal surface around each fragment of the restored model having the second property associated therewith includes finding corresponding points between the reference model and the fragment of the restored model having the first property associated therewith that have (a) corresponding points are close to each other, (b) corresponding points have similar surface normal, and (c) corresponding points lie near to the surface normal of each other, determining a surface deformation on a salient surface of the reference model in accordance with the corresponding points and lateral symmetry to the fragment of the restored model having the first property associated therewith, and cutting the surface deformation on the salient surface of the reference model leaving only the portions that correspond to closest points on the fragment of the restored model having the second property associated therewith, the above steps being repeated until convergence of the estimated normal surface.
- the surface continuity-constrained registration includes an iterative calculation until convergence that registers each fragment of the restored model having the second property associated therewith to an estimated normal surface around it in response to (a) closest points between a salient surface of the fragment of the restored model having the second property associated therewith and an estimated normal surface around it, (b) the landmark points identified by the user on the fragment of the restored model having the second property associated therewith and the perpendicular projections of the landmark points on corresponding planes of the restored model, and (c) weighted rigid transformation of corresponding points between the fragment of the restored model having the second property associated therewith and the estimated normal surface around the fragment of the restored model having the second property associated therewith, wherein larger weights are given to points near boundaries of the fragment of the restored model having the second property associated therewith adjacent to fragments of the restored model having the first property associated therewith.
- a method for reconstructive surgery planning includes providing images of one or more bones in an area to be reconstructed, the one or more bones including one or more bone fragments, segmenting the one or more bone fragments into a plurality of three-dimensional mesh representations such that each of the one or more bone fragments are a separate one of the plurality of three-dimensional mesh representations, assigning one of a first property or a second property to each of the plurality of three-dimensional mesh representations, and reconstructing the area by repositioning the three-dimensional mesh representations having the second property in response to ones of the three-dimensional mesh representations having the first property and a reference model.
- the step of providing images includes providing computed tomography (CT) images of the one or more bones in the area to be reconstructed.
- CT computed tomography
- the step of reconstructing the area comprises reconstructing the area by repositioning the three-dimensional mesh representations having the second property in response to (a) identified landmarks, (b) the ones of the three-dimensional mesh representations having the first property, (c) identified salient surfaces of the three-dimensional mesh representations, and (d) the reference model, wherein the salient surfaces of the three-dimensional mesh representations are surfaces of the three-dimensional mesh representations that should be flush with surfaces of the three-dimensional mesh representations of adjacent ones of the one or more bone fragments after reconstructing the area.
- the reconstructive surgery includes craniomaxillofacial surgery and/or orthopedic surgery
- ones of the plurality of three-dimensional mesh representations having the first property include three-dimensional mesh representations of fixed bone fragments
- ones of the plurality of three-dimensional mesh representations having the second property include three-dimensional mesh representations of movable bone fragments.
- the reconstructive surgery is craniomaxillofacial surgery and salient surfaces of the three-dimensional mesh representations of the one or more bone fragments include surfaces of the three-dimensional mesh representations of the one or more bone fragments that should be flush with surfaces of the three-dimensional mesh representations of adjacent ones of the one or more bone fragments after reconstruction and the salient surfaces include outer surfaces of the skull and surfaces around eye sockets.
- the landmarks comprise landmarks on a Frankfurt plane (FP) and a mid-sagittal plane (MSP)
- the reference model includes a mesh model of a normal skull having a MSP that serves as a laterally symmetric plane and includes a subset of surfaces corresponding to the salient surfaces of the three-dimensional mesh representations of the one or more bone fragments.
- the reference model may also be selected from a plurality of reference models in response to one or more similarities with a skull of a patient of the craniomaxillofacial surgery.
- the step of assigning ones of the plurality of three-dimensional mesh representations of a skull of a patient as movable bone fragments may include selecting ones of the plurality of three-dimensional mesh representations in a computer display of the plurality of three-dimensional mesh representations.
- This aspect of the embodiment may further include identifying a subset of the three-dimensional mesh representations of movable bone fragments as three-dimensional mesh representations of movable bone fragments having teeth, and the step of reconstructing the area may include reconstructing the area by repositioning the three-dimensional mesh representations having movable bone fragments in response to (a) the identified landmarks, (b) the ones of the three-dimensional mesh representations having fixed bone fragments, (c) the identified salient surfaces of the three-dimensional mesh representations of the bone fragments, (d) the reference model, and (e) ones of the three-dimensional mesh representations of movable bone fragments having teeth.
- a method for computer-aided craniomaxillofacial surgery planning includes taking computed tomography (CT) images of a patient’s skull and segmenting and constructing a three-dimensional deformed model of the patient’s skull from the CT images, wherein the three-dimensional deformed model include a plurality of three-dimensional meshes, each of the plurality of three-dimensional meshes corresponding to one of a plurality of bone fragments of the patient’s skull.
- CT computed tomography
- the method further includes indicating ones of the plurality of bone fragments to be repositioned by selecting the ones of the plurality of bone fragments on a computer display of the plurality of three-dimensional meshes corresponding to one of a plurality of bone fragments, wherein ones of the plurality of bone fragments on the computer display not selected is identified as bone fragments not to be repositioned.
- the method includes indicating ones of the plurality of bone fragments to be repositioned that contain teeth and identifying salient surfaces on the ones of the plurality of bone fragments to be repositioned, wherein salient surfaces are surfaces of the ones of the plurality of bone fragments that should be flush with surfaces of other ones of the plurality of bone fragments after restoration and include outer surfaces of the patient’s skull and surfaces around eye sockets of the patient’s skull.
- the method also includes identifying landmarks associated with a mid-sagittal plane (MSP) and a Frankfurt plane (FP) and generating a restored model from the three-dimensional deformed model of the patient’s skull in response to the three-dimensional deformed model of the patient’s skull and a reference model of a normal person’s skull by repositioning the plurality of bone fragments in the three-dimensional deformed model to normalize an overall shape of the patient’s skull by (a) locating the MSP and FP landmarks proximately close to a calculated MSP and FP of the patient’s skull, (b) placing the movable bones proximately close to an estimated normal and laterally symmetric shape, and (c) placing boundaries of adjacent bone fragments flush with each other.
- MSP mid-sagittal plane
- FP Frankfurt plane
- the method includes exporting data on the repositioned plurality of bone fragments for completion of a surgery plan and/or surgical verification with post-operative CT scans.
- the exported data can include meshes and synthesized DICOM images of the repositioned plurality of bone fragments.
- the step of segmenting and constructing the three-dimensional deformed model of the patient’s skull includes segmenting only ones of the plurality of bone fragments that need to be repositioned in the surgery as individual ones of the plurality of three-dimensional meshes, the ones of the plurality of bone fragments identified as bone fragments not to be repositioned being segmented as a single mesh.
- FIG. 1 depicts a flow diagram of the computer-aided CMF surgery planning procedure in accordance with the present embodiment.
- FIG. 1 depicts results of each stage of the planning procedure in accordance with the present embodiment, including (a) Input CT (b) Patient’s deformed model generated from CT images (c) Bone fragments to be repositioned (d) Identified salient surfaces, MSP landmarks, and FP landmarks (e) Restored model generated by the skull restoration algorithm (f) Synthesized DICOM image of the restored model.
- FIG. 1 depicts a planning tool for indicating movable bone fragments and anatomical landmarks in accordance with the present embodiment, including (a) GUI of planning tool (b) Identification of movable bone fragments to be repositioned (c) Identification of a MSP landmark.
- the Identified salient surfaces include eye orbit regions and outer surfaces detected automatically.
- FIG. 1 depicts Restored skull vs deformed skull in accordance with the present embodiment, including (a) MSP alignment, (b) FP alignment, (c) Lateral symmetry and surface continuity, (Top) deformed skull, (bottom) restored skull, MSP landmarks, FP landmarks.
- FIG. 1 depicts synthesized DICOM images of the restored model in accordance with the present embodiment, including (a) CT images of the deformed skull (b) synthesized DICOM images of the restored model.
- Red surfaces are salient surfaces and line indicates the MSP.
- the bones include the unfractured, undisplaced (“fixed”) bone, and the movable bone.
- the computer-aided CMF surgery planning procedure takes CT images of a patient’s skull as input and generates a surgery plan for restoring the patient’s skull by bone repositioning. It consists of 6 stages as depicted in FIG. 1.
- the surgeon applies software tools to segment and construct a 3D model of the patient’s skull from CT images 114 (FIG 2(a)).
- the skull model consists of separate 3D meshes, one for each bone fragment in the skull 116 (FIG 2(b)).
- the surgeon may segment only the bones that need to be repositioned in the surgery as an individual mesh, while all other bones are segmented as a single mesh. In general, more bone fragments could be segmented.
- the surgeon indicates the bone fragments that need to be repositioned using a GUI planning tool as shown in FIG 3. Bone fragments that are not selected are fixed bones. The surgeon also indicates whether a bone contains teeth.
- Salient surfaces are surfaces that should be flushed (i.e. continuous) after restoration. They consist of outer surfaces of the skull that can be automatically detected and surfaces around the eye sockets that are more difficult to detect automatically, which need to be indicated by the surgeon (FIG 4).
- the surgeon applies an automatic restoration algorithm to generate the restored model 124 (FIG. 2(e)) from the deformed model.
- the restoration algorithm takes a patient’s deformed model and a reference model of a normal person as inputs and generates a restored model by repositioning the bones in the deformed model.
- the fractured bones in the skull are displaced from their correct positions, making the overall shape abnormal.
- the boundaries of adjacent bones are discontinuous and the MSP landmarks 506 and FP landmarks 504 may not lie on the MSP and FP.
- FIG 5(2) (a)-(c) the overall shape becomes normal, the boundaries of adjacent bones are flushed and the MSP landmarks 506and FP landmarks 504 line close to the MSP and FP.
- the surgeon instructs the planning tool to export the repositioned bones in STL format.
- the STL files can be imported into existing planning software to overcome the limitation of the existing planning software to unilateral fractures.
- the planning tool also synthesized DICOM images of the restored model (FIG 2(f) and FIG 6) for the surgeon’s verification with post-operative CT scans.
- the skull restoration algorithm in the fifth stage 110 is an algorithm that iteratively repositions fractured bone fragments.
- the inputs to the restoration algorithm include Reference model (FIG 7) and Deformed Model (FIG 8).
- the Reference model (FIG 7) is a mesh model of a normal skull. It has a MSP 704 that serves as the laterally symmetric plane. It also contains a subset of surfaces which are the salient surfaces 702.
- the Deformed model (FIG 8) contains a set of bone fragment models 800. These bone fragments are divided into a subset of fixed bone fragments 802 and a subset of movable bone fragments 808. Some movable bones that contain teeth are also known. Each bone fragment has a mesh representing the bone’s surface. It also contains a set of salient surfaces and a set of anatomical landmarks located on it, which can be empty for bone fragments that do not pass through any anatomical plane.
- the output of the algorithm is the restored model, which is produced by repositioning the movable bone fragments in the deformed model.
- the restoration algorithm uses a reference model as a guide.
- the preparation of the reference model includes three steps.
- 3D mesh models of the skull and the lower jaw are segmented from CT images and reconstructed.
- a human expert indicates a set of MSP landmarks 806 on the models using the planning GUI.
- the MSP is then fitted to the MSP landmarks 806 by minimizing its mean squared distance to the landmarks.
- the origin of the plane is the mean position of these landmarks, and the normal of the plane is the principal component corresponding to the smallest eigenvalue of the covariance matrix of these landmarks.
- the laterally symmetrical corresponding points are computed by with respect to the MSP.
- salient surfaces of the reference model are identified in a similar manner as that of the patient’s deformed model.
- the reference model all the salient surfaces are identified.
- multiple reference models are prepared, and the one that is most similar to the patient’s skull is used by the skull restoration algorithm.
- the restoration algorithm is an iterative optimization algorithm that repositions each bone fragment one at a time. It includes three steps. In the first step, given a deformed model, the reference model that is most similar to the deformed model is selected. In the second step, the restored model is initialized as the deformed model. In the third step, several sub-steps are repeated until convergence. The several sub-steps include a. Register reference model to the restored model by plane-fitting registration b. Find corresponding points between the reference and restored model. For each point on the reference model, its corresponding point is the closest point on the restored model. c. Compute MSP and FP of the restored model d. Mark fixed bone fragments as confident and movable bone fragments as nonconfident. e.
- Compute restoration order of the nonconfident bone fragments f For each nonconfident bone fragment in decreasing order, generate the estimated normal surface around the bone fragment, and register the bone fragment to estimated surface around it by surface continuity-constrained registration, and then mark the repositioned bone as confident.
- the method for selecting the reference model that is most similar to the deformed model comprises the following steps:
- the plane-fitting registration algorithm registers the reference model to the restored model robustly and also ensures that the MSP of the reference model matches the MSP landmarks of the restored model. It is an iterative algorithm that repeating the following steps until convergence.
- a. Find correspondence between reference model and restored model. For a point on the restored model, its corresponding point is its closest point on the reference model. For a MSP landmark on the restored model, its corresponding point is its perpendicular projection on the MSP of the reference model. b. Find inlier subset of corresponding points with small distances between the corresponding points. c. Apply the algorithm to compute weighted similarity transformation using correspondence of inlier points. Larger weights are given to correspondences generated from MSP landmarks. d. Apply the computed similarity transformation on all points of the reference model.
- the method for computing MSP and FP of restored model comprises the following steps: 1. Transfer the MSP of the reference model to the restored model. 2. Fit the FP to the FP landmarks identified by the user while constraining it to be orthogonal to the MSP.
- the step 2 includes the following sub-steps: a. Project FP landmarks to the MSP orthogonally. b. Fit a line to the projected landmarks by minimizing least squared distance from the line to the projected landmarks. c. Compute the plane that is orthogonal to MSP and passes through the fitted line. The origin is any point on the fitted line, and the normal is the cross product the plane normal and the direction of the fitted line.
- the method for computing the repositioning order of the nonconfident bones computes the following scores: 1. Number of anatomical landmarks on the surface of a bone fragment. 2. Fraction of salient surface vertices whose symmetric points are on a confident bone. The symmetry point relationship is transferred from the reference model using the correspondence computed in (b) of the third step of Skull restoration algorithm. 3. Fraction of salient surface vertices that are at the boundary adjacent to confident bones. The boundaries are identified on the reference model at the edges where the two neighboring points lie on two different bone fragments in the restored model. They are transferred to the restored model using the correspondence computed in (b) of the third step of Skull restoration algorithm.
- a weighted sum of these scores is computed for each nonconfident bone fragment, and the bones are sorted in decreasing order of their weighted sum of scores. After that, the bones that contain teeth are moved to the end of the sorted lists so that they are repositioned last.
- the method for generating the estimated normal surface around a bone fragment comprises the following steps: 1. Find corresponding points between the reference model and the confident parts of the restoration model that meet several requirements. 2. Apply Laplacian surface deformation algorithm on salient surface of the reference model with corresponding points as hard constraints and lateral symmetry as soft constraints. 3. Cut the deformed surface leaving only the portions that correspond to the bone fragment using the corresponding computed in (b) of the third step of Skull restoration algorithm. 4. Repeating the above steps until convergence of the estimated normal surface.
- Step 1 includes: a. Corresponding points are close to each other. b. Corresponding points have similar surface normal. c. Corresponding points lie near to the surface normal of each other.
- the surface continuity-constrained registration algorithm is an iterative algorithm that registers a nonconfident bone fragment to the estimated normal surface around. It comprises the following steps: Repeat until convergence: a. Find corresponding points between the salient surface of a nonconfident bone and the estimated normal surface around it. For a point on the salient surface, its corresponding point is the closest point on the estimated normal surface. b. Find correspondence for anatomical landmarks. For a MSP or FP landmark on the nonconfident bone, its corresponding point is its orthogonal projection on the MSP or FP of the restored model, respectively. c. Apply the algorithm to compute weighted rigid transformation using the point correspondence. Larger weights are given to the points near the boundaries of nonconfident bone adjacent to confident bones. d. Apply the computed rigid transformation to all the points of the nonconfident bone.
- This embodiment provides a computer-aided procedure for a surgeon to derive a surgery plan for restoring a patient’s deformed skull or pelvis back to the normal state by bone respositioning.
- the embodiment has several advantages over existing methods. Firstly, the output of the planning procedure is a restored model which directly results in a feasible surgery plan. Secondly, the planning procedure is applicable to both unilateral and bilateral fractures. Thirdly, the skull restoration algorithm generates a restored model with anatomical plane fitting, shape normality, surface continuity, and lateral symmetry as algorithmic constraints. This makes the result more accurate and reliable than existing manual methods, symmetry-based methods, geometric reconstruction methods, statistical reconstruction methods, and fracture surface matching methods.
- the proposed planning procedure has several advantages over existing procedures.
- the proposed procedure provides an automatic FP, MSP and landmark identification method for skulls with minor injuries.
- the proposed procedure replaces the reflection method of existing procedure by an automatic restoration algorithm.
- the proposed procedure generates a restored model by bone repositioning whereas the existing procedure generates a partially reconstructed model by lateral symmetry.
- This embodiment provides more accurate pre-operative planning of craniomaxillofacial (CMF) surgery for the skull and orthopedic surgery for the pelvis.
- CMF craniomaxillofacial
- This embodiment also provides training of CMF and orthopedic surgeons in precision recognition of deformed bones as well as planning and execution of the plan in the operating theatre.
- This embodiment is applicable to large bone structures such as the skull and the pelvis where all the surgical decisions are made according to the shape of the bone.
- the surgical decisions are based on the relative orientations of the fractures bones rather than the shape of the bones.
- salient surfaces can be detected by an automatic algorithm.
- the outer surface is identified automatically, and the eye socket region is identified manually.
- the eye socket region can also be identified automatically by fitting a sphere to the eye socket region in a manner similar to the foramen magnum identification method.
- reference model can be an active shape model.
- the reference model is a single normal model selected from a database. However, it can also be generated by fitting an active shape model to the normal part of the patient’s model, as done for statistical reconstruction.
- the skull restoration algorithm does not explicitly detect collisions of the restored bone fragments. Instead, it relies on the accurate estimation of normal surfaces and registration of bone fragments to the estimated normal surface to naturally avoid collisions. Experiments showed that the collision error is small or most cases. For cases where the collision is not negligible, collision detection avoidance can be performed. Collision detection is a well solved problem. There are various algorithms and packages available, e.g. proximity query package and vtkbioeng, etc. One modification is to check for collision after the skull is restored, and move the collided bones apart to reduce or remove collisions.
- the present embodiment can be applied to orthopedic surgery of pelvis bone.
- CMF surgery is used as an example to describe the procedure and the algorithm.
- a similar procedure and algorithm can be applied to orthopedic surgery of pelvis bones.
- Frankfurt plan is not defined for pelvis.
- pelvis restoration either a different horizontal plan is defined or the horizontal plane is ignored in the restoration algorithm.
- corresponding salient surfaces need to be either detected automatically or identified manually.
- Flow diagram of the computer-aided CMF surgery planning procedure 102 Segmentation and 3D model construction 104 Identification of bone fragments to be repositioned 106 Identification of salient surfaces on the skull 108 Identification of MSP and FP landmarks 110 Execution of skull restoration algorithm 112 Generation of planning output 114 CT images 116 Deformed model 118 Bones to be moved 120 Salient surfaces 122 MSP/FP landmarks 124 Restored model 126 Output files 200 Each stage of the planning procedure 202 Bone fragments to be repositioned 204 FP landmarks 206 MSP landmarks 300 Planning tool for indicating movable bone fragments and anatomical landmarks 400 Identification of salient surfaces 500 Comparison of Restored skull and Deformed skull 504 FP landmarks 506 MSP landmarks 600 Synthesized DICOM images of the restored model 700 Reference model 702 Salient surface 704 MSP 800 Patient’s deformed model 802 Unfractured undisplaced (“fixed”) bone 804 FP landmarks 806 MSP landmarks 808 Movable bone
Landscapes
- Health & Medical Sciences (AREA)
- Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Surgery (AREA)
- Medical Informatics (AREA)
- Public Health (AREA)
- Heart & Thoracic Surgery (AREA)
- Veterinary Medicine (AREA)
- General Health & Medical Sciences (AREA)
- Animal Behavior & Ethology (AREA)
- Molecular Biology (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Physics & Mathematics (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Radiology & Medical Imaging (AREA)
- Optics & Photonics (AREA)
- Dentistry (AREA)
- Neurosurgery (AREA)
- High Energy & Nuclear Physics (AREA)
- Pathology (AREA)
- Neurology (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Theoretical Computer Science (AREA)
- Robotics (AREA)
- Apparatus For Radiation Diagnosis (AREA)
Abstract
A procedure for computer-aided planning of craniomaxillofacial (CMF) and orthopedic surgery and scheme for generating a restored skull or pelvis model from a patient's fractured skull or pelvis model for computer-aided surgery planning are provided. In one embodiment, an algorithm for generating a restored model from a patient's deformed model by bone repositioning is presented along with a computer aided procedure for assisting a surgeon in deriving a CMF or orthopedic surgery plan.
Description
The present application claims priority to
Singapore Patent Application No. 10201400827T.
The present invention relates to surgery. In
particular, it relates to computer-aided planning of
craniomaxillofacial and orthopedic surgery.
Many patients suffer from skull deformity and
pelvis deformity. For example, in North America, every
year, around 1.58 million people suffer from head and
facial injuries due to traffic accidents, work
accidents, home accidents, sports injuries, and
violence. About 10% of trauma patients treated by a
Level 1 Trauma Centre (e.g., a general hospital) have
pelvic fracture, and the mortality rate of patients with
pelvic fracture ranges from 10% to 16%. In Singapore,
National University Hospital (NUH) alone receives about
250–350 patients with facial fracture every year. Bone
deformation often leads to shape distortion as well as
incomplete functionality because the muscles and skin
attached to a deformed bone are also deformed. These
deformities may greatly affect the patients’ life
quality and even threaten their lives.
To restore the bones to their normal states,
craniomaxillofacial (CMF) and orthopedic reconstructive
surgeries are performed. These surgeries involve
complex operations on the skull or pelvis. In addition
to the procedural complexity, difficulties of performing
such surgeries also result from the variation of
patients’ anatomical structures and the severity of the
deformation. Therefore, careful pre-operative planning
of surgery is crucial for the success of the operation.
There are two related problems in CMF and
orthopedic surgery planning, namely restoration and
reconstruction. Restoration seeks to restore a deformed
skull or pelvis back to its normal state by
repositioning the fractured bones, which directly
results in a feasible surgery plan. Reconstruction, on
the other hand, derives an estimation of the normal
shape from the deformed skull or pelvis by shape
similarity. The detailed shapes of the bones in the
reconstructed model are not necessarily the same as
those in the patient’s skull or pelvis. It may not be
possible to obtain the reconstructed model by
repositioning fractured bones in the deformed model. To
use the reconstructed model for surgery planning, the
surgeon needs to manually work out how to reposition the
patient’s fractured bones to match a reference as given
by the reconstructed model.
Some existing computer-aided CMF or orthopedic
surgery systems can reconstruct estimations of the skull
or pelvis before injury. But, there is no existing
planning system that generates a restored skull or
pelvis from a patient’s deformed skull or pelvis by
repositioning fractured bones.
Conventional methods for solving the
restoration and reconstruction problems include manual
manipulation, symmetry-based reconstruction, geometric
reconstruction, statistical reconstruction and fracture
surface matching. Manual manipulation methods display a
three-dimensional (3D) model of the patient’s skull in
the computer monitor and allow the surgeon to manually
cut and reposition the bones in the 3D model to generate
a restored model. Such methods are relatively easy to
implement, and give the surgeon maximum control over how
the bones in the deformed model should be repositioned.
However, as bone repositioning is planned by the surgeon
manually, such manual manipulation methods can be very
tedious, time-consuming, and inaccurate.
Symmetry-based reconstruction methods reflect
the healthy parts of a deformed model about a symmetry
plane and use the reflected parts as estimates of the
normal shapes of the fractured parts. While the
symmetry-based reconstruction methods do not require any
reference model, they do require the presence of healthy
symmetric parts to reconstruct normal shape on the
fractured side of the skull. When both sides of the
skull are fractured, which is common in impact injuries,
these methods are inapplicable.
Geometric reconstruction methods register a
reference model to the healthy parts of a deformed model
and use the registered reference model to estimate the
normal shapes of the fractured parts. These methods do
not require the presence of healthy symmetric parts.
However, they use a single reference model to generate
the reconstructed model and their reconstruction
accuracy depends on the similarity between the reference
model and the normal model, as well as the degree of the
correlation between the healthy parts and deformed
parts. Statistical reconstruction methods overcome the
limitation of geometric reconstruction by matching a
statistical reference model to the healthy parts of a
deformed model and using the matched statistical model
to infer the normal shapes of the fractured parts of the
deformed model. As such methods require a statistical
model, statistical reconstruction methods require a
large amount of training samples of healthy skull models
to construct the statistical model. In addition, their
reconstruction accuracy depends on how well the
statistical model captures the normal shape variation of
the patient’s normal shape. In the case where such
normal shape variation is not adequately captured, only
the mean shape of the statistical reference can be used.
Then, statistical reconstruction approximates geometric
reconstruction and exhibits the problems associated with
such conventional methods. Moreover, similar to
geometric reconstruction methods, statistical
reconstruction methods rely on the correlation between
the healthy parts and the deformed parts.
Fracture surface matching methods reposition
fractured bones by determining their correct relative
positions and orientations based on shape
complementarity of adjacent fracture surfaces. They do
not require any reference models or the presence of
healthy parts in the deformed model. Instead, they
produce the restored model by computing the correct
positions and orientations of the fractured bone
fragments. These methods, however, require the features
of the fracture surfaces to be well captured in the
deformed model. In the case of impact injuries where
the fracture surfaces abrade each other destroying shape
features, fracture surface matching methods become inaccurate.
Computer-aided systems have also been
developed for planning CMF surgery and orthopedic
surgery. Some of these conventional systems allow the
surgeon to manually reposition the bones and then
predict the facial appearance. However, as described
above, manual repositioning of bones is tedious,
time-consuming and highly inaccurate. Some other
conventional systems generate reconstructed models that
can be used to guide manual planning, such as generating
reconstructed models using symmetry-based method. These
methods, however, are not applicable to patients with
bilateral fractures, i.e., fractures on both sides of
the skull. Further, the output of these methods is
typically a reconstructed model instead of a restored
model, which can only be used as a reference model for
surgery (i.e., the surgeon still needs to manually work
out how to reposition the bones according to the reference).
Another conventional software package provides
manual surgery planning service to surgeons where the
planning can be performed manually by the surgeon’s
staff. This procedure, however, shares the same
limitation as manual restoration including the
limitation that it is tedious, time-consuming and
inaccurate. Several other software systems can generate
surgery plans for mandible restoration using a fractured
surface matching method. Yet, this method is also not
reliable because it incorporates the limitations of the
fractured surface matching method used.
Thus, what is needed is an automatic, reliable
and accurate method for generating a restored model of a
patient’s skull or pelvis from his/her deformed skull or
pelvis. More particularly, what is needed is a
computer-aided system that provides assistance for
generating a CMF or orthopedic surgery plan based on
restoration of fractured skull or pelvis models.
Furthermore, other desirable features and
characteristics will become apparent from the subsequent
detailed description and the appended claims, taken in
conjunction with the accompanying drawings and this
background of the disclosure.
This invention relates to a procedure for
computer-aided planning of craniomaxillofacial and
orthopedic surgery and a scheme for generating a
restored skull or pelvis model from a patient’s
fractured skull or pelvis model for computer-aided
surgery planning. In one embodiment, a novel algorithm
for generating a restored model from a patient’s
deformed model by bone repositioning is presented along
with a computer-aided procedure for assisting a surgeon
in deriving a CMF or orthopedic surgery plan.
In accordance with this first embodiment a
method for reconstructive modeling of a deformed model
is provided which includes the steps of initializing a
restored model as the deformed model and registering a
reference model to the restored model by plane-fitting
registration. The method also includes identifying
corresponding points between the reference model and the
restored model, wherein a corresponding point on the
restored model for each point on the reference model is
the closest point on the restored model, and computing a
mid-sagittal plane (MSP) and a Frankfurt plane (FP) of
the restored model. The method further includes
associating one of a first property and a second
property with fragments of the restored model and
calculating a restoration order of fragments of the
restored model having the second property associated
therewith. And, for each of the fragments of the
restored model having the second property associated
therewith and in accordance with the calculated
restoration order, the method includes generating an
estimated normal surface around the fragment,
registering the fragment to an estimated surface
proximate thereto by a surface continuity-constrained
registration, and associating the first property with
fragments repositioned. Finally, the embodiment
includes repeating all steps of the method except the
initializing step until convergence of all of the
fragments of the restored model.
In one aspect, the method includes selecting
the reference model from a plurality of reference models
in response to a similarity between the reference model
and the deformed model. In another aspect, the step of
selecting the reference model includes registering each
of the plurality of reference models to the deformed
model by the plane-fitting registration, calculating
mean surface distances to the fragments of the restored
model having the first property associated therewith,
and selecting one of the plurality of the reference
models as the reference model in response to the one of
the plurality of the reference models having a smallest
mean surface distance over the fragments of the restored
model having the first property associated therewith.
In yet another aspect of the embodiment, the
plane-fitting registration includes finding a
correspondence between the reference model and the
restored model by defining a corresponding point of a
point on the restored model as a closest point on the
reference model and defining a corresponding point of a
landmark on the restored model as a perpendicular
projection of a corresponding landmark on the
mid-sagittal plane (MSP) of the reference model,
defining inlier points as an inlier subset of
corresponding points in response to small distances
between the corresponding points, calculating a weighted
similarity transformation in response to a
correspondence of the inlier points wherein larger
weights are given to correspondences generated from the
corresponding landmarks, applying the calculated
similarity transformation on all points of the reference
model, and repeating these steps until convergence of
all the points on the restored model.
In yet another aspect, the method of
computing the mid-sagittal plane (MSP) and Frankfurt
plane (FP) of the restored model includes transferring
the MSP of the reference model to the restored model and
fitting the FP to FP landmarks identified by the user
while constraining it to be orthogonal to the MSP by
projecting the FP landmarks to the MSP orthogonally,
fitting a line to the projected FP landmarks by
minimizing a least squared distance from the line to the
projected FP landmarks, and computing a plane that is
orthogonal to the MSP and passes through the fitted
line, wherein an origin is any point on the fitted line
and a normal is a cross product of a normal of the plane
and a direction of the fitted line.
In accordance with another aspect of the
embodiment, the fragments of the restored model includes
bone fragments, the first property comprises a confident
property, the second property comprises a nonconfident
property, and the bone fragments comprise fixed bone
fragments and movable bone fragments. In regards to
this aspect, the step of associating the first property
and the second property with the bone fragments of the
restored model includes associating the confident
property with the fixed bone fragments and associating
the nonconfident property with the movable bone
fragments. In accordance with another aspect of the
embodiment, the step of calculating the restoration
order includes computing a repositioning order of the
movable bone fragments in response to weighted sums of
the following for each of the movable bone fragments:
(a) a number of anatomical landmarks on a
surface of one of the movable bone fragments,
(b) a fraction of salient surface vertices on
the one of the movable bone fragments whose symmetric
points are on one of the bone fragments of the restored
model having the first property associated therewith, and
(c) a fraction of salient surface vertices on
the one of the movable bone fragments that are at a
boundary adjacent to one of the bone fragments of the
restored model having the first property associated therewith.
This step of computing the repositioning order
of the movable bone fragments in response to the
weighted sums of each of the movable bone fragments
includes computing the repositioning order of the
movable bone fragments in response to a decreasing value
of the weighted sums of each of the movable bone
fragments. This step of computing the repositioning
order of the movable bone fragments in response to a
decreasing value of the weighted sums of each of the
movable bone fragments may also include identifying ones
of the movable bone fragments that contain teeth,
computing the repositioning order of the movable bone
fragments in response to a decreasing value of the
weighted sums of each of the movable bone fragments, and
moving the ones of the movable bone fragments that
contain teeth to the end of the repositioning order so
that the ones of the movable bone fragments that contain
teeth are repositioned last.
In accordance with another aspect of the
embodiment, the step of generating an estimated normal
surface around each fragment of the restored model
having the second property associated therewith includes
finding corresponding points between the reference model
and the fragment of the restored model having the first
property associated therewith that have (a)
corresponding points are close to each other, (b)
corresponding points have similar surface normal, and
(c) corresponding points lie near to the surface normal
of each other, determining a surface deformation on a
salient surface of the reference model in accordance
with the corresponding points and lateral symmetry to
the fragment of the restored model having the first
property associated therewith, and cutting the surface
deformation on the salient surface of the reference
model leaving only the portions that correspond to
closest points on the fragment of the restored model
having the second property associated therewith, the
above steps being repeated until convergence of the
estimated normal surface.
And in accordance with a final aspect of the
embodiment, the surface continuity-constrained
registration includes an iterative calculation until
convergence that registers each fragment of the restored
model having the second property associated therewith to
an estimated normal surface around it in response to (a)
closest points between a salient surface of the fragment
of the restored model having the second property
associated therewith and an estimated normal surface
around it, (b) the landmark points identified by the
user on the fragment of the restored model having the
second property associated therewith and the
perpendicular projections of the landmark points on
corresponding planes of the restored model, and (c)
weighted rigid transformation of corresponding points
between the fragment of the restored model having the
second property associated therewith and the estimated
normal surface around the fragment of the restored model
having the second property associated therewith, wherein
larger weights are given to points near boundaries of
the fragment of the restored model having the second
property associated therewith adjacent to fragments of
the restored model having the first property associated therewith.
In accordance with another embodiment, a
method for reconstructive surgery planning is provided.
The method includes providing images of one or more
bones in an area to be reconstructed, the one or more
bones including one or more bone fragments, segmenting
the one or more bone fragments into a plurality of
three-dimensional mesh representations such that each of
the one or more bone fragments are a separate one of the
plurality of three-dimensional mesh representations,
assigning one of a first property or a second property
to each of the plurality of three-dimensional mesh
representations, and reconstructing the area by
repositioning the three-dimensional mesh representations
having the second property in response to ones of the
three-dimensional mesh representations having the first
property and a reference model.
In accordance with an aspect of this
embodiment, the step of providing images includes
providing computed tomography (CT) images of the one or
more bones in the area to be reconstructed. In
accordance with another aspect, the step of
reconstructing the area comprises reconstructing the
area by repositioning the three-dimensional mesh
representations having the second property in response
to (a) identified landmarks, (b) the ones of the
three-dimensional mesh representations having the first
property, (c) identified salient surfaces of the
three-dimensional mesh representations, and (d) the
reference model, wherein the salient surfaces of the
three-dimensional mesh representations are surfaces of
the three-dimensional mesh representations that should
be flush with surfaces of the three-dimensional mesh
representations of adjacent ones of the one or more bone
fragments after reconstructing the area.
In accordance with yet another aspect of this
embodiment, the reconstructive surgery includes
craniomaxillofacial surgery and/or orthopedic surgery,
and ones of the plurality of three-dimensional mesh
representations having the first property include
three-dimensional mesh representations of fixed bone
fragments, and ones of the plurality of
three-dimensional mesh representations having the second
property include three-dimensional mesh representations
of movable bone fragments.
In accordance with another aspect, the
reconstructive surgery is craniomaxillofacial surgery
and salient surfaces of the three-dimensional mesh
representations of the one or more bone fragments
include surfaces of the three-dimensional mesh
representations of the one or more bone fragments that
should be flush with surfaces of the three-dimensional
mesh representations of adjacent ones of the one or more
bone fragments after reconstruction and the salient
surfaces include outer surfaces of the skull and
surfaces around eye sockets. In accordance with this
aspect, the landmarks comprise landmarks on a Frankfurt
plane (FP) and a mid-sagittal plane (MSP), and the
reference model includes a mesh model of a normal skull
having a MSP that serves as a laterally symmetric plane
and includes a subset of surfaces corresponding to the
salient surfaces of the three-dimensional mesh
representations of the one or more bone fragments. The
reference model may also be selected from a plurality of
reference models in response to one or more similarities
with a skull of a patient of the craniomaxillofacial
surgery. The step of assigning ones of the plurality of
three-dimensional mesh representations of a skull of a
patient as movable bone fragments may include selecting
ones of the plurality of three-dimensional mesh
representations in a computer display of the plurality
of three-dimensional mesh representations. This aspect
of the embodiment may further include identifying a
subset of the three-dimensional mesh representations of
movable bone fragments as three-dimensional mesh
representations of movable bone fragments having teeth,
and the step of reconstructing the area may include
reconstructing the area by repositioning the
three-dimensional mesh representations having movable
bone fragments in response to (a) the identified
landmarks, (b) the ones of the three-dimensional mesh
representations having fixed bone fragments, (c) the
identified salient surfaces of the three-dimensional
mesh representations of the bone fragments, (d) the
reference model, and (e) ones of the three-dimensional
mesh representations of movable bone fragments having teeth.
In accordance with a further embodiment, a
method for computer-aided craniomaxillofacial surgery
planning is provided. The method includes taking
computed tomography (CT) images of a patient’s skull and
segmenting and constructing a three-dimensional deformed
model of the patient’s skull from the CT images, wherein
the three-dimensional deformed model include a plurality
of three-dimensional meshes, each of the plurality of
three-dimensional meshes corresponding to one of a
plurality of bone fragments of the patient’s skull. The
method further includes indicating ones of the plurality
of bone fragments to be repositioned by selecting the
ones of the plurality of bone fragments on a computer
display of the plurality of three-dimensional meshes
corresponding to one of a plurality of bone fragments,
wherein ones of the plurality of bone fragments on the
computer display not selected is identified as bone
fragments not to be repositioned. And the method
includes indicating ones of the plurality of bone
fragments to be repositioned that contain teeth and
identifying salient surfaces on the ones of the
plurality of bone fragments to be repositioned, wherein
salient surfaces are surfaces of the ones of the
plurality of bone fragments that should be flush with
surfaces of other ones of the plurality of bone
fragments after restoration and include outer surfaces
of the patient’s skull and surfaces around eye sockets
of the patient’s skull. The method also includes
identifying landmarks associated with a mid-sagittal
plane (MSP) and a Frankfurt plane (FP) and generating a
restored model from the three-dimensional deformed model
of the patient’s skull in response to the
three-dimensional deformed model of the patient’s skull
and a reference model of a normal person’s skull by
repositioning the plurality of bone fragments in the
three-dimensional deformed model to normalize an overall
shape of the patient’s skull by (a) locating the MSP and
FP landmarks proximately close to a calculated MSP and
FP of the patient’s skull, (b) placing the movable bones
proximately close to an estimated normal and laterally
symmetric shape, and (c) placing boundaries of adjacent
bone fragments flush with each other. Finally, the
method includes exporting data on the repositioned
plurality of bone fragments for completion of a surgery
plan and/or surgical verification with post-operative CT
scans. The exported data can include meshes and
synthesized DICOM images of the repositioned plurality
of bone fragments.
In accordance with an aspect of this
embodiment, the step of segmenting and constructing the
three-dimensional deformed model of the patient’s skull
includes segmenting only ones of the plurality of bone
fragments that need to be repositioned in the surgery as
individual ones of the plurality of three-dimensional
meshes, the ones of the plurality of bone fragments
identified as bone fragments not to be repositioned
being segmented as a single mesh.
The accompanying figures, where like reference
numerals refer to identical or functionally similar
elements throughout the separate views and which
together with the detailed description below are
incorporated in and form part of the specification,
serve to illustrate various embodiments and to explain
various principles and advantages in accordance with a
present embodiment.
Skilled artisans will appreciate that elements
in the figures are illustrated for simplicity and
clarity and have not necessarily been depicted to scale.
For example, the dimensions of some of the elements in
the illustrations, block diagrams or flowcharts may be
exaggerated in respect to other elements to help to
improve understanding of the present embodiments.
The following detailed description is merely
exemplary in nature and is not intended to limit the
invention or the application and uses of the invention.
Furthermore, there is no intention to be bound by any
theory presented in the preceding background of the
invention or the following detailed description.
Herein, a computer-aided CMF and orthopedic surgery
planning procedure and an algorithm for the generation
of a restored model from a patient’s deformed model.
Similar procedure and algorithm can be applied to
orthopedic surgery of pelvis bones.
The computer-aided CMF surgery planning
procedure takes CT images of a patient’s skull as input
and generates a surgery plan for restoring the patient’s
skull by bone repositioning. It consists of 6 stages as
depicted in FIG. 1.
In the first stage 102, the surgeon applies
software tools to segment and construct a 3D model of
the patient’s skull from CT images 114 (FIG 2(a)). The
skull model consists of separate 3D meshes, one for each
bone fragment in the skull 116 (FIG 2(b)). In practice,
the surgeon may segment only the bones that need to be
repositioned in the surgery as an individual mesh, while
all other bones are segmented as a single mesh. In
general, more bone fragments could be segmented.
In the second stage 104, the surgeon indicates
the bone fragments that need to be repositioned using a
GUI planning tool as shown in FIG 3. Bone fragments that
are not selected are fixed bones. The surgeon also
indicates whether a bone contains teeth.
In the third stage 106, salient surfaces are
identified on the bone fragments 120 (FIG 2(d)). Salient
surfaces are surfaces that should be flushed (i.e.
continuous) after restoration. They consist of outer
surfaces of the skull that can be automatically detected
and surfaces around the eye sockets that are more
difficult to detect automatically, which need to be
indicated by the surgeon (FIG 4).
In the fourth stage 108, Frankfurt plane (FP),
mid-sagittal plane (MSP) and their landmarks are
identified as shown in 120 (FIG 2(d)). The surgeon
manually indicates FP landmarks 204 and MSP landmarks
206 on the deformed model using the GUI planning tool as
shown in FIG 3 (c).
In the fifth stage 110, the surgeon applies an
automatic restoration algorithm to generate the restored
model 124 (FIG. 2(e)) from the deformed model. The
restoration algorithm takes a patient’s deformed model
and a reference model of a normal person as inputs and
generates a restored model by repositioning the bones in
the deformed model. Before restoration (FIG 5(1)
(a)-(c)), the fractured bones in the skull are displaced
from their correct positions, making the overall shape
abnormal. The boundaries of adjacent bones are
discontinuous and the MSP landmarks 506 and FP landmarks
504 may not lie on the MSP and FP. After restoration
(FIG 5(2) (a)-(c)), the overall shape becomes normal,
the boundaries of adjacent bones are flushed and the MSP
landmarks 506and FP landmarks 504 line close to the MSP
and FP.
In the sixth stage 112, the surgeon instructs
the planning tool to export the repositioned bones in
STL format. The STL files can be imported into existing
planning software to overcome the limitation of the
existing planning software to unilateral fractures. The
planning tool also synthesized DICOM images of the
restored model (FIG 2(f) and FIG 6) for the surgeon’s
verification with post-operative CT scans.
The skull restoration algorithm in the fifth
stage 110 is an algorithm that iteratively repositions
fractured bone fragments. The inputs to the restoration
algorithm include Reference model (FIG 7) and Deformed
Model (FIG 8).
The Reference model (FIG 7) is a mesh model of
a normal skull. It has a MSP 704 that serves as the
laterally symmetric plane. It also contains a subset of
surfaces which are the salient surfaces 702.
The Deformed model (FIG 8) contains a set of
bone fragment models 800. These bone fragments are
divided into a subset of fixed bone fragments 802 and a
subset of movable bone fragments 808. Some movable bones
that contain teeth are also known. Each bone fragment
has a mesh representing the bone’s surface. It also
contains a set of salient surfaces and a set of
anatomical landmarks located on it, which can be empty
for bone fragments that do not pass through any
anatomical plane.
The output of the algorithm is the restored
model, which is produced by repositioning the movable
bone fragments in the deformed model.
The restoration algorithm uses a reference
model as a guide. The preparation of the reference model
includes three steps. In the first step, 3D mesh models
of the skull and the lower jaw are segmented from CT
images and reconstructed. In the second step, a human
expert indicates a set of MSP landmarks 806 on the
models using the planning GUI. The MSP is then fitted to
the MSP landmarks 806 by minimizing its mean squared
distance to the landmarks. The origin of the plane is
the mean position of these landmarks, and the normal of
the plane is the principal component corresponding to
the smallest eigenvalue of the covariance matrix of
these landmarks. The laterally symmetrical corresponding
points are computed by with respect to the MSP. In the
third step, salient surfaces of the reference model are
identified in a similar manner as that of the patient’s
deformed model. For the reference model, all the salient
surfaces are identified. In practice, multiple reference
models are prepared, and the one that is most similar to
the patient’s skull is used by the skull restoration algorithm.
The restoration algorithm is an iterative
optimization algorithm that repositions each bone
fragment one at a time. It includes three steps. In the
first step, given a deformed model, the reference model
that is most similar to the deformed model is selected.
In the second step, the restored model is initialized as
the deformed model. In the third step, several sub-steps
are repeated until convergence. The several sub-steps
include
a. Register reference model to the restored model by plane-fitting registration
b. Find corresponding points between the reference and restored model. For each point on the reference model, its corresponding point is the closest point on the restored model.
c. Compute MSP and FP of the restored model
d. Mark fixed bone fragments as confident and movable bone fragments as nonconfident.
e. Compute restoration order of the nonconfident bone fragments
f. For each nonconfident bone fragment in decreasing order, generate the estimated normal surface around the bone fragment, and register the bone fragment to estimated surface around it by surface continuity-constrained registration, and then mark the repositioned bone as confident.
a. Register reference model to the restored model by plane-fitting registration
b. Find corresponding points between the reference and restored model. For each point on the reference model, its corresponding point is the closest point on the restored model.
c. Compute MSP and FP of the restored model
d. Mark fixed bone fragments as confident and movable bone fragments as nonconfident.
e. Compute restoration order of the nonconfident bone fragments
f. For each nonconfident bone fragment in decreasing order, generate the estimated normal surface around the bone fragment, and register the bone fragment to estimated surface around it by surface continuity-constrained registration, and then mark the repositioned bone as confident.
The method for selecting the reference model
that is most similar to the deformed model comprises the
following steps:
Register the reference models to the
deformed model using Plane-Fitting Registration
Algorithm as described in details below and compute
their mean surface distances to the deformed model
Select the reference model with the
smallest mean surface distance over the fixed bone
fragments of the deformed model.
The plane-fitting registration algorithm
registers the reference model to the restored model
robustly and also ensures that the MSP of the reference
model matches the MSP landmarks of the restored model.
It is an iterative algorithm that repeating the
following steps until convergence.
a. Find correspondence between reference
model and restored model. For a point on the restored
model, its corresponding point is its closest point on
the reference model. For a MSP landmark on the restored
model, its corresponding point is its perpendicular
projection on the MSP of the reference model.
b. Find inlier subset of corresponding points with small distances between the corresponding points.
c. Apply the algorithm to compute weighted similarity transformation using correspondence of inlier points. Larger weights are given to correspondences generated from MSP landmarks.
d. Apply the computed similarity transformation on all points of the reference model.
b. Find inlier subset of corresponding points with small distances between the corresponding points.
c. Apply the algorithm to compute weighted similarity transformation using correspondence of inlier points. Larger weights are given to correspondences generated from MSP landmarks.
d. Apply the computed similarity transformation on all points of the reference model.
The method for computing MSP and FP of
restored model comprises the following steps:
1. Transfer the MSP of the reference model to the restored model.
2. Fit the FP to the FP landmarks identified by the user while constraining it to be orthogonal to the MSP.
1. Transfer the MSP of the reference model to the restored model.
2. Fit the FP to the FP landmarks identified by the user while constraining it to be orthogonal to the MSP.
The step 2 includes the following
sub-steps:
a. Project FP landmarks to the MSP orthogonally.
b. Fit a line to the projected landmarks by minimizing least squared distance from the line to the projected landmarks.
c. Compute the plane that is orthogonal to MSP and passes through the fitted line. The origin is any point on the fitted line, and the normal is the cross product the plane normal and the direction of the fitted line.
a. Project FP landmarks to the MSP orthogonally.
b. Fit a line to the projected landmarks by minimizing least squared distance from the line to the projected landmarks.
c. Compute the plane that is orthogonal to MSP and passes through the fitted line. The origin is any point on the fitted line, and the normal is the cross product the plane normal and the direction of the fitted line.
The method for computing the repositioning
order of the nonconfident bones computes the following
scores:
1. Number of anatomical landmarks on the surface of a bone fragment.
2. Fraction of salient surface vertices whose symmetric points are on a confident bone. The symmetry point relationship is transferred from the reference model using the correspondence computed in (b) of the third step of Skull restoration algorithm.
3. Fraction of salient surface vertices that are at the boundary adjacent to confident bones. The boundaries are identified on the reference model at the edges where the two neighboring points lie on two different bone fragments in the restored model. They are transferred to the restored model using the correspondence computed in (b) of the third step of Skull restoration algorithm.
1. Number of anatomical landmarks on the surface of a bone fragment.
2. Fraction of salient surface vertices whose symmetric points are on a confident bone. The symmetry point relationship is transferred from the reference model using the correspondence computed in (b) of the third step of Skull restoration algorithm.
3. Fraction of salient surface vertices that are at the boundary adjacent to confident bones. The boundaries are identified on the reference model at the edges where the two neighboring points lie on two different bone fragments in the restored model. They are transferred to the restored model using the correspondence computed in (b) of the third step of Skull restoration algorithm.
A weighted sum of these scores is computed for
each nonconfident bone fragment, and the bones are
sorted in decreasing order of their weighted sum of
scores. After that, the bones that contain teeth are
moved to the end of the sorted lists so that they are
repositioned last.
The method for generating the estimated normal
surface around a bone fragment comprises the following
steps:
1. Find corresponding points between the reference model and the confident parts of the restoration model that meet several requirements.
2. Apply Laplacian surface deformation algorithm on salient surface of the reference model with corresponding points as hard constraints and lateral symmetry as soft constraints.
3. Cut the deformed surface leaving only the portions that correspond to the bone fragment using the corresponding computed in (b) of the third step of Skull restoration algorithm.
4. Repeating the above steps until convergence of the estimated normal surface.
1. Find corresponding points between the reference model and the confident parts of the restoration model that meet several requirements.
2. Apply Laplacian surface deformation algorithm on salient surface of the reference model with corresponding points as hard constraints and lateral symmetry as soft constraints.
3. Cut the deformed surface leaving only the portions that correspond to the bone fragment using the corresponding computed in (b) of the third step of Skull restoration algorithm.
4. Repeating the above steps until convergence of the estimated normal surface.
Requirements in Step 1 includes:
a. Corresponding points are close to each other.
b. Corresponding points have similar surface normal.
c. Corresponding points lie near to the surface normal of each other.
a. Corresponding points are close to each other.
b. Corresponding points have similar surface normal.
c. Corresponding points lie near to the surface normal of each other.
The surface continuity-constrained
registration algorithm is an iterative algorithm that
registers a nonconfident bone fragment to the estimated
normal surface around. It comprises the following
steps:
Repeat until convergence:
a. Find corresponding points between the salient surface of a nonconfident bone and the estimated normal surface around it. For a point on the salient surface, its corresponding point is the closest point on the estimated normal surface.
b. Find correspondence for anatomical landmarks. For a MSP or FP landmark on the nonconfident bone, its corresponding point is its orthogonal projection on the MSP or FP of the restored model, respectively.
c. Apply the algorithm to compute weighted rigid transformation using the point correspondence. Larger weights are given to the points near the boundaries of nonconfident bone adjacent to confident bones.
d. Apply the computed rigid transformation to all the points of the nonconfident bone.
Repeat until convergence:
a. Find corresponding points between the salient surface of a nonconfident bone and the estimated normal surface around it. For a point on the salient surface, its corresponding point is the closest point on the estimated normal surface.
b. Find correspondence for anatomical landmarks. For a MSP or FP landmark on the nonconfident bone, its corresponding point is its orthogonal projection on the MSP or FP of the restored model, respectively.
c. Apply the algorithm to compute weighted rigid transformation using the point correspondence. Larger weights are given to the points near the boundaries of nonconfident bone adjacent to confident bones.
d. Apply the computed rigid transformation to all the points of the nonconfident bone.
This embodiment provides a computer-aided
procedure for a surgeon to derive a surgery plan for
restoring a patient’s deformed skull or pelvis back to
the normal state by bone respositioning.
In comparison, the embodiment has several
advantages over existing methods. Firstly, the output of
the planning procedure is a restored model which
directly results in a feasible surgery plan. Secondly,
the planning procedure is applicable to both unilateral
and bilateral fractures. Thirdly, the skull restoration
algorithm generates a restored model with anatomical
plane fitting, shape normality, surface continuity, and
lateral symmetry as algorithmic constraints. This makes
the result more accurate and reliable than existing
manual methods, symmetry-based methods, geometric
reconstruction methods, statistical reconstruction
methods, and fracture surface matching methods.
Also, the proposed planning procedure has
several advantages over existing procedures. The
proposed procedure provides an automatic FP, MSP and
landmark identification method for skulls with minor
injuries. And the proposed procedure replaces the
reflection method of existing procedure by an automatic
restoration algorithm. Furthermore, the proposed
procedure generates a restored model by bone
repositioning whereas the existing procedure generates a
partially reconstructed model by lateral symmetry.
This embodiment provides more accurate
pre-operative planning of craniomaxillofacial (CMF)
surgery for the skull and orthopedic surgery for the
pelvis. This embodiment also provides training of CMF
and orthopedic surgeons in precision recognition of
deformed bones as well as planning and execution of the
plan in the operating theatre.
This embodiment is applicable to large bone
structures such as the skull and the pelvis where all
the surgical decisions are made according to the shape
of the bone. For orthopedic surgeries on long tubular
bones, such as the arm and thigh bones, the surgical
decisions are based on the relative orientations of the
fractures bones rather than the shape of the bones.
In addition, there are four possible
modifications of the preferred embodiments. Firstly,
salient surfaces can be detected by an automatic
algorithm. In the present embodiment, the outer surface
is identified automatically, and the eye socket region
is identified manually. However, the eye socket region
can also be identified automatically by fitting a sphere
to the eye socket region in a manner similar to the
foramen magnum identification method.
Secondly, reference model can be an active
shape model. In the present embodiment, the reference
model is a single normal model selected from a database.
However, it can also be generated by fitting an active
shape model to the normal part of the patient’s model,
as done for statistical reconstruction.
Thirdly, collision can be avoided. In the
above embodiment, the skull restoration algorithm does
not explicitly detect collisions of the restored bone
fragments. Instead, it relies on the accurate estimation
of normal surfaces and registration of bone fragments to
the estimated normal surface to naturally avoid
collisions. Experiments showed that the collision error
is small or most cases. For cases where the collision is
not negligible, collision detection avoidance can be
performed. Collision detection is a well solved problem.
There are various algorithms and packages available,
e.g. proximity query package and vtkbioeng, etc. One
modification is to check for collision after the skull
is restored, and move the collided bones apart to reduce
or remove collisions.
Fourthly, the present embodiment can be
applied to orthopedic surgery of pelvis bone. In the
above embodiments, CMF surgery is used as an example to
describe the procedure and the algorithm. However, a
similar procedure and algorithm can be applied to
orthopedic surgery of pelvis bones. One of the
differences is that Frankfurt plan is not defined for
pelvis. For pelvis restoration, either a different
horizontal plan is defined or the horizontal plane is
ignored in the restoration algorithm. One of the
differences is that corresponding salient surfaces need
to be either detected automatically or identified manually.
It should further be appreciated that the
exemplary embodiments are only examples, and are not
intended to limit the , applicability, operation, or
configuration of the invention in any way. Rather, the
foregoing detailed description will provide those
skilled in the with a convenient road map for
implementing an exemplary embodiment of the invention,
it being understood that various changes may be made in
the function and arrangement of elements and method of
operation described in an exemplary embodiment without
departing from the scope of the invention as set forth
in the appended claims.
100 Flow diagram of the computer-aided CMF
surgery planning procedure
102 Segmentation and 3D model construction
104 Identification of bone fragments to be repositioned
106 Identification of salient surfaces on the skull
108 Identification of MSP and FP landmarks
110 Execution of skull restoration algorithm
112 Generation of planning output
114 CT images
116 Deformed model
118 Bones to be moved
120 Salient surfaces
122 MSP/FP landmarks
124 Restored model
126 Output files
200 Each stage of the planning procedure
202 Bone fragments to be repositioned
204 FP landmarks
206 MSP landmarks
300 Planning tool for indicating movable bone fragments and anatomical landmarks
400 Identification of salient surfaces
500 Comparison of Restored skull and Deformed skull
504 FP landmarks
506 MSP landmarks
600 Synthesized DICOM images of the restored model
700 Reference model
702 Salient surface
704 MSP
800 Patient’s deformed model
802 Unfractured undisplaced (“fixed”) bone
804 FP landmarks
806 MSP landmarks
808 Movable bone
102 Segmentation and 3D model construction
104 Identification of bone fragments to be repositioned
106 Identification of salient surfaces on the skull
108 Identification of MSP and FP landmarks
110 Execution of skull restoration algorithm
112 Generation of planning output
114 CT images
116 Deformed model
118 Bones to be moved
120 Salient surfaces
122 MSP/FP landmarks
124 Restored model
126 Output files
200 Each stage of the planning procedure
202 Bone fragments to be repositioned
204 FP landmarks
206 MSP landmarks
300 Planning tool for indicating movable bone fragments and anatomical landmarks
400 Identification of salient surfaces
500 Comparison of Restored skull and Deformed skull
504 FP landmarks
506 MSP landmarks
600 Synthesized DICOM images of the restored model
700 Reference model
702 Salient surface
704 MSP
800 Patient’s deformed model
802 Unfractured undisplaced (“fixed”) bone
804 FP landmarks
806 MSP landmarks
808 Movable bone
Claims (23)
- A method for reconstructive modeling of a deformed model, the method comprising:initializing a restored model as the deformed model;registering a reference model to the restored model by plane-fitting registration;identifying corresponding points between the reference model and the restored model, wherein a corresponding point on the restored model for each point on the reference model is the closest point on the restored model;computing a mid-sagittal plane (MSP) and a Frankfurt plane (FP) of the restored model;associating one of a first property and a second property with fragments of the restored model;calculating a restoration order of fragments of the restored model having the second property associated therewith;for each of the fragments of the restored model having the second property associated therewith and in accordance with the calculated restoration order:generating an estimated normal surface around the fragment;registering the fragment to an estimated surface proximate thereto by a surface continuity-constrained registration;associating the first property with fragments repositioned; andrepeating all steps except the initializing step until convergence of all of the fragments of the restored model.
- The method in accordance with Claim 1 further comprising selecting the reference model from a plurality of reference models in response to a similarity between the reference model and the deformed model.
- The method in accordance with Claim 2 wherein the step of selecting the reference model comprises:registering each of the plurality of reference models to the deformed model by the plane-fitting registration;calculating mean surface distances to the fragments of the restored model having the first property associated therewith; andselecting one of the plurality of the reference models as the reference model in response to the one of the plurality of the reference models having a smallest mean surface distance over the fragments of the restored model having the first property associated therewith.
- The method in accordance with any of the foregoing claims wherein the plane-fitting registration comprises:finding a correspondence between the reference model and the restored model by defining a corresponding point of a point on the restored model as a closest point on the reference model and defining a corresponding point of a landmark on the restored model as a perpendicular projection of a corresponding landmark on the mid-sagittal plane (MSP) of the reference model;defining inlier points as an inlier subset of corresponding points in response to small distances between the corresponding points;calculating a weighted similarity transformation in response to a correspondence of the inlier points wherein larger weights are given to correspondences generated from the corresponding landmarks;applying the calculated similarity transformation on all points of the reference model; andrepeating these steps until convergence of all the points on the restored model.
- The method in accordance with any of the foregoing claims wherein the method comprises a method for computing a Frankfurt plane (FP) and a mid-sagittal plane (MSP) of the restored model.
- The method in accordance with Claim 5 wherein the step of computing the Frankfurt plane (FP) and the mid-sagittal plane (MSP) of the restored model comprises the steps of:transferring the MSP of the reference model to the restored model; andfitting the FP to FP landmarks identified by the user while constraining it to be orthogonal to the MSP by projecting the FP landmarks to the MSP orthogonally, fitting a line to the projected FP landmarks by minimizing a least squared distance from the line to the projected FP landmarks, and computing a plane that is orthogonal to the MSP and passes through the fitted line, wherein an origin is any point on the fitted line and a normal is a cross product of a normal of the plane and a direction of the fitted line.
- The method in accordance with any of the foregoing claims wherein the fragments of the restored model comprise bone fragments, the first property comprises a confident property, the second property comprises a nonconfident property, and the bone fragments comprise fixed bone fragments and movable bone fragments, and wherein the step of associating the first property and the second property with the bone fragments of the restored model comprises associating the confident property with the fixed bone fragments and associating the nonconfident property with the movable bone fragments.
- The method in accordance with Claim 7 wherein the step of calculating the restoration order comprises computing a repositioning order of the movable bone fragments in response to weighted sums of the following for each of the movable bone fragments:(a) a number of anatomical landmarks on a surface of one of the movable bone fragments,(b) a fraction of salient surface vertices on the one of the movable bone fragments whose symmetric points are on one of the bone fragments of the restored model having the first property associated therewith, and(c) a fraction of salient surface vertices on the one of the movable bone fragments that are at a boundary adjacent to one of the bone fragments of the restored model having the first property associated therewith.
- The method in accordance with Claim 8 wherein the step of computing the repositioning order of the movable bone fragments in response to the weighted sums of each of the movable bone fragments comprises computing the repositioning order of the movable bone fragments in response to a decreasing value of the weighted sums of each of the movable bone fragments.
- The method in accordance with Claim 9 wherein the step of computing the repositioning order of the movable bone fragments in response to a decreasing value of the weighted sums of each of the movable bone fragments comprises:identifying ones of the movable bone fragments that contain teeth;computing the repositioning order of the movable bone fragments in response to a decreasing value of the weighted sums of each of the movable bone fragments; andmoving the ones of the movable bone fragments that contain teeth to the end of the repositioning order so that the ones of the movable bone fragments that contain teeth are repositioned last.
- The method in accordance with any of the foregoing claims wherein the step of generating an estimated normal surface around each fragment of the restored model having the second property associated therewith comprises:finding corresponding points between the reference model and the fragment of the restored model having the first property associated therewith that have (a) corresponding points are close to each other, (b) corresponding points have similar surface normal, and (c) corresponding points lie near to the surface normal of each other;determining a surface deformation on a salient surface of the reference model in accordance with the corresponding points and lateral symmetry to the fragment of the restored model having the first property associated therewith;cutting the surface deformation on the salient surface of the reference model leaving only the portions that correspond to closest points on the fragment of the restored model having the second property associated therewith; andrepeating the above steps until convergence of the estimated normal surface.
- The method in accordance with any of the foregoing claims wherein the surface continuity-constrained registration comprises an iterative calculation until convergence that registers each fragment of the restored model having the second property associated therewith to an estimated normal surface around it in response to (a) closest points between a salient surface of the fragment of the restored model having the second property associated therewith and an estimated normal surface around it, (b) the landmark points identified by the user on the fragment of the restored model having the second property associated therewith and the perpendicular projections of the landmark points on corresponding planes of the restored model, (c) weighted rigid transformation of corresponding points between the fragment of the restored model having the second property associated therewith and the estimated normal surface around the fragment of the restored model having the second property associated therewith, wherein larger weights are given to points near boundaries of the fragment of the restored model having the second property associated therewith adjacent to fragments of the restored model having the first property associated therewith, and (d) rigid transformation to all the points of the fragment of the restored model having the second property associated therewith.
- A method for reconstructive surgery planning comprising:providing images of one or more bones in an area to be reconstructed, the one or more bones including one or more bone fragments;segmenting the one or more bones into a plurality of three-dimensional mesh representations such that each of the one or more bone fragments are a separate one of the plurality of three-dimensional mesh representations;assigning one of a first property or a second property to each of the plurality of three-dimensional mesh representations; andreconstructing the area by repositioning the three-dimensional mesh representations having the second property in response to ones of the three-dimensional mesh representations having the first property and a reference model.
- The method in accordance with Claim 13 wherein the step of providing images comprises providing computed tomography (CT) images of the one or more bones in the area to be reconstructed.
- The method in accordance with either Claim 13 or Claim 14 wherein the step of reconstructing the area comprises reconstructing the area by repositioning the three-dimensional mesh representations having the second property in response to (a) identified landmarks, (b) the ones of the three-dimensional mesh representations having the first property, (c) identified salient surfaces of the three-dimensional mesh representations, and (d) the reference model, wherein the salient surfaces of the three-dimensional mesh representations are surfaces of the three-dimensional mesh representations that should be flush with surfaces of the three-dimensional mesh representations of adjacent ones of the one or more bone fragments after reconstructing the area.
- The method in accordance with any of Claims 13 to 15 wherein the reconstructive surgery comprises craniomaxillofacial surgery and/or orthopedic surgery, and wherein ones of the plurality of three-dimensional mesh representations having the first property comprise three-dimensional mesh representations of fixed bone fragments, and wherein ones of the plurality of three-dimensional mesh representations having the second property comprise three-dimensional mesh representations of movable bone fragments.
- The method in accordance with Claim 16 wherein the reconstructive surgery comprises craniomaxillofacial surgery, and wherein salient surfaces of the three-dimensional mesh representations of the one or more bone fragments comprise surfaces of the three-dimensional mesh representations of the one or more bone fragments that should be flush with surfaces of the three-dimensional mesh representations of adjacent ones of the one or more bone fragments after reconstruction and the salient surfaces include outer surfaces of the skull and surfaces around eye sockets.
- The method in accordance with Claim 17 wherein the identified landmarks comprise identified landmarks on a Frankfurt plane (FP) and a mid-sagittal plane (MSP), and wherein the reference model comprises a mesh model of a normal skull having a MSP that serves as a laterally symmetric plane and comprises a subset of surfaces corresponding to the salient surfaces of the three-dimensional mesh representations of the one or more bone fragments.
- The method in accordance with Claim 18 wherein the reference model is selected from a plurality of reference models in response to one or more similarities with a skull of a patient of the craniomaxillofacial surgery.
- The method in accordance with either Claim 18 or Claim 19 wherein the step of assigning ones of the plurality of three-dimensional mesh representations of a skull of a patient as movable bone fragments comprises selecting ones of the plurality of three-dimensional mesh representations in a computer display of the plurality of three-dimensional mesh representations.
- The method in accordance with Claim 20 further comprising identifying a subset of the three-dimensional mesh representations of movable bone fragments as three-dimensional mesh representations of movable bone fragments having teeth, and wherein the step of reconstructing the area comprises reconstructing the area by repositioning the three-dimensional mesh representations having movable bone fragments in response to (a) the identified landmarks, (b) the ones of the three-dimensional mesh representations having fixed bone fragments, (c) the identified salient surfaces of the three-dimensional mesh representations of the bone fragments, (d) the reference model, and (e) ones of the three-dimensional mesh representations of movable bone fragments having teeth.
- A method for computer-aided craniomaxillofacial surgery planning comprising:taking computed tomography (CT) images of a patient’s skull;segmenting and constructing a three-dimensional deformed model of the patient’s skull from the CT images, wherein the three-dimensional deformed model comprises a plurality of three-dimensional meshes, each of the plurality of three-dimensional meshes corresponding to one of a plurality of bone fragments of the patient’s skull;indicating ones of the plurality of bone fragments to be repositioned by selecting the ones of the plurality of bone fragments on a computer display of the plurality of three-dimensional meshes corresponding to one of a plurality of bone fragments, wherein ones of the plurality of bone fragments on the computer display not selected being identified as bone fragments not to be repositioned;indicating ones of the plurality of bone fragments to be repositioned that contain teeth;identifying salient surfaces on the ones of the plurality of bone fragments to be repositioned, wherein salient surfaces are surfaces of the ones of the plurality of bone fragments that should be flush with surfaces of other ones of the plurality of bone fragments after restoration and include outer surfaces of the patient’s skull and surfaces around eye sockets of the patient’s skull;identifying landmarks associated with a mid-sagittal plane (MSP) and a Frankfurt plane (FP) ;generating a restored model from the three-dimensional deformed model of the patient’s skull in response to the three-dimensional deformed model of the patient’s skull and a reference model of a normal person’s skull by repositioning the plurality of bone fragments in the three-dimensional deformed model to normalize an overall shape of the patient’s skull by locating the MSP and FP landmarks proximately close to a calculated MSP and FP of the patient’s skull, placing the movable bones proximately close to an estimated normal and laterally symmetric shape, and placing boundaries of adjacent bone fragments flush with each other; andexporting data on the repositioned plurality of bone fragments for completion of a surgery plan and/or surgical verification with post-operative CT scans.
- The method in accordance with Claim 22 wherein the step of segmenting and constructing the three-dimensional deformed model of the patient’s skull comprises segmenting only ones of the plurality of bone fragments that need to be repositioned in the surgery as individual ones of the plurality of three-dimensional meshes, the ones of the plurality of bone fragments identified as bone fragments not to be repositioned being segmented as a single mesh.
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
SG10201400827T | 2014-03-20 | ||
SG10201400827T | 2014-03-20 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2015142291A1 true WO2015142291A1 (en) | 2015-09-24 |
Family
ID=54145079
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/SG2015/050045 WO2015142291A1 (en) | 2014-03-20 | 2015-03-20 | Computer-aided planning of craniomaxillofacial and orthopedic surgery |
Country Status (1)
Country | Link |
---|---|
WO (1) | WO2015142291A1 (en) |
Cited By (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN105963005A (en) * | 2016-04-25 | 2016-09-28 | 华南理工大学 | Method for producing funnel chest correction plate |
CN111513719A (en) * | 2020-04-30 | 2020-08-11 | 赤峰学院附属医院 | Analysis method and device for craniomaxillary surface state and electronic equipment |
CN111968236A (en) * | 2020-07-17 | 2020-11-20 | 哈尔滨理工大学 | K-D tree method-based contraposition to line fracture reduction system |
US10937250B2 (en) | 2016-12-06 | 2021-03-02 | National University Of Singapore | Methods of reconstructing skulls |
CN115153838A (en) * | 2022-05-16 | 2022-10-11 | 上海交通大学医学院附属第九人民医院 | Rapid craniomaxillofacial bone tissue operation navigation registration guide plate and registration method thereof |
WO2023153606A1 (en) * | 2022-02-14 | 2023-08-17 | 주식회사 에이치디엑스윌 | Device and method for reconstructing three-dimensional oral scan data by using computed tomography image |
Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2002021452A2 (en) * | 2000-09-11 | 2002-03-14 | Siemens Aktiengesellschaft | Separating objects out of a medical 3d data set |
WO2006000063A1 (en) * | 2004-06-25 | 2006-01-05 | Medicim Nv | Method for deriving a treatment plan for orthognatic surgery and devices therefor |
-
2015
- 2015-03-20 WO PCT/SG2015/050045 patent/WO2015142291A1/en active Application Filing
Patent Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2002021452A2 (en) * | 2000-09-11 | 2002-03-14 | Siemens Aktiengesellschaft | Separating objects out of a medical 3d data set |
WO2006000063A1 (en) * | 2004-06-25 | 2006-01-05 | Medicim Nv | Method for deriving a treatment plan for orthognatic surgery and devices therefor |
Non-Patent Citations (1)
Title |
---|
M. J. TROULIS ET AL.: "Development of a three-dimensional treatment planning system based on computed tomographic data", INT. J. ORAL MAXILLOFAC. SURG., vol. 31, no. 4, August 2002 (2002-08-01), pages 349 - 357, XP008053607 * |
Cited By (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN105963005A (en) * | 2016-04-25 | 2016-09-28 | 华南理工大学 | Method for producing funnel chest correction plate |
US10937250B2 (en) | 2016-12-06 | 2021-03-02 | National University Of Singapore | Methods of reconstructing skulls |
CN111513719A (en) * | 2020-04-30 | 2020-08-11 | 赤峰学院附属医院 | Analysis method and device for craniomaxillary surface state and electronic equipment |
CN111513719B (en) * | 2020-04-30 | 2021-08-17 | 赤峰学院附属医院 | Analysis method and device for craniomaxillary surface state and electronic equipment |
CN111968236A (en) * | 2020-07-17 | 2020-11-20 | 哈尔滨理工大学 | K-D tree method-based contraposition to line fracture reduction system |
WO2023153606A1 (en) * | 2022-02-14 | 2023-08-17 | 주식회사 에이치디엑스윌 | Device and method for reconstructing three-dimensional oral scan data by using computed tomography image |
CN115153838A (en) * | 2022-05-16 | 2022-10-11 | 上海交通大学医学院附属第九人民医院 | Rapid craniomaxillofacial bone tissue operation navigation registration guide plate and registration method thereof |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US10991070B2 (en) | Method of providing surgical guidance | |
US11937888B2 (en) | Artificial intelligence intra-operative surgical guidance system | |
WO2015142291A1 (en) | Computer-aided planning of craniomaxillofacial and orthopedic surgery | |
US10937250B2 (en) | Methods of reconstructing skulls | |
JP6404310B2 (en) | Planning system and method for surgical correction of abnormal bone | |
US20200188026A1 (en) | Systems and methods for preoperative planning and postoperative analysis of surgical procedures | |
US20160331463A1 (en) | Method for generating a 3d reference computer model of at least one anatomical structure | |
Grupp et al. | Pose estimation of periacetabular osteotomy fragments with intraoperative X-ray navigation | |
Albrecht et al. | Automatic fracture reduction | |
US11540794B2 (en) | Artificial intelligence intra-operative surgical guidance system and method of use | |
Irwansyah et al. | Algorithm for segmentation and reduction of fractured bones in computer-aided preoperative surgery | |
Morooka et al. | A survey on statistical modeling and machine learning approaches to computer assisted medical intervention: Intraoperative anatomy modeling and optimization of interventional procedures | |
Lu et al. | Preoperative virtual reduction planning algorithm of fractured pelvis based on adaptive templates | |
Probst et al. | Accuracy of free-hand positioned patient specific implants (PSI) in primary reconstruction after inferior and/or medial orbital wall fractures | |
Kronman et al. | Automatic bone fracture reduction by fracture contact surface identification and registration | |
Rasoulian et al. | Group-wise feature-based registration of CT and ultrasound images of spine | |
US11452566B2 (en) | Pre-operative planning for reorientation surgery: surface-model-free approach using simulated x-rays | |
Khallaghi et al. | Biomechanically constrained groupwise statistical shape model to ultrasound registration of the lumbar spine | |
US20240206990A1 (en) | Artificial Intelligence Intra-Operative Surgical Guidance System and Method of Use | |
Sun | A Review of 3D-2D Registration Methods and Applications based on Medical Images | |
Bataeva et al. | Intraoperative guidance of orthopaedic instruments using 3D correspondence of 2D object instance segmentations | |
Sepasian et al. | Bone reposition planning for corrective surgery using statistical shape model: assessment of differential geometrical features | |
Luan et al. | Point extraction from cross sections of fractured long bones for registration | |
Zeng et al. | A bidirectional framework for fracture simulation and deformation-based restoration prediction in pelvic fracture surgical planning | |
JP2024099519A (en) | Artificial intelligence intraoperative surgical guidance systems and methods of use |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 15764528 Country of ref document: EP Kind code of ref document: A1 |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 15764528 Country of ref document: EP Kind code of ref document: A1 |