WO2015142291A1

WO2015142291A1 - Computer-aided planning of craniomaxillofacial and orthopedic surgery

Info

Publication number: WO2015142291A1
Application number: PCT/SG2015/050045
Authority: WO
Inventors: Yuan Cheng; Wee Kheng Leow; Thiam Chye Lim
Original assignee: National University Of Singapore
Priority date: 2014-03-20
Filing date: 2015-03-20
Publication date: 2015-09-24

Abstract

A procedure for computer-aided planning of craniomaxillofacial (CMF) and orthopedic surgery and scheme for generating a restored skull or pelvis model from a patient's fractured skull or pelvis model for computer-aided surgery planning are provided. In one embodiment, an algorithm for generating a restored model from a patient's deformed model by bone repositioning is presented along with a computer aided procedure for assisting a surgeon in deriving a CMF or orthopedic surgery plan.

Description

COMPUTER-AIDED PLANNING OF CRANIOMAXILLOFACIAL AND ORTHOPEDIC SURGERY

Priority Claim

The present application claims priority to Singapore Patent Application No. 10201400827T.

The present invention relates to surgery. In particular, it relates to computer-aided planning of craniomaxillofacial and orthopedic surgery.

Many patients suffer from skull deformity and pelvis deformity. For example, in North America, every year, around 1.58 million people suffer from head and facial injuries due to traffic accidents, work accidents, home accidents, sports injuries, and violence. About 10% of trauma patients treated by a Level 1 Trauma Centre (e.g., a general hospital) have pelvic fracture, and the mortality rate of patients with pelvic fracture ranges from 10% to 16%. In Singapore, National University Hospital (NUH) alone receives about 250–350 patients with facial fracture every year. Bone deformation often leads to shape distortion as well as incomplete functionality because the muscles and skin attached to a deformed bone are also deformed. These deformities may greatly affect the patients’ life quality and even threaten their lives.

To restore the bones to their normal states, craniomaxillofacial (CMF) and orthopedic reconstructive surgeries are performed. These surgeries involve complex operations on the skull or pelvis. In addition to the procedural complexity, difficulties of performing such surgeries also result from the variation of patients’ anatomical structures and the severity of the deformation. Therefore, careful pre-operative planning of surgery is crucial for the success of the operation.

There are two related problems in CMF and orthopedic surgery planning, namely restoration and reconstruction. Restoration seeks to restore a deformed skull or pelvis back to its normal state by repositioning the fractured bones, which directly results in a feasible surgery plan. Reconstruction, on the other hand, derives an estimation of the normal shape from the deformed skull or pelvis by shape similarity. The detailed shapes of the bones in the reconstructed model are not necessarily the same as those in the patient’s skull or pelvis. It may not be possible to obtain the reconstructed model by repositioning fractured bones in the deformed model. To use the reconstructed model for surgery planning, the surgeon needs to manually work out how to reposition the patient’s fractured bones to match a reference as given by the reconstructed model.

Some existing computer-aided CMF or orthopedic surgery systems can reconstruct estimations of the skull or pelvis before injury. But, there is no existing planning system that generates a restored skull or pelvis from a patient’s deformed skull or pelvis by repositioning fractured bones.

Conventional methods for solving the restoration and reconstruction problems include manual manipulation, symmetry-based reconstruction, geometric reconstruction, statistical reconstruction and fracture surface matching. Manual manipulation methods display a three-dimensional (3D) model of the patient’s skull in the computer monitor and allow the surgeon to manually cut and reposition the bones in the 3D model to generate a restored model. Such methods are relatively easy to implement, and give the surgeon maximum control over how the bones in the deformed model should be repositioned. However, as bone repositioning is planned by the surgeon manually, such manual manipulation methods can be very tedious, time-consuming, and inaccurate.

Symmetry-based reconstruction methods reflect the healthy parts of a deformed model about a symmetry plane and use the reflected parts as estimates of the normal shapes of the fractured parts. While the symmetry-based reconstruction methods do not require any reference model, they do require the presence of healthy symmetric parts to reconstruct normal shape on the fractured side of the skull. When both sides of the skull are fractured, which is common in impact injuries, these methods are inapplicable.

Geometric reconstruction methods register a reference model to the healthy parts of a deformed model and use the registered reference model to estimate the normal shapes of the fractured parts. These methods do not require the presence of healthy symmetric parts. However, they use a single reference model to generate the reconstructed model and their reconstruction accuracy depends on the similarity between the reference model and the normal model, as well as the degree of the correlation between the healthy parts and deformed parts. Statistical reconstruction methods overcome the limitation of geometric reconstruction by matching a statistical reference model to the healthy parts of a deformed model and using the matched statistical model to infer the normal shapes of the fractured parts of the deformed model. As such methods require a statistical model, statistical reconstruction methods require a large amount of training samples of healthy skull models to construct the statistical model. In addition, their reconstruction accuracy depends on how well the statistical model captures the normal shape variation of the patient’s normal shape. In the case where such normal shape variation is not adequately captured, only the mean shape of the statistical reference can be used. Then, statistical reconstruction approximates geometric reconstruction and exhibits the problems associated with such conventional methods. Moreover, similar to geometric reconstruction methods, statistical reconstruction methods rely on the correlation between the healthy parts and the deformed parts.

Fracture surface matching methods reposition fractured bones by determining their correct relative positions and orientations based on shape complementarity of adjacent fracture surfaces. They do not require any reference models or the presence of healthy parts in the deformed model. Instead, they produce the restored model by computing the correct positions and orientations of the fractured bone fragments. These methods, however, require the features of the fracture surfaces to be well captured in the deformed model. In the case of impact injuries where the fracture surfaces abrade each other destroying shape features, fracture surface matching methods become inaccurate.

Computer-aided systems have also been developed for planning CMF surgery and orthopedic surgery. Some of these conventional systems allow the surgeon to manually reposition the bones and then predict the facial appearance. However, as described above, manual repositioning of bones is tedious, time-consuming and highly inaccurate. Some other conventional systems generate reconstructed models that can be used to guide manual planning, such as generating reconstructed models using symmetry-based method. These methods, however, are not applicable to patients with bilateral fractures, i.e., fractures on both sides of the skull. Further, the output of these methods is typically a reconstructed model instead of a restored model, which can only be used as a reference model for surgery (i.e., the surgeon still needs to manually work out how to reposition the bones according to the reference).

Another conventional software package provides manual surgery planning service to surgeons where the planning can be performed manually by the surgeon’s staff. This procedure, however, shares the same limitation as manual restoration including the limitation that it is tedious, time-consuming and inaccurate. Several other software systems can generate surgery plans for mandible restoration using a fractured surface matching method. Yet, this method is also not reliable because it incorporates the limitations of the fractured surface matching method used.

Thus, what is needed is an automatic, reliable and accurate method for generating a restored model of a patient’s skull or pelvis from his/her deformed skull or pelvis. More particularly, what is needed is a computer-aided system that provides assistance for generating a CMF or orthopedic surgery plan based on restoration of fractured skull or pelvis models. Furthermore, other desirable features and characteristics will become apparent from the subsequent detailed description and the appended claims, taken in conjunction with the accompanying drawings and this background of the disclosure.

This invention relates to a procedure for computer-aided planning of craniomaxillofacial and orthopedic surgery and a scheme for generating a restored skull or pelvis model from a patient’s fractured skull or pelvis model for computer-aided surgery planning. In one embodiment, a novel algorithm for generating a restored model from a patient’s deformed model by bone repositioning is presented along with a computer-aided procedure for assisting a surgeon in deriving a CMF or orthopedic surgery plan.

In accordance with this first embodiment a method for reconstructive modeling of a deformed model is provided which includes the steps of initializing a restored model as the deformed model and registering a reference model to the restored model by plane-fitting registration. The method also includes identifying corresponding points between the reference model and the restored model, wherein a corresponding point on the restored model for each point on the reference model is the closest point on the restored model, and computing a mid-sagittal plane (MSP) and a Frankfurt plane (FP) of the restored model. The method further includes associating one of a first property and a second property with fragments of the restored model and calculating a restoration order of fragments of the restored model having the second property associated therewith. And, for each of the fragments of the restored model having the second property associated therewith and in accordance with the calculated restoration order, the method includes generating an estimated normal surface around the fragment, registering the fragment to an estimated surface proximate thereto by a surface continuity-constrained registration, and associating the first property with fragments repositioned. Finally, the embodiment includes repeating all steps of the method except the initializing step until convergence of all of the fragments of the restored model.

In one aspect, the method includes selecting the reference model from a plurality of reference models in response to a similarity between the reference model and the deformed model. In another aspect, the step of selecting the reference model includes registering each of the plurality of reference models to the deformed model by the plane-fitting registration, calculating mean surface distances to the fragments of the restored model having the first property associated therewith, and selecting one of the plurality of the reference models as the reference model in response to the one of the plurality of the reference models having a smallest mean surface distance over the fragments of the restored model having the first property associated therewith.

In yet another aspect of the embodiment, the plane-fitting registration includes finding a correspondence between the reference model and the restored model by defining a corresponding point of a point on the restored model as a closest point on the reference model and defining a corresponding point of a landmark on the restored model as a perpendicular projection of a corresponding landmark on the mid-sagittal plane (MSP) of the reference model, defining inlier points as an inlier subset of corresponding points in response to small distances between the corresponding points, calculating a weighted similarity transformation in response to a correspondence of the inlier points wherein larger weights are given to correspondences generated from the corresponding landmarks, applying the calculated similarity transformation on all points of the reference model, and repeating these steps until convergence of all the points on the restored model.

In yet another aspect, the method of computing the mid-sagittal plane (MSP) and Frankfurt plane (FP) of the restored model includes transferring the MSP of the reference model to the restored model and fitting the FP to FP landmarks identified by the user while constraining it to be orthogonal to the MSP by projecting the FP landmarks to the MSP orthogonally, fitting a line to the projected FP landmarks by minimizing a least squared distance from the line to the projected FP landmarks, and computing a plane that is orthogonal to the MSP and passes through the fitted line, wherein an origin is any point on the fitted line and a normal is a cross product of a normal of the plane and a direction of the fitted line.

In accordance with another aspect of the embodiment, the fragments of the restored model includes bone fragments, the first property comprises a confident property, the second property comprises a nonconfident property, and the bone fragments comprise fixed bone fragments and movable bone fragments. In regards to this aspect, the step of associating the first property and the second property with the bone fragments of the restored model includes associating the confident property with the fixed bone fragments and associating the nonconfident property with the movable bone fragments. In accordance with another aspect of the embodiment, the step of calculating the restoration order includes computing a repositioning order of the movable bone fragments in response to weighted sums of the following for each of the movable bone fragments:

(a) a number of anatomical landmarks on a surface of one of the movable bone fragments,

(b) a fraction of salient surface vertices on the one of the movable bone fragments whose symmetric points are on one of the bone fragments of the restored model having the first property associated therewith, and

(c) a fraction of salient surface vertices on the one of the movable bone fragments that are at a boundary adjacent to one of the bone fragments of the restored model having the first property associated therewith.

This step of computing the repositioning order of the movable bone fragments in response to the weighted sums of each of the movable bone fragments includes computing the repositioning order of the movable bone fragments in response to a decreasing value of the weighted sums of each of the movable bone fragments. This step of computing the repositioning order of the movable bone fragments in response to a decreasing value of the weighted sums of each of the movable bone fragments may also include identifying ones of the movable bone fragments that contain teeth, computing the repositioning order of the movable bone fragments in response to a decreasing value of the weighted sums of each of the movable bone fragments, and moving the ones of the movable bone fragments that contain teeth to the end of the repositioning order so that the ones of the movable bone fragments that contain teeth are repositioned last.

In accordance with another aspect of the embodiment, the step of generating an estimated normal surface around each fragment of the restored model having the second property associated therewith includes finding corresponding points between the reference model and the fragment of the restored model having the first property associated therewith that have (a) corresponding points are close to each other, (b) corresponding points have similar surface normal, and (c) corresponding points lie near to the surface normal of each other, determining a surface deformation on a salient surface of the reference model in accordance with the corresponding points and lateral symmetry to the fragment of the restored model having the first property associated therewith, and cutting the surface deformation on the salient surface of the reference model leaving only the portions that correspond to closest points on the fragment of the restored model having the second property associated therewith, the above steps being repeated until convergence of the estimated normal surface.

And in accordance with a final aspect of the embodiment, the surface continuity-constrained registration includes an iterative calculation until convergence that registers each fragment of the restored model having the second property associated therewith to an estimated normal surface around it in response to (a) closest points between a salient surface of the fragment of the restored model having the second property associated therewith and an estimated normal surface around it, (b) the landmark points identified by the user on the fragment of the restored model having the second property associated therewith and the perpendicular projections of the landmark points on corresponding planes of the restored model, and (c) weighted rigid transformation of corresponding points between the fragment of the restored model having the second property associated therewith and the estimated normal surface around the fragment of the restored model having the second property associated therewith, wherein larger weights are given to points near boundaries of the fragment of the restored model having the second property associated therewith adjacent to fragments of the restored model having the first property associated therewith.

In accordance with another embodiment, a method for reconstructive surgery planning is provided. The method includes providing images of one or more bones in an area to be reconstructed, the one or more bones including one or more bone fragments, segmenting the one or more bone fragments into a plurality of three-dimensional mesh representations such that each of the one or more bone fragments are a separate one of the plurality of three-dimensional mesh representations, assigning one of a first property or a second property to each of the plurality of three-dimensional mesh representations, and reconstructing the area by repositioning the three-dimensional mesh representations having the second property in response to ones of the three-dimensional mesh representations having the first property and a reference model.

In accordance with an aspect of this embodiment, the step of providing images includes providing computed tomography (CT) images of the one or more bones in the area to be reconstructed. In accordance with another aspect, the step of reconstructing the area comprises reconstructing the area by repositioning the three-dimensional mesh representations having the second property in response to (a) identified landmarks, (b) the ones of the three-dimensional mesh representations having the first property, (c) identified salient surfaces of the three-dimensional mesh representations, and (d) the reference model, wherein the salient surfaces of the three-dimensional mesh representations are surfaces of the three-dimensional mesh representations that should be flush with surfaces of the three-dimensional mesh representations of adjacent ones of the one or more bone fragments after reconstructing the area.

In accordance with yet another aspect of this embodiment, the reconstructive surgery includes craniomaxillofacial surgery and/or orthopedic surgery, and ones of the plurality of three-dimensional mesh representations having the first property include three-dimensional mesh representations of fixed bone fragments, and ones of the plurality of three-dimensional mesh representations having the second property include three-dimensional mesh representations of movable bone fragments.

In accordance with another aspect, the reconstructive surgery is craniomaxillofacial surgery and salient surfaces of the three-dimensional mesh representations of the one or more bone fragments include surfaces of the three-dimensional mesh representations of the one or more bone fragments that should be flush with surfaces of the three-dimensional mesh representations of adjacent ones of the one or more bone fragments after reconstruction and the salient surfaces include outer surfaces of the skull and surfaces around eye sockets. In accordance with this aspect, the landmarks comprise landmarks on a Frankfurt plane (FP) and a mid-sagittal plane (MSP), and the reference model includes a mesh model of a normal skull having a MSP that serves as a laterally symmetric plane and includes a subset of surfaces corresponding to the salient surfaces of the three-dimensional mesh representations of the one or more bone fragments. The reference model may also be selected from a plurality of reference models in response to one or more similarities with a skull of a patient of the craniomaxillofacial surgery. The step of assigning ones of the plurality of three-dimensional mesh representations of a skull of a patient as movable bone fragments may include selecting ones of the plurality of three-dimensional mesh representations in a computer display of the plurality of three-dimensional mesh representations. This aspect of the embodiment may further include identifying a subset of the three-dimensional mesh representations of movable bone fragments as three-dimensional mesh representations of movable bone fragments having teeth, and the step of reconstructing the area may include reconstructing the area by repositioning the three-dimensional mesh representations having movable bone fragments in response to (a) the identified landmarks, (b) the ones of the three-dimensional mesh representations having fixed bone fragments, (c) the identified salient surfaces of the three-dimensional mesh representations of the bone fragments, (d) the reference model, and (e) ones of the three-dimensional mesh representations of movable bone fragments having teeth.

In accordance with a further embodiment, a method for computer-aided craniomaxillofacial surgery planning is provided. The method includes taking computed tomography (CT) images of a patient’s skull and segmenting and constructing a three-dimensional deformed model of the patient’s skull from the CT images, wherein the three-dimensional deformed model include a plurality of three-dimensional meshes, each of the plurality of three-dimensional meshes corresponding to one of a plurality of bone fragments of the patient’s skull. The method further includes indicating ones of the plurality of bone fragments to be repositioned by selecting the ones of the plurality of bone fragments on a computer display of the plurality of three-dimensional meshes corresponding to one of a plurality of bone fragments, wherein ones of the plurality of bone fragments on the computer display not selected is identified as bone fragments not to be repositioned. And the method includes indicating ones of the plurality of bone fragments to be repositioned that contain teeth and identifying salient surfaces on the ones of the plurality of bone fragments to be repositioned, wherein salient surfaces are surfaces of the ones of the plurality of bone fragments that should be flush with surfaces of other ones of the plurality of bone fragments after restoration and include outer surfaces of the patient’s skull and surfaces around eye sockets of the patient’s skull. The method also includes identifying landmarks associated with a mid-sagittal plane (MSP) and a Frankfurt plane (FP) and generating a restored model from the three-dimensional deformed model of the patient’s skull in response to the three-dimensional deformed model of the patient’s skull and a reference model of a normal person’s skull by repositioning the plurality of bone fragments in the three-dimensional deformed model to normalize an overall shape of the patient’s skull by (a) locating the MSP and FP landmarks proximately close to a calculated MSP and FP of the patient’s skull, (b) placing the movable bones proximately close to an estimated normal and laterally symmetric shape, and (c) placing boundaries of adjacent bone fragments flush with each other. Finally, the method includes exporting data on the repositioned plurality of bone fragments for completion of a surgery plan and/or surgical verification with post-operative CT scans. The exported data can include meshes and synthesized DICOM images of the repositioned plurality of bone fragments.

In accordance with an aspect of this embodiment, the step of segmenting and constructing the three-dimensional deformed model of the patient’s skull includes segmenting only ones of the plurality of bone fragments that need to be repositioned in the surgery as individual ones of the plurality of three-dimensional meshes, the ones of the plurality of bone fragments identified as bone fragments not to be repositioned being segmented as a single mesh.

The accompanying figures, where like reference numerals refer to identical or functionally similar elements throughout the separate views and which together with the detailed description below are incorporated in and form part of the specification, serve to illustrate various embodiments and to explain various principles and advantages in accordance with a present embodiment.

Fig.1

depicts a flow diagram of the computer-aided CMF surgery planning procedure in accordance with the present embodiment.

Fig.2

depicts results of each stage of the planning procedure in accordance with the present embodiment, including (a) Input CT (b) Patient’s deformed model generated from CT images (c) Bone fragments to be repositioned (d) Identified salient surfaces, MSP landmarks, and FP landmarks (e) Restored model generated by the skull restoration algorithm (f) Synthesized DICOM image of the restored model.

Fig.3

depicts a planning tool for indicating movable bone fragments and anatomical landmarks in accordance with the present embodiment, including (a) GUI of planning tool (b) Identification of movable bone fragments to be repositioned (c) Identification of a MSP landmark.

Fig4

depicts an identification of salient surfaces in accordance with the present embodiment. The Identified salient surfaces include eye orbit regions and outer surfaces detected automatically.

Fig.5

depicts Restored skull vs deformed skull in accordance with the present embodiment, including (a) MSP alignment, (b) FP alignment, (c) Lateral symmetry and surface continuity, (Top) deformed skull, (bottom) restored skull, MSP landmarks, FP landmarks.

Fig.6

depicts synthesized DICOM images of the restored model in accordance with the present embodiment, including (a) CT images of the deformed skull (b) synthesized DICOM images of the restored model.

Fig.7

depicts reference model in accordance with the present embodiment. Red surfaces are salient surfaces and line indicates the MSP.

Fig.8

depicts patients’ deformed model in accordance with the present embodiment. Salient surfaces of the bones are indicated. Small balls indicate MSP landmarks and FP landmarks. The bones include the unfractured, undisplaced (“fixed”) bone, and the movable bone.

Skilled artisans will appreciate that elements in the figures are illustrated for simplicity and clarity and have not necessarily been depicted to scale. For example, the dimensions of some of the elements in the illustrations, block diagrams or flowcharts may be exaggerated in respect to other elements to help to improve understanding of the present embodiments.

The following detailed description is merely exemplary in nature and is not intended to limit the invention or the application and uses of the invention. Furthermore, there is no intention to be bound by any theory presented in the preceding background of the invention or the following detailed description. Herein, a computer-aided CMF and orthopedic surgery planning procedure and an algorithm for the generation of a restored model from a patient’s deformed model. Similar procedure and algorithm can be applied to orthopedic surgery of pelvis bones.

Computer-Aided CMF surgery Planning Procedure

The computer-aided CMF surgery planning procedure takes CT images of a patient’s skull as input and generates a surgery plan for restoring the patient’s skull by bone repositioning. It consists of 6 stages as depicted in FIG. 1.

In the first stage 102, the surgeon applies software tools to segment and construct a 3D model of the patient’s skull from CT images 114 (FIG 2(a)). The skull model consists of separate 3D meshes, one for each bone fragment in the skull 116 (FIG 2(b)). In practice, the surgeon may segment only the bones that need to be repositioned in the surgery as an individual mesh, while all other bones are segmented as a single mesh. In general, more bone fragments could be segmented.

In the second stage 104, the surgeon indicates the bone fragments that need to be repositioned using a GUI planning tool as shown in FIG 3. Bone fragments that are not selected are fixed bones. The surgeon also indicates whether a bone contains teeth.

In the third stage 106, salient surfaces are identified on the bone fragments 120 (FIG 2(d)). Salient surfaces are surfaces that should be flushed (i.e. continuous) after restoration. They consist of outer surfaces of the skull that can be automatically detected and surfaces around the eye sockets that are more difficult to detect automatically, which need to be indicated by the surgeon (FIG 4).

In the fourth stage 108, Frankfurt plane (FP), mid-sagittal plane (MSP) and their landmarks are identified as shown in 120 (FIG 2(d)). The surgeon manually indicates FP landmarks 204 and MSP landmarks 206 on the deformed model using the GUI planning tool as shown in FIG 3 (c).

In the fifth stage 110, the surgeon applies an automatic restoration algorithm to generate the restored model 124 (FIG. 2(e)) from the deformed model. The restoration algorithm takes a patient’s deformed model and a reference model of a normal person as inputs and generates a restored model by repositioning the bones in the deformed model. Before restoration (FIG 5(1) (a)-(c)), the fractured bones in the skull are displaced from their correct positions, making the overall shape abnormal. The boundaries of adjacent bones are discontinuous and the MSP landmarks 506 and FP landmarks 504 may not lie on the MSP and FP. After restoration (FIG 5(2) (a)-(c)), the overall shape becomes normal, the boundaries of adjacent bones are flushed and the MSP landmarks 506and FP landmarks 504 line close to the MSP and FP.

In the sixth stage 112, the surgeon instructs the planning tool to export the repositioned bones in STL format. The STL files can be imported into existing planning software to overcome the limitation of the existing planning software to unilateral fractures. The planning tool also synthesized DICOM images of the restored model (FIG 2(f) and FIG 6) for the surgeon’s verification with post-operative CT scans.

Skull restoration algorithm

The skull restoration algorithm in the fifth stage 110 is an algorithm that iteratively repositions fractured bone fragments. The inputs to the restoration algorithm include Reference model (FIG 7) and Deformed Model (FIG 8).

The Reference model (FIG 7) is a mesh model of a normal skull. It has a MSP 704 that serves as the laterally symmetric plane. It also contains a subset of surfaces which are the salient surfaces 702.

The Deformed model (FIG 8) contains a set of bone fragment models 800. These bone fragments are divided into a subset of fixed bone fragments 802 and a subset of movable bone fragments 808. Some movable bones that contain teeth are also known. Each bone fragment has a mesh representing the bone’s surface. It also contains a set of salient surfaces and a set of anatomical landmarks located on it, which can be empty for bone fragments that do not pass through any anatomical plane.

The output of the algorithm is the restored model, which is produced by repositioning the movable bone fragments in the deformed model.

The restoration algorithm uses a reference model as a guide. The preparation of the reference model includes three steps. In the first step, 3D mesh models of the skull and the lower jaw are segmented from CT images and reconstructed. In the second step, a human expert indicates a set of MSP landmarks 806 on the models using the planning GUI. The MSP is then fitted to the MSP landmarks 806 by minimizing its mean squared distance to the landmarks. The origin of the plane is the mean position of these landmarks, and the normal of the plane is the principal component corresponding to the smallest eigenvalue of the covariance matrix of these landmarks. The laterally symmetrical corresponding points are computed by with respect to the MSP. In the third step, salient surfaces of the reference model are identified in a similar manner as that of the patient’s deformed model. For the reference model, all the salient surfaces are identified. In practice, multiple reference models are prepared, and the one that is most similar to the patient’s skull is used by the skull restoration algorithm.

The restoration algorithm is an iterative optimization algorithm that repositions each bone fragment one at a time. It includes three steps. In the first step, given a deformed model, the reference model that is most similar to the deformed model is selected. In the second step, the restored model is initialized as the deformed model. In the third step, several sub-steps are repeated until convergence. The several sub-steps include
a. Register reference model to the restored model by plane-fitting registration
b. Find corresponding points between the reference and restored model. For each point on the reference model, its corresponding point is the closest point on the restored model.
c. Compute MSP and FP of the restored model
d. Mark fixed bone fragments as confident and movable bone fragments as nonconfident.
e. Compute restoration order of the nonconfident bone fragments
f. For each nonconfident bone fragment in decreasing order, generate the estimated normal surface around the bone fragment, and register the bone fragment to estimated surface around it by surface continuity-constrained registration, and then mark the repositioned bone as confident.

Selecting the Reference Model

The method for selecting the reference model that is most similar to the deformed model comprises the following steps:

Register the reference models to the deformed model using Plane-Fitting Registration Algorithm as described in details below and compute their mean surface distances to the deformed model

Select the reference model with the smallest mean surface distance over the fixed bone fragments of the deformed model.

Plane-Fitting Registration Algorithm

The plane-fitting registration algorithm registers the reference model to the restored model robustly and also ensures that the MSP of the reference model matches the MSP landmarks of the restored model. It is an iterative algorithm that repeating the following steps until convergence.

a. Find correspondence between reference model and restored model. For a point on the restored model, its corresponding point is its closest point on the reference model. For a MSP landmark on the restored model, its corresponding point is its perpendicular projection on the MSP of the reference model.
b. Find inlier subset of corresponding points with small distances between the corresponding points.
c. Apply the algorithm to compute weighted similarity transformation using correspondence of inlier points. Larger weights are given to correspondences generated from MSP landmarks.
d. Apply the computed similarity transformation on all points of the reference model.

Computing MSP and FP of Restored Model

The method for computing MSP and FP of restored model comprises the following steps:
1. Transfer the MSP of the reference model to the restored model.
2. Fit the FP to the FP landmarks identified by the user while constraining it to be orthogonal to the MSP.

The step 2 includes the following sub-steps:
a. Project FP landmarks to the MSP orthogonally.
b. Fit a line to the projected landmarks by minimizing least squared distance from the line to the projected landmarks.
c. Compute the plane that is orthogonal to MSP and passes through the fitted line. The origin is any point on the fitted line, and the normal is the cross product the plane normal and the direction of the fitted line.

Computing Bone Repositioning Order

The method for computing the repositioning order of the nonconfident bones computes the following scores:
1. Number of anatomical landmarks on the surface of a bone fragment.
2. Fraction of salient surface vertices whose symmetric points are on a confident bone. The symmetry point relationship is transferred from the reference model using the correspondence computed in (b) of the third step of Skull restoration algorithm.
3. Fraction of salient surface vertices that are at the boundary adjacent to confident bones. The boundaries are identified on the reference model at the edges where the two neighboring points lie on two different bone fragments in the restored model. They are transferred to the restored model using the correspondence computed in (b) of the third step of Skull restoration algorithm.

A weighted sum of these scores is computed for each nonconfident bone fragment, and the bones are sorted in decreasing order of their weighted sum of scores. After that, the bones that contain teeth are moved to the end of the sorted lists so that they are repositioned last.

Generating Estimated Normal Surface

The method for generating the estimated normal surface around a bone fragment comprises the following steps:
1. Find corresponding points between the reference model and the confident parts of the restoration model that meet several requirements.
2. Apply Laplacian surface deformation algorithm on salient surface of the reference model with corresponding points as hard constraints and lateral symmetry as soft constraints.
3. Cut the deformed surface leaving only the portions that correspond to the bone fragment using the corresponding computed in (b) of the third step of Skull restoration algorithm.
4. Repeating the above steps until convergence of the estimated normal surface.

Requirements in Step 1 includes:
a. Corresponding points are close to each other.
b. Corresponding points have similar surface normal.
c. Corresponding points lie near to the surface normal of each other.

Surface Continuity-Constrained Registration

The surface continuity-constrained registration algorithm is an iterative algorithm that registers a nonconfident bone fragment to the estimated normal surface around. It comprises the following steps:
Repeat until convergence:
a. Find corresponding points between the salient surface of a nonconfident bone and the estimated normal surface around it. For a point on the salient surface, its corresponding point is the closest point on the estimated normal surface.
b. Find correspondence for anatomical landmarks. For a MSP or FP landmark on the nonconfident bone, its corresponding point is its orthogonal projection on the MSP or FP of the restored model, respectively.
c. Apply the algorithm to compute weighted rigid transformation using the point correspondence. Larger weights are given to the points near the boundaries of nonconfident bone adjacent to confident bones.
d. Apply the computed rigid transformation to all the points of the nonconfident bone.

This embodiment provides a computer-aided procedure for a surgeon to derive a surgery plan for restoring a patient’s deformed skull or pelvis back to the normal state by bone respositioning.

In comparison, the embodiment has several advantages over existing methods. Firstly, the output of the planning procedure is a restored model which directly results in a feasible surgery plan. Secondly, the planning procedure is applicable to both unilateral and bilateral fractures. Thirdly, the skull restoration algorithm generates a restored model with anatomical plane fitting, shape normality, surface continuity, and lateral symmetry as algorithmic constraints. This makes the result more accurate and reliable than existing manual methods, symmetry-based methods, geometric reconstruction methods, statistical reconstruction methods, and fracture surface matching methods.

Also, the proposed planning procedure has several advantages over existing procedures. The proposed procedure provides an automatic FP, MSP and landmark identification method for skulls with minor injuries. And the proposed procedure replaces the reflection method of existing procedure by an automatic restoration algorithm. Furthermore, the proposed procedure generates a restored model by bone repositioning whereas the existing procedure generates a partially reconstructed model by lateral symmetry.

This embodiment provides more accurate pre-operative planning of craniomaxillofacial (CMF) surgery for the skull and orthopedic surgery for the pelvis. This embodiment also provides training of CMF and orthopedic surgeons in precision recognition of deformed bones as well as planning and execution of the plan in the operating theatre.

This embodiment is applicable to large bone structures such as the skull and the pelvis where all the surgical decisions are made according to the shape of the bone. For orthopedic surgeries on long tubular bones, such as the arm and thigh bones, the surgical decisions are based on the relative orientations of the fractures bones rather than the shape of the bones.

Further preferred embodiments

In addition, there are four possible modifications of the preferred embodiments. Firstly, salient surfaces can be detected by an automatic algorithm. In the present embodiment, the outer surface is identified automatically, and the eye socket region is identified manually. However, the eye socket region can also be identified automatically by fitting a sphere to the eye socket region in a manner similar to the foramen magnum identification method.

Secondly, reference model can be an active shape model. In the present embodiment, the reference model is a single normal model selected from a database. However, it can also be generated by fitting an active shape model to the normal part of the patient’s model, as done for statistical reconstruction.

Thirdly, collision can be avoided. In the above embodiment, the skull restoration algorithm does not explicitly detect collisions of the restored bone fragments. Instead, it relies on the accurate estimation of normal surfaces and registration of bone fragments to the estimated normal surface to naturally avoid collisions. Experiments showed that the collision error is small or most cases. For cases where the collision is not negligible, collision detection avoidance can be performed. Collision detection is a well solved problem. There are various algorithms and packages available, e.g. proximity query package and vtkbioeng, etc. One modification is to check for collision after the skull is restored, and move the collided bones apart to reduce or remove collisions.

Fourthly, the present embodiment can be applied to orthopedic surgery of pelvis bone. In the above embodiments, CMF surgery is used as an example to describe the procedure and the algorithm. However, a similar procedure and algorithm can be applied to orthopedic surgery of pelvis bones. One of the differences is that Frankfurt plan is not defined for pelvis. For pelvis restoration, either a different horizontal plan is defined or the horizontal plane is ignored in the restoration algorithm. One of the differences is that corresponding salient surfaces need to be either detected automatically or identified manually.

It should further be appreciated that the exemplary embodiments are only examples, and are not intended to limit the , applicability, operation, or configuration of the invention in any way. Rather, the foregoing detailed description will provide those skilled in the with a convenient road map for implementing an exemplary embodiment of the invention, it being understood that various changes may be made in the function and arrangement of elements and method of operation described in an exemplary embodiment without departing from the scope of the invention as set forth in the appended claims.

Reference Numerals

100 Flow diagram of the computer-aided CMF surgery planning procedure
102 Segmentation and 3D model construction
104 Identification of bone fragments to be repositioned
106 Identification of salient surfaces on the skull
108 Identification of MSP and FP landmarks
110 Execution of skull restoration algorithm
112 Generation of planning output
114 CT images
116 Deformed model
118 Bones to be moved
120 Salient surfaces
122 MSP/FP landmarks
124 Restored model
126 Output files
200 Each stage of the planning procedure
202 Bone fragments to be repositioned
204 FP landmarks
206 MSP landmarks
300 Planning tool for indicating movable bone fragments and anatomical landmarks
400 Identification of salient surfaces
500 Comparison of Restored skull and Deformed skull
504 FP landmarks
506 MSP landmarks
600 Synthesized DICOM images of the restored model
700 Reference model
702 Salient surface
704 MSP
800 Patient’s deformed model
802 Unfractured undisplaced (“fixed”) bone
804 FP landmarks
806 MSP landmarks
808 Movable bone

Claims

A method for reconstructive modeling of a deformed model, the method comprising:
initializing a restored model as the deformed model;

registering a reference model to the restored model by plane-fitting registration;

identifying corresponding points between the reference model and the restored model, wherein a corresponding point on the restored model for each point on the reference model is the closest point on the restored model;

computing a mid-sagittal plane (MSP) and a Frankfurt plane (FP) of the restored model;

associating one of a first property and a second property with fragments of the restored model;

calculating a restoration order of fragments of the restored model having the second property associated therewith;

for each of the fragments of the restored model having the second property associated therewith and in accordance with the calculated restoration order:

generating an estimated normal surface around the fragment;

registering the fragment to an estimated surface proximate thereto by a surface continuity-constrained registration;

associating the first property with fragments repositioned; and

repeating all steps except the initializing step until convergence of all of the fragments of the restored model.
The method in accordance with Claim 1 further comprising selecting the reference model from a plurality of reference models in response to a similarity between the reference model and the deformed model.
The method in accordance with Claim 2 wherein the step of selecting the reference model comprises:
registering each of the plurality of reference models to the deformed model by the plane-fitting registration;

calculating mean surface distances to the fragments of the restored model having the first property associated therewith; and

selecting one of the plurality of the reference models as the reference model in response to the one of the plurality of the reference models having a smallest mean surface distance over the fragments of the restored model having the first property associated therewith.
The method in accordance with any of the foregoing claims wherein the plane-fitting registration comprises:
finding a correspondence between the reference model and the restored model by defining a corresponding point of a point on the restored model as a closest point on the reference model and defining a corresponding point of a landmark on the restored model as a perpendicular projection of a corresponding landmark on the mid-sagittal plane (MSP) of the reference model;

defining inlier points as an inlier subset of corresponding points in response to small distances between the corresponding points;

calculating a weighted similarity transformation in response to a correspondence of the inlier points wherein larger weights are given to correspondences generated from the corresponding landmarks;

applying the calculated similarity transformation on all points of the reference model; and

repeating these steps until convergence of all the points on the restored model.
The method in accordance with any of the foregoing claims wherein the method comprises a method for computing a Frankfurt plane (FP) and a mid-sagittal plane (MSP) of the restored model.
The method in accordance with Claim 5 wherein the step of computing the Frankfurt plane (FP) and the mid-sagittal plane (MSP) of the restored model comprises the steps of:
transferring the MSP of the reference model to the restored model; and

fitting the FP to FP landmarks identified by the user while constraining it to be orthogonal to the MSP by projecting the FP landmarks to the MSP orthogonally, fitting a line to the projected FP landmarks by minimizing a least squared distance from the line to the projected FP landmarks, and computing a plane that is orthogonal to the MSP and passes through the fitted line, wherein an origin is any point on the fitted line and a normal is a cross product of a normal of the plane and a direction of the fitted line.
The method in accordance with any of the foregoing claims wherein the fragments of the restored model comprise bone fragments, the first property comprises a confident property, the second property comprises a nonconfident property, and the bone fragments comprise fixed bone fragments and movable bone fragments, and wherein the step of associating the first property and the second property with the bone fragments of the restored model comprises associating the confident property with the fixed bone fragments and associating the nonconfident property with the movable bone fragments.
The method in accordance with Claim 7 wherein the step of calculating the restoration order comprises computing a repositioning order of the movable bone fragments in response to weighted sums of the following for each of the movable bone fragments:
(a) a number of anatomical landmarks on a surface of one of the movable bone fragments,

(b) a fraction of salient surface vertices on the one of the movable bone fragments whose symmetric points are on one of the bone fragments of the restored model having the first property associated therewith, and

(c) a fraction of salient surface vertices on the one of the movable bone fragments that are at a boundary adjacent to one of the bone fragments of the restored model having the first property associated therewith.
The method in accordance with Claim 8 wherein the step of computing the repositioning order of the movable bone fragments in response to the weighted sums of each of the movable bone fragments comprises computing the repositioning order of the movable bone fragments in response to a decreasing value of the weighted sums of each of the movable bone fragments.
The method in accordance with Claim 9 wherein the step of computing the repositioning order of the movable bone fragments in response to a decreasing value of the weighted sums of each of the movable bone fragments comprises:
identifying ones of the movable bone fragments that contain teeth;

computing the repositioning order of the movable bone fragments in response to a decreasing value of the weighted sums of each of the movable bone fragments; and

moving the ones of the movable bone fragments that contain teeth to the end of the repositioning order so that the ones of the movable bone fragments that contain teeth are repositioned last.
The method in accordance with any of the foregoing claims wherein the step of generating an estimated normal surface around each fragment of the restored model having the second property associated therewith comprises:
finding corresponding points between the reference model and the fragment of the restored model having the first property associated therewith that have (a) corresponding points are close to each other, (b) corresponding points have similar surface normal, and (c) corresponding points lie near to the surface normal of each other;

determining a surface deformation on a salient surface of the reference model in accordance with the corresponding points and lateral symmetry to the fragment of the restored model having the first property associated therewith;

cutting the surface deformation on the salient surface of the reference model leaving only the portions that correspond to closest points on the fragment of the restored model having the second property associated therewith; and

repeating the above steps until convergence of the estimated normal surface.
The method in accordance with any of the foregoing claims wherein the surface continuity-constrained registration comprises an iterative calculation until convergence that registers each fragment of the restored model having the second property associated therewith to an estimated normal surface around it in response to (a) closest points between a salient surface of the fragment of the restored model having the second property associated therewith and an estimated normal surface around it, (b) the landmark points identified by the user on the fragment of the restored model having the second property associated therewith and the perpendicular projections of the landmark points on corresponding planes of the restored model, (c) weighted rigid transformation of corresponding points between the fragment of the restored model having the second property associated therewith and the estimated normal surface around the fragment of the restored model having the second property associated therewith, wherein larger weights are given to points near boundaries of the fragment of the restored model having the second property associated therewith adjacent to fragments of the restored model having the first property associated therewith, and (d) rigid transformation to all the points of the fragment of the restored model having the second property associated therewith.
A method for reconstructive surgery planning comprising:
providing images of one or more bones in an area to be reconstructed, the one or more bones including one or more bone fragments;

segmenting the one or more bones into a plurality of three-dimensional mesh representations such that each of the one or more bone fragments are a separate one of the plurality of three-dimensional mesh representations;

assigning one of a first property or a second property to each of the plurality of three-dimensional mesh representations; and

reconstructing the area by repositioning the three-dimensional mesh representations having the second property in response to ones of the three-dimensional mesh representations having the first property and a reference model.
The method in accordance with Claim 13 wherein the step of providing images comprises providing computed tomography (CT) images of the one or more bones in the area to be reconstructed.
The method in accordance with either Claim 13 or Claim 14 wherein the step of reconstructing the area comprises reconstructing the area by repositioning the three-dimensional mesh representations having the second property in response to (a) identified landmarks, (b) the ones of the three-dimensional mesh representations having the first property, (c) identified salient surfaces of the three-dimensional mesh representations, and (d) the reference model, wherein the salient surfaces of the three-dimensional mesh representations are surfaces of the three-dimensional mesh representations that should be flush with surfaces of the three-dimensional mesh representations of adjacent ones of the one or more bone fragments after reconstructing the area.
The method in accordance with any of Claims 13 to 15 wherein the reconstructive surgery comprises craniomaxillofacial surgery and/or orthopedic surgery, and wherein ones of the plurality of three-dimensional mesh representations having the first property comprise three-dimensional mesh representations of fixed bone fragments, and wherein ones of the plurality of three-dimensional mesh representations having the second property comprise three-dimensional mesh representations of movable bone fragments.
The method in accordance with Claim 16 wherein the reconstructive surgery comprises craniomaxillofacial surgery, and wherein salient surfaces of the three-dimensional mesh representations of the one or more bone fragments comprise surfaces of the three-dimensional mesh representations of the one or more bone fragments that should be flush with surfaces of the three-dimensional mesh representations of adjacent ones of the one or more bone fragments after reconstruction and the salient surfaces include outer surfaces of the skull and surfaces around eye sockets.
The method in accordance with Claim 17 wherein the identified landmarks comprise identified landmarks on a Frankfurt plane (FP) and a mid-sagittal plane (MSP), and wherein the reference model comprises a mesh model of a normal skull having a MSP that serves as a laterally symmetric plane and comprises a subset of surfaces corresponding to the salient surfaces of the three-dimensional mesh representations of the one or more bone fragments.
The method in accordance with Claim 18 wherein the reference model is selected from a plurality of reference models in response to one or more similarities with a skull of a patient of the craniomaxillofacial surgery.
The method in accordance with either Claim 18 or Claim 19 wherein the step of assigning ones of the plurality of three-dimensional mesh representations of a skull of a patient as movable bone fragments comprises selecting ones of the plurality of three-dimensional mesh representations in a computer display of the plurality of three-dimensional mesh representations.
The method in accordance with Claim 20 further comprising identifying a subset of the three-dimensional mesh representations of movable bone fragments as three-dimensional mesh representations of movable bone fragments having teeth, and wherein the step of reconstructing the area comprises reconstructing the area by repositioning the three-dimensional mesh representations having movable bone fragments in response to (a) the identified landmarks, (b) the ones of the three-dimensional mesh representations having fixed bone fragments, (c) the identified salient surfaces of the three-dimensional mesh representations of the bone fragments, (d) the reference model, and (e) ones of the three-dimensional mesh representations of movable bone fragments having teeth.
A method for computer-aided craniomaxillofacial surgery planning comprising:
taking computed tomography (CT) images of a patient’s skull;

segmenting and constructing a three-dimensional deformed model of the patient’s skull from the CT images, wherein the three-dimensional deformed model comprises a plurality of three-dimensional meshes, each of the plurality of three-dimensional meshes corresponding to one of a plurality of bone fragments of the patient’s skull;

indicating ones of the plurality of bone fragments to be repositioned by selecting the ones of the plurality of bone fragments on a computer display of the plurality of three-dimensional meshes corresponding to one of a plurality of bone fragments, wherein ones of the plurality of bone fragments on the computer display not selected being identified as bone fragments not to be repositioned;

indicating ones of the plurality of bone fragments to be repositioned that contain teeth;

identifying salient surfaces on the ones of the plurality of bone fragments to be repositioned, wherein salient surfaces are surfaces of the ones of the plurality of bone fragments that should be flush with surfaces of other ones of the plurality of bone fragments after restoration and include outer surfaces of the patient’s skull and surfaces around eye sockets of the patient’s skull;

identifying landmarks associated with a mid-sagittal plane (MSP) and a Frankfurt plane (FP) ;

generating a restored model from the three-dimensional deformed model of the patient’s skull in response to the three-dimensional deformed model of the patient’s skull and a reference model of a normal person’s skull by repositioning the plurality of bone fragments in the three-dimensional deformed model to normalize an overall shape of the patient’s skull by locating the MSP and FP landmarks proximately close to a calculated MSP and FP of the patient’s skull, placing the movable bones proximately close to an estimated normal and laterally symmetric shape, and placing boundaries of adjacent bone fragments flush with each other; and

exporting data on the repositioned plurality of bone fragments for completion of a surgery plan and/or surgical verification with post-operative CT scans.
The method in accordance with Claim 22 wherein the step of segmenting and constructing the three-dimensional deformed model of the patient’s skull comprises segmenting only ones of the plurality of bone fragments that need to be repositioned in the surgery as individual ones of the plurality of three-dimensional meshes, the ones of the plurality of bone fragments identified as bone fragments not to be repositioned being segmented as a single mesh.