JP5882743B2 - Customized orthopedic implants and related methods and intelligent cartilage systems - Google Patents

Description

The present disclosure relates to systems and methods for manufacturing customized surgical devices, and more particularly, the present disclosure relates to automated systems and methods for arthroplasty cutting guides, in generating computer models of knee joints. The present invention relates to a system and method for segmenting an image.
(Cross-reference of related applications)
This application is a provisional US application Ser. No. 61 / 208,509 entitled DEFORMABLE ARTICULATING TEMPLATE filed on Feb. 25, 2009, the disclosure of which is incorporated herein by reference, and July 2009. Claims priority based on US Provisional Application No. 61 / 222,560 entitled CUSTOMIZED ORTHOPAEDIC IMPLANTS AND RELATED METHODS filed on the 2nd.

  The success of TKA surgery depends on the recovery of accurate knee alignment. The key to knee joint stability has been shown to be the recovery of a functional axis with a balanced flexion and extension gap. Traditionally, intramedullary and extramedullary jigs have been used to assist in the orientation of femoral and tibial components. Computer-assisted surgery has been developed to assist surgeons in the proper positioning and orientation of components. However, surgical navigation systems are not widely used in hospitals. The main argument against surgical navigation systems is that, in addition to the difficulty of learning, they are expensive and take more time than necessary in the operating room.

  The need for a simple and accurate system as an alternative has stimulated the orthopedic industry and technologies have been developed that include the use of patient-specific cutting jigs. Magnetic resonance images (MRI) of the patient's hips, knees, and ankles are used to generate a model of a particular patient's anatomy. From these images, a virtual 3D model of the femur and tibia is created using software and the bones are oriented in space. The size of the implant is determined by the software, the virtual bone excision is mapped, and the implant is positioned and aligned using a disposable specialized guide that fits into the patient's bone. A standard ablation instrument pin position is determined.

  However, there are a number of drawbacks associated with MRI, including scanning costs, increased scanning time, geometric distortions, and the need to standardize scanning protocols among different MRI vendors. In recent years, computed tomography CT has also been used, but the radiation associated with this procedure may be undesirable for some patients. An alternative approach to MRI or CT is to use statistical anatomical shape analysis methodology to accurately model hyaline cartilage using X-rays and / or ultrasound.

  Statistical anatomical shape analysis has rapidly established itself as a valuable tool in the design process of orthopedic implants and patient-specific solutions. Therefore, an intelligent cartilage system (iCS) with the goal of performing statistical anatomical shape analysis, accurately modeling cartilage thickness and contour, and creating specialized cutting guides (or jigs) Was devised. The guide is designed to make a cut so that the knee returns to its normal anatomical state before the cartilage deteriorates. iCS systems are built on a fundamental fusion of statistics and 3D bone modeling.

  The iCS platform represents a cohesive software system that includes integrated multidimensional medical imaging, computer aided design (CAD) and computer graphics mechanisms to design patient-specific cutting jigs. Coordinated interactions, efficiencies, and the prospect of extensive customization allow iCS to accurately and quickly handle the complexity of patient volume growth. A high performance database of bone and cartilage atlases provides the technology used in customized modules for generating specialized jigs. By reducing the time required for current specialized jig manufacturing processes, the iCS platform minimizes the risk of bottlenecks.

  The database stores patient information according to HIPPA regulations. Different imaging modalities are applied to each patient, including MRI / CT, X-ray images, and DICOM from ultrasound. The reconstructed bone and cartilage are stored in the database. Virtual template data including calculated landmarks, axes, implant sizing, and placement information is also stored in the database. This component implements both a relational schema and an XML schema, providing a powerful means for data storage and manipulation.

Osteochondral reconstruction subsystem. This module can be segmented from MRI, CT, PET or ultrasound, or directly reconstructed from RF signals as in microwave imaging and US, or 3D bones from 2D X-ray images include reconstruction of bone and soft tissue with either of the reconstruction, the flow chart showing the outline of this sub-system can be found in Figure 5.

  Medical image segmentation can be broadly divided into two categories: structural and statistical. The structural approach is based on the spatial characteristics of the image pixel, such as edges and regions. Statistical methods are based on intensity value probability distributions to label image regions. Image intensity and their corresponding class labels are considered to be random variables. Thus, the statistical approach is likely to solve the problem of evaluating class labels when given image intensity values. Previous segmentation attempts have combined one or more of these methods in an attempt to overcome the limitations of individual methods.

  Most segmentation using edge-oriented techniques is semi-automatic segmentation, which is used to extract a contour, optimize the contour, and propagate it to the adjacent slice. Image pre-processing and gradient operators can be used to improve typical region expansion methods. If the placement of the seed (manually or automatically) indicates that an organ is present in a particular region of the image, this region can be expanded until the contrast boundary of the organ is reached. The parameters can be changed to change the expansion momentum, making the algorithm somewhat more sensitive to small changes in grayscale.

  Region-oriented segmentation with knowledge-based labeling is a pixel classification technique based on the homogeneity of several features of an object. Various techniques such as knowledge based on uncertainty inference, static domain knowledge from higher-order feature extraction, fuzzy logic, long and short-term memory modeling, and unsupervised clustering are used in this field. These give very good results with MRI images due to the high contrast between soft tissues. The overall interpretation error of the brain is 3.1%, and the interpretation error of a small area of the brain is 9%.

  The watershed technique is used in combination with various other segmentation methods in the hope of improving accuracy. The watershed algorithm is briefly described by a 3D plot of the grayscale intensity of the image, where “water” fills the valleys until the two valleys of the image meet. This provides connection information for different areas of the image based on grayscale values. Preprocessing by edge enhancement such as Sobel filter and texture analysis can be useful for detecting different organs.

  The clustering algorithm is an unsupervised algorithm that iterates between image segmentation and characterization of each class until a distinct image cluster is formed. Examples of such algorithms are the K-means algorithm, fuzzy C-means algorithm, expectation maximization (EM) algorithm, and self-organizing neural network. The K-means clustering algorithm clusters the data by iterating to calculate the average intensity of each class, classify each pixel in the class by nearest neighbor, and segment the image. This was used to segment the brain. The number of classes was assumed to be 3 representing cerebrospinal fluid, gray matter and white matter.

  The fuzzy C average algorithm generalizes the K average algorithm and allows soft segmentation based on fuzzy set theory. The EM algorithm applies the same clustering principle, assuming that the data follows a mixed Gaussian distribution model. This iterates between the calculation of the posterior probabilities and the calculation of the maximum likelihood estimate of the mean, the covariance, and the model mixture coefficients. Since the algorithms do not directly incorporate spatial modeling, they are sensitive to noise and intensity inhomogeneities. This can be overcome by using Markov random field modeling.

  Morphological operators such as binary morphological operations are used in some segmentation systems, and the basic concept in morphology is to convolve an image with a given mask (known as a structuring element) Using a given function, the result of convolution is binarized. Examples include shrinkage, expansion, opening and closing.

  Statistical techniques such as thresholding label regions of the image using their intensity value histograms. The maximum and minimum thresholds define the area of interest according to the knowledge base. For example, in CT, rough segmentation of organs can be achieved by image thresholding by the range of the hounsfield units of the organ of interest. This is typically applied in digital mammography where there are two classes of tissue, healthy and tumorous. The limitation of relying solely on such methods is due to the normal overlap of organ intensity intervals, intensity non-uniformities, and sensitivity to noise and image artifacts. Since thresholding does not consider the spatial characteristics of the image, any artifact that distorts the image histogram may ultimately affect the segmentation results. Nevertheless, thresholding remains the first step in many segmentation algorithms. With respect to classical thresholding, variants have been proposed for segmenting medical images that incorporate information based on local strength and connectivity.

  Deformable template matching, including contour detection, tracking, matching, and match error optimization, is performed using 2D deformable contours. In segmentation, atlases are applied to 2D deformable contours, which allow mapping between image data and atlases with conditional minimization of predetermined data. A 3D deformable surface is also realized by tracking the change in contour between slides. In order to separate the organ of interest from neighboring organs, Bayesian statistics are usually used to determine model prior information or probabilities. In general, this approach produces good results for small and local shape changes. This is also suitable for large and general misalignments or deformations. The main improvement was obtained by using a main geodesic analysis to determine the average shape and main deformation mode of the object and an m-rep model in which the object is represented as a connected mesh of a set of medial atoms. This method gives excellent results with 0.12 cm average surface separation and 88.8% volume overlay compared to manual segmentation for automatic segmentation of the kidney.

  Knowledge-based approaches using blackboard architecture systems are generally in the field of shared memory that includes problems to be solved and many different processes. The blackboard is built and updated continuously along the inference process. The anatomical structure of the image data is labeled by adapting the structure in the image to the corresponding object in the model. Data from images and models are converted to a common parameter feature space for comparison. The extracted low and high level features are written to the blackboard and compared to the model. This result is also written on the black board to guide further matching. Long-term memory and object relationships for descriptions can be written to the knowledge base.

  A four-dimensional Markov random field (MRF) provides a way to segment moving targets such as the heart using a 4D stochastic atlas. Atlas predicts temporal and spatial changes based on prior information, and the algorithm also incorporates spatial and temporal context information using a 4D Markov random field (MRF). The overall connection filter completes the segmentation using the largest connection structure as a starting point. The results given are highly favorable for heart segmentation and are not limited to MRI images. Compared to the manually segmented model, the left ventricular (LV) result is 96%, the myocardial result is 92%, and the right ventricular (RV) result is 92%.

  The system uses information from any 3D imaging modality to extract gradient information combined with a statistical atlas to extract bone boundaries and cartilage interfaces. The following is a detailed description of each of these bone reconstruction modalities.

1 is an overview of an exemplary iCS system. FIG. 2 is a diagram of an exemplary data upload subsystem shown in FIG. 2 is an illustration of the exemplary database shown in FIG. 2 is a list of exemplary database table details for the database shown in FIG. FIG. 2 is an exemplary schematic diagram of the bone / cartilage reconstruction subsystem shown in FIG. 1. FIG. 2 is an exemplary diagram of a virtual templating subsystem shown in FIG. 1. FIG. 2 is an exemplary diagram of a jig prototype manufacturing subsystem shown in FIG. 1. FIG. 2 is an exemplary diagram illustrating an algorithmic process for automatic segmentation within a segmentation process according to the present disclosure. FIG. 3 is an exemplary diagram of a process of an automatic alignment algorithm according to the present disclosure. It is a figure which shows the bone ridgeline calculated as an example of the characteristic extracted from the mesh of the femur distal part. It is a figure which shows the profile search along the normal line direction of CT. It is a figure which shows the profile search along the normal line direction of MRI. FIG. 6 is a graph showing profiles at various stages and new edge positions relative to old edge positions. FIG. This is a segmented image with edge relaxation. It is a segmented image. It is a segmented image before relaxation. It is the segmented image after relaxation. FIG. 6 is an exemplary diagram illustrating the process of cartilage segmentation from MRI. It is CT image image | photographed using the contrast agent which enables visualization of a cartilage tissue. 2 is an image of two femur surfaces for profile calculation, where image (a) includes a tibial contact surface and image (b) includes a tibial non-contact surface. It is a graph of the average profile of class 1 (a), class 2 (b), and class 3 (c). Figure 2 is an image of segmented bone and cartilage from MRI. FIG. 3 is an exemplary diagram of an X-ray 3D model reconstruction process flow. 2 is an image of an exemplary calibration target. FIG. 6 is an exemplary image showing beads appearing on a radiographic image. FIG. Fig. 6 is a radiograph of a leg with a calibration target attached. It is a segmented image of a femur distal part. It is an image showing a complex pose exploration space and how the particle filter succeeds in finding the optimal pose. It is an image which shows the projection of the template bone | frame on a radiograph. It is the image of the outline mapped by 3D. FIG. 3 is an exemplary diagram illustrating a 3D reconstruction process. FIG. 4 is an exemplary diagram for training a predictive model for cartilage thickness. FIG. 6 is an exemplary diagram for cartilage reconstruction using a trained prediction model. FIG. 4 is a diagram of estimated cartilage thickness from MRI. FIG. 6 is a map of an exemplary cartilage template thickness. FIG. 3 is an exemplary image of predicted cartilage on the femur and tibia. 1 is an exemplary diagram of a UWB imaging system. FIG. FIG. 6 is an exemplary diagram showing how a signal reflected from the knee is received by all of the other UWB antennas while one signal acts as a transmitter. It is an image which shows the experimental setting in which a UWB antenna array surrounds the circumference | surroundings of a knee. FIG. 3 is a diagram of a microwave imaging process for detecting tissue interfaces in the femur and tibia. 2 is an exemplary sample of an ultrasound image acquired and recorded from an anterior distal femur. FIG. 6 is an exemplary image of a bone model fitted to an acquired ultrasound image of a distal femur. FIG. FIG. 6 shows steps performed to segment using ultrasound. It is a screen shot of the subcomponent of virtual templating. FIG. 6 is an exemplary diagram of a jig creation process. It is a continuous image representing a specific step of creating a jig. FIG. 5 is a series of images representing different jig designs with different fixation methods (a. Fixation of medial and lateral condyles, b. Curve fixation, c. Groove fixation). Figure 6 is a series of images showing a femur and tibia cutting jig for use in knee correction surgery. It is a screen shot of a CAD editor for modifying a 3D output jig model. It is a screen shot of evaluation of a jig about original CT data.

  Exemplary embodiments of the present disclosure provide a method and apparatus for designing a patient-specific prosthetic cutting jig, more specifically, an apparatus and method for segmenting a knee bone, and resulting Is described and shown as including the cutting guide itself. Of course, it will be apparent to those skilled in the art that the preferred embodiments described below are exemplary in nature and can be reconfigured without departing from the scope and spirit of the invention. However, for clarity and accuracy, the exemplary embodiments described below may include optional steps, methods, and features that one skilled in the art will recognize need not fall within the scope of the invention. .

  Referring to FIG. 1, an exemplary overall system overview includes a surgeon who creates a new symptom, requests a specialized jig, and uploads patient imaging data, after which the company's A system of notifying engineers follows. The next step involves the creation of patient-specific bone and cartilage, which are then used to find the implant that best fits the patient, and once the pre-operative plan is complete, the surgeon reviews and approves the plan. To be notified. Once the surgeon approves the plan, a specialized cutting jig is automatically created for the patient and the preoperative plan is transferred to the operating room.

  2-7 show an overview of the major components of the system, including a data upload component, a database component, an osteochondral reconstruction component, a virtual templating and jig generation component.

  An overview of the automatic segmentation process is shown in FIG. The first step in the segmentation process is to adjust the base mesh from the statistical atlas to the volume, and an automatic adjustment algorithm was developed to make an accurate adjustment. This adjustment process3. a. 1 is outlined in FIG. This process includes extracting the isosurface by simple thresholding. An isosurface is basically a surface mesh generated from all of the bone-like tissue in the volume. The isosurface is noisy and cannot distinguish distinct bones. Extract features from the isosurface and mean atlas models (this is done early and can be easily loaded). These features can be, but are not limited to, folds, or bone ridges, umbilical points in FIG. 10, or any other surface descriptor. The feature points on the average model and the feature points on the isosurface are matched by nearest neighbor or other matching methods. The result here is noisy, which means that there are some mismatches, but the subset is correct. Some robust fitting method, such as the RANSAC algorithm or least mean square method, is used to find an image transform that minimizes the error between matching points while maximizing the number of matches.

  When the previous adjustment step is completed, an iterative warping procedure is performed that uses the information in the atlas as constraints on model deformation. The initial parameters define the number of major components at the start, the initial search length and the minimum possible search length. The first step is to calculate the vertex normal at each vertex on the bone mesh. These normal directions represent the deformation direction of each vertex. These normals are calculated by averaging the normals of adjacent triangles in the mesh. The search line for each vertex defined by the normal direction is a path that extends inward and outward from the bone model and is centered on the vertex. The length of the probe line decreases as the process proceeds.

  The probe line intensity value is calculated by three-line interpolation because the probe line sampling rate can be higher than the resolution of a given volume. The profile is first smoothed by some denoising filter, here using Sabitsuki-Golay filtering. This is particularly important when a noisy MRI image is given. After denoising, the gradient along the profile is calculated. This initial gradient is insufficient to define the bone edge position because there are several tissue interfaces with strong gradients, such as the skin-air interface. This can be seen in FIGS. It is safe to assume that the patient-specific bone margin is located near the adjusted apex because the initial automatic adjustment is considered very accurate. To model this assumption, the gradient profile is weighted with a Gaussian function so that the central vertex is given an initial weight of 1.0 and the weight decreases as the search proceeds further from the central location. . A diagram of the profile at various stages can be seen in FIG. After weighting, the absolute maximum value of the gradient is determined along with its position. This position represents the bone edge and the old vertex position is now replaced with the new edge position.

  For CT, the process looks for the location of the falling edge, i.e. the edge that moves from high intensity to low intensity as the probe moves from inside to outside. The reverse is true for MR. Therefore, if necessary, the profile is inverted before the above steps to take into account differences in modalities. After the transformation for each vertex is made, the model can be adjusted to the atlas using the inverse of the transform calculated in the initial adjustment step. This model can be considered noisy because some edge positions are placed at inaccurate positions. In order to constrain the deformation and remove as many noise points as possible, a noisy model vertex is projected onto the atlas space using a certain number of major components, as in FIG. Is determined by a parameter that changes based on The resulting model is a healthy femur that best represents the patient-specific noisy model. These models are then transformed back into volume space.

  After projection, the parameters are updated so that the exploration length is reduced unless new major components are added. The search length is then increased slightly so that new local information can be captured. When the residual RMS reaches a sufficiently low threshold, a new major component is added. The above process starting with the calculation of the normal direction is repeated until all major components are used and the residual RMS is sufficiently low or until a predetermined maximum number of iterations is reached. The resulting noisy model contains patient-specific information, which is then smoothed and remeshed at high resolution [FIGS. 15, 16]. The higher resolution allows the capture of small local deformations such as bone growths that may have been missed by the low resolution segmentation process. The high-resolution model then uses a single iteration of the segmentation procedure that stops before the bone is projected onto the atlas, using a probe length that is small enough to prevent inaccurate bone positioning. Alleviated. The resulting bone is a highly accurate representation of the patient's anatomy. Thus, the output of the segmentation is a high-resolution patient-specific bone model and smoothing model that represents the closest healthy bone generated by the atlas before relaxation [FIG. 17].

  After completion of the automatic segmentation, two bones are obtained. Bone projected on the atlas and relaxed patient-specific bone. If the bone is highly deformed, unconstrained relaxation can be performed using a gradient vector flow (GVF) snake. These snakes respond to gradient information present in the image, and the gradient flow acts as a force that locally expands or contracts the snake's contour to a boundary that minimizes the force on the contour. If the contour is not highly denatured, the initial relaxation phase is very close to the actual patient anatomy in most cases and no snake method is required. Snake and interactive segmentation tools can be seen in FIGS. The final segmentation step before model generation is an interactive intervention to correct any errors in the segmentation process. Several tools are provided for interactive segmentation, such as adding or removing contours, coloring, etc. Once segmentation is approved, a final model is generated by interpolating the contours at a high resolution that ensures smooth results.

  The process of cartilage segmentation from MRI is shown in FIG. 18, and patient volume segmentation results in patient-specific using a patient-specific bone model of the resulting distal femur, proximal tibia and patella. A cartilage model can be obtained. If sufficient information is present in the scan results, this is the case when MRI is used or CT using contrast agents that highlight cartilage tissue (FIG. 19) is used, but is patient specific By using prior information together with information, cartilage can be segmented. Prior information can be considered as a feature vector.

ここで、ｐ（ｘ｜ｍ）は、ｍすなわち測定値が与えられる事後確率である。値ｐ（ｍ｜ｘ）は、ｘの値が与えられる場合に所与の測定値ｍが生じる可能性を表わす公算関数である。ｐ（ｍ）項は規格化項である。事前確率は、ｐｐｒ（ｘ）すなわち事前確率密度内に存在する。何らかの所与の測定値ｍが与えられている場合、ｘの値について行うことができる最良の推量は、事後確率を最大にするｘである。これは、帰納的最大確率（ＭＡＰ）として知られている。カートリッジ厚の推定（ｘ）に関しては、関節間隔（ｍ）及び事前厚マップ（ｐｐｒ（ｘ））が与えられている場合、ＭＡＰ推定値が求められる。この同じ概念をＢＣＩ位置に適用することができる。 In the case of cartilage, this can be thickness or position information about the bone of the joint. The reliability placed in the prior data is represented by a probabilistic model of each feature. For example, the confidence that a cartilage is xmm thick at a particular point on the bone is modeled in a cartilage thickness map constructed from a previously segmented cartilage model. In order to determine the posterior probability, for example, the cartilage thickness at the new point, the following Bayesian inference model is used.

Here, p (x | m) is m, that is, a posterior probability that a measurement value is given. The value p (m | x) is a probable function representing the probability that a given measurement m will occur when a value of x is given. The p (m) term is a normalization term. Prior probabilities exist within ppr (x), the prior probability density. Given any given measurement m, the best guess that can be made for the value of x is x which maximizes the posterior probability. This is known as the maximum recursive probability (MAP). With respect to the cartridge thickness estimate (x), a MAP estimate is determined if a joint spacing (m) and a pre-thickness map (ppr (x)) are given. This same concept can be applied to BCI locations.

  Initially, exploration is limited to the joint surface. This is done using prior information that exists in the atlas. The contact surface is most likely on the bone-cartilage interface (BCI) and consists of vertices in the atlas that make most contact with the opposing bone. Non-contact surfaces may include cartilage but are limited to vertices that do not contact bone. These surfaces can be seen in FIG.

  Each profile of the contact surface is calculated along a path between the current bone vertex and the nearest vertex on the contact bone. The non-contact surface profile is calculated up to 1 cm along the normal direction of the bone surface. The local maxima and minima of each profile are calculated and the profiles are placed in one of three different classes. The average profile for each class is shown in FIG. If the profile contains a single maximum, it belongs to class 1. These are the shortest profiles and correspond to places where the cartilage of the tibia and femur are in close contact and indistinguishable from each other. A profile containing two maxima and one minima is said to belong to class 2. These correspond to an intermediate length profile with a clear gap between the femur and tibia cartilage. The class 3 profile is the longest profile and femoral cartilage is usually well represented, but the curves in class 3 vary widely and are often irregular.

  Any vertex with a profile belonging to class 1 or class 2 can be immediately classified as belonging to BCI. A class 3 profile is added to the BCI based on whether the intensity level is close to the intensity level of other BCI points and the point may belong to the BCI as determined from the BCI probability map, or Removed from BCI.

  After the BCI is automatically determined, the user is given the option of manual confirmation and can edit the BCI using a number of tools similar to those found in the bone segmentation editor. If it is determined that the BCI is sufficiently accurate, segmentation of the cartilage model proceeds. The cartilage is segmented along the profile dimensions using gradient information from the scan volume coupled with prior knowledge of cartilage thickness. This prior knowledge gives an initial estimate of the potential cartilage margin, which is then updated by determining the local maximum of the profile gradient. For class 1 profiles, the cartilage margin is considered to be at the maximum. For class 2 and class 3 profiles, the local maximum of the gradient is used.

Cartilage segmentation can then be adjusted interactively, if necessary, before the final cartilage model is output. The segmented femoral cartilage can be seen in FIG.
An overview of the x-ray bone reconstruction process is shown in FIG.
X-ray images are taken using either conventional fluoroscopy or radiography. The number of images may be one or more. Image projections are taken to maximize the information that can be acquired by scanning with a large angular difference. The accuracy of the system is directly proportional to the number of images, but the speed is inversely proportional. The focal length and image resolution (of the camera digitizer or film scanner) of radiographic scene characteristics are manually entered into the system if not immediately available in the image file header.

  The goal of preprocessing is to reduce image noise, increase image contrast to improve the input image, and prepare the image for further analysis. This is done automatically, although there may be manual intervention for extreme image distortion. A Gaussian filter, a median filter, and a Sobel filter are used.

  Calibration involves the extraction of imaged bone poses in the radiographic scene. Prior to image acquisition, a calibration target is attached to the patient's leg [FIG. 24]. This target will contain beads that are opaque to radiation, which will appear on the acquired image [FIGS. 25, 26]. The bead projection is automatically extracted from the image and used to roughly estimate the bone pose to the x-ray source.

  The marker can then be automatically detected in the image using morphological operations. The placement of the markers on the object is selected to cover as large an image area as possible in every frame. Using enough markers allowed accurate pose estimation even if the feature detection algorithm missed some of the markers or some of them were out of the field of view.

  These beads can be separated by thresholding at various levels and removing (by morphological manipulation) any objects that contain lines longer than the bead diameter. The bead centroid is found by finding the connected components and then determining the centroid of each component. The calibration target is designed to minimize the possibility of overlapping bead projections, thereby maximizing the number of beads detected.

  The sensor pose is calculated by finding the correct association (or correspondence) between the detected bead location and the 3D bead location in the image. This is achieved using an interpretation tree search method. In this method, each level of the tree represents a correspondence between an image point and all possible model points, and each path from root to leaf node in the complete tree is one possible correspondence. Represents a relationship. When an exploration down the tree reaches a node where at least four correspondences are assumed, the correspondence is used to calculate a pose solution. In general, only three points are needed to obtain a pose solution, but since the three points are always on the same plane, there can be more than one solution. At least four points need to be used.

が得られる。 Once there are four correspondences, the pose can be calculated. This pose solution is used to project the object points back onto the image for comparison with the detected markers as a measure of how well this solution fits the complete data set. Used. If the pose solution fits well with the data, the correspondence and the calculated pose are appropriate. If the pose solution does not match the data, the correspondence is inappropriate and the tree is further traversed to check for another correspondence. For a large number of points, this tree can be very large and does not explore the whole tree. In practice, the tree is further explored only if the current correspondence results in an error greater than a certain threshold. In the following cases, it is assumed that a correct correspondence is found.
Once the correct correspondence is found, the pose is calculated using a non-linear least squares method. Specifically, given a six-way vector containing image points P, 3D model points Q, and appropriate pose parameters, the transformation projected onto the image is a function such as P = f (β, Q) Suppose that it is given by f (β, Q). The vector function f also represents a perspective projection whose parameters are known from calibration. By linearizing around the solution β of the current pose,

Is obtained.

Given the correspondence of at least four points, the correction term Δβ can be solved. This process is repeated until the solution converges. The final solution is a pose β that minimizes the square of the Euclidean distance between the observed image point and the projected model point on the image plane.
The bone image segmentation block uses the preprocessed image as input. The goal of this is to extract the actual bone contours of all images. This is done automatically using a bone contour statistical atlas generated using the 3D bone database of the present disclosure. The template average bone contour from the statistical atlas is first placed in the image and then translated and rotated to align with the bone image. Thereafter, contour deformation is statistically performed to fit the target bone image based on the intensity values of the image and the edges obtained from the preprocessing steps. Manual and semi-automatic contour editing tools can also be used for verification of the automated process [FIG. 27].

ここでｗ_jはｊ番目の粒子の重みである。 The feature extraction module is responsible for extracting image parameters from the image. These parameters are extracted from both the preprocessed and segmented versions of the image. Feature types include information extracted from segmented bone contours (curvature, geometric shape, aspect ratio, etc.) or information extracted from pixel intensity values (texture features, such as Haralick and Tamura texture features). Including.
This process is necessary because the calibration block is expected to cause errors due to the relative conversion between the calibration target and the actual bone.
In the case of fluoroscopy, the number of bone images is usually large and the difference in bone pose between coherent images is usually very small. Therefore, pose tracking using a particle filter is applied.
This approach has been found to be useful for handling applications where the state vector is complex and the image contains a large amount of clutter. The basic idea is to represent posterior probabilities by state or weighted sampling of particles. Given enough samples, even very complex probability distributions can be represented.
When measurements are taken, the weight of the particles is adjusted by the probable model using the following equation:

Here, w _j is the weight of the j-th particle.

  The main advantage of this representation is that given enough particles, it can represent a large number of peaks in the posterior probability distribution (FIG. 28).

  Once the measurement is obtained, the tracking algorithm adjusts the weights, and when enough data is obtained to eliminate the ambiguity of the state, eventually one of the particles will have a higher weight and all the other Has a very small weight. Another advantage of this approach is that it can determine when a unique solution is obtained.

  Particle resampling. It is important to ensure that the state sampling properly represents the posterior probability distribution. Specifically, it is desirable that a large number of samples approach a peak having a large weight. To ensure this, at each step, the probability distribution is resampled by generating additional particles near large peaks and discarding particles with very small weights. Emphasis sampling is used to input samples based on bottom-up information.

  Probability function design. It is important to design the likelihood function as smooth as possible. Given a large number of narrow peaks, some of these peaks can be missed unless a very large sample is used. What is important is to use a similarity measure with broad support for each hypothesis. In other words, the similarity needs to gradually increase as the assumed state approaches the correct state and should not increase rapidly.

に基づくものであり、ここで、ｘ₁、ｘ_nは画像特徴及び骨のポーズの変数を表わす。 In the case of radiography, the number of images is usually limited, so the features will be used within the Bayesian network framework, where the current set of image features is given The predicted output is a bone pose. This method can also be used to initialize the particle filter in the case of fluoroscopy. Bayesian networks are built as directed acyclic graphs, where each node represents an image feature and the node connection represents a conditional dependency. The output will be the pose with the highest probability given a set of input image features. The probability of network output is the probability chain rule

Where x ₁ and x _n represent image feature and bone pose variables.

  A 3D reconstruction algorithm based on GPU rendering simulation is used. The input to this method is a set of segmented images and their corresponding bone poses. The number of images and the various projection poses indicate the amount of information that can be obtained about the shape of the bone and thus the accuracy of the output. For each input image, a graph reconstruction of the radiation scene used to capture the image is performed. The x-ray source is represented by a perspective camera setting for simulating the spread of the radiation beam. Within the camera's field of view, the template bone model is placed in a pose that mimics the actual bone pose in the radiation scene [FIG. 29].

  Once the graph scene is set, bone projection images are synthesized and their contours are mapped to 3D points on the surface of the template bone using depth mapping and rendering functions (FIG. 30). These 3D points are then systematically transformed in space to eliminate 2D contour errors between the composite image and the original radiographic image. As a result, a 3D point cloud that generates a bone projection similar to the bone projection of the input X-ray image is generated using the contour data from all images [FIG. 31].

  This translates the problem into a 3D to 3D optimization problem. To quickly and uniquely find the best shape that matches both the statistical atlas as well as the generated cloud of points, the POCS (alternate projection onto the convex hull) method is used. POCS is a powerful tool that has been successfully used for numerous signal and image reproduction and compositing problems. This is particularly useful in defect setting problems, and in some cases the problem can be regularized by imposing a non-linear convex constraint on the set of solutions. By iteratively projecting onto these convex sets, a solution that matches all the desired properties is obtained. A brief description of this method follows.

及び

のときかつそのときに限って凸型であり、その場合、

である。換言すれば、ｘとｙを接続する線分は、ρに完全に含まれる。２つの点を接続する弦のいずれかの部分が集合の外にある場合には、その集合は凸型ではない。凸集合への射影は、以下のように定義される。すべての閉凸集合ρ及びヒルベルト空間内のすべてのベクトルｘに対して、ｘに最も近い一意的なベクトルがρの中に存在する。このベクトルは、ＰＣｘと表示され、ρへのｘの射影である。ＰＯＣＳの最も有用な態様は、非空共通集合を有する２つ又はそれ以上の凸集合が与えられる場合、集合の間の交互の射影が共通集合に含まれる一点に収束することである。 In vector space, the set ρ is

as well as

And only then is convex, in which case

It is. In other words, the line segment connecting x and y is completely included in ρ. If any part of the string connecting two points is outside the set, the set is not convex. Projection to a convex set is defined as follows: For every closed convex set ρ and every vector x in Hilbert space, there is a unique vector in ρ that is closest to x. This vector is denoted PCx and is the projection of x onto ρ. The most useful aspect of POCS is that given two or more convex sets with a non-empty common set, the alternating projections between the sets converge to one point included in the common set.

  If the two convex sets do not intersect, the convergence goes to a limit cycle that is a root mean square solution to the problem. Specifically, this cycle is between the points closest to the other set in the mean square sense within each set.

  In this method, 1. a collection of all bones that can belong to a statistical bone atlas; There are two convex sets with the set of all bones (the other vertices can have any value) with constraint values for the selected number of vertices. For the selected vertex, a corresponding point can be seen on the contour of the image. The bone vector can be projected onto the second set simply by replacing each selected vertex with the corresponding estimated point.

  By alternately projecting from one convex set to the other, then the other convex set can be quickly converged to a solution that fits both sets.

  Once a patient-specific bone model is obtained, cartilage must be added to complete the mounting surface on which the jig is fitted. Using the X-ray image database of the present disclosure and their corresponding MRI scan data, a Bayesian network was created that identifies the level of cartilage degradation from the inter-knee distance profile of the X-ray image. These pieces of information are used as input to the cartilage generation module.

  The system includes a high performance database that stores patient information according to HIPPA regulations. Data from different imaging modalities including MRI / CT, X-ray images, and DICOM from ultrasound are attached to each patient. The reconstructed bone and cartilage are stored in the database. Virtual template data including calculated landmarks, axes, implant sizing, and placement information is also stored in the database. This component implements both relational and XML schemas and provides a powerful tool for data storage and manipulation.

  FIG. 32 outlines the process of creating a predictive model for reconstructing articular cartilage given the femur, tibia, and patella. To build this model, a 2000 MRI scan database was used. Bone and articular cartilage were first segmented from these scans. Bones were added to the statistical atlas to obtain point correspondences, and the probability of the bone-cartilage interface region was calculated for each bone. The distance from bone to bone was calculated across the bone-cartilage interface region by finding the closest distance between the two bone surfaces at each vertex. Cartilage thickness was also calculated at each of these locations and used as a target to train the neural network and build a Bayesian belief network to predict cartilage thickness. Inputs to these systems include, but are not limited to, bone-to-bone distance, knee degeneration and deformation classification, and varus and valgus angle measurements. The output for this is a prediction system that can construct cartilage in CT, X-ray, US, microwave, and guide cartilage segmentation in MRI [FIG. 33]. FIG. 34 shows the process of identifying the cartilage interface in the MRI training data set, while FIG. 35 shows the average cartilage map. FIG. 36 shows the output of the prediction model in one test case.

  A microwave imaging system consists of an array in which each element operates as both a transmitter and a receiver. The system configuration is shown in FIG. 37, where a low noise system clock (clock crystal) triggers a baseband UWB pulse generator (eg, a step recovery diode (SRD) pulse generator). The baseband pulse is converted to a higher frequency by a local oscillator through a double balanced wideband mixer. The signal converted to a high frequency is amplified and filtered. Finally, the signal is transmitted via a directional microwave antenna. The signal is received at all of the other antennas in the array, filtered, amplified, converted to a lower frequency, and processed with a low pass filter. The pulses are then time extended to 1000-100,000x using a subsampling mixer triggered by the same low noise system clock. This effectively reduces the bandwidth of the UWB pulse and allows sampling by a conventional analog-to-digital converter (ADC). Finally, as shown in FIG. 38, beam forming and final cross-sectional image generation are performed using a specialized digital signal processing algorithm. The image is recovered using a near-field delay and sum beamformer. Target scattered signals received by different Rx antennas are equalized in amplitude to compensate for different scattering ratios, propagation losses, and attenuation. Different phase delays of the Rx signal are used for beam manipulation. Interfaces between various types of tissues are detected as shown in FIG. 15 (air-skin, fat-muscle, cartilage-bone, etc.). An experimental configuration showing a UWB antenna array, knee bones, and muscles surrounding the knee is shown in FIG.

  The resulting received signal is used to detect tissue interfaces (ie, cartilage-bone, fat-muscle, air-skin) from various angles and can be further transformed into a 2D cross-sectional image. Tissue interface data is added to the patient's existing bone and cartilage atlas information. This results in the expanded feature vector containing UWB imaging data. This process is outlined in FIG. The tracking UWB antenna array is moved along the knee and multiple cross-sectional images are acquired. Cross-sectional images are recorded together, allowing full 3D analysis of the various tissue interfaces (emphasis is placed on the cartilage-bone interface). Information about these tissue interfaces is added to the patient feature vector. This results in additional tissue interface information (ie, cartilage-bone, soft tissue-cartilage) being used for the creation of bone and cartilage atlas and various automated measurements for knee articular cartilage and bone. To. Finally, this information can be included in the Bayesian estimation process for bone and cartilage atlases.

  This system extends the diagnostic B-mode ultrasound machine and adds the ability to create 3D models of patient-specific joint bones and cartilages (eg, knees). A location probe (optical or magnetic) is rigidly attached to the ultrasound probe and tracks its movement while the operator is scanning the joint (eg, knee). The transformation between the motion tracking probe coordinate frame and the ultrasound probe coordinate frame is determined by a calibration process. Since the motion tracking probe gives the position (translation) and orientation of each acquired B-mode image (frame), the acquired image is recorded in 3D Cartesian space as shown in FIG. The acquired ultrasound image set then reconstructs the scanned anatomical volume (similar to the volume reconstructed from CT or MRI), along with their acquired position and orientation. Used for

Next, three or more alignment landmarks (such as a predetermined landmark, the most prominent point on the epicondyle of the femur) are obtained using a tracking A-mode probe or B-mode probe, then An average model of an atlas of bone (eg, a bone to be modeled like a femur) is recorded in the volume reconstructed using registration landmarks obtained using paired point recording. Next, the proposed automated segmentation used for CT or MRI is applied to the reconstructed ultrasound volume, resulting in a segmented bone model as shown in FIG.
A flowchart describing the ultrasound reconstruction system is shown in FIG.

  Virtual templating results in implant sizing, placement, and visualization of selected patient bones. The landmark is automatically calculated with high accuracy and reproducibility. The size adjustment of the implant is performed together with the placement of the implant. The user can then select a specific implant and implant family to perform these functions. The user can select from predefined or user-defined surgical techniques to place the components of the implant, and the user can define new surgical techniques for placement of both femoral and tibia components. Can do. For example, based on the landmarks and axes, the user can visualize an excision, a complete intact bone, and / or an implant placed on the excised bone. For example, in an exemplary embodiment, a user may be given three 2D orthogonal views and one 3D view for visualization and implant manipulation. The user can modify the implant size, family and manufacturer information. Visualization can include axes and landmarks that are overlaid on the bone. The adapted results can be stored in a sophisticated database. The surgeon uses the various functions described herein to perform virtual templating, implant placement, virtual resection, and implant manipulation, thereby pre-operative templating for patient-implant alignment. Can produce quantitative results. This surgeon version of virtual ablation is performed over the Internet using 3D technology in an applet (Java® 3d). The surgeon can modify, approve, or reject the templated results. Various exercises are performed to verify the alignment and placement of the implant. The placed tool is then used to convert the cutting tool to the same alignment as the patient. These screw holes are then transferred to a jig and this arrangement is transferred to the operating room [FIG. 44].

  45 and 46 show an overview of the jig creation process, which uses a template jig placed on the average bone from the statistical atlas of the present disclosure. The jig is then resized to fit the size of the patient's femoral condyle and tibial plateau. Next, both the tibial model and the femur model are crossed with the patient-specific 3D bone model to create a unique patient stamp on the inside of the jig, which ensures a tight fit during surgery.

FIG. 47 highlights different jig design strategies for different levels of joint alteration. FIG. 48 shows the femur and tibia jig output from the system.
FIG. 49 shows an editor for polishing and validating automatically generated jigs.
FIG. 50 illustrates the process of verifying the output jig by projecting it onto the same space as the patient volume data.

  While the foregoing description and summary of the invention, the methods and apparatus described herein constitute exemplary embodiments of the invention, the invention contained herein is intended to be in this exact embodiment. It should be apparent to those skilled in the art that the embodiments can be modified without departing from the scope of the invention as defined by the claims and without departing from the scope of the invention. Further, the present invention is defined by the claims, and any limitation or element that describes the exemplary embodiments described herein is not intended to be limited unless such limitation or element is expressly stated. It should be understood that it is not intended to be incorporated into the interpretation of the elements of a claim. Similarly, the invention is defined by the following claims, and may exist where an intrinsic and / or unexpected advantage of the invention may not be expressly described herein. It should be understood that it is not necessary to meet any or all of the identified advantages or objectives of the invention disclosed herein to fall within the scope of any claim.

1: Data upload 2: Database 3: Bone / cartilage reconstruction 4: Virtual template 5: Jig prototype

Claims (8)

A method of constructing a 3D image bone shell and a 3D image cartilage shell for the generation of a patient specific bone shell representing at least a portion of the current patient bone and cartilage , comprising:
A plurality of 2D image slices of the patient anatomy are imaged at right angles to an axis extending through the patient anatomy, imaging at least a portion of the patient anatomy, each A 2D image slice is created to include at least one of a bone contour including a closed boundary corresponding to the outer boundary of the patient's bone and a cartilage contour including a boundary corresponding to the outer boundary of the patient's cartilage. Steps,
Constructing a 3D image crust of at least a portion of the patient's bone from which the plurality of 2D image slices of the patient's bone were created;
Constructing a 3D image cartilage shell of at least a portion of the patient cartilage from which the plurality of 2D image slices of the patient cartilage were created ;
Including
The step of constructing the bone shell of the 3D image includes recognizing the closed boundary of each bone contour using a bone shell of a template 3D image that is not patient-specific to indicate the current state of the bone of the patient. Including using software to build a bone shell,
The step of constructing the cartilage shell of the 3D image recognizes the closed boundary of each cartilage contour by using the cartilage shell of the template 3D image that is not patient specific, and indicates the current state of the cartilage of the patient. Using software to construct a cartilage shell of
Wherein a call.   The method of claim 1, wherein the imaging step includes at least one of magnetic resonance tomography and computed tomography. The method of claim 1, wherein the patient's bone includes at least one of a femur, a tibia, and a patella . A software component further includes generating a 3D image surgical jig to mate with the 3D image bone shell and the 3D image cartilage shell, wherein the 3D image surgical jig comprises a bone of the 3D image. The method of claim 1, including customized topographic features in the outer boundary features of the shell and the cartilage shell of the 3D image.   5. The software component of claim 4, wherein the software component is operative to output a command file for manufacturing a surgical jig having tangible topographic features of the 3D image surgical jig. Method. A method for generating a patient-specific cutting jig representing at least a portion of an existing patient anatomy comprising:
A plurality of 2D image slices of the patient anatomy are imaged at right angles to an axis extending through the patient anatomy, imaging at least a portion of the patient anatomy, each Creating a 2D image slice to include a bone contour including a closed boundary corresponding to an outer boundary of the patient's bone;
Constructing a 3D image crust of at least a portion of the patient's bone from which the plurality of 2D image slices of the patient's bone were created;
Including
The step of constructing the bone shell of the 3D image includes recognizing the closed boundary of each bone contour using a bone shell of a template 3D image that is not patient-specific to indicate the current state of the bone of the patient. Including using software to build a bone shell,
The step of constructing the bone shell of the 3D image includes isosurface extraction and shape feature extraction, and fitting a point on the bone shell of the template 3D image to a point on the plurality of 2D image slices It includes steps to create a mesh,
The method for generating a cutting jig comprises:
Creating a 3D image surgical cutting jig using the 3D image bone shell ;
Creating a surgical cutting jig using the 3D image surgical cutting jig created in the step of creating a 3D image surgical cutting jig ;
Further comprising :
Method.   7. The 2D image slice is created using at least one of magnetic resonance tomography, computed tomography, X-ray imaging, and ultrasound imaging. The method described.   Creating the surgical cutting jig comprises using a software component to output a command file for manufacturing the surgical cutting jig including tangible topographic features of the 3D image surgical jig. The method of claim 6 comprising:

Images